nucleic-purpose-classifier/README.md

# Purpose classifier

This directory is the reproducible data and training pipeline for
`docs/PURPOSE_CLASSIFIER.md`. The current dataset is the promoted Sol-high v2 reset. Its
dataset lineage, completed lite/deep experiments, artifact hashes, decisions, and bounded
next steps are locked in `experiments/sol-high-v2.json`. Older v1 results below are kept
as implementation history and are not label-compatible candidates for Sol-high v2.

## Data contract

The canonical generated sources are listed in `data/generation-manifest.json`.
`round2-NN.jsonl` files are retained generation batches and intentionally duplicate
`purpose-prompts-round2.jsonl`; they are provenance, not additional training input.

`prepare_data.py`:

- validates the strict generated-record schema;
- removes exact and high-overlap word-trigram duplicates;
- fails for review if a high-overlap pair has conflicting labels;
- keeps shipped fixtures completely outside source data;
- holds every `vague-eval` record out of training;
- optionally applies a completed, versioned human-review ledger before splitting;
- stratifies by primary purpose, slice, and primary language; and
- verifies that the deterministic test partition still matches the versioned
  `data/frozen-test-v1.jsonl`.

The Sol-high v2 frozen test set is the synthetic JSONL plus the 89 classifiable records in
`Tests/NucleicCoreTests/Fixtures/purpose-prompts.json`. The fixture file's three `general`
records are excluded because `general` is deliberately not a model label. The exact
membership and hashes are locked in `data/dataset-v1-manifest.json`.

## Full Sol-high label reset

`rebuild_sol_high.py` is the end-to-end workflow for intentionally invalidating the old
labels and rebuilding both classifiers from one teacher. It freezes and relabels every
canonical public prompt, every shipped fixture, local Nucleic first-prompt history, and
the pinned SWE-chat candidates with `gpt-5.6-sol` at high reasoning effort. The teacher
settings are not configurable in this workflow. Batches default to eight prompts and
failed structured responses are retried up to ten times.

The reset is staged below the ignored `.artifacts/sol-high-reset/` directory. Snapshot
hashes prevent a resumed run from silently mixing input revisions or teacher settings.
Run these commands from the repository root with the purpose-classifier environment:

```bash
"$PY" ml/purpose-classifier/rebuild_sol_high.py snapshot

"$PY" ml/purpose-classifier/rebuild_sol_high.py label \
  --python "$PY"

"$PY" ml/purpose-classifier/rebuild_sol_high.py status
```

Rerunning `label` resumes every existing decision log and does not relabel completed
lines. Once `status` reports `"complete": true`, promote the staged result explicitly:

```bash
"$PY" ml/purpose-classifier/rebuild_sol_high.py promote \
  --confirm overwrite-all-labels-with-sol-high
```

If promotion reports a near-duplicate label conflict in the combined real-session
population, inspect both prompts and exclude the reviewed bad/ambiguous copy explicitly.
The 1-based line is relative to the reported `combined-real.labeled.jsonl`; repeat the
option for multiple decisions:

```bash
"$PY" ml/purpose-classifier/rebuild_sol_high.py promote \
  --confirm overwrite-all-labels-with-sol-high \
  --exclude-real-line <line>
```

Each exclusion is bound to the prompt hash, prior purpose, and review reason in the
combined dataset manifest and promotion result. This option does not change a teacher
label or relax duplicate detection.

Promotion first validates and curates the complete staged population. It then replaces
the two canonical source files, regenerates all round-two mirrors, relabels shipped
fixtures (teacher-rejected fixtures become the runtime `general` fallback), rebuilds the
frozen public evaluation split, and replaces `.artifacts/dataset-v1/` with the combined
training dataset. Nucleic history and SWE-chat are deduplicated against public evaluation
data and added to training only; `vague-eval` records remain optimization exclusions.
The prior files are moved into a timestamped ignored backup before replacement.

The old human/semantic review assertions are marked superseded because they were tied to
the invalidated label population. Consequently, Sol-high v2 evaluation measures agreement
with the new teacher and is not directly comparable with the old v1 scores.

Print the two clean, from-pretrained-base training commands after promotion:

```bash
"$PY" ml/purpose-classifier/rebuild_sol_high.py train-commands
```

By default the generated commands use the Python interpreter that invoked
`rebuild_sol_high.py`, so running this from an activated virtual environment emits that
environment's concrete Python path. Pass `--python <path>` only to override it.

The purpose-lite command starts from the pinned MiniLM revision. The purpose-deep command
starts from the pinned ModernBERT base variant; neither command supplies a prior classifier
checkpoint or continuation flag.

### Sol-high v2 promotion and training result

Promotion completed on 2026-08-02/03 with 12,164 combined training records, 1,208
validation records, and 1,119 synthetic test records. It explicitly excluded the reviewed
real-population conflicts at labeled source lines 1201 and 1481. The ignored combined
training manifest is bound by SHA-256 in the versioned experiment ledger without exposing
private prompt text.

| Run | Selected validation | Frozen evaluation | Decision |
|---|---:|---:|---|
| lite, from pretrained | 91.28% primary | 93.29% overall; 88.84% hard | rejected |
| lite, boundary continuation | 92.04% primary | 93.72% overall; 91.16% hard | rejected; 13 correct decisions short of 95% |
| deep base | 73.39% primary; 67.50% hard | not opened | rejected on validation |
| deep distilled continuation | 77.64% primary; 72.92% hard | not opened | rejected on validation |

The boundary continuation is useful as the selected lite validation baseline and cached
teacher, but it is not a shipping candidate. Do not tune another lite continuation against
this frozen set, start QAT/export, chain another deep continuation, or start ModernBERT
large/purpose-max.

The next result-bearing work is a validation-only representation probe: compare the
current pretrained prediction-head CLS representation with attention-masked mean pooling
using a regularized primary-only linear head. A pooling change earns one full
ModernBERT-base run only if it gains at least three points on validation overall or hard,
reaches at least 85% overall, and causes no per-purpose collapse. Before any deep frozen
look, the revised model must come within two points of lite overall and beat lite by at
least three points on the identical validation hard slice. If the probe fails, compare a
sentence-trained encoder with the same protocol or remove the deep tier; increasing model
size is not the next variable.

For lite, use validation-only error analysis and data improvements, then reserve a new
independent holdout before the next candidate cycle. QAT, export, target-runtime parity,
latency, energy, and residency resume only after a new float candidate qualifies. See
`experiments/sol-high-v2.json` for exact metrics, hashes, and gates.

## Label local Nucleic history

`export_nucleic_prompts.py` extracts the first `userText` event from every local
transcript. `label_nucleic_prompts.py` then removes malformed, empty, NUL-containing, and
normalized-duplicate lines before asking `gpt-5.6-terra` to reject semantic junk and label
the retained prompts. The result uses the exact canonical seven-field source-data
contract. All generated files stay under the gitignored `.artifacts/` directory because
they contain private prompt history.

```bash
python3 ml/purpose-classifier/export_nucleic_prompts.py \
  --sessions-dir "$HOME/Library/Application Support/Nucleic/sessions" \
  --output ml/purpose-classifier/.artifacts/nucleic-history-first-prompts.unlabeled.jsonl \
  --manifest ml/purpose-classifier/.artifacts/nucleic-history-first-prompts.manifest.json

python3 ml/purpose-classifier/label_nucleic_prompts.py
```

The labeler writes the dataset, a rejection audit, and an append-only state file. If a
Codex call or the process stops partway through, continue without re-labeling completed
batches:

```bash
python3 ml/purpose-classifier/label_nucleic_prompts.py --resume
```

By default Codex runs ephemerally at low reasoning effort, ignores user configuration and
project rules, and is instructed not to use tools. `--codex-isolation auto` uses Codex's
read-only isolation on a host and the existing outer isolation when the script runs in a
Nucleic managed container.

## SWE-chat v2 import (gated source)

The SWE-chat source is not downloaded by this repository. After accepting the dataset's
Hugging Face conditions, place a **pinned** Parquet snapshot below the ignored
`.artifacts/swe-chat/raw/` directory, record its immutable revision, then run the
streaming extractor. It reads only the needed columns, takes the first qualifying human
prompt plus its conversational agent response, and writes the prompt plus a response hash.
Do not use `main` as a revision.

```bash
ml/purpose-classifier/.venv/bin/pip install -r \
  ml/purpose-classifier/requirements-swe-chat.txt
ml/purpose-classifier/.venv/bin/python \
  ml/purpose-classifier/export_swe_chat.py \
  --revision <accepted-immutable-hf-revision>
```

The export and manifest remain ignored because candidate JSONL temporarily contains the
prompt and agent response. Run the one-record schema/availability canary before the 100-session dry
run; both use Luna through subscription-backed `codex exec`, not an API key. The labeler
writes only the first message to canonical source JSONL; state and audit sidecars retain
the response solely as a hash and source ID.

The labeler sends all remaining explanatory response prose to Luna. It replaces fenced
code and structured tool/reasoning payloads with size-marked placeholders, and fails on
oversized remaining prose rather than silently truncating it. The default response limit
is 48,000 characters; raise `--max-response-chars` only after inspecting an outlier.

```bash
ml/purpose-classifier/.venv/bin/python \
  ml/purpose-classifier/label_swe_chat_prompts.py --limit-sessions 1
ml/purpose-classifier/.venv/bin/python \
  ml/purpose-classifier/label_swe_chat_prompts.py --limit-sessions 100
```

Use a fresh `--output` path for the dry run, then manually audit it before invoking the
full resumable run. Cases marked `recoverableFromFirst=false` remain `vague-eval`
abstention evidence and are excluded from optimization.

## Assistant-prose slice

Context Switch (`docs/CONTEXT_SWITCH.md` §3.2) classifies the agent's settled-turn reply as
well as the user's prompt, but everything above trains on prompts only. This slice adds the
missing register from the same pinned snapshot: turn-ending `assistant_response` rows, with
the student text produced by `prose_extract.py` — a port of the runtime's own
`HeuristicSummary.contextSwitchReplyProse`, pinned to it by
`Tests/NucleicCoreTests/Fixtures/context-switch-prose.json` so the model trains on exactly
the text it is later asked to classify.

```bash
ml/purpose-classifier/.venv/bin/python \
  ml/purpose-classifier/export_swe_chat_prose.py \
  --revision <accepted-immutable-hf-revision>
ml/purpose-classifier/.venv/bin/python \
  ml/purpose-classifier/label_swe_chat_prose.py --limit-replies 1
ml/purpose-classifier/.venv/bin/python \
  ml/purpose-classifier/label_swe_chat_prose.py --limit-replies 100
```

The teacher reads the reply prose and the user message that opened the turn; only the prose
reaches the canonical record, with the user message kept as a hash in the audit sidecar.
`recoverableFromProse=false` is the reply-side counterpart of `recoverableFromFirst` and is
likewise `vague-eval` evidence, never training data. A reply that only *offers* work
("Should I start on the settings screen?") must be rejected rather than labeled with the
work it asks about; the run prints `endingInQuestion` alongside `endingInQuestionRejected`
so that rule can be verified on the dry run instead of assumed.

## Prepare

From the repository root:

```bash
python3 ml/purpose-classifier/validate-data.py
python3 ml/purpose-classifier/prepare_data.py
python3 -m unittest discover -s ml/purpose-classifier/tests -p 'test_*.py'
```

For a newly generated raw 200-record batch, enable batch-shape checks explicitly with
`validate-data.py path/to/batch.jsonl --batch-size 200 --expected-total 200`.

The generated train/validation copies land under `.artifacts/dataset-v1/` and are
gitignored. A source, curation, seed, or split-policy change that moves the frozen test
set fails closed. After reviewing such a change, intentionally version it with:

```bash
python3 ml/purpose-classifier/prepare_data.py --refresh-frozen-test
```

## Train purpose-lite

Use a dedicated virtual environment. The base model is pinned to a specific
`sentence-transformers/all-MiniLM-L6-v2` commit: a 6-layer, 384-dimensional encoder. The
training collator always pads/truncates to 128 tokens so the later ONNX/Core ML export
can expose a fixed `1 x 128` runtime shape. Long prompts preserve both ends as
`[CLS]` + 63 head tokens + `[SEP]` + 62 tail tokens + `[SEP]`; this keeps the ask when it
follows a pasted log or stack trace while retaining enough leading context to interpret it.

```bash
python3 -m venv ml/purpose-classifier/.venv
ml/purpose-classifier/.venv/bin/pip install -r ml/purpose-classifier/requirements.txt
ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/train.py
```

The default requirements use PyTorch's CPU-only wheel on Linux, avoiding an accidental
multi-gigabyte CUDA install in CI and development containers. For NVIDIA, AMD, or Intel
accelerator training, install the platform's `torch==2.13.0` build using PyTorch's
platform selector, then install `requirements-base.txt`.

Training writes a local checkpoint, `calibration.json`, and `metrics.json` under
`outputs/purpose-lite-v1/`. It selects checkpoints and fits temperature on label-scorable
validation records. When deriving nested HIGH/MEDIUM/LOW cutoffs, every `vague-eval`
record counts as an abstention miss even if its synthetic label happens to match. The
incoming checkpoint is scored and retained as epoch zero, so a continuation run cannot
silently replace it with a regression. Validation early stopping defaults to two epochs
without an improvement greater than 0.05 points.

Continuation training accepts a local checkpoint. `--boundary-weight` is an opt-in,
validation-selected loss weight for the measured weakest slice; it does not add held-out
fixtures to training:

```bash
ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/train.py \
  --model ml/purpose-classifier/outputs/purpose-lite-v1/model \
  --epochs 3 --learning-rate 3e-6 --warmup-ratio 0 \
  --boundary-weight 2 \
  --output-dir ml/purpose-classifier/outputs/purpose-lite-v1-boundary-tune \
  --overwrite-output
```

For QAT, `--quantization-aware` replaces the model's linear and embedding forwards with
straight-through fake quantization matching the shipping QDQ graph: per-tensor uint8
embeddings, per-channel symmetric int8 linear weights, and per-tensor uint8 activations.
Parameter names remain unchanged, so the selected checkpoint reopens as an ordinary
Transformers model and uses the same `export.py` path. Keep the incoming checkpoint as
epoch zero and select QAT only on validation. Training logs progress every 50 batches by
default (`--progress-steps 0` disables it), so a long CPU run remains observable:

```bash
ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/train.py \
  --model ml/purpose-classifier/outputs/purpose-lite-v1-boundary-tune/model \
  --epochs 2 --learning-rate 1e-6 --warmup-ratio 0 \
  --early-stopping-patience 1 --boundary-weight 2 --quantization-aware \
  --output-dir ml/purpose-classifier/outputs/purpose-lite-v1-qat1 \
  --overwrite-output
```

On dataset v1, that validation-selected run produced a 23,148,500-byte int8 graph at
94.88% frozen accuracy (889/937), 94.46% scored-hard accuracy, and 98.19% scorable
PyTorch↔ONNX agreement. It was the pre-distillation quantized candidate and remained two correct
predictions below the 95% gate. A subsequent validation-selected `5e-7` epoch improved
int8 validation accuracy from 93.31% to 93.71% but regressed frozen accuracy to 94.34%;
it is rejected. Do not continue optimizer-only QAT sweeps on this split. The next model
iteration should incorporate reviewed boundary data and be selected on a revised
validation/frozen dataset version.

To target only the remaining float→int8 decision drift, cache the float teacher in a
separate inference process and use its logits for QAT distillation. Keeping teacher and
student models out of the same process avoids doubling peak resident memory:

```bash
ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/cache_teacher.py \
  --model ml/purpose-classifier/outputs/purpose-lite-v1-boundary-tune/model \
  --output ml/purpose-classifier/outputs/purpose-lite-v1-boundary-teacher.pt \
  --overwrite-output
ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/train.py \
  --model ml/purpose-classifier/outputs/purpose-lite-v1-boundary-tune/model \
  --distillation-cache \
    ml/purpose-classifier/outputs/purpose-lite-v1-boundary-teacher.pt \
  --distillation-weight 0.9 --distillation-temperature 2 \
  --distillation-selection-weight 0.5 --quantization-aware \
  --epochs 2 --learning-rate 1e-6 --warmup-ratio 0 --boundary-weight 1 \
  --output-dir ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat \
  --overwrite-output
```

The cache binds each logit row to normalized prompt hash plus expected label. Training
fails closed if either split changes. Selection combines label accuracy with float-teacher
agreement, retains the incoming checkpoint as epoch zero, and logs label/distillation loss
separately. A 64-record wiring run exercised cache loading, shuffled row alignment,
backpropagation, selection, and ordinary checkpoint reload. The current shared CPU runtime
then showed severe post-batch throttling, so no full candidate result is claimed from that
canary.

### Native Apple Silicon training with MLX

Use the MLX backend when training on Apple Silicon. It implements the same six-layer BERT
classifier, fixed head-tail tokenization, export-matched QAT graph, cached-teacher
distillation, validation selection, and early stopping with native MLX arrays. Fake
quantization is decomposed into Metal-supported round, clip, and straight-through-gradient
operations, avoiding PyTorch's unsupported MPS fake-quant operator. Selected weights are
written back with the original Hugging Face parameter names, so the existing PyTorch
`export.py` and `eval.py` paths remain unchanged.

Install the additional pinned dependency into the macOS virtual environment:

```bash
ml/purpose-classifier/venv/bin/python -m pip install \
  -r ml/purpose-classifier/requirements-mlx.txt
```

Before the first full run on a new MLX or Transformers version, run the fail-closed parity
check. It requires exact fake-quant primitives, float-logit parity, matching QAT
predictions with bounded backend drift, healthy QAT gradients, and an exact Hugging Face
→ MLX → Hugging Face weight round trip:

```bash
ml/purpose-classifier/venv/bin/python ml/purpose-classifier/verify_mlx.py \
  --model ml/purpose-classifier/outputs/purpose-lite-v1-boundary-tune/model
```

Then run the distilled QAT candidate natively on Metal:

```bash
ml/purpose-classifier/venv/bin/python -u ml/purpose-classifier/train_mlx.py \
  --model ml/purpose-classifier/outputs/purpose-lite-v1-boundary-tune/model \
  --distillation-cache \
    ml/purpose-classifier/outputs/purpose-lite-v1-boundary-teacher.pt \
  --distillation-weight 0.9 --distillation-temperature 2 \
  --distillation-selection-weight 0.5 --quantization-aware \
  --epochs 4 --early-stopping-patience 1 \
  --learning-rate 1e-6 --warmup-ratio 0 --boundary-weight 1 \
  --progress-steps 1 \
  --output-dir ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e \
  --overwrite-output
```

The full Metal run stopped after epoch three and selected epoch two at 94.75% fake-quant
validation accuracy. Its 23,148,500-byte int8-QDQ export scores **95.20% frozen
(892/937)**, 95.19% macro recall, 94.17% scored-hard accuracy, and 98.08% scorable
PyTorch↔ONNX agreement. Every purpose recall is above 91%, and the vague-abstention and
routing-tier-drift gates pass. This became the accepted artifact for the historical v1
label contract; it is not a Sol-high v2 candidate. Its remaining latency and
energy/residency work is relevant only as runtime evidence unless a new candidate adopts
the same export path.

### First-prompt history augmentation experiment

`prepare_history_experiment.py` appends the labeled Nucleic first-prompt corpus to
training only. It preserves validation and test byte-for-byte, excludes `vague-eval`
records from optimization, removes exact base/evaluation overlap, and applies the
canonical 0.92 near-duplicate guard against evaluation fixtures and earlier history
records. Every exclusion is represented only by hashes and source line in
`history-exclusions.jsonl`; `manifest.json` binds all input and output hashes.

Build the augmented split and its teacher cache:

```bash
ml/purpose-classifier/venv/bin/python \
  ml/purpose-classifier/prepare_history_experiment.py
ml/purpose-classifier/venv/bin/python -u \
  ml/purpose-classifier/cache_teacher.py \
  --dataset-dir \
    ml/purpose-classifier/.artifacts/dataset-v1-history-first-prompts \
  --model ml/purpose-classifier/outputs/purpose-lite-v1-boundary-tune/model \
  --output \
    ml/purpose-classifier/outputs/purpose-lite-v1-history-first-prompts-teacher.pt \
  --device mps --batch-size 16 --progress-steps 25
```

Then run the same validation-selected MLX recipe as the accepted baseline:

```bash
ml/purpose-classifier/venv/bin/python -u ml/purpose-classifier/train_mlx.py \
  --device metal \
  --dataset-dir \
    ml/purpose-classifier/.artifacts/dataset-v1-history-first-prompts \
  --model ml/purpose-classifier/outputs/purpose-lite-v1-boundary-tune/model \
  --distillation-cache \
    ml/purpose-classifier/outputs/purpose-lite-v1-history-first-prompts-teacher.pt \
  --distillation-weight 0.9 --distillation-temperature 2 \
  --distillation-selection-weight 0.5 --quantization-aware \
  --epochs 4 --early-stopping-patience 1 \
  --learning-rate 1e-6 --warmup-ratio 0 --boundary-weight 1 \
  --progress-steps 1 \
  --output-dir \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-history-v1
```

MLX remains Metal-first. `--device cpu` is an explicit diagnostic fallback for parity
checks and bounded smoke tests; it is not an acceptable full-training path when Metal is
available.

The completed Metal run stopped after epoch two and selected epoch one at 94.42%
fake-quant validation accuracy and a 95.59% teacher-aware selection score. Its selected
float checkpoint improves frozen v1 to **95.09% (891/937)**, two correct decisions above
the accepted baseline's float checkpoint. That gain does not survive export: the matched
23,148,500-byte int8-QDQ graph scores **94.66% (887/937)**, 94.66% macro recall, and
94.17% scored-hard accuracy. It is five correct decisions behind the accepted int8
baseline and fails the 95% shipping gate. Preserve the artifact as a rejected historical
experiment; `purpose-lite-v1-distilled-qat-mlx-4e` remains the accepted v1 artifact only.

## Purpose-deep implementation and historical v1 experiments

The Sol-high v2 base and distilled runs described above supersede the experimental
sequence in this section. Keep the commands and results below for implementation lineage;
do not run another continuation or the large rung from them. The representation probe in
`experiments/sol-high-v2.json` is the current deep next step.

The next classifier tier is an MLX-native ModernBERT multi-task model. It keeps the
primary eight-way purpose output and jointly learns secondary purpose, a mixed-intent
flag, and advisory difficulty. Both upstream rungs are immutable: base is ModernBERT
149M at revision `8949b909ec900327062f0ebf497f51aef5e6f0c8`; large is ModernBERT
395M at revision `45bb4654a4d5aaff24dd11d4781fa46d39bf8c13`.

Before the first run on a new MLX/Transformers version, compare the real pinned backbone
against Hugging Face. The check crosses ModernBERT's local-attention window and fails if
pooled-representation drift exceeds `5e-4`:

```bash
ml/purpose-classifier/venv/bin/python ml/purpose-classifier/verify_deep_mlx.py \
  --variant base
```

Start with the base ablation rung and the validated first-prompt history augmentation.
The trainer downloads the pinned checkpoint on first use, fixes every input at 512 tokens
(`255` head + `254` tail + three special tokens for long prompts), and uses gradient
checkpointing by default. Checkpoint selection is half scored overall accuracy and half
scored hard-slice accuracy; auxiliary heads are reported independently and cannot hide a
primary-purpose regression.

```bash
ml/purpose-classifier/venv/bin/python -u \
  ml/purpose-classifier/train_deep_mlx.py \
  --variant base \
  --dataset-dir \
    ml/purpose-classifier/.artifacts/dataset-v1-history-first-prompts \
  --epochs 3 --early-stopping-patience 1 \
  --progress-steps 10 \
  --output-dir ml/purpose-classifier/outputs/purpose-deep-v1-base-mlx \
  --overwrite-output
```

The defaults use batch size 4 and learning rate `2e-5` for base (2 and `1e-5` for
large). If unified memory is tight, lower `--batch-size` before disabling gradient
checkpointing. `--device cpu` is diagnostic only: a real 512-token backward pass is
expected to be extremely slow there. Each improved epoch atomically rewrites `model/`
and updates `training-state.json`, so progress is visible and an interrupted run retains
the last selected checkpoint.

To continue a completed run without discarding its trained task heads, pass its selected
`model/` directory through `--resume-from` and write to a new output directory.
Continuation restores the backbone and all four heads strictly, then starts a fresh
optimizer and learning-rate schedule; `--model` remains reserved for an untrained local
upstream checkpoint. The first base run was still improving when its three-epoch schedule
ended, so its selected checkpoint was continued conservatively before changing
architecture:

```bash
ml/purpose-classifier/venv/bin/python -u \
  ml/purpose-classifier/train_deep_mlx.py \
  --variant base \
  --resume-from \
    ml/purpose-classifier/outputs/purpose-deep-v1-base-mlx/model \
  --dataset-dir \
    ml/purpose-classifier/.artifacts/dataset-v1-history-first-prompts \
  --epochs 3 \
  --learning-rate 1e-5 \
  --early-stopping-patience 2 \
  --progress-steps 10 \
  --output-dir \
    ml/purpose-classifier/outputs/purpose-deep-v1-base-mlx-cont-3e \
  --overwrite-output
```

That continuation reached 80.31% primary and 78.28% hard-slice validation accuracy;
calibrated mixed F1 reached 67.12%. It remained far below purpose-lite, while primary
training loss and validation accuracy were still improving. Do not chain another plain
continuation. The bounded next experiment distills the mature purpose-lite boundary
teacher into the continued deep checkpoint while retaining direct primary labels and all
three auxiliary losses.

Create a teacher cache bound to the history-augmented split:

```bash
ml/purpose-classifier/venv/bin/python -u \
  ml/purpose-classifier/cache_teacher.py \
  --dataset-dir \
    ml/purpose-classifier/.artifacts/dataset-v1-history-first-prompts \
  --model \
    ml/purpose-classifier/outputs/purpose-lite-v1-boundary-tune/model \
  --output \
    ml/purpose-classifier/outputs/purpose-lite-v1-history-first-prompts-teacher.pt \
  --device mps \
  --batch-size 16 \
  --progress-steps 25 \
  --overwrite-output
```

Then run two validation-selected distilled continuation epochs:

```bash
ml/purpose-classifier/venv/bin/python -u \
  ml/purpose-classifier/train_deep_mlx.py \
  --variant base \
  --resume-from \
    ml/purpose-classifier/outputs/purpose-deep-v1-base-mlx-cont-3e/model \
  --dataset-dir \
    ml/purpose-classifier/.artifacts/dataset-v1-history-first-prompts \
  --distillation-cache \
    ml/purpose-classifier/outputs/purpose-lite-v1-history-first-prompts-teacher.pt \
  --distillation-weight 0.5 \
  --distillation-temperature 2 \
  --epochs 2 \
  --learning-rate 1e-5 \
  --early-stopping-patience 1 \
  --progress-steps 10 \
  --output-dir \
    ml/purpose-classifier/outputs/purpose-deep-v1-base-mlx-distilled \
  --overwrite-output
```

The trainer records the resumed checkpoint as epoch zero before updating anything, so a
distillation regression cannot overwrite the 80.31% candidate. Teacher agreement is
reported for diagnosis but does not enter deep checkpoint selection; overall and hard
primary label accuracy remain the only selection inputs.

### Pause and resume purpose-deep training

`train_deep_mlx.py` handles both `Ctrl-C` (`SIGINT`) and `SIGTERM` gracefully. Press
`Ctrl-C` once: the current optimizer step finishes, then the trainer atomically saves the
current model, AdamW state and learning-rate cursor, epoch and next batch, exact shuffle
order, accumulated losses, RNG state, best-selection state, and history under
`<output-dir>/resume/`. It exits with the conventional signal-derived status only after
that checkpoint is durable. A second `Ctrl-C` forces an immediate interruption.

The pause message prints the complete resume command. For the distilled run above it is:

```bash
ml/purpose-classifier/venv/bin/python -u \
  ml/purpose-classifier/train_deep_mlx.py \
  --resume-training \
    ml/purpose-classifier/outputs/purpose-deep-v1-base-mlx-distilled/resume
```

`--resume-training` is sufficient by itself: it restores the original dataset,
distillation, schedule, loss, and selection arguments from the checkpoint. It also verifies
hashes of the training split, validation split, and teacher cache before loading. The
checkpoint resumes at the next unprocessed batch—even if stopped after the last training
batch but before epoch evaluation. Normal completion removes the large optimizer resume
artifact while retaining the selected `model/` and reports.

This applies to processes started with the updated script. A process already running older
code cannot acquire signal handling retroactively.

The historical large-rung rule required base to qualify first, large to beat base by at
least two hard-slice points, and deep to reach 97% scored overall while beating lite by
five hard-slice points. Neither the v1 nor Sol-high v2 base result unlocked that rung.

### Convert and validate Core ML

Core ML Tools no longer maintains the legacy ONNX converter, so the Apple artifact is
converted directly from the selected Hugging Face checkpoint. `convert_coreml.py` uses a
fixed-shape export-only BERT forward to avoid dynamic Transformers masking helpers, checks
that forward against Transformers before conversion, writes an ML Program package, and
records hashes for every package file.

Install the pinned converter in the macOS environment and create the package:

```bash
ml/purpose-classifier/venv/bin/python -m pip install \
  -r ml/purpose-classifier/requirements-coreml.txt
ml/purpose-classifier/venv/bin/python ml/purpose-classifier/convert_coreml.py \
  --model-dir \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e/model \
  --output \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e/coreml/purpose-lite-v1-fp16.mlpackage \
  --overwrite-output
```

The direct float16 package is the conversion baseline, not the accepted Apple artifact.
On the first physical Apple-Silicon run it scored 94.98% (890/937), two correct decisions
behind the accepted ONNX graph, with 97.97% scorable label agreement. Calibrate a
Core ML-native W8A8 candidate with the same deterministic 256-record sample and QDQ policy
as the ONNX exporter:

```bash
ml/purpose-classifier/venv/bin/python ml/purpose-classifier/quantize_coreml.py \
  --model \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e/coreml/purpose-lite-v1-fp16.mlpackage \
  --model-dir \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e/model \
  --output \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e/coreml/purpose-lite-v1-w8a8.mlpackage \
  --overwrite-output
```

Activation calibration writes its temporary packages under the candidate output directory
and removes each package immediately after prediction; this avoids Core ML Tools retaining
one full weight copy per calibration step until process exit. It prints progress while it
runs. The successfully rewritten A8 package is cached beside the W8A8 output and reused
only when its source hash, Core ML Tools version, activation policy, calibration seed, and
prompt hashes match exactly. This prevents a later weight-stage failure from forcing
another calibration. The candidate uses per-tensor asymmetric uint8 activations,
per-channel symmetric int8 linear weights, and per-tensor asymmetric uint8 embedding
weights. Activation quantization is limited to floating-point linear operations; applying
Core ML Tools' global policy also selects integer embedding-index additions and produces
an invalid quantize operation. It fails the command if the resulting package exceeds
25 MiB.

Run the frozen gate with CPU+Neural Engine placement and compare labels directly with the
accepted int8 ONNX artifact. Gated Core ML evaluation fails closed without
`--compare-onnx`, and requires at least 99.5% scorable label agreement:

```bash
ml/purpose-classifier/venv/bin/python ml/purpose-classifier/eval.py \
  --model-dir \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e/model \
  --calibration \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e/calibration.json \
  --coreml-model \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e/coreml/purpose-lite-v1-w8a8.mlpackage \
  --coreml-compute-units cpu-and-ne \
  --compare-onnx \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e/export/purpose-lite-v1-int8-qdq.onnx \
  --report \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e/coreml/ane-frozen-eval.json
```

Record the compute plan separately; this reports both operation-count and estimated-cost
ANE shares. Repeat evaluation with `--coreml-compute-units cpu-only --no-gate` before the
energy comparison in `ENERGY_AND_RESIDENCY.md`:

```bash
ml/purpose-classifier/venv/bin/python ml/purpose-classifier/inspect_coreml.py \
  --model \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e/coreml/purpose-lite-v1-w8a8.mlpackage \
  --compute-units cpu-and-ne \
  --report \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e/coreml/ane-compute-plan.json
```

For a wiring smoke test, use a small deterministic prefix:

```bash
ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/train.py \
  --epochs 1 --max-train-records 64 --max-validation-records 64 \
  --output-dir ml/purpose-classifier/outputs/smoke --overwrite-output
```

## Evaluate

```bash
ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/eval.py
```

The command returns failure unless label-scorable frozen accuracy is at least 95%, every
purpose recall is at least 85%, at least 90% of the deliberately context-free
`vague-eval` slice resolves LOW, every misroute stays within one routing cost tier, and
measured batch-one p95 is at most 20 ms. Use `--no-gate` only for diagnostic runs.

## Export and score ONNX

```bash
ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/export.py
ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/eval.py \
  --onnx-model ml/purpose-classifier/outputs/purpose-lite-v1/export/purpose-lite-v1-int8-qdq.onnx \
  --report ml/purpose-classifier/outputs/purpose-lite-v1/export/int8-frozen-eval.json
```

`export.py` emits fixed-shape opset-17 fp16 and int8-QDQ graphs, a tokenizer/
normalization contract, golden tokenizations, shared calibration config, graph checks,
artifact hashes, and a size report. Its default 256-record quantization calibration sample
is deterministic and stratified by purpose, slice, and primary language; the export report
records the seed, distribution, and prompt hashes. The int8 graph is the ≤25 MiB shipping
candidate; the fp16 graph remains the accelerator-oriented conversion input.

When scoring ONNX, add `--compare-pytorch` to measure artifact drift against
`--model-dir`. The report then includes overall, label-scorable, and per-slice label
agreement plus every correct→incorrect, incorrect→correct, and changed-wrong-label
transition:

```bash
ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/eval.py \
  --onnx-model ml/purpose-classifier/outputs/purpose-lite-v1/export/purpose-lite-v1-int8-qdq.onnx \
  --compare-pytorch --no-gate
```

## Audit curation

Run the semantic embedding duplicate audit. It also emits the deterministic,
purpose/slice/language-stratified 10% human label-and-difficulty review CSV:

```bash
ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/audit_data.py
```

The semantic pass uses the same commit-pinned MiniLM encoder and fixed 128-token input as
`purpose-lite`. Similarity only proposes review candidates; it never edits source data or
the frozen split automatically. The 18 word-trigram exclusions and the human-review
completion rule are recorded in `data/curation-review-v1.json`; the semantic report is
versioned as `data/semantic-audit-v1.json`.

## Optional human review

The dataset owner accepted the curated generated labels and difficulty metadata as-is on
2026-07-31, so the blank 1,219-row review sample is not a training or rollout blocker. It
remains available as an optional future audit. Check its progress without running the
embedding audit again:

```bash
ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/review_data.py
```

Mark each row `accept`, `relabel`, or `reject`. `accept` and `reject` leave the four
`reviewed*` fields blank; `reject` requires notes. For `relabel`, blank reviewed fields
retain their generated value, `<none>` clears a secondary purpose, and notes are required.
If a secondary purpose is added or removed, set `reviewedSlice` consistently (`mixed`
when a secondary is present). The validator rejects stale generated columns, missing or
duplicate sample rows, invalid label combinations, and partially completed rows.

When every row has a human decision, write the versionable ledger:

```bash
ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/review_data.py --finalize
```

Build an isolated candidate split first:

```bash
ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/prepare_data.py \
  --human-review ml/purpose-classifier/data/human-review-v1.json \
  --output-dir ml/purpose-classifier/.artifacts/reviewed-candidate \
  --frozen-test ml/purpose-classifier/.artifacts/reviewed-frozen-candidate.jsonl \
  --manifest ml/purpose-classifier/.artifacts/reviewed-manifest-candidate.json \
  --refresh-frozen-test
```

Inspect the ledger, decision summary, candidate manifest, and split diff. Only then rerun
the same command with the three candidate-path overrides removed to intentionally replace
the versioned frozen dataset and manifest.

`--regenerate` recreates a blank CSV in the current schema and is only appropriate before
review begins.

The one-time, hardware-bound energy and accelerator-residency procedure is in
`ENERGY_AND_RESIDENCY.md`.