Merge nucleic/plucky-north-vole-sdna into dev
This commit is contained in:
@@ -60,6 +60,40 @@ project rules, and is instructed not to use tools. `--codex-isolation auto` uses
|
||||
read-only isolation on a host and the existing outer isolation when the script runs in a
|
||||
Nucleic managed container.
|
||||
|
||||
## SWE-chat v2 import (gated source)
|
||||
|
||||
The SWE-chat source is not downloaded by this repository. After accepting the dataset's
|
||||
Hugging Face conditions, place a **pinned** Parquet snapshot below the ignored
|
||||
`.artifacts/swe-chat/raw/` directory, record its immutable revision, then run the
|
||||
streaming extractor. It reads only the needed columns, takes the first three qualifying
|
||||
human prompts per session, and writes the first prompt plus hashes for the two context
|
||||
turns. Do not use `main` as a revision.
|
||||
|
||||
```bash
|
||||
ml/purpose-classifier/.venv/bin/pip install -r \
|
||||
ml/purpose-classifier/requirements-swe-chat.txt
|
||||
ml/purpose-classifier/.venv/bin/python \
|
||||
ml/purpose-classifier/export_swe_chat.py \
|
||||
--revision <accepted-immutable-hf-revision>
|
||||
```
|
||||
|
||||
The export and manifest remain ignored because candidate JSONL temporarily contains all
|
||||
three messages. Run the one-record schema/availability canary before the 100-session dry
|
||||
run; both use Luna through subscription-backed `codex exec`, not an API key. The labeler
|
||||
writes only the first message to canonical source JSONL; state and audit sidecars retain
|
||||
the other turns solely as hashes and source IDs.
|
||||
|
||||
```bash
|
||||
ml/purpose-classifier/.venv/bin/python \
|
||||
ml/purpose-classifier/label_swe_chat_prompts.py --limit-sessions 1
|
||||
ml/purpose-classifier/.venv/bin/python \
|
||||
ml/purpose-classifier/label_swe_chat_prompts.py --limit-sessions 100
|
||||
```
|
||||
|
||||
Use a fresh `--output` path for the dry run, then manually audit it before invoking the
|
||||
full resumable run. Cases marked `recoverableFromFirst=false` remain `vague-eval`
|
||||
abstention evidence and are excluded from optimization.
|
||||
|
||||
## Prepare
|
||||
|
||||
From the repository root:
|
||||
|
||||
Reference in New Issue
Block a user