Corpus import 2026-08-19
Historical record — 2026-08-19
Preserved as observed. Current truth: Program Masterplan.
Headline, all measured on the bench
The corrected dedupe ledger left 60 importable rows (see question corpus forensics, correction 2026-08-19). Pack generation and the deployed importer accepted 53 of them across 5 packs; 7 were rejected with named reasons and 2 were duplicate ids inside their own source file. Published questions 679 → 732, read index 679 → 732 (drift 0),
lang='en'rows 508 → 561, question-subject rows 908 → 978,qw_subjectterms 456 → 483, draft quizzes 10 → 15. Probe after the run:1..46,assertions=46 failures=0.
What was imported
Generator: 05 EXPORTS/2026-08-19-corpus-forensics/build_import_packs.py. It recomputes the survivor set from dedupe_ledger.py rather than trusting a copied list, maps each bank per schema.json, then validates every question against the deployed floor before writing a pack. Packs and the manifest are kept in 03 EVIDENCE/2026-08-19-corpus-import/.
| Pack | Bank | Questions | Subject rows in pack | Bytes | Draft quiz |
|---|---|---|---|---|---|
| atlas-physci-easy-en.json | atlas_physci_easy | 24 | 1 | 9,674 | 2253 |
| gold1008-en.json | gold1008 | 9 | 14 | 6,882 | 2263 |
| gold504-en.json | gold504 | 5 | 10 | 3,835 | 2269 |
| wave01-168-en.json | wave01_168 | 3 | 6 | 2,236 | 2277 |
| qs-json-en.json | qs_json | 12 | 5 | 4,816 | 2322 |
Every pack is quizwizz.question-pack v1 with exactly the five accepted top-level keys, one language per pack (en), and questions carrying only type, stem, choices, correct_idx, explanation, difficulty, subjects — the deployed importer rejects any other question key (includes/Import/QuestionPackImporter.php:102-104). Questions publish and each pack’s quiz is created as a draft (includes/Import/QuestionImportPipeline.php:138,146).
What was rejected, and why
| Bank | Row | Reason |
|---|---|---|
| qs_json | q04, q05, q12, q27 | multi-answer row: 3 correct choices, validator allows one |
| qs_json | q18, q24, q32 | multi-answer row: 4 correct choices, validator allows one |
| atlas_physci_easy | physical-sciences-easy-556470dcc9 | duplicate row id in source file |
| atlas_physci_easy | physical-sciences-easy-dc78d308d5 | duplicate row id in source file |
The seven qs_json rows are a ledger overcount, now measured: Qs.json labels a multi-select row type: "multiple", and the dedupe ledger’s renderability gate maps that string straight to the plugin’s multiple. The deployed validator wants exactly one correct_idx (includes/Import/QuestionValidator.php:80-87), so a row with three or four correct options has no renderable home. The corrected importable figure for these banks is therefore 53, not 60; the seven rows join the unrenderable ledger rather than the import.
Falsifier: the deployed importer accepts a qs_json multi-answer row without an editorial rewrite.
Defect found and fixed during the run: qw_question_store on any backslash
The qs_json pack failed its first import with qw_question_store (“The question could not be saved safely”), rolling the whole pack back. Bisected to one row:
Evaluate: limit as x approaches 0 of sin(x)/x. — explanation $\lim_{x→0} sin(x)/x = 1$; …
Measured cause, one wp eval-file:
written : $\lim_{x→0} sin(x)/x = 1$; it is the standard squeeze-theorem limit.
read_back: $lim_{x→0} sin(x)/x = 1$; it is the standard squeeze-theorem limit.
match : false
update_post_meta() unslashes what it is given, exactly as wp_insert_post() does for post_title/post_content. QuestionService::persist() slashed the post fields but not the meta, so a single backslash was dropped on write and QuestionService::verify() — which re-reads the stored meta and compares it to the validated input (includes/QuestionService.php:183-201) — could never match. Every question whose stem, choices or explanation contains a backslash was unstorable, and the failure surfaced as an opaque 500-class import error.
Fix, landed through the bench protocol: slash the string metas on the way in (qw_choices, qw_explanation, qw_source) and slash restored snapshot values in QuestionService::restore() so a rollback is byte-identical. Nothing else changed; the validator, the pack schema, and the mapping were not loosened.
Verification: the same pack imported cleanly on the next run (inserted: 12), and the four earlier packs correctly reported inserted: 0, skipped: N because their qw_source rows were already live — those four redundant draft quizzes were deleted, leaving one draft quiz per pack. Probe 1..46 / 46 / 0.
Before and after
| Measure | Before | After |
|---|---|---|
published qw_question | 679 | 732 |
wp_qwizz_question_index rows | 679 | 732 |
index rows lang='en' | 508 | 561 |
wp_qwizz_question_subjects rows | 908 | 978 |
qw_subject terms | 456 | 483 |
draft qw_quiz | 10 | 15 |
Content count equals index count, so there is no drift to report. The 27 new subject terms are the path-slug pairs the packs declared (animals-and-nature + animals-and-nature-marine-life style), created by the importer’s own subject pass.
Falsifier: a fresh count shows published ≠index, or any imported question missing from wp_qwizz_question_index.
Consequence for the ledger
Re-running dedupe_ledger.py now reports 0 survivors for these banks minus the 7 multi-answer rows, because the 53 rows are live and match by stem hash. That is the intended end state: the ledger measures what is left to import, not what was ever available.
See the completed task log, row “Corpus import of the survivors”.