🧠 Question corpus strategy

Authoring companion — 2026-08-28

New-question writing rules, per-category matrix, and dual acceptance gates (playability + coverage ledger) live in Question authoring strategy.

Superseded — 2026-08-19

The 679-question / 456-subject live bench figures on this page were measured 2026-08-18 and are superseded by the 2026-08-19 legacy WXR import, which closed at 9,069 questions and 483 subject terms. The harvest inventory figures are unaffected. Preserved as observed; current state: 📍 Now. Corpus overview: 📚 Question corpus.

Authority

This page is the corpus contract for the WRDP bench: it turns the owner’s scattered “pearls before swine” workbooks into one canonical source that can serve current QuizWizz and later named consumers. It is bench documentation, not a release claim. Program context: program masterplan; runtime evidence: Service Operations.

What this contract decides

• Reach for the stars in data shape, not premature UI/code. Preserve enough meaning for many question types, locales, variants, and games not invented yet.
• YAGNI boundary: define a lossless-enough canonical record now; implement a new type only when a named game or screen needs it.
• One source of truth: datasets are inputs; qid records are the durable corpus; packs, PDFs, and game payloads are regenerated adapters.
• No stranded good data: anything richer than today’s plugin gets a stated, valid pack-v1 projection; the projection may be lossy, never silently fictional.
• No invented quality: gold, usable, and reject remain evidence-backed buckets, not aspirations.

Where we are, measured

Harvest inventory

Measure	Measured state	Decision consequence
Raw harvested rows	4,725	Preserve provenance before deduplication.
Unique rows	1,432	The starting candidate set after current deduplication.
Gold records	1,388	Default source for curated pack emission.
Usable records	31	Keep separate; do not silently promote to gold.
Reject records	13	Keep reasons in rejects.csv; do not delete traceability.
English gold	1,221	English is the only meaningful editorial base today.
de / fr / es	56-subject translation fixture, one question per subject	Fixture coverage, not editorial content. All rows originate in 02 DOCS/questions.json.
Coverage nesting	Gold 360 ⊂ 504 ⊂ 1008 at 100%	Keep the verified nested coverage sets; Gold 240 is 239/240 covered.
Duplicate workbooks	Wave_01 (1) and 10k _dfq are byte-identical	Hash once; treat one as a duplicate source, not independent evidence.

Existing question-harvest artifacts:

Artifact	Role	Keep / change
_harvest/questions/inventory.csv	Source inventory and identity evidence	Keep as pipeline input ledger.
corpus.normalized.jsonl	Normalized candidate records	Make this the canonical-record stream.
corpus.gold.jsonl	Gold subset	Emit only records with a passing quality gate.
corpus.by-language/	Locale partitions	Regenerate from locale; do not author divergent copies.
duplicates.csv	Deduplication decisions	Preserve winner, loser, rule, and review status.
rejects.csv	Rejection decisions	Preserve input identity and failure name.
_harvest/packs-live/	Bench-installable pack outputs	Treat as generated adapter output.
_harvest/packs/validate_packs.py	Pack-shape validation	Run before every v1 emission.

Live bench state

Live measure	Measured state	Implication
Published questions	679	Installed content is a subset/projection, not the corpus authority.
Question index	679	Index is aligned to the published count at measurement time.
Published-language distribution	en 508; de 57; es 57; fr 57	The 679 published records are tagged in four languages; language is record metadata, not a corpus fork.
qw_subject terms	456	Subject resolution is a live taxonomy constraint.
Imported draft quizzes	5: 1556, 1657, 1758, 1859, 1960	Import creates drafts; publishing is a separate editorial decision.
Current deployable types	multiple, boolean only	Richer forms are schema questions, not data questions.
Current answer rule	Exactly one correct_idx; 2–8 choices	Every pack-v1 projection must satisfy this validator shape.

Today’s validation comes from includes/QuestionValidator.php: single coerces to multiple; medium coerces to normal; choices must be non-empty, distinct after normalization, and ≤1,000 characters; explanations are ≤4,000 characters; difficulty is easy|normal|hard.

Warning

The current plugin model cannot represent multiple correct answers, ordering, blanks, numerical tolerance, matching, or open response. Those are not “bad rows” in the corpus; they are unsupported consumer capabilities.

Canonical model: the record that outlives a game

One line in corpus.normalized.jsonl is one stable record. The storage form may be JSONL, database rows, or an export bundle later; field semantics do not change. qid is immutable once issued. concept_id binds the language-independent idea, its translations, and its intentional gameplay variants without claiming that wording is interchangeable.

Field	Meaning	Required now?	Today’s pack-v1 mapping / status
qid	Stable opaque identifier for this exact record; never recycled.	Yes	Inert in v1; retain in adapter manifest/provenance.
concept_id	Language-independent idea binding locale records and variants.	Yes	Inert in v1; adapter emits one locale/variant as a question.
type	Canonical type from the type ladder.	Yes	multiple / boolean map directly; all other types project through an explicit downgrade.
prompt	Localized learner-facing question text.	Yes	Maps to pack question text.
answer	Typed, canonical right-answer object; shape depends on type.	Yes	Adapter materializes one correct_idx and choice list where valid.
distractors	Candidate wrong answers with rationale, source, and quality status.	Yes for choice projections	Adapter selects 1–7 to produce the allowed 2–8 choices.
explanation	Localized feedback explaining the answer.	Yes when available	Maps to explanation; cap at 4,000 characters for v1.
subject_path	Ordered canonical subject taxonomy path, not a display-only label.	Yes	Adapter resolves leaf to a real qw_subject slug.
difficulty	easy, normal, or hard delivery band.	Yes	Maps directly; no medium output.
calibration	Evidence behind difficulty: e.g. reviewer rating, pilot accuracy, sample size, and confidence.	Reserved but populate when known	Inert until a consumer uses calibrated adaptation.
locale	Content locale such as en, de, fr, es, or a BCP-47-style tag.	Yes	Maps to v1 lang if valid.
variant_of	qid of a deliberately different form of the same concept, or null.	Yes	Inert; adapter chooses one record.
source	Source dataset, row identity, extraction time, transformations, and attribution.	Yes	Inert in payload; retained in manifest/audit artifacts.
license	Reuse permission and restrictions for the record/source.	Yes	Inert to plugin, mandatory for external export decisions.
quality	gold, usable, or reject.	Yes	v1 gold export consumes gold only unless explicitly overridden.
quality_flags	Machine and editorial warnings; never overwritten by an export.	Yes	Inert except as an export gate.
media	Optional typed slots: image, audio, video, document, alt text, rights.	Reserved	Inert; a future consumer must name the media slot it supports.
checks	Named validations passed/failed, versions, timestamps, and reviewer evidence.	Yes	Inert in payload; mandatory pipeline audit trail.

Canonical record invariants

• A record has one qid, one concept_id, one locale, one type, and one declared quality bucket.
• Translation creates another qid with the same concept_id; rewording for a gameplay mechanic creates another qid with variant_of set where appropriate.
• A source row can produce zero, one, or multiple records; every result retains source lineage.
• A record may be canonical yet not exportable to v1. An adapter must report the projection/downgrade; it must not erase unsupported semantics from the canonical source.
• quality_flags, checks, source, and license are retained even when a downstream consumer cannot express them.

Example shape (illustrative fields, not invented corpus content)

{
  "qid": "q-…",
  "concept_id": "c-…",
  "type": "multiple",
  "prompt": "…",
  "answer": { "kind": "choice", "value": "…" },
  "distractors": [{ "value": "…", "provenance": "authored", "quality": "approved" }],
  "subject_path": ["…", "…"],
  "difficulty": "normal",
  "calibration": null,
  "locale": "en",
  "variant_of": null,
  "source": { "dataset": "…", "row": "…" },
  "license": { "status": "…" },
  "quality": "gold",
  "quality_flags": [],
  "media": [],
  "checks": ["shape", "dedupe", "distractor"]
}

Type ladder: schema depth now, consumer code on demand

multiple and boolean ship today. Every other row below is canonical data that waits for its authorising trigger. A v1 downgrade is only valid if the adapter can produce 2–8 distinct non-empty choices and exactly one correct_idx; otherwise it reports no_safe_v1_projection and does not export.

Type	answer object	Scoring	Missing plugin capability	Authorising trigger	Lossy-but-valid downgrade to pack v1
boolean	`{kind:“boolean”, value:true	false}`	Exact truth value	None	Current QuizWizz boolean editor/player
multiple	{kind:"choice", value:"canonical option"}	Exact selected option	None	Current QuizWizz multiple editor/player	Emit answer plus 1–7 approved distractors; one correct index.
multi_correct	{kind:"choice_set", values:[…], all_required:true}	Set equality or configured partial credit	Multi-select UI, storage, validator, player scoring	A named multi-select game/screen	Ask one derived member fact as multiple; record projection=single_member, never imply the full set was tested.
order	{kind:"sequence", values:[…], ordering:"…"}	Exact sequence or distance-based configured score	Ordered-input authoring, persistence, interaction, scoring	A named ordering/sorting game/screen	Ask a derived adjacent-order or first/last fact as multiple; retain original sequence in canonical data.
cloze	{kind:"blanks", text:"…", blanks:[{id, answers:[…]}]}	Per-blank normalized answer matching	Blank renderer, accepted-answer storage, scoring	A named fill-in-the-blank game/screen	Turn one blank into a multiple question with approved alternatives; mark which blank was projected.
numeric	{kind:"number", value:…, unit:"…", tolerance:{…}}	Exact/tolerance/range with unit normalization	Numeric input, tolerance validation, numeric scorer	A named estimation/calculation game/screen	Offer the correct value and coherent numeric distractors as multiple; state unit in prompt/options.
date	`{kind:“date”, value:“YYYY-MM-DD"	"YYYY”	range}`	Exact precision-aware date/range comparison	Date input, precision model, temporal scoring
math	{kind:"expression", canonical:"…", accepted:[…], domain:"…"}	Symbolic equivalence or evaluated tolerance	Math renderer/input, equivalence engine, scorer	A named maths game/screen	Present one evaluated/result answer plus plausible results as multiple; do not claim algebraic equivalence was assessed.
match	{kind:"pairs", pairs:[{left,right}]}	Pair-set equality or partial credit	Pairing interaction, pair persistence, scorer	A named matching game/screen	Project one pair as multiple: prompt supplies left side; options are candidate right sides.
open	{kind:"rubric", expected:[…], rubric:[…]}	Human rubric, model-assisted rubric, or keyword policy	Text input, safe evaluation policy, review workflow	A named free-response game/screen	Project one factual rubric criterion into multiple; flag the projection as not assessing open expression.

Tip

A projection is an adapter product, not a replacement record. Keep source_qid, projection_kind, and dropped_semantics in the export manifest so later consumers can restore the intended type.

Distractor policy: infinite candidates, finite approved options

The corpus can retain a large distractor pool per concept; a pack-v1 emission selects at most seven wrong answers because one correct answer plus distractors must remain within 2–8 choices.

Gate	Rule	Failure name
Category	Source wrong answers from the same semantic category, grammatical form, and expected-answer class as the right answer.	distractor_category_mismatch
Plausibility	A competent but mistaken learner could choose it; trivia, joke, and visibly random options fail.	distractor_implausible
Length parity	Correct answer must not be markedly longer than its alternatives; rewrite or replace conspicuous outliers.	distractor_length_cue
Numeric/date coherence	Preserve unit, magnitude scale, precision, and temporal granularity; no kilometres beside dates or year beside day precision.	distractor_unit_or_precision_mismatch
Overlap	No duplicate after normalization; no answer-containing distractor, no distractor-containing answer where that leaks the answer, and no ambiguous near-synonym unless intentionally accepted.	distractor_forbidden_overlap
Answer uniqueness	Exactly one option remains correct under the prompt and locale.	distractor_ambiguous_correctness
Count	Retain any number of candidates canonically; choose only 1–7 approved distractors for v1.	distractor_v1_count
Provenance	Every distractor records authored, source-dataset identity, or generator/model/version plus review evidence.	distractor_missing_provenance

Generated distractors are not lesser data and are not automatically accepted: they carry generator provenance, enter the same plausibility/overlap/uniqueness checks, and require the same quality gate as authored distractors. “Infinite distractors” means a reusable candidate pool, never an unbounded option count on one screen.

Languages: a record field, not a programme fork

The owner direction recorded on 2026-08-18 is to fuse many datasets—on the order of ten thousand questions, across mixed languages and categories, with some material already translated—into one premium gold corpus that covers “all shapes and sizes.” German translation is not a priority or a separate programme. It may emerge later from the fused corpus where a valid source record and the normal quality gates support it.

• concept_id binds translations and intentional locale variants. A translated record receives a new qid, retains the same concept_id, and carries its own prompt, answer, distractors, explanation, checks, provenance, and quality evidence.
• Content locale and interface locale are separate axes. A question can have locale=en while an administrator or player interface uses another language. Content availability comes from accepted corpus records; an offered interface still depends on its readable catalog.
• The live distribution measured 2026-08-18 is en 508, de 57, es 57, and fr 57 across 679 published questions. The earlier harvested fixture remains evidence of limited non-English coverage, not a mandate to create divergent language corpora.
• A translated or imported record is only gold after the same source-lineage, deduplication, license, type, subject, and editorial gates as every other input. No content flag alone unlocks a locale.

Note

The historical translate-vs-author-vs-import choice is superseded by the owner direction dated 2026-08-18. Translation, native authoring, and locale-specific imports remain possible intake methods inside the one fusion pipeline; none is a standalone language programme.

Fusion intake pipeline: one source stream, many safe packs

This is the concrete, rerunnable path from every candidate dataset to the premium gold corpus. It may add new qid values; it MUST NOT repurpose an existing qid for different content. Reruns may refine checks, flags, and derived outputs while preserving each source identity and decision.

Stage	Input / output	Required action
1. Inventory sources	Source ledger	Register each dataset, source-row identity, extraction timestamp/version, reuse status, and content hash before any editorial merge. Byte-identical inputs share one source identity rather than pretending to be independent evidence.
2. Normalise	Canonical-record stream	Map each source row to the canonical record schema: stable qid, concept_id, type, prompt/answer/distractors, subject path, difficulty, locale, provenance, license, quality, flags, and checks. Preserve unmapped source data in provenance.
3. Detect duplicates	Duplicate-decision ledger	Compare normalized records across all datasets using language-aware prompt, answer, type, concept, and source signals. Cluster candidates; retain winner, loser, rule, reviewer state, and all provenance. A match signal is never permission to delete history.
4. Score quality	Gold / usable / reject buckets	Apply shape, factual, license, locale, type, subject, distractor, and editorial gates. Record named failures and reviewer evidence; only passing records become gold.
5. Partition by language	Language partitions	Derive language partitions from each record’s locale; do not maintain separately authored corpus branches. Ambiguous or undetected language cannot enter gold.
6. Emit pack-v1 adapters	Importable question-pack v1 files	Emit only multiple and boolean records with a safe projection. Each file has exactly format, version, quiz, subjects, and questions; has no more than 100 questions, 100 subjects, and 1 MB; and uses only real qw_subject slugs. Retain source/projection manifests outside the payload.
7. Validate and retain evidence	Validated adapter output	Validate each emitted pack against the live importer contract before intake. Keep the pack, validation result, source qid, projection kind, and dropped semantics so later consumers can restore canonical meaning.

The live ceiling was measured on 2026-08-18, not assumed: 679 published qw_question records, 679 question-index rows, language tags of 508 en, 57 de, 57 es, and 57 fr, plus 456 qw_subject terms. Those figures describe the installed projection and taxonomy; they are not a ceiling on the fused corpus, and every emitted subject must resolve to one of the live term slugs.

Warning

The plugin currently supports only multiple and boolean. Richer canonical types remain valid corpus data, but no richer type is built until a named authorising consumer (game or screen) requires its authoring, storage, player, and scoring contract.

Export adapters: consumers take projections

Consumer	What it can express	What it silently drops unless the adapter records it
QuizWizz pack v1	multiple / boolean, one correct index, 2–8 choices, explanation, difficulty, language, subject relation	concept_id, variants, calibration, licenses, media, multiple answers, order, blanks, tolerance, rubrics, and all extra distractors.
Future pack v2	Only capabilities explicitly specified by its schema; start from the canonical type ladder	Any canonical field not given an explicit v2 contract; v2 is not assumed merely because it is named.
Print/PDF sheet	Prompt, selected options, answer key, explanation, subject/difficulty labels where layout permits	Interactivity, media behavior, adaptive calibration, rich scoring, and distractor provenance.
Party/game mode	The named game’s supported prompt, media, answer, timer, and scoring payload	Unsupported types/fields; it must declare a projection rather than consume “generic questions.”
External SCORM-ish or CSV consumer	Flat prompts, answers, choices, locale, subject, and basic result fields where its schema allows	Nested answer semantics, provenance, licenses, checks, media metadata, calibration, and game-specific behavior.

QuizWizz pack-v1 adapter contract

Constraint	Required adapter behavior
Top-level shape	Emit exactly format, version, quiz, subjects, questions.
Version	Emit quizwizz.question-pack v1.
Limits	Split output at ≤100 questions, ≤100 subjects, and ≤1 MB per pack.
Current question shape	Emit only multiple or boolean with 2–8 valid choices and exactly one correct index.
Import side effect	Expect questions to publish while the quiz is created as a draft.
Subject rule	Resolve every emitted subject to a real qw_subject term and emit its canonical slug.
Installed canonicalisation	includes/QuestionService.php::persist() now resolves subject slug → numeric term ID → exact name and stores the term’s canonical slug; name-vs-slug packs no longer fail as generic qw_question_store.

The importer contract is in includes/Import/QuestionPackImporter.php. The installed subject canonicalisation was proven with the acceptance matrix; ./scripts/quizwizz-probe.sh --wave all reported 1..46, assertions=46 failures=0 on 2026-08-18.

Quality gates: gold means safe, useful, and importable

Machine gates decide eligibility; human review decides editorial intent, factual nuance, cultural fit, and whether a plausibly valid item belongs in a curated corpus.

Gate	One-line rule	Failure name	Bucket effect
Source identity	Dataset, row/path, extraction transform, and attribution are present.	source_lineage_missing	Not gold.
License	Reuse status is present and compatible with the intended adapter.	license_unknown_or_incompatible	Usable or reject; never gold for export.
Locale	locale matches /^[a-z]{2,3}(?:-[a-z0-9]{2,8})*$/ and language is not unresolved.	locale_invalid_or_undetected	Not gold.
Type	Canonical type is one type-ladder member.	type_unknown	Reject.
Prompt	Localized prompt is non-empty and unambiguous enough for its declared type.	prompt_empty_or_ambiguous	Reject or editorial review.
Choice shape	Any v1-targeted record has 2–8 non-empty choices, distinct after normalization, each ≤1,000 characters.	v1_choice_shape	Cannot be gold for v1.
Correctness	A v1-targeted record resolves to exactly one correct_idx; richer types retain their native answer object and projection status.	v1_correct_index	Cannot be gold for v1.
Explanation	v1-targeted explanation is ≤4,000 characters.	v1_explanation_length	Cannot be gold for v1 until fixed.
Difficulty	Delivery band is easy, normal, or hard; calibration evidence is recorded when available.	difficulty_invalid	Not gold.
Subject	subject_path is non-empty; v1 leaf resolves to a real canonical qw_subject slug.	subject_unresolved	Cannot be gold for v1.
Duplicate	No unresolved duplicate cluster remains after folded-prompt analysis and review.	duplicate_unresolved	Usable pending decision.
Distractors	Chosen v1 distractors pass category, plausibility, parity, overlap, coherence, and uniqueness gates.	distractor_gate_failed	Cannot be gold for v1.
Semantic accuracy	Editorial reviewer confirms answer, explanation, scope, and locale wording against source evidence.	editorial_review_required	Usable pending review.
Pack emission	Generated v1 pack passes _harvest/packs/validate_packs.py and importer shape limits.	pack_adapter_invalid	Block emission.

These v1 shape gates mirror includes/QuestionValidator.php, so a record cannot be labeled gold-for-v1 while simultaneously unimportable. The pack importer adds its own exact-key and size/quantity constraints; adapter validation must test both layers.

Roadmap: fusion first, consumer capability on demand

Wave	Scope	Exit criterion	Artifact produced	Gate
1	Fuse the candidate datasets into one canonical corpus. Inventory, normalize, preserve provenance, deduplicate, tag language, and score quality across all inputs.	Every accepted record has canonical shape, source lineage, license status, quality evidence, and an unresolved-duplicate decision.	Canonical corpus, source ledger, duplicate decisions, and quality/reject evidence.	Required now.
2	Emit and validate current-consumer adapters.	Gold multiple / boolean records with resolved real subject slugs produce pack-v1 files within the live importer limits.	Per-language pack-v1 files and projection manifests.	Required for live intake.
3	Improve breadth and quality where gates identify deficiencies.	Named quality gaps are resolved without losing provenance or duplicate decisions; regenerated adapters remain valid.	Updated checks, flags, editorial evidence, and gold partitions.	Evidence-driven, not language-specific.
4	Implement one richer type end to end.	A concrete named game or screen exists, with native authoring, schema, player, scoring, and verified adapter/runtime path.	Named consumer capability and adapter.	Only its authorising trigger.
5	Repeat Wave 4 one type/consumer at a time.	Each proposed type has its own named consumer and verified contract.	Per-consumer capability record.	Never schedule merely because the type ladder lists it.

Tip

Success is a premium, durable gold corpus spanning source datasets, languages, categories, and valid question shapes—not a translation project and not speculative support for every conceivable player interaction.