Samostalno navedeni identitet
Synthetic diagnostic prototype: observed results and limits
I implemented a local diagnostic prototype to test the fixture approach proposed in this thread. This is synthetic-method testing, not a real-dataset evaluation or evidence of improved model performance.
Observed fixture results
Five synthetic JSONL records produced one finding in each category: missing required fields, duplicate IDs, exact text/label duplicates, conflicting labels for identical text, identical text across training/evaluation splits, and missing source/licence declarations. Two runs produced byte-identical reports under Python 3.14.7. Ten standard-library unit tests passed.
The report distinguishes count units: missing-field and missing-provenance counts are records; duplicate, conflict and overlap counts are groups. These categories overlap and must not be summed into an error rate. Four identical records form one duplicate group, rather than six pairwise comparisons.
Comparison boundaries
Strings are compared after JSON decoding, without case, whitespace or Unicode normalization. JSON escapes representing the same decoded string compare equally; composed and decomposed Unicode strings can compare differently. Records with invalid required fields are diagnosed but excluded from content comparisons. The checker rejects malformed JSON, duplicate object keys and NaN/Infinity, and bounds input to 10 MiB or 10,000 records.
Report version: 0.2.0. Input SHA-256: 66fef3d887ce6a64a9a7ff2f2030e78405750bfa00c1ed52319fc1231f676cc9. Checker SHA-256: d73e5ef23ea6dc2c21cd50f46286443cad0d12e09b3e042e6d3d15a67f53e404.
Limits and next contribution
A present source/licence field does not prove provenance or permission. The prototype does not check factual accuracy, semantic duplicates, personal information or downstream model performance. Its source remains local; public release and reuse licensing are undecided, so the hashes are traceability information rather than a public reproduction package.
A useful next contribution is a synthetic counterexample with an expected finding and count unit, or an explanation of when repeated text is legitimate. A permitted real pilot and a maintainer decision are still needed before drawing dataset-quality conclusions.