Dataset quality audit, 18 September 2026

This is the historical pre-cleanup audit. See the 27 September cleanup for applied decisions and current counts; the original queues below remain preserved.

The next pass should improve the evidence behind existing judgments before expanding keyword searches. The current register contains 8,629 inclusions, 3,255 exclusions, and 334 deferrals. Of the deferrals, 168 already have an abstract, and 39 have reasons referring to title-only or unavailable evidence despite a cached abstract. Several of those abstracts are quoted in the original review evidence itself. This is a screening-consistency issue, not simply a download backlog.

This audit changes no inclusion, exclusion, identity, or human-confirmation status. Its queues are agent-generated follow-up indicators. Counts describe canonical records; they do not measure unique papers or classification accuracy.

Measured priorities

Priority	Current evidence	Next bounded action	Success measure
1. Reassess existing evidence	39 deferred reasons need checking; 168 deferred records have abstracts	Read all 39 flagged cases against the search plan, then finish the other 129 abstract-bearing deferrals in batches of at most 50	Each gets a current, paper-specific reason; ambiguities remain deferred
2. Resolve identity and metadata conflicts	825 accepted records have existing identity flags; an additional test finds 41 groups with identical abstracts but different normalized titles	Start with the 11 shared-abstract groups containing multiple included records; compare identifiers, authors, dates, and repository manifestations	Explicit same-work, distinct-work, or unresolved decisions with source evidence; no automatic merges
3. Recover useful missing evidence	1,656 accepted and 166 deferred records lack abstracts	First check existing manifestations for the 166 deferrals; then obtain primary source, PDF, or HTML for unresolved high-value cases in batches of 25	Count usable abstracts/full texts and resulting evidence-supported decisions, rather than successful HTTP requests
4. Measure screening precision	Earlier targeted audits found substantial false inclusions and exclusions; present judgments are not independently calibrated	Draw a reproducible stratified sample across decision, discovery period, and abstract availability; review without seeing the old decision first	Publish disagreements and adjudications by stratum; use sampling weights for any overall estimate
5. Repair demonstrated recall gaps	Both historical arXiv families have zero successful batches; the known-paper audit already found keyword misses	Retry one saved historical request at a time with recorded stopping conditions; expand an independent DOI/arXiv benchmark across topics and periods	Recovery rate for held-out benchmark papers, new relevant unique works, and exact provider continuation status

Priority 2 also matters for citation-based rankings: duplicated manifestations can inflate record counts or split incoming links. Separately, 2,879 accepted records have no resolved edge in the current 58,634-edge graph. An absent edge is a missing-data indicator, not evidence that a paper is scientifically isolated. Reference enrichment should target accepted seeds and resolve source identities before recursively expanding their neighborhoods.

Cases that explain the priorities

3930, “Time Evolution in Superconducting Nanostructures.” The deferred reason says accessible detail does not identify an Andreev/subgap/Josephson focus. The cached abstract instead describes a quantum dot coupled to superconducting leads and relaxation affected by Andreev bound states. Its original OpenAlex payload contains that abstract. This is a strong candidate for renewed inclusion assessment; verify the thesis/repository record before recording a new decision.

10952, “Analysis of Superconducting Microstructures: Formulation.” The abstract identifies a finite-element treatment of the superconducting proximity effect in the dirty limit. Its review evidence quotes this abstract, while the reason says the title alone is insufficient. Research about relevant numerical methods is explicitly in scope. This should be reviewed from the existing mechanism evidence before requesting full text merely because it is deferred. The publisher/repository endpoint was inaccessible through the browsing tool in this audit; no new primary-text verification is claimed.

10842, “Frustrated XY model with screening,” and 11024, “Mesoscopics with Superconductivity.” Their cached abstracts describe Josephson/proximity-coupled arrays and normal-metal proximity physics respectively, but their reasons repeat the same generic title-only deferral. These need explicit scope judgments. The chapter particularly needs its relevant sections identified, as the search plan requires. Physics vocabulary alone is not a reason to include either record.

8425, “Magnetic and quasiparticle excitations in cuprates.” The cached abstract instead describes ferromagnet/superconductor interfaces. Reconstruction of abstract_inverted_index from source record 11992 exactly matches the stored work abstract. The mismatched text is therefore present in the retained OpenAlex response; this check does not support blaming local abstract reconstruction. The publisher endpoint was inaccessible in this audit. Preserve the conflict and seek correct primary metadata before using this abstract for relevance or topics.

8580 and 8593, two different proceedings volumes. A cosmology proceedings record and a condensed-matter symposium share an identical contents-style abstract. Both original provider payloads contain the same text (source records 12215 and 12239). Record 8593 also displays an entomology venue. Treat these as metadata conflicts, not a same-work merge inferred from matching abstracts.

5393 and 7932, carbon-nanotube Andreev spectroscopy. Both are included but have different titles and identical abstracts. Record 5393 already combines arXiv 1005.0443 with DOI 10.1038/nphys1811; record 7932 points to HAL-00480211. The primary arXiv record, checked during this audit, displays record 7932's title and links that DOI. This is strong evidence for a preprint/journal relationship, and motivates checking the HAL identifier before an explicit identity decision. The HAL page itself was not retrievable through the browsing tool. Matching the preferred canonical title alone misses this candidate; compare titles and identifiers of every source manifestation.

Missing abstracts are concentrated in the older extension

These rows are grouped by discovery batch, not by publication year.

Discovery batch	Accepted records	Accepted without abstracts	Deferred records	Deferred without abstracts
Original snapshot	3,190	325 (10.2%)	13	4
2000 backfill	3,468	660 (19.0%)	34	28
1985 backfill	1,971	671 (34.0%)	287	134

Overall, 19.2% of accepted records have no cached abstract. There are also 17 accepted records without author metadata and 432 without a venue. Missing DOI or arXiv identifiers alone are not errors: some eligible literature has neither. Read alternate cached manifestations before fetching again, and record which source supports each repaired field. Do not replace a known conflict with an unattributed “best” abstract.

Parsing the bibliography for every screened record also found malformed author fields in 4064, 4363, and 11277: affiliations and postal addresses are stored as personal names with too many commas for BibTeX. All three records are excluded; this does not invalidate the accepted bibliography. The Atlas now validates entries with Pybtex after PatchScholar serialization and marks these three as unavailable. Selecting one prevents a partial or malformed shortlist download. Fix the underlying author metadata in PatchScholar if exporting these records becomes useful; do not invent names or silently discard authors.

The 41 shared-abstract groups contain 97 records. Eleven groups contain multiple inclusions, and 26 accepted records in these groups have no existing identity flag. The groups include legitimate versions, translations, deposits, corrections, and metadata contamination. They are an investigative queue rather than 41 confirmed duplicates. The existing 403 shared-title groups remain a separate queue.

Make review quality measurable

A practical first validation sample is 20 records from each nonempty stratum of discovery batch × decision × abstract presence, taking all records in smaller strata. Select deterministically with a recorded seed and retain the IDs before reviewing. Hide the prior judgment during the first assessment. Adjudicate changes with the scope document and original evidence; obtain human confirmation for representative boundary cases. Report weighted error estimates and their uncertainty only after completing that process. The present targeted examples cannot estimate a corpus-wide error rate.

For recall, build a benchmark independently of the query results: select well-established papers from reviews and bibliographies across periods and Andreev reflection, MAR, ABS, crossed reflection/pair splitting, YSR, proximity, and hybrid circuits. Keep some benchmark papers out of query design and measure their later recovery. The failed historical arXiv requests are a real gap, but retrying them cannot recover pre-arXiv work; retain publisher/reference searches for the early period. Do not interpret query exhaustion as complete recall.

Reusable field provenance, identity adjudication, provider retries, and metadata repair belong in PatchScholar. Corpus-specific evidence, benchmark papers, review batches, and acceptance decisions belong here. The existing classifications are navigation aids: do not use unreviewed text suggestions as a ground truth for topic coverage, especially when their input abstract is already in dispute.

Artifacts and reproduction

The summary records the exact register and snapshot hashes. The audit recipe reads the enriched snapshot, requires a matching register hash, and writes:

Deferred records, ordered with the 39 reason/evidence checks first, followed by the other abstract-bearing cases.
Accepted records missing abstracts.
Shared abstracts across different titles.
Targeted provider-source checks, recorded separately from the snapshot-only recipe.

The shared-abstract test requires more than 100 characters and compares text after case folding and whitespace normalization. The reason check searches four literal phrases listed in the recipe. Neither heuristic diagnoses all errors; neither makes a relevance or identity judgment. All 12,218 records were scanned; the explanatory source checks above are a small targeted sample. No discovery queries or failed-provider retries were executed for this audit.

pixi run explorer-data  # Read existing corpus metadata; no provider fetches
pixi run audit-quality

Review scope and prior audits: search plan, 2000 backfill, and 1985 extension.