Dataset cleanup, 27 September 2026

The active corpus now contains 8,067 accepted records, 3,869 exclusions, and 282 deferrals. All 12,218 IDs remain available. The new register replaces the 14 September register for the explorer; older registers and bibliographies remain historical snapshots. These are agent assessments, not human-confirmed judgments.

Strategies transferred from the QN publication audit

The t3code thread ab746574-56b8-4dca-a0b0-dc1ce80204f1 and the local QN audit's DATA_CLEANUP.md, validation/luna_full_review_method.md, and scripts/reconcile_reviewed_works.py supplied four useful practices:

• Check publication eligibility across all manifestations. Provider types can mislabel meeting abstracts as articles, and a journal manifestation should not be discarded because another source represents a talk.
• Adjudicate identity explicitly as same work, distinct, or uncertain. Shared titles and abstracts supply candidates, not automatic identity decisions.
• Resolve unclear relevance using source evidence. Record the exact download, hash, member and inspected section; distinguish unavailable evidence from evidence of irrelevance.
• Freeze candidate IDs and input hashes, retain individual reasons, and check exact output coverage. Preserve an auditable history rather than deleting raw records or rewriting earlier judgments.

The user explicitly extended the conference exclusion to this collection during this pass. Conference abstracts, presentations, proceedings contributions and proceedings volumes are now excluded. A preprint of the same conference work is also excluded. Independently identified journal publications and standalone manuscripts remain eligible. Reviews and non-conference book chapters remain eligible when substantively relevant. This policy is recorded in the search plan.

Applied changes

The conference review combines source types, venue, publication references, DOIs and original source URLs. It includes proceedings mislabelled as chapters, articles or preprints. Candidate selection deliberately overmatches: PNAS, Proceedings of the IEEE and AIP Advances are retained as journal publications. IEEE identifies its Proceedings as a review/survey/tutorial journal, while the publisher of physica status solidi (c) explicitly describes conference contributions. These distinctions are supported by the IEEE journal description and Wiley editorial. Repository-only conference classifications have weaker evidence than explicit proceedings DOIs; their provider type and source URLs are retained for review. This pass does not certify that every conference contribution has been found.

The seven duplicate mappings are 3759→256, 8528→8549, 8786→8785, 8876→8873, 8968→8969, 9023→9026, and 1800→435. Shared arXiv identifiers or persistent thesis repository identifiers support these decisions. The misleading meeting venue on 1800 does not override its ADS arXiv identifier and matching primary preprint. Record 5540 remains distinct from 5639 despite an erroneous shared OpenAlex arXiv location. Original article 892 and Author Correction 1752 also remain separate. Similar abstracts alone did not resolve the other five groups.

Four original arXiv source files were downloaded through PatchScholar in source/PDF/HTML preference order. All succeeded as source downloads. The originals were retained unchanged; TeX was inspected without compilation or execution. The comment on surface pair breaking (10306) and thesis on NS/SNS physics (10967) qualify. The Hubbard-chain shadow-band paper (11091) is a surname false positive: its Ogata–Shiba wave function is unrelated to superconducting impurity states. The spin-gap proximity paper (11017) remains a scope question after reading its pair-hopping argument; it is no longer marked as lacking full text. The precise reading limits are in source reviews.

Evidence, application and reproduction

The input manifest freezes candidate IDs, local packet hashes and provider payload fingerprints. The publication-type table, relevance table, identity decisions, and conference scope table record the assessments. Conference scope is applied last and supersedes interim inclusions in the relevance/type tables.

Run pixi run cleanup-replay to reproduce the cleaned CSV, final review pack and summary from the frozen baseline and these decisions. This command does not write the live database. packs/2026-09-27-cleanup.yaml is the complete final PatchScholar review pack for a database at the original baseline. It has already been applied here in three stages; rerunning review packs unnecessarily creates additional history events. The subsequent packs are packs/2026-09-27-conference-scope.yaml and packs/2026-09-27-deferred-conferences.yaml.

PatchScholar added 321 initial review events, 589 conference-scope events, and 18 further exclusions from the deferred queue. All applications preserved all earlier events, untouched reviews, source-to-work identities and canonical metadata. The initial application check the conference application check, and final application check record the hashes and local backups. The final register's decisions and reasons match the live corpus. Original source types remain in the database; the reviewed publication type is a collection-specific field used by the explorer and its BibTeX projection. No destructive canonical merging was performed.

The explorer displays reviewed types, links duplicate records to their retained representative, and keeps original IDs and browser-local notes intact. Accepted counts omit excluded duplicates and conference records. Citation edges remain attached to the original canonical IDs; this is not a deduplicated citation network, and citations have not been transferred between versions.

The search ledger records the exact four-ID acquisition, its initial sandbox DNS failure, the bounded cached review and the subsequent conference scope pass. None is called provider exhaustion or proof of coverage. The runtime used PatchScholar revision ee243c90dd572a6390f981dcd875bbba4399938b with a clean sibling working tree.

Validation covers the register and unchanged historical IDs, negative identity cases, conference/journal distinctions, all accepted BibTeX entries, and the desktop/mobile explorer and reading workflows. The validation record contains the artifact hashes and exact checks. The accepted export contains 8,067 parseable entries with exactly the accepted register IDs; its fields agree with the explorer's bibliography.

Remaining quality work

The corpus still has 1,345 accepted records without abstracts. This is a follow-up queue, not a reason to exclude otherwise evidenced literature. Prioritize primary evidence for title-only admissions, then missing/incorrect metadata and coverage of underrepresented periods and topics. The refreshed audit and its adjacent CSV inventories record the remaining work against the final register. The September 18 inventories remain unchanged.

Five shared-abstract identity groups remain open. Records 5496 and 6109 have conflicting conference/preprint/journal evidence and are deferred; primary arXiv pages do not settle the version relationship. Records 2005, 4098 and 4116 are Figshare collection candidates requiring an artifact/identity decision; this conference pass did not conflate a publisher name with a conference venue. The earlier audit's wrong-source abstracts on 8425, 8580 and 8593 also need replacement from checked primary documents, not guessed corrections.

For further cleanup, inspect both source manifests and primary text, use explicit same-work/distinct/uncertain decisions, and record the consequence for the accepted view separately from the raw citation graph. Do not turn similarity, missing abstracts or a full local batch into automatic exclusions or a coverage claim.