All articles

Tasuki Keifu: Data Governance Agent Product Requirements

Personal projectsRead in Chinese

Data Governance Agent Product Requirements

Updated November 18, 2026. This is the current V2 contract, replacing obsolete pilot plans and status notes. See the optimization history for the evolution and technical design for implementation and limitations.

1. Goals and scope

An incremental event import is the primary entry point. Each leg or heat reads the current database, governs incoming and affected existing records, commits, and lets the next batch use those changes. A full database scan is not required. Targeted Membership reruns are supported.

Balance correctness, execution time, model cost and maintainability rather than exhaust all public information. People mainly revise business rules and settle a few difficult cases, instead of approving ordinary writes. Operator repairs must not be counted as automatic Graph outcomes.

Local deterministic preparation creates or reuses Competition, Edition, Race and Source from official material, without writing people, Memberships, PBs or results. Keep ordinary and NCG competition semantics separate. Respect the selected import scope, including gender; a mixed source is not permission to import excluded events.

2. Data and identity contract

Organization inputBehavior
junior_high_school, high_school, universitySource-confirmed schools may have formal organizations and sourced Memberships
corporate_teamConfirmed stable corporate teams may have formal organizations and Memberships; repeated multi-athlete participation can support confirmation
club, company, unknownNo formal Organization, public organization page or Membership; preserve a plain-text race label
Regional representatives, selections, organizersPreserve their event semantics; representation alone is not Membership evidence

Do not guess a school or corporate-team type to eliminate unknown. Reuse confirmed organization names, short names, aliases and source mappings first. A university club is not the university. RaceResult.organizationNameSnapshot is independent of the organization foreign key; unconfirmed labels have no organization link. Preserve snapshots and provenance before clearing legacy relations.

Japanese/Chinese names containing Han characters lose internal spaces. Non-Han names retain separators after whitespace folding. Display normalization does not establish identity, and source text remains traceable.

Same kana recalls candidates, not identities. Clearly different written names without supporting links can be distinct; possible missing characters or transcription errors receive explainable lightweight triage and, when needed, research. Visual similarity alone never authorizes merging. Names, kana, dates, school stages, historical results and provenance jointly support decisions. Normal high-school-to-university progression and absent Membership are not conflicts by themselves.

An unconfirmed label must not block an otherwise unique identity: one row under that organization key in the batch, one same-name candidate and no birth-date contradiction allow a rule-based match. New competing school histories or other material conflicts require further assessment.

DNS, DNF and DQ all retain race records. DNS is not a start; DNF/DQ indicate participation without a valid finish. None excludes source-supported school/team Membership governance. Current DNS identity cases skip lightweight/heavy research and retain safe rule mappings or independent source people. Historical DNS can support registration identity.

3. Full workflow

Official material → local event shell and standard input
 → organization rules → whole-group organization lightweight assessment
 → person rules → whole-group person lightweight assessment
 → one cross-check → bounded organization adjudication
 → single-organization web verification → affected-person reassessment once
 → necessary individual identity research
 → Membership governance → optional person facts
 → transactional write → observation/notification → next batch

This is a fixed DAG without loops. Nodes may skip calls. Lightweight assessment uses no web tools and does not silently split a group. Cross-check reviews both domains and independent evidence; it neither orchestrates freely nor writes the database. Organization corrections reassess only affected people once.

Every valid source row needs an explicit outcome: existing person, independent source person, or supported correction, with its result and provenance. Inconclusive evidence or exhausted research budget can leave unresolved findings without dropping rows or failing unrelated results. Structural errors, invalid targets and transaction failures are technical failures, not ordinary unresolved decisions. Retry resumes unfinished batches and preserves committed predecessors.

Current limitation: identity-unresolved rows skip Membership and person-fact writes while their people and results are retained. Do not report those fields as fully governed; settle identity and arrange a follow-up. The product goal remains an explicit write or non-write reason for all applicable data.

4. Membership

  • Confirmed school/team representation generates an observation even without an explicit memberships array.
  • Missing grade or dates does not prevent a sourced relationship with null dates. Include DNS and DNF.
  • Avoid repeating unchanged relationships; do not collapse multiple periods merely because the organization is the same.
  • Infer dates only from reliable academic anchors using Japan's April boundary; fill known school stages in either direction without inventing schools, grades or graduation.
  • Sourced dates may correct inferred dates; inference fills blanks. Missing end dates do not mean ongoing, and a new team does not automatically close the old one.
  • Use rules for routine cases. Only same-stage school contradictions, explicit exclusivity or unexplained timeline conflicts justify LLM/web research.
  • Include new and existing people. membership_only reuses imported identities and does not rewrite race results or rerun normalization.

5. Facts and research

Process only supplied hometown/PB facts. Fill empty supported fields and improve comparable PBs under source-quality rules; newer slower results do not replace better marks. Skip ordinary missing fields without web research. Current facts processing skips DNS and unresolved identities.

Heavy research targets a single serious identity/organization ambiguity or substantive Membership conflict. Tool citations must be traceable, and lightweight hypotheses are not independent evidence. Stop when evidence suffices or further search has little value; preserve the decision, confidence, evidence and stopping reason.

Persist budgets, timeouts and finite retries. Distinguish protocol/network errors from insufficient evidence. Count rule decisions, lightweight/heavy requests, investigated objects and web calls separately.

6. Observation and notifications

Keep event, batch, attempt, Agent, request, finding, change and delivery records outside business entities. Show readable summaries before raw detail; unknown usage/cost is not zero.

Capture wall time, queue/execution/retry time, rule and stage outcomes, lightweight/heavy requests, searches, model/token usage, estimated/unit-row cost, planned/applied changes, imported rows, unresolved reasons and failures. Failed tasks retain completed progress. Comparisons use consistent scope and versions, separating automatic results from manual repairs.

Do not send routine per-batch email. Serious unresolved cases or technical failures trigger an alert naming the event and leg/heat. Send an event-level terminal summary naming the event and covering completed/unfinished work, writes, repairs and unresolved cases. Delivery retries do not roll back data; review real-world alert volume as needed.

7. Current delivery

V2 is deployed and has handled the 333rd Nittai incremental import and Membership reruns. Its final unresolved count includes manual review and production corrections; it is not proof of 100% autonomous resolution or correctness on a new event.

Continue new events through the complete workflow, stopping for blockers and preserving ordinary evidence-based decisions. This optimization round is closed; new systemic issues use a new history document. Keep this PRD and the design current rather than retaining obsolete development plans.

Tasuki Keifu: Data Governance Agent Product Requirements | Xuhuan / Gelatoni