Corpus ingestion
Prepared imports and indexing exist. Full acquisition, enrichment generation and complete legacy-corpus import do not.
The corpus worker currently accepts prepared source records and prepared global representations. It stores source versions and builds embeddings. Automated acquisition from every source, enrichment generation and the full corpus import remain unimplemented.
flowchart LR Prepared["Prepared source import"] --> Snapshot["Immutable snapshot and original reference"] Snapshot --> Jobs["Index jobs for supported profiles"] Jobs --> Vectors["Provider embedding batches"] Vectors --> Publish["Validated publication"] Global["Prepared summary, article or act representation"] --> GlobalStore["Global embeddings per profile"]
What the import establishes
Section titled “What the import establishes”services/legal-core-api/internal/indexing/source.go distinguishes original-backed imports from explicitly text-only imports. A text-only record has no fabricated original checksum or download. global.go verifies the retained input hash, owner, profile and vector before importing a global representation.
Idempotent imports matter because acquisition jobs can retry after a lost response. Reusing the same identity with conflicting content must fail rather than silently change a citation. source_schema_integration_test.go exercises each source kind and retained-input profile fan-out; postgres_integration_test.go exercises concurrent imports and provider failures.
Passage indexing does not generate judgment summaries or act profiles. Those are prepared inputs today. New source snapshots and legal amendments are different events, as described in Legal database.
The approved Qdrant target adds a derived-index projection after source/representation validation. PostgreSQL retains exact embedding input, profiles and durable projection work; Qdrant receives vectors and only the allowlisted search payload. The worker must not copy the whole import record into the index. Acknowledgement, revision checks, coverage and retry ordering follow the projection publication contract. The current importer still writes SQL vectors; the Qdrant projector is not implemented yet.
Storage & indexes owns the completeness rules. Workers & recovery owns the stale-worker rule. Erasure records the implemented cleanup scope and remaining gaps.