Skip to content

Legal database

The legal database is the authoritative catalog and search index for the shared legal corpus. Only Legal Core and the corpus worker own this boundary. Product applications, public clients, and future MCP clients use the Legal Core API instead of database credentials.

erDiagram
    DOCUMENT ||--o{ DOCUMENT_VERSION : has_immutable_versions
    DOCUMENT ||--o{ INDEX_GENERATION : builds
    DOCUMENT_VERSION ||--o{ INDEX_GENERATION : source_for
    INDEX_GENERATION ||--|{ DOCUMENT_CHUNK : publishes
    DOCUMENT o|--o| INDEX_GENERATION : activates
    JOB ||--o{ JOB_ATTEMPT : retries

    DOCUMENT {
        uuid id PK
        text canonical_key UK
        text source_name
        text source_id
        uuid active_generation_id FK
        timestamptz withdrawn_at
    }
    DOCUMENT_VERSION {
        uuid id PK
        uuid document_id FK
        bigint version
        text content_sha256
        text artifact_key
        text artifact_sha256
    }
    INDEX_GENERATION {
        uuid id PK
        uuid document_id FK
        uuid source_version_id FK
        text embedding_model
        int chunk_count
    }
    DOCUMENT_CHUNK {
        uuid id PK
        uuid document_id FK
        uuid generation_id FK
        int chunk_no
        vector embedding
    }
    JOB {
        uuid id PK
        text job_type
        text status
        int fence
    }
    JOB_ATTEMPT {
        uuid id PK
        uuid job_id FK
        int attempt_no
        text status
    }

The database separates three concerns:

  • legal stores canonical documents and immutable source versions.
  • search stores publish-at-once index generations and their chunks.
  • ops stores durable ingestion jobs and attempts.

A document exposes only its active index generation. A new generation can be built completely, checked, and then activated without serving a partially written index.

flowchart LR
    Source[Approved legal source] --> Worker[Corpus worker]
    Worker --> Original[(Private corpus blob container<br/>sha256/content hash)]
    Worker --> Version[(legal.document_versions<br/>immutable metadata and text)]
    Version --> Generation[(search.index_generations)]
    Generation --> Chunks[(search.document_chunks)]
    Chunks --> ItalianFTS[Italian full-text GIN]
    Chunks --> HNSW[pgvector HNSW<br/>cosine, vector 1024]
    ItalianFTS --> Core[Legal Core API]
    HNSW --> Core
    Original -->|checksum-verified read| Core
    Core --> Consumers[Kesita, owned apps,<br/>future public API and MCP]

The private corpus container preserves original artifacts under content-addressed keys such as sha256/<digest>. PostgreSQL stores the expected digest, size, media type, extracted text, and source metadata. Legal Core verifies the artifact against that metadata when it needs the original.

Search chunks use a 1024-dimensional pgvector embedding with an HNSW cosine index and an Italian full-text GIN index. The recorded embedding model belongs to an index generation, so model upgrades create and publish a new generation.

  • legal_api reads corpus and search data; it does not publish corpus versions.
  • legal_worker ingests documents and builds generations, but immutable document versions cannot be updated or deleted.
  • reporting_reader is read-only.
  • External consumers authenticate to Legal Core. They never receive PostgreSQL, pgvector, or blob credentials.