Quivr Ingestion and Retrieval

This context defines the domain-neutral language of the Quivr ingestion and retrieval engine. Vertical concepts such as a news-agency dispatch or monitoring signal belong to plugins rather than the engine vocabulary.

​

Content model

Corpus: A logical collection of records that share an access and retrieval boundary. Avoid: Index, database Record: A stable, typed logical item owned by exactly one corpus, independent of any particular representation or revision. Avoid: Document as a universal term, publication, signal Record Version: An immutable representation of a record observed at a particular point in its history. A source correction creates a new version, while progressive enrichment does not; distinct ingestion receipts may converge on the same version when source revision and content agree. A correction back to an earlier version’s content is a new version, never a return to the old one. Avoid: Mutable record, overwrite Record Version Manifest: The atomically published, immutable description of a record version’s verified parts, blobs, checksums, provenance, and plugin-produced structure. Avoid: Database row set, search projection Part: A typed, potentially hierarchical component owned by exactly one record version, with at most one parent and a key unique within its manifest. Avoid: Attachment as a universal term Normalized Content: A source-faithful representation of content independent of retrieval-specific segmentation, embedding, or ranking choices. Avoid: Chunked content, search projection Segmentation: A versioned derivation that divides normalized content into retrieval or processing units without changing the record version; multiple segmentations may coexist. Avoid: Record version, source structure Derivation: A reproducible output whose identity is determined by its inputs, capability, producer digest, model, parameters, and output role. Identical executions converge; divergent output for the same identity is a conflict rather than an overwrite. Avoid: Untracked generated file Blob: Immutable stored bytes whose identity and deduplication scope never cross an organization boundary; one blob may be referenced by multiple parts without sharing their provenance or business context. Avoid: File when referring to stored content Upload Session: A short-lived, organization-scoped grant to transfer exact bytes to storage, carrying the expected size, checksum and media type. It yields a reusable Blob identity only after read-after-write verification; a session or its transfer URL is not itself content or a durable Blob identity. Avoid: Transfer URL as Blob identity Relation: A typed link between records or parts. Avoid: Dependency, association Annotation: A versioned derived fact about a record, record version, or part, produced without changing its identity. Avoid: Metadata for derived results Projection: A rebuildable representation optimized for retrieval or presentation and derived from durable records and annotations. Avoid: Source of truth, primary record Vector Space: A named and versioned embedding representation whose dimensions, distance metric, and generating model remain consistent within that space. Avoid: Universal embedding, vector column Space Owner: The single ingestion plugin, or the engine for its built-in space, that declares a vector space and alone produces its document and query vectors. A space never changes owner; vectors from another model or template are a new space version. Avoid: Shared space, embedder Served Space: The vector space a deployment answers search with; each projection generation pins one when it is built. Avoid: Default embedding, active model Evaluation Space: A vector space a deployment indexes on the same segments as its served space so the two can be compared, but never uses to answer search. Avoid: Shadow index, experimental space Embedding Artifact: A durable derived representation of a part in a vector space, retained independently of any particular search projection. Avoid: Vector index entry Projection Generation: An internally consistent, rebuildable search representation that pins its derivations and vector spaces and can coexist with another generation during validation and cutover. Avoid: In-place index migration Archive Projection: A retrieval representation of historical records with its own availability, latency, and storage policy. Avoid: Backup, source of truth Cross-modal Vector Space: A vector space in which queries and parts of different modalities can be compared directly. Avoid: Modality-specific embedding Retrieval Candidate: A current segment the caller may read, which the core found in the index for one candidate request and served to the retrieval plugin with its text and score. A ranking may hold only candidates served in the same search. Avoid: Final search result, raw index hit Retrieval Round: One exchange of a search with the retrieval plugin: the plugin either asks for candidates, which the core serves before the next round, or returns the final ranking. A search has at most three. Avoid: Retry, iteration Retrieval Profile: A named latency and cost budget under which a search runs, such as default or deep; the retrieval plugin declares its profiles and the strategy behind each. Avoid: Search engine configuration, mode

​

Continuous retrieval

Saved Query: A stable, organization-owned identity for a retrieval request retained for repeated or automated evaluation. Avoid: Subscription, alert Saved Query Version: An immutable definition of a saved query that fixes its query expression, corpus scope, retrieval profile, and temporal policy. Avoid: Mutable saved query, subscription version Subscription: A stable, organization-owned instruction to evaluate a saved query continuously. Avoid: Saved query, notification channel Subscription Owner: An opaque reference to one end user of a client application, attached to a subscription when it is created and fixed for its life; Quivr stores and echoes it without interpreting it. A subscription without one is global to its organization. Avoid: User account, API key Subscription Version: An immutable subscription configuration that pins one saved query version and its evaluation and delivery policy. Avoid: Match, delivery attempt Match: A durable, idempotent determination that one record version satisfies one version of a subscription. Reevaluation after progressive enrichment converges on the same match; a materially relevant source correction may create a linked match for the new record version. Avoid: Delivery, search result Delivery: A stable logical external notification for one match, destination, and event kind; transport attempts may repeat without creating another delivery. Avoid: Match, delivery attempt Delivery Attempt: One transport attempt for a delivery, recorded separately because external effects are at-least-once rather than transactional with Quivr. Avoid: Logical delivery, match Record Key: The source-provided stable identity of a record within an organization, corpus, and source namespace, preserved across all of its versions. Avoid: Version identifier, blob hash Source Namespace: A durable identity partition for record keys that survives connector replacement or reconfiguration. Avoid: Connector instance, transport endpoint Source Position: An optional monotonic position supplied within a source namespace to order record revisions independently of delivery time; durable acceptance order is the fallback when none exists. Avoid: Arbitrary source timestamp, ingestion receipt identifier Current Record Version: The eligible version selected for a record, replaced atomically only after a successor becomes searchable. Avoid: Latest submitted version, latest created row Version Availability: The canonical readiness of a record version for retrieval, tracked independently from whether it is the record’s current version and from optional enrichment progress. Avoid: Receipt state, workflow status Ingestion Receipt: The immutable acknowledgement that the engine has durably accepted responsibility for one submission; it resolves exactly once as created, duplicate, withdrawal applied, or conflict. Processing and search availability belong to the associated record version rather than being copied onto the receipt. Avoid: Processing completion, search availability Operation: A durable, trackable execution of a long-running administrative command such as a backfill, rebuild, cold-data restoration, or purge. Technical retries and restarts preserve its identity; rerunning a terminal operation creates a new linked operation. Avoid: Workflow, ingestion receipt Change Event: An immutable, uniquely identified public fact describing a committed domain change, ordered within its organization for clients that consume the resumable change feed. Avoid: Temporal history event, internal task Change Cursor: An opaque organization-scoped position from which a client can resume at-least-once consumption of the public change feed within its retention window; an expired cursor requires explicit resynchronization. Avoid: Database offset, page number Tombstone: An explicit durable marker that immediately and permanently withdraws a record from retrieval, monitoring, and new delivery without erasing its identity or history; physical deletion remains a separate retention operation. Avoid: Hard delete, missing record

​

Governance

Organization: The ownership and security boundary that contains one or more corpora. Avoid: Corpus, deployment, user account Retention Policy: A versioned set of lifecycle rules governing the availability, tiering, and eventual deletion of records and their derivatives. Avoid: Garbage collection policy Legal Hold: An explicit override that prevents destructive retention actions on protected content and the canonical artifacts it still references. Avoid: Permanent retention policy Lifecycle State: The current logical availability of content or an artifact, independent of the storage provider implementing it. Avoid: S3 storage class Purge Candidate: An item with no canonical reference, active use, or legal hold that has satisfied its retention policy and is awaiting verification and a recovery grace period before physical deletion. Avoid: Deleted item Searchable Record: A materialized record version for which the engine has published the mandatory retrieval baseline, independently of optional multimodal enrichments that may arrive later. Avoid: Fully processed record Progressive Enrichment: The availability model in which a record version becomes searchable as soon as its mandatory retrieval baseline is ready, then gains optional text, image, audio, or video derivations without changing its identity. Avoid: Waiting for full processing, creating a new record version for derived output Backfill: The controlled processing or reprocessing of an existing historical range without changing the identity of its records. Avoid: Real-time ingestion, replay when referring only to transport redelivery

​

Extensibility

Connector Instance: A configured acquisition endpoint bound to exactly one corpus and one source namespace that introduces external content into that corpus through the same ingestion commands as any client. Avoid: Source when referring to collection mechanics Acquisition Checkpoint: The durable, connector-defined position from which a connector instance resumes acquisition, advanced only after the items fetched before it were durably accepted; re-fetching after a crash converges on the same receipts. Avoid: Source position, change cursor Deposited Credential: A write-only secret supplied for a connector instance, encrypted at rest, versioned by replacement and optionally carrying an expiry; it is never returned or logged. Avoid: API key, stored password Connector Health: The committed, evaluated condition of a connector instance’s collection (active, silent, access error, credential expiring or disabled) that distinguishes a source refusing access from a source that simply published nothing new. Avoid: Uptime, workflow status Connector Usage: The per-UTC-day count of source resources a connector instance read (current and previous day), reported by kinds whose source bills or rate-limits per resource; an estimate of what the source bills. Avoid: Quota, cost Plugin: A versioned installation unit that contributes one or more extensions to the engine through public contracts. Avoid: One plugin type per extension point Plugin Package: An immutable distribution unit containing a plugin manifest, schemas, and references to its executable artifacts. The operator deploys and runs it; the engine never starts it and only records the artifact digest the plugin reports. Avoid: Mutable image tag, contribution Plugin Registration: A plugin version the operator runs at an address, as the engine records it: identity, version, endpoint, manifest digest, the roles its manifest declares and a lifecycle state (registered, validated, active, draining, inactive or rejected). The first start registers the plugins pinned in the startup configuration. Avoid: Installation, pin, deployment Contribution: A named, typed extension supplied by a plugin, such as a connector, normalizer, enricher, projector, retriever, or subscription. Avoid: Plugin when referring to one capability inside a plugin Capability: A named and versioned public contract that a contribution provides or consumes without depending on another plugin’s identity. Avoid: Plugin dependency, internal service Plugin Requirements: The data access, secrets, network dependencies, and resources a plugin declares so an installer can configure and assess it. Avoid: Enforced sandbox policy, plugin configuration Plugin Trust Level: The support and provenance classification assigned by an installer to a plugin it has chosen to trust. Avoid: Runtime permission system, sandbox profile Plugin Admission Policy: The deployment-specific rules that determine whether a plugin package may be installed or activated based on provenance and integrity evidence. Avoid: Permission grant, runtime sandbox Plugin Generation: An activated, internally consistent set of plugin registrations and configuration, captured as one Pipeline Plan, that owns the work started under it while newer generations may coexist and older ones drain. Avoid: Deployment, plugin version, started process Pipeline Plan: The immutable mapping of every role of a deployment (a normalizer per media type, an alert rule per plugin, a connector per kind) to the plugin registration that serves it. One plan is active at a time, and each bounded processing execution keeps the plan it started with. Avoid: Live plugin registry, mutable workflow configuration Quarantine: A durable hold that withholds an accepted submission or record version from normal availability because a mandatory contribution could not safely complete. Avoid: Retry queue, deletion Plugin Worker: An independently operated process that executes remote contributions through public engine contracts. Avoid: In-process plugin, engine instance Plugin Manifest: The declaration a plugin ships to describe itself: its identity and version, compatibility ranges, contributions, configuration schema, required secrets, owned extension namespaces and limits. Avoid: Record Version Manifest, package metadata Plugin Protocol: The versioned, language-neutral contract through which the engine discovers, checks and invokes a plugin’s contributions. Avoid: SDK API, internal plugin interface Plugin API Version: The semantic version of the plugin protocol, versioned independently of the engine; a plugin declares the range it supports and the engine refuses one outside it. A minor version only adds to the previous one, so the engine keeps serving plugins built for an earlier minor version and speaks the highest version their range admits. Avoid: Engine version, plugin version Normalizer: A contribution that turns one accepted blob of a routed media type into the parts, relations and extensions of that record version’s manifest, without changing the record version’s identity. Avoid: Parser, converter, enricher Alert Rule: A subscription contribution: it decides, for one record version and a batch of distinct saved query expressions and subscription configurations, whether each one is a match, no match or not ready yet, with bounded evidence for a match. It declares the schemas of the expressions and configurations it interprets. It sees the record version’s text parts and metadata (source identity, acceptance time, provenance, extensions); several installed plugins can each provide one, and a subscription version names the one it uses by plugin id and version. Avoid: Evaluator plugin, matcher, filter Normalizer Route: The installation’s mapping from an accepted blob media type to the normalizer that handles it, marked required or optional. Avoid: Plugin registration, media type support Plugin Invocation: One uniquely identified call of a contribution for a specific input; retries of the same logical call share an idempotency key and must converge on the same output. Avoid: Plugin request, job Invocation Fixture: A language-neutral local test input for a normalizer: an input file, its media type and optional configuration, which tools turn into a plugin invocation through a local file reference. Avoid: Test case, sample request Subscription Fixture: A language-neutral local test input for an alert rule: a record version’s text parts and the evaluations to decide, each with an optional expected decision, which tools turn into batched plugin invocations. Avoid: Test case, sample alert Plugin Contract Runner: The tool that checks a plugin against the plugin protocol and normative fixtures using the engine’s own validation, so passing it means the engine accepts the plugin. Avoid: SDK test suite, integration test