> ## Documentation Index
> Fetch the complete documentation index at: https://docs.quivr.thevibecompany.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Write an ingestion plugin

> Decide how articles are cut and embedded, and move a Corpus onto it

An ingestion plugin decides how Quivr cuts each article into segments and turns
them into vectors, and encodes search queries the same way. The core keeps
what must stay safe and uniform: the index, the vector space registry,
rebuilds, authorization and withdrawal. The contract is the
[ingestion Contribution](/contracts/plugins/v0#ingestion-contribution);
this page takes a plugin from its manifest to searchable Records.

<h2 id="declare-its-spaces">
  Declare its spaces
</h2>

A vector space is one embedding model with its dimensions and distance. The
plugin owns every space it declares, so each space id is the plugin id or
starts with it:

```yaml theme={null}
id: acme.embedder
version: 0.1.0
compatibility: {engine: ">=0.1.0 <0.2.0", plugin_api: ">=0.8.0 <0.9.0"}
contributions:
  ingestion:
    spaces:
      acme.embedder.base:
        version: "1"
        model: acme/text-embed-base
        dimensions: 768
        metric: cosine
        indexes: [text]
        query_modalities: [text]
      acme.embedder.large:
        version: "1"
        model: acme/text-embed-large
        dimensions: 1024
        metric: cosine
        indexes: [text]
        query_modalities: [text]
```

Bump a space's `version` whenever its vectors change: a new model, new weights
or a new input template. Quivr treats `acme.embedder.base@2` as a new space,
never mixing its vectors with those of version 1.

<h2 id="implement-it">
  Implement it
</h2>

With the [Go SDK](/sdks/go), one type answers both operations:

* `SegmentAndEmbed` returns the segments of the Version's text Parts, in
  reading order, each with Unicode code point offsets into its Part and one
  vector per requested space. It may add a `LexicalText` (folded or stemmed
  words, indexed in a separate keyword field while excerpts keep the source
  text) and a `Provenance` map.
* `EmbedQuery` returns the vector of one query in one space.

Both must be deterministic: rebuilds reuse the stored segments and vectors, and
a different answer for the same Version is refused. With Plugin API 0.8 a
request may name no space: Quivr asks for the segments alone when a Version
arrives, so it is searchable by keyword even while your embedding backend is
down, then for the vectors. Return the same segments both times. Return a retryable error
when a backend is down, a terminal one for content you can never handle.
`sdks/go/examples/hash-embedder` is a complete example with two spaces.

<h2 id="certify-it">
  Certify it
</h2>

```sh theme={null}
quivr plugin test --report report.json .
```

The Contract Runner sends the normative article and your own ingestion
fixtures (`fixtures/*.json` with a top-level `ingestion`), checks every segment
and vector as the engine will, replays each fixture, encodes the fixture
queries in every space, and checks that invalid requests are refused and that
no declared secret appears in an answer or your plugin's output. Set the
fixture's `expect.segments` to pin your segmentation.

<h2 id="pin-it">
  Pin it
</h2>

Add it to `plugins` in the `QUIVR_CONFIG` file of every `api`, `worker` and
`migrate` process, and say which space answers search:

```json theme={null}
{"plugins": [
  {"manifest": "/etc/quivr/plugins/acme-embedder/quivr-plugin.yaml",
   "endpoint": "http://127.0.0.1:9910",
   "spaces": {"acme.embedder.base": "served", "acme.embedder.large": "evaluation"}}
]}
```

* **Served and evaluation.** Exactly one space is served. Evaluation spaces are
  embedded on the same segments and indexed, so you can compare them, but
  search never uses them. With one declared space, `spaces` may be omitted.
* **One plugin.** A deployment pins one ingestion plugin
  (`ingestion_conflict`); compare embedders as several spaces of it.
* **Startup.** The spaces are registered with their owner. Startup refuses a
  space another plugin already registered (`space_owner_conflict`) and a
  registered space whose model, dimensions or metric changed without a new
  version (`space_changed`). An unreachable plugin does not prevent startup.

<h2 id="move-a-corpus-onto-the-plugin">
  Move a Corpus onto the plugin
</h2>

Each Corpus keeps the spaces it was built with. A Corpus created before the
pin stays on its current space, searchable as before, until you rebuild it
(`POST /v0/corpora/{corpus_id}/rebuilds`, see the
[HTTP API reference](/reference/http-api)). The rebuild asks the plugin for every Version that has no stored vectors in the
new spaces, then switches search over at once. From then on new Records go
through the plugin, and search encodes queries with it.
`GET /v0/corpora/{corpus_id}/vector-spaces` shows the Corpus's spaces, their
owner and role, and how many segments hold a vector in each.

<h2 id="when-something-fails">
  When something fails
</h2>

| What happens | Effect |
| - | - |
| The plugin is down or answers a retryable error | The Version waits and is retried with backoff; search on a plugin space answers 503 |
| The plugin answers a terminal error, or segments or vectors the checks refuse | The Version is blocked as `ingestion_refused`; a rebuild fails with `ingestion_refused` |
| The segments with vectors differ from the segments first returned alone | The Version's enrichment is blocked as `derivation_conflict` |
| `embed_query` answers a terminal error | The search is refused with 422 `unsupported_search` |
| The plugin is unpinned while Corpora use its spaces | Their new Versions wait, and semantic search on them answers 422, until it is pinned again or they are rebuilt |