Skip to main content
Run a second embedding model (a model that turns text into numeric vectors) beside the current one. Fill it for existing documents, compare searches, then switch with a way back and update alerts that match by meaning. On this page
  • Steps: prepare the second model, run it for evaluation, fill existing documents, compare searches, switch, then move meaning alerts.
  • Check it worked, roll back, and clean up once you no longer need the way back.

Prerequisites

  • A running Quivr with an ingestion plugin pinned. This guide keeps core.ingest serving and adds hosted.embed, the hosted text embedding plugin.
  • curl, jq, Go and the quivr CLI (curl --version, jq --version, go version, command -v quivr). Run package commands from a Quivr checkout.
  • Your API address in QUIVR_API_URL and the id of a collection of documents (Corpus) in CORPUS_ID.
  • API keys with the following permissions and grants for the Corpus:
All three keys for a Corpus must belong to its Organization. Plugin, plan and promotion requests affect the whole deployment; backfills and coverage reads see only the key’s Organization and grants. You also need a reachable model endpoint, its model name and dimensions, and its credential in the plugin process environment. The plugin reads EMBED_API_KEY for authenticated providers. The examples use a hosted Cohere v2 endpoint and 1024 dimensions; choose a model that supports them.
Hosted embedding sends document text and search queries to your provider and can incur charges, including during certification and evaluation. Keep provider keys out of configuration files and API requests.
The package generation runs offline. The API sequence is checked with a local fake embedding provider; the provider configuration below is an example, not a live provider run. Replace placeholders with your installation’s values. This procedure changes the plugin that produces segments and vectors (the ingestion owner). If your new model is another space of the same plugin, follow Fill a new vector space, then use this page’s comparison checks and move meaning alerts. That path promotes a space; this example activates a registration. Keep the former model reachable and filling new documents throughout the trial and switch, so rollback remains possible.

Steps

1

Prepare the second model

Build the plugin, then write its configuration. Replace the URL and model placeholders before generating the manifest:
For Cohere’s own API, use "base_url":"https://api.cohere.com/v2" and "auth":"bearer" in both configuration copies instead.For exact token budgets, prepare the model’s matching tokenizer.json before generating the manifest:
Preparation installs the hash-pinned tokenizers 0.23.2 runtime when absent and downloads the requested model’s tokenizer file, requiring network access during setup. Startup stays offline. Without a tokenizer, the plugin counts UTF-8 bytes conservatively and underfills passages. The published hosted image includes tokenizers 0.23.2: mount the model file read-only and use /opt/tokenizer/bin/python plus its container model path. Follow the tokenizer preparation and image instructions for E5 and Gemma pins, mounts and manifest generation. A tokenizer change creates a new recipe and requires a rebuild for existing Corpora.Generate the manifest after adding the tokenizer and any container path changes:
For archive imports, hosted.embed combines concurrent document inputs from the same Organization within each plugin process. The batch_wait_ms window (default 25, range 0–100 ms) can extend while awaiting provider admission. Batches respect batch_size, max_batch_tokens and the shared max_concurrent_requests cap. Set batch_wait_ms: 0 to disable collection. Queries bypass the collection window. See the plugin’s limits and defaults for cancellation, refusal isolation and batch usage logs.With a matching local tokenizer configured, document cutting uses a pool of persistent helpers. tokenizer_processes: 0 (the default) uses up to four helpers, limited by the CPUs available to the plugin’s Go runtime. Set it to 1–32 to override the pool size. Each loaded helper holds its own runtime and tokenizer vocabulary. A local EmbeddingGemma 2 sample used about 440 MiB per loaded helper, or 1.7 GiB for four. Allow startup and request headroom; choose a smaller pool when memory is limited. Pool tuning and adding its execution-only declaration preserve the active ingestion recipe when semantic settings are unchanged. Pinned work can use the active equivalent registration after an in-place update. Follow the plugin’s throughput registration steps to change it while earlier work drains.The generated manifest binds the model and settings to a distinct vector space, the coordinate system for its numeric vectors. Use that exact configuration when pinning or registering it. Copy the space name from contributions.ingestion.spaces; its API id is that name followed by @ and the declared space version.Certify with a fixture carrying the same configuration. Inject EMBED_API_KEY into the certification command’s environment so its plugin subprocess receives it. This step calls your provider:
Check that the report says CERTIFIED. Start the plugin in a separate terminal, with the provider key and this plugin’s signing key ring injected from your secret manager. Its QUIVR_PLUGIN_SIGNING_KEYS ring must match the plugin’s entry in QUIVR_ENGINE_PLUGIN_KEYS on the API and workers; see signed engine requests. Choose a free port reachable by Quivr’s API and workers:
Set document_template and query_template explicitly for models with special prompts; the model name never selects a template. EmbeddingGemma 2 has its own example configuration.Changing the model, dimensions, format, metric, input templates or model_revision creates another space. A running deployment needs fresh ingestion or a rebuild before it serves that space. Set model_revision when the provider changes weights behind a stable model name. A changed package needs a new immutable registration: increment plugin_version, regenerate and certify it. To compare several hosted models at once, give each a distinct plugin_id before generation.
2

Install it for evaluation

Keep your current served routing, and add hosted.embed under ingestion.evaluation for each source format you want to compare. An ingestion owner is the plugin that produces a document’s segments and vectors. Evaluation adds another owner’s output without replacing the served owner’s output.For example, merge these settings into each API, worker and migrate configuration, preserving existing pins and routes:
Replace the pin’s entire configuration object above with the contents of .scratch/hosted-embed/configuration.json, including tokenizer when prepared. The engine pin and the generated manifest must carry the same settings and paths.Two settings use the same words. The spaces pin picks which of this plugin’s spaces it uses: hosted.embed has only one, so it is served. ingestion.evaluation decides whether its output reaches ordinary search: it does not until you activate it in step 5. Apply the pin; for a runtime registration, follow Upgrade or switch a plugin.Routing uses the original source media type: a PDF normalized to text still follows application/pdf. Both owners read the normalized document pieces (canonical Parts), but each keeps its own segment cuts and vectors.Ordinary search reads the served owner’s output; an explicit evaluation search reads the second owner’s output.In words: each model builds its own segments and vectors from the same document. Ordinary search uses the served model; evaluation search names the candidate and its space. An evaluation failure is recorded in document diagnostics and does not change served readiness.Find the active hosted.embed registration and save its registration_id as REGISTRATION_ID:
Extract SPACE_ID from the generated manifest, which is JSON-compatible YAML. Save the current plan for an exact rollback target:
3

Fill existing documents

Live ingestion fills a space once the Corpus’s search index carries it. Installing the plugin alone does not prepare every existing Corpus. A backfill can add the space and fill old documents. Run it for every Corpus in the deployment, in every Organization, including empty ones, without date filters.Use your deployment’s list of Organizations and each Organization’s keys. List its authorized Corpora:
Follow next_page_cursor by sending it as page_cursor until it is absent (displayed here as null). Quivr cannot list all Organizations’ Corpora with one key. Ensure the key grants access to every Corpus in its Organization.Make a dry run with an explicit registration and its primary space:
For two short documents, the dry run includes these fields (trimmed; your counts vary):
Read versions, segments, input_tokens, estimated_seconds and confirmation_required. Cost is unknown when estimated_cost_usd is absent. A plugin without its own segments yet is estimated using the current owner’s cuts; completion uses the new owner’s actual cuts.The start request must reuse the dry run’s key and scope, changing dry_run to false. Add confirm_cost:true only after accepting an estimate that requires confirmation:
Save the returned operation_id in OPERATION_ID. An Operation records this long-running task. Follow it until state is succeeded, failed or canceled. Continue only after succeeded; otherwise inspect the failure or rerun the fill:
See Fill a new vector space for pausing, resuming and failures. The same accepted key and body return the same Operation; a later fill needs a fresh key and its own dry run before starting. One unfinished backfill per Corpus is allowed.
4

Compare searches and coverage

Use the same query, Corpora, mode, search profile, filters and limit. These two requests differ only in selecting the evaluation path:
Both evaluation fields are required together. The space must belong to that plugin and be carried by every requested Corpus; invalid selections return 422 unsupported_search.Each hit is one segment (passage), so one document can appear several times. Compare relevant document ids and passage order across several queries, including paraphrases and other languages. Owners can cut documents differently, so segment offsets need not match. Raw similarity scores from different models are not comparable probabilities. Check hit vector_space_id to confirm which model answered.Read each Corpus’s coverage:
In the same two-document run, the candidate entry reads (trimmed):
For each space, coverage.segments counts vectors present, coverage.total_segments counts that owner’s current segments, and coverage.versions_covered counts current document revisions fully covered. Compare segment counts within that owner; the response’s top-level segments describes served segments and can differ.Before switching, require coverage.segments to equal coverage.total_segments and the backfill Operation to be succeeded for each Corpus. An empty Corpus reads 0/0, which is fine. These counts describe stored segments; skipped documents can still lack candidate output. The switch rechecks coverage across the deployment and refuses if anything is missing.
5

Switch the model

Fill and compare before switching ordinary search; migrate meaning alerts immediately afterward.In words: run both models, fill existing documents, compare, then switch and update alerts. If results worsen, return to the saved plan and update alerts again. Changing spaces within one owner uses space promotion; changing the ingestion owner uses registration activation.Different owner, as in this hosted.embed example: activate the already active evaluation registration:
The response is 202 Accepted with an Operation and a Location header. Follow the Operation with your operator key. After success, admin.plan_id identifies the plan to inspect with GET /v0/admin/plugins/plans/{plan_id}.This switches every source format that names this plugin as an evaluation owner, moving its segments and vectors together. Its primary space must be carried by every Corpus’s index across every Organization, including empty Corpora. Every current document routed to the candidate must have all its candidate segments and primary vectors.Missing coverage fails the Operation with coverage_incomplete and leaves the plan unchanged; an unreachable candidate fails it with plugin_unreachable. Retrying with the same idempotency key returns the same Operation. The previous owner becomes an evaluation owner for those formats and continues filling new documents asynchronously. If a configuration change leaves an existing Corpus serving another owner, new documents also fill the active plan’s default or source-route owner. Keep these plugins reachable and check coverage before returning to either model.Same owner, another space: follow Fill a new vector space, which ends with a space promotion. That API allows force as an explicit exception for incomplete coverage; documents without vectors then disappear from semantic results until filled. Registration activation has no force option.
6

Move vector meaning alerts

Switching search does not re-encode the query vectors of alerts that match by meaning, so those alerts return not_ready until you re-create them. Do it right away: follow Move meaning alerts to a new vector model.

Check it worked

Repeat the current-model search without evaluation fields:
With matching documents in the switched formats, expect your candidate’s space id:
Read the active plan’s served routes:
API and worker processes poll for plan changes. If loading a plan fails, they keep the old plan and do not retry that plan id until another plan appears; inspect their logs and fix the reported error before applying another plan change. Inspect every Corpus’s coverage. Submit a new document and confirm the new model fills it. Check the migrated alerts as Move meaning alerts shows.

Roll back

Keep the previous plugin reachable with its expected build, and keep its evaluation coverage filling new documents. Rollback accepts missing vectors or owner projections and starts background recovery. Affected documents retain their exact outgoing projection and encoder until recovery completes. Do not stop it while work or alerts still pin it. For an owner switch, return to the saved plan, using a new key for this rollback attempt:
Use drain for sound output; use stop for bad output or hangs. The same key and body replay this rollback. Keep the explicit target: another untargeted rollback can return to the plan you just left. Unreachable returning builds fail the Operation with plugin_unreachable and leave the plan unchanged. Follow its Location until success before inspecting admin.plan_id. During recovery, keep the outgoing build available for queries against documents still using its vectors. Lexical reads remain available if that outgoing encoder fails; hybrid search degrades and semantic search reports the failure. For a same-owner space switch, promote the returned previous_space_id back with the space-promotion request. It needs complete coverage too. Immediately after either rollback, move meaning alerts again to the restored space; the earlier migrations do not reverse themselves. Compatible redeploys preserve activated serving and evaluation routes; see Redeploy a configured build for the compatibility conditions. Earlier plans and pinned work still require their original builds. If a redeploy makes the saved plan unreachable, restore that build or follow the linked procedure to activate the returning owner’s current evaluation registration, then move meaning alerts again.

Encode queries on a local CPU

If you use hosted.embed with an OpenAI-compatible model, you can keep documents on the remote service and encode search questions beside the API. The optional local text encoder must use the same model revision, dimensions and input prefix. Changing where that computation runs leaves the vector space unchanged, so this execution change needs no Corpus rebuild or alert migration. Before enabling local routing, compare at least 200 distinct queries: cosine similarity must be at least 0.999, and the ordered top-10 evaluation results must match. Measure the encoder and deployed search under representative concurrency; local query-encoding p95 must be well below 100 ms before rollout. For the Railway reference deployment, build the image with QUIVR_BUILD_LOCAL_QUERY_ENCODER=1, then set QUIVR_LOCAL_QUERY_ENCODER=1 on the API. Start with four CPU cores and 4 GB RAM, and measure actual memory use. Documents continue remotely. The API refuses startup without the offline assets or matching encoder readiness. A local outage returns a bounded query error, without automatically falling back to the remote provider. Unset QUIVR_LOCAL_QUERY_ENCODER and redeploy the API to restore remote queries. The deployment instructions list the pinned runtime, thread/startup variables, parity CLI and validation gates. For other deployments, set QUIVR_HOSTED_QUERY_URL only in the API’s hosted plugin process. Use a loopback HTTP /v1 base, and keep the API and worker plugin pins identical. The same plugin reference describes the required readiness metadata and supported format.

Clean up

Retaining an evaluation owner is useful for rollback but continues provider calls. Once you no longer need that rollback option:
  1. Copy the active plan’s default, served routes and evaluation owners into ingestion.default, ingestion.routes and ingestion.evaluation in every process’s configuration. Apply this configuration first, so it records the routing you chose by activation or rollback.
  2. Remove the retiring owner from each ingestion.evaluation list and apply the configuration again. After rollback in this example, remove hosted.embed; if you keep the candidate serving, remove the former owner instead. This stops new evaluation calls for those formats.
  3. If that plugin no longer serves any format or other plugin role, remove its plugins pin too. Keep it reachable until pinned work and Subscriptions release it and its registration becomes inactive; see the plugin lifecycle. A plugin still serving another format or role must keep running.
Preserve the served routes and remaining owners’ pins throughout cleanup. Stored alert Versions and Matches remain readable.

Next

A refused space promotion appears as state: failed with coverage_incomplete in errors[0].code on the accepted Operation. The served space stays unchanged.