- Steps: prepare the second model, run it for evaluation, fill existing documents, compare searches, switch, then move meaning alerts.
- Check it worked, roll back, and clean up once you no longer need the way back.
Prerequisites
- A running Quivr with an ingestion plugin pinned. This guide keeps
core.ingestserving and addshosted.embed, the hosted text embedding plugin. curl,jq, Go and thequivrCLI (curl --version,jq --version,go version,command -v quivr). Run package commands from a Quivr checkout.- Your API address in
QUIVR_API_URLand the id of a collection of documents (Corpus) inCORPUS_ID. - API keys with the following permissions and grants for the Corpus:
All three keys for a Corpus must belong to its Organization. Plugin, plan and promotion requests affect the whole deployment; backfills and coverage reads see only the key’s Organization and grants.
You also need a reachable model endpoint, its model name and dimensions, and its credential in the plugin process environment. The plugin reads
EMBED_API_KEY for authenticated providers. The examples use a hosted Cohere v2 endpoint and 1024 dimensions; choose a model that supports them.
The package generation runs offline. The API sequence is checked with a local fake embedding provider; the provider configuration below is an example, not a live provider run. Replace placeholders with your installation’s values.
This procedure changes the plugin that produces segments and vectors (the ingestion owner). If your new model is another space of the same plugin, follow Fill a new vector space, then use this page’s comparison checks and move meaning alerts. That path promotes a space; this example activates a registration.
Keep the former model reachable and filling new documents throughout the trial and switch, so rollback remains possible.
Steps
1
Prepare the second model
Build the plugin, then write its configuration. Replace the URL and model placeholders before generating the manifest:For Cohere’s own API, use Preparation installs the hash-pinned tokenizers 0.23.2 runtime when absent and
downloads the requested model’s tokenizer file, requiring network access during
setup. Startup stays offline. Without a tokenizer, the
plugin counts UTF-8 bytes conservatively and underfills passages. The published
hosted image includes tokenizers 0.23.2: mount the model file read-only and use
For archive imports, Check that the report says Set
"base_url":"https://api.cohere.com/v2" and "auth":"bearer" in both configuration copies instead.For exact token budgets, prepare the model’s matching tokenizer.json before
generating the manifest:/opt/tokenizer/bin/python plus its container model path. Follow the
tokenizer preparation and image instructions
for E5 and Gemma pins, mounts and manifest generation. A tokenizer change creates
a new recipe and requires a rebuild for existing Corpora.Generate the manifest after adding the tokenizer and any container path changes:hosted.embed combines concurrent document inputs from
the same Organization within each plugin process. The batch_wait_ms window
(default 25, range 0–100 ms) can extend while awaiting provider admission.
Batches respect batch_size, max_batch_tokens and the shared
max_concurrent_requests cap. Set batch_wait_ms: 0 to disable collection.
Queries bypass the collection window. See the plugin’s
limits and defaults
for cancellation, refusal isolation and batch usage logs.With a matching local tokenizer configured, document cutting uses a pool of
persistent helpers. tokenizer_processes: 0 (the default) uses up to four
helpers, limited by the CPUs available to the plugin’s Go runtime. Set it to
1–32 to override the pool size. Each loaded helper holds its own runtime and
tokenizer vocabulary. A local EmbeddingGemma 2 sample used about 440 MiB per
loaded helper, or 1.7 GiB for four. Allow startup and request headroom; choose a
smaller pool when memory is limited. Pool tuning and adding its execution-only
declaration preserve the active ingestion recipe when semantic settings are
unchanged. Pinned work can use the active equivalent registration after an
in-place update. Follow the plugin’s
throughput registration steps
to change it while earlier work drains.The generated manifest binds the model and settings to a distinct vector space, the coordinate system for its numeric vectors. Use that exact configuration when pinning or registering it. Copy the space name from contributions.ingestion.spaces; its API id is that name followed by @ and the declared space version.Certify with a fixture carrying the same configuration. Inject EMBED_API_KEY into the certification command’s environment so its plugin subprocess receives it. This step calls your provider:CERTIFIED. Start the plugin in a separate terminal,
with the provider key and this plugin’s signing key ring injected from your secret
manager. Its QUIVR_PLUGIN_SIGNING_KEYS ring must match the plugin’s entry in
QUIVR_ENGINE_PLUGIN_KEYS on the API and workers; see
signed engine requests.
Choose a free port reachable by Quivr’s API and workers:document_template and query_template explicitly for models with special prompts;
the model name never selects a template. EmbeddingGemma 2 has its own
example configuration.Changing the model, dimensions, format, metric, input templates or model_revision creates another space. A running deployment needs fresh ingestion or a rebuild
before it serves that space. Set model_revision when the provider changes weights behind a stable model name. A changed package needs a new immutable registration: increment plugin_version, regenerate and certify it. To compare several hosted models at once, give each a distinct plugin_id before generation.2
Install it for evaluation
Keep your current served routing, and add Replace the pin’s entire Extract
hosted.embed under ingestion.evaluation for each source format you want to compare. An ingestion owner is the plugin that produces a document’s segments and vectors. Evaluation adds another owner’s output without replacing the served owner’s output.For example, merge these settings into each API, worker and migrate configuration, preserving existing pins and routes:configuration object above with the contents of
.scratch/hosted-embed/configuration.json, including tokenizer when prepared.
The engine pin and the generated manifest must carry the same settings and paths.Two settings use the same words. The spaces pin picks which of this plugin’s spaces it uses: hosted.embed has only one, so it is served. ingestion.evaluation decides whether its output reaches ordinary search: it does not until you activate it in step 5. Apply the pin; for a runtime registration, follow Upgrade or switch a plugin.Routing uses the original source media type: a PDF normalized to text still follows application/pdf. Both owners read the normalized document pieces (canonical Parts), but each keeps its own segment cuts and vectors.Ordinary search reads the served owner’s output; an explicit evaluation search reads the second owner’s output.In words: each model builds its own segments and vectors from the same document. Ordinary search uses the served model; evaluation search names the candidate and its space. An evaluation failure is recorded in document diagnostics and does not change served readiness.Find the active hosted.embed registration and save its registration_id as REGISTRATION_ID:SPACE_ID from the generated manifest, which is JSON-compatible YAML. Save the current plan for an exact rollback target:3
Fill existing documents
Live ingestion fills a space once the Corpus’s search index carries it. Installing the plugin alone does not prepare every existing Corpus. A backfill can add the space and fill old documents. Run it for every Corpus in the deployment, in every Organization, including empty ones, without date filters.Use your deployment’s list of Organizations and each Organization’s keys. List its authorized Corpora:Follow For two short documents, the dry run includes these fields (trimmed; your counts vary):Read Save the returned See Fill a new vector space for pausing, resuming and failures. The same accepted key and body return the same Operation; a later fill needs a fresh key and its own dry run before starting. One unfinished backfill per Corpus is allowed.
next_page_cursor by sending it as page_cursor until it is absent (displayed here as null). Quivr cannot list all Organizations’ Corpora with one key. Ensure the key grants access to every Corpus in its Organization.Make a dry run with an explicit registration and its primary space:versions, segments, input_tokens, estimated_seconds and confirmation_required. Cost is unknown when estimated_cost_usd is absent. A plugin without its own segments yet is estimated using the current owner’s cuts; completion uses the new owner’s actual cuts.The start request must reuse the dry run’s key and scope, changing dry_run to false. Add confirm_cost:true only after accepting an estimate that requires confirmation:operation_id in OPERATION_ID. An Operation records this long-running task. Follow it until state is succeeded, failed or canceled. Continue only after succeeded; otherwise inspect the failure or rerun the fill:4
Compare searches and coverage
Use the same query, Corpora, mode, search profile, filters and limit. These two requests differ only in selecting the evaluation path:Both evaluation fields are required together. The space must belong to that plugin and be carried by every requested Corpus; invalid selections return In the same two-document run, the candidate entry reads (trimmed):For each space,
422 unsupported_search.Each hit is one segment (passage), so one document can appear several times. Compare relevant document ids and passage order across several queries, including paraphrases and other languages. Owners can cut documents differently, so segment offsets need not match. Raw similarity scores from different models are not comparable probabilities. Check hit vector_space_id to confirm which model answered.Read each Corpus’s coverage:coverage.segments counts vectors present, coverage.total_segments counts that owner’s current segments, and coverage.versions_covered counts current document revisions fully covered. Compare segment counts within that owner; the response’s top-level segments describes served segments and can differ.Before switching, require coverage.segments to equal coverage.total_segments and the backfill Operation to be succeeded for each Corpus. An empty Corpus reads 0/0, which is fine. These counts describe stored segments; skipped documents can still lack candidate output. The switch rechecks coverage across the deployment and refuses if anything is missing.5
Switch the model
Fill and compare before switching ordinary search; migrate meaning alerts immediately afterward.In words: run both models, fill existing documents, compare, then switch and update alerts. If results worsen, return to the saved plan and update alerts again. Changing spaces within one owner uses space promotion; changing the ingestion owner uses registration activation.Different owner, as in this The response is
hosted.embed example: activate the already active evaluation registration:202 Accepted with an Operation and a Location header. Follow the Operation with your operator key. After success, admin.plan_id identifies the plan to inspect with GET /v0/admin/plugins/plans/{plan_id}.This switches every source format that names this plugin as an evaluation owner, moving its segments and vectors together. Its primary space must be carried by every Corpus’s index across every Organization, including empty Corpora. Every current document routed to the candidate must have all its candidate segments and primary vectors.Missing coverage fails the Operation with coverage_incomplete and leaves the plan unchanged; an unreachable candidate fails it with plugin_unreachable. Retrying with the same idempotency key returns the same Operation. The previous owner becomes an evaluation owner for those formats and continues filling new documents asynchronously. If a configuration change leaves an existing Corpus serving another owner, new documents also fill the active plan’s default or source-route owner. Keep these plugins reachable and check coverage before returning to either model.Same owner, another space: follow Fill a new vector space, which ends with a space promotion. That API allows force as an explicit exception for incomplete coverage; documents without vectors then disappear from semantic results until filled. Registration activation has no force option.6
Move vector meaning alerts
Switching search does not re-encode the query vectors of alerts that match by meaning, so those alerts return
not_ready until you re-create them. Do it right away: follow Move meaning alerts to a new vector model.Check it worked
Repeat the current-model search without evaluation fields:Roll back
Keep the previous plugin reachable with its expected build, and keep its evaluation coverage filling new documents. Rollback accepts missing vectors or owner projections and starts background recovery. Affected documents retain their exact outgoing projection and encoder until recovery completes. Do not stop it while work or alerts still pin it. For an owner switch, return to the saved plan, using a new key for this rollback attempt:drain for sound output; use stop for bad output or hangs. The same key and body replay this rollback. Keep the explicit target: another untargeted rollback can return to the plan you just left. Unreachable returning builds fail the Operation with plugin_unreachable and leave the plan unchanged. Follow its Location until success before inspecting admin.plan_id. During recovery, keep the outgoing build available for queries against documents still using its vectors. Lexical reads remain available if that outgoing encoder fails; hybrid search degrades and semantic search reports the failure.
For a same-owner space switch, promote the returned previous_space_id back with the space-promotion request. It needs complete coverage too. Immediately after either rollback, move meaning alerts again to the restored space; the earlier migrations do not reverse themselves.
Compatible redeploys preserve activated serving and evaluation routes; see Redeploy a configured build for the compatibility conditions. Earlier plans and pinned work still require their original builds. If a redeploy makes the saved plan unreachable, restore that build or follow the linked procedure to activate the returning owner’s current evaluation registration, then move meaning alerts again.
Encode queries on a local CPU
If you usehosted.embed with an OpenAI-compatible model, you can keep documents
on the remote service and encode search questions beside the API. The optional
local text encoder must use the same model revision, dimensions and input prefix.
Changing where that computation runs leaves the vector space unchanged, so this
execution change needs no Corpus rebuild or alert migration.
Before enabling local routing, compare at least 200 distinct queries: cosine
similarity must be at least 0.999, and the ordered top-10 evaluation results must
match. Measure the encoder and deployed search under representative concurrency;
local query-encoding p95 must be well below 100 ms before rollout.
For the Railway reference deployment, build the image with
QUIVR_BUILD_LOCAL_QUERY_ENCODER=1, then set QUIVR_LOCAL_QUERY_ENCODER=1 on the
API. Start with four CPU cores and 4 GB RAM, and measure actual memory use.
Documents continue remotely. The API refuses startup without the offline assets
or matching encoder readiness. A local outage returns a bounded query error,
without automatically falling back to the remote provider.
Unset QUIVR_LOCAL_QUERY_ENCODER and redeploy the API to restore remote queries.
The deployment instructions
list the pinned runtime, thread/startup variables, parity CLI and validation gates.
For other deployments, set QUIVR_HOSTED_QUERY_URL only in the API’s hosted
plugin process. Use a loopback HTTP /v1 base, and keep the API and worker plugin
pins identical. The same plugin reference
describes the required readiness metadata and supported format.
Clean up
Retaining an evaluation owner is useful for rollback but continues provider calls. Once you no longer need that rollback option:- Copy the active plan’s default, served routes and evaluation owners into
ingestion.default,ingestion.routesandingestion.evaluationin every process’s configuration. Apply this configuration first, so it records the routing you chose by activation or rollback. - Remove the retiring owner from each
ingestion.evaluationlist and apply the configuration again. After rollback in this example, removehosted.embed; if you keep the candidate serving, remove the former owner instead. This stops new evaluation calls for those formats. - If that plugin no longer serves any format or other plugin role, remove its
pluginspin too. Keep it reachable until pinned work and Subscriptions release it and its registration becomesinactive; see the plugin lifecycle. A plugin still serving another format or role must keep running.
Next
- Fill a new vector space for estimates and Operation controls.
- Plugin lifecycle for draining work and deciding when a plugin can stop.
- Move meaning alerts to a new vector model for the alert migration after a switch.
state: failed with coverage_incomplete in errors[0].code on the accepted Operation. The served space stays unchanged.