Skip to main content
A plugin that refuses an article, or stops while processing it, leaves its Version quarantined: Quivr keeps the article and its reason but withholds it from search. Once a fixed plugin version is active, or a rollback restored a working one, a reprocess runs those articles again and makes the ones that succeed searchable. It follows a rollback in Upgrade a plugin with no downtime.

​
Prerequisites

  • The plan that should process the articles is active: the fixed plugin version is activated (see Switch plugins without restarting), or the plan was rolled back.
  • A key of the Organization that owns the Corpus with the actions plugins:admin, operations:read (to follow the reprocess) and operations:write (to pause, resume or cancel it), in QUIVR_OPERATOR_KEY.

​
Steps

1

List the stuck articles

stage is the step that failed: normalization (the normalizer failed, so the article was kept as submitted) or ingestion (segmenting or embedding it was refused or stopped). Filter with corpus_id, plugin, code, quarantined_after and quarantined_before; page_cursor returns the next page. An article quarantined before Quivr recorded structured reasons has only its code and no plugin. An article whose Record got a newer revision, or was withdrawn, is not listed: it can never become current.Keep the Corpus id and the code you want to reprocess:
2

Count them

A dry run is required. It counts the articles the reprocess would take, by stage and by code:
Leave out code to take every stuck article of the Corpus, or narrow with plugin, quarantined_after and quarantined_before.
3

Start it

Send the same body with dry_run set to false:
The answer is an Operation, Quivr’s record of a long-running administrative task. It takes the articles in scope at this moment and runs them with the plan active now. The same key and body return the same Operation. Keep its id:
4

Follow it

versions_recovered are searchable now. versions_quarantined failed again and stay quarantined with their new reason. versions_skipped counts the articles it left as they were, each under skipped_<reason>:Pause, resume or cancel it with POST /v0/operations/{id}/pause, /resume or /cancel, each with an idempotency_key body. POST /v0/operations/{id}/rerun on a finished reprocess takes what is still stuck in the same scope, with the plan active then.

​
Check it worked

List the Corpus again: the recovered articles are gone, and those that failed again show their new reason.

​
What a reprocess does

  • Normalization stage. The normalizer of the active plan runs again. When it answers, the article is published with the normalized content, as it would have been the first time, and processed. The change feed announces it with record.materialized.
  • Ingestion stage. The ingestion plugin of the active plan segments and embeds the article again.
  • As a first success. A recovered article becomes its Record’s current Version if the Record still wants it, is searchable, is evaluated by the alerts of its Corpus, and appears in the change feed with record.retrieval_ready and record.enrichment_available.
  • Pace. It runs on the backfill queue, at most backfill.rate articles per second (see Configuration), so live ingestion keeps its capacity. One reprocess of a Corpus runs at a time.
  • Plan. It is pinned to the plan active when it was accepted, like other work. A plugin of that plan that becomes unreachable quarantines the article again with pinned_plugin_unavailable.
  • Interruptions. A restart resumes the article it was processing. Canceling it puts an article it had started back in quarantine with its previous reason.

​
Troubleshooting