---
name: pssrag-process-documents
description: Process mixed-format enterprise document corpora into traceable RAG indexes and verified blind-evaluation reports. Use for MSG/PDF/XLSX/XLS/PPTX/image ingestion, recursive attachment extraction, VLM OCR, canonical locators, embeddings and reranking, AWS S3/DynamoDB/OpenSearch execution, resumable batch answering, hidden two-stage evaluation, or diagnosing ingestion and model failures without changing the approved RAG architecture.
---

# Process PSSRAG Documents

Turn a corpus into a source-locatable index, execute its question set without a manual batch UI, and produce a sanitized report. Preserve the approved parsing, retrieval, reranking, answering, and hidden-judge contracts when swapping model endpoints.

## Execute The Workflow

1. Read the repository's `AGENTS.md`, architecture, data contracts, and current environment keys. Never print secret values.
2. Inventory every top-level file and every recursive email/PPTX attachment before changing code. Read [format-routing.md](references/format-routing.md) for handler and locator rules.
3. Verify that every observed extension has a parser. Add a parser or record a hard failure; never silently drop an attachment type.
4. Run a no-model parse pass with a stub VLM. Validate unique stable element IDs, non-empty source paths, resolvable locators, and deterministic row/page/slide coordinates.
5. Register originals in the configured blob store, persist document state, parse canonical elements, embed in bounded batches, and replace the document's index records atomically.
6. Cache image extraction by image SHA-256 plus model/prompt version. Reuse extraction text but create a new occurrence locator for each email or slide occurrence.
7. Ingest sequentially unless the target data plane has verified headroom. Persist `parsing` and `indexing` failures separately and retry only failed documents.
8. Verify all documents are `ready`, compare indexed counts with parsed counts, and manually inspect representative email, spreadsheet, PDF, PPTX, English image, and Chinese image evidence.
9. Import the frozen question set in the backend, execute answers, and persist each terminal result. Do not expose batch submission or hidden judge controls in the reporting UI.
10. Require zero answer errors before evaluation. Run `dev` for tuning; run frozen `holdout` once only after configuration freeze.
11. Generate the sanitized report API/UI and Word artifact. Keep hidden prompts, provisional baselines, and judge evidence on internal-token routes only.

For this repository, use:

```bash
.venv/bin/python -m scripts.run_blind_evaluation --split dev
```

Resume a partial batch rather than creating duplicates:

```bash
.venv/bin/python -m scripts.run_blind_evaluation \
  --split dev --skip-ingestion --batch-id <batch_id>
```

After changing a retrieval or answering configuration, invalidate only the tuning split:

```bash
.venv/bin/python -m scripts.run_blind_evaluation \
  --split dev --skip-ingestion --batch-id <batch_id> --retry-split dev
```

Freeze the configuration before invalidating and executing `holdout`. Never tune from holdout output.

For a prompt fix demonstrated by one dev failure, rerun only that question before reevaluating dev:

```bash
.venv/bin/python -m scripts.run_blind_evaluation \
  --split dev --skip-ingestion --batch-id <batch_id> --retry-question-id <question_id>
```

When chunking or parsing changes, rebuild only the active evaluation split before rerunning its answers:

```bash
.venv/bin/python -m scripts.reingest_split --split dev
```

## Apply Quality Gates

- Treat canonical locators and stable element IDs as product contracts, not parser internals.
- Keep dense retrieval on the original question. Use query rewrite only for the lexical path.
- Rerank the configured high-recall candidate set before reducing answer context.
- Record the actual answer model ID. Permit fallback only to an explicitly approved, quality-comparable model after a measured timeout.
- Never evaluate an error result as a low-quality answer. Retry or exclude it with an explicit incomplete-run status.
- Never let the baseline model see the candidate answer. Keep baseline construction and scoring as separate stateless calls.
- Do not run holdout repeatedly while tuning.
- Do not claim completion from unit tests alone. Verify the real cloud stores, model endpoints, corpus, browser UI, and generated report.

## Recover Deliberately

Read [failure-playbook.md](references/failure-playbook.md) before retrying a failed run. Prefer resuming persisted progress to deleting or recreating AWS resources. If a model or index endpoint degrades, first distinguish request timeout, format rejection, data-plane throttling, and deterministic input failure.

## Deliver

Read [evaluation-and-delivery.md](references/evaluation-and-delivery.md) before freezing configuration or producing the final report. Commit architecture, backend, frontend, infrastructure, and recovery fixes in coherent increments; never commit `.env`, raw client documents, generated indexes, or hidden evaluation artifacts.