Methodology

The pipeline

1. Step 1

   Fetch

   Paste an allowlisted paper URL or upload a PDF. The source adapter creates one canonical identity, normalized document, page ledger, and deterministic structural pre-scan before model work begins.

2. Step 2

   Classify

   A structural classifier resolves a paper type from source-bound records. If the proposal is missing, ambiguous, invalid, or weakly supported, the closed fallback path is selected deterministically.

3. Step 3

   Extract

   Four independent model lanes produce typed records for claims, evidence, methodology, and paper context. Quotes and references must resolve to the normalized source ledger; model prose is never treated as an assessment result.

4. Step 4

   Assess

   Four owner lanes judge claim alignment, evidence quality, methodology, and rhetorical restraint. Each owner returns closed findings, confidence, and source references that are validated independently.

5. Step 5

   Score

   Deterministic rubric code aggregates criteria into dimensions, applies explicit caps and minimum-evidence rules, derives confidence and verdict, and renders closed rationale text.

The assessment contract

The model supplies typed, source-bound findings. The deterministic assembler owns the public assessment result.

Why it's built this way

• Evidence stays attached to the paper. Every claim, finding, and rationale reference must resolve against the normalized source ledger.
• Owners stay independent. Separate assessment lanes reduce cross-contamination and make each judgment auditable.
• Unknown is not a zero. Missing, unusable, or unresolved evidence remains explicit and can produce an insufficient-evidence response.
• Caps are visible. Critical failures, minimum evidence, and low confidence can constrain the result only through named, ordered rules.
• Serialization is a boundary. HTTP, cache replay, persistence, SSE, and frontend mapping all validate the same ready/unavailable union.

Assessment dimensions

Each dimension is assembled from its named criteria and preserves its applicability, evidence references, confidence, and rationale.

Terminology

Why the assessment is trustworthy

Source-bound. Quotes, structural signals, and references are checked against one normalized source ledger.

Separated responsibilities. Model producers return typed findings; deterministic code owns IDs, aggregation, caps, confidence, verdicts, and rationale.

Fail closed. Invalid model output, unresolved references, insufficient evidence, malformed state, and persistence failures map to explicit stable states.

Validated everywhere. The same strict union is validated before HTTP return, cache replay, persistence, SSE completion, and frontend rendering.

Privacy-aware. Structured events contain operational metadata only; source content, prompts, provider responses, and credentials never belong in logs.