# Validation: DeepSeek core instrument factorial ## Текущее состояние - Study status: complete - Outcome: fail - Paid screening calls: 24 / 24 - Paid confirmation calls: 0 / 18, not run by preregistered stop - Selected config: none; `no_eligible_config` - Operational blocker: none; confirmation is forbidden by the frozen selector ## Criteria matrix | Criterion | Состояние | Evidence | |---|---|---| | AC-01 | pass before paid calls | protocol-v2 + identity, canonical preflight-v2 audit, prompt/scorer/rubric/source hashes frozen; v1 preserved with 0 paid calls | | AC-02 | pass | 24 unique submitted jobs, 24 terminal ready, 24 capture-authoritative raw files, 0 missing/failed; full transport binding recomputed | | AC-03 | pass | frozen scorer evaluated 24/24; independent metadata-hidden review supplied 24 exact aliases/hashes | | AC-04 | pass | prompt, reasoning and interaction effects published from the complete 2×2 design | | AC-05 | pass | full-chain selector deterministically returned `no_eligible_config`; no reduced denominator | | AC-06 | not run — preregistered stop | no config met deterministic 6/6 + semantic floor 6/6 + zero critical failures, so confirmation was forbidden | | AC-07 | pass with explicit null telemetry | model/prompt/reasoning/seed/workflow/job/raw hashes bound; unavailable credits/tokens/latency/retries stored as `null`; provider label drift preserved | ## Frozen protocol evidence | Проверка | Фактический результат | |---|---| | Protocol v2 file SHA-256 | `1806fee835210fcd1c902b3bea12c827145acc59eba67a0e9d33d732b149b223` | | Factorial manifest hash | `3a36376394e7723adfc969ed9b1c991778a7b5003b7b7f8e2152ce0eff495482` | | Canonical prepared manifest file | `034aa6a40bbffe798fdbc64522ca44f17e1832c526b4be1d5b468a7eb88af82c` | | Canonical prepared tree | `6ba93cc0137dee0962b910ce520aef243455a6b5eabb7e92c0a8468415fc5c35` | | Preflight v2 audit file | `ced0c45798ccf7a8006b5de7a418dc5bb37d671579094e3af6411f4ed90a618a` | | Live catalog exact-model status | `confirmed_exact_match`; 33 entries; options hash `aa6ec51e7958356c88c214ac63f35ae7a44aa5dcb6bb6946bd94871b81ab73b3` | | Semantic invariant block SHA-256 | `15fc3866392d1cfb1a3143af62fafef0a691f7565155719e67c30c049f391c47` | | RU baseline / treatment prompt | `07fedbf8ec56e8f8b85db60338561294f0418b9ef8bb09c1a096cec7643d1c7b` / `ecd7a19207acb34b28668bb10bbfa3e1fd6716cefec93ef3ef1d09d087c87958` | | EN baseline / treatment prompt | `cb995c1879d845b2ed2690bcde5c18e4c2b4803fddc882e5778c16213883ef03` / `2188edf9c458bad58a0b307ab47491a960898f1dc3c102534af99d73413282e1` | | Frozen evaluator file | `22f2877444133b149624018cea155efac0749025eaf70343a1e2811dd4f9f425` | | Frozen semantic rubric | `3d90905f2649ad52663775805cfd7152313d272306fbb38d9ee3f4cbd35cb3f3` | | Repository tests | 139 technical + 5 research-site, 0 failed | | TypeScript | `npm run typecheck`: exit 0 | ## Pre-paid checks ```text TECHNICAL: 139 tests passed, 0 failed RESEARCH SITE: 5 tests passed, 0 failed TYPECHECK: exit 0 PREFLIGHT: 72/72 prompt/workflow/payload bindings REPRODUCTION: 50/50 files, 0 byte mismatches ``` Проверки доказывают: - exact 24-cell matrix и unique request IDs; - prompt delta не меняет output contract; - `off`/`medium` отличаются только dynamic reasoning input; - selector сохраняет all-cell hard gates; - capture-authoritative raw и transport-v2 не допускают подмену source bytes; - fresh recomputation и full-chain finalization отклоняют forged scorecard; - все evidence bundles публикуются atomic/no-overwrite и отклоняют symlink targets/ancestors. ## Terminal screening evidence | Проверка | Фактический результат | |---|---| | Submission | один confirmed batch; 24 unique request-to-job bindings | | Terminal provider state | 24 ready, 0 pending, 0 failed | | Raw capture | 24 exact files, 191730 bytes, no extras | | Transport | 24 records; SHA-256 `f65f9452a33f4c50c27e1c6afa877f7cb94fd0fc0564c68a832a726d11d83b3b` | | Deterministic config passes | baseline/off 4/6; semantic/off 4/6; baseline/medium 3/6; semantic/medium 5/6 | | Blind semantic review | 24 reviews; score 0: 12, score 1: 9, score 2: 3; critical: 12 | | Selector | `no_eligible_config`; confirmation not materialized | Config-level verified join: | Config | Observed | Deterministic | Semantic ≥1 | Semantic 2 | Semantic total | Critical | |---|---:|---:|---:|---:|---:|---:| | `baseline-v1--off` | 6 | 4 | 1 | 0 | 1 | 5 | | `semantic-v2--off` | 6 | 4 | 4 | 1 | 5 | 2 | | `baseline-v1--medium` | 6 | 3 | 3 | 1 | 4 | 3 | | `semantic-v2--medium` | 6 | 5 | 4 | 1 | 5 | 2 | Counts reconcile: `4 × 6 = 24`, deterministic passes `16/24`, semantic-floor passes `12/24`, semantic-2 cells `3/24`, critical failures `12/24`. ## Terminal artifact bindings | Artifact | SHA-256 | |---|---| | batch capture | `253197e4f7e12c639f218e2dbd05d0e0292145a255d804ccde4b31ee661cc832` | | provider transport audit | `00ee107c4275cd8bbc611adf8251cc2060e71009cc2bc010eba429ef39ed858d` | | scorecard | `dac5eb5d71bcd750c4cbabdd3833d0533261be4fdf8a83e9cb7d4f69e492f86c` | | blind semantic review | `ed5df980bda88c12af6dffa53e622240dd1e6afde1bdbd39829eec374c72e536` | | screening evidence | `0d57129271a97b59e0eac9ed96a4922e45b01d380b0694ce1e0d75fbb9bdb074` | | factorial analysis | `87e5e93c8e7db734a4b5a2b69e02741e8ce7af261226523a28510ef5686f6d22` | | selection | `1654c760effda3f3cb6f8bb5605f7d241a68b508d54ad8921d261ac5e61b4e73` | MCP не сообщил actual credit delta, token counts, per-job latency или retry counts. Capture фиксирует эти пять полей как `null`; значения не оценивались. Submitted и terminal batch tokens не byte-equal: provider добавил один дефис в label последнего job. IDs всех 24 jobs, terminal set и output set совпали exactly, поэтому request-to-raw binding остаётся однозначным. Расхождение сохранено в provider transport audit. ## Fresh terminal verification | Проверка | Результат | |---|---| | Full-chain recomputation | fresh `finalize-screening` из canonical raw bytes повторил все три output SHA-256 byte-for-byte и снова вернул `no_eligible_config` | | Independent integrity audit | 24/24 exact request → job → raw → score → blind review → evidence bindings; 9/9 terminal artifact hashes и 17/17 header/file bindings совпали | | `npm run check` | 5/5 research-site и 139/139 technical tests; typecheck exit 0; 13 studies valid; 109 documents built; 309 HTML pages, 32435 local links и 3764 anchors checked; 0 failures | | Local Chrome E2E | library search → click `TECH.IDX` → rendered report; `complete`, `fail`, `24/24`, `5/6`, `no_eligible_config` и confirmation stop видимы; console errors 0 | | Cloudflare Pages | `npm run deploy` exit 0; canonical production report повторно проверен в браузере | | 390×844 production E2E | report → click `Проверка и итог` → terminal validation; document width 390 px, horizontal overflow false; console errors 0 | | Design detector | bounded `impeccable detect` завершён; три non-blocking findings: inherited black skip-link color, em-dash cadence и AC-01…AC-06 sequence | ## Verdict `fail`. Полный verified screening показал полезный средний prompt effect, но ни один config не прошёл all-cell hard gate. Лучший deterministic результат `semantic-v2--medium` равен 5/6, с semantic floor 4/6 и двумя critical failures. Confirmation и production transfer запрещены; следующий шаг требует отдельного исследования FactLedger или bounded repair без ослабления gates.