Model measurement register
Register
ct-bench / snapshot 2026-10-21Operator-verified measurements, not independent replication. Classics: partial coverage; coding and ARC re-running at 8,192 tokens. Speed summaries supplied by operator; source logs linked where available.
13 task records
| Compare | Model / arm | Task / suite | Score | n | Recorded UTC | Evidence | ||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek-V4.1-Flash-1017ARM 1 / styles01 | ARCClassics | —% / re-running | — | |||||||||||||||||||||||||||||||
Run evidenceARM 1 / styles01
Per-task measurements
Hardware / self-reported 2 × NVIDIA DGX Spark / TP2 / EXL3 2.9bpw 262K server context; vision on Fresh ct-bench lm-eval artifact Completed 1024-token run; harness context 16,384. Displayed scores and stderr are percentages. Coding and ARC are re-running at 8192, not zero. Pending re-run with max_gen_toks=8192; invalid 1024-token coding/ARC results excluded. Normalized JSONSource evidence | ||||||||||||||||||||||||||||||||||
| DeepSeek-V4.1-Flash-1017ARM 1 / styles01 | gsm8k / flexible-extractClassics | 94.31% / verified | 1319 | |||||||||||||||||||||||||||||||
Run evidenceARM 1 / styles01
Per-task measurements
Hardware / self-reported 2 × NVIDIA DGX Spark / TP2 / EXL3 2.9bpw 262K server context; vision on Fresh ct-bench lm-eval artifact Completed 1024-token run; harness context 16,384. Displayed scores and stderr are percentages. Coding and ARC are re-running at 8192, not zero. Normalized JSONSource evidence | ||||||||||||||||||||||||||||||||||
| DeepSeek-V4.1-Flash-1017ARM 1 / styles01 | humanevalClassics | —% / re-running | — | |||||||||||||||||||||||||||||||
Run evidenceARM 1 / styles01
Per-task measurements
Hardware / self-reported 2 × NVIDIA DGX Spark / TP2 / EXL3 2.9bpw 262K server context; vision on Fresh ct-bench lm-eval artifact Completed 1024-token run; harness context 16,384. Displayed scores and stderr are percentages. Coding and ARC are re-running at 8192, not zero. Pending re-run with max_gen_toks=8192; invalid 1024-token coding/ARC results excluded. Normalized JSONSource evidence | ||||||||||||||||||||||||||||||||||
| DeepSeek-V4.1-Flash-1017ARM 1 / styles01 | ifeval / prompt-strictClassics | 51.39% / verified | 541 | |||||||||||||||||||||||||||||||
Run evidenceARM 1 / styles01
Per-task measurements
Hardware / self-reported 2 × NVIDIA DGX Spark / TP2 / EXL3 2.9bpw 262K server context; vision on Fresh ct-bench lm-eval artifact Completed 1024-token run; harness context 16,384. Displayed scores and stderr are percentages. Coding and ARC are re-running at 8192, not zero. Normalized JSONSource evidence | ||||||||||||||||||||||||||||||||||
| DeepSeek-V4.1-Flash-1017ARM 1 / styles01 | math500 / math_verifyClassics | 60.00% / verified | 500 | |||||||||||||||||||||||||||||||
Run evidenceARM 1 / styles01
Per-task measurements
Hardware / self-reported 2 × NVIDIA DGX Spark / TP2 / EXL3 2.9bpw 262K server context; vision on Fresh ct-bench lm-eval artifact Completed 1024-token run; harness context 16,384. Displayed scores and stderr are percentages. Coding and ARC are re-running at 8192, not zero. Normalized JSONSource evidence | ||||||||||||||||||||||||||||||||||
| DeepSeek-V4.1-Flash-1017ARM 1 / styles01 | mbppClassics | —% / re-running | — | |||||||||||||||||||||||||||||||
Run evidenceARM 1 / styles01
Per-task measurements
Hardware / self-reported 2 × NVIDIA DGX Spark / TP2 / EXL3 2.9bpw 262K server context; vision on Fresh ct-bench lm-eval artifact Completed 1024-token run; harness context 16,384. Displayed scores and stderr are percentages. Coding and ARC are re-running at 8192, not zero. Pending re-run with max_gen_toks=8192; invalid 1024-token coding/ARC results excluded. Normalized JSONSource evidence | ||||||||||||||||||||||||||||||||||
| DeepSeek-V4.1-Flash-1017ARM 2 / MiaAI kit | codeDecode speed | 38.2tok/s / verified | 3 | |||||||||||||||||||||||||||||||
Run evidenceARM 2 / MiaAI kit
Per-task measurements
Hardware / self-reported 2 × NVIDIA DGX Spark / TP2 / EXL3 2.9bpw Not recorded Operator-verified summary supplied 2026-10-21 Median of 3 greedy 512-token trials. Original per-trial speed artifact not located; timestamp is archive date, not execution time. Normalized JSONSource evidence | ||||||||||||||||||||||||||||||||||
| DeepSeek-V4.1-Flash-1017ARM 2 / MiaAI kit | proseDecode speed | 23.9tok/s / verified | 3 | |||||||||||||||||||||||||||||||
Run evidenceARM 2 / MiaAI kit
Per-task measurements
Hardware / self-reported 2 × NVIDIA DGX Spark / TP2 / EXL3 2.9bpw Not recorded Operator-verified summary supplied 2026-10-21 Median of 3 greedy 512-token trials. Original per-trial speed artifact not located; timestamp is archive date, not execution time. Normalized JSONSource evidence | ||||||||||||||||||||||||||||||||||
| DeepSeek-V4.1-Flash-1017ARM 2 / MiaAI kit | structuredDecode speed | 49.2tok/s / verified | 3 | |||||||||||||||||||||||||||||||
Run evidenceARM 2 / MiaAI kit
Per-task measurements
Hardware / self-reported 2 × NVIDIA DGX Spark / TP2 / EXL3 2.9bpw Not recorded Operator-verified summary supplied 2026-10-21 Median of 3 greedy 512-token trials. Original per-trial speed artifact not located; timestamp is archive date, not execution time. Normalized JSONSource evidence | ||||||||||||||||||||||||||||||||||
| DeepSeek-V4.1-Flash-1017ARM 1 / styles01 | needle / 194,871 tokens / depth 60%Long context | 100.6s / pass | 1 | |||||||||||||||||||||||||||||||
Run evidenceARM 1 / styles01
Per-task measurements
Hardware / self-reported 2 × NVIDIA DGX Spark / TP2 / EXL3 2.9bpw 262K server context; vision on Fresh Spark needle log; depth from operator record Generation cap and temperature not recorded in log. Timestamp is archive date. PASS: violet-harbor-7291 recovered. Total elapsed time; lower is faster. Normalized JSONSource evidence | ||||||||||||||||||||||||||||||||||
| DeepSeek-V4.1-Flash-1017ARM 1 / styles01 | codeDecode speed | 91.2tok/s / verified | 3 | |||||||||||||||||||||||||||||||
Run evidenceARM 1 / styles01
Per-task measurements
Hardware / self-reported 2 × NVIDIA DGX Spark / TP2 / EXL3 2.9bpw 262K server context; vision on Operator-verified summary supplied 2026-10-21 Median of 3 greedy 512-token trials. Original per-trial speed artifact not located; timestamp is archive date, not execution time. Normalized JSONSource evidence | ||||||||||||||||||||||||||||||||||
| DeepSeek-V4.1-Flash-1017ARM 1 / styles01 | proseDecode speed | 49.5tok/s / verified | 3 | |||||||||||||||||||||||||||||||
Run evidenceARM 1 / styles01
Per-task measurements
Hardware / self-reported 2 × NVIDIA DGX Spark / TP2 / EXL3 2.9bpw 262K server context; vision on Operator-verified summary supplied 2026-10-21 Median of 3 greedy 512-token trials. Original per-trial speed artifact not located; timestamp is archive date, not execution time. Normalized JSONSource evidence | ||||||||||||||||||||||||||||||||||
| DeepSeek-V4.1-Flash-1017ARM 1 / styles01 | structuredDecode speed | 102.0tok/s / verified | 3 | |||||||||||||||||||||||||||||||
Run evidenceARM 1 / styles01
Per-task measurements
Hardware / self-reported 2 × NVIDIA DGX Spark / TP2 / EXL3 2.9bpw 262K server context; vision on Operator-verified summary supplied 2026-10-21 Median of 3 greedy 512-token trials. Original per-trial speed artifact not located; timestamp is archive date, not execution time. Normalized JSONSource evidence | ||||||||||||||||||||||||||||||||||
No records match these filters. Try another model or suite.
Evaluation queue
No scores until completed- TensorFold 512Kqueued—
- Qwen3.8-27B NVFP4queued—
- Muse Glimmer 30Bqueued—
- Nemotron 3.5 Lightningqueued—
- Nemotron 3.5 Boltqueued—
- Nemotron 3.5 SuperVLqueued—
- GLM-5.3-Flashqueued—
- gpt-oss-120bqueued—
- Qwen3.5-35B-A3Bqueued—
HLE / 2,158 questions / text-only. Nightly schedule begins 21 October; no completed frontier score published.