# Remix Mode — archived development observations

These excerpts preserve the project’s frame comparison and numerical pilot,
recorded in `scripts/variorum-scorer/README.md` at repository commit
`6678cf609c4611d5237c1df274242eba58b6db1e`. They were archived for _The unchosen word_
on 3 October 2026. These are development observations, not a peer-reviewed
validation study. The underlying pilot logs are not included in this archive.
The precision pilot predates the current exhaustive joint-table format and does
not provide an error bound for the essay’s frozen record.

**Editorial qualification.** The precision table below describes 0.9–1.3 nats
as error relative to the poet’s reading, while the accompanying paragraph
reports a maximum 0.18-nat menu change in a five-poem check. The surviving notes
do not establish how these metrics or experimental populations relate. Without
the underlying logs, their apparent discrepancy cannot be resolved here. The
excerpts are preserved as development history, not as a reconciled accuracy claim.

## Precision (original heading: M5 Max, 2026-09-28)

These measurements cover arithmetic on the NVFP4 weights as shipped. They say
nothing about how closely NVFP4 tracks the original bf16 model.

The reference is the same NVFP4 weights dequantized to float32 and applied with
plain float32 matmuls, TF32 off. Error is in a poem's log-prob relative to the
poet's reading, which is all a menu's softmax sees:

| Setting                                  | Error          | Speed      |
| ---------------------------------------- | -------------- | ---------- |
| float32 activations, `MLX_ENABLE_TF32=0` | ≤ 0.03 nats    | 204 tok/s  |
| float32, TF32 on (MLX's default; used)   | 0.9–1.3 nats   | 621 tok/s  |
| bfloat16                                 | 3.4–5.4 nats   | ~750 tok/s |
| float16                                  | NaN (overflow) |            |

- `score.py` runs float32 activations with TF32 on, MLX's default: float32
  matmuls, quantized matmuls and attention run as TF32 (MLX
  `docs/src/usage/precision.rst`; on the M5, NAX MMAs with relaxed precision),
  about 4e-4 relative error per matmul, amplified across 60 layers. A menu is a
  softmax over candidates scored the same way, so most of that error is shared
  and cancels: rescoring the shift step of five poems in TF32 moved the odds of
  their 352 shift menus by at most 0.18 nats, and 6 menus exchanged one
  near-tied last word; a review (2026-09-29) found no bar moved by more than
  2.7 percentage points. The whole run uses one setting, because a mix of
  arithmetic would not cancel. `MLX_ENABLE_TF32=0` in the environment
  scores in full float32 at a third of the speed; each file records which.
- `check.py` runs in full float32 and compares the scorer's cached, batched
  path with a plain forward pass of each sequence alone, summed token by token through the end-of-turn
  token. It covers multi-token words, the words after each substitution, any
  padding length, pad token or batch size, candidates at several words in one
  batch (contexts sliced from one pass over the poem) and every reading scored
  as one prefix tree. Earlier runs matched within 0.0002 nats.
- Loader: with the reply opener Ollama sends (`<|channel>thought\n<channel|>`),
  our logprobs for Ollama's own replies match the ones Ollama reports to within
  0.1 to 0.2 nats.
- Peak memory 19 to 28 GB.

## Frame comparison

The model sees no titles: the collection's titles were added after the poems
were written. Over 24 of Jindal's poems, ln P per poem token:

| Frame                                               | Mean  | Median | Words after the first |
| --------------------------------------------------- | ----- | ------ | --------------------- |
| Plain text (the poem alone)                         | −9.47 | −8.51  |                       |
| Reply to "Write a poem."                            | −6.74 | −8.11  | −5.75                 |
| Reply to "Write a poem. … without a title."         | −6.83 | −7.93  | −5.89                 |
| Reply to "Write a poem.", opening with a blank line | −6.37 | −7.37  | −5.46                 |
| Reply headed "Untitled"                             | −6.39 | −7.33  | −5.53                 |

`score.py` uses the blank-line reply. In every frame the model opens a reply
with "The" at probability ≈ 1, so a poem's first word scores about −27 nats
whatever it is, and the first word is nearly always marked. That is kept on
purpose: the model's habits are part of what the page shows. The
instruction-tuned model assigns plain text odd probabilities: for example,
−19 nats for "Course" in "When in the Course of human events".