Testing recurrent latent reasoning on arithmetic.

I implemented three families of GPT-2-based models: direct answer prediction, visible chain-of-thought generation and recurrent latent computation. Holding the task family and evaluation pipeline constant made architectural differences easier to interpret.

Carry-balanced addition was used because carries create identifiable difficulty. It supports overall accuracy measurement while also allowing errors to be grouped by carry pattern.

I created deterministic data generation and shared training and evaluation workflows so model variants could be compared against the same distributions. The system records accuracy and latency, with further analysis by digit length and carry vector.

• Controlled generation of arithmetic examples.
• Common experiment configuration across model variants.
• Evaluation by exact answer accuracy and inference cost.
• Failure analysis that goes beyond aggregate scores.

The recurrent variant reuses a computation block in latent space before producing an answer. This creates a tunable internal-compute budget without requiring the model to emit an equivalent number of reasoning tokens.

I also examined adaptive routing: allocating additional recurrence when an example appears to need it, rather than applying the maximum compute depth to every input.

Curriculum experiments progressively increased arithmetic difficulty instead of training only on the final digit length. The five-digit curriculum run reached 99.25% exact-answer accuracy.

I treated that result as evidence for the specific experiment, not a universal model claim. Carry-vector breakdowns, latency and transfer behaviour remained necessary to understand where performance came from and where it failed.

Model configuration, data generation, training and analysis were kept as distinct concerns. This reduced the chance that a change intended for one variant silently altered the evaluation of another.

The work strengthened my understanding of experiment architecture: deterministic inputs, explicit configuration and failure-level analysis are as important as the model implementation itself.

The experiments operate on a narrow synthetic task and small GPT-2 variants. Strong arithmetic accuracy does not establish equivalent gains on open-ended language tasks.

The next useful step is to test whether routing decisions remain stable on broader algorithmic tasks, then compare accuracy and latency at matched compute budgets.