Skip to content

Add speculative decoding docs and tests#133

Draft
wirybeaver wants to merge 2 commits into
skyzh:mainfrom
wirybeaver:xuanyili/specdecode
Draft

Add speculative decoding docs and tests#133
wirybeaver wants to merge 2 commits into
skyzh:mainfrom
wirybeaver:xuanyili/specdecode

Conversation

@wirybeaver

@wirybeaver wirybeaver commented Jun 13, 2026

Copy link
Copy Markdown

Summary

  • add the Week 3 Day 4 speculative decoding tutorial and book navigation
  • add deterministic tests for speculative accept/reject, rewind, EOS, and cache release behavior
  • correct the roadmap interpretation: the reference implementation already existed in tiny_llm_ref.speculative_generate; this PR adds the missing docs and tests

TODO

  • Run pdm install -v in a local project environment.
  • Run pdm run check-installation.
  • Run pdm run test-refsol --week 3 --day 4.
  • Run pdm run test --week 3 --day 4.
  • Run pdm run main --solution tiny_llm_ref --loader week2 --model qwen3-4b --draft-model qwen3-0.6b with local model weights.

Tests Run

  • uv run --python 3.12 --with pytest --with mlx --with mlx-lm --with numpy --with setuptools --with nanobind==2.4.0 python -m pytest -q tests_refsol/test_week_3_day_4.py
  • python3 -m py_compile tests_refsol/test_week_3_day_4.py
  • mdbook build book (passes; existing environment warns that mdbook-katex is not installed and mdbook-toc was built for an older mdbook)

Note: I currently use uv for the targeted pytest run. Will use pdm for verifycation

@wirybeaver wirybeaver changed the title Add speculative decoding lesson Add speculative decoding docs and tests Jun 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant