ReaDy authors report 3.16x Qwen3.8–27B decoding speedup

Bohan Sun and five co-authors reported ReaDy, a speculative-decoding method, delivered an average 3.16 times speedup over autoregressive decoding in an arXiv preprint submitted Oct. 14.

The Oct. 14 preprint matters because faster inference can lower latency and GPU time for large language models; the authors said ReaDy beat DSpark by 7.97% in throughput across seven benchmarks.

Speculative decoding is an inference technique that drafts multiple tokens before a target model verifies them. The authors tested ReaDy with Qwen3.8-27B on a single NVIDIA B200 GPU, spanning mathematics, code generation and dialogue tasks.

The method targets hybrid linear attention, or HLA, models that keep compact recurrent states carrying prior context. “We introduce ReaDy, the first speculative decoding method, to the best of our knowledge, to leverage HLA recurrent states for position-specific context retrieval during drafting,” the authors wrote in the abstract.

ReaDy gives each draft position a learned query that retrieves additional context from the target model’s committed recurrent states, according to the preprint. A lightweight state reader then folds that information into draft representations through residual updates, while low-rank state reads and fused operators are designed to limit extra computation and data movement.

The authors positioned ReaDy as an extension of DSpark, using DSpark’s parallel backbone and sequential prediction head. In the reported tests, ReaDy outperformed DSpark on every benchmark and raised average acceptance length by 8.50%, a measure of how many drafted tokens are accepted before verification rejects or stops the draft.

The paper is an arXiv version 1 preprint, which means it has not been peer reviewed. The abstract does not provide independent replication, production workload results, cost measurements or evidence that the same gains hold beyond the tested Qwen3.8-27B setup and single NVIDIA B200 GPU.

The authors said the 13-page paper includes six figures and eight tables, with ablation studies examining reader capacity, state sources and residual placement. They also said the code is available through a GitHub repository linked from the arXiv record.

Source: arXiv preprint, Oct. 14, 2026