UC-RAMP cuts MoE token latency by up to 22.3% in preprint

UC-RAMP reduced measured time per output token by 20.2% to 22.3% in a single-A100 test at 25% pruning, according to an arXiv preprint published Oct. 20.

The Oct. 20 preprint by Zukang Xu, Xinpei Gao, Xing Hu, Zhe Jiang and Dawei Yang matters for mixture-of-experts models because UC-RAMP also led evaluated methods in ten-task mean accuracy in five of six settings.

A reduction of about 20% means roughly one-fifth less measured decoding time than the unpruned model, according to Atlas360 calculations. The hardware scale is limited: the deployment result came from one A100 accelerator, not a larger reported cluster.

Mixture-of-experts, or MoE, models route computation through selected expert modules rather than use all parameters for every token. Pruning removes parts of a model to lower compute needs, but the authors said lower language-modeling perplexity, a measure of prediction error, does not always preserve task performance.

The authors describe UC-RAMP as a two-stage post-training pruning framework, meaning it is applied after a model has already been trained. In their formulation, one decision allocates remaining capacity across experts, while a second decision chooses which channels stay inside that capacity.

The preprint said: “finer pruning granularity alone does not guarantee the preservation of downstream capabilities, as similar language-modeling perplexity can accompany substantially different task performance.” That is the central claim behind separating expert-width allocation from channel selection.

At 50% routed-expert channel pruning, UC-RAMP beat the strongest evaluated baseline by 8.19, 3.96 and 6.14 percentage points across the three MoE models, respectively, according to the preprint. At both 25% and 50% pruning, it produced the highest ten-task mean accuracy among evaluated pruning methods in five of six settings.

In the authors’ design, CAPE reallocates how wide each expert remains by using calibration evidence across domains while enforcing an exact budget for each layer. RAMP then adjusts channel choices at those fixed widths through one-out, one-in swaps guided by a surrogate for channel interactions.

The deployment result is narrower than the accuracy test. The authors measured inference on a single A100 setup at 25% pruning and reported a 19.7% to 23.9% reduction in time to first token, the delay before generation starts, versus the unpruned model.

The evidence remains preliminary. The paper is version 1 on arXiv, is 14 pages with three figures, and arXiv preprints are not peer reviewed.

The arXiv page says the authors have made code available through a GitHub repository linked from the paper.

Source: arXiv preprint, arXiv cs.AI, Oct. 20, 2026