MAPLE puts Gemma 4 26B-A4B on 6GB GPUs at 45.4 tokens/sec

Aman Sachan’s MAPLE GitHub repository said on Nov. 8 that it compressed Gemma 4 26B-A4B to a 4.75 GB GGUF file that generated 45.4 tokens a second.

The developer-published benchmark matters for edge AI because the repository compares the 4.75 GB container with a 9.92 GB base model and says it cut VRAM use by 32% on an RTX 4050 6GB laptop GPU.

The repo names Sachan as author and Independent Systems Research as organization. Its README describes the project as “Sub-2-BPW Structural Compression for Mixture-of-Experts (MoE) LLMs.”

MAPLE, short for Manifold-Activated Pruning of Latent Experts, targets mixture-of-experts models, which route each token through selected expert subnetworks rather than using the full model for every step. Tokens are the text units processed by large language models.

The repository says the checkpoint was profiled for Python programming, algorithm synthesis, data structures and formal STEM reasoning. It says MAPLE removed 80 dormant multilingual and general conversational experts, leaving an active 48-expert structure.

In the benchmark table, the original 128-expert Gemma 4 26B-A4B model used 6,141 MiB of paged VRAM, offloaded 16 of 30 layers and generated 6.9 tokens a second. The MAPLE container used 4,180 MiB of VRAM, kept all 30 layers on the GPU and generated 45.4 tokens a second, according to the README.

The repository says the pruning pipeline profiles routing frequency and activation energy with llama-imatrix, scores experts with a composite metric and assembles a sub-2-BPW GGUF container. BPW means bits per weight, a compression measure for model parameters.

The claims should be read as developer-published benchmark results rather than independently validated performance data. The repository links to a Hugging Face model repository and a direct download for the 4.75 GB GGUF file.

Source: GitHub repository, AmSach/maple, Nov. 8, 2026; Hacker News post, Nov. 8, 2026.