SGQN ranks first on 4 of 8 continuous-control tasks

Selective Growing Q-Networks ranked first on four of eight DeepMind Control Suite tasks in an arXiv preprint submitted Oct. 14 by Parham Nooranbakht, Seyed Mahdi Basiri Azad and Joschka Bödecker.

The result gives value-based Q-learning, a reinforcement-learning approach that estimates action returns, a competitive showing in continuous-control problems: SGQN finished within 1.1% of the best method on three more tasks, the preprint said.

SGQN keeps decoupled Q-learning but changes how possible actions are represented. Instead of expanding every action bin uniformly, it adds bins near high estimated Q-values, prunes low-value bins and uses epsilon-greedy choice, which usually tests random actions with a small probability.

The authors wrote that SGQN used an average 7.5 of 9 available bins per action dimension across tasks with 2 to 12 dimensions. “We present SGQN as a competitive value-based method and as an analysis of which ingredients of a growing action grid matter,” they wrote in the preprint.

Action discretization converts a continuous command into a set of selectable bins. The trade-off is that a coarse grid can help early search, while a finer grid can improve precision later. Growing Q-Networks use that idea by moving from coarse to fine resolution uniformly.

The paper’s central caution is that the adaptive grid was not the explanation the authors could support. Growth activated every candidate bin and functioned like uniform growth, while removing pruning changed final return by no more than 1.1%, according to the preprint.

Two other design choices mattered more in the authors’ tests. Starting with all 9 bins produced collapsed seeds, while replacing epsilon-greedy selection with an annealed softmax, a probability-weighted action rule, cut returns by 23% to 100%.

The comparison also leaves a confound the authors did not control. SGQN used one gradient update per environment step, compared with one per eight for the GQN and DecQN baselines, so the paper does not assign its advantage to one mechanism.

The paper is 19 pages with 6 figures and 6 tables, and arXiv lists it as under review at Transactions on Machine Learning Research. As an arXiv preprint, it has not completed peer review.

Source: arXiv preprint, Oct. 14, 2026