Two-Reward Method Trains AI Models Without Human Labels
Researchers led by Shourov Joarder propose combining answer-level and completion-level internal signals to train language models.
Updated on Nov. 5, 2026 in Language Learning
Shourov Joarder and three co-authors propose a training method for language models that pairs a cluster-voting answer-level reward with a token-level self-certainty reward. The authors report results approaching those of supervised RLVR on mathematical reasoning and code-generation benchmarks.
Why it matters
Current reinforcement learning for language models often depends on human annotations or gold-standard solutions, which are costly to produce. The authors say single-reward self-training approaches can suffer reward hacking, entropy collapse, and weakened reasoning structure, which this two-reward design aims to address.
The method uses reinforcement learning from internal feedback, in which a model generates its own training signals rather than relying on human annotations. The authors report stability and robustness gains over earlier unsupervised approaches on mathematical and code benchmarks, though exact figures were not provided.
The players
Shourov Joarder
Lead author of the paper proposing the two-reward internal-feedback training method for language models.
The details
The method combines a cluster-voting answer-level signal with a token-by-token self-certainty completion-level signal, with GDPO-based normalization balancing the two scales. KL-Cov regularization targets low-entropy token distributions. The authors plan to release the code.
Culture Shift
The method extends reinforcement learning with verifiable rewards (RLVR), which relies on human annotations or gold-standard solutions, by replacing those labels with signals the model generates itself. It reflects a broader research push to reduce the human-labeling bottleneck in training capable language models.
Readers who use AI tools may eventually see models trained with fewer human-labeled datasets, which could affect how quickly new capabilities reach consumer products. The authors plan to release code, which would let developers test the approach on their own tasks.
The takeaway
Training AI without human-labeled data is a growing research priority, and this two-reward design shows one way to pursue it while guarding against common failure modes. Readers interested in how models learn to reason can follow the authors' planned code release for hands-on testing.
Further reading
Explore more on Language Learning to see how training methods shape the way models handle reasoning tasks.
Source note: This article includes information reported by TokenPost.
