How rounding can amplify rollout–trainer differences, why sequence log-ratios develop a drift, and what a controlled experiment can tell us about choosing a precision format.
Selective human review changes which model errors we observe, and therefore the correction an estimate needs. Routing and correction have to be designed together.
A practical map of how GRPO-family methods estimate advantages, constrain policy updates, and balance reward improvement with continued exploration.
When a detector decides what gets investigated, it shapes the labels used to train and evaluate its successor.
