Reinforcement Learning & Post-Training

Reinforcement Learning Research: Inference, Fairness, Agents, and Applications

• A paper by Zijie Cheng, Yang Peng, and Zhihua Zhang establishes functional central limit theorems for both synchronous and asynchronous quantile temporal difference (QTD) learning in distributional RL, showing averaged QTD iterates converge weakly to a rescaled Brownian motion. Building on this, they propose an online inference method using random scaling that constructs an asymptotically pivotal statistic from the entire QTD path without storing the full iterate trajectory, substantially reducing memory requirements. ^[1]
• A paper by Jianhan Zhang et al. proposes a data-preprocessing algorithm for counterfactually fair RL (CF-RL) using a novel quantile distribution mapping method to sequentially estimate counterfactual states and rewards, subsuming common additivity assumptions as a special case. The authors theoretically prove bounds on per-step counterfactual unfairness and the infinite-horizon suboptimality gap, and validate the approach on numerical experiments and a real-world interventional digital health dataset. ^[2]
• Researchers introduced TRIAL (Trajectory-Relative Hindsight Distillation), an agentic RL framework that assigns turn-aligned supervision by computing a signed log-probability gap between ordinary and hindsight-conditioned response evaluations, then normalizing turn-level magnitudes across the realized trajectory. On WebShop with Qwen3-1.7B, TRIAL raised the success rate from 56.4% to 75.2% and the task score from 78.7% to 85.7%, outperforming GRPO across all eight tested combinations of backbone, environment, and evaluation metric. ^[3]
• A paper published in Scientific Reports proposes a digital twin-driven RL framework for software-defined electric vehicles (SDEVs) that unifies adaptive energy management, predictive health monitoring, and physics-informed virtual modeling in a single closed-loop architecture. The framework integrates battery modeling, remaining useful life estimation, and RL-based torque control with an adaptive digital twin that continuously synchronizes virtual and physical system states via online parameter updating. ^[4]
• A paper by Chien- et al. frames distributed quantum circuit compilation as a constrained Markov Decision Process, using a heterogeneous graph model to represent hardware, logical qubits, and circuit operations, with a policy trained via Proximal Policy Optimization to minimize EPR-pair consumption and communication makespan. Evaluation on benchmark circuits shows the RL policy matches state-of-the-art heuristics on structured workloads, with lookahead reward shaping yielding modest gains on unstructured circuits, though scalability is identified as a key bottleneck. ^[5]