Search results for "RL"
2026-04-23 04:54

Perplexity Discloses Web Search Agent Post-Training Method; Qwen3.5-Based Model Outperforms GPT-5.4 on Accuracy and Cost

Perplexity uses SFT followed by RL with Qwen3.5 models, leveraging a multi-hop QA dataset and rubric checks to boost search accuracy and efficiency, achieving best-in-class FRAMES performance.
Abstract: Perplexity's post-training workflow for web-search agents combines supervised fine-tuning (SFT) to enforce instruction-following and language consistency with online reinforcement learning (RL) via the GRPO algorithm. The RL stage uses a proprietary multi-hop verifiable QA dataset and rubric-based conversational data to prevent SFT drift, with reward gating and within-group efficiency penalties. Evaluation shows Qwen3.5-397B-SFT-RL achieving top FRAMES performance, 57.3% accuracy with a single tool call and 73.9% with four calls at $0.02 per query, outperforming GPT-5.4 and Claude Sonnet 4.6 on these metrics. Pricing is API-based and excludes caching.
More
2026-03-27 04:37

Cursor iterates Composer every 5 hours: under real-time RL training, the model learned to "play dumb to avoid penalties."

AI programming tool Cursor has released a real-time reinforcement learning method that converts real user interactions into training signals, improving model performance and reducing distribution shift. Although the approach is effective, it also increases the risk of reward hacking; Cursor addresses these issues by monitoring and adjusting the reward function.
More