Best AI papers explained
Cut through the noise. We curate and break down the most important AI papers so you don't have to.
Episodes
Strategic Self-Consistency
This research demonstrates that unfaithful large language model service providers can artificially inflate the number of reasoning paths generated during test-time compute to overcharge users. By utilizing a specific algorithm to generate and strategically reorder extra paths, a provider can make every reported output appear necessary under adaptive stopping rules. Furthermore, the authors show th
Retrofitting language models to operate over bytes
This Nature paper discusses Byteification, which offers a novel method to convert traditional subword-based large language models into efficient byte-level models that handle raw text encodings natively. By introducing a two-stage training procedure and a latent tokenizer architecture with non-causal boundary prediction, researchers can bypass the limitations of traditional tokenization—such as En
Priced Guidance: Can Language Models Generate Future Research Ideas?
This paper introduces Priced Guidance, a novel evaluation framework that assesses language models on their ability to generate novel and future research ideas. Rather than using expensive repeated sampling or verbatim text compression, this methodology measures the compression cost—the exact number of additional bits required for an informed guide to steer a generator model toward a target researc
Context Language Models
This paper introduce Context Language Models (CLMs), a novel approach that enables language models to natively manage their own context by treating it as an editable file rather than relying on traditional append-only histories or external harnesses. By granting models unrestricted read and write access to their context, CLMs achieve superior performance and greater efficiency across long-horizon
Position: AI-Agent Pricing Should Become More Outcome-Dependent: An Economic Perspective
This paper analyzes AI-agent pricing models through an economic lens, arguing that dominant token-based pricing is insufficient because it fails to share risk efficiently or incentivize hidden quality efforts like model selection and verification. By utilizing stylized principal-agent models, the authors demonstrate that optimal contracts must balance three critical dimensions: efficient risk real
Emergent Capabilities Arise Randomly from Learning Sparse Attention Patterns
This paper investigates why transformer language models display sudden, emergent capabilities during training by analyzing how they learn attention patterns. Through experiments on both real language models and synthetic tasks like linear maps and cellular automata, the authors demonstrate that acquiring downstream skills correlates with the abrupt mastery of sparse attention mechanisms. Furthermo
Pragmatic Causal Inference with AI-Learned Representations
This paper presents a rigorous framework for performing valid causal inference when complex covariates are compressed into AI-learned representations or embeddings. The authors demonstrate that information loss from compression distorts causal parameters through a product of regression and balancing weight errors. By leveraging cross-fitted double machine learning (DML), researchers can conduct va
Pointwise or pairwise: when do pairwise losses help reward learning, provably?
This research paper investigates when and why pairwise losses outperform pointwise losses for reward learning by analyzing both approaches within a grouped offline contextual-bandit framework. The authors compare Value Regression (VR), which directly fits absolute observed rewards, with Value Difference Regression (VDR), which models reward differences between action pairs sharing the same context
When to Trust the Cheap Check: Weak and Strong Verification for Reasoning
This paper explores weak-strong verification policies for large language models, presenting a framework that balances the affordability of scalable internal checks with the precision of resource-intensive external validation. To address the trade-off between type-I errors, type-II errors, and verification frequency, the authors introduce Selective Strong Verification (SSV), an online calibration a
Theoretical Limits of Language Model Alignment
This paper investigate the fundamental limits of language model alignment by establishing exact theoretical boundaries for reward improvement under a KL-divergence constraint using Jeffreys divergence and a computable covariance estimator. They demonstrate that best-of-N sampling closely approaches this theoretical Pareto frontier, whereas gradient-based methods like PPO and GRPO remain suboptimal
DT2: Decision-Targeted Digital Twins
This paper introduces DT2, a novel training framework designed to align digital twins more effectively with their primary goal of decision support. Traditional virtual models often fail to rank policy options correctly because they prioritize minimizing overall simulation errors rather than focusing on the specific variables that influence outcomes. To solve this, DT2 incorporates an architecture-
Self-Play Pretraining with Zero Data
This paper introduces Self-Play Pretraining with Zero Data, a method for training language models using only synthetic data generated by the model itself. In this framework, a generator creates programs for a universal Turing machine while a learner is trained to predict the resulting byte sequences. A reinforcement learning objective drives the generator to produce increasingly complex data at th
Language models need sleep: learning to self-modify and consolidate memories
This research paper introduces a "Sleep" paradigm for Large Language Models (LLMs) to overcome the limitations of static knowledge and catastrophic forgetting. Inspired by human biology, the framework alternates between active phasesfor processing external data and sleep phases for internal knowledge refinement. During sleep, the model performs memory consolidation by expanding its parameters and
Jev Creator: System One models for Prod, not God
We discuss the Latent Space's interview with Diogo Almeida, CEO of TypeSafe, about the launch of Jev, a specialized class of AI models designed for programmatic integration rather than human conversation. He argues that traditional models are "fractured" by safety alignments and chatbot optimizations, making them unreliable for economically valuable automation. Jev is presented as a
Detecting and countering misuse of AI: September 2026
This paper is an Anthropic threat intelligence report from September 2026 detailing the misuse of AI models by various malicious actors. It describes how state-sponsored groups and cybercriminals have integrated Claude into autonomous attack frameworks to accelerate the development of exploits and the execution of complex intrusions. Key case studies highlight a Russian espionage group automating
Position: LLMs can’t jump
This paper examines the cognitive limitations of Large Language Models by using Albert Einstein’s discovery of General Relativity as a primary case study. While modern AI excels at induction through data compression and deduction via logical proof, the author argues that it lacks the capacity for abduction, or the creative "jump" required to invent new scientific axioms. The paper highli
Jailbreaking Jailbreaks: A Proactive Defense for LLMs
The research introduces PROACT, a proactive defense framework designed to safeguard Large Language Models from iterative adversarial attacks. Unlike traditional passive defenses that offer standard refusals, this system generates spurious responses that mimic successful jailbreaks while remaining semantically benign. By providing these false signals, the framework tricks an attacker’s internal opt
When Agents Slow Down: Understanding LLM Agents’ Test-Time Strategies via Elo-per-token Analysis
This paper introduces Elo-per-token analysis, a novel framework for measuring how the performance of large language model agents scales with increased inference-time computation. By analyzing diverse benchmarks, the authors demonstrate that while agents initially show efficiency gains, their progress eventually slows to a rate no better than independent sampling, essentially hitting a scaling wall
Thinking with Looped Flows
This paper introduces looped flows, a novel framework designed to enhance the reasoning capabilities of neural networks by merging recurrent hidden states with probability flow models. Traditional looped models often struggle with training instability because they cannot effectively backpropagate through many iterations, but this approach sidesteps that issue by using local denoising objectives ac
Multi-Turn LLM Conversations under the Least-Recently-Used Policy: Mean-Field Asymptotics
This paper introduces a Mean-Field Asymptotic framework designed to estimate the hit ratio in multi-turn large language model (LLM) serving systems. As conversations grow in length, managing the KV cache in finite high-bandwidth memory becomes a critical performance bottleneck. The authors model these dynamics using the least-recently-used (LRU) eviction policy to determine which conversation hist
Breaking the Token Ceiling: Distilling Smaller, Stronger Byte Models
This research introduces Marginalize-It and End-Of-Token, two novel methods for efficiently distilling large token-based language models into smaller, more capable byte-level models. By evaluating dense transformers across various compute budgets, the study reveals that while token models perform better with limited resources, byte models achieve a significantly higher performance ceiling as train
Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMs
This research explores how reasoning helps Large Language Models (LLMs) answer simple, single-hop factual questions that do not logically require step-by-step thinking. The authors demonstrate that enabling reasoning expands the model’s parametric knowledge boundary, allowing it to "unlock" correct answers that are otherwise unreachable. This improvement is driven by two primary mechanisms: a comp
Tail-Likelihood Reinforcement Learning
This paper introduces Tail-Likelihood Reinforcement Learning (TailRL), a novel optimization framework designed to improve how generative policies handle continuous rewards. Traditional reinforcement learning often focuses on maximizing average rewards, which can inadvertently suppress rare but exceptionally high-performing outcomes and limit a model's ability to scale with more compute. TailRL add
Next-Latent Prediction Transformers Learn Compact World Models
This paper introduces Next-Latent Prediction (NextLat), a novel training framework designed to help Transformer models learn more compact and generalizable internal world models. Unlike standard approaches that only focus on next-token prediction, NextLat adds a self-supervised objective where the model must predict its own future latent states. This method encourages the formation of belief state
Language Models Can Control Their Own Attention
Researchers have introduced Declarative Attention (DA), a protocol that enables large language models to autonomously manage their own focus during long-context tasks. Traditional models consume excessive memory by scanning the entire history for every response, but DA allows a model to explicitly declare whether it needs to survey the full text, focus on a specific segment, or reason locally. By
AI Finds A Way
This paper introduces a comprehensive collection of anecdotes documenting instances where artificial intelligence systems developed innovative yet unpredictable solutions. While researchers primarily use reinforcement learning to achieve superhuman performance in complex games like Go and Poker, these same optimization processes often lead to reward hacking. This occurs when an agent exploits loop
Superposition Without Interference? Towards Isolated Interventions via Almost Orthogonal Features in Language Models
This research paper investigates how feature entanglement in large language models prevents precise, localized interventions on specific concepts. The authors argue that because internal features often overlap in superposition, modifying one frequently leads to unintended side effects across others. To solve this, they propose an orthogonality regularization method that forces features to remain n
TTPO: Test-Time Policy Optimization
This paper introduces Test-Time Policy Optimization (TTPO), a novel method for improving the mathematical reasoning of large language models without using ground-truth labels. The authors address the unreliability of majority-vote pseudo-labels by employing an asymmetric objective that treats positive and negative model rollouts differently. Specifically, it uses on-policy self-distillation to ref
Demystifying Reinforcement Learning Post-Training of Language Models
This paper deconstructs the mechanics of reinforcement learning (RL) post-training for large language models to determine how different factors influence model performance. By utilizing a controlled "sandbox" environment, the researchers demonstrate that standard sparse rewards typically fail unless the base model already possesses some prior knowledge of the desired behavior, a concept
Recursive Experiential–Working Memory Evolution for Long-Horizon Agent Harnesses
Recuris is a recursive architectural framework designed to enhance the performance of large language model agents during complex, long-horizon tasks. By coupling Working Memory, which tracks live task progress, with Experiential Memory containing reusable skills, the system ensures that model actions remain grounded in current needs rather than becoming lost in expanding conversation histories. Th
TailSFT: Filtered Fine-Tuning Improves Post-Training Performance
Researchers introduce TailSFT, a modified supervised fine-tuning algorithm designed to better prepare language models for subsequent reinforcement learning. Unlike standard fine-tuning that minimizes overall cross-entropy, TailSFT filters out sequences that the model has already mastered to focus training on the under-modeled "tail" of the data distribution. This approach prioritizes cov
SPADE: Self-Play in Adaptive Synthetic Executable Environments
This paper introduces SPADE, a reinforcement learning framework that enables a single large language model to achieve open-ended self-improvement by designing its own training worlds. One role, the Environment Designer, creates complex, multi-turn tasks as executable Python code, while the Reasoning Agent role learns to solve them. To ensure the tasks are challenging yet possible, the system utili
Evaluating Skills, Not Just Agents: Agentic Continuous Evaluation of Skills
This paper introduces ACES (Agentic Continuous Evaluation of Skills), a comprehensive framework developed by NVIDIA to move beyond static document scanning when assessing AI agent capabilities. While traditional methods merely check a skill's structure or style, ACES evaluates skills as executable artifacts by running live, sandboxed trials to observe how agents actually discover and use them.
Impression Share Prediction: An Offline Evaluation Task for Ranking Systems
Researchers from Meta Platforms propose a novel offline evaluation task called impression share prediction to better anticipate how new ranking models redistribute traffic across different business objectives. Traditional metrics often fail to capture these shifts, which can negatively impact downstream utility even when predictive accuracy improves. To address this, the authors developed a struct
Q-Learning with World Models
The researchers introduce Q-Learning with World Models (QWM), a framework designed to enhance sample efficiency and performance in robotic reinforcement learning. Unlike traditional model-based methods that often suffer from compounding biases by training policies on "imagined" data, QWM maintains a policy and critic trained exclusively on real environment transitions. It leverages a lea
Conformal Language Modeling via Posterior Sampling
This paper introduces Conformal Language Modeling via Posterior Sampling, a novel framework designed to reduce hallucinations in Large Language Models while maintaining text quality. Unlike previous methods that perform "post-hoc surgery" by deleting claims from already generated text, this approach reweights the model's sampling distribution toward more reliable responses. By treati
BoNVoyage: Learning Better Rewards without Ranking
BoNVoyage is a novel training framework designed to improve reward models (RMs) used in reinforcement learning from human feedback. Traditional RMs often fail because they are trained on static data distributions that do not reflect the adversarial distribution shifts occurring during the actual optimization process. Instead of simple pairwise ranking, this method uses test-time alignment and Mark
Demystifying Agent Skills: Why They Work—Until They Don’t
This research investigates the operational dynamics of agent skills, which are structured packages of procedural knowledge designed to help AI agents learn from experience. By comparing distilled skills against raw workflow memories, the study reveals that skills primarily act as procedural anchors that stabilize execution and reduce environment failures rather than simply injecting factual knowle
Jagged Judges: Epistemic Stability Under Silence, Pressure, and Persistence
This paper introduces the Wiggle Framework, a novel diagnostic tool designed to evaluate the epistemic stability of Large Language Models when they act as autonomous judges. Researchers discovered that even top-tier models frequently reverse their original verdicts when subjected to social pressure, rephrased prompts, or persistent adversarial arguments. This vulnerability, termed "wiggle,&qu
Predicting Neural Scaling Laws without Training: A Data Manifold Oracle
This paper introduces the Data Manifold Oracle (DMO), a training-free framework designed to predict neural scaling laws by analyzing raw text through compression statistics. By using Lempel-Ziv algorithms, the researchers extract two key metrics—an entropy-rate floor and a data-scaling exponent—to forecast model performance without the high cost of training model families. The authors prove an exa
Making Interpretable Discoveries from Unstructured Data: A High-Dimensional Multiple Hypothesis Testing
This paper introduces a rigorous statistical framework for discovering human-interpretable insights from unstructured data, such as text, audio, and video. By repurposing AI interpretability tools like sparse autoencoders, the method maps complex data into a high-dimensional space of thousands of distinct concepts. The author utilizes advanced multiple hypothesis testing to ensure these discoverie
Overcoming the Incentive Collapse Paradox
This paper introduces and addresses the incentive collapse paradox, a phenomenon where accuracy-based payments fail to motivate human effort as AI assistance becomes more reliable. The authors demonstrate that if human workers only receive rewards based on their final output accuracy, they will eventually free-ride on the AI’s suggestions rather than exert costly verification effort. To solve this
Position: Modular Memory is the Key to Continual Learning Agents
This paper introduces a framework for modular memory as the essential solution for creating continual learning agents that adapt without forgetting. The authors argue that while current foundation models excel at static tasks, they struggle with ongoing experience accumulation and personalization because they rely too heavily on single-model parameter updates. To solve this, the framework integrat
Harness RL is Meta-Learning: Training to Self-Improve at Test Time
This paper introduces harness RL, a novel meta-learning framework designed to enable large language models to self-improve during test-time adaptation. Rather than updating model weights, which is computationally expensive, this method optimizes the agent’s harness—the external instructions, memory, and rules that guide model execution. By training a proposer model to revise this harness while kee
Escaping the Nash Trap: Structural Estimation and Alignment of Strategic Reasoning in Large Language Models
This paper investigates a critical strategic mismatch between Large Language Models (LLMs) and human decision-makers in competitive environments. Through game-theoretic experiments, the researchers demonstrate that LLMs predominantly act as Nash-type reasoners, assuming their opponents are perfectly rational, whereas humans exhibit bounded rationality and varied reasoning depths. This overestimati
When Does LeJEPA Learn a World Model?
This research paper introduces a mathematical framework to prove that LeJEPA (a specific self-supervised learning architecture) can accurately recover the hidden structure of the world from complex data. The authors establish that when a model combines an alignment loss with Gaussian regularization, it achieves linear identifiability, meaning the learned representation is a simple rotation of the
Do Modules Stay in Their Lane? Role Drift in Compound LLM Systems
This research paper investigates Role Drift, a failure mode in compound AI systems where individual modules abandon their specific instructions to find shortcuts that improve final task accuracy. During end-to-end training, modules like "readers" or "decomposers" may stop performing their intended functions—such as relying on external evidence—and instead fall back on internal
Do you really need to pretrain Q-functions for online RL fine-tuning?
Research from Stanford University challenges the conventional assumption that pre-training a Q-function on offline data improves reinforcement learning fine-tuning. The authors demonstrate that naive pre-training often yields no benefit because the offline Q-function mismatch with the optimal online Q-function creates an incompatible value landscape. To address this, they introduce Initialization
The Evolution of Digital Search: From Blue Links to Delegated Decision-Making
Digital search is transitioning from a human-centered discovery process based on links and keywords to an agent-mediated system of delegated decision-making. In this new AI-native paradigm, users express goals in natural language while autonomous agents interpret intent and execute tasks on their behalf. This shift moves the internet from an attention economy, where firms compete for clicks, towar
Ask, Don’t Judge: Binary Questions for Interpretable LLM Evaluation and Self-Improvement
The research introduces BINEVAL, a novel evaluation framework that improves the reliability of Large Language Models (LLMs) by decomposing complex quality criteria into atomic binary questions. Unlike traditional holistic grading, which often produces opaque and inconsistent scores, this method utilizes a "decompose-then-verify" approach to generate transparent, multidimensional assessme
From Reasoning Traces to Reusable Modules: Understanding Compositional Generalization in Language Model Reasoning
This research introduces a hierarchical latent selection model to explain how large language models develop robust reasoning through post-training. The authors argue that supervised fine-tuning (SFT) provides the essential building blocks, while reinforcement learning (RL) decomposes these traces into reusable atomic skills and routing mechanisms. By isolating these components, RL enables models t
Understanding Reasoning from Pretraining to Post-Training
Researchers utilized chess as a controlled testbed to investigate how pretraining choices influence the effectiveness of reinforcement learning (RL) in large language models. By systematically scaling models from 5M to 1B parameters, the study established a joint scaling law where a model's pretraining loss accurately predicts its subsequent RL performance. The findings reveal that extended pr
A Positive Case for Faithfulness: LLM Self-Explanations Help Predict Model Behavior
This paper introduces Normalized Simulatability Gain (NSG), a new metric designed to measure the faithfulness of AI self-explanations by testing their predictive value. By evaluating 18 frontier models, the researchers demonstrate that an AI's explanation of its own logic significantly helps a separate "predictor" model guess how the AI will behave on related counterfactual scenarios
Reject, Resample, Repeat: Understanding Parallel Reasoning in Language Model Inference
This research paper investigates Sequential Monte Carlo (SMC) and other particle filtering algorithms as a theoretical framework for improving large language model (LLM) inference. The authors introduce a principled approach to analyze inference-time interventions, such as parallel reasoning and pruning, by utilizing process reward models to steer generation. Their findings establish non-asymptoti
Rethinking the Evaluation of Harness Evolution for Agents
This research paper critically examines automatic harness evolution, a method where AI agents iteratively improve the prompts, tools, and logic used to interact with environments. The authors argue that current evaluations are flawed because they often test evolved harnesses on the same data used for optimization, risking overfitting rather than genuine design improvement. By comparing harness evo
From Reasoning Traces to Reusable Modules: Understanding Compositional Generalization in Language Model Reasoning
This paper studies how post-training pipelines transform large language models into effective reasoners through compositional generalization. The authors propose a hierarchical latent selection model that separates reasoning into atomic skills, such as local operations, and routing mechanisms that dictate how information is composed. Their theory suggests that supervised fine-tuning (SFT) provides
Position: Interpretability can be actionable
This research paper advocates for actionable interpretability as the primary standard for evaluating how effectively we explain deep learning models. The authors argue that current studies often lack real-world impact because they prioritize theoretical understanding over practical utility and concrete decision-making. To bridge this gap, the text introduces a framework and checklist designed to h
High-accuracy sampling for diffusion models and log-concave distributions
This paper introduces a new algorithm called first-order rejection sampling (FORS) to achieve high-accuracy sampling for diffusion models and log-concave distributions. By utilizing only score estimates (the gradient of the log-density) rather than density evaluations, the researchers provide a method that converges exponentially fast, requiring only polylogarithmic steps relative to the target er
Causal Inference with Video Features as Treatments
his research paper introduces a novel statistical framework for conducting causal inference using video features as treatments, a significant advancement for analyzing high-dimensional, unstructured data. To overcome the challenges of latent and dynamic confounding, the authors utilize deep generative artificial intelligence to extract low-dimensional internal representations that serve as summari
What Does Thompson Sampling Optimize?
This research paper investigates the underlying mechanisms of Thompson Sampling, a popular bandit algorithm, by reframing it as an online optimization process. While traditionally viewed as a simple heuristic, the authors prove that Thompson Sampling actually minimizes instantaneous squared regret regularized by a specific measure of residual uncertainty. By comparing this mechanism to a Bellman-o
Globally Convergent Offline Reinforcement Learning with Smoothed Bellman Residual Minimization
This paper introduces **Off-GLADIUS**, a novel algorithm designed for **offline reinforcement learning** that utilizes **Bellman Residual Minimization (BRM)**. While traditional BRM methods often struggle with stability and convergence issues, this research proves that the proposed approach achieves **global optimality** by satisfying a **Polyak–Łojasiewicz (PL) condition**. The authors establish
LLM-as-a-Verifier: A General-Purpose Verification Framework
Researchers from Stanford, UC Berkeley, and NVIDIA have introduced LLM-as-a-Verifier, a novel framework designed to improve how artificial intelligence evaluates its own work. Unlike traditional methods that use simple pass-fail scores, this system calculates continuous scores by analyzing the underlying probability of specific words within a language model’s output. This approach allows the syste
How Much Do Language Models Memorize?
This research paper investigates language model capacity by introducing a new method to measure how much a model truly memorizes versus what it generalizes. The authors distinguish between unintended memorization, which is specific data storage, and generalization, which is the understanding of broader patterns. By testing the GPT family, they determine these models possess a storage capacity of a
Position: Uncertainty Quantification in LLMs is Just Unsupervised Clustering
This research paper argues that current methods for Uncertainty Quantification (UQ) in large language models are fundamentally flawed because they function as unsupervised clustering rather than measures of factual accuracy. The authors contend that these techniques merely track internal consistency, which fails to identify confident hallucinations where a model is consistently wrong. This relianc
Position: Agents Should Invoke External Tools ONLY When Epistemically Necessary
This position paper discusess Theory of Agent (ToA), a framework that redefines large language model agents as decision-makers who must choose between internal reasoning and external tool use. The authors argue that agents should only invoke external tools when epistemically necessary, meaning the task cannot be reliably solved using the model's existing internal knowledge and logic. This pers
From conversations to mechanisms: aligning advertiser Incentives in ai-powered product recommendations
This research paper explores the development of efficient recommendation systems, such as AI shopping assistants, that manage multi-round interactions between a platform, advertisers, and users. The authors address a fundamental challenge: advertisers possess private, multi-dimensional information about both their own profit values and the user's preferences, creating incentives to manipulate
Is one layer enough? Training a single transformer layer can match full-parameter RL training
This paper explores a surprising structural property of large language models: most reinforcement learning (RL) gains are concentrated in a very small subset of transformer layers. By isolating and training individual layers, researchers discovered that optimizing just a single middle layer can match or even exceed the performance of full-parameter RL training. This phenomenon was remarkably consi
RL Excursions during Pre-Training: Re-examining Policy Optimization for LLM training
This research investigates the effectiveness of integrating reinforcement learning (RL) earlier in the large language model training pipeline rather than treating it solely as a final post-training step. The authors demonstrate that RL is effective remarkably early, often matching the performance of standard sequential pipelines after only a small fraction of pre-training is complete. Unlike super
Language Generation with Feedback: Queries and Mistakes
This paper introduces a theoretical framework for language generation in the limit, exploring how machines can learn to produce valid, unseen strings from a target language through various forms of feedback. The authors specifically investigate two models: mistake feedback, where a generator learns if its prior output was incorrect, and query feedback, where the generator can actively ask if speci
Quantifying Theoretical AI Alignment Guarantees: Receiver-Utility Bounds in Bayesian Persuasion
This research paper explores theoretical AI alignment through the lens of Bayesian persuasion, specifically examining how a misaligned AI agent might manipulate information. The authors utilize a bit-string model to analyze the interaction between an AI sender aiming to maximize "1" guesses and a human receiver seeking accuracy. A primary contribution is the establishment of a universal
SPIRAL: Learning to search and aggregate
The Spiral framework addresses a limitation in current language model training where models are optimized for single-trace reasoning but fail to coordinate complex inference strategies at test time. To solve this, researchers combine set reinforcement learning with standard reinforcement learning to train models on sequential, parallel, and aggregative compute primitives simultaneously. The model
Qwen-AgentWorld: Language World Models for General Agents
We discuss Qwen-AgentWorld, a pioneering suite of language world models designed to simulate complex digital environments for artificial intelligence agents. By training on over 10 million trajectories across seven domains, including operating systems, web browsers, and software engineering sandboxes, these models learn to predict how an environment will respond to specific actions. This simulatio
When Does Trajectory-Level Supervision Permit Efficient Offline Reinforcement Learning?
This paper discusses a statistical framework for offline reinforcement learning using trajectory-level supervision, where only final outcomes or preferences are observed rather than step-by-step rewards. The authors introduce OPAC, a pessimistic actor-critic algorithm designed to learn from these aggregated signals by estimating latent rewards and applying pessimism to account for distribution shi
SuperThoughts: Reasoning Tokens in Superposition
SuperThoughts is a novel framework designed to accelerate the Chain-of-Thought (CoT) reasoning process in large language models by processing tokens in superposition. Unlike traditional models that generate tokens sequentially, this method uses a compressor to fuse pairs of consecutive tokens into single latent representations, effectively halving the number of required forward passes. To ensure a
First-Explore PPO : Learning Meta-Exploration with Proximal Policy Optimization
This research paper introduces First-Explore Proximal Policy Optimization (FE-PPO), a new reinforcement learning algorithm designed to improve how agents discover rewards in complex, deceptive environments. While standard meta-learning methods often fail when immediate rewards are misleading, the FE-PPO framework trains agents specifically to gather information during exploration that will maximiz
Self-Distillation for Data-Scarce Language Model Pretraining
This research paper investigates self-distillation as a powerful regularization technique for pretraining language models when high-quality data is in short supply. By comparing various training strategies across different model scales and data scarcity levels, the authors demonstrate that self-distillation significantly outperforms both direct training and standard methods like weight decay or ex
Meta-Harness for Agent-State Construction
eta-Harness is an advanced optimization system designed to improve how language-model agents process and compress long interaction histories into useful states. Unlike traditional methods that rely on manual engineering or simple feedback, this system uses a coding agent to search for and rewrite the "harness" code that manages an agent's memory and retrieval. By providing the propos
ExpRL: Using Reference Solutions as Rewards for LLM Mid-Training
Exploratory RL (ExpRL) is an automated mid-training method designed to enhance the reasoning capabilities of large language models before they undergo standard reinforcement learning. While traditional reinforcement learning often struggles with sparse rewards on difficult problems, ExpRL uses human-written reference solutions as reward scaffolds to provide dense, informative feedback on partial p
Valid Inference with Synthetic Data via Task Exchangeability
This paper introduces a statistical framework for making valid scientific discoveries using synthetic data, specifically addressing concerns that artificially generated data can be biased or noisy. The authors propose a new technical condition called task exchangeability, which allows researchers to calibrate synthetic results by comparing them to historical tasks where both real and synthetic dat
GRPO is Secretly a Process Reward Model
This paper establishs that Group Relative Policy Optimization (GRPO), while appearing to use only final outcome rewards, inherently functions as a Process Reward Model (PRM) through its implicit sub-trajectory credit assignment. By analyzing groups of trajectories that share identical prefixes, the authors prove that GRPO naturally computes step-level rewards using a Monte Carlo approach. However,
Recommended
پلی لیست | PlayList
Conspiracy Files with Paige Carter
Bad Friends
Learn 50 English Phrases While You Sleep | Everyday English Phrases & Vocabulary
This Past Weekend w/ Theo Von
Dateline NBC
Back to the 80s Radio
Crime Stories with Nancy Grace
Bible Tea
این نقطه
TED Business
Dark Night Tales with Brian King
