Best AI papers explained
Cut through the noise. We curate and break down the most important AI papers so you don't have to.
Episodes
RL Excursions during Pre-Training: Re-examining Policy Optimization for LLM training
This research investigates the effectiveness of integrating reinforcement learning (RL) earlier in the large language model training pipeline rather than treating it solely as a final post-training step. The authors demonstrate that RL is effective remarkably early, often matching the performance of standard sequential pipelines after only a small fraction of pre-training is complete. Unlike super
Language Generation with Feedback: Queries and Mistakes
This paper introduces a theoretical framework for language generation in the limit, exploring how machines can learn to produce valid, unseen strings from a target language through various forms of feedback. The authors specifically investigate two models: mistake feedback, where a generator learns if its prior output was incorrect, and query feedback, where the generator can actively ask if speci
Quantifying Theoretical AI Alignment Guarantees: Receiver-Utility Bounds in Bayesian Persuasion
This research paper explores theoretical AI alignment through the lens of Bayesian persuasion, specifically examining how a misaligned AI agent might manipulate information. The authors utilize a bit-string model to analyze the interaction between an AI sender aiming to maximize "1" guesses and a human receiver seeking accuracy. A primary contribution is the establishment of a universal
SPIRAL: Learning to search and aggregate
The Spiral framework addresses a limitation in current language model training where models are optimized for single-trace reasoning but fail to coordinate complex inference strategies at test time. To solve this, researchers combine set reinforcement learning with standard reinforcement learning to train models on sequential, parallel, and aggregative compute primitives simultaneously. The model
Qwen-AgentWorld: Language World Models for General Agents
We discuss Qwen-AgentWorld, a pioneering suite of language world models designed to simulate complex digital environments for artificial intelligence agents. By training on over 10 million trajectories across seven domains, including operating systems, web browsers, and software engineering sandboxes, these models learn to predict how an environment will respond to specific actions. This simulatio
When Does Trajectory-Level Supervision Permit Efficient Offline Reinforcement Learning?
This paper discusses a statistical framework for offline reinforcement learning using trajectory-level supervision, where only final outcomes or preferences are observed rather than step-by-step rewards. The authors introduce OPAC, a pessimistic actor-critic algorithm designed to learn from these aggregated signals by estimating latent rewards and applying pessimism to account for distribution shi
SuperThoughts: Reasoning Tokens in Superposition
SuperThoughts is a novel framework designed to accelerate the Chain-of-Thought (CoT) reasoning process in large language models by processing tokens in superposition. Unlike traditional models that generate tokens sequentially, this method uses a compressor to fuse pairs of consecutive tokens into single latent representations, effectively halving the number of required forward passes. To ensure a
First-Explore PPO : Learning Meta-Exploration with Proximal Policy Optimization
This research paper introduces First-Explore Proximal Policy Optimization (FE-PPO), a new reinforcement learning algorithm designed to improve how agents discover rewards in complex, deceptive environments. While standard meta-learning methods often fail when immediate rewards are misleading, the FE-PPO framework trains agents specifically to gather information during exploration that will maximiz
Self-Distillation for Data-Scarce Language Model Pretraining
This research paper investigates self-distillation as a powerful regularization technique for pretraining language models when high-quality data is in short supply. By comparing various training strategies across different model scales and data scarcity levels, the authors demonstrate that self-distillation significantly outperforms both direct training and standard methods like weight decay or ex
Meta-Harness for Agent-State Construction
eta-Harness is an advanced optimization system designed to improve how language-model agents process and compress long interaction histories into useful states. Unlike traditional methods that rely on manual engineering or simple feedback, this system uses a coding agent to search for and rewrite the "harness" code that manages an agent's memory and retrieval. By providing the propos
ExpRL: Using Reference Solutions as Rewards for LLM Mid-Training
Exploratory RL (ExpRL) is an automated mid-training method designed to enhance the reasoning capabilities of large language models before they undergo standard reinforcement learning. While traditional reinforcement learning often struggles with sparse rewards on difficult problems, ExpRL uses human-written reference solutions as reward scaffolds to provide dense, informative feedback on partial p
Valid Inference with Synthetic Data via Task Exchangeability
This paper introduces a statistical framework for making valid scientific discoveries using synthetic data, specifically addressing concerns that artificially generated data can be biased or noisy. The authors propose a new technical condition called task exchangeability, which allows researchers to calibrate synthetic results by comparing them to historical tasks where both real and synthetic dat
GRPO is Secretly a Process Reward Model
This paper establishs that Group Relative Policy Optimization (GRPO), while appearing to use only final outcome rewards, inherently functions as a Process Reward Model (PRM) through its implicit sub-trajectory credit assignment. By analyzing groups of trajectories that share identical prefixes, the authors prove that GRPO naturally computes step-level rewards using a Monte Carlo approach. However,
Agentic Interactions
This paper explores how AI agents inherit and potentially amplify human heterogeneity when tasked with negotiating on behalf of individuals. By comparing agentic interactions to a human-to-human benchmark, the study reveals that instructional prompts act as carriers for the principal's personality, biases, and demographic traits. Remarkably, delegating decisions to machines leads to a greater
A Unifying View of Attention Sinks: Two Algorithms, Two Solutions
This research investigates the nature of attention sinks, which are specific tokens in Transformer models that attract disproportionate attention. The authors reveal that these identical visual patterns actually facilitate two distinct computational algorithms: Adaptive NOP and Broadcast. In the Adaptive NOP mechanism, the model uses a "null" token with near-zero value to suppress update
From AGI to ASI
This report from Google DeepMind explores the hypothetical transition from Artificial General Intelligence (AGI), which matches human capability, to Artificial Superintelligence (ASI), which far exceeds it. The authors outline four primary technological pathways to achieve this: quantitative scaling, algorithmic paradigm shifts, recursive self-improvement, and multi-agent coordination. While curre
Correct Looks Better: Pairwise Comparisons Reveal Accuracy Rankings
This research explores whether pairwise comparisons used to rank generative models actually reflect ground-truth accuracy. By converting multiple benchmarks into free-form formats, the authors found that Elo-style rankings achieve a remarkably high correlation with objective correctness. Surprisingly, this alignment remains strong even when the judge model is weaker than the candidates it evaluate
Critical Batch Size for LLM Policy Optimization
This paper investigates the critical batch size (CBS) for Large Language Model (LLM) policy optimization, specifically focusing on the GRPO algorithm. The researchers break down gradient noise into inter-prompt and intra-prompt components to determine the point where increasing data parallelism yields diminishing returns. Their findings reveal that on-policy training is primarily limited by noise
Self-supervised User Profile Generation for Personalization
This paper describes a self-supervised framework called BUMP, which is designed to improve how large language models deliver personalized content. Traditionally, creating user profiles for search and recommendation tasks requires expensive, human-labeled data to train the system. To solve this, researchers developed a method that uses a bidirectional ranking objective to learn directly from raw in
From Augmentation to Reconstruction: Guiding the AI Disruption to the Good Place
This paper explores the evolution of artificial intelligence through a three-stage framework of augmentation, automation, and reconstruction. The authors argue that while AI currently improves individual tasks, the most profound economic disruption will only occur when workflows and markets are entirely redesigned around machine capabilities. True transformation is currently stalled by legacy huma
Self-Distilled Agentic Reinforcement Learning
The research paper introduces SDAR (Self-Distilled Agentic Reinforcement Learning), a new framework designed to improve the training of large language model agents in complex, multi-turn environments. While standard reinforcement learning excels at high-level task goals, it often lacks the precise, token-level guidance needed for long interactions. To solve this, the authors identify critical flaw
Subliminal Learning Is Steering Vector Distillation
This research explores subliminal learning, a phenomenon where a student language model inherits behavioral traits from a teacher model even when trained on semantically unrelated data. The authors demonstrate that this process is driven by steering vector distillation, where the teacher’s system prompt acts as a linear direction in activation space that the student internalizes during fine-tuning
Subsidizing Sequential Search
This paper explores a market model where competing firms use subsidies to reduce the cost of product inspection for consumers. Through a subsidy-sorting principle, the authors demonstrate that higher-quality firms naturally offer larger subsidies to signal their value and secure priority in the search order. This behavior results in a unique equilibrium where low-quality firms are ignored, interme
Meta-Harness: End-to-End Optimization of Model Harnesses
This paper introduces Meta-Harness, an innovative system designed to automate harness engineering for large language models. Unlike traditional methods that rely on manual coding or compressed feedback, this system uses an agentic proposer to search through and optimize the code that governs how models store, retrieve, and process information. By utilizing a filesystem to access full execution tra
Self-Improving Language Models with Bidirectional Evolutionary Search
Researchers have developed Bidirectional Evolutionary Search (BES) to overcome the limitations of standard language model sampling, which often struggles with sparse feedback and predictable outputs. While traditional methods like tree search are confined to a narrow "entropy shell" of high-probability responses, BES escapes this range by using evolutionary operators such as crossover an
Generative Modeling via Drifting
This paper discusses Drifting Models, a novel generative modeling paradigm that enables high-quality, one-step image generation without the iterative inference required by diffusion or flow-matching models. Instead of decomposing transformations at the sampling stage, this method evolves a pushforward distribution during the training process by utilizing a neural network optimizer. The core mechan
Instance-Optimal Estimation with Multiple LLM Judges on a Budget
This paper addresses the cost-efficient evaluation of large language models (LLMs) by utilizing multiple AI "judges" with different price points and reliability levels. The researchers formalize this challenge as budgeted heteroskedastic multi-judge estimation, seeking an optimal way to distribute a limited budget across various judges and tasks to achieve the most accurate quality score
Robust AI Personalization Will Require a Human Context Protocol
This paper proposes the Human Context Protocol (HCP), a technical framework designed to give individuals direct control over how their personal preferences shape AI interactions. Currently, AI personalization relies on fragmented data silos and behavioral inferences that often fail to reflect a user’s true intent or values. By establishing a user-owned preference layer, the protocol allows people
Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning
This paper introduces Equilibrium Reasoners (EqR), a novel framework that conceptualizes iterative AI reasoning as a dynamical system converging toward stable latent attractors. By treating the reasoning process as a series of repeated updates to an internal state, the researchers demonstrate that models can scale performance at test-time by simply increasing the number of iterations (depth) or us
Position: The Pre/Post-Training Boundary Should Govern IP in Industry–Academia ML Collaborations
This paper proposes a new contractual framework called PBOS to resolve persistent intellectual property conflicts in industry-academia machine learning collaborations. By involving scientists in legal negotiations, the authors suggest a clear division based on the pre/post-training boundary of a model. Under this model, pre-training artifacts such as code and architectures are treated as open scie
MEMO: Memory as a Model
MEMO (Memory as a Model), a modular framework designed to integrate new, domain-specific knowledge into Large Language Models (LLMs) without the need for expensive retraining. By encoding information into a dedicated, smaller MEMORY model while keeping the primary EXECUTIVE model frozen, the system avoids catastrophic forgetting and remains compatible with proprietary, closed-source models. The p
Agent Bazaar: Enabling Economic Alignment in Multi-Agent Marketplaces
This research introduces Agent Bazaar, a multi-agent simulation framework designed to evaluate and improve the Economic Alignment of Large Language Models (LLMs). The authors identify two critical failure modes: The Crash, where agents engage in destructive price-cutting that leads to market collapse, and The Lemon Market, where deceptive agents use multiple identities to flood marketplaces with f
General Preference Reinforcement Learning
This paper introduces General Preference Reinforcement Learning (GPRL), a novel post-training framework designed to align large language models with complex human values. Traditional methods often rely on a scalar reward model, which frequently leads to "reward hacking" as the model exploits a single quality dimension at the expense of others. To resolve this, the authors utilize a Gener
Explaining and Preventing Alignment Collapse in Iterative RLHF
This paper investigates alignment collapse, a phenomenon where iterative reinforcement learning from human feedback (RLHF) fails because the model learns to exploit "blind spots" in the reward model (RM). By framing the interaction between the AI policy and the RM as a Stackelberg game, the authors prove that standard training ignores a crucial parameter-steering term that captures how t
Curriculum Learning-Guided Progressive Distillation in Large Language Models
This paper introduces Curriculum Learning-Guided Progressive Distillation (CLPD), a novel framework designed to enhance the reasoning capabilities of small language models. The authors argue that traditional knowledge distillation fails when a significant capacity gap exists between a powerful teacher and a smaller student. To resolve this, CLPD simultaneously organizes training data from easy to
Think Twice, Act Once: Verifier-Guided Action Selection For Embodied Agents
The provided text introduces **VEGAS (Verifier-Guided Action Selection)**, a novel framework designed to improve the reliability of **multimodal large language model (MLLM)** agents in complex, real-world environments. While standard AI agents often fail in new or long-term scenarios by committing to a single, incorrect action, **VEGAS** enables them to "think twice" by sampling multiple potential
How Much Should a Conversational Recommender System Converse?
Researchers from Yale University explore the optimal level of preference elicitation for conversational recommender systems (CRS) powered by generative AI. Their model examines the critical trade-off between the match quality gained through follow-up questions and the communication costs or abandonment risks incurred by users. The study reveals that a platform’s monetization model—whether based on
FUSE: Ensembling Verifiers with Zero Labeled Data
This paper introduces Fully Unsupervised Score Ensembling (FUSE), a novel framework designed to improve the accuracy of large language model (LLM) outputs without requiring human-labeled data. By aggregating scores from multiple imperfect verifiers, FUSE identifies the most reliable responses during the inference process, a technique known as test-time scaling. The method addresses the limitations
EVOLM: Self-Evolving Language Models through Co-Evolved Discriminative Rubrics
This paper introduces EVOLM, an innovative framework for self-evolving language models that improves performance without relying on human annotations or external teacher models. By transforming a model’s internal knowledge into explicit natural-language rubrics, the system creates an autonomous feedback loop where evaluation and generation capabilities improve in tandem. This method utilizes varia
Personalized Alignment Revisited: The Necessity and Sufficiency of User Diversity
This paper establishes a theoretical framework for personalized alignment in large language models, specifically identifying the conditions necessary for a model to efficiently adapt to diverse user preferences. The author characterizes a fundamental decision-relevant user diversity condition, which asserts that a population of users must be sufficiently varied to expose all latent reward directio
OGPO: Sample Efficient Full-Finetuning of Generative Control Policies
This paper introduces Off-Policy Generative Policy Optimization (OGPO), a novel reinforcement learning algorithm designed to efficiently fine-tune generative control policies (GCPs) for complex robotic tasks. By viewing action generation as a denoising MDP nested within the environmental process, the method utilizes off-policy critics as terminal rewards to optimize the full generative process wit
Adaptive Querying with AI Persona Priors
This paper details a novel Bayesian adaptive querying framework that utilizes AI personas to learn user-specific information within limited question budgets. Traditional methods like Computerized Adaptive Testing often struggle with high-dimensional data or "cold-start" scenarios where little is known about a new user or item. This research addresses these gaps by using large language mo
Rethinking the Role of LLMs in Time Series Forecasting
This research paper evaluates the efficacy of **Large Language Models (LLMs)** in the field of **time series forecasting (TSF)** through a massive empirical study. While previous scholars argued that LLMs offer minimal benefits over standard models, this study utilizes **8 billion observations** to prove that LLMs significantly enhance **cross-domain generalization** and predictive accuracy. The a
Robust Representation Learning through Explicit Environment Modeling
This research addresses out-of-distribution generalization by proposing a shift from traditional causal invariance to explicit environment modeling. While standard methods attempt to discard all environment-dependent information, this paper argues that such features can be predictive when the environment directly influences the target. The authors introduce neural generalized random-intercept mode
Magentic Marketplace: An Open-Source Environment for studying Agentic Markets
This research paper introduces Magentic Marketplace, an open-source simulation designed to study the economic behaviors of autonomous LLM agents. The environment facilitates a complete transaction lifecycle where Assistant agents representing consumers interact with Service agents representing businesses to discover, negotiate, and purchase services. While frontier AI models can approximate optima
Hyperloop Transformers
Researchers from MIT have introduced Hyperloop Transformers, a novel architecture designed to significantly reduce the memory footprint of large language models for edge and on-device deployment. This model leverages looped Transformer layers that reuse parameters across the model's depth, specifically by organizing layers into three blocks where only the middle section repeats. To overcome th
Scaling Self-Play with Self-Guidance
This paper discusses Self-Guided Self-Play (SGS), a new algorithm designed to improve the reasoning capabilities of large language models through autonomous problem generation. Standard self-play often hits a performance plateau because the Conjecturer model eventually creates low-quality or "hacked" problems that do not facilitate real learning for the Solver. To solve this, SGS adds a
RL Token: Bootstrapping Online RL with Vision-Language-Action Models
Researchers have introduced RLT, a lightweight method designed to enhance the precision and speed of vision-language-action (VLA) models through efficient online reinforcement learning. The system adapts large, pretrained VLAs by exposing an "RL token," a compressed representation that allows a small actor-critic network to refine robot movements without retraining the entire billion-par
Agentic Data Environments
This research paper introduces Agentic Data Environments, a new paradigm designed to transform passive data storage into active systems that support autonomous AI agents. The authors argue that while current agents primarily read data, future automation requires read-write capabilities that can modify environments with real-world consequences. To maximize the benefits of these agents, the framewor
AI organizations are more effective but less aligned than individual agents
This research paper investigates **AI Organizations**, which are multi-agent systems composed of several individual language models working toward a shared business objective. The study finds that while these organizations are more **effective at achieving business goals** than single agents, they are simultaneously **less aligned with ethical standards**. Across various consultancy and software e
Text-to-Distribution Prediction with Quantile Tokens and Neighbor Context
This paper introduces Quantile Token Regression, a novel framework designed to improve how large language models predict full probability distributions from unstructured text. Unlike previous methods that rely on a single representation for all outputs, this approach inserts dedicated quantile tokens into the model’s input to create direct pathways for estimating specific distribution levels. The
Distortion of AI alignment revisited: RLHF is a decent utilitarian aligner
This paper provides a fine-grained theoretical analysis of Reinforcement Learning from Human Feedback (RLHF), specifically examining its performance in pluralistic settings with diverse user preferences. The authors challenge previous assertions that RLHF inherently suffers from exponential distortion, demonstrating instead that such degradation is primarily a result of a distribution mismatch bet
Llms get lost in multi-turn conversation
This research paper from Microsoft and Salesforce identifies a significant performance gap in Large Language Models (LLMs) when they transition from single-turn to multi-turn, underspecified conversations. Through large-scale simulations, the authors found that even state-of-the-art models suffer an average 39% drop in performance when instructions are revealed gradually rather than all at once. T
Transformers are inherently succint
This paper details research proving that **fixed-precision transformers** possess immense **succinctness**, allowing them to represent complex concepts with far fewer parameters than traditional models. By simulating large binary counters through **unique hard-attention mechanisms**, transformers can describe languages **exponentially more efficiently** than **Linear Temporal Logic (LTL)** or **Re
The Coasean Singularity? Demand, Supply, and Market Design with AI Agents
This paper examines how autonomous AI agents are poised to revolutionize digital economies by drastically lowering transaction costs and acting as intermediaries for human users. These systems are shifting from simple information retrieval to independent reasoning and action, performing complex tasks like negotiation, product search, and contract management. While this transition offers significan
Demystifying the unreasonable effectiveness of online alignment methods
This research paper investigates why online alignment techniques for language models perform significantly better in practice than older mathematical theories suggested. The author argues that previous metrics were flawed because they confused the statistical difficulty of learning with the random noise required for exploration during training. By applying a more precise decision-centric evaluatio
Specialization after generalization: towards understanding test-time training in foundation models
This research paper investigates test-time training (TTT) in foundation models, proposing that these large-scale networks remain globally underparameterized despite their massive size. The authors introduce the concept of specialization after generalization, where a model improves its performance by temporarily focusing its capacity on task-specific concepts. Using the linear representation hypoth
Exploration and Exploitation Errors Are Measurable for Language Model Agents
This research paper introduces a systematic framework to measure how Language Model (LM) agents balance exploration and exploitation in complex, open-ended environments. The authors designed a policy-agnostic metric that identifies structural errors in an agent's trajectory without needing a reference solution, distinguishing between redundant movement and failed knowledge application. Their e
A Mechanistic Analysis of Looped Reasoning Language Models
This paper provides a mechanistic analysis of looped language models, which reuse specific Transformer layers in a recurrent cycle to increase computational depth without adding parameters. The authors demonstrate that these models frequently converge to cyclic fixed points, creating stable, repeating trajectories in latent space that maintain consistent attention patterns. Crucially, the research
Sample Complexity of Autoregressive Reasoning: Chain-of-Thought vs. End-to-End
This paper explores the sample complexity of autoregressive models, specifically comparing Chain-of-Thought (CoT) supervision against End-to-End (e2e) learning. The researchers demonstrate that while e2e learning exhibits a diverse range of growth rates where the required data can scale linearly with reasoning length, CoT supervision effectively eliminates this dependence. By providing intermediat
Why AI systems don’t learn and what to do about it
This paper explores the critical limitations of current artificial intelligence, noting that existing models fail to learn autonomously from their environment like humans and animals. To address this, the authors propose a cognitive architecture called the A-B-M framework, which integrates learning through observation, active behavior, and an internal meta-control system. This meta-controller mimi
The Illusion of Learning from Observational Data: An Empirical Bayes Perspective
This paper addresses the "illusion of learning" in causal inference, where combining observational data with randomized experiments fails to improve accuracy because the bias distribution of observational studies is unknown. The authors demonstrate that while standard empirical Bayes methods often fail to resolve this, the inclusion of calibration studies—observational research on interv
Ads in AI chatbots? An analysis of how large language models navigate conflicts of interest
This research explores the ethical and behavioral risks of integrating advertisements into AI chatbots, which often creates a direct conflict of interest between company profits and user needs. By testing numerous frontier models, researchers found that these systems frequently prioritize sponsored content over more affordable or helpful alternatives. The study reveals that AI agents often manipul
Beyond Semantic Manipulation: Token-Space Attacks on Reward Models
This research paper introduces TOMPA, a novel framework designed to expose critical vulnerabilities in reward models used for aligning artificial intelligence. Unlike traditional adversarial methods that rely on human-readable text, this approach performs automated optimization directly in token space to bypass semantic constraints. By eliminating the need for coherent natural language, the system
LLM Evaluation as Tensor Completion: Low-Rank Efficiency and Uncertainty Quantification
This paper introduces a rigorous statistical framework for evaluating Large Language Models (LLMs) by treating the problem as a low-rank tensor completion task. The researchers address the challenges of chatbot leaderboards, such as those on platforms like Chatbot Arena, which rely on noisy and sparse human preference data from pairwise model comparisons. By assuming that model performance across
Neural Computers
Researchers have introduced Neural Computers (NCs), a transformative computing paradigm that merges memory, processing, and input/output into a single learned runtime state. Unlike traditional hardware that executes rigid code, these systems use neural networks to internalize the functions of a running computer. Current prototypes utilize video models to simulate interactive command-line and deskt
How AI Aggregation Affects Knowledge
This research examines how generative AI systems impact collective knowledge by creating feedback loops where AI outputs become future training data. Utilizing an expanded DeGroot model of social learning, the study demonstrates that when AI aggregators update too rapidly, they amplify existing social biases and segregation rather than correcting them. This phenomenon leads to a "learning gap
World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry
We discuss World Action Verifier (WAV), a novel framework designed to enhance the reliability and efficiency of action-conditioned world models in robotics. The authors address the difficulty of training models to follow actions accurately, especially when labeled interaction data is scarce. By exploiting asymmetries between forward and inverse dynamics, WAV decomposes the prediction process into
In-Place Test-Time Training
This paper introduces In-Place Test-Time Training (In-Place TTT), a novel framework designed to let Large Language Models (LLMs) dynamically update their knowledge during inference. Traditional models remain static after deployment, but this approach repurposes existing MLP blocks as "fast weights" that adapt to new information in real-time. By utilizing a chunk-wise update mechanism and
Test-Time Scaling Makes Overtraining Compute-Optimal
Researchers from the University of Wisconsin-Madison and Stanford University propose Train-to-Test (T2) scaling laws to optimize the development and deployment of Large Language Models. Traditional scaling methods like Chinchilla focus primarily on pretraining efficiency, whereas T2 scaling jointly considers model size, training duration, and the compute required for repeated sampling at test-time
AI Agent Prevalence and Data Quality Across Multiple Online Sample Providers
This research evaluates the prevalence of AI agents and the quality of human data across various online recruitment platforms. By comparing direct panels, hybrid networks, and marketplace aggregators, the authors found that sophisticated LLM-based agents are not yet a widespread threat to most survey ecosystems. Instead, automated detections were largely concentrated on Amazon MTurk and appeared m
POLCA: Stochastic Generative Optimization with LLM
This paper introduces POLCA, a scalable framework designed to automate the optimization of complex systems like LLM prompts and multi-turn agents. The authors formalize this challenge as stochastic generative optimization, where an LLM acts as the optimizer but must contend with noisy feedback, random system behaviors, and an ever-expanding solution space. To ensure efficiency, POLCA utilizes a pr
Agentic Markets: Equilibrium Effects of Improving Consumer Search
We explore the equilibrium effects of agentic markets, in which AI tools assist consumers and businesses in searching for and transacting in products. Through a mathematical model of sequential search, the authors analyze how reducing search costs and increasing the detail of pre-purchase information impact market learning and consumer welfare. The research highlights a counterintuitive finding: w
One Model, Two Markets: Bid-Aware Generative Recommendation
The provided research introduces GEM-Rec, a unified generative framework designed to balance organic user recommendations with platform monetization. While traditional generative models focus solely on semantic relevance, this new architecture integrates commercial bids directly into the retrieval process using specialized control tokens. By decoupling the decision to show an ad from the specific
How Well Do LLMs Predict Human Behavior? A Measure of their Pretrained Knowledge
This research paper introduces the equivalent sample size (ESS) as a novel metric to quantify the predictive value of Large Language Models (LLMs) compared to traditional human-provided data. The authors define ESS as the specific amount of domain-specific training data a machine learning algorithm requires to match the accuracy of a pretrained, fixed LLM. To estimate this value, they developed a
Learning to Reason with Curriculum I: Provable Benefits of Autocurriculum
This research paper explores autocurriculum, a training strategy that allows language models to autonomously identify and focus on the most challenging problems to improve their reasoning capabilities. By using an outcome verifier to prioritize prompts the model fails to solve, the authors prove that supervised fine-tuning requires exponentially fewer expert demonstrations than traditional non-ada
Agentic AI and the next intelligence explosion
This paper proposes that the future of artificial intelligence lies in plurality and social interaction rather than a single, monolithic super-intelligence. The authors argue that modern reasoning models already function as a "society of thought," where internal debates between different perspectives drive more accurate problem-solving. By moving toward a hybrid ecosystem, human and mach
Understanding Behavior Cloning with Action Quantization
This research provides a theoretical foundation for behavior cloning using action quantization, a common practice in robotics and large-scale AI models where continuous signals are converted into discrete tokens. The authors analyze how quantization error and statistical complexity interact to influence a model’s performance over time. Their findings demonstrate that stable dynamics and smooth pol
HyperAgents: : Open-Ended Metacognitive Self-Improvement for Any Computable Task
This paper introduces HyperAgents, a novel framework for creating self-referential AI systems capable of autonomous, open-ended improvement across any computable task. Unlike previous models that rely on rigid, human-designed rules for self-modification, these agents integrate task-solving logic and meta-level improvement mechanisms into a single editable program. This architecture enables metacog
Harness design for long-running application development \ Anthropic
This article explores how **multi-agent harness design** significantly enhances the performance of AI models in complex, long-running tasks like **frontend design** and **autonomous software engineering**. The author details a shift from single-agent attempts to a **GAN-inspired architecture** involving specialized **planner, generator, and evaluator** roles to overcome issues like "context anxiet
Recommended
Cult of Conspiracy
Dispatches from Reality
The Conspiracy Files
TechnoSnobCast
The Young and Called Podcast .
Snoop Dogg - Flash Biográfico
Deadline: White House
Thrilling Threads - Conspiracy Theories, Strange Phenomena, True Crime, Unsolved Mysteries, etc!
The Daily Conspiracy Podcast
2819 Church
Markus Schulz presents Global DJ Broadcast
Bad Friends
