Research Papers

Trains LLMs to adaptively retrieve information and self-critique outputs using special reflection tokens, improving both accuracy and attribution.

Comprehensive survey of RAG techniques covering retrieval methods, generation approaches, and augmentation strategies for enhancing LLMs with external knowledge.

Introduces generative agents with memory, reflection, and planning capabilities that produce believable human-like behavior in a simulated environment.

Proposes decoupling memory from model parameters, using a frozen LLM with trainable memory retrieval to enable unlimited context through a memory bank.

Introduces a memory management system inspired by OS virtual memory, enabling LLMs to handle unlimited context through hierarchical memory and self-directed memory operations.

Shows that retrieval from a massive corpus (2 trillion tokens) can match the performance of 25x larger models, demonstrating retrieval as an efficient alternative to scaling parameters.

Introduces the ReAct paradigm where language models interleave reasoning traces with actions, enabling more grounded and interpretable agent behavior.

Demonstrates that LLMs can learn to use external tools (calculator, search, etc.) in a self-supervised way by generating and filtering their own training data.

Introduces Dense Passage Retrieval (DPR), showing that learned dense embeddings significantly outperform sparse methods like BM25 for open-domain QA retrieval.

Introduces end-to-end training of retrieval-augmented language models, jointly learning what to retrieve and how to use retrieved information during pre-training.