Retrieval plays a central role in multi-hop question answering (QA), where
answering complex questions requires gathering multiple pieces of evidence. We
introduce an Agentic Retrieval System that leverages large language models
(LLMs) in a structured loop to retrieve relevant evidence with high precision
and recall. Our framework consists of three specialized agents: a Question
Analyzer that decomposes a multi-hop question into sub-questions, a Selector
that identifies the most relevant context for each sub-question (focusing on
precision), and an Adder that brings in any missing evidence (focusing on
recall). The iterative interaction between Selector and Adder yields a compact
yet comprehensive set of supporting passages. In particular, it achieves higher
retrieval accuracy while filtering out distracting content, enabling downstream
QA models to surpass full-context answer accuracy while relying on
significantly less irrelevant information. Experiments on four multi-hop QA
benchmarks -- HotpotQA, 2WikiMultiHopQA, MuSiQue, and MultiHopRAG --
demonstrates that our approach consistently outperforms strong baselines.
Authors' comments: 18 pages
Retrieval-Augmented Generation (RAG) systems enable large language models
(LLMs) instant access to relevant information for the generative process,
demonstrating their superior performance in addressing common LLM challenges
such as hallucination, factual inaccuracy, and the knowledge cutoff.
Graph-based RAG further extends this paradigm by incorporating knowledge graphs
(KGs) to leverage rich, structured connections for more precise and inferential
responses. A critical challenge, however, is that most Graph-based RAG systems
rely on LLMs for automated KG construction, often yielding noisy KGs with
redundant entities and unreliable relationships. This noise degrades retrieval
and generation performance while also increasing computational cost. Crucially,
current research does not comprehensively address the denoising problem for
LLM-generated KGs. In this paper, we introduce DEnoised knowledge Graphs for
Retrieval Augmented Generation (DEG-RAG), a framework that addresses these
challenges through: (1) entity resolution, which eliminates redundant entities,
and (2) triple reflection, which removes erroneous relations. Together, these
techniques yield more compact, higher-quality KGs that significantly outperform
their unprocessed counterparts. Beyond the methods, we conduct a systematic
evaluation of entity resolution for LLM-generated KGs, examining different
blocking strategies, embedding choices, similarity metrics, and entity merging
techniques. To the best of our knowledge, this is the first comprehensive
exploration of entity resolution in LLM-generated KGs. Our experiments
demonstrate that this straightforward approach not only drastically reduces
graph size but also consistently improves question answering performance across
diverse popular Graph-based RAG variants.
Retrieval Augmented Generation (RAG) has advanced software engineering tasks but remains underexplored in unit test generation. To bridge this gap, we investigate the efficacy of RAG-based unit test generation for machine learning (ML/DL) APIs and analyze the impact of different knowledge sources on their effectiveness. We examine three domain-specific sources for RAG: (1) API documentation (official guidelines), (2) GitHub issues (developer-reported resolutions), and (3) StackOverflow Q&As (community-driven solutions). Our study focuses on five widely used Python-based ML/DL libraries, TensorFlow, PyTorch, Scikit-learn, Google JAX, and XGBoost, targeting the most-used APIs. We evaluate four state-of-the-art LLMs -- GPT-3.5-Turbo, GPT-4o, Mistral MoE 8x22B, and Llama 3.1 405B -- across three strategies: basic instruction prompting, Basic RAG, and API-level RAG. Quantitatively, we assess syntactical and dynamic correctness and line coverage. While RAG does not enhance correctness, RAG improves line coverage by 6.5% on average. We found that GitHub issues result in the best improvement in line coverage by providing edge cases from various issues. We also found that these generated unit tests can help detect new bugs. Specifically, 28 bugs were detected, 24 unique bugs were reported to developers, ten were confirmed, four were rejected, and ten are awaiting developers' confirmation. Our findings highlight RAG's potential in unit test generation for improving test coverage with well-targeted knowledge sources. Future work should focus on retrieval techniques that identify documents with unique program states to optimize RAG-based unit test generation further.
Authors' comments: 11 pages + reference. Accepted as Research Track Paper at ICSE'26
Inspired by the success of reinforcement learning (RL) in Large Language
Model (LLM) training for domains like math and code, recent works have begun
exploring how to train LLMs to use search engines more effectively as tools for
retrieval-augmented generation. Although these methods achieve performance
improvement across QA benchmarks, many prioritize final answer correctness
while overlooking the quality of intermediate reasoning steps, which may lead
to chain-of-thought unfaithfulness. In this paper, we first introduce a
comprehensive evaluation framework for evaluating RL-based search agents,
covering three distinct faithfulness metrics: information-think faithfulness,
think-answer faithfulness, and think-search faithfulness. Our evaluations
reveal that a prototypical RL-based search agent, Search-R1, has significant
room for improvement in this regard. To foster faithful reasoning, we introduce
VERITAS (Verifying Entailed Reasoning through Intermediate Traceability in
Agentic Search), a novel framework that integrates fine-grained faithfulness
rewards into the reinforcement learning process. Our experiments show that
models trained with VERITAS not only significantly improve reasoning
faithfulness, but also achieve comparable task performance across seven QA
benchmarks.
Large Language Models (LLMs) have emerged as promising assistants for
scientific writing. However, there have been concerns regarding the quality and
reliability of the generated text, one of which is the citation accuracy and
faithfulness. While most recent work relies on methods such as LLM-as-a-Judge,
the reliability of LLM-as-a-Judge alone is also in doubt. In this work, we
reframe citation evaluation as a problem of citation attribution alignment,
which is assessing whether LLM-generated citations match those a human author
would include for the same text. We propose CiteGuard, a retrieval-aware agent
framework designed to provide more faithful grounding for citation validation.
CiteGuard improves the prior baseline by 12.3%, and achieves up to 65.4%
accuracy on the CiteME benchmark, on par with human-level performance (69.7%).
It also enables the identification of alternative but valid citations.
Biomedical queries often rely on a deep understanding of specialized knowledge such as gene regulatory mechanisms and pathological processes of diseases. They require detailed analysis of complex physiological processes and effective integration of information from multiple data sources to support accurate retrieval and reasoning. Although large language models (LLMs) perform well in general reasoning tasks, their generated biomedical content often lacks scientific rigor due to the inability to access authoritative biomedical databases and frequently fabricates protein functions, interactions, and structural details that deviate from authentic information. Therefore, we present BioMedSearch, a multi-source biomedical information retrieval framework based on LLMs. The method integrates literature retrieval, protein database and web search access to support accurate and efficient handling of complex biomedical queries. Through sub-queries decomposition, keywords extraction, task graph construction, and multi-source information filtering, BioMedSearch generates high-quality question-answering results. To evaluate the accuracy of question answering, we constructed a multi-level dataset, BioMedMCQs, consisting of 3,000 questions. The dataset covers three levels of reasoning: mechanistic identification, non-adjacent semantic integration, and temporal causal reasoning, and is used to assess the performance of BioMedSearch and other methods on complex QA tasks. Experimental results demonstrate that BioMedSearch consistently improves accuracy over all baseline models across all levels. Specifically, at Level 1, the average accuracy increases from 59.1% to 91.9%; at Level 2, it rises from 47.0% to 81.0%; and at the most challenging Level 3, the average accuracy improves from 36.3% to 73.4%. The code and BioMedMCQs are available at: https://github.com/CyL-ucas/BioMed_Search
Can we bring large-scale reasoning to local-scale compute? Small language models (SLMs) are increasingly attractive because they run efficiently on local hardware, offering strong privacy, low cost, and reduced environmental impact. Yet they often struggle with tasks that require multi-step reasoning or domain-specific knowledge. We address this limitation through instruction intervention at inference time, where an SLM retrieves structured reasoning procedures rather than generating them from scratch. Our method builds an Instruction Corpus by grouping similar training questions and creating instructions via GPT-5. During inference, the SLM retrieves the most relevant instructions and follows their steps. Unlike retrieval-augmented generation, which retrieves text passages, instruction retrieval gives the model structured guidance for reasoning. We evaluate this framework on MedQA (medical board exams), MMLU Professional Law, and MathQA using models from 3B to 14B parameters without any additional fine-tuning. Instruction retrieval yields consistent gains: 9.4% on MedQA, 7.9% on MMLU Law, and 5.1% on MathQA. Concise instructions outperform longer ones, and the magnitude of improvement depends strongly on model family and intrinsic reasoning ability.
Due to the exponential growth of big data in this digital era, an advanced method for effective information retrieval becomes essential. The basic objective of this paper is to propose a topology-based method for cognitive information retrieval (CIR) in big data environments. By using concepts such as cognitive similarity distances, metric spaces, retrieval topologies, etc., this paper aims to propose the semantic alignment between user queries and document repositories. The paper also extends this approach to incorporate logical connectives in cognitive information retrieval.
Large language models (LLMs) generate high-dimensional embeddings that
capture rich semantic and syntactic information. However, high-dimensional
embeddings exacerbate computational complexity and storage requirements,
thereby hindering practical deployment. To address these challenges, we propose
a novel training framework named Sequential Matryoshka Embedding Compression
(SMEC). This framework introduces the Sequential Matryoshka Representation
Learning(SMRL) method to mitigate gradient variance during training, the
Adaptive Dimension Selection (ADS) module to reduce information degradation
during dimension pruning, and the Selectable Cross-batch Memory (S-XBM) module
to enhance unsupervised learning between high- and low-dimensional embeddings.
Experiments on image, text, and multimodal datasets demonstrate that SMEC
achieves significant dimensionality reduction while maintaining performance.
For instance, on the BEIR dataset, our approach improves the performance of
compressed LLM2Vec embeddings (256 dimensions) by 1.1 points and 2.7 points
compared to the Matryoshka-Adaptor and Search-Adaptor models, respectively.
Authors' comments: Accepted by EMNLP2025
Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm to
enhance the factuality of Large Language Models (LLMs). However, existing RAG
systems often suffer from an unfaithfulness issue, where the model's response
contradicts evidence from the retrieved context. Existing approaches to
improving contextual faithfulness largely rely on external interventions, such
as prompt engineering, decoding constraints, or reward-based fine-tuning. These
works treat the LLM as a black box and overlook a crucial question: how does
the LLM internally integrate retrieved evidence with its parametric memory,
particularly under knowledge conflicts? To address this gap, we conduct a
probing-based analysis of hidden-state representations in LLMs and observe
three findings: knowledge integration occurs hierarchically, conflicts manifest
as latent signals at the sentence level, and irrelevant context is often
amplified when aligned with parametric knowledge. Building on these findings,
we propose CLEAR (Conflict-Localized and Enhanced Attention for RAG), a
framework that (i) decomposes context into fine-grained sentence-level
knowledge, (ii) employs hidden-state probing to localize conflicting knowledge,
and (iii) introduces conflict-aware fine-tuning to guide the model to
accurately integrate retrieved evidence. Extensive experiments across three
benchmarks demonstrate that CLEAR substantially improves both accuracy and
contextual faithfulness, consistently outperforming strong baselines under
diverse conflict conditions. The related resources are available at
https://github.com/LinfengGao/CLEAR.
The widespread adoption of Retrieval-Augmented Image Generation (RAIG) has
raised significant concerns about the unauthorized use of private image
datasets. While these systems have shown remarkable capabilities in enhancing
generation quality through reference images, protecting visual datasets from
unauthorized use in such systems remains a challenging problem. Traditional
digital watermarking approaches face limitations in RAIG systems, as the
complex feature extraction and recombination processes fail to preserve
watermark signals during generation. To address these challenges, we propose
ImageSentinel, a novel framework for protecting visual datasets in RAIG. Our
framework synthesizes sentinel images that maintain visual consistency with the
original dataset. These sentinels enable protection verification through
randomly generated character sequences that serve as retrieval keys. To ensure
seamless integration, we leverage vision-language models to generate the
sentinel images. Experimental results demonstrate that ImageSentinel
effectively detects unauthorized dataset usage while preserving generation
quality for authorized applications. Code is available at
https://github.com/luo-ziyuan/ImageSentinel.
Authors' comments: Accepted at NeurIPS 2025
Text--image retrieval is necessary for applications such as product
recommendation. Embedding-based approaches like CLIP enable efficient
large-scale retrieval via vector similarity search, but they are primarily
trained on literal caption-like text--image pairs and often fail to capture
abstract or persona-driven attributes common in product recommendation
applications (e.g., ``a gift for a mother who loves gardening''). In contrast,
state-of-the-art vision--language models (vLLMs) can align text with images in
a flexible manner, but their limited context window prevents them from directly
handling retrieval over large catalogs. We propose a framework that distills
the preference rankings of a powerful vLLM into an embedding-based system,
transferring its nuanced alignment abilities while maintaining the
inference-time scalability of an embedding-based approach. Experiments on
persona-driven product recommendation tasks demonstrate that our method
significantly outperforms existing embedding-based baselines, providing an
efficient solution for personalized text--image retrieval.
Large Language Models (LLMs) can serve as world models to enhance agent decision-making in digital environments by simulating future states and predicting action outcomes, potentially eliminating costly trial-and-error exploration. However, this capability is fundamentally limited by LLMs' tendency toward hallucination and their reliance on static training knowledge, which can lead to compounding errors that inhibit long-horizon simulations. To systematically investigate whether LLMs are appropriate for world modeling, we probe two core capabilities of world models--future state prediction and reward estimation--through three tasks: next-state identification, full-procedure planning alignment, and milestone transition recognition. Our analysis shows that while LLMs effectively capture immediate next states and identify meaningful state transitions, their performance rapidly degrades in full-procedure planning. This highlights LLMs' limitations in reliably modeling environment dynamics over long horizons. To address these limitations, we propose the Retrieval-augmented World Model (R-WoM), which grounds LLM simulations by incorporating factual, up-to-date knowledge retrieved from external tutorials. Experiments show that R-WoM achieves substantial improvements of up to 25.3% (OSWorld) and 18.1% (WebArena) compared to baselines, with particular advantages in longer-horizon simulations.
Designing protein binders targeting specific sites, which requires to
generate realistic and functional interaction patterns, is a fundamental
challenge in drug discovery. Current structure-based generative models are
limited in generating nterfaces with sufficient rationality and
interpretability. In this paper, we propose Retrieval-Augmented Diffusion for
Aligned interface (RADiAnce), a new framework that leverages known interfaces
to guide the design of novel binders. By unifying retrieval and generation in a
shared contrastive latent space, our model efficiently identifies relevant
interfaces for a given binding site and seamlessly integrates them through a
conditional latent diffusion generator, enabling cross-domain interface
transfer. Extensive exeriments show that RADiAnce significantly outperforms
baseline models across multiple metrics, including binding affinity and
recovery of geometries and interactions. Additional experimental results
validate cross-domain generalization, demonstrating that retrieving interfaces
from diverse domains, such as peptides, antibodies, and protein fragments,
enhances the generation performance of binders for other domains. Our work
establishes a new paradigm for protein binder design that successfully bridges
retrieval-based knowledge and generative AI, opening new possibilities for drug
discovery.
Safety alignment in large language models (LLMs) induces over-refusals --
where LLMs decline benign requests due to aggressive safety filters. We analyze
this phenomenon in retrieval-augmented generation (RAG), where both the query
intent and retrieved context properties influence refusal behavior. We
construct RagRefuse, a domain-stratified benchmark spanning medical, chemical,
and open domains, pairing benign and harmful queries with controlled context
contamination patterns and sizes. Our analysis shows that context arrangement /
contamination, domain of query and context, and harmful-text density trigger
refusals even on benign queries, with effects depending on model-specific
alignment choices. To mitigate over-refusals, we introduce
\textsc{SafeRAG-Steering}, a model-centric embedding intervention that steers
the embedding regions towards the confirmed safe, non-refusing output regions
at inference time. This reduces over-refusals in contaminated RAG pipelines
while preserving legitimate refusals.
Authors' comments: Preprint
Retrieval-augmented generation (RAG) systems trained using reinforcement
learning (RL) with reasoning are hampered by inefficient context management,
where long, noisy retrieved documents increase costs and degrade performance.
We introduce RECON (REasoning with CONdensation), a framework that integrates
an explicit summarization module to compress evidence within the reasoning
loop. Our summarizer is trained via a two-stage process: relevance pretraining
on QA datasets, followed by multi-aspect distillation from proprietary LLMs to
ensure factuality and clarity. Integrated into the Search-R1 pipeline, RECON
reduces total context length by 35\%, leading to improved training speed and
inference latency, while simultaneously improving RAG performance on downstream
QA benchmarks. Notably, it boosts the average EM score of the 3B model by
14.5\% and the 7B model by 3.0\%, showing particular strength in multi-hop QA.
RECON demonstrates that learned context compression is essential for building
practical, scalable, and performant RAG systems. Our code implementation is
made available at https://github.com/allfornancy/RECON.
Accurate recall from large scale memories remains a core challenge for memory
augmented AI assistants performing question answering (QA), especially in
similarity dense scenarios where existing methods mainly rely on semantic
distance to the query for retrieval. Inspired by how humans link information
associatively, we propose AssoMem, a novel framework constructing an
associative memory graph that anchors dialogue utterances to automatically
extracted clues. This structure provides a rich organizational view of the
conversational context and facilitates importance aware ranking. Further,
AssoMem integrates multi-dimensional retrieval signals-relevance, importance,
and temporal alignment using an adaptive mutual information (MI) driven fusion
strategy. Extensive experiments across three benchmarks and a newly introduced
dataset, MeetingQA, demonstrate that AssoMem consistently outperforms SOTA
baselines, verifying its superiority in context-aware memory recall.
Retrieval-Augmented Generation (RAG) is widely used to mitigate
hallucinations of Large Language Models (LLMs) by leveraging external
knowledge. While effective for simple queries, traditional RAG systems struggle
with large-scale, unstructured corpora where information is fragmented. Recent
advances incorporate knowledge graphs to capture relational structures,
enabling more comprehensive retrieval for complex, multi-hop reasoning tasks.
However, existing graph-based RAG (GraphRAG) methods rely on unstable and
costly relation extraction for graph construction, often producing noisy graphs
with incorrect or inconsistent relations that degrade retrieval quality. In
this paper, we revisit the pipeline of existing GraphRAG systems and propose
LinearRAG (Linear Graph-based Retrieval-Augmented Generation), an efficient
framework that enables reliable graph construction and precise passage
retrieval. Specifically, LinearRAG constructs a relation-free hierarchical
graph, termed Tri-Graph, using only lightweight entity extraction and semantic
linking, avoiding unstable relation modeling. This new paradigm of graph
construction scales linearly with corpus size and incurs no extra token
consumption, providing an economical and reliable indexing of the original
passages. For retrieval, LinearRAG adopts a two-stage strategy: (i) relevant
entity activation via local semantic bridging, followed by (ii) passage
retrieval through global importance aggregation. Extensive experiments on four
datasets demonstrate that LinearRAG significantly outperforms baseline models.
Scientific document retrieval is a critical task for enabling knowledge
discovery and supporting research across diverse domains. However, existing
dense retrieval methods often struggle to capture fine-grained scientific
concepts in texts due to their reliance on holistic embeddings and limited
domain understanding. Recent approaches leverage large language models (LLMs)
to extract fine-grained semantic entities and enhance semantic matching, but
they typically treat entities as independent fragments, overlooking the
multi-faceted nature of scientific concepts. To address this limitation, we
propose Pairwise Semantic Matching (PairSem), a framework that represents
relevant semantics as entity-aspect pairs, capturing complex, multi-faceted
scientific concepts. PairSem is unsupervised, base retriever-agnostic, and
plug-and-play, enabling precise and context-aware matching without requiring
query-document labels or entity annotations. Extensive experiments on multiple
datasets and retrievers demonstrate that PairSem significantly improves
retrieval performance, highlighting the importance of modeling multi-aspect
semantics in scientific information retrieval.
Retrieving relevant instructional videos from multilingual medical archives
is crucial for answering complex, multi-hop questions across language
boundaries. However, existing systems either compress hour-long videos into
coarse embeddings or incur prohibitive costs for fine-grained matching. We
tackle the Multilingual Video Corpus Retrieval (mVCR) task in the NLPCC-2025
M4IVQA challenge with a multi-stage framework that integrates multilingual
semantics, domain terminology, and efficient long-form processing. Video
subtitles are divided into semantically coherent chunks, enriched with concise
knowledge-graph (KG) facts, and organized into a hierarchical tree whose node
embeddings are generated by a language-agnostic multilingual encoder. At query
time, the same encoder embeds the input question; a coarse-to-fine tree search
prunes irrelevant branches, and only the top-ranked chunks are re-scored by a
lightweight large language model (LLM). This design avoids exhaustive
cross-encoder scoring while preserving chunk-level precision. Experiments on
the mVCR test set demonstrate state-of-the-art performance, and ablation
studies confirm the complementary contributions of KG enrichment, hierarchical
indexing, and targeted LLM re-ranking. The proposed method offers an accurate
and scalable solution for multilingual retrieval in specialized medical video
collections.
Authors' comments: Accepted to NLPCC 2025 (Springer), to appear November 2025
