Retrieval-augmented generation (RAG) enhances large language models (LLMs) by
retrieving external documents. As an emerging form of RAG, parametric
retrieval-augmented generation (PRAG) encodes documents as model parameters
(i.e., LoRA modules) and injects these representations into the model during
inference, enabling interaction between the LLM and documents at parametric
level. Compared with directly placing documents in the input context, PRAG is
more efficient and has the potential to offer deeper model-document
interaction. Despite its growing attention, the mechanism underlying parametric
injection remains poorly understood. In this work, we present a systematic
study of PRAG to clarify the role of parametric injection, showing that
parameterized documents capture only partial semantic information of documents,
and relying on them alone yields inferior performance compared to interaction
at text level. However, these parametric representations encode high-level
document information that can enhance the model's understanding of documents
within the input context. When combined parameterized documents with textual
documents, the model can leverage relevant information more effectively and
become more robust to noisy inputs, achieving better performance than either
source alone. We recommend jointly using parameterized and textual documents
and advocate for increasing the information content of parametric
representations to advance PRAG.
Knowledge Hypergraphs (KHs) have recently emerged as a knowledge
representation for retrieval-augmented generation (RAG), offering a paradigm to
model multi-entity relations into a structured form. However, existing KH-based
RAG methods suffer from three major limitations: static retrieval planning,
non-adaptive retrieval execution, and superficial use of KH structure and
semantics, which constrain their ability to perform effective multi-hop
question answering. To overcome these limitations, we propose PRoH, a dynamic
Planning and Reasoning over Knowledge Hypergraphs framework. PRoH incorporates
three core innovations: (i) a context-aware planning module that sketches the
local KH neighborhood to guide structurally grounded reasoning plan generation;
(ii) a structured question decomposition process that organizes subquestions as
a dynamically evolving Directed Acyclic Graph (DAG) to enable adaptive,
multi-trajectory exploration; and (iii) an Entity-Weighted Overlap (EWO)-guided
reasoning path retrieval algorithm that prioritizes semantically coherent
hyperedge traversals. Experiments across multiple domains demonstrate that PRoH
achieves state-of-the-art performance, surpassing the prior SOTA model
HyperGraphRAG by an average of 19.73% in F1 and 8.41% in Generation Evaluation
(G-E) score, while maintaining strong robustness in long-range multi-hop
reasoning tasks.
Almost all previous text-to-video retrieval works ideally assume that videos
are pre-trimmed with short durations containing solely text-related content.
However, in practice, videos are typically untrimmed in long durations with
much more complicated background content. Therefore, in this paper, we focus on
the more practical yet challenging task of Partially Relevant Video Retrieval
(PRVR), which aims to retrieve partially relevant untrimmed videos with the
given query. To tackle this task, we propose a novel framework that distills
generalization knowledge from a powerful large-scale vision-language
pre-trained model and transfers it to a lightweight, task-specific PRVR
network. Specifically, we introduce a Dual Learning framework with Dynamic
Knowledge Distillation (DL-DKD++), where a large teacher model provides
supervision to a compact dual-branch student network. The student model
comprises two branches: an inheritance branch that absorbs transferable
knowledge from the teacher, and an exploration branch that learns task-specific
information from the PRVR dataset to address domain gaps. To further enhance
learning, we incorporate a dynamic soft-target construction mechanism. By
replacing rigid hard-target supervision with adaptive soft targets that evolve
during training, our method enables the model to better capture the
fine-grained, partial relevance between videos and queries. Experiment results
demonstrate that our proposed model achieves state-of-the-art performance on
TVR, ActivityNet, and Charades-STA datasets for PRVR. The code is available at
https://github.com/HuiGuanLab/DL-DKD.
Real-world use cases often present RAG systems with complex queries for which
relevant information is missing from the corpus or is incomplete. In these
settings, RAG systems must be able to reject unanswerable, out-of-scope queries
and identify failures of retrieval and multi-hop reasoning. Despite this,
existing RAG benchmarks rarely reflect realistic task complexity for multi-hop
or out-of-scope questions, which often can be cheated via disconnected
reasoning (i.e., solved without genuine multi-hop inference) or require only
simple factual recall. This limits the ability for such benchmarks to uncover
limitations of existing RAG systems. To address this gap, we present the first
pipeline for automatic, difficulty-controlled creation of
un$\underline{c}$heatable, $\underline{r}$ealistic, $\underline{u}$nanswerable,
and $\underline{m}$ulti-hop $\underline{q}$uerie$\underline{s}$ (CRUMQs),
adaptable to any corpus and domain. We use our pipeline to create CRUMQs over
two popular RAG datasets and demonstrate its effectiveness via benchmark
experiments on leading retrieval-augmented LLMs. Results show that compared to
prior RAG benchmarks, CRUMQs are highly challenging for RAG systems and achieve
up to 81.0\% reduction in cheatability scores. More broadly, our pipeline
offers a simple way to enhance benchmark difficulty and realism and drive
development of more capable RAG systems.
Retrieval-Augmented Generation (RAG) has emerged as a crucial approach for
enhancing the responses of large language models (LLMs) with external knowledge
sources. Despite the impressive performance in complex question-answering
tasks, RAG still struggles with hallucinations. Attributing RAG-generated
content through in-line citations has demonstrated potential in reducing
hallucinations and facilitating human verification. Existing citation
generation methods primarily rely on either fine-tuning the generator or
employing post-processing approaches for citation matching. However, the former
approach demands substantial annotated data and computational resources, while
the latter often encounters difficulties in managing multiple citations and
frequently produces suboptimal results. In this paper, we introduce a novel
framework, called VeriCite, designed to rigorously validate supporting evidence
and enhance answer attribution. Specifically, VeriCite breaks down into a
three-stage generation: 1) The initial answer generation first generates a
response based on all available contexts and has its claims verified through
the NLI model; 2) the supporting evidence selection assesses the utility of
each document and extracts useful supporting evidences; 3) the final answer
refinement integrates the initial response and collected evidences to produce
the final, refined answer.We conduct experiments across five open-source LLMs
and four datasets, demonstrating that VeriCite can significantly improve
citation quality while maintaining the correctness of the answers.
Modern enterprises manage vast knowledge distributed across heterogeneous
systems such as Jira, Git repositories, Confluence, and wikis. Conventional
retrieval methods based on keyword search or static embeddings often fail to
answer complex queries that require contextual reasoning and multi-hop
inference across artifacts. We present a modular hybrid retrieval framework for
adaptive enterprise information access that integrates Knowledge Base
Language-Augmented Models (KBLam), DeepGraph representations, and
embedding-driven semantic search. The framework builds a unified knowledge
graph from parsed repositories including code, pull requests, and commit
histories, enabling semantic similarity search, structural inference, and
multi-hop reasoning. Query analysis dynamically determines the optimal
retrieval strategy, supporting both structured and unstructured data sources
through independent or fused processing. An interactive interface provides
graph visualizations, subgraph exploration, and context-aware query routing to
generate concise and explainable answers. Experiments on large-scale Git
repositories show that the unified reasoning layer improves answer relevance by
up to 80 percent compared with standalone GPT-based retrieval pipelines. By
combining graph construction, hybrid reasoning, and interactive visualization,
the proposed framework offers a scalable, explainable, and user-centric
foundation for intelligent knowledge assistants in enterprise environments.
Given an inner product space $V$ and a group $G$ of linear isometries, max filtering offers a rich class of convex $G$-invariant maps. In this paper, we identify sufficient conditions under which these maps are locally bilipschitz on $R(G)$, the set of orbits with maximal dimension, with respect to the quotient metric on the orbit space $V/G$. Central to our proof is a desingularization theorem, which applies to open, dense neighborhoods around each orbit in $R(G)/G$ and may be of independent interest.
As an application, we provide guarantees for stable weighted phase retrieval. That is, we construct componentwise convex bilipschitz embeddings of weighted complex (resp.\ quaternionic) projective spaces. These spaces arise as quotients of direct sums of nontrivial unitary irreducible complex (resp.\ quaternionic) representations of the group of unit complex numbers $S^1\cong \operatorname{SO}(2)$ (resp.\ unit quaternions $S^3\cong \operatorname{SU}(2)$).
We also discuss the relevance of such embeddings to a nearest-neighbor problem in single-particle cryogenic electron microscopy (cryo-EM), a leading technique for resolving the spatial structure of biological molecules.
Multimodal large language models (MLLMs) often fail in fine-grained visual
question answering, producing hallucinations about object identities,
positions, and relations because textual queries are not explicitly anchored to
visual referents. Retrieval-augmented generation (RAG) alleviates some errors,
but it fails to align with human-like processing at both the retrieval and
augmentation levels. Specifically, it focuses only on global-level image
information but lacks local detail and limits reasoning about fine-grained
interactions. To overcome this limitation, we present Human-Like
Retrieval-Augmented Generation (HuLiRAG), a framework that stages multimodal
reasoning as a ``what--where--reweight'' cascade. Queries are first anchored to
candidate referents via open-vocabulary detection (what), then spatially
resolved with SAM-derived masks to recover fine-grained precision (where), and
adaptively prioritized through the trade-off between local and global alignment
(reweight). Mask-guided fine-tuning further injects spatial evidence into the
generation process, transforming grounding from a passive bias into an explicit
constraint on answer formulation. Extensive experiments demonstrate that this
human-like cascade improves grounding fidelity and factual consistency while
reducing hallucinations, advancing multimodal question answering toward
trustworthy reasoning.
Authors' comments: 12 pages, 5 figures
Generative retrieval (GR) reformulates information retrieval (IR) by framing
it as the generation of document identifiers (docids), thereby enabling an
end-to-end optimization and seamless integration with generative language
models (LMs). Despite notable progress under supervised training, GR still
struggles to generalize to zero-shot IR scenarios, which are prevalent in
real-world applications. To tackle this challenge, we propose \textsc{ZeroGR},
a zero-shot generative retrieval framework that leverages natural language
instructions to extend GR across a wide range of IR tasks. Specifically,
\textsc{ZeroGR} is composed of three key components: (i) an LM-based docid
generator that unifies heterogeneous documents (e.g., text, tables, code) into
semantically meaningful docids; (ii) an instruction-tuned query generator that
generates diverse types of queries from natural language task descriptions to
enhance corpus indexing; and (iii) a reverse annealing decoding strategy to
balance precision and recall during docid generation. We investigate the impact
of instruction fine-tuning scale and find that performance consistently
improves as the number of IR tasks encountered during training increases.
Empirical results on the BEIR and MAIR benchmarks demonstrate that
\textsc{ZeroGR} outperforms strong dense retrieval and generative baselines in
zero-shot settings, establishing a new state-of-the-art for instruction-driven
GR.
Designing protein sequences that fold into a target three-dimensional structure, known as the inverse folding problem, is central to protein engineering but remains challenging due to the vast sequence space and the importance of local structural constraints. Existing deep learning approaches achieve strong recovery rates, yet they lack explicit mechanisms to reuse fine-grained structure-sequence patterns that are conserved across natural proteins. We present PRISM, a multimodal retrieval-augmented generation framework for inverse folding that retrieves fine-grained representations of potential motifs from known proteins and integrates them with a hybrid self-cross attention decoder. PRISM is formulated as a latent-variable probabilistic model and implemented with an efficient approximation, combining theoretical grounding with practical scalability. Across five benchmarks (CATH-4.2, TS50, TS500, CAMEO 2022, and the PDB date split), PRISM establishes new state of the art in both perplexity and amino acid recovery, while also improving foldability metrics (RMSD, TM-score, pLDDT), demonstrating that fine-grained multimodal retrieval is a powerful and efficient paradigm for protein sequence design.
Medical Visual Question Answering (MedVQA) enables natural language queries over medical images to support clinical decision-making and patient care. The MEDIQA-WV 2025 shared task addressed wound-care VQA, requiring systems to generate free-text responses and structured wound attributes from images and patient queries. We present the MasonNLP system, which employs a general-domain, instruction-tuned large language model with a retrieval-augmented generation (RAG) framework that incorporates textual and visual examples from in-domain data. This approach grounds outputs in clinically relevant exemplars, improving reasoning, schema adherence, and response quality across dBLEU, ROUGE, BERTScore, and LLM-based metrics. Our best-performing system ranked 3rd among 19 teams and 51 submissions with an average score of 41.37%, demonstrating that lightweight RAG with general-purpose LLMs -- a minimal inference-time layer that adds a few relevant exemplars via simple indexing and fusion, with no extra training or complex re-ranking -- provides a simple and effective baseline for multimodal clinical NLP tasks.
The performance of Large Language Models (LLMs) often degrades when crucial
information is in the middle of a long context, a "lost-in-the-middle"
phenomenon that mirrors the primacy and recency effects in human memory. We
propose that this behavior is not simply a flaw indicative of information loss
but an adaptation to different information retrieval demands during
pre-training: some tasks require uniform recall across the entire input (a
long-term memory demand), while others prioritize the most recent information
(a short-term memory demand). Consistent with this view, we show that this
U-shaped performance curve emerges when LLMs (GPT-2 and Llama variants) are
trained from scratch on two simple human memory paradigms simulating long-term
and short-term memory demands. Our analysis reveals that while the recency
effect directly aligns with short-term memory demand in the training data, the
primacy effect is induced by the uniform long-term memory demand and is
additionally influenced by the model's autoregressive properties and the
formation of attention sinks. Our main findings from simple human memory
paradigms also generalize to a sequence completion task, which more closely
resembles the next-token prediction process in LLM pre-training. Together, our
findings reveal how information retrieval demands, model architecture, and
structural attention dynamics during model training can jointly produce
positional bias observed in LLMs.
Unmanned aerial vehicles (UAVs) have become powerful platforms for real-time,
high-resolution data collection, producing massive volumes of aerial videos.
Efficient retrieval of relevant content from these videos is crucial for
applications in urban management, emergency response, security, and disaster
relief. While text-video retrieval has advanced in natural video domains, the
UAV domain remains underexplored due to limitations in existing datasets, such
as coarse and redundant captions. Thus, in this work, we construct the Drone
Video-Text Match Dataset (DVTMD), which contains 2,864 videos and 14,320
fine-grained, semantically diverse captions. The annotations capture multiple
complementary aspects, including human actions, objects, background settings,
environmental conditions, and visual style, thereby enhancing text-video
correspondence and reducing redundancy. Building on this dataset, we propose
the Text-Conditioned Multi-granularity Alignment (TCMA) framework, which
integrates global video-sentence alignment, sentence-guided frame aggregation,
and word-guided patch alignment. To further refine local alignment, we design a
Word and Patch Selection module that filters irrelevant content, as well as a
Text-Adaptive Dynamic Temperature Mechanism that adapts attention sharpness to
text type. Extensive experiments on DVTMD and CapERA establish the first
complete benchmark for drone text-video retrieval. Our TCMA achieves
state-of-the-art performance, including 45.5% R@1 in text-to-video and 42.8%
R@1 in video-to-text retrieval, demonstrating the effectiveness of our dataset
and method. The code and dataset will be released.
Retrieval-Augmented Generation (RAG) systems based on Large Language Models
(LLMs) have become a core technology for tasks such as question-answering (QA)
and content generation. However, by injecting poisoned documents into the
database of RAG systems, attackers can manipulate LLMs to generate text that
aligns with their intended preferences. Existing research has primarily focused
on white-box attacks against simplified RAG architectures. In this paper, we
investigate a more complex and realistic scenario: the attacker lacks knowledge
of the RAG system's internal composition and implementation details, and the
RAG system comprises components beyond a mere retriever. Specifically, we
propose the RIPRAG attack framework, an end-to-end attack pipeline that treats
the target RAG system as a black box, where the only information accessible to
the attacker is whether the poisoning succeeds. Our method leverages
Reinforcement Learning (RL) to optimize the generation model for poisoned
documents, ensuring that the generated poisoned document aligns with the target
RAG system's preferences. Experimental results demonstrate that this method can
effectively execute poisoning attacks against most complex RAG systems,
achieving an attack success rate (ASR) improvement of up to 0.72 compared to
baseline methods. This highlights prevalent deficiencies in current defensive
methods and provides critical insights for LLM security research.
Image segmentation, the process of dividing images into meaningful regions,
is critical in medical applications for accurate diagnosis, treatment planning,
and disease monitoring. Although manual segmentation by healthcare
professionals produces precise outcomes, it is time-consuming, costly, and
prone to variability due to differences in human expertise. Artificial
intelligence (AI)-based methods have been developed to address these
limitations by automating segmentation tasks; however, they often require
large, annotated datasets that are rarely available in practice and frequently
struggle to generalize across diverse imaging conditions due to inter-patient
variability and rare pathological cases. In this paper, we propose Joint
Retrieval Augmented Segmentation (J-RAS), a joint training method for guided
image segmentation that integrates a segmentation model with a retrieval model.
Both models are jointly optimized, enabling the segmentation model to leverage
retrieved image-mask pairs to enrich its anatomical understanding, while the
retrieval model learns segmentation-relevant features beyond simple visual
similarity. This joint optimization ensures that retrieval actively contributes
meaningful contextual cues to guide boundary delineation, thereby enhancing the
overall segmentation performance. We validate J-RAS across multiple
segmentation backbones, including U-Net, TransUNet, SAM, and SegFormer, on two
benchmark datasets: ACDC and M&Ms, demonstrating consistent improvements. For
example, on the ACDC dataset, SegFormer without J-RAS achieves a mean Dice
score of 0.8708$\pm$0.042 and a mean Hausdorff Distance (HD) of
1.8130$\pm$2.49, whereas with J-RAS, the performance improves substantially to
a mean Dice score of 0.9115$\pm$0.031 and a mean HD of 1.1489$\pm$0.30. These
results highlight the method's effectiveness and its generalizability across
architectures and datasets.
Document expansion (DE) via query generation tackles vocabulary mismatch in
sparse retrieval, yet faces limitations: uncontrolled generation producing
hallucinated or redundant queries with low diversity; poor generalization from
in-domain training (e.g., MS MARCO) to out-of-domain data like BEIR; and noise
from concatenation harming dense retrieval. While Large Language Models (LLMs)
enable cross-domain query generation, basic prompting lacks control, and
taxonomy-based methods rely on domain-specific structures, limiting
applicability. To address these challenges, we introduce Doc2Query++, a DE
framework that structures query generation by first inferring a document's
latent topics via unsupervised topic modeling for cross-domain applicability,
then using hybrid keyword selection to create a diverse and relevant keyword
set per document. This guides LLM not only to leverage keywords, which ensure
comprehensive topic representation, but also to reduce redundancy through
diverse, relevant terms. To prevent noise from query appending in dense
retrieval, we propose Dual-Index Fusion strategy that isolates text and query
signals, boosting performance in dense settings. Extensive experiments show
Doc2Query++ significantly outperforms state-of-the-art baselines, achieving
substantial gains in MAP, nDCG@10 and Recall@100 across diverse datasets on
both sparse and dense retrieval.
Authors' comments: 11 pages, 4 figures
We introduce MRMR, the first expert-level multidisciplinary multimodal
retrieval benchmark requiring intensive reasoning. MRMR contains 1,502 queries
spanning 23 domains, with positive documents carefully verified by human
experts. Compared to prior benchmarks, MRMR introduces three key advancements.
First, it challenges retrieval systems across diverse areas of expertise,
enabling fine-grained model comparison across domains. Second, queries are
reasoning-intensive, with images requiring deeper interpretation such as
diagnosing microscopic slides. We further introduce Contradiction Retrieval, a
novel task requiring models to identify conflicting concepts. Finally, queries
and documents are constructed as image-text interleaved sequences. Unlike
earlier benchmarks restricted to single images or unimodal documents, MRMR
offers a realistic setting with multi-image queries and mixed-modality corpus
documents. We conduct an extensive evaluation of 4 categories of multimodal
retrieval systems and 14 frontier models on MRMR. The text embedding model
Qwen3-Embedding with LLM-generated image captions achieves the highest
performance, highlighting substantial room for improving multimodal retrieval
models. Although latest multimodal models such as Ops-MM-Embedding perform
competitively on expert-domain queries, they fall short on reasoning-intensive
tasks. We believe that MRMR paves the way for advancing multimodal retrieval in
more realistic and challenging scenarios.
The COVID-19 infodemic calls for scalable fact-checking solutions that handle long-form misinformation with accuracy and reliability. This study presents SAFE (system for accurate fact extraction and evaluation), an agent system that combines large language models with retrieval-augmented generation (RAG) to improve automated fact-checking of long-form COVID-19 misinformation. SAFE includes two agents - one for claim extraction and another for claim verification using LOTR-RAG, which leverages a 130,000-document COVID-19 research corpus. An enhanced variant, SAFE (LOTR-RAG + SRAG), incorporates Self-RAG to refine retrieval via query rewriting. We evaluated both systems on 50 fake news articles (2-17 pages) containing 246 annotated claims (M = 4.922, SD = 3.186), labeled as true (14.1%), partly true (14.4%), false (27.0%), partly false (2.2%), and misleading (21.0%) by public health professionals. SAFE systems significantly outperformed baseline LLMs in all metrics (p < 0.001). For consistency (0-1 scale), SAFE (LOTR-RAG) scored 0.629, exceeding both SAFE (+SRAG) (0.577) and the baseline (0.279). In subjective evaluations (0-4 Likert scale), SAFE (LOTR-RAG) also achieved the highest average ratings in usefulness (3.640), clearness (3.800), and authenticity (3.526). Adding SRAG slightly reduced overall performance, except for a minor gain in clearness. SAFE demonstrates robust improvements in long-form COVID-19 fact-checking by addressing LLM limitations in consistency and explainability. The core LOTR-RAG design proved more effective than its SRAG-augmented variant, offering a strong foundation for scalable misinformation mitigation.
Multimodal Retrieval-Augmented Generation (MRAG) enables Multimodal Large
Language Models (MLLMs) to generate responses with external multimodal
evidence, and numerous video-based MRAG benchmarks have been proposed to
evaluate model capabilities across retrieval and generation stages. However,
existing benchmarks remain limited in modality coverage and format diversity,
often focusing on single- or limited-modality tasks, or coarse-grained scene
understanding. To address these gaps, we introduce CFVBench, a large-scale,
manually verified benchmark constructed from 599 publicly available videos,
yielding 5,360 open-ended QA pairs. CFVBench spans high-density formats and
domains such as chart-heavy reports, news broadcasts, and software tutorials,
requiring models to retrieve and reason over long temporal video spans while
maintaining fine-grained multimodal information. Using CFVBench, we
systematically evaluate 7 retrieval methods and 14 widely-used MLLMs, revealing
a critical bottleneck: current models (even GPT5 or Gemini) struggle to capture
transient yet essential fine-grained multimodal details. To mitigate this, we
propose Adaptive Visual Refinement (AVR), a simple yet effective framework that
adaptively increases frame sampling density and selectively invokes external
tools when necessary. Experiments show that AVR consistently enhances
fine-grained multimodal comprehension and improves performance across all
evaluated MLLMs
Despite remarkable advances in Large Language Model capabilities, tool
retrieval for agent-based systems remains fundamentally limited by reliance on
semantic similarity, which fails to capture functional viability. Current
methods often retrieve textually relevant but functionally inoperative tools
due to parameter mismatches, authentication failures, and execution
constraints--a phenomenon we term the semantic-functional gap. We introduce
GRETEL, to address this gap through systematic empirical validation. GRETEL
implements an agentic workflow that processes semantically retrieved candidates
through sandboxed plan-execute-evaluate cycles, generating execution-grounded
evidence to distinguish truly functional tools from merely descriptive matches.
Our comprehensive evaluation on the ToolBench benchmark demonstrates
substantial improvements across all metrics: Pass Rate (at 10) increases from
0.690 to 0.826, Recall (at 10) improves from 0.841 to 0.867, and NDCG (at 10)
rises from 0.807 to 0.857.. These results establish that execution-based
validation provides a more reliable foundation for tool selection than semantic
similarity alone, enabling more robust agent performance in real-world
applications.
Authors' comments: 5 pages, 1 figures, 5 tables
