Retrieval-Augmented Generation (RAG) has emerged as an important means of enhancing the performance of large language models (LLMs) in knowledge-intensive tasks. However, most existing RAG strategies treat retrieved passages in a flat and unstructured way, which prevents the model from capturing structural cues and constrains its ability to synthesize knowledge from dispersed evidence across documents. To overcome these limitations, we propose Disco-RAG, a discourse-aware framework that explicitly injects discourse signals into the generation process. Our method constructs intra-chunk discourse trees to capture local hierarchies and builds inter-chunk rhetorical graphs to model cross-passage coherence. These structures are jointly integrated into a planning blueprint that conditions the generation. Experiments on question answering and long-document summarization benchmarks show the efficacy of our approach. Disco-RAG achieves state-of-the-art results on the benchmarks without fine-tuning. These findings underscore the important role of discourse structure in advancing RAG systems.
This paper studies phase and norm retrieval by subspaces. We first investigate norm retrieval by hyperplanes. We show that if $N$ hyperplanes $\{\varphi_i^\perp\}_{i=1}^N\subset \mathbb{R}^N$ allow norm retrieval and the vectors $\{\varphi_i\}_{i=1}^N$ are linearly independent, then these vectors must be an orthonormal basis for $\mathbb{R}^N$. We then present several new properties of subspaces that allow phase and norm retrieval. In particular, we provide a complete classification of two proper subspaces that perform norm retrieval. It is known that the collection of norm-retrievable frames $\{\varphi_i\}_{i=1}^M$ in $\mathbb{R}^N$ is not dense in the set of all $M$-element frames when $M < 2N-1$. We extend this result to subspaces. Several alternative proofs of fundamental results in phase and norm retrieval are also provided.
Recent work has shown that directly fine-tuning large language models (LLMs) for dense retrieval yields strong performance, but their substantial parameter counts make them computationally inefficient. While prior studies have revealed significant layer redundancy in LLMs for generative tasks, it remains unclear whether similar redundancy exists when these models are adapted for retrieval tasks, which require encoding entire sequences into fixed representations rather than generating tokens iteratively. To this end, we conduct a comprehensive analysis of layer redundancy in LLM-based dense retrievers. We find that, in contrast to generative settings, MLP layers are substantially more prunable, while attention layers remain critical for semantic aggregation. Building on this insight, we propose EffiR, a framework for developing efficient retrievers that performs large-scale MLP compression through a coarse-to-fine strategy (coarse-grained depth reduction followed by fine-grained width reduction), combined with retrieval-specific fine-tuning. Across diverse BEIR datasets and LLM backbones, EffiR achieves substantial reductions in model size and inference cost while preserving the performance of full-size models.
Multimodal document retrieval systems have shown strong progress in aligning visual and textual content for semantic search. However, most existing approaches remain heavily English-centric, limiting their effectiveness in multilingual contexts. In this work, we present M3DR (Multilingual Multimodal Document Retrieval), a framework designed to bridge this gap across languages, enabling applicability across diverse linguistic and cultural contexts. M3DR leverages synthetic multilingual document data and generalizes across different vision-language architectures and model sizes, enabling robust cross-lingual and cross-modal alignment. Using contrastive training, our models learn unified representations for text and document images that transfer effectively across languages. We validate this capability on 22 typologically diverse languages, demonstrating consistent performance and adaptability across linguistic and script variations. We further introduce a comprehensive benchmark that captures real-world multilingual scenarios, evaluating models under monolingual, multilingual, and mixed-language settings. M3DR generalizes across both single dense vector and ColBERT-style token-level multi-vector retrieval paradigms. Our models, NetraEmbed and ColNetraEmbed achieve state-of-the-art performance with ~150% relative improvements on cross-lingual retrieval.
Retrieval-augmented models couple parametric predictors with non-parametric memories, but their use in streaming supervised learning with concept drift is not well understood. We study online classification in non-stationary environments and propose Retrieval-Augmented Memory for Online Learning (RAM-OL), a simple extension of stochastic gradient descent that maintains a small buffer of past examples. At each time step, RAM-OL retrieves a few nearest neighbours of the current input in the hidden representation space and updates the model jointly on the current example and the retrieved neighbours. We compare a naive replay variant with a gated replay variant that constrains neighbours using a time window, similarity thresholds, and gradient reweighting, in order to balance fast reuse of relevant past data against robustness to outdated regimes. From a theoretical perspective, we interpret RAM-OL under a bounded drift model and discuss how retrieval can reduce adaptation cost and improve regret constants when patterns recur over time. Empirically, we instantiate RAM-OL on a simple online multilayer perceptron and evaluate it on three real-world data streams derived from electricity pricing, electricity load, and airline delay data. On strongly and periodically drifting streams, RAM-OL improves prequential accuracy by up to about seven percentage points and greatly reduces variance across random seeds, while on a noisy airline stream the gated variant closely matches the purely online baseline. These results show that retrieval-augmented memory is a practical and robust tool for online learning under concept drift.
Authors' comments: 11 pages, 3 figures
This paper investigates noise-robust phase retrieval by enhancing the prDeep architecture with difference of convex functions (DC) and DnCNN-based denoising regularization. This research introduces two novel algorithms, prDeep-DC and prDeep-L2, which demonstrably achieve excellent quantitative and visual performance, as confirmed by extensive numerical experiments.
ColBERT introduced a late interaction mechanism that independently encodes queries and documents using BERT, and computes similarity via fine-grained interactions over token-level vector representations. This design enables expressive matching while allowing efficient computation of scores, as the multi-vector document representations could be pre-computed offline. ColBERT models distance using a Chamfer-style function: for each query token, it selects the closest document token and sums these distances across all query tokens.
In our work, we explore enhancements to the Chamfer distance function by computing a weighted sum over query token contributions, where weights reflect the token importance. Empirically, we show that this simple extension, requiring only token-weight training while keeping the multi-vector representations fixed, further enhances the expressiveness of late interaction multi-vector mechanism. In particular, on the BEIR benchmark, our method achieves an average improvement of 1.28\% in Recall@10 in the zero-shot setting using IDF-based weights, and 3.66\% through few-shot fine-tuning.
Gabor phase retrieval stands for recovering a square integrable function up to a global phase from absolute values of its Gabor transform. In this paper, we study Gabor phase retrieval from discrete samples. We consider three types of sampling sequences, which include square root lattices, square root sequences on two intersecting lines and on three parallel lines respectively. In all cases we give the optimal sampling density for a sequence to do Gabor phase retrieval.
Authors' comments: 22 pages
Open-domain question answering over datalakes requires retrieving and composing information from multiple tables, a challenging subtask that demands semantic relevance and structural coherence (e.g., joinability). While exact optimization methods like Mixed-Integer Programming (MIP) can ensure coherence, their computational complexity is often prohibitive. Conversely, simpler greedy heuristics that optimize for query coverage alone often fail to find these coherent, joinable sets. This paper frames multi-table retrieval as an iterative search process, arguing this approach offers advantages in scalability, interpretability, and flexibility. We propose a general framework and a concrete instantiation: a fast, effective Greedy Join-Aware Retrieval algorithm that holistically balances relevance, coverage, and joinability. Experiments across 5 NL2SQL benchmarks demonstrate that our iterative method achieves competitive retrieval performance compared to the MIP-based approach while being 4-400x faster depending on the benchmark and search space settings. This work highlights the potential of iterative heuristics for practical, scalable, and composition-aware retrieval.
Authors' comments: Accepted @ the AI for Tabular Data Workshop, EurIPS 2025
Visual document retrieval requires understanding heterogeneous and multi-modal content to satisfy information needs. Recent advances use screenshot-based document encoding with fine-grained late interaction, significantly improving retrieval performance. However, retrievers are still trained with coarse global relevance labels, without revealing which regions support the match. As a result, retrievers tend to rely on surface-level cues and struggle to capture implicit semantic connections, hindering their ability to handle non-extractive queries. To alleviate this problem, we propose a \textbf{A}ttention-\textbf{G}rounded \textbf{RE}triever \textbf{E}nhancement (AGREE) framework. AGREE leverages cross-modal attention from multimodal large language models as proxy local supervision to guide the identification of relevant document regions. During training, AGREE combines local signals with the global signals to jointly optimize the retriever, enabling it to learn not only whether documents match, but also which content drives relevance. Experiments on the challenging ViDoRe V2 benchmark show that AGREE significantly outperforms the global-supervision-only baseline. Quantitative and qualitative analyses further demonstrate that AGREE promotes deeper alignment between query terms and document regions, moving beyond surface-level matching toward more accurate and interpretable retrieval. Our code is available at: https://anonymous.4open.science/r/AGREE-2025.
Dynamic recommendation systems aim to provide personalized suggestions by modeling temporal user-item interactions across time-series behavioral data. Recent studies have leveraged pre-trained dynamic graph neural networks (GNNs) to learn user-item representations over temporal snapshot graphs. However, fine-tuning GNNs on these graphs often results in generalization issues due to temporal discrepancies between pre-training and fine-tuning stages, limiting the model's ability to capture evolving user preferences. To address this, we propose TarDGR, a task-aware retrieval-augmented framework designed to enhance generalization capability by incorporating task-aware model and retrieval-augmentation. Specifically, TarDGR introduces a Task-Aware Evaluation Mechanism to identify semantically relevant historical subgraphs, enabling the construction of task-specific datasets without manual labeling. It also presents a Graph Transformer-based Task-Aware Model that integrates semantic and structural encodings to assess subgraph relevance. During inference, TarDGR retrieves and fuses task-aware subgraphs with the query subgraph, enriching its representation and mitigating temporal generalization issues. Experiments on multiple large-scale dynamic graph datasets demonstrate that TarDGR consistently outperforms state-of-the-art methods, with extensive empirical evidence underscoring its superior accuracy and generalization capabilities.
Authors' comments: AAAI 2026
The convergence of deep learning and formal mathematics has spurred research in formal verification. Statement autoformalization, a crucial first step in this process, aims to translate informal descriptions into machine-verifiable representations but remains a significant challenge. The core difficulty lies in the fact that existing methods often suffer from a lack of contextual awareness, leading to hallucination of formal definitions and theorems. Furthermore, current retrieval-augmented approaches exhibit poor precision and recall for formal library dependency retrieval, and lack the scalability to effectively leverage ever-growing public datasets. To bridge this gap, we propose a novel retrieval-augmented framework based on DDR (\textit{Direct Dependency Retrieval}) for statement autoformalization. Our DDR method directly generates candidate library dependencies from natural language mathematical descriptions and subsequently verifies their existence within the formal library via an efficient suffix array check. Leveraging this efficient search mechanism, we constructed a dependency retrieval dataset of over 500,000 samples and fine-tuned a high-precision DDR model. Experimental results demonstrate that our DDR model significantly outperforms SOTA methods in both retrieval precision and recall. Consequently, an autoformalizer equipped with DDR shows consistent performance advantages in both single-attempt accuracy and multi-attempt stability compared to models using traditional selection-based RAG methods.
Organizations handling sensitive documents face a critical dilemma: adopt cloud-based AI systems that offer powerful question-answering capabilities but compromise data privacy, or maintain local processing that ensures security but delivers poor accuracy. We present a question-answering system that resolves this trade-off by combining semantic understanding with keyword precision, operating entirely on local infrastructure without internet access. Our approach demonstrates that organizations can achieve competitive accuracy on complex queries across legal, scientific, and conversational documents while keeping all data on their machines. By balancing two complementary retrieval strategies and using consumer-grade hardware acceleration, the system delivers reliable answers with minimal errors, letting banks, hospitals, and law firms adopt conversational document AI without transmitting proprietary information to external providers. This work establishes that privacy and performance need not be mutually exclusive in enterprise AI deployment.
Authors' comments: 10 pages, 5 figures, 3 tables; conference-style (ACL format); fully local RAG system
This work introduces Text-based Aerial-Ground Person Retrieval (TAG-PR), which aims to retrieve person images from heterogeneous aerial and ground views with textual descriptions. Unlike traditional Text-based Person Retrieval (T-PR), which focuses solely on ground-view images, TAG-PR introduces greater practical significance and presents unique challenges due to the large viewpoint discrepancy across images. To support this task, we contribute: (1) TAG-PEDES dataset, constructed from public benchmarks with automatically generated textual descriptions, enhanced by a diversified text generation paradigm to ensure robustness under view heterogeneity; and (2) TAG-CLIP, a novel retrieval framework that addresses view heterogeneity through a hierarchically-routed mixture of experts module to learn view-specific and view-agnostic features and a viewpoint decoupling strategy to decouple view-specific features for better cross-modal alignment. We evaluate the effectiveness of TAG-CLIP on both the proposed TAG-PEDES dataset and existing T-PR benchmarks. The dataset and code are available at https://github.com/Flame-Chasers/TAG-PR.
In this work, we introduce TurkEmbed4Retrieval, a retrieval specialized variant of the TurkEmbed model originally designed for Natural Language Inference (NLI) and Semantic Textual Similarity (STS) tasks. By fine-tuning the base model on the MS MARCO TR dataset using advanced training techniques, including Matryoshka representation learning and a tailored multiple negatives ranking loss, we achieve SOTA performance for Turkish retrieval tasks. Extensive experiments demonstrate that our model outperforms Turkish colBERT by 19,26% on key retrieval metrics for the Scifact TR dataset, thereby establishing a new benchmark for Turkish information retrieval.
Authors' comments: 4 pages, in Turkish language, 1 figure, conference
Deep learning models such as MLP, Transformer, and TCN have achieved
remarkable success in univariate time series forecasting, typically relying on
sliding window samples from historical data for training. However, while these
models implicitly compress historical information into their parameters during
training, they are unable to explicitly and dynamically access this global
knowledge during inference, relying only on the local context within the
lookback window. This results in an underutilization of rich patterns from the
global history. To bridge this gap, we propose Predicting the Future by
Retrieving the Past (PFRP), a novel approach that explicitly integrates global
historical data to enhance forecasting accuracy. Specifically, we construct a
Global Memory Bank (GMB) to effectively store and manage global historical
patterns. A retrieval mechanism is then employed to extract similar patterns
from the GMB, enabling the generation of global predictions. By adaptively
combining these global predictions with the outputs of any local prediction
model, PFRP produces more accurate and interpretable forecasts. Extensive
experiments conducted on seven real-world datasets demonstrate that PFRP
significantly enhances the average performance of advanced univariate
forecasting models by 8.4\%. Codes can be found in
https://github.com/ddz16/PFRP.
Authors' comments: Accepted by AAAI 2026
Code review is essential for maintaining software quality but is
labor-intensive. Automated code review generation offers a promising solution
to this challenge. Both deep learning-based generative techniques and
retrieval-based methods have demonstrated strong performance in this task.
However, despite these advancements, there are still some limitations where
generated reviews can be either off-point or overly general. To address these
issues, we introduce Retrieval-Augmented Reviewer (RARe), which leverages
Retrieval-Augmented Generation (RAG) to combine retrieval-based and generative
methods, explicitly incorporating external domain knowledge into the code
review process. RARe uses a dense retriever to select the most relevant reviews
from the codebase, which then enrich the input for a neural generator,
utilizing the contextual learning capacity of large language models (LLMs), to
produce the final review. RARe outperforms state-of-the-art methods on two
benchmark datasets, achieving BLEU-4 scores of 12.32 and 12.96, respectively.
Its effectiveness is further validated through a detailed human evaluation and
a case study using an interpretability tool, demonstrating its practical
utility and reliability.
Generative Retrieval (GR) differs from the traditional index-then-retrieve
pipeline by storing relevance in model parameters and directly generating
document identifiers. However, GR often struggles to generalize and is costly
to scale. We introduce QUESTER (QUEry SpecificaTion gEnerative Retrieval),
which reframes GR as query specification generation - in this work, a simple
keyword query handled by BM25 - using a (small) LLM. The policy is trained
using reinforcement learning techniques (GRPO). Across in- and out-of-domain
evaluations, we show that our model is more effective than BM25, and
competitive with neural IR models, while maintaining a good efficiency
We introduce Trove, an easy-to-use open-source retrieval toolkit that
simplifies research experiments without sacrificing flexibility or speed. For
the first time, we introduce efficient data management features that load and
process (filter, select, transform, and combine) retrieval datasets on the fly,
with just a few lines of code. This gives users the flexibility to easily
experiment with different dataset configurations without the need to compute
and store multiple copies of large datasets. Trove is highly customizable: in
addition to many built-in options, it allows users to freely modify existing
components or replace them entirely with user-defined objects. It also provides
a low-code and unified pipeline for evaluation and hard negative mining, which
supports multi-node execution without any code changes. Trove's data management
features reduce memory consumption by a factor of 2.6. Moreover, Trove's
easy-to-use inference pipeline incurs no overhead, and inference times decrease
linearly with the number of available nodes. Most importantly, we demonstrate
how Trove simplifies retrieval experiments and allows for arbitrary
customizations, thus facilitating exploratory research.
Although Large Language Models (LLMs) demonstrate significant capabilities,
their reliance on parametric knowledge often leads to inaccuracies. Retrieval
Augmented Generation (RAG) mitigates this by incorporating external knowledge,
but these methods may introduce irrelevant retrieved documents, leading to
inaccurate responses. While the integration methods filter out incorrect
answers from multiple responses, but lack external knowledge like RAG methods,
and their high costs require balancing overhead with performance gains. To
address these issues, we propose an Efficient Test-Time Retrieval-Augmented
Generation Framework named ET2RAG to improve the performance of LLMs while
maintaining efficiency. Specifically, ET2RAG is a training-free method, that
first retrieves the most relevant documents and augments the LLMs to
efficiently generate diverse candidate responses by managing response length.
Then we compute the similarity of candidate responses and employ a majority
voting mechanism to select the most suitable response as the final output. In
particular, we discover that partial generation is sufficient to capture the
key information necessary for consensus calculation, allowing us to effectively
perform majority voting without the need for fully generated responses. Thus,
we can reach a balance between computational cost and performance by managing
the response length for the number of retrieved documents for majority voting.
Experimental results demonstrate that ET2RAG significantly enhances performance
across three tasks, including open-domain question answering, recipe generation
and image captioning.
