In large-scale industrial recommendation systems, retrieval must produce high-quality candidates from massive corpora under strict latency. Recently, Generative Retrieval (GR) has emerged as a viable alternative to Embedding-Based Retrieval (EBR), which quantizes items into a finite token space and decodes candidates autoregressively, providing a scalable path that explicitly models target-history interactions via cross-attention. However, three challenges persist: 1) how to balance users' long-term and short-term interests , 2) noise interference when generating hierarchical semantic IDs (SIDs), 3) the absence of explicit modeling for negative feedback such as exposed items without clicks. To address these challenges, we propose DualGR, a generative retrieval framework that explicitly models dual horizons of user interests with selective activation. Specifically, DualGR utilizes Dual-Branch Long/Short-Term Router (DBR) to cover both stable preferences and transient intents by explicitly modeling users' long- and short-term behaviors. Meanwhile, Search-based SID Decoding (S2D) is presented to control context-induced noise and enhance computational efficiency by constraining candidate interactions to the current coarse (level-1) bucket during fine-grained (level-2/3) SID prediction. % also reinforcing intra-class consistency. Finally, we propose an Exposure-aware Next-Token Prediction Loss (ENTP-Loss) that treats "exposed-but-unclicked" items as hard negatives at level-1, enabling timely interest fade-out. On the large-scale Kuaishou short-video recommendation system, DualGR has achieved outstanding performance. Online A/B testing shows +0.527% video views and +0.432% watch time lifts, validating DualGR as a practical and effective paradigm for industrial generative retrieval.
Information retrieval (IR) systems play a critical role in navigating information overload across various applications. Existing IR benchmarks primarily focus on simple queries that are semantically analogous to single- and multi-hop relations, overlooking \emph{complex logical queries} involving first-order logic operations such as conjunction ($\land$), disjunction ($\lor$), and negation ($\lnot$). Thus, these benchmarks can not be used to sufficiently evaluate the performance of IR models on complex queries in real-world scenarios. To address this problem, we propose a novel method leveraging large language models (LLMs) to construct a new IR dataset \textbf{ComLQ} for \textbf{Com}plex \textbf{L}ogical \textbf{Q}ueries, which comprises 2,909 queries and 11,251 candidate passages. A key challenge in constructing the dataset lies in capturing the underlying logical structures within unstructured text. Therefore, by designing the subgraph-guided prompt with the subgraph indicator, an LLM (such as GPT-4o) is guided to generate queries with specific logical structures based on selected passages. All query-passage pairs in ComLQ are ensured \emph{structure conformity} and \emph{evidence distribution} through expert annotation. To better evaluate whether retrievers can handle queries with negation, we further propose a new evaluation metric, \textbf{Log-Scaled Negation Consistency} (\textbf{LSNC@$K$}). As a supplement to standard relevance-based metrics (such as nDCG and mAP), LSNC@$K$ measures whether top-$K$ retrieved passages violate negation conditions in queries. Our experimental results under zero-shot settings demonstrate existing retrieval models' limited performance on complex logical queries, especially on queries with negation, exposing their inferior capabilities of modeling exclusion.
Authors' comments: Accepted by AAAI 2026
Large Language Models (LLMs) demonstrate strong reasoning capabilities but struggle with hallucinations and limited transparency. Recently, KG-enhanced LLMs that integrate knowledge graphs (KGs) have been shown to improve reasoning performance, particularly for complex, knowledge-intensive tasks. However, these methods still face significant challenges, including inaccurate retrieval and reasoning failures, often exacerbated by long input contexts that obscure relevant information or by context constructions that struggle to capture the richer logical directions required by different question types. Furthermore, many of these approaches rely on LLMs to directly retrieve evidence from KGs, and to self-assess the sufficiency of this evidence, which often results in premature or incorrect reasoning. To address the retrieval and reasoning failures, we propose ProgRAG, a multi-hop knowledge graph question answering (KGQA) framework that decomposes complex questions into sub-questions, and progressively extends partial reasoning paths by answering each sub-question. At each step, external retrievers gather candidate evidence, which is then refined through uncertainty-aware pruning by the LLM. Finally, the context for LLM reasoning is optimized by organizing and rearranging the partial reasoning paths obtained from the sub-question answers. Experiments on three well-known datasets demonstrate that ProgRAG outperforms existing baselines in multi-hop KGQA, offering improved reliability and reasoning quality.
Text-to-Image Person Retrieval (TIPR) aims to retrieve person images based on natural language descriptions. Although many TIPR methods have achieved promising results, sometimes textual queries cannot accurately and comprehensively reflect the content of the image, leading to poor cross-modal alignment and overfitting to limited datasets. Moreover, the inherent modality gap between text and image further amplifies these issues, making accurate cross-modal retrieval even more challenging. To address these limitations, we propose the Generation-Enhanced Alignment (GEA) from a generative perspective. GEA contains two parallel modules: (1) Text-Guided Token Enhancement (TGTE), which introduces diffusion-generated images as intermediate semantic representations to bridge the gap between text and visual patterns. These generated images enrich the semantic representation of text and facilitate cross-modal alignment. (2) Generative Intermediate Fusion (GIF), which combines cross-attention between generated images, original images, and text features to generate a unified representation optimized by triplet alignment loss. We conduct extensive experiments on three public TIPR datasets, CUHK-PEDES, RSTPReid, and ICFG-PEDES, to evaluate the performance of GEA. The results justify the effectiveness of our method. More implementation details and extended results are available at https://github.com/sugelamyd123/Sup-for-GEA.
Authors' comments: 8pages,3figures
Large transformer models, trained on diverse datasets, have demonstrated impressive few-shot performance on previously unseen tasks without requiring parameter updates. This capability has also been explored in Reinforcement Learning (RL), where agents interact with the environment to retrieve context and maximize cumulative rewards, showcasing strong adaptability in complex settings. However, in cooperative Multi-Agent Reinforcement Learning (MARL), where agents must coordinate toward a shared goal, decentralized policy deployment can lead to mismatches in task alignment and reward assignment, limiting the efficiency of policy adaptation. To address this challenge, we introduce Multi-agent In-context Coordination via Decentralized Memory Retrieval (MAICC), a novel approach designed to enhance coordination by fast adaptation. Our method involves training a centralized embedding model to capture fine-grained trajectory representations, followed by decentralized models that approximate the centralized one to obtain team-level task information. Based on the learned embeddings, relevant trajectories are retrieved as context, which, combined with the agents' current sub-trajectories, inform decision-making. During decentralized execution, we introduce a novel memory mechanism that effectively balances test-time online data with offline memory. Based on the constructed memory, we propose a hybrid utility score that incorporates both individual- and team-level returns, ensuring credit assignment across agents. Extensive experiments on cooperative MARL benchmarks, including Level-Based Foraging (LBF) and SMAC (v1/v2), show that MAICC enables faster adaptation to unseen tasks compared to existing methods. Code is available at https://github.com/LAMDA-RL/MAICC.
Dynamic retrieval-augmented generation (RAG) allows large language models (LLMs) to fetch external knowledge on demand, offering greater adaptability than static RAG. A central challenge in this setting lies in determining the optimal timing for retrieval. Existing methods often trigger retrieval based on low token-level confidence, which may lead to delayed intervention after errors have already propagated. We introduce Entropy-Trend Constraint (ETC), a training-free method that determines optimal retrieval timing by modeling the dynamics of token-level uncertainty. Specifically, ETC utilizes first- and second-order differences of the entropy sequence to detect emerging uncertainty trends, enabling earlier and more precise retrieval. Experiments on six QA benchmarks with three LLM backbones demonstrate that ETC consistently outperforms strong baselines while reducing retrieval frequency. ETC is particularly effective in domain-specific scenarios, exhibiting robust generalization capabilities. Ablation studies and qualitative analyses further confirm that trend-aware uncertainty modeling yields more effective retrieval timing. The method is plug-and-play, model-agnostic, and readily integrable into existing decoding pipelines. Implementation code is included in the supplementary materials.
Authors' comments: AAAI'26, Oral Paper
Retrieval-Augmented Generation (RAG) improves factuality but retrieving for every query often hurts quality while inflating tokens and latency. We propose Training-free Adaptive Retrieval Gating (TARG), a single-shot policy that decides when to retrieve using only a short, no-context draft from the base model. From the draft's prefix logits, TARG computes lightweight uncertainty scores: mean token entropy, a margin signal derived from the top-1/top-2 logit gap via a monotone link, or small-N variance across a handful of stochastic prefixes, and triggers retrieval only when the score exceeds a threshold. The gate is model agnostic, adds only tens to hundreds of draft tokens, and requires no additional training or auxiliary heads. On NQ-Open, TriviaQA, and PopQA, TARG consistently shifts the accuracy-efficiency frontier: compared with Always-RAG, TARG matches or improves EM/F1 while reducing retrieval by 70-90% and cutting end-to-end latency, and it remains close to Never-RAG in overhead. A central empirical finding is that under modern instruction-tuned LLMs the margin signal is a robust default (entropy compresses as backbones sharpen), with small-N variance offering a conservative, budget-first alternative. We provide ablations over gate type and prefix length and use a delta-latency view to make budget trade-offs explicit.
Financial markets are inherently non-stationary: structural breaks and macroeconomic regime shifts often cause forecasting models to fail when deployed out of distribution (OOD). Conventional multimodal approaches that simply fuse numerical indicators and textual sentiment rarely adapt to such shifts. We introduce macro-contextual retrieval, a retrieval-augmented forecasting framework that grounds each prediction in historically analogous macroeconomic regimes. The method jointly embeds macro indicators (e.g., CPI, unemployment, yield spread, GDP growth) and financial news sentiment in a shared similarity space, enabling causal retrieval of precedent periods during inference without retraining.
Trained on seventeen years of S&P 500 data (2007-2023) and evaluated OOD on AAPL (2024) and XOM (2024), the framework consistently narrows the CV to OOD performance gap. Macro-conditioned retrieval achieves the only positive out-of-sample trading outcomes (AAPL: PF=1.18, Sharpe=0.95; XOM: PF=1.16, Sharpe=0.61), while static numeric, text-only, and naive multimodal baselines collapse under regime shifts. Beyond metric gains, retrieved neighbors form interpretable evidence chains that correspond to recognizable macro contexts, such as inflationary or yield-curve inversion phases, supporting causal interpretability and transparency. By operationalizing the principle that "financial history may not repeat, but it often rhymes," this work demonstrates that macro-aware retrieval yields robust, explainable forecasts under distributional change.
All datasets, models, and source code are publicly available.
Authors' comments: Accepted in IEEE BigData 2025
Generative retrieval (GR) re-frames document retrieval as a sequence-based document identifier (DocID) generation task, memorizing documents with model parameters and enabling end-to-end retrieval without explicit indexing. Existing GR methods are based on auto-regressive generative models, i.e., the token generation is performed from left to right. However, such auto-regressive methods suffer from: (1) mismatch between DocID generation and natural language generation, e.g., an incorrect DocID token generated in early left steps would lead to totally erroneous retrieval; and (2) failure to balance the trade-off between retrieval efficiency and accuracy dynamically, which is crucial for practical applications. To address these limitations, we propose generative document retrieval with diffusion language models, dubbed DiffuGR. It models DocID generation as a discrete diffusion process: during training, DocIDs are corrupted through a stochastic masking process, and a diffusion language model is learned to recover them under a retrieval-aware objective. For inference, DiffuGR attempts to generate DocID tokens in parallel and refines them through a controllable number of denoising steps. In contrast to conventional left-to-right auto-regressive decoding, DiffuGR provides a novel mechanism to first generate more confident DocID tokens and refine the generation through diffusion-based denoising. Moreover, DiffuGR also offers explicit runtime control over the qualitylatency tradeoff. Extensive experiments on benchmark retrieval datasets show that DiffuGR is competitive with strong auto-regressive generative retrievers, while offering flexible speed and accuracy tradeoffs through variable denoising budgets. Overall, our results indicate that non-autoregressive diffusion models are a practical and effective alternative for generative document retrieval.
Authors' comments: This paper is under review
Hard negatives are essential for training effective retrieval models. Hard-negative mining typically relies on ranking documents using cross-encoders or static embedding models based on similarity metrics such as cosine distance. Hard negative mining becomes challenging for biomedical and scientific domains due to the difficulty in distinguishing between source and hard negative documents. However, referenced documents naturally share contextual relevance with the source document but are not duplicates, making them well-suited as hard negatives. In this work, we propose BiCA: Biomedical Dense Retrieval with Citation-Aware Hard Negatives, an approach for hard-negative mining by utilizing citation links in 20,000 PubMed articles for improving a domain-specific small dense retriever. We fine-tune the GTE_small and GTE_Base models using these citation-informed negatives and observe consistent improvements in zero-shot dense retrieval using nDCG@10 for both in-domain and out-of-domain tasks on BEIR and outperform baselines on long-tailed topics in LoTTE using Success@5. Our findings highlight the potential of leveraging document link structure to generate highly informative negatives, enabling state-of-the-art performance with minimal fine-tuning and demonstrating a path towards highly data-efficient domain adaptation.
Authors' comments: Accepted for oral presentation at AAAI 2026
In high-stakes information domains such as healthcare, where large language
models (LLMs) can produce hallucinations or misinformation, retrieval-augmented
generation (RAG) has been proposed as a mitigation strategy, grounding model
outputs in external, domain-specific documents. Yet, this approach can
introduce errors when source documents contain outdated or contradictory
information. This work investigates the performance of five LLMs in generating
RAG-based responses to medicine-related queries. Our contributions are
three-fold: i) the creation of a benchmark dataset using consumer medicine
information documents from the Australian Therapeutic Goods Administration
(TGA), where headings are repurposed as natural language questions, ii) the
retrieval of PubMed abstracts using TGA headings, stratified across multiple
publication years, to enable controlled temporal evaluation of outdated
evidence, and iii) a comparative analysis of the frequency and impact of
outdated or contradictory content on model-generated responses, assessing how
LLMs integrate and reconcile temporally inconsistent information. Our findings
show that contradictions between highly similar abstracts do, in fact, degrade
performance, leading to inconsistencies and reduced factual accuracy in model
answers. These results highlight that retrieval similarity alone is
insufficient for reliable medical RAG and underscore the need for
contradiction-aware filtering strategies to ensure trustworthy responses in
high-stakes domains.
Ingesting data for Retrieval-Augmented Generation (RAG) involves either
fine-tuning the embedding model directly on the target corpus or parsing
documents for embedding model encoding. The former, while accurate, incurs high
computational hardware requirements, while the latter suffers from suboptimal
performance when extracting tabular data. In this work, we address the latter
by presenting TabRAG, a parsing-based RAG pipeline designed to tackle
table-heavy documents via structured language representations. TabRAG
outperforms existing popular parsing-based methods for generation and
retrieval. Code is available at https://github.com/jacobyhsi/TabRAG.
Authors' comments: NeurIPS 2025 AI4Tab
Nearest-neighbour retrieval is central to classification and explainable-AI
pipelines, but current practice relies on hand-tuning feature layers and
distance metrics. We propose Targeted Manifold Manipulation-Nearest Neighbour
(TMM-NN), which reconceptualises retrieval by assessing how readily each sample
can be nudged into a designated region of the feature manifold; neighbourhoods
are defined by a sample's responsiveness to a targeted perturbation rather than
absolute geometric distance. TMM-NN implements this through a lightweight,
query-specific trigger patch. The patch is added to the query image, and the
network is weakly ``backdoored'' so that any input with the patch is steered
toward a dummy class. Images similar to the query need only a slight shift and
are classified as the dummy class with high probability, while dissimilar ones
are less affected. By ranking candidates by this confidence, TMM-NN retrieves
the most semantically related neighbours. Robustness analysis and benchmark
experiments confirm this trigger-based ranking outperforms traditional metrics
under noise and across diverse tasks.
Despite the popularity of retrieval-augmented generation (RAG) as a solution
for grounded QA in both academia and industry, current RAG methods struggle
with questions where the necessary information is distributed across many
documents or where retrieval needs to be combined with complex reasoning.
Recently, the LOFT study has shown that this limitation also applies to
approaches based on long-context language models, with the QUEST benchmark
exhibiting particularly large headroom. In this paper, we provide an in-depth
analysis of the factors contributing to the poor performance on QUEST-LOFT,
publish updated numbers based on a thorough human evaluation, and demonstrate
that RAG can be optimized to significantly outperform long-context approaches
when combined with a structured output format containing reasoning and
evidence, optionally followed by answer re-verification.
Industrial part specification extraction from unstructured text remains a persistent challenge in manufacturing, procurement, and maintenance, where manual processing is both time-consuming and error-prone. This paper introduces a retrieval-augmented multi-LLM ensemble framework that orchestrates nine state-of-the-art Large Language Models (LLMs) within a structured three-phase pipeline. RAGsemble addresses key limitations of single-model systems by combining the complementary strengths of model families including Gemini (2.0, 2.5, 1.5), OpenAI (GPT-4o, o4-mini), Mistral Large, and Gemma (1B, 4B, 3n-e4b), while grounding outputs in factual data using FAISS-based semantic retrieval. The system architecture consists of three stages: (1) parallel extraction by diverse LLMs, (2) targeted research augmentation leveraging high-performing models, and (3) intelligent synthesis with conflict resolution and confidence-aware scoring. RAG integration provides real-time access to structured part databases, enabling the system to validate, refine, and enrich outputs through similarity-based reference retrieval. Experimental results using real industrial datasets demonstrate significant gains in extraction accuracy, technical completeness, and structured output quality compared to leading single-LLM baselines. Key contributions include a scalable ensemble architecture for industrial domains, seamless RAG integration throughout the pipeline, comprehensive quality assessment mechanisms, and a production-ready solution suitable for deployment in knowledge-intensive manufacturing environments.
Authors' comments: The 17th International Conference on Knowledge and Systems Engineering
Chunking quality determines RAG system performance. Current methods partition documents individually, but complex queries need information scattered across multiple sources: the knowledge fragmentation problem. We introduce Cross-Document Topic-Aligned (CDTA) chunking, which reconstructs knowledge at the corpus level. It first identifies topics across documents, maps segments to each topic, and synthesizes them into unified chunks.
On HotpotQA multi-hop reasoning, our method reached 0.93 faithfulness versus 0.83 for contextual retrieval and 0.78 for semantic chunking, a 12% improvement over current industry best practice (p < 0.05). On UAE Legal texts, it reached 0.94 faithfulness with 0.93 citation accuracy. At k = 3, it maintains 0.91 faithfulness while semantic methods drop to 0.68, with a single CDTA chunk containing information requiring multiple traditional fragments.
Indexing costs are higher, but synthesis produces information-dense chunks that reduce query-time retrieval needs. For high-query-volume applications with distributed knowledge, cross-document synthesis improves measurably over within-document optimization.
Understanding 3D scenes in open-world settings poses fundamental challenges
for vision and robotics, particularly due to the limitations of
closed-vocabulary supervision and static annotations. To address this, we
propose a unified framework for Open-World 3D Scene Graph Generation with
Retrieval-Augmented Reasoning, which enables generalizable and interactive 3D
scene understanding. Our method integrates Vision-Language Models (VLMs) with
retrieval-based reasoning to support multimodal exploration and language-guided
interaction. The framework comprises two key components: (1) a dynamic scene
graph generation module that detects objects and infers semantic relationships
without fixed label sets, and (2) a retrieval-augmented reasoning pipeline that
encodes scene graphs into a vector database to support text/image-conditioned
queries. We evaluate our method on 3DSSG and Replica benchmarks across four
tasks-scene question answering, visual grounding, instance retrieval, and task
planning-demonstrating robust generalization and superior performance in
diverse environments. Our results highlight the effectiveness of combining
open-vocabulary perception with retrieval-based reasoning for scalable 3D scene
understanding.
Authors' comments: Accepted by AAAI 2026
Zero-shot dense retrieval is a challenging setting where a document corpus is
provided without relevant queries, necessitating a reliance on pretrained dense
retrievers (DRs). However, since these DRs are not trained on the target
corpus, they struggle to represent semantic differences between similar
documents. To address this failing, we introduce a training-free representation
sharpening framework that augments a document's representation with information
that helps differentiate it from similar documents in the corpus. On over
twenty datasets spanning multiple languages, the representation sharpening
framework proves consistently superior to traditional retrieval, setting a new
state-of-the-art on the BRIGHT benchmark. We show that representation
sharpening is compatible with prior approaches to zero-shot dense retrieval and
consistently improves their performance. Finally, we address the
performance-cost tradeoff presented by our framework and devise an
indexing-time approximation that preserves the majority of our performance
gains over traditional retrieval, yet suffers no additional inference-time
cost.
Authors' comments: 15 pages, 4 figures
Retrieval-Augmented Generation (RAG) utilizes external knowledge to augment
Large Language Models' (LLMs) reliability. For flexibility, agentic RAG employs
autonomous, multi-round retrieval and reasoning to resolve queries. Although
recent agentic RAG has improved via reinforcement learning, they often incur
substantial token overhead from search and reasoning processes. This trade-off
prioritizes accuracy over efficiency. To address this issue, this work proposes
TeaRAG, a token-efficient agentic RAG framework capable of compressing both
retrieval content and reasoning steps. 1) First, the retrieved content is
compressed by augmenting chunk-based semantic retrieval with a graph retrieval
using concise triplets. A knowledge association graph is then built from
semantic similarity and co-occurrence. Finally, Personalized PageRank is
leveraged to highlight key knowledge within this graph, reducing the number of
tokens per retrieval. 2) Besides, to reduce reasoning steps, Iterative
Process-aware Direct Preference Optimization (IP-DPO) is proposed.
Specifically, our reward function evaluates the knowledge sufficiency by a
knowledge matching mechanism, while penalizing excessive reasoning steps. This
design can produce high-quality preference-pair datasets, supporting iterative
DPO to improve reasoning conciseness. Across six datasets, TeaRAG improves the
average Exact Match by 4% and 2% while reducing output tokens by 61% and 59% on
Llama3-8B-Instruct and Qwen2.5-14B-Instruct, respectively. Code is available at
https://github.com/Applied-Machine-Learning-Lab/TeaRAG.
Authors' comments: 32 pages
We present \texttt{EMPEROR}, an open-source Python framework designed for
efficient exoplanet detection and characterisation with radial velocities (RV).
\texttt{EMPEROR} integrates Dynamic Nested Sampling (DNS) and Adaptive Parallel
Tempering (APT) Markov Chain Monte Carlo (MCMC), supporting multiple noise
models such as Gaussian Processes (GPs) and Moving Averages (MA). The framework
enables systematic model comparison using statistical metrics, including
Bayesian evidence ($\ln{\mathcal{Z}}$) and Bayesian Information Criterion
(BIC), while providing automated, publish-ready visualisations.
\texttt{EMPEROR} is evaluated across three distinct systems to assess its
capabilities in different detection scenarios. Sampling performance, model
selection, and the search for Earth-mass planets are evaluated in data for 51
Pegasi, HD 55693 and Barnard's Star (GJ 699). For 51 Pegasi, APT achieves an
effective sampling increase over DNS by a factor 3.76, while retrieving tighter
parameter estimates. For HD 55693 the stellar rotation
$P_{\text{rot}}=29.72^{+0.01}_{-0.02}$ and magnetic cycle
$P_{\text{mag}}=2557.0^{+70.1}_{-36.7}$ are recovered, while demonstrating the
sensitivity of $\ln{\mathcal{Z}}$ to prior selection. For Barnard's star,
several noise models are compared, and the confirmed planet parameters are
successfully retrieved with all of them. The best model shows a period of
3.1536$\pm$0.0003~d, minimum mass of 0.38$\pm$0.03 M$_{\rm{\oplus}}$, and
semi-major axis of 0.02315$\pm$0.00039~AU. Purely statistical inference might
be insufficient on its own for robust exoplanet detection. Effective
methodologies must integrate domain knowledge, heuristic criteria, and
multi-faceted model comparisons. The versatility of \texttt{EMPEROR} in
handling diverse noise structures, its systematic model selection, and its
improved performance make it a valuable tool for RV exoplanetary studies.
Authors' comments: Accepted for publication in A&A Sect. 15. Numerical methods and
codes. The official acceptance date is 19/10/2025
