Visual retrieval-augmented generation (VRAG) augments vision-language models (VLMs) with external visual knowledge to ground reasoning and reduce hallucinations. Yet current VRAG systems often fail to reliably perceive and integrate evidence across multiple images, leading to weak grounding and erroneous conclusions. In this paper, we propose EVisRAG, an end-to-end framework that learns to reason with evidence-guided multi-image to address this issue. The model first observes retrieved images and records per-image evidence, then derives the final answer from the aggregated evidence. To train EVisRAG effectively, we introduce Reward-Scoped Group Relative Policy Optimization (RS-GRPO), which binds fine-grained rewards to scope-specific tokens to jointly optimize visual perception and reasoning abilities of VLMs. Experimental results on multiple visual question answering benchmarks demonstrate that EVisRAG delivers substantial end-to-end gains over backbone VLM with 27\% improvements on average. Further analysis shows that, powered by RS-GRPO, EVisRAG improves answer accuracy by precisely perceiving and localizing question-relevant evidence across multiple images and deriving the final answer from that evidence, much like a real detective.
Vision-and-Language Navigation (VLN) requires agents to follow natural
language instructions through environments, with memory-persistent variants
demanding progressive improvement through accumulated experience. Existing
approaches for memory-persistent VLN face critical limitations: they lack
effective memory access mechanisms, instead relying on entire memory
incorporation or fixed-horizon lookup, and predominantly store only
environmental observations while neglecting navigation behavioral patterns that
encode valuable decision-making strategies. We present Memoir, which employs
imagination as a retrieval mechanism grounded by explicit memory: a world model
imagines future navigation states as queries to selectively retrieve relevant
environmental observations and behavioral histories. The approach comprises: 1)
a language-conditioned world model that imagines future states serving dual
purposes: encoding experiences for storage and generating retrieval queries; 2)
Hybrid Viewpoint-Level Memory that anchors both observations and behavioral
patterns to viewpoints, enabling hybrid retrieval; and 3) an
experience-augmented navigation model that integrates retrieved knowledge
through specialized encoders. Extensive evaluation across diverse
memory-persistent VLN benchmarks with 10 distinctive testing scenarios
demonstrates Memoir's effectiveness: significant improvements across all
scenarios, with 5.4% SPL gains on IR2R over the best memory-persistent
baseline, accompanied by 8.3x training speedup and 74% inference memory
reduction. The results validate that predictive retrieval of both environmental
and behavioral memories enables more effective navigation, with analysis
indicating substantial headroom (73.3% vs 93.4% upper bound) for this
imagination-guided paradigm. Code at https://github.com/xyz9911/Memoir.
Authors' comments: 14 pages, 6 figures, 13 tables
The remarkable success of large language models (LLMs) stems from their
ability to consolidate vast amounts of knowledge into the memory during
pre-training and to retrieve it from the memory during inference, enabling
advanced capabilities such as knowledge memorization, instruction-following and
reasoning. However, the mechanisms of memory retrieval and consolidation in
LLMs remain poorly understood. In this paper, we propose the function token
hypothesis to explain the workings of LLMs: During inference, function tokens
activate the most predictive features from context and govern next token
prediction (memory retrieval). During pre-training, predicting the next tokens
(usually content tokens) that follow function tokens increases the number of
learned features of LLMs and updates the model parameters (memory
consolidation). Function tokens here roughly correspond to function words in
linguistics, including punctuation marks, articles, prepositions, and
conjunctions, in contrast to content tokens. We provide extensive experimental
evidence supporting this hypothesis. Using bipartite graph analysis, we show
that a small number of function tokens activate the majority of features. Case
studies further reveal how function tokens activate the most predictive
features from context to direct next token prediction. We also find that during
pre-training, the training loss is dominated by predicting the next content
tokens following function tokens, which forces the function tokens to select
the most predictive features from context.
Composed Image Retrieval (CIR), which aims to find a target image from a
reference image and a modification text, presents the core challenge of
performing unified reasoning across visual and semantic modalities. While
current approaches based on Vision-Language Models (VLMs, e.g., CLIP) and more
recent Multimodal Large Language Models (MLLMs, e.g., Qwen-VL) have shown
progress, they predominantly function as ``black boxes." This inherent opacity
not only prevents users from understanding the retrieval rationale but also
restricts the models' ability to follow complex, fine-grained instructions. To
overcome these limitations, we introduce CIR-CoT, the first end-to-end
retrieval-oriented MLLM designed to integrate explicit Chain-of-Thought (CoT)
reasoning. By compelling the model to first generate an interpretable reasoning
chain, CIR-CoT enhances its ability to capture crucial cross-modal
interactions, leading to more accurate retrieval while making its decision
process transparent. Since existing datasets like FashionIQ and CIRR lack the
necessary reasoning data, a key contribution of our work is the creation of
structured CoT annotations using a three-stage process involving a caption,
reasoning, and conclusion. Our model is then fine-tuned to produce this
structured output before encoding its final retrieval intent into a dedicated
embedding. Comprehensive experiments show that CIR-CoT achieves highly
competitive performance on in-domain datasets (FashionIQ, CIRR) and
demonstrates remarkable generalization on the out-of-domain CIRCO dataset,
establishing a new path toward more effective and trustworthy retrieval
systems.
Modern software teams frequently encounter delays in resolving recurring or related issues due to fragmented knowledge scattered across JIRA tickets, developer discussions, and GitHub pull requests (PRs). To address this challenge, we propose a Retrieval-Augmented Generation (RAG) framework that integrates Sentence-Transformers for semantic embeddings with FAISS-based vector search to deliver context-aware ticket resolution recommendations. The approach embeds historical JIRA tickets, user comments, and linked PR metadata to retrieve semantically similar past cases, which are then synthesized by a Large Language Model (LLM) into grounded and explainable resolution suggestions. The framework contributes a unified pipeline linking JIRA and GitHub data, an embedding and FAISS indexing strategy for heterogeneous software artifacts, and a resolution generation module guided by retrieved evidence. Experimental evaluation using precision, recall, resolution time reduction, and developer acceptance metrics shows that the proposed system significantly improves resolution accuracy, fix quality, and knowledge reuse in modern DevOps environments.
Authors' comments: 13 Pages
Sarcasm is a subtle form of non-literal language that poses significant
challenges for speech synthesis due to its reliance on nuanced semantic,
contextual, and prosodic cues. While existing speech synthesis research has
focused primarily on broad emotional categories, sarcasm remains largely
unexplored. In this paper, we propose a Large Language Model (LLM)-enhanced
Retrieval-Augmented framework for sarcasm-aware speech synthesis. Our approach
combines (1) semantic embeddings from a LoRA-fine-tuned LLaMA 3, which capture
pragmatic incongruity and discourse-level cues of sarcasm, and (2) prosodic
exemplars retrieved via a Retrieval Augmented Generation (RAG) module, which
provide expressive reference patterns of sarcastic delivery. Integrated within
a VITS backbone, this dual conditioning enables more natural and contextually
appropriate sarcastic speech. Experiments demonstrate that our method
outperforms baselines in both objective measures and subjective evaluations,
yielding improvements in speech naturalness, sarcastic expressivity, and
downstream sarcasm detection.
We consider image transmission via deep joint source-channel coding
(DeepJSCC) over multi-hop additive white Gaussian noise (AWGN) channels by
training a DeepJSCC encoder-decoder pair with a pre-trained deep hash
distillation (DHD) module to semantically cluster images, facilitating
security-oriented applications through enhanced semantic consistency and
improving the perceptual reconstruction quality. We train the DeepJSCC module
to both reduce mean square error (MSE) and minimize cosine distance between DHD
hashes of source and reconstructed images. Significantly improved perceptual
quality as a result of semantic alignment is illustrated for different
multi-hop settings, for which classical DeepJSCC may suffer from noise
accumulation, measured by the learned perceptual image patch similarity (LPIPS)
metric.
The application of advanced generative artificial intelligence in education is often constrained by the lack of real-time adaptability, personalization, and reliability of the content. To address these challenges, we propose ExpertAgent - an intelligent agent framework designed for personalized education that provides reliable knowledge and enables highly adaptive learning experiences. Therefore, we developed ExpertAgent, an innovative learning agent that provides users with a proactive and personalized learning experience. ExpertAgent dynamic planning of the learning content and strategy based on a continuously updated student model. Therefore, overcoming the limitations of traditional static learning content to provide optimized teaching strategies and learning experience in real time. All instructional content is grounded in a validated curriculum repository, effectively reducing hallucination risks in large language models and improving reliability and trustworthiness.
Authors' comments: Manuscript previously submitted to the NeurIPS 2025 Workshop on Bridging Language, Agent, and World Models (LAW 2025)
We present a novel quantum storage algorithm for k binary vectors of dimension m into a superposition of a m qubit quantum state based on a permutation technique. We compare this algorithm to the storage algorithm proposed by Ventura and Martinez. The permutation technique is simpler and can lead to an additional reduction through the reduce algorithm. To retrieve a binary vector from the superposition of k vectors represented by a m qubit quantum state, we must use a modified version of Grover algorithm, as Grover algorithm does not function correctly for non uniform distributions. We introduce the permutation trick that enables an exhaustive search by Grover algorithm in square root of k steps for k patterns, independent of n equal two power m. We compare this trick to the Ventura and Martinez trick, which requires square root of n steps for k patterns.
Wikipedia is a critical resource for modern NLP, serving as a rich repository of up-to-date and citation-backed information on a wide variety of subjects. The reliability of Wikipedia -- its groundedness in its cited sources -- is vital to this purpose. This work analyzes both how grounded Wikipedia is and how readily fine-grained grounding evidence can be retrieved. To this end, we introduce PeopleProfiles -- a large-scale, multi-level dataset of claim support annotations on biographical Wikipedia articles. We show that: (1) ~22% of claims in Wikipedia lead sections are unsupported by the article body; (2) ~30% of claims in the article body are unsupported by their publicly accessible sources; and (3) real-world Wikipedia citation practices often differ from documented standards. Finally, we show that complex evidence retrieval remains a challenge -- even for recent reasoning rerankers.
Reasoning over structured graphs remains a fundamental challenge for Large Language Models (LLMs), particularly when scaling to large graphs. Existing approaches typically follow the retrieval-augmented generation (RAG) paradigm: first retrieving subgraphs relevant to the query and then generating answers conditioned on the retrieved subgraphs. However, such two-phase pipelines often struggle to faithfully incorporate graph structure, since the generation process is ultimately constrained by the quality and completeness of the retrieved subgraph. Although many advanced retrievers have been proposed recently to mitigate this issue, they are usually tailored to the training graphs and generalize poorly to unseen graphs, which limits their practical applicability. In this work, we propose Reasoning by Exploration (RoE), a novel approach that unifies retrieval and generation by framing reasoning over graphs as a process of graph exploration. At each step, the LLM selects candidate nodes and edges to explore, gradually constructing reasoning paths and generating answers along the way. To enable effective exploration, RoE is trained in two stages: supervised fine-tuning (SFT) on gold reasoning paths, followed by reinforcement learning (RL) to enhance exploration effectiveness and generalization. Experiments on benchmark datasets demonstrate that RoE achieves substantial overall improvements over baselines, while also generalizing effectively to unseen graphs.
The Structure-Aware Temporal Graph RAG (SAT-Graph RAG) addresses core
limitations of standard Retrieval-Augmented Generation in the legal domain by
providing a verifiable knowledge graph that models hierarchical structure,
temporal evolution, and causal events of legal norms. However, a critical gap
remains: how to reliably query this structured knowledge without sacrificing
its deterministic properties. This paper introduces the SAT-Graph API, a formal
query execution layer centered on canonical actions-atomic, composable, and
auditable primitives that isolate probabilistic discovery from deterministic
retrieval. These actions enable: (i) high-precision hybrid search; (ii) robust
reference resolution; (iii) point-in-time version retrieval; and (iv) auditable
causal tracing. We demonstrate how planner-guided agents can decompose complex
queries into Directed Acyclic Graphs (DAGs) of these actions. This two-layer
architecture transforms retrieval from an opaque black box to a transparent,
auditable process, directly addressing Explainable AI (XAI) requirements for
high-stakes domains.
Agentic Retrieval-Augmented Generation (Agentic RAG) enhances the processing
capability for complex tasks through dynamic retrieval and adaptive workflows.
Recent advances (e.g., Search-R1) have shown that outcome-supervised
reinforcement learning demonstrate strong performance. However, this approach
still suffers from inefficient exploration, sparse reward signals, and
ambiguous global reward feedback. To address these challenges, we propose
DecEx-RAG, which models RAG as a Markov Decision Process (MDP) incorporating
decision-making and execution, while introducing an efficient pruning strategy
to optimize data expansion. Through comprehensive process-level policy
optimization, DecEx-RAG significantly enhances the autonomous task
decomposition, dynamic retrieval, and high-quality answer generation
capabilities of large language models (LLMs). Experiments show that DecEx-RAG
achieves an average absolute performance improvement of $6.2\%$ across six
datasets, significantly outperforming existing baselines. Moreover, the pruning
strategy improves data construction efficiency by nearly $6 \times$, providing
an efficient solution for process-supervised RAG training. The code is
available at https://github.com/sdsxdxl/DecEx-RAG.
Existing Visual Language Models (VLMs) suffer structural limitations where a
few low contribution tokens may excessively capture global semantics,
dominating the information aggregation process and suppressing the
discriminative features in text-driven image retrieval tasks. To address this,
we introduce \textbf{CalibCLIP}, a training-free method designed to calibrate
the suppressive effect of dominant tokens. Specifically, in the visual space,
we propose the Contrastive Visual Enhancer (CVE), which decouples visual
features into target and low information regions. Subsequently, it identifies
dominant tokens and dynamically suppresses their representations.In the textual
space, we introduce the Discriminative Concept Calibrator (DCC), which aims to
differentiate between general and discriminative concepts within the text
query. By mitigating the challenges posed by generic concepts and improving the
representations of discriminative concepts, DCC strengthens the differentiation
among similar samples. Finally, extensive experiments demonstrate consistent
improvements across seven benchmarks spanning three image retrieval tasks,
underscoring the effectiveness of CalibCLIP. Code is available at:
https://github.com/kangbin98/CalibCLIP
Authors' comments: ACMMM2025(oral)
We present ARRC (Advanced Reasoning Robot Control), a practical system that
connects natural-language instructions to safe local robotic control by
combining Retrieval-Augmented Generation (RAG) with RGB-D perception and
guarded execution on an affordable robot arm. The system indexes curated robot
knowledge (movement patterns, task templates, and safety heuristics) in a
vector database, retrieves task-relevant context for each instruction, and
conditions a large language model (LLM) to produce JSON-structured action
plans. Plans are executed on a UFactory xArm 850 fitted with a Dynamixel-driven
parallel gripper and an Intel RealSense D435 camera. Perception uses AprilTag
detections fused with depth to produce object-centric metric poses. Execution
is enforced via software safety gates: workspace bounds, speed and force caps,
timeouts, and bounded retries. We describe the architecture, knowledge design,
integration choices, and a reproducible evaluation protocol for tabletop scan,
approach, and pick-place tasks. Experimental results demonstrate the efficacy
of the proposed approach. Our design shows that RAG-based planning can
substantially improve plan validity and adaptability while keeping perception
and low-level control local to the robot.
This work draws on the conjecture that fingerprints of stochastic event sequences can be retrieved from electroencephalographic data (EEG) recorded during a behavioral task. To test this, we used the Goalkeeper Game (game.numec.prp.usp.br). Acting as a goalkeeper, the participant predicted each kick in a probabilistic sequence while EEG activity was recorded. At each trial, driven by a context tree, the kicker chose one of three options: left, center, or right. The goalkeeper then predicted the next kick by pressing a button. Tree estimation was performed by applying the Context Algorithm to EEG segments locked to the button press (-300 to 0 ms). We calculated the distance between the penalty taker's tree and the trees retrieved per participant and electrode. This metric was then correlated with the goalkeeper's success rates. We observed a clear reduction in the overall distance distribution over time for a subset of electrodes, indicating that EEG dependencies become more congruent with the penalty taker's tree as the goalkeeper learns the sequence. This distance is inversely proportional to the goalkeepers' success rates, indicating a clear relationship between performance and the neural signatures associated with the sequence structure.
Authors' comments: 18 pages,7 figures, 1 table
Recent advancements in large language models (LLMs) have substantially
improved automated code generation. While function-level and file-level
generation have achieved promising results, real-world software development
typically requires reasoning across entire repositories. This gives rise to the
challenging task of Repository-Level Code Generation (RLCG), where models must
capture long-range dependencies, ensure global semantic consistency, and
generate coherent code spanning multiple files or modules. To address these
challenges, Retrieval-Augmented Generation (RAG) has emerged as a powerful
paradigm that integrates external retrieval mechanisms with LLMs, enhancing
context-awareness and scalability. In this survey, we provide a comprehensive
review of research on Retrieval-Augmented Code Generation (RACG), with an
emphasis on repository-level approaches. We categorize existing work along
several dimensions, including generation strategies, retrieval modalities,
model architectures, training paradigms, and evaluation protocols. Furthermore,
we summarize widely used datasets and benchmarks, analyze current limitations,
and outline key challenges and opportunities for future research. Our goal is
to establish a unified analytical framework for understanding this rapidly
evolving field and to inspire continued progress in AI-powered software
engineering.
Retrieval-Augmented Generation (RAG) is a powerful technique for enriching
Large Language Models (LLMs) with external knowledge, allowing for factually
grounded responses, a critical requirement in high-stakes domains such as
healthcare. However, the efficacy of RAG systems is fundamentally restricted by
the performance of their retrieval module, since irrelevant or semantically
misaligned documents directly compromise the accuracy of the final generated
response. General-purpose dense retrievers can struggle with the nuanced
language of specialised domains, while the high accuracy of in-domain models is
often achieved at prohibitive computational costs. In this work, we aim to
address this trade-off by developing and evaluating a two-stage retrieval
architecture that combines a lightweight ModernBERT bidirectional encoder for
efficient initial candidate retrieval with a ColBERTv2 late-interaction model
for fine-grained re-ranking. We conduct comprehensive evaluations of our
retriever module performance and RAG system performance in the biomedical
context, fine-tuning the IR module using 10k question-passage pairs from
PubMedQA. Our analysis of the retriever module confirmed the positive impact of
the ColBERT re-ranker, which improved Recall@3 by up to 4.2 percentage points
compared to its retrieve-only counterpart. When integrated into the biomedical
RAG, our IR module leads to a state-of-the-art average accuracy of 0.4448 on
the five tasks of the MIRAGE question-answering benchmark, outperforming strong
baselines such as MedCPT (0.4436). Our ablation studies reveal that this
performance is critically dependent on a joint fine-tuning process that aligns
the retriever and re-ranker; otherwise, the re-ranker might degrade the
performance.
Accurate auto-formalization of theorem statements is essential for advancing
automated discovery and verification of research-level mathematics, yet remains
a major bottleneck for LLMs due to hallucinations, semantic mismatches, and
their inability to synthesize new definitions. To tackle these issues, we
present Aria (Agent for Retrieval and Iterative Autoformalization), a system
for conjecture-level formalization in Lean that emulates human expert reasoning
via a two-phase Graph-of-Thought process: recursively decomposing statements
into a dependency graph and then constructing formalizations from grounded
concepts. To ensure semantic correctness, we introduce AriaScorer, a checker
that retrieves definitions from Mathlib for term-level grounding, enabling
rigorous and reliable verification. We evaluate Aria on diverse benchmarks. On
ProofNet, it achieves 91.6% compilation success rate and 68.5% final accuracy,
surpassing previous methods. On FATE-X, a suite of challenging algebra problems
from research literature, it outperforms the best baseline with 44.0% vs. 24.0%
final accuracy. On a dataset of homological conjectures, Aria reaches 42.9%
final accuracy while all other models score 0%.
In the current paradigm of image captioning, deep learning models are trained to generate text from image embeddings of latent features. We challenge the assumption that fine-tuning of large, bespoke models is required to improve model generation accuracy. Here we propose Label Boosted Retrieval Augmented Generation (LaB-RAG), a small-model-based approach to image captioning that leverages image descriptors in the form of categorical labels to boost standard retrieval augmented generation (RAG) with pretrained large language models (LLMs). We study our method in the context of radiology report generation (RRG) over MIMIC-CXR and CheXpert Plus. We argue that simple classification models combined with zero-shot embeddings can effectively transform X-rays into text-space as radiology-specific labels. In combination with standard RAG, we show that these derived text labels can be used with general-domain LLMs to generate radiology reports. Without ever training our generative language model or image embedding models specifically for the task, and without ever directly "showing" the LLM an X-ray, we demonstrate that LaB-RAG achieves better results across natural language and radiology language metrics compared with other retrieval-based RRG methods, while attaining competitive results compared to other fine-tuned vision-language RRG models. We further conduct extensive ablation experiments to better understand the components of LaB-RAG. Our results suggest broader compatibility and synergy with fine-tuned methods to further enhance RRG performance.
