Vision-Language Model (VLM) based retrievers have advanced visual document retrieval (VDR) to impressive quality. They require the same multi-billion parameter encoder for both document indexing and query encoding, incurring high latency and GPU dependence even for plain-text queries. We observe that this design is unnecessarily symmetric: documents are visually complex and demand strong visual understanding, whereas queries are just short text strings. NanoVDR exploits this query--document asymmetry by decoupling the two encoding paths: a frozen 2B VLM teacher indexes documents offline, while a distilled text-only student as small as 69M parameters encodes queries at inference. The key design choice is the distillation objective. Through systematic comparison of six objectives across three backbones and 22 ViDoRe benchmark datasets, we find that pointwise cosine alignment on query text consistently outperforms ranking-based and contrastive alternatives, while requiring only pre-cached teacher query embeddings and no document processing during training. Furthermore, we identify cross-lingual transfer as the primary performance bottleneck, and resolve it cheaply by augmenting training data with machine-translated queries. The resulting NanoVDR-S-Multi (DistilBERT, 69M) retains 95.1\% of teacher quality and outperforms DSE-Qwen2 (2B) on v2 and v3 with 32$\times$ fewer parameters and 50$\times$ lower CPU query latency, at a total training cost under 13 GPU-hours.
Retrieval augmented generation RAG is widely deployed to improve factual accuracy in language models yet it remains unclear whether smaller models of size 7B parameters or less can effectively utilize retrieved information. To investigate this question we evaluate five model sizes from 360M to 8B across three architecture families SmolLM2 Qwen2.5 and Llama 3.1 under four retrieval conditions including no retrieval BM25 dense retrieval using E5 large v2 and oracle retrieval where the retrieved passage is guaranteed to contain the answer. We introduce a parametric knowledge split that separates questions a model can already answer from those that require external knowledge which allows us to isolate utilization failure from retrieval quality failure. We find three main results. First even with oracle retrieval models of size 7B or smaller fail to extract the correct answer 85 to 100 percent of the time on questions they cannot answer alone which indicates a fundamental utilization bottleneck. Second adding retrieval context destroys 42 to 100 percent of answers the model previously knew suggesting a distraction effect driven by the presence of context rather than its quality. Third an error analysis of 2588 oracle failures shows that the dominant failure mode is irrelevant generation where the model ignores the provided context entirely. These patterns hold across multiple prompt templates and retrieval methods. The results indicate that for models below 7B parameters the main limitation of RAG is context utilization rather than retrieval quality and that deploying RAG at this scale can lead to a net negative trade off under standard evaluation conditions.
Authors' comments: 10 pages, 5 figures, planning to submit to arr march 2026. Code and evaluation data: https://anonymous.4open.science/r/rag-utilization-study-C67F . Earlier draft preprint available on Zenodo: https://zenodo.org/records/18870116 (note: this arXiv submission is an updated draft)
2763.
Higress-RAG: A Holistic Optimization Framework for Enterprise Retrieval-Augmented Generation via Dual Hybrid Retrieval, Adaptive Routing, and CRAG
The integration of Large Language Models (LLMs) into enterprise knowledge management systems has been catalyzed by the Retrieval-Augmented Generation (RAG) paradigm, which augments parametric memory with non-parametric external data. However, the transition from proof-of-concept to production-grade RAG systems is hindered by three persistent challenges: low retrieval precision for complex queries, high rates of hallucination in the generation phase, and unacceptable latency for real-time applications. This paper presents a comprehensive analysis of the Higress RAG MCP Server, a novel, enterprise-centric architecture designed to resolve these bottlenecks through a "Full-Link Optimization" strategy. Built upon the Model Context Protocol (MCP), the system introduces a layered architecture that orchestrates a sophisticated pipeline of Adaptive Routing, Semantic Caching, Hybrid Retrieval, and Corrective RAG (CRAG). We detail the technical implementation of key innovations, including the Higress-Native Splitter for structure-aware data ingestion, the application of Reciprocal Rank Fusion (RRF) for merging dense and sparse retrieval signals, and a 50ms-latency Semantic Caching mechanism with dynamic thresholding. Experimental evaluations on domain-specific Higress technical documentation and blogs verify the system's architectural robustness. The results demonstrate that by optimizing the entire retrieval lifecycle - from pre-retrieval query rewriting to post-retrieval corrective evaluation - the Higress RAG system offers a scalable, hallucination-resistant solution for enterprise AI deployment.
Authors' comments: 7 pages,5 figures, our submissions are not yet published
Recent advancements in Large Language Models (LLMs) have transformed Natural Language Processing (NLP), enabling complex information retrieval and generation tasks. Retrieval-Augmented Generation (RAG) has emerged as a key innovation, enhancing factual accuracy and contextual grounding by integrating external knowledge sources with generative models. Although RAG demonstrates strong performance on unstructured text, its application to structured knowledge graphs presents challenges: scaling retrieval across connected graphs and preserving contextual relationships during response generation. Cross-encoders refine retrieval precision, yet their integration with structured data remains underexplored. Addressing these challenges is crucial for developing domain-specific assistants that operate in production environments. This study presents the design and comparative evaluation of multiple Retriever-Reranker pipelines for knowledge graph natural language queries in e-Commerce contexts. Using the STaRK Semi-structured Knowledge Base (SKB), a production-scale e-Commerce dataset, we evaluate multiple RAG pipeline configurations optimized for language queries. Experimental results demonstrate substantial improvements over published benchmarks, achieving 20.4% higher Hit@1 and 14.5% higher Mean Reciprocal Rank (MRR). These findings establish a practical framework for integrating domain-specific SKBs into generative systems. Our contributions provide actionable insights for the deployment of production-ready RAG systems, with implications that extend beyond e-Commerce to other domains that require information retrieval from structured knowledge bases.
Authors' comments: This manuscript is under review at the Springer journal Knowledge and Information Systems
Recent advancements in Retrieval-Augmented Generation (RAG) have enabled Large Language Models (LLMs) to access multimodal knowledge bases containing both text and visual information such as charts, diagrams, and tables in financial documents. However, existing multimodal RAG systems rely on LLM-based summarization to convert images into text during preprocessing, storing only text representations in vector databases, which causes loss of contextual information and visual details critical for downstream retrieval and question answering. To address this limitation, we present a comprehensive comparative analysis of two retrieval approaches for multimodal RAG systems, including text-based chunk retrieval (where images are summarized into text before embedding) and direct multimodal embedding retrieval (where images are stored natively in the vector space). We evaluate all three approaches across 6 LLM models and a two multi-modal embedding models on a newly created financial earnings call benchmark comprising 40 question-answer pairs, each paired with 2 documents (1 image and 1 text chunk). Experimental results demonstrate that direct multimodal embedding retrieval significantly outperforms LLM-summary-based approaches, achieving absolute improvements of 13% in mean average precision (mAP@5) and 11% in normalized discounted cumulative gain. These gains correspond to relative improvements of 32% in mAP@5 and 20% in nDCG@5, providing stronger evidence of their practical impact. We additionally find that direct multimodal retrieval produces more accurate and factually consistent answers as measured by LLM-as-a-judge pairwise comparisons. We demonstrate that LLM summarization introduces information loss during preprocessing, whereas direct multimodal embeddings preserve visual context for retrieval and inference.
Since the launch of JWST, observations of exoplanetary atmospheres have seen
a revolution in data quality. Given that atmospheric parameter inferences
depend heavily on the underlying data, a re-evaluation of current methodologies
is warranted to assess the reliability of these results. We investigate the
impact of variations in input spectra on atmospheric retrievals for the hot
Jupiter WASP-39 b using JWST transit data. Specifically, we analyse the
reliability of parameter estimations from random perturbations of the
underlying spectrum and their sensitivity to three transmission spectra derived
from the same observational data. Using the NIRSpec PRISM observation from a
single transit of WASP-39 b, we perform retrievals with the TauREx framework.
As a baseline, we use a spectrum derived with the Eureka! data reduction
pipeline. To evaluate retrieval reliability, we analyse posterior distributions
under deviations from this spectrum. We simulate random noise by performing
retrievals on scattered instances of this spectrum and compare them with
retrievals based on existing spectra reduced from the same raw observation. Our
analysis identifies three types of posterior distributions: (1) Stable,
Gaussian distributions for species constrained across the entire spectrum
(e.g., H2O, CO2); (2) Uniform posteriors with upper bounds for weakly
constrained species (e.g., CO, CH4); and (3) Unstable, heavy-tailed posteriors
for species constrained by minor spectrum features (e.g., SO2, C2H2). We find
that other parameters, such as the planetary radius and p-T profile, are stable
under spectral perturbations. Posterior distributions differ for retrievals on
independently reduced transmission spectra from the same raw data, complicating
interpretation, particularly for skewed distributions. Based on this, we
advocate for careful assessment and selection of credible interval sizes to
reflect this.
Authors' comments: 20 pages, 12 figures. Accepted for publication in Astronomy and
Astrophysics
Slide decks, serving as digital reports that bridge the gap between
presentation slides and written documents, are a prevalent medium for conveying
information in both academic and corporate settings. Their multimodal nature,
combining text, images, and charts, presents challenges for retrieval-augmented
generation systems, where the quality of retrieval directly impacts downstream
performance. Traditional approaches to slide retrieval often involve separate
indexing of modalities, which can increase complexity and lose contextual
information. This paper investigates various methodologies for effective slide
retrieval, including visual late-interaction embedding models like ColPali, the
use of visual rerankers, and hybrid retrieval techniques that combine dense
retrieval with BM25, further enhanced by textual rerankers and fusion methods
like Reciprocal Rank Fusion. A novel Vision-Language Models-based captioning
pipeline is also evaluated, demonstrating significantly reduced embedding
storage requirements compared to visual late-interaction techniques, alongside
comparable retrieval performance. Our analysis extends to the practical aspects
of these methods, evaluating their runtime performance and storage demands
alongside retrieval efficacy, thus offering practical guidance for the
selection and development of efficient and robust slide retrieval systems for
real-world applications.
Social media users often make scientific claims without citing where these
claims come from, generating a need to verify these claims. This paper details
work done by the DS@GT team for CLEF 2025 CheckThat! Lab Task 4b Scientific
Claim Source Retrieval which seeks to find relevant scientific papers based on
implicit references in tweets. Our team explored 6 different data augmentation
techniques, 7 different retrieval and reranking pipelines, and finetuned a
bi-encoder. Achieving an MRR@5 of 0.58, our team ranked 16th out of 30 teams
for the CLEF 2025 CheckThat! Lab Task 4b, and improvement of 0.15 over the BM25
baseline of 0.43. Our code is available on Github at
https://github.com/dsgt-arc/checkthat-2025-swd/tree/main/subtask-4b.
Graph-based retrieval-augmented generation (RAG) enables large language models (LLMs) to ground responses with structured external knowledge from up-to-date knowledge graphs (KGs) and reduce hallucinations. However, LLMs often rely on a weak retriever in graph-based RAG: I) Due to the lack of ground truth, the retriever is often trained on weak supervision, which often introduces spurious signals to the LLMs. II) Due to the abstraction of graph data, the retrieved knowledge is often presented in unorganized forms. To mitigate the issue, we present Refined Graph-based RAG (ReG) to align weak retrievers to LLMs for graph-based RAG. Specifically, ReG incorporates LLM feedback to get rid of spurious signals and improve the quality of the supervision. Meanwhile, ReG introduces a structure-aware reorganization module to refactor the retrieval results into logically coherent evidence chains. Experiments on prominent benchmarks demonstrate that ReG significantly and consistently brings improvements across different LLM backbones by up to 10%. The improved supervision quality enables ReG to match the state-of-the-art performance with 5% training data and to transfer to out-of-distribution KGs. Notably, when adopted to reasoning-based LLMs, ReG reduces the reasoning token cost by up to 30% and improves the performance by up to 4%.
This paper describes the participation of QUST_NLP in the SemEval-2025 Task 7. We propose a three-stage retrieval framework specifically designed for fact-checked claim retrieval. Initially, we evaluate the performance of several retrieval models and select the one that yields the best results for candidate retrieval. Next, we employ multiple re-ranking models to enhance the candidate results, with each model selecting the Top-10 outcomes. In the final stage, we utilize weighted voting to determine the final retrieval outcomes. Our approach achieved 5th place in the monolingual track and 7th place in the crosslingual track. We release our system code at: https://github.com/warmth27/SemEval2025_Task7
We present the methodology and results of the Deep Retrieval team for subtask
4b of the CLEF CheckThat! 2025 competition, which focuses on retrieving
relevant scientific literature for given social media posts. To address this
task, we propose a hybrid retrieval pipeline that combines lexical precision,
semantic generalization, and deep contextual re-ranking, enabling robust
retrieval that bridges the informal-to-formal language gap. Specifically, we
combine BM25-based keyword matching with a FAISS vector store using a
fine-tuned INF-Retriever-v1 model for dense semantic retrieval. BM25 returns
the top 30 candidates, and semantic search yields 100 candidates, which are
then merged and re-ranked via a large language model (LLM)-based cross-encoder.
Our approach achieves a mean reciprocal rank at 5 (MRR@5) of 76.46% on the
development set and 66.43% on the hidden test set, securing the 1st position on
the development leaderboard and ranking 3rd on the test leaderboard (out of 31
teams), with a relative performance gap of only 2 percentage points compared to
the top-ranked system. We achieve this strong performance by running
open-source models locally and without external training data, highlighting the
effectiveness of a carefully designed and fine-tuned retrieval pipeline.
Large language models excel at many tasks but often incur high inference
costs during deployment. To mitigate hallucination, many systems use a
knowledge graph to enhance retrieval-augmented generation (KG-RAG). However,
the large amount of retrieved knowledge contexts increase these inference costs
further. A promising solution to balance performance and cost is LLM routing,
which directs simple queries to smaller LLMs and complex ones to larger LLMs.
However, no dedicated routing methods currently exist for RAG, and existing
training-based routers face challenges scaling to this domain due to the need
for extensive training data. We observe that the score distributions produced
by the retrieval scorer strongly correlate with query difficulty. Based on
this, we propose a novel, training-free routing framework, the first tailored
to KG-RAG that effectively balances performance and cost in a plug-and-play
manner. Experiments show our method reduces calls to larger LLMs by up to 50%
without sacrificing response quality, demonstrating its potential for efficient
and scalable LLM deployment.
Online disinformation poses a global challenge, placing significant demands
on fact-checkers who must verify claims efficiently to prevent the spread of
false information. A major issue in this process is the redundant verification
of already fact-checked claims, which increases workload and delays responses
to newly emerging claims. This research introduces an approach that retrieves
previously fact-checked claims, evaluates their relevance to a given input, and
provides supplementary information to support fact-checkers. Our method employs
large language models (LLMs) to filter irrelevant fact-checks and generate
concise summaries and explanations, enabling fact-checkers to faster assess
whether a claim has been verified before. In addition, we evaluate our approach
through both automatic and human assessments, where humans interact with the
developed tool to review its effectiveness. Our results demonstrate that LLMs
are able to filter out many irrelevant fact-checks and, therefore, reduce
effort and streamline the fact-checking process.
With rapid advancements in artificial intelligence, question-answering (Q&A)
systems have become essential in intelligent search engines, virtual
assistants, and customer service platforms. However, in dynamic domains like
smart grids, conventional retrieval-augmented generation(RAG) Q&A systems face
challenges such as inadequate retrieval quality, irrelevant responses, and
inefficiencies in handling large-scale, real-time data streams. This paper
proposes an optimized iterative retrieval-based Q&A framework called Chats-Grid
tailored for smart grid environments. In the pre-retrieval phase, Chats-Grid
advanced query expansion ensures comprehensive coverage of diverse data
sources, including sensor readings, meter records, and control system
parameters. During retrieval, Best Matching 25(BM25) sparse retrieval and BAAI
General Embedding(BGE) dense retrieval in Chats-Grid are combined to process
vast, heterogeneous datasets effectively. Post-retrieval, a fine-tuned large
language model uses prompt engineering to assess relevance, filter irrelevant
results, and reorder documents based on contextual accuracy. The model further
generates precise, context-aware answers, adhering to quality criteria and
employing a self-checking mechanism for enhanced reliability. Experimental
results demonstrate Chats-Grid's superiority over state-of-the-art methods in
fidelity, contextual recall, relevance, and accuracy by 2.37%, 2.19%, and 3.58%
respectively. This framework advances smart grid management by improving
decision-making and user interactions, fostering resilient and adaptive smart
grid infrastructures.
Authors' comments: 12 pages, 10 figures
Accessing accurate medication insights is vital for enhancing patient safety,
minimizing errors, and supporting clinical decision-making. However, healthcare
professionals in Africa often rely on manual and time-consuming processes to
retrieve drug information, exacerbated by limited access to pharmacists due to
brain drain and healthcare disparities. This paper presents "Drug Insights," an
open-source Retrieval-Augmented Generation (RAG) chatbot designed to streamline
medication lookup for healthcare workers in Africa. By leveraging a corpus of
Nigerian pharmaceutical data and advanced AI technologies, including Pinecone
databases and GPT models, the system delivers accurate, context-specific
responses with minimal hallucination. The chatbot integrates prompt engineering
and S-BERT evaluation to optimize retrieval and response generation.
Preliminary tests, including pharmacist feedback, affirm the tool's potential
to improve drug information access while highlighting areas for enhancement,
such as UI/UX refinement and extended corpus integration.
Authors' comments: 4 pages, 2 tables and 3 figures
Composed Image Retrieval (CIR) aims to retrieve target images that closely
resemble a reference image while integrating user-specified textual
modifications, thereby capturing user intent more precisely. Existing
training-free zero-shot CIR (ZS-CIR) methods often employ a two-stage process:
they first generate a caption for the reference image and then use Large
Language Models for reasoning to obtain a target description. However, these
methods suffer from missing critical visual details and limited reasoning
capabilities, leading to suboptimal retrieval performance. To address these
challenges, we propose a novel, training-free one-stage method, One-Stage
Reflective Chain-of-Thought Reasoning for ZS-CIR (OSrCIR), which employs
Multimodal Large Language Models to retain essential visual information in a
single-stage reasoning process, eliminating the information loss seen in
two-stage methods. Our Reflective Chain-of-Thought framework further improves
interpretative accuracy by aligning manipulation intent with contextual cues
from reference images. OSrCIR achieves performance gains of 1.80% to 6.44% over
existing training-free methods across multiple tasks, setting new
state-of-the-art results in ZS-CIR and enhancing its utility in vision-language
applications. Our code will be available at
https://github.com/Pter61/osrcir2024/.
In Retrieval-Augmented Generation (RAG) tasks using Large Language Models
(LLMs), the quality of retrieved information is critical to the final output.
This paper introduces the IRSC benchmark for evaluating the performance of
embedding models in multilingual RAG tasks. The benchmark encompasses five
retrieval tasks: query retrieval, title retrieval, part-of-paragraph retrieval,
keyword retrieval, and summary retrieval. Our research addresses the current
lack of comprehensive testing and effective comparison methods for embedding
models in RAG scenarios. We introduced new metrics: the Similarity of Semantic
Comprehension Index (SSCI) and the Retrieval Capability Contest Index (RCCI),
and evaluated models such as Snowflake-Arctic, BGE, GTE, and M3E. Our
contributions include: 1) the IRSC benchmark, 2) the SSCI and RCCI metrics, and
3) insights into the cross-lingual limitations of embedding models. The IRSC
benchmark aims to enhance the understanding and development of accurate
retrieval systems in RAG tasks. All code and datasets are available at:
https://github.com/Jasaxion/IRSC_Benchmark
Retrieval-augmented generation (RAG) techniques leverage the in-context
learning capabilities of large language models (LLMs) to produce more accurate
and relevant responses. Originating from the simple 'retrieve-then-read'
approach, the RAG framework has evolved into a highly flexible and modular
paradigm. A critical component, the Query Rewriter module, enhances knowledge
retrieval by generating a search-friendly query. This method aligns input
questions more closely with the knowledge base. Our research identifies
opportunities to enhance the Query Rewriter module to Query Rewriter+ by
generating multiple queries to overcome the Information Plateaus associated
with a single query and by rewriting questions to eliminate Ambiguity, thereby
clarifying the underlying intent. We also find that current RAG systems exhibit
issues with Irrelevant Knowledge; to overcome this, we propose the Knowledge
Filter. These two modules are both based on the instruction-tuned Gemma-2B
model, which together enhance response quality. The final identified issue is
Redundant Retrieval; we introduce the Memory Knowledge Reservoir and the
Retriever Trigger to solve this. The former supports the dynamic expansion of
the RAG system's knowledge base in a parameter-free manner, while the latter
optimizes the cost for accessing external knowledge, thereby improving resource
utilization and response efficiency. These four RAG modules synergistically
improve the response quality and efficiency of the RAG system. The
effectiveness of these modules has been validated through experiments and
ablation studies across six common QA datasets. The source code can be accessed
at https://github.com/Ancientshi/ERM4.
Authors' comments: ECAI2024 #1304
Generating high-quality answers consistently by providing contextual
information embedded in the prompt passed to the Large Language Model (LLM) is
dependent on the quality of information retrieval. As the corpus of contextual
information grows, the answer/inference quality of Retrieval Augmented
Generation (RAG) based Question Answering (QA) systems declines. This work
solves this problem by combining classical text classification with the Large
Language Model (LLM) to enable quick information retrieval from the vector
store and ensure the relevancy of retrieved information. For the same, this
work proposes a new approach Context Augmented retrieval (CAR), where
partitioning of vector database by real-time classification of information
flowing into the corpus is done. CAR demonstrates good quality answer
generation along with significant reduction in information retrieval and answer
generation time.
2780.
Npix2Cpix: A GAN-based Image-to-Image Translation Network with
Retrieval-Classification Integration for Watermark Retrieval from Historical
Document Images
Utsab Saha, Sawradip Saha, Shaikh Anowarul Fattah, Mohammad Saquib
The identification and restoration of ancient watermarks have long been a
major topic in codicology and history. Classifying historical documents based
on watermarks is challenging due to their diversity, noisy samples, multiple
representation modes, and minor distinctions between classes and intra-class
variations. This paper proposes a modified U-net-based conditional generative
adversarial network (GAN) named Npix2Cpix to translate noisy raw historical
watermarked images into clean, handwriting-free watermarked images by
performing image translation from degraded (noisy) pixels to clean pixels.
Using image-to-image translation and adversarial learning, the network creates
clutter-free images for watermark restoration and categorization. The generator
and discriminator of the proposed GAN are trained using two separate loss
functions, each based on the distance between images, to learn the mapping from
the input noisy image to the output clean image. After using the proposed GAN
to pre-process noisy watermarked images, Siamese-based one-shot learning is
employed for watermark classification. Experimental results on a large-scale
historical watermark dataset demonstrate that cleaning the noisy watermarked
images can help to achieve high one-shot classification accuracy. The
qualitative and quantitative evaluation of the retrieved watermarked image
highlights the effectiveness of the proposed approach.
