In Retrieval-Augmented Generation (RAG) tasks using Large Language Models
(LLMs), the quality of retrieved information is critical to the final output.
This paper introduces the IRSC benchmark for evaluating the performance of
embedding models in multilingual RAG tasks. The benchmark encompasses five
retrieval tasks: query retrieval, title retrieval, part-of-paragraph retrieval,
keyword retrieval, and summary retrieval. Our research addresses the current
lack of comprehensive testing and effective comparison methods for embedding
models in RAG scenarios. We introduced new metrics: the Similarity of Semantic
Comprehension Index (SSCI) and the Retrieval Capability Contest Index (RCCI),
and evaluated models such as Snowflake-Arctic, BGE, GTE, and M3E. Our
contributions include: 1) the IRSC benchmark, 2) the SSCI and RCCI metrics, and
3) insights into the cross-lingual limitations of embedding models. The IRSC
benchmark aims to enhance the understanding and development of accurate
retrieval systems in RAG tasks. All code and datasets are available at:
https://github.com/Jasaxion/IRSC_Benchmark
Retrieval-augmented generation (RAG) techniques leverage the in-context
learning capabilities of large language models (LLMs) to produce more accurate
and relevant responses. Originating from the simple 'retrieve-then-read'
approach, the RAG framework has evolved into a highly flexible and modular
paradigm. A critical component, the Query Rewriter module, enhances knowledge
retrieval by generating a search-friendly query. This method aligns input
questions more closely with the knowledge base. Our research identifies
opportunities to enhance the Query Rewriter module to Query Rewriter+ by
generating multiple queries to overcome the Information Plateaus associated
with a single query and by rewriting questions to eliminate Ambiguity, thereby
clarifying the underlying intent. We also find that current RAG systems exhibit
issues with Irrelevant Knowledge; to overcome this, we propose the Knowledge
Filter. These two modules are both based on the instruction-tuned Gemma-2B
model, which together enhance response quality. The final identified issue is
Redundant Retrieval; we introduce the Memory Knowledge Reservoir and the
Retriever Trigger to solve this. The former supports the dynamic expansion of
the RAG system's knowledge base in a parameter-free manner, while the latter
optimizes the cost for accessing external knowledge, thereby improving resource
utilization and response efficiency. These four RAG modules synergistically
improve the response quality and efficiency of the RAG system. The
effectiveness of these modules has been validated through experiments and
ablation studies across six common QA datasets. The source code can be accessed
at https://github.com/Ancientshi/ERM4.
Authors' comments: ECAI2024 #1304
Generating high-quality answers consistently by providing contextual
information embedded in the prompt passed to the Large Language Model (LLM) is
dependent on the quality of information retrieval. As the corpus of contextual
information grows, the answer/inference quality of Retrieval Augmented
Generation (RAG) based Question Answering (QA) systems declines. This work
solves this problem by combining classical text classification with the Large
Language Model (LLM) to enable quick information retrieval from the vector
store and ensure the relevancy of retrieved information. For the same, this
work proposes a new approach Context Augmented retrieval (CAR), where
partitioning of vector database by real-time classification of information
flowing into the corpus is done. CAR demonstrates good quality answer
generation along with significant reduction in information retrieval and answer
generation time.
2884.
Npix2Cpix: A GAN-based Image-to-Image Translation Network with
Retrieval-Classification Integration for Watermark Retrieval from Historical
Document Images
Utsab Saha, Sawradip Saha, Shaikh Anowarul Fattah, Mohammad Saquib
The identification and restoration of ancient watermarks have long been a
major topic in codicology and history. Classifying historical documents based
on watermarks is challenging due to their diversity, noisy samples, multiple
representation modes, and minor distinctions between classes and intra-class
variations. This paper proposes a modified U-net-based conditional generative
adversarial network (GAN) named Npix2Cpix to translate noisy raw historical
watermarked images into clean, handwriting-free watermarked images by
performing image translation from degraded (noisy) pixels to clean pixels.
Using image-to-image translation and adversarial learning, the network creates
clutter-free images for watermark restoration and categorization. The generator
and discriminator of the proposed GAN are trained using two separate loss
functions, each based on the distance between images, to learn the mapping from
the input noisy image to the output clean image. After using the proposed GAN
to pre-process noisy watermarked images, Siamese-based one-shot learning is
employed for watermark classification. Experimental results on a large-scale
historical watermark dataset demonstrate that cleaning the noisy watermarked
images can help to achieve high one-shot classification accuracy. The
qualitative and quantitative evaluation of the retrieved watermarked image
highlights the effectiveness of the proposed approach.
Recent studies have demonstrated that the ability of dense retrieval models
to generalize to target domains with different distributions is limited, which
contrasts with the results obtained with interaction-based models. Prior
attempts to mitigate this challenge involved leveraging adversarial learning
and query generation approaches, but both approaches nevertheless resulted in
limited improvements. In this paper, we propose to combine the query-generation
approach with a self-supervision approach in which pseudo-relevance labels are
automatically generated on the target domain. To accomplish this, a T5-3B model
is utilized for pseudo-positive labeling, and meticulous hard negatives are
chosen. We also apply this strategy on conversational dense retrieval model for
conversational search. A similar pseudo-labeling approach is used, but with the
addition of a query-rewriting module to rewrite conversational queries for
subsequent labeling. This proposed approach enables a model's domain adaptation
with real queries and documents from the target dataset. Experiments on
standard dense retrieval and conversational dense retrieval models both
demonstrate improvements on baseline models when they are fine-tuned on the
pseudo-relevance labeled data.
Authors' comments: 12 pages, accepted by COLING 2024
2886.
Exploiting Semantic Role Contextualized Video Features for
Multi-Instance Text-Video Retrieval EPIC-KITCHENS-100 Multi-Instance
Retrieval Challenge 2022
In this report, we present our approach for EPIC-KITCHENS-100 Multi-Instance
Retrieval Challenge 2022. We first parse sentences into semantic roles
corresponding to verbs and nouns; then utilize self-attentions to exploit
semantic role contextualized video features along with textual features via
triplet losses in multiple embedding spaces. Our method overpasses the strong
baseline in normalized Discounted Cumulative Gain (nDCG), which is more
valuable for semantic similarity. Our submission is ranked 3rd for nDCG and
ranked 4th for mAP.
Authors' comments: Ranked joint 3rd place in the Multi-Instance Retrieval Challenge at
EPIC@CVPR2022. (v2: ref error is corrected)
In recent years, there has been a notable increase in the level of attention that is given to algorithms based on deep learning in the context of medical image segmentation. Nevertheless, the reliability of the field has been hindered due to the absence of a standardized methodology for performance analysis and the utilization of different datasets in previous research. The primary objective of the research is to comprehensively evaluate contemporary segmentation frameworks combined with state-of-the-art pre-trained backbones in order to accurately predict COVID-19 lesions in CT images. Moreover, this evaluation can serve as a point of reference for the segmentation of images in various other imaging scenarios. In order to accomplish this, we integrate four distinct deep learning architectures, namely Unet, PSPNet, Linknet, and FPN, with six pre-trained encoders, including VGG 19, DenseNet 121, Inception ResNet V2, MobileNet V2, SeresNet 101, and EfficientNet B0. This approach enables the development of diverse testing architectures. In the context of image segmentation, our research encompassed both binary and multi-class experimentation. The findings derived from our analysis of three distinct COVID-19 CT segmentation datasets indicate that deep learning architectures yield precise and efficient segmentation outcomes. Significantly, a maximum F1-Score of 98% was attained for binary class segmentation, while multi-class segmentation yielded F1-Scores of 75% and 77% across two separate datasets. The utilization of artificial intelligence and deep learning enhances the diagnostic process for pandemic diseases across multiple dimensions.
Authors' comments: 7 pages, 6 figures, 4 tables
Response-free item difficulty modelling promises to reduce reliance on response-based calibration but is intrinsically difficult on reading-comprehension multiple-choice items, where difficulty depends on inferential demands across wording components. Whereas most existing approaches extract item-text features and pass them to a separate statistical or machine-learning model, we fine-tune transformer encoders end-to-end on the item wording, eliminating the manual feature engineering and preprocessing that discards information. Moreover, two extensions to this joint-encoding approach are proposed: a component-wise variant that encodes wording components separately through a shared encoder, and a multi-task variant that retains joint encoding and adds an auxiliary multiple-choice question answering objective on the shared encoder. Each method is evaluated under a Monte Carlo subsampling design at three training-set sizes on a held-out test set. We find that joint encoding is a viable end-to-end alternative to feature-engineering pipelines; while the component-wise variant shows no detectable benefit, consistent with self-attention already harvesting the cross-component signal, the multi-task variant delivers significant paired improvements in the smallest-sample regime. Transformer fine-tuning, especially if regularised by a suitable auxiliary task, recovers a substantial share of the wording-derivable signal at training-set sizes typical of applied measurement. The framework provides a customisable interface for psychometrically motivated extensions.
Medical vision-language models (VLMs) and AI agents have made significant progress in learning to analyze and reason about clinical images. However, existing medical visual question answering (VQA) benchmarks collapse model capabilities into a single accuracy score, obscuring where and why models fail. We propose DeepTumorVQA, a hierarchical benchmark that follows the multi-stage evidence chain in tumor diagnosis and decomposes 3D CT reasoning into four stages: recognition, measurement, visual reasoning, and medical reasoning. Higher-level questions remain independently scorable, while their ground-truth evidence chains are defined over lower-level primitives. The benchmark contains 476K questions across 42 clinical subtypes on 9,262 3D CT volumes. In addition to a direct reasoning mode for VLMs, DeepTumorVQA provides tool-interaction environments for agent evaluation, where a model can call external tools, including segmentation models, measurement programs, and medical knowledge modules, before answering the question. Evaluating over 30 model configurations, we find that reliable quantitative measurement is the primary bottleneck, making later-stage visual and medical reasoning harder for VLMs, while tool augmentation substantially mitigates this issue. When tools are available, leveraging medical knowledge and tools to reason about medical images becomes a new challenge. We further show that ground-truth step-by-step tool-use traces from DeepTumorVQA can supervise agents and reduce tool-use and reasoning failures. This stage-wise progression from recognition to measurement to visual and medical reasoning provides a concrete roadmap for future medical VLM and AI agent studies. All data and code are released at https://github.com/Schuture/DeepTumorVQA.
The WISE source, J141046.00+740511.2, has been recently observed from radio to $γ$ rays. Although the optical spectrum is consistent with a BL Lacertae (BL Lac) object, the source displays unusually weak radio emission, which challenges standard interpretations. Our aim is to understand the origin of the broadband emission from J141046.00+740511.2, using a leptonic model of an extended jet. To obtain the distribution of electrons along the conical jet, we solved a steady-state convective transport equation. Emissivities were computed along the jet and integrated over the cone volume to obtain the observed flux. Our model successfully reproduces the observed multiwavelength spectral energy distribution from radio to $γ$ rays and naturally accounts for the source's low radio flux without invoking extra emission zones. We also reproduce the mid-IR emission within the same framework. These results demonstrate that extended jet leptonic models can robustly describe the broadband physics of radio-weak BL Lacs.
Authors' comments: 7 pages, 4 figures, accepted for publication in A&A
This work presents ROSA, a microring-based optical neural network architecture that improves robustness and energy efficiency using an optical shift-and-add (OSA) module and a layer-wise hybrid mapping strategy. It introduces a noise-aware voltage-to-weight model considering DAC and thermal variations, and a workload-aware framework to co-optimize MRR array size and layer-wise dataflow. Optimized arrays reduce the aggregated relative energy-delay product (EDP) by 64% and 26% compared with DEAP-CNNs and a general compact array, respectively. OSA further contributes 29% EDP reduction. The proposed hybrid mapping strategy improves CIFAR-10 accuracy by 8.3% over weight-stationary mapping while achieving an average 54.7% lower EDP than DEAP-CNNs.
Long-context inference in Large Language Models (LLMs) is bottlenecked by the linear growth of Key-Value (KV) cache memory. Existing KV cache compression paradigms are fundamentally limited by heuristics: heuristic budgeting relies on statistical priors rather than task objectives, causing resource misallocation, while heuristic selection relies on coupled query-key interactions or static inductive biases (e.g., attention sinks). To address this limitation, we introduce LKV (Learned KV Eviction), which formulates KV compression as an end-to-end differentiable optimization problem. LKV integrates LKV-H to learn task-optimized global budgets, and LKV-T to derive intrinsic KV importance without materializing attention matrices. This design bypasses heuristic proxies, strictly aligning compression with task objectives. Extensive evaluations demonstrate that LKV achieves state-of-the-art performance on both LongBench and RULER benchmarks at high compression rates. In particular, on LongBench, LKV achieves near-lossless performance with only 15\% KV cache retention. Crucially, our analysis identifies learned budgeting as the dominant driver of fidelity, demonstrating that data-driven allocation is essential to overcome the limitations of hand-crafted heuristics.
The use of Large Language Models (LLMs) to support patients in addressing medical questions is becoming increasingly prevalent. However, most of the measures currently used to evaluate the performance of these models in this context only measure how closely a model's answers match semantically, and therefore do not provide a true indication of the model's medical accuracy or of the health equity risks associated with it. To address these shortcomings, we present a new evaluation framework for medical question answering called VB-Score (Verification-Based Score) that provides a separate evaluation of the four components of entity recognition, semantic similarity, factual consistency, and structured information completeness for medical question-answering models. We perform rigorous reviews of the performance of three well-known and widely used LLMs on 48 public health-related topics taken from high-quality, authoritative information sources. Based on our analyses, we discover a major discrepancy between the models' semantic and entity accuracy. Our assessments of the performance of all three models show that each of them has almost uniformly severe performance failures when evaluated against our criteria. Our findings indicate alarming performance disparities across various public health topics, with most of the models exhibiting 13.8% lower performance (compared to an overall average) for all the public health topics that relate to chronic conditions that occur in older and minority populations, which indicates the existence of what's known as condition-based algorithmic discrimination. Our findings also demonstrate that prompt engineering alone does not compensate for basic architectural limitations on how these models perform in extracting medical entities and raise the question of whether semantic evaluation alone is a sufficient measure of medical AI safety.
Authors' comments: Accepted in the Ninth Annual ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT) 2026
In LLM-based code generation, multiple code candidates are often generated in parallel from the same prompt -- for example, in best-of-N sampling or multi-candidate code completion. These requests can share KV caches through a common prefix, yet the extent to which their Mixture-of-Experts (MoE) expert routing overlaps, and how this overlap varies across layers, remains insufficiently understood. We study Qwen3.5-35B-A3B-FP8 (256 routed experts, top-8) by performing tree-search-based branching generation from a shared prefix (851 completed codes, temperature 0.7) and analyzing the results with a compiler-output-based alignment (gcc -S -O0 assembly) that controls for token-identity confounds. Our findings are threefold: (1) At positions where both sequences generated the same token, Jaccard similarity reaches 0.649 (40x random), while even at positions with different tokens it remains 0.175 (11x random). (2) A layer-wise decomposition reveals a crossing pattern: same-token routing similarity exceeds different-token similarity across all layers, but dips in the middle layers (L14-20), while different-token similarity peaks in the middle layers at 14x random. (3) In tree-search code generation, 67% of successfully compiled codes concentrate in the top three assembly-equivalent groups, and 99.6% of within-group differences consist of comments and blank lines. We show that diversity in top-P search, including beam search, poses a significant challenge. These results refine the "context-independent routing" claim of prior work through layer-wise decomposition and suggest opportunities for improving search efficiency in LLM code generation.
Digital twin technology, when combined with physics-informed machine learning with simulation results of Aspen, offers transformative capabilities for industrial process monitoring, control, and optimization. In this work, the proposed model presents a Physics-Informed Neural Network (PINN) digital twin framework for the dynamic, tray-wise modeling of binary distillation columns operating under transient conditions. The architecture of the proposed model embeds fundamental thermodynamic constraints, including vapor-liquid equilibrium (VLE) described by modified Raoult's law, tray-level mass and energy balances, and the McCabe-Thiele graphical methodology directly into the neural network loss function via physics residual terms. The model is trained and evaluated on a high-fidelity synthetic dataset of 961 timestamped measurements spanning 8 hours of transient operation, generated in Aspen HYSYS for a binary HX/TX distillation system comprising 16 sensor streams. An adaptive loss-weighting scheme balances the data fidelity and physics consistency objectives during training. Compared to five data-driven baselines (LSTM, vanilla MLP, GRU, Transformer, DeepONet), the proposed PINN achieves an RMSE of 0.00143 for HX mole fraction prediction (R^2 = 0.9887), representing a 44.6% reduction over the best data-only baseline, while strictly satisfying thermodynamic constraints. Tray-wise temperature and composition profiles predicted under transient perturbations demonstrate that the digital twin accurately captures column dynamics including feed tray responses, reflux ratio variations, and pressure transients. These results establish the proposed PINN digital twin as a robust foundation for real-time soft sensing, model-predictive control, and anomaly detection in industrial distillation processes.
Authors' comments: 17 pages, 10 figures
Diffusion Transformers (DiTs) achieve strong video generation quality but suffer from high inference cost due to dense 3D attention, leading to the development of sparse attention technologies to improve efficiency. However, existing training-free sparse attention methods in video generation still face two unresolved limitations: ignoring layer heterogeneity in attention pruning and ignoring query-key coupling in block partitioning, which hinder a better quality-speedup trade-off. In this work, we uncover a critical insight that the attention sparsity of each layer is its intrinsic property, with minor effects across different inputs. Motivated by this, we propose SVOO, a training-free Sparse attention framework for fast Video generation via Offline layer-wise sparsity profiling and Online bidirectional co-clustering. Specifically, SVOO adopts a two-stage paradigm: (i) offline layer-wise sensitivity profiling to derive intrinsic per-layer pruning levels, and (ii) online block-wise sparse attention via a novel bidirectional co-clustering algorithm. Extensive experiments on seven widely used video generation models demonstrate that SVOO achieves a superior quality-speedup trade-off over state-of-the-art methods, delivering up to $1.93\times$ speedup while maintaining a PSNR of up to 29 dB on Wan2.1.
Layer-wise mixed-precision quantization (LMPQ) enables effective compression under extreme low-bit settings by allocating higher precision to sensitive layers. However, existing methods typically treat all intra-layer weight modules uniformly and rely on a single numerical property when estimating sensitivity, overlooking their distinct operational roles and structural characteristics. To address this, we propose NSDS, a novel calibration-free LMPQ framework driven by Numerical and Structural Dual-Sensitivity. Specifically, it first mechanistically decomposes each layer into distinct operational roles and quantifies their sensitivity from both numerical and structural perspectives. These dual-aspect scores are then aggregated into a unified layer-wise metric through a robust aggregation scheme based on MAD-Sigmoid and Soft-OR to guide bit allocation. Extensive experiments demonstrate that NSDS consistently achieves superior performance compared to various baselines across diverse models and downstream tasks, without relying on any calibration data.
Machine learning (ML) techniques have been demonstrated to improve the accuracy and efficiency of anomaly detection (AD) when compared to conventional methods. This has led to the adoption of ML for data quality monitoring (DQM) use cases in order to monitor the operation of certain systems to ensure that they are free of undesirable or potentially deleterious anomalies. For applications in the field of High-Energy physics (HEP), where detectors must operate in long-running, harsh environments, ML models used in DQM that have been trained on static datasets are bound to experience degraded performance due to distributional shifts that naturally occur in the incoming data streams, unless directly mitigated via the inclusion of continual ML techniques. This work introduces DepthViT, a lightweight masked autoencoder architecture that employs unique depth-wise embeddings and cross-depth attention, to perform computationally efficient AD tasks. A continual learning framework is developed in which DepthViT models trained on the most recent data streams are ensembled with older models to create a robust overall system which is more resilient to shifts in incoming data streams. When evaluated on occupancy maps from the Compact Muon Solenoid (CMS) hadron calorimeter across multiple data-taking campaigns, the proposed method maintains precision above 99\% and stable ratio of correct anomaly predictions to number of anomalies both under small and large distributional shifts. Beyond HEP, the same ensembling-based continual adaptation strategy can be directly applied to industrial monitoring environments where data also naturally evolve over time. This work therefore presents a path toward adaptive anomaly detection systems capable of sustained operation in dynamic data environments.
Mixture-of-Experts (MoE) architectures have emerged as a powerful paradigm for scaling neural networks while maintaining computational efficiency. However, standard MoE implementations rely on two rigid design assumptions: (1) fixed Top-K routing where exactly K experts are activated per token, and (2) uniform expert allocation across all layers. This paper introduces DynaMoE, a novel MoE framework that relaxes both constraints through dynamic token-level expert activation and layer-wise adaptive capacity allocation. DynaMoE introduces a principled routing mechanism where the number of active experts per token varies based on input complexity. Concurrently, the framework implements six distinct scheduling strategies for distributing expert capacity across network depth, including descending, ascending, pyramid, and wave patterns. We theoretically analyze the expressivity gains of dynamic routing and derive bounds on computational efficiency. Through extensive experiments on MNIST, Fashion-MNIST, CIFAR-10 (image classification), and Recycling-the-Web (language modeling) across multiple model scales, we demonstrate that DynaMoE achieves superior parameter efficiency compared to static baselines. Our key finding is that optimal expert schedules are task- and scale-dependent: descending schedules (concentrating capacity in early layers) outperform uniform baselines on image classification. For language modeling, optimal schedules vary by model size, descending for Tiny, ascending for Small, and uniform for Medium. Furthermore, dynamic routing reduces gradient variance during training, leading to improved convergence stability. DynaMoE establishes a new framework for adaptive computation in neural networks, providing principled guidance for MoE architecture design.
Fingerspelling is a component of sign languages in which words are spelled out letter by letter using specific hand poses. Automatic fingerspelling recognition plays a crucial role in bridging the communication gap between Deaf and hearing communities, yet it remains challenging due to the signing-hand ambiguity issue, the lack of appropriate training losses, and the out-of-vocabulary (OOV) problem. Prior fingerspelling recognition methods rely on explicit signing-hand detection, which often leads to recognition failures, and on a connectionist temporal classification (CTC) loss, which exhibits the peaky behavior problem. To address these issues, we develop OpenFS, an open-source approach for fingerspelling recognition and synthesis. We propose a multi-hand-capable fingerspelling recognizer that supports both single- and multi-hand inputs and performs implicit signing-hand detection by incorporating a dual-level positional encoding and a signing-hand focus (SF) loss. The SF loss encourages cross-attention to focus on the signing hand, enabling implicit signing-hand detection during recognition. Furthermore, without relying on the CTC loss, we introduce a monotonic alignment (MA) loss that enforces the output letter sequence to follow the temporal order of the input pose sequence through cross-attention regularization. In addition, we propose a frame-wise letter-conditioned generator that synthesizes realistic fingerspelling pose sequences for OOV words. This generator enables the construction of a new synthetic benchmark, called FSNeo. Through comprehensive experiments, we demonstrate that our approach achieves state-of-the-art performance in recognition and validate the effectiveness of the proposed recognizer and generator. Codes and data are available in: https://github.com/JunukCha/OpenFS.
Authors' comments: Accepted to CVPR 2026
