Coherent, broadband pulses of extreme ultraviolet (XUV) light provide a new
and exciting tool for exploring attosecond electron dynamics. Using
photoelectron streaking, interferometric spectrograms can be generated that
contain a wealth of information about the phase properties of the
photoionization process. If properly retrieved, this phase information reveals
attosecond dynamics during photoelectron emission such as multielectron
dynamics and resonance processes. However, until now, the full retrieval of the
continuous electron wavepacket phase from isolated attosecond pulses has
remained challenging. Here, after elucidating key approximations and
limitations that hinder one from extracting the coherent electron wavepacket
dynamics using available retrieval algorithms, we present a new method called
Absolute Complex Dipole transmission matrix element reConstruction (ACDC). We
apply the ACDC method to experimental spectrograms to resolve the phase and
group delay difference between photoelectrons emitted from Ne and Ar. Our
results reveal subtle dynamics in this group delay difference of photoelectrons
emitted form Ar. These group delay dynamics were not resolvable with prior
methods that were only able to extract phase information at discrete energy
levels, emphasizing the importance of a complete and continuous phase retrieval
technique such as ACDC. Here we also make this new ACDC retrieval algorithm
available with appropriate citation in return.
Language model pre-training has been shown to capture a surprising amount of
world knowledge, crucial for NLP tasks such as question answering. However,
this knowledge is stored implicitly in the parameters of a neural network,
requiring ever-larger networks to cover more facts.
To capture knowledge in a more modular and interpretable way, we augment
language model pre-training with a latent knowledge retriever, which allows the
model to retrieve and attend over documents from a large corpus such as
Wikipedia, used during pre-training, fine-tuning and inference. For the first
time, we show how to pre-train such a knowledge retriever in an unsupervised
manner, using masked language modeling as the learning signal and
backpropagating through a retrieval step that considers millions of documents.
We demonstrate the effectiveness of Retrieval-Augmented Language Model
pre-training (REALM) by fine-tuning on the challenging task of Open-domain
Question Answering (Open-QA). We compare against state-of-the-art models for
both explicit and implicit knowledge storage on three popular Open-QA
benchmarks, and find that we outperform all previous methods by a significant
margin (4-16% absolute accuracy), while also providing qualitative benefits
such as interpretability and modularity.
Phase retrieval is in general a non-convex and non-linear task and the
corresponding algorithms struggle with the issue of local minima. We consider
the case where the measurement samples within typically very small and
disconnected subsets are coherently linked to each other - which is a
reasonable assumption for our objective of antenna measurements. Two classes of
measurement setups are discussed which can provide this kind of extra
information: multi-probe systems and holographic measurements with multiple
reference signals. We propose several formulations of the corresponding phase
retrieval problem. The simplest of these formulations poses a linear system of
equations similar to an eigenvalue problem where a unique non-trivial
null-space vector needs to be found. Accurate phase reconstruction for
partially coherent observations is, thus, possible by a reliable solution
process and with judgment of the solution quality. Under ideal, noise-free
conditions, the required sampling density is less than two times the number of
unknowns. Noise and other observation errors increase this value slightly.
Simulations for Gaussian random matrices and for antenna measurement scenarios
demonstrate that reliable phase reconstruction is possible with the presented
approach.
Authors' comments: 12 pages, 14 figures
Over the last several years, spectroscopic observations of transiting
exoplanets have begun to uncover information about their atmospheres, including
atmospheric composition and indications of the presence of clouds and hazes.
Spectral retrieval is the leading technique for interpretation of transmission
spectra and is employed by several teams using a variety of forward models and
parameter estimation algorithms. However, different model suites have mostly
been used in isolation and so it is unknown whether the results from each are
comparable. As we approach the launch of the James Webb Space Telescope we
anticipate advances in wavelength coverage, precision, and resolution of
transit spectroscopic data, so it is important that the tools that will be used
to interpret these information rich spectra are validated. To this end, we
present an inter-model comparison of three retrieval suites: TauREx, NEMESIS
and CHIMERA. We demonstrate that the forward model spectra are in good
agreement (residual deviations on the order of 20 - 40 ppm), and discuss the
results of cross retrievals between the three tools. Generally, the constraints
from the cross retrievals are consistent with each other and with input values
to within 1 sigma However, for high precision scenarios with error envelopes of
order 30 ppm, subtle differences in the simulated spectra result in
discrepancies between the different retrieval suites, and inaccuracies in
retrieved values of several sigma. This can be considered analogous to
substantial systematic/astrophysical noise in a real observation, or
errors/omissions in a forward model such as molecular linelist incompleteness
or missing absorbers.
Authors' comments: 25 pages, 21 figures. Accepted in MNRAS
This paper democratizes neural information retrieval to scenarios where large
scale relevance training signals are not available. We revisit the classic IR
intuition that anchor-document relations approximate query-document relevance
and propose a reinforcement weak supervision selection method, ReInfoSelect,
which learns to select anchor-document pairs that best weakly supervise the
neural ranker (action), using the ranking performance on a handful of relevance
labels as the reward. Iteratively, for a batch of anchor-document pairs,
ReInfoSelect back propagates the gradients through the neural ranker, gathers
its NDCG reward, and optimizes the data selection network using policy
gradients, until the neural ranker's performance peaks on target relevance
metrics (convergence). In our experiments on three TREC benchmarks, neural
rankers trained by ReInfoSelect, with only publicly available anchor data,
significantly outperform feature-based learning to rank methods and match the
effectiveness of neural rankers trained with private commercial search logs.
Our analyses show that ReInfoSelect effectively selects weak supervision
signals based on the stage of the neural ranker training, and intuitively picks
anchor-document pairs similar to query-document pairs.
Authors' comments: Accepted by WWW 2020
Consider the problem of Private Information Retrieval (PIR), where a user
wishes to retrieve a single message from $N$ non-communicating and
non-colluding databases (servers). All servers store the same set of $M$
messages and they respond to the user through a block fading Gaussian Multiple
Access Channel (MAC). The goal in this setting is to keep the index of the
required message private from the servers while minimizing the overall
communication overhead.
This work provides joint privacy and channel coding retrieval schemes for the
Gaussian MAC with and without fading. The schemes exploit the linearity of the
channel while using the Compute and Forward (CF) coding scheme. Consequently,
single-user encoding and decoding are performed to retrieve the private
message. In the case of a channel without fading, the achievable retrieval rate
is shown to outperform a separation-based scheme, in which the retrieval and
the channel coding are designed separately. Moreover, this rate is
asymptotically optimal as the SNR grows, and are up to a constant gap of $2$
bits per channel use from the channel capacity without privacy constraints, for
all SNR values. When the channel suffers from fading, the asymmetry between the
servers' channels forces a more complicated solution, which involves a hard
optimization problem. Nevertheless, we provide coding scheme and lower bounds
on the expected achievable retrieval rate which are shown to have the same
scaling laws as the channel capacity, both in the number of servers and the
SNR.
Active learning is a machine learning method aiming at optimal design for
model training. At variance with supervised learning, which labels all samples,
active learning provides an improved model by labeling samples with maximal
uncertainty according to the estimation model. Here, we propose the use of
active learning for efficient quantum information retrieval, which is a crucial
task in the design of quantum experiments. Meanwhile, when dealing with large
data output, we employ active learning for the sake of classification with
minimal cost in fidelity loss. Indeed, labeling only 5% samples, we achieve
almost 90% rate estimation. The introduction of active learning methods in the
data analysis of quantum experiments will enhance applications of quantum
technologies.
In this paper, we propose the application of conditional generative
adversarial networks to solve various phase retrieval problems. We show that
including knowledge of the measurement process at training time leads to an
optimization at test time that is more robust to initialization than existing
approaches involving generative models. In addition, conditioning the generator
network on the measurements enables us to achieve much more detailed results.
We empirically demonstrate that these advantages provide meaningful solutions
to the Fourier and the compressive phase retrieval problem and that our method
outperforms well-established projection-based methods as well as existing
methods that are based on neural networks. Like other deep learning methods,
our approach is very robust to noise and can therefore be very useful for
real-world applications.
Authors' comments: Accepted at the 25th International Conference on Pattern Recognition
2020 (ICPR)
This article presents a summary graph to show the relationships between
Information Retrieval (IR) and other related disciplines. The figure tells the
key differences between them and the conditions under which one would
transition into another.
Generally, phase retrieval problem can be viewed as the reconstruction of a
function/signal from only the magnitude of the linear measurements. These
measurements can be, for example, the Fourier transform of the density
function. Computationally the phase retrieval problem is very challenging. Many
algorithms for phase retrieval are based on i.i.d. Gaussian random
measurements. However, Gaussian random measurements remain one of the very few
classes of measurements. In this paper, we develop an efficient phase retrieval
algorithm for sub-gaussian random frames. We provide a general condition for
measurements and develop a modified spectral initialization. In the algorithm,
we first obtain a good approximation of the solution through the
initialization, and from there we useWirtinger Flow to solve for the solution.
We prove that the algorithm converges to the global minimizer linearly.
Authors' comments: 20 pages, 2 figures
This paper provides a tutorial of iterative phase retrieval algorithms based
on the Gerchberg-Saxton (GS) algorithm applied in digital holography. In
addition, a novel GS-based algorithm that allows reconstruction of 3D samples
is demonstrated. The GS-based algorithms recover complex-valued wavefront by
wavefront back-and forth propagation between two planes with constraints
superimposed in these two planes. Iterative phase retrieval allows
quantitatively correct and twin-image-free reconstructions of object amplitude
and phase distributions from its in-line hologram. The present work derives the
quantitative criteria on how many holograms are required to reconstruct a
complex-valued object distribution, be it a 2D or 3D sample. It is shown that
for a sample that can be approximated as a 2D sample, a single-shot in-line
hologram is sufficient to reconstruct the absorption and phase distributions of
the sample. Previously, the GS-based algorithms have been successfully employed
to reconstruct samples that are limited to a 2D plane. However, realistic
physical objects always have some finite thickness and therefore are 3D rather
than 2D objects. This study demonstrates that 3D samples, including 3D phase
objects, can be reconstructed from two or more holograms. It is shown that in
principle, two holograms are sufficient to recover the entire wavefront
diffracted by a 3D sample distribution. In this method, the reconstruction is
performed by applying iterative phase retrieval between the planes where
intensity was measured. The recovered complex-valued wavefront is then
propagated back to the sample planes, thus reconstructing the 3D distribution
of the sample. This method can be applied for 3D samples such as 3D
distribution of particles, thick biological samples, and other 3D phase
objects. Examples of reconstructions of 3D objects, including phase objects,
are provided.
Recent work has shown the surprising ability of multi-lingual BERT to serve
as a zero-shot cross-lingual transfer model for a number of language processing
tasks. We combine this finding with a similarly-recently proposal on
sentence-level relevance modeling for document retrieval to demonstrate the
ability of multi-lingual BERT to transfer models of relevance across languages.
Experiments on test collections in five different languages from diverse
language families (Chinese, Arabic, French, Hindi, and Bengali) show that
models trained with English data improve ranking quality, without any special
processing, both for (non-English) mono-lingual retrieval as well as
cross-lingual retrieval.
Work on retrieval-based chatbots, like most sequence pair matching tasks, can
be divided into Cross-encoders that perform word matching over the pair, and
Bi-encoders that encode the pair separately. The latter has better performance,
however since candidate responses cannot be encoded offline, it is also much
slower. Lately, multi-layer transformer architectures pre-trained as language
models have been used to great effect on a variety of natural language
processing and information retrieval tasks. Recent work has shown that these
language models can be used in text-matching scenarios to create Bi-encoders
that perform almost as well as Cross-encoders while having a much faster
inference speed. In this paper, we expand upon this work by developing a
sequence matching architecture that %takes into account contexts in the
training dataset at inference time. utilizes the entire training set as a
makeshift knowledge-base during inference. We perform detailed experiments
demonstrating that this architecture can be used to further improve Bi-encoders
performance while still maintaining a relatively high inference speed.
Authors' comments: 8 pages, 1 figure, 3 tables
Retrieving videos of a particular person with face image as a query via
hashing technique has many important applications. While face images are
typically represented as vectors in Euclidean space, characterizing face videos
with some robust set modeling techniques (e.g. covariance matrices as exploited
in this study, which reside on Riemannian manifold), has recently shown
appealing advantages. This hence results in a thorny heterogeneous spaces
matching problem. Moreover, hashing with handcrafted features as done in many
existing works is clearly inadequate to achieve desirable performance for this
task. To address such problems, we present an end-to-end Deep Heterogeneous
Hashing (DHH) method that integrates three stages including image feature
learning, video modeling, and heterogeneous hashing in a single framework, to
learn unified binary codes for both face images and videos. To tackle the key
challenge of hashing on the manifold, a well-studied Riemannian kernel mapping
is employed to project data (i.e. covariance matrices) into Euclidean space and
thus enables to embed the two heterogeneous representations into a common
Hamming space, where both intra-space discriminability and inter-space
compatibility are considered. To perform network optimization, the gradient of
the kernel mapping is innovatively derived via structured matrix
backpropagation in a theoretically principled way. Experiments on three
challenging datasets show that our method achieves quite competitive
performance compared with existing hashing methods.
Authors' comments: 14 pages, 17 figures, 4 tables, accepted by IEEE Transactions on
Image Processing (TIP) 2019
We consider the problem of conjugate phase retrieval in Paley-Wiener space
$PW_{\pi}$. The goal of conjugate phase retrieval is to recover a signal $f$
from the magnitudes of linear measurements up to unknown phase factor and
unknown conjugate, meaning $f(t)$ and $\overline{f(t)}$ are not necessarily
distinguishable from the available data. We show that conjugate phase retrieval
can be accomplished in $PW_{\pi}$ by sampling only on the real line by using
structured convolutions. We also show that conjugate phase retrieval can be
accomplished in $PW_{\pi}$ by sampling both $f$ and $f^{\prime}$ only on the
real line. Moreover, we demonstrate experimentally that the Gerchberg-Saxton
method of alternating projections can accomplish the reconstruction from
vectors that do conjugate phase retrieval in finite dimensional spaces.
Finally, we show that generically, conjugate phase retrieval can be
accomplished by sampling at three times the Nyquist rate, whereas phase
retrieval requires sampling at four times the Nyquist rate.
Authors' comments: 5 color figures
Phase retrieval with prior information can be cast as a nonsmooth and
nonconvex optimization problem. We solve the problem by graph projection
splitting (GPS), where the two proximity subproblems and the graph projection
step can be solved efficiently. With slight modification, we also propose a
robust graph projection splitting (RGPS) method to stabilize the iteration for
noisy measurements. Contrary to intuition, RGPS outperforms GPS with fewer
iterations to locate a satisfying solution even for noiseless case. Based on
the connection between GPS and Douglas-Rachford iteration, under mild
conditions on the sampling vectors, we analyze the fixed point sets and provide
the local convergence of GPS and RGPS applied to noiseless phase retrieval
without prior information. For noisy case, we provide the error bound of the
reconstruction. Compared to other existing methods, thanks for the splitting
approach, GPS and RGPS can efficiently solve phase retrieval with prior
information regularization for general sampling vectors which are not
necessarily isometric. For Gaussian phase retrieval, compared to existing
gradient flow approaches, numerical results show that GPS and RGPS are much
less sensitive to the initialization. Thus they markedly improve the phase
transition in noiseless case and reconstruction in the presence of noise
respectively. GPS shows sharpest phase transition among existing methods
including RGPS, while it needs more iterations than RGPS when the number of
measurement is large enough. RGPS outperforms GPS in terms of stability for
noisy measurements. When applying RGPS to more general non-Gaussian
measurements with prior information, such as support, sparsity and TV
minimization, RGPS either outperforms state-of-the-art solvers or can be
combined with state-of-the-art solvers to improve their reconstruction quality.
Studying human behaviour through lifelogging has seen an increase in
attention from researchers over the past decade. The opportunities that
lifelogging offers are based on the fact that a lifelog, as a "black box" of
our lives, offers rich contextual information, which has been an Achilles heel
of information discovery. While lifelog data has been put to use in various
contexts, its application to indoor environment scenario remains unexplored. In
this proposal, I plan to design a method that enables us to capture and record
indoor lifelog data of a person's life in order to facilitate healthcare
systems, emergency response, item tracking etc. To this end, we aim to build an
Indoor Information Retrieval system that can be queried with natural language
queries over lifelog data. Judicious use of the lifelog data for the indoor
application may enable us to solve very fundamental but non-avoidable problems
of our daily life. Analysis of lifelog data coupled with Information Retrieval
is not only a promising research topic, but the possibility of its indoor
application especially for healthcare, lost-item tracking would be an
innovative research idea to the best of our knowledge.
Motivated by the promising performance of pre-trained language models, we
investigate BERT in an evidence retrieval and claim verification pipeline for
the FEVER fact extraction and verification challenge. To this end, we propose
to use two BERT models, one for retrieving potential evidence sentences
supporting or rejecting claims, and another for verifying claims based on the
predicted evidence sets. To train the BERT retrieval system, we use pointwise
and pairwise loss functions, and examine the effect of hard negative mining. A
second BERT model is trained to classify the samples as supported, refuted, and
not enough information. Our system achieves a new state of the art recall of
87.1 for retrieving top five sentences out of the FEVER documents consisting of
50K Wikipedia pages, and scores second in the official leaderboard with the
FEVER score of 69.7.
Pattern counting in graphs is a fundamental primitive for many network
analysis tasks, and a number of methods have been developed for scaling
subgraph counting to large graphs. Many real-world networks carry a natural
notion of strength of connection between nodes, which are often modeled by a
weighted graph, but existing scalable graph algorithms for pattern mining are
designed for unweighted graphs. Here, we develop a suite of deterministic and
random sampling algorithms that enable the fast discovery of the 3-cliques
(triangles) with the largest weight in a graph, where weight is measured by a
generalized mean of a triangle's edges. For example, one of our proposed
algorithms can find the top-1000 weighted triangles of a weighted graph with
billions of edges in thirty seconds on a commodity server, which is orders of
magnitude faster than existing "fast" enumeration schemes. Our methods thus
open the door towards scalable pattern mining in weighted graphs.
Most research on dialogue has focused either on dialogue generation for
openended chit chat or on state tracking for goal-directed dialogue. In this
work, we explore a hybrid approach to goal-oriented dialogue generation that
combines retrieval from past history with a hierarchical, neural
encoder-decoder architecture. We evaluate this approach in the customer support
domain using the Multiwoz dataset (Budzianowski et al., 2018). We show that
adding this retrieval step to a hierarchical, neural encoder-decoder
architecture leads to significant improvements, including responses that are
rated more appropriate and fluent by human evaluators. Finally, we compare our
retrieval-based model to various semantically conditioned models explicitly
using past dialog act information, and find that our proposed model is
competitive with the current state of the art (Chen et al., 2019), while not
requiring explicit labels about past machine acts.
