Automatic source detection and classification tools based on machine learning
(ML) algorithms are growing in popularity due to their efficiency when dealing
with large amounts of data simultaneously and their ability to work in
multidimensional parameter spaces. In this work, we present a new, automated
method of outlier selection based on support vector machine (SVM) algorithm
called one-class SVM (OCSVM), which uses the training data as one class to
construct a model of 'normality' in order to recognize novel points. We test
the performance of OCSVM algorithm on \textit{Wide-field Infrared Survey
Explorer (WISE)} data trained on the Sloan Digital Sky Survey (SDSS) sources.
Among others, we find $\sim 40,000$ sources with abnormal patterns which can be
associated with obscured and unobscured active galactic nuclei (AGN) source
candidates. We present the preliminary estimation of the clustering properties
of these objects and find that the unobscured AGN candidates are preferentially
found in less massive dark matter haloes ($M_{DMH}\sim10^{12.4}$) than the
obscured candidates ($M_{DMH}\sim 10^{13.2}$). This result contradicts the
unification theory of AGN sources and indicates that the obscured and
unobscured phases of AGN activity take place in different evolutionary paths
defined by different environments.
Authors' comments: 4 figures, 6 pages
Delineation of curvilinear structures is an important problem in Computer
Vision with multiple practical applications. With the advent of Deep Learning,
many current approaches on automatic delineation have focused on finding more
powerful deep architectures, but have continued using the habitual pixel-wise
losses such as binary cross-entropy. In this paper we claim that pixel-wise
losses alone are unsuitable for this problem because of their inability to
reflect the topological impact of mistakes in the final prediction. We propose
a new loss term that is aware of the higher-order topological features of
linear structures. We also introduce a refinement pipeline that iteratively
applies the same model over the previous delineation to refine the predictions
at each step while keeping the number of parameters and the complexity of the
model constant.
When combined with the standard pixel-wise loss, both our new loss term and
our iterative refinement boost the quality of the predicted delineations, in
some cases almost doubling the accuracy as compared to the same classifier
trained with the binary cross-entropy alone. We show that our approach
outperforms state-of-the-art methods on a wide range of data, from microscopy
to aerial images.
Information Bottleneck (IB) is a generalization of rate-distortion theory
that naturally incorporates compression and relevance trade-offs for learning.
Though the original IB has been extensively studied, there has not been much
understanding of multiple bottlenecks which better fit in the context of neural
networks. In this work, we propose Information Multi-Bottlenecks (IMBs) as an
extension of IB to multiple bottlenecks which has a direct application to
training neural networks by considering layers as multiple bottlenecks and
weights as parameterized encoders and decoders. We show that the multiple
optimality of IMB is not simultaneously achievable for stochastic encoders. We
thus propose a simple compromised scheme of IMB which in turn generalizes
maximum likelihood estimate (MLE) principle in the context of stochastic neural
networks. We demonstrate the effectiveness of IMB on classification tasks and
adversarial robustness in MNIST and CIFAR10.
Authors' comments: published in Entropy journal
Sparse inverse covariance selection is a fundamental problem for analyzing
dependencies in high dimensional data. However, such a problem is difficult to
solve since it is NP-hard. Existing solutions are primarily based on convex
approximation and iterative hard thresholding, which only lead to sub-optimal
solutions. In this work, we propose a coordinate-wise optimization algorithm to
solve this problem which is guaranteed to converge to a coordinate-wise minimum
point. The algorithm iteratively and greedily selects one variable or swaps two
variables to identify the support set, and then solves a reduced convex
optimization problem over the support set to achieve the greatest descent. As a
side contribution of this paper, we propose a Newton-like algorithm to solve
the reduced convex sub-problem, which is proven to always converge to the
optimal solution with global linear convergence rate and local quadratic
convergence rate. Finally, we demonstrate the efficacy of our method on
synthetic data and real-world data sets. As a result, the proposed method
consistently outperforms existing solutions in terms of accuracy.
We present a new paradigm for the simulation of arrays of Imaging Atmospheric
Cherenkov Telescopes (IACTs) which overcomes limitations of current approaches.
Up to now, all major IACT experiments rely on the same Monte-Carlo simulation
strategy, using predefined observation and instrument settings. Simulations
with varying parameters are generated to provide better estimates of the
Instrument Response Functions (IRFs) of different observations. However, a
large fraction of the simulation configuration remains preserved, leading to
complete negligence of all related influences. Additionally, the simulation
scheme relies on interpolations between different array configurations, which
are never fully reproducing the actual configuration for a given observation.
Interpolations are usually performed on zenith angles, off-axis angles, array
multiplicity, and the optical response of the instrument. With the advent of
hybrid systems consisting of a large number of IACTs with different sizes,
types, and camera configurations, the complexity of the interpolation and the
size of the phase space becomes increasingly prohibitive. Going beyond the
existing approaches, we introduce a new simulation and analysis concept which
takes into account the actual observation conditions as well as individual
telescope configurations of each observation run of a given data set. These
run-wise simulations (RWS) thus exhibit considerably reduced systematic
uncertainties compared to the existing approach, and are also more
computationally efficient and simple. The RWS framework has been implemented in
the H.E.S.S. software and tested, and is already being exploited in science
analysis.
Authors' comments: Proceedings of the 35th International Cosmic Ray Conference; Busan,
Korea
This paper uses the multi-epoch astrometry from the Wide-field Infrared
Survey Explorer (WISE) to demonstrate a method to measure proper motions and
trigonometric parallaxes with precisions of $\sim$4 mas yr$^{-1}$ and $\sim$7
mas, respectively, for low-mass stars and brown dwarfs. This method relies on
WISE single exposures (Level 1b frames) and a Markov Chain Monte Carlo method.
The limitations of Gaia in observing very low-mass stars and brown dwarfs are
discussed, and it is shown that WISE will be able to measure astrometry past
the 95% completeness limit and magnitude limit of Gaia (L, T, and Y dwarfs
fainter than $G\approx19$ and $G=21$, respectively). This method is applied to
WISE data of 20 nearby ($\lesssim17$ pc) dwarfs with spectral types between
M6-Y2 and previously measured trigonometric parallaxes. Also provided are WISE
astrometric measurements for 23 additional low-mass dwarfs with spectral types
between M6-T7 and estimated photometric distances $<17$ pc. Only nine of these
objects contain parallaxes within Gaia Data Release 2.
Authors' comments: Accepted to ApJ
SD-WISE is a complete software-defined solution for wireless sensor (and
actuator) networks (WSNs). SD-WISE has several unique features making it a
flexible and expandable solution that can be applied in heterogeneous
application domains. Its fundamental feature is that it provides software
abstractions of the nodes' resources on both the controller and the nodes
sides. By leveraging these abstractions, SD-WISE (i) extends the Software
Defined Networking (SDN) approach to WSNs, introducing a more flexible way to
define flows as well as the possibility to control the duty cycles of the node
radios to increase energy efficiency; (ii) enables network function
virtualization (NFV) in WSNs; (iii) leverages the tight interplay between
trusted hardware and software to support regulation compliant behavior of
sensor nodes. In this paper SD-WISE is introduced, its major operations are
described, and its features are demonstrated in three relevant scenarios, thus
assessing the effectiveness of the approach.
We present accurate resolved $WISE$ photometry of galaxies in the combined
SINGS and KINGFISH sample. The luminosities in the W3 12$\mu$m and W4 23$\mu$m
bands are calibrated to star formation rates (SFRs) derived using the total
infrared luminosity, avoiding UV/optical uncertainties due to dust extinction
corrections. The W3 relation has a 1-$\sigma$ scatter of 0.15 dex over nearly 5
orders of magnitude in SFR and 12$\mu$m luminosity, and a range in host stellar
mass from dwarf (10$^7$ M$_\odot$) to $\sim3\times$M$_\star$ (10$^{11.5}$
M$_\odot$) galaxies. In the absence of deep silicate absorption features and
powerful active galactic nuclei, we expect this to be a reliable SFR indicator
chiefly due to the broad nature of the W3 band. By contrast the W4 SFR relation
shows more scatter (1-$\sigma =$ 0.18 dex). Both relations show reasonable
agreement with radio continuum-derived SFRs and excellent accordance with
so-called "hybrid" H$\alpha + 24 \mu$m and FUV$+$24$\mu$m indicators. Moreover,
the $WISE$ SFR relations appear to be insensitive to the metallicity range in
the sample. We also compare our results with IRAS-selected luminous infrared
galaxies, showing that the $WISE$ relations maintain concordance, but
systematically deviate for the most extreme galaxies. Given the all-sky
coverage of $WISE$ and the performance of the W3 band as a SFR indicator, the
$L_{12\mu \rm m}$ SFR relation could be of great use to studies of nearby
galaxies and forthcoming large area surveys at optical and radio wavelengths.
Authors' comments: 34 pages, 6 tables include complete WISE photometry of the combined
SINGS+KINGFISH sample. Accepted for publication in The Astrophysical Journal
We show that mid-infrared data from the all-sky WISE survey can be used as a
robust photometric redshift indicator for powerful radio AGN, in the absence of
other spectroscopic or multi-band photometric information. Our work is
motivated by a desire to extend the well-known K-z relation for radio galaxies
to the wavelength range covered by the all-sky WISE mid-infrared survey. Using
the LARGESS radio spectroscopic sample as a training set, and the mid-infrared
colour information to classify radio sources, we generate a set of redshift
probability distributions for the hosts of high-excitation and low-excitation
radio AGN. We test the method using spectroscopic data from several other radio
AGN studies, and find good agreement between our WISE-based redshift estimates
and published spectroscopic redshifts out to z ~ 1 for galaxies and z ~ 3-4 for
radio-loud QSOs. Our chosen method is also compared against other
classification methods and found to perform reliably. This technique is likely
to be particularly useful in the analysis of upcoming large-area radio surveys
with SKA pathfinder telescopes, and our code is publicly available. As a
consistency check, we show that our WISE-based redshift estimates for sources
in the 843 MHz SUMSS survey reproduce the redshift distribution seen in the
CENSORS study up to z ~ 2. We also discuss two specific applications of our
technique for current and upcoming radio surveys; an interpretation of large
scale HI absorption surveys, and a determination of whether low-frequency
peaked spectrum sources lie at high redshift.
Authors' comments: 18 pages, 11 figures, 11 tables; submitted to MNRAS
Training deep neural networks is known to require a large number of training
samples. However, in many applications only few training samples are available.
In this work, we tackle the issue of training neural networks for
classification task when few training samples are available. We attempt to
solve this issue by proposing a new regularization term that constrains the
hidden layers of a network to learn class-wise invariant representations. In
our regularization framework, learning invariant representations is generalized
to the class membership where samples with the same class should have the same
representation. Numerical experiments over MNIST and its variants showed that
our proposal helps improving the generalization of neural network particularly
when trained with few samples. We provide the source code of our framework
https://github.com/sbelharbi/learning-class-invariant-features .
Authors' comments: Submitted to ELSEVIER, 13 pages, 5 figures
Contexts play an important role in the saliency detection task. However,
given a context region, not all contextual information is helpful for the final
task. In this paper, we propose a novel pixel-wise contextual attention
network, i.e., the PiCANet, to learn to selectively attend to informative
context locations for each pixel. Specifically, for each pixel, it can generate
an attention map in which each attention weight corresponds to the contextual
relevance at each context location. An attended contextual feature can then be
constructed by selectively aggregating the contextual information. We formulate
the proposed PiCANet in both global and local forms to attend to global and
local contexts, respectively. Both models are fully differentiable and can be
embedded into CNNs for joint training. We also incorporate the proposed models
with the U-Net architecture to detect salient objects. Extensive experiments
show that the proposed PiCANets can consistently improve saliency detection
performance. The global and local PiCANets facilitate learning global contrast
and homogeneousness, respectively. As a result, our saliency model can detect
salient objects more accurately and uniformly, thus performing favorably
against the state-of-the-art methods.
We introduce an accurate lung segmentation model for chest radiographs based
on deep convolutional neural networks. Our model is based on atrous
convolutional layers to increase the field-of-view of filters efficiently. To
improve segmentation performances further, we also propose a multi-stage
training strategy, network-wise training, which the current stage network is
fed with both input images and the outputs from pre-stage network. It is shown
that this strategy has an ability to reduce falsely predicted labels and
produce smooth boundaries of lung fields. We evaluate the proposed model on a
common benchmark dataset, JSRT, and achieve the state-of-the-art segmentation
performances with much fewer model parameters.
Authors' comments: Accepted to the 3rd Workshop on Deep Learning in Medical Image
Analysis (DLMIA 2017), MICCAI 2017
We probe the isotropy of the Universe with the largest all-sky photometric
redshift dataset currently available, namely WISE~$\times$~SuperCOSMOS. We
search for dipole anisotropy of galaxy number counts in multiple redshift
shells within the $0.10 < z < 0.35$ range, for two subsamples drawn from the
same parent catalogue. Our results show that the dipole directions are in good
agreement with most of the previous analyses in the literature, and in most
redshift bins the dipole amplitudes are well consistent with $\Lambda$CDM-based
mocks in the cleanest sample of this catalogue. In the $z<0.15$ range, however,
we obtain a persistently large anisotropy in both subsamples of our dataset.
Overall, we report no significant evidence against the isotropy assumption in
this catalogue except for the lowest redshift ranges. The origin of the latter
discrepancy is unclear, and improved data may be needed to explain it.
Authors' comments: 5 pages, 4 figures, 2 tables. Published in MNRAS
List-wise learning to rank methods are considered to be the state-of-the-art.
One of the major problems with these methods is that the ambiguous nature of
relevance labels in learning to rank data is ignored. Ambiguity of relevance
labels refers to the phenomenon that multiple documents may be assigned the
same relevance label for a given query, so that no preference order should be
learned for those documents. In this paper we propose a novel sampling
technique for computing a list-wise loss that can take into account this
ambiguity. We show the effectiveness of the proposed method by training a
3-layer deep neural network. We compare our new loss function to two strong
baselines: ListNet and ListMLE. We show that our method generalizes better and
significantly outperforms other methods on the validation and test sets.
Delensing, the removal of the limiting lensing B-mode background, is crucial
for the success of future cosmic microwave background (CMB) surveys in
constraining inflationary gravitational waves (IGWs). In recent work, delensing
with large-scale structure tracers has emerged as a promising method both for
improving constraints on IGWs and for testing delensing methods for future use.
However, the delensing fractions (i.e., the fraction of the lensing-B mode
power removed) achieved by recent efforts have been only $20-30\%$. In this
work, we provide a detailed characterization of a full-sky, dust-cleaned cosmic
infrared background (CIB) map for delensing and construct a further-improved
delensing template by adding additional tracers to increase delensing
performance. In particular, we build a multitracer delensing template by
combining the dust-cleaned Planck CIB map with a reconstructed CMB lensing map
from Planck and a galaxy number density map from the Wide-field Infrared Survey
Explorer (WISE) satellite. For this combination, we calculate the relevant
weightings by fitting smooth templates to measurements of all the cross- and
auto-spectra of these maps. On a large fraction of the sky
($f_\mathrm{sky}=0.43$), we demonstrate that our maps are capable of providing
a delensing factor of $43 \pm 1\%$; using a more restrictive mask
($f_\mathrm{sky}=0.11$), the delensing factor reaches $48 \pm 1\%$. For
low-noise surveys, our delensing maps, which cover much of the sky, can thus
improve constraints on the tensor-to-scalar ratio ($r$) by nearly a factor of
2. The delensing tracer maps are made publicly available, and we encourage
their use in ongoing and upcoming B-mode surveys.
Authors' comments: 10 pages, 7 figures, data products available at
http://www.sns.ias.edu/~jch/delens/
We present an approach for the verification of feed-forward neural networks
in which all nodes have a piece-wise linear activation function. Such networks
are often used in deep learning and have been shown to be hard to verify for
modern satisfiability modulo theory (SMT) and integer linear programming (ILP)
solvers.
The starting point of our approach is the addition of a global linear
approximation of the overall network behavior to the verification problem that
helps with SMT-like reasoning over the network behavior. We present a
specialized verification algorithm that employs this approximation in a search
process in which it infers additional node phases for the non-linear nodes in
the network from partial node phase assignments, similar to unit propagation in
classical SAT solving. We also show how to infer additional conflict clauses
and safe node fixtures from the results of the analysis steps performed during
the search. The resulting approach is evaluated on collision avoidance and
handwritten digit recognition case studies.
The $L_1$-regularized models are widely used for sparse regression or
classification tasks. In this paper, we propose the orthant-wise passive
descent algorithm (OPDA) for optimizing $L_1$-regularized models, as an
improved substitute of proximal algorithms, which are the standard tools for
optimizing the models nowadays. OPDA uses a stochastic variance-reduced
gradient (SVRG) to initialize the descent direction, then apply a novel
alignment operator to encourage each element keeping the same sign after one
iteration of update, so the parameter remains in the same orthant as before. It
also explicitly suppresses the magnitude of each element to impose sparsity.
The quasi-Newton update can be utilized to incorporate curvature information
and accelerate the speed. We prove a linear convergence rate for OPDA on
general smooth and strongly-convex loss functions. By conducting experiments on
$L_1$-regularized logistic regression and convolutional neural networks, we
show that OPDA outperforms state-of-the-art stochastic proximal algorithms,
implying a wide range of applications in training sparse models.
Authors' comments: Accepted to The Thirty-Second AAAI Conference on Artificial
Intelligence (AAAI-18). Feb 2018, New Orleans
Hamiltonian systems such as the gravitational N-body problem have
time-reversal symmetry. However, all numerical N-body integration schemes,
including symplectic ones, respect this property only approximately. In this
paper, we present the new N-body integrator JANUS, for which we achieve exact
time-reversal symmetry by combining integer and floating point arithmetic.
JANUS is explicit, formally symplectic and satisfies Liouville's theorem
exactly. Its order is even and can be adjusted between two and ten. We discuss
the implementation ofJANUS and present tests of its accuracy and speed by
performing and analyzing long-term integrations of the Solar System. We show
that JANUS is fast and accurate enough to tackle a broad class of dynamical
problems. We also discuss the practical and philosophical implications of
running exactly time-reversible simulations.
Authors' comments: Accepted for publication by MNRAS, 7 pages, 4 figures, source code
available at https://github.com/hannorein/rebound , iPython notebooks to
reproduce figures available at https://github.com/hannorein/JanusPaper
Deep learning has shown promising results in many machine learning
applications. The hierarchical feature representation built by deep networks
enable compact and precise encoding of the data. A kernel analysis of the
trained deep networks demonstrated that with deeper layers, more simple and
more accurate data representations are obtained. In this paper, we propose an
approach for layer-wise training of a deep network for the supervised
classification task. A transformation matrix of each layer is obtained by
solving an optimization aimed at a better representation where a subsequent
layer builds its representation on the top of the features produced by a
previous layer. We compared the performance of our approach with a DNN trained
using back-propagation which has same architecture as ours. Experimental
results on the real image datasets demonstrate efficacy of our approach. We
also performed kernel analysis of layer representations to validate the claim
of better feature encoding.
Mid-IR colour selection techniques have proved to be very efficient in
finding AGN. This is because the AGN heats the surrounding dust producing warm
mid-IR colours. Using the WISE 3.6, 4.5 and 12 $\mu m$ colours, the largest
sample of IR selected AGN has already been produced containing 1.4 million AGN
over the whole sky. Here, we explore the X-ray properties of this AGN sample by
cross-correlating it with the subsample of the 3XMM X-ray catalogue that has
available X-ray spectra and at the same time optical spectroscopy from SDSS.
Our goal is to find rare luminous obscured AGN. Our final sample contains 65
QSOs with $\rm{log}\,\nu L_\nu \ge 46.2$\,erg\,s$^{-1}$. This IR luminosity cut
corresponds to $\rm{log}\,L_X \approx 45$\,erg\,s$^{-1}$, at the median
redshift of our sample ($z=2.3$), that lies at the bright end of the X-ray
luminosity function at $z>2$. The X-ray spectroscopic analysis reveals seven
obscured AGN having a column density $\rm N_H>10^{22} cm^{-2}$. Six of them
show evidence for broad [CIV] absorption lines and five are classified as
BALQSOs. We fit the optical spectra of our X-ray absorbed sources to estimate
the optical reddening. We find that none of these show any obscuration
according to the optical continuum. These sources add to the growing evidence
for populations of luminous QSOs with evidence for substantial absorption by
outflowing ionised material, similar to those expected to be emerging from
their absorbing cocoons in the framework of AGN/galaxy co-evolution.
Authors' comments: 10 pages, 5 figures, 3 Tables, MNRAS accepted
