We explore the presence of active galactic nuclei (AGN)/black holes (BH) in
Green Pea galaxies (GPs), motivated by the presence of high ionization emission
lines such as HeII and [NeIII] in their optical spectra. In order to identify
AGN candidates, we used mid-infrared (MIR) photometric observations from the
all-sky Wide-field Infrared Survey Explorer (WISE) mission for a sample of 1004
GPs. Considering only $>5\sigma$ detections with no contamination from
neighboring sources in AllWISE, we select 31 GPs out of 134 as candidate AGN
based on a stringent 3-band WISE color diagnostic. Using multi-epoch photometry
in W1 and W2 bands based on time-resolved unWISE coadd images, we find two
sources exhibiting variability in both the WISE bands among 112 GPs with
W1$\leqslant16$ mag and no contamination from neighboring sources in unWISE.
These two variable sources were selected as AGN by the WISE 3-band color
diagnostic as well. Compared to variable AGN fractions observed among low-mass
galaxy samples in previous studies, we find a higher fraction ($\sim1.8\%$) of
MIR variable sources among GPs, which demonstrates the uniqueness and
importance of studying these extreme objects. Through this work, we demonstrate
that MIR diagnostics are promising tools to select AGN that may be missed by
other selection techniques (including optical emission-line ratios and X-ray
emission) in star-formation dominated, low-mass, low-metallicity galaxies.
Authors' comments: 10 pages, 6 figures; Resubmitted to ApJ
In this short communication, I gave a generalization of measurement postulate
in quantum mechanics. It is regarding the case with partial measurement,
namely, measurement on only part of a wave function. Upon a partial
measurement, the wavefunction will either collapse in one of the eigenstate
covered by the measured partial wavefunction; or collapses out of the measured
part and shifts to the unmeasured part. An explanation in the WISE
(Wavefunction Is System Entity) interpretation is given.
Authors' comments: 2 pages. SCIENCE CHINA Physics, Mechanics & Astronomy, (2021)
Matrix completion has attracted attention in many fields, including
statistics, applied mathematics, and electrical engineering. Most of the works
focus on the independent sampling models under which the observed entries are
sampled independently. Motivated by applications in the integration of
knowledge graphs derived from multi-source biomedical data such as those from
Electronic Health Records (EHR) and biomedical text, we propose the {\bf
B}lock-wise {\bf O}verlapping {\bf N}oisy {\bf M}atrix {\bf I}ntegration
(BONMI) to treat blockwise missingness of symmetric matrices representing
relatedness between entity pairs. Our idea is to exploit the orthogonal
Procrustes problem to align the eigenspace of the two sub-matrices, then
complete the missing blocks by the inner product of the two low-rank
components. Besides, we prove the statistical rate for the eigenspace of the
underlying matrix, which is comparable to the rate under the independently
missing assumption. Simulation studies show that the method performs well under
a variety of configurations. In the real data analysis, the method is applied
to two tasks: (i) the integrating of several point-wise mutual information
matrices built by English EHR and Chinese medical text data, and (ii) the
machine translation between English and Chinese medical concepts. Our method
shows an advantage over existing methods.
Themis family is one of the largest and oldest asteroid populations in the
main-belt. Water-ice may widely exist on the parent body (24) Themis. In this
work, we employ the Advanced Thermophysical Model as well as mid-infrared
measurements from NASA's Wide-Field Infrared Survey Explorer to explore thermal
parameters of 20 Themis family members. Here we show that the average thermal
inertia and geometric albedo are ~$39.5\pm26.0 ~\rm J m^{-2} s^{-1/2} K^{-1}$
and $0.067\pm0.018$, respectively. The family members have a relatively
moderate roughness fraction on their surfaces. We find that the relatively low
albedos of Themis members are consistent with the typical values of B-type and
C-type asteroids. As aforementioned, Themis family bears a very low thermal
inertia, which indicates a fine and mature regolith on their surfaces. The
resemblance of thermal inertia and geometric albedo of Themis members may
reveal their close connection in origin and evolution. In addition, we present
the compared results of thermal parameters for several prominent families.
Authors' comments: 22 pages, 25 figures, accepted for publication in AJ
Contrastive learning (CL) aims to learn useful representation without relying
on expert annotations in the context of medical image segmentation. Existing
approaches mainly contrast a single positive vector (i.e., an augmentation of
the same image) against a set of negatives within the entire remainder of the
batch by simply mapping all input features into the same constant vector.
Despite the impressive empirical performance, those methods have the following
shortcomings: (1) it remains a formidable challenge to prevent the collapsing
problems to trivial solutions; and (2) we argue that not all voxels within the
same image are equally positive since there exist the dissimilar anatomical
structures with the same image. In this work, we present a novel Contrastive
Voxel-wise Representation Learning (CVRL) method to effectively learn low-level
and high-level features by capturing 3D spatial context and rich anatomical
information along both the feature and the batch dimensions. Specifically, we
first introduce a novel CL strategy to ensure feature diversity promotion among
the 3D representation dimensions. We train the framework through bi-level
contrastive optimization (i.e., low-level and high-level) on 3D images.
Experiments on two benchmark datasets and different labeled settings
demonstrate the superiority of our proposed framework. More importantly, we
also prove that our method inherits the benefit of hardness-aware property from
the standard CL approaches.
Reliably identifying active galactic nuclei (AGNs) in dwarf galaxies is key
to understanding black hole demographics at low masses and constraining models
for black hole seed formation. Here we present Chandra X-ray Observatory
observations of eleven dwarf galaxies that were chosen as AGN candidates using
Wide-field Infrared Survey Explorer (WISE) mid-infrared (mid-IR) color-color
selection. Hubble Space Telescope images are also presented for ten of the
galaxies. Based on Sloan Digital Sky Survey spectroscopy, six galaxies in our
sample have optical evidence for hosting AGNs and five are classified as
star-forming. We detect X-ray point sources with luminosities above that
expected from X-ray binaries in the nuclei of five of the six galaxies with
optical evidence of AGNs. However, the X-ray emission from these AGNs is
generally much lower than expected based on AGN scaling relations with infrared
and optical tracers. We do not find compelling evidence for AGNs in the five
optically-selected star-forming galaxies despite having red mid-IR colors. Only
two are detected in X-rays and their properties are consistent with
stellar-mass X-ray binaries. Based on this multiwavelength study, we conclude
that two-color mid-IR AGN diagnostics at the resolution of WISE cannot be used
to reliably select AGNs in optically-star-forming dwarf galaxies. Future
observations in the infrared with the James Webb Space Telescope offer a
promising path forward.
Authors' comments: 16 pages, 8 figures, accepted for publication in ApJ
This paper investigates the model-based methods in multi-agent reinforcement
learning (MARL). We specify the dynamics sample complexity and the opponent
sample complexity in MARL, and conduct a theoretic analysis of return
discrepancy upper bound. To reduce the upper bound with the intention of low
sample complexity during the whole learning process, we propose a novel
decentralized model-based MARL method, named Adaptive Opponent-wise Rollout
Policy Optimization (AORPO). In AORPO, each agent builds its multi-agent
environment model, consisting of a dynamics model and multiple opponent models,
and trains its policy with the adaptive opponent-wise rollout. We further prove
the theoretic convergence of AORPO under reasonable assumptions. Empirical
experiments on competitive and cooperative tasks demonstrate that AORPO can
achieve improved sample efficiency with comparable asymptotic performance over
the compared MARL methods.
Authors' comments: Paper accepted at IJCAI 2021
In this paper, we propose to enhance the pair-wise aspect and opinion terms
extraction (PAOTE) task by incorporating rich syntactic knowledge. We first
build a syntax fusion encoder for encoding syntactic features, including a
label-aware graph convolutional network (LAGCN) for modeling the dependency
edges and labels, as well as the POS tags unifiedly, and a local-attention
module encoding POS tags for better term boundary detection. During pairing, we
then adopt Biaffine and Triaffine scoring for high-order aspect-opinion term
pairing, in the meantime re-harnessing the syntax-enriched representations in
LAGCN for syntactic-aware scoring. Experimental results on four benchmark
datasets demonstrate that our model outperforms current state-of-the-art
baselines, meanwhile yielding explainable predictions with syntactic knowledge.
Authors' comments: IJCAI2021
For fixed training data and network parameters in the other layers the L1
loss of a ReLU neural network as a function of the first layer's parameters is
a piece-wise affine function. We use the Deep ReLU Simplex algorithm to
iteratively minimize the loss monotonically on adjacent vertices and analyze
the trajectory of these vertex positions. We empirically observe that in a
neighbourhood around a local minimum, the iterations behave differently such
that conclusions on loss level and proximity of the local minimum can be made
before it has been found: Firstly the loss seems to decay exponentially slow at
iterated adjacent vertices such that the loss level at the local minimum can be
estimated from the loss levels of subsequently iterated vertices, and secondly
we observe a strong increase of the vertex density around local minima. This
could have far-reaching consequences for the design of new gradient-descent
algorithms that might improve convergence rate by exploiting these facts.
Authors' comments: 4 pages, 5 figures
Standard losses for training deep segmentation networks could be seen as
individual classifications of pixels, instead of supervising the global shape
of the predicted segmentations. While effective, they require exact knowledge
of the label of each pixel in an image.
This study investigates how effective global geometric shape descriptors
could be, when used on their own as segmentation losses for training deep
networks. Not only interesting theoretically, there exist deeper motivations to
posing segmentation problems as a reconstruction of shape descriptors:
Annotations to obtain approximations of low-order shape moments could be much
less cumbersome than their full-mask counterparts, and anatomical priors could
be readily encoded into invariant shape descriptions, which might alleviate the
annotation burden. Also, and most importantly, we hypothesize that, given a
task, certain shape descriptions might be invariant across image acquisition
protocols/modalities and subject populations, which might open interesting
research avenues for generalization in medical image segmentation.
We introduce and formulate a few shape descriptors in the context of deep
segmentation, and evaluate their potential as standalone losses on two
different challenging tasks. Inspired by recent works in constrained
optimization for deep networks, we propose a way to use those descriptors to
supervise segmentation, without any pixel-level label. Very surprisingly, as
little as 4 descriptors values per class can approach the performance of a
segmentation mask with 65k individual discrete labels. We also found that shape
descriptors can be a valid way to encode anatomical priors about the task,
enabling to leverage expert knowledge without additional annotations. Our
implementation is publicly available and can be easily extended to other tasks
and descriptors: https://github.com/hkervadec/shape_descriptors
Authors' comments: Accepted at Medical Imaging with Deep Learning (MIDL) 2021
This work presents a novel approach for semi-supervised semantic
segmentation. The key element of this approach is our contrastive learning
module that enforces the segmentation network to yield similar pixel-level
feature representations for same-class samples across the whole dataset. To
achieve this, we maintain a memory bank continuously updated with relevant and
high-quality feature vectors from labeled data. In an end-to-end training, the
features from both labeled and unlabeled data are optimized to be similar to
same-class samples from the memory bank. Our approach outperforms the current
state-of-the-art for semi-supervised semantic segmentation and semi-supervised
domain adaptation on well-known public benchmarks, with larger improvements on
the most challenging scenarios, i.e., less available labeled data.
https://github.com/Shathe/SemiSeg-Contrastive
One of the main challenges since the advancement of convolutional neural
networks is how to connect the extracted feature map to the final
classification layer. VGG models used two sets of fully connected layers for
the classification part of their architectures, which significantly increased
the number of models' weights. ResNet and the next deep convolutional models
used the Global Average Pooling (GAP) layer to compress the feature map and
feed it to the classification layer. Although using the GAP layer reduces the
computational cost, but also causes losing spatial resolution of the feature
map, which results in decreasing learning efficiency. In this paper, we aim to
tackle this problem by replacing the GAP layer with a new architecture called
Wise-SrNet. It is inspired by the depthwise convolutional idea and is designed
for processing spatial resolution while not increasing computational cost. We
have evaluated our method using three different datasets: Intel Image
Classification Challenge, MIT Indoors Scenes, and a part of the ImageNet
dataset. We investigated the implementation of our architecture on several
models of the Inception, ResNet, and DenseNet families. Applying our
architecture has revealed a significant effect on increasing convergence speed
and accuracy. Our Experiments on images with 224*224 resolution increased the
Top-1 accuracy between 2% to 8% on different datasets and models. Running our
models on 512*512 resolution images of the MIT Indoors Scenes dataset showed a
notable result of improving the Top-1 accuracy within 3% to 26%. We will also
demonstrate the GAP layer's disadvantage when the input images are large and
the number of classes is not few. In this circumstance, our proposed
architecture can do a great help in enhancing classification results. The code
is shared at https://github.com/mr7495/image-classification-spatial.
Authors' comments: The code is shared at
https://github.com/mr7495/image-classification-spatial
In this paper, we propose a novel DNN watermarking method that utilizes a
learnable image transformation method with a secret key. The proposed method
embeds a watermark pattern in a model by using learnable transformed images and
allows us to remotely verify the ownership of the model. As a result, it is
piracy-resistant, so the original watermark cannot be overwritten by a pirated
watermark, and adding a new watermark decreases the model accuracy unlike most
of the existing DNN watermarking methods. In addition, it does not require a
special pre-defined training set or trigger set. We empirically evaluated the
proposed method on the CIFAR-10 dataset. The results show that it was resilient
against fine-tuning and pruning attacks while maintaining a high
watermark-detection accuracy.
International Classification of Disease (ICD) coding procedure which refers
to tagging medical notes with diagnosis codes has been shown to be effective
and crucial to the billing system in medical sector. Currently, ICD codes are
assigned to a clinical note manually which is likely to cause many errors.
Moreover, training skilled coders also requires time and human resources.
Therefore, automating the ICD code determination process is an important task.
With the advancement of artificial intelligence theory and computational
hardware, machine learning approach has emerged as a suitable solution to
automate this process. In this project, we apply a transformer-based
architecture to capture the interdependence among the tokens of a document and
then use a code-wise attention mechanism to learn code-specific representations
of the entire document. Finally, they are fed to separate dense layers for
corresponding code prediction. Furthermore, to handle the imbalance in the code
frequency of clinical datasets, we employ a label distribution aware margin
(LDAM) loss function. The experimental results on the MIMIC-III dataset show
that our proposed model outperforms other baselines by a significant margin. In
particular, our best setting achieves a micro-AUC score of 0.923 compared to
0.868 of bidirectional recurrent neural networks. We also show that by using
the code-wise attention mechanism, the model can provide more insights about
its prediction, and thus it can support clinicians to make reliable decisions.
Our code is available online (https://github.com/biplob1ly/TransICD)
Authors' comments: 10 pages, 4 figures
Domain mismatch is a noteworthy issue in acoustic event detection tasks, as
the target domain data is difficult to access in most real applications. In
this study, we propose a novel CNN-based discriminative training framework as a
domain compensation method to handle this issue. It uses a parallel CNN-based
discriminator to learn a pair of high-level intermediate acoustic
representations. Together with a binary discriminative loss, the discriminators
are forced to maximally exploit the discrimination of heterogeneous acoustic
information in each audio clip with target events, which results in a robust
paired representations that can well discriminate the target events and
background/domain variations separately. Moreover, to better learn the
transient characteristics of target events, a frame-wise classifier is designed
to perform the final classification. In addition, a two-stage training with the
CNN-based discriminator initialization is further proposed to enhance the
system training. All experiments are performed on the DCASE 2018 Task3
datasets. Results show that our proposal significantly outperforms the official
baseline on cross-domain conditions in AUC by relative $1.8-12.1$% without any
performance degradation on in-domain evaluation conditions.
Drop-wise condensation (DWC) has been the focus of scientific research in
vapor condensation technologies since the 20th century. Improvement of
condensation rate in DWC is limited by the maximum droplet a condensation
surface could sustain. Furthermore, the presence of non-condensable gases (NCG)
reduces the condensation rate significantly. Here, we present continuous
drop-wise condensation (CDC) to overcome the need of hydrophobic surfaces while
yet maintaining micron-sized droplets. By shifting focus from surface treatment
to the force required to sweep off a droplet, we were able to utilize
stagnation pressure of jet impingement to tune the shed droplet size. The
results show that droplet size being shed can be tuned effectively by tuning
the jet parameters. our experimental observations showed that the effect of NCG
is greatly alleviated by utilizing our technique. An improvement by at least
six folds in mass transfer compactness factor compared to state-of-the-art
dehumidification technology was possible.
Authors' comments: Videos are available upon request
A myriad of recent breakthroughs in hand-crafted neural architectures for
visual recognition have highlighted the urgent need to explore hybrid
architectures consisting of diversified building blocks. Meanwhile, neural
architecture search methods are surging with an expectation to reduce human
efforts. However, whether NAS methods can efficiently and effectively handle
diversified search spaces with disparate candidates (e.g. CNNs and
transformers) is still an open question. In this work, we present Block-wisely
Self-supervised Neural Architecture Search (BossNAS), an unsupervised NAS
method that addresses the problem of inaccurate architecture rating caused by
large weight-sharing space and biased supervision in previous methods. More
specifically, we factorize the search space into blocks and utilize a novel
self-supervised training scheme, named ensemble bootstrapping, to train each
block separately before searching them as a whole towards the population
center. Additionally, we present HyTra search space, a fabric-like hybrid
CNN-transformer search space with searchable down-sampling positions. On this
challenging search space, our searched model, BossNet-T, achieves up to 82.5%
accuracy on ImageNet, surpassing EfficientNet by 2.4% with comparable compute
time. Moreover, our method achieves superior architecture rating accuracy with
0.78 and 0.76 Spearman correlation on the canonical MBConv search space with
ImageNet and on NATS-Bench size search space with CIFAR-100, respectively,
surpassing state-of-the-art NAS methods. Code:
https://github.com/changlin31/BossNAS
Authors' comments: Accepted to ICCV 2021
Safety is one of the fundamental problems in robotics. Recently, a quadratic
program-based control barrier function (CBF) method has emerged as a way to
enforce safety-critical constraints. Together with control Lyapunov function
(CLF), it forms a safety-critical control strategy, named CLF-CBF-QP, which can
mediate between achieving the control objective and ensuring safety, while
being executable in real-time. However, once additional constraints such as
input constraints are introduced, the CLF-CBF-QP may encounter infeasibility.
In order to address the challenge that arises due to the infeasibility, we
propose an optimal-decay form for safety-critical control wherein the decay
rate of the CBF is optimized point-wise in time so as to guarantee point-wise
feasibility when the state lies inside the safe set. The proposed control
design is numerically validated using an adaptive cruise control example.
Glacier calving front position (CFP) is an important glaciological variable.
Traditionally, delineating the CFPs has been carried out manually, which was
subjective, tedious and expensive. Automating this process is crucial for
continuously monitoring the evolution and status of glaciers. Recently, deep
learning approaches have been investigated for this application. However, the
current methods get challenged by a severe class-imbalance problem. In this
work, we propose to mitigate the class-imbalance between the calving front
class and the non-calving front class by reformulating the segmentation problem
into a pixel-wise regression task. A Convolutional Neural Network gets
optimized to predict the distance values to the glacier front for each pixel in
the image. The resulting distance map localizes the CFP and is further
post-processed to extract the calving front line. We propose three
post-processing methods, one method based on statistical thresholding, a second
method based on conditional random fields (CRF), and finally the use of a
second U-Net. The experimental results confirm that our approach significantly
outperforms the state-of-the-art methods and produces accurate delineation. The
Second U-Net obtains the best performance results, resulting in an average
improvement of about 21% dice coefficient enhancement.
Randomized smoothing (RS) is an effective and scalable technique for
constructing neural network classifiers that are certifiably robust to
adversarial perturbations. Most RS works focus on training a good base model
that boosts the certified robustness of the smoothed model. However, existing
RS techniques treat every data point the same, i.e., the variance of the
Gaussian noise used to form the smoothed model is preset and universal for all
training and test data. This preset and universal Gaussian noise variance is
suboptimal since different data points have different margins and the local
properties of the base model vary across the input examples. In this paper, we
examine the impact of customized handling of examples and propose Instance-wise
Randomized Smoothing (Insta-RS) -- a multiple-start search algorithm that
assigns customized Gaussian variances to test examples. We also design Insta-RS
Train -- a novel two-stage training algorithm that adaptively adjusts and
customizes the noise level of each training example for training a base model
that boosts the certified robustness of the instance-wise Gaussian smoothed
model. Through extensive experiments on CIFAR-10 and ImageNet, we show that our
method significantly enhances the average certified radius (ACR) as well as the
clean data accuracy compared to existing state-of-the-art provably robust
classifiers.
Authors' comments: We plan to make major modifications to this paper including rewriting
the entire text, rewriting the proofs and adding experiments. Given that the
paper will be completely different, we decided to take this paper down
temporarily
