Parameter Space Noise for Exploration

Deep reinforcement learning (RL) methods generally engage in exploratory behavior through noise injection in the action space. An alternative is to add noise directly to the agent’s parameters, which can lead to more consistent exploration and a richer set of behaviors. Methods such as evolutionary strategies use parameter perturbations, but discard all temporal structure in the process and require significantly more samples. Combining parameter noise with traditional RL methods allows to combine the best of both worlds. We demonstrate that both off- and on-policy methods benefit from this approach through experimental comparison of DQN, DDPG, and TRPO on high-dimensional discrete action environments as well as continuous control tasks. Our results show that RL with parameter noise learns more efficiently than traditional RL with action space noise and evolutionary strategies individually.

Causality network retrieval from short time series

We investigate how efficiently a known underlying causality structure of a simulated multivariate process can be retrieved from the analysis of time-series. Causality is quantified from conditional transfer entropy and the network is constructed by retaining only the statistically validated contributions. We compare results from three methodologies: two commonly used regularization methods, Glasso and ridge, and a newly introduced technique, LoGo, based on the combination of information filtering network and graphical modelling. For these three methodologies we explore the regions of time series lengths and model-parameters where a significant fraction of true causality links is retrieved. We conclude that, when time-series are short, with length of the time series shorter than the number of variables, sparse models are better suited to uncover true causality links with LoGo retrieving the true causality network more accurately than Glasso and ridge.

Synergistic Union of Word2Vec and Lexicon for Domain Specific Semantic Similarity

Semantic similarity measures are an important part in Natural Language Processing tasks. However Semantic similarity measures built for general use do not perform well within specific domains. Therefore in this study we introduce a domain specific semantic similarity measure that was created by the synergistic union of word2vec, a word embedding method that is used for semantic similarity calculation and lexicon based (lexical) semantic similarity methods. We prove that this proposed methodology out performs word embedding methods trained on generic corpus and methods trained on domain specific corpus but do not use lexical semantic similarity methods to augment the results. Further, we prove that text lemmatization can improve the performance of word embedding methods.

Deep Convolutional Decision Jungle for Image Classification

We propose a novel method called deep convolutional decision jungle (CDJ) and its learning algorithm for image classification. The CDJ maintains the structure of standard convolutional neural networks (CNNs), i.e. multiple layers of multiple response maps fully connected. Each response map-or node-in both the convolutional and fully-connected layers selectively respond to class labels s.t. each data sample travels via a specific soft route of those activated nodes. The proposed method CDJ automatically learns features, whereas decision forests and jungles require pre-defined feature sets. Compared to CNNs, the method embeds the benefits of using data-dependent discriminative functions, which better handles multi-modal/heterogeneous data; further,the method offers more diverse sparse network responses, which in turn can be used for cost-effective learning/classification. The network is learnt by combining conventional softmax and proposed entropy losses in each layer. The entropy loss,as used in decision tree growing, measures the purity of data activation according to the class label distribution. The back-propagation rule for the proposed loss function is derived from stochastic gradient descent (SGD) optimization of CNNs. We show that our proposed method outperforms state-of-the-art methods on three public image classification benchmarks and one face verification dataset. We also demonstrate the use of auxiliary data labels, when available, which helps our method to learn more discriminative routing and representations and leads to improved classification.

Network Sketching: Exploiting Binary Structure in Deep CNNs

Convolutional neural networks (CNNs) with deep architectures have substantially advanced the state-of-the-art in computer vision tasks. However, deep networks are typically resource-intensive and thus difficult to be deployed on mobile devices. Recently, CNNs with binary weights have shown compelling efficiency to the community, whereas the accuracy of such models is usually unsatisfactory in practice. In this paper, we introduce network sketching as a novel technique of pursuing binary-weight CNNs, targeting at more faithful inference and better trade-off for practical applications. Our basic idea is to exploit binary structure directly in pre-trained filter banks and produce binary-weight models via tensor expansion. The whole process can be treated as a coarse-to-fine model approximation, akin to the pencil drawing steps of outlining and shading. To further speedup the generated models, namely the sketches, we also propose an associative implementation of binary tensor convolutions. Experimental results demonstrate that a proper sketch of AlexNet (or ResNet) outperforms the existing binary-weight models by large margins on the ImageNet large scale classification task, while the committed memory for network parameters only exceeds a little.

Imposing Hard Constraints on Deep Networks: Promises and Limitations

Imposing constraints on the output of a Deep Neural Net is one way to improve the quality of its predictions while loosening the requirements for labeled training data. Such constraints are usually imposed as soft constraints by adding new terms to the loss function that is minimized during training. An alternative is to impose them as hard constraints, which has a number of theoretical benefits but has not been explored so far due to the perceived intractability of the problem. In this paper, we show that imposing hard constraints can in fact be done in a computationally feasible way and delivers reasonable results. However, the theoretical benefits do not materialize and the resulting technique is no better than existing ones relying on soft constraints. We analyze the reasons for this and hope to spur other researchers into proposing better solutions.

Semi-Supervised Phoneme Recognition with Recurrent Ladder Networks

Ladder networks are a notable new concept in the field of semi-supervised learning by showing state-of-the-art results in image recognition tasks while being compatible with many existing neural architectures. We present the recurrent ladder network, a novel modification of the ladder network, for semi-supervised learning of recurrent neural networks which we evaluate with a phoneme recognition task on the TIMIT corpus. Our results show that the model is able to consistently outperform the baseline and achieve fully-supervised baseline performance with only 75% of all labels which demonstrates that the model is capable of using unsupervised data as an effective regulariser.

The Temporal Event Graph

Temporal networks are increasingly being used to model the interactions of complex systems. Most studies require the temporal aggregation of edges (or events) into discrete time steps to perform analysis. In this article we describe a static, lossless, and unique representation of a temporal network, the temporal event graph (TEG). The TEG describes the temporal network in terms of both the inter-event time and two-event temporal motif distributions. By considering these distributions in unison we provide a new method to characterise the behaviour of individuals and collectives in temporal networks as well as providing a natural decomposition of the network. We illustrate the utility of the TEG by providing examples on both synthetic and real temporal networks.

BiSeg: Simultaneous Instance Segmentation and Semantic Segmentation with Fully Convolutional Networks

We present a simple and effective framework for simultaneous semantic segmentation and instance segmentation with Fully Convolutional Networks (FCNs). The method, called BiSeg, predicts instance segmentation as a posterior in Bayesian inference, where semantic segmentation is used as a prior. We extend the idea of position-sensitive score maps used in recent methods to a fusion of multiple score maps at different scales and partition modes, and adopt it as a robust likelihood for instance segmentation inference. As both Bayesian inference and map fusion are performed per pixel, BiSeg is a fully convolutional end-to-end solution that inherits all the advantages of FCNs. We demonstrate state-of-the-art instance segmentation accuracy on PASCAL VOC.

A Bootstrap Lasso + Partial Ridge Method to Construct Confidence Intervals for Parameters in High-dimensional Sparse Linear Models

For high-dimensional sparse linear models, how to construct confidence intervals for coefficients remains a difficult question. The main reason is the complicated limiting distributions of common estimators such as the Lasso. Several confidence interval construction methods have been developed, and Bootstrap Lasso+OLS is notable for its simple technicality, good interpretability, and comparable performance with other more complicated methods. However, Bootstrap Lasso+OLS depends on the beta-min assumption, a theoretic criterion that is often violated in practice. In this paper, we introduce a new method called Bootstrap Lasso+Partial Ridge (LPR) to relax this assumption. LPR is a two-stage estimator: first using Lasso to select features and subsequently using Partial Ridge to refit the coefficients. Simulation results show that Bootstrap LPR outperforms Bootstrap Lasso+OLS when there exist small but non-zero coefficients, a common situation violating the beta-min assumption. For such coefficients, compared to Bootstrap Lasso+OLS, confidence intervals constructed by Bootstrap LPR have on average 50% larger coverage probabilities. Bootstrap LPR also has on average 35% shorter confidence interval lengths than the de-sparsified Lasso methods, regardless of whether linear models are misspecified. Additionally, we provide theoretical guarantees of Bootstrap LPR under appropriate conditions and implement it in the R package ‘HDCI.’

Inductive Representation Learning on Large Graphs

Low-dimensional embeddings of nodes in large graphs have proved extremely useful in a variety of prediction tasks, from content recommendation to identifying protein functions. However, most existing approaches require that all nodes in the graph are present during training of the embeddings; these previous approaches are inherently transductive and do not naturally generalize to unseen nodes. Here we present GraphSAGE, a general, inductive framework that leverages node feature information (e.g., text attributes) to efficiently generate node embeddings for previously unseen data. Instead of training individual embeddings for each node, we learn a function that generates embeddings by sampling and aggregating features from a node’s local neighborhood. Our algorithm outperforms strong baselines on three inductive node-classification benchmarks: we classify the category of unseen nodes in evolving information graphs based on citation and Reddit post data, and we show that our algorithm generalizes to completely unseen graphs using a multi-graph dataset of protein-protein interactions.

Gated Recurrent Neural Tensor Network

Recurrent Neural Networks (RNNs), which are a powerful scheme for modeling temporal and sequential data need to capture long-term dependencies on datasets and represent them in hidden layers with a powerful model to capture more information from inputs. For modeling long-term dependencies in a dataset, the gating mechanism concept can help RNNs remember and forget previous information. Representing the hidden layers of an RNN with more expressive operations (i.e., tensor products) helps it learn a more complex relationship between the current input and the previous hidden layer information. These ideas can generally improve RNN performances. In this paper, we proposed a novel RNN architecture that combine the concepts of gating mechanism and the tensor product into a single model. By combining these two concepts into a single RNN, our proposed models learn long-term dependencies by modeling with gating units and obtain more expressive and direct interaction between input and hidden layers using a tensor product on 3-dimensional array (tensor) weight parameters. We use Long Short Term Memory (LSTM) RNN and Gated Recurrent Unit (GRU) RNN and combine them with a tensor product inside their formulations. Our proposed RNNs, which are called a Long-Short Term Memory Recurrent Neural Tensor Network (LSTMRNTN) and Gated Recurrent Unit Recurrent Neural Tensor Network (GRURNTN), are made by combining the LSTM and GRU RNN models with the tensor product. We conducted experiments with our proposed models on word-level and character-level language modeling tasks and revealed that our proposed models significantly improved their performance compared to our baseline models.

Adaptive Robust Control Under Model Uncertainty

In this paper we propose a new methodology for solving an uncertain stochastic Markovian control problem in discrete time. We call the proposed methodology the adaptive robust control. We demonstrate that the uncertain control problem under consideration can be solved in terms of associated adaptive robust Bellman equation. The success of our approach is to the great extend owed to the recursive methodology for construction of relevant confidence regions. We illustrate our methodology by considering an optimal portfolio allocation problem, and we compare results obtained using the adaptive robust control method with some other existing methods.

Efficient Reinforcement Learning via Initial Pure Exploration

In several realistic situations, an interactive learning agent can practice and refine its strategy before going on to be evaluated. For instance, consider a student preparing for a series of tests. She would typically take a few practice tests to know which areas she needs to improve upon. Based of the scores she obtains in these practice tests, she would formulate a strategy for maximizing her scores in the actual tests. We treat this scenario in the context of an agent exploring a fixed-horizon episodic Markov Decision Process (MDP), where the agent can practice on the MDP for some number of episodes (not necessarily known in advance) before starting to incur regret for its actions. During practice, the agent’s goal must be to maximize the probability of following an optimal policy. This is akin to the problem of Pure Exploration (PE). We extend the PE problem of Multi Armed Bandits (MAB) to MDPs and propose a Bayesian algorithm called Posterior Sampling for Pure Exploration (PSPE), which is similar to its bandit counterpart. We show that the Bayesian simple regret converges at an optimal exponential rate when using PSPE. When the agent starts being evaluated, its goal would be to minimize the cumulative regret incurred. This is akin to the problem of Reinforcement Learning (RL). The agent uses the Posterior Sampling for Reinforcement Learning algorithm (PSRL) initialized with the posteriors of the practice phase. We hypothesize that this PSPE + PSRL combination is an optimal strategy for minimizing regret in RL problems with an initial practice phase. We show empirical results which prove that having a lower simple regret at the end of the practice phase results in having lower cumulative regret during evaluation.

Graph Convolutional Matrix Completion

In this paper we revisit matrix completion for recommender systems from the point of view of link prediction on graphs. Interaction data such as movie ratings can be represented by a bipartite user-item graph with labeled edges representing observed ratings. Building on recent progress in deep learning on graph-structured data, we propose a graph auto-encoder framework based on differentiable message passing on the bipartite interaction graph. This framework can be viewed as an important first step towards end-to-end learning in settings where the interaction data is integrated into larger graphs such as social networks or knowledge graphs, circumventing the need for multistage frameworks. Our model achieves competitive performance on standard collaborative filtering benchmarks, significantly outperforming related methods in a recommendation task with side information.

An infinite family of 3d Floquet topological paramagnets
Full Quantification of Left Ventricle via Deep Multitask Learning Network Respecting Intra- and Inter-Task Relatedness
Nature vs. Nurture in Discrete Spin Dynamics
Distributed Hierarchical Control for State Estimation With Robotic Sensor Networks
Square function and the Hamming cube: duality
An adverse selection approach to power pricing
Resource Allocation Strategies for Secure WPCN Multiantenna Multicasting Systems
Upper bounds on the smallest size of a complete cap in $\mathrm{PG}(N,q)$, $N\ge3$, under a certain probabilistic conjecture
Practical Integer-to-Binary Mapping for Quantum Annealers
Robust Convex Quadratically Constrained Quadratic Programming with Mixed-Integer Uncertainty
Discrete Adaptive Second Order Sliding Mode Controller Design with Application to Automotive Control Systems with Model Uncertainties
Bayesian Formulations of Multidimensional Barcode Inversion
Discounted Penalty Function at Parisian Ruin for Lévy Insurance Risk Process
Inference for heavy tailed stationary time series based on sliding blocks
Deep Learning: Generalization Requires Deep Compositional Feature Space Design
Measuring Offensive Speech in Online Political Discourse
Unsupervised Neural-Symbolic Integration
Scaling and Saturation in Infinite-Dimensional Control Problems with Applications to Stochastic Partial Differential Equations
Approximation Algorithms for the Open Shop Problem with Delivery Times
Question Answering and Question Generation as Dual Tasks
Relationship of Two Formulations for Shortest Bibranchings
Optimal Power Control for Transmitting Correlated Sources with Energy Harvesting Constraints
An Opportunistic-Bit Scheme with IP Styled Communication
Coupling and Decoupling to bound an approximating Markov Chain
DeepSketch2Face: A Deep Learning Based Sketching System for 3D Face and Caricature Modeling
Conditional independence test for categorical data using Poisson log-linear model
Stacked Convolutional and Recurrent Neural Networks for Bird Audio Detection
Epistemic Logic with Functional Dependency Operator
Automatic Emphysema Detection using Weakly Labeled HRCT Lung Images
Are Saddles Good Enough for Deep Learning?
Unsupervised Place Discovery for Place-Specific Change Classifier
Early Experiences with Crowdsourcing Airway Annotations in Chest CT
An Extended Relevance Model for Session Search
Fast Eigen Decomposition for Low-Rank Matrix Approximation
DeLiGAN : Generative Adversarial Networks for Diverse and Limited Data
Approximate Shifted Combinatorial Optimization
Preliminary Performance Estimations and Benchmark Results for a Software-based Fault-Tolerance Approach aboard Miniaturized Satellite Computers
Multi-Robot Data Gathering Under Buffer Constraints and Intermittent Communication
Cascade Ranking for Operational E-commerce Search
Macquarie University at BioASQ 5b — Query-based Summarisation Techniques for Selecting the Ideal Answers
Parameter-space study of kinetic-impactor mission design
Symmetric loss functions in restricted parameter spaces
Guided Interaction Exploration in Artifact-centric Process Models
Energy Efficient Scheduling of Application Components via Brownout and Approximate Markov Decision Process
On maximizing the fundamental frequency of the complement of an obstacle
How Important is Syntactic Parsing Accuracy? An Empirical Evaluation on Sentiment Analysis
Variance bounding of delayed-acceptance kernels
Output feedback exponential stabilization of a nonlinear 1-D wave equation with boundary input
Usage Bibliometrics as a Tool to Measure Research Activity
Output feedback exponential stabilization for 1-D unstable wave equations with boundary control matched disturbance
Schubert polynomials, theta and eta polynomials, and Weyl group invariants
Propagation of regularity in $L^p$-spaces for Kolmogorov type hypoelliptic operators
Synthesizing Filamentary Structured Images with GANs
Kernel Regression for Signals over Graphs
Non-triviality of the vacancy phase transition for the Boolean model
Bounds and approximation results for scheduling coupled-tasks with compatibility constraints
Isomorphic coupled-task scheduling problem with compatibility constraints on a single processor
Asymptotic topology of random subcomplexes in a finite simplicial complex
Compression, inversion, and approximate PCA of dense kernel matrices at near-linear computational complexity
On The Communication Complexity of High-Dimensional Permutations
Improving Max-Sum through Decimation to Solve Loopy Distributed Constraint Optimization Problems
Distributed Optimization of Multi-Beam Directional Communication Networks
Some complexity and approximation results for coupled-tasks scheduling problem according to topology
Asymptotic measures and links in simplicial complexes
Recurrent computations for visual pattern completion
Insights into Analogy Completion from the Biomedical Domain
Stochastic Global Optimization Algorithms: A Systematic Formal Approach
Comparative Analysis of Open Source Frameworks for Machine Learning with Use Case in Single-Threaded and Multi-Threaded Modes
A Mention-Ranking Model for Abstract Anaphora Resolution
Driver Action Prediction Using Deep (Bidirectional) Recurrent Neural Network
InfoVAE: Information Maximizing Variational Autoencoders
Phase Transitions in Edge-Weighted Exponential Random Graphs: Near-Degeneracy and Universality
Handling Model and Implementation Uncertainties via an Adaptive Discrete Sliding Mode Controller Design
Can Computers overcome Humans? Consciousness interaction and its implications
Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
A Note on Multiparty Communication Complexity and the Hales-Jewett Theorem
A multilevel block search algorithm for fast modeling generalized separable systems
Propensity score-based estimators with multiple error-prone covariates