Multi-Advisor Reinforcement Learning

This article deals with a novel branch of Separation of Concerns, called Multi-Advisor Reinforcement Learning (MAd-RL), where a single-agent RL problem is distributed to n learners, called advisors. Each advisor tries to solve the problem with a different focus. Their advice is then communicated to an aggregator, which is in control of the system. For the local training, three off-policy bootstrapping methods are proposed and analysed: local-max bootstraps with the local greedy action, rand-policy bootstraps with respect to the random policy, and agg-policy bootstraps with respect to the aggregator’s greedy policy. MAd-RL is positioned as a generalisation of Reinforcement Learning with Ensemble methods. An experiment is held on a simplified version of the Ms. Pac-Man Atari game. The results confirm the theoretical relative strengths and weaknesses of each method.


A Genetic Programming Approach to Designing Convolutional Neural Network Architectures

The convolutional neural network (CNN), which is one of the deep learning models, has seen much success in a variety of computer vision tasks. However, designing CNN architectures still requires expert knowledge and a lot of trial and error. In this paper, we attempt to automatically construct CNN architectures for an image classification task based on Cartesian genetic programming (CGP). In our method, we adopt highly functional modules, such as convolutional blocks and tensor concatenation, as the node functions in CGP. The CNN structure and connectivity represented by the CGP encoding method are optimized to maximize the validation accuracy. To evaluate the proposed method, we constructed a CNN architecture for the image classification task with the CIFAR-10 dataset. The experimental result shows that the proposed method can be used to automatically find the competitive CNN architecture compared with state-of-the-art models.


Restricted Recurrent Neural Tensor Networks

Increasing the capacity of recurrent neural networks (RNN) usually involves augmenting the size of the hidden layer, resulting in a significant increase of computational cost. An alternative is the recurrent neural tensor network (RNTN), which increases capacity by employing distinct hidden layer weights for each vocabulary word. The disadvantage of RNTNs is that memory usage scales linearly with vocabulary size, which can reach millions for word-level language models. In this paper, we introduce restricted recurrent neural tensor networks (r-RNTN) which reserve distinct hidden layer weights for frequent vocabulary words while sharing a single set of weights for infrequent words. Perplexity evaluations using the Penn Treebank corpus show that r-RNTNs improve language model performance over standard RNNs using only a small fraction of the parameters of unrestricted RNTNs.


Time Series Cluster Kernel for Learning Similarities between Multivariate Time Series with Missing Data

Similarity-based approaches represent a promising direction for time series analysis. However, many such methods rely on parameter tuning and have shortcomings if the time series are multivariate (MTS) and contain missing data. In this paper, we address these challenges within the powerful context of kernel methods by proposing the robust \emph{time series cluster kernel} (TCK). The approach taken is to leverage the missing data handling properties of Gaussian mixture models (GMM) augmented with informative prior distributions. An ensemble learning approach is exploited to ensure robustness to parameters by combining the clustering results of many GMM to form the final kernel. We evaluate the TCK on synthetic and real data and compare to other state-of-the-art techniques. The experimental results demonstrate that the TCK is robust to parameter choices, provides competitive results for MTS without missing data and outstanding results for missing data.


On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learning

In this paper, we utilize results from convex analysis and monotone operator theory to derive additional properties of the softmax function not yet covered in the existing literature. In particular, we show that the softmax function is the monotone gradient map of the log-sum-exp function. We show that the inverse temperature parameter determines the Lipschitz and co-coercivity properties of the softmax function. We demonstrate the usefulness of these properties through an application in population games and reinforcement learning.


Interpretation of Semantic Tweet Representations

Research in analysis of microblogging platforms is experiencing a renewed surge with a large number of works applying representation learning models for applications like sentiment analysis, semantic textual similarity computation, hashtag prediction, etc. Although the performance of the representation learning models has been better than the traditional baselines for such tasks, little is known about the elementary properties of a tweet encoded within these representations, or why particular representations work better for certain tasks. Our work presented here constitutes the first step in opening the black-box of vector embeddings for tweets. Traditional feature engineering methods for high-level applications have exploited various elementary properties of tweets. We believe that a tweet representation is effective for an application because it meticulously encodes the application-specific elementary properties of tweets. To understand the elementary properties encoded in a tweet representation, we evaluate the representations on the accuracy to which they can model each of those properties such as tweet length, presence of particular words, hashtags, mentions, capitalization, etc. Our systematic extensive study of nine supervised and four unsupervised tweet representations against most popular eight textual and five social elementary properties reveal that Bi-directional LSTMs (BLSTMs) and Skip-Thought Vectors (STV) best encode the textual and social properties of tweets respectively. FastText is the best model for low resource settings, providing very little degradation with reduction in embedding size. Finally, we draw interesting insights by correlating the model performance obtained for elementary property prediction tasks with the high-level downstream applications.


Converging High-Throughput and High-Performance Computing: A Case Study

The computing systems used by LHC experiments has historically consisted of the federation of hundreds to thousands of distributed resources, ranging from small to mid-size resource. In spite of the impressive scale of the existing distributed computing solutions, the federation of small to mid-size resources will be insufficient to meet projected future demands. This paper is a case study of how the ATLAS experiment has embraced Titan — a DOE leadership facility in conjunction with traditional distributed high-throughput computing to reach sustained production scales of approximately 51M core-hours a years. The three main contributions of this paper are: (i) a critical evaluation of design and operational considerations to support the sustained, scalable and production usage of Titan; (ii) a preliminary characterization of a next generation executor for PanDA to support new workloads and advanced execution modes; and (iii) early lessons for how current and future experimental and observational systems can be integrated with production supercomputers and other platforms in a general and extensible manner.


Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory

Emotional intelligence is one of the key factors to the success of dialogue systems or conversational agents. In this paper, we propose Emotional Chatting Machine (ECM) which generates responses that are appropriate not only at the content level (relevant and grammatical) but also at the emotion level (consistent emotional expression). To the best of our knowledge, this is the first work that addresses the emotion factor in large-scale conversation generation. ECM addresses the factor in three ways: modeling high-level abstraction of emotion expression by embedding emotion categories, changing of implicit internal emotion states, and using explicit emotion expressions with an external emotion vocabulary. Experiments show that our model can generate responses appropriate not only in content but also in emotion.


Probabilistic Search for Structured Data via Probabilistic Programming and Nonparametric Bayes

Databases are widespread, yet extracting relevant data can be difficult. Without substantial domain knowledge, multivariate search queries often return sparse or uninformative results. This paper introduces an approach for searching structured data based on probabilistic programming and nonparametric Bayes. Users specify queries in a probabilistic language that combines standard SQL database search operators with an information theoretic ranking function called predictive relevance. Predictive relevance can be calculated by a fast sparse matrix algorithm based on posterior samples from CrossCat, a nonparametric Bayesian model for high-dimensional, heterogeneously-typed data tables. The result is a flexible search technique that applies to a broad class of information retrieval problems, which we integrate into BayesDB, a probabilistic programming platform for probabilistic data analysis. This paper demonstrates applications to databases of US colleges, global macroeconomic indicators of public health, and classic cars. We found that human evaluators often prefer the results from probabilistic search to results from a standard baseline.


Many-body Localization Transition: Schmidt Gap, Entanglement Length & Scaling

Ore’s theorem on subfactor planar algebras

Data-Injection Attacks in Stochastic Control Systems: Detectability and Performance Tradeoffs

Voltage Control Using Limited Communication

Unsupervised Action Proposal Ranking through Proposal Recombination

AMC: Attention guided Multi-modal Correlation Learning for Image Search

Quantum Algorithms for Graph Connectivity and Formula Evaluation

Sequential Active Detection of Anomalies in Heterogeneous Processes

Geometric Insights into Support Vector Machine Behavior using the KKT Conditions

A comparative study of counterfactual estimators

The Unbounded-Error Communication Complexity of symmetric XOR functions

What is the meaning of the graph energy after all?

Brief Notes on Hard Takeoff, Value Alignment, and Coherent Extrapolated Volition

Online and Linear-Time Attention by Enforcing Monotonic Alignments

Towards generic adiabatic elimination for bipartite open quantum systems

Design and development of a software system for swarm intelligence based research studies

On the idea of a new artificial intelligence based optimization algorithm inspired from the nature of vortex

Regularization by noise in (2x 2) hyperbolic systems of conservation law

Synchronization Strings: Codes for Insertions and Deletions Approaching the Singleton Bound

Principal Inertia Components and Applications

SAFE2: A Hierarchical Model of Pitch Framing

A Probabilistic Linear Genetic Programming with Stochastic Context-Free Grammar for solving Symbolic Regression problems

Online deforestation detection

Locally Self-Adjusting Skip Graphs

Cascaded Segmentation-Detection Networks for Word-Level Text Spotting

Near-Optimality of Linear Recovery from Indirect Observations

t-unique reductions for Mészáros’s subdivision algebra

The Cloudlet Bazaar Dynamic Markets for the Small Cloud

Guided Proofreading of Automatic Segmentations for Connectomics

Voice Conversion from Unaligned Corpora using Variational Autoencoding Wasserstein Generative Adversarial Networks

Estimating the spectral gap of a trace-class Markov operator

Some Schubert shenanigans

A History of Metaheuristics

On polytopes close to being simple

Population games and Discrete optimal transport

Simultaneous Feature Aggregating and Hashing for Large-scale Image Search

Linear Openness and Feedback Stabilization of Nonlinear Control Systems

The string of diamonds is tight for rumor spreading

HAlign-II: efficient ultra-large multiple sequence alignment and phylogenetic tree reconstruction with distributed and parallel computing

Nearly Equal Distributions of the Rank and the Crank of Partitions

A Branch-and-Bound Algorithm for Checkerboard Extraction in Camera-Laser Calibration

Crystal analysis of type $C$ Stanley symmetric functions

Dynamic Rank Maximal Matchings

Quantitative Reductions and Vertex-Ranked Infinite Games

(1, k)-Swap Local Search for Maximum Clique Problem

MIMO Underwater Visible Light Communications: Comprehensive Channel Study, Performance Analysis, and Multiple-Symbol Detection

Space and time inversions of stochastic processes and Kelvin transform

Deriving Probability Density Functions from Probabilistic Functional Programs

Japanese Sentiment Classification using a Tree-Structured Long Short-Term Memory with Attention

On transformation semigroups based on digraphs

Measure-Valued Generators of General Piecewise Deterministic Markov Processes

Fortia-FBK at SemEval-2017 Task 5: Bullish or Bearish? Inferring Sentiment towards Brands from Financial News Headlines

A Universal Approximation Theorem for Gaussian-Gated Mixture of Experts Models

Blind Signal Detection in Massive MIMO: Exploiting the Channel Sparsity

Spatial clustering and heterogeneity in evolutionary games on structured populations: a unified framework

Stress Testing German Industry Sectors: Results from a Vine Copula Based Quantile Regression

Using clustering of rankings to explain brand preferences with personality and socio-demographic variables

Stability tests for a class of switched descriptor systems with non-homogenous indices

Adaptive Motion Gaming AI for Health Promotion

Bayesian optimization with virtual derivative sign observations

On inverse Wiener interval problem of trees

A large deviations principle for the polar empirical measure in the two-dimensional symmetric simple exclusion process

Optic Disc and Cup Segmentation Methods for Glaucoma Detection with Modification of U-Net Convolutional Neural Network

Probabilistic approach to finite state mean field games

Modelling and Filtering for Non-Markovian Quantum Systems

Learning Filter Functions in Regularisers by Minimising Quotients

Testing isomorphism of central Cayley graphs over almost simple groups in polynomial time

A New Transmitted Reference Pulse Cluster Based Ultra-Wideband Transmitter Design

A Backward Algorithm for the Multiprocessor Online Feasibility of Sporadic Tasks

Minimum Rainbow $H$-Decompositions of Graphs

On the Combinatorial Power of the Weisfeiler-Lehman Algorithm

Comparison of mean-field based theoretical analysis methods for SIS model

Uplink Performance Analysis in D2D-Enabled mmWave Cellular Networks

Strongly angle-dependent magnetoresistance in Weyl semimetals with long-range disorder

Polynomial Time and Sample Complexity for Non-Gaussian Component Analysis: Spectral Methods

Charting the replica symmetric phase

Using Echo State Networks for Cryptography

OctNetFusion: Learning Depth Fusion from Data

McKean-Vlasov limit for interacting systems with simultaneous jumps

Network-ensemble comparisons with stochastic rewiring and von Neumann entropy

One-step Local M-estimator for Integrated Jump-Diffusion Models

Sparse optimal control for fractional diffusion

Tests for qualitative features in the random coefficients model

ME R-CNN: Multi-Expert Region-based CNN for Object Detection

Computing top-k Closeness Centrality Faster in Unweighted Graphs

Homotopy Parametric Simplex Method for Sparse Learning

A Feasibility-Enforcing Primal-Decomposition SQP Algorithm for Optimal Vehicle Coordination

Deep Depth From Focus

Stein’s method for nonconventional sums

Fringe Analysis of Plane Trees Related to Cutting and Pruning

On Resource-bounded versions of the van Lambalgen theorem

Communication Complexity of Correlated Equilibrium in Two-Player Games

Betti splitting from a topological and computational point of view

Upper bounds for the spectral function on homogeneous spaces via volume growth

A note on quadratic forms with applications in optimization

Damped Posterior Linearization Filter

Advertisements