Large Linear Multi-output Gaussian Process Learning for Time Series

Gaussian processes, or distributions over arbitrary functions in a continuous domain, can be generalized to the multi-output case: a linear model of coregionalization (LMC) is one approach. LMCs estimate and exploit correlations across the multiple outputs. While model estimation can be performed efficiently for single-output GPs, these assume stationarity, but in the multi-output case the cross-covariance interaction is not stationary. We propose Large Linear GPs (LLGPs), which circumvent the need for stationarity by using LMC’s structure, enabling optimization of GP hyperparameters for multi-dimensional outputs and one-dimensional inputs. When applied to real time series data, we find our theoretical improvement relative to the current state of the art is realized with LLGP being generally an order of magnitude faster while improving or maintaining predictive accuracy.

Dynamics Based Features For Graph Classification

Numerous social, medical, engineering and biological challenges can be framed as graph-based learning tasks. Here, we propose a new feature based approach to network classification. We show how dynamics on a network can be useful to reveal patterns about the organization of the components of the underlying graph where the process takes place. We define generalized assortativities on networks and use them as generalized features across multiple time scales. These features turn out to be suitable signatures for discriminating between different classes of networks. Our method is evaluated empirically on established network benchmarks. We also introduce a new dataset of human brain networks (connectomes) and use it to evaluate our method. Results reveal that our dynamics based features are competitive and often outperform state of the art accuracies.

Surface Networks

We study data-driven representations for three-dimensional triangle meshes, which are one of the prevalent objects used to represent 3D geometry. Recent works have developed models that exploit the intrinsic geometry of manifolds and graphs, namely the Graph Neural Networks (GNNs) and its spectral variants, which learn from the local metric tensor via the Laplacian operator. Despite offering excellent sample complexity and built-in invariances, intrinsic geometry alone is invariant to isometric deformations, making it unsuitable for many applications. To overcome this limitation, we propose several upgrades to GNNs to leverage extrinsic differential geometry properties of three-dimensional surfaces, increasing its modeling power. In particular, we propose to exploit the Dirac operator, whose spectrum detects principal curvature directions — this is in stark contrast with the classical Laplace operator, which directly measures mean curvature. We coin the resulting model the \emph{Surface Network (SN)}. We demonstrate the efficiency and versatility of SNs on two challenging tasks: temporal prediction of mesh deformations under non-linear dynamics and generative models using a variational autoencoder framework with encoders/decoders given by SNs.

Objective-Reinforced Generative Adversarial Networks (ORGAN) for Sequence Generation Models

In unsupervised data generation tasks, besides the generation of a sample based on previous observations, one would often like to give hints to the model in order to bias the generation towards desirable metrics. We propose a method that combines Generative Adversarial Networks (GANs) and reinforcement learning (RL) in order to accomplish exactly that. While RL biases the data generation process towards arbitrary metrics, the GAN component of the reward function ensures that the model still remembers information learned from data. We build upon previous results that incorporated GANs and RL in order to generate sequence data and test this model in several settings for the generation of molecules encoded as text sequences (SMILES) and in the context of music generation, showing for each case that we can effectively bias the generation process towards desired metrics.

Sparse canonical correlation analysis

Canonical correlation analysis was proposed by Hotelling [6] and it measures linear relationship between two multidimensional variables. In high dimensional setting, the classical canonical correlation analysis breaks down. We propose a sparse canonical correlation analysis by adding l1 constraints on the canonical vectors and show how to solve it efficiently using linearized alternating direction method of multipliers (ADMM) and using TFOCS as a black box. We illustrate this idea on simulated data.

High Dimensional Structured Superposition Models

High dimensional superposition models characterize observations using parameters which can be written as a sum of multiple component parameters, each with its own structure, e.g., sum of low rank and sparse matrices, sum of sparse and rotated sparse vectors, etc. In this paper, we consider general superposition models which allow sum of any number of component parameters, and each component structure can be characterized by any norm. We present a simple estimator for such models, give a geometric condition under which the components can be accurately estimated, characterize sample complexity of the estimator, and give high probability non-asymptotic bounds on the componentwise estimation error. We use tools from empirical processes and generic chaining for the statistical analysis, and our results, which substantially generalize prior work on superposition models, are in terms of Gaussian widths of suitable sets.

FALKON: An Optimal Large Scale Kernel Method

Kernel methods provide a principled way to perform non linear, nonparametric learning. They rely on solid functional analytic foundations and enjoy optimal statistical properties. However, at least in their basic form, they have limited applicability in large scale scenarios because of stringent computational requirements in terms of time and especially memory. In this paper, we take a substantial step in scaling up kernel methods, proposing FALKON, a novel algorithm that allows to efficiently process millions of points. FALKON is derived combining several algorithmic principles, namely stochastic projections, iterative solvers and preconditioning. Our theoretical analysis shows that optimal statistical accuracy is achieved requiring essentially O(n) memory and O(n\sqrt{n}) time. Extensive experiments show that state of the art results on available large scale datasets can be achieved even on a single machine.

SuperSpike: Supervised learning in multi-layer spiking neural networks

A vast majority of computation in the brain is performed by spiking neural networks. Despite the ubiquity of such spiking, we currently lack an understanding of how biological spiking neural circuits learn and compute in-vivo, as well as how we can instantiate such capabilities in artificial spiking circuits in-silico. Here we revisit the problem of supervised learning in temporally coding multi-layer spiking neural networks. First, by using a surrogate gradient approach, we derive SuperSpike, a nonlinear voltage-based three factor learning rule capable of training multi-layer networks of deterministic integrate-and-fire neurons to perform nonlinear computations on spatiotemporal spike patterns. Second, inspired by recent results on feedback alignment, we compare the performance of our learning rule under different credit assignment strategies for propagating output errors to hidden units. Specifically, we test uniform, symmetric and random feedback, finding that simpler tasks can be solved with any type of feedback, while more complex tasks require symmetric feedback. In summary, our results open the door to obtaining a better scientific understanding of learning and computation in spiking neural networks by advancing our ability to train them to solve nonlinear problems involving transformations between different spatiotemporal spike-time patterns.

How a small quantum bath can thermalize long localized chains

Interplay between disorder and Coulomb interaction in nodal-line semimetals

Special cases of the orbifold version of Zvonkine’s $r$-ELSV formula

Character Composition Model with Convolutional Neural Networks for Dependency Parsing on Morphologically Rich Languages

Practical Neural Network Performance Prediction for Early Stopping

Minimizing the Cost of Team Exploration

Accuracy First: Selecting a Differential Privacy Level for Accuracy-Constrained ERM

Substitution Markov chains and Martin boundaries

Experience Replay Using Transition Sequences

Interaction and association of effect causes

A Tale of Two Animats: What does it take to have goals?

Generic Tubelet Proposals for Action Localization

Lifelong Multi-Agent Path Finding for Online Pickup and Delivery Tasks

Working hard to know your neighbor’s margins:Local descriptor learning loss

Deep Learning for Environmentally Robust Speech Recognition: An Overview of Recent Developments

A Hierarchical Model to Evaluate Policies for Reducing Vehicle Speed in Major American Cities

A general theory of singular values with applications to signal denoising

Morphological Error Detection in 3D Segmentations

Optimization of Tree Ensembles

Sparse and low-rank approximations of large symmetric matrices using biharmonic interpolation

Identification of Gaussian Process State Space Models

Operations preserving equivalence relations

Distributed Functional Observers for LTI Systems

Asymptotics of the spectral radius for directed Chung-Lu random graphs with community structure

Serial Correlations in Single-Subject fMRI with Sub-Second TR

Towards Learned Clauses Database Reduction Strategies Based on Dominance Relationship

Propositional Knowledge Representation in Restricted Boltzmann Machines

Does the Geometry of Word Embeddings Help Document Classification? A Case Study on Persistent Homology Based Representations

Weakly Supervised Generative Adversarial Networks for 3D Reconstruction

Unsupervised Learning of Disentangled Representations from Video

The ALAMO approach to machine learning

Network-based identification of disease genes in expression data: the GeneSurrounder method

Saving Critical Nodes with Firefighters is FPT

Sequential Dynamic Decision Making with Deep Neural Nets on a Test-Time Budget

A combinatorial proof of a formula of Biane and Chapuy

Naturally Combined Shape-Color Moment Invariants under Affine Transformations

Adversarial Generation of Natural Language

Micro Fourier Transform Profilometry ($μ$FTP): 3D shape measurement at 10,000 frames per second

Learning Graphs with Monotone Topology Properties and Multiple Connected Components

Long-time asymptotic of stable Dawson-Watanabe processes in supercritical regimes

Planar arcs

Spectral Norm Regularization for Improving the Generalizability of Deep Learning

Complex Quadrature Spatial Modulation

Bridge Simulation and Metric Estimation on Landmark Manifolds

A remark on the paper ‘properties of intersecting families of ordered sets’ by O. Einstein

Distributed Simulation Platform for Autonomous Driving

Optimal Selection of Small-Scale Hybrid PV-battery Systems to Maximize Economic Benefit Based on Temporal Load Data

Analysis of the Effect of Dependency Information on Predicate-Argument Structure Analysis and Zero Anaphora Resolution

A Reduction for the Distinct Distances Problem in ${\mathbb R}^d$

Max-Min Fair Transmit Precoding for Multi-group Multicasting in Massive MIMO

Reflected Solutions of BSDEs Driven by G-Brownian Motion

Skew Brownian motion with dry friction: The Pugachev-Sveshnikov equation approach

Class Specific Feature Selection for Interval Valued Data Through Interval K-Means Clustering

Succinct Partial Sums and Fenwick Trees

Spatial asymptotics at infinity for heat kernels of integro-differential operators

Non-Markovian Control with Gated End-to-End Memory Policy Networks

Correlations in magnitude series to assess nonlinearities: application to multifractal models and heartbeat fluctuations

The Atari Grand Challenge Dataset

Deep Supervised Discrete Hashing

Adversarial Ranking for Language Generation

Two bosonic quantum walkers in one-dimensional optical lattices

Congruent families and invariant tensors

Criticality & Deep Learning II: Momentum Renormalisation Group

Optimal control of reaction-diffusion systems with hysteresis

Application of projection algorithms to differential equations: boundary value problems

Chomp on numerical semigroups

Large Deviation Multifractal Analysis of a Process Modeling TCP CUBIC

End-to-end Differentiable Proving

Greedy Algorithms for Cone Constrained Optimization with Convergence Guarantees

Implicit Consensus: Blockchain with Unbounded Throughput

Weighted estimates for the bilinear maximal operator on filtered measure spaces

Neuron Segmentation Using Deep Complete Bipartite Networks

Statistical Analysis of Precipitation Events

Obtaining a Proportional Allocation by Deleting Items

Bayesian significance test for discriminating between survival distributions

Generalised Precoded Spatial Modulation for Integrated Wireless Information and Power Transfer

EvaluationNet: Can Human Skill be Evaluated by Deep Networks?

Bayesian multi-parameter evidence synthesis to inform decision-making: a case study in hormone-refractory metastatic prostate cancer

Effects of nonmagnetic disorder on the energy of Yu-Shiba-Rusinov states

Uniform random colored complexes

HiNet: Hierarchical Classification with Neural Network

Information Theoretic Properties of Markov Random Fields, and their Algorithmic Applications

Identification of points using disks

Complexity Certification of a Distributed Augmented Lagrangian Method

Controllable Invariance through Adversarial Feature Learning

Bayesian Distributional Non-Linear Multilevel Modeling with the R Package brms

Proper efficiency and cone efficiency

On the Sublinear Regret of Distributed Primal-Dual Algorithms for Online Constrained Optimization

A lower bound on the order of the largest induced linear forest in triangle-free planar graphs

Representation Learning by Rotating Your Faces

Variational Sequential Monte Carlo

Extreme-Scale De Novo Genome Assembly

Information transmission and criticality in the contact process

Models and information-theoretic bounds for nanopore sequencing

Reinforcement Learning for Learning Rate Control

Learning When to Attend for Neural Machine Translation

The Tutte embedding of the mated-CRT map converges to Liouville quantum gravity

Decremental Single-Source Reachability in Planar Digraphs

Adversarial Inversion: Inverse Graphics with Adversarial Priors

Are distributional representations ready for the real world? Evaluating word vectors for grounded perceptual meaning

Long-term Correlation Tracking using Multi-layer Hybrid Features in Sparse and Dense Environments

Gradient and Stability Estimates of Heat Kernels for Fractional Powers of Elliptic Operator

U-Phylogeny: Undirected Provenance Graph Construction in the Wild

The Morphospace of Consciousness

Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols