Generalized linear models with low rank effects for network data

Networks are a useful representation for data on connections between units of interests, but the observed connections are often noisy and/or include missing values. One common approach to network analysis is to treat the network as a realization from a random graph model, and estimate the underlying edge probability matrix, which is sometimes referred to as network denoising. Here we propose a generalized linear model with low rank effects to model network edges. This model can be applied to various types of networks, including directed and undirected, binary and weighted, and it can naturally utilize additional information such as node and/or edge covariates. We develop an efficient projected gradient ascent algorithm to fit the model, establish asymptotic consistency, and demonstrate empirical performance of the method on both simulated and real networks.

Building effective deep neural network architectures one feature at a time

Successful training of convolutional neural networks is often associated with the training of sufficiently deep architectures composed of high amounts of features while relying on a variety of regularization and pruning techniques to converge to less redundant states. We introduce an easy to compute metric, based on feature time evolution, to evaluate feature importance during training and demonstrate its potency in determining a networks effective capacity. In consequence we propose a novel algorithm to evolve fixed-depth architectures starting from just a single feature per layer to attain effective representational capacities needed for a specific task by greedily adding feature by feature. We revisit popular CNN architectures and demonstrate how evolved architectures not only converge to similar topologies that benefit from less parameters or improved accuracy, but furthermore exhibit systematic correspondence in representational complexity with the specified task. In contrast to conventional design patterns that typically have a monotonic increase in the amount of features with increased depth, we observe that CNNs perform better when there is a peak in learnable parameters in intermediate, with falloffs to earlier and later layers.

Learning Convolutional Text Representations for Visual Question Answering

Visual question answering is a recently proposed artificial intelligence task that requires a deep understanding of both images and texts. In deep learning, images are typically modeled through convolutional neural networks, and texts are typically modeled through recurrent neural networks. While the requirement for modeling images is similar to traditional computer vision tasks, such as object recognition and image classification, visual question answering raises a different need for textual representation as compared to other natural language processing tasks. In this work, we perform a detailed analysis on natural language questions in visual question answering. Based on the analysis, we propose to rely on convolutional neural networks for learning textual representations. By exploring the various properties of convolutional neural networks specialized for text data, such as width and depth, we present our ‘CNN Inception + Gate’ model. We show that our model improves question representations and thus the overall accuracy of visual question answering models. We also show that the text representation requirement in visual question answering is more complicated and comprehensive than that in conventional natural language processing tasks, making it a better task to evaluate textual representation methods. Shallow models like fastText, which can obtain comparable results with deep learning models in tasks like text classification, are not suitable in visual question answering.

A Survey of Neuromorphic Computing and Neural Networks in Hardware

Neuromorphic computing has come to refer to a variety of brain-inspired computers, devices, and models that contrast the pervasive von Neumann computer architecture. This biologically inspired approach has created highly connected synthetic neurons and synapses that can be used to model neuroscience theories as well as solve challenging machine learning problems. The promise of the technology is to create a brain-like ability to learn and adapt, but the technical challenges are significant, starting with an accurate neuroscience model of how the brain works, to finding materials and engineering breakthroughs to build devices to support these models, to creating a programming framework so the systems can learn, to creating applications with brain-like capabilities. In this work, we provide a comprehensive survey of the research and motivations for neuromorphic computing over its history. We begin with a 35-year review of the motivations and drivers of neuromorphic computing, then look at the major research areas of the field, which we define as neuro-inspired models, algorithms and learning approaches, hardware and devices, supporting systems, and finally applications. We conclude with a broad discussion on the major research topics that need to be addressed in the coming years to see the promise of neuromorphic computing fulfilled. The goals of this work are to provide an exhaustive review of the research conducted in neuromorphic computing since the inception of the term, and to motivate further work by illuminating gaps in the field where new research is needed.

The Landscape of Deep Learning Algorithms

This paper studies the landscape of empirical risk of deep neural networks by theoretically analyzing its convergence behavior to the population risk as well as its stationary points and properties. For an l-layer linear neural network, we prove its empirical risk uniformly converges to its population risk at the rate of \mathcal{O}(r^{2l}\sqrt{d\log(l)}/\sqrt{n}) with training sample size of n, the total weight dimension of d and the magnitude bound r of weight of each layer. We then derive the stability and generalization bounds for the empirical risk based on this result. Besides, we establish the uniform convergence of gradient of the empirical risk to its population counterpart. We prove the one-to-one correspondence of the non-degenerate stationary points between the empirical and population risks with convergence guarantees, which describes the landscape of deep neural networks. In addition, we analyze these properties for deep nonlinear neural networks with sigmoid activation functions. We prove similar results for convergence behavior of their empirical risks as well as the gradients and analyze properties of their non-degenerate stationary points. To our best knowledge, this work is the first one theoretically characterizing landscapes of deep learning algorithms. Besides, our results provide the sample complexity of training a good deep neural network. We also provide theoretical understanding on how the neural network depth l, the layer width, the network size d and parameter magnitude determine the neural network landscapes.

Classification revisited: a web of knowledge

The vision of the Semantic Web (SW) is gradually unfolding and taking shape through a web of linked data, a part of which is built by capturing semantics stored in existing knowledge organization systems (KOS), subject metadata and resource metadata. The content of vast bibliographic collections is currently categorized by some widely used bibliographic classification and we may soon see them being mined for information and linked in a meaningful way across the Web. Bibliographic classifications are designed for knowledge mediation which offers both a rich terminology and different ways in which concepts can be categorized and related to each other in the universe of knowledge. From 1990-2010 they have been used in various resource discovery services on the Web and continue to be used to support information integration in a number of international digital library projects. In this chapter we will revisit some of the ways in which universal classifications, as language independent concept schemes, can assist humans and computers in structuring and presenting information and formulating queries. Most importantly, we highlight issues important to understanding bibliographic classifications, both in terms of their unused potential and technical limitations.

The Bag Semantics of Ontology-Based Data Access

Ontology-based data access (OBDA) is a popular approach for integrating and querying multiple data sources by means of a shared ontology. The ontology is linked to the sources using mappings, which assign views over the data to ontology predicates. Motivated by the need for OBDA systems supporting database-style aggregate queries, we propose a bag semantics for OBDA, where duplicate tuples in the views defined by the mappings are retained, as is the case in standard databases. We show that bag semantics makes conjunctive query answering in OBDA coNP-hard in data complexity. To regain tractability, we consider a rather general class of queries and show its rewritability to a generalisation of the relational calculus to bags.

Prediction of Individual Outcomes for Asthma Sufferers

On Variations of Nim and Chomp

A quartet of fermionic expressions for $M(k,2k\pm1)$ Virasoro characters via half-lattice paths

Why Noise and Dispersion may Seriously Hamper Nonlinear Frequency-Division Multiplexing

Comprehensive Modeling of Three-Phase Distribution Systems via the Bus Admittance Matrix

Chimera states: Effects of different coupling topologies

Interference Alignment with Power Splitting Relays in Multi-User Multi-Relay Networks

Being even slightly shallow makes life hard

Asymptotic Average Number of Different Categories of Trapping Sets, Absorbing Sets and Stopping Sets in Random Regular and Irregular LDPC Code Ensembles

Joint Uplink and Downlink Coverage Analysis of Cellular-based RF-powered IoT Network

Good bounds in certain systems of true complexity $1$

Antenna Arrays for Line-of-Sight Massive MIMO: Half Wavelength is not Enough

Parallel replica dynamics method for bistable stochastic reaction networks: simulation and sensitivity analysis

Analysis of Thompson Sampling for Gaussian Process Optimization in the Bandit Setting

Stochastic Setup-Cost Inventory Model with Backorders and Quasiconvex Cost Functions

Minimal contagious sets in random graphs

Pixel Deconvolutional Networks

Spatial Variational Auto-Encoding via Matrix-Variate Normal Distributions

Simulations, Computations, and Statistics for Longest Common Subsequences

Agent-based simulation of the learning dissemination on a Project-Based Learning context considering the human aspects

Exploring the structure of a real-time, arbitrary neural artistic stylization network

Lattice exit models

Deep-LK for Efficient Adaptive Object Tracking

The Conference Paper Assignment Problem: Using Order Weighted Averages to Assign Indivisible Goods

Avalanches and Plastic Flow in Crystal Plasticity: An Overview

Modeling Geometrical Mysteries of Cafe Wall illusions

Syndrome-Coupled Rate-Compatible Error-Correcting Codes

Online Signature Verification using Recurrent Neural Network and Length-normalized Path Signature

Using a Hamiltonian cycle problem algorithm to assist in solving difficult instances of Traveling Salesman Problem

Origin of Non-cubic Scaling Law in Disordered Granular Packing

Beyond Massive-MIMO: The Potential of Positioning with Large Intelligent Surfaces

Prediction of Sea Surface Temperature using Long Short-Term Memory

ADMM-Net: A Deep Learning Approach for Compressive Sensing MRI

Fiber Orientation Estimation Guided by a Deep Network

Affine-Gradient Based Local Binary Pattern Descriptor for Texture Classiffication

An upper bound for the critical probability on the Cartesian product graph of a regular tree and a line

A Representation of Generalized Convex Polyhedra and Applications

Efficient Solutions in Generalized Linear Vector Optimization

Low-Complexity Iterative Algorithms for (Discrete) Compressed Sensing

Exact simulation of the first-passage time of diffusions

A Unified Framework for Stochastic Matrix Factorization via Variance Reduction

Energy-Efficient Resource Allocation for Elastic Optical Networks using Convex Optimization

On Some Generalized Polyhedral Convex Constructions

Piecewise Linear Vector Optimization Problems on Locally Convex Hausdorff Topological Vector Spaces

Practical Algorithms for Best-K Identification in Multi-Armed Bandits

Disordered statistical physics in low dimensions: extremes, glass transition, and localization

CDS Rate Construction Methods by Machine Learning Techniques

Local Shape Spectrum Analysis for 3D Facial Expression Recognition

Evidence for mixed rationalities in preference formation

Ultra-Reliable and Low Latency Communication in mmWave-Enabled Massive MIMO Networks

Unbiased estimates for linear regression via volume sampling

Beyond similarity assessment: Selecting the optimal model for sequence alignment via the Factorized Asymptotic Bayesian algorithm

R Package ASMap: Efficient Genetic Linkage Map Construction and Diagnosis

New symmetry tests based on characterization by squares of linear statistics, and their efficiencies

Hyperspectral Band Selection Using Unsupervised Non-Linear Deep Auto Encoder to Train External Classifiers

Spectral-graph Based Classifications: Linear Regression for Classification and Normalized Radial Basis Function Network

Weak convergence of the weighted empirical beta copula process

Foundations of Declarative Data Analysis Using Limit Datalog Programs

Colourings of cubic graphs inducing isomorphic monochromatic subgraphs

Atari games and Intel processors

Proposal for a Leaky Integrate Fire Spiking Neuron Using Voltage Driven Domain Wall Motion

The Kinetics Human Action Video Dataset

Diffusion limit for the partner model at the critical value

Beam Design and User Scheduling for Non-Orthogonal Multiple Access with Multiple Antennas Based on Pareto-Optimality

Parameter Adaptation and Criticality in Particle Swarm Optimization

Evolutionary games on scale-free multiplex networks

End-to-End Cross-Modality Retrieval with CCA Projections and Pairwise Ranking Loss

Cooperative Spectrum Sensing over Generalized Fading Channels Based on Energy Detection

Performance Analysis of Energy Detection over Composite kappa-miu Shadowed Fading Channels

Cooperative spectrum sensing with enhanced energy detection under GAUSSIAN noise uncertainty in cognitive radios

Near-optimality of sequential joint detection and estimation via online mirror descent

A lower bound on the positive semidefinite rank of convex bodies

A High-Performance Algorithm for Identifying Frequent Items in Data Streams

Bayesian Nonparametric Poisson Process Allocation

A Lightweight Regression Method to Infer Psycholinguistic Properties for Brazilian Portuguese

Fundamental mode exact schemes for unsteady problems

Achievable Information Rates for Coded Modulation with Hard Decision Decoding for Coherent Fiber-Optic Systems

A note on the metastability in three modifications of the standard Ising model

Segmentation of 3D High-frequency Ultrasound Images of Human Lymph Nodes Using Graph Cut with Energy Functional Adapted to Local Intensity Distribution

On Packet Scheduling with Adversarial Jamming and Speedup

Distribution-Free Causal Inference via Counterfactual Prediction

Learning Effective Representations from Clinical Notes

Hamilton cycles in infinite cubic graphs

Posterior sampling for reinforcement learning: worst-case regret bounds

Standardizing densities on Gaussian spaces

Faceted classification: management and use

Linear regression without correspondence

What are the Receptive, Effective Receptive, and Projective Fields of Neurons in Convolutional Neural Networks?

Speeding up Memory-based Collaborative Filtering with Landmarks

Masked Autoregressive Flow for Density Estimation

MRI-PET Registration with Automated Algorithm in Pre-clinical Studies

Cycle decompositions of pathwidth-6 graphs

CacheShuffle: An Oblivious Shuffle Algorithm Using Caches

EE-Grad: Exploration and Exploitation for Cost-Efficient Mini-Batch SGD

On the number of faces of Gelfand-Zetlin polytopes

What do We Learn by Semantic Scene Understanding for Remote Sensing imagery in CNN framework?

Scalable Variational Inference for Dynamical Systems

Bitwise Operations of Cellular Automaton on Gray-scale Images

Secure Computation of Randomized Functions: Further Results

Generalized bilinear forms graphs and MDR codes

Estimating Accuracy from Unlabeled Data: A Probabilistic Logic Approach

CNN-Based Joint Clustering and Representation Learning with Feature Drift Compensation for Large-Scale Image Data

Induction of Interpretable Possibilistic Logic Theories from Relational Data

Optimal bounds and extremal trajectories for time averages in dynamical systems

Machine learning for classification and quantification of monoclonal antibody preparations for cancer therapy

Feature-rich bifurcations in a simple electronic circuit

Efficient Learning of Harmonic Priors for Pitch Detection in Polyphonic Music

Gradient Estimators for Implicit Models

Snapshot Difference Imaging using Time-of-Flight Sensors

Deep adversarial neural decoding

The Kernel Mixture Network: A Nonparametric Method for Conditional Density Estimation of Continuous Random Variables

Fast Singular Value Shrinkage with Chebyshev Polynomial Approximation Based on Signal Sparsity

A Comparison of Reinforcement Learning Techniques for Fuzzy Cloud Auto-Scaling

Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics