KATE: K-Competitive Autoencoder for Text

Autoencoders have been successful in learning meaningful representations from image datasets. However, their performance on text datasets has not been widely studied. Traditional autoencoders tend to learn possibly trivial representations of text documents due to their confounding properties such as high-dimensionality, sparsity and power-law word distributions. In this paper, we propose a novel k-competitive autoencoder, called KATE, for text documents. Due to the competition between the neurons in the hidden layer, each neuron becomes specialized in recognizing specific data patterns, and overall the model can learn meaningful representations of textual data. A comprehensive set of experiments show that KATE can learn better representations than traditional autoencoders including denoising, contractive, variational, and k-sparse autoencoders. Our model also outperforms deep generative models, probabilistic topic models, and even word representation models (e.g., Word2Vec) in terms of several downstream tasks such as document classification, regression, and retrieval.


Exponential scaling of neural algorithms – a future beyond Moore’s Law?

Although the brain has long been considered a potential inspiration for future computing, Moore’s Law – the scaling property that has seen revolutions in technologies ranging from supercomputers to smart phones – has largely been driven by advances in materials science. As the ability to miniaturize transistors is coming to an end, there is increasing attention on new approaches to computation, including renewed enthusiasm around the potential of neural computation. Recent advances in neurotechnologies, many of which have been aided by computing’s rapid progression over recent decades, are now reigniting this opportunity to bring neural computation insights into broader computing applications. As we understand more about the brain, our ability to motivate new computing paradigms with continue to progress. These new approaches to computing, which we are already seeing in techniques such as deep learning, will themselves improve our ability to learn about the brain and accordingly can be projected to give rise to even further insights. Such a positive feedback has the potential to change the complexion of how computing sciences and neurosciences interact, and suggests that the next form of exponential scaling in computing may emerge from our progressive understanding of the brain.


A Survey of Shortest-Path Algorithms

A shortest-path algorithm finds a path containing the minimal cost between two vertices in a graph. A plethora of shortest-path algorithms is studied in the literature that span across multiple disciplines. This paper presents a survey of shortest-path algorithms based on a taxonomy that is introduced in the paper. One dimension of this taxonomy is the various flavors of the shortest-path problem. There is no one general algorithm that is capable of solving all variants of the shortest-path problem due to the space and time complexities associated with each algorithm. Other important dimensions of the taxonomy include whether the shortest-path algorithm operates over a static or a dynamic graph, whether the shortest-path algorithm produces exact or approximate answers, and whether the objective of the shortest-path algorithm is to achieve time-dependence or is to only be goal directed. This survey studies and classifies shortest-path algorithms according to the proposed taxonomy. The survey also presents the challenges and proposed solutions associated with each category in the taxonomy.


A Bayesian Stochastic Approximation Method

Motivated by the goal of improving the efficiency of small sample design, we propose a novel Bayesian stochastic approximation method to estimate the root of a regression function. The method features adaptive local modelling and nonrecursive iteration. Strong consistency of the Bayes estimator is obtained. Simulation studies show that our method is superior in finite-sample performance to Robbins–Monro type procedures. Extensions to searching for extrema and a version of generalized multivariate quantile are presented.


SLDR-DL: A Framework for SLD-Resolution with Deep Learning

This paper introduces an SLD-resolution technique based on deep learning. This technique enables neural networks to learn from old and successful resolution processes and to use learnt experiences to guide new resolution processes. An implementation of this technique is named SLDR-DL. It includes a Prolog library of deep feedforward neural networks and some essential functions of resolution. In the SLDR-DL framework, users can define logical rules in the form of definite clauses and teach neural networks to use the rules in reasoning processes.


A vector linear programming approach for certain global optimization problems

Global optimization problems with a quasi-concave objective function and linear constraints are studied. We point out that various other classes of global optimization problems can be expressed in this way. We present two algorithms, which can be seen as slight modifications of Benson-type algorithms for multiple objective linear programs. The modification of the MOLP algorithms results into a more efficient treatment of the studied optimization problems. This paper generalizes and improves results of Schulz and Mittal on quasi-concave problems, Shao and Ehrgott on multiplicative linear programs and L\’ohne and Wagner on minimizing the difference f=g-h of two convex functions g, h where either g or h is polyhedral. Numerical examples are given and the results are compared with the global optimization software BARON.


Case studies in network community detection

Community structure describes the organization of a network into subgraphs that contain a prevalence of edges within each subgraph and relatively few edges across boundaries between subgraphs. The development of community-detection methods has occurred across disciplines, with numerous and varied algorithms proposed to find communities. As we present in this Chapter via several case studies, community detection is not just an ‘end game’ unto itself, but rather a step in the analysis of network data which is then useful for furthering research in the disciplinary domain of interest. These case-study examples arise from diverse applications, ranging from social and political science to neuroscience and genetics, and we have chosen them to demonstrate key aspects of community detection and to highlight that community detection, in practice, should be directed by the application at hand.


Machine Learning $\mathbb{Z}_{2}$ Quantum Spin Liquids with Quasi-particle Statistics

Many-body localization transition through pairwise correlations

Efficiently decodable codes for the binary deletion channel

Makespan Minimization via Posted Prices

A Workflow for Visual Diagnostics of Binary Classifiers using Instance-Level Explanations

A Cheeger-Buser-Type inequality on CW complexes

Zarankiewicz’s problem for semi-algebraic hypergraphs

Dynamic ASEP, duality and continuous $q^{-1}$-Hermite polynomials

Sharp Models on Dull Hardware: Fast and Accurate Neural Machine Translation Decoding on the CPU

Edges not in any monochromatic copy of a fixed graph

Analytic solutions to the coherent control of the Dirac equation and beyond

Surrogate-based Ensemble Grouping Strategies for Embedded Sampling-based Uncertainty Quantification

Barabanov norms, Lipschitz continuity and monotonicity for the max algebraic joint spectral radius

On Identifying Disaster-Related Tweets: Matching-based or Learning-based?

Regression Driven F–Transform and Application to Smoothing of Financial Time Series

Machine Comprehension by Text-to-Text Neural Question Generation

Approximation of corner polyhedra with families of intersection cuts

Senti17 at SemEval-2017 Task 4: Ten Convolutional Neural Network Voters for Tweet Polarity Classification

Inferring the Partial Correlation Structure of Allelic Effects and Incorporating it in Genome-wide Prediction

Adaptive Mirror Descent for Constrained Optimization

Adaptive Stochastic Mirror Descent for Constrained Optimization

Persistence Terrace for Topological Inference of Point Cloud Data

Streaming Algorithm for Euler Characteristic Curves of Multidimensional Images

Matrix Factorization with Side and Higher Order Information

Maximum vanishing subspace problem, CAT(0)-space relaxation, and block-triangularization of partitioned matrix

Schubert polynomials, 132-patterns, and Stanley’s conjecture

Ovoids of Generalized Quadrangles of Order $(q, q^2-q)$ and Delsarte Cocliques in Related Strongly Regular Graphs

Optimal Power Control and Scheduling under Hard Deadline Constraints for Continuous Fading Channels

Cross-lingual Distillation for Text Classification

Crowdsourcing Argumentation Structures in Chinese Hotel Reviews

Motion Prediction Under Multimodality with Conditional Stochastic Networks

A Probabilistic Model for Collaborative Filtering with Implicit and Explicit Feedback Data

A fundamental theorem of asset pricing for continuous time large financial markets in a two filtration setting

GRASS: Generative Recursive Autoencoders for Shape Structures

Optimizing the Finite Length Performance of Sparse Superposition Codes

Characterizing and Improving Stability in Neural Style Transfer

TALL: Temporal Activity Localization via Language Query

Phase Congruency Parameter Optimization for Enhanced Detection of Image Features for both Natural and Medical Applications

Blind Detection of Polar Codes

Fluctuations of the Empirical Measure of Freezing Markov Chains

Terminal-Pairability in $K_{n,n}$ revisited

Networks of reinforced stochastic processes: asymptotics for the empirical means

A Note on Hardness of Diameter Approximation

Joint estimation of genetic and parent-of-origin effects using RNA-seq data from human

A spectral approach for quenched limit theorems for random expanding dynamical systems

Joint RNN Model for Argument Component Boundary Detection

D2D User Selection For Simultaneous Spectrum Sharing And Energy Harvesting

Bridging between Computer and Robot Vision through Data Augmentation: a Case Study on Object Recognition

Part-based Deep Hashing for Large-scale Person Re-identification

Social Media Advertisement Outreach: Learning the Role of Aesthetics

Unified Embedding and Metric Learning for Zero-Exemplar Event Detection

Shrinking Horizon Model Predictive Control with Signal Temporal Logic Constraints under Stochastic Disturbances

Leontief Meets Shannon – Measuring the Complexity of the Economic System

A New Sparse and Robust Adaptive Lasso Estimator for the Independent Contamination Model

Lines in Euclidean Ramsey theory

Distributed Online Learning of Event Definitions

Discrete Modeling of Multi-Transmitter Neural Networks with Neuron Competition

Finite-time Consensus Protocols for Multi-dimensional Multi-agent Systems

Unsupervised learning of object landmarks by factorized spatial embeddings

Online Covering with Sum of $\ell_q$-Norm Objectives

Group invariance principles for causal generative models

Resource Allocation for Secure Full-Duplex OFDMA Radio Systems

S-OHEM: Stratified Online Hard Example Mining for Object Detection

A Polya Contagion Model for Networks

Power Allocation and Cooperative Diversity in Two-Way Non-Regenerative Cognitive Radio Networks

Data Readiness Levels

Distributed Task Encoding

In-place Parallel Super Scalar Samplesort (IPSSSSo)

Detecting Adversarial Samples Using Density Ratio Estimates

Computing Constrained Approximate Equilibria in Polymatrix Games

Sequential Attention

A Dissipation Theory for Three-Dimensional FDTD with Application to Stability Analysis and Subgridding

Mixing properties and central limit theorem for associated point processes

Slower deviations of the branching Brownian motion and of branching random walks

Pathwise differentiability of reflected diffusions in convex polyhedral domains

The Stochastic Matching Problem: Beating Half with a Non-Adaptive Algorithm

Probabilistically-Shaped Coded Modulation with Hard Decision Decoding for Coherent Optical Systems

Graph matching the matchable nodes when some nodes are unmatchable

Consistent Sensor, Relay, and Link Selection in Wireless Sensor Networks

Analysis and Design of Convolutional Networks via Hierarchical Tensor Decompositions

Fundamental Limits of Covert Communication over MIMO AWGN Channel

Deep Speaker: an End-to-End Neural Speaker Embedding System

A Time-Vertex Signal Processing Framework

Efficient Parallel Strategy Improvement for Parity Games

Building Morphological Chains for Agglutinative Languages

ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases

Fairness Incentives for Myopic Agents

Advertisements