PMLB: A Large Benchmark Suite for Machine Learning Evaluation and Comparison

The selection, development, or comparison of machine learning methods in data mining can be a difficult task based on the target problem and goals of a particular study. Numerous publicly available real-world and simulated benchmark datasets have emerged from different sources, but their organization and adoption as standards have been inconsistent. As such, selecting and curating specific benchmarks remains an unnecessary burden on machine learning practitioners and data scientists. The present study introduces an accessible, curated, and developing public benchmark resource to facilitate identification of the strengths and weaknesses of different machine learning methodologies. We compare meta-features among the current set of benchmark datasets in this resource to characterize the diversity of available data. Finally, we apply a number of established machine learning methods to the entire benchmark suite and analyze how datasets and algorithms cluster in terms of performance. This work is an important first step towards understanding the limitations of popular benchmarking suites and developing a resource that connects existing benchmarking standards to more diverse and efficient standards in the future.

Human Interaction with Recommendation Systems: On Bias and Explorationn

Recommendation systems rely on historical user data to provide suggestions. We propose an explicit and simple model for the interaction between users and recommendations provided by a platform, and relate this model to the multi-armed bandit literature. First, we show that this interaction leads to a bias in naive estimators due to selection effects. This bias leads to suboptimal outcomes, which we quantify in terms of linear regret. We end the first part by discussing ways to obtain unbiased estimates. The second part of this work considers exploration of alternatives. We show that although agents are myopic, agents’ heterogeneous preferences ensure that recommendation systems ‘learn’ about all alternatives without explicitly incentivizing this exploration. This work provides new and practical insights relevant to a wide range of systems designed to help users make better decisions.

Evolving Deep Neural Networks

The success of deep learning depends on finding an architecture to fit the task. As deep learning has scaled up to more challenging tasks, the architectures have become difficult to design by hand. This paper proposes an automated method, CoDeepNEAT, for optimizing deep learning architectures through evolution. By extending existing neuroevolution methods to topology, components, and hyperparameters, this method achieves results comparable to best human designs in standard benchmarks in object recognition and language modeling. It also supports building a real-world application of automated image captioning on a magazine website. Given the anticipated increases in available computing power, evolution of deep networks is promising approach to constructing deep learning applications in the future.

Change Detection under Global Viewpoint Uncertainty

This paper addresses the problem of change detection from a novel perspective of long-term map learning. We are particularly interested in designing an approach that can scale to large maps and that can function under global uncertainty in the viewpoint (i.e., GPS-denied situations). Our approach, which utilizes a compact bag-of-words (BoW) scene model, makes several contributions to the problem: 1) Two kinds of prior information are extracted from the view sequence map and used for change detection. Further, we propose a novel type of prior, called motion prior, to predict the relative motions of stationary objects and anomaly ego-motion detection. The proposed prior is also useful for distinguishing stationary from non-stationary objects. 2) A small set of good reference images (e.g., 10) are efficiently retrieved from the view sequence map by employing the recently developed Bag-of-Local-Convolutional-Features (BoLCF) scene model. 3) Change detection is reformulated as a scene retrieval over these reference images to find changed objects using a novel spatial Bag-of-Words (SBoW) scene model. Evaluations conducted of individual techniques and also their combinations on a challenging dataset of highly dynamic scenes in the publicly available Malaga dataset verify their efficacy.

Scattertext: a Browser-Based Tool for Visualizing how Corpora Differ

Scattertext is an open source tool for visualizing linguistic variation between document categories in a language-independent way. The tool presents a scatterplot, where each axis corresponds to the rank-frequency a term occurs in a category of documents. Through a tie-breaking strategy, the tool is able to display thousands of visible term-representing points and find space to legibly label hundreds of them. Scattertext also lends itself to a query-based visualization of how the use of terms with similar embeddings differs between document categories, as well as a visualization for comparing the importance scores of bag-of-words features to univariate metrics.

Introduction to Nonnegative Matrix Factorization

In this paper, we introduce and provide a short overview of nonnegative matrix factorization (NMF). Several aspects of NMF are discussed, namely, the application in hyperspectral imaging, geometry and uniqueness of NMF solutions, complexity, algorithms, and its link with extended formulations of polyhedra. In order to put NMF into perspective, the more general problem class of constrained low-rank matrix approximation problems is first briefly introduced.

A Robust Adaptive Stochastic Gradient Method for Deep Learning

Stochastic gradient algorithms are the main focus of large-scale optimization problems and led to important successes in the recent advancement of the deep learning algorithms. The convergence of SGD depends on the careful choice of learning rate and the amount of the noise in stochastic estimates of the gradients. In this paper, we propose an adaptive learning rate algorithm, which utilizes stochastic curvature information of the loss function for automatically tuning the learning rates. The information about the element-wise curvature of the loss function is estimated from the local statistics of the stochastic first order gradients. We further propose a new variance reduction technique to speed up the convergence. In our experiments with deep neural networks, we obtained better performance compared to the popular stochastic gradient algorithms.

Meta Networks

Deep neural networks have been successfully applied in applications with a large amount of labeled data. However, there are major drawbacks of the neural networks that are related to rapid generalization with small data and continual learning of new concepts without forgetting. We present a novel meta learning method, Meta Networks (MetaNet), that acquires a meta-level knowledge across tasks and shifts its inductive bias via fast parameterization for the rapid generalization. When tested on the standard one-shot learning benchmarks, our MetaNet models achieved near human-level accuracy. We demonstrated several appealing properties of MetaNet relating to generalization and continual learning.

Being Robust (in High Dimensions) Can Be Practical

Robust estimation is much more challenging in high dimensions than it is in one dimension: Most techniques either lead to intractable optimization problems or estimators that can tolerate only a tiny fraction of errors. Recent work in theoretical computer science has shown that, in appropriate distributional models, it is possible to robustly estimate the mean and covariance with polynomial time algorithms that can tolerate a constant fraction of corruptions, independent of the dimension. However, the sample and time complexity of these algorithms is prohibitively large for high-dimensional applications. In this work, we address both of these issues by establishing sample complexity bounds that are optimal, up to logarithmic factors, as well as giving various refinements that allow the algorithms to tolerate a much larger fraction of corruptions. Finally, we show on both synthetic and real data that our algorithms have state-of-the-art performance and suddenly make high-dimensional robust estimation a realistic possibility.

Deterministic Distributed Matching: Simpler, Faster, Better

We present improved deterministic distributed algorithms for a number of well-studied matching problems, which are simpler, faster, more accurate, and/or more general than their known counterparts. The common denominator of these results is a deterministic distributed rounding method for certain linear programs, which is the first such rounding method, to our knowledge. A sampling of our end results is as follows: — An O(\log^2 \Delta \log n)-round deterministic distributed algorithm for computing a maximal matching, in n-node graphs with maximum degree \Delta. This is the first improvement in about 20 years over the celebrated O(\log^4 n)-round algorithm of Hanckowiak, Karonski, and Panconesi [SODA’98, PODC’99]. — An O(\log^2 \Delta \log \frac{1}{\varepsilon} + \log^ * n)-round deterministic distributed algorithm for a (2+\varepsilon)-approximation of maximum matching. This is exponentially faster than the classic O(\Delta +\log^* n)-round 2-approximation of Panconesi and Rizzi [DIST’01]. With some modifications, the algorithm can also find an almost maximal matching which leaves only an \varepsilon-fraction of the edges on unmatched nodes. — An O(\log^2 \Delta \log \frac{1}{\varepsilon} \log_{1+\varepsilon} W + \log^ * n)-round deterministic distributed algorithm for a (2+\varepsilon)-approximation of a maximum weighted matching, and also for the more general problem of maximum weighted b-matching. Here, W denotes the maximum normalized weight. These improve over the O(\log^4 n \log_{1+\varepsilon} W)-round (6+\varepsilon)-approximation algorithm of Panconesi and Sozio [DIST’10].

Reactive Trajectory Generation in an Unknown Environment

Confidence Bands for Coefficients in High Dimensional Linear Models with Error-in-variables

Multidimensional Sampling of Isotropically Bandlimited Signals

Reinforcement Learning for Pivoting Task

A Distortion Based Approach for Protecting Inferences

Truth and Regret in Online Scheduling

Centered Sobolev inequality and exponential convergence in $Φ$-entropy

Chvátal’s conjecture for downsets of small rank

Making 360° Video Watchable in 2D: Learning Videography for Click Free Viewing

Infinity-Norm Permutation Covering Codes from Cyclic Groups

Learning Social Affordance Grammar from Videos: Transferring Human Interactions to Human-Robot Interactions

Non asymptotic distributional bounds for the Dickman Approximation of the running time of the Quickselect algorithm

Exponential convergence in the Wasserstein metric $W_1$ for one dimensional diffusions

Identifying leading indicators of product recalls from online reviews using positive unlabeled learning and domain adaptation

Understanding Synthetic Gradients and Decoupled Neural Interfaces

ISIC 2017 – Skin Lesion Analysis Towards Melanoma Detection

Centralized Network Utility Maximization over Aggregate Flows

Stability and optimality of distributed secondary frequency control schemes in power networks

Spatial evolution of human dialects

Skin cancer reorganization and classification with deep neural network

Unsupervised Ensemble Ranking of Terms in Electronic Health Record Notes Based on Their Importance to Patients

Learning Determinantal Point Processes with Moments and Cycles

A note on the approximate admissibility of regularized estimators in the Gaussian sequence model

Simplified Algorithmic Metatheorems Beyond MSO: Treewidth and Neighborhood Diversity

Label Refinement Network for Coarse-to-Fine Semantic Segmentation

A Deep Cascade of Convolutional Neural Networks for MR Image Reconstruction

Conversion Rate Optimization through Evolutionary Computation

Diffusion Independent Semi-Bandit Influence Maximization

Optimal Topology Design for Disturbance Minimization in Power Grids

An Analytical Formula of Population Gradient for two-layered ReLU network and its Applications in Convergence and Critical Point Analysis

Signal-based Bayesian Seismic Monitoring

MoleculeNet: A Benchmark for Molecular Machine Learning

Structural Embedding of Syntactic Trees for Machine Comprehension

Generalization and Equilibrium in Generative Adversarial Nets (GANs)

On the NP-hardness of scheduling with time restrictions

Skin Lesion Analysis Towards Melanoma Detection Using Deep Learning Network

Active Learning for Accurate Estimation of Linear Models

A novel image tag completion method based on convolutional neural network

Positive-Unlabeled Learning with Non-Negative Risk Estimator

The Second Order Linear Model

Marcinkiewicz’s strong law of large numbers for non-additive expectation

Coloring ($P_6$, diamond, $K_4$)-free graphs

Discovery of Evolving Semantics through Dynamic Word Embedding Learning

Impact of Optimal Storage Allocation on Price Volatility in Electricity Markets

The generalized k-resultant modulus set problem in finite fields

In Search of an Entity Resolution OASIS: Optimal Asymptotic Sequential Importance Sampling

The pitfalls of planar spin-glass benchmarks: Raising the bar for quantum annealers (again)

The RowHammer Problem and Other Issues We May Face as Memory Becomes Denser

A Dominant Strategy Truthful, Deterministic Multi-Armed Bandit Mechanism with Logarithmic Regret

Faster truncated integer multiplication

Learning Mixtures of Sparse Linear Regressions Using Sparse Graph Codes

TumorNet: Lung Nodule Characterization Using Multi-View Convolutional Neural Network with Gaussian Process

Inference for Multiple Change-points in Linear and Non-linear Time Series Models

Nonparametric estimation of galaxy cluster’s emissivity and point source detection in astrophysics with two lasso penalties

Time-varying Bang-bang Property of Minimal Controls for Approximately Null-controllable Heat Equations

Traffic-Aware Transmission Mode Selection in D2D-enabled Cellular Networks with Token System

On a User-Centric Base Station Cooperation Scheme for Reliable Communications

Pathwise uniqueness for a class of SPDEs driven by cylindrical $α$-stable processes

Vertex-quasiprimitive $2$-arc-transitive digraphs

A resource-frugal probabilistic dictionary and applications in bioinformatics

Adaptive Matching for Expert Systems with Uncertain Task Types

A Unifying View of Explicit and Implicit Feature Maps for Structured Data: Systematic Studies of Graph Kernels

The Rohde–Schramm theorem, via the Gaussian free field

Parity Games, Imperfect Information and Structural Complexity

Rationality of the zeta function of the subgroups of abelian $p$-groups

BoxCars: Improving Vehicle Fine-Grained Recognition using 3D Bounding Boxes in Traffic Surveillance

Even faster sorting of (not only) integers

Even better correction of genome sequencing data

Artificial Noise-Aided Biobjective Transmitter Optimization for Service Integration in Multi-User MIMO Gaussian Broadcast Channel

Exact algorithms for the picking problem

Unveiling Bias Compensation in Turbo-Based Algorithms for (Discrete) Compressed Sensing

Fixing number of co-noraml product of graphs

Wireless Power Transfer for Distributed Estimation in Sensor Networks

Secrecy and Robustness for Active Attack in Secure Network Coding

Mixing Complexity and its Applications to Neural Networks

Distributed Bayesian Matrix Factorization with Minimal Communication

Wireless Interference Identification with Convolutional Neural Networks

Comparison of Lasserre’s measure–based bounds for polynomial optimization to bounds obtained by simulated annealing

Peterson-Gorenstein-Zierler algorithm for skew RS codes

A yield-cost tradeoff governs Escherichia coli’s decision between fermentation and respiration in carbon-limited growth

Predicting Rankings of Software Verification Competitions

Sampling Variations of Lead Sheets

On a class of constacyclic codes over the non-principal ideal ring $\mathbb{Z}_{p^s}+u\mathbb{Z}_{p^s}$

Hankel determinants of harmonic numbers and related topics

Attentive Recurrent Comparators

Particle picture representation of the non-symmetric Rosenblatt process and Hermite processes of any order

Lock-Free Parallel Perceptron for Graph-based Dependency Parsing

A Generic Online Parallel Learning Framework for Large Margin Models

Linearly constrained Gaussian processes

Robust Spatial Filtering with Graph Convolutional Neural Networks

Unsupervised Steganalysis Based on Artificial Training Sets

A Simple, Fast and Fully Automated Approach for Midline Shift Measurement on Brain Computed Tomography

Opening the Black Box of Deep Neural Networks via Information

Sandpiles on the square lattice

General and Robust Communication-Efficient Algorithms for Distributed Clustering

Face Image Reconstruction from Deep Templates

SLIM: Semi-Lazy Inference Mechanism for Plan Recognition

Encrypted accelerated least squares regression

Renormalized asymptotic enumeration of Feynman diagrams

Towards CNN Map Compression for camera relocalisation

Exact Topology Reconstruction of Radial Dynamical Systems with Applications to Distribution System of the Power Grid

Unsupervised Image-to-Image Translation Networks

Wireless Node Cooperation with Resource Availability Constraints

Learning the Structure of Generative Models without Labeled Data

Araguaia Medical Vision Lab at ISIC 2017 Skin Lesion Classification Challenge

On the minimum trace norm of (0,1)-matrices

Binarized Convolutional Landmark Localizers for Human Pose Estimation and Face Alignment with Limited Resources

The Unreasonable Effectiveness of Random Orthogonal Embeddings

On Certain Properties of Convex Functions

Using Synthetic Data to Train Neural Networks is Model-Based Reasoning

Bootstrap confidence sets for spectral projectors of sample covariance

Second order necessary and sufficient optimality conditions for singular solutions of partially-affine control problems

A Dichotomy for Sampling Barrier-Crossing Events of Random Walks with Regularly Varying Tails

Gowers norms control Diophantine inequalities

How to Escape Saddle Points Efficiently

Quantum Harmonic Analysis of the Density Matrix: Basics

Global Multiple SLEs for $κ\leq 4$ and Connection Probabilities for Level Lines of GFF