Paranom: A Parallel Anomaly Dataset Generator

In this paper, we present Paranom, a parallel anomaly dataset generator. We discuss its design and provide brief experimental results demonstrating its usefulness in improving the classification correctness of LSTM-AD, a state-of-the-art anomaly detection model.

Greenhouse: A Zero-Positive Machine Learning System for Time-Series Anomaly Detection

This short paper describes our ongoing research on Greenhouse – a zero-positive machine learning system for time-series anomaly detection.

Precision and Recall for Range-Based Anomaly Detection

Classical anomaly detection is principally concerned with point-based anomalies, anomalies that occur at a single data point. In this paper, we present a new mathematical model to express range-based anomalies, anomalies that occur over a range (or period) of time.

Multivariate Bayesian Structural Time Series Model

This paper deals with inference and prediction for multiple correlated time series, where one has also the choice of using a candidate pool of contemporaneous predictors for each target series. Starting with a structural model for the time-series, Bayesian tools are used for model fitting, prediction, and feature selection, thus extending some recent work along these lines for the univariate case. The Bayesian paradigm in this multivariate setting helps the model avoid overfitting as well as capture correlations among the multiple time series with the various state components. The model provides needed flexibility to choose a different set of components and available predictors for each target series. The cyclical component in the model can handle large variations in the short term, which may be caused by external shocks. We run extensive simulations to investigate properties such as estimation accuracy and performance in forecasting. We then run an empirical study with one-step-ahead prediction on the max log return of a portfolio of stocks that involve four leading financial institutions. Both the simulation studies and the extensive empirical study confirm that this multivariate model outperforms three other benchmark models, viz. a model that treats each target series as independent, the autoregressive integrated moving average model with regression (ARIMAX), and the multivariate ARIMAX (MARIMAX) model.

BigRoots: An Effective Approach for Root-cause Analysis of Stragglers in Big Data System

Stragglers are commonly believed to have a great impact on the performance of big data system. However, the reason to cause straggler is complicated. Previous works mostly focus on straggler detection, schedule level optimization and coarse-grained cause analysis. These methods cannot provide valuable insights to help users optimize their programs. In this paper, we propose BigRoots, a general method incorporating both framework and system features for root-cause analysis of stragglers in big data system. BigRoots considers features from big data framework such as shuffle read/write bytes and JVM garbage collection time, as well as system resource utilization such as CPU, I/O and network, which is able to detect both internal and external root causes of stragglers. We verify BigRoots by injecting high resource utilization across different system components and perform case studies to analyze different workloads in Hibench. The experimental results demonstrate that BigRoots is effective to identify the root cause of stragglers and provide useful guidance for performance optimization.

Chameleon: A Hybrid Secure Computation Framework for Machine Learning Applications

We present Chameleon, a novel hybrid (mixed-protocol) framework for secure function evaluation (SFE) which enables two parties to jointly compute a function without disclosing their private inputs. Chameleon combines the best aspects of generic SFE protocols with the ones that are based upon additive secret sharing. In particular, the framework performs linear operations in the ring \mathbb{Z}_{2^l} using additively secret shared values and nonlinear operations using Yao’s Garbled Circuits or the Goldreich-Micali-Wigderson protocol. Chameleon departs from the common assumption of additive or linear secret sharing models where three or more parties need to communicate in the online phase: the framework allows two parties with private inputs to communicate in the online phase under the assumption of a third node generating correlated randomness in an offline phase. Almost all of the heavy cryptographic operations are precomputed in an offline phase which substantially reduces the communication overhead. Chameleon is both scalable and significantly more efficient than the ABY framework (NDSS’15) it is based on. Our framework supports signed fixed-point numbers. In particular, Chameleon’s vector dot product of signed fixed-point numbers improves the efficiency of mining and classification of encrypted data for algorithms based upon heavy matrix multiplications. Our evaluation of Chameleon on a 5 layer convolutional deep neural network shows 133x and 4.2x faster executions than Microsoft CryptoNets (ICML’16) and MiniONN (CCS’17), respectively.

‘Robust-squared’ Imputation Models Using BART

Examples of ‘doubly robust’ estimator for missing data include augmented inverse probability weighting (AIPWT) models (Robins et al., 1994) and penalized splines of propensity prediction (PSPP) models (Zhang and Little, 2009). Doubly-robust estimators have the property that, if either the response propensity or the mean is modeled correctly, a consistent estimator of the population mean is obtained. However, doubly-robust estimators can perform poorly when modest misspecification is present in both models (Kang and Schafer, 2007). Here we consider extensions of the AIPWT and PSPP models that use Bayesian Additive Regression Trees (BART; Chipman et al., 2010) to provide highly robust propensity and mean model estimation. We term these ‘robust-squared’ in the sense that the propensity score, the means, or both can be estimated with minimal model misspecification, and applied to the doubly-robust estimator. We consider their behavior via simulations where propensities and/or mean models are misspecified. We apply our proposed method to impute missing instantaneous velocity (delta-v) values from the 2014 National Automotive Sampling System Crashworthiness Data System dataset and missing Blood Alcohol Concentration values from the 2015 Fatality Analysis Reporting System dataset. We found that BART applied to PSPP and AIPWT, provides a more robust and efficient estimate compared to PSPP and AIPWT, with the BART-estimated propensity score combined with PSPP providing the most efficient estimator with close to nominal coverage.

An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos

Videos represent the primary source of information for surveillance applications and are available in large amounts but in most cases contain little or no annotation for supervised learning. This article reviews the state-of-the-art deep learning based methods for video anomaly detection and categorizes them based on the type of model and criteria of detection. We also perform simple studies to understand the different approaches and provide the criteria of evaluation for spatio-temporal anomaly detection.

Adaptive Graph Convolutional Neural Networks

Graph Convolutional Neural Networks (Graph CNNs) are generalizations of classical CNNs to handle graph data such as molecular data, point could and social networks. Current filters in graph CNNs are built for fixed and shared graph structure. However, for most real data, the graph structures varies in both size and connectivity. The paper proposes a generalized and flexible graph CNN taking data of arbitrary graph structure as input. In that way a task-driven adaptive graph is learned for each graph data while training. To efficiently learn the graph, a distance metric learning is proposed. Extensive experiments on nine graph-structured datasets have demonstrated the superior performance improvement on both convergence speed and predictive accuracy.

eCommerceGAN : A Generative Adversarial Network for E-commerce

E-commerce companies such as Amazon, Alibaba and Flipkart process billions of orders every year. However, these orders represent only a small fraction of all plausible orders. Exploring the space of all plausible orders could help us better understand the relationships between the various entities in an e-commerce ecosystem, namely the customers and the products they purchase. In this paper, we propose a Generative Adversarial Network (GAN) for orders made in e-commerce websites. Once trained, the generator in the GAN could generate any number of plausible orders. Our contributions include: (a) creating a dense and low-dimensional representation of e-commerce orders, (b) train an ecommerceGAN (ecGAN) with real orders to show the feasibility of the proposed paradigm, and (c) train an ecommerce-conditional-GAN (ec^2GAN) to generate the plausible orders involving a particular product. We propose several qualitative methods to evaluate ecGAN and demonstrate its effectiveness. The ec^2GAN is used for various kinds of characterization of possible orders involving a product that has just been introduced into the e-commerce system. The proposed approach ec^2GAN performs significantly better than the baseline in most of the scenarios.

Unsupervised Despeckling

Contrast and quality of ultrasound images are adversely affected by the excessive presence of speckle. However, being an inherent imaging property, speckle helps in tissue characterization and tracking. Thus, despeckling of the ultrasound images requires the reduction of speckle extent without any oversmoothing. In this letter, we aim to address the despeckling problem using an unsupervised deep adversarial approach. A despeckling residual neural network (DRNN) is trained with an adversarial loss imposed by a discriminator. The discriminator tries to differentiate between the despeckled images generated by the DRNN and the set of high-quality images. Further to prevent the developed DRNN from oversmoothing, a structural loss term is used along with the adversarial loss. Experimental evaluations show that the proposed DRNN is able to outperform the state-of-the-art despeckling approaches.

Expected Policy Gradients for Reinforcement Learning

We propose expected policy gradients (EPG), which unify stochastic policy gradients (SPG) and deterministic policy gradients (DPG) for reinforcement learning. Inspired by expected sarsa, EPG integrates (or sums) across actions when estimating the gradient, instead of relying only on the action in the sampled trajectory. For continuous action spaces, we first derive a practical result for Gaussian policies and quadric critics and then extend it to an analytical method for the universal case, covering a broad class of actors and critics, including Gaussian, exponential families, and reparameterised policies with bounded support. For Gaussian policies, we show that it is optimal to explore using covariance proportional to the matrix exponential of the scaled Hessian of the critic with respect to the actions. EPG also provides a general framework for reasoning about policy gradient methods, which we use to establish a new general policy gradient theorem, of which the stochastic and deterministic policy gradient theorems are special cases. Furthermore, we prove that EPG reduces the variance of the gradient estimates without requiring deterministic policies and with little computational overhead. Finally, we show that EPG outperforms existing approaches on six challenging domains involving the simulated control of physical systems.

Reasoning about Unforeseen Possibilities During Policy Learning

Methods for learning optimal policies in autonomous agents often assume that the way the domain is conceptualised—its possible states and actions and their causal structure—is known in advance and does not change during learning. This is an unrealistic assumption in many scenarios, because new evidence can reveal important information about what is possible, possibilities that the agent was not aware existed prior to learning. We present a model of an agent which both discovers and learns to exploit unforeseen possibilities using two sources of evidence: direct interaction with the world and communication with a domain expert. We use a combination of probabilistic and symbolic reasoning to estimate all components of the decision problem, including its set of random variables and their causal dependencies. Agent simulations show that the agent converges on optimal polices even when it starts out unaware of factors that are critical to behaving optimally.

A Polynomial Algorithm for Balanced Clustering via Graph Partitioning

The objective of clustering is to discover natural groups in datasets and to identify geometrical structures which might reside there, without assuming any prior knowledge on the characteristics of the data. The problem can be seen as detecting the inherent separations between groups of a given point set in a metric space governed by a similarity function. The pairwise similarities between all data objects form a weighted graph adjacency matrix which contains all necessary information for the clustering process, which can consequently be formulated as a graph partitioning problem. In this context, we propose a new cluster quality measure which uses the maximum spanning tree and allows us to compute the optimal clustering under the min-max principle in polynomial time. Our algorithm can be applied when a load-balanced clustering is required.

Blessing of dimensionality: mathematical foundations of the statistical physics of data

The concentration of measure phenomena were discovered as the mathematical background of statistical mechanics at the end of the XIX – beginning of the XX century and were then explored in mathematics of the XX-XXI centuries. At the beginning of the XXI century, it became clear that the proper utilisation of these phenomena in machine learning might transform the curse of dimensionality into the blessing of dimensionality. This paper summarises recently discovered phenomena of measure concentration which drastically simplify some machine learning problems in high dimension, and allow us to correct legacy artificial intelligence systems. The classical concentration of measure theorems state that i.i.d. random points are concentrated in a thin layer near a surface (a sphere or equators of a sphere, an average or median level set of energy or another Lipschitz function, etc.). The new stochastic separation theorems describe the thin structure of these thin layers: the random points are not only concentrated in a thin layer but are all linearly separable from the rest of the set, even for exponentially large random sets. The linear functionals for separation of points can be selected in the form of the linear Fisher’s discriminant. All artificial intelligence systems make errors. Non-destructive correction requires separation of the situations (samples) with errors from the samples corresponding to correct behaviour by a simple and robust classifier. The stochastic separation theorems provide us by such classifiers and a non-iterative (one-shot) procedure for learning.

Net2Vec: Quantifying and Explaining how Concepts are Encoded by Filters in Deep Neural Networks

In an effort to understand the meaning of the intermediate representations captured by deep networks, recent papers have tried to associate specific semantic concepts to individual neural network filter responses, where interesting correlations are often found, largely by focusing on extremal filter responses. In this paper, we show that this approach can favor easy-to-interpret cases that are not necessarily representative of the average behavior of a representation. A more realistic but harder-to-study hypothesis is that semantic representations are distributed, and thus filters must be studied in conjunction. In order to investigate this idea while enabling systematic visualization and quantification of multiple filter responses, we introduce the Net2Vec framework, in which semantic concepts are mapped to vectorial embeddings based on corresponding filter responses. By studying such embeddings, we are able to show that 1., in most cases, multiple filters are required to code for a concept, that 2., often filters are not concept specific and help encode multiple concepts, and that 3., compared to single filter activations, filter embeddings are able to better characterize the meaning of a representation and its relationship to other concepts.

Existence and continuity of solution trajectories of generalized equations with application in electronics
A detailed treatment of Doob’s theorem
Recognizing Material Properties from Images
Robust Propensity Score Computation Method based on Machine Learning with Label-corrupted Data
Convergence Analysis of Gradient Descent Algorithms with Proportional Updates
Deep In-GPU Experience Replay
Comparing heterogeneous entities using artificial neural networks of trainable weighted structural components and machine-learned activation functions
Visual and Semantic Knowledge Transfer for Large Scale Semi-supervised Object Detection
Moments in Time Dataset: one million videos for event understanding
A New Coding Paradigm for the Primitive Relay Channel
A Formalization of Kant’s Second Formulation of the Categorical Imperative
$\mathcal{NP}$-Completeness and Inapproximability of the Virtual Network Embedding Problem and Its Variants
Sufficient Conditions for the Tightness of Shannon’s Capacity Bounds for Two-Way Channels
Algebraic differential formulas for the shuffle, stuffle and duality relations of iterated integrals
Optimal Allocation of Series FACTS Devices in Large Scale Systems
A Benchmark for Breast Ultrasound Image Segmentation (BUSIS)
Known Boundary Emulation of Complex Computer Models
A note on strict functional covariate overlap in causal inference problems with high-dimensional covariates
Measure-valued spline curves: an optimal transport viewpoint
Risk-Averse Matchings over Uncertain Graph Databases
The Hypothesis About Expansion of Multiple Stratonovich Stochastic Integrals of Arbitrary Multiplicity
An Entropy Lower Bound for Non-Malleable Extractors
Stochastic global maximum principle for general mean-field forward backward control systems with jumps
Many-Body Localization in a finite-range Sachdev-Ye-Kitaev model
Fano Resonances in Flat Band Networks
A counterexample to De Pierro’s conjecture on the convergence of under-relaxed cyclic projections
Reduced critical processes for small populations
Replica symmetry breaking in trajectory space for diffusion in logarithmically correlated random potentials
FWLBP: A Scale Invariant Descriptor for Texture Classification
Supervised and Unsupervised Tumor Characterization in the Deep Learning Era
Eliciting Worker Preference for Task Completion
Combating Error Propagation in Window Decoding of Braided Convolutional Codes
Symmetry reduction for dynamic programming
Generalized Linear Models with Linear Constraints for Microbiome Compositional Data
Instance Map based Image Synthesis with a Denoising Generative Adversarial Network
FPT algorithms for embedding into low complexity graphic metrics
When Exploiting Individual User Preference Is Beneficial for Caching at Base Stations?
Translating Pro-Drop Languages with Reconstruction Models
Exploiting structure of chance constrained programs via submodularity
Asymptotically Optimal Scheduling Algorithm for Compute-and-Forward
More Adaptive Algorithms for Adversarial Bandits
Improved Time of Arrival measurement model for non-convex optimization
Single-photon quantum filtering with multiple measurements
Markovian tricks for non-Markovian tree: contour processes, extinction and scaling limits
A Composition Theorem via Conflict Complexity
Convergence of Pascal-Like Triangles in Parry-Bertrand Numeration Systems
Simultaneous Tensor Completion and Denoising by Noise Inequality Constrained Convex Optimization
Shapley effects for sensitivity analysis with dependent inputs: bootstrap and kriging-based algorithms
Probabilistic performance estimators for computational chemistry methods: the empirical cumulative distribution function of absolute errors
On minimal actions of countable groups
No eigenvalues outside the limiting support of the spectral distribution of general sample covariance matrices
Weakly Supervised One-Shot Detection with Attention Siamese Networks
Gaussian fluctuations for linear spectral statistics of Wigner beta ensembles
Applications of deep learning to relativistic hydrodynamics
BSDE formulation of combined regular and singular stochastic control problems
Distribution of the absolute indicator of random Boolean functions
Fooling End-to-end Speaker Verification by Adversarial Examples
The critical 1-arm exponent for the ferromagnetic Ising model on the Bethe lattice
Optimal functional supervised classification with separation condition
Statistical Blockage Modeling and Robustness of Beamforming in Millimeter Wave Systems
Side Disks of a Spherical Great Polygon
Mean-field backward stochastic differential equations and applications
A triple comparison between anticipating stochastic integrals in financial modeling
Planning with Pixels in (Almost) Real Time
Axiomatizations of inconsistency indices for triads
Multicanonical Sampling of the Space of States of H(2,n)-Vector Models
Mean-Field Delayed BSDEs with Jumps
Effects of heterogeneous social interactions on flocking dynamics
Data-driven forecasting of solar irradiance
On Maximally Recoverable Codes for Product Topologies
Asymptotics of the density of parabolic Anderson random fields
Characterizing subclasses of cover-incomparability graphs by forbidden subposets
Age of Information: Whittle Index for Scheduling Stochastic Arrivals
A Smoothed Analysis of the Greedy Algorithm for the Linear Contextual Bandit Problem
Inferring a Third Spatial Dimension from 2D Histological Images
On the determinants and permanents of matrices with restricted entries over prime fields
Approximation beats concentration? An approximation view on inference with smooth radial kernels
A class of $L_1$-to-$L_1$ and $L_\infty$-to-$L_\infty$ interval observers for (delayed) Markov jump linear systems
Unsupervised Real-to-Virtual Domain Unification for End-to-End Highway Driving
Sequential decomposition of dynamic games with asymmetric information and dependent states
MilkQA: a Dataset of Consumer Questions for the Task of Answer Selection
Online Maximum Matching with Recourse
Stability analysis and state-feedback control of LPV systems with piecewise constant parameters subject to spontaneous Poissonian jumps
Strong existence and uniqueness for stable stochastic differential equations with distributional drift
Gaussian Latent Factor Analysis under Graphical Constraints
Buying Online – A Characterization of Rational Buying Procedures
Focus: Querying Large Video Datasets with Low Latency and Low Cost
On the Secure Degrees of Freedom of the K-user Interference Channel with Delayed CSIT