PANFIS++: A Generalized Approach to Evolving Learning

The concept of evolving intelligent system (EIS) provides an effective avenue for data stream mining because it is capable of coping with two prominent issues: online learning and rapidly changing environments. We note at least three uncharted territories of existing EISs: data uncertainty, temporal system dynamic, redundant data streams. This book chapter aims at delivering a concrete solution of this problem with the algorithmic development of a novel learning algorithm, namely PANFIS++. PANFIS++ is a generalized version of the PANFIS by putting forward three important components: 1) An online active learning scenario is developed to overcome redundant data streams. This module allows to actively select data streams for the training process, thereby expediting execution time and enhancing generalization performance, 2) PANFIS++ is built upon an interval type-2 fuzzy system environment, which incorporates the so-called footprint of uncertainty. This component provides a degree of tolerance for data uncertainty. 3) PANFIS++ is structured under a recurrent network architecture with a self-feedback loop. This is meant to tackle the temporal system dynamic. The efficacy of the PANFIS++ has been numerically validated through numerous real-world and synthetic case studies, where it delivers the highest predictive accuracy while retaining the lowest complexity.

DCA: Dynamic Correlation Analysis

In high-throughput data, dynamic correlation between genes, i.e. changing correlation patterns under different biological conditions, can reveal important regulatory mechanisms. Given the complex nature of dynamic correlation, and the underlying conditions for dynamic correlation may not manifest into clinical observations, it is difficult to recover such signal from the data. Current methods seek underlying conditions for dynamic correlation by using certain observed genes as surrogates, which may not faithfully represent true latent conditions. In this study we develop a new method that directly identifies strong latent signals that regulate the dynamic correlation of many pairs of genes, named DCA: Dynamic Correlation Analysis. At the center of the method is a new metric for the identification of gene pairs that are highly likely to be dynamically correlated, without knowing the underlying conditions of the dynamic correlation. We validate the performance of the method with extensive simulations. In real data analysis, the method reveals novel latent factors with clear biological meaning, bringing new insights into the data.

A Study and Comparison of Human and Deep Learning Recognition Performance Under Visual Distortions

Deep neural networks (DNNs) achieve excellent performance on standard classification tasks. However, under image quality distortions such as blur and noise, classification accuracy becomes poor. In this work, we compare the performance of DNNs with human subjects on distorted images. We show that, although DNNs perform better than or on par with humans on good quality images, DNN performance is still much lower than human performance on distorted images. We additionally find that there is little correlation in errors between DNNs and human subjects. This could be an indication that the internal representation of images are different between DNNs and the human visual system. These comparisons with human performance could be used to guide future development of more robust DNNs.

Machine Learning with World Knowledge: The Position and Survey

Machine learning has become pervasive in multiple domains, impacting a wide variety of applications, such as knowledge discovery and data mining, natural language processing, information retrieval, computer vision, social and health informatics, ubiquitous computing, etc. Two essential problems of machine learning are how to generate features and how to acquire labels for machines to learn. Particularly, labeling large amount of data for each domain-specific problem can be very time consuming and costly. It has become a key obstacle in making learning protocols realistic in applications. In this paper, we will discuss how to use the existing general-purpose world knowledge to enhance machine learning processes, by enriching the features or reducing the labeling work. We start from the comparison of world knowledge with domain-specific knowledge, and then introduce three key problems in using world knowledge in learning processes, i.e., explicit and implicit feature representation, inference for knowledge linking and disambiguation, and learning with direct or indirect supervision. Finally we discuss the future directions of this research topic.

TaskUniVerse: A Task-Based Unified Interface for Versatile Parallel Execution

Task based parallel programming has shown competitive outcomes in many aspects of parallel programming such as efficiency, performance, productivity and scalability. Different approaches are used by different software development frameworks to provide these outcomes to the programmer, while making the underlying hardware architecture transparent to her. However, since programs are not portable between these frameworks, using one framework or the other is still a vital decision by the programmer whose concerns are expandability, adaptivity, maintainability and interoperability of the programs. In this work, we propose a unified programming interface that a programmer can use for working with different task based parallel frameworks transparently. In this approach we abstract the common concepts of task based parallel programming and provide them to the programmer in a single programming interface uniformly for all frameworks. We have tested the interface by running programs which implement matrix operations within frameworks that are optimized for shared and distributed memory architectures and accelerators, while the cooperation between frameworks is configured externally with no need to modify the programs. Further possible extensions of the interface and future potential research are also described.

Non-negative Matrix Factorization via Archetypal Analysis

Given a collection of data points, non-negative matrix factorization (NMF) suggests to express them as convex combinations of a small set of `archetypes’ with non-negative entries. This decomposition is unique only if the true archetypes are non-negative and sufficiently sparse (or the weights are sufficiently sparse), a regime that is captured by the separability condition and its generalizations. In this paper, we study an approach to NMF that can be traced back to the work of Cutler and Breiman (1994) and does not require the data to be separable, while providing a generally unique decomposition. We optimize the trade-off between two objectives: we minimize the distance of the data points from the convex envelope of the archetypes (which can be interpreted as an empirical risk), while minimizing the distance of the archetypes from the convex envelope of the data (which can be interpreted as a data-dependent regularization). The archetypal analysis method of (Cutler, Breiman, 1994) is recovered as the limiting case in which the last term is given infinite weight. We introduce a `uniqueness condition’ on the data which is necessary for exactly recovering the archetypes from noiseless data. We prove that, under uniqueness (plus additional regularity conditions on the geometry of the archetypes), our estimator is robust. While our approach requires solving a non-convex optimization problem, we find that standard optimization methods succeed in finding good solutions both for real and synthetic data.

Noisy independent component analysis of auto-correlated components

CaloGAN: Simulating 3D High Energy Particle Showers in Multi-Layer Electromagnetic Calorimeters with Generative Adversarial Networks

Solving (most) of a set of quadratic equalities: Composite optimization for robust phase retrieval

Transmit Array Interpolation for DOA Estimation via Tensor Decomposition in 2D MIMO Radar

Supervised Learning of Universal Sentence Representations from Natural Language Inference Data

Exploration of Large Networks via Fast and Universal Latent Space Model Fitting

Parameter-dependent Stochastic Optimal Control in Finite Discrete Time

Noncentral limit theorem for the generalized Rosenblatt process

Ramsey properties and extending partial automorphisms for classes of finite structures

On Algorithms for $L$-bounded Cut Problem

Bayesian Ensembles of Binary-Event Forecasts

Weights at the Bottom Matter When the Top is Heavy

Optimal Power Control and Scheduling for Real-Time and Non-Real-Time Data

Face Detection, Bounding Box Aggregation and Pose Estimation for Robust Facial Landmark Localisation in the Wild

DeepCorrect: Correcting DNN models against Image Distortions

Knowledge-Guided Deep Fractal Neural Networks for Human Pose Estimation

Max-Pooling Loss Training of Long Short-Term Memory Networks for Small-Footprint Keyword Spotting

A comprehensive study of batch construction strategies for recurrent neural networks in MXNet

Generalized permutahedra to Grothendieck polynomials via flow polytopes

An Ensemble Approach to Predicting the Impact of Vaccination on Rotavirus Disease in Niger

A Passivity-Based Approach to Nash Equilibrium Seeking over Networks

Analogical Inference for Multi-Relational Embeddings

Deep Patch Learning for Weakly Supervised Object Classification and Discovery

Sparse Representation-based Open Set Recognition

Nonlinear Information Bottleneck

Face Super-Resolution Through Wasserstein GANs

Comments on `High-dimensional simultaneous inference with the bootstrap’

On Continuity Properties for Infinite Rectangle Packing

On human motion prediction using recurrent neural networks

A Generative Model of a Pronunciation Lexicon for Hindi

Density Evolution on a Class of Smeared Random Graphs: A Theoretical Framework for Fast MRI

Millimeter Wave Channel Estimation via Exploiting Joint Sparse and Low-Rank Structures

Sequential Double Robustness in Right-Censored Longitudinal Models

Image Annotation using Multi-Layer Sparse Coding

Distributed, scalable and gossip-free consensus optimization with application to data analysis

Periodic Steiner graphs minimizing length

Price sensitivities for a general stochastic volatility model

Metacognitive Learning Approach for Online Tool Condition Monitoring

On the maximum and minimum multiplicative Zagreb indices of graphs with given number of cut edges

Proper vertex connection and graph operations

Total proper connection and graph operations

Tight Lower Bounds on the Contact Distance Distribution in Poisson Hole Process

Learning Distributed Representations of Texts and Entities from Knowledge Base

Pascal Triangle and Restricted Words

Linearized ADMM for Non-convex Non-smooth Optimization with Convergence Analysis

Context-Aware Trajectory Prediction

Smooth approximation of Yang–Mills theory on $\mathbb{R}^{2}$: a rough path approach

A generalized Gaussian process model for computer experiments with binary time series

Epistemic Model Checking of Atomic Commitment Protocols with Byzantine Failures

Exploring Latent Semantic Factors to Find Useful Product Reviews

Item Recommendation with Evolving User Preferences and Experience

Status Updates Over Unreliable Multiaccess Channels

People on Drugs: Credibility of User Statements in Health Communities

Largest reduced neighborhood clique cover number revisited

Small partial Latin squares that embed in an infinite group but not into any finite group

Deep Visual Attention Prediction

Experimental results : Reinforcement Learning of POMDPs using Spectral Methods

Performance Limits on the Classification of Kronecker-structured Models

Learning Discriminative Relational Features for Sequence Labeling

A Design Methodology for Efficient Implementation of Deconvolutional Neural Networks on an FPGA

On the structure of large sum-free sets of integers

Horizontal Product Differentiation in Varian’s Model of Sales

On the boundary theory of subordinate killed Lévy processes

Simultaneous Super-Resolution and Cross-Modality Synthesis of 3D Medical Images using Weakly-Supervised Joint Convolutional Sparse Coding

Emptiness Problems for Distributed Automata

Incremental DFS algorithms: a theoretical and experimental study

A New Medical Diagnosis Method Based on Z-Numbers

Learning of Gaussian Processes in Distributed and Communication Limited Systems

TrajectoryNet: An Embedded GPS Trajectory Representation for Point-based Classification Using Recurrent Neural Networks

Codes for Graph Erasures

DropIn: Making Reservoir Computing Neural Networks Robust to Missing Inputs by Dropout

The growing ratios of hyperbolic regular mosaics with bounded cells

Towards Applying the OPRA Theory to Shape Similarity

SOS for bounded rationality

Homomesy in products of three chains and multidimensional recombination

Leveraging Joint Interactions for Credibility Analysis in News Communities with Continuous Conditional Random Field

Credible Review Detection with Limited Information using Consistency Analysis

Continuous Experience-aware Language Model for Recommender Systems using Brownian Motion

Metacontrol for Adaptive Imagination-Based Optimization

Lightweight Robust Framework for Workload Scheduling in Clouds

Applications of some special numbers obtained from a difference equation of degree three

Large scale digital prostate pathology image analysis combining feature extraction and deep neural network

A Concentration Inequality Based Methodology for Sparse Covariance Estimation

Handwritten Bangla Digit Recognition Using Deep Learning

Finding Bottlenecks: Predicting Student Attrition with Unsupervised Classifier

AirDraw: Leveraging Smart Watch Motion Sensors for Mobile Human Computer Interactions

Gaps in full homomorphism order

A bijective proof of Straub’s conjecture on the number of $(s, s+2)$-core partitions with distinct parts

Multimodal Affect Analysis for Product Feedback Assessment

Spatiotemporal Recurrent Convolutional Networks for Traffic Prediction in Transportation Networks

Generating Memorable Mnemonic Encodings of Numbers

Egocentric Effective Conductance Centrality and General Degree

Linear Network Coding for Two-Unicast-Z Networks: A Commutative Algebraic Perspective and Fundamental Limits

On Time-Reversal Imaging by Statistical Testing

On the optimality of some group testing algorithms

Residual Diffusivity in Elephant Random Walk Models with Stops

Regularized Spatial Maximum Covariance Analysis

Integrated PV Charging of EV Fleet Based on Dynamic Energy Prices and Offer of Reserves

Joint Trajectory and Communication Design for Multi-UAV Enabled Wireless Networks

Automatic Recognition of Mammal Genera on Camera-Trap Images using Multi-Layer Robust Principal Component Analysis and Mixture Neural Networks

Respondent driven sampling and sparse graph convergence

Combating Human Trafficking with Deep Multimodal Models

Multiple Imputation Using Deep Denoising Autoencoders

Polarization Shift Keying (PolarSK): System Scheme and Performance Analysis

Quantifying non-monotonicity of functions and the lack of positivity in signed measures

ChineseFoodNet: A large-scale Image Dataset for Chinese Food Recognition

Pricing of Tiered cloud storage via two-stage, latency-aware bidding

Pathwise continuous time spectrum degeneracy at a single point and weak predictability

Computing an Approximately Optimal Agreeable Set of Items

Density Estimation for Geolocation via Convolutional Mixture Density Network

High-Level Concepts for Affective Understanding of Images

An O(n\log n)-Time Algorithm for the k-Center Problem in Trees

Optimizing Pilot Overhead for Ultra-Reliable Short-Packet Transmission

What Can Help Pedestrian Detection?

Deep Descriptor Transforming for Image Co-Localization

On toric face rings I

On toric face rings II

Covariance structure associated with equality between two general ridge estimators

Integral and measure-turnpike properties for infinite-dimensional optimal control systems

Optimal shape design for 2D heat equations in large time

‘Convex Until Proven Guilty’: Dimension-Free Acceleration of Gradient Descent on Non-Convex Functions

Inductive Freeness of Ziegler’s Canonical Multiderivations for Reflection Arrangements

Scene Text Eraser

The concurrence of form and function in developing networks: An explanation for synaptic pruning

Network Coherence Time Matters – Aligned Image Sets and the Degrees of Freedom of Interference Networks with Finite Precision CSIT and Perfect CSIR

The stochastic interpolation method: A simple scheme to prove replica formulas in Bayesian inference

Face Recognition Machine Vision System Using Eigenfaces

A local ensemble transform Kalman particle filter for convective scale data assimilation

Mnemonic Reader for Machine Comprehension

Directed Information as Privacy Measure in Cloud-based Control

Towards Reduced Instruction Sets for Synchronization

Finite-Blocklength Bounds on the Maximum Coding Rate of Rician Fading Channels with Applications to Pilot-Assisted Transmission

On degree sum conditions for 2-factors with a prescribed number of cycles

Finite-time attitude synchronization with distributed discontinuous protocols

Rank Vertex Cover as a Natural Problem for Algebraic Compression

Some regional control problems for population dynamics

Discriminant analysis in small and large dimensions

Faster tuple lattice sieving using spherical locality-sensitive filters

Random time changes of Feller processes

Discretisation of regularity structures

The Edge-Wiener Index, the Szeged Indices and the PI Index of Benzenoid Systems in Sub-Linear Time

Dynamics of homogeneous shear turbulence: A key role of the nonlinear transverse cascade in the bypass concept

Block-Parallel IDA* for GPUs (Extended Manuscript)

Semi-Markov models and motion in heterogeneous media

Goldberg, Fuller, Caspar, Klug and Coxeter and a general approach to local symmetry-preserving operations

Flat Parallelization

Geometric Properties of Isostables and Basins of Attraction of Monotone Systems

Video Processing for Barycenter Trajectory Identification in Diving

Random Inscribed Polytopes Have Similar Radius Functions as Poisson-Delaunay Mosaics

A Dual-Source Approach for 3D Human Pose Estimation from a Single Image

Proving Correctness of Concurrent Objects by Validating Linearization Points

Generative Cooperative Net for Image Generation and Data Augmentation

Geometry and Dynamics for Markov Chain Monte Carlo

Multi Resolution LSTM For Long Term Prediction In Neural Activity Video

Geometric GAN

Covariance Matrix Estimation in Massive MIMO

Lower Bounds for Asymptotic Consensus in Dynamic Networks

Teaching Concurrent Software Design: A Case Study Using Android

Keeping the Bad Guys Out: Protecting and Vaccinating Deep Learning with JPEG Compression

Ontology-Aware Token Embeddings for Prepositional Phrase Attachment

Cross-label Suppression: A Discriminative and Fast Dictionary Learning with Group Regularization

Elementary abelian groups of rank 5 are DCI-groups

Duality for pathwise superhedging in continuous time

Hardness Results for Structured Linear Systems

The Query Complexity of Cake Cutting

Large Deviations for Small Noise Diffusions in a Fast Markovian Environment

Learning non-maximum suppression

Temporal Segment Networks for Action Recognition in Videos

Safe and Nested Subgame Solving for Imperfect-Information Games

Multi-Fidelity Surrogate Based on Single Linear Regression

Game Theoretic Dynamic Channel Allocation for Frequency-Selective Interference Channels

A branch-and-price approach with MILP formulation to modularity density maximization on graphs

You said that?

Energy-Throughput Tradeoff in Sustainable Cloud-RAN with Energy Harvesting

On variance reduction for stochastic smooth convex optimization with multiplicative noise

A Functorial Construction of Quantum Subtheories

Community Detection in Hypergraphs, Spiked Tensor Models, and Sum-of-Squares

On the Achievable Rates of Decentralized Equalization in Massive MU-MIMO Systems

Optimally-Tuned Nonparametric Linear Equalization for Massive MU-MIMO Systems

Ramsey theorem for designs

K-classes of Brill-Noether loci and a determinantal formula

Quantum Chaos on random Cayley graphs of ${\rm SL}_2[\mathbb{Z}/p\mathbb{Z}]$

Developing All-Skyrmion Spiking Neural Network

Light Field Video Capture Using a Learning-Based Hybrid Imaging System

Real-Time User-Guided Image Colorization with Learned Deep Priors