Distributed Least-Squares Iterative Methods in Networks: A Survey

Many science and engineering applications involve solving a linear least-squares system formed from some field measurements. In the distributed cyber-physical systems (CPS), often each sensor node used for measurement only knows partial independent rows of the least-squares system. To compute the least-squares solution they need to gather all these measurement at a centralized location and then compute the solution. These data collection and computation are inefficient because of bandwidth and time constraints and sometimes are infeasible because of data privacy concerns. Thus distributed computations are strongly preferred or demanded in many of the real world applications e.g.: smart-grid, target tracking etc. To compute least squares for the large sparse system of linear equation iterative methods are natural candidates and there are a lot of studies regarding this, however, most of them are related to the efficiency of centralized/parallel computations while and only a few are explicitly about distributed computation or have the potential to apply in distributed networks. This paper surveys the representative iterative methods from several research communities. Some of them were not originally designed for this need, so we slightly modified them to suit our requirement and maintain the consistency. In this survey, we sketch the skeleton of the algorithm first and then analyze its time-to-completion and communication cost. To our best knowledge, this is the first survey of distributed least-squares in distributed networks.

The energy landscape of a simple neural network

We explore the energy landscape of a simple neural network. In particular, we expand upon previous work demonstrating that the empirical complexity of fitted neural networks is vastly less than a naive parameter count would suggest and that this implicit regularization is actually beneficial for generalization from fitted models.

MAGIX: Model Agnostic Globally Interpretable Explanations

Explaining the behavior of a black box machine learning model at the instance level is useful for building trust. However, what is also important is understanding how the model behaves globally. Such an understanding provides insight into both the data on which the model was trained and the generalization power of the rules it learned. We present here an approach that learns rules to explain globally the behavior of black box machine learning models. Collectively these rules represent the logic learned by the model and are hence useful for gaining insight into its behavior. We demonstrate the power of the approach on three publicly available data sets.

Explaining Recurrent Neural Network Predictions in Sentiment Analysis

Recently, a technique called Layer-wise Relevance Propagation (LRP) was shown to deliver insightful explanations in the form of input space relevances for understanding feed-forward neural network classification decisions. In the present work, we extend the usage of LRP to recurrent neural networks. We propose a specific propagation rule applicable to multiplicative connections as they arise in recurrent network architectures such as LSTMs and GRUs. We apply our technique to a word-based bi-directional LSTM model on a five-class sentiment prediction task, and evaluate the resulting LRP relevances both qualitatively and quantitatively, obtaining better results than a gradient-based related method which was used in previous work.

Explanation in Artificial Intelligence: Insights from the Social Sciences

There has been a recent resurgence in the area of explainable artificial intelligence as researchers and practitioners seek to provide more transparency to their algorithms. Much of this research is focused on explicitly explaining decisions or actions to a human observer, and it should not be controversial to say that, if these techniques are to succeed, the explanations they generate should have a structure that humans accept. However, it is fair to say that most work in explainable artificial intelligence uses only the researchers’ intuition of what constitutes a `good’ explanation. There exists vast and valuable bodies of research in philosophy, psychology, and cognitive science of how people define, generate, select, evaluate, and present explanations. This paper argues that the field of explainable artificial intelligence should build on this existing research, and reviews relevant papers from philosophy, cognitive psychology/science, and social psychology, which study these topics. It draws out some important findings, and discusses ways that these can be infused with work on explainable artificial intelligence.

A New Sequence Counted by OEIS Sequence A006012
Response theory of the ergodic many-body delocalized phase: Keldysh Finkel’stein sigma models and the 10-fold way
Interior-proximal primal-dual methods
CAN: Creative Adversarial Networks, Generating ‘Art’ by Learning About Styles and Deviating from Style Norms
Laplacian Simplices
The influence of periodic external fields in multi-agent models with language dynamics
Constrained Bayesian Optimization with Noisy Experiments
K-Adaptability in Two-Stage Mixed-Integer Robust Optimization
A hybrid supervised/unsupervised machine learning approach to solar flare prediction
Cluster Analysis is Convex
‘Parallel Training Considered Harmful?’: Comparing Series-Parallel and Parallel Feedforward Network Training
A 2-spine Decomposition of the Critical Galton-Watson Tree and a Probabilistic Proof of Yaglom’s Theorem
Convergence and Stationary Distributions for Walsh Diffusions
Multiscale Information Decomposition: Exact Computation for Multivariate Gaussian Processes
Generating Long-term Trajectories Using Deep Hierarchical Networks
Balanced Quantization: An Effective and Efficient Approach to Quantized Neural Networks
The Charming Leading Eigenpair
A Useful Motif for Flexible Task Learning in an Embodied Two-Dimensional Visual Environment
On the Enumeration and Congruences for m-ary Partitions
Multiplicative Pacing Equilibria in Auction Markets
Personalized Automatic Estimation of Self-reported Pain Intensity from Facial Expressions
Comparison of Time-Frequency Representations for Environmental Sound Classification using Convolutional Neural Networks
A Novel VHR Image Change Detection Algorithm Based on Image Fusion and Fuzzy C-Means Clustering
A bijection of plane increasing trees with relaxed binary trees of right height at most one
Curvature-aware Manifold Learning
Some remarks on boundary operators of Bessel extensions
Shape recognition of volcanic ash by simple convolutional neural network
RelNet: End-to-end Modeling of Entities & Relations
Compressive Statistical Learning with Random Feature Moments
Equilibria, information and frustration in heterogeneous network games with conflicting preferences
The Best-or-Worst and the Postdoc problems
High-Performance Out-of-core Block Randomized Singular Value Decomposition on GPU
Continuum Limit of Posteriors in Graph Bayesian Inverse Problems
From here to infinity – sparse finite versus Dirichlet process mixtures in model-based clustering
Synthesis of Near-regular Natural Textures
Living Labs – An Ethical Challenge for Researchers and Platform Providers
Bounds on energy absorption in quantum systems with long-range interactions
Refined restricted inversion sequences
Distributed Matching between Individuals and Activities with Additively Separable Preferences
Restricted inversion sequences and enhanced $3$-noncrossing partitions
GraphHP: A Hybrid Platform for Iterative Graph Processing
Localization and mobility edges in the off-diagonal quasiperiodic model with slowly varying potentials
Monotonicity Methods for Input-to-State Stability of Nonlinear Parabolic PDEs with Boundary Disturbances
Solidification of porous interfaces and disconnection
Gated-Attention Architectures for Task-Oriented Language Grounding
Convolved subsampling estimation with applications to block bootstrap
Automatic Quality Estimation for ASR System Combination
A Self-Adaptive Proposal Model for Temporal Action Detection based on Reinforcement Learning
Characterization of the ranking induced by the Logarithmic Least Squares Method
Scalable Multi-Class Gaussian Process Classification using Expectation Propagation
Fast Estimation of Haemoglobin Concentration in Tissue Via Wavelet Decomposition
Fractional Partial Differential Equations with Boundary Conditions
Nonlinear Acceleration of Stochastic Algorithms
A note on edge degree and spanning trail containing given edges
Decomposing $C_4$-free graphs under degree constraints
A Minimal Developmental Model Can Increase Evolvability in Soft Robots
Path-dependent Hamilton-Jacobi equations in infinite dimensions
Notes on the replica symmetric solution of the classical and quantum SK model, including the matrix of second derivatives and the spin glass susceptibility
On the non-existence of $srg(76,21,2,7)$
Targeted Undersmoothing
Antimagic orientation of biregular bipartite graphs
Polluted Bootstrap Percolation in Three Dimensions
An End-to-End Computer Vision Pipeline for Automated Cardiac Function Assessment by Echocardiography
Deep Supervision for Pancreatic Cyst Segmentation in Abdominal CT Scans
An approach to reachability analysis for feed-forward ReLU neural networks
Strong Disorder Renormalization for the dynamics of Many-Body-Localized systems : iterative elimination of the fastest degree of freedom via the Floquet expansion
Three-dimensional Cardiovascular Imaging-Genetics: A Mass Univariate Framework
Efficient Convex Optimization with Membership Oracles
Reconstructing the Forest of Lineage Trees of Diverse Bacterial Communities Using Bio-inspired Image Analysis
Tracking Single-Cells in Overcrowded Bacterial Colonies
Pixels to Graphs by Associative Embedding
Constrained Ordered Equilibrium Problems
Girsanov Theorem for Multifractional Brownian Processes
Crystallization of random matrix orbits
Fine-Grained Categorization via CNN-Based Automatic Extraction and Integration of Object-Level and Part-Level Features
Geometric Understanding of the Stability of Power Flow Solutions
On the Complexity and Approximation of the Maximum Expected Value All-or-Nothing Subset
Data-adaptive smoothing for optimal-rate estimation of possibly non-regular parameters
Universal Sampling Rate Distortion
Rational coordination with no communication or conventions
Optimal General Matchings
Single Classifier-based Passive System for Source Printer Classification using Local Texture Features