An introduction to MM algorithms for machine learning and statistical estimation

MM (majorization-minimization) algorithms are an increasingly popular tool for solving optimization pro- blems in machine learning and statistical estimation. This article introduces the MM algorithm framework in general and via three popular example applications: Gaussian mixture regressions, multinomial logistic regres- sions, and support vector machines. Speci?c algorithms for the three examples are derived and numerical demonstrations are presented. Theoretical and practical aspects of MM algorithm design are discussed.


Feature Engineering with Tidyverse

In this blog post, I will discuss feature engineering using the Tidyverse collection of libraries. Feature engineering is crucial for a variety of reasons, and it requires some care to produce any useful outcome. In this post, I will consider a dataset that contains description of crimes in San Francisco between years 2003-2015. The data can be downloaded from Kaggle. One issue with this data is that there are only a few useful columns that is readily available for modelling. Therefore, it is important in this specific case to construct new features from the given data to improve the accuracy of predictions.


Multi-Agent Diverse Generative Adversarial Networks

Why to employ Multiple Generators?
• As we can see from the 4 randomly sampled images from Imagenet below, the variations in the image samples is staggering.
• As we have seen previously that single GANs (DCGAN) have experienced great success in modeling restricted domains of images such as celebrity faces and our motivation is to distribute the tasks of the Generators so that they can master disjoint modes of the data distribution.


Introducing tf-seq2seq: An Open Source Sequence-to-Sequence Framework in TensorFlow

Last year, we announced Google Neural Machine Translation (GNMT), a sequence-to-sequence (“seq2seq”) model which is now used in Google Translate production systems. While GNMT achieved huge improvements in translation quality, its impact was limited by the fact that the framework for training these models was unavailable to external researchers. Today, we are excited to introduce tf-seq2seq, an open source seq2seq framework in TensorFlow that makes it easy to experiment with seq2seq models and achieve state-of-the-art results. To that end, we made the tf-seq2seq codebase clean and modular, maintaining full test coverage and documenting all of its functionality. Our framework supports various configurations of the standard seq2seq model, such as depth of the encoder/decoder, attention mechanism, RNN cell type, or beam size. This versatility allowed us to discover optimal hyperparameters and outperform other frameworks, as described in our paper, “Massive Exploration of Neural Machine Translation Architectures.”


Interpretability via attentional and memory-based interfaces, using TensorFlow

This article is a gentle introduction to attentional and memory-based interfaces in deep neural architectures, using TensorFlow. Incorporating attention mechanisms is very simple and can offer transparency and interpretability to our complex models. We conclude with extensions and caveats of the interfaces. As you read the article, please access all of the code on GitHub and view the IPython notebook here; all code is compatible with TensorFlow version 1.0. The intended audience for this notebook are developers and researchers who have some basic understanding of TensorFlow and fundamental deep learning concepts. Check out this post for a nice introduction to TensorFlow.
Advertisements