Cubic and Smoothing Splines in R

Splines are a smooth and flexible way of fitting Non linear Models and learning the Non linear interactions from the data.In most of the methods in which we fit Non linear Models to data and learn Non linearities is by transforming the data or the variables by applying a Non linear transformation.

Introducing SurvivalStan

Survival analysis is an important and useful tool in biostatistics. It is commonly used in the analysis of clinical trial data, where the time to a clinical event is a primary endpoint. This endpoint may or may not be observed for all patients during the study’s follow-up period. At the core of survival analysis is the relationship between hazard and survival. The observed time to event t t or Survival is often modeled as the result of an accumulation of event-related risks or hazards at each moment up to that time t t . Factors that modify the time to event do so by reducing or increasing the instantaneous risk of the event in a particular time period. The basic survival model posits quite an elegant relationship between covariates and the dependent variable. In addition, there are several analytical problems that survival analysis attempts to address, which may not be obvious at first glance.

Our quest for robust time series forecasting at scale

We were part of a team of data scientists in Search Infrastructure at Google that took on the task of developing robust and automatic large-scale time series forecasting for our organization. In this post, we recount how we approached the task, describing initial stakeholder needs, the business and engineering contexts in which the challenge arose, and theoretical and pragmatic choices we made to implement our solution.

2017 Data Science Bowl, Predicting Lung Cancer: 2nd Place Solution Write-up, Daniel Hammack and Julian de Wit

Julian and I independently wrote summaries of our solution to the 2017 Data Science Bowl. What is below is my (Daniel’s) summary. For the other half of the story, see Julian’s post here. Julian is a freelance software/machine learning engineer so check out his site and work if you are looking to apply machine intelligence to your work. He won 3rd in last year’s Data Science Bowl too! This blog post describes the story behind my contribution to the 2nd place solution to the 2017 Data Science Bowl. I will try to describe here why and when I did certain things but avoid the deep details on exactly how everything works. For those details see my technical report which has more of an academic flavor. I’ll try to go roughly in chronological order here.

Artificial intelligence in the software engineering workflow

The workflow of the AI researcher has been quite different from the workflow of the software developer. Peter Norvig explores how the two can come together.

Algebra of Sets in R

Part 4 of 4 in the series Set Theory

Data Manipulation with data.table (part -2)

In the last set of exercise of data.table ,we saw some interesting features of data.table .In this set we will cover some of the advanced features like set operation ,join in data.table.You should ideally complete the first part before attempting this one . Answers to the exercises are available here. If you obtained a different (correct) answer than those listed on the solutions page, please feel free to post your answer as a comment on that page.