Module_0_Introduction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# HackerMath for ML\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "\n",
    "Intro to Stats & Maths for Machine Learning\n",
    "\n",
    "<br>\n",
    "*Amit Kapoor*\n",
    "@amitkaps\n",
    "<br>\n",
    "*Bargava Subramanian*\n",
    "@bargava\n",
    "\n",
    "---\n",
    "\n",
    "> What I cannot create, I do not understand \n",
    "-- Richard Feynman\n",
    "\n",
    "---\n",
    "\n",
    "# Philosophy of HackerMath\n",
    "\n",
    "> Hacker literally means developing mastery over something.\n",
    " -- Paul Graham\n",
    "\n",
    "<br>\n",
    "\n",
    "Here we will aim to learn Math essential for Data Science in this hacker way.\n",
    "\n",
    "---\n",
    "\n",
    "# **Three Key Questions**\n",
    "\n",
    "- Why do you need to understand the math?\n",
    "- What math knowledge do you need?\n",
    "- Why approach it the hacker's way?\n",
    "\n",
    "---\n",
    "\n",
    "# Approach\n",
    "- Understand the Math.\n",
    "- Code it to learn it.\n",
    "- Play with code.\n",
    "\n",
    "---\n",
    "\n",
    "# Module 1: Linear Algebra\n",
    "## Supervised ML - Regression, Classification\n",
    "- Solve $Ax = b$ for $ n \\times n$\n",
    "- Solve $Ax = b$ for $ n \\times p + 1$\n",
    "- Linear Regression\n",
    "- Ridge Regularization (L2)\n",
    "- Bootstrapping\n",
    "- Logistic Regression (Classification)\n",
    "\n",
    "---\n",
    "\n",
    "# Module 2: Statistics\n",
    "## Hypothesis Testing: A/B Testing\n",
    "- Basic Statistics\n",
    "- Distributions\n",
    "- Shuffling\n",
    "- Bootstrapping & Simulation\n",
    "- A/B Testing\n",
    "\n",
    "---\n",
    "\n",
    "# Module 3: Linear Algebra contd.\n",
    "## Unsupervised ML: Dimensionality Reduction\n",
    "- Solve $Ax = \\lambda x$ for $ n \\times n$\n",
    "- Eigenvectors & Eigenvalues\n",
    "- Principle Component Analysis\n",
    "- Cluster Analysis (K-Means)\n",
    "\n",
    "---\n",
    "\n",
    "# Schedule\n",
    "- 0900 - 1000: Breakfast\n",
    "- 1000 - 1130: Session 1\n",
    "- 1130 - 1145: Tea Break\n",
    "- 1145 - 1315: Session 2\n",
    "- 1315 - 1400: Lunch\n",
    "- 1400 - 1530: Session 3\n",
    "- 1530 - 1545: Tea Break\n",
    "- 1545 - 1700: Session 4\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> It’s tough to make predictions, especially about the future.\n",
    "-- Yogi Berra\n",
    "\n",
    "## What is Machine Learning (ML)?\n",
    "\n",
    "> [Machine learning is the] field of study that gives computers the ability to learn without being explicitly programmed.\n",
    "-- *Arthur Samuel*\n",
    "\n",
    "> Machine learning is the study of computer algorithm that improve automatically through experience\n",
    "-- *Tom Mitchell*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ML Problems\n",
    "- “Is this cancer?”\n",
    "- “What is the market value of this house?”\n",
    "- “Which of these people are friends?”\n",
    "- “Will this person like this movie?”\n",
    "- “Who is this?”\n",
    "- “What did you say?”\n",
    "- “How do you fly this thing?”. \n",
    "\n",
    "## ML in use Everyday\n",
    "- Search\n",
    "- Photo Tagging\n",
    "- Spam Filtering\n",
    "- Recommendation\n",
    "- ...\n",
    "\n",
    "## Broad ML Application\n",
    "- Database Mining e.g. Clickstream data, Business data\n",
    "- Automating e.g. Handwriting, Natural Language Processing, Computer Vision\n",
    "- Self Customising Program e.g. Recommendations\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ML Thought Process\n",
    "\n",
    "![](img/thought.jpg)\n",
    "\n",
    "\n",
    "## Learning Paradigm\n",
    "- *Supervised* Learning\n",
    "- *Unsupervised* Learning\n",
    "- *Reinforcement* Learning\n",
    "- *Online* Learning\n",
    "\n",
    "## Supervised Learning\n",
    "- Regression\n",
    "- Classification\n",
    "\n",
    "![](img/supervised.png)\n",
    "\n",
    "## Unsupervised Learning\n",
    "- Clustering\n",
    "- Dimensionality Reduction\n",
    "\n",
    "![](img/unsupervised.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ML Pipeline\n",
    "\n",
    "- *Frame*: Problem definition\n",
    "- *Acquire*: Data ingestion \n",
    "- *Refine*: Data wrangline\n",
    "- *Transform*: Feature creation \n",
    "- *Explore*: Feature selection \n",
    "- *Model*: Model creation & assessment\n",
    "- *Insight*: Communication "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Linear Regression\n",
    "\n",
    "![](img/linear_regression.png)\n",
    "\n",
    "---\n",
    "\n",
    "### Linear Relationship\n",
    "\n",
    "$$ y_i = \\alpha + \\beta_1 x_1 + \\beta_2 x_2 + .. $$\n",
    "\n",
    "### Objective Function\n",
    "\n",
    "$$ \\epsilon = \\sum_{k=1}^n (y_i - \\hat{y_i} ) ^ 2 $$\n",
    "\n",
    "*Interactive Example: [http://setosa.io/ev/](http://setosa.io/ev/ordinary-least-squares-regression/)*\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Logit Function\n",
    "\n",
    "$$ \\sigma (t)={\\frac {e^{t}}{e^{t}+1}}={\\frac {1}{1+e^{-t}}}$$\n",
    "\n",
    "![](img/logistic-curve.png)\n",
    "\n",
    "\n",
    "## Logistic Regression\n",
    "\n",
    "![](img/logistic_regression.png)\n",
    "\n",
    "\n",
    "## Logistic Relationship\n",
    "\n",
    "Find the $ \\beta $  parameters that best fit:\n",
    "$ y=1 $  if $\\beta _{0}+\\beta _{1}x+\\epsilon > 0$\n",
    "$ y=0$, otherwise\n",
    "\n",
    "Follows:\n",
    "\n",
    "$$ P(x)={\\frac {1}{1+e^{-(\\beta _{0}+\\beta _{1}x)}}} $$\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fitting a Model\n",
    "\n",
    "![fit inline](img/overfitting_2.png)\n",
    "\n",
    "## Bias-Variance Tradeoff\n",
    "\n",
    "![fit inline](img/biasvariance_2.png)\n",
    "\n",
    "## Train and Test Datasets\n",
    "\n",
    "Split the Data - 80% / 20%\n",
    "\n",
    "![fit inline](img/train_test2.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train and Test Datasets\n",
    "\n",
    "Measure the error on Test data\n",
    "\n",
    "![](img/test_train.jpg)\n",
    "\n",
    "\n",
    "## Model Complexity\n",
    "![](img/model_complexity.png)\n",
    "\n",
    "\n",
    "## Cross Validation\n",
    "![](img/cross_validation.png)\n",
    "\n",
    "## Regularization\n",
    "\n",
    "Attempts to impose Occam's razor on the solution\n",
    "\n",
    "![](img/regularization.png)\n",
    "\n",
    "\n",
    "## Model Evaluation\n",
    "\n",
    "Mean Squared Error\n",
    "\n",
    "$$ MSE = 1/n \\sum_{k=1}^n (y_i - \\hat{y_i} ) ^ 2 $$\n",
    "\n",
    "\n",
    "## Model Evaluation \n",
    "\n",
    "Confusion Matrix\n",
    "\n",
    "![](img/confusion_matrix2.png)\n",
    "\n",
    "\n",
    "## Model Evaluation \n",
    "\n",
    "**Classification Metrics**\n",
    "\n",
    "![](img/precision_recall.png)\n",
    "\n",
    "Recall (TPR) = TP / (TP + FN)\n",
    "<br>\n",
    "Precision = TP / (TP + FP)\n",
    "<br>\n",
    "Specificity (TNR) =  TN / (TN + FP)\n",
    "\n",
    "## Model Evaluation\n",
    "\n",
    "**Receiver Operating Characteristic Curve** \n",
    "\n",
    "Plot of TPR vs FPR at different discrimination threshold\n",
    " \n",
    "![](img/roc-curves.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Decision Tree\n",
    "\n",
    "Example: Survivor on Titanic\n",
    "\n",
    "![](img/tree_titanic.png)\n",
    "\n",
    "## Decision Tree\n",
    "\n",
    "- Easy to interpret\n",
    "- Little data preparation\n",
    "- Scales well with data\n",
    "- White-box model\n",
    "- Instability – changing variables, altering sequence\n",
    "- Overfitting\n",
    "\n",
    "## Bagging\n",
    "- Also called bootstrap aggregation, reduces variance\n",
    "- Uses decision trees and uses a model averaging approach\n",
    "\n",
    "## Random Forest\n",
    "- Combines bagging idea and random selection of features.\n",
    "- Similar to decision trees are constructed – but at each split, a random subset of features is used. \n",
    "\n",
    "![](img/random_forest.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Challenges\n",
    "\n",
    "> If you torture the data enough, it will confess.\n",
    "-- Ronald Case\n",
    "\n",
    "- Data Snooping\n",
    "- Selection Bias\n",
    "- Survivor Bias \n",
    "- Omitted Variable Bias\n",
    "- Black-box model Vs White-Box model\n",
    "- Adherence to regulations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda root]",
   "language": "python",
   "name": "conda-root-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}