Linear_Regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# Linear Regression 1\n",
    "\n",
    "The following cell provides you with a function <code>ones(m)</code> that creates an $m$-dimensional vector where each component is $1$.  Be sure to evaluate the cell below to get started."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
   ],
   "source": [
    "def ones(m):\n",
    "    return vector([1] * m)\n",
    "\n",
    "def vectors2matrix(vectors):\n",
    "    return matrix(vectors).transpose()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### Problem 1\n",
    "\n",
    "The cell below defines a data set <code>crickets</code> and two vectors <code>chirps</code> and <code>temperature</code> that put the data into vectors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
   ],
   "source": [
    "crickets = [[20, 88.6],\n",
    "            [16, 71.6],\n",
    "            [19.8, 93.3],\n",
    "            [18.4, 84.3],\n",
    "            [17.1, 80.6],\n",
    "            [15.5, 75.2],\n",
    "            [14.7, 69.7],\n",
    "            [15.7, 71.6],\n",
    "            [15.4, 69.4],\n",
    "            [16.3, 83.3],\n",
    "            [15, 79.6 ],\n",
    "            [17.2, 82.6],\n",
    "            [16, 80.6 ],\n",
    "            [17, 83.5 ],\n",
    "            [14.4, 76.3]]\n",
    "\n",
    "chirps, temperature = map(vector, zip(*crickets))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Construct a matrix $A$ and vector ${\\mathbf b}$ so that the solution to the linear system $A{\\mathbf x} = {\\mathbf b}$ gives a line passing through all of these data points."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
   ],
   "source": [
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Write the normal equations $A^TA\\widehat{\\mathbf x} = A^T{\\mathbf b}$ and solve this system to  find the least squares approximate solution $\\widehat{\\mathbf x}$ to the equation $A{\\mathbf x} = {\\mathbf b}$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
   ],
   "source": [
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Estimate the temperature if a cricket is chirping at 22 chirps per second."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
   ],
   "source": [
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### Problem 3\n",
    "\n",
    "If you evaluate the following cell, you will have a function <code>QR</code> that forms that $QR$ factorization of a matrix $A$:\n",
    "<code>\n",
    "Q, R = QR(A)\n",
    "</code>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
   ],
   "source": [
    "def projection(b, basis):\n",
    "    return sum([b.dot_product(v)/v.dot_product(v)*v for v in basis])\n",
    "\n",
    "def unit(v):\n",
    "    return v/v.norm()\n",
    "\n",
    "def vectors2matrix(vectors):\n",
    "    return matrix(vectors).transpose()\n",
    "\n",
    "def gs(basis):\n",
    "    onbasis = []\n",
    "    for b in basis:\n",
    "        if len(onbasis) == 0: onbasis.append(b)\n",
    "        else: onbasis.append(b-projection(b, onbasis))\n",
    "    return map(unit, onbasis)\n",
    "\n",
    "def QR(A):\n",
    "    Q = vectors2matrix(gs(A.columns()))\n",
    "    return Q, Q.T*A"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "The data below give vectors for the measurements of the 10 patients.  Evaluate this cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
   ],
   "source": [
    "weight=vector([154,173,154,185,184,210,181,176,191,199])\n",
    "height=vector([68,72,66,72,71,75,70,73,74,74])\n",
    "abdomen=vector([85,83,88,86,100,94,91,89,83,89])\n",
    "wrist=vector([17,18,17,18,18,19,18,19,18,19])\n",
    "neck=vector([36,39,34,37,34,39,36,38,38,42])\n",
    "BFI = vector([13,7,25,11,28,21,19,13,5,12])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Suppose that we want to model $BFI$ as a linear function of weight, height, abdomen, wrist, and neck so that\n",
    "$$\n",
    "BFI = \\beta_0 + \\beta_1 w + \\beta_2 h + \\beta_3 a + \\beta_4 r + \\beta_5 n.\n",
    "$$\n",
    "\n",
    "Set up a matrix $A$ and vector ${\\mathbf b}$ giving the linear system $A{\\mathbf x} = {\\mathbf b}$ whose solution is the coefficients $\\beta_0,\\beta_1,\\ldots,\\beta_5$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
   ],
   "source": [
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Form the $QR$ factorization of $A$ and use it to find the least squares approximate solution $\\widehat{\\mathbf x}$ to the equation $A{\\mathbf x} = {\\mathbf b}$; that is, solve the equation\n",
    "$\n",
    "R\\widehat{\\mathbf x} = Q^T{\\mathbf b}.\n",
    "$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
   ],
   "source": [
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Estimate the $BFI$ of a patient with $w=190$, $h=70$, $a = 90$, $r = 18$, and $n =35$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
   ],
   "source": [
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "SageMath (stable)",
   "language": "sagemath",
   "name": "sagemath"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}