Regression-Multiple Linear Regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "<center>\n",
    "    <img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-ML0101EN-SkillsNetwork/labs/Module%202/images/IDSNlogo.png\" width=\"300\" alt=\"cognitiveclass.ai logo\"  />\n",
    "</center>\n",
    "\n",
    "# Multiple Linear Regression\n",
    "\n",
    "Estimated time needed: **15** minutes\n",
    "\n",
    "## Objectives\n",
    "\n",
    "After completing this lab you will be able to:\n",
    "\n",
    "*   Use scikit-learn to implement Multiple Linear Regression\n",
    "*   Create a model, train it, test it and use the model\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>Table of contents</h1>\n",
    "\n",
    "<div class=\"alert alert-block alert-info\" style=\"margin-top: 20px\">\n",
    "    <ol>\n",
    "        <li><a href=\"https://#understanding-data\">Understanding the Data</a></li>\n",
    "        <li><a href=\"https://#reading_data\">Reading the Data in</a></li>\n",
    "        <li><a href=\"https://#multiple_regression_model\">Multiple Regression Model</a></li>\n",
    "        <li><a href=\"https://#prediction\">Prediction</a></li>\n",
    "        <li><a href=\"https://#practice\">Practice</a></li>\n",
    "    </ol>\n",
    "</div>\n",
    "<br>\n",
    "<hr>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Importing Needed packages\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "import pylab as pl\n",
    "import numpy as np\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Downloading Data\n",
    "\n",
    "To download the data, we will use !wget to download it from IBM Object Storage.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2022-02-04 10:12:41--  https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-ML0101EN-SkillsNetwork/labs/Module%202/data/FuelConsumptionCo2.csv\n",
      "Resolving cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud (cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud)... 169.63.118.104\n",
      "Connecting to cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud (cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud)|169.63.118.104|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 72629 (71K) [text/csv]\n",
      "Saving to: ‘FuelConsumption.csv’\n",
      "\n",
      "FuelConsumption.csv 100%[===================>]  70.93K  --.-KB/s    in 0.002s  \n",
      "\n",
      "2022-02-04 10:12:41 (33.8 MB/s) - ‘FuelConsumption.csv’ saved [72629/72629]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget -O FuelConsumption.csv https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-ML0101EN-SkillsNetwork/labs/Module%202/data/FuelConsumptionCo2.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Did you know?** When it comes to Machine Learning, you will likely be working with large datasets. As a business, where can you host your data? IBM is offering a unique opportunity for businesses, with 10 Tb of IBM Cloud Object Storage: [Sign up now for free](http://cocl.us/ML0101EN-IBM-Offer-CC)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "<h2 id=\"understanding_data\">Understanding the Data</h2>\n",
    "\n",
    "### `FuelConsumption.csv`:\n",
    "\n",
    "We have downloaded a fuel consumption dataset, **`FuelConsumption.csv`**, which contains model-specific fuel consumption ratings and estimated carbon dioxide emissions for new light-duty vehicles for retail sale in Canada. [Dataset source](http://open.canada.ca/data/en/dataset/98f1a129-f628-4ce4-b24d-6f16bf24dd64?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkML0101ENSkillsNetwork20718538-2021-01-01)\n",
    "\n",
    "*   **MODELYEAR** e.g. 2014\n",
    "*   **MAKE** e.g. Acura\n",
    "*   **MODEL** e.g. ILX\n",
    "*   **VEHICLE CLASS** e.g. SUV\n",
    "*   **ENGINE SIZE** e.g. 4.7\n",
    "*   **CYLINDERS** e.g 6\n",
    "*   **TRANSMISSION** e.g. A6\n",
    "*   **FUELTYPE** e.g. z\n",
    "*   **FUEL CONSUMPTION in CITY(L/100 km)** e.g. 9.9\n",
    "*   **FUEL CONSUMPTION in HWY (L/100 km)** e.g. 8.9\n",
    "*   **FUEL CONSUMPTION COMB (L/100 km)** e.g. 9.2\n",
    "*   **CO2 EMISSIONS (g/km)** e.g. 182   --> low --> 0\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "<h2 id=\"reading_data\">Reading the data in</h2>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>MODELYEAR</th>\n",
       "      <th>MAKE</th>\n",
       "      <th>MODEL</th>\n",
       "      <th>VEHICLECLASS</th>\n",
       "      <th>ENGINESIZE</th>\n",
       "      <th>CYLINDERS</th>\n",
       "      <th>TRANSMISSION</th>\n",
       "      <th>FUELTYPE</th>\n",
       "      <th>FUELCONSUMPTION_CITY</th>\n",
       "      <th>FUELCONSUMPTION_HWY</th>\n",
       "      <th>FUELCONSUMPTION_COMB</th>\n",
       "      <th>FUELCONSUMPTION_COMB_MPG</th>\n",
       "      <th>CO2EMISSIONS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2014</td>\n",
       "      <td>ACURA</td>\n",
       "      <td>ILX</td>\n",
       "      <td>COMPACT</td>\n",
       "      <td>2.0</td>\n",
       "      <td>4</td>\n",
       "      <td>AS5</td>\n",
       "      <td>Z</td>\n",
       "      <td>9.9</td>\n",
       "      <td>6.7</td>\n",
       "      <td>8.5</td>\n",
       "      <td>33</td>\n",
       "      <td>196</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2014</td>\n",
       "      <td>ACURA</td>\n",
       "      <td>ILX</td>\n",
       "      <td>COMPACT</td>\n",
       "      <td>2.4</td>\n",
       "      <td>4</td>\n",
       "      <td>M6</td>\n",
       "      <td>Z</td>\n",
       "      <td>11.2</td>\n",
       "      <td>7.7</td>\n",
       "      <td>9.6</td>\n",
       "      <td>29</td>\n",
       "      <td>221</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2014</td>\n",
       "      <td>ACURA</td>\n",
       "      <td>ILX HYBRID</td>\n",
       "      <td>COMPACT</td>\n",
       "      <td>1.5</td>\n",
       "      <td>4</td>\n",
       "      <td>AV7</td>\n",
       "      <td>Z</td>\n",
       "      <td>6.0</td>\n",
       "      <td>5.8</td>\n",
       "      <td>5.9</td>\n",
       "      <td>48</td>\n",
       "      <td>136</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2014</td>\n",
       "      <td>ACURA</td>\n",
       "      <td>MDX 4WD</td>\n",
       "      <td>SUV - SMALL</td>\n",
       "      <td>3.5</td>\n",
       "      <td>6</td>\n",
       "      <td>AS6</td>\n",
       "      <td>Z</td>\n",
       "      <td>12.7</td>\n",
       "      <td>9.1</td>\n",
       "      <td>11.1</td>\n",
       "      <td>25</td>\n",
       "      <td>255</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2014</td>\n",
       "      <td>ACURA</td>\n",
       "      <td>RDX AWD</td>\n",
       "      <td>SUV - SMALL</td>\n",
       "      <td>3.5</td>\n",
       "      <td>6</td>\n",
       "      <td>AS6</td>\n",
       "      <td>Z</td>\n",
       "      <td>12.1</td>\n",
       "      <td>8.7</td>\n",
       "      <td>10.6</td>\n",
       "      <td>27</td>\n",
       "      <td>244</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   MODELYEAR   MAKE       MODEL VEHICLECLASS  ENGINESIZE  CYLINDERS  \\\n",
       "0       2014  ACURA         ILX      COMPACT         2.0          4   \n",
       "1       2014  ACURA         ILX      COMPACT         2.4          4   \n",
       "2       2014  ACURA  ILX HYBRID      COMPACT         1.5          4   \n",
       "3       2014  ACURA     MDX 4WD  SUV - SMALL         3.5          6   \n",
       "4       2014  ACURA     RDX AWD  SUV - SMALL         3.5          6   \n",
       "\n",
       "  TRANSMISSION FUELTYPE  FUELCONSUMPTION_CITY  FUELCONSUMPTION_HWY  \\\n",
       "0          AS5        Z                   9.9                  6.7   \n",
       "1           M6        Z                  11.2                  7.7   \n",
       "2          AV7        Z                   6.0                  5.8   \n",
       "3          AS6        Z                  12.7                  9.1   \n",
       "4          AS6        Z                  12.1                  8.7   \n",
       "\n",
       "   FUELCONSUMPTION_COMB  FUELCONSUMPTION_COMB_MPG  CO2EMISSIONS  \n",
       "0                   8.5                        33           196  \n",
       "1                   9.6                        29           221  \n",
       "2                   5.9                        48           136  \n",
       "3                  11.1                        25           255  \n",
       "4                  10.6                        27           244  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(\"FuelConsumption.csv\")\n",
    "\n",
    "# take a look at the dataset\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's select some features that we want to use for regression.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ENGINESIZE</th>\n",
       "      <th>CYLINDERS</th>\n",
       "      <th>FUELCONSUMPTION_CITY</th>\n",
       "      <th>FUELCONSUMPTION_HWY</th>\n",
       "      <th>FUELCONSUMPTION_COMB</th>\n",
       "      <th>CO2EMISSIONS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2.0</td>\n",
       "      <td>4</td>\n",
       "      <td>9.9</td>\n",
       "      <td>6.7</td>\n",
       "      <td>8.5</td>\n",
       "      <td>196</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2.4</td>\n",
       "      <td>4</td>\n",
       "      <td>11.2</td>\n",
       "      <td>7.7</td>\n",
       "      <td>9.6</td>\n",
       "      <td>221</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.5</td>\n",
       "      <td>4</td>\n",
       "      <td>6.0</td>\n",
       "      <td>5.8</td>\n",
       "      <td>5.9</td>\n",
       "      <td>136</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3.5</td>\n",
       "      <td>6</td>\n",
       "      <td>12.7</td>\n",
       "      <td>9.1</td>\n",
       "      <td>11.1</td>\n",
       "      <td>255</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3.5</td>\n",
       "      <td>6</td>\n",
       "      <td>12.1</td>\n",
       "      <td>8.7</td>\n",
       "      <td>10.6</td>\n",
       "      <td>244</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>3.5</td>\n",
       "      <td>6</td>\n",
       "      <td>11.9</td>\n",
       "      <td>7.7</td>\n",
       "      <td>10.0</td>\n",
       "      <td>230</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>3.5</td>\n",
       "      <td>6</td>\n",
       "      <td>11.8</td>\n",
       "      <td>8.1</td>\n",
       "      <td>10.1</td>\n",
       "      <td>232</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>3.7</td>\n",
       "      <td>6</td>\n",
       "      <td>12.8</td>\n",
       "      <td>9.0</td>\n",
       "      <td>11.1</td>\n",
       "      <td>255</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>3.7</td>\n",
       "      <td>6</td>\n",
       "      <td>13.4</td>\n",
       "      <td>9.5</td>\n",
       "      <td>11.6</td>\n",
       "      <td>267</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   ENGINESIZE  CYLINDERS  FUELCONSUMPTION_CITY  FUELCONSUMPTION_HWY  \\\n",
       "0         2.0          4                   9.9                  6.7   \n",
       "1         2.4          4                  11.2                  7.7   \n",
       "2         1.5          4                   6.0                  5.8   \n",
       "3         3.5          6                  12.7                  9.1   \n",
       "4         3.5          6                  12.1                  8.7   \n",
       "5         3.5          6                  11.9                  7.7   \n",
       "6         3.5          6                  11.8                  8.1   \n",
       "7         3.7          6                  12.8                  9.0   \n",
       "8         3.7          6                  13.4                  9.5   \n",
       "\n",
       "   FUELCONSUMPTION_COMB  CO2EMISSIONS  \n",
       "0                   8.5           196  \n",
       "1                   9.6           221  \n",
       "2                   5.9           136  \n",
       "3                  11.1           255  \n",
       "4                  10.6           244  \n",
       "5                  10.0           230  \n",
       "6                  10.1           232  \n",
       "7                  11.1           255  \n",
       "8                  11.6           267  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cdf = df[['ENGINESIZE','CYLINDERS','FUELCONSUMPTION_CITY','FUELCONSUMPTION_HWY','FUELCONSUMPTION_COMB','CO2EMISSIONS']]\n",
    "cdf.head(9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's plot Emission values with respect to Engine size:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEHCAYAAABBW1qbAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAAAwDElEQVR4nO2df5xddXnn389MJiGTqMAksoGQGUqBdmItypTWTX+gAbHRF2i3unFHzUt9NZjQFu36smazW7XddNnWttLdJjYVJJqpLFvtyotSlERtV0ulAQFJkCUuCQSykIA/QBBI8uwf59yZM3fOuefHPeeeO3c+79fr+7r3fs/5nvPcm8z3Od/neb7PY+6OEEIIAdBXtwBCCCG6BykFIYQQk0gpCCGEmERKQQghxCRSCkIIISaRUhBCCDHJvCovbmYHgKeB48Axdx8zs1OB/wGMAAeAt7n798LzNwHvDc//bXf/UqvrL1myxEdGRqoSXwghepI777zzqLsvjTtWqVIIea27H418/jCw292vNrMPh59/18xGgbXASuB0YJeZnevux5MuPDIywp49e6qUXQgheg4zO5h0rA7z0eXAjvD9DuDNkf4b3P15d38I2A9c2HnxhBBi7lK1UnDgy2Z2p5mtD/tOc/fDAOHry8P+M4BHImMPhX1CCCE6RNXmo1Xu/piZvRy4zcy+0+Jci+mbkYMjVC7rAVasWFGOlEIIIYCKVwru/lj4+gTwtwTmoMfNbBlA+PpEePoh4MzI8OXAYzHX3O7uY+4+tnRprJ9ECCFEQSpTCma2yMxe0ngPvB64D7gJWBeetg74Yvj+JmCtmS0ws7OAc4A7qpJPCCHETKpcKZwGfN3M7iGY3P/O3W8FrgYuMbMHgUvCz7j7XuBGYB9wK3Blq8gjIcTcYmICRkagry94nZioW6LexGZz6uyxsTFXSKoQvc/EBKxfD88+O9U3OAjbt8P4eH1yzVbM7E53H4s7ph3NQoiuZ/Pm6QoBgs+bN9cjTy8jpSCE6HoefjhfvyiOlIIQoutJij5XVHr5SCkIIbqeLVsCH0KUwcGgX5SLlIIQousZHw+cysPDYBa8yslcDZ1IiCeEEG0zPi4l0Am0UhBCCDGJlIIQQohJpBSEEEJMIqUghBBiEikFIYQQk0gpCCGEmERKQQghxCRSCkIIISaRUhBCCDGJlIIQQohJpBSEEEJMUrlSMLN+M/uWmd0cfv6omT1qZneHbU3k3E1mtt/MHjCzS6uWTQghxHQ6kRDvKuB+4KWRvj9z949HTzKzUWAtsBI4HdhlZueqTrMQQnSOSlcKZrYceCPwqQynXw7c4O7Pu/tDwH7gwirlE0IIMZ2qzUefAD4EnGjq/00zu9fMrjOzU8K+M4BHIuccCvuEEEJ0iMqUgpm9CXjC3e9sOrQNOBs4HzgM/EljSMxlPOa6681sj5ntOXLkSIkSCyGEqHKlsAq4zMwOADcArzOzne7+uLsfd/cTwF8xZSI6BJwZGb8ceKz5ou6+3d3H3H1s6dKlFYovhBBzj8qUgrtvcvfl7j5C4ED+iru/w8yWRU57C3Bf+P4mYK2ZLTCzs4BzgDuqkk8IIcRM6ijH+Udmdj6BaegAcAWAu+81sxuBfcAx4EpFHgkhRGfpyOY1d/+au78pfP9Od/8Zd3+lu1/m7ocj521x97Pd/Tx3//tOyCaEmB1MTMDICPT1Ba8TE3VL1JtoR7MQIpWNG2HePDALXjdu7Oz9JyZg/Xo4eBDcg9f166UYqkBKQQjRko0bYds2OB4ac48fDz53UjFs3gzPPju979lng35RLuY+I+pz1jA2NuZ79uypWwwhepq+vuDpvBkzONG8A6mHZeglzOxOdx+LO6aVghCiJUnPjZ18nlyxIl+/KI6UghCi69myBQYHp/cNDgb9olykFIQQXc/4OGzfDsPDgcloeDj4PD5et2S9h5SCEKIlGzbk66+K8XE4cCDwIRw4IIVQFVIKQvQAVcbwb90aKID+/uBzf3/weevW9q6rfQfdiZSCEBVTdYx/XAz/u98NS5aUN+Fu3QrHjgXXP3asHIWgfQfdiUJShaiQRox/M2U8aTcYGQkm1VYMDnaXDT5J5uHhwDQkqqVVSKqUghAVMm/e1KavKP39wRN3GSTF8DfTTROu9h3Ui/YpCFETcQqhVX8RssbqP/xw8XukmcDy+ge076B7kVIQokIsrnRUi/4ixMXwx1F0wk1Lc1HEP6B9B92LlIIQFVLGbuC0p/TmGP6hIZg/f/o57Uy4n/xk6/4ieYm076B7kVIQc5q6s3+mkTUZXTSG/+hReO97p4eQrltXfMJNU2xJZqk0c5X2HXQnUgpiztKJ7J+NiTlrfzPbt+frh8Bss2PH9O+1Y0d14Z7yD/QWUgpizlJkws3L+vX5+psp4qjudJpp+Qd6i8qVgpn1m9m3zOzm8POpZnabmT0Yvp4SOXeTme03swfM7NKqZRNzm05EBrW7G7jISqOoOSeJtDQX4+OBeaosc5Wol06sFK4C7o98/jCw293PAXaHnzGzUWAtsBJ4A7DVzDIusoXIT7umnay0sxu4yEqjbHNOmmLrtLlKVEulSsHMlgNvBD4V6b4c2BG+3wG8OdJ/g7s/7+4PAfuBC6uUT8xt2jXtdIJVqwIHeJR584L+JOLMOQMD8MwzxdNetFJsqorWW1S9UvgE8CEgukfxNHc/DBC+vjzsPwN4JHLeobBPiEqoKtFbmWzePHPn87FjMyfc6OaxzZsD8000RNUMnnyymjxDZZurRL1UphTM7E3AE+5+Z9YhMX0zguHMbL2Z7TGzPUeOHGlLRiHKTvRWNlkm3LjNYzt2BCuGEydg8WJ44YXp48t8klf0UW9R5UphFXCZmR0AbgBeZ2Y7gcfNbBlA+PpEeP4h4MzI+OXAY80Xdfft7j7m7mNLly6tUHwh6ifLhJtmvqn6Sb6q6KOVK4MVTqOtXNne9UQ2KlMK7r7J3Ze7+wiBA/kr7v4O4CZgXXjaOuCL4fubgLVmtsDMzgLOAe6oSj4hZgNZJty0Sb/qJ/kqdievXAn79k3v27dPiqET1LFP4WrgEjN7ELgk/Iy77wVuBPYBtwJXunuJwYFCzD6yTLhpk34n9hGUvTu5WSGk9c8lKt+F7+6ztl1wwQUuRN3s3Ok+POxuFrzu3Nn5+w8OugcehaANDk6Xo24Z8xL9Ls1tLrNhQ/xvsmFDvusAezxhXtWOZiHaoBsqiGVZTTQ/yYNKYc5GOrELX0pBiDaYjTH6VSiysp3Co6P5+ucKndiFL6UgRBvUEaMfnXwbLW2Sj+5jWLcuvyK7+OLp97v44qljVTiF9+6dqQBGR4P+uUwnduFLKQjRBp2O0U8qztNqkm9eGSQ9VSbVeb74Yti9e3rf7t1TiqEqp/DevdMt53NdIUBnduFLKQjRBt2cIbSxWokzccWR9LTZrBDS+kV1dGIXvpSCEG3QzRXEGquVrKasMu3Sojqq3oUvpSBEDxJdrWQ1ZQ0PVyePmD1IKQjRBt0QktogabUSZ+KKY82a+P7Vq/P1i9mNlIIQLYhG7cTF83c6JLVVveSkHcXNJq4k38Ett8T379o1UwGsXh30Q+fqUojOIKUgRAJZVgF1hKTG7WlNS30Q3bx24kTsZVvKvGvX9Ps1FAKUFxFTefoGkY2krc6zoSnNhaiS4eH4lALDw1PnDA3FnzM0VJ1cGza49/cH9+nvdx8dzZf6IMv3ysvq1dOvtXp1/u9URvoGkQ2U5kKI/HRj8ZiNG2HbtumlL5P2A2zbFm/2KjuMdmICbr99et/tt880tbUyxXUifYPISJK2mA1NKwVRJVmeqM3izzErT45oMrtWieJatSoT5CX9TkNDU/cYGnKfPz9ZJiXA6yxopSBEfrI8USeFe/b1lZNsrtmvUZRnnw3SWzRkgtaprlultWgmaSf0k09Oyf3kk62rv8lZ3T1IKQiRQJaNaUnhnsePlxOimnU3chayypSW1qJMGqa4iy6KP57UL6rDvJ3Hj5oZGxvzPXv21C2GmONMTAST98MPB0/icTuDh4enUlbnoa+vvRVCK/r7g5XCihWBcmsou6T8StFx69dP7aRNO78Vjd9lZCR+xVH0dxOtMbM73X0s7phWCkK0Sbvhnq1CMatKrAfTVw7veU/21czx44ETuyFnURNP1BTXjU79uUplSsHMTjKzO8zsHjPba2YfC/s/amaPmtndYVsTGbPJzPab2QNmdmlVsglRFYsW5euPiyaKTrg/+ZPlyxjHCy/AVVflG9OIDDrvvGznDwzA0FC8Ka7T2WZFC5I80O02wIDF4fsB4JvALwAfBT4Yc/4ocA+wADgL+C7Q3+oeij4S3UDW6KA4GvsNmlt/f+vj0Xb66TPLcRZt7jP3HKSd3+p7Z41wylJSVJQHdUQfhfd+Jvw4ELZW1tHLgRvc/Xl3fwjYD1xYlXxClEG70UFplbRaZS5tTJ+PPpotjUV/fzb7f1xai6TrNeRIolWEU5RuzjY716jUp2Bm/WZ2N/AEcJu7fzM89Jtmdq+ZXWdmp4R9ZwCPRIYfCvuE6Bh5Uy20Gx2UFoqZNVTzG9+AQ4eCCfrEiUD2KIODsGNHcGxoKP6a0f5oWosNG+LPL7OwC8ysIy2FUA+VKgV3P+7u5wPLgQvN7BXANuBs4HzgMPAn4elxzzAznkHMbL2Z7TGzPUeOHKlEbjE3SbPvx5EUo5+VJHt8oz9p4j1+fGoPwfz50+V2D/LsL1489dS9bl2gwPrCv/hmpTIwANdcE3+vtMIuJ50UPy6pX3Q3HQtJNbOPAD9y949H+kaAm939FWa2CcDd/0t47EvAR9399rjrgUJSRbnMmxdvrunvDybZPGOaSQpVbYSFtjp/48bAlJK3CE5D7oaJK7qiGRiAl74UnnpqZkhqXpYsCTanNTM0BEePFrumqJZaQlLNbKmZnRy+XwhcDHzHzJZFTnsLcF/4/iZgrZktMLOzgHOAO6qST4hm0uz7eY9FSQpVzdIfrbSVh4ZscSauF18MJnL3wOz0jW/ku3aUp57K1y+6m3nppxRmGbDDzPoJlM+N7n6zmX3WzM4nMA0dAK4AcPe9ZnYjsA84Blzp7ioQKDpGf3/yJN9w0I6OTi8gPzyczYRUR1WzhrknLda/YSaDYqUdk1ZSSlExO6ky+uhed3+Vu7/S3V/h7r8f9r/T3X8m7L/M3Q9Hxmxx97Pd/Tx3//uqZBOiQdSxnOWpf98+WLly6nPWqmad2m8QpeGPyBrrXzQjaZJpLalfdDfa0SwKk1aVrNtlaHYsN0gL24ymqm4OpUzia1/LLlcrTj45vn9gINkRnFVx5fVZiB4laQPDbGjavFYf3bDZqF0Z0jaOFUnnnHdM3vOLFqPJssGu8b3zUuR3EvVCi81rSognCtENCczalaHVk717+vE48kYw5b1HkQipZhorpGaiq4s8FPmdRL20HX0URhL9BzPbHm44u87MritXTDGb6IYEZu3KkLYxbHQ0/nhzf9SEtXBh/JiyNnoViZBqZtWqqf0KDfr6gv4iJG1uS+oX3U1Wn8IXgZcBu4C/izQxR+mGBGZZZGjlc0grOL9370wF0Bx91Jzm4plnZk64q1cnP4EnPWUn9ZdRjGbz5pmhsCdOTBW8iaMb/EeiQyTZlaINuDvLeZ1u8inUx2zwKWSRccOGKd9Cf3/+QvFJpSiz/i6d8ilEyVtCNO13TPPNiO6DFj6FrErhPwNrspzbySalUC/tTqhl0KrWcJYay+1c3z173eSkexaRsSpFVlRGOZpnH62UQlbz0VXAzWb2YzN7Omw/rGTpImYFExNBgrVonqAdO7rLrNCuz6HZNBRXxjKruSzpnlnqQDcT3eF87Fh+5/CaNfn6037HvCYw0eUkaYvZ0LRSqI8ynsLbJc2ssWhRvIyLFmW7fpbvuHOn+/z56SuF/v7k1UanV1x1rRTSVl2ic9Cu+Si4BpcBHw/bm7KOq7JJKdRHXrt0FvJOGlWbNbKM37nTfWAgXSkkKa648QMDrb973Pl5KNunkPV3ar6vmRRDXbStFICrgd3Ae8J2G3B1lrFVNimF+ih7pVDEcZ02uXVCKST9Do2VQZITtvE7DQ3FHx8aipcpSQHlUQxF/u1aKewsv1PSamr+/Oxyi/IoQyncC/RFPvcD92YZW2WTUqiPsqOP0ibXuJVDN6wU2lVMeWVs9zu5x5u85s8v/m+XZeVRhtyiPFophTy5j06OvH9ZYSeG6AnKLp+Y5Mw8fjyYOuKcvEWctGWTtleijH0FVRA82yV/zsP73pevX3Q5Sdoi2oC3AweB64EdwEPA2ixjq2xaKfQOWeL940wc7Zo1WpFlfLv29qpWCq2c11UECaQ5y7VS6C4oydG8jMDZfDnwr7KOq7JJKfQOO3e69/VlUwxZI3WymDVaKZXR0fjxo6MzZS+6VyLvZJnFp5C2wa2KIIE0Vq+Ov+fq1dXdUyRTWCkAPxW+vjqutRrbiSal0DskTWRZWpJiWLw4/vzFi4PjWWzrzYqhWSGkUUbkTjNp0UdJyrWvL9vvUhXNikEKoT7aUQrbw9evxrSvtBrbiSalUC9lxp0nRelkXTnEkTbh5o38yUrz77JhQ3UmrjjSrtlqF7b2EMwNSjEf5W3ASQQ1lu8B9gIfC/tPJQhpfTB8PSUyZhOwH3gAuDTtHlIK9VF29FFRhdBqAq2iXkIaeX+XOpRClt+z03msRGdppRQy1VMws7cCt7r702b2H0Pz0R+4+7dajDFgkbs/Y2YDwNcJ0mX8GvCUu19tZh8OlcLvmtko8DngQuB0goys53qLOs2qp1AfZddTSKoTkIWitQqqqAOQ93epQoa+vvixZkE21Ky/dSdrY4jO0nY9BeA/hQrhF4FLCSKQPtlqQKiQngk/DoTNCRzVO8L+HcCbw/eXAze4+/Pu/hDBiuHCjPKJDpNUrD5LEfs42qk3kDR28eLW/UND8ceT+rOQFFp78GB5aaejdaXnzQs+R0lSJo3+iy7Kdp9O1sYQ3UNWpdB4rngjsM3dvwjMTxtkZv1mdjfwBHCbu38TOM3dDwOEry8PTz8DeCQy/FDYJ7qQsuPv4wq/xLF48fRaxKtXwy23xE+4P/pR/DUa/ddcE9Q2jjIwEPTnITpJt3q6dw+Uw7vfXVwxNNeVPn48+NysGFqxf3+28zpZG0N0EUl2pWgDbgb+EvguwSa2BcA9WcaG408mcE6/Avh+07Hvha9/Abwj0n8t8G9irrUe2APsWbFiRXlGNpGLLLbwPI7orPsUYOqaQ0Mzo4eK5ORpx1leNGqq4cwu20+Sds2sqb7z+hSU7G52QQlpLgYJfAHnhJ+XAa/PMjZyjY8AHyRwIi+LXOeB8P0mYFPk/C8Br2l1TTma6yMtcqdMh2veVmae/4ULp49buHD68VZRU2kTcJbfMc/vVMZvWWRSL5LUT9RLK6WQ1dF8NnDI3Z83s4uAVwKfcffvtxizFHjR3b9vZguBLwP/FfgV4EmfcjSf6u4fMrOVwF8z5WjeHSohOZq7kLQC8mU6XPPScKgWceJu3Bik62jliF24EJ59dupere6Rdnzx4ngz16JFQWnPZrJ8p3Z+ywzTwQyWLIEnn5zZPzQER48Wl0VURxmO5s8Dx83sJwnMOmcRTOCtWAZ81czuBf6FwKdwM0HG1UvM7EHgkvAz7r4XuBHYB9wKXNlKIfQ6ZdfETXNO5iWtgHy7BW7aoagtvNlen8Rzz029b9e3kub3mA3EKYRW/aLLSVpCRBtwV/j6IeC3wvffyjK2ytar5qOy9wCUUde3mTQzRt78OmWZjtrZLZxnA13W3zbtGnllrNp8VISyryeqhxJ8Ct8kSIp3H3BW2HdflrFVtl5VCmUnLKuisHraRNBpn0IZu4WLTp6tksGl+QyqUAp5nPbRVjTtRFU7w0V1tFIKWc1H7wZeA2xx94fM7CxgZ3nrFRGlbNNLmqmnCspOrZ3GiROBr6Kq60dZuHD651WrYPny4HsuXx58blBW2GuDDRvS++NSijdMhxCYtk4/ffrx1ath165iMl1zDcxvClCfP7/4dxQ1k6QtZkPr1ZVC2U9eaQnSilC2yaCd3EdJ98z7O2YJ12z+zbKsiFqFaxbJHpqlpnP0nkNDM6ODyk5joZDU2QVFVwpmdmP4+m0zuzfSvh06kMUsoPnJNq2/DqpYtZx/fr5+9/Rrnjgx3Um/efNUJFKDZ5+FdeumggQgWMXErWbOPTf+Pkn9AFu3BhFe7sHr1q0zzxkfn7rn4sXw4oszZdy8OfkeeYner1MrNlENLUNSzWyZux82s+G44+5eMKlBOfRqSGpa7pq6r9cYm0SWybWZl7wkPgQzK3H3TAubzXp+q/FJv22UwcFk01leGWF62Gx/f5DmI04xNKji31/MbgqHpPpUOoqDoQL4HvB0pIkKSCvxWPf1qqCKEMy8vpSs+Zei47P8hq2eyvPKWCTNRSf+/csOoRY1kmRXijbgCuBx4ABBKc6HgP+bZWyVrVd9CmUXVm/X7h1H2T6FdvwJ0RYtglMk6ipqr09q0fFxv22e3yWvjEW+U9khzp2+vigfSghJfRBYkuXcTrZeVgplpw1oNekXuV8RpVC0nnJRxVBkf0ZUKSQ5npvHR79XFkXSfL88MhZVxlU6gquo+SyqpQylcCswmOXcTrZeVQqd/iMrEu2Ud3JqpyxlkdYgS6RO9NxW10wbX+R3yStjFXtO2qWOms+iPVophay5j14FfJpgE9vzEdPTb5dmxyqAHM3lUMRpnGXMxERgS3/44eA7xdnJG7mQysx9FJUhD2U4z/PmMsrLypWwb9/M/tFR2Lu3/esXoeyCS6J6ysh99JfAV4B/Bu6MNFEBzRuP0vq7kYmJwHF78GAwoSY5TosW5elWmsNT0/rz8sAD+fo7QdxmucHBoF/MPrIqhWPu/jvu/ml339FolUo2h4kmXMvS3y5VVCCLi9+Po2hRnlaMjpZ/zawkrSharTTyRO7UsTs9jU7vXhfVklUpfNXM1pvZMjM7tdEqlWwOk2QiqiqmvIo0BVlXAGVPZu2YUZLMR2WbtqJMTASV2BorqrTKbGVXvCsLbV7rHbIqhX9HUATnn5gyHfWeMb+HaH763Lgx+Wl0fByuu276k95117X3h511kmpnNRKl4d5sVgh5nsLf9758/WVw1VUzdxu/+GLQH0fSXop2alwLMY0kD/RsaL0afVQ07LBBltj5vr72whLTZMwaJZQlW2iWlhRqmzd+Pk8kUJHfpd3zy5CxCpT7aHZB0ZBU4EOR929tOvaHrcZ2okkpxJM1dfKiRdXJmJSEr10lktaim/zqiJ/vhFLoNrR5bfbRSimkmY/WRt5vajr2hpIWK6KJdh2/We35VVb3yuP/6MtqxMzACy9MmV7qqP6W99+uCid/p0lKClhmwj3ROdL+HC3hfdzn6QfNzjSzr5rZ/Wa218yuCvs/amaPmtndYVsTGbPJzPab2QNmdmmub9JDzLX89MHCszwaZSDryPmU99+uF/6t6yy9KiogaQkRrDCCMpzN7+M+x4xdBrw6fP8S4P8Ao8BHgQ/GnD8K3AMsIKgB/V2gv9U9etV85N6ejTarqSVaG6Ds3EdZahNU2RrfqQ6zRt7fcrbb45XmYvZBC/PRvBSd8bNm9kOCVcHC8D3h55NSlM1hoJFl9Wkzux84o8WQy4Eb3P154CEz2w9cCNyeIqMoyBVXBK+NjWYNE8DBg1PRLEUjkMp++i9CQ/bGruoVK4INVVWHS46P57tH3vO7jS1bpv//AW1em9UkaYsyGzACPAy8lGClcAC4F7gOOCU8578D74iMuRb49VbX7dWVws6dM5+0zbI/QaY9RTdHrBR50kt7Sk/Kp9TJlcJcptOrj9m+2plr0G7uo3Yws8XAPxDUd/6CmZ0GHAUc+ANgmbu/x8z+Arjd3XeG464FbnH3zzddbz2wHmDFihUXHOy1PAnAggWBw7SZ+fPh+edn9jeTN4dPkVxLafdYsmTKtl8H3bBSqYvmlR+0LvQj5h5l5D4qeuMB4PPAhLt/AcDdH3f34+5+AvgrAhMRwCHgzMjw5cBjzdd09+3uPubuY0uXLq1S/NqIUwit+tulCofsU08VHyvaQ9FAoh0qUwpmZgQmoPvd/U8j/csip70FuC98fxOw1swWmNlZwDnAHVXJJ6aoIqHZqUqCUhuKBhLtUOVKYRXwTuB1TeGnf2Rm3zaze4HXAh8AcPe9wI3APoL6DVe6e41pvnoTs6AucLR8YycTmvX1VZtLSMyO8quie0mLPiqMu3+d+L0Mt7QYswVQzELFNOr6QuuC76046ST48Y/j+yHZfOQe+CmqVAyzaeNXFSgaSLRDpT4FUQ9ZU0dv3x68TkzAu941PVPnu97VOnncT/xE6/66nlYHBmbXxq8qUCpr0Q5SChWQJzNnFcRV5oqjkbb6iitmRhmdODG1jyHPPRr9a9bEH0/qb4fFi6cmv09/WpMfKJW1KE5l5qO5ShUbwaL09ZW/CSspB1I7uZFuSTASJvW3w3PPVVdrQoi5hlYKJVN1OGDDvLN+ffsrkCrt+p2MgKmz6pgQvYaUQsl0ajIsQ9FUucGrkz6FuquOCdFLSCmUTCcnw26OO0/b+1DmRK6qY0KUh5RCyXTSwdrNcedpETDnnVf82g2F0t8PGzYUD6sVQsyk8txHVTI2NuZ79nRXqeiknD9DQ3D0aLZrZLH1t8plk8dX4F5N7qM05s0r5gvImv9JCJFMbbmP5iJJSeDKSg5XRdx5HQXrizqHf+mXypVDCDEdhaTOMqoIvWyYX7ZvDybr/v7ATl+lWaa/v5hi+NrXShdFCBFBK4Ua2LgxMJ/E5SGqi61b4dixwPRz7Fj1dvqizmGFnwpRLVoplIxZsn0eAgXQyDsE5eQhqoO075lG8+okKwo/FaJatFIoQKs0FklO1kZ/I99QM0n9nSLv6uWnfzpffxzR1cnpp2cbo/BTIapFK4WctJvGIumpuE6zSJHVy/335+tPY2Cg9fFO+DmEEApJzc3ISKAImhkeDhKPpYVqJoVi9vcHT83Qfrhn3pDULDLluUeR/1JlX08IkYxCUkskqSR01lLRF12Ur79KFi4MXrth9ZLkK5APQYjOIqWQk3Ynr7vvztdfJc89F7wW+U6LF+frT6MbFJMQotoazWea2VfN7H4z22tmV4X9p5rZbWb2YPh6SmTMJjPbb2YPmNmlVcnWDu1OXlVvbitCkvO2lVP3k58MzE5R5s0L+oswPJyvXwhRDVWuFI4B/97dfxr4BeBKMxsFPgzsdvdzgN3hZ8Jja4GVwBuArWbWdcaD2TB5nXxyvvO3bg1yCOXJKTQ+DtdfPz230fXXF99l3cmcUUKIZCpTCu5+2N3vCt8/DdwPnAFcDuwIT9sBvDl8fzlwg7s/7+4PAfuBC6uSryizYfL6/vfzjymyea3M6l6dLMojhEimIz4FMxsBXgV8EzjN3Q9DoDiAl4ennQE8Ehl2KOzrKpImqe3bg30LohidLMojhEim8mnMzBYDnwfe7+4/bHVqTN+MYEQzW29me8xsz5EjR8oSMzNJUUbHj3cmdLKuus9V08k6FEKIZCpVCmY2QKAQJtz9C2H342a2LDy+DHgi7D8EnBkZvhx4rPma7r7d3cfcfWzp0qXVCZ9A3SGSZZbj7CbSivIIITpDldFHBlwL3O/ufxo5dBOwLny/DvhipH+tmS0ws7OAc4A7qpKvKN0SIllm3eeitEr3kZfxcVi3brqze9268tKDCyGyUeVKYRXwTuB1ZnZ32NYAVwOXmNmDwCXhZ9x9L3AjsA+4FbjS3btkCu5OkuztixZlG59n53MzjXQfBw+Ws3qZmAjCWRtK9/jx4HMvrYaEmA0ozUVO2plI3bOlc8h6j0ZqjWYuvhh2704fv2gRPPNMtns1k5buIy8nnRRfUW3BAvjxj/NfTwiRjNJc9CCt7O1f+Uq2a/zoR8XvX3a0UFKJTZXeFKKzSCnMMrKU4+zE4k/RQkL0JkqdPcuoohxnEbZsmZ5CHBQtJEQvoJVCD9KO3yMr4+PBaiWa5qLV6iWN1avz9QshqkGO5pzU7Wgus57C0BAcPZrt3E7Q7CBfvRp27apPHiF6FTma5xhZk/O97W3VypGXc8+dvk/h3HPrlUeIuYiUQg8Stzs4js98pnpZstIoCRrdp7BtW3qtaCFEuUgp9CDN9v4k2glJLZvt2/P1CyGqQUqhR4mmtZ4NqPKaEN2BlEKHSUpBEe0vOxInabXQiSglIcTsQkqhQzSqoWWZoPfvjz8nqV8IIcpCSqFDvOxlwWtSrqFof1LNhqT+NJLCWLspGnk2lDkVYi4gpdAhVEGsNaqnIER3IKUQw8aNMG9eYNKZN6+csMhTT23/Gr1M2TukhRDFUO6jJhrx8g0a8fKQrZh9N7J6dXwq7W5LITE+LiUgRN1opdBEWrz80FCx6z71VLFxZbBr10wFoBQSQog4pBSaSIuXL5oaou6U0rt2BY7lRpNCEELEUWWN5uvM7Akzuy/S91Eze7SpPGfj2CYz229mD5jZpVXJBa1rCzdy7zTT6L/llvz3GxiYcphmCUmdPz/+nKR+IYQoiypXCtcDb4jp/zN3Pz9stwCY2SiwFlgZjtlqZgnTc3uk1RZevz5+XKO/SBRRdMLPEh563XUzlYdZ0C+EEFVSmVJw938EslrSLwducPfn3f0hYD9wYRVybd48vTAMBJ83bw7eb90KGzZMz9a5YcOUkznJDNTfH0zccSuNF16Yun6WePzxcfjsZ6dH4nz2s3LCCiGqpw6fwm+a2b2heemUsO8M4JHIOYfCvtLJUlt41SpYvjyYkJcvDz43SIqn37EjyDOUlGuocf2s8fjR3EUHDkghCCE6Q6eVwjbgbOB84DDwJ2F/nKU91tBiZuvNbI+Z7Tly5EhuAZJSSjf608xLcfH069YFK4G+vqDF0VhhZI3Hb+X3yEK744UQcxR3r6wBI8B9aceATcCmyLEvAa9Ju/4FF1zgeenri8bgTLW+vuD48HD88f5+d7Pg+M6dU9fbudN9cDB+TKMNDk4fk0bcNfNcY+dO9/nzp4+fPz+fDEKI3gXY4wnzaqXlOM1sBLjZ3V8Rfl7m7ofD9x8Aft7d15rZSuCvCfwIpwO7gXPcvWXi5CLlONNKXfb1pecEGhycerofGYnPSdTfH5h+VqwITEN5zD9J1xweDkxJaSxZAk8+ObO/28pvCiHqoZZynGb2OeB24DwzO2Rm7wX+yMy+bWb3Aq8FPgDg7nuBG4F9wK3AlWkKoShpIadZ9hNEHdNJPoqGf6GIPyCL36MVcQqh0V9m6g4hRO9RZfTR2919mbsPuPtyd7/W3d/p7j/j7q9098saq4bw/C3ufra7n+fuf1+VXGkhp2vWxB9vpjFBJymRdjarJeVJKit/kkpdCiGSmHM7mtNCTrNuTmtM+klKJKtyqYKsqThU6lII0cycUwoQKIBjxwLfwbFj0xPdZTHRRENIk5RIkZ3PDZLyJGXNn3TNNcEu6jRU6lII0cycVAqtSNuc1hxC2q79P48MWU1S4+Pw6U9Phb0mkeRfEULMXaQUmkjbnNbsOC4ygaftISij4Ex081tSiuyLLsp+PSHE3EBKoYm8xV7yTuBpm+OKyJCGaj4LIbIipRBDnhQTeSfwtNxLVVCFiUsI0ZtIKRSg2fwD2ZVI3Ka05v4sq4k8VBE2K4ToTaQUctLuhJ22eQ7KX02U4aMQQswNpBRy0u6EnVbZDco395TtoxBC9C7z6hZgttHuhD08nJzXqMGKFfHntGPuGR+XEhBCpKOVQk7atc9nMeXI3COEqAsphZy0O2FnMeXI3COEqItKU2dXTZHU2WUwMRH4EB5+uFhqbCGEqJNaUmf3Mu2WysxSFU2V04QQdSClUAJ5JvAsIa1l71MQQoisyHzUJo0JPBqmGq3M1kyWqmrtVl4TQohWtDIfSSm0Sd4JPKncp1lgjsp6jhBCFKWucpzXmdkTZnZfpO9UM7vNzB4MX0+JHNtkZvvN7AEzu7Qqucom776FLCGtSkshhKiLKn0K1wNvaOr7MLDb3c8BdoefMbNRYC2wMhyz1cxmRbb/vBO49ikIIbqZKms0/yPQXCvscmBH+H4H8OZI/w3u/ry7PwTsBy6sSrYyyTuBa5+CEKKb6XSai9Pc/TCAux82s5eH/WcA/xw571DY1/U0Juo8+xaypJxQWgohRB10S+6juKKRsR5wM1sPrAdY0SVGdk3gQoheodP7FB43s2UA4esTYf8h4MzIecuBx+Iu4O7b3X3M3ceWLl1aqbBCCDHX6LRSuAlYF75fB3wx0r/WzBaY2VnAOcAdHZZNCCHmPJWZj8zsc8BFwBIzOwR8BLgauNHM3gs8DLwVwN33mtmNwD7gGHCluydUHhBCCFEVlSkFd397wqHVCedvARR0KYQQNaLcR0IIISaZ1WkuzOwIEJNkIjNLgKMliVMVkrEcJGM5SMZyqFvGYXePjdSZ1UqhXcxsT1L+j25BMpaDZCwHyVgO3SyjzEdCCCEmkVIQQggxyVxXCtvrFiADkrEcJGM5SMZy6FoZ57RPQQghxHTm+kpBCCFEhDmnFOKK/3QbZnammX3VzO43s71mdlXdMjVjZieZ2R1mdk8o48fqlikJM+s3s2+Z2c11y5KEmR0ws2+b2d1mVm85wQTM7GQz+xsz+074f/M1dcsUxczOC3+/Rvuhmb2/brmaMbMPhH8z95nZ58zspLplijLnzEdm9svAM8Bn3P0VdcsTR5gscJm732VmLwHuBN7s7vtqFm0SMzNgkbs/Y2YDwNeBq9z9n1OGdhwz+x1gDHipu7+pbnniMLMDwJi7d218vZntAP63u3/KzOYDg+7+/ZrFiiUs0vUo8PPu3s5eplIxszMI/lZG3f25ML3PLe5+fb2STTHnVgoJxX+6Cnc/7O53he+fBu6ny+pLeMAz4ceBsHXdE4aZLQfeCHyqbllmM2b2UuCXgWsB3P2FblUIIauB73aTQogwD1hoZvOAQRIyQtfFnFMKsw0zGwFeBXyzZlFmEJpl7iZIgX6bu3edjMAngA8BJ2qWIw0Hvmxmd4Y1Q7qNnwCOAJ8OTXGfMrNFdQvVgrXA5+oWohl3fxT4OEFC0MPAD9z9y/VKNR0phS7GzBYDnwfe7+4/rFueZtz9uLufT1D/4kIz6ypznJm9CXjC3e+sW5YMrHL3VwO/ClwZmjm7iXnAq4Ft7v4q4EeENda7jdC0dRnwP+uWpRkzO4Wg/PBZwOnAIjN7R71STUdKoUsJ7fSfBybc/Qt1y9OK0IzwNeAN9Uoyg1XAZaG9/gbgdWa2s16R4nH3x8LXJ4C/pftqlB8CDkVWg39DoCS6kV8F7nL3x+sWJIaLgYfc/Yi7vwh8AfjXNcs0DSmFLiR04l4L3O/uf1q3PHGY2VIzOzl8v5DgP/t3ahWqCXff5O7L3X2EwJzwFXfvqqcyADNbFAYUEJpkXg90VXScu/8/4BEzOy/sWk1Q/6QbeTtdaDoKeRj4BTMbDP/OVxP4DLuGOacUwuI/twPnmdmhsOBPt7EKeCfBk20jvG5N3UI1sQz4qpndC/wLgU+ha0M+u5zTgK+b2T0EFQf/zt1vrVmmOH4LmAj/zc8H/rBecWZiZoPAJQRP4F1HuNL6G+Au4NsEc3BX7W6ecyGpQgghkplzKwUhhBDJSCkIIYSYREpBCCHEJFIKQgghJpFSEEIIMYmUgpgzmNnxpiyahXfkmtk/lSlb07XHzOzPq7q+EK1QSKqYM5jZM+6+uG45hOhmtFIQc56wlsHHzOyusKbBT4X9S83strD/L83soJktCY89E75eZGZfi9QZmAh3qmJmF5jZP4RJ7r4UpkRvvvdbw7z695jZP0aueXP4/pbIyuYHZrYuTET4x2b2L2Z2r5ld0anfSvQ+UgpiLrGwyXz0byPHjoYJ6bYBHwz7PkKQGuPVBPmIViRc91XA+4FRgmyiq8LcVf8N+HV3vwC4DtgSM/b3gEvd/WcJkrhNw93XhEkH3wscBP5X+P4H7v5zwM8Bv2FmZ2X8DYRoyby6BRCigzwXTrBxNNIi3An8Wvj+F4G3ALj7rWb2vYSxd7j7IYAwlfgI8H3gFcBt4cKhnyBVcjPfAK4Pi63EpmYIVyefBd7m7j8ws9cDrzSzXw9PeRlwDvBQgnxCZEZKQYiA58PX40z9XVjOsdHxBux195YlK939fWb28wSFgO42s/Ojx8MKYjcAv+/ujSR5BvyWu38po3xCZEbmIyGS+TrwNoDw6fyUHGMfAJZaWMfYzAbMbGXzSWZ2trt/091/DzgKnNl0ytXAve5+Q6TvS8CG0ESFmZ3b5QVvxCxCKwUxl1gYmnca3OrurcJSPwZ8LvQ9/AOB+efpLDdy9xdC886fm9nLCP7WPgHsbTr1j83sHIKn/93APcCvRI5/ENgbkfv3CEqLjgB3hU7tI8Cbs8glRBoKSRUiATNbABx392PhE/+2Fj4JIXoCrRSESGYFcKOZ9QEvAL9RszxCVI5WCkIIISaRo1kIIcQkUgpCCCEmkVIQQggxiZSCEEKISaQUhBBCTCKlIIQQYpL/D/ZoRF15citXAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(cdf.ENGINESIZE, cdf.CO2EMISSIONS,  color='blue')\n",
    "plt.xlabel(\"Engine size\")\n",
    "plt.ylabel(\"Emission\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Creating train and test dataset\n",
    "\n",
    "Train/Test Split involves splitting the dataset into training and testing sets respectively, which are mutually exclusive. After which, you train with the training set and test with the testing set.\n",
    "This will provide a more accurate evaluation on out-of-sample accuracy because the testing dataset is not part of the dataset that have been used to train the model. Therefore, it gives us a better understanding of how well our model generalizes on new data.\n",
    "\n",
    "We know the outcome of each data point in the testing dataset, making it great to test with! Since this data has not been used to train the model, the model has no knowledge of the outcome of these data points. So, in essence, it is truly an out-of-sample testing.\n",
    "\n",
    "Let's split our dataset into train and test sets. Around 80% of the entire dataset will be used for training and 20% for testing. We create a mask to select random rows using the  **np.random.rand()** function:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "msk = np.random.rand(len(df)) < 0.8\n",
    "train = cdf[msk]\n",
    "test = cdf[~msk]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Train data distribution\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEHCAYAAABBW1qbAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAAAu80lEQVR4nO3df7RddXnn8fdzfxByc1FIiEwEci9jA/XitCgp6qTLUgNC0SXaqUycK2YVV6MJttjW5ZBmpupM02H6UzrToBEjcXJHyqgdGEpBEqGO1opBAQnIIi0BIhkIIApig0me+WPvc+++5+6f5+x99jnnfl5r7XXO+Z699/nem5v9nP398XzN3REREQEYqLsCIiLSPRQURERkmoKCiIhMU1AQEZFpCgoiIjJNQUFERKYNVXlyM9sHPA8cAQ67+0ozWwz8FTAO7AMucfcfhPtvBN4X7v9b7n5b2vlPPPFEHx8fr6r6IiJ96e67737a3ZfGvVdpUAj9srs/HXl9JbDL3a8ysyvD1//ezCaANcCZwCuBnWZ2ursfSTrx+Pg4u3fvrrLuIiJ9x8weTXqvjuaji4Ht4fPtwDsi5de7+yF3fwTYC5zT+eqJiMxfVQcFB75sZneb2bqw7CR3PwAQPr4iLD8ZeDxy7P6wTEREOqTq5qNV7v6Emb0CuN3Mvpeyr8WUzcnBEQaXdQDLly8vp5YiIgJUfKfg7k+Ej08Bf03QHPSkmS0DCB+fCnffD5waOfwU4ImYc25195XuvnLp0th+EhERaVFlQcHMFpnZcY3nwFuA+4GbgLXhbmuBG8PnNwFrzGyBmZ0GrADuqqp+IiIyV5V3CicBXzOzewku7n/j7rcCVwHnm9nDwPnha9x9D3AD8ABwK3B52sgjEZlfpqZgfBwGBoLHqam6a9SfrJdTZ69cudI1JFWk/01Nwbp18OKLM2UjI7B1K0xO1levXmVmd7v7yrj3NKNZRLrepk2zAwIErzdtqqc+/UxBQUS63mOPFSuX1ikoiEjXSxp9rlHp5VNQEJGut3lz0IcQNTISlEu5FBREpOtNTgadymNjYBY8qpO5Gp1IiCci0rbJSQWBTtCdgoiITFNQEBGRaQoKIiIyTUFBRESmKSiIiMg0BQUREZmmoCAiItMUFEREZJqCgoiITFNQEBGRaQoKIiIyrfKgYGaDZvYdM7s5fP0xM/u+md0TbhdF9t1oZnvN7CEzu6DquomIyGydSIh3BfAg8LJI2Z+7+59EdzKzCWANcCbwSmCnmZ2udZpFRDqn0jsFMzsFeCtwbY7dLwaud/dD7v4IsBc4p8r6iYjIbFU3H30C+AhwtKn8g2Z2n5ltM7MTwrKTgccj++wPy0REpEMqCwpm9jbgKXe/u+mta4BXAWcBB4A/bRwScxqPOe86M9ttZrsPHjxYYo1FRKTKO4VVwNvNbB9wPfBmM9vh7k+6+xF3Pwp8mpkmov3AqZHjTwGeaD6pu29195XuvnLp0qUVVl9EZP6pLCi4+0Z3P8Xdxwk6kL/i7u8xs2WR3d4J3B8+vwlYY2YLzOw0YAVwV1X1ExGRuepYjvOPzOwsgqahfcD7Adx9j5ndADwAHAYu18gjEZHO6sjkNXe/093fFj6/1N3/lbv/nLu/3d0PRPbb7O6vcvcz3P1vO1E3EekNU1MwPg4DA8Hj1FTdNepPmtEsIpnqviBPTcG6dfDoo+AePK5bp8BQBQUFEUk1NQWXXTb7gnzZZZ29IG/aBC++OLvsxReDcimXgoKIpLriCnjppdllL70UlHfKY48VK5fWKSiISKpnnilWXoXly4uVS+sUFESk623eDCMjs8tGRoJyKZeCgoikWrKkWHkVJidh61YYGwOz4HHr1qBcyqWgICKprr4ahodnlw0PB+WdNDkJ+/bB0aPBowJCNRQURPpAlUNGJyfhs5+d/S39s59t/6Jc9zBXiaegIFKxDRtgaCi4oA4NBa/L1Ikx/GV/S9e8g+5l7nMSkfaMlStX+u7du+uuhkiiDRvgmmvmlq9fD1u2lPMZ4+PBRbXZ2FhwAe9GvVjnfmJmd7v7ytj3FBREqjM0BEdiMngNDsLhw+V8xsBA8G27mVnwzb4b9WKd+0laUFDzkUiF4gJCWnkrOjGGP6sJrGj/gOYddC8FBZEKWdzSUSnlrah6DH+jCawRyI4cCV43AkMr/QOad9DF3L1nt7PPPttFullwmYzf8lq/3n1wMDhmcDB43WzHDvexMXez4HHHjrJ+guCccfU3C94fG4t/f2ws/bxV1lnSAbs94bqqOwWZ16oeGdSurG/pDc2jg6C84Z5J3Y6N8lbzEmneQXdSUJB5K+8Ftx2Dg8XKm23dWqwcOj/cU/0D/UVBQeatVi64Ra1bV6y8WSsd1Z1OM63+gf5SeVAws0Ez+46Z3Ry+Xmxmt5vZw+HjCZF9N5rZXjN7yMwuqLpuMr91YmTQli3BnITGncHgYLE5Cq3caZSdZnr9+vRy5SXqL524U7gCeDDy+kpgl7uvAHaFrzGzCWANcCZwIbDFzHLeZIsU127TTl5btgRzEtyDxyKT1lq50yi7OSdPYFP/QP+oNCiY2SnAW4FrI8UXA9vD59uBd0TKr3f3Q+7+CLAXOKfK+sn81m7TTiesWhV0gEcNDQXlSeKac4aH4YUXWu94biewSW+p+k7hE8BHgOgcxZPc/QBA+PiKsPxk4PHIfvvDMpFKtNu00wmbNs2d+Xz48Nz+gejksU2bYO3ameacJUuCx2eeUZ4hyVZZUDCztwFPufvdeQ+JKZszGM7M1pnZbjPbffDgwbbqKNLt34Dz9A/EjTbavj24Yzh6FEZH5y6nqfWNJUmVdwqrgLeb2T7geuDNZrYDeNLMlgGEj0+F++8HTo0cfwrwRPNJ3X2ru69095VLly6tsPoi9cvTP5A12qhX1zc+77zgDqexnXde3TWaHyoLCu6+0d1Pcfdxgg7kr7j7e4CbgLXhbmuBG8PnNwFrzGyBmZ0GrADuqqp+Ir0gz3DPrIt+L84jOO882LVrdtmuXQoMnVDHPIWrgPPN7GHg/PA17r4HuAF4ALgVuNzdSxwcKNJ78gz3zLro9+I8guaAkFU+n1Q+Cz8p/0UvbMp9JBLkDBoZmZ13aGRkdi6hXsszVEbOqH60fn387yQuH1YaUnIfaT0FkT4wNRX0ITz2WHCHsHlzb88VSMsi28OXrLaVtT6H1lMQkUqV3aSxenWx8vmiE7PwFRREeszg4OxROWbFE+AVXRQnbSRQFYkFd+6cGwBWrw7K57NOzMJX85FIDxkczL9cZdJ6x415DdFhrCMjyfmK4kYCwcxFWktrdk5Za35rjWaRPlFkxbaki/L4eHA30SwpiGS176v9v7M2bAgC+JEjwZeEdeuKT7pUUBDpE0WCQtJFvug3ewWF/qOOZpF5Jm0eQi9OZpPOUVAQ6SEDKf9j865ncNFFxco1Emh+GcreRUS6RaMdOdrMMzBQbEjiLbcUK9+5c25nc3Qk0OBg8th56T26UxBpU9Hhne06cmT2fNYjR4rNE2glQd7OnbM/Mzo0tKx1KTr9e5QESVOde2FTmgupW54UE2Vbv959cDD4rMFB94mJYqkPxsbi9x8bK69ORdMu1PF7nM9ISXNR+4W9nU1BQepWxQU2TVLum6QtLtdRXRfgtPxLnf49zncKCiIVMUu+GJclejEtEhA6mSAv7k4h+hlLlrgfc0xynTrxe5QZaUFBfQoibUgaxjkwUE7bePOqaq168cVgic5GnSCYw3D0aPDYPFKpyAI3SWku3vvemXo/80z66m8aJts9FBRE2hC3VgHMdAa3ux5y3Kpqrcpbp6IL3GzdGl+eJ8VFo3O76DBZqY6CgkgbmhfBiRuG2c56yFUtmZlWp7QFbuJGN7WTobNxJ1B0mKxUR0FBpE2TkzNNMUnfjtMu7mlDMatsPmk14DRnQW11PkJ01nWvriPdjyoLCmZ2rJndZWb3mtkeM/t4WP4xM/u+md0TbhdFjtloZnvN7CEzu6CquolUZfHiYuXNfQbNTTs/8zPV1BPaDziNZqMzzsi3//AwLFkSP+tafQpdJKkHut0NMGA0fD4MfBN4A/Ax4MMx+08A9wILgNOAfwQG0z5Do4+kG+QZHbRoUfyxWUMxGyN60raJiblDTIuORopavTr/edzTf+68I5w0T6GzqGP0UfjZL4Qvh8MtbfzExcD17n7I3R8B9gLnVFU/kTLkHR304x/Hl2c1m6S11zcun3v2ZPdrwMziPFm5keIWuEk6X6MeSdJGOEU1981k1VGqU2mfgpkNmtk9wFPA7e7+zfCtD5rZfWa2zcxOCMtOBh6PHL4/LBPpmKKpFtodHZTVbJJ3pa2vfx327w8u0EePBp3BUSMjsH17vgs0zE5rsX59/D5F01hkifbN5KmjVKPSoODuR9z9LOAU4Bwzew1wDfAq4CzgAPCn4e5xWdnnfAcxs3VmttvMdh88eLCSesv8lNW+HydvR+iSJfHlSX0GjfKkC+/ChTOB67zzZs8TcA8WcR8dnfnWvXZtEMBamTuxZUsQGBqBaHBw9kpfxx4bf1xSuXS5pHalsjfgozT1JQDjwP3h843Axsh7twFvTDun+hSkTK2kWkg6Jrodc0xy23hSn8Hg4Mw+0dnCZu5DQ/na+xvnqLq9fsmS+M9fsqSc80v5qKNPwcyWmtnx4fOFwHnA98xsWWS3dwL3h89vAtaY2QIzOw1YAdxVVf1EmrUyLDJu8lrzKJtt25KbQpL6DKLlW7YE3/zdg2alw4eT6xN3jrgmrhdfhPe8J19W1SzPPlusXLpblc1Hy4A7zOw+4FsEfQo3A39kZt8Ny38Z+G0Ad98D3AA8ANwKXO7ubUyLESkmqX3ffSbdw5lnzn4vroP0TW+C554Ljtu/P2jvL0uRcfuN5p6sY5rnHRS1YEGxculuWqNZ5rXoIuiNkTtZ38QnJoIRP0nnu+aaueXRNvioousbj48HfR15ND4z7zF5fvY4WqO592iNZqlENyyK0k4dmhO5NXfQJnnggeT3kvIAJZUXtXnz3JFFQ0PBENKkjuCk/EzN2klXIf1Dy3FKSxojdRpt1Y2ROtC5oYTt1iHpQv2TnwTDItMCQ5I8fQTt+PrX536bP3wYTj999mpoUY3fxaZNQVNS0rd3LZ8poOYjaVFSk8TYWDDGvBfqkNXs0UqzyNBQ8nrFcU0zRT+j6PnjFG3iyrJwIfzzP88tP/bYIMBK92m7+SgcSfR7ZrY1nHC2zcy2lVtN6SXdkMCs3TpkTQybmIh/v7k82oS1cGH8MUnzDRYtKlZexp3IqlVBXaMGBoLyVlx7bbFy6W55+xRuBF4O7AT+JrLJPNUNCczy1CGtzyFrwfk9e+YGgOZO5uYJby+8EHybb9wBNLfvNys68SvvDOc0mzbNzeZ69Gh6eu+032PS6KoyR11JByVNYIhuwD159uv0pslr9emGBGZZdchTx3YXnG93beGiy1AmrdFcpN5FPzPr95hnAp50F9pdoxn4A+CiPPt2clNQqFe7F9QyVL0YfNZaxu2uLdxKHTsdyLL2T5tVLd2pjKDwPHAU+Ofw+fPAj/IcW+WmoFCfbrhTyNLuBTvPz9hu4Knj91j0biPr99ju71k6r+2g0K2bgkJ9yvgWXrVFi+LrmLS2QbM8P+OOHUFuo+j7zbmOsu42st4vW113Cp3+OSVZKUEBeDvwJ+H2trzHVbkpKNSnim+HZV802m3WyHP8jh3uw8Oz3xseLtavUdTChbPPt3BhsePL7lPI+3tq/lwzBYa6lNF8dBWwC7gs3G4HrspzbJWbgkJ9yr5TqOLi2YmgkPV7yPN7mpiY/d7ERHKdmgNCK4GhlX+7tICd5/fUfDcVvauSzisjKNwHDEReDwL35Tm2yk1BoT5lX8STLlSDg63fOXQiKGR96846R3NAyAoM7f5M7vmavIrIc+dRRr2lPGlBoUjuo+Mjz19e4DjpQ2Uvn5g04ezIkeDSkWfBmzq0u3JaUh6ltPxKZQi+2yW/LuIDHyhWLl0uKVpEN+DdwKPAdcB24BFgTZ5jq9x0p9A/8ixWU7R5qhN3Cu22txetY97904atVjFIIGuYrO4UugsldTQvI+hsvhj4F3mPq3JTUOgfO3a4DwzkCwx5x+jnadZIayvP27TTzlyJohfLPH0KWUNO6xhCunp1/GeuXl3dZ0qyloMC8LPh4+vitrRjO7EpKPSPpAtZni0pMIyOxu8/Ohq8n6dtvUgncJysO4mkpTWHhpLPmTX6KCm4Dgzk+71UpTkwKCDUp52gsDV8vCNm+0rasZ3YFBTqVeYQ0qRUCXnvHOJkfQvv1NrC7Y7cKSrrnEl3Co27Bc0h6H9pQaGy1NlmdizwVWABwboNX3D3j5rZYuCvgHFgH3CJu/8gPGYj8D7gCPBb7n5b2mcodXZ9mtcygGAhl1Y7m1tZuyAq7s84K810N6wYVkUd2kkJ3tDOv6V0vzJSZ7/LzI4Ln/8HM/uSmb0247BDwJvd/eeBs4ALzewNwJXALndfQTD34crwvBPAGuBM4EJgi5lp2Y8ulbQYfFqmzTTtLPCSdGzVC950q6SLfjRza5Z2/i2lt+Udkvof3f15M/tF4AKCEUifTDsgvEt5IXw5HG5O0FG9PSzfDrwjfH4xcL27H3L3R4C9wDl5fxDprKQ1f/OuH9wsKY11O8eOjqaXL1kS/35SebfYsGEmPffQUPA6KukOo1F+7rn5PqeTa2NI98gbFBrfrd4KXOPuNwLHZB1kZoNmdg/wFHC7u38TOMndDwCEj68Idz8ZeDxy+P6wTLpQGXn9o+IWfmm2enWwNkF0LeLVq+GWW+Lz/P/4x/HnaZRffTUMD89+b3g4KC8i6yKdthZB0d9j87rSR44Er5s/M83evfn26+TaGNJFkjobohtwM/Ap4B8JJrEtAO7Nc2x4/PEEndOvAZ5reu8H4eNfAu+JlH8G+Dcx51oH7AZ2L1++vMSuFykiTwdpkY7ovPMUmhPNlZGTp53O8qzhn1l1LDo8NM/aBWk/d1onc1Id81Cyu95CCWkuRoBfBVaEr5cBb8lzbOQcHwU+DDwELIuc56Hw+UZgY2T/24A3pp1To4/qkzVyp2gajFZGGnUiz//xx88+7vjjZ7+fdZHOqmPREVB5fqa8v8u4gNHKRT0rKaB0n7SgkGv0kZm9Ctjv7ofM7Fzg54DPuftzKccsBX7q7s+Z2ULgy8B/BX4JeMbdrzKzK4HF7v4RMzsT+J8E/QivJOiEXuHuid2CGn1UnwUL4KWX5pYfcwwcOhQ0k8T1L4yNwb59c8uLjD4aGwvau5P+dM2C5SVbGdmzYUMw6iatM/r44+EHP5j5rLTPyHp/dDS+mWvRomBpz2Z5fqZWR3INDLTWCX/iifDMM3PLlyyBp59urS5SrbZHHwFfBI6Y2c8QNOucRnABT7MMuMPM7gO+RdCncDNBxtXzzexh4PzwNe6+B7gBeAC4Fbg8LSD0u7R26FZktXsXFRcQouVJnZRldF421kNO0mpbeHN7fZLnnpt5ntUnkPV+Vr9HJzWv25xXXEBIK5cul3QLEd2Ab4ePHwF+M3z+nTzHVrn1a/NR2RlIy1jXt1lWM0bR/DqtNnmktYUXbT4qMoEu7+826xxF61hl81GrayoX/RmkfpTQp/BNgqR49wOnhWX35zm2yq1fg0LZCcuqWFg9K5VCVX0KSVsZs4WLfF5UO8nnqggKeTvtkwJZUZ2aGS7lKSMoTAB/Abw7fH0acGWeY6vc+jUolJ2wrIpvcnkSnBUZkdJuUCjj5877Wc2dzWnKGCEVleeuL+4zh4Zm/q4GB4McTnkTC+b5Gctcn0Gq13ZQ6NatX4NC2d+8sr7Vt6Ld9Y+btZP7KOkCWvT3mGe4ZtGlL93Tg2Mr2UOz0lQ3f+aSJXNHB7W7ql2Rn1G6T1pQSO1oNrMbwsfvmtl9ke27YQey9ICFC4uV51F2B2k7qSeSRttcckmxcvd8n5XV6d88SACCEVdHjwaP0XxCp58ef46kcoAtW4LcTe7B45Ytc/eZnJz5zNFR+OlPZ79fdhqL6Oc1/4zSW1KHpJrZMnc/YGZjce+7e4tJDcrRr0NSBwbiL1CNoZZ1n69xbJI8F9dmxx0XPwQzr7jPLDosNimBXt7joXiiwKykfUmfsWlTMJJr+XLYvDn9IlzFv7/0tpaHpPpMOopHwwDwA+D5yCYVyFrise7zVaGKIZhFh8Xmzb+UNqy2aKLAokn7GkGnMSw3zzKlnfj3L3sItdQoqV0pugHvB54kSHX9SLj9U55jq9z6tU+h7I67PCOBirYJF23fz9JOf0J0iy6C08oormh7fdKWdnzR30vRkWGt/ExlD3Hu9PmlfJQw+uhh4MQ8+3Zy6+egUHbagLSLfiufV3ZQaLejOS4wtHKxareDtuhFvugcklZHplXZEVzFms9SrTKCwq3ASJ59O7n1a1Do9H+yVkY7lR0UktZDbnVrKHIxjFsnuhEcqryDyjOaqKEbL8B1rPks7UkLCnlzH70W+CzBJLZDkaan3yqtHasF6mguRyudxq0ck9ZBOjhY7s+W4896jqJ5iOJUnQfovPNg16655atXw86d7Z+/FUU79KV+ZeQ++hTwFeAfgLsjm1RgZKRYeS/I6iDthlEwZQyzPXSoWHlRd95ZrLwTNm+e+7c5MhKUS+/JGxQOu/vvuPtn3X17Y6u0ZvPYT35SrLxdVa1AFh2RsnZtuct3ppmYKP+ceSXdUaTdaRQZudONS4xOTgZDbsfGgjvIsTGt79zLhnLud4eZrQP+D7Obj56tpFbzXNK35qq+TV99NVx22ezMp8ccU3wFsqjm8fpJF62yl3ycmIA9e1o7dmAg/nectSJcO6am4Nd/fWZy2aOPBq8h/qI6OJg8r6FOk5MKAv0i75/7vyNYBOfvmWk66r/G/D5S5Nvn5CRs2zb7m962be39J48brx9n8eLWPyOq0b3ZakAAeP/7i5XHWbSoWPkVV8ydbfzTnwblcZLmUrSzxrXILEk90L2w9evoo3ZH9sQNxRweLjaKpt065h0l1Bjh1O5oo7J+riIjgeJUsZJa2XWsgnIf9RZaHZIKfCTy/F1N7/1h2rGd2BQU4uVJndzu5KKsOiYl4Ws3iGRtdWfnLDo8s91/626gyWu9Jy0oZDUfrYk839j03oUl3axIk3Y7fuOGBzarqpO3oUj/R5lt9i+9lNz00glFU0pU1cnfSUVTe0h3y/rvaAnP417PftPsVDO7w8weNLM9ZnZFWP4xM/u+md0TbhdFjtloZnvN7CEzu6DQT9JHrr466OiNarfjN07ZnbytCm48y1PnMpBFh2d26t+6SlUuvSqdlxUUPOF53Otmh4HfdfdXA28ALjezxmDBP3f3s8LtFoDwvTXAmQR3IVvMrOYxFfWoouM3TvTba9kJzVpdPL7XFR2e2al/6yr1QsJFyS9rSOrPm9mPCO4KFobPCV8fm3agBxlWG1lWnzezB4GTUw65GLje3Q8Bj5jZXuAc4BvZP4YUFf322jx8tDGxDFq/OJX97b+XFB2e2evDOTdvjk8XrslrvSkrdfagu7/M3Y9z96HweeP1cN4PMbNx4LUEaTIAPhgu1rPNzE4Iy04GHo8ctp/0INK3pqbg0ktnz/699NJy0hHHfXutok24zjbx0dH6PrtbdDKVtSav9ZdcuY/a+gCzUeDvgM3u/iUzOwl4mqD56T8Dy9z9MjP7S+Ab7r4jPO4zwC3u/sWm860D1gEsX7787Efz9Kr2mAULZk8kazjmmHzpEormJWol11LWZyTlAOqEsvIM9aqiC/3I/FNG7qNWP3gY+CIw5e5fAnD3J939iLsfBT5N0EQEwZ3BqZHDTwGeaD6nu29195XuvnLp0qVVVr82cQEhrbxdVbQJP1vjXPc6O5q7gUYDSTsqCwpmZsBngAfd/c8i5csiu70TuD98fhOwxswWmNlpwArgrqrqJzOqSGhW1kzlVtSd8qFuGg0k7cib+6gVq4BLge+a2T1h2e8B7zazswiaj/YRrOqGu+8xsxuABwhGLl3u7jWm+Zo/on0Ledf9bVWjqarKVss6k8N1g+XL4+eqaDSQ5FF5n0KV+nU9hVbWKogqY12ALMcdF3+u0VF4/vnsfooqh6zO9zz+6lOQLLX1KUg93vCGYuUAGzbA0FBwsR4aCl6nef3r08vrGruuoZAaDSTtUVCoQCeHA8b5ylfiy3ftiq/Thg1wzTUzzS5HjgSv0wJD0mc0yi+6KP79pPJ2jI7q4tdscjK4Wzp6NHjU70TyUvNRycq4dU9rWjHLbvPP0zQTrdPQUHKO/sOHi3+Ge/YSjWU2H6XVU0TmSms+UlAoWRnr1Ra9qLdyfLROVazR3Ok+hR7+MxbpOPUpdFCnhgOWMe68ynl/nexTmO9DUEXKpKBQsk5eDLt53HnW3IcyL+RadUykPAoKJetkB2s3jzvPGgFz7rmtn7sRUAYHYf162LKl7eqKSEh9CiVLyvlTJB9Pp/oUID1PUVqd251LkdT3kqXMuRYi85X6FDooKe9OWfl4qhh6efXVMNyU83Z4uNqFXlpt+kqbayEi7asyzYVUoMgyl3l1Ms1FQ1Iqhix33ll6VUQkQncKNSg6e7ioV76y+DGdnuwU1xGdx3zPayRSNQWFkiW1tTfKW5k9XNSBA+WdK0nWz5mluSM6Lw0/FamWgkLJkjpZG+Vbt8a/n1ReZh3SFL17efWri5XHid6dTExk7g5o+KlI1RQUOiyp+aPOZpFW7l4efLBYeZY9e+Y2ey1cqOGnIp2moNCCdhLeJTV/1Nks0srdS9YdUVFTU/Dcc7PLzGD79uCchw8rIIh0goJCQY2Ed48+GlysHn00eJ03MCQ1f9TRLHL88cFjN9y9aAlJke6goFBQuxevVauCNvuooaGgvNMa38xbuXsZHS1WnkVLSIp0hyrXaD7VzO4wswfNbI+ZXRGWLzaz283s4fDxhMgxG81sr5k9ZGYXVFW3drR78dq0aW6a58OH6/1G3Mrdyyc/GR/cPvnJ1upQ16I8IjJblXcKh4HfdfdXA28ALjezCeBKYJe7rwB2ha8J31sDnAlcCGwxs64bgNjuxStpwlaZGUuLzlPYsiXoxC3SqTs5CdddNzu30XXXtT6/oZM5o0QkWcdyH5nZjcB/D7dz3f2AmS0D7nT3M8xsI4C7/5dw/9uAj7n7N5LOWUfuo6xFdLJyAuXJGdRuXqGiuY+6QRnrUIhIPrXnPjKzceC1wDeBk9z9AED4+Ipwt5OBxyOH7Q/LusrkJKxdO/tb9RvfGDT/DKiHpmXqUxDpDpVfxsxsFPgi8CF3/1HarjFlc77Hmtk6M9ttZrsPHjxYVjVzm5oKhklGx/Tv2jUzGqlqda37XDX1KYh0h0qDgpkNEwSEKXf/Ulj8ZNhsRPj4VFi+Hzg1cvgpwBPN53T3re6+0t1XLl26tLrKJ4gbfdRJrQyD7QVZi/KISGdUOfrIgM8AD7r7n0XeuglYGz5fC9wYKV9jZgvM7DRgBXBXVfVrVZVLWBaRNgx20aLO1KGdSXzN4prl1q6tPjGfiMxW5Z3CKuBS4M1mdk+4XQRcBZxvZg8D54evcfc9wA3AA8CtwOXurpyYKZLa2z/1qXydze3Mom53El/c+T796dnNcp/+dH/dDYn0Aq28VlCRkT3N3INv1XG/crOZtRLyfkbSyJypKXjve7PXXmgnl1DZo4WOOy5+RbXRUXj++eLnE5FktY8+khll5QxKa2+/4op8i/HccEOxz4wqe7RQ0hKbWnpTpLMUFHpMnuU48y792c4SoRotJNKftBxnj6liOc5WbN4cP4mv1dFCZsnNaiLSObpT6EOdmETXvHJa1t1Llg98oFi5iFRDdwp9KO/dxJIl7X3O5GR5Q0YbHd5btwYjjwYHgzsRraEg0lm6U+hDY2P59rvkkmrrISK9R0GhD8XNDo7zuc9VX5e8WlkSVETKp6DQh5rb+5P8+Medq1OWVpYEFZHyKSj0qcnJYBJZt4xWytINS4KKiIJCxyXlJYqWr14dv09SeZakuwUN9xSRZgoKBbWabK6xGtqxx8a/Hy3fuzd+n6RyEZGyKCgU1Go7/PBw8Jg0izhaXvaSnWWl1qhS0oipvCOpRKQcCgod0sgJlJSZtJ2Mpf1A6ymIdAcFhRgbNsDQUNDmPjRUzrDIxYuDR3Woxit7hrSItEYzmps0xss3NMbLQzmza+vI8bN6dbBkaFx5NylzhrSItEZ3Ck2yxsu3mhri2WeDxzra93funBsAVq8OykVEohQUmmQ177SaGqLulNI7dwaBp7EpIIhInCrXaN5mZk+Z2f2Rso+Z2febludsvLfRzPaa2UNmdkFV9YL0tYWzOoJvuaX45w0Pz3SYJmUwjZZrJI6I1KXKO4XrgAtjyv/c3c8Kt1sAzGwCWAOcGR6zxcwqGY+TtbbwunXxxzXKW1lZLNpfkDTDOFqukTgiUpfKgoK7fxV4NufuFwPXu/shd38E2AucU0W9Nm2avTAMBK83bQqeb9kSrF3cuDMYHJy9lnFSM9DgYHDxj7vTeOmlmfPnuQvQSBwRqUsdfQofNLP7wualE8Kyk4HHI/vsD8tKl2dt4S1b4PDh4E7i8OHZo46SvsVv3x5820+6E2icP+9dQDR30b59Cggi0hmdDgrXAK8CzgIOAH8alscNyIwdj2Nm68xst5ntPnjwYOEKJKWUzpNqGuK/xa9dG9wJDAwk9xk07jDy3gWk9Xvk0e7xIjJPuXtlGzAO3J/1HrAR2Bh57zbgjVnnP/vss72ogYHoGJyZbWAg+ZgdO9zHxtzNgscdO2a/NzISf87GNjIy+5gscecsco52jxeR/gbs9qTrdtIbZWzNQQFYFnn+2wT9CBB0MN8LLABOA/4JGMw6fytBIe3iHSfrAjs2Fn+uwcH4IJJH0jnHxjpzvIj0t7SgYMH75TOzzwPnAicCTwIfDV+fRdA0tA94v7sfCPffBFwGHAY+5O5/m/UZK1eu9N27dxeq19BQ/FyEwcGg/wCCppZNm4J+gIGB+P3HxoK2/oGB5BnKra5l0O45k45vnGP58qAPQ/0UIvOTmd3t7ivj3qsszYW7vzum+DMp+28GKh90uW7d7DQW0XKYGbLaGKGUNJmt0XG8fHl89tJ2JqstXhyfTbWRPylLUp1g9jBcUGAQkdnm3YzmrCGncUNW4zQu+hddFP9+Unkn5FmjOToMV0SkobLmo05opfkoS1rTS8PIyMyIofHx+G/ljealMutQpEkq2gSW1pTUK8t1ikh50pqP5t2dQpasyWnNQ0jzzHsoqw5FmqSi8xySJszVnY9JRLqPgkKTrMlpzRPJWrmAZ80hKDvNRTc2cYlId1JQaFI0xUTRC3hW7qVW6pAlKYlfK8n9RKS/KSjEKJJiougFPCv3UhWqaOISkf6koNCC5uYfyB9EkoaKRsvz3E0UUUYfhYjMDwoKBbV7wc5arwHKv5tQKm4RyUtBoaB2L9hZK7tB+c09SsUtInlVNqO5X7V7wR4bS57X0FDFLOnJSQUBEcmmO4WC2m2fz9OUo+YeEamLgkJB7V6w8zTlqLlHROqiNBctiKaQUMZREek1tWRJ7WdqnxeRfqXmoxJo6UsR6Re6U2hT8/oLWqtARHqZ7hTaVEfaChGRqlQWFMxsm5k9ZWb3R8oWm9ntZvZw+HhC5L2NZrbXzB4yswuqqlfZlFdIRPpJlXcK1wEXNpVdCexy9xXArvA1ZjYBrAHODI/ZYmYJCSG6i/IKiUg/qSwouPtXgWebii8GtofPtwPviJRf7+6H3P0RYC9wTlV1K5MmmolIP+l0n8JJ7n4AIHx8RVh+MvB4ZL/9YVnX00QzEekn3TL6yGLKYmfVmdk6YB3A8i5po9G8BRHpF52+U3jSzJYBhI9PheX7gVMj+50CPBF3Anff6u4r3X3l0qVLK62siMh80+mgcBOwNny+FrgxUr7GzBaY2WnACuCuDtdNRGTeq6z5yMw+D5wLnGhm+4GPAlcBN5jZ+4DHgHcBuPseM7sBeAA4DFzu7gkrD4iISFUqCwru/u6Et1Yn7L8Z0JgdEZEaaUaziIhM6+nU2WZ2EIhZoyy3E4GnS6pOVVTHcqiO5VAdy1F3HcfcPXakTk8HhXaZ2e6knOLdQnUsh+pYDtWxHN1cRzUfiYjINAUFERGZNt+Dwta6K5CD6lgO1bEcqmM5uraO87pPQUREZpvvdwoiIhIx74JC3OI/3cbMTjWzO8zsQTPbY2ZX1F2nZmZ2rJndZWb3hnX8eN11SmJmg2b2HTO7ue66JDGzfWb2XTO7x8x2112fOGZ2vJl9wcy+F/5tvrHuOkWZ2Rnh76+x/cjMPlR3vZqZ2W+H/2fuN7PPm9mxddcpat41H5nZm4AXgM+5+2vqrk+cMFngMnf/tpkdB9wNvMPdH6i5atPMzIBF7v6CmQ0DXwOucPd/qLlqc5jZ7wArgZe5+9vqrk8cM9sHrHT3rh1fb2bbgf/r7tea2THAiLs/V3O1YoWLdH0feL27tzOXqVRmdjLB/5UJd/9JmN7nFne/rt6azZh3dwoJi/90FXc/4O7fDp8/DzxIl60v4YEXwpfD4dZ13zDM7BTgrcC1ddell5nZy4A3AZ8BcPeXujUghFYD/9hNASFiCFhoZkPACAkZoesy74JCrzGzceC1wDdrrsocYbPMPQQp0G93966rI/AJ4CPA0ZrrkcWBL5vZ3eGaId3mXwIHgc+GTXHXmtmiuiuVYg3w+bor0czdvw/8CUFC0APAD939y/XWajYFhS5mZqPAF4EPufuP6q5PM3c/4u5nEax/cY6ZdVVznJm9DXjK3e+uuy45rHL31wG/AlweNnN2kyHgdcA17v5a4MeEa6x3m7Bp6+3A/6q7Ls3M7ASC5YdPA14JLDKz99Rbq9kUFLpU2E7/RWDK3b9Ud33ShM0IdwIX1luTOVYBbw/b668H3mxmO+qtUjx3fyJ8fAr4a7pvjfL9wP7I3eAXCIJEN/oV4Nvu/mTdFYlxHvCIux90958CXwL+dc11mkVBoQuFnbifAR509z+ruz5xzGypmR0fPl9I8Mf+vVor1cTdN7r7Ke4+TtCc8BV376pvZQBmtigcUEDYJPMWoKtGx7n7/wMeN7MzwqLVBOufdKN304VNR6HHgDeY2Uj4/3w1QZ9h15h3QSFc/OcbwBlmtj9c8KfbrAIuJfhm2xhed1HdlWqyDLjDzO4DvkXQp9C1Qz673EnA18zsXoIVB//G3W+tuU5xfhOYCv/NzwL+sN7qzGVmI8D5BN/Au054p/UF4NvAdwmuwV01u3neDUkVEZFk8+5OQUREkikoiIjINAUFERGZpqAgIiLTFBRERGSagoLMG2Z2pCmLZsszcs3s78usW9O5V5rZX1R1fpE0GpIq84aZveDuo3XXQ6Sb6U5B5r1wLYOPm9m3wzUNfjYsX2pmt4flnzKzR83sxPC9F8LHc83szsg6A1PhTFXM7Gwz+7swyd1tYUr05s9+V5hX/14z+2rknDeHz2+J3Nn80MzWhokI/9jMvmVm95nZ+zv1u5L+p6Ag88nCpuajfxt57+kwId01wIfDso8SpMZ4HUE+ouUJ530t8CFggiCb6Kowd9V/A37N3c8GtgGbY479feACd/95giRus7j7RWHSwfcBjwL/O3z+Q3f/BeAXgN8ws9Ny/g5EUg3VXQGRDvpJeIGN00iLcDfwq+HzXwTeCeDut5rZDxKOvcvd9wOEqcTHgeeA1wC3hzcOgwSpkpt9HbguXGwlNjVDeHfyP4BL3P2HZvYW4OfM7NfCXV4OrAAeSaifSG4KCiKBQ+HjEWb+X1jBY6PHG7DH3VOXrHT3D5jZ6wkWArrHzM6Kvh+uIHY98J/cvZEkz4DfdPfbctZPJDc1H4kk+xpwCUD47fyEAsc+BCy1cB1jMxs2szObdzKzV7n7N93994GngVObdrkKuM/dr4+U3QasD5uoMLPTu3zBG+khulOQ+WRh2LzTcKu7pw1L/Tjw+bDv4e8Imn+ez/NB7v5S2LzzF2b2coL/a58A9jTt+sdmtoLg2/8u4F7glyLvfxjYE6n37xMsLToOfDvs1D4IvCNPvUSyaEiqSAIzWwAccffD4Tf+a1L6JET6gu4URJItB24wswHgJeA3aq6PSOV0pyAiItPU0SwiItMUFEREZJqCgoiITFNQEBGRaQoKIiIyTUFBRESm/X9NQB4UQUJFHQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(train.ENGINESIZE, train.CO2EMISSIONS,  color='blue')\n",
    "plt.xlabel(\"Engine size\")\n",
    "plt.ylabel(\"Emission\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "<h2 id=\"multiple_regression_model\">Multiple Regression Model</h2>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In reality, there are multiple variables that impact the co2emission. When more than one independent variable is present, the process is called multiple linear regression. An example of multiple linear regression is predicting co2emission using the features FUELCONSUMPTION_COMB, EngineSize and Cylinders of cars. The good thing here is that multiple linear regression model is the extension of the simple linear regression model.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Coefficients:  [[ 9.07707365  7.93000774 10.03575069]]\n"
     ]
    }
   ],
   "source": [
    "from sklearn import linear_model\n",
    "regr = linear_model.LinearRegression()\n",
    "x = np.asanyarray(train[['ENGINESIZE','CYLINDERS','FUELCONSUMPTION_COMB']])\n",
    "y = np.asanyarray(train[['CO2EMISSIONS']])\n",
    "regr.fit (x, y)\n",
    "# The coefficients\n",
    "print ('Coefficients: ', regr.coef_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As mentioned before, **Coefficient** and **Intercept**  are the parameters of the fitted line.\n",
    "Given that it is a multiple linear regression model with 3 parameters and that the parameters are the intercept and coefficients of the hyperplane, sklearn can estimate them from our data. Scikit-learn uses plain Ordinary Least Squares method to solve this problem.\n",
    "\n",
    "#### Ordinary Least Squares (OLS)\n",
    "\n",
    "OLS is a method for estimating the unknown parameters in a linear regression model. OLS chooses the parameters of a linear function of a set of explanatory variables by minimizing the sum of the squares of the differences between the target dependent variable and those predicted by the linear function. In other words, it tries to minimizes the sum of squared errors (SSE) or mean squared error (MSE) between the target variable (y) and our predicted output ($\\hat{y}$) over all samples in the dataset.\n",
    "\n",
    "OLS can find the best parameters using of the following methods:\n",
    "\n",
    "*   Solving the model parameters analytically using closed-form equations\n",
    "*   Using an optimization algorithm (Gradient Descent, Stochastic Gradient Descent, Newton’s Method, etc.)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 id=\"prediction\">Prediction</h2>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Residual sum of squares: 582.12\n",
      "Variance score: 0.87\n"
     ]
    }
   ],
   "source": [
    "y_hat= regr.predict(test[['ENGINESIZE','CYLINDERS','FUELCONSUMPTION_COMB']])\n",
    "x = np.asanyarray(test[['ENGINESIZE','CYLINDERS','FUELCONSUMPTION_COMB']])\n",
    "y = np.asanyarray(test[['CO2EMISSIONS']])\n",
    "print(\"Residual sum of squares: %.2f\"\n",
    "      % np.mean((y_hat - y) ** 2))\n",
    "\n",
    "# Explained variance score: 1 is perfect prediction\n",
    "print('Variance score: %.2f' % regr.score(x, y))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Explained variance regression score:**\\\n",
    "Let $\\hat{y}$ be the estimated target output, y the corresponding (correct) target output, and Var be the Variance (the square of the standard deviation). Then the explained variance is estimated as follows:\n",
    "\n",
    "$\\texttt{explainedVariance}(y, \\hat{y}) = 1 - \\frac{Var{ y - \\hat{y}}}{Var{y}}$\\\n",
    "The best possible score is 1.0, the lower values are worse.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 id=\"practice\">Practice</h2>\n",
    "Try to use a multiple linear regression with the same dataset, but this time use FUELCONSUMPTION_CITY and FUELCONSUMPTION_HWY instead of FUELCONSUMPTION_COMB. Does it result in better accuracy?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Coefficients:  [[9.13199983 7.7081688  6.11790593 3.71269608]]\n",
      "Residual sum of squares: 579.86\n",
      "Variance score: 0.87\n"
     ]
    }
   ],
   "source": [
    "# write your code here\n",
    "regr = linear_model.LinearRegression()\n",
    "x = np.asanyarray(train[['ENGINESIZE','CYLINDERS','FUELCONSUMPTION_CITY','FUELCONSUMPTION_HWY']])\n",
    "y = np.asanyarray(train[['CO2EMISSIONS']])\n",
    "regr.fit (x, y)\n",
    "print ('Coefficients: ', regr.coef_)\n",
    "y_= regr.predict(test[['ENGINESIZE','CYLINDERS','FUELCONSUMPTION_CITY','FUELCONSUMPTION_HWY']])\n",
    "x = np.asanyarray(test[['ENGINESIZE','CYLINDERS','FUELCONSUMPTION_CITY','FUELCONSUMPTION_HWY']])\n",
    "y = np.asanyarray(test[['CO2EMISSIONS']])\n",
    "print(\"Residual sum of squares: %.2f\"% np.mean((y_ - y) ** 2))\n",
    "print('Variance score: %.2f' % regr.score(x, y))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details><summary>Click here for the solution</summary>\n",
    "\n",
    "```python\n",
    "regr = linear_model.LinearRegression()\n",
    "x = np.asanyarray(train[['ENGINESIZE','CYLINDERS','FUELCONSUMPTION_CITY','FUELCONSUMPTION_HWY']])\n",
    "y = np.asanyarray(train[['CO2EMISSIONS']])\n",
    "regr.fit (x, y)\n",
    "print ('Coefficients: ', regr.coef_)\n",
    "y_= regr.predict(test[['ENGINESIZE','CYLINDERS','FUELCONSUMPTION_CITY','FUELCONSUMPTION_HWY']])\n",
    "x = np.asanyarray(test[['ENGINESIZE','CYLINDERS','FUELCONSUMPTION_CITY','FUELCONSUMPTION_HWY']])\n",
    "y = np.asanyarray(test[['CO2EMISSIONS']])\n",
    "print(\"Residual sum of squares: %.2f\"% np.mean((y_ - y) ** 2))\n",
    "print('Variance score: %.2f' % regr.score(x, y))\n",
    "\n",
    "```\n",
    "\n",
    "</details>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "<h2>Want to learn more?</h2>\n",
    "\n",
    "IBM SPSS Modeler is a comprehensive analytics platform that has many machine learning algorithms. It has been designed to bring predictive intelligence to decisions made by individuals, by groups, by systems – by your enterprise as a whole. A free trial is available through this course, available here: <a href=\"https://www.ibm.com/analytics/spss-statistics-software?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkML0101ENSkillsNetwork20718538-2021-01-01\">SPSS Modeler</a>\n",
    "\n",
    "Also, you can use Watson Studio to run these notebooks faster with bigger datasets. Watson Studio is IBM's leading cloud solution for data scientists, built by data scientists. With Jupyter notebooks, RStudio, Apache Spark and popular libraries pre-packaged in the cloud, Watson Studio enables data scientists to collaborate on their projects without having to install anything. Join the fast-growing community of Watson Studio users today with a free account at <a href=\"https://www.ibm.com/cloud/watson-studio?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkML0101ENSkillsNetwork20718538-2021-01-01\">Watson Studio</a>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Thank you for completing this lab!\n",
    "\n",
    "## Author\n",
    "\n",
    "Saeed Aghabozorgi\n",
    "\n",
    "### Other Contributors\n",
    "\n",
    "<a href=\"https://www.linkedin.com/in/joseph-s-50398b136/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkML0101ENSkillsNetwork20718538-2021-01-01\" target=\"_blank\">Joseph Santarcangelo</a>\n",
    "\n",
    "## Change Log\n",
    "\n",
    "| Date (YYYY-MM-DD) | Version | Changed By | Change Description                 |\n",
    "| ----------------- | ------- | ---------- | ---------------------------------- |\n",
    "| 2020-11-03        | 2.1     | Lakshmi    | Made changes in URL                |\n",
    "| 2020-08-27        | 2.0     | Lavanya    | Moved lab to course repo in GitLab |\n",
    "|                   |         |            |                                    |\n",
    "|                   |         |            |                                    |\n",
    "\n",
    "## <h3 align=\"center\"> © IBM Corporation 2020. All rights reserved. <h3/>\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python",
   "language": "python",
   "name": "conda-env-python-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.12"
  },
  "widgets": {
   "state": {},
   "version": "1.1.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}