Ubaydulloh_AI_SL-Regression-codes.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Ubaydulloh[22212410]_AI-Intro_ass1.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "8tDVB54iM208",
        "VbzRvEtP0Bi6",
        "aqJ1FF771T0i",
        "K1Byt5xSxSaW",
        "F-lunXMR2hSV"
      ]
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "#   Boston Housing Data\n",
        "\n",
        "- Creator: Harrison, D. and Rubinfeld, D.L. 'Hedonic prices and the demand for clean air', J. Environ. Economics & Management, vol.5, 81-102, 1978. \n",
        "- :Purpose: To predic housing price in Boston area\n",
        "- Feature: \n",
        "1. CRIM           per capita crime rate by town \n",
        "2. ZN               proportion of residential land zoned for lots over 25,000 sq.ft. \n",
        "3. INDUS         proportion of non-retail business acres per town \n",
        "4. CHAS          Charles River dummy variable (= 1 if tract bounds river; 0 otherwise) \n",
        "5. NOX            nitric oxides concentration (parts per 10 million) \n",
        "6. RM              average number of rooms per dwelling \n",
        "7. AGE             proportion of owner-occupied units built prior to 1940 \n",
        "8. DIS              weighted distances to five Boston employment centres \n",
        "9. RAD            index of accessibility to radial highways \n",
        "10. TAX             full-value property-tax rate per $10,000 \n",
        "11. PTRATIO    pupil-teacher ratio by town \n",
        "12. B               1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town \n",
        "13. LSTAT         % lower status of the population \n",
        "\n",
        "- target: MEDV Median housing price value of owner-occupied homes in $1000's \n",
        "- Repository: https://raw.githubusercontent.com/sesillim/ai/main/Housing.csv"
      ],
      "metadata": {
        "id": "d2kgGqxiLfkS"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 1 How many observations are in the data?"
      ],
      "metadata": {
        "id": "8tDVB54iM208"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "z2L5TQPyKCkC"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "import sklearn\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "url = 'https://raw.githubusercontent.com/sesillim/ai/main/Housing.csv'\n",
        "data= pd.read_csv(url)"
      ],
      "metadata": {
        "id": "1es4A5nnTTKX"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "data.keys()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8BKoaAo8XLr1",
        "outputId": "6580c8fc-4170-41d7-986e-e07a8a475a24"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Index(['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE', 'DIS', 'RAD', 'TAS',\n",
              "       'PTRATIO', 'B', 'LSTAT', 'MEDV'],\n",
              "      dtype='object')"
            ]
          },
          "metadata": {},
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "data.shape\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Y18Y0jReTjV3",
        "outputId": "e011e067-356b-4dc4-8fcf-e80c80477a79"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(506, 14)"
            ]
          },
          "metadata": {},
          "execution_count": 34
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**There are 506 observations in the data for 14 variables with the orice of houses in Boston**"
      ],
      "metadata": {
        "id": "sQgRiRVJziiD"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "data.head()\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "rJqQEgwQT7lw",
        "outputId": "cb96e099-0667-4669-b226-b3264a151c4d"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "      CRIM    ZN  INDUS  CHAS    NOX     RM   AGE     DIS  RAD  TAS  PTRATIO  \\\n",
              "0  0.00632  18.0   2.31     0  0.538  6.575  65.2  4.0900    1  296     15.3   \n",
              "1  0.02731   0.0   7.07     0  0.469  6.421  78.9  4.9671    2  242     17.8   \n",
              "2  0.02729   0.0   7.07     0  0.469  7.185  61.1  4.9671    2  242     17.8   \n",
              "3  0.03237   0.0   2.18     0  0.458  6.998  45.8  6.0622    3  222     18.7   \n",
              "4  0.06905   0.0   2.18     0  0.458  7.147  54.2  6.0622    3  222     18.7   \n",
              "\n",
              "        B  LSTAT  MEDV  \n",
              "0  396.90   4.98  24.0  \n",
              "1  396.90   9.14  21.6  \n",
              "2  392.83   4.03  34.7  \n",
              "3  394.63   2.94  33.4  \n",
              "4  396.90   5.33  36.2  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-c2f52355-8c14-4055-ba60-bee8db475093\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>CRIM</th>\n",
              "      <th>ZN</th>\n",
              "      <th>INDUS</th>\n",
              "      <th>CHAS</th>\n",
              "      <th>NOX</th>\n",
              "      <th>RM</th>\n",
              "      <th>AGE</th>\n",
              "      <th>DIS</th>\n",
              "      <th>RAD</th>\n",
              "      <th>TAS</th>\n",
              "      <th>PTRATIO</th>\n",
              "      <th>B</th>\n",
              "      <th>LSTAT</th>\n",
              "      <th>MEDV</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0.00632</td>\n",
              "      <td>18.0</td>\n",
              "      <td>2.31</td>\n",
              "      <td>0</td>\n",
              "      <td>0.538</td>\n",
              "      <td>6.575</td>\n",
              "      <td>65.2</td>\n",
              "      <td>4.0900</td>\n",
              "      <td>1</td>\n",
              "      <td>296</td>\n",
              "      <td>15.3</td>\n",
              "      <td>396.90</td>\n",
              "      <td>4.98</td>\n",
              "      <td>24.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.02731</td>\n",
              "      <td>0.0</td>\n",
              "      <td>7.07</td>\n",
              "      <td>0</td>\n",
              "      <td>0.469</td>\n",
              "      <td>6.421</td>\n",
              "      <td>78.9</td>\n",
              "      <td>4.9671</td>\n",
              "      <td>2</td>\n",
              "      <td>242</td>\n",
              "      <td>17.8</td>\n",
              "      <td>396.90</td>\n",
              "      <td>9.14</td>\n",
              "      <td>21.6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0.02729</td>\n",
              "      <td>0.0</td>\n",
              "      <td>7.07</td>\n",
              "      <td>0</td>\n",
              "      <td>0.469</td>\n",
              "      <td>7.185</td>\n",
              "      <td>61.1</td>\n",
              "      <td>4.9671</td>\n",
              "      <td>2</td>\n",
              "      <td>242</td>\n",
              "      <td>17.8</td>\n",
              "      <td>392.83</td>\n",
              "      <td>4.03</td>\n",
              "      <td>34.7</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0.03237</td>\n",
              "      <td>0.0</td>\n",
              "      <td>2.18</td>\n",
              "      <td>0</td>\n",
              "      <td>0.458</td>\n",
              "      <td>6.998</td>\n",
              "      <td>45.8</td>\n",
              "      <td>6.0622</td>\n",
              "      <td>3</td>\n",
              "      <td>222</td>\n",
              "      <td>18.7</td>\n",
              "      <td>394.63</td>\n",
              "      <td>2.94</td>\n",
              "      <td>33.4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0.06905</td>\n",
              "      <td>0.0</td>\n",
              "      <td>2.18</td>\n",
              "      <td>0</td>\n",
              "      <td>0.458</td>\n",
              "      <td>7.147</td>\n",
              "      <td>54.2</td>\n",
              "      <td>6.0622</td>\n",
              "      <td>3</td>\n",
              "      <td>222</td>\n",
              "      <td>18.7</td>\n",
              "      <td>396.90</td>\n",
              "      <td>5.33</td>\n",
              "      <td>36.2</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-c2f52355-8c14-4055-ba60-bee8db475093')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-c2f52355-8c14-4055-ba60-bee8db475093 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-c2f52355-8c14-4055-ba60-bee8db475093');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 35
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 2 Build a linear regression model. Print the coefficients of the model and assess the fit. "
      ],
      "metadata": {
        "id": "VbzRvEtP0Bi6"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# divide data into x and y\n",
        "# split data into training and test (30%)\n",
        "X =data.drop(['MEDV'], axis='columns')\n",
        "y=data['MEDV']\n",
        "\n",
        "from sklearn.model_selection import train_test_split\n",
        "X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.3,random_state = 0)"
      ],
      "metadata": {
        "id": "tYQ78MRiZY8E"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# build model and train the model with training set\n",
        "from sklearn.linear_model import LinearRegression\n",
        "lr = LinearRegression()\n",
        "lr.fit(X_train, y_train)"
      ],
      "metadata": {
        "id": "pnmCQ-uaaT7W",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "bf2d4f78-4b2a-450b-8f4a-28948f78b83d"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LinearRegression()"
            ]
          },
          "metadata": {},
          "execution_count": 37
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Baseline: Linear Regression\n",
        "# possibility of overfitting\n",
        "df=pd.DataFrame(lr.coef_,X.columns, columns=['Coefficient'])\n",
        "print(\" A linear regression with non-scaled data:\")\n",
        "print(df)\n",
        "print(\"R-squared for a training set:\", lr.score(X_train, y_train))\n",
        "print(\"R-squared for a test set:\", lr.score(X_test, y_test))\n",
        "y_pred=lr.predict(X_test)\n",
        "from sklearn import metrics\n",
        "print(\"MAE:\", metrics.mean_absolute_error(y_test,y_pred))\n",
        "print(\"MSE:\", metrics.mean_squared_error(y_test,y_pred))\n",
        "print(\"RMSE:\", np.sqrt(metrics.mean_squared_error(y_test,y_pred)))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wU8fnpcujHMB",
        "outputId": "b63f4978-d270-4ef7-c0d9-3729736efc8d"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " A linear regression with non-scaled data:\n",
            "         Coefficient\n",
            "CRIM       -0.121310\n",
            "ZN          0.044466\n",
            "INDUS       0.011342\n",
            "CHAS        2.511246\n",
            "NOX       -16.231253\n",
            "RM          3.859068\n",
            "AGE        -0.009985\n",
            "DIS        -1.500270\n",
            "RAD         0.242143\n",
            "TAS        -0.011072\n",
            "PTRATIO    -1.017753\n",
            "B           0.006814\n",
            "LSTAT      -0.486738\n",
            "R-squared for a training set: 0.7645451026942549\n",
            "R-squared for a test set: 0.6733825506400184\n",
            "MAE: 3.609904060381819\n",
            "MSE: 27.195965766883308\n",
            "RMSE: 5.214975145375413\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.linear_model import Ridge\n",
        "ridge10 = Ridge(alpha=10).fit(X_train,y_train)"
      ],
      "metadata": {
        "id": "HBAl8xRtxsdw"
      },
      "execution_count": 75,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df_ridge10=pd.DataFrame(ridge10.coef_,X.columns, columns=['Coefficient'])\n",
        "print(\"ridge regresssion with lambda 10 & non-scaled data:\")\n",
        "print(df_ridge10)\n",
        "print(\"R-squared for a training set:\", ridge10.score(X_train,y_train))\n",
        "print(\"R-squared for a test set:\", ridge10.score(X_test,y_test))\n",
        "y_pred=ridge10.predict(X_test)\n",
        "from sklearn import metrics\n",
        "print(\"MAE:\", metrics.mean_absolute_error(y_test,y_pred))\n",
        "print(\"MSE:\", metrics.mean_squared_error(y_test,y_pred))\n",
        "print(\"RMSE:\", np.sqrt(metrics.mean_squared_error(y_test,y_pred)))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jaZ8O6ToxyIQ",
        "outputId": "acd38845-9dc0-41cd-b59c-324ed7829b7d"
      },
      "execution_count": 78,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "ridge regresssion with lambda 10 & non-scaled data:\n",
            "         Coefficient\n",
            "CRIM       -0.116008\n",
            "ZN          0.048609\n",
            "INDUS      -0.047175\n",
            "CHAS        1.785961\n",
            "NOX        -1.540350\n",
            "RM          3.620640\n",
            "AGE        -0.021636\n",
            "DIS        -1.309680\n",
            "RAD         0.211507\n",
            "TAS        -0.012664\n",
            "PTRATIO    -0.876398\n",
            "B           0.007688\n",
            "LSTAT      -0.524099\n",
            "R-squared for a training set: 0.7561651276850343\n",
            "R-squared for a test set: 0.656796352915132\n",
            "MAE: 3.6459408352561233\n",
            "MSE: 28.577023840824758\n",
            "RMSE: 5.345748202153255\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Here, i have build a Linear Regression model and printed the coefficients of model with the assessment of the fit.**"
      ],
      "metadata": {
        "id": "qlMWbsnjDtlV"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "data.describe()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 364
        },
        "id": "JvhAHRU8jbIQ",
        "outputId": "1417836f-e50e-4f7d-d025-6b1d7615ee5e"
      },
      "execution_count": 84,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "             CRIM          ZN       INDUS        CHAS         NOX          RM  \\\n",
              "count  506.000000  506.000000  506.000000  506.000000  506.000000  506.000000   \n",
              "mean     3.613524   11.363636   11.136779    0.069170    0.554695    6.284634   \n",
              "std      8.601545   23.322453    6.860353    0.253994    0.115878    0.702617   \n",
              "min      0.006320    0.000000    0.460000    0.000000    0.385000    3.561000   \n",
              "25%      0.082045    0.000000    5.190000    0.000000    0.449000    5.885500   \n",
              "50%      0.256510    0.000000    9.690000    0.000000    0.538000    6.208500   \n",
              "75%      3.677083   12.500000   18.100000    0.000000    0.624000    6.623500   \n",
              "max     88.976200  100.000000   27.740000    1.000000    0.871000    8.780000   \n",
              "\n",
              "              AGE         DIS         RAD         TAS     PTRATIO           B  \\\n",
              "count  506.000000  506.000000  506.000000  506.000000  506.000000  506.000000   \n",
              "mean    68.574901    3.795043    9.549407  408.237154   18.455534  356.674032   \n",
              "std     28.148861    2.105710    8.707259  168.537116    2.164946   91.294864   \n",
              "min      2.900000    1.129600    1.000000  187.000000   12.600000    0.320000   \n",
              "25%     45.025000    2.100175    4.000000  279.000000   17.400000  375.377500   \n",
              "50%     77.500000    3.207450    5.000000  330.000000   19.050000  391.440000   \n",
              "75%     94.075000    5.188425   24.000000  666.000000   20.200000  396.225000   \n",
              "max    100.000000   12.126500   24.000000  711.000000   22.000000  396.900000   \n",
              "\n",
              "            LSTAT        MEDV  \n",
              "count  506.000000  506.000000  \n",
              "mean    12.653063   22.532806  \n",
              "std      7.141062    9.197104  \n",
              "min      1.730000    5.000000  \n",
              "25%      6.950000   17.025000  \n",
              "50%     11.360000   21.200000  \n",
              "75%     16.955000   25.000000  \n",
              "max     37.970000   50.000000  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-3e8849b0-eb70-4dda-98b1-814f65d10cee\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>CRIM</th>\n",
              "      <th>ZN</th>\n",
              "      <th>INDUS</th>\n",
              "      <th>CHAS</th>\n",
              "      <th>NOX</th>\n",
              "      <th>RM</th>\n",
              "      <th>AGE</th>\n",
              "      <th>DIS</th>\n",
              "      <th>RAD</th>\n",
              "      <th>TAS</th>\n",
              "      <th>PTRATIO</th>\n",
              "      <th>B</th>\n",
              "      <th>LSTAT</th>\n",
              "      <th>MEDV</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>506.000000</td>\n",
              "      <td>506.000000</td>\n",
              "      <td>506.000000</td>\n",
              "      <td>506.000000</td>\n",
              "      <td>506.000000</td>\n",
              "      <td>506.000000</td>\n",
              "      <td>506.000000</td>\n",
              "      <td>506.000000</td>\n",
              "      <td>506.000000</td>\n",
              "      <td>506.000000</td>\n",
              "      <td>506.000000</td>\n",
              "      <td>506.000000</td>\n",
              "      <td>506.000000</td>\n",
              "      <td>506.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>3.613524</td>\n",
              "      <td>11.363636</td>\n",
              "      <td>11.136779</td>\n",
              "      <td>0.069170</td>\n",
              "      <td>0.554695</td>\n",
              "      <td>6.284634</td>\n",
              "      <td>68.574901</td>\n",
              "      <td>3.795043</td>\n",
              "      <td>9.549407</td>\n",
              "      <td>408.237154</td>\n",
              "      <td>18.455534</td>\n",
              "      <td>356.674032</td>\n",
              "      <td>12.653063</td>\n",
              "      <td>22.532806</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>8.601545</td>\n",
              "      <td>23.322453</td>\n",
              "      <td>6.860353</td>\n",
              "      <td>0.253994</td>\n",
              "      <td>0.115878</td>\n",
              "      <td>0.702617</td>\n",
              "      <td>28.148861</td>\n",
              "      <td>2.105710</td>\n",
              "      <td>8.707259</td>\n",
              "      <td>168.537116</td>\n",
              "      <td>2.164946</td>\n",
              "      <td>91.294864</td>\n",
              "      <td>7.141062</td>\n",
              "      <td>9.197104</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>0.006320</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.460000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.385000</td>\n",
              "      <td>3.561000</td>\n",
              "      <td>2.900000</td>\n",
              "      <td>1.129600</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>187.000000</td>\n",
              "      <td>12.600000</td>\n",
              "      <td>0.320000</td>\n",
              "      <td>1.730000</td>\n",
              "      <td>5.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>0.082045</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>5.190000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.449000</td>\n",
              "      <td>5.885500</td>\n",
              "      <td>45.025000</td>\n",
              "      <td>2.100175</td>\n",
              "      <td>4.000000</td>\n",
              "      <td>279.000000</td>\n",
              "      <td>17.400000</td>\n",
              "      <td>375.377500</td>\n",
              "      <td>6.950000</td>\n",
              "      <td>17.025000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>0.256510</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>9.690000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.538000</td>\n",
              "      <td>6.208500</td>\n",
              "      <td>77.500000</td>\n",
              "      <td>3.207450</td>\n",
              "      <td>5.000000</td>\n",
              "      <td>330.000000</td>\n",
              "      <td>19.050000</td>\n",
              "      <td>391.440000</td>\n",
              "      <td>11.360000</td>\n",
              "      <td>21.200000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>3.677083</td>\n",
              "      <td>12.500000</td>\n",
              "      <td>18.100000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.624000</td>\n",
              "      <td>6.623500</td>\n",
              "      <td>94.075000</td>\n",
              "      <td>5.188425</td>\n",
              "      <td>24.000000</td>\n",
              "      <td>666.000000</td>\n",
              "      <td>20.200000</td>\n",
              "      <td>396.225000</td>\n",
              "      <td>16.955000</td>\n",
              "      <td>25.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>88.976200</td>\n",
              "      <td>100.000000</td>\n",
              "      <td>27.740000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.871000</td>\n",
              "      <td>8.780000</td>\n",
              "      <td>100.000000</td>\n",
              "      <td>12.126500</td>\n",
              "      <td>24.000000</td>\n",
              "      <td>711.000000</td>\n",
              "      <td>22.000000</td>\n",
              "      <td>396.900000</td>\n",
              "      <td>37.970000</td>\n",
              "      <td>50.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-3e8849b0-eb70-4dda-98b1-814f65d10cee')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-3e8849b0-eb70-4dda-98b1-814f65d10cee button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-3e8849b0-eb70-4dda-98b1-814f65d10cee');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 84
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 3 Examine whether the scaling improves the fit of the linear regression model. You can choose any scaling method for the examination."
      ],
      "metadata": {
        "id": "aqJ1FF771T0i"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# MinMax Scalar\n",
        "from sklearn.preprocessing import MinMaxScaler\n",
        "scaler = MinMaxScaler()\n",
        "\n",
        "scaler.fit(X_train)\n",
        "X_train_scaled = scaler.transform(X_train)\n",
        "X_test_scaled = scaler.transform(X_test)"
      ],
      "metadata": {
        "id": "QDShvTK_jqX6"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Standard Scaler\n",
        "from sklearn.preprocessing import StandardScaler\n",
        "scaler = StandardScaler()\n",
        "\n",
        "scaler.fit(X_train)\n",
        "X_train_scaled = scaler.transform(X_train)\n",
        "X_test_scaled = scaler.transform(X_test)"
      ],
      "metadata": {
        "id": "Y868GU0rkBk5"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "lr_scale = LinearRegression()\n",
        "lr_scale .fit(X_train_scaled,y_train)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "F7AbtM0Bl4_D",
        "outputId": "fae7f196-5b70-4db5-87ea-638e2acdee02"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LinearRegression()"
            ]
          },
          "metadata": {},
          "execution_count": 46
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df_scale=pd.DataFrame(lr_scale.coef_,X.columns, columns=['Coefficient'])\n",
        "print(\"A linear regresssion with scaled data:\")\n",
        "print(df_scale)\n",
        "print(\"R-squared for a training set:\", lr_scale.score(X_train_scaled,y_train))\n",
        "print(\"R-squared for a test set:\", lr_scale.score(X_test_scaled,y_test))\n",
        "y_pred=lr_scale.predict(X_test_scaled)\n",
        "from sklearn import metrics\n",
        "print(\"MAE:\", metrics.mean_absolute_error(y_test,y_pred))\n",
        "print(\"MSE:\", metrics.mean_squared_error(y_test,y_pred))\n",
        "print(\"RMSE:\", np.sqrt(metrics.mean_squared_error(y_test,y_pred)))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bNbZ7gWNkFGQ",
        "outputId": "33ba942d-4fd5-4f9b-d0ca-4c8a6b4d3bab"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "A linear regresssion with scaled data:\n",
            "         Coefficient\n",
            "CRIM       -1.011901\n",
            "ZN          1.050280\n",
            "INDUS       0.079210\n",
            "CHAS        0.618962\n",
            "NOX        -1.873691\n",
            "RM          2.705270\n",
            "AGE        -0.279573\n",
            "DIS        -3.097665\n",
            "RAD         2.096900\n",
            "TAS        -1.886063\n",
            "PTRATIO    -2.261105\n",
            "B           0.582643\n",
            "LSTAT      -3.440498\n",
            "R-squared for a training set: 0.7645451026942549\n",
            "R-squared for a test set: 0.6733825506400195\n",
            "MAE: 3.6099040603818127\n",
            "MSE: 27.195965766883212\n",
            "RMSE: 5.214975145375403\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**After utilizing the feature scaling, it improved the fit of the linear regression model with 1 to 2 percents better output. and i have experienced Minmax and Standard Feature Scaling methods on them.** \n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "0PIcizuw2_UD"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 4 Examine whether implementing L2 regularization with scaling improves model performance. To optimize the hyperparameter, you have to compare at least 5 different hyperparameter values. "
      ],
      "metadata": {
        "id": "K1Byt5xSxSaW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.linear_model import Ridge\n",
        "ridge10_scale = Ridge(alpha=10).fit(X_train_scaled,y_train)\n"
      ],
      "metadata": {
        "id": "-E6iqNZulxDR"
      },
      "execution_count": 87,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df_ridge10_scale=pd.DataFrame(ridge10_scale.coef_,X.columns, columns=['Coefficient'])\n",
        "print(\"ridge regresssion with lambda 10 & scaled data:\")\n",
        "print(df_ridge10_scale)\n",
        "print(\"R-squared for a training set:\", ridge10_scale.score(X_train_scaled,y_train))\n",
        "print(\"R-squared for a test set:\", ridge10_scale.score(X_test_scaled,y_test))\n",
        "y_pred=ridge10_scale.predict(X_test_scaled)\n",
        "from sklearn import metrics\n",
        "print(\"MAE:\", metrics.mean_absolute_error(y_test,y_pred))\n",
        "print(\"MSE:\", metrics.mean_squared_error(y_test,y_pred))\n",
        "print(\"RMSE:\", np.sqrt(metrics.mean_squared_error(y_test,y_pred)))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "z_9v4EMwmgDO",
        "outputId": "f7f03bbe-8701-4108-9eb6-e87498aa942a"
      },
      "execution_count": 95,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "ridge regresssion with lambda 10 & scaled data:\n",
            "         Coefficient\n",
            "CRIM       -0.933398\n",
            "ZN          0.901779\n",
            "INDUS      -0.130099\n",
            "CHAS        0.653141\n",
            "NOX        -1.554675\n",
            "RM          2.780153\n",
            "AGE        -0.318141\n",
            "DIS        -2.751812\n",
            "RAD         1.470846\n",
            "TAS        -1.327215\n",
            "PTRATIO    -2.183673\n",
            "B           0.586609\n",
            "LSTAT      -3.297412\n",
            "R-squared for a training set: 0.7633167147255946\n",
            "R-squared for a test set: 0.6674648670481825\n",
            "MAE: 3.608315093238685\n",
            "MSE: 27.68870465973234\n",
            "RMSE: 5.262005763939483\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.linear_model import Ridge\n",
        "ridge100_scale = Ridge(alpha=100).fit(X_train_scaled,y_train)"
      ],
      "metadata": {
        "id": "2qBnfHZcmsZ7"
      },
      "execution_count": 85,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df_ridge100_scale=pd.DataFrame(ridge100_scale.coef_,X.columns, columns=['Coefficient'])\n",
        "print(\"ridge regresssion with lambda 100 & scaled data:\")\n",
        "print(df_ridge100_scale)\n",
        "print(\"R-squared for a training set:\", ridge100_scale.score(X_train_scaled,y_train))\n",
        "print(\"R-squared for a test set:\", ridge100_scale.score(X_test_scaled,y_test))\n",
        "y_pred=ridge100_scale.predict(X_test_scaled)\n",
        "from sklearn import metrics\n",
        "print(\"MAE:\", metrics.mean_absolute_error(y_test,y_pred))\n",
        "print(\"MSE:\", metrics.mean_squared_error(y_test,y_pred))\n",
        "print(\"RMSE:\", np.sqrt(metrics.mean_squared_error(y_test,y_pred)))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LRJgIVaunj_T",
        "outputId": "b39f402b-85ed-428b-d71b-8127d63d0329"
      },
      "execution_count": 96,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "ridge regresssion with lambda 100 & scaled data:\n",
            "         Coefficient\n",
            "CRIM       -0.715429\n",
            "ZN          0.557659\n",
            "INDUS      -0.462932\n",
            "CHAS        0.675175\n",
            "NOX        -0.669166\n",
            "RM          2.685603\n",
            "AGE        -0.324439\n",
            "DIS        -1.408948\n",
            "RAD         0.296800\n",
            "TAS        -0.580408\n",
            "PTRATIO    -1.838525\n",
            "B           0.583344\n",
            "LSTAT      -2.592446\n",
            "R-squared for a training set: 0.7389220452609574\n",
            "R-squared for a test set: 0.6318772220574332\n",
            "MAE: 3.6863461673345483\n",
            "MSE: 30.651927772241887\n",
            "RMSE: 5.536418316225923\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.linear_model import Ridge\n",
        "ridge1000_scale = Ridge(alpha=1000).fit(X_train_scaled,y_train)"
      ],
      "metadata": {
        "id": "RcS9qStU6GXN"
      },
      "execution_count": 89,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df_ridge1000_scale=pd.DataFrame(ridge1000_scale.coef_,X.columns, columns=['Coefficient'])\n",
        "print(\"ridge regresssion with lambda 1000 & scaled data:\")\n",
        "print(df_ridge1000_scale)\n",
        "print(\"R-squared for a training set:\", ridge1000_scale.score(X_train_scaled,y_train))\n",
        "print(\"R-squared for a test set:\", ridge1000_scale.score(X_test_scaled,y_test))\n",
        "y_pred=ridge1000_scale.predict(X_test_scaled)\n",
        "from sklearn import metrics\n",
        "print(\"MAE:\", metrics.mean_absolute_error(y_test,y_pred))\n",
        "print(\"MSE:\", metrics.mean_squared_error(y_test,y_pred))\n",
        "print(\"RMSE:\", np.sqrt(metrics.mean_squared_error(y_test,y_pred)))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tNJdK_QH6KmH",
        "outputId": "8a00cac4-0622-41b8-8313-cbf8ef5ad3c1"
      },
      "execution_count": 97,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "ridge regresssion with lambda 1000 & scaled data:\n",
            "         Coefficient\n",
            "CRIM       -0.423946\n",
            "ZN          0.375638\n",
            "INDUS      -0.476137\n",
            "CHAS        0.335314\n",
            "NOX        -0.341437\n",
            "RM          1.228876\n",
            "AGE        -0.305634\n",
            "DIS        -0.081153\n",
            "RAD        -0.256937\n",
            "TAS        -0.441192\n",
            "PTRATIO    -0.873286\n",
            "B           0.370682\n",
            "LSTAT      -1.115034\n",
            "R-squared for a training set: 0.5371333525433009\n",
            "R-squared for a test set: 0.4612028177403027\n",
            "MAE: 4.677071843748582\n",
            "MSE: 44.863217665624404\n",
            "RMSE: 6.698001020127155\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.linear_model import Ridge\n",
        "ridge5_scale = Ridge(alpha=5).fit(X_train_scaled,y_train)"
      ],
      "metadata": {
        "id": "NJ27zTAj6j0W"
      },
      "execution_count": 91,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df_ridge5_scale=pd.DataFrame(ridge5_scale.coef_,X.columns, columns=['Coefficient'])\n",
        "print(\"ridge regresssion with lambda 5 & scaled data:\")\n",
        "print(df_ridge5_scale)\n",
        "print(\"R-squared for a training set:\", ridge5_scale.score(X_train_scaled,y_train))\n",
        "print(\"R-squared for a test set:\", ridge5_scale.score(X_test_scaled,y_test))\n",
        "y_pred=ridge5_scale.predict(X_test_scaled)\n",
        "from sklearn import metrics\n",
        "print(\"MAE:\", metrics.mean_absolute_error(y_test,y_pred))\n",
        "print(\"MSE:\", metrics.mean_squared_error(y_test,y_pred))\n",
        "print(\"RMSE:\", np.sqrt(metrics.mean_squared_error(y_test,y_pred)))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4p4bF6gC6kDO",
        "outputId": "95a34a44-a3a7-406d-a4d9-33ffc37f7f06"
      },
      "execution_count": 98,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "ridge regresssion with lambda 5 & scaled data:\n",
            "         Coefficient\n",
            "CRIM       -0.967683\n",
            "ZN          0.966353\n",
            "INDUS      -0.045437\n",
            "CHAS        0.639325\n",
            "NOX        -1.697762\n",
            "RM          2.750044\n",
            "AGE        -0.303631\n",
            "DIS        -2.913691\n",
            "RAD         1.729902\n",
            "TAS        -1.550811\n",
            "PTRATIO    -2.219011\n",
            "B           0.584446\n",
            "LSTAT      -3.364826\n",
            "R-squared for a training set: 0.7641599771836047\n",
            "R-squared for a test set: 0.6703585664250096\n",
            "MAE: 3.6099315279962014\n",
            "MSE: 27.447759329511776\n",
            "RMSE: 5.23906092057649\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.linear_model import Ridge\n",
        "ridge1_scale = Ridge(alpha=1).fit(X_train_scaled,y_train)"
      ],
      "metadata": {
        "id": "mtsThKTb64Dn"
      },
      "execution_count": 93,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df_ridge1_scale=pd.DataFrame(ridge1_scale.coef_,X.columns, columns=['Coefficient'])\n",
        "print(\"ridge regresssion with lambda 1 & scaled data:\")\n",
        "print(df_ridge1_scale)\n",
        "print(\"R-squared for a training set:\", ridge1_scale.score(X_train_scaled,y_train))\n",
        "print(\"R-squared for a test set:\", ridge1_scale.score(X_test_scaled,y_test))\n",
        "y_pred=ridge1_scale.predict(X_test_scaled)\n",
        "from sklearn import metrics\n",
        "print(\"MAE:\", metrics.mean_absolute_error(y_test,y_pred))\n",
        "print(\"MSE:\", metrics.mean_squared_error(y_test,y_pred))\n",
        "print(\"RMSE:\", np.sqrt(metrics.mean_squared_error(y_test,y_pred)))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "W-3hXEeh64MZ",
        "outputId": "7dd1727f-9f4c-48fe-828a-8d1118c3d94b"
      },
      "execution_count": 99,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "ridge regresssion with lambda 1 & scaled data:\n",
            "         Coefficient\n",
            "CRIM       -1.002007\n",
            "ZN          1.031479\n",
            "INDUS       0.049805\n",
            "CHAS        0.623750\n",
            "NOX        -1.835263\n",
            "RM          2.715728\n",
            "AGE        -0.285448\n",
            "DIS        -3.058900\n",
            "RAD         2.011591\n",
            "TAS        -1.806511\n",
            "PTRATIO    -2.251977\n",
            "B           0.582930\n",
            "LSTAT      -3.424557\n",
            "R-squared for a training set: 0.7645258699709746\n",
            "R-squared for a test set: 0.6727737684528233\n",
            "MAE: 3.6101074898564725\n",
            "MSE: 27.24665632109248\n",
            "RMSE: 5.2198329782755\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**First, here I have build L2 Regression Ridge model and implemented feature scaling with it and compared 5 hyperparameters values with alpha[10,100,1000,5,1] respectively, which i have found that using scaling is beneficial and decreasing alpha rate impoves the output accuracy which error rate decreases.**"
      ],
      "metadata": {
        "id": "NBsCEpWs7TPo"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 5. Examine whether implementing L1 regularization with scaling improves model performance. To optimize the hyperparameter, you have to compare at least 5 different hyperparameter values."
      ],
      "metadata": {
        "id": "F-lunXMR2hSV"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.linear_model import Lasso\n",
        "lasso01_scale = Lasso(alpha=0.1).fit(X_train_scaled,y_train)"
      ],
      "metadata": {
        "id": "aKfSe5mJoJl4"
      },
      "execution_count": 101,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df_lasso01_scale=pd.DataFrame(lasso01_scale.coef_,X.columns, columns=['Coefficient'])\n",
        "print(\"lasso regresssion lambda with 0.1 & scaled data:\")\n",
        "print(df_lasso01_scale)\n",
        "print(\"R-squared for a training set:\", lasso01_scale.score(X_train_scaled,y_train))\n",
        "print(\"R-squared for a test set:\", lasso01_scale.score(X_test_scaled,y_test))\n",
        "y_pred=lasso01_scale.predict(X_test_scaled)\n",
        "from sklearn import metrics\n",
        "print(\"MAE:\", metrics.mean_absolute_error(y_test,y_pred))\n",
        "print(\"MSE:\", metrics.mean_squared_error(y_test,y_pred))\n",
        "print(\"RMSE:\", np.sqrt(metrics.mean_squared_error(y_test,y_pred)))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GbdW-z0co5iK",
        "outputId": "b73e2bb9-3483-4d8e-9a2e-8a3704a0cfb8"
      },
      "execution_count": 103,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "lasso regresssion lambda with 0.1 & scaled data:\n",
            "         Coefficient\n",
            "CRIM       -0.713303\n",
            "ZN          0.701583\n",
            "INDUS      -0.069505\n",
            "CHAS        0.603998\n",
            "NOX        -1.444227\n",
            "RM          2.834286\n",
            "AGE        -0.089646\n",
            "DIS        -2.345932\n",
            "RAD         0.639348\n",
            "TAS        -0.657379\n",
            "PTRATIO    -2.163676\n",
            "B           0.472380\n",
            "LSTAT      -3.504427\n",
            "R-squared for a training set: 0.759129270664182\n",
            "R-squared for a test set: 0.6599647116559878\n",
            "MAE: 3.6229879738141193\n",
            "MSE: 28.313208860876916\n",
            "RMSE: 5.321015773409896\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.linear_model import Lasso\n",
        "lasso01 = Lasso(alpha=0.1).fit(X_train,y_train)"
      ],
      "metadata": {
        "id": "xjc0QJs7nwYn"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df_lasso01=pd.DataFrame(lasso01.coef_,X.columns, columns=['Coefficient'])\n",
        "print(\"lasso regresssion lambda with 0.1 & non-scaled data:\")\n",
        "print(df_lasso01)\n",
        "print(\"R-squared for a training set:\", lasso01.score(X_train,y_train))\n",
        "print(\"R-squared for a test set:\", lasso01.score(X_test,y_test))\n",
        "y_pred=lasso01.predict(X_test)\n",
        "from sklearn import metrics\n",
        "print(\"MAE:\", metrics.mean_absolute_error(y_test,y_pred))\n",
        "print(\"MSE:\", metrics.mean_squared_error(y_test,y_pred))\n",
        "print(\"RMSE:\", np.sqrt(metrics.mean_squared_error(y_test,y_pred)))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_TcR0rMloDU-",
        "outputId": "8a64bc50-d970-4c44-de20-741a23366812"
      },
      "execution_count": 104,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "lasso regresssion lambda with 0.1 & non-scaled data:\n",
            "         Coefficient\n",
            "CRIM       -0.113118\n",
            "ZN          0.047251\n",
            "INDUS      -0.039925\n",
            "CHAS        0.964789\n",
            "NOX        -0.000000\n",
            "RM          3.722896\n",
            "AGE        -0.021431\n",
            "DIS        -1.233704\n",
            "RAD         0.204690\n",
            "TAS        -0.012944\n",
            "PTRATIO    -0.852690\n",
            "B           0.007958\n",
            "LSTAT      -0.523924\n",
            "R-squared for a training set: 0.7531274572554778\n",
            "R-squared for a test set: 0.6532086050344972\n",
            "MAE: 3.6392670911559044\n",
            "MSE: 28.875759467880663\n",
            "RMSE: 5.373616981873631\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.linear_model import Lasso\n",
        "lasso0001_scale = Lasso(alpha=0.001).fit(X_train_scaled,y_train)"
      ],
      "metadata": {
        "id": "aF--JwhzpMsM"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df_lasso0001_scale=pd.DataFrame(lasso0001_scale.coef_,X.columns, columns=['Coefficient'])\n",
        "print(\"lasso regresssion lambda with 0.001 & scaled data:\")\n",
        "print(df_lasso0001_scale)\n",
        "print(\"R-squared for a training set:\", lasso0001_scale.score(X_train_scaled,y_train))\n",
        "print(\"R-squared for a test set:\", lasso0001_scale.score(X_test_scaled,y_test))\n",
        "y_pred=lasso0001_scale.predict(X_test_scaled)\n",
        "from sklearn import metrics\n",
        "print(\"MAE:\", metrics.mean_absolute_error(y_test,y_pred))\n",
        "print(\"MSE:\", metrics.mean_squared_error(y_test,y_pred))\n",
        "print(\"RMSE:\", np.sqrt(metrics.mean_squared_error(y_test,y_pred)))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "f7O1spq4penq",
        "outputId": "dbc41850-991e-4683-c7b8-ccb037f35163"
      },
      "execution_count": 105,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "lasso regresssion lambda with 0.001 & scaled data:\n",
            "         Coefficient\n",
            "CRIM       -1.009102\n",
            "ZN          1.046310\n",
            "INDUS       0.069836\n",
            "CHAS        0.619218\n",
            "NOX        -1.867103\n",
            "RM          2.706109\n",
            "AGE        -0.278011\n",
            "DIS        -3.091935\n",
            "RAD         2.078839\n",
            "TAS        -1.868239\n",
            "PTRATIO    -2.259217\n",
            "B           0.581396\n",
            "LSTAT      -3.440498\n",
            "R-squared for a training set: 0.7645442937237866\n",
            "R-squared for a test set: 0.6732910521770625\n",
            "MAE: 3.609961304106649\n",
            "MSE: 27.203584432301042\n",
            "RMSE: 5.215705554601509\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.linear_model import Lasso\n",
        "lasso1_scale = Lasso(alpha=1).fit(X_train_scaled,y_train)"
      ],
      "metadata": {
        "id": "uvhFSYOaplnw"
      },
      "execution_count": 106,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df_lasso1_scale=pd.DataFrame(lasso1_scale.coef_,X.columns, columns=['Coefficient'])\n",
        "print(\"lasso regresssion lambda with 1 & scaled data:\")\n",
        "print(df_lasso1_scale)\n",
        "print(\"R-squared for a training set:\", lasso1_scale.score(X_train_scaled,y_train))\n",
        "print(\"R-squared for a test set:\", lasso1_scale.score(X_test_scaled,y_test))\n",
        "y_pred=lasso1_scale.predict(X_test_scaled)\n",
        "from sklearn import metrics\n",
        "print(\"MAE:\", metrics.mean_absolute_error(y_test,y_pred))\n",
        "print(\"MSE:\", metrics.mean_squared_error(y_test,y_pred))\n",
        "print(\"RMSE:\", np.sqrt(metrics.mean_squared_error(y_test,y_pred)))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YKI8G3BbAHbu",
        "outputId": "0af7ff30-8ec7-42cb-b6c1-1ec65e00b487"
      },
      "execution_count": 109,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "lasso regresssion lambda with 1 & scaled data:\n",
            "         Coefficient\n",
            "CRIM       -0.034090\n",
            "ZN          0.000000\n",
            "INDUS      -0.000000\n",
            "CHAS        0.000000\n",
            "NOX        -0.000000\n",
            "RM          2.675475\n",
            "AGE        -0.000000\n",
            "DIS        -0.000000\n",
            "RAD        -0.000000\n",
            "TAS        -0.119743\n",
            "PTRATIO    -1.784836\n",
            "B           0.002440\n",
            "LSTAT      -3.404284\n",
            "R-squared for a training set: 0.6927580317165543\n",
            "R-squared for a test set: 0.5999442961470397\n",
            "MAE: 3.957998413357712\n",
            "MSE: 33.31083886715502\n",
            "RMSE: 5.771554285212521\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.linear_model import Lasso\n",
        "lasso10_scale = Lasso(alpha=10).fit(X_train_scaled,y_train)"
      ],
      "metadata": {
        "id": "am9i__ASAPx5"
      },
      "execution_count": 110,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df_lasso10_scale=pd.DataFrame(lasso10_scale.coef_,X.columns, columns=['Coefficient'])\n",
        "print(\"lasso regresssion lambda with 10 & scaled data:\")\n",
        "print(df_lasso10_scale)\n",
        "print(\"R-squared for a training set:\", lasso10_scale.score(X_train_scaled,y_train))\n",
        "print(\"R-squared for a test set:\", lasso10_scale.score(X_test_scaled,y_test))\n",
        "y_pred=lasso10_scale.predict(X_test_scaled)\n",
        "from sklearn import metrics\n",
        "print(\"MAE:\", metrics.mean_absolute_error(y_test,y_pred))\n",
        "print(\"MSE:\", metrics.mean_squared_error(y_test,y_pred))\n",
        "print(\"RMSE:\", np.sqrt(metrics.mean_squared_error(y_test,y_pred)))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RGWFa021BPay",
        "outputId": "c4d7b4d1-9262-4fb3-8d25-fe4dcf9676de"
      },
      "execution_count": 111,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "lasso regresssion lambda with 10 & scaled data:\n",
            "         Coefficient\n",
            "CRIM            -0.0\n",
            "ZN               0.0\n",
            "INDUS           -0.0\n",
            "CHAS             0.0\n",
            "NOX             -0.0\n",
            "RM               0.0\n",
            "AGE             -0.0\n",
            "DIS              0.0\n",
            "RAD             -0.0\n",
            "TAS             -0.0\n",
            "PTRATIO         -0.0\n",
            "B                0.0\n",
            "LSTAT           -0.0\n",
            "R-squared for a training set: 0.0\n",
            "R-squared for a test set: -0.0060197319476869016\n",
            "MAE: 6.6181274159976216\n",
            "MSE: 83.76673764512785\n",
            "RMSE: 9.152417038418204\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "***Here I have build L1 Regression Lasso model and implemented feature scaling with it and one without and compared 5 hyperparameters values with alpha[0.1,0.1,0.001,1,10] respectively, which i have found that using scaling is beneficial and decreasing alpha rate impoves the output accuracy which error rate decreases. By the way for our 506 observation dataset, using Ridge regression model gives much better result and accuracy. ***"
      ],
      "metadata": {
        "id": "H1bONgy-CiPt"
      }
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "esExNKXrDJc1"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}