homework_4_week[reu_ml_school].ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/sage-artem/REU_course/blob/main/homework_4_week%5Breu_ml_school%5D.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# <center>**ML School by REU Data Scinece Club**</center>\n",
        "\n",
        "---\n",
        "## <center> Неделя №4 </center>\n"
      ],
      "metadata": {
        "id": "QWjiOr9XkQ61"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "На этой неделе вам предстоит написать свою логистическую регрессию на чистом numpy и оптимизировать ее спомощью градиентного спуска. Сторонние биболетки (например, если знакомы с Torch) для вычисления использовать нельзя"
      ],
      "metadata": {
        "id": "Oo0jo4Y3kUor"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Импортируем библиотеки, которые могут вам понадобиться, а затем загрузим данные"
      ],
      "metadata": {
        "id": "Bd12XvWWkk8G"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.metrics import f1_score"
      ],
      "metadata": {
        "id": "wA-FYP50kf5f"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Мы возьмем синтетические данные (созданные нами искутсвенно). Эти данные не нуждаются в обратботке"
      ],
      "metadata": {
        "id": "zKVOtf3ro0LR"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!git clone https://github.com/MeinRespect/REU-Ml-SCHOOL.git"
      ],
      "metadata": {
        "id": "z4bTlAsODdRU",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a2e798b0-c049-43e9-b5ac-5c0c6800b096"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Cloning into 'REU-Ml-SCHOOL'...\n",
            "remote: Enumerating objects: 13, done.\u001b[K\n",
            "remote: Counting objects: 100% (13/13), done.\u001b[K\n",
            "remote: Compressing objects: 100% (12/12), done.\u001b[K\n",
            "remote: Total 13 (delta 2), reused 0 (delta 0), pack-reused 0\u001b[K\n",
            "Unpacking objects: 100% (13/13), 1.16 MiB | 1.17 MiB/s, done.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df = pd.read_csv('/content/REU-Ml-SCHOOL/simple_data_for_linear_regression.csv',\n",
        "                 index_col='Unnamed: 0')\n",
        "\n",
        "df.head(10)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 363
        },
        "id": "NsDCr50QlDMD",
        "outputId": "8c0efced-d55a-433e-f5e8-915c9eb186a4"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "          1         2         3         4         5         6         7  \\\n",
              "0 -0.825654  0.996750  2.172271  0.296957 -0.463338 -0.349884  2.511452   \n",
              "1 -0.557834  0.160590  0.286331 -0.041377 -1.448752  0.027275  0.048385   \n",
              "2  0.309318 -0.638823  0.937555  1.098689  2.917309 -0.305654  0.647202   \n",
              "3  0.814625 -1.468853 -2.514925 -0.247016  0.276103 -0.445736  0.393316   \n",
              "4  0.529211  1.362208  0.026132  1.244459  0.165382  0.625437 -1.267929   \n",
              "5 -0.140911 -1.729456 -0.254853  0.259032 -0.661892 -0.691096  1.171270   \n",
              "6 -0.962663 -0.990447 -1.106779  1.147607 -1.251957  0.127369 -1.557314   \n",
              "7  2.018122  0.725902  0.144987 -0.316239 -0.621207  0.092930  0.348007   \n",
              "8 -0.017818  0.785941  0.457947  0.146765 -0.600139  0.095527  0.398463   \n",
              "9 -0.614641  1.036744  0.581185 -1.217846  0.120342  0.505202 -1.089338   \n",
              "\n",
              "          8         9  Target  \n",
              "0 -0.928644 -2.778066       1  \n",
              "1 -0.764340 -0.049713       1  \n",
              "2 -0.108898 -0.740039       0  \n",
              "3  1.142564 -0.478847       0  \n",
              "4  0.604447  1.452787       1  \n",
              "5  0.186937 -1.354727       0  \n",
              "6  0.895921  1.712605       0  \n",
              "7 -2.453892 -0.369115       1  \n",
              "8  0.572963 -0.423850       0  \n",
              "9 -0.163125  1.244560       1  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-0f85a243-4926-4f60-88b8-6e701620f226\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>1</th>\n",
              "      <th>2</th>\n",
              "      <th>3</th>\n",
              "      <th>4</th>\n",
              "      <th>5</th>\n",
              "      <th>6</th>\n",
              "      <th>7</th>\n",
              "      <th>8</th>\n",
              "      <th>9</th>\n",
              "      <th>Target</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>-0.825654</td>\n",
              "      <td>0.996750</td>\n",
              "      <td>2.172271</td>\n",
              "      <td>0.296957</td>\n",
              "      <td>-0.463338</td>\n",
              "      <td>-0.349884</td>\n",
              "      <td>2.511452</td>\n",
              "      <td>-0.928644</td>\n",
              "      <td>-2.778066</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>-0.557834</td>\n",
              "      <td>0.160590</td>\n",
              "      <td>0.286331</td>\n",
              "      <td>-0.041377</td>\n",
              "      <td>-1.448752</td>\n",
              "      <td>0.027275</td>\n",
              "      <td>0.048385</td>\n",
              "      <td>-0.764340</td>\n",
              "      <td>-0.049713</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0.309318</td>\n",
              "      <td>-0.638823</td>\n",
              "      <td>0.937555</td>\n",
              "      <td>1.098689</td>\n",
              "      <td>2.917309</td>\n",
              "      <td>-0.305654</td>\n",
              "      <td>0.647202</td>\n",
              "      <td>-0.108898</td>\n",
              "      <td>-0.740039</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0.814625</td>\n",
              "      <td>-1.468853</td>\n",
              "      <td>-2.514925</td>\n",
              "      <td>-0.247016</td>\n",
              "      <td>0.276103</td>\n",
              "      <td>-0.445736</td>\n",
              "      <td>0.393316</td>\n",
              "      <td>1.142564</td>\n",
              "      <td>-0.478847</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0.529211</td>\n",
              "      <td>1.362208</td>\n",
              "      <td>0.026132</td>\n",
              "      <td>1.244459</td>\n",
              "      <td>0.165382</td>\n",
              "      <td>0.625437</td>\n",
              "      <td>-1.267929</td>\n",
              "      <td>0.604447</td>\n",
              "      <td>1.452787</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>-0.140911</td>\n",
              "      <td>-1.729456</td>\n",
              "      <td>-0.254853</td>\n",
              "      <td>0.259032</td>\n",
              "      <td>-0.661892</td>\n",
              "      <td>-0.691096</td>\n",
              "      <td>1.171270</td>\n",
              "      <td>0.186937</td>\n",
              "      <td>-1.354727</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>-0.962663</td>\n",
              "      <td>-0.990447</td>\n",
              "      <td>-1.106779</td>\n",
              "      <td>1.147607</td>\n",
              "      <td>-1.251957</td>\n",
              "      <td>0.127369</td>\n",
              "      <td>-1.557314</td>\n",
              "      <td>0.895921</td>\n",
              "      <td>1.712605</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>2.018122</td>\n",
              "      <td>0.725902</td>\n",
              "      <td>0.144987</td>\n",
              "      <td>-0.316239</td>\n",
              "      <td>-0.621207</td>\n",
              "      <td>0.092930</td>\n",
              "      <td>0.348007</td>\n",
              "      <td>-2.453892</td>\n",
              "      <td>-0.369115</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>-0.017818</td>\n",
              "      <td>0.785941</td>\n",
              "      <td>0.457947</td>\n",
              "      <td>0.146765</td>\n",
              "      <td>-0.600139</td>\n",
              "      <td>0.095527</td>\n",
              "      <td>0.398463</td>\n",
              "      <td>0.572963</td>\n",
              "      <td>-0.423850</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>-0.614641</td>\n",
              "      <td>1.036744</td>\n",
              "      <td>0.581185</td>\n",
              "      <td>-1.217846</td>\n",
              "      <td>0.120342</td>\n",
              "      <td>0.505202</td>\n",
              "      <td>-1.089338</td>\n",
              "      <td>-0.163125</td>\n",
              "      <td>1.244560</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-0f85a243-4926-4f60-88b8-6e701620f226')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-0f85a243-4926-4f60-88b8-6e701620f226 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-0f85a243-4926-4f60-88b8-6e701620f226');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "X = df.drop('Target', axis=1)\n",
        "y = df['Target']"
      ],
      "metadata": {
        "id": "5dHhS_LoD0Gh"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Теперь нужно разделить на train и test c помощью train_test_split из sklearn, мы его уже импортировали.\n",
        "В качестве test_size выберите 0.25, а random_state = 1. Random state указывает, как должны генериться случаные числа. Указав его, мы будем всегда получать одни и те же значения, это поможет нам воспроизводить результат заново"
      ],
      "metadata": {
        "id": "PcPq0TmzpMXJ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Ваш код тут\n",
        "X_train, X_test, y_train, y_test = train_test_split(X.to_numpy(), y.to_numpy(), test_size=0.25, random_state=42)"
      ],
      "metadata": {
        "id": "qakSPLbYtm3E"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Теперь все готово к написанию самой модели"
      ],
      "metadata": {
        "id": "P5oacblkugxR"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Задание 1. Градиентный спуск (9 баллов)**"
      ],
      "metadata": {
        "id": "PTIuiMyLuof7"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Наша модель должна быть в матричном виде. Bias будет включен в матрицу весов. Реализуем градиентный спуск.\n",
        "\n",
        "Вот несколько нужных формул:\n",
        "\n",
        "p - наши предсказанные значения\n",
        "\n",
        "1. Градиентный спуск:   $$\\nabla L(w) = - lr \\frac{\\partial}{\\partial w} L(w)$$\n",
        "\n",
        "2. Бинарная кросс энтропия:   $$ L(y, p) = -[y\\log(p) + (1-y)\\log(1-p)] $$\n",
        "\n",
        "3. Сигмоидное преобразование:  $$ p = \\sigma(z) = \\frac{1}{1 + e^{-z}} $$\n",
        "\n",
        "4. Регрессия $$ {z} = X w $$\n",
        "\n",
        "В нашем случае нам нужно получить производную функции потерь (Бинарная кросс энтропия) по W. Это и есть наши веса или коэффицниенты. Производная в данном случае будет сложная, и именно ее значения мы будем домножать на learning rate. Вы можете вычислить на прямую: $$ grad = \\frac{\\partial L(w)}{\\partial w} $$\n",
        "Но это тяжело, поэтому можно также сделать по-другому, воспользовавшись цепным правилом(Chain rule): $$ grad = \\frac{\\partial L( p )}{\\partial p} \\frac{\\partial \\sigma(z) }{\\partial z} \\frac{\\partial z }{\\partial w}$$\n",
        "\n",
        "Получится то же самое, но посчитать будет легче. Просто так получить градиент не выйдет, поле нахождения производных нужно сократить некоторые части (кстати, максимально сократить громоздкую формулу можно к одному перемножению и одному вычитанию)."
      ],
      "metadata": {
        "id": "ZUG8bu2Xu9GG"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "$$ \\frac{\\partial L}{\\partial w} = X^T (y_\\text{pred} - y_\\text{real})$$\\"
      ],
      "metadata": {
        "id": "y1YLK4jwG0eb"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def gradient_descent(X, weights, y_pred, y_real, learning_rate):\n",
        "\n",
        "    m, n = X.shape\n",
        "\n",
        "    # Дополните grad и обновите веса\n",
        "    grad = (1/m) * X.T @ (y_pred - y_real)\n",
        "    weights -= learning_rate * grad\n",
        "\n",
        "    return weights\n",
        "\n",
        "def sigmoid(z):\n",
        "    return 1/(1+np.exp(-z))"
      ],
      "metadata": {
        "id": "WIr4LGJqntAm"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Эта часть кода покажет правильно ли вы реализовали градиентный спуск. Нужно его просто запустить, и если все было сделано правильно, то ошибок не произойдет."
      ],
      "metadata": {
        "id": "r-3Ch-1Znuo0"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "X = np.array([[1, 2], [3, 4], [5, 6]])\n",
        "weights = np.array([0.1, 0.2])\n",
        "y_real = np.array([3, 7, 11])\n",
        "\n",
        "y_pred = X @ weights\n",
        "\n",
        "learning_rate = 0.01\n",
        "weights_updated = gradient_descent(X, weights, y_pred, y_real, learning_rate)\n",
        "expected_weights_updated = np.array([0.32233333, 0.48133333])\n",
        "np.testing.assert_allclose(weights_updated, expected_weights_updated, rtol=1e-5)"
      ],
      "metadata": {
        "id": "fBo03fAJ9-Jp"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "sigmoid(np.arange(12))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rsZGZROyOB3P",
        "outputId": "7316342f-c2d2-45a2-d7dd-657d71fb7be0"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([0.5       , 0.73105858, 0.88079708, 0.95257413, 0.98201379,\n",
              "       0.99330715, 0.99752738, 0.99908895, 0.99966465, 0.99987661,\n",
              "       0.9999546 , 0.9999833 ])"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Задание 2. Логистическая регрессия (5 баллов)**"
      ],
      "metadata": {
        "id": "-t3Mnf2fzJni"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Не забудьте, что у нас все в матричном виде, поэтому X и матрицу весов нужно немного изменить. В качестве метрики используйте f1_score. Мы пройдем ее на следующей неделе, пока нужно знать только, что чем ближе она к 1, тем лучше.\n",
        "Применяется она так f1_score(y_true, y_pred), где y_true - правильные значение, а y_pred - значения, предсказанные моделью.\n",
        "\n",
        "Если вам нужно, то краткий экскурс в ООП вы найдете тут: https://pythonchik.ru/osnovy/osnovy-oop-v-python-klassy-obekty-metody"
      ],
      "metadata": {
        "id": "fJnwEdpEzw7I"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "class Logistic_Regression():\n",
        "    def __init__(self, num_of_columns):\n",
        "\n",
        "        # Задайте веса рандомными значениями.\n",
        "        #Не забудьте что в них должен быть вклдючен bias\n",
        "        self.weights = np.random.randn(num_of_columns + 1)\n",
        "\n",
        "    def train(self, X, y, learning_rate, iterations_num):\n",
        "\n",
        "        # Добавьте к X единичный столбец для bias\n",
        "\n",
        "        X = np.hstack((X, np.ones(X.shape[0]).reshape(-1, 1)))\n",
        "\n",
        "        # Цикл обучения, обновляйте каждую итерацию веса.\n",
        "        # Выведите также первый и конечный резульат ошибки f1.\n",
        "        # Так как f1 не принимает веротяности, y_pred надо округлить(np.where)\n",
        "        print(f1_score(y, np.where(sigmoid(X@self.weights) > 0.5, 1, 0)))\n",
        "        for step in range(iterations_num):\n",
        "            y_pred = sigmoid(X@self.weights)\n",
        "            self.weights = gradient_descent(X, self.weights, y_pred, y, learning_rate)\n",
        "        print(f1_score(y, np.where(y_pred > 0.5, 1, 0)))\n",
        "\n",
        "    def predict(self, X):\n",
        "        # Напишите предскзаание y по X\n",
        "        X = np.hstack((X, np.ones(X.shape[0]).reshape(-1, 1)))\n",
        "        # Ваш код тут\n",
        "        preds = sigmoid(X@self.weights)\n",
        "\n",
        "        # Округляем значения\n",
        "        return np.where(preds > 0.5, 1, 0)"
      ],
      "metadata": {
        "id": "RQhPCybFzI-O"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Теперь обучим модель и проверим ее на тестовой выборке с помощью f1_score. Она должна быть больше 0.85 на тестовой выборке. Если у вас меньше, то вам нужно поменять параметры, например, увеличить learning rate или количество итераций"
      ],
      "metadata": {
        "id": "Al0K_8lkT9zw"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Ваш код тут\n",
        "model =  Logistic_Regression(X_train.shape[1])\n",
        "_ = model.train(X_train, y_train, 0.01, 1500)\n",
        "y_pred = model.predict(X_test)\n",
        "print(f1_score(y_test, y_pred))"
      ],
      "metadata": {
        "id": "wlvzW0PA8Ord",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "360b04c0-4a69-47f1-b385-de0f69b0f99d"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "0.6176470588235295\n",
            "0.9500000000000001\n",
            "0.9318181818181818\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Задание 3. Логистическая регрессия в Sklearn (6 баллов)**"
      ],
      "metadata": {
        "id": "KNKr36SqBBYL"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.svm import SVC"
      ],
      "metadata": {
        "id": "oQivcBsqfQXt"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Теперь, написав свою модель, ее можно сравнить с моделью из Sklearn, обучите LogisticRegression и также проверьте на тестовой выборке по f1_score"
      ],
      "metadata": {
        "id": "usoTGzh7VA9E"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Ваш код тут\n",
        "model = LogisticRegression(penalty=None, fit_intercept=True) #убираем регуляризацию и добавляем bias\n",
        "_ = model.fit(X_train, y_train)\n",
        "y_pred = model.predict(X_test)\n",
        "print(f1_score(y_test, y_pred))"
      ],
      "metadata": {
        "id": "za_HS8PQBNU6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "380eb92e-fd0b-43df-d541-c7897584c3e3"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "0.9195402298850575\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "получили такой же результат..."
      ],
      "metadata": {
        "id": "Mboe-Xhm5sZZ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Логичесческая регрессия модель хорошая, но не лучшая. Давайте теперь попробуем построить SVM из Sklearn. Все также: обучите ее и проверьте на тестовой выборке."
      ],
      "metadata": {
        "id": "cbTTLlfYPn10"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Ваш код тут\n",
        "model = SVC(kernel='linear')\n",
        "_ = model.fit(X_train, y_train)\n",
        "y_pred = model.predict(X_test)\n",
        "print(f1_score(y_test, y_pred))"
      ],
      "metadata": {
        "id": "-f26KCTTP9hg",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "2f2a2d29-a746-41cc-91fb-59c9441076bc"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "0.9195402298850575\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model = SVC(kernel='rbf') #убираем регуляризацию и добавляем bias\n",
        "_ = model.fit(X_train, y_train)\n",
        "y_pred = model.predict(X_test)\n",
        "print(f1_score(y_test, y_pred))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "W-OYsgzT58_p",
        "outputId": "6843239d-ffe6-4a02-c618-8bec0def7789"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "0.9195402298850575\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "видимо качество лучше получить нельзя\n"
      ],
      "metadata": {
        "id": "M_G63QiM6AMV"
      }
    }
  ]
}