week01_intro/sem01_pytorch_intro_solution.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center>\n",
    "<img src=\"https://camo.githubusercontent.com/1faf678af2925dc98a2fb877b80461f1c992b9e7e229412e5d6b729c49246ffa/68747470733a2f2f7079746f7263682e6f72672f7475746f7269616c732f5f7374617469632f7079746f7263682d6c6f676f2d6461726b2e737667\" height=\"400\" width=\"800\"> \n",
    "\n",
    "# Семинар 1: введение в PyTorch\n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Фрэймворком для обучения нейросеток на нашем курсе будет PyTorch. Раньше это был Tensorflow. [В прошлом запуске курса](https://github.com/FUlyankin/deep_learning_tf) можно найти много тетрадок с кодом на второй версии библиотеки. \n",
    "\n",
    "В этом году я окончательно решил перевести весь курс на PyTorch. Кажется, что [Google признал его победу](https://t.me/gonzo_ML/1026) и переходит на новую библиотеку, JAX. Если кому-то интересно, он может подробнее почитать про [раличия между Tensorflow и Pytorch.](https://towardsdatascience.com/pytorch-vs-tensorflow-spotting-the-difference-25c75777377b)\n",
    "\n",
    "Ну а мы помянем Tensorflof, нажмём дружно F и больше не будем про него вспоминать в этом курсе."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Устанавливаем библиотеку\n",
    "\n",
    "```\n",
    "pip install torch torchvision\n",
    "```\n",
    "\n",
    "Дополнительно с торчом выходят `torchvision` и `torchaudio`. В них куча всяких методов дял работы с изображениями и звуком. Обе эти библиотеки тоже можно поставить виесте с торчом. Первая из них уже сегодня нам понадобится. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.23.5'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "np.__version__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.13.1'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch\n",
    "torch.__version__ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Тензоры и базовые операции над ними\n",
    "\n",
    "Основной объект в PyTorch - это **тензор.** Или многомерный массив чисел. Чтобы не нужно было учить кучу новых команд, PyTorch косплеит numpy. \n",
    "\n",
    "```\n",
    "    np.zeros -> torch.zeros\n",
    "    np.sin -> torch.sin\n",
    "    np.cumsum -> torch.cumsum\n",
    "    \n",
    "    x.reshape([1,2,8]) -> x.view(1,2,8)\n",
    "    x.sum(axis=-1) -> x.sum(dim=-1)\n",
    "    x.astype('int64') -> x.type(torch.LongTensor)\n",
    "```\n",
    "\n",
    "Правда говоря, не совсем косплеит. Но чаще всего оказывается довольно близок. Для помощи вам есть [таблица](https://github.com/torch/torch7/wiki/Torch-for-Numpy-users), которая поможет вам найти аналог операции в numpy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([1, 2, 3, 4])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# как в нумпае можно объявить вектор\n",
    "x = torch.tensor([1, 2, 3, 4])\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0., 0., 0., 0.],\n",
       "        [0., 0., 0., 0.],\n",
       "        [0., 0., 0., 0.]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# можно задать тензор из нулей\n",
    "torch.zeros([3, 4])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[[1., 1.],\n",
       "         [1., 1.],\n",
       "         [1., 1.],\n",
       "         [1., 1.]],\n",
       "\n",
       "        [[1., 1.],\n",
       "         [1., 1.],\n",
       "         [1., 1.],\n",
       "         [1., 1.]],\n",
       "\n",
       "        [[1., 1.],\n",
       "         [1., 1.],\n",
       "         [1., 1.],\n",
       "         [1., 1.]]])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# или из единиц, но уже более сложной размерности\n",
    "torch.ones([3, 4, 2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.1091, -0.6293],\n",
       "        [-0.5046, -1.1700]])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# можно сгенерировать тензор из стандартного нормального распределения\n",
    "torch.randn([2,2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1.1210e-44, 0.0000e+00, 0.0000e+00],\n",
       "        [0.0000e+00, 0.0000e+00, 0.0000e+00],\n",
       "        [0.0000e+00, 0.0000e+00, 3.5733e-43],\n",
       "        [0.0000e+00, 0.0000e+00, 0.0000e+00],\n",
       "        [0.0000e+00, 0.0000e+00, 0.0000e+00]])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = torch.empty(5, 3) # пустой тензор\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0., 0., 0.],\n",
      "        [0., 0., 0.],\n",
      "        [0., 0., 0.],\n",
      "        [0., 0., 0.],\n",
      "        [0., 0., 0.]], dtype=torch.float64)\n"
     ]
    }
   ],
   "source": [
    "# тензор с нулями и указанием типов чисел\n",
    "x = torch.zeros(5, 3, dtype=torch.double) \n",
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1., 1., 1.],\n",
      "        [1., 1., 1.],\n",
      "        [1., 1., 1.],\n",
      "        [1., 1., 1.],\n",
      "        [1., 1., 1.]], dtype=torch.float64)\n"
     ]
    }
   ],
   "source": [
    " # используем уже созданный тензор для создания тензора из единичек\n",
    "x = x.new_ones(5, 3, dtype=torch.double)\n",
    "print(x) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-0.7176, -0.0266,  0.5595],\n",
      "        [-0.9849,  0.7153,  0.3862],\n",
      "        [-2.2734, -0.2512,  0.9891],\n",
      "        [ 0.4731, -1.0718, -0.8225],\n",
      "        [-0.5787, -0.7352,  1.3117]])\n"
     ]
    }
   ],
   "source": [
    "# создаем матрицу с размерами как у x\n",
    "x = torch.randn_like(x, dtype=torch.float)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-0.7176, -0.0266,  0.5595],\n",
       "        [-0.9849,  0.7153,  0.3862],\n",
       "        [-2.2734, -0.2512,  0.9891],\n",
       "        [ 0.4731, -1.0718, -0.8225],\n",
       "        [-0.5787, -0.7352,  1.3117]], dtype=torch.float64)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.double() # можно поменяить тип данных"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0,  0,  0],\n",
       "        [ 0,  0,  0],\n",
       "        [-2,  0,  0],\n",
       "        [ 0, -1,  0],\n",
       "        [ 0,  0,  1]], dtype=torch.int32)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.int()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-0.7176, -0.0266,  0.5595],\n",
       "        [-0.9849,  0.7153,  0.3862],\n",
       "        [-2.2734, -0.2512,  0.9891],\n",
       "        [ 0.4731, -1.0718, -0.8225],\n",
       "        [-0.5787, -0.7352,  1.3117]])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# как думаете, почему не пишет dtype? \n",
    "x.float()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Все срезы, операции, размерности работают как в numpy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([5, 3])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.size()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.float32"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.dtype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([5, 3])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([-0.7176, -0.0266,  0.5595])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([-0.0266,  0.7153, -0.2512, -1.0718, -0.7352])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[:,1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 9.2824,  9.9734, 10.5595],\n",
       "        [ 9.0151, 10.7153, 10.3862],\n",
       "        [ 7.7266,  9.7488, 10.9891],\n",
       "        [10.4731,  8.9282,  9.1775],\n",
       "        [ 9.4213,  9.2648, 11.3117]])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x + 10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.4879, 0.9737, 1.7497],\n",
       "        [0.3735, 2.0449, 1.4713],\n",
       "        [0.1030, 0.7779, 2.6889],\n",
       "        [1.6049, 0.3424, 0.4393],\n",
       "        [0.5606, 0.4794, 3.7124]])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.exp(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0.5595, 0.7153, 0.3862, 0.9891, 0.4731, 1.3117])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[x > 0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.7538, 0.4221, 0.9308],\n",
       "        [0.1871, 0.8619, 0.3082],\n",
       "        [0.1547, 0.8429, 0.5990],\n",
       "        [0.5575, 0.4682, 0.1802],\n",
       "        [0.6793, 0.1510, 0.9434]])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = torch.rand(5, 3)\n",
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 0.0363,  0.3954,  1.4902],\n",
      "        [-0.7978,  1.5773,  0.6944],\n",
      "        [-2.1187,  0.5917,  1.5882],\n",
      "        [ 1.0306, -0.6036, -0.6423],\n",
      "        [ 0.1006, -0.5841,  2.2550]])\n"
     ]
    }
   ],
   "source": [
    "print(x + y) # операция сложение"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 0.0363,  0.3954,  1.4902],\n",
      "        [-0.7978,  1.5773,  0.6944],\n",
      "        [-2.1187,  0.5917,  1.5882],\n",
      "        [ 1.0306, -0.6036, -0.6423],\n",
      "        [ 0.1006, -0.5841,  2.2550]])\n"
     ]
    }
   ],
   "source": [
    "z = torch.add(x, y) # очередная операция сложения\n",
    "print(z)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 0.0363,  0.3954,  1.4902],\n",
      "        [-0.7978,  1.5773,  0.6944],\n",
      "        [-2.1187,  0.5917,  1.5882],\n",
      "        [ 1.0306, -0.6036, -0.6423],\n",
      "        [ 0.1006, -0.5841,  2.2550]])\n"
     ]
    }
   ],
   "source": [
    "torch.add(x, y, out=z) # и наконец последний вид\n",
    "print(z)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-0.5409, -0.0112,  0.5207],\n",
      "        [-0.1843,  0.6166,  0.1190],\n",
      "        [-0.3517, -0.2117,  0.5925],\n",
      "        [ 0.2637, -0.5018, -0.1482],\n",
      "        [-0.3931, -0.1110,  1.2374]])\n"
     ]
    }
   ],
   "source": [
    "print(x * y) # поэлементное умножение"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-0.0314,  0.0152,  0.2017, -0.3117,  0.0363],\n",
      "        [-0.0811,  0.5513,  0.6819, -0.1446, -0.1967],\n",
      "        [-0.8991, -0.3370,  0.0291, -1.2068, -0.6490],\n",
      "        [-0.8613, -1.0888, -1.3229, -0.3862, -0.6164],\n",
      "        [ 0.4744, -0.3376,  0.0765, -0.4305,  0.7333]])\n"
     ]
    }
   ],
   "source": [
    "print(x @ y.t()) # матричное умножение"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-0.0314,  0.0152,  0.2017, -0.3117,  0.0363],\n",
      "        [-0.0811,  0.5513,  0.6819, -0.1446, -0.1967],\n",
      "        [-0.8991, -0.3370,  0.0291, -1.2068, -0.6490],\n",
      "        [-0.8613, -1.0888, -1.3229, -0.3862, -0.6164],\n",
      "        [ 0.4744, -0.3376,  0.0765, -0.4305,  0.7333]])\n"
     ]
    }
   ],
   "source": [
    "# и опять матричное умножение\n",
    "print(x.mm(y.t()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([1, 5, 3])\n"
     ]
    }
   ],
   "source": [
    "# добавили измерение в начало, аналог броадкастинга \n",
    "print(x.unsqueeze(0).shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[[-0.7176, -0.0266,  0.5595],\n",
       "         [-0.9849,  0.7153,  0.3862],\n",
       "         [-2.2734, -0.2512,  0.9891],\n",
       "         [ 0.4731, -1.0718, -0.8225],\n",
       "         [-0.5787, -0.7352,  1.3117]]])"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.unsqueeze(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([5, 3])\n"
     ]
    }
   ],
   "source": [
    "# убрали измерение в начале, аналог броадкастинга \n",
    "print(x.unsqueeze(0).squeeze(0).shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-0.7176, -0.0266,  0.5595],\n",
       "        [-0.9849,  0.7153,  0.3862],\n",
       "        [-2.2734, -0.2512,  0.9891],\n",
       "        [ 0.4731, -1.0718, -0.8225],\n",
       "        [-0.5787, -0.7352,  1.3117]])"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.unsqueeze(0).squeeze(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Мы также можем переводить матрицы назад в numpy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.717573  , -0.02663185,  0.5594581 ],\n",
       "       [-0.9849378 ,  0.7153478 ,  0.38616174],\n",
       "       [-2.2733572 , -0.25119457,  0.9891233 ],\n",
       "       [ 0.47308052, -1.0717893 , -0.8224664 ],\n",
       "       [-0.5786742 , -0.7351583 ,  1.3116655 ]], dtype=float32)"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Задание 1: \n",
    "\n",
    "- При помощи numpy посчитайте сумму квадратов чисел от 1 до 10000\n",
    "- Сделайте то же самое с помощью pytorch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "333383335000"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.square(np.arange(1, 10_000+1)).sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "333383335000"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# .item() возвращает число из тензора. \n",
    "# Без этого будет тензор размерности 1x1\n",
    "\n",
    "torch.square(torch.arange(1, 10_000+1)).sum().item()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Задание 2:\n",
    "\n",
    "Реализуйте на PyTorch сигмоиду \n",
    "\n",
    "$$ \\sigma(x) = \\frac{1}{1 + e^{-x}} $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGdCAYAAADAAnMpAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/P9b71AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAzeklEQVR4nO3de3RU5b3/8c/MJJkkhCRAroRAuCgXUVCQnHhptaZEammt1kXVn1COl5+KVo09VVTASzUcL8ipxaZekJ51joXaX7WtIlajaJVUlIsXFJRrIpCEAMmEhEySmef3R5iBSIBMyGTP5f1aa1bIzt6Z7+40Mx+f/X2ebTPGGAEAAFjEbnUBAAAguhFGAACApQgjAADAUoQRAABgKcIIAACwFGEEAABYijACAAAsRRgBAACWirG6gK7wer3atWuX+vbtK5vNZnU5AACgC4wxamho0MCBA2W3H3v8IyzCyK5du5Sbm2t1GQAAoBsqKys1aNCgY/48LMJI3759JbWfTHJyssXVAACArnC5XMrNzfV/jh9LWIQR36WZ5ORkwggAAGHmRC0WNLACAABLEUYAAIClCCMAAMBShBEAAGApwggAALAUYQQAAFiKMAIAACxFGAEAAJYijAAAAEsFHEbee+89TZ06VQMHDpTNZtMrr7xywmNWrlyps846S06nUyNGjNCSJUu6USoAAIhEAYeRxsZGjRs3TosWLerS/tu2bdMll1yiCy+8UOvXr9ftt9+u6667Tm+88UbAxQIAgMgT8L1ppkyZoilTpnR5/9LSUg0dOlRPPPGEJGn06NF6//339eSTT6qoqCjQpwcAABEm6DfKKy8vV2FhYYdtRUVFuv322495jNvtltvt9n/vcrmCVR4AhDRjjFo8XjW6PWp0t8nd5lFzq1fuNq9a2rxyt3nU6jFq83jV4vGqzWPU5vXK45U8Xq88XqM2r5ExktcYecyhf3uNvEYyOvT10HYj31f5v28vRL5/+evSoX061NvpOXThPDs9Er3p388dqtz+iZY8d9DDSFVVlTIzMztsy8zMlMvl0sGDB5WQkHDUMSUlJXrggQeCXRoAWKKppU2V+w6qytWsalezquubVd3QrP1NrapvalXdwRbVNbXKdbBVTS0etXn5oEbwTR03MHLDSHfMnj1bxcXF/u9dLpdyc3MtrAgAAufxGm2scmldRZ021xzQlj0HtKXmgHbVN3fr9zlj7EqIc8gZY5czpv1rXIxdsQ67Yh02xdjtinHYFOuwy2G3yWGzyeE49NVuk80m2W3t39tskt1uk90m2XT4Z5Lku9u7b7vt0DabzaYON4I/Yr8Omzu5W/zxbyB/bCe48zx6UGZyvGXPHfQwkpWVperq6g7bqqurlZyc3OmoiCQ5nU45nc5glwYAPcrrNVpXWaf3v67Vxzv2aV1FnQ642zrdNyUhVtkp8cpMjldmslOZyfEa0CdOqYlxSkmMVWpCrJITYpXkjFFinEOJcTFy2PlkRmQKehgpKCjQ8uXLO2x78803VVBQEOynBoCgM8Zowy6X/v7pLr36yW7trDvY4edJzhidOThVY7KTNTw9ScMz+mhYWpL69YmzqGIg9AQcRg4cOKDNmzf7v9+2bZvWr1+v/v37a/DgwZo9e7Z27typ//7v/5Yk3Xjjjfrtb3+rX/3qV/r3f/93vf322/rTn/6k1157refOAgB6WUubVy+tqdTz/9ymrbWN/u1JzhhdMDJd+UP7a8KQ/hqZ1ZcRDeAEAg4jH3/8sS688EL/977ejhkzZmjJkiXavXu3Kioq/D8fOnSoXnvtNd1xxx36r//6Lw0aNEjPPfcc03oBhKVWj1cvr92p37z9tb7Z3z4K4oyxq3B0pqaOy9YFIzMUH+uwuEogvNiM6cqkK2u5XC6lpKSovr5eycnJVpcDIEr9/ZNdevwfm7Rjb5MkKb2vUzdfMFxXTMxVkjMk5wMAlurq5zd/PQBwAk0tbZrzygb9v7XfSJIG9InTTRcM19X5Q5QQxygIcLIIIwBwHF9VN+jm/12rzTUHZLdJt3zvFP3f7wxTH0ZCgB7DXxMAHMOfPq7U3L9+ruZWrzKTnfrNz85U/rABVpcFRBzCCAB0Yv7rG1X67hZJ0vmnpOnJaeOVlsT6R0AwEEYA4Ft+/+4WfxC58/unataFI2Rnei4QNIQRADjCSx9XquT1jZKke34wSjd8Z7jFFQGRz251AQAQKt76olp3/+UzSdL//c4wggjQSwgjACDpo+37NOvFtfJ4jX46YZDunjLK6pKAqEEYARD1dtUd1LVLPpK7zauLRmVo/mWny8btYoFeQxgBENWMMZr718/lam7TuNxU/faqsxTj4K0R6E38xQGIaq9/XqW3vqxRrMOmx396BiuqAhYgjACIWvUHWzXvbxskSTdfMEKnZPa1uCIgOhFGAESt/1yxUXsa3BqW3kc3X8jMGcAqhBEAUWn1tn168cMKSVLJT06XM4bLM4BVCCMAoo67zaPZf/lUknTlpFzuNwNYjDACIOo88+5WbdnTqLQkp+6+eLTV5QBRjzACIKo0utv07D+3SpLm/HC0UhJjLa4IAGEEQFT585pv5GpuU96ARE09Y6DV5QAQYQRAFPF4jZ5/f5sk6drzhnInXiBEEEYARI03v6hWxb4mpSbG6vIJg6wuB8AhhBEAUeO5Q70iV+cPVmJcjMXVAPAhjACICusq9uvjHfsV67BpRkGe1eUAOAJhBEBUeO5Qr8iPxuUoIzne4moAHIkwAiDiVe5r0uuf7ZYkXXf+UIurAfBthBEAEW/Jqu3yGum8EWkanZ1sdTkAvoUwAiCiuZpbteyjSkmMigChijACIKKt+KxKB9xtGp7eR989Nd3qcgB0gjACIKK9/nl7r8iPx+fIZmORMyAUEUYARCxXc6s+2LxXkjRlbJbF1QA4FsIIgIj1zsYatXi8Gp7eR6dk9rW6HADHQBgBELFe/6xKkjRlbLbFlQA4HsIIgIjU1NKmlV/VSJIu5hINENIIIwAi0rub9qi51atB/RJ02kDWFgFCGWEEQER6/XPfJZosZtEAIY4wAiDiuNs8enuj7xIN/SJAqCOMAIg4H2yu1QF3mzKTnTozN9XqcgCcAGEEQMTxzaK5+LQs2e1cogFCHWEEQERp9Xj15pfVkqQiZtEAYYEwAiCifLh1n+qaWtW/T5wm5fW3uhwAXUAYARBRfPeimTwmUzEO3uKAcMBfKoCIsnLTHklcogHCCWEEQMT4Zn+TdtYdVIzdpvyhXKIBwgVhBEDEWL1tnyRpbE6KEuNiLK4GQFcRRgBEDF8YYVQECC+EEQARwxdGJhFGgLBCGAEQEWpczdpa2yibTZo4hDAChBPCCICIsHp7+6jIqKxkpSTGWlwNgEAQRgBEBPpFgPBFGAEQEegXAcIXYQRA2KtratHGqgZJ0tksAQ+EHcIIgLD30fb9kqRh6X2U3tdpcTUAAkUYARD2Vm/bK0nKHzrA4koAdAdhBEDYo3kVCG+EEQBh7YC7TZ/vckmieRUIV4QRAGFt7Y798niNBvVL0MDUBKvLAdANhBEAYY0pvUD4I4wACGv0iwDhr1thZNGiRcrLy1N8fLzy8/O1evXq4+6/cOFCjRw5UgkJCcrNzdUdd9yh5ubmbhUMAD7NrR6tr6yTJE1iJg0QtgIOI8uWLVNxcbHmzZuntWvXaty4cSoqKlJNTU2n+7/44ou6++67NW/ePH355Zd6/vnntWzZMt1zzz0nXTyA6PZJZZ1aPF6l93Uqb0Ci1eUA6KaAw8iCBQt0/fXXa+bMmRozZoxKS0uVmJioxYsXd7r/qlWrdO655+qqq65SXl6eJk+erCuvvPKEoykAcCK+UZGJQ/rJZrNZWwyAbgsojLS0tGjNmjUqLCw8/AvsdhUWFqq8vLzTY8455xytWbPGHz62bt2q5cuX6wc/+MExn8ftdsvlcnV4AMC3bTg0pXdsTorFlQA4GTGB7FxbWyuPx6PMzMwO2zMzM7Vx48ZOj7nqqqtUW1ur8847T8YYtbW16cYbbzzuZZqSkhI98MADgZQGIApt2FUvSRozMNniSgCcjKDPplm5cqUeeeQRPf3001q7dq3+8pe/6LXXXtNDDz10zGNmz56t+vp6/6OysjLYZQIIM00tbdpa2yhJOi2bMAKEs4BGRtLS0uRwOFRdXd1he3V1tbKysjo9Zs6cObrmmmt03XXXSZJOP/10NTY26oYbbtC9994ru/3oPOR0OuV0crMrAMe2sapBxkhpSU5lJMdbXQ6AkxDQyEhcXJwmTJigsrIy/zav16uysjIVFBR0ekxTU9NRgcPhcEiSjDGB1gsAkg73i3CJBgh/AY2MSFJxcbFmzJihiRMnatKkSVq4cKEaGxs1c+ZMSdL06dOVk5OjkpISSdLUqVO1YMECnXnmmcrPz9fmzZs1Z84cTZ061R9KACBQXxwKI6cRRoCwF3AYmTZtmvbs2aO5c+eqqqpK48eP14oVK/xNrRUVFR1GQu677z7ZbDbdd9992rlzp9LT0zV16lQ9/PDDPXcWAKLOF4eaVwkjQPizmTC4VuJyuZSSkqL6+nolJ/PGA0S7No9Xp817Q+42r96+87salp5kdUkAOtHVz2/uTQMg7GytbZS7zas+cQ7lDehjdTkAThJhBEDY8a0vMjo7WXY7K68C4Y4wAiDsfMFMGiCiEEYAhJ0NzKQBIgphBEBYMcboi92HRkayuScNEAkIIwDCyq76ZtU1tSrGbtOpWcyiASIBYQRAWNmws715dURGkpwxLJwIRALCCICw4r9EQ78IEDEIIwDCyuHmVfpFgEhBGAEQVrgnDRB5CCMAwkZdU4t21h2U1L7gGYDIQBgBEDZ8oyK5/ROUkhBrcTUAegphBEDYOLy+CKMiQCQhjAAIGzSvApGJMAIgbPhukEfzKhBZCCMAwkKrx6utexolSaO4TANEFMIIgLCwY2+T2rxGiXEODUyJt7ocAD2IMAIgLGyuOSBJGp6eJJvNZnE1AHoSYQRAWNiyxxdG+lhcCYCeRhgBEBZ8YWREBnfqBSINYQRAWNhyxGUaAJGFMAIg5BljtOXQTJrhjIwAEYcwAiDk1TS4dcDdJofdpiEDEq0uB0API4wACHm+mTSD+yfKGeOwuBoAPY0wAiDkMZMGiGyEEQAhz7/GCP0iQEQijAAIeYdHRggjQCQijAAIeVtqDs2kIYwAEYkwAiCkNTS3qsrVLEkaQRgBIhJhBEBI892pNy3JqZTEWIurARAMhBEAIY2ZNEDkI4wACGm+mTTckwaIXIQRACGNmTRA5COMAAhp3JMGiHyEEQAhq9Xj1fba9jDCZRogchFGAISsin1NavMaJcQ6lJ0cb3U5AIKEMAIgZG3xLwPfR3a7zeJqAAQLYQRAyNpM8yoQFQgjAEIWy8AD0YEwAiBk+ab10rwKRDbCCICQZIw53DPCyAgQ0QgjAELSnga3GtxtstukvLREq8sBEESEEQAhyde8Orh/opwxDourARBMhBEAIYlLNED0IIwACEnbapskSUPTuFsvEOkIIwBC0o697dN68wgjQMQjjAAISdt9YWQAYQSIdIQRACHH4zWq3HdQkjRkADNpgEhHGAEQcnbXH1SLx6tYh00DUxOsLgdAkBFGAIScHXvbm1dz+yfKwQ3ygIhHGAEQcugXAaILYQRAyPGNjNAvAkQHwgiAkOOb1jukP2EEiAaEEQAhxz8ywhojQFQgjAAIKcYYekaAKEMYARBSahrcam71ymG3KYdpvUBU6FYYWbRokfLy8hQfH6/8/HytXr36uPvX1dVp1qxZys7OltPp1Kmnnqrly5d3q2AAkW17bfuoSE5qguJi+O8lIBrEBHrAsmXLVFxcrNLSUuXn52vhwoUqKirSpk2blJGRcdT+LS0t+v73v6+MjAz9+c9/Vk5Ojnbs2KHU1NSeqB9AhGEmDRB9Ag4jCxYs0PXXX6+ZM2dKkkpLS/Xaa69p8eLFuvvuu4/af/Hixdq3b59WrVql2NhYSVJeXt7JVQ0gYtEvAkSfgMZAW1patGbNGhUWFh7+BXa7CgsLVV5e3ukxf/vb31RQUKBZs2YpMzNTY8eO1SOPPCKPx3PM53G73XK5XB0eAKIDIyNA9AkojNTW1srj8SgzM7PD9szMTFVVVXV6zNatW/XnP/9ZHo9Hy5cv15w5c/TEE0/o17/+9TGfp6SkRCkpKf5Hbm5uIGUCCGM79jEyAkSboHeHeb1eZWRk6JlnntGECRM0bdo03XvvvSotLT3mMbNnz1Z9fb3/UVlZGewyAYQAY4x21LaPjOSlMTICRIuAekbS0tLkcDhUXV3dYXt1dbWysrI6PSY7O1uxsbFyOBz+baNHj1ZVVZVaWloUFxd31DFOp1NOpzOQ0gBEgH2NLWpwt8lmkwb1I4wA0SKgkZG4uDhNmDBBZWVl/m1er1dlZWUqKCjo9Jhzzz1Xmzdvltfr9W/76quvlJ2d3WkQARC9th/qF8lOjld8rOMEewOIFAFfpikuLtazzz6rP/zhD/ryyy910003qbGx0T+7Zvr06Zo9e7Z//5tuukn79u3Tbbfdpq+++kqvvfaaHnnkEc2aNavnzgJARPDfk4Z+ESCqBDy1d9q0adqzZ4/mzp2rqqoqjR8/XitWrPA3tVZUVMhuP5xxcnNz9cYbb+iOO+7QGWecoZycHN1222266667eu4sAEQE38gI/SJAdLEZY4zVRZyIy+VSSkqK6uvrlZycbHU5AILktqXr9Nf1u3T3lFG68bvDrS4HwEnq6uc3ay0DCBn+kRHWGAGiCmEEQMiooGcEiEqEEQAhob6pVfubWiWx+ioQbQgjAEKCb+XVjL5OJcYF3FsPIIwRRgCEhMP9IlyiAaINYQRASNhR2z4yMphLNEDUIYwACAnMpAGiF2EEQEhg9VUgehFGAISEHfvoGQGiFWEEgOUa3W3a0+CWRM8IEI0IIwAsV3FoVKRfYqxSEmItrgZAbyOMALCcL4wM7s+oCBCNCCMALFdxaCbNYPpFgKhEGAFgucMjIwkWVwLACoQRAJbbwWUaIKoRRgBYrtIfRrhMA0QjwggAS3m8Rt/s9/WMMDICRCPCCABL7a4/qFaPUZzDrqzkeKvLAWABwggAS/lm0gzqlyCH3WZxNQCsQBgBYCn/TBou0QBRizACwFLMpAFAGAFgKVZfBUAYAWAp/+qrhBEgahFGAFjKNzIyhKXggahFGAFgmfqmVtUfbJUk5bIUPBC1CCMALOMbFUnv61RiXIzF1QCwCmEEgGVoXgUgEUYAWGjHvkZJ0hDCCBDVCCMALOO7QV4uYQSIaoQRAJbZsdc3k4YwAkQzwggAy9AzAkAijACwSEubV7vqDkrivjRAtCOMALDErrqD8hopPtau9CSn1eUAsBBhBIAljrxBns1ms7gaAFYijACwxOF+EZaBB6IdYQSAJSr2tq8xQvMqAMIIAEscvkEeYQSIdoQRAJbwrTHCyAgAwgiAXmeM8a++yrReAIQRAL1uX2OLGls8stmkQf0SrC4HgMUIIwB6nW9ab3ZyvJwxDourAWA1wgiAXscN8gAciTACoNdxgzwARyKMAOh1zKQBcCTCCIBeV7GvfcGzIQNYfRUAYQSABbYfGhnJI4wAEGEEQC9rdLdpT4NbEmuMAGhHGAHQq3z9Iv37xCklIdbiagCEAsIIgF61Y6+vX4RREQDtCCMAehX9IgC+jTACoFf5RkaY1gvAhzACoFdtPxRG8tIIIwDaEUYA9KrDq69ymQZAO8IIgF7T3OrR7vpmSfSMADiMMAKg1/hukNc3Pkb9EpnWC6AdYQRArzlyJo3NZrO4GgCholthZNGiRcrLy1N8fLzy8/O1evXqLh23dOlS2Ww2XXrppd15WgBhjjVGAHQm4DCybNkyFRcXa968eVq7dq3GjRunoqIi1dTUHPe47du365e//KXOP//8bhcLILxtJ4wA6ETAYWTBggW6/vrrNXPmTI0ZM0alpaVKTEzU4sWLj3mMx+PR1VdfrQceeEDDhg07qYIBhC9m0gDoTEBhpKWlRWvWrFFhYeHhX2C3q7CwUOXl5cc87sEHH1RGRoauvfbaLj2P2+2Wy+Xq8AAQ/vxrjBBGABwhoDBSW1srj8ejzMzMDtszMzNVVVXV6THvv/++nn/+eT377LNdfp6SkhKlpKT4H7m5uYGUCSAEtbR5tXP/QUlSHpdpABwhqLNpGhoadM011+jZZ59VWlpal4+bPXu26uvr/Y/KysogVgmgN3yzv0leIyXEOpTe12l1OQBCSEwgO6elpcnhcKi6urrD9urqamVlZR21/5YtW7R9+3ZNnTrVv83r9bY/cUyMNm3apOHDhx91nNPplNPJmxUQSQ73iyQyrRdABwGNjMTFxWnChAkqKyvzb/N6vSorK1NBQcFR+48aNUqfffaZ1q9f73/86Ec/0oUXXqj169dz+QWIIjvoFwFwDAGNjEhScXGxZsyYoYkTJ2rSpElauHChGhsbNXPmTEnS9OnTlZOTo5KSEsXHx2vs2LEdjk9NTZWko7YDiGy+Bc+GcIM8AN8ScBiZNm2a9uzZo7lz56qqqkrjx4/XihUr/E2tFRUVsttZ2BVAR/4Fz/ozMgKgI5sxxlhdxIm4XC6lpKSovr5eycnJVpcDoBu+9/hKba1t1IvX5eucEV1vaAcQvrr6+c0QBoCga/N4Vbnfd5mGkREAHRFGAATd7vpmtXqM4mLsyk6Ot7ocACGGMAIg6Hwrrw7unyi7nWm9ADoijAAIOt9MGlZeBdAZwgiAoNtR67tbL/0iAI5GGAEQdNuPWH0VAL6NMAIg6Cr2MTIC4NgIIwCCyus1/vvS0DMCoDOEEQBBVd3QLHebVzF2m3JSE6wuB0AIIowACKotNe2XaHL7JyrGwVsOgKPxzgAgqLbsOSBJGp6eZHElAEIVYQRAUPnCyIgMwgiAzhFGAATV5hrfyAgzaQB0jjACIKj8l2kYGQFwDIQRAEHT0NyqapdbEj0jAI6NMAIgaLbsaZ9Jk97XqZSEWIurARCqCCMAgmYL/SIAuoAwAiBoNjOTBkAXEEYABM3hkRHCCIBjI4wACBoWPAPQFYQRAEHR6vH6b5DHZRoAx0MYARAUO/Y2qc1rlBjnUHZKvNXlAAhhhBEAQXHkJRqbzWZxNQBCGWEEQFCwDDyAriKMAAgKmlcBdBVhBEBQ+FZfpXkVwIkQRgD0OGPM4TVGCCMAToAwAqDH1TS4dcDdJrtNGjIg0epyAIQ4wgiAHucbFRkyoI+cMQ6LqwEQ6ggjAHrc5j3MpAHQdYQRAD2Oe9IACARhBECP882koXkVQFcQRgD0uM2MjAAIAGEEQI864G5TlatZkjSCMAKgCwgjAHqUr18kLcmplMRYi6sBEA4IIwB61BZm0gAIEGEEQI/yhRGWgQfQVYQRAD3K17w6jH4RAF1EGAHQo77c3SBJGp3V1+JKAIQLwgiAHuNqblXFviZJ0piByRZXAyBcEEYA9JgvdrkkSTmpCUpNjLO4GgDhgjACoMf4wsjobEZFAHQdYQRAj9lwKIycxiUaAAEgjADoMRt21UsijAAIDGEEQI9wt3n803ppXgUQCMIIgB7xdfUBtXmNUhJilZOaYHU5AMIIYQRAj/jiiH4Rm81mcTUAwglhBECP8PWLjGEmDYAAEUYA9Aj/TJocwgiAwBBGAJw0r9foy93tYWRMdorF1QAIN4QRACdtx74mNbZ45Iyxa3h6H6vLARBmCCMATpqvX2RUVl/FOHhbARAY3jUAnDTfTBrWFwHQHYQRACdtgz+M0C8CIHCEEQAn7Qt/8yojIwACRxgBcFJqGpq1p8Etm00and3X6nIAhKFuhZFFixYpLy9P8fHxys/P1+rVq4+577PPPqvzzz9f/fr1U79+/VRYWHjc/QGEF98lmmFpfZQYF2NxNQDCUcBhZNmyZSouLta8efO0du1ajRs3TkVFRaqpqel0/5UrV+rKK6/UO++8o/LycuXm5mry5MnauXPnSRcPwHpf0C8C4CQFHEYWLFig66+/XjNnztSYMWNUWlqqxMRELV68uNP9//d//1c333yzxo8fr1GjRum5556T1+tVWVnZSRcPwHpH3pMGALojoDDS0tKiNWvWqLCw8PAvsNtVWFio8vLyLv2OpqYmtba2qn///sfcx+12y+VydXgACE2+5lXCCIDuCiiM1NbWyuPxKDMzs8P2zMxMVVVVdel33HXXXRo4cGCHQPNtJSUlSklJ8T9yc3MDKRNALzngbtO22kZJzKQB0H29Optm/vz5Wrp0qV5++WXFx8cfc7/Zs2ervr7e/6isrOzFKgF0le8STWayUwOSnBZXAyBcBdT6npaWJofDoerq6g7bq6urlZWVddxjH3/8cc2fP19vvfWWzjjjjOPu63Q65XTyxgaEuo+275MknTW4n8WVAAhnAY2MxMXFacKECR2aT33NqAUFBcc87tFHH9VDDz2kFStWaOLEid2vFkBIWb2tPYxMGnrsHjAAOJGAFwUoLi7WjBkzNHHiRE2aNEkLFy5UY2OjZs6cKUmaPn26cnJyVFJSIkn6z//8T82dO1cvvvii8vLy/L0lSUlJSkpK6sFTAdCb2jxerdmxXxJhBMDJCTiMTJs2TXv27NHcuXNVVVWl8ePHa8WKFf6m1oqKCtnthwdcfve736mlpUU//elPO/yeefPm6f777z+56gFY5svdDTrgblPf+BiNyqJ5FUD3dWu5xFtuuUW33HJLpz9buXJlh++3b9/enacAEOI+3LZXknR2Xn857DaLqwEQzrg3DYBuoV8EQE8hjAAImNdr/DNpCCMAThZhBEDANu85oP1NrUqIdWgs96QBcJIIIwAC9uGhSzRnDUlVXAxvIwBODu8iAALm7xfJG2BxJQAiAWEEQECMMVp9aCYN/SIAegJhBEBAKvY1qdrlVqzDpjMHp1pdDoAIQBgBEJAPt7Zfohk3KFXxsQ6LqwEQCQgjAALyIeuLAOhhhBEAAVm9nX4RAD2LMAKgy3bVHVTlvoOy26QJQ/pZXQ6ACEEYAdBlvlVXTxuYor7xsRZXAyBSEEYAdBn9IgCCgTACoEuMMfpgc60kwgiAnkUYAdAlG6satGNvk+Ji7Dp3RJrV5QCIIIQRAF3y+udVkqTvnJKuJGeMxdUAiCSEEQBd8sahMDJlbJbFlQCINIQRACe0dc8BbapuUIzdpsLRmVaXAyDCEEYAnJDvEk3B8AFKSWRKL4CeRRgBcEIr/Jdosi2uBEAkIowAOK7KfU36bGe97DZp8mlcogHQ8wgjAI7rjQ3toyJn5/VXWpLT4moARCLCCIDjWsEsGgBBRhgBcEw1rmatqdgvSbqYfhEAQUIYAXBMb2yokjHSmYNTlZUSb3U5ACIUYQTAMfmm9F58GpdoAAQPYQRAp/Y1tvjv0suUXgDBRBgB0KlXP90lj9doTHayBg9ItLocABGMMALgKB6v0eL3t0mSpp2da3E1ACIdYQTAUcq+rNb2vU1KSYjVFRMHWV0OgAhHGAFwlOf+2T4qcnX+YCXGxVhcDYBIRxgB0MEnlXVavX2fYh02zTgnz+pyAEQBwgiADp471CsyddxAZSaztgiA4COMAPDbWXdQyz/bLUm67rxhFlcDIFoQRgD4Lflgmzxeo3NHDNCYgclWlwMgShBGAEiSGppbtXR1pSRGRQD0LsIIAEnSso8q1eBu04iMJH331HSrywEQRQgjANTobtPzhxpXrz1vqOx2m8UVAYgmhBEAeuIfX2l3fbMG9UvQT87MsbocAFGGMAJEuU8q67RkVfuoyK8vHav4WIfFFQGINoQRIIq1ery6+y+fyWukH48fqAtGZlhdEoAoRBgBotji97fpy90upSbGas4Px1hdDoAoRRgBolTF3iY9+dZXkqR7fzBaaUlOiysCEK0II0AUMsbo3lc+U3OrVwXDBuinE7gzLwDrEEaAKPQ//9qhf35dq7gYux657HTZbEzlBWAdwggQZV7/bLfm/W2DJOnO75+qoWl9LK4IQLQjjABRZNWWWt22dL28Rrpy0mDd8B2WfQdgPcIIECU+31mvG/57jVo8Xl18WpZ+felYLs8ACAmEESAKbKtt1IzFq3XA3aaCYQO08Gfj5WDJdwAhgjACRLh1Fft11bP/0t7GFp02MFnPTJ/AKqsAQkqM1QUACA5jjJ5/f5vmv75RbV6jYel9tGTmJPWNj7W6NADogDACRKC6phb98qVP9daX1ZKkS07PVsnlpyuZIAIgBBFGgAhijNE7m2o055UN2ll3UHEOu+ZMHaP/kz+YZlUAIYswAkQAY4z++XWtFrz5ldZX1kmShgxI1KKrztLYnBRriwOAEyCMAGHM4zX6YHOtnnr7a320fb8kKT7WrukFebrleyO4LAMgLBBGgDBjjNG6yjr9/ZNdeu3T3appcEuS4mLs+j/5Q3TjBcOU0Tfe4ioBoOu6NbV30aJFysvLU3x8vPLz87V69erj7v/SSy9p1KhRio+P1+mnn67ly5d3q1ggGhljVLmvSX9Z+43uefkznf/oO7rs6VV64YPtqmlwKzk+Rj8/J0///NWFmjt1DEEEQNgJeGRk2bJlKi4uVmlpqfLz87Vw4UIVFRVp06ZNysjIOGr/VatW6corr1RJSYl++MMf6sUXX9Sll16qtWvXauzYsT1yEkCkaGhu1dY9jdqy54C27mnU1zUNWldR5x/98EmMc2jymExNHTdQ55+SrrgYlgwCEL5sxhgTyAH5+fk6++yz9dvf/laS5PV6lZubq1tvvVV33333UftPmzZNjY2NevXVV/3b/u3f/k3jx49XaWlpl57T5XIpJSVF9fX1Sk5ODqRcwBLGGLnbvGp0t6nR7VFjS5sOuNtU19SquqYW1R9sVV1Tq2oamlXlcqu6vllVrmbVH2zt9PfFOmw6bWCKJg7pp7OH9td3TklXQhwLlwEIbV39/A5oZKSlpUVr1qzR7Nmz/dvsdrsKCwtVXl7e6THl5eUqLi7usK2oqEivvPLKMZ/H7XbL7T78X4IulyuQMrvs+fe36Zv9TUH53ZEksLgajOfvWIDxbz9ym+mwreM+h39mTPu+3iP+LSN5Tftv8BrJ6zXymvaHx6tDXzs+Wr1etXq8avMYtXq8avF45W49/NXd5pG3m/+7pfd1alhaHw3PSNKwtD46PSdF43JTWTUVQMQKKIzU1tbK4/EoMzOzw/bMzExt3Lix02Oqqqo63b+qquqYz1NSUqIHHnggkNK65bVPd2ltRV3QnwfRLSHWoT5Oh/o4Y5SaGKfUhFilJsYqJSFWGX2dykiOV1ZyvDKT45WdGs8MGABRJyRn08yePbvDaIrL5VJubm6PP8/lEwapYPiAHv+9aGdT9xbZ6mxtrqM2dbKT7Vs/sskmm63jdpvNt+3wz+y+bbb2ih12m+w2yW63yW6zyWGzyWHv+Iix2xQbY1es3a4Yh02xDrviY+1yxjjkjLHLGWtXn7gYJcQ6ZOeGdABwXAGFkbS0NDkcDlVXV3fYXl1draysrE6PycrKCmh/SXI6nXI6nYGU1i1X5w8J+nMAAIDjC6gFPy4uThMmTFBZWZl/m9frVVlZmQoKCjo9pqCgoMP+kvTmm28ec38AABBdAr5MU1xcrBkzZmjixImaNGmSFi5cqMbGRs2cOVOSNH36dOXk5KikpESSdNttt+m73/2unnjiCV1yySVaunSpPv74Yz3zzDM9eyYAACAsBRxGpk2bpj179mju3LmqqqrS+PHjtWLFCn+TakVFhez2wwMu55xzjl588UXdd999uueee3TKKafolVdeYY0RAAAgqRvrjFiBdUYAAAg/Xf38ZtlGAABgKcIIAACwFGEEAABYijACAAAsRRgBAACWIowAAABLEUYAAIClCCMAAMBShBEAAGCpgJeDt4JvkViXy2VxJQAAoKt8n9snWuw9LMJIQ0ODJCk3N9fiSgAAQKAaGhqUkpJyzJ+Hxb1pvF6vdu3apb59+8pms/XY73W5XMrNzVVlZWXE3vMm0s+R8wt/kX6OnF/4i/RzDOb5GWPU0NCggQMHdriJ7reFxciI3W7XoEGDgvb7k5OTI/L/YEeK9HPk/MJfpJ8j5xf+Iv0cg3V+xxsR8aGBFQAAWIowAgAALBXVYcTpdGrevHlyOp1WlxI0kX6OnF/4i/Rz5PzCX6SfYyicX1g0sAIAgMgV1SMjAADAeoQRAABgKcIIAACwFGEEAABYKirDyPbt23Xttddq6NChSkhI0PDhwzVv3jy1tLR02O/TTz/V+eefr/j4eOXm5urRRx+1qOLAPfzwwzrnnHOUmJio1NTUTvex2WxHPZYuXdq7hZ6ErpxjRUWFLrnkEiUmJiojI0P/8R//oba2tt4ttAfl5eUd9ZrNnz/f6rK6bdGiRcrLy1N8fLzy8/O1evVqq0vqMffff/9Rr9WoUaOsLqvb3nvvPU2dOlUDBw6UzWbTK6+80uHnxhjNnTtX2dnZSkhIUGFhob7++mtriu2mE53jz3/+86Ne04svvtiaYgNUUlKis88+W3379lVGRoYuvfRSbdq0qcM+zc3NmjVrlgYMGKCkpCRdfvnlqq6u7pX6ojKMbNy4UV6vV7///e+1YcMGPfnkkyotLdU999zj38flcmny5MkaMmSI1qxZo8cee0z333+/nnnmGQsr77qWlhZdccUVuummm4673wsvvKDdu3f7H5deemnvFNgDTnSOHo9Hl1xyiVpaWrRq1Sr94Q9/0JIlSzR37txerrRnPfjggx1es1tvvdXqkrpl2bJlKi4u1rx587R27VqNGzdORUVFqqmpsbq0HnPaaad1eK3ef/99q0vqtsbGRo0bN06LFi3q9OePPvqofvOb36i0tFQffvih+vTpo6KiIjU3N/dypd13onOUpIsvvrjDa/rHP/6xFyvsvnfffVezZs3Sv/71L7355ptqbW3V5MmT1djY6N/njjvu0N///ne99NJLevfdd7Vr1y5ddtllvVOggTHGmEcffdQMHTrU//3TTz9t+vXrZ9xut3/bXXfdZUaOHGlFed32wgsvmJSUlE5/Jsm8/PLLvVpPMBzrHJcvX27sdrupqqryb/vd735nkpOTO7yu4WTIkCHmySeftLqMHjFp0iQza9Ys//cej8cMHDjQlJSUWFhVz5k3b54ZN26c1WUExbffO7xer8nKyjKPPfaYf1tdXZ1xOp3mj3/8owUVnrzO3h9nzJhhfvzjH1tST0+rqakxksy7775rjGl/vWJjY81LL73k3+fLL780kkx5eXnQ64nKkZHO1NfXq3///v7vy8vL9Z3vfEdxcXH+bUVFRdq0aZP2799vRYlBMWvWLKWlpWnSpElavHjxCW/zHE7Ky8t1+umnKzMz07+tqKhILpdLGzZssLCykzN//nwNGDBAZ555ph577LGwvOzU0tKiNWvWqLCw0L/NbrersLBQ5eXlFlbWs77++msNHDhQw4YN09VXX62KigqrSwqKbdu2qaqqqsPrmZKSovz8/Ih6PSVp5cqVysjI0MiRI3XTTTdp7969VpfULfX19ZLk/9xbs2aNWltbO7yGo0aN0uDBg3vlNQyLG+UF2+bNm/XUU0/p8ccf92+rqqrS0KFDO+zn+1CrqqpSv379erXGYHjwwQf1ve99T4mJifrHP/6hm2++WQcOHNAvfvELq0vrEVVVVR2CiNTxNQxHv/jFL3TWWWepf//+WrVqlWbPnq3du3drwYIFVpcWkNraWnk8nk5fn40bN1pUVc/Kz8/XkiVLNHLkSO3evVsPPPCAzj//fH3++efq27ev1eX1KN/fU2evZ7j+rXXm4osv1mWXXaahQ4dqy5YtuueeezRlyhSVl5fL4XBYXV6Xeb1e3X777Tr33HM1duxYSe2vYVxc3FH9d731GkbUyMjdd9/daVPmkY9vv9Ht3LlTF198sa644gpdf/31FlXeNd05v+OZM2eOzj33XJ155pm666679Ktf/UqPPfZYEM/gxHr6HMNBIOdcXFysCy64QGeccYZuvPFGPfHEE3rqqafkdrstPgt825QpU3TFFVfojDPOUFFRkZYvX666ujr96U9/sro0dNPPfvYz/ehHP9Lpp5+uSy+9VK+++qo++ugjrVy50urSAjJr1ix9/vnnITVhIaJGRu688079/Oc/P+4+w4YN8/97165duvDCC3XOOecc1ZialZV1VBex7/usrKyeKThAgZ5foPLz8/XQQw/J7XZbdo+CnjzHrKyso2ZnWP0aduZkzjk/P19tbW3avn27Ro4cGYTqgiMtLU0Oh6PTv7FQem16Umpqqk499VRt3rzZ6lJ6nO81q66uVnZ2tn97dXW1xo8fb1FVwTds2DClpaVp8+bNuuiii6wup0tuueUWvfrqq3rvvfc0aNAg//asrCy1tLSorq6uw+hIb/1NRlQYSU9PV3p6epf23blzpy688EJNmDBBL7zwguz2joNEBQUFuvfee9Xa2qrY2FhJ0ptvvqmRI0dadokmkPPrjvXr16tfv36W3iypJ8+xoKBADz/8sGpqapSRkSGp/TVMTk7WmDFjeuQ5esLJnPP69etlt9v95xcu4uLiNGHCBJWVlflncHm9XpWVlemWW26xtrggOXDggLZs2aJrrrnG6lJ63NChQ5WVlaWysjJ/+HC5XPrwww9POKMvnH3zzTfau3dvhwAWqowxuvXWW/Xyyy9r5cqVR7UhTJgwQbGxsSorK9Pll18uSdq0aZMqKipUUFDQKwVGnW+++caMGDHCXHTRReabb74xu3fv9j986urqTGZmprnmmmvM559/bpYuXWoSExPN73//ewsr77odO3aYdevWmQceeMAkJSWZdevWmXXr1pmGhgZjjDF/+9vfzLPPPms+++wz8/XXX5unn37aJCYmmrlz51pceded6Bzb2trM2LFjzeTJk8369evNihUrTHp6upk9e7bFlXfPqlWrzJNPPmnWr19vtmzZYv7nf/7HpKenm+nTp1tdWrcsXbrUOJ1Os2TJEvPFF1+YG264waSmpnaY/RTO7rzzTrNy5Uqzbds288EHH5jCwkKTlpZmampqrC6tWxoaGvx/Y5LMggULzLp168yOHTuMMcbMnz/fpKammr/+9a/m008/NT/+8Y/N0KFDzcGDBy2uvOuOd44NDQ3ml7/8pSkvLzfbtm0zb731ljnrrLPMKaecYpqbm60u/YRuuukmk5KSYlauXNnhM6+pqcm/z4033mgGDx5s3n77bfPxxx+bgoICU1BQ0Cv1RWUYeeGFF4ykTh9H+uSTT8x5551nnE6nycnJMfPnz7eo4sDNmDGj0/N75513jDHGvP7662b8+PEmKSnJ9OnTx4wbN86UlpYaj8djbeEBONE5GmPM9u3bzZQpU0xCQoJJS0szd955p2ltbbWu6JOwZs0ak5+fb1JSUkx8fLwZPXq0eeSRR8LijfBYnnrqKTN48GATFxdnJk2aZP71r39ZXVKPmTZtmsnOzjZxcXEmJyfHTJs2zWzevNnqsrrtnXfe6fTvbcaMGcaY9um9c+bMMZmZmcbpdJqLLrrIbNq0ydqiA3S8c2xqajKTJ0826enpJjY21gwZMsRcf/31YROej/WZ98ILL/j3OXjwoLn55ptNv379TGJiovnJT37S4T/Sg8l2qEgAAABLRNRsGgAAEH4IIwAAwFKEEQAAYCnCCAAAsBRhBAAAWIowAgAALEUYAQAAliKMAAAASxFGAACApQgjAADAUoQRAABgKcIIAACw1P8Hn97TFmSu4G4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "x = torch.linspace(-20, 20, 100)\n",
    "\n",
    "def sigm(x):\n",
    "    return 1/(1 + torch.exp(-x))\n",
    "\n",
    "y = sigm(x)\n",
    "\n",
    "plt.plot(x, y);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Задание 3:\n",
    "\n",
    "Реализуйте на PyTorch среднюю квадратичную ошибку. \n",
    "\n",
    "$$ \n",
    "MSE(\\hat y, y) = \\frac{1}{n} \\cdot \\sum_{i=1}^n (\\hat y - y)^2\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = torch.tensor([1, 2, 3, 4, 5], dtype=torch.float)\n",
    "y_test = torch.tensor([-1, 3, 4, 6, -5], dtype=torch.float)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "22.0"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def mse(y_true, y_predict):\n",
    "    return torch.mean((y_true - y_pred)**2)\n",
    "\n",
    "z = mse(y_test, y_pred)\n",
    "z.item()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Задание 4:\n",
    "\n",
    "Что будет в переменной `x` в результате выполнения следующего кода? Почему?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "x = torch.tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])\n",
    "y = x\n",
    "y[2] = torch.ones(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1, 2, 3],\n",
       "        [4, 5, 6],\n",
       "        [1, 1, 1]])"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1, 2, 3],\n",
       "        [4, 5, 6],\n",
       "        [1, 1, 1]])"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1, 2, 3],\n",
       "        [4, 5, 6],\n",
       "        [7, 8, 9]])"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = torch.tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])\n",
    "y = x.clone()\n",
    "y[2] = torch.ones(3)\n",
    "\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "------------------------------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Производные да градиенты\n",
    "\n",
    "Если в PyTorch те же самые операции, что и в numpy, то на кой чёрт он нам нужен? В отличие от numpy, в PyTorch есть возможность при создании тензора указывать нужно ли считать по нему градиент или нет, с помощью параметра `requires_grad`. \n",
    "\n",
    "Когда `requires_grad=True` мы сообщаем фреймворку, о том, что мы хотим следить за всеми тензорами, которые получаются из созданного. Иными словами, у любого тензора, у которого указан данный параметр, будет доступ к цепочке операций и преобразований совершенными с ними. Если эти функции дифференцируемые, то у тензора появляется параметр `.grad`, в котором хранится значение градиента.\n",
    "\n",
    "Он проходит по всем операциям, которые фигурируют в графе вычислений, и применяет к ним chain rule:\n",
    "\n",
    "$$ {\\partial f(g(x)) \\over \\partial x} = {\\partial f(g(x)) \\over \\partial g(x)}\\cdot {\\partial g(x) \\over \\partial x} $$\n",
    "\n",
    "Давайте попробуем!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.tensor(\n",
    "    [0.3, 1], requires_grad=True\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = torch.sum(x)\n",
    "z = y ** 2 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Если к результирующему тензору применить метод `.backward()`, то фреймворк посчитает по цепочке градиенты для всех тензоров, у которых `requires_grad=True.`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None <- gradient\n"
     ]
    }
   ],
   "source": [
    "# пока градиента нет\n",
    "print(x.grad, '<- gradient')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([2.6000, 2.6000]) <- gradient\n"
     ]
    }
   ],
   "source": [
    "z.backward() # считаем градиент\n",
    "print(x.grad, '<- gradient')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Важно:** если вы вычисляете градиент дважды, градиенты будут суммироваться в тензорах, поэтому полезно обнулять градиенты между итерациями.\n",
    "\n",
    "Обычно мы обучаем модель не по одному объекту, а на большой выборке. Вся выборка обычно в память не влезает. Мы прогоняем через нейросеть данные батчами. PyTorch позволяет накапливать градиенты по батчам не помещая в память все данные. Иногда это оказывается полезным."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<PowBackward0 object at 0x11e978280>\n",
      "<SumBackward0 object at 0x11e978700>\n",
      "<AccumulateGrad object at 0x11e978280>\n"
     ]
    }
   ],
   "source": [
    "# можно посмотреть на порядок действий\n",
    "print(z.grad_fn)\n",
    "print(z.grad_fn.next_functions[0][0])\n",
    "print(z.grad_fn.next_functions[0][0].next_functions[0][0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Задание 5:\n",
    "\n",
    "Реализуйте расчёт градиента для функции \n",
    "\n",
    "$$\n",
    "f(w) = \\prod_{i,j} \\ln(\\ln(w_{ij} + 7) \n",
    "$$\n",
    "\n",
    "в точке `w = [[5,10], [1,2]]`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0.0201, 0.0109],\n",
      "        [0.0449, 0.0351]])\n"
     ]
    }
   ],
   "source": [
    "# ваш код\n",
    "w =  torch.tensor([[5.,10.],[1.,2.]], requires_grad=True)\n",
    "\n",
    "function =  torch.prod(torch.log(torch.log(w + 7)))\n",
    "function.backward()\n",
    "\n",
    "print(w.grad)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "------------------------------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Оптимизация\n",
    "\n",
    "Итак, PyTorch умеет искать производные, осталось научиться применять его для оптимизации. \n",
    "\n",
    "Давайте обучим линейную регрессию."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/P9b71AAAACXBIWXMAAA9hAAAPYQGoP6dpAABd0ElEQVR4nO3dfXhU5Zk/8O8k5B0yIUQzQXmJvMcICCuQorbGUFC0VNnfr77tqmWxIvRSsC3iVaoudlG7W2pX1NZa2F0Fqq6UIpb98SYsmIACESNKIQahkkAJZBICeTFzfn/EM8xMzstzzpxz5szM93NduVqSyZlnZuI89zz3/dyPR5IkCUREREQOSYn1AIiIiCi5MPggIiIiRzH4ICIiIkcx+CAiIiJHMfggIiIiRzH4ICIiIkcx+CAiIiJHMfggIiIiR/WK9QAiBQIBnDhxAn369IHH44n1cIiIiEiAJEloaWlB//79kZKivbbhuuDjxIkTGDBgQKyHQURERCYcP34cl19+ueZtXBd89OnTB0D34HNzc2M8GiIiIhLR3NyMAQMGBOdxLa4LPuRUS25uLoMPIiKiOCNSMsGCUyIiInIUgw8iIiJyFIMPIiIichSDDyIiInIUgw8iIiJyFIMPIiIichSDDyIiInIUgw8iIiJylOuajNmlKyBhT90ZNPgv4ExrB/J7Z8CXm4kJxflITVFviNIVkPD+kdP4731/xfmOr/B3A/MxwtcHe744gxNnL6B/Xha+MaQA1xTn44O6M6j8/DQAD8qG9MOkK/r1uLbeOOSfn2ppw6V99Mdn5XMT7X3qXScWj42IiNzHI0mSJHrjJ598Ek899VTY90aMGIHPPvsMANDW1oZHH30Ua9asQXt7O6ZOnYoXX3wRhYWFwgNqbm6G1+uF3++3rMPpxpp6PLX+IOr9bT1+VuTNxBO3lmBaaZHi7y144yOc7+gydb952Wl45vargtfWG8d3xhThTx/Vh/1ca3xWUBqTmfvUu45V90NERO5kZP42HHy89dZb2Lx5c/B7vXr1QkFBAQBgzpw52LBhA1auXAmv14t58+YhJSUFu3btsmXwIjbW1GPOa/ug9SA9AF66Z1zYJLixph4PvrYv6vsHgJfvGQcAuuNQGxsUxmcFtefG6H3qXeeB64vx2x11Ud8PERG5l5H523DapVevXvD5fD2+7/f78eqrr2LVqlUoLy8HAKxYsQKjRo1CVVUVJk2aZPSuotYVkPDU+oNCE/5T6w9iSokvmPp4Yt0nlo3jqfUHIUmS4cADACR0T9Kh47OC1nNj5D5FrvPK//YMPIzeDxERJQ7DBaeHDx9G//79ccUVV+Duu+/GsWPHAAB79+5FZ2cnKioqgrcdOXIkBg4ciMrKStXrtbe3o7m5OezLKnvqziimOCJJAOr9bdhTdyb4eydb2i0bR72/DQ3N5q8XOT4r6D03ovcpcp2ARtRlx2MjIiJ3MxR8TJw4EStXrsTGjRvx0ksvoa6uDtdddx1aWlrQ0NCA9PR05OXlhf1OYWEhGhoaVK+5dOlSeL3e4NeAAQNMPRAlp1r0Aw+l2xv9PadYOS7Ra+ndzqoxufU5JyIi6xlKu9x0003B/z969GhMnDgRgwYNwhtvvIGsrCxTA1i0aBEWLFgQ/Hdzc7NlAcilfTJN3d7o7znFynGJXkvvdlaNya3PORERWS+qPh95eXkYPnw4jhw5Ap/Ph46ODjQ1NYXd5uTJk4o1IrKMjAzk5uaGfVllQnE+irz6k5oH3TsvJhTnB3+vsE+GZeMo8mbCl2v+epHjs4L83KhVWYjep8h1tEo57HhsRETkblEFH+fOnUNtbS2Kioowfvx4pKWlYcuWLcGfHzp0CMeOHUNZWVnUAzUjNcWDJ24tUZ0YQz1xa0mw4DE1xYOnZlxp2TieuLUET37nSqFxRJJ/J3R8VpCfm9D7MHOfIteZfV0xPFHeDxERJQ5DwcePfvQjbN++HUePHsX777+P2267Dampqbjzzjvh9Xoxa9YsLFiwANu2bcPevXtx//33o6ysLCY7XWTTSovw0j3jVFdAiryZils9p5UW4eV7xiE7PdX0fffNTsPLX19bZBw/uL64x899KuPrCkiorG3EuuovUVnbiC6tqk4V8ph8gvdp9jqLbi6x5H6IiCgxGOrzcccdd2DHjh1obGzEJZdcgmuvvRY///nPMWTIEAAXm4ytXr06rMmYVtolkh1NxoDE6nBqdcMudjglIqJo2dZkzAl2BR+JwqrGYERERFYyMn/zYDmLWZEO0bq2VkMvoLthl5X3SUREZLWkOVjOCUrpkLysNNw/eTDmlQ+LOsVgpDFY2ZB+Ud0XERGRXbjyYRE5HRIZHDRd6MSyzYcx/ulN2FhTH9V9WNUYjIiIKJYYfFhA5AyZpvOdmPPavqgCEKsagxEREcUSgw8LGDlDJpqaDKsagxEREcUSgw8LGElzRHOImlWNwYiIiGKJwYcFjKY5oqnJsKoxGBERUaxwt4sF5HSISOoFiL4mY1ppEaaU+Niwi4iI4hKDDwvI6ZAHX9uneTsPulcorKjJSE3xcDstERHFJaZdLCKfBZOXnab4c9ZkEBERdWPwYaFppUXY+9MpmF8xHHlZ4UEIazKIiIi68WwXm/AQNSIiSiZG5m/WfNiENRlERETKGHzYgKseRERE6hh8WEzpcLkibyaeuLWE9R5ERERgwaml1A6Xa/C3RX2uCxERUaJg8GERrcPl5O9Fc64LERFRomDwYRG9w+UkRHeuCxERUaJgzYcgvSJS0fNaojnXhYiIKBEw+BAgUkQqel5LtOe6EBERxTumXXSIFpHKh8upbaj1oDtgseJcFyIionjG4EODkSJS+XA5AD0CEJ7rQkREdBGDDw1Gi0inlRbhpXvGwecNT63wXBciIqKLWPOhwWgRaVdAgjcrHT+ZNhJnzrUjPycdPm8WO5wSERGFYPChwUgRqVZRKgMPIiKii5h20SBaRHq2tZ2dTYmIiAQx+NAgUkS6eHoJlmz4lJ1NiYiIBDH40KFXRNo3J12oKHXZpkOorG1kEEJEREnPI0mSq2bD5uZmeL1e+P1+5Obmxno4QWodTtdVf4mH11QLX4cn3BIRUSIyMn+z4FRQaooHZUP69fi+0Y6lch0It94SEVGyYtolSnpFqZFYB0JERMmOwUeUtIpS1fCEWyIiSmYMPiygVpSqhyfcEhFRMmLNhwFqRadAdwAypcSHPXVnsOvIabyw7Yju9XjCLRERJSMGH4K0OpjKhaNyUeqE4nz8976/osHfptj/w4Purbo84ZaIiJIR0y4CNtbUa3YwffdAPSprG7Gu+ktU1jYCAE+4JSIiUsE+Hzq6AhKufXarZiOxFA8QunFFXhEBoLtaQkRElAjY58NCe+rOaAYeQHjgAYT38ti5sFy1ToSIiCgZMfjQYWZHioTu9MpT6w9iSolPsTkZERFRsmLNhw6zO1LYy4OIiEgZVz50TCjOhy83Aw3N7aZ+3829PLS2DhMREdmFwYeO1BQP7pwwEMs2Hzb1+27t5SGydZiIiMgOTLsIGFyQY/h3POiezN3Yy0Nv6/DGmvoYjYyIiJIBgw8BZlcv3NjLoysg4an1BxWbn/HQOyIicgKDDwFGT64FgAeuL3Zl+kJv6zALZYmIyG4MPgQYPbnWA+BPH9W7cvVAtADWzYWyREQU3xh8CJpWWoTld41D35x03du6efVANIXk1kJZIiKKfww+BG2sqceSDQdxprVD+HfcuHqgl0Jyc6EsERElBgYfAtR2h+hx4+qBVgqJh94REZETGHzo0NodosbtqwfTSovw0j3j4POGB0c+byZeumecKwtliYgocbDJWAiljp8iB8uFipfVg2mlRZhS4mOHUyIichyDj6+pdfy8qdRn6Dq+OOoSmpri4aF3RETkOAYfuFjTEZlaafC34fe7jgpdY94NQzF5aAFXD4iIiHQkffCh1/HTA8DjAdRadnjQvdoxf8pw4aCDB7oREVEyS/rgQ6Tjp/R14OEBwoIUM/UdPNCNiIiSXdLvdhHtxTFr8uCod4fwQDciIiKufAj34qgo8eHx6SWm0yUi6Z2n1h/ElBIfUzBERJTQkj74kDt+NvjbFAMDuaZDDjTM7g4xcqAbd6AQEVEiS/q0i1MdP3mgGxERUbekDz4AZzp+8kA3IiKibkmfdpHZ3fHTSHqHiIgokTH4CGFnx085vTPntX2WbNklIiKKV0y7OIgHuhEREXHlw3E80I2IiJJdVCsfzzzzDDweDx555JHg99ra2jB37lz069cPvXv3xsyZM3Hy5Mlox5lQ5PTOjLGXoWxIPwYeRESUVEwHHx988AF+85vfYPTo0WHfnz9/PtavX48333wT27dvx4kTJ3D77bdHPVCyV1dAQmVtI9ZVf4nK2kZ0qR1mQ0REFCVTaZdz587h7rvvxiuvvIKnn346+H2/349XX30Vq1atQnl5OQBgxYoVGDVqFKqqqjBp0iRrRk2W4nkzRETkJFMrH3PnzsX06dNRUVER9v29e/eis7Mz7PsjR47EwIEDUVlZqXit9vZ2NDc3h32Rc3jeDBEROc1w8LFmzRrs27cPS5cu7fGzhoYGpKenIy8vL+z7hYWFaGhoULze0qVL4fV6g18DBgwwOiQySe+8GaD7vBmmYIiIyEqGgo/jx4/j4Ycfxuuvv47MTGs6cS5atAh+vz/4dfz4cUuuS/qMnDdDRERkFUPBx969e3Hq1CmMGzcOvXr1Qq9evbB9+3b8+te/Rq9evVBYWIiOjg40NTWF/d7Jkyfh8/kUr5mRkYHc3NywL3IGz5shIqJYMFRweuONN+Ljjz8O+97999+PkSNHYuHChRgwYADS0tKwZcsWzJw5EwBw6NAhHDt2DGVlZdaNmizB82aIiCgWDAUfffr0QWlpadj3cnJy0K9fv+D3Z82ahQULFiA/Px+5ubn44Q9/iLKyMu50cSGeN0NERLFgeYfTZcuWISUlBTNnzkR7ezumTp2KF1980eq7iStdAcmVHU153gwREcWCR5IkV21laG5uhtfrhd/vT4j6j3jooREPYyQiInczMn8z+LCR3EMj8gmW1xHcdJicW1dniIgoPhiZv3mwnE30emh40N1DY0qJzxWTvHzeDBERkd2iOliO1LGHBhERkTIGHzZhDw0iIiJlTLvYRLQ3RkFOBiprG1lrQURESYPBh01Eemh4s9Pw6JsfoaGZu0yIiCh5MO1iQFdAQmVtI9ZVf4nK2kbNA9fkHhrAxd0tMrmnRtP5zrDAA+BpskRElPi48iHITC+MaaVFeOmecT1+rzA3A21fBdB0vrPH77hxJwwREZGV2OdDgF6/juV3XY2+ORmqdRuRPTQCkoS7f7db935Xz57E7a9ERBQX2OfDQnr9OgBg3ur9CM3ARK6IRPbQWFf9pdB9R6ZkiIiIEgFrPnTo9esAgMjSD726DdGdMEve+YS1H0RElHAYfOgw04dDjkWeWn9QsShV3gmjV81xprUTD7L4lIiIEgyDDx2iqxSRtDqYau2EUfLY2x9r7qwhIiKKJww+dIiuUqhRWzmRd8L0zUnXvUbT+U68sPWIyREQERG5C4MPHUZXKSJprZxMKy3C4umjhK6z4v26mK1+GOlvQkREpIe7XXR0BSR4s9Jx/+TB+GP1CZxp7Qj+LMXTs9hU5gHg83Zvu9Xi82YJjaPpfCf21J1xfOutmf4mREREWhh8aFCaePNz0nDb2MtQUeLD2dYOzF21DwDCtuLKKyRP3Fqi2yRsQnE+8rLS0HShZ8OxSE4fQqfW30TezfPSPeMYgBARkWFMu6iQJ97IbbZnWzvx+11H4b/QgZtHd9dt+LzhqRWfN1N4Yk5N8eD+yYOFxlTQO0N4/EqMpE9E+puo7eaJ9r6JiCixscOpgq6AhGuf3ara30NOqexcWI7UFE+PDqZGT6btCkgY//QmxXbroXy5GXjyO1eaWm0wkj7pCkhYuasOSzZ8qntdkS6sTN0QESU+I/M3Vz4U6DUWi9xGK3cwnTH2MpQN6Wf4PJbUFA+euf0q3YLWk83tpg6dU1vFUWqGtrGmHtc+u1Uo8AD0U0FG7puIiJIDgw8ForUVVtZgyFtvfbnqu2PMpjtE0ydqgYIWrd08VqduiIgoMTD4UCDaWMxsAzI100qL8G//Z4zmbbSalykRXcWp+rxRNVBQ4kF36kRrN4/RFSQiIkoO3O2iQG4s1uBvU5yMRbfRqgmtESnonQFIwOnWdlzaJ9PyVRfR21XWNgqveIju5onFChIREbkfgw8FcmOxOa/tgwfmt9EqUSq+DJUv0PEUsGN1Rjz14RMsFo3VChIREbkb0y4qgjUYUWyjjSRSU3E2pImZEpF0B3Bxa2uD/wLyc9JVi1nl65VdUaA9+K8tnj4KOxeWCz1+vdb0oo+FiIgSC1c+NEwrLcKUEl9U22hlWsWXobR+Lrrqore6onS9SUP6CaWa7ptcLPz47VxBIiKi+JWUKx9GGl5Fu41Wpld8qaRvdnhsWJibobvqYmTHSugqjtYZNtEECnasIBERUXxLupWPWDW8MlNU2f5VZFCkPfGLrK7k56Rh8S1XwpfbcxVHDhQinx/RGg81Vq4gERFR/EuqDqdqZ5XIU6DVn8RDd7WcbmkXbtylRm+clbWNuPOVKt3r6HUljbZjKxERJR8j83fSrHzoNbzyoLvh1ZQSnyUTrWjdhRF647Rqa6ucaiIiIrJD0tR8ONnwykynUFFa4xTdsnr0dKvFoyIiIhKXNMGHUw2vRHe1REtpnBOK8zXbs8tW7znGluZERBQzSRN8ONXwysyuFjOUxpma4sGdEwbq/m5DcztbmhMRUcwkTfDhVMMru1uF641zcEG20HXY0pyIiGIlaYIPu/pYRLKiVXjf7DQA5sbp9pbmRnqsEBFRYkqa3S6AfX0sQukdSgcAKR4gdM715WbgzgkDMbggJ7i1ddPBBlPjtPtQvGjEqscKERG5S1L1+ZDZ3cdC3u0CKLcUX37X1eibk6F7/2bHqXf/segs6nSPFSIicpaR+Tspgw+jzAQBsfyU3xWQ8MLWw1ix6yiaLnQ6fv9K47n22a2qhbjyaszOheVsZkZEFKfYZMxCZoOIaaVFKB9ZiP+qPIovzpzHoPxs/EPZYKT3srfMRmm8eVlpuH9yMeaVD43J5G6kxwqbmxERJT4GHxrUUgUN/jbMeW2fZqpAKQj43c46y1ceugISqj5vRGVtI2r/1oI/15zscRv/hU78avNfMMLXOyapDad6rBARUXxg8KEimnbsakFLvb8ND762Dy9HBC3R1HY89vbHaDrfqXm7aNvHR1sj4/YdOERE5CwGHyrMpgpEOpz+6M2PcKEzAF9uJs62dmDJBuNpnY019Xjw66JSEWZTG1bUrrh5Bw4RETkvafp8GGU2VSDS4fRcexfm/6Ead75ShYdW9TwDRk7rbKypV/z9roCEJ/90UGh8euPVonZGjd74IjnVY4WIiOIDgw8VZlMFVtQtyKsDT60/iK6A1KMxV1VtIxqazd2P6OPSSzuFjk+E3GPF5w2/f583k9tsiYiSDNMuKsymCqyqW5DTJC9sPYI1HxwLW33ITk81fD2jqQ07dqhMKy3ClBKfrT1WiIjI/Rh8qJBTBXNe2wcPlJt1yamEytrG4GQ6flBf5GWlhfXXiMayzX/p8b3zHV2GrmEmtWHXDpXUFA+30xIRJTkGHxr02rED6NE8q8ibieuG9cP6Aw2Oj1eNmfbxBTkZQrfjDhUiIjKKwYcOtVTBpoMNqj1A1h9oQE56KloNrlBYbdbkwago8RlObWysqdctaOUOFSIiMosFpyaI9ABJ65XSY2eHqGgrIPKy0/DyPePw+PTu1Zl3DpwQPkFW3uGiVdDKHSpERBQNrnzoUOpzkZ+TjjOtHaq/IwFoOt+J+RXDexSLivB5M3HHNQOwbPNh4d/5dkkhhhf2RtkVBZg0pB82HWxQTAlppV9EepQAQGFuBp78zpXcoUJERKYw+NCg1qlUK/AINbggGzsXlgdTNkdPt2L1nmNoaG4P3qbIm4nF00f1OOW2KyDhPyq/EL6v+ycXBws5zbaFF+lRAgD/9n/HYvLQAqFxERERRWLwoUJ0FUDL6ZZ2vHPgBC7tk4lbRvdHaooH88qH6W41lVdbRAKPyNqLjq8CeHztx6bawovuXDl9rl3/RkRERCoYfKgQXQVQ4/EASzZ8Gvx3aMpDXqFQOjNFrZBV8T6+/l+59mJjTT0eX1uDM63q23y1+nPwDBYiInICgw8V0XYqlSKih8iUh1ItiS83E21fdQmvtoRuoVVLtahRenw8g4WIiJzA4EOF6Kf7/Jw0zZUGWWjKIxAAHlrV81A40Zbp824YgslDLwmmbMykiJQen2hjNe5wISKiaHCrrQp5FUBtmvWgO5Wya+GNyM9JE7qmnPL4yX8fiGpswwr7oGxIv2AQYCRFJI9bbfUi3s5giTz3RvSsGSIiih2ufKgQXQWoPt4ktPIR6lz7V1GNLdrD7PRWL+LlDBal1JXedmIiIoo9rnxoEFkFsOIUW1FqqxZGUkSiqxfyGSwzxl4WtsriFnKNS+SKj1xbs7GmPkYjIyIiPVz50KG3CuDUzg+tmgu9QlEA6JeTjspFNyK9V/zHmyIdZtW2E8ea0g4nt42RiMhuDD4EaJ3EKjLxWyE/Jx0/v61UcdVCJEX089tKEyLwAPRrXLS2E8cS00RERN0SYzaKIXniB6I/k0XLT6eP0pyg4q1QNBqiqS4nU2J6mCYiIrqIKx8WkCf+yE+1uZm90NwWXXGpzOfNEhqHk4WisUohxFsztHhOExER2YHBh0WUJv4G/wXMf+OjqK5rtLGXWopIJFAwEkzEMoUQb83Q4jVNRERkFwYfFoqc+CtrG6O6nlWNvUQCBSPBhNmD66wSb83Q4jFNRERkJ0M1Hy+99BJGjx6N3Nxc5ObmoqysDH/+85+DP29ra8PcuXPRr18/9O7dGzNnzsTJkyctH3S80GtUpseKeg2RWgMj9Qh6KQSgO4Vgd7MvrRqX5XddDW9Wumsaj8VbmoiIyG4eSYo8hUTd+vXrkZqaimHDhkGSJPzHf/wHfvGLX2D//v248sorMWfOHGzYsAErV66E1+vFvHnzkJKSgl27dgkPqLm5GV6vF36/H7m5uaYelJtsrKnHg6/1bKWuZ/H0UbhvcnFUn967AhKufXar5pJ/kTcTkiShoVn5pFo5hbFzYTlSUzyorG3Ena9U6d736tmTHEkhRKaKzrZ2YMkGd+0okV8HvTSR/BwTEcUjI/O3oZWPW2+9FTfffDOGDRuG4cOH4+c//zl69+6Nqqoq+P1+vPrqq/jlL3+J8vJyjB8/HitWrMD777+Pqir9ySpe6bX3nlZahPkVw4SvJzcSizbwAMTartf721QDDyC8HgFwXwohtBma/0IH5q6ydkeJFe3btXZEuTFNRERkN9M1H11dXXjzzTfR2tqKsrIy7N27F52dnaioqAjeZuTIkRg4cCAqKysxadIkxeu0t7ejvf3i5Nfc3Gx2SI4TrZMYXJBj6LryRBTtbhLRg+pEyMGEW1MIduwosbKoVm1HlI99PogoCRkOPj7++GOUlZWhra0NvXv3xtq1a1FSUoLq6mqkp6cjLy8v7PaFhYVoaGhQvd7SpUvx1FNPGR54rBkpujQyET9SMRzTSossmfjOnFNf0TBKfgxu3Wli9Y4SO4pq4+XMHCIiuxluMjZixAhUV1dj9+7dmDNnDu69914cPHjQ9AAWLVoEv98f/Dp+/LjpaznFaNHlhOJ85GWJnXw7uCDbsoZU+TnpQrfzZvbSPb1XDibcmkKwMh1kZ1Gt28/MISJyguHgIz09HUOHDsX48eOxdOlSjBkzBs8//zx8Ph86OjrQ1NQUdvuTJ0/C5/OpXi8jIyO4e0b+cjsjn7KB7gnn/snFQtcu6J1h2cQn0pgMAKaU+IKpiVDyvxdPH4U9dWeCdQ9TSnyu66ZqZTrI6OtLRETGRN3nIxAIoL29HePHj0daWhq2bNmCmTNnAgAOHTqEY8eOoaysLOqBuomZT9nDLu3doydFpBQP8IGF6QM5RaJXdPrWvr8iL7t7ZabpfGfw+z5vJr4zpghLNnyqmP7ZubDcNSkEK9NBbiuqJSJKNIaCj0WLFuGmm27CwIED0dLSglWrVuG9997D//zP/8Dr9WLWrFlYsGAB8vPzkZubix/+8IcoKytTLTaNV0Y/ZW+sqcfcVT3rByIFJOBXWw4LXVtk4gttxqV33/6vg475FcMxuCA7uG1VadxONRMzwsrGY24tqiUiShSG0i6nTp3CP/7jP2LEiBG48cYb8cEHH+B//ud/MGXKFADAsmXLcMstt2DmzJm4/vrr4fP58Pbbb9sy8FjSax4WWiehVT8QjdMt7WGpF7UtofIuiyKv9kQpX2nNB8dwy+j+mFCcjyUb7Kl7sGL7qhKrDtcz8voSEZFxhpqMOSFemoy9e+AEHlq1v8f35QlLnuxEm3KZIac/AOjujOkKSFi5qw5LNnyqe93Vs7tXquxoJubEmTBWHHgnF/0Cyqsoblr1ISJyAyPzN892MWFjTb3qJB7Zt8HOuoAGf5tq91Q5NfJISBolv3eG0HWNjDn0tnqTvlNnwqgdrmcE+3IQEdmHwYdBahOobPH08InJzroArSUr+WfLNv8l+L38HLHtvqdb2jHSJ7bqFFrXorWiEY/HyrMvBxGRPRh8GKBXv+EBsGTDQUwtvTiB6u3CcNKZ1k79GwFYsuFT+HIzkZedBv/5Tt3dI3orGsvvGod6/4W4PFbeilUUIiIKZ7jPRzIz0/9BqylXLOmN5WRzG5q+Djy0mokB0FzRkADMW71PqNYEsDdNZVehKxERGcOVDwPM9n9Qqx+Ipb456TjT2qH6cznoyMtOQ0avlLDD50LrHiprG3Ufk5E53q40lROFrkREJIbBhwHR9H8IrR/YdLABv9911OLRGbN4+iicae3QXJGQAJw934nX/2kiUjwexboHq1Yq7DwTxqlCVyIiEsPgw4Bou2impngwoTgfC96otmQ8oc209LqnRjrT2oEvzpwXuu3pc+2YMfYyxZ9ZtVIhAbjjmgGWXCtUPBa6JiIrtj8TUeJg8GGAFV009epGjPBp9PnQkuKBcA0GoB1gWFlQu2zzYaz54LilqRCrT7sl45jyIqJILDg1KNoumlakKf6xbBBWz56EnQvLMa20CNNKi7BzYTlWz56E5+8Yi/kVwwCoF5WK1mCIdPK0uqDW6Mm9enhOS2xZdUIzESUWrnyYEE3/ByvSFDeVFvX4lB65JXSEr0+PT5spHmOBByB2HopaQa2R+5NZnQrhOS3OiUytjB/UlykvIlLE4MMks/0fok1T5OekYfygvrq3iwyQTre0G0q1GO3kqRSQVX3eiOcFD8oLZWUqxMrTbkmdUmolPydNs7cMU15EyYtpF4dppSk8Kv8/1JnWTnzzF9uElqvlAGnG2MtQ0Eestbps8fRRhvPxofdXNqQfrrgkx9DvR7IiFSLyfIuedkvK1FIrok3tmPIiSj4MPmJAq27k5XvG4WWFn4Uyky83klbo7tT6adRNuKJNZViVCrHqtFvqyYpTm5nyIko+TLvESGiaoqG5DWfOtSM/Jx3erHRMKM5H+chCTFq6RbERmPxG//jaj3GhMwBfbs+aE6X8u2i6x6rlcLMpJjtSITynxR7R7N5iyosoeTH4iKHUFA/8Fzrw3MbPemxDvOOaAZodSIHuZe35f6gO/o5co6G2tfE7Y4rw2x11wuPbdeR0VBO01tZkNXamQnhOi/XMpkyY8iJKbh5Jklx1wEVzczO8Xi/8fj9yc8VOVo1Xap03jTYMk38HAB64vhi/3VGneE3552/u/VI3sJHp9WMQaR61saYeT/7pk7AW7Wbvz62StYlWZW0j7nylSvd2+RHt/OP1dSYidUbmbwYfMdIVkHDts1stPevFA8Cjsb1VXube+ui3MPnZLUIFgfL0qVQbYaR51K4jp3H373br3t/rsyZi8rAC3du5STI30ZL/jvV2E23/8Q3Y+8XZpAvOiJKJkfmbBacxYmWnU5kE7b4aci1H9fEm/MttV3UHKwLXBLr7MYQWoBptHnX6nP6qBwCcbhW7nVskexMt0d1E6b1SwnZCMfAgSm4MPmLESK7c6rfpUy1tqjtAlIQWoAL656UAPYOVRGz2ZeZ5SETcTURERrHgNEZEJ9n5FcOx5oNjlq6SFOR09/yQd4As23QIL2yr1f09OWAyc15KIjb74rkxF3E3EREZwZWPGJEnY7W3ZvlclXnlQ4Pntiz7v2OQn5Me/UpIyAVSUzyYPPQSoV+TAyYz56UkYrMvnhsTLrLJXDy9lkTkLAYfMWJkMpbf1G8bdzn+5bZSzd8REVl/IRoIyasSZlMoibY8n4ipJCIiJzD4iCEzk7HW78yvGC50v5GTodFVCaPBSuT4Q0/gDT2dN95E8zwQESUzbrV1AaM9IroCEqpqG1H5+WkA3asik67orikQ2fa4c2G54vWNbBmVd3kA4T1JtLbmJiI+D0RE3djnI4HpBQhqjcuA7glRbzI0EggZ7W+RqI24krnPBxGRjMFHAlCaqDcdbFANLADgplIfhlySg7rTrfhzTUNYz48UDzD7umL8ZNoo3QDASJAgett3D9Tjp+tqwrpc5uek47tj+2NKiS/uA5FEDayIiEQx+IhzSp+kfbmZaPuqC03nxY4pjyS3bM/LTgu7RuQndDs+xS999yB+o3OmDFcKiIjiG4MPh1n5qVcrbWIHeZSPVAyH/0IHfr/rqOptzNQvvHvgBB5atV/49t+fPDghVkKIiJINgw8HWblS0BWQMPmZLUIHsDlNr1hVSVdAwt89vQlnTazWcCWEiCi+8GwXh1h9rscLW4+4MvAAerZYF7Gn7oypwANw99koXQEJlbWNWFf9JSprGxO+fToRkdXYXt0kvXM9POg+12NKiU9opWBjTT2Wbf6L1cO0nGi3zq6AhF1H/mb6frSeQ700l53Fn9GsdLEolYioG4MPk6w810MOZOKBSLdOpQnaDKXnUGSrsV3bXtXqceRVGq2aGG7HJSK6iGkXk6w810MvkHGLvtlput061VJR0ZCfQ70019J3D9p2vH00J9hanZ4jIop3DD5MsvJcj3g5eCxyWo2sfej4KqA6QUfj0j6ZupO/BOCV/62z7Xh7IytdoaIJWoiIEhXTLiZZeUT80dOtlo/PDk3nO4MpEKU0Qn5OelgTsWiFPociq0Na87ccHKzcVYeCPhmGay7MrnRZmZ4jIkoUDD5Mkg9jm/PavmADL5mRI+K7C00P2zVMy51qaVOtfbA68AAuPoebDjZYct0lGz4N/n8jNRdmV7qsTM8RuR2LqkkUg48oyCfM9uhGKjCpdQUkVH3eiMf++2MnhmqZgpwM/OitjyxPrUQGcKHPYVdAwh+rT1h8j2KFojKzK11WpueI3IxF1WQEg48oTSstwpQSn6Fo36rdIE7Ly04DPLBl3BKAxdNHKaZE9tSdsXRVJfQ+RbdEm13psjI9R+RW0ewEo+TEglMLpKZ0H2s/Y+xlKBvSTzfwsHo3iFM8ADZblP5QUtAnQ/E5tDMlYaR52rTSIiy/axz65qSHfd/nzVR9c5WDFuBikCIzkp4jcisWVZMZXPlwkNZ/pPHg7PlOrHj/C9uuf7qlHeuqv0RBTgbgAU6fa8elfTK7/21AXlYami4Y66yqF+B0BSS8sPUwVuw6Gnbt/Jw0LJ4+SjHwkPPf7V8F8EjFMKzafQwnWy52sBVJz9kpmvw8c/skY1E1mcHgw0F29fPonZGKc+1dll/XSSme8GLQUN6sNOSkp6K1Q+wxLr9rHFJSPDjV0obTLe2q1w2lVXOxsaYej739seKJwmdbOzF31X68lOIJCyKUUmuRc3Msj1WKJj/P3D6FYlE1mcG0i4Ps+o+vsyte11Iu0lqR9V/oFA48+manYdKQfsE02H2Ti1HkzeyR8giVl52GQEDSbBCmFHgAysvKaqm1yMufbG6PSZOxaJqesWEaRWJRNZnB4MNBdv3H1/5VwJbrOsHqlfqz5zvx9Duf4NX//Rxr9/0Ve+rOYPF05ZoLWdP5Ttz96m5c++zWsMlTNE0WuqxsJLUWi3x4NPl55vZJiVxUrfbflwfdK2MsqqZQTLs4aEJxPvKy01Q/RScDX24mrh3aD9kZvSBJEv6r6pjl9xFZl1LkzcQD1xfjTx/Va6a96v1tePC1fZhfMQzzyocZTpOdamkz/DtO58Ojyc8zt09KrOp5RMmFwYeDNh1s0A08bhx5CfYfb8KZ1sQLUG4d7cMHR8/irX1fOnq/9f42/HZHHZbfdTW82emY+/o+zYLUZZsP4/e7juIbBifQS/tkmk6tOZUPF72fXUdO9ygmZW6f1ETT84iSE4MPh4ieXLvls+5j6PNz0tHW2YXzgrUOblbkzcR3xhThtzuUz15xgoTugtZ//fsxQjth/Bc68eca8W3FRSFt4M0QTclFu8tE9H5e2HYk+P/lYlLm9kmLmZ5HlLwYfDjE6HK8HU21YiE/Jw1bH/0Wyv/tvZhvMa73t6Hy89O2XFteVtZrKhbJSJMxK3aZGB0fcLGYdPldV7NhGmmSex4R6WHBqUOSdSn6TGsnVu3+wkVN1az9FJaXnYaXQxqMaTUVUxuJ6BlAVuwyMTI+mRxoLNnwqWrxLnP7RGQEgw+HJPNS9Bdnzsd6CEFlQ/rpbr0VkZedhvkVw7H3p1OCgUdXQEJlbePXTcWGozA3/DWPnJO1OqOGsnqXiZyf93nF/yblYtK+OemKvyv6WIiIAKZdHGNmuTtRDMrPjvUQAHSngE61tOOOawaYOkl47reGYLivj2IuWyklEhngXNI7HXdNHITBBTmG8uF27DKJzM8fPtmCF7bV6v7eqZY2zBh7GXP7RBQVBh8O0dqOlqjkGoBLc92x6nOmtRPz/1AN4OtD8iQJTRe+Ev79E00XcO2wSxQDD6VDtSL/faqlA7/afBgv3TNOOEjoCkjYdUSsTkWkRXxkwCCPo7K2USj4kFfw3JDbZ4t3ovjF4MNBatvREpEcYE29shBP/OmTWA+nB//XW55vGV2Edw6I1UusrT6BtdUnwoo8jTYVEz1FFzB++rFei3itYtUJxfnw5Wagobld8ffdVkzKFu9E8c0jxfKACQXNzc3wer3w+/3Izc2N9XBsEfqJ7ejpVizbfDgmqyHzK4Zj9Z5jaGi2PhCy6/F4PIBVf7HyhPp//24Ant9iLA3jAfDSPePgzUrHna9UGb7vxdNHoaBPhuondrXVFLWx+LyZ2LmwXDGgUbuWfMuX7hkHAKrn14Tezg0Tu8jjccM4iZKNkfmbKx8xELlkPcLXx/HVkCJvJuaVD8W88qF4YethUzUQkXpnpKIrIOFCZ8C2QEqSgMenjcSL22sNn1zb41rorpXQ+9Sv9rtPrT+In0wbaeq+Qw+7i/zEbmQ1RW+XiV6xqgfdQYf/fKfq/Xmz0/DM7Ve5YkIXeTyiK0tEFDvc7eIC00qLsHNhORZPH+XYfcqTVWqKBw9XDMfL94xDkYHdD0rOtXfhQqf958w0t3+FZ2ZeZdn1Kmsb8bNbrjT8e/X+Npw5Jx6wqIncLmukJ4zeLhORYtUmjcADALLSUjGlxCc0HrsZKb4lIvdi8OESqSkeoRNYo+UB8OJdV/eYrKIJgFI8QE56qkUj1Ff7txZMKfEhL8uahbsXth3Bkg0HccOISwz/bl5WGvJz0qO6/8jtsqI9YebdMAQ7F5ZrrkhY0V/GTZM5W7wTJQYGHy6i1QDK8/XX/IrheP6OsZhfMQy+3Iyw2+Rlp+nex/K7xuHm0f1V77+gT4biz7RkpaUIH3lvhT/XnMQLW48I7VRJSxUL5er9bdh26G+Gx9J0oRPfHav8fBoR+oldtCfM5KGX6KYWrOov45bJnC3eiRIDaz5cxsgBTfLJq6FbDTcdbFAsHMxOT8UPrr8CU0u1l8+NvGlnp6fgfEcArR32p1pCeQCseL9O6LadXfaW8R5tbMXgfjmWXe9USxtuGd3fsjbmVvWXcctkrvd43LYrh4iUcbeLS0X2MBg/qC/2fnFWqKdBV0BC1eeNeK3qC+z4y9/CViX0tiN2BSRc8/NNQqfq9s5Ixbn2+D/4zgopHkCrwWhuZi80t+mv1KyePQllQ/oFd3QAykeUG9nRoXUtCd0rZmoFp3o7aWLByueGiKxjZP5m2sWl5B0xM8ZeBv+FDnzzF9tw5ytVeHhNNe58pQrXPrtV9TyP1BQPWto6sbGmoUc6ROQskInFYs2jYh145GWl2VofY4ReZ/PvXTNAs57Hg+7AcPygviEt2oehMCK1ZqaNuVo7dZ83Ey/fMw7P3H5VcAyRYwLcd16L1uNh4EEUH7jy4XJmehp0BSRc++xW1V0Bap9mjTa1irX5FcPwq82H46JbbJE3E4unj8LcVfsBKH9if+D6Yvzpo/rwdFtuJr53zQB0BQIAugPSSVf0MxUMaHUEXfruQbzyv3VhQVSKB5h9XTEW3Vxi+L6cwA6nRO5iZP5m8OFiZoOIytpGocZX8hI/YKyplRukeIDPltyEzQcbMG/1ft2VBzeYXzFMsadLkTcT3xlThN/uqBN6/q3u5Kn12svN1LiaQER6mHZJEGZ7GhjdjmikqZVbBCRg7xdn0TcnIy4CDwBYtvkwPq1vxu3jLsNtY/tj7reG4PVZE7H9xzfgTx/VCz//IqkzUSKvvZETc+OdfDLxuuovUVnbmDSPm8hphna7LF26FG+//TY+++wzZGVl4Rvf+AaeffZZjBgxInibtrY2PProo1izZg3a29sxdepUvPjiiygsLLR88InObE8Do9sRjTS1chO3bP804vktR8L+/fqeY7j/G8WGnn+5k+eTf/oEfTLTcPpcu+m0gx0n5sYrnhdD5BxDKx/bt2/H3LlzUVVVhU2bNqGzsxPf/va30draGrzN/PnzsX79erz55pvYvn07Tpw4gdtvv93ygScDsz0N5O2IesWN8nbEeJzEge7HffR0q/4NXazpfCeWbf6L4d+TADQ0t+Pu3+0WKkJWw6Zd3eTUU2QgZuUqExFdZCj42LhxI+677z5ceeWVGDNmDFauXIljx45h7969AAC/349XX30Vv/zlL1FeXo7x48djxYoVeP/991FVZfzwrWRnNIiQ6TUrA8J3MLilh4Oo0J0hq/cci/VwXMPMRBlPTbvsSononRcDJFfqicgJUdV8+P1+AEB+fvfkt3fvXnR2dqKioiJ4m5EjR2LgwIGorKxUvEZ7ezuam5vDvqib0SAilJHtiHpBjpuEPu69X5w1dBhcopNw8cA70YnSTIAbi7qIjTX1uPbZrcLbzY3geTFEzjPd4TQQCOCRRx7B5MmTUVpaCgBoaGhAeno68vLywm5bWFiIhoYGxessXboUTz31lNlhJDwjHU+VfndKiU93O6Ic5Mx5bV+w8ZRbhT7uddVfxno4lrPi+a/3t+GFrYfxcMVw3dtqvfZKAW4s6iLUduPIKz3R7sZh6onIeaaDj7lz56KmpgY7d+6MagCLFi3CggULgv9ubm7GgAEDorpmohENIpTIzcpE7kMpyJHlZaeh46sAzjt4hkvk/S+/cxwmDekXt+kiPfMrhmHNB8ctKf5dtvkwRvj6CE3KogGu3UGAEr2UiAfdKz1TSnyme3yYTT2xzwiReaaCj3nz5uGdd97Bjh07cPnllwe/7/P50NHRgaamprDVj5MnT8LnUz5TJCMjAxkZxg8zSzaiQUQ0ppUWIRAAHlq1r8fP1NpvO+WawX0xeVhB2PesOrfEDfJz0jDnW0Mx7NLeeOjrRmTRMjIpRwa4BTkZgAc4fa4dlbWNGD+or+1BgBLRlEjV542YPLRA9XZazJwXw50xRNExVPMhSRLmzZuHtWvXYuvWrSguLg77+fjx45GWloYtW7YEv3fo0CEcO3YMZWVl1oyYbNMVkLBkw0HFn8kTTKw+2G06eArvHgjP72vVxMSbM62duP65bXj8jzWWXTO0TkGkTkMOcDN6peBHb30UtpNm0tItMamLEE11zH3d/I4Uo7VV3BlDFD1DKx9z587FqlWrsG7dOvTp0ydYx+H1epGVlQWv14tZs2ZhwYIFyM/PR25uLn74wx+irKwMkyZNsuUBkHVEPmXK/XBjURuyeF0NppaGf7LWSxe5SXZ6qmbaqqHZ+vGfamnDuwfq8dN1NTjT2hH8fne79xL0zUnvcSqyUmol9Hf17s9KoimRpgudUaV+RFNPTqSBiJKBoeDjpZdeAgB861vfCvv+ihUrcN999wEAli1bhpSUFMycOTOsyRi5n+jEMWvyYLxb0xBVi3AzGls7FJtdySmDZZv+ghe2HVH57djpnZGKf/0/YzClxIdfbT6EF7bWOha4bTp4Eu8c6PlJvN7f1iO95svNQNtXgajGZnUdjtHUWjQTv0htFZuyEVnDUPAhcgxMZmYmli9fjuXLl5seFMWG6MRRUeLD49NLFN+krx7YF4+9/TGaznfaMsZdR06r7tiZPLTAdcFHfnYaqh6vQHqvFGysqce/b6115H49ALxZvRQDDzXRbFtWqouwQuhuHD1WTPx6tVXcGUNkDZ7tQkETivORl52m+vPQng/ym/SMsZehLGQHyrTSIuz96RTcMtqeorsXth1R7e/gpn4lnq+//uX2q5DeKyW4XO8UCXBsZ5Jez5loySmRvCz1v81Qdk788dSUjcjNGHxQ0KaDDZorFhLEJ5i9X5y1cGTh6lUK++RPyW7Y+eLNSsMjFcMwpaR7l1cszs/p6HLmmVBqXGe1aaVFWH7XOKHb2jnxm+06TEThGHwQAAh9Ms9JT0WfzDTdjpZOTLShnTxDd3Icajhn6/3qyUlPBdBdALls8+HgKo2RT+M5Ganw5cbH9vP5FcOxc2G5I9tLJw3pF/OJP5quw0R0EYMPAiAWMLR2dOHu3+3WbWvtVL5b7uQZ2nZb9JA2OS3yg+uL4cuN/pOynK5qjUh1yNsvj54+L3wtDzyYeqVyXxw38QBY84FzZ+u4ZeI3cnQBESnzSCJVpA5qbm6G1+uF3+9Hbm5urIeTNNZVf4mH11QL3VZ+a1d7o62sbcSdr7j7IMHQhlC7Dp/G3a/uNnyNrLQU/Py7V6EwNxOPvvmR6lZZuRhTkiTXn0XjAeDNTkNmr1Thrb+rZ09ydGeHWxp8scMp8W8gnJH523R7dUosRvLkev0M3N55dN4NQzF/yvDguE+3mgsI/u3/jMHNo/ujsrZRc6KWd2HMrxiGZZsPm7ovJz1z+1Vfb10+hBe26e/OcXpnRzTHDVjJia7DscAJVYxbguB4xeCDABgPGLS2NRo9qC43sxea274yOXLjJg8tCL6ZdgUknG4xHnz84Ppi3Dy6PwDxyXdgvxz8/bjL8NY+dxyI5/FcbBoHdHevnX1dcfCNMy01Veg6sdjZkagTf6xxQhUTi3OOEg2DDwJg/mRbtYlXrWOkUmfNBv8FzH/jo+gfhI7IXhRKb7R6+uWkY8mMUtwcspVYdPL95/Wf4KyF/U8WTx+F/Jx0LNnwKc62dhheZYpMuEoS8Nsddbh6YF8AwK8E6mf6ZqdxZ0eC4IQqhl1urcHgg4LMtCrXmnhFl8craxsNj/WW0UWGGmhFFiSqvdGqubm0EHdPHBx2qq5MdNXIqsBDDqLum1yM1BQPstJT8aBAEy49oW+ckiQJr4BR/OOEKo5dbq3B3S4UZlppEXYuLMfrsyZqNnUS3dao1owslNHmYLOvK8bzd1xt6HdCdyJovdGqebfmJH701kfYdLChx8+cPOBOaVfHtNIizK8YZsn15TdO0cLYpvOdlh8mp0XkgLx4Y/Qx2fEcGJlQkx273FqDKx/UQ2qKB5OHFeCZmVcF21qHvr1Zva3RSAttD4B3DtTjsZtGqaaJ5H/PrxiGwQU5PVZczPYh0Vp+duqAu8iDzmSDC3Jsu089am+yVhcuJmI9gtHHZNdzwAlVHLvcWoMrH6TKyX4G8n3l52i30A79BKY1vpfvGYeHK4YrrriYfQOVAxy5uZnSY9i5sByv/5P2qpER8qi/P3kwVs+epNrQK5ZvdEr3vbGmPqz/yp2vVOn2h9GSiMfYG31Mdj4HnFDFscutNbjyQZqc3NY4rbQIFzoDmP+Hat3bygGEmfEZafgVSS+fm5riQYrHg6YL1tR3qK10RJLfEKNddQntSXKyuV03NaX0Jmt14WIi1iMYfUx2Pwd6dUt2HRwYj7SK89nlVhxXPkiXSN2GVUS7jYZ+AjMyvq6AhNV7ou/KqbV6YtXS9OLpo4Rbl4fWnYjS6hL65HeuFLrGHdcMCPu33iQJqK8cqUnEegSjj8nu58At3WPjBbvcRo/BB7mK3Uuae+rOCHfu1KK1/BzNyoosxQP8Q9lgQ2/23YWnw4VuO79iuOYbp/zmWuTVDgaXbT6Ma36+Ge9+vfPIjkkyEesRjD4mJ54DTqjGyGnW1bMn4fk7xmqmRaknpl0oKlYXFdq9pBntBKW3/GzVykpA6j4ZWGSrXuhr8HeD+6KwTzpOtnQo3lYe/7zyoZhXPlTztQtNaTU0t2HX4b8pNkg709qBh1btww/+WoyS/l6hx2fkdUjEegSjj8mp58CONGsid0xlszvzGHyQaXZV3qvtHBGtf9ASzZuzSPBj1coKIDZBK70G8iF3IsGb3hun/ObaFZDw3MbPNG/7mx11ePjGobpjBoCC3hmorG0UmpASsR7B6GNy8jmwckJNxB1KZA0GH2SK3d0QzXwCE/mEFc25MyLBj5VL/3qBktpr4P+6mZk3Ow1NIY3N8nPSMWNsf3iz0tEVkAx9+hTdnvyflV/Al5uJk83qk2RedhoefaM6rJeI1oSUiAV+Rh9TPD4H7JhKWniqLRnWFZBw7bNbVScj+VPYzoXljr0Z6n3CCg1Mjp5uxbLNhzXbyCu1gRdZLrbqRN+8rDTsXTxF9f5EX4N//fsx2PrZSayt/hJnWkMDkTQ8PaM0eD6NHiOnHs+vGB5sza7Uf0VtvID6SclAYn6KdkufD6u58T2C7MdTbclWbmsvrPcJ64Hri/Gnj+oVUxNNKi3PJUlCSop+WiKSVSf63j9Zu9hU9DX48Isz+P2uoz3Gcqa1Ew+t2o8f/LUJi27W3yVjJF01uCBbNW12obNL8TkX2S7qltNsrWT0McXLc+C29whyHwYfZJibdh+IbO38zY66Hj+TUxNqZ8ScbG43tTQsL49Hc9ZK3+w0zCvXbpcu+tyuUAg8Qv1mRx3GXN437KA8JROK85Gfk44zrcqFrKEu7ZOJsiH9ekySgYCEu1/drfp7IhNSIhb4GX1M8fAcuOk9gtyJW23JMDftPjDbKl2ekN/9WLkrpNmeFAAwpcQXXFkxygNg6e1X6X6SFX1uRZqdLV5Xo/sYU1M8eHpGqe61QrdBR/ZfOd0qdl4MJyRtRs52idVZOG56jyB34soHGeam3QfRTFQSeh4rH/lzM0vDe+rOqKZztHg8wAPXFQuttJxtbUeKp3tLbrQaWzuEHuPNo4vwg78WK64kAd2vu1bRIyek6Bmp+YhlfYib3iPInbjyQYa5qRuiExOV0QDH9NkxEvDbHXW6Z3RsrKnH3FX7LQk8ZJsVTutVsujmErx419U9zuDJz0nD8ruu1pzUrG4gl4gn3GoxcrZLrM/CcdN7BLkTgw8yxS3dEPUmNCsYDXCiDYi0Uj1aNS7ReGPvcazd3z2Jd3wV0JzUp5YW4R8nDUbvjNTg9860dmLJhk81JzUrJySrD65zOyNt6+1ocW+GW94jyJ241Zai4obuhfKnPEB9K6caj0c99WJ2O6C8zTCaHS+rZ09STINYtZVXS2Q6J3SpfmNNPR57+2PFtJLIdlkg+nSA2u4m0fuPR6Kv++rZkwBA+LZOFK664T2CnMGttuQYqyrvo3mDUuuIWuTNxHfGFKnWKAAXAw/Rxk0i49RqCCVKLXXjRDFm5Afi0C3Lms8lxE5XVdsuCkC362kinnArwo7dI04V9sbD7hxyHoMPijkrCuO0JrQ/fPhX1QJQD7o7gWb2Sg1ri67UzdTIONUCovyctLBmX2rUUjexKMaUJ/VX/lc98Ai9rUiRbuSEJPrcJmv/CDuKdVnYS7HE4INiysoWzEqfsCprGzV3nkjobjT2+qxxSEnxqH7qNjNOpYBo/KC++OYvtpneBTChOB95EW3TnaC3MyiSkU/VRp7bZO0fYXT3CHeakNux4JRixonCONFJ6HRre1hPishUi9lxRva6SO+VElXR5aaDDY4HHmaIfqo2+twm63ZdI8W6sdxpkmw7kMg8rnxQzDixhB7NZCXXd+w68jdLx2n21F55otaiVUDrFK3tspE1M4GAZOi5Teb+EUb+buw8GVqNFenTeCxOjccxuwGDD4oZJ5bQzU5WSm+kVo7TzBkdIt1cJQlYPH0Uzp7vwAvbaoXHYyW1T9VKz2lellgnWPm5jcfTXa1k5O/GyXNgrEifxsuheaHiccxuwbQLxYwTS+hmlqDVGjRZPc7IlIzSpBC6jL3ryGmh6xb0ycD8KSNQ5HU+9TBr8mDV01iVnlOR9u9A+HOb7P0jRP5uzNzWLCvSp7FuimZGPI7ZTbjyQTHj1BK6kSVoM0287FrqN7P6AnRP1FYccGdGRYmvx/c6vgrg8bUfm9pyrPbcuul012Rfdo82fRqP26fjccxuw+CDYsbJJXTRycroQXV2LfWrLWPrjscDfPVVAF0BCdNKi/DiXeMwb/U+S1uxK94v1NNXj6+tEdperHRNQP25tbN/hGhAwWX36NOn8bh9Oh7H7DYMPiimnCyME5msjNaXWDHOyIlu/KC+pluoSxLwDyv2BCfAm0cX4QVcjYdW7Tc9Pj166SvRx5GXlRaWhrGzOFKLaEBh5TZxp1m5WhNt+jQet0+LjmXXkb+5biXMLSt1DD4o5ty0hC76RjrvhqGYPLQg6nEqTXSijci0hE6AN4/uj5dTPKZSOCKsSl8tv3scUjzKvVacesMUDSjiednd6tWaaNOn8bh9WnQsL2yrxX/v+9I1K2FuWqlj8EGu4JYWzKJvpPOnDI96UlGb6KINPICLKazH136MC50B+HIzsf3HN2DvF2fR4L+AJRs+xdnWjqgPqFs8fRTum1wcdfoqPycNp1ra4cvNxC2j+/dYQXHiDdNIQBGvy+52rNZEmz6Nx+3TemMO5ZaVMLet1HG3C1EIpxo02XU6baQzrZ2Y/4fuU1+/+Ytt8F/owHfGXoZ7ywZFdd8edAcASoEHYHyJPHScoafTOrmjwEhA4bZUgUhzLzub+kWzAymWTdHM0hpzJCdPElbjlpOOQ3HlgyiCE3UoRlcGrNDgb8ODr+2zrD271oQQzRK5HFgsv2sclmzQfsN89I2PkJPeC98YWhD15GQkoHBTqsAt5+JEkz6NRVO0aKmNWUmsV8LcuFLH4INIgd11KGY/EcvL2r0zeuFc+1eGfleesKMNPHIzU/Hc34/RnBBElqXVurHKKY7F62rQ2NqhOZbWji78w+/3IC87Dc/cflVUk5RooHD45DkU9M5AXlYvNF1Qfg2cShW47VycaNKnbqr9EiWPedmmv+CFbUd0bx+rolm3rdQBTLsQqbKzQZPoRJefkx7277zsNORlpxkOPKyUmqL9tiEXh95c6lPN4QPabeAlQDfwCNV0vhMPRpmGGTsgT+h2L2w7grt/t1s18AC6x293qsDoUvrR061C141lYacTTdGslpriweShBUK3jdVz66aVOhlXPohiQLTITi4SPdXShqOnW7Fs82Gnh9rD2fOdqgVqSimAFA/C+oz4vJm4qdSH3+86avnYnvzTJ4Z3mMjB0hsfHLNsHH2z0zBFoeGalYwspU8ozsfqPfqPT+tcHlLn9qJZN46PKx9EMSBaZJfeKwVlQ/rhltH9seaD446OUY/8qVoudlyy/hM8qFAcKq9wfH/yYKyePQk7F5YLT8z5OWm6BX2hGprbsafujO7t5DH/8/pPcM3PN+POV6qwtvqEgXvSdvZ8p9A4omFkKX1P3Rk0NLfr3vaOawbGxWqD27i9aNaN42PwQRQjRnYIiBaoZvTyoHeG/Qua8qfqF7YewbXPbsWdr1ThVZWVDLmG4881DcEcvvxJTO2tTt5N8/SMUsNj05uUN9bUB8f8+11HccZAesfKcUTLyFK66FgGF2RHM6Sk5vYzh9w2PqZdiGJItMhOdPJo/0pC+1fG60HkZdfF00uwZIN4M7Jlm/8idLvIanrR3hDTSovwUooHj6/9WLj/idakbLZtvRl258+NLKWLrsK4qZFXPHJ70aybxsfggyjGRHYI2DkpRE72U0t9WLmrDks2fGr5fYUGUaLbK6eVFqF8ZCEmLd2iu0rhy81QzVs71VtFnvQDAQnrqr+07Q3eSHMvN+b8E5VbGiaqccv4GHwQxQEjHRWNipzsU1M8uG9yMX63s87y+yvonRH2bzmw+K/Ko/jizHkMys/GP5QNRnqv8Ixweq8U/Mttpbqn9D75nStVJ3kneqvIQcCFzi7c/eru4PftamEtGsA5eYgjkQiPJGlteHNec3MzvF4v/H4/cnNzYz0cItcwfdItoPg7syYPRkWJT/PE1jlfT/aRk5XZN43XZ03E5GEXtyUabZ2+saYej739cY9eJSJ9PtZVf4mH11QbGu+3SwqRm9kLb+37Uuhx981Ow1mNPip6z7lZPIWX3MDI/M3ggxS55eRDCtd9RL14/cOsyYPxbk2D6clGbbK645oBprb9Pn/HWMwYe1nw2krBlPxXplYE1xWQUPV5IyprGwFIKLuiAJMEekJU1jbizleqhMYppyH+9e/H4HRrO46ebsXqPcfCdowUeTOxePoo9M3JwKmWNhT0zsCjb1QL7SqJ5YTP/7bJLgw+KCr8dOQso5NBx1cBjH96E1ra9AtLV8+ehAnF+aiqbUTl56cBdOd7J10h3sBJaXwAcO2zWw2nZVbPnoSyIf3QFZBw7bNbVdMg8uS/c2G5ZROjfJ96Y5ZXOCLb0PtyM3HnhIEYXJCNS/tkYvygvsEeLJf26a7xCE21aNELsOIJgxmSMfgg08x+GiVzzAZ67x44gYdW7de8dtHXk/emgw22BJNqaRklkcGE6CqEHKxYRWTMaqmT0P8GAPR4TvOy0tB0Qbx1vR0BltP4QYVCGZm/2eeDgtx48mEii+bE1ptH98cPri9W/bkH3QWEmw422HYqrFrfAKWxAOEFjaJbhxua23RPazVCbcz5OWmYNXkwXv+nicjopfy2KN/zY29/rPicGgk85OvJ24/jkZMnDlPi4W4XCnLjyYeJSi/Q86A70NNqFb7o5hKMubwvfrquJmwLqvzJc0qJD9c+uzWq+9AT2TdAqTZC6WRS0a3DS975JKy+xYpP1Vq9DiprGzVrNiREfzBfpFgdNhYNK/5+Kbkx+KAgN558mKisCvRuHt3dl0NtInUimIzsGzCvfJhuDYDo1uHIwlql01qtGLMsFn/b8djYix9UKFoMPijIjScfJiorA71oJ1KrJ1yRJkZ6fSfUAhK7P1Vb9bctUv8Rz429+EGFosWaDwoSPW8jHt8sY0k+xCy0bsGJQM/twaR6/UW65u/Jn6qrPm9UfG6jofffgKjld4/D6tmT8P3JgwG45zAvq7j9b4vcjysfFMQuiNZT2w2wePoo29tdx0NLbaX6iwb/Bcx/4yPd3539nx8ivVdKWA1GtDUhof8NmCE/p/JW5rIh/TChOF+3A2m8iYe/LXI3brWlHrh9zhp625YfuL4Yv91RB0A50LNiW7NWl1Kr7sNqRpqBRbLqcW2sqcdj//2x4a2zavediL0w4vFvi+zFPh8UtUR8s3SSaBMtpVNkrQ704i2YFG0GpkZ+brf/+IawJmBG/4Z3HT4t3DQMcPdzKsrof/fx9rdF9mLwQRRjRppoyUee2xnodXwV0D28zU3MnmMTKj8nXXELsuikqBdAht9XGqoWVTj+nFr5IcFsIMEPKiRj8EEUY6KHmIWedWKXeP10aib1ocVMOmBjTb3uSboyq7ux6rHydWVnY7ICO5wSxZhVuwGi3c0Rz10op5UWYfld4yy7npkuvdNKizDr6x0repzcVmrl6xrrzsZW71ii+MDdLkQ2sGI3QLSfbBOhC+WkIf2EmpGJMtP8qqLEh1d3HdW9nVPbSq1+XWPZMCxeV+UoeoZXPnbs2IFbb70V/fv3h8fjwR//+Mewn0uShJ/97GcoKipCVlYWKioqcPiw8aO3ieKZvGUTMNfjwYpPtkYmFbfSeh6jYWSVwm39b6x+XWPVMCyeV+UoeoaDj9bWVowZMwbLly9X/Plzzz2HX//613j55Zexe/du5OTkYOrUqWhrY6c7Si5qTbR83kzNHLpVy+CJ0oVS7XnMy05DXnaaqWsaWaWINpA0QiQFYfXrGouGYbFO9VDsGU673HTTTbjpppsUfyZJEn71q1/hpz/9KWbMmAEA+M///E8UFhbij3/8I+64447oRksUZ7QOMVNj1TK42UnFjbsX1J7HqtpGQ9thzTa/kgMgO5uFiaYgrA4WYtEwjGfDkKU1H3V1dWhoaEBFRUXwe16vFxMnTkRlZaVi8NHe3o729ounSDY3N1s5JKKYEznrJJRVn2zNTCpuzsErPY+nW9VPoI0U7SqFUgA0flBf7P3iLNZVfxlVoKa220TpID2rg4VYdDZOlFU5Ms/S3S4NDQ0AgMLCwrDvFxYWBn8WaenSpfB6vcGvAQMGWDkkorhj1Sdbo+mCeMzBG0kF6KW7RMgB0Iyxl8F/oQPf/MU23PlKFR5eU407X6nCtc9uNfw8GU1B2JEGMpsiNItnw1DMt9ouWrQIfr8/+HX8+PFYD4kopqwscBSdVOI1By9yEFxeVhpe/6eJ2Lmw3NKusVYFamYKSO0IFqaVFmHnwnKsnj0Jz98xFqtnT7L0OQvltiJecp6laRefzwcAOHnyJIqKLv7Bnjx5EmPHjlX8nYyMDGRkZFg5DKK4ZvUyuEjdSbzm4EWeq2dmXoXJQwssu0+rt7qaTUGYqSfSYzRFGM398BDL5GbpykdxcTF8Ph+2bNkS/F5zczN2796NsrIyK++KKKFZ/ck2NF1QNqRfjzf1eM7BO50ysHqrazQpCL3X1c2cft3IXQyvfJw7dw5HjhwJ/ruurg7V1dXIz8/HwIED8cgjj+Dpp5/GsGHDUFxcjMWLF6N///747ne/a+W4iRKeHZ9s1bgpB29mt42Tz5XVgVoyH08/rbQI5SML4+rcIbKG4eDjww8/xA033BD894IFCwAA9957L1auXImf/OQnaG1txQMPPICmpiZce+212LhxIzIzWThEZJRTy+BumQCj2W3j1HNldaBmNAXhxq3QZim93r/bWeeK3VVkLx4sR0QALhZRAsoToBVL4VoTZ7wcbiafdqsXqO1cWG4oKBAJvMwEZ24NVuLl9SZxPNWWiEyxs8+H1rWnlPg0j683O6FriWZStitQszo4c2vfFjmAc/L1toJbAzm3YPBBRKbZ8Qb77oETeGjV/h7fl6/6SMUwLNusfwaUVcfWWzEpq13jjmsGYnBBtqWTk5nJ2s0rC5W1jbjzlSrd21n1elvBrYGcmxiZv3mqLRGFsbp24t0D9Zi3umfgAVzcmrpC4NRYwJrdNka6iWqJLHI9eroVq/ccw7LNfwnexqrJyehW6FifaKwXwMbb7iqr/mboIpYUE5FtNtbU46FV+6DVm0wC0HShU+h60e62sbqZmhyoZfRKwa82H0ZDc3i7d6u6wxqdrGN5ovHGmnpc++xWzc6vbtpdpSdeG/C5HYMPIrKF/KYtKi8rzfaOl3ZMyk5MTkYn61itLIh2fo2nDqexDOQSGYMPIrKF3pt2pPsnFwOw99h6OyZlJyYno5N1LFYWjARhdpxPY5d4SxHFCwYfRGQLI2/GRd5MzCsfanvHSzsmZScmJ6OTdSxWFowGYfHS4TSeUkTxhAWnRGQLI2/G8sRpd6dSO5qpWT05qRVrypN15I4Ln0JRayzOTjEThDnZmdYstzTgSzQMPojIFnpv2gCQ4gFeuPPqHhOnXdsr7ZiUrZyc9LZzGpmsjQQrVjAbhFnxetvZf4OH4NmDfT6IyDZqzbhkL941DlNLnf/ka3XPBiuajtnVl8OOiVnpmgCi7vxqZqxO9d9gnw99bDJGRK6h9aYNIGZv6FZPytFMTvHU8VPv9TQbhJl5/pxupMYOp9oYfBCRqyi9aW862ODaDpwi1D79m5mc4qXjp8hkDxgPKM0EEfEUsNnFbcEQO5wSkatE5vVj3YEzWlYvwcfDdk7R12znwnJDRaRm/xaMdn1NNPGeBuJWWyJyXDw3bhJtpGVEPGznNPKaycHmjLGXoWxIP80A0uzfQjwEbHax42/QaQw+iMhx8Tpx2NXNNB46ftr1mpm9bjwEbHZIlHbvDD6IyHHxOnHYtWITDx0/7XrNzF43HgI2O8TzqmEoBh9E5Lh4nTjsXLFxe8dPs69ZV0BCZW0j1lV/icraxh6fyM1eNx4CNjvE66phJBacEpHj4rVxk90rNm7u+GnmNRMpiozmb8HpRmpuEK+rhpG41ZaIYibeKvbl7Z3RNNKKd6KvmdHts9H2SXFjwGYHN/8Nss8HEcWNeJs4rOhmGu/0XjOzPTji7W8hVtz6N8jgg4jIRvG2YuO0eGmaFs/c+DfIJmNERDZyU22GG1cLEqUo0s3c9DdoBoMPIiIT7Dx9V5QbP/0CiVMU6XZu+Bs0i1ttiYjikJu7XMbrVmpyDoMPIqI44/Yul8nag4PEMfggoqSn1wjLbeKhy6Vo07R4e+7JGqz5IKKk5ta6CS3xUtCpVxQZj889WYMrH0SUtNxcN6Elngo61U64jdfnnqzB4IOIkpLb6ya0xHtBZzw/92QNBh9ElJTioW5CTbwXdMbzc0/WYPBBREkpXuom1Lj9FFwt8f7cU/RYcEpESSme6ibUxGuXy0R47ik6DD6IKCnJdRN6p4O6tW5CFo9dLhPluSfzmHYhoqQU73UT8YzPPTH4IKKkFc91E/GOz31y80iS5Kq9TEaO5CUisoIbT4ZNFnzuE4eR+Zs1H0SU9OKxbiJR8LlPTky7EBERkaMYfBAREZGjGHwQERGRoxh8EBERkaMYfBAREZGjGHwQERGRoxh8EBERkaMYfBAREZGjGHwQERGRo1zX4VTu9t7c3BzjkRAREZEoed4WObXFdcFHS0sLAGDAgAExHgkREREZ1dLSAq/Xq3kb1x0sFwgEcOjQIZSUlOD48eNJd7hcc3MzBgwYkHSPnY+bjzsZJOvjBpL3sSfT45YkCS0tLejfvz9SUrSrOly38pGSkoLLLrsMAJCbm5vwL5aaZH3sfNzJhY87+STrY0+Wx6234iFjwSkRERE5isEHEREROcqVwUdGRgaeeOIJZGRkxHoojkvWx87HzcedDJL1cQPJ+9iT9XHrcV3BKRERESU2V658EBERUeJi8EFERESOYvBBREREjmLwQURERI5yZfCxfPlyDB48GJmZmZg4cSL27NkT6yHZ6sknn4TH4wn7GjlyZKyHZYsdO3bg1ltvRf/+/eHxePDHP/4x7OeSJOFnP/sZioqKkJWVhYqKChw+fDg2g7WQ3uO+7777evwNTJs2LTaDtcjSpUtxzTXXoE+fPrj00kvx3e9+F4cOHQq7TVtbG+bOnYt+/fqhd+/emDlzJk6ePBmjEVtH5LF/61vf6vGaP/jggzEasTVeeukljB49OthQq6ysDH/+85+DP0/U11vvcSfiax0t1wUff/jDH7BgwQI88cQT2LdvH8aMGYOpU6fi1KlTsR6ara688krU19cHv3bu3BnrIdmitbUVY8aMwfLlyxV//txzz+HXv/41Xn75ZezevRs5OTmYOnUq2traHB6ptfQeNwBMmzYt7G9g9erVDo7Qetu3b8fcuXNRVVWFTZs2obOzE9/+9rfR2toavM38+fOxfv16vPnmm9i+fTtOnDiB22+/PYajtobIYweA2bNnh73mzz33XIxGbI3LL78czzzzDPbu3YsPP/wQ5eXlmDFjBj755BMAift66z1uIPFe66hJLjNhwgRp7ty5wX93dXVJ/fv3l5YuXRrDUdnriSeekMaMGRPrYTgOgLR27drgvwOBgOTz+aRf/OIXwe81NTVJGRkZ0urVq2MwQntEPm5JkqR7771XmjFjRkzG45RTp05JAKTt27dLktT92qalpUlvvvlm8DaffvqpBECqrKyM1TBtEfnYJUmSvvnNb0oPP/xw7AblkL59+0q/+93vkur1lqSLj1uSkue1NsJVKx8dHR3Yu3cvKioqgt9LSUlBRUUFKisrYzgy+x0+fBj9+/fHFVdcgbvvvhvHjh2L9ZAcV1dXh4aGhrDX3+v1YuLEiQn/+gPAe++9h0svvRQjRozAnDlz0NjYGOshWcrv9wMA8vPzAQB79+5FZ2dn2Os9cuRIDBw4MOFe78jHLnv99ddRUFCA0tJSLFq0COfPn4/F8GzR1dWFNWvWoLW1FWVlZUnzekc+blkiv9ZmuOpgudOnT6OrqwuFhYVh3y8sLMRnn30Wo1HZb+LEiVi5ciVGjBiB+vp6PPXUU7juuutQU1ODPn36xHp4jmloaAAAxddf/lmimjZtGm6//XYUFxejtrYWjz/+OG666SZUVlYiNTU11sOLWiAQwCOPPILJkyejtLQUQPfrnZ6ejry8vLDbJtrrrfTYAeCuu+7CoEGD0L9/fxw4cAALFy7EoUOH8Pbbb8dwtNH7+OOPUVZWhra2NvTu3Rtr165FSUkJqqurE/r1VnvcQOK+1tFwVfCRrG666abg/x89ejQmTpyIQYMG4Y033sCsWbNiODJyyh133BH8/1dddRVGjx6NIUOG4L333sONN94Yw5FZY+7cuaipqUnYWiYtao/9gQceCP7/q666CkVFRbjxxhtRW1uLIUOGOD1My4wYMQLV1dXw+/146623cO+992L79u2xHpbt1B53SUlJwr7W0XBV2qWgoACpqak9qp9PnjwJn88Xo1E5Ly8vD8OHD8eRI0diPRRHya9xsr/+AHDFFVegoKAgIf4G5s2bh3feeQfbtm3D5ZdfHvy+z+dDR0cHmpqawm6fSK+32mNXMnHiRACI+9c8PT0dQ4cOxfjx47F06VKMGTMGzz//fMK/3mqPW0mivNbRcFXwkZ6ejvHjx2PLli3B7wUCAWzZsiUsd5bozp07h9raWhQVFcV6KI4qLi6Gz+cLe/2bm5uxe/fupHr9AeCvf/0rGhsb4/pvQJIkzJs3D2vXrsXWrVtRXFwc9vPx48cjLS0t7PU+dOgQjh07Fvevt95jV1JdXQ0Acf2aKwkEAmhvb0/o11uJ/LiVJOprbUisK14jrVmzRsrIyJBWrlwpHTx4UHrggQekvLw8qaGhIdZDs82jjz4qvffee1JdXZ20a9cuqaKiQiooKJBOnToV66FZrqWlRdq/f7+0f/9+CYD0y1/+Utq/f7/0xRdfSJIkSc8884yUl5cnrVu3Tjpw4IA0Y8YMqbi4WLpw4UKMRx4drcfd0tIi/ehHP5IqKyuluro6afPmzdK4ceOkYcOGSW1tbbEeumlz5syRvF6v9N5770n19fXBr/Pnzwdv8+CDD0oDBw6Utm7dKn344YdSWVmZVFZWFsNRW0PvsR85ckT653/+Z+nDDz+U6urqpHXr1klXXHGFdP3118d45NF57LHHpO3bt0t1dXXSgQMHpMcee0zyeDzS//t//0+SpMR9vbUed6K+1tFyXfAhSZL07//+79LAgQOl9PR0acKECVJVVVWsh2Sr733ve1JRUZGUnp4uXXbZZdL3vvc96ciRI7Eeli22bdsmAejxde+990qS1L3ddvHixVJhYaGUkZEh3XjjjdKhQ4diO2gLaD3u8+fPS9/+9relSy65REpLS5MGDRokzZ49O+4DbqXHC0BasWJF8DYXLlyQHnroIalv375Sdna2dNttt0n19fWxG7RF9B77sWPHpOuvv17Kz8+XMjIypKFDh0o//vGPJb/fH9uBR+n73/++NGjQICk9PV265JJLpBtvvDEYeEhS4r7eWo87UV/raHkkSZKcW2chIiKiZOeqmg8iIiJKfAw+iIiIyFEMPoiIiMhRDD6IiIjIUQw+iIiIyFEMPoiIiMhRDD6IiIjIUQw+iIiIyFEMPoiIiMhRDD6IiIjIUQw+iIiIyFEMPoiIiMhR/x96ouMhYG7pmwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "data_url = \"http://lib.stat.cmu.edu/datasets/boston\"\n",
    "raw_df = pd.read_csv(data_url, sep=\"\\s+\", skiprows=22, header=None)\n",
    "data = np.hstack([raw_df.values[::2, :], raw_df.values[1::2, :2]])\n",
    "target = raw_df.values[1::2, 2]\n",
    "\n",
    "plt.scatter(data[:, -1], target);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Нам надо обучить линейную регрессию. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "w = torch.randn(1, requires_grad=True)\n",
    "b = torch.randn(1, requires_grad=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([-1.1203], requires_grad=True), tensor([1.1006], requires_grad=True))"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w, b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.tensor(data[:, -1] / data[:, -1].max(), dtype=torch.float32)\n",
    "y = torch.tensor(target, dtype=torch.float32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Наша модель\n",
    "def linear_regression(x):\n",
    "    return w*x + b\n",
    "\n",
    "# Ошибка для модели\n",
    "def mean_square(y_pred, y_true):\n",
    "    return torch.mean((y_pred-y_true)**2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([506])"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pred = linear_regression(x)\n",
    "pred.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(557.0925, grad_fn=<MeanBackward0>)"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loss = mean_square(pred, y)\n",
    "loss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "# считаем градиенты\n",
    "loss.backward()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dL/dw = \n",
      " tensor([-12.0651])\n",
      "dL/db = \n",
      " tensor([-43.6111])\n"
     ]
    }
   ],
   "source": [
    "print(\"dL/dw = \\n\", w.grad)\n",
    "print(\"dL/db = \\n\", b.grad)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Обучим нашу линейную регрессию."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/P9b71AAAACXBIWXMAAA9hAAAPYQGoP6dpAABp+ElEQVR4nO3deXwTdfoH8E/SNiktbXoBKVigtFzlFBSoIK5YBEFExVVBXUEXFcFV2HUFlUUWXTz2J+gqHqyKioCiInJYV0BhwSIuUKWWm3IobZEW2lLolczvjzilaXPMTCaZSfp5v17dlXQy+WYonSff7/N9HoMgCAKIiIiIAsSo9QCIiIioeWHwQURERAHF4IOIiIgCisEHERERBRSDDyIiIgooBh9EREQUUAw+iIiIKKAYfBAREVFAhWs9gMbsdjtOnjyJmJgYGAwGrYdDREREEgiCgIqKCrRt2xZGo+e5Dd0FHydPnkRKSorWwyAiIiIFTpw4gUsuucTjMboLPmJiYgA4Bh8bG6vxaIiIiEiK8vJypKSk1N/HPdFd8CEutcTGxjL4ICIiCjJSUiaYcEpEREQBxeCDiIiIAorBBxEREQUUgw8iIiIKKAYfREREFFAMPoiIiCigGHwQERFRQDH4ICIiooDSXZExf7HZBewoKEVR2QWUVtYgoaUZ1thIDEhNQJjRfUEUm13At4dO45NdP+N8TR0ua5+ArtYY7DhWipNnLqBtXAtckZaEy1MT8H1BKXKOnAZgQGZaIgZ1Smxybm/jEL9/qqIKrWO8j0/Na+Pra3o7jxbvjYiI9McgCIIg9eCnnnoKc+fOdXqsa9eu2LdvHwCgqqoKf/7zn7FixQpUV1djxIgRWLRoEdq0aSN5QOXl5bBYLCgrK1Otwml2XiHmrslHYVlVk+8lWyIxZ0wGRvZMdvm8GR/9gPM1NkWvGxcVgWdv7lV/bm/juKFPMj7/odDp+57GpwZXY1Lymt7Oo9brEBGRPsm5f8sOPj7++GNs2LCh/rHw8HAkJSUBAKZMmYJ169ZhyZIlsFgsmDZtGoxGI7Zt2+aXwUuRnVeIKUt3wdObNAB47c5+TjfB7LxCPLB0l8+vDwCv39kPALyOw93Y4GJ8anB3beS+prfz3Dc0FW9uKfD5dYiISL/k3L9lL7uEh4fDarU2ebysrAxvvfUWli1bhmHDhgEA3nnnHXTv3h3bt2/HoEGD5L6Uz2x2AXPX5Eu64c9dk4/hGdb6pY85q39SbRxz1+RDEATZgQcACHDcpBuOTw2ero2c15RynsX/bRp4yH0dIiIKHbITTg8ePIi2bduiU6dOuOOOO3D8+HEAwM6dO1FbW4usrKz6Y7t164b27dsjJyfH7fmqq6tRXl7u9KWWHQWlLpc4GhMAFJZVYUdBaf3ziiuqVRtHYVkVisqVn6/x+NTg7dpIfU0p57F7iLr88d6IiEjfZAUfAwcOxJIlS5CdnY3XXnsNBQUFuPLKK1FRUYGioiKYTCbExcU5PadNmzYoKipye8758+fDYrHUf6WkpCh6I66cqvAeeLg6Xu7zAkXNcUk9l7fj1BqTXq85ERGpT9ayy3XXXVf/371798bAgQPRoUMHfPTRR2jRooWiAcyaNQszZsyo/3N5eblqAUjrmEhFx8t9XqCoOS6p5/J2nFpj0us1JyIi9flU5yMuLg5dunTBoUOHYLVaUVNTg7NnzzodU1xc7DJHRGQ2mxEbG+v0pZYBqQlItni/qRng2HkxIDWh/nltYsyqjSPZEglrrPLzNR6fGsRr4y7LQuprSjmPp1QOf7w3IiLSN5+Cj3PnzuHw4cNITk5G//79ERERgY0bN9Z/f//+/Th+/DgyMzN9HqgSYUYD5ozJcHtjbGjOmIz6hMcwowFzx/ZQbRxzxmTgqRt6SBpHY+JzGo4PcCR65hwuwercX5BzuAQ2T4kVLojXpuFreHtNpeeZfGUqDD6+DhERhQ5ZW23/8pe/YMyYMejQoQNOnjyJOXPmIDc3F/n5+WjVqhWmTJmC9evXY8mSJYiNjcVDDz0EAPj2228lDyiU6nzER0Vgvh/qfKhZM4N1PoiISA1+q/Nx++23Y8uWLSgpKUGrVq0wZMgQPPPMM0hLSwNwscjY8uXLnYqMeVp28WXwcgSqwum3h0/j5NkLaBvfAld0SsKgtESXVT6VVjhVqzaHq2vDCqdERKSU34KPQPBX8BEI/v50b7MLGPLcJrdbWw0ArJZIbH1sGG/qREQUUHLu32wspxJxRqJxYFBYVoUHlu7C+h9P+vwaatXmICIi0hKDDxVIqaQ6bflurP+x0KfXUas2BxERkZYYfKhASiVVuwA8uGwXsvOUByBq1eYgIiLSEoMPFciZaZi7Jl/2tliRWrU5iIiItMTgQwVyZhp8yclQqzYHERGRlhh8qEBqJVWRLzkZI3sm47U7+8Ha6PWslki2picioqAgq7cLuSbOSDywdJek433NyRjZMxnDM6ysmUFEREGJwYdKRvZMxqIJl2La8t1uW8iLdTjUyMkIMxqQmZbo83mIiIgCjcsuKhrVuy1eGd/P5feYk0FEROTA4ENlo3on4/U7+zXJAWFOBhERkQOXXfyAORlERETucebDD9hAjYiIyD3OfKiMreOJiIg848yHitw1lysqq8KUpb6VViciIgoVDD5U4qm5nPiYL6XViYiIQgWDD4lsdgE5h0uwOvcX5BwuaRJEsN09ERGRNMz5kEBKHgfb3RMREUnDmQ8vpOZxsN09ERGRNAw+PJCTx8F290RERNIw+PBATh5HmNGA2aO7uwxUWFqdiIjoIuZ8eCAnjyM7rxDz1u11+X0r63wQERHVY/DhgdT8jKOnK7Fww0GXsx4AMHs0Aw8iIiIRl108kJrHsXzHcbeBhwHAvHWs70FERCRi8OFBmNGAOWMyAKBJACL++fbL26OovNrtOcS8kAVf7XdZH4SIiKi5YfDhxcieyXjtzn6wWpyXYKyWSLx2Zz90TIqSdJ5Xvj6M8Yu3Y8hzm1hmnYiImjXmfEgwsmcyhmdYXXaqzTlcIutcYn2Q1+7sxzwQIiJqlhh8SBRmNCAzLbHJ42JeSFFZldu8j4YEOJZs5q7Jx/AMK7feEhFRs8NlFx95ygtxh31eiIioOWPwoQJ3eSHesM8LERE1R1x2kcFmF1zmfQDOeSHbDp3GK18f8no+9nkhIqLmiMGHRJ462zZORv3TNZ3xya6f3eaBGODYLcM+L0RE1Bwx+JBA7GzbOJAoKqvCA0t3IS4qAmfP19Y/nmyJxA19kvHmlgIYAKfnsc8LERE1d8z58EJKZ9uGgQfgCEre3FKA+4amuq0Pwm22RETUXHHmwwtvnW1dEbfTfv5DITY/ejV2HjvjMk+EiIioOWLw4YXSHSnidtqdx864rA9CRETUXDH48MLXHSl63k7rafcOERGRvzD48GJAagISoiNQWlnr/WAX9Lqd1tPuHeajEBGRPzHh1IswowE39W0n+3kGOG7metxOK+7eaZzLIvadYeM7IiLyJwYfEmRlWBU9T4/baaXs3pm7Jh82u5RONURERPIx+JBAbB4nJ4y4b2iqLpcvvO3eYd8ZIiLyNwYfEjRsHieFuM1Wj7MHUhNg9ZwoS0REwY3BhwyWqAhJx+l59kBqAqxeE2WJiCj4MfiQQEzQbFzJ1Bs9zh54W0LSc6IsERGFBgYfXnhK0PRGj7MHDZeQGgcg7DtDRESBwOCjAZtdQM7hEqzO/QU5h0vqi3DJLa+u99mDkT2T8dqd/dh3hoiINMEiY79xV3Trup7yttkGy+zByJ7JGJ5hZYVTIiIKOAYfuJjT0XhppaisCm9vOyrrXNYgqhIaZjSw7wwREQVcsw8+vBXdMgAwGABPu2YToiMw+/oesMZKmz1gTxUiImrOmn3wIaXolvBb4GEAnIIUMVz4x029JM90sKcKERE1d80+4VTqdth7B3f0OUGTPVWIiIg48yF5O2xWhhWPj85QvFwiZXln7pp8DM+wcgmGiIhCWrMPPsSiW0VlVS4DAwMcMxxioKE0QVNOTxUmgRIRUShr9ssugSq6xZ4qREREDs0++AACU3SLPVWIiIgcmv2yi8jfRbfkLO8QERGFMgYfDfiz6Ja4vDNl6S63W3b1XhWViIhIDT4tuzz77LMwGAx45JFH6h+rqqrC1KlTkZiYiJYtW2LcuHEoLi72dZwhgT1ViIiIfJj5+P777/HGG2+gd+/eTo9Pnz4d69atw8qVK2GxWDBt2jTcfPPN2LZtm8+DDQV67anCqqtERBQoioKPc+fO4Y477sDixYvx9NNP1z9eVlaGt956C8uWLcOwYcMAAO+88w66d++O7du3Y9CgQeqMOsjpracKq64SEVEgKVp2mTp1KkaPHo2srCynx3fu3Ina2lqnx7t164b27dsjJyfH5bmqq6tRXl7u9EWBw6qrREQUaLKDjxUrVmDXrl2YP39+k+8VFRXBZDIhLi7O6fE2bdqgqKjI5fnmz58Pi8VS/5WSkiJ3SKSQt6qrgKPqqs1TVz0iIiKZZAUfJ06cwMMPP4wPPvgAkZHq1KOYNWsWysrK6r9OnDihynnJOzlVV4mIiNQiK/jYuXMnTp06hX79+iE8PBzh4eHYvHkzXn75ZYSHh6NNmzaoqanB2bNnnZ5XXFwMq9Xq8pxmsxmxsbFOXxQYrLpKRERakJVwes0112DPnj1Oj02aNAndunXDY489hpSUFERERGDjxo0YN24cAGD//v04fvw4MjMz1Rs1qYJVV4mISAuygo+YmBj07NnT6bHo6GgkJibWP37vvfdixowZSEhIQGxsLB566CFkZmY2650uet3GyqqrRESkBdUrnC5YsABGoxHjxo1DdXU1RowYgUWLFqn9MkFDz9tYWXWViIi0YBAEQVdbGcrLy2GxWFBWVhb0+R/iNtbGF1i8leulqqmeAyQiIgoOcu7f7O3iJ962sRrg2MY6PMOq+cyCXquuEhFRaGLw4SdytrHqodqp3qquEhFR6GLw4SdSt6cWlV1AzuESzjgQEVGzweDDT6RuT523bi9KK2vq/8xcCyIiCnWKers0Vza7gJzDJVid+wtyDpd4LDsubmP1NofRMPAA2FOFiIhCH2c+JJK7I8TTNlZP9JaMSkREpDbOfEjgrfPr+h9PupwRGdkzGa/d2Q9Wi/MSTEJ0hMfXY08VIiIKZZz58EJK59dpy3ej4QpMwxkRV9tYi8qrMP3DXK+vXVTOnipERBR6OPPhhbctswDQOPWjcd6GuI11bN92yExLhDVWWjLq31bvwfofTyoaNxERkV4x+PBCSUdXMRaZuybfZVKq1GTUiiobHly2G/PX58seAxERkV4x+PBCaUdXT3kbYjIqAK8BCAC8saUA63/k7hciIgoNDD68kDpL4Y67mRMxGTU+2iTpPLNX53nc2ktERBQsGHxIcPvl7SVvlW3M08zJyJ7JmD26u6TzlFTWaLb7RU59EyIiIm+428UDV7U9GjIamiabigwArBZHuXRPrJYWksejJP/EV+x4S0REauPMhxvuanuIpmd1xivj+8GApnkb4p/njMnwWiRsQGqC17ofoqSWZknHuSN3BsNbfRM5VVg5e0JERCKDIAi6uguUl5fDYrGgrKwMsbGxmozBZhcw5LlNbgMPcVZj62PD8FV+kc8zA+t/PIkHl+32epw11oynbuihaMZBzgyGzS5g++ESTF22C2cv1Lo8X8Nr4C3A4uwJEVHok3P/ZvDhQs7hEoxfvN3rccsnD0JmWiJsdsGpiJiSzrTz1+fjjS0FHo8Rz/janf1k3bTFGYzGf9Guzudtqakx8Rqo8dpERBS85Ny/uezigtTcCvG4xkXElPRjmTUqA4sm9EOCh90v3uqHuCKlQqt4Pm9LTa54ulZyXpuIiJoPBh8uSK3tobQGiDujeifjX7df6vEYuX1fvFVoFc+3/UiJ20DBE0/XQOprs4cNEVHzwt0uLoi1PYrKqlzejKXuZHGn4TJNUkszIACnK6vROiZS9qyLWsflHC6RNeMh5RqoPUYiIgoNDD5cECuQTlm6CwbAKQCRs5PFFW85FZ6WXRryNusiBjgHiyskjkz6nIfUa6DVDBIREekbl13cECuQWi3ON0arJVJxkqSUnIrSyhqP5zDAsVPE04xDdl4hhjy3CeMXb8crXx+WdL7MTkkej2tI6jXwVh1WynshIqLQ0yxnPqTuThnZMxnDM6w+72QRX1NJToUrnmYc3O0ucaXhDMagtESPS00AENciAq/e0Q+DOklLqvXnDBIREQWvZhd8yK05Ie5k8ZW35EspjAZg8pWpbmcc5AY41kbv21ug8Oy4XhicLn2GBLg4g9T4mjd+bSIiaj6aVZ2PQNecaDjDcrD4HF75+pBP5/M2Tqn1SaZdnY7B6UkuZ3H8VRBMjVooRESkX3Lu381m5sNbzQkDHDUnhmdYVbkpyi3WJYW3cUrdNdK5TUu3szlqLjU1pNYMEhERBb9mE3zIqTnh601STt6FXJ7GKXXXyNHTlR6/z0CBiIj8qdnsdglUzQk1E0s9cTXOAakJsMZ6D0CW7zjOqqJERKSZZhN8BKrmhBqJpVK4GmeY0YDxA9p7fW5ReTWrihIRkWaaTfARqJoT/q7W6W2cHZOiJJ2HVUWJiEgrzSb4EGtOAGgSgKhZc0KNap3xUREAlI1T71VFbXYBOYdLsDr3F+QcLuHyDxFRM9RsEk6BwNSc8NYXBnDU62h4z7XGmjF+QHt0TIqu313yVX6RonH6uy+NL/y1jZeIiIJLs6rzIZJbc0Lu8eJuF8B1sa5XJ1yK+Giz1/MprY2x/seTeHDZ7iaP+6ueiRSBrrFCRESBJef+3SyDDzmUflrPzivEU5//hKLy6vrHrLFmPHVDD7/eZD3VF9FqlsFmFzDkuU1uE3HF2Zitjw1j4TEioiDFImMqcfdpvaisClOW7pLwad35RlpdZ4fdRY6DL9U/bXYB24+UIOdwCQ7/WoEv8ordHjt7tDbLG4GssUJERPrH4MMNXyqiugtazpyvxYPLduP+n89i1qiM+mOV5kFk5xVi5qd7cPZ8rdf3YwAwb10+RvSUX8HV19LogaqxQkREwaF5BR/VpUBtGRBhAcyeEy6VflqXUmTsjS0FqLEJiGthwsINBxTNrGTnFeKB3/JKpFA6u6BGkqjed+AQEVFghX7wUV0KHHoDOLIEqDhw8fHoVKDjHUC36S4DEaWf1qUWGXtn21G332s8syKeV5x56N8hHk99ni9pfN7G64nvy04Oet6BQ0REgRfawcfJbCDnLqD6dNPvVRYAPz0N/PQPIPUPQL//cwpClH5aV2vpQJypeGXTIaz4/rhTQNPSHIZz1TZF55X6vtRsxCfWWJmydBcMcL0DSI0aK0REFBxCt8jYyWxg8xjXgYcTO1CwBFjVFji2sv5RqRVR+3eIdyqalRRtVukNOCzYcKDJTIqSwENuBVc5y05SiDVWrBbn4MdqieQ2WyKiZiY0Zz6qSx0zHkKd9OfYq4FttwLl84BeT0r6tH5Dn2Rc9cLXzoXAYiMRbQ5DpcKZCX9QMrtQVHZB0nFyZnpG9kzG8AyrT8mrREQU/EIz+Dj0hoQZDzf2zAZM8UDH8R4rot7QJxlvbilosixRXO6+sqlW5FZwzc4rxLx1eyUdKzdJNMxo4HZaIqJmLjSDjyNLfHv+zmmOr5guGNlpIob9aTLe31WOY6Xn0SEhChMGdsCw//vGYz5ElCkMlTXKZj8az7QoNemKDrgkPgoJ0SZYWphgswteZxncJZm6GiOTRImISInQCz6qS513tfii4gDww+O4sGs+zp4ag73nemB1TSe8+s1hlFbWuH2aAKCyxoZHrumM97Yf83isK1ZLJG6/PAULNhxUNOy4qAjcdtkl+PyHQllbZKVsE26ISaJERKRE6AUftWWqn9ISVoE/Jy+r/3OlzYSlp0dh0enbUGaLcfu8OruA2aO7o7SyBj+fuYB3vj3a5BhxlmN6VmenxnI2u4B3c6QHLjf1bYt28S0wsGMidh4/g4UbmwYu3rbISt0mnBAdgX/c1ItJokREpEjoBR8RFr+/RHRYDe5v8xkmtfoMj//8J3x89lqXx73y9aH6/062ROL+oalNZiPioyNwU992GJCaWJ98KRb2khJ4iMsf/7y1L77KL8Kjn/yIonLXAYS3LbJSk0dnX+/f/jRERBTaQrOx3Jqu6i29SFAnGPHWr2Ox6Ndb3c6ENO5o+1V+ET7LPekUYCR7SGT1dM7X7uwHAJJyNUTLJw9qkviZc7gE4xdvV/RcIiJq3uTcv0OzzkeniQF9uXCDHfe3XoXdGePxUKulLo8Rg4J56/biTGUN3t52tMnMRlFZFd6QGHgAF2tkDM+wysrVAFzPckitbcIkUyIi8kVoBh/p9wPmpIC/rNEA/Dl5BQp6XY/vuk5ARuQhp++Lhbn++smPLp8vNXiYdnUalk8ehK2PDcPInsmSczUacrVFVqxtAjTux6vfSqQ2u+BU5M3momswERHpS+jlfACOMumZ7wPfXA8g8MW+DAagjbkc6zo/gkqbGbceeQ75Ven13z9XLaP4mQud28Q4LXvIKfTlbYusp9omcmqFBIIaTe+IiCjwQjP4AIC2I4HfrQW23QbUlmsyBIMBaBlejXWdH8HhqnYYd+SfHnfHSNV41kJuoS9vsxfBUIlUraZ3gWazC7q+rkREgRCaCacNVZcCW28Hir/y/Vw+EgTg4IV2uPf4UzhRI//GKM5abH1smNMNy2YXMOS5TW67xoqssWY8dUPw71QR36+7pSZ310lrnKkholDGhNOGzAnANf8BBn8EhEVpOhSDAegS9Qu2dJ2MT1KnwxJWIfscrmYtPOVqiKZndcG2mdeExE1O7aZ3gSDO1DQetzhTk51XqNHIiIgCL/SDD1GH3wM3ngD6zAdapmk6FIMB6B9zELszxmNZx0clBSFxUREelxLcdY1NtkTi9Tv74eGszqrPAmiV7Ck1x0VOLow/eaocKz42d00+k2WJqNkI3ZwPV8wJQI+Zjq+aM8DPq4HvJsvrfqsiowG4InYvcjPGo9ZuwP3HHsfX5zJdHvvq+H4Y3NnzDh5PuRpScg3k5CNouYQgNcdFbi6Mv8iZqWH9FCJqDppX8NGQKd5RDyTSCuT8Aaj+VbOhGAyAKUzA26nPwCYAE448gx3n+zi+B0f+wiCJNyVXXWOlBApyggmtkz3FeiTuclz01vQu2GZqiIj8Tdayy2uvvYbevXsjNjYWsbGxyMzMxBdffFH//aqqKkydOhWJiYlo2bIlxo0bh+LiYtUHraq2I4Hr9zmWY2K6aDoUgwEINwIfpj2B/T3GYmDUDwB8q60hJddATj6CHpYQvNUjEQDcfnkK1v54Uhe1P4JtpoaIyN9k7XZZs2YNwsLC0LlzZwiCgHfffRcvvPACdu/ejR49emDKlClYt24dlixZAovFgmnTpsFoNGLbtm2SB6T6bhe5as4AXw4GKvYG/rVdsMMAY+cHgd5/dywbyeBtVwjgmN0QBAFF5dUuv99454ieSrC7mq2Ji4oAAJw9X1v/mNY7SrztRtLr7hwiIjnk3L993mqbkJCAF154AbfccgtatWqFZcuW4ZZbbgEA7Nu3D927d0dOTg4GDRqk+uD9avPNwC+rtHt9Vy65GRi4WHIQIjVQkEIMJlbn/oKHV+R6Pf6l2/tibN92qry2Jw3zVI6ePo+FGw40ucE37IEjNwBRqy6HOLsEOFey9WVsRER6Iuf+rTjnw2azYeXKlaisrERmZiZ27tyJ2tpaZGVl1R/TrVs3tG/f3mPwUV1djerqi5+6y8u1KQjWxFWfAiU7gQ2/A2zntB6Nw8+fOr56zwd6zvR6uLvutkqI+Qh6W0IQc1zE2QV3y0Geuvm6o2ZSbTBVjiUi8jfZwceePXuQmZmJqqoqtGzZEqtWrUJGRgZyc3NhMpkQFxfndHybNm1QVFTk9nzz58/H3LlzZQ88IBL7A7dVOIKQzdcDVe7fR0D9OMvx1eFO4LKX3M6ElJ5zvZSihBhM6DXZU+0dJf5Iqg2GyrFERIEgu85H165dkZubi++++w5TpkzB3Xffjfz8fMUDmDVrFsrKyuq/Tpw4ofhcfpPYH7i5EJggAFd+Dnt4DHRRF/bYUuCTRGDHNJffTog2STpNdIT7H4PGnWz12nxOzR0l/kyqFWdqxvZth8y0RAYeRNQsyQ4+TCYT0tPT0b9/f8yfPx99+vTBSy+9BKvVipqaGpw9e9bp+OLiYlitVrfnM5vN9btnxC9dSxmD7/oXYPTBhSipbamPIOTQq8AyA7AqxTFL8xurpYWkp1fW2l0+Lt4WZ4/ujh0FpfXFxIZnWF0WNLNaIjXLXVBzOSgYK6gSEQUTn+t82O12VFdXo3///oiIiMDGjRsxbtw4AMD+/ftx/PhxZGa6LpwVrE5VVCG/Kh39966AJawCi1LmITMmH5p/iL3wM/DlZUBEPDDyewxI7YRkS6THG6knVkskbuiTjHnr9rrMe9j62DDdLCGouRzEuhxERP4lK/iYNWsWrrvuOrRv3x4VFRVYtmwZvvnmG3z55ZewWCy49957MWPGDCQkJCA2NhYPPfQQMjMzJe90CRYNPz2X2WJwx9HnYQmrwJsd/o4B0Xth0DoIqT0DYU06wlpm4OnrVuKPKwo8NpwTGeBYqnlydHdYLS1wprIGU5epX0zMH51dxeWgKUt31df6EMldDtJbUi0RUaiRFXycOnUKf/jDH1BYWAiLxYLevXvjyy+/xPDhwwEACxYsgNFoxLhx41BdXY0RI0Zg0aJFfhm4lgakJsAaa3aqjVFmi8FtR15AiqkQy1NnoZ3ptKZBiAEAzuXjmvwe2NenJe7/5Tl8c7qDx+cIAEoqa2C1tMCA1ATVd48A/i3LrtaOEr0m1RIRhQqf63yoTTd1PjzIzivEzE/3OBWyEom34SW/j8dVJ++FULHPbafZQBJ++/rT0RlYWz7M47Ev3d4XrWMiZRcT8zaj4W4Hidq1LtSYWWFdDiIieQJS56O5cncDFVmiIvDszb1wVc9koP9e/G//IVj+ezU6t/hZ85kQA4B/dXgRLwkvYnyD/jGNna6ohl3iTg4x78HbjIa3HSRKZ1JccdXfRi7W5SAi8h/OfMggtVx5wzLZ4nMSqvOwMu0vaGGs0z4nBIAgALV2A+4seNplEJIQbUJpZY3X8yyfPAhlF2o8zmi8OqEfCssuYN467yXrA1GWXQ5/5KcQEYUiznz4ibctmEDTQlYXEyGr0OOnz9A98hBWpD6KmPBabWdCfuuk+2HaEzhVG4drD76GMltM/ffPeAk8xLyH/h3icdULX3usiTFt+S5ILYnhzx0kSgIJNWZRiIjIGYMPGZRuwWw4hZ9flo7ee1chxVSIlal/RhtTueZBSBvTWeRmjMd5ewQmFTyFHef7eNwd03D3yM5jZ7wGZHJqcflrB4k/E12JiEge2UXGmjNftmCO7JmMrY8Nw/LJg3DP4I44UZOMQfuXYej+xThZk6B5sTKDAYgOq8WHaU/gYM8xGBD1Q/33EqIjnI5tWExMrZmKxpVU1STm6TQOksQtw9l5haq/JhERuceZDxl83YIZZjRgQGoCZnyUW//YiZpkXLHvPaSYCrGq03QkRpzTfCYkwuBYjqmzA9NPzMA1182ANTbS5XKFWjMVAoDbL09R5VwNBTLRldxj7gwRNcSEU5l83YLprc19RuQhvNVhLqymM7pITAV+u0knDAQGfwDEpDl9T0yodReQyaX2Uoi36y3SW6JrKOGSF1HzIOf+zWUXmcT8DaV9TbwtU+RXpSNz//tI3bMWc07cA5sd2i/JAEDpd8CadGBND6D6Yk8TT43mlFB7KYSl0gPHZheQc7ikvgeQzS5wyYuIXOKyiwK+tEaXs0zx7pmb8e6Zm3Ff4keYmfwejHoIFSvyHZ10Ta2Bq9cDif3d1sQwGuQlmwLqL4WwVHpguJrdsMaaUVVn55IXETXBZZcAU7pMYQmrwD1JqzEl6UOYwnT0VxabAQz/L2BOaLKuv/1ICV7aeFDxqdVYCvF2vcU8nYa1WUgeb4X3vOGSF1Fo4LKLjnlaphCrkN4/NBXJjZZ1ymwxWFB8J7r8tAa3HnoG1bZwVXIsfFaeD3ySBBxbWV8TY2zfdshMS0SnVtE+nVqNpRBv1xuQ3nCOmvKU0CsVl7yImh8GHxpwlzdiiYrAI1ld8NeR3bH50auREG1y+fwd5/ug60+f4blTU2APxIC9EoBttwLLIoCdf6nPCfF1KUOtpRBf83TIPSmF97zhkhdR88NlFw3Z7AJe2XQQ72w7irMXLjapS7ZE4vbLU7Bgg/clC0tYBea2fR03xG2Grj6893sZti7TFC0x+WspxJ/bPZvrVtLVub/g4RW5ip7LJS+i0CLn/s3gQ0OeurzK/UuJC6vAn1otxd2t1yFMpfH5zoDjcbfjhm034GyD0u2en+EQTDMSzXkrqdStzI0F498zEXnG4CMISGlSJ5cBjiJh/SN/wL87Po3Y8Au6qBUiADhU3QH3FDyJEzWebzSJ0SY8c1PPoLkheQoggdC/uUpJ6LVERSAyPAxF5c0vOCNqThh8BAGlnxjlsIRVYHH7p3B5y/36CEIE4IfznXD30Wecmtg1tODWPrip3yUBHpky3gLI5rKsIKXwntKt6UQUPLjbJQjIyfBX+iu6zBaDWwv+D33zl+O7iu7aFyszAH2jjyA3Yzw+6zgVlrCKJsdYLS00GJky3pItBVzschzKpCT0Nt4JxcCDqHljkTGNSM3wn57VBSu+P+7T8kyZLQa3FbyAFFMh3us4Gx0ji1SpRqqUwQD0jT2G3IzxEARg7slJeK90nMe+OHrE6qkX+VJ4j4iaHwYfGpHapG7asHRMG5aOHQWlKCq7gHnr9uJMZY2iugonapJx9YF/45VbOuD6n0cD54/7+C58YzA4vuZe8g6eavcOTsSPR1jtpYA5OAIQVk91Js5uEBF5w2UXjcgpfiX+Ur+p3yX4x009XZ5PzufLxPhk4MZjwIj/AWEtZY/dHwwGoP3Z5cAnrYBjK7UejiRiAOnu2hvgSKwMptkcIqJAYPChIaXFryxREU0ei4uKwKIJ/ZpURm3M6WaY2B+4rQIYcwiIiNdHxVTYHQXLvhrm1MBOj1g9lYhIGe520QFXBaoANHnsq/wijz00rutpBSDgi7xit6+1aMKliI82u16XL9kJ++YbYKg6qWlOSBPt7wAuf1m3yzHNuc4HEZGIW22DnOsOoZGoqrPh7PlaD8/0Li4qwukcjW+S4ms/FP0sxidu0MUW3Xptrwcy39VlENJcK5wSEYkYfASYmjceXzuEyiWO8pGsLii7UIO3tx2t/16KqRCL289D1xbHGYQQEZFHDD4CSM0pd5tdwOBnN6KovFrtYfrEElaBN9rPw8CW+QxCiIjIJQYfAaJ2ae2XNhzEgg0HVBuf2lJMhcju8zyiq703vAsUAYAQ3w/GIR8BMWkBeU2lM11cmiGiUCbn/s06HwrZ7ALmrsl3uTwiwBGAzF2Tj+EZVkk3mOy8Ql0HHoCjTsiGtK8xtnsLYNsEoOhLrYfk6GdzZheENekwdJ0O9H/Rrzd5pTNdTEolIrqIW20VUrO0thjIBIPWMZGOZY5h2cC4EiB5jNZDAvDbbNP+BbAvi8CKN27FA29/hYdX5GL84u0Y8twmZOcV+vwa4kxX47/3orIqTFm6y+1rKH0eEVGoYvChkJqltb0FMnphNAD9O8RffMCcAFz9uSMISblFu4E1YEQd7rB8jNyM8Vjcfi4sYRWq3OS9zXQBjpkum935CKXPIyIKZQw+FFKztPaG/CJfhxMQdgHYeexM/Z9tdgE5h0uweu8F5LR9HTVjf8W35/tr3sAOcFRMHR73PXZ3H49prd4H4LjJbzt4Gqtzf0HO4RJZN3ylM11sPkdE1BRzPhSS2pvFW2nt7LxCvNVge6veiTM5rnIYEqJNKK10zDY80uoD/CFpLcI0Dm+NRuDPyR9iuvVD7DrXFdPf/wtO1DhyLOTkXCid6WLzOWpOmFRNUnHmQyFfS2vb7AK2HTqNmZ/s8d8g/aB1TKTbHIbSyhoAji66c4seQFreWkw68iRqbNB8NsRoAC6L2Y8tXSfj2bYvApCXc6F0povN56i5yM4rxJDnNmH84u2q51tR6GHw4QOlvVnEf6R3/Ps7nL3gW8XSQIqLikD/DvF46nPXOQyufH1uELr8tBajDy7EuTqT5kGIwQDcnrQJBb2ux+6M2zAg6gdJORdKm8ix+Rw1B0yqJrlY50MFcqYaA13BVE3xURH4Q2YHvLTxkOJzpJgKsSptBhLDK3RRsEwQgANV7VA+5Btc3jXd47Fy67qIPxcb8otcLq0prQejB5xeJ5HNLmDIc5vc5jaJS9BbHxvGn5EQxzofASa2vPfG086HYHDmfK1PgQfgqBVy2d7lyIg8hI/SHkO0sVrTIMRgALq2+AXY2RnYHQl0mQr0eLxJ1VSbXcD+ogq0MIXhfI3N6XtxURGYf3MvpwDCVU6M0eBI2hVZNa7zoTSAYM0SakhOUrWU35PUPDD4CKBg2VIbCPlV6ej50yfIiDyEj9P+ghbGOu1nQuxVwL7/c3wNeBtInwTAcbOd+eket039zjR63N0MSeOVHS0nHX0plubqvYnT68E4i0O+YVI1KcGcjwDiP76m8qvSkfHTZxh9cCEq6yI0zwmpt+MeYJkBR7MnYebyzR67CYvVbG12QdbsVnF5tSbr4UrX51mzhFxhUjUpweAjgPiPz738qnT0yF+F0QcXorwuUjdBSMfSJdiVMR5/Slrq9piG08pyZre0uGH7EkCwZgm5wqRqUoLBRwCd+W0rKrmXX5WO3vkfY+j+xdh9rnOTpQotGA3AjLYrcLjn9XjKugiWsAqXx52qqJI9uxXoG7bUAGLJtoImxdg4vU6u+Fp2gJon5nwEiM0uYN664OjfogcnapJx05EFAIB2EUV40vomRsbt0DQvJMwITGy9Hne3Wo8vyjIx65c/ocwWU/99X2a2pN6wfd1lIvV15q3bW//fYi4Ip9fJHbHsQOM8Iq2Tqkm/GHwESHNONrXGmlFVZ/eYN+HJL7VWTDnxN4wq+y8Wtv8nTEab9yf5kcEAjIrLwUhLDuYX3o3Fp3/vNK3sqfKtO1Ju2GrsMlESGIi5IK9OuFSVqr4Umkb2TMbwDCu3YJMkXHYJkOY8FX3vkE6KA4+G1pdficv3LsVnZ67SRU6I0QA8nvwuDvYYg4W/K0WY0eBxCtoVqevhahVx8rY+74p4qeet24vZozm9Tu6JZQfG9m2HzLRE/iyQWww+AqQ5T0WfOHNetXOV2WLwyIlH0Td/OTaVXap5EGIwABFhAgYW3Ar7Rxbs+ul/qK6zY3TvZK9LRFJv2GruMpEbHDV8ncKyKsRHmxRV9SUiaojLLgHirRFdKDtfXaf6OctsMbjn2DxYwirwaOu3MSHxKxi1bmJXV45Lcy+HsTINP/w8E3bB841Yynq4zS5gybYCVYo4ifki1XV2PJLVBct3HEdRubwZuVMVVRjbt50uptdZZZUoeLG8egCJU+cAmk0AYmkRjjCDEaXn/b/TZ3LiSsxMflfzTrqiGjuwrGQ0Fpy60ykxNa5FBF69ox8GdfI8Le0qx8OTl27vi7F920k+lzXWjPED2qNjUjROV1Q7JZm6s3zyIF1UqWSVVSL9kXP/1smv6ebBXSO6UFZ2oS4ggQcALC75PfrtXY7nCu9GWV1UQF7TE5MRmNhqHXIzxmN1x4fqt+ievVCLfYXlWPvjSaetrA25y/HwxN3SnrtzFZdXY+GGgzCHGzFxcCqssWa359ZTrQY2MSMKfpz50EDD6eKjpyuxYMNBGBD42ZDpCqfeg0X3yEP4oOPjiI84r33pdjia2J2us+DugrnIr7rYxK7xJ3Zvjboa89S4S2rTr9mjM/D4Z65LyOupAR6bmBHpF2c+dK5hRvjDWV3wugazIcmWSEwblo5tM4dhelbngL62r+4c2B5xLSK8Hre3Kh399n2E0QcX4mxdtC6SU1tFlGFd50fwbZe76mdCGn9il7Mt21vSqtSiYg8u2+V2R5IlKkIXgQfAKqtEoYLBhw6M7JmMrY8Nw+zR3QP2muLNKsxoqA+AkoNkOejEmQv4x029JB+fX5WOvvkfYuj+xSisjtNFENI28gxyM8bjUM8xGB27CcDFHStytmV722WixhbvFhFhGJ5h9fk8amCVVaLQwOBDJ8KMBkwcnCq7BoNcRgOwaMKlTW5WYgA07ep0N8/Uj80HfsXBU+dkP+9ETTIy9y/F0P2LceDCJboIQsKNAv7V4UXs73E94qvzsKOgVPK27Nmju2PrY8M8zkioscVbTzMJrLJKFBoYfOiItx4JBjjyNF66vS+mZ3VukiCYbInE8IzWHl/j5dsuxajebd2+/uD0JMnjNYVps6ZuAPDOtwWKn3+iJhnXHnwdffOX48OSq2G3qzc2JQwGwBQGrOv8CAZ8l4wBLXIlNeqaODjVa16DkqJiruhlJoFNzIhCAxNOdcjdNsLbL2+PjklR9TUNALisc7D+x5N4cnUeSiubruF7245oswu4/JmvXD63sZbmMJyr1rbUuVr+r90LuDlhsy4SUwGgFi1w56EnseN8H6dEZCXJn+62eMtJctbLFlvA8/sB9JEYS9Qcybl/M/jQqcY7Yhy7Uqrrv+8tiFj/YyEeXLaryePefkHb7AKmLduFL/KKVHkf/hTXIgJlF2pV2yWUYirEstTHcYnpV90EITbBgO3nemLmL3/CiZpkxbUs3AW0s0d3x7x1ez0mcSbrcPcI63wQ6Q+DjxAifspr/JfkKYhQuh1RblErrU3P6oyFGw6qvkU5xVSIFZ0eQ9uIUt0EIQKAk4Ze+PnST3BZl3RFgYC7iqDz1+fjjS3ul7HuH5qKWaMyfBi9f7DCKZG+cKttiFDa00PJdkQlRa20ZDQAU36XjlcnXAq17zcnapIxeN97GLp/MfIq2kNCyxS/MwBoJ+zBgJ1dULI0Cdt2ZMs+h6umXza7gM9/8FyU6/MfCiX1jQk0NjEjCl4MPnRMaU0DudsRPQU5emUXgJ3HziA+2uy34OBETTKuL1iETnvW4tXimzXfHQM4klNbh5fiioPXoejzLOQcLsHq3F/cVkr1RkpNET3tdvE3m13w+ZoSkXeygo/58+fj8ssvR0xMDFq3bo0bb7wR+/fvdzqmqqoKU6dORWJiIlq2bIlx48ahuLhY1UE3F0prGsjdjiinqJWenKqoCtgujBeK70Hf/OV4/dTNqLFr34/RYACs5zbi8pwkVG6ZhAfe/gpDntsku7Q462ZclJ1XiCHPbcL4xdvx8IpcjF+8XdE1JSLvZAUfmzdvxtSpU7F9+3Z89dVXqK2txbXXXovKysr6Y6ZPn441a9Zg5cqV2Lx5M06ePImbb75Z9YE3B0prGsjdjhisN5bWMZE4errS+4EqKbPF4Nmie9Al7zMM2bsYuefSNJ8NCTcCE5K+cvSPSR6Hf338kaybZTDVzfDnrAT7xRAFlk8Jp7/++itat26NzZs3Y+jQoSgrK0OrVq2wbNky3HLLLQCAffv2oXv37sjJycGgQYO8npMJpxeJiaNFZVUul0Q89bGQsx0x53AJxi/ervr4/UV835sfvRpDn9/ktAso0FJMhfhP5wcRaazVRXKqIABHa9uh/W0/IKyF962xSn7GtEj09OfuFvaLIVJHwBJOy8rKAAAJCY5P0Dt37kRtbS2ysrLqj+nWrRvat2+PnJwcl+eorq5GeXm50xc5eCs6Brjv6eGug66rctxqFaIKhIbve+exM5oGHoAjL6T7T6vw0LEZqLMbNJ8JMRiAVNMvMK5KAjaNAKo952rI/RnTYmnC37MSSnOrmB9CpJzi4MNut+ORRx7B4MGD0bNnTwBAUVERTCYT4uLinI5t06YNiopc142YP38+LBZL/VdKSorSIYUkOUGEq+dufWwYlk8ehJdu74vlkwe5LMft6QakNw3ft56Wi9aWD0N63hr0zV+OQ5VttA9CAKDoP8AnicDWCR6DEKk/Y1osTSjd8SWHkrwX5ocQ+UZx5tzUqVORl5eHrVu3+jSAWbNmYcaMGfV/Li8vZwDSyMieyRieYVU01S1uR5TyGq/d2Q8zP3XdVl1rvdrF4vFRGU7vWw95CI2V2WKQdfgtpJgKsbj9PHRtcVz75ZjjyyEc/wgH0t/EPtMIlz8/jX/GkqLNgAE4fa4aOYdL0L9DvMcgwABHEDA8w6rq0oTUWYntR0pktQZoSG7ei7vaO2IQxgqrRN4pCj6mTZuGtWvXYsuWLbjkkkvqH7daraipqcHZs2edZj+Ki4thtbruimk2m2E2m11+jy6SGkT4YniGFU99ng/AdfBh+O1/tPhUv+eXcpyprHG6sYnLRe7yFbR0oiYZIw8tgiWsAh93nIH0qEJNgxADbOhy8F60t4VjzsnJmCGMa5IvIf6MZecV4i8f/+B000+INqG0ssbt+RsuTaj5cyp1VmLqB7vw7Lheim763n6OxJyPAakJXmdi/BWEEYUaWcsugiBg2rRpWLVqFTZt2oTU1FSn7/fv3x8RERHYuHFj/WP79+/H8ePHkZmZqc6IyW92FJSiqNzzp0wx8NDi1+rs1XlO0+vBsFxUZovB8MOLMXT/Yuyu7KzpcozBALQIr8Pz7V/Dt+2zMHhXJ6xc+65TzoK7pRVPgUdDai+FSZ2VOHuhVvHSj5y8F6X5IUTkTFbwMXXqVCxduhTLli1DTEwMioqKUFRUhAsXLgAALBYL7r33XsyYMQNff/01du7ciUmTJiEzM1PSThfSltQbx72DOzbJD0i2ROL+oal+DQJKKmuwZFuBUwDiLl9BTyLDjRg16ApMKnwFqXvWYk1ppvY5IQYgJrwKt5RNxGW7e+OBt7/C4Gc3Yuane3yaRVJ7KUxuMrTS/A+peS+si0KkDllbbQ1u5o3feecdTJw4EYCjyNif//xnLF++HNXV1RgxYgQWLVrkdtmlMW611Y7ULbfLJw/CgNQElzko2XmFfs8bcbXF0mYXsGRbAeat2+u311UiPioC3z2ehTCjAYOf3Vi/OyfFVIgX2i3AgJb5qpeHV8JmBx77+U/4+Oy1ip7vz+2o7nIs3PGlA6+3bcRy/o3opQswUaCwsRwp4q4TbkNSOpzW1NnRb95XOFddp/YQ6xnQtKmet3oNWlg04VKM6t3W402rXUQRrondjj8mfoYU82lNc0Pq7MBHpcPxXPE9KLPFSHpOIFrZZ+cVYuYne3D2gveg9qXb+2Js33Z+GYcvtXeIQh0by5FsNruAeevyvR5322WXeD1m57Ezfg08AMfaujjFLtZbWPvjSfTvEO/X15Vr3rq9yM4r9DgN/0utFe+V3IihB5bgweOPadrITqyYujtjPN7sMBcpJu85FFK2fftqZM9kvDqhn6Rj/bkLypfaO0R0kfZNKkgXpPZ3WbjxED78388eK0sGar27sKwKr2w6iBXfn9B8tiOuRTjOXmgacInbLx/J6iLpPF+UXYlJBVF4rcPTiArTbsuz0QBca/ke11q+x5m6llj86834oPS6JrMhidEmbH70apjC/f85ZlBaouRdKf4k5oc0rrhqVaniKlFzwGUXAgCszv0FD6/IlXSst2n2YCjXLr6H+4amYuXOXyTv5mioRYQRz9zYC21iI/HnlT+43Skk3hQFQZBckdUSVoEpSR9hYtJaRGoYhDQkCMCiUzfhheJ7nR4PZH6DnLYB/qZFmflACNX35Q+8Vs6Y80GyyQ0YPK1te1sX14OGSaurdv2M6R/9IPscUvI5GnrkmnQs3HhI9utcYirC6ynPoEdUgfYFy+BITt1+rhdmnvwTTtQk+zXHwhV/9nlp7nhtpeO1aorBB8mmNGBw96nX3SdUV2Ijw1Fe5d8ckYZmj+6OiYNTEWY0KN4lc//QVMwa5Vj7lzprZGkRgTIJCZPuXGf5L15OeQERRrvic6jJLgCflQ5F8qhPA76zg5841eduV5EWs0p6x2vlGoMPUkROwCDy9KnX3SeD2aMzEB9tqr9xFJVdUDTzIFfj2RpX4/MmMdqEeWN7YlTvi79YArnMZAmrwJRWH+GPSasRrpMgRABgiO4EDPkISOyv9XBIAXb2lY7Xyj05928mnFI9d4l0nnjaWSC1J03O4RLZY72+dzLW/ii9mmXjnQhya0cAjhLjjQMPILBl3stsMXiu6F68e2Eq3ri2HF0OTIJZKNO0wqsBACqPAF9eBrTsAozIAcz+SfoMxRkPue/JH9dATuXW5l6/hNdKHQw+yIkYMGw/XIKpy3a5rasgdWeBlJ40cm/e8VERePHWvth57Izk5zTcieCpP4cnZyprMHXZLrxm7NekJ8qcMRmYsnQXDJA+a6REwyCqd89koP9NQHUpDnx6Azrbt2mfE3LugKOTbofxsPX7F3achGo3yVBcY5f7nvx1DVi5VTpeK3Wwzgc1EWY0YHDnJDw7rhcM8H89g4a1E6Q4c74WO4+d8VpvYXpWZ7x0e18snzwIWx8bVv/LWeq24sY8tXAPVJl3lzU1zAkoGbAaffOXI/98R81LtwMAji2H4dNWOJM9Bn9b+V+fW8676zkjbmUOxlb2ct+TP6+B3M6+zRmvlToYfJBbUvtdqPlacS0iJB1/qqLK4/hev7MfHs7qgrF92yEzLdEpSPoqv0jxOD01DhvZMxlbHxuG2aO7Kz5/Y+Ko7xncsUkQ1dCA1AREtWyFUYdeweiDC3HeFqF5EGI0CBgVn4PcjPFY13kaIi4UKLpJeuskCyjv6aIVue/J39fAWw8dAxwzLP6uoRIMeK3UwWUX8khq3oZarxUTGYE7/v2d12PFTxVyx2ezC/gs96TPY3U3pRpmNCApxuzz+UVSC1eJs0cPLN2F/Kp0ZPy0CgOifsDS1CcQYYSmyzEGA9CjxVFs7joZ28/1xFPr/o7hGTdK/hkKxTV2ue/J39fA09IhK7c647VSB2c+yCsxb8PVLIJY2rxhW3ZfDOqUKPtThafxNbajoFRRQbHGPE2pHj193ufzGwzA+5MGuJ3pcGVkz2RMb1BJdcf5Pujy01qMPrgQJbUtNZ8JMRiAzJg8fHHJzTi94Q6gWlrb+VBcY5f7ngJxDfw106n27wg9COSscKjizAcp5o/kN39/qvD1BuUt0dZmF7B8x3GfXgNwVBMNDzdKep8Ndz9c1jEebWJMKK64GGDlV6Wj/94VsIRVYGWnR9E58mdNZ0KMBqDN6eWwf/IhcpOfR3X7iR5nq0JxjV3uewrUNVB7pjMUk4RFgZwVDkUMPkgRd1tVxeQ3X6J/pb0zpGxB9OWXs5TgZ0dBqdsy63JJCZRc/XKPi3LkzTQO3spsMbj24Ot4Zmg1xl+YAmP1KVXGqZQRdlx68i84dexp3PbpS/jjdcPd5rPooaeLmuS+p0BeAyk71KTw5+8IvVDrWjVHDD5INm/JbwY4kt+GZ1gVfwqQ+6nC2ycsMTApKq9CQnQESivlVxqVkn+h5tS/t0DJ3S/3svOO92aJisDZ803f5xNbzPi/6Pfx3Oh2GF58N3Bmp1pDls1gANqYzmJl27uxd3sHfFfxAgZm/t7pmFBcY5f7noLtGgTidwQFN1Y4JdmkVvQMVMMxb6WO7xuais9/KJS1vdZoAO4dkoph3drImlJVq9ppXIsI7Jw93GPirJQqi7dedgle8tBP5v6hqZh1pRnI7g/Ulvk8bjW4q5gailP4eqnzoTa9/Y6gwGCFU/IrPSUAStmC+MaWAtnntQvAv/9bgP4d4mU1TROnx5XUEWlo0uCOXhNnpex+eH/7MY+v88aWAvS5pB9G/f4sULgJ2DIWsJ1TOGp1OFVMje0JDN8MmBNCco1d7ntSsrtLi+ulp98RpE8MPkg2PSUAKi0YBvx2kzPA4y4QuVPDYUYDZo/OwIPLdikaE+Co4DptWGePx0j9pS1leWn26jyM6GlFWPIw4LYKx4M/rwW2/xGoKZb0On5TnueomHrJOODS5xAWkxZyn5T9lTeg5SyJnn5HkD4x+CDZ9JQA6MsnJ6H+f9x/X0nthPhok+IxAcD8m3t5DXaOnq706TUaKqmsafoeL7keuKUIqDgMbBkHlPm/8Z9HP3/i+DK3AdInA92mwxYRH1KzIFJIDSi0TvbU0+8I0ifW+SDZGpZD93fpdW8C8clJboDj76nk7LxCLNhwUNVzbnBX9TUmDRidC/zuCwDSqs/6VXUx8NPTED5JwheLR+KBt7/CwytyfS7fHgykllfXQ0VYPf2OIH1i8EGK6KXIjrdSx2qQG+D4up3X041BvLGo7aOdJ7Bqt6MIVE2dvWlRqLYjgXFFQNe/oA7aT5UbIOD6mP9gZ/cJuC/xIwDB3efFGzkBhZxqqP6kl98RpE9cdiHF1EwAVJoY58+Osq6mhqWMU26X3oa8LfX4kuPiSUWVDdM/zAXg2OnTMPZpOK2/PmI6pu35HVoaz6F/5E+Y3HoVBrXMg1YfYMONAh5v9x5mtn0P35/LwKO/TA/JLZxyAgo9JXuGYpIwqYPBB/lEjWQ5XxPj3BUlS7ZE4vreyVj8X++7XaTUTpA6TjUCInc3hkDcMBpPuogzCvcNTa3fOVRua4mvKwfi64KBsIRV4LX2zyCzZZ5mlVONBmBgTD42d52M7yu7YdfBNbi8a7o2g/EDOQGF3pI9WYiLXOGyC2lKrTbhYkfZ5ZMH4aXb+9Z3gB3WrY2k5zdOEm08NSx3nO6mnBOipeVNuLsxaLE7QIxF3AVxZbYYTCh4FkP3L8ZP51M17SFjMAADWu5Dv53dgZPZ2g1EZXICCnZdpWDAmQ/SjNpVEF19wpL6iXH26O6wWlo4TQ0DjmJJRWUXMG/dXtnjdDXl3L9DPK564WvFuwDOqNAUTwkBnrckA8CJmmSMPvQvWMIqsK3/U2hZtT8gY3MlDHXAN9cBqROBfv8HmIP7Ritn94iW1VC1qitCwYfBB2kmEK3SpX5itFpaOL2GqyUWJeN0FRApvTHY7ALmrVM/2VRtUS1bocWNe4EfngD2ztd2MAVLgJ9XAQMWAx1+7/VwvZIbUCjtj+QLNeqKBGPwEoxj1gMGH6SZQCTGKak34K5GglrjVHpj8Feyqdrqb4KX/gNIvxf4eiSEc4f8uiPJo9oyYNutwK52QJepQPr9QTkTIvfnJpDJnmrUFQmW0vENBeOY9YK9XUgzger/IP5iBFx/Ymz4i9FbzxQ1xynlE1PDYw4Wn8MrX7vv0yKadnU60lpFY966vSgN8DLNpCs6Ys4NPZweq6mzY9wL7+L5pKfQLeq4Zkmp9cxJQOb7ju3DQUhvn7Sl9hna+tgwj40hPfVn0uPW3GAcs7+xtwsFhUBVQZTziVHJ7ILScXrbBSBn6aehwelJyExLRAtTGB5YqrzMuxLX9rA6/Tk7rxCPr8pDaWVrXFe2CJawCjzQaiXuTlyHqLDqgI6tXvVpYPMY4IplQOJlQITFp5mQQAcDets94uvyaTB2wA3GMesNgw/STCAT46ROQctd4lFjnK5uXl/lFyla+jEYgLo6O2x2ASN7JmPRhH6YtnxXk+2z/tB4B4WrT4Zlthg8V3QPniu6BwOifsQr7Z9Hq4izgZ8NEeocSzGilmlA2r31SzJSA4pgnXZXM2Dydfk0ELlfapM65iXbCjBxcKquAhC9zJwx+CBNBTIxTsonRrlbWX0dp6ublzXWjKo6u6L6IIIA3PXOjvob4KjeyXgFl+LBZbsVjU8KVwGYp0+Goh3ne2PAvqWwhFXguXYLcK3lexgNGq0CnzsM/PA4sGcufkyej/u/7a37/ilKqR0w+VpXRE9F0aSSOpZ56/bi31sLdBOM6ilYZs4H6YJeonFx/dpTddKE6AjMvr4HrLG+jVNpYqsU4kzS9KzO6JgUjaOnK7F8x3EUlau/1OHql5fUfB5RQnQE5o5IRrtTi5FRuQKRtl9VH6ccF2wmvFsyCq/9ehvKbDFN1vHVyHPQgj/yFLz9m/F2LQKV+6UmOT/feskBCUSOipz7N4uMkS6IsxJj+7ZDZlqiZr+wvTXEMgD4x029cNOlvo1TysyAL8TzLthwEA+vyP2tEZ0B07O6YMGtfXBLv0tUeZ3Zo7tj62PDmvzSkvsptbSyFg99ehw3bx2Bq48sx+au3wItu6oyRiVahNXggdafYVfGeLzfcRYuMTmKyOmtf4rIZhea9uNxcYw/Gs752kQuGIuiyekpFahmfp7oodlgYww+iBoJREMsLbbNFpdXYcGGA5i7Nh8f7/rZp3OJNwR369m+VGItKqvCxJWlyO70NU5abtC0YmqYAbgydg/+220yDva8AfdFvYhdBw/raqkgO68QQ57bhPGLt3vs8OvPgMmXfzPB2AHX05hdCXQw2pjegmWAOR9ELvm7RoIW69fiPfzs+VqX3zeHG1FdZ5d8PimfZj0tXxkMrqumNtwtIAgPYXBYRzzb7hVEGKWPzR/CjXZMarUetTv/g/KEBQBSvT7H3+Xw5eSd+Dtg8uXfjBZF0XzlbsyeaJW3oqdgWcTgg8gNf25p1KJHizdSAw8DgPuGprq9IYjLEqN6WvHWtqMun++tXLv4SQwAPsG12FCeiXuSVuO+Vh+jhbFO0jj9JQJ1uKb0IfzYowWW/HoD3iq5EWW2mCbH+XupQO52z6OnKyWd15efTV/+zQRjB1xxzEu2FWDeur1ej9fq373emg0CXHYh0oSUde64qAhYY/UXpAgA3txS4LLpX8MlADHwaHzvsFoicc/gjrJes8wWgwXFd6J73me47sBC7KroDJu2EyGIDbuAP1k/xI7ud2By0kdNvj97dHe/3jjlTKXb7AKW7zju9Zxa51boJfdLjjCjARMHp+o6b0WPeTUMPog0IGWd+9mbe2HbTEen3gW39ZXcETdQxAQ1Mdlx3pqf8ICLzr/iDMc9gzvWdxsenmF1cUZp9lal4+aCBUjLW4sZxx9BWV2UL2/DZ2ajHU+0fQ8Fva7HN13vRUakowptfLTZr68rZyp9R0GppJ1Ot1/ePihu+Hqj97wVPY6PwQeRRqQk6YmfBK2xkSitdJ2roQXxU/Urmw41melwdawBwBd5RfXT6FI/iVljzR4T+j49m4Wh+9/Cc4V342SNttswDQago7kY6zo/gi/TH8CZs01nhtQkZypdaqDSMUnbQC6YBSJR3Rd6Gx/rfBBpTEqNk9W5v+DhFbl+G4MBgCUqAmW/JaP665dCw1oNUnruAJBVC+U6y1a8nPYGIurOqDNgH9hgxI727wLJ1/kld0FOfY0dBaVBV0sjWOmlZpE7/hwf63wQBREp69z+TARruMzj6pORmhp+ApfySaz+mFhpSxi7hWtgHHMQ6DMfiOmi6tjlCoMdg47dhVPZN2PUC5+5zJHx6fwyptL1uOYfqvSet6KX8XHmgygISKm8qlTjCqU2u4DtR0ow9YNdOHtB3aWeD/44EIPTk5weq6mz4/2cozhWeh4dEqJwV2ZHmMKdPxfZ7AJe2XQICzYc8Hj+1xtPH9ecAWrOAsXfQPjuHkk1GfzBJhjx6ZmrcbT93zGkR1dVP21KLZktp7szkRJy7t8MPsglvU8dNkdKy7E3btonundwR2RlWD02TXN3s1L6S+ODewdicOeLwYfcXhPZeYWY+emeJrVK4qIi8OzNvTzePPd+9Vd0O/VC4JvYNVBhi8RjJx7GbsNwVetXhHojPAoODD7IJ/wFFVhyAj1Hi/o9kpNPp2d1xorvTzj9XSZER+DpsT0xqndbr89397Nw++Upv5Vsl+el2/tibN929edW0mtCnJnJOVwCQEBmpyQMkjB9nHO4BO999AIWXPIiIsO0Td4trEnE+yWj0f+ax3BN34yAvrbaHyz4QYVEDD5IsUA0H6KLlAR6NXV2DPzHBpxxU6m04Xm2PjYMX+YV4cnVeSitrJH8Gg25urkAULQMJCY0atGYTXzNC+d+xePJb+GW+I3addH9zRmbBbHDliOs3XWajkMpflChhhh8kCLB2qkzWPkS6GXnFeKB35ZE3Hndw24RNYJJd8syrjT+2ZHaFfSDPw6E0WBQ7VN1wzHHhlVgSquVuDNxPVqGadeu3W4Ih/GKZUDiZUCEBTAHR9InP6hQYww+SJFgbG0drNQI9NzlP8RHRWD+zb0wPMPq92DS1SdfV68DON+MpG4djmsR4ZT0qsanaldj7mn5FW93+DtaoUCzpNR6LdOAS24COj8AxKRpPRqX+EGFXJFz/2ZvF6qnx+ZDoUpOaWx3gZ7YV2L74RLkHDkNwLGFblCnxPrZBV9fw5vG/TiOnq7E8h3HnappumoOJnXrcOPdNq4apvk65oszKhPxdc4XSNs3ESmmU9olpp47DOz7p+MrMhno+hCQfr+uZkTU+Pml5o3BB9XTY/OhUKVWoBdmNGBw5ySnHSRqv4Y3jZuJTRvW2WsCopSut640bpgGQFGyo7sGaJGtB2Do6rcBAHcmrMWTyYsRGWaTMUKVVRUCPzwO5D0NDFoCdPi9dmNpgB9UyFcMPqietxuCOJXKQkTyuErYDESgp1UwKaWzqVgga8rSXbK37oqfqv+18SA+/N8JVZMdG/4bWFp6PdaUXYW5bV/HDXFbtE1OtZ0Htt0KlM8Dej2p3Th+ww8q5CtWOKV6emw+FOwadnl9eEUuxi/ejiHPbcKZymq/V5zUe1VLdxVO46KkNdBbuPFgk6l/cVlGaTXRhv8GAEc33UdOPIpL85dhYfF41NjCFJ1XNXtmA8dWajsG6P9ni/SPCafUBLfPqcPbboD7hqbizS0FAPxXcTIYqlo2nhmy2wXc8dZ3is8nztBtfvRq7Dx2RtFOmey8Qsz8ZI/LCq/z2y3A7QkbtcsJCY8GrvwcqP4ViEkHEvurdmq5NWf0/rNFgcXdLuQzFg7yjdTdALNHZ2DeOv8Getl5hXjq83wUlQdHMKlWKfmEaJPi2iYAsO3gabdBUIqpEM+2+xcyW/4Irf9ZCAAqwy/B0bSX0f3SG1XdueTtmvGDCjXE4INIY3K2LQ9ITXAb6KkRBDqCj5+cd6DEmvHUDT10e4NQWkreE7mfyL0FkKJxcRvw93aLEB1W4/G4QDhvM+Nk8hSkXzlb1u4YX2p2+Pozyg86oYPBB5HGpNaxaFhuvDE1PlUGcyEoT0sfSsmtPyGlmBsAWMIq8PkVq9Ch/BNA0HB3zG8EGGBofztw+StegxAta3Zw5iS0yLl/y0443bJlC8aMGYO2bdvCYDDgs88+c/q+IAj429/+huTkZLRo0QJZWVk4eFB+DwiiYObrbgAxaPAlodJmFzB3Tb7L2QPxsblr8mGz6+rzR72RPZPx6oR+qp6zYf0JqWOYntXZ63Flthic7LYIuPkU0ONvgLmNjyP1jQECcHw58FkKcDLb47FyanaoSY2fcQpesoOPyspK9OnTB6+++qrL7z///PN4+eWX8frrr+O7775DdHQ0RowYgaoq7vem5sOX3QBqBQ1a3VTUNCgt0eN1VEpO/YlpwzrDGus+mHT6uzQnAH3mAuOKgCEfASaNC2zZzgPfjAKOLAGqXf89a1GzI9gDY/Kd7ODjuuuuw9NPP42bbrqpyfcEQcDChQvx5JNPYuzYsejduzfee+89nDx5sskMCVEo82XbslpBg9Kbis0uIOdwCVbn/oKcwyWa3gA8XUdfyKk/EWY04KkbMmBwMQaPf5ftfw+MOQD0mQ/EdPFluD4SgO2TgE8Sgc/TgZ/mOwUiWtTsCIXAmHyjap2PgoICFBUVISsrq/4xi8WCgQMHIicnR82XItI9d3UsrJZIj7kWan0SVXJTcVeXRMspcHfXMdkSiYlXdJB1LqX1Jzz9XT6S1QXVdXbXgZo5AegxExizH7ilFLjhCDD4I8CgUX3Hc4cdFVM/S6mvF6JFzQ5WSCVV/wUUFRUBANq0cV7vbNOmTf33GquurkZ19cUs/PLycjWHRKQp931E3H+OV+uTqNyKte6SU9Xop+Ird9dxR0Eplnx7TNI5fC2U566PzYINB+qP8ZgsaYp3fLVMBSJigJw/OGp1aEGsmFr8IML6zHNbbdZfxQVZIZU0r3A6f/58WCyW+q+UlBSth0SkKrHc+Ni+7ZCZluj1l7han0TlLP0Ewxq8q+vo7Vo15G3GSc4YzOFGLNxw0Gn7MiAjWbLtSOD6fUCf+RC0zAs5tAj4JBEj9/fGV1kb0CXeebuwGtfMFVZIJVWDD6vV0eipuLjY6fHi4uL67zU2a9YslJWV1X+dOHFCzSERBR01y9xLXfoJ1jV4KTkh9w7uiOWTB2HrY8NUuYmqFqj9tiSz/dL9+POJR3DBbvZ5bIpVn0b6qYXI7jQRX/y+Ei/d3lfVa9YYWzmQqssuqampsFqt2LhxI/r27QvAsYzy3XffYcqUKS6fYzabYTZr+I+OSIfEoKFxDQRX7emlnMvb0k8wr8G7u1b+qhehdjv5UxVV+ORMFjaUD8SEhGzclvAfdDRrk2NjqC1H9/3j0b3/y0D8FUCtQVaxMjnU/Bmn4CM7+Dh37hwOHTpU/+eCggLk5uYiISEB7du3xyOPPIKnn34anTt3RmpqKmbPno22bdvixhtvVHPcRCFPSb6IO946zeppDV5JxUs1r5U3agdq4jUts8XgtV9/j9d+/T1iw84h1ngOyRHF+CjtiQD3kbEDO6dd/KMpCej8ANBtuuqByMieyRjWrQ3ezzmKY6Xn0SEhCndldoQpXPOMAPIz2cHH//73P1x99dX1f54xYwYA4O6778aSJUvw17/+FZWVlbjvvvtw9uxZDBkyBNnZ2YiMZOIQkVxS2tOrQW5yqr/4UvEyUNdK7UDN1bUvt7VEua0lfq614pMzwzAufpN2jexqTgM/PQ0ceBkY/KEjX0Ulrv6+/721gDMfzQDLqxMRgMB0KfU0qxHoUvBKe4p4a3ynpBy5t2v/36FL0O7sx6oXW5PPCPxunSoBSDCW/mcfGs/Y24WIFPFXrw2bXcArmw7inW1HnXq1iOcenmENaH8RX9+np2BBAHDP4I4YnmGVdXPyNKbhGVbc++Jr+FvCM0g1n9RuFgRwVG0dc8CnJRgt+8koxT403jH4ICLF1P50l51XiJmf7sHZ800bxIlnfSSrMxZs8N4DavnkQT4vraj1idvVzchoABpucpF7c3J37Rt3Sf5j0id41PouzEa7pPOqrs98IP0+oLYMiLA0CUS8/QzJ6fociKU0b4JxlkYLcu7fGpXZIyK9UjN3wt0vbZEAxy/wd7YdlXQ+X3fbeNsma4Bjm+zwDKusJNcN+UV4a9tRNN5dK7dAm7tr3/h9//v0OKw8cy3uSVqNuxLXISG8wuu5VfXj34AfZl38c8s0IOUmIP0BZB+L8jpDEEy7q9T8maGLmFJMRH7h6Zd2QwLgtBTjia+7bdSuZyIWOluf57qCs1oF2ly97zJbDBYU34l++cvR+6cVuO7Ayzjabo7i15BFaPT3de4wsPefwJp09PlfL9xkeg+WsIsBUePia3raXeVNsNbA0TsGH0TkF95+aTcW1yLC7xUv/fGJOxA3J28VQStsLXHWnIGUK+cAveZ5Dfj8KdlUgr8mv4dvuk7GVS13AmgahAVThdNgmqUJJgw+iMgv5P4ynjQ4FYB/K1764xO32jcnV12FZVUE7fUkDqS/hUqbtrMG8eHnsCR1DtanT0OKqdApCAumCqfBNEsTTJjzQUR+IeeXcbIlEtOGpaOrtaVfK176o56JmjcnbzsqpFYETb9sEkZ9HY9h4asxMWkN2kRosyRgMAAZUUexpetk/Hg+HbN+mYZTFX0BqFvh1J9bYPVSAyfUcLcLEfmFt3oYDU3P6oyOSdFoHROJ/h3isfPYGb/VUlC7noladT+k7qiQeqNt+D5Hxm7FcykLERum/dKAHWEwWq8BLl8ExKQ5vZ+kaDNgAE6fq5b8dx+ILbCBqIETCrjVloh0wd0vbVGUKQymcKPTNtxA1E5Q+4bl683JX3UvGr5PS1gF7klcjT+0Wo/4sHLJ5/AfA9Dr70CvJwEo+zsJ5BZY1vnwjsEHEemGq1/acVERGJKehHU/FmpWO8HXqfrGzz9TWY156/Yqujn5s+6Fy/dZdxY4dww4vQ3Y+TAg2GSdU1W95iHbcK/sIEKLQmV6q3Cqt/GwzgcR6Yarpm/9O8Tjqhe+1rR2gi/1TNx9Cp49OgPx0SbZNwN/7qhw+T5N8UBCPJDQ11GjY9vtjoJhGhD2/A0bi2wYFVuJgpp2yK9Kv/g9uP9ZULu7sBSB6h8kRbDPxDD4ICK/a/xLO+dwScBvHGpxN9VfVFaFqcscBcXG9m0n65ya7qhoOxK44QiwfyFw8DWg+nSjA8IA+G9mxAABL1ifqv+zIACFtYl44ucH8PW5TLc/C815C6ynn0E5Re20xK22RBRwwXrj8FbtElBWUEzzuhfmBKD334FxvwK3lALX7QZG7nb8d595/nlNNwwGoK2pBO90egaHe12PdzvORoqpsMnPQnPdAuuvn8FAY/BBRAEXrDcOfxUU01XdC1M8EN/XsSRjigfS7wfMSf5/XRfCDMBVsbuxuet9uLT8VafvaR6waSRUKq4y+CCigAvWG4c/Z2zEuhdWi3PAZbVEajuNbk4AMt+HYNBuld5oEJDy83xgz9P1j+kqYAugYJ01bIw5H0QUcOKNY8rSXfVt6EV6vnH4e8bGVXKu1jsYAABtR8Jw1RpU//dOmG0lmgzBAAB7/gZ0HA/EpAFQt1BZsAjWWcPGuNWWiDQTbBn7ahUUC1rVpdi/7Z+I/3kxWoc3TkwNEOsIYFi200N623LqT3r+GWSdDyIKGsF242C1S8ff2Q97dyFl/xQkVX3vdvnMLwwRwPiaQL6i7uj1Z5DBBxGRH+lpxkbz4G3P08Ce2YF7PcCxNbhlqvNj1aWOWiURFkeeSojT08+giMEHEZGfaX7Th45uQBWHga23Amd2Beb1rtkMtBnqCDgOvQEcWQJUHLj4/ZguQKeJv+3UCd1ARA8/gw0x+CAiCnGB7GsiWXUpkP8shENvwFDrx/4xNxwByvcDOXe5KIrWgDkJyHzfUUiN/E7O/ZtbbYmIgoxuC02ZE4BLn4fh92XYlv4Ftlf0QI09TN3XMEQ4Ao/NYzwHHoDj+5vHACezPR9HAcettkTU7Olt+tobLfqayDV4wEhkR/XBVWvyYTx/FG0jfkW0sRLvpD4Ngy+XttUQx4yHUCfteKEOyPkDcP2+kF6CCTYMPoioWdNN3oQMwVJoylXdEnvxMYQdfV/hGQ1AQj/g1Nfynlb9K3DoTaDHTIWvS2rjsgsRNVti3kTjWQSxQVd2XqFGI/MsmApNiU0Fx/Zth8y0RIRd8R7Q8Q8ul4y86vV34Jc1ygZy5B1lzyO/YPBBRM2SbvMmJAjW8vQi26AlmHTyVRypagtpWx4MQK95QJcHnXe1yFFxAKg5o+y5pDoGH0TULAVzg65g72uyo6AU35zugGEH3kTqnrWYeHg2fjrfETV25/HaDRGOiqZjDgK9nnTU8fBFzVnfnk+qYc4HETVLwZI34U4w9zVpfE2/qRyIbw4NBAC0iyhC24hfcbK2Ff46biTG9m138cAIi28vbIrz7fmkGgYfRNQsBVPehDu6bUTnhadr+kutFb/UWl0fZ05wFBBTsvQS0wUwxct/HvkFgw8iapbEvAlvDbr0mjchEhM6g4lP177TROCHx+W/aKdJ8p9DfsOcDyJqloI9byKY+XTt0+93VC6Vw9wKSL9P9jjJfxh8EFGzJeZNWC3O0/tWS2Sz6E6rJcXX3pzgKJlukDhxbwgHMt9Tr8BYdSlwrsDx/6QYe7sQUbMXbBVOQ4nia38y21G5tPpX98eYWzkCD197uzTzBnZSsbEcERGFvupSR+XSI++4CAomOZZafA0KTmZ7b2AXEQf0ngd0nNCsgxAGH0RE1LzUnHHU8TDFqber5WS2ozGd1D4yAGBKAjo/AHSb3uwCEXa1JSKi5sUUD7RMVS/wqC6V18BOVHMa+OlpYHUH4NhKdcYSghh8EBERNXboDc9LLd7UnQO23QqsugT4aT4TVBth8EFERNTYkSXqnOfCL466JGu7OpZxCACDDyIiImfVpcob2Lk952lH/ggDEAAMPoiIiJz52sDOHaHut+3BXIJh8EFERNSQrw3sPKn+1bE9uJlj8EFERNSQ2MDOX468479zBwkGH0RERI11mui/c1cccNQlacYYfBARETWmpIGdHDVn/XfuIMDgg4iIqDG5DezkMsX557xBgsEHERGRK21HAletcTSoU1NMF/UqsQYpBh9ERETutB0JXL8P6DMfaNFOnXN2mqTOeYIYgw8iIiJPzAlAj5nATT8Dgz8CwmN8OFcrR7fdZo7BBxERkVQdfg+MPQr0nC0/IdUQDmS+1+y63brC4IOIiEgOcwLQ++/AuF+BW0qBy14BIrzkcJhbOfJH2o5UdyzVpcC5gqCrmuqnNF4iIqJmwBQPdJkKdBjvqFx65B3nvjAxXRw5Hun3qTfjUV3q6Lp7ZImL15r42zZhfc+uGARBELQeREPl5eWwWCwoKytDbGys1sMhIiKSp+aMo46HKU79XS0ns4GcuxyN6twxJzm2Cas9y+KFnPs3l12IiIjUZIoHWqb6J/DYPMZz4AEERQddBh9ERER6V13qmPEQ6qQdr/MOugw+iIiI9O7QG95nPBpz10FXB0mqTDglIiLSuyNLFD7vHUeNEp0lqfpt5uPVV19Fx44dERkZiYEDB2LHjh3+eikiIqLQVV3qHDDIUXEAOLYSWNsV+OHxpuepOOB4fG3XgOaI+CX4+PDDDzFjxgzMmTMHu3btQp8+fTBixAicOnXKHy9HREQUumrLfHv+tvG6S1L1S/Dx4osvYvLkyZg0aRIyMjLw+uuvIyoqCm+//bY/Xo6IiCh0RVh8PIFN2mEBTFJVPfioqanBzp07kZWVdfFFjEZkZWUhJyenyfHV1dUoLy93+iIiIqLfmBMcuRmB4C5JVWWqBx+nT5+GzWZDmzZtnB5v06YNioqKmhw/f/58WCyW+q+UlBS1h0RERBTcOk0M3GsdecfvL6H5VttZs2ahrKys/uvEiRNaD4mIiEhf0u+X38hOqYoDjiqtfqR68JGUlISwsDAUFxc7PV5cXAyr1drkeLPZjNjYWKcvIiIiasCc4CiZbpBYIUPqce7UnPXt+V6oHnyYTCb0798fGzdurH/Mbrdj48aNyMzMVPvliIiImoe2Ix2dcc2tPB9nbgVcscy31zLF+fZ8L/xSZGzGjBm4++67cdlll2HAgAFYuHAhKisrMWnSJH+8HBERUfPQdiRw/T5pHXR/7KKsPkhMF/X70jTil+Djtttuw6+//oq//e1vKCoqQt++fZGdnd0kCZWIiIhkMic4qpb2mOm5g26niY4CYnJ18v9EgUEQBMHvryKDnJa8RERE5EZ1qaNyqZyeMOZWjpkVBaXW5dy/Nd/tQkRERH6gJEk1872A9Hhh8EFERBSq5CSpXrXGcXwAMPggIiIKZWKSap/5TSulxnRxPH79voAFHoCfEk6JiIhIR6QmqQYIgw8iIqLmxBSvWdAh4rILERERBRSDDyIiIgooBh9EREQUUAw+iIiIKKAYfBAREVFAMfggIiKigNLdVlux1Ux5ebnGIyEiIiKpxPu2lJZxugs+KioqAAApKSkaj4SIiIjkqqiogMVi8XiM7rra2u127N+/HxkZGThx4gQ72wZYeXk5UlJSeO0DjNddO7z22uB1146/rr0gCKioqEDbtm1hNHrO6tDdzIfRaES7du0AALGxsfyh1AivvTZ43bXDa68NXnft+OPae5vxEDHhlIiIiAKKwQcREREFlC6DD7PZjDlz5sBsNms9lGaH114bvO7a4bXXBq+7dvRw7XWXcEpEREShTZczH0RERBS6GHwQERFRQDH4ICIiooBi8EFEREQBpVnw8eqrr6Jjx46IjIzEwIEDsWPHDo/Hr1y5Et26dUNkZCR69eqF9evXB2ikoUfOtV+8eDGuvPJKxMfHIz4+HllZWV7/rsg1uT/zohUrVsBgMODGG2/07wBDmNxrf/bsWUydOhXJyckwm83o0qULf+coIPe6L1y4EF27dkWLFi2QkpKC6dOno6qqKkCjDQ1btmzBmDFj0LZtWxgMBnz22Wden/PNN9+gX79+MJvNSE9Px5IlS/w+TggaWLFihWAymYS3335b+Omnn4TJkycLcXFxQnFxscvjt23bJoSFhQnPP/+8kJ+fLzz55JNCRESEsGfPngCPPPjJvfYTJkwQXn31VWH37t3C3r17hYkTJwoWi0X4+eefAzzy4Cb3uosKCgqEdu3aCVdeeaUwduzYwAw2xMi99tXV1cJll10mjBo1Sti6datQUFAgfPPNN0Jubm6ARx7c5F73Dz74QDCbzcIHH3wgFBQUCF9++aWQnJwsTJ8+PcAjD27r168XnnjiCeHTTz8VAAirVq3yePyRI0eEqKgoYcaMGUJ+fr7wr3/9SwgLCxOys7P9Ok5Ngo8BAwYIU6dOrf+zzWYT2rZtK8yfP9/l8bfeeqswevRop8cGDhwo3H///X4dZyiSe+0bq6urE2JiYoR3333XX0MMSUque11dnXDFFVcI//73v4W7776bwYdCcq/9a6+9JnTq1EmoqakJ1BBDktzrPnXqVGHYsGFOj82YMUMYPHiwX8cZyqQEH3/961+FHj16OD122223CSNGjPDjyAQh4MsuNTU12LlzJ7KysuofMxqNyMrKQk5Ojsvn5OTkOB0PACNGjHB7PLmm5No3dv78edTW1iIhIcFfwww5Sq/73//+d7Ru3Rr33ntvIIYZkpRc+88//xyZmZmYOnUq2rRpg549e+If//gHbDZboIYd9JRc9yuuuAI7d+6sX5o5cuQI1q9fj1GjRgVkzM2VVvfXgDeWO336NGw2G9q0aeP0eJs2bbBv3z6XzykqKnJ5fFFRkd/GGYqUXPvGHnvsMbRt27bJDyu5p+S6b926FW+99RZyc3MDMMLQpeTaHzlyBJs2bcIdd9yB9evX49ChQ3jwwQdRW1uLOXPmBGLYQU/JdZ8wYQJOnz6NIUOGQBAE1NXV4YEHHsDjjz8eiCE3W+7ur+Xl5bhw4QJatGjhl9flbheS7Nlnn8WKFSuwatUqREZGaj2ckFVRUYG77roLixcvRlJSktbDaXbsdjtat26NN998E/3798dtt92GJ554Aq+//rrWQwtp33zzDf7xj39g0aJF2LVrFz799FOsW7cO8+bN03po5AcBn/lISkpCWFgYiouLnR4vLi6G1Wp1+Ryr1SrreHJNybUX/fOf/8Szzz6LDRs2oHfv3v4cZsiRe90PHz6Mo0ePYsyYMfWP2e12AEB4eDj279+PtLQ0/w46RCj5mU9OTkZERATCwsLqH+vevTuKiopQU1MDk8nk1zGHAiXXffbs2bjrrrvwxz/+EQDQq1cvVFZW4r777sMTTzwBo5Gflf3B3f01NjbWb7MegAYzHyaTCf3798fGjRvrH7Pb7di4cSMyMzNdPiczM9PpeAD46quv3B5Prim59gDw/PPPY968ecjOzsZll10WiKGGFLnXvVu3btizZw9yc3Prv2644QZcffXVyM3NRUpKSiCHH9SU/MwPHjwYhw4dqg/4AODAgQNITk5m4CGRkut+/vz5JgGGGAAKbEHmN5rdX/2azurGihUrBLPZLCxZskTIz88X7rvvPiEuLk4oKioSBEEQ7rrrLmHmzJn1x2/btk0IDw8X/vnPfwp79+4V5syZw622Csm99s8++6xgMpmEjz/+WCgsLKz/qqio0OotBCW5170x7nZRTu61P378uBATEyNMmzZN2L9/v7B27VqhdevWwtNPP63VWwhKcq/7nDlzhJiYGGH58uXCkSNHhP/85z9CWlqacOutt2r1FoJSRUWFsHv3bmH37t0CAOHFF18Udu/eLRw7dkwQBEGYOXOmcNddd9UfL261ffTRR4W9e/cKr776auhutRUEQfjXv/4ltG/fXjCZTMKAAQOE7du313/vqquuEu6++26n4z/66COhS5cugslkEnr06CGsW7cuwCMOHXKufYcOHQQATb7mzJkT+IEHObk/8w0x+PCN3Gv/7bffCgMHDhTMZrPQqVMn4ZlnnhHq6uoCPOrgJ+e619bWCk899ZSQlpYmREZGCikpKcKDDz4onDlzJvADD2Jff/21y9/Z4rW+++67hauuuqrJc/r27SuYTCahU6dOwjvvvOP3cRoEgfNZREREFDjM4CEiIqKAYvBBREREAcXgg4iIiAKKwQcREREFFIMPIiIiCigGH0RERBRQDD6IiIgooBh8EBERUUAx+CAiIqKAYvBBREREAcXgg4iIiAKKwQcREREF1P8DtADvW+jIQDAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss =  38.60055\n"
     ]
    }
   ],
   "source": [
    "from IPython.display import clear_output\n",
    "\n",
    "epochs = 1000 # число эпох \n",
    "\n",
    "w = torch.randn(1, requires_grad=True)\n",
    "b = torch.randn(1, requires_grad=True)\n",
    "\n",
    "for i in range(epochs):\n",
    "\n",
    "    y_pred = w * x + b\n",
    "    loss = torch.mean((y_pred - y)**2)\n",
    "    loss.backward()\n",
    "\n",
    "    # делаем шаг градиентного спуска с lr = .05\n",
    "    w.data -= 0.05 * w.grad.data \n",
    "    b.data -= 0.05 * b.grad.data\n",
    "\n",
    "    # обнуляем градиенты, чтобы на следующем шаге опять посчитать и не аккумулировать их\n",
    "    w.grad.data.zero_()\n",
    "    b.grad.data.zero_()\n",
    "\n",
    "    # рисуем картинки\n",
    "    if (i+1) % 5 == 0:\n",
    "        clear_output(True)\n",
    "        plt.scatter(x.data.numpy(), y.data.numpy())\n",
    "        plt.scatter(x.data.numpy(), y_pred.data.numpy(),\n",
    "                    color='orange', linewidth=5)\n",
    "        plt.show()\n",
    "\n",
    "        print(\"loss = \", loss.data.numpy())\n",
    "        if loss.data.numpy() < 0.5:\n",
    "            print(\"Done!\")\n",
    "            break\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Задание 6:\n",
    "\n",
    "Реализуйте для функции \n",
    "\n",
    "$$\n",
    "f(w) = \\prod_{i,j} \\ln(\\ln(w_{ij} + 7) \n",
    "$$\n",
    "\n",
    "процедуру градиентного спуска. Каким получилось минимальное значение? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[4.9900, 9.9948],\n",
      "        [0.9775, 1.9825]], requires_grad=True)\n"
     ]
    }
   ],
   "source": [
    "w = torch.tensor([[5., 10.], [1., 2.]], requires_grad=True)\n",
    "alpha = 0.001\n",
    "\n",
    "for _ in range(500):\n",
    "    # it's critical to calculate function inside the loop:\n",
    "    function = (w + 7).log().log().prod()\n",
    "    function.backward()\n",
    "    w.data -= alpha * w.grad\n",
    "    w.grad.zero_()\n",
    "\n",
    "print(w)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. Моя первая нейросеть \n",
    "\n",
    "Для того, чтобы разобраться как обучать нейросетки, нужно освоить три вещи: \n",
    "\n",
    "1. Как обрабатывать поток данных и пихать его в сетку\n",
    "2. Как сделать сетку\n",
    "3. Как написать цикл обучения"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.1 Как формировать батчи и пихать их в сетку\n",
    "\n",
    "С данными помогают работать две абстракции: `Dataset` и `DataLoader`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Dataset` умеет выдавать по индексу некоторый элемент и помогает итерироваться по данным. Чтобы работать с данными и применять к ним преобразования, например, аугментации, о которых вы узнаете позже — нужно создать свой класс унаследованный от `torch.utils.data.Dataset`.\n",
    "\n",
    "Вот пример из документации:\n",
    "\n",
    "```\n",
    "class FaceLandmarksDataset(torch.utils.data.Dataset):\n",
    "    \"\"\"Face Landmarks dataset.\"\"\"\n",
    "\n",
    "    def __init__(self, csv_file, root_dir, transform=None):\n",
    "        \"\"\"\n",
    "        Args:\n",
    "            csv_file (string): Path to the csv file with annotations.\n",
    "            root_dir (string): Directory with all the images.\n",
    "            transform (callable, optional): Optional transform to be applied\n",
    "                on a sample.\n",
    "        \"\"\"\n",
    "        self.landmarks_frame = pd.read_csv(csv_file)\n",
    "        self.root_dir = root_dir\n",
    "        self.transform = transform\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.landmarks_frame)\n",
    "\n",
    "    def __getitem__(self, idx):\n",
    "        if torch.is_tensor(idx):\n",
    "            idx = idx.tolist()\n",
    "\n",
    "        img_name = os.path.join(self.root_dir,\n",
    "                                self.landmarks_frame.iloc[idx, 0])\n",
    "        image = io.imread(img_name)\n",
    "        landmarks = self.landmarks_frame.iloc[idx, 1:]\n",
    "        landmarks = np.array([landmarks])\n",
    "        landmarks = landmarks.astype('float').reshape(-1, 2)\n",
    "        sample = {'image': image, 'landmarks': landmarks}\n",
    "\n",
    "        if self.transform:\n",
    "            sample = self.transform(sample)\n",
    "\n",
    "        return sample\n",
    "```\n",
    "\n",
    "Как вы видите, у такого класса должно быть два метода: \n",
    "\n",
    "* `__len__` — возвращает информацию о том, сколько объектов у нас в датасете\n",
    "* `__getitem__` — возвращает семпл и таргет к нему\n",
    "\n",
    "Теперь давайте напишем такой сами, в качестве датасета сгенерируем рандомные данные."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "class RandomDataset(torch.utils.data.Dataset):\n",
    "    \"\"\"Our random dataset\"\"\"\n",
    "    \n",
    "    def __init__(self, x, y):\n",
    "        self.x=x\n",
    "        self.y=y\n",
    "        \n",
    "    def __len__(self):\n",
    "        return len(self.x)\n",
    "    \n",
    "    def __getitem__(self, idx):\n",
    "        return {'sample': torch.tensor(x[idx, :], dtype=torch.float), 'target': y[idx]}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = np.random.rand(1000, 5)\n",
    "y = np.random.rand(1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "our_dataset = RandomDataset(x, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'sample': tensor([0.1201, 0.6425, 0.5698, 0.3123, 0.7891]),\n",
       " 'target': 0.08797364268406249}"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "our_dataset.__getitem__(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Dataset` умеет говорить сколько в выборке объектов и брать из неё очередной. Этого мало. Данные хочется как-то видоизменять. Для этого есть такая сущность как `DataLoader`.\n",
    "\n",
    "Он принимает на вход класс унаследованный от `torch.utils.data.Dataset` и преобразовывает его."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataloader = torch.utils.data.DataLoader(our_dataset, batch_size=4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Работают с ним следующим образом:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sample: tensor([[0.5543, 0.6750, 0.4421, 0.6813, 0.5724],\n",
      "        [0.1201, 0.6425, 0.5698, 0.3123, 0.7891],\n",
      "        [0.2113, 0.8836, 0.1088, 0.0501, 0.3684],\n",
      "        [0.4632, 0.4599, 0.6145, 0.0718, 0.7054]])\n",
      "Target: tensor([0.0595, 0.0880, 0.8279, 0.9854], dtype=torch.float64)\n"
     ]
    }
   ],
   "source": [
    "for batch in dataloader:\n",
    "    batch_x = batch['sample']\n",
    "    batch_y = batch['target']\n",
    "    break\n",
    "    \n",
    "print('Sample:', batch_x)\n",
    "print('Target:', batch_y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.2 Нейронная сеть\n",
    "\n",
    "Есть несколько способов собрать нейросетку."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Способ 1:__ Для того, чтобы в high-level pytorch создавать нейросети используется модуль `nn`. Нейросеть должна быть унаследована от класса `nn.Module`. Пример как это может выглядеть:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch import nn\n",
    "import torch.nn.functional as F\n",
    "\n",
    "class Model(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Model, self).__init__()\n",
    "        self.layer1 = nn.Linear(5, 3)\n",
    "        self.layer2 = nn.Linear(3, 1)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = F.relu(self.layer1(x))\n",
    "        return F.relu(self.layer2(x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Как мы видим на данном примере, у данного класса должно быть метод `forward`, который определяет прямой проход нейросети. Также из класса выше видно, что модуль `nn` содержит в себе реализацию большинства слоев, а модуль `nn.functional`—  функций активаций."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.],\n",
       "        [0.],\n",
       "        [0.],\n",
       "        [0.]], grad_fn=<ReluBackward0>)"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = Model()\n",
    "model(batch_x) # получили предсказания модели"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Если вы не очень поняли, что произошло, не пугайтесь. На следующих семинарах мы более подробно будем говорить про объектно-ориентированное программирование и научимся писать подобные модели. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Способ 2:__ Можно собрать модель с помощью класса `Sequential`. В его рамках сбор модели будет выглядить как строительство башни из конструктора LEGO. Модель описывается последовательно. Мы как бы создаём коробочку `model` и постепенно добавляем туда детальки нашей сетки. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = nn.Sequential()                 # создаем пустую модель, в которую будем добавлять слои\n",
    "model.add_module('l1', nn.Linear(5, 3)) # добавили слой с 5-ю нейронами на вход и 3-мя на выход\n",
    "model.add_module('l2', nn.ReLU())       # добавили функцию активации\n",
    "model.add_module('l3', nn.Linear(3, 1)) # добавили слой с 3-мя нейронами на вход и 5-ю на выход"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.1964],\n",
       "        [0.2025],\n",
       "        [0.2482],\n",
       "        [0.2245]], grad_fn=<AddmmBackward0>)"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model(batch_x) # получили предсказания модели"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Способ 3:__ Описать модель функционально, в явном виде прописав какие аргументы идут на вход какому слою. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.2817],\n",
       "        [0.2817],\n",
       "        [0.2817],\n",
       "        [0.2817]], grad_fn=<AddmmBackward0>)"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "h = nn.Linear(5, 3)(batch_x)\n",
    "h = nn.ReLU()(h)\n",
    "out = nn.Linear(3, 1)(h)\n",
    "out"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Обычно модель объявляют в виде класса, а второй и третий способ используют, чтобы вычисления внутри него выглядели более красиво."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.3 Цикл для обучения модели\n",
    "\n",
    "Осталось научиться писать циклы для обучения. Давайте подгрузим настоящие данные, соберём модель и обучим её."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torchvision.datasets import MNIST\n",
    "from torch.utils.data import Dataset, DataLoader\n",
    "\n",
    "# внутри модуля transforms есть много полезных преобразований\n",
    "# пока что нам из него понадобится только T.ToTensor() \n",
    "# для конвертации картинки в тензор \n",
    "import torchvision.transforms as T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Не будем выпендриваться и как все обучим [наборе рукопистных цифр MNIST.](http://yann.lecun.com/exdb/mnist/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_set = MNIST('.MNIST', transform=T.ToTensor(), train=True, download=True)\n",
    "test_set = MNIST('.MNIST', transform=T.ToTensor(), train=False, download=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`train_set` в данном случае это датасет. Давайте нарисуем несколько изображений. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABPAAAAGCCAYAAABuET1uAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/P9b71AAAACXBIWXMAAA9hAAAPYQGoP6dpAAA7YUlEQVR4nO3dd5hV1b0//jUUlSCCoKhIsUTEigVFQINEojexYIOINZbovWIBY40xYCcaozEoGmMwKhZiCUKuBbkCUawQVOIVS4xKESXSRUVmfn/cX/xmn7X1HI5nZu+Zeb2eJ8+T9X7W3vMhWe4z5+NmraqampqaAAAAAADkUpOsCwAAAAAAvpoGHgAAAADkmAYeAAAAAOSYBh4AAAAA5JgGHgAAAADkmAYeAAAAAOSYBh4AAAAA5JgGHgAAAADkWLNyL6yurg7z588PrVq1ClVVVZWsiXqopqYmLF++PHTo0CE0aVJ7fWHrjn9n3ZGFulp3IVh7JHnmkQXrjiz4rCUrnnlkodR1V3YDb/78+aFTp07lXk4D9f7774eOHTvW2v2tO9JYd2ShttddCNYe6TzzyIJ1RxZ81pIVzzyyUGzdld1SbtWqVbmX0oDV9rqw7khj3ZGFulgX1h5pPPPIgnVHFnzWkhXPPLJQbF2U3cDzmidpantdWHekse7IQl2sC2uPNJ55ZMG6Iws+a8mKZx5ZKLYuHGIBAAAAADmmgQcAAAAAOaaBBwAAAAA5poEHAAAAADmmgQcAAAAAOaaBBwAAAAA5poEHAAAAADmmgQcAAAAAOaaBBwAAAAA5poEHAAAAADmmgQcAAAAAOdYs6wIAyIftt98+Mf7BD34QzRkwYECUtW3bNjFu0iT+d0N9+/aNsg8//HBtSwQAAOrQvvvuG2VPPfVUSddWVVVVuJr/Z8SIESVlDYk38AAAAAAgxzTwAAAAACDHNPAAAAAAIMc08AAAAAAgxxxiAdAI/ehHP4qyW265JTFeZ511KvbzHn300ShLOyRj4cKFFfuZ1F89evSIsmHDhkXZMcccUxfl0Eh07do1yqZPnx5lCxYsSIx32mmnWqsJ1sbgwYOj7Morr4yyGTNmJMYDBw6stZqoXy666KIou+KKKxLjO++8M5pz4okn1lpN1L3CQytKPbBiypQplS9mLaUduJGHuirFG3gAAAAAkGMaeAAAAACQYxp4AAAAAJBjGngAAAAAkGMOsfg3Rx55ZJT9/Oc/T4x32GGHku7VpEmyN/ruu+9Gcx5++OEoGzVqVJS9/fbbJf1MgO7du0fZyJEjo2ybbbaJslIOrVi2bFmUbbDBBkWva9GiRZQtWrSo6HU0DptttlliPH78+GjOSy+9VFfl0Eh16NAhytq0aRNlhYdYQBbOP//8KLv00kujLO2zvdQN6SGEEGpqar52TMOTdhBEobSDIfr161f5YtbS8OHDo8whFgAAAABAndDAAwAAAIAc08ADAAAAgBxrtHvg9ejRI8puvfXWKGvdunViXOrf+a+urk6MO3bsGM0588wzo+y0006LsjvvvDMxvuCCC6I5S5cuLakusvXDH/4wyk466aSy7vXEE09E2Zw5c4peN3PmzCibP39+WTWQP2l72x1wwAFl3euFF16IsqFDh0bZMccckxgPGTIkmrP++utHWdu2baPso48+WosKaSjWXXfdxLhwTzzIk2233TYx7t27dzRn+vTpdVUOjUTTpk0T46OOOiqak7bf3TPPPBNlP/vZzypXGI1O2r7xV155ZZTZx73+6tu3b9E5edjvLk0p+/fVZ97AAwAAAIAc08ADAAAAgBzTwAMAAACAHNPAAwAAAIAcaxSHWLRv3z7K7r333igrPLAizXvvvRdlkyZNirKqqqrEuH///tGczp07R1na5rOnnHJKYvz8889Hc8aMGRMXS50q/P/8wgsvjOacd955UdamTZuyfl7amipWUwghLFq0KMpefPHFKBs4cGBi/Mknn6xFdWTl008/jbK0Q25WrFgRZd///vcT4zfffDOa89lnn0XZF198kRinHWKRdpDPdtttF2UOsSCEEGbNmhVlaeuxcF3NnTu3tkqiEfjb3/4WZfPmzYuywnXXvHnzWqsJ/uWSSy5JjLt3717SdePHj4+yBQsWVKQmGqdvfetbUdasWaNoKzQaDe0giBEjRnztuD7xBh4AAAAA5JgGHgAAAADkmAYeAAAAAOSYBh4AAAAA5Fij2G1yhx12iLItt9wyympqaqLsvvvuS4wvvfTSaM5bb71VtIZ11103yo4++ugo+/Wvfx1lhRuF3nLLLdGcyZMnR1nagRtURtrhEAMGDEiMr7zyymhO2gESN954Y5QdeOCBiXGHDh2iOccee2zROrt06RJlZ555ZpQVHl4QQggLFy5MjK+55ppozu233x5l8+fPL1oXtWfixIlRtuuuu0bZP/7xj4r9zLR1Bt/ElClTouzOO++MslWrVtVBNTQWaYfoLFu2rOh1O+64Y5RNnTq1IjXBv5TyWVtdXR1lr7zySm2UQwO11VZbZV0Cdaw+H1iR9vvi8OHDi2YOsQAAAAAAaoUGHgAAAADkmAYeAAAAAORYg9wDr3nz5onxBRdcUNJ1N998c5SdffbZFanps88+i7IxY8ZE2ZtvvhllhX+3u2nTptGcH//4x1F2ySWXrEWFrI1BgwZF2b333psYz5s3L5pTuLddCOl7k+y+++6Jcbt27aI5Dz/8cNE609xxxx1RlrZH2rBhwxLjtP0fTz311KLZo48+upYVUmmV3O9u/fXXj7K0NVUobV+eL774ohIl0QAMHDgwMR46dGg0Z+utt46ywr1HoS4U7oO7zz77RHNuuummuiqHRiJtP+1CM2fOjLJJkybVRjk0UCeddFKUpe0TT8PR0PbAS8sK/4xpe+DVl33xvIEHAAAAADmmgQcAAAAAOaaBBwAAAAA5poEHAAAAADnWIA+x+M///M/EuH///iVdd8UVV9RGOWvl+eefj7LCjZCHDBkSzenatWut1dTYNWsW/2Py85//PMoWLVqUGJd6YEXaoSSF2YwZM4rWWaolS5ZE2VNPPRVl06ZNS4zTNrVNO9ji7rvvToz79OkTzXn99deLlUlOpa2Dli1bFr1u1KhRUTZ9+vSK1ET9973vfS/rEqBkNnSntm2xxRZRdtRRRyXGaevwpZdeqq2S4EtphwTMnz+/7guBFFOnTo2y+nxQRyFv4AEAAABAjmngAQAAAECOaeABAAAAQI5p4AEAAABAjtX7QyyaN28eZQcddFDR68aNGxdl//znPytS0zexevXqKPvlL3+ZGB922GHRnMMPPzzKjjnmmMR47Nix37C6xqnwf/8QQthuu+2i7MYbb0yM0w6sSLPTTjtFWbdu3RLjUtZ0pa1ZsyYxvu2226I5ixcvjrLCf7YmTZoUzenRo0eULVy4cG1LpJY1aRL/O560Q1dKceWVV37TcmjA8vD5C2kHB2y88cZ1XwiN3vnnn1/WdWmH4cFXGThwYFnXvffee1G2fPnyb1oOVMSIESOibPjw4V87/qrr8sgbeAAAAACQYxp4AAAAAJBjGngAAAAAkGMaeAAAAACQY/X+EItzzjknyvbbb7/EeNWqVdGcq666Ksqqq6srV1gFvf/++4nx3LlzozkdOnSIsvXWW6/WamrIOnfunBgff/zx0ZwvvvgiykaOHFnWz3vrrbeirFevXonxnDlzyrp3bXvggQei7IorrkiMf/azn0Vzrr766ig76aSTKlcYFXHEEUdE2XXXXVf0ukWLFkVZ2oEn8C8XXXRRYnzUUUdFcwoP90nLXn/99coWRqOSdojFRhttVPS6efPm1UI1NGZdu3YtOqfw+0EIITz11FO1UQ4N1C677JJ1CZAbaYdY5PFgC2/gAQAAAECOaeABAAAAQI5p4AEAAABAjtX7PfCOPvroonNeeeWVKJs9e3ZtlJMrhx56aGJ8++23Z1NIPfOTn/wkMW7Tpk00Z/To0VH2wQcflPXzVqxYEWV53fOuFIXr7MILL4zm/OhHP4oye+Blb911102Mzz333LLuc80110RZ2r6R8FVqamqibJtttomyI488MjEu3IMT6sIjjzySdQnUYxtvvHGUlbI32a233hplafviAXydKVOmRNnw4cPrvpBa1K9fv8Q4bb/Qvn371lU534g38AAAAAAgxzTwAAAAACDHNPAAAAAAIMc08AAAAAAgx+r9IRZ8tU6dOmVdQr101llnJcZpm6nbsPqrvfvuu4nx448/Hs058MAD66oc1sKIESMS4z322KOk65588snE+LrrrqtUSTQSa9asSYxXrlwZzWnZsmWUDRkyJDF2iAWVVlVVFWVNmvj331TOscceG2Vt27aNssK1OH369FqriYap8LCyXr16RXPSnm/V1dWJcdpzkfor7RCLUuy7774Vu1dtK6WutD9PHvkNBAAAAAByTAMPAAAAAHJMAw8AAAAAckwDDwAAAAByrN4fYpG2iWZhtmzZsroqp06U8mcOIYRp06bVRTmN0pw5c7IuIbc6dOiQGO+9994ZVcLXSds0e+jQoUWve/DBB6PszDPPTIzTDn6Br7N8+fLEeMaMGdGc73znO3VVDo3UgAEDoiztebZkyZLE+KOPPqqtkmgETjvttChLW3d//etfv3YMxRQeYpH2uVp4YEUI8Xr0e17DV3joQ9oBD8OHDy96XV6l1ekQCwAAAADgG9PAAwAAAIAc08ADAAAAgByr93vgvfvuu1G2/fbbJ8arV6+O5rRu3TrKli5dWrnCatF7770XZXvssUeUffzxx3VRDiT07ds3MU77Z+3Xv/51XZVDCKFr165R9vvf/z7KmjUr/pHw0EMPRdkHH3xQXmHw/yvcl6dTp04ZVUJjVriH61eZO3duYvzaa6/VRjk0ULvssktiXOrz7sYbb0yMG9oe39S+Cy64oCL3mTVrVkXuQ35NnTo1MU7bH66+7BmXpvDPF0Jpf8Y87PHnDTwAAAAAyDENPAAAAADIMQ08AAAAAMgxDTwAAAAAyLF6f4jF+PHjo+z73/9+YvyDH/wgmtOtW7coe/755ytXWC3605/+FGVHHHFElH3yySd1UA2N2WabbRZll1xySWK8ZMmSaM6vfvWr2iqp0UvbhP3BBx+MslIOrJgwYUJJWbn23nvvKGvXrl1inPb83mCDDaJs8ODBFauLurdw4cLE+J577onmXHzxxVHWpk2bxPioo46K5tx3333frDiACrrssssS4/XWW6+k6woPT4Gvs8kmm0TZaaedVta9xo4dmxjffPPNZd2Hhm/EiBElZZTPG3gAAAAAkGMaeAAAAACQYxp4AAAAAJBjGngAAAAAkGP1/hCLl156KcpWr16dGDdv3jya06JFi1qrqdIK6z/rrLOiOWkHVjgooDz/+7//mxinHXiStlH6yJEja62mvLrwwgujrGPHjonxYYcdFs15//33a62mxm6//faLsh122CHKqqqqoqympiYx/uyzz6I555xzTpT17ds3MU77ZyZN4YEVIYSwzjrrFL1u0aJFJd2f+uuZZ56JslWrVkVZ4Wf59ttvX2s1AVRCv379is554oknomzy5Mm1UQ4NVNphZYUHP5Wq8EC6L774oqz7UH8UHjxR+Lt+CCHsu+++UTZ8+PAomzJlyteOs5D250mTh1oLeQMPAAAAAHJMAw8AAAAAckwDDwAAAAByTAMPAAAAAHKs3h9iMWvWrCibMGFCYnz44YdHc37/+99H2S677BJly5YtK7u2Srn44osT4z322COa8/bbb0eZDUbLc/nllyfGY8eOjeYMGDAgyn7zm98kxitXrqxsYXWsa9euifEDDzwQzUk7HOG2225LjG26XLdOOOGEkuYVHliR5sgjjywpq2sbbLBBlBXWlbZeqT8ee+yxKJs5c2aU9enTJzHu3LlzNGe99daLsk8//fQbVEdjknbgD6Rp0iR+L+InP/lJlBU+k6qrq6M5V155ZeUKg29o3rx5WZdAxtIO3ynlu0QIITz11FOJcRafq4WHcqQdwFFfeAMPAAAAAHJMAw8AAAAAckwDDwAAAAByrN7vgZdm4sSJiXHaHnhp++Qcf/zxUTZq1KjKFVaC7t27R9l+++2XGL/77rvRnIMPPrjWampsCtfPyy+/HM3p2bNnlI0fPz4xHjp0aDRn9uzZ36y4Cthxxx2j7KyzzoqyE088MTFu2rRpNCdtL8nTTz/9G1TH2tpiiy0S4169emVTSC2ZM2dOlC1atCjKvvWtb9VFOWRo8ODBUVa4H2nhXqQhxPuahpC+byyNT+FeZB07dozmlLrHD+y6665RNnLkyKLXPf300yVlsDY23njjit1r9OjRFbsXDUfavniF+92lSftcnTJlSpRNnTq1rLqGDx9e1nX1hTfwAAAAACDHNPAAAAAAIMc08AAAAAAgxzTwAAAAACDHGuQhFo899lhivGrVqmhOixYtouyqq66Ksm222SYxvuCCC6I5n376adGa0g4ASKth0qRJUda2bdvE+K677ormvPHGG0VroDTLly9PjIcNGxbNSVsr3/3udxPjZ555Jppz2WWXRdmf//znKHv99deL1ll4uEkI8Xo99dRTozlbbbVVlG2wwQZRtnLlysT46KOPjuakrdfq6uq4WGrNxx9/nBj//e9/j+bssMMOZd174cKFUbbJJptEWSkHAjz88MNRlvZsXn/99RPjX/3qV0XnhJB+2AUNy9y5c6Os8PPw2GOPjebcfPPNUXbAAQdUrjDqrcLPvrQDqtKkPbtoXFq2bBllab+fl+K00077puXQyHXo0CHKxo0bV9a9Cr8HheB3e9KlHTxR7sEW++67b0lZpaTVfumll9baz6skb+ABAAAAQI5p4AEAAABAjmngAQAAAECOaeABAAAAQI41yEMsPvzww8R46623juY88sgjUdajR48oGzJkSGJ88MEHR3OefPLJKPvLX/6SGK+zzjrRnFtvvTXK0rz44ouJcdpBGtSetE0u+/btG2U///nPE+Mf//jH0Zxrr702yn7xi19EWSmbxaYdjFJVVZUYL1myJJrz6quvRtlDDz0UZbfccktibNPufFq2bFliPGrUqGjO6NGjo6zwsJ8Q4sNwfvnLX0ZzCg/VCSH9cIFCixcvjrKampqi18HXKVz///3f/x3N2XvvveuqHBqJMWPGZF0CGSs8NCyEELbddtuSri085Czt8ClYG2kHOKUdWleKm266KcoKD7aDr5L2vbnw++mIESPKunfa9+80U6dOjbJyf2YeeQMPAAAAAHJMAw8AAAAAckwDDwAAAAByrEHugVeocE+8EEIYMGBAlE2cODHKdt1118S4c+fO0ZyTTjopyk4++eTEuNS9nmbOnBllhx56aGKc9uehbq1evTrKLrnkksQ4bQ+JU089Ncp69+5dsbruv//+xLhwL8YQQnjrrbcq9vPIn7S9NUvdb7MUpex3B3ly5513Zl0COVW4t+vbb78dzUnbR/lvf/tbrdVE/fDxxx+XlKXtG3v88ccnxp9//nnlCqNR6tmzZ8Xu9cQTT1TsXpCmIe1HlwVv4AEAAABAjmngAQAAAECOaeABAAAAQI5p4AEAAABAjlXVlHq6QoFly5aF1q1bV7qeTLVp0ybKjjzyyMT4oIMOiuakZVVVVYlx2v/MY8eOjbKf/OQnUbZo0aIoy6ulS5eGDTbYoNbu3xDXHd+cdUcWanvdhWDtkc4zjyxYd2TBZy1Z8cwjC8XWnTfwAAAAACDHNPAAAAAAIMc08AAAAAAgxzTwAAAAACDHmmVdQJ4sWbIkyn73u9997RgAAAAAapM38AAAAAAgxzTwAAAAACDHNPAAAAAAIMc08AAAAAAgxzTwAAAAACDHNPAAAAAAIMc08AAAAAAgxzTwAAAAACDHNPAAAAAAIMc08AAAAAAgxzTwAAAAACDHNPAAAAAAIMfKbuDV1NRUsg4aiNpeF9Ydaaw7slAX68LaI41nHlmw7siCz1qy4plHFoqti7IbeMuXLy/3Uhqw2l4X1h1prDuyUBfrwtojjWceWbDuyILPWrLimUcWiq2LqpoyW7/V1dVh/vz5oVWrVqGqqqqs4mg4ampqwvLly0OHDh1Ckya19zezrTv+nXVHFupq3YVg7ZHkmUcWrDuy4LOWrHjmkYVS113ZDTwAAAAAoPY5xAIAAAAAckwDDwAAAAByTAMPAAAAAHJMAw8AAAAAckwDDwAAAAByTAMPAAAAAHJMAw8AAAAAckwDDwAAAAByTAPvG7jpppvCFltsEdZbb73Qs2fP8MILL2RdEo2EtUdduvrqq8Mee+wRWrVqFdq3bx8OPfTQMGfOnKzLohGYNm1aOPjgg0OHDh1CVVVV+NOf/pR1STQC1h1ZsO7IirVHVnynXXsaeGW6//77wznnnBOGDx8eZs6cGbp37x4OOOCA8OGHH2ZdGg2ctUddmzp1ahgyZEh47rnnwqRJk8Lq1avD/vvvH1auXJl1aTRwK1euDN27dw833XRT1qXQiFh3ZMG6IyvWHlnwnbY8VTU1NTVZF1Ef9ezZM+yxxx5h1KhRIYQQqqurQ6dOncKZZ54ZLrzwwoyroyGz9sjaRx99FNq3bx+mTp0avvOd72RdDo1EVVVVePjhh8Ohhx6adSk0ItYdWbDuyIq1R13xnbY83sArw+effx5mzJgR+vfv/2XWpEmT0L9///Dss89mWBkNnbVHHixdujSEEELbtm0zrgQAAKhPfKctnwZeGRYtWhTWrFkTNtlkk0S+ySabhA8++CCjqmgMrD2yVl1dHYYOHRr69OkTdtxxx6zLAQAA6hHfacvXLOsCAKg/hgwZEmbPnh2efvrprEsBAABoNDTwyrDRRhuFpk2bhoULFybyhQsXhk033TSjqmgMrD2ydMYZZ4SJEyeGadOmhY4dO2ZdDgAAUM/4Tls+f4W2DOuss07Yfffdw+TJk7/Mqqurw+TJk0OvXr0yrIyGztojCzU1NeGMM84IDz/8cPif//mfsOWWW2ZdEgAAUA/5Tls+b+CV6ZxzzgknnHBC6NGjR9hzzz3DDTfcEFauXBlOPPHErEujgbP2qGtDhgwJ99xzTxg/fnxo1arVl3tTtG7dOrRo0SLj6mjIVqxYEd56660vx++8806YNWtWaNu2bejcuXOGldGQWXdkwbojK9YeWfCdtjxVNTU1NVkXUV+NGjUqXHvtteGDDz4Iu+yyS7jxxhtDz549sy6LRsDaoy5VVVWl5mPGjAk/+tGP6rYYGpUpU6aEfv36RfkJJ5wQ7rjjjroviEbBuiML1h1ZsfbIiu+0a08DDwAAAAByzB54AAAAAJBjGngAAAAAkGMaeAAAAACQYxp4AAAAAJBjGngAAAAAkGMaeAAAAACQYxp4AAAAAJBjGngAAAAAkGMaeAAAAACQYxp4AAAAAJBjGngAAAAAkGMaeAAAAACQYxp4AAAAAJBjGngAAAAAkGMaeAAAAACQYxp4AAAAAJBjGngAAAAAkGMaeAAAAACQYxp4AAAAAJBjGngAAAAAkGMaeAAAAACQYxp4AAAAAJBjGngAAAAAkGMaeAAAAACQYxp4AAAAAJBjGngAAAAAkGMaeAAAAACQYxp4AAAAAJBjGngAAAAAkGMaeAAAAACQYxp4AAAAAJBjGngAAAAAkGMaeAAAAACQYxp4AAAAAJBjGngAAAAAkGMaeAAAAACQYxp4AAAAAJBjGngAAAAAkGMaeAAAAACQYxp4AAAAAJBjGngAAAAAkGMaeAAAAACQYxp4AAAAAJBjGngAAAAAkGMaeAAAAACQYxp4AAAAAJBjGngAAAAAkGMaeAAAAACQYxp4AAAAAJBjGngAAAAAkGMaeAAAAACQYxp4AAAAAJBjGngAAAAAkGMaeAAAAACQYxp4AAAAAJBjzcq9sLq6OsyfPz+0atUqVFVVVbIm6qGampqwfPny0KFDh9CkSe31ha07/p11Rxbqat2FYO2R5JlHFqw7suCzlqx45pGFUtdd2Q28+fPnh06dOpV7OQ3U+++/Hzp27Fhr97fuSGPdkYXaXnchWHuk88wjC9YdWfBZS1Y888hCsXVXdku5VatW5V5KA1bb68K6I411RxbqYl1Ye6TxzCML1h1Z8FlLVjzzyEKxdVF2A89rnqSp7XVh3ZHGuiMLdbEurD3SeOaRBeuOLPisJSueeWSh2LpwiAUAAAAA5JgGHgAAAADkmAYeAAAAAOSYBh4AAAAA5JgGHgAAAADkmAYeAAAAAORYs6wLaAiGDRsWZT/72c8S46OOOiqaM2nSpFqrCQAAAICGwRt4AAAAAJBjGngAAAAAkGMaeAAAAACQYxp4AAAAAJBjDrFYS4cddliUXXPNNVF26qmnJsZTp06ttZoAAID6p0WLFlH2ySefRFlNTU1iPGvWrGjO4YcfHmX/+Mc/yq4NgHzxBh4AAAAA5JgGHgAAAADkmAYeAAAAAOSYBh4AAAAA5JhDLIoo3Fj2d7/7XTRn7ty5UfbYY48lxp9//nllCwPIiUGDBiXGZ599djSnd+/eUVZdXR1lzz///NfeO4T0Zy4A1EdnnXVWlKV9Phbq3r17lP3Hf/xHlN1yyy3lFQZA7ngDDwAAAAByTAMPAAAAAHJMAw8AAAAAckwDDwAAAAByzCEWRRx55JGJ8YYbbhjNOeWUU6JswYIFtVYTQF3Ya6+9omzo0KFRNnDgwMQ4bfPtUrOePXt+7TgEh1jw/4wbNy4xPuyww6I5aet4xowZtVYTwL9sttlmUTZ79uzEuGXLlhX7eddee22U/elPf4qyDz74oGI/k+x16NAhyp599tkoW7RoUWK8++6711pNQO3wBh4AAAAA5JgGHgAAAADkmAYeAAAAAOSYPfD+TevWraNs5MiRRa+bPn16bZQD39i3v/3tKHv00UejrE2bNlG2zz77JMavv/56xeqibqXtAVa4b10I8f52VVVV0ZyampooK5zXpEn874bS7lXKvLTraPiaNm0aZXfeeWeUFe5Tm7ZeOnfuHGX2wCNNly5domzAgAFRVvj87NOnT0n3HzNmTJSdddZZifHKlStLuhf1Q9o+2Wm/c1VK2u94y5cvr7WfRz60bds2yjbffPOi2S677BLNmTVrVtGfd9xxx0XZEUccEWXnn39+lL3xxhtF7w98NW/gAQAAAECOaeABAAAAQI5p4AEAAABAjmngAQAAAECOOcTi37Ro0SLKNt1008Q4bePNFStW1FpN1F+bbLJJlP32t7+NsrTN1C+77LKK1NC+ffsoSzvY4rXXXouyd955pyI1ULfuv//+KOvZs2eUderUKcqqq6sT47RDJgrnpM0rZU6p89IOzaDhu/jii6Ps6KOPzqASGor11lsvygo3Yj/wwAOjOYccckiUvfLKK4nxmjVrojlpB7GceOKJUfb73/8+MX7mmWeiOdQPab9zDRo0qNZ+3t133x1lZ555ZpQ5GIWvUsqBFWm6du0aZQcddFCUvfvuu1F29tlnl/Uzgf/jDTwAAAAAyDENPAAAAADIMQ08AAAAAMgxDTwAAAAAyDGHWPyb8847r+icIUOGRJnNYQkhhHXXXTcxnjRpUjRnp512Kimr1CEWAwcOLGneokWLouyzzz6rSA1UTtrBE/fdd19i3Lt372hOqYdKTJ8+PTF+6KGHojnXX3990boKawohhF69epVUw7nnnpsYP/jgg9Ec6reNNtooyp588snEuHv37tGcv/71r1G2ePHixPi73/3uN6yOhqBVq1ZRduyxx0bZNddckxgffvjh0ZzHH388ym6//fbEeKuttorm9OvXL8ratm0bZc2a+VW8Pko7sCLtUIntt9++1mo47bTTouzTTz+ttZ9HPlRVVUXZfvvtV9K1hb/nlatPnz4lzUt75kFebb755lG28847J8atW7eO5myxxRZRlrb2X3311cT4rrvuWssK/4838AAAAAAgxzTwAAAAACDHNPAAAAAAIMca7cYbafuVnHzyyUWvmzx5cm2UQwOwevXqxHjq1KnRnLT97v7yl79UrIbCvaWOO+64it2b7O21115RtueeeybGafvdpWVp+6AMHjw4MZ47d25JdRXueVdYUwgh1NTURNl1110XZWl77FF/NW3aNMoK9w8LId7zbtq0adGcwvUZQghjx45NjNPWmT2hGrYddtghyvr37x9lp556apQtWLAgMU7buzYtK/T666+XlLVs2TLK1qxZU/T+5E/aHnil7kNWKO25NX/+/Cgr/Mz8/PPPy/p51G9bb711lKX9PpXmz3/+c0Vq6NGjR0XuA//SokWLKDv99NMT4w022CCas+uuu0bZlltuGWWFz+xPPvkkmpP2+2Lh96V58+ZFc1588cUoS/P++++XNK8Yb+ABAAAAQI5p4AEAAABAjmngAQAAAECOaeABAAAAQI412kMsDjjggChL2xjxmmuuqYtyaADatWuXGJ9xxhklXbdw4cKK1dClS5evremrpG0YT/788Y9/jLKOHTsmxmkHXaQdDPHcc8+VVcP9998fZb169UqM0zbkTtv09aGHHiqrBuqPO+64I8oOOeSQKJsxY0ZifNttt0Vz0g6f2meffRLj5cuXR3MeffTRYmVSj/3nf/5nlP34xz+OsmeffTbKBg0aVCs1hZB+YEXa4UETJ05MjC+//PJojoNYGrZVq1ZFWefOnTOohDwq/N1+woQJJV339NNPR9kNN9xQiZKg4n76059G2cUXX5wYp32XGDVqVJSlfdYW/nOTdqDEsmXLitaZB97AAwAAAIAc08ADAAAAgBzTwAMAAACAHNPAAwAAAIAca7SHWOy2225RlrZx4ejRo+uiHBqxqVOnVuxe3/72t8u6Lm2zT+qHtAMqKiXtQIyePXtGWeGhFdXV1dGctDVW7kEa5NO4ceOi7Mgjj4yyN954I8pOPvnkxPjll1+O5px22mlR1rRp08R40aJFReukYdl7772jbMGCBVF24oknRtlHH31U9P6tWrWKsvXXXz8xXrFiRTTnoosuirKddtqpaJa2Qb1nZf6MGDGiYvdavXp1Wdf17t07yjbZZJMoSzusZcMNN0yM0w5ee+utt8qqi8rq27dvYrzNNttEc9J+7xo5cmSUffbZZxWpKe2gqbPPPrsi96ZxWrNmTZQVHlrRp0+faM57771XazXllTfwAAAAACDHNPAAAAAAIMc08AAAAAAgxzTwAAAAACDHGsUhFmkbsR9//PFR9s9//jPK3n333VqpiYanX79+ReesWrUqyubMmRNlrVu3LnqvpUuXRtnAgQOLXvfqq69G2eTJk4teR+OTdvBE4YEVIYRQVVWVGKdtuH7UUUdVrjDqXOFhESGEcPHFFyfGac+ftMMEDjnkkCgrfA62aNEimpO2QXbhpsdXXHFFNIeGbeLEiVFWuDZDCOH111+PsldeeaXo/du2bRtlhc+8+++/P5qzzz77FL13mnXWWaes66hdu+yyS2Kctpl6uW666aYoK+Uz84477oiy5s2bl1XDtGnTouycc86JssLfF0s5CIbS7bnnnlH2i1/8ouh1v/nNb6Lsscceq0hNaT755JNauzf1R9p6TfsO26NHj8R4gw02iOZcc801UTZq1KjE2EFl/8cbeAAAAACQYxp4AAAAAJBjGngAAAAAkGONYg+8tL9n3axZ/Ed/8skn66IcGqi0PRQLpe3rNH78+CgrXLNNmsS99nnz5kXZzjvvXLSG5cuXR9nnn39e9DoavmHDhiXGafvdVVdXR1nh+rzhhhsqWhfZ69ChQ5RdeumlifHHH38czSnc9ySEEObPn1/05w0YMCDKtttuuyibNWtWYjxmzJii965t7du3j7K0PdnS9vRj7T3wwANRdvTRR0fZxhtvHGWFa/Hggw+O5qxcuTLKtt5668Q4bb/k9dZbLy42ReHefDNmzCjpOurWsccemxin/XNerp/+9KcVu1e5NtlkkygbO3ZslPXt2zcxtgde+dZdd90oS9sHrPDZlbaf9nXXXVexutKelb169UqMBw0aVNK99ttvvyg78cQTE+M8fG5TXNp+dw8++GCUTZgwIcoKP9fGjRsXzVm8ePE3qK5x8QYeAAAAAOSYBh4AAAAA5JgGHgAAAADkmAYeAAAAAORYozjEIk1VVVWUffDBBxW7f+HGpIWbfYeQvil42kaeTz31VMXqova88sorZV23/fbbl3Vd2vopRdpBFxtttFGULVq0qKz7Uz906tQpyo488sjEOO05mXagSuG8tOuo3wo3cE9z/fXXR1kpB1akOeKII0qal3ZwRrkKN+7efPPNozlph3J07949MT755JOjOX/729++YXV8lZdffjnKCg+ZCCGEzTbbLMoKP+cKN+gPIf3/u3I37v/000+j7JRTTkmM0w7NoG7tvffeUXbWWWdlUAkNWdp3vrS1VyjtgJxp06aVVUPa72tp90875KQUaYe9pB2mR/4VflaFkN47WbJkSZRNnTo1MX777bcrVldj5A08AAAAAMgxDTwAAAAAyDENPAAAAADIMQ08AAAAAMixRnGIxRtvvBFlaRto7rbbbmXdP+0wgQcffDAx3nPPPaM5aRuH/vCHP4yyJ554IjE++uijozk2BM3e4sWLE+O0DY8LN0n/KoX/f86ePTuac/rpp0fZQQcdVPTeU6ZMibKlS5eWVBcNx3333Rdlhc+pmpqaaE51dXWUPffcc187pv4r5bCdtM33+/XrF2UzZ86MsoMPPvhrx1+l8Jk6dOjQkmro0qVLlLVr1y4x7tixYzQnbf2vWLEiMb7rrruiOcOHD48y6taCBQuKznnyySdLulfPnj0T4+bNm0dzPvzwwyjbbrvtoqzwdweyl3ZYU9OmTTOohIYs7dCcUg4BS5uz5ZZbllVD2lpP+5wrV9p3lQceeKBi96funHPOOVGW9vvVD37wgyibOHFiYpx2wNlxxx0XZfPmzVubEhsNb+ABAAAAQI5p4AEAAABAjmngAQAAAECONYo98P7xj39EWdpePWl/H7tQt27douzmm2+Osh122CExHjt2bDTnoYceirJtttkmykaOHJkY77XXXtGcSZMmxcVSp7744ovE+De/+U2t/rxTTjml6JxXX301yg4//PAoW716dUVqInudOnWKsrT97nr16hVlhXvepe2zkra/3T777LM2JVIP3XrrrVF27LHHJsa/+tWv6qqcL+20006J8fXXX1/SdWn7xhbuD5r250nbj9Tnb+Mzbty4xDhtf7TJkydHmf3uqKQ333wzytK+R5BPaXu9Fn7nCyF+3ixcuDCaU7iHawghtGjRIsrmzp27NiV+pbTfK9ddd90omzVrVkV+Htkr3O83hBAmTJhQUla4Fq+44opoTlpfpHCfyE8//bRonY2BN/AAAAAAIMc08AAAAAAgxzTwAAAAACDHNPAAAAAAIMcaxSEWu+++e5RtuOGGUda/f/+i97rtttuibLvttit6rxdeeKHovUMIoXv37lGWtqEpjcu3v/3tKEvb/LbQqlWrosyBFQ1b2sbCe+65Z5QVHlgRQgjV1dWJcZMm8b/jueGGG8ovrhYNGzas6JxSDzgg9uyzz0bZRhttlBj37t07mnPRRRdFWeHBEyGEsP766xetIe1Qnj/84Q+J8fjx46M5aQcHrFmzJsqWLFlStAYatrTDKEaMGBFlaYcFFTr77LMrURJ86f7770+M0z7HHWJRf8ycOTPK9t9//wwqWXtpn6Fpjj766Ch7/vnnK10OOVf4ffS8886L5jz66KNRVvhd949//GNlC6unvIEHAAAAADmmgQcAAAAAOaaBBwAAAAA5poEHAAAAADnWKA6xmDFjRpSlbVa97rrrRtkhhxySGO+9997RnLQNOks5tKJdu3ZRdvvtt0fZsmXLEuPZs2cXvTcNyzrrrBNlzZs3L3rdY489VhvlkCOFm1r36tUrmpO20XVVVVWUFR5akbbRcNrBFgMHDixaZyXttddeUTZ06NDE+IEHHqijahqHtA2r//nPfybGEyZMiOakZWnPpQMOOKBoDWn3uu6664peB6Xq2LFjlF188cVl3avUTd7Jn08//TTKCr83tGnTpm6K+Td9+vRJjDfffPOK3TvtgLMbb7wxytIOXoCv0rZt26xLoASbbbZZlBX2KSrZfyg8NC+EEKZMmRJlhX0Xh1j8H2/gAQAAAECOaeABAAAAQI5p4AEAAABAjmngAQAAAECONYpDLNI8+eSTUZZ2GMW9996bGM+dOzeaM3369Chr3759Yty7d+9oTtqBFWmHFRTWsGDBgmgOpFmxYkXWJVBBaYc39OzZMzFOO7AibbPYtMMoCucV3juEEO65556i9yr351XyXtdff300h7qXdjjURhttVPS6tGfXTTfdVJGa4F++9a1vJcaPPvpoSdcVHgJ0ySWXRHOWLl1afmFkKu0gujFjxiTGw4YNq6tyvpR2yEqlpB1Ycf7559fazwPyY/78+VFWeEjYueeeW7Gft9tuu0XZ6aefHmWeQem8gQcAAAAAOaaBBwAAAAA5poEHAAAAADnWaPfAO/XUU0uad8QRRyTGm2++eTTn6aefLnqfDh06RNnnn38eZZdddlmUXX311UXvT8O26667lnXdhAkTKlwJdSVtv7u0/TYL97wr3JsphPQ940qZV+69yv15pd5r3rx5UTZo0KDE+LnnnovmUPe22GKLKNt9992LXpe21tP2aIFSNW/ePMoGDBiQGHfr1q2kew0fPjwxvuqqq6I5aft3Un/dfPPNifGBBx4YzenatWtdlbNWlixZEmV33XVXYjxixIi6KYYG4Y033oiyXXbZpe4LoSKeeOKJKDv55JMT46233jqa8+abb0ZZixYtomybbbZJjNOelZdeemmUjRs3Li4Wb+ABAAAAQJ5p4AEAAABAjmngAQAAAECOaeABAAAAQI412kMsVq1aFWXHHXdclF1//fWJ8SGHHBLNKdzkMYQQJk+eXLSGG2+8McpmzpxZ9Doan7SNQ9O88847iXGbNm1qoRrqwtChQ6Os8MCKEOKN0tMOfUjbTL2UeeXeq9yflzbvuuuui+Y89NBDUebQinw6/PDDy7ru5ZdfrnAlNHZbbbVVlI0dO7bodcuXL4+ytAPHaNj+/ve/J8bf+973ojlpB0EcfPDBUbbRRhtVrK5S7LjjjlG2YMGCOq2BhiXte65DLOqvAw44IMoKPzM7d+4czdl0002j7PXXX4+yjz76KDFOO4yO0nkDDwAAAAByTAMPAAAAAHJMAw8AAAAAckwDDwAAAAByrNEeYlGqwkMl0g6ZSNu0FrKw5ZZbJsa//vWvozk9e/asq3JYC4MGDUqMBw4cGM2pqqqKssJDH9LmpB3wkHYQROGhPZCVP/zhD1mXQAOT9kwtlHZgRdohBDB37twoO+WUU6LsO9/5TpQVHjA2fPjwaE65BwLccsstUbZ48eKy7gVfpfB3VhqewoN7Csdkxxt4AAAAAJBjGngAAAAAkGMaeAAAAACQY/bAg3rg0EMPLeu6BQsWVLYQas306dO/dhxCCL17946y6urqxHjw4MHRnLQ98NL274GsPP3004nxG2+8kVElNAT7779/lJ133nlRVvj8vOKKK6I506ZNq1xhNDqlrJ9HHnmkDiqByrn77ruj7KKLLsqgEmh8vIEHAAAAADmmgQcAAAAAOaaBBwAAAAA5poEHAAAAADnmEAuoBz788MOyrvvjH/9Y4UqoLYWHSuyzzz4ZVQKV89BDD0XZ6aefHmWXXXZZYrx69epaq4mGZYsttoiyBx54IMrWX3/9KJs1a1ZifO2111aqLIAG65VXXomytIPznnjiibooBxoVb+ABAAAAQI5p4AEAAABAjmngAQAAAECOaeABAAAAQI45xALqgdtvvz3KevfuHWVjx45NjO+5555aqwmgmDlz5kRZp06dMqiEhurjjz+OsvHjx0fZMcccE2WFn5kAFJd2SJ6D86BueAMPAAAAAHJMAw8AAAAAckwDDwAAAAByTAMPAAAAAHLMIRZQD9gsFgBiy5Yti7LjjjuupAwAoD7xBh4AAAAA5JgGHgAAAADkmAYeAAAAAOSYBh4AAAAA5JgGHgAAAADkmAYeAAAAAOSYBh4AAAAA5FjZDbyamppK1kEDUdvrwrojjXVHFupiXVh7pPHMIwvWHVnwWUtWPPPIQrF1UXYDb/ny5eVeSgNW2+vCuiONdUcW6mJdWHuk8cwjC9YdWfBZS1Y888hCsXVRVVNm67e6ujrMnz8/tGrVKlRVVZVVHA1HTU1NWL58eejQoUNo0qT2/ma2dce/s+7IQl2tuxCsPZI888iCdUcWfNaSFc88slDquiu7gQcAAAAA1D6HWAAAAABAjmngAQAAAECOaeABAAAAQI5p4AEAAABAjmngAQAAAECOaeABAAAAQI5p4AEAAABAjmngAQAAAECOaeCVad68eeHYY48N7dq1Cy1atAg77bRTeOmll7Iui0Zk5MiRoaqqKgwdOjTrUmjgrr766rDHHnuEVq1ahfbt24dDDz00zJkzJ+uyaGQ886gry5cvD0OHDg1dunQJLVq0CL179w4vvvhi1mXRCPh+QdZ81lJXRowYEaqqqhL/6datW9Zl5V6zrAuojxYvXhz69OkT+vXrFx599NGw8cYbhzfffDNsuOGGWZdGI/Hiiy+GW2+9Ney8885Zl0IjMHXq1DBkyJCwxx57hC+++CL89Kc/Dfvvv3947bXXQsuWLbMuj0bAM4+6dMopp4TZs2eHu+66K3To0CHcfffdoX///uG1114Lm2++edbl0UD5fkHWfNZS13bYYYfw5JNPfjlu1kx7qhj/C5XhF7/4RejUqVMYM2bMl9mWW26ZYUU0JitWrAjHHHNMuO2228IVV1yRdTk0Ao899lhifMcdd4T27duHGTNmhO985zsZVUVj4ZlHXVq1alV48MEHw/jx4798vo0YMSJMmDAhjB492hqk1vh+QZZ81pKFZs2ahU033TTrMuoVf4W2DI888kjo0aNHGDhwYGjfvn3Yddddw2233ZZ1WTQSQ4YMCQceeGDo379/1qXQSC1dujSEEELbtm0zroTGwDOPuvTFF1+ENWvWhPXWWy+Rt2jRIjz99NMZVUVj4PsFWfJZSxbefPPN0KFDh7DVVluFY445Jrz33ntZl5R7Gnhl+Pvf/x5Gjx4dttlmm/D444+H//qv/wpnnXVW+MMf/pB1aTRw9913X5g5c2a4+uqrsy6FRqq6ujoMHTo09OnTJ+y4445Zl0MD55lHXWvVqlXo1atXuPzyy8P8+fPDmjVrwt133x2effbZsGDBgqzLowHz/YKs+KwlCz179gx33HFHeOyxx8Lo0aPDO++8E/bZZ5+wfPnyrEvLNX+FtgzV1dWhR48e4aqrrgohhLDrrruG2bNnh1tuuSWccMIJGVdHQ/X++++Hs88+O0yaNCl6MwDqypAhQ8Ls2bO9iUKt88wjK3fddVc46aSTwuabbx6aNm0adttttzB48OAwY8aMrEujAfP9giz4rCUr3//+97/87zvvvHPo2bNn6NKlSxg3blw4+eSTM6ws37yBV4bNNtssbL/99olsu+2288ontWrGjBnhww8/DLvttlto1qxZaNasWZg6dWq48cYbQ7NmzcKaNWuyLpEG7owzzggTJ04MTz31VOjYsWPW5dDAeeaRla233jpMnTo1rFixIrz//vvhhRdeCKtXrw5bbbVV1qXRgPl+QRZ81pIXbdq0CV27dg1vvfVW1qXkmjfwytCnT58wZ86cRPbGG2+ELl26ZFQRjcF+++0XXn311UR24oknhm7duoULLrggNG3aNKPKaOhqamrCmWeeGR5++OEwZcoUm2pTJzzzyFrLli1Dy5Ytw+LFi8Pjjz8errnmmqxLogHz/YIs+KwlL1asWBHefvvtcNxxx2VdSq5p4JVh2LBhoXfv3uGqq64KgwYNCi+88EL47W9/G377299mXRoNWKtWraI9x1q2bBnatWtnLzJq1ZAhQ8I999wTxo8fH1q1ahU++OCDEEIIrVu3Di1atMi4Ohoqzzyy8vjjj4eampqw7bbbhrfeeiucd955oVu3buHEE0/MujQaMN8vyILPWrJy7rnnhoMPPjh06dIlzJ8/PwwfPjw0bdo0DB48OOvScs1foS3DHnvsER5++OFw7733hh133DFcfvnl4YYbbgjHHHNM1qUBVNzo0aPD0qVLw7777hs222yzL/9z//33Z10aQMUtXbo0DBkyJHTr1i0cf/zxYe+99w6PP/54aN68edal0YD5fgE0JnPnzg2DBw8O2267bRg0aFBo165deO6558LGG2+cdWm5VlVTU1OTdREAAAAAQDpv4AEAAABAjmngAQAAAECOaeABAAAAQI5p4AEAAABAjmngAQAAAECOaeABAAAAQI5p4AEAAABAjmngAQAAAECOaeABAAAAQI5p4AEAAABAjmngAQAAAECOaeABAAAAQI79f2G0riHAta0UAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1600x500 with 16 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cols = 8\n",
    "rows = 2\n",
    "fig = plt.figure(figsize=(2 * cols, 2.5 * rows))\n",
    "for i in range(cols):\n",
    "    for j in range(rows):\n",
    "        random_index = np.random.randint(0, len(train_set))\n",
    "        ax = fig.add_subplot(rows, cols, i * rows + j + 1)\n",
    "        ax.grid(False)\n",
    "        ax.set_xticks([])\n",
    "        ax.set_yticks([])\n",
    "        ax.imshow(train_set[random_index][0].squeeze(0).numpy().reshape([28, 28]), cmap = 'gray')\n",
    "        ax.set_xlabel(train_set[random_index][1])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Каждая картинка это матрица из чисел. Если число большое - пиксель яркий. Если маленькое - тёмный."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([1, 28, 28])"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_set[5][0].shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Будем прогнозировать тип картинки по её пикселям. Подготовим данные с помощью `DataLoader`. Пока что попробуем настроить его максимально просто. В будущем мы увидим с вами другую магию, на которую способна эта абстракция. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Задание 7:\n",
    "\n",
    "Напишите `DataLoader` для тренировочной и тестовой выборок. Попробуйте проитерироваться по нескольким его первым объектам с помощью цикла.\n",
    "\n",
    "В обучающей выборке данные должны перемешиваться каждую эпоху. Размер батча поставьте равным 64. В тестовой выборке данных перемешивать не надо."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [],
   "source": [
    "# берем датасет, перемешиваем его и формируем батчи\n",
    "train_loader = torch.utils.data.DataLoader(\n",
    "    train_set, batch_size=64, shuffle=True\n",
    ")\n",
    "\n",
    "test_loader = torch.utils.data.DataLoader(\n",
    "    test_set, batch_size=64, shuffle=False\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([64, 1, 28, 28]) torch.Size([64])\n"
     ]
    }
   ],
   "source": [
    "for images, labels in train_loader:\n",
    "    print(images.shape, labels.shape)\n",
    "    break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Задание 8:\n",
    "\n",
    "Напишите  двухслойную полносвязную нейросеть. Выходной слой должен состоять из $10$ нейронов с `Softmax` в качестве функции активации, так как мы решаем задачу классификации $10$ классов. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [],
   "source": [
    "class MLP(nn.Module):\n",
    "    def __init__(self, in_features, num_classes, hidden_size):\n",
    "        super().__init__()\n",
    "        self.model = nn.Sequential(\n",
    "            nn.Linear(in_features=in_features, out_features=hidden_size),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(in_features=hidden_size, out_features=hidden_size, bias=False),\n",
    "            nn.LeakyReLU(0.1),\n",
    "            nn.Linear(in_features=hidden_size, out_features=num_classes),\n",
    "            # nn.LogSoftmax(), # в функцию потерь пойдут логиты\n",
    "        )\n",
    "\n",
    "    def forward(self, x):\n",
    "        return self.model(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = MLP(28**2, 10, 20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Веса моделей хранятся в виде матриц и выглядят так:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [],
   "source": [
    "# [x for x in model.named_parameters()] "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Объявляем модель и оптимизатор."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [],
   "source": [
    "IMG_SIZE = 28\n",
    "NUM_CLASSES = 10\n",
    "HIDDEN_SIZE = 64\n",
    "NUM_EPOCHS = 10\n",
    "\n",
    "device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')\n",
    "model = MLP(in_features=IMG_SIZE ** 2, num_classes=NUM_CLASSES, hidden_size=HIDDEN_SIZE).to(device)\n",
    "\n",
    "# оптимайзер\n",
    "optimizer = torch.optim.SGD(model.parameters(), lr=0.01, momentum=0.9)\n",
    "\n",
    "# функция потерь \n",
    "criterion = nn.CrossEntropyLoss()  # nn.NLLLoss()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Осталось всё это дело обучить. Для этого нам придётся написать цикл"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "7038438a6add484fb56cf8b35a002172",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/938 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "64a52e69616546349a0b1c73e3534ee8",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating:   0%|          | 0/157 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0, loss: 0.24048808217048645, accuracy: 0.9299\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9cf43d0329b14c978510b738dfb485f3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/938 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9df2b90889644cd58de90d7e50deeda0",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/938 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e3e551c6f50e4585bd55e2ca9d230666",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating:   0%|          | 0/157 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 2, loss: 0.13008937239646912, accuracy: 0.9602\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "76e91501906247fdb28815a897bd522f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/938 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c7836277e5e046af9f19ddd25fd926c9",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/938 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "589fffcd3c104a79bbdbbfa7e34683ef",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating:   0%|          | 0/157 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 4, loss: 0.102723129093647, accuracy: 0.9691\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "502321ad1c28429db8cf447e28861fbc",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/938 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fd8dec6e4d8a4ac2ba74f35aa6c3b616",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/938 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0d0dac09e63f480daf292820fb8754d9",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating:   0%|          | 0/157 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 6, loss: 0.09784935414791107, accuracy: 0.9706\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "7fa489e5d7f54e029b916c99a9b8e305",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/938 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f8f97b42ae7d44a8bded58023c65a214",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/938 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8951506fb53e4a9eb9ced97cba7ad7d2",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating:   0%|          | 0/157 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 8, loss: 0.08746642619371414, accuracy: 0.9742\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0aace361dca44fe198e5d1c8a5eddb70",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/938 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from tqdm.notebook import tqdm\n",
    "\n",
    "# NUM_EPOCHS раз подряд пройдемся по всем батчам из трейна\n",
    "for epoch in range(NUM_EPOCHS):\n",
    "    \n",
    "    # берем батч из трейн лоадера\n",
    "    for images, labels in tqdm(train_loader):\n",
    "        \n",
    "        # отправляем тензоры на девайс к сетке\n",
    "        images = images.to(device)\n",
    "        labels = labels.to(device)\n",
    "        \n",
    "        ### Пишем самые важные 5 строк\n",
    "        ### Мы их дальше будем встречать везде\n",
    "        \n",
    "        # зануляем старые градиенты иначе они сложатся\n",
    "        optimizer.zero_grad()\n",
    "        \n",
    "        # вызываем модель\n",
    "        logits = model(torch.flatten(images, start_dim=1))\n",
    "        # (batch_size, x, num_classes) - размерность\n",
    "        \n",
    "        # считаем функцию потерь \n",
    "        loss = criterion(logits, labels)\n",
    "        \n",
    "        # считаем градиенты обратным проходом\n",
    "        loss.backward() \n",
    "        \n",
    "        # обновляем параметры сети (шаг спуска)\n",
    "        optimizer.step()\n",
    "        \n",
    "    if epoch % 2 == 0:\n",
    "        mean_val_loss = [] # сюда будем складывать средний лосс по батчам\n",
    "        val_accuracy = []\n",
    "        \n",
    "        # берем батч из вал лоадера для валидации\n",
    "        for images, labels in tqdm(test_loader, desc='Validating'):\n",
    "            images = images.to(device)\n",
    "            labels = labels.to(device)\n",
    "            \n",
    "            # мы считаем качество, поэтому мы запрещаем фреймворку считать градиенты по параметрам\n",
    "            with torch.no_grad(): \n",
    "                # делаем предсказания\n",
    "                logits = model(torch.flatten(images, start_dim=1))\n",
    "                \n",
    "                # считаем ошибку на валидации\n",
    "                loss = criterion(logits, labels)\n",
    "                \n",
    "                mean_val_loss.append(loss.numpy()) # добавляем в массив \n",
    "                val_accuracy.extend((torch.argmax(logits, dim=-1) == labels).numpy().tolist())\n",
    "        \n",
    "        # выводим статистику\n",
    "        print('Epoch: {epoch}, loss: {loss}, accuracy: {accuracy}'.format(\n",
    "                epoch=epoch, loss=np.mean(mean_val_loss), accuracy=np.mean(val_accuracy)\n",
    "        )) \n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Кажется, что всё работает. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Ещё одно задание:\n",
    "\n",
    "На следующей паре мы продолжим обучать эту сетку и попробуем улучшить получившийся у нас пайплайн. Попробуйте для удобства вынести один шаг обучения модели и один шаг валидации модели в отдельные небольшие функции, чтобы код стал более читаемым. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Почиташки\n",
    "\n",
    "__Откуда взялась эта тетрадка?__ \n",
    "\n",
    "Когда я писал эту тетрадку, я ориентировался на четыре источника:\n",
    "\n",
    "- [Введение в pytorch от ШАД](https://github.com/yandexdataschool/Practical_DL/blob/fall22/week02_autodiff/seminar_pytorch.ipynb)\n",
    "- [Тетрадка с введением в pytorch с ИАД](https://github.com/hse-ds/iad-deep-learning/blob/master/2022/seminars/sem01/sem01.ipynb) (её взяли из ШАД)\n",
    "- [Введение в pytorch от Samsung](https://github.com/SlinkoIgor/Neural_Networks_and_CV)\n",
    "- [Экспромт Ильдуса с семинаров по DL на ФКН](https://github.com/isadrtdinov/intro-to-dl-hse/tree/2022-2023/seminars/202)\n",
    "\n",
    "Я добавил к ним свою больную фантазию и получилось то, что получилось.\n",
    "\n",
    "\n",
    "__Что можно почитать?__\n",
    "\n",
    "* [Документация торча](https://pytorch.org/docs/master/)\n",
    "* [Использование pytorch на GPU](https://pytorch.org/docs/master/notes/cuda.html)\n",
    "* [Хорошая книга про pytorch](http://download.webclark.org/Deep-Learning-with-PyTorch.pdf)\n",
    "* [Pytorch за 60 минут](http://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html)\n",
    "* [Как устроено автоматическое дифференцирование в pytorch](http://videolectures.net/site/normal_dl/tag=1129745/deeplearning2017_johnson_automatic_differentiation_01.pdf)\n",
    "* [PyTorch examples](https://github.com/pytorch/examples)\n",
    "* [Practical pytorch tutorials](https://github.com/spro/practical-pytorch)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}