{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Hands-On 1: Set Up Your Environment and Debug a CNN"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Introduction\n",
    "\n",
    "This is the first hands-on session of the course. It has two parts.\n",
    "\n",
    "**Part A: Set up your environment.** In this part, you:\n",
    "\n",
    "* connect this notebook to the course Python environment\n",
    "* make sure that the necessary packages are installed\n",
    "* make sure that PyTorch can use the GPU\n",
    "\n",
    "In later sessions, you use [Anemoi](https://anemoi.readthedocs.io) to train and run data-driven weather forecast models. These sessions use the same environment and the same GPU. If there is a problem, find it now. Then we can repair it before you need it.\n",
    "\n",
    "**Part B: Debug a CNN.** In this part, you:\n",
    "\n",
    "* train a small Convolutional Neural Network (CNN) on the MNIST dataset\n",
    "* add errors on purpose and examine the results\n",
    "* use a step-by-step procedure to find errors\n",
    "* use tools to examine tensor shapes and gradients\n",
    "\n",
    "Anemoi models are much larger than this CNN. But they usually fail for the same causes: incorrect tensor shapes, a bad learning rate or bad data. Thus, the procedure in Part B also applies to Anemoi models."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part A: Set Up Your Environment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A.1 The Course Environment\n",
    "\n",
    "The course VM has a Python environment that is ready to use. It contains PyTorch, Anemoi and all other packages for this course. You do not install packages.\n",
    "\n",
    "The environment is in this directory:\n",
    "\n",
    "```\n",
    "/opt/ml-course/env\n",
    "```\n",
    "\n",
    "**CAUTION:** Do not use `pip install` in this course. A change to the environment can cause errors in later sessions. If a package is missing, tell an instructor."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A.2 Connect the Environment to Jupyter\n",
    "\n",
    "Jupyter runs the code of a notebook in a **kernel**. Each kernel uses one Python environment. To use the course environment, you must register it as a kernel. Do this procedure one time only. The kernel is then available for all notebooks in this course.\n",
    "\n",
    "1. In JupyterLab, select **File > New > Terminal**.\n",
    "2. In the terminal, type these commands:\n",
    "\n",
    "   ```bash\n",
    "   source /opt/ml-course/env/bin/activate\n",
    "   which python\n",
    "   python -m ipykernel install --user --name ml-course --display-name \"Python (ML course)\"\n",
    "   ```\n",
    "\n",
    "   The `which python` command must show `/opt/ml-course/env/bin/python`.\n",
    "3. Go back to this notebook.\n",
    "4. Select **Kernel > Change Kernel...**.\n",
    "5. Select **Python (ML course)**.\n",
    "6. Make sure that the top-right corner of the notebook shows **Python (ML course)**.\n",
    "\n",
    "**NOTE:** In later sessions, you also run Anemoi commands in a terminal. Before you do this, type the `source` command of step 2 in each new terminal."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A.3 Do a Check of the Kernel\n",
    "\n",
    "Run the cell below. It shows the Python environment that this notebook uses. If it is not the course environment, the cell stops with an error."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Python executable: /perm/naco/venvs/aifs-raps-ac/bin/python\n",
      "Python version:    3.11.10\n",
      "Environment:       /perm/naco/venvs/aifs-raps-ac\n",
      "\n",
      "OK: This notebook uses the course environment.\n"
     ]
    }
   ],
   "source": [
    "import sys\n",
    "from pathlib import Path\n",
    "\n",
    "COURSE_ENV = Path(\"/opt/ml-course/env\")\n",
    "\n",
    "print(\"Python executable:\", sys.executable)\n",
    "print(\"Python version:   \", sys.version.split()[0])\n",
    "print(\"Environment:      \", sys.prefix)\n",
    "\n",
    "if Path(sys.prefix).resolve() != COURSE_ENV.resolve():\n",
    "    raise RuntimeError(\n",
    "        f\"This notebook does not use the course environment ({COURSE_ENV}). \"\n",
    "        \"Do the steps in Section A.2 again.\"\n",
    "    )\n",
    "print(\"\\nOK: This notebook uses the course environment.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A.4 Do a Check of the Packages\n",
    "\n",
    "Run the cell below. It imports each package that this course uses and shows its version.\n",
    "\n",
    "A package can be installed but still fail when you import it. For example, a library file can be missing. Thus, the cell imports each package. It does not only read the version.\n",
    "\n",
    "**NOTE:** The first import of some packages can take up to one minute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OK     torch              2.9.0\n",
      "OK     torchvision        0.24.0\n",
      "OK     numpy              1.26.4\n",
      "OK     matplotlib         3.10.8\n",
      "OK     scikit-learn       1.8.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/perm/naco/venvs/aifs-raps-ac/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OK     torch-geometric    2.7.0\n",
      "OK     earthkit-data      0.18.4\n",
      "OK     anemoi-datasets    0.5.28\n",
      "OK     anemoi-graphs      0.8.3\n",
      "OK     anemoi-models      0.9.5.post153\n",
      "OK     anemoi-training    0.6.5.post153\n",
      "OK     anemoi-inference   0.0.0.post444\n",
      "\n",
      "OK: All packages are available.\n"
     ]
    }
   ],
   "source": [
    "import importlib\n",
    "from importlib.metadata import version\n",
    "\n",
    "# Package name (for pip) -> module name (for import)\n",
    "PACKAGES = {\n",
    "    \"torch\": \"torch\",\n",
    "    \"torchvision\": \"torchvision\",\n",
    "    \"numpy\": \"numpy\",\n",
    "    \"matplotlib\": \"matplotlib\",\n",
    "    \"scikit-learn\": \"sklearn\",\n",
    "    \"torch-geometric\": \"torch_geometric\",\n",
    "    \"earthkit-data\": \"earthkit.data\",\n",
    "    \"anemoi-datasets\": \"anemoi.datasets\",\n",
    "    \"anemoi-graphs\": \"anemoi.graphs\",\n",
    "    \"anemoi-models\": \"anemoi.models\",\n",
    "    \"anemoi-training\": \"anemoi.training\",\n",
    "    \"anemoi-inference\": \"anemoi.inference\",\n",
    "}\n",
    "\n",
    "failed = []\n",
    "for package, module in PACKAGES.items():\n",
    "    try:\n",
    "        importlib.import_module(module)\n",
    "        print(f\"OK     {package:<18} {version(package)}\")\n",
    "    except Exception as e:\n",
    "        print(f\"ERROR  {package:<18} {type(e).__name__}: {e}\")\n",
    "        failed.append(package)\n",
    "\n",
    "if failed:\n",
    "    print(f\"\\n{len(failed)} package(s) failed: {', '.join(failed)}. Look at Section A.6.\")\n",
    "else:\n",
    "    print(\"\\nOK: All packages are available.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A.5 Do a Check of the GPU\n",
    "\n",
    "A GPU makes training much faster. The small models in Part B also train on a CPU. But Anemoi training in later sessions needs a GPU.\n",
    "\n",
    "Do these three checks:\n",
    "\n",
    "1. `nvidia-smi` shows the GPU, the driver and the processes that use the GPU. If the command is not found, the VM has no NVIDIA driver.\n",
    "2. PyTorch shows if it can find the GPU.\n",
    "3. A small calculation runs on the CPU and on the GPU. The two results must agree.\n",
    "\n",
    "**NOTE:** The CPU and GPU results are not exactly the same. The two devices add numbers in a different sequence, and floating-point addition is not exact. Small differences (approximately 1e-5) are normal."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fri Nov 13 08:17:13 2026       \n",
      "+-----------------------------------------------------------------------------------------+\n",
      "| NVIDIA-SMI 595.91.07              Driver Version: 595.91.07      CUDA Version: 13.2     |\n",
      "+-----------------------------------------+------------------------+----------------------+\n",
      "| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |\n",
      "| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |\n",
      "|                                         |                        |               MIG M. |\n",
      "|=========================================+========================+======================|\n",
      "|   0  NVIDIA A100-SXM4-40GB          On  |   00000000:C4:00.0 Off |                    0 |\n",
      "| N/A   40C    P0             58W /  400W |       4MiB /  40960MiB |      0%      Default |\n",
      "|                                         |                        |             Disabled |\n",
      "+-----------------------------------------+------------------------+----------------------+\n",
      "\n",
      "+-----------------------------------------------------------------------------------------+\n",
      "| Processes:                                                                              |\n",
      "|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |\n",
      "|        ID   ID                                                               Usage      |\n",
      "|=========================================================================================|\n",
      "|  No running processes found                                                             |\n",
      "+-----------------------------------------------------------------------------------------+\n"
     ]
    }
   ],
   "source": [
    "!nvidia-smi"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PyTorch version:        2.9.0+cu128\n",
      "PyTorch built for CUDA: 12.8\n",
      "CUDA available:         True\n",
      "GPU 0: NVIDIA A100-SXM4-40GB, 42.6 GB memory\n",
      "bfloat16 supported:     True\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "\n",
    "print(\"PyTorch version:       \", torch.__version__)\n",
    "print(\"PyTorch built for CUDA:\", torch.version.cuda)\n",
    "print(\"CUDA available:        \", torch.cuda.is_available())\n",
    "\n",
    "if torch.cuda.is_available():\n",
    "    for i in range(torch.cuda.device_count()):\n",
    "        props = torch.cuda.get_device_properties(i)\n",
    "        print(f\"GPU {i}: {props.name}, {props.total_memory / 1e9:.1f} GB memory\")\n",
    "    print(\"bfloat16 supported:    \", torch.cuda.is_bf16_supported())\n",
    "else:\n",
    "    print(\"\\nWARNING: PyTorch cannot find a GPU. Look at Section A.6.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Device: cuda\n",
      "Largest difference between CPU and cuda result: 6.20e-05\n",
      "\n",
      "OK: The results agree.\n"
     ]
    }
   ],
   "source": [
    "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "\n",
    "a = torch.randn(1000, 1000)\n",
    "b = torch.randn(1000, 1000)\n",
    "\n",
    "cpu_result = a @ b\n",
    "device_result = (a.to(device) @ b.to(device)).cpu()  # Calculate on the device, then copy back to the CPU\n",
    "\n",
    "max_diff = (cpu_result - device_result).abs().max().item()\n",
    "print(f\"Device: {device}\")\n",
    "print(f\"Largest difference between CPU and {device} result: {max_diff:.2e}\")\n",
    "\n",
    "if torch.allclose(cpu_result, device_result, atol=1e-3):\n",
    "    print(\"\\nOK: The results agree.\")\n",
    "else:\n",
    "    print(\"\\nERROR: The results do not agree. Tell an instructor.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A.6 If a Check Fails\n",
    "\n",
    "| Symptom | Possible cause | Action |\n",
    "|---|---|---|\n",
    "| Section A.3 shows an error. | The notebook uses the wrong kernel. | Do Section A.2 again. |\n",
    "| **Python (ML course)** is not in the kernel list. | The kernel is not registered, or JupyterLab did not update the list. | Do step 2 of Section A.2 again. Then reload the browser page. |\n",
    "| `ModuleNotFoundError` | The notebook uses the wrong kernel, or the package is not installed. | Make sure that Section A.3 shows OK. If it does, tell an instructor. |\n",
    "| `nvidia-smi: command not found` | The VM has no NVIDIA driver. | Tell an instructor. |\n",
    "| `nvidia-smi` shows a GPU, but PyTorch shows `CUDA available: False`. | The CUDA version of PyTorch does not agree with the driver. | Tell an instructor. |\n",
    "| `CUDA out of memory` | An other kernel uses the GPU memory. | Open **Running Terminals and Kernels** in the left sidebar. Shut down the kernels that you do not use. |\n",
    "| The kernel stops (\"Kernel died\") during an import. | Not sufficient memory, or a damaged installation. | Restart the kernel and run the cell again. If the problem continues, tell an instructor. |\n",
    "| The MNIST download in Section 4 fails. | The VM has no internet connection. | Tell an instructor. |\n",
    "\n",
    "After you change the kernel or repair the environment, select **Kernel > Restart Kernel**. Then run Part A again."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A.7 Summary\n",
    "\n",
    "Make sure that all these items are correct:\n",
    "\n",
    "- [ ] The notebook uses the kernel **Python (ML course)**.\n",
    "- [ ] All packages show `OK`.\n",
    "- [ ] `nvidia-smi` shows a GPU.\n",
    "- [ ] PyTorch shows `CUDA available: True`.\n",
    "- [ ] The CPU and GPU results agree.\n",
    "\n",
    "If you cannot repair a problem, tell an instructor now. Do not wait for a later session."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part B: Debug a CNN"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wM3Kt5u9wtWS"
   },
   "source": [
    "## 2. Why Convolutional Neural Networks for Images?\n",
    "\n",
    "In previous sessions, you utilized fully connected neural networks, also known as Multi-Layer Perceptrons (MLPs). While powerful, MLPs face two significant limitations when processing image data.\n",
    "\n",
    "### 2.1 Parameter Explosion\n",
    "\n",
    "Images are composed of high-dimensional data. A 28x28 grayscale image has 28 x 28 = 784 input features.\n",
    "If a hidden layer has 100 neurons, that layer needs 784 x 100 = 78,400 weights. For modern high-definition images, the number of parameters becomes computationally prohibitive and leads to rapid overfitting.\n",
    "\n",
    "### 2.2 Lack of Spatial Awareness\n",
    "\n",
    "Flattening an image into a 1D vector destroys the spatial relationships between pixels. If a digit shifts by just a few pixels, the resulting vector changes drastically. The model would then need to learn the same feature (like a curve or an edge) separately for every possible location in the image.\n",
    "\n",
    "### 2.3 The CNN Solution\n",
    "\n",
    "CNNs utilize two fundamental concepts to solve these issues:\n",
    "\n",
    "1. **Local Connectivity:** Instead of every neuron connecting to every pixel, filters (kernels) look at small local regions (e.g., 3x3).\n",
    "2. **Weight Sharing:** The same filter is slid across the entire image. This allows the model to detect the same feature regardless of its position, drastically reducing the parameter count.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Import the Libraries\n",
    "\n",
    "Import the libraries for tensors, neural network layers, optimization and plots."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "cPc_kx2LrlrH"
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.optim as optim\n",
    "import torchvision\n",
    "import torchvision.transforms as transforms\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Set seed for reproducibility\n",
    "torch.manual_seed(42)\n",
    "np.random.seed(42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "L0igvk9yrlrI"
   },
   "source": [
    "## 4. Loading and Normalizing the MNIST Dataset\n",
    "\n",
    "Normalization is a critical preprocessing step. By scaling pixel values (originally 0-255) to a **mean of 0.1307** and a **standard deviation of 0.3081**, we ensure that gradients propagate more effectively during training.\n",
    "\n",
    "### Why these specific values?\n",
    "\n",
    "These are the global statistics of the MNIST dataset. Centering and scaling the data provides:\n",
    "\n",
    "* **Faster Convergence:** It makes the \"loss surface\" more spherical, allowing the optimizer to take a direct path to the minimum rather than oscillating.\n",
    "* **Optimal Activation:** Most activation functions (like ReLU or Tanh) are designed to work best with inputs centered around zero. This prevents **Gradient Saturation**, where neurons \"die\" and stop learning.\n",
    "* **Numerical Stability:** It prevents weights from exploding to massive values or vanishing to zero during backpropagation.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "F3IGQQCorlrI",
    "outputId": "9c762c9b-053b-4285-ff63-a01bc56d5ab8"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 9.91M/9.91M [00:01<00:00, 5.28MB/s]\n",
      "100%|██████████| 28.9k/28.9k [00:00<00:00, 285kB/s]\n",
      "100%|██████████| 1.65M/1.65M [00:00<00:00, 2.29MB/s]\n",
      "100%|██████████| 4.54k/4.54k [00:00<00:00, 9.82MB/s]\n"
     ]
    }
   ],
   "source": [
    "transform = transforms.Compose([\n",
    "    # Converts a PIL Image or numpy.ndarray (0-255) to a FloatTensor (0.0-1.0).\n",
    "    transforms.ToTensor(),\n",
    "\n",
    "    # Standardizes the data by subtracting the mean and dividing by the std dev.\n",
    "    # 0.1307 and 0.3081 are the global mean and standard deviation of MNIST.\n",
    "    # This helps the model converge faster by centering inputs around zero.\n",
    "    transforms.Normalize((0.1307,), (0.3081,))\n",
    "])\n",
    "\n",
    "# 'train=True' loads the 60,000 images used for model training.\n",
    "train_dataset = torchvision.datasets.MNIST(\n",
    "    root='./data',\n",
    "    train=True,         # Specify training set\n",
    "    download=True,\n",
    "    transform=transform # Apply the normalization defined above\n",
    ")\n",
    "\n",
    "# 'train=False' loads the 10,000 images used to evaluate the model.\n",
    "test_dataset = torchvision.datasets.MNIST(\n",
    "    root='./data',\n",
    "    train=False,        # Specify test set\n",
    "    download=True,\n",
    "    transform=transform\n",
    ")\n",
    "\n",
    "# Loaders handle batching, shuffling, and multi-process data loading.\n",
    "# Batch size of 64 means the model looks at 64 images before updating weights.\n",
    "train_loader = torch.utils.data.DataLoader(\n",
    "    train_dataset,\n",
    "    batch_size=64,\n",
    "    shuffle=True        # Shuffle every epoch\n",
    ")\n",
    "\n",
    "test_loader = torch.utils.data.DataLoader(\n",
    "    test_dataset,\n",
    "    batch_size=64,\n",
    "    shuffle=False       # Shuffling is not necessary for evaluation\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kL4cGdAYrlrJ"
   },
   "source": [
    "\n",
    "## 5. Visualizing Data\n",
    "\n",
    "Before training, always inspect your data to ensure the loaders and transforms are functioning correctly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 194
    },
    "id": "UT3a-FA5rlrJ",
    "outputId": "bf6f45f5-7d58-48a1-91b7-234ae58c75cd"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAoAAAACxCAYAAABUdAwCAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAGShJREFUeJzt3XlwldX9x/HvJQlJgJRAWEUIpuwCRWFYHJBQQWCMFMoiCA3KUqaCUAYE7chire1QoGVYBFG2FkQpAxQQ7UwUHCK7FIpWkC1sRUpkSVgD5Pn9wc/U534P5nJzc7fzfs3wx/lwnueeXI4PXx7PeR6P4ziOAAAAwBplQj0AAAAABBcFIAAAgGUoAAEAACxDAQgAAGAZCkAAAADLUAACAABYhgIQAADAMhSAAAAAlqEABAAAsIz1BWBOTo54PB6ZMWNGwM65ZcsW8Xg8smXLloCdE9GNeYhwwDxEqDEHgyciC8ClS5eKx+ORPXv2hHoopeLQoUMyduxYeeyxxyQhIUE8Ho/k5OSEeljwEu3zcM2aNfLMM89IWlqalCtXTho2bCjjxo2TS5cuhXpo+J5on4ciIllZWdKpUyepUqWKJCcnS+vWreWvf/1rqIeF/xftc3Dq1Kni8XjUr4SEhFAPrURiQz0AaNu3b5fZs2dLkyZNpHHjxrJv375QDwkW+uUvfykPPPCADBo0SOrUqSMHDhyQuXPnyqZNm2Tv3r2SmJgY6iHCAuvXr5eePXtKu3btiv4iXrVqlWRmZkpubq6MHTs21EOEJebPny8VKlQoasfExIRwNCVHARiGevToIZcuXZKkpCSZMWMGBSBCYvXq1ZKenu7KWrZsKYMHD5YVK1bIsGHDQjMwWGXu3LlSs2ZN+eSTTyQ+Pl5EREaMGCGNGjWSpUuXUgAiaPr06SNVqlQJ9TACJiL/F7AvCgoKZPLkydKyZUupWLGilC9fXjp06CCbN2++5zF//vOfJTU1VRITE6Vjx47yxRdfqD4HDx6UPn36SOXKlSUhIUFatWol69evL3Y8165dk4MHD0pubm6xfStXrixJSUnF9kP4i+R56F38iYj06tVLRES++uqrYo9H+IjkeZiXlyeVKlUqKv5ERGJjY6VKlSrchY4gkTwHv+M4juTl5YnjOD4fE86itgDMy8uTd955R9LT02XatGkydepUOX/+vHTt2tV4R+0vf/mLzJ49W0aOHCmvvPKKfPHFF/LTn/5Uzp07V9Tnyy+/lLZt28pXX30lL7/8ssycOVPKly8vPXv2lLVr1/7geHbt2iWNGzeWuXPnBvpHRRiLtnn4zTffiIhE1b+CbRDJ8zA9PV2+/PJLmTRpkhw5ckSOHj0qr7/+uuzZs0cmTJhw398FQiOS5+B30tLSpGLFipKUlCSDBg1yjSUiORFoyZIljog4u3fvvmef27dvOzdv3nRlFy9edKpXr+4MGTKkKDt+/LgjIk5iYqJz+vTponznzp2OiDhjx44typ544gmnWbNmzo0bN4qywsJC57HHHnPq169flG3evNkREWfz5s0qmzJlyn39rNOnT3dExDl+/Ph9HYfSZ9M8/M7QoUOdmJgY5+uvv/breARetM/DK1euOP369XM8Ho8jIo6IOOXKlXPWrVtX7LEIjmifg7NmzXJGjRrlrFixwlm9erUzZswYJzY21qlfv75z+fLlYo8PV1F7BzAmJkbKli0rIiKFhYVy4cIFuX37trRq1Ur27t2r+vfs2VNq1apV1G7durW0adNGNm3aJCIiFy5ckE8++UT69esn+fn5kpubK7m5ufLtt99K165d5fDhw3LmzJl7jic9PV0cx5GpU6cG9gdFWIumefjuu+/KokWLZNy4cVK/fv37Ph6hE8nzMD4+Xho0aCB9+vSRlStXyvLly6VVq1YyaNAg2bFjx31+EwiVSJ6DY8aMkTlz5sizzz4rvXv3llmzZsmyZcvk8OHD8uabb97nNxE+orYAFBFZtmyZNG/eXBISEiQlJUWqVq0qH3zwgVy+fFn1Nf2F1qBBg6LHrxw5ckQcx5FJkyZJ1apVXb+mTJkiIiL//e9/S/XnQWSKhnm4detWGTp0qHTt2lXeeOONgJ8fpS9S5+GoUaNkw4YN8t5770n//v1l4MCBkpWVJTVr1pQxY8YE5DMQHJE6B02effZZqVGjhmRlZZXaZ5S2qN0FvHz5cnnuueekZ8+e8tJLL0m1atUkJiZG/vCHP8jRo0fv+3yFhYUiIjJ+/Hjp2rWrsU+9evVKNGZEn2iYh/v375cePXpI06ZNZfXq1RIbG7WXjagVqfOwoKBAFi1aJBMmTJAyZf53vyIuLk66d+8uc+fOlYKCgqI7SwhfkToHf0jt2rXlwoULpfoZpSlqr+SrV6+WtLQ0WbNmjXg8nqL8u38ZeDt8+LDKvv76a6lbt66I3F38KXL3wtO5c+fADxhRKdLn4dGjR6Vbt25SrVo12bRpk+sZWIgckToPv/32W7l9+7bcuXNH/d6tW7eksLDQ+HsIP5E6B+/FcRzJycmRRx55JOifHShR+7+Av3tAo/O97do7d+6U7du3G/uvW7fOtV5g165dsnPnTunevbuIiFSrVk3S09PlrbfekrNnz6rjz58//4Pj8WfLOSJfJM/Db775Rp588kkpU6aM/OMf/5CqVasWewzCU6TOw2rVqklycrKsXbtWCgoKivIrV67Ihg0bpFGjRjwKJkJE6hy817nmz58v58+fl27duhV7fLiK6DuAixcvlo8++kjlY8aMkYyMDFmzZo306tVLnnrqKTl+/LgsWLBAmjRpIleuXFHH1KtXT9q3by+/+tWv5ObNmzJr1ixJSUlxPWZg3rx50r59e2nWrJkMHz5c0tLS5Ny5c7J9+3Y5ffq07N+//55j3bVrl3Tq1EmmTJlS7KLTy5cvy5w5c0RE5LPPPhORuw9DTU5OluTkZBk1apQvXw+CJFrnYbdu3eTYsWMyYcIEyc7Oluzs7KLfq169unTp0sWHbwfBEo3zMCYmRsaPHy+vvvqqtG3bVjIzM+XOnTuyaNEiOX36tCxfvvz+viSUqmicgyIiqamp8swzz0izZs0kISFBsrOz5b333pMWLVrIiBEjfP+Cwk1I9h6X0Hdbzu/169SpU05hYaHz+9//3klNTXXi4+OdRx55xNm4caMzePBgJzU1tehc3205nz59ujNz5kyndu3aTnx8vNOhQwdn//796rOPHj3qZGZmOjVq1HDi4uKcWrVqORkZGc7q1auL+pR0y/l3YzL9+v7YEVrRPg9/6Gfr2LFjCb45BFK0z0PHcZwVK1Y4rVu3dpKTk53ExESnTZs2rs9AaEX7HBw2bJjTpEkTJykpyYmLi3Pq1avnTJw40cnLyyvJ1xZyHseJkkdaAwAAwCdRuwYQAAAAZhSAAAAAlqEABAAAsAwFIAAAgGUoAAEAACxDAQgAAGAZCkAAAADL+PwmkO+/uw/4vmA+SpJ5iHsJ1jxkDuJeuBYiHPg6D7kDCAAAYBkKQAAAAMtQAAIAAFiGAhAAAMAyFIAAAACWoQAEAACwDAUgAACAZSgAAQAALEMBCAAAYBkKQAAAAMtQAAIAAFiGAhAAAMAyFIAAAACWoQAEAACwDAUgAACAZSgAAQAALEMBCAAAYBkKQAAAAMvEhnoAMGvRooWr/frrr6s+GRkZKktJSVHZhQsXAjYu3FtycrKrPWzYMNVnzpw5KqtatarKpkyZojLT+Xxx5MgRlT3xxBMqO3v2rMpu3brl12cCAMIbdwABAAAsQwEIAABgGQpAAAAAy1AAAgAAWMbjOI7jU0ePp7THYq24uDiVLV682NUeOHCg6mNaoF+jRg2VXbx4sQSjK56PUyggwmUemsaxcOFCV3vo0KGlOoY7d+6o7Pr16yqLiYlRmWnOrV27VmUDBgxwtQsLC+9niEEVrHkYLnPQJD4+3tUuW7as6lO9enWVDR482Kfze29Eqlat2n2Mzi0rK0tlP//5z1V29epVvz8j2Gy8Fvpr9OjRKjNdl2bOnBmM4UQVX+chdwABAAAsQwEIAABgGQpAAAAAy1AAAgAAWIY3gQSZaVH222+/rTLvTR83b95Uffr166ey0t7wgbv69++vskBu+sjPz1eZ96afadOmqT7Tp09XWd26dVXWtGlTlZnePuLd71//+pfqg9BITU1VmfdGnubNm5fqGEqy6aFz584qM72NplevXq72xx9/7PdnIjQSExNVNnLkSJV5b6RD6eIOIAAAgGUoAAEAACxDAQgAAGAZCkAAAADLsAkkyEybB37xi18Ue9yCBQtUtmHDhoCMCT8sISFBZS+//HKxx508eVJl27ZtU9nWrVtV9uGHH6osJyen2M80MR1nyipUqKCyV1991dX2fjOIiPmNJCh9x44dU1kw30RRGsqVK6cy740BP/nJT1SfK1eulNqYUHLjx49XWf369UMwEnwfdwABAAAsQwEIAABgGQpAAAAAy1AAAgAAWMbj+Lhq2OPxlPZYok779u1V5v2kfhGRlJQUlV29etXVbtSokepz5syZEowucIK58DwU8zAuLk5lmZmZxR6XnZ2tskOHDgVkTKXBtEHp3XffdbVNbxUxbXYJhWDNw3C5Fpo23/jyHVy/fl1lvs5L701MGRkZqk+dOnV8OpfpezSN3/stSD/72c9Un6ysLJU1aNBAZS1btlRZz549VfbRRx+52kuWLFF9TKL9WuirBx54wNXesWOH6vPggw+q7MSJEyozvR3GX1u2bFGZ6e1cb775psoKCgpc7dOnTwdsXIHm6zzkDiAAAIBlKAABAAAsQwEIAABgGdYABkj58uVV9tlnn6msefPmKsvPz1fZwIEDXe2NGzeWYHSli3Uv0cGXNYCPPvqo6rNv377SGtJ9YQ2gb9/Bb37zG5X98Y9/9GsMpnVcTZs2VVm/fv1UNnjwYJWZxv+3v/3N1X7llVdUn3bt2qls9uzZKqtUqZLKTLzXoj388MOqz40bN1TGtfCusWPHutozZ84M0UgC5/z586527969VR/Tuu9QYA0gAAAAjCgAAQAALEMBCAAAYBkKQAAAAMvEhnoAkci04WP+/PkqM234yMvLU5npocLhvOkD9ho5cqTKhg8fHoKRwF+mDRP+Mj0M15SZFsz7qm/fvq62afymzSglkZqa6mp3795d9TE91B93ef+Z+cr0cgPTg8vr1avnal+6dEn1OX78uF9j8FXDhg1VFi6bQHzFHUAAAADLUAACAABYhgIQAADAMhSAAAAAlmETiB8yMjJUNmjQIJ+OXbFihcrWr19f4jEBJVWjRo1i+6xatSoII4EvFi9erLLnn3++2OM6d+6ssokTJ6rM9CYjXxa5DxgwQGWmt37cunVLZfv371dZy5YtXe1Ab/jwhffbSEREYmP56zPQZs2apTLTd++9yeTw4cOqD3+vFo87gAAAAJahAAQAALAMBSAAAIBlKAABAAAs43Ecx/Gpo8dT2mMJW6+99pqr/cILL6g+KSkpKlu3bp3KTIu0L1++7P/gwoCPUyggbJ6HgVS5cmWV7d69W2UPPfSQq123bl3V5+TJkwEbV0kEax6GyxysXbu2ygL59oNr166pzPutL++//77qc+DAAZU1btxYZabvMZB/hnPmzFHZvHnzVDZixAiVJScnF3t+0xtwuBbetW3bNle7bdu2Ph135coVlV29etWvMZg2lOzZs8evc4mI7Nq1y9XOz8/3+1ylzdd5yB1AAAAAy1AAAgAAWIYCEAAAwDIUgAAAAJZhE4gX01PyV65c6WqbNnxcvHhRZR06dFDZv//97xKMLjyx8Dny/PrXv1bZn/70J5WdO3fO1W7RokWxfULFtk0g8fHxKvPe+DBkyJCAfub169dd7by8PNWnatWqKitTRt9rCOQmkJ07d6qsU6dOKisoKPDr/L7iWniXv5tAwtmCBQtcbdNm0HDBJhAAAAAYUQACAABYhgIQAADAMlavAUxLS1OZaS2J95o/03q/zMxMlX3wwQclGF3kYN1LeDOtyfJeoyMi8uMf/1hl3g9B926HE9vWAJokJSW52p9++qnq07x582AN5wf5uwZw/vz5KpswYYLKvNcrBgPXwrsaNGjgah88eDCg5//8889d7RMnTvh9rscff1xlVapUUZn3n+2tW7dUn4YNG6qsJGPzF2sAAQAAYEQBCAAAYBkKQAAAAMtQAAIAAFgmNtQDCBbTA2wnTpyoMtNDnnfv3u1q//a3v1V9bNnwgbvKli2rsl69ernaY8eO9elcW7duVVm9evVU9s4777jagwYNUn0eeughlSUnJ6vMtOHD5D//+Y+r/dRTT/l0nK+OHDmiskOHDgX0M2ySn5/vaj/66KOqz8iRI1U2e/bsUhvTvZgeDp2bm6uyTZs2udovvvhiqY0JgXHmzBlXu3fv3gE9/z//+U9XOycnx+9ztW/fXmWmjXMzZ850tevWrav6hPPGHBPuAAIAAFiGAhAAAMAyFIAAAACWoQAEAACwTFS+CaRcuXIq815ILGJ+AvilS5dU1qVLF1fb+ynktov2p9+bFtIvWbJEZc2aNQvGcKLK7373O5VNnjzZr3PxJhDfmBa9b9myJejjMH2PY8aMUdncuXODMZyAiPZroc327t3raps2lg4ZMkRlS5cuLaUR3RtvAgEAAIARBSAAAIBlKAABAAAsQwEIAABgmah4E0hiYqKrbVqgb9rwcfnyZZUNHDhQZWz6sEdMTIzKpkyZojJfNnxcvXpVZRs3bvRvYAbp6ekqq169ut/nO3nypMoqV67saleoUMHv8yP4GjVqpLJly5aFYCS+GTBggMr27dvnamdnZwdpNLCV6ToXFxdX7HHDhw9XWSg2gfiKO4AAAACWoQAEAACwDAUgAACAZSgAAQAALBMVm0C8N3j07dvXp+NMbwf58MMPAzImRKbRo0er7Omnn1bZzZs3Vfb888+72jt27FB9cnJy/B+cl1q1aqls7dq1KqtZs6bKTIvtjxw5orL4+HhXe9y4capPxYoVVWb62WfMmKGyDh06qAyB89JLL6ksNTXV7/NNnDjR1T5x4oTq8+KLL6qsXbt2KjNtuGrTpo3Kxo8f72qzCQSBZNrwsXLlSpU9/PDDxZ4rkJv8goE7gAAAAJahAAQAALAMBSAAAIBlKAABAAAs43Ecx/Gpo8dT2mPxSUJCgsq8N2507NhR9TEtjs/MzFSZ6e0N+GE+TqGAKO15uGHDBpU9+eSTKuvfv7/KTHPMX1WqVFHZyJEjXW3vTSciIrdv31ZZnz59VOb9doVgqF27tsry8/NVdunSJb/OH6x5GC7XQtMmii5durjab731lupj2jxkYprPQ4cOdbXz8vJ8OtfChQuLPZeI+c/Qe3OS6e0m4SKaroW2ML05bPDgwSrz/r5Nf9ZpaWkqC+TGP1/5Og+5AwgAAGAZCkAAAADLUAACAABYJuIeBF2mjK5ZTQ9y9Fa3bl2Vsd4P3jIyMlR248YNlR07dqzYcyUlJamsR48eKjOt5Rk4cKDKWrRo4WqvWbNG9Xn77bdVFor1fianTp0K9RCiSrly5VTmy4NoTev23n//fZV5P/TZdOyDDz6o+pjW+zVs2LDYcd3L8uXL/T4W9oqLi1PZ5MmTVZaenu7T+bzX1WVlZak+/q5fDhXuAAIAAFiGAhAAAMAyFIAAAACWoQAEAACwTMRtAklMTFRZy5Ytiz3Ol40igEl8fLzKsrOzVTZ69GhX27TJaNmyZSpbv369ymbPnq2yc+fOudp79+7VgwWKYVocb3pY7QsvvKCyvn37utqVK1dWfUwP/DYxPbTctLHJ9N8a8H1t27ZV2bhx41TWu3dvvz/D+4UTpo16bAIBAABAWKMABAAAsAwFIAAAgGUoAAEAACzjcbwfb32vjoa3FYRCSkqKys6fP1/scaYf89q1ayqbP3++yiZMmODj6Ozk4xQKiNKeh9u2bVOZaYGxSUFBgattemtNbKzed9W/f3+VrVq1yqfPxP8Eax6Gy7XQ9KaZixcvhmAk/nnuuedUFulv/Yima2Egde/eXWWmty6ZNgF17txZZT/60Y9c7REjRqg+puuvrz7++GOVDRgwwNXOzc31+/ylzdd5yB1AAAAAy1AAAgAAWIYCEAAAwDIUgAAAAJaJuE0gpnFUqlTJ1c7KylJ96tSpo7LXXntNZfPmzVNZYWHh/QzROtG08DkmJkZljz/+uMrKly+vMu9FwZ9//rlPn3nnzh2VMefun22bQExvqNm+fbur3bx582ANp8j+/ftVNm3aNJVF40anaLoWBtKkSZNUZvr798aNGyozzXNffnbTm5LOnj2rsjfeeENlpjl8/fr1Yj8zXLAJBAAAAEYUgAAAAJahAAQAALAMBSAAAIBlIm4TCMIPC58RDmzbBGLi/XYF0xsYTG9NMDl69KjKvDduHDhwQPX5+9//rrKbN2/69JmRjmuhWdOmTVX29NNPq8y0IcNk4cKFrvann36q+mRnZ6vs1KlTPp0/0rEJBAAAAEYUgAAAAJahAAQAALAMBSAAAIBl2ASCEmPhM8IBm0AQalwLEQ7YBAIAAAAjCkAAAADLUAACAABYhgIQAADAMhSAAAAAlqEABAAAsAwFIAAAgGUoAAEAACxDAQgAAGAZCkAAAADLUAACAABYhgIQAADAMhSAAAAAlvE4juOEehAAAAAIHu4AAgAAWIYCEAAAwDIUgAAAAJahAAQAALAMBSAAAIBlKAABAAAsQwEIAABgGQpAAAAAy1AAAgAAWOb/AGmGRqP/406cAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 800x400 with 4 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "examples = iter(train_loader)\n",
    "images, labels = next(examples)\n",
    "\n",
    "plt.figure(figsize=(8, 4))\n",
    "for i in range(4):\n",
    "    plt.subplot(1, 4, i+1)\n",
    "    plt.imshow(images[i][0], cmap=\"gray\")\n",
    "    plt.title(f\"Label: {labels[i].item()}\")\n",
    "    plt.axis('off')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "YWRyf79mrlrJ"
   },
   "source": [
    "## 6. Understanding Shape Flow in CNNs\n",
    "\n",
    "The most common source of runtime errors in PyTorch is a **shape mismatch**, particularly at the transition between convolutional layers and fully connected (Linear) layers. To debug effectively, you must understand how each operation transforms the 4D input tensor: `(Batch Size, Channels, Height, Width)`.\n",
    "\n",
    "### The Shape Transformation Formula\n",
    "\n",
    "For any convolutional layer, the output spatial dimension (O) is calculated based on the input size (W), kernel size (K), padding (P), and stride (S):\n",
    "\n",
    "O = [(W - K + 2P) / S] + 1\n",
    "\n",
    "\n",
    "**Shape Flow for MNIST (28x28 images):**\n",
    "\n",
    "1. **Input:** `(batch_size, 1, 28, 28)` — MNIST is grayscale, so it has only **1 channel**.\n",
    "2. **Conv2d(1 -> 2, kernel=3, padding=1):** Calculation: [(28 - 3 + 2*1) / 1] + 1 = 28\n",
    "\n",
    "* The spatial size stays the same because `padding=1` compensates for the `3x3` kernel.\n",
    "* **Output:** `(batch_size, 2, 28, 28)`.\n",
    "\n",
    "\n",
    "3. **MaxPool2d(kernel=2, stride=2):** Pooling layers divide the height and width by the stride.\n",
    "* Calculation: 28 / 2 = 14\n",
    "* **Output:** `(batch_size, 2, 14, 14)`.\n",
    "\n",
    "\n",
    "4. **Flatten:** This collapses the spatial dimensions and channels into a single vector per image.\n",
    "* Calculation: 2 * 14 * 14 = 392\n",
    "* **Output:** `(batch_size, 392)`.\n",
    "\n",
    "\n",
    "5. **Linear(392 -> 8):** A fully connected hidden layer. This is often where \"Size Mismatch\" errors occur if the input features (392) don't match the Flatten output.\n",
    "\n",
    "* **Output:** `(batch_size, 8)`.\n",
    "\n",
    "6. **Linear(8 -> 10):** Final output producing \"logits\" for each of the **10 classes** (digits 0-9).\n",
    "\n",
    "* **Output:** `(batch_size, 10)`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wDQ-ioUO1iBG"
   },
   "source": [
    "\n",
    "## 7. Defining a Tiny CNN\n",
    "\n",
    "We will implement a minimal architecture called `TinyMNISTNet`. We keep the number of channels very low (only 2) to make it easier to visualize the weight matrices and understand how features are extracted.\n",
    "\n",
    "In PyTorch, we define the layers in `__init__` and the data flow in the `forward` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "id": "pKdHX2SjrlrJ"
   },
   "outputs": [],
   "source": [
    "class TinyMNISTNet(nn.Module):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "\n",
    "        # 1. Feature Extraction: Extracts spatial patterns from the image\n",
    "        self.features = nn.Sequential(\n",
    "            nn.Conv2d(1, 2, kernel_size=3, padding=1), # 1 input channel -> 2 feature maps\n",
    "            nn.ReLU(),                                 # Non-linearity\n",
    "            nn.MaxPool2d(2),                           # Halves resolution: 28x28 -> 14x14\n",
    "        )\n",
    "\n",
    "        # 2. Classification: Maps extracted features to class scores\n",
    "        self.classifier = nn.Sequential(\n",
    "            nn.Flatten(),                              # Converts 3D maps to a 1D vector (392 elements)\n",
    "            nn.Linear(2 * 14 * 14, 8),                 # Compressed hidden layer\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(8, 10)                           # Final layer: 10 outputs (one per digit)\n",
    "        )\n",
    "\n",
    "    def forward(self, x):\n",
    "        # Data flows through features first, then the classifier\n",
    "        x = self.features(x)\n",
    "        x = self.classifier(x)\n",
    "        return x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xgFqUtJ3rlrJ"
   },
   "source": [
    "\n",
    "## 8. Shape Debugging\n",
    "\n",
    "This utility function allows you to identify exactly where a tensor's shape might be deviating from your expectations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "BO00CkYO2hJZ",
    "outputId": "648bcb91-7ff8-48f4-bf47-a75d040be7c8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=== Debugging Forward Pass ===\n",
      "Initial Input: [1, 1, 28, 28]\n",
      "After features  : [1, 2, 14, 14]\n",
      "After classifier: [1, 10]\n",
      "\n",
      "Final Prediction Shape: [1, 10]\n"
     ]
    }
   ],
   "source": [
    "def debug_forward_pass(model, input_shape=(1, 1, 28, 28)):\n",
    "    print(\"=== Debugging Forward Pass ===\")\n",
    "\n",
    "\n",
    "    # Create a dummy 'image' with the expected input dimensions\n",
    "    # (Batch Size, Channels, Height, Width)\n",
    "    x = torch.randn(input_shape)\n",
    "    print(f\"Initial Input: {list(x.shape)}\")\n",
    "\n",
    "    # We iterate through the main components\n",
    "    # Note: Named_children only shows top-level attributes\n",
    "    for name, block in model.named_children():\n",
    "        try:\n",
    "            x = block(x)\n",
    "            print(f\"After {name:10}: {list(x.shape)}\")\n",
    "        except Exception as e:\n",
    "            print(f\"Error in block '{name}': {e}\")\n",
    "            return\n",
    "\n",
    "    print(f\"\\nFinal Prediction Shape: {list(x.shape)}\")\n",
    "\n",
    "# Usage\n",
    "model = TinyMNISTNet()\n",
    "debug_forward_pass(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aqTlOMOR5NA2"
   },
   "source": [
    "## 9. The ML Debugging Checklist\n",
    "\n",
    "When your model performs poorly or crashes, follow this sequence:\n",
    "\n",
    "\n",
    "### Data & Inputs\n",
    "\n",
    "* **Normalization Check:** Ensure pixels are scaled (e.g., mean 0.1307, std 0.3081). Raw 0-255 values cause unstable gradients.\n",
    "* **Label Integrity:** Verify that labels correctly map to images. Shuffled targets make the model learn \"noise.\"\n",
    "* **Class Balance:** Check if one class dominates. Imbalance leads to a model that only predicts the majority class.\n",
    "\n",
    "### Architecture\n",
    "\n",
    "* **Shape Flow Trace:** Ensure the output of the last Conv layer matches the input of the first Linear layer.\n",
    "* **The \"Overfit Single Batch\" Test:** Try to reach 100% accuracy on 5 images. If this fails, your loss function or training loop logic is broken.\n",
    "\n",
    "\n",
    "### Optimization\n",
    "\n",
    "* **Learning Rate (LR) :**  **Too High:** Loss \"explodes,\" bounces wildly, or results in `NaN`.**Too Low:** Loss remains perfectly flat for many epochs.\n",
    "\n",
    "* Ensure gradients aren't vanishing (becoming too small) in early layers.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tNsUCv_W5riG"
   },
   "source": [
    "\n",
    "## 10. Training with an Intentional Bug\n",
    "\n",
    "We will now intentionally use an excessively high learning rate to observe the failure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "A8f8uk3zrlrK",
    "outputId": "f6479251-cca6-46e1-84db-d4bf51bca02c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Starting training with high Learning Rate...\n",
      "Epoch 0: Loss=6.8173, Acc=0.1017\n",
      "Epoch 1: Loss=2.3723, Acc=0.1003\n",
      "Epoch 2: Loss=2.3724, Acc=0.1027\n"
     ]
    }
   ],
   "source": [
    "model = TinyMNISTNet()\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "# INTENTIONAL BUG: Learning rate set to 1.0 (normally 0.001)\n",
    "optimizer = optim.Adam(model.parameters(), lr=1.0)\n",
    "\n",
    "def train(model, loader, optimizer):\n",
    "    model.train()\n",
    "    total_loss, correct, total = 0, 0, 0\n",
    "    for data, target in loader:\n",
    "        optimizer.zero_grad()\n",
    "        output = model(data)\n",
    "        loss = criterion(output, target)\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "\n",
    "        total_loss += loss.item()\n",
    "        _, predicted = torch.max(output, 1)\n",
    "        total += target.size(0)\n",
    "        correct += (predicted == target).sum().item()\n",
    "    return total_loss / len(loader), correct / total\n",
    "\n",
    "print(\"Starting training with high Learning Rate...\")\n",
    "for epoch in range(3):\n",
    "    loss, acc = train(model, train_loader, optimizer)\n",
    "    print(f\"Epoch {epoch}: Loss={loss:.4f}, Acc={acc:.4f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qk1wP6_Z6SgL"
   },
   "source": [
    "### Why the Model Failed\n",
    "\n",
    "The training logs show the model is **stagnant**, not learning. Even though the loss dropped initially, it settled at a \"random guess\" baseline.\n",
    "\n",
    "* **Accuracy stuck at ~10%:** In a 10-class dataset like MNIST, 10% is the result of random guessing.\n",
    "* A cross-entropy loss of roughly **2.3** (−ln(0.1)) is the mathematical signature of a model that is completely uncertain, assigning an equal 10% probability to every digit.\n",
    "* The learning rate is so high that the optimizer \"jumps\" across the loss landscape, overshooting the optimal solution and landing on a flat plateau where it can no longer find its way down.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-06Ptk2O6n2Z"
   },
   "source": [
    "## 11. Gradient Diagnostic\n",
    "We have adjusted the learning rate to **0.001** to ensure stable optimization. Before proceeding to full-scale training, we perform a **Gradient Inspection** to verify the integrity of the backpropagation signal.\n",
    "\n",
    "\n",
    "The **magnitude (norm)** of the gradients represents the strength of the update signal. Monitoring this helps identify numerical instability early:\n",
    "\n",
    "* **Optimal Flow (0.01 – 1.0):** The signal is stable. Weights will update at a controlled pace, allowing for smooth convergence.\n",
    "* **Vanishing Gradients (< 1e-6):** The signal attenuates as it moves backward. This often occurs in deep networks, leaving early layers \"frozen\" and incapable of learning.\n",
    "* **Exploding Gradients (> 10.0):** The signal grows exponentially. This leads to massive weight updates that \"overshoot\" the loss minimum, often resulting in `NaN` values.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "DrrmTp2mrlrL",
    "outputId": "e1ede1e8-eb11-4047-c85d-ca327de24010"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Layer Name                | Gradient Norm  \n",
      "---------------------------------------------\n",
      "features.0.weight         | 0.055391\n",
      "features.0.bias           | 0.015171\n",
      "classifier.1.weight       | 0.418048\n",
      "classifier.1.bias         | 0.049468\n",
      "classifier.3.weight       | 0.039150\n",
      "classifier.3.bias         | 0.100659\n"
     ]
    }
   ],
   "source": [
    "def inspect_gradients(model):\n",
    "\n",
    "    print(f\"{'Layer Name':<25} | {'Gradient Norm':<15}\")\n",
    "    print(\"-\" * 45)\n",
    "\n",
    "    for name, param in model.named_parameters():\n",
    "        if param.grad is not None:\n",
    "            # The 'norm' is the total magnitude of all updates for this layer\n",
    "            norm = param.grad.norm().item()\n",
    "            print(f\"{name:<25} | {norm:.6f}\")\n",
    "        else:\n",
    "            print(f\"{name:<25} | No Gradient (Check your code!)\")\n",
    "\n",
    "# Reset the model to clear the weights from before\n",
    "model = TinyMNISTNet()\n",
    "\n",
    "optimizer = optim.Adam(model.parameters(), lr=0.001)\n",
    "# Clear any old gradients\n",
    "optimizer.zero_grad()\n",
    "\n",
    "data, target = next(iter(train_loader))\n",
    "output = model(data)\n",
    "loss = criterion(output, target)\n",
    "\n",
    "# Backward pass: This calculates the gradients\n",
    "loss.backward()\n",
    "inspect_gradients(model)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LgA7qGpVrlrL"
   },
   "source": [
    "\n",
    "## 12. Exercise: Data Corruption (Label Shuffling)\n",
    "\n",
    "A model may appear to be \"learning\" (loss goes down) but fail validation if the evaluation data is corrupted.\n",
    "\n",
    "Your **Training Accuracy** will increase because the model is learning from correct data. However, your **Test Accuracy** will plummet to roughly **10%** (random chance).\n",
    "If you see a massive gap between Train and Test performance, it usually means one of two things:\n",
    "1. **Overfitting:** The model is memorizing the training data.\n",
    "2. **Data Corruption:** The test set is broken, mislabeled, or follows a completely different distribution.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "b3dwaeqcrlrL",
    "outputId": "95432437-ca9b-4a27-a66b-f7b19503bb74"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training on valid data...\n",
      "Epoch 0: Train Acc=0.7346\n",
      "Epoch 1: Train Acc=0.8896\n",
      "\n",
      "[!] Test Accuracy on Corrupted Labels: 0.1020\n"
     ]
    }
   ],
   "source": [
    "def corrupt_labels(dataset, seed=42):\n",
    "    \"\"\"\n",
    "    Randomly shuffles the labels of a dataset while keeping the images intact.\n",
    "    This simulates a broken data pipeline or mislabeled 'ground truth'.\n",
    "    \"\"\"\n",
    "    np.random.seed(seed)\n",
    "    # Convert targets to a numpy array, shuffle them, and put them back\n",
    "    shuffled_targets = np.random.permutation(dataset.targets.numpy())\n",
    "    dataset.targets = torch.tensor(shuffled_targets)\n",
    "\n",
    "# We corrupt ONLY the test set to see the impact on evaluation\n",
    "corrupt_labels(test_dataset)\n",
    "\n",
    "def evaluate(model, loader):\n",
    "    \"\"\"Standard evaluation loop to calculate accuracy.\"\"\"\n",
    "    model.eval()\n",
    "    correct, total = 0, 0\n",
    "    with torch.no_grad(): # Disable gradient tracking for speed and memory efficiency\n",
    "        for data, target in loader:\n",
    "            output = model(data)\n",
    "            _, predicted = torch.max(output, 1) # Get the index of the highest score\n",
    "            total += target.size(0)\n",
    "            correct += (predicted == target).sum().item()\n",
    "    return correct / total\n",
    "\n",
    "# Train normally on GOOD data, but evaluate on CORRUPTED data\n",
    "print(\"Training on valid data...\")\n",
    "for epoch in range(2):\n",
    "    loss, acc = train(model, train_loader, optimizer)\n",
    "    print(f\"Epoch {epoch}: Train Acc={acc:.4f}\")\n",
    "\n",
    "# Observe the 'Evaluation Gap'\n",
    "test_acc = evaluate(model, test_loader)\n",
    "print(f\"\\n[!] Test Accuracy on Corrupted Labels: {test_acc:.4f}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZQPQMzihrlrL"
   },
   "source": [
    "\n",
    "\n",
    "## 13. The \"Overfit 10 Samples\" Debugging Technique\n",
    "\n",
    "If you are unsure if your architecture is capable of learning, try to train it on a tiny batch. If it cannot reach 100% accuracy on 10 images, there is a fundamental bug in your code.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "WmnEg2JsrlrM",
    "outputId": "f9e4e657-fc66-4a78-c34e-8a0da826e1e6"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Attempting to overfit 10 samples...\n",
      "Epoch 10: Loss=0.7232, Acc=0.90\n",
      "Epoch 20: Loss=0.2113, Acc=1.00\n",
      "Epoch 30: Loss=0.0353, Acc=1.00\n",
      "Epoch 40: Loss=0.0075, Acc=1.00\n",
      "Epoch 50: Loss=0.0027, Acc=1.00\n"
     ]
    }
   ],
   "source": [
    "# Create a 10-sample \"mini-world\"\n",
    "small_data, small_targets = next(iter(train_loader))\n",
    "small_data, small_targets = small_data[:10], small_targets[:10]\n",
    "\n",
    "model_test = TinyMNISTNet()\n",
    "optimizer_test = optim.Adam(model_test.parameters(), lr=0.01) # Higher LR to force memorization\n",
    "\n",
    "print(\"Attempting to overfit 10 samples...\")\n",
    "for epoch in range(50):\n",
    "    optimizer_test.zero_grad()\n",
    "    output = model_test(small_data)\n",
    "    loss = criterion(output, small_targets)\n",
    "    loss.backward()\n",
    "    optimizer_test.step()\n",
    "\n",
    "    if (epoch + 1) % 10 == 0:\n",
    "        _, predicted = torch.max(output, 1)\n",
    "        acc = (predicted == small_targets).sum().item() / 10\n",
    "        print(f\"Epoch {epoch+1:2}: Loss={loss.item():.4f}, Acc={acc:.2f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oat_-iEcrlrM"
   },
   "source": [
    "\n",
    "## 14. Advanced Challenge: Designing a High-Accuracy CNN\n",
    "\n",
    "To reach over 98% accuracy, a network needs more capacity.\n",
    "\n",
    "**Recommended Design:**\n",
    "\n",
    "* **Block 1:** Conv(1 -> 8), ReLU, MaxPool\n",
    "* **Block 2:** Conv(8 -> 16), ReLU, MaxPool\n",
    "* **Final Spatial Size:** After two MaxPool layers, a 28x28 image becomes 7x7.\n",
    "* **Flatten Size:** The input to your first Linear layer must be 16×7×7=784.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "id": "ksHRbJVmrlrM"
   },
   "outputs": [],
   "source": [
    "class ImprovedCNN(nn.Module):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        self.features = nn.Sequential(\n",
    "            nn.Conv2d(1, 8, 3, padding=1),\n",
    "            nn.ReLU(),\n",
    "            nn.MaxPool2d(2),\n",
    "            nn.Conv2d(8, 16, 3, padding=1),\n",
    "            nn.ReLU(),\n",
    "            nn.MaxPool2d(2)\n",
    "        )\n",
    "        self.classifier = nn.Sequential(\n",
    "            nn.Flatten(),\n",
    "            nn.Linear(16 * 7 * 7, 64),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(64, 10)\n",
    "        )\n",
    "\n",
    "    def forward(self, x):\n",
    "        return self.classifier(self.features(x))\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "xyfdYmXQERI1",
    "outputId": "3915581d-6770-42b3-e60f-a9b229532700"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test labels restored. Ready for final evaluation.\n",
      "Training ImprovedCNN on cuda...\n",
      "Epoch 1: Loss=0.2333, Train Acc=0.9300, Test Acc=0.9740\n",
      "Epoch 2: Loss=0.0736, Train Acc=0.9770, Test Acc=0.9776\n",
      "Epoch 3: Loss=0.0530, Train Acc=0.9839, Test Acc=0.9831\n",
      "Epoch 4: Loss=0.0415, Train Acc=0.9871, Test Acc=0.9872\n",
      "Epoch 5: Loss=0.0347, Train Acc=0.9892, Test Acc=0.9882\n"
     ]
    }
   ],
   "source": [
    "# Reload the test dataset to remove the corruption from Section 12\n",
    "test_dataset = torchvision.datasets.MNIST(\n",
    "    root='./data',\n",
    "    train=False,\n",
    "    download=True,\n",
    "    transform=transform\n",
    ")\n",
    "\n",
    "test_loader = torch.utils.data.DataLoader(\n",
    "    test_dataset,\n",
    "    batch_size=64,\n",
    "    shuffle=False\n",
    ")\n",
    "\n",
    "print(\"Test labels restored. Ready for final evaluation.\")\n",
    "\n",
    "\n",
    "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "model = ImprovedCNN().to(device)\n",
    "optimizer = optim.Adam(model.parameters(), lr=0.001)\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "\n",
    "print(f\"Training ImprovedCNN on {device}...\")\n",
    "for epoch in range(5):\n",
    "    model.train()\n",
    "    total_loss, correct, total = 0, 0, 0\n",
    "\n",
    "    for images, labels in train_loader:\n",
    "        images, labels = images.to(device), labels.to(device) # Move to GPU\n",
    "\n",
    "        optimizer.zero_grad()\n",
    "        outputs = model(images)\n",
    "        loss = criterion(outputs, labels)\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "\n",
    "        total_loss += loss.item()\n",
    "        _, predicted = torch.max(outputs, 1)\n",
    "        total += labels.size(0)\n",
    "        correct += (predicted == labels).sum().item()\n",
    "\n",
    "    # Evaluation Phase\n",
    "    model.eval()\n",
    "    test_correct, test_total = 0, 0\n",
    "    with torch.no_grad():\n",
    "        for images, labels in test_loader:\n",
    "            images, labels = images.to(device), labels.to(device) # Move to GPU\n",
    "            outputs = model(images)\n",
    "            _, predicted = torch.max(outputs, 1)\n",
    "            test_total += labels.size(0)\n",
    "            test_correct += (predicted == labels).sum().item()\n",
    "\n",
    "    print(f\"Epoch {epoch+1}: Loss={total_loss/len(train_loader):.4f}, \"\n",
    "          f\"Train Acc={correct/total:.4f}, Test Acc={test_correct/test_total:.4f}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kA6USA7qHqWN"
   },
   "source": [
    "## 15. Advanced Diagnostic Visualizations\n",
    "\n",
    "When a model reaches high accuracy, a simple \"Loss\" value no longer provides enough information to improve the model further and we might want to look inside the \"Black Box\" using advanced visualizations.\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KwqXihiPIhSF"
   },
   "source": [
    "\n",
    "### The Confusion Matrix\n",
    "\n",
    "A Confusion Matrix compares the model's predictions against the actual labels in a grid format.\n",
    "\n",
    "* Look for high numbers outside the diagonal line. For example, if you see a high count where \"Actual 4\" meets \"Predicted 9,\" your model is struggling with vertical strokes.\n",
    "* You may need more convolutional filters or specific data augmentation (like slight rotations) to help the model tell these shapes apart."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 696
    },
    "id": "VWa3o_ESHCDr",
    "outputId": "92ee189d-d795-4297-cee9-9a026386fa4a"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAp4AAAJOCAYAAAAeSAe8AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjUsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvWftoOwAAAAlwSFlzAAAPYQAAD2EBqD+naQAAiHVJREFUeJzs3Xl8FPX9x/HX5NrcGxLIBYFwKPd9GRBFRSgi1Wq1KG0Rz1r4qcUTrYpapeKF99FaRSui9a6tB0IFUZTIIZfcVzhCgBybBHLtzu8PZHUNCLtkd3bH9/PxmMeDnZ2d/XwyO8t3P/P9fscwTdNERERERCTIoqwOQERERER+HtTwFBEREZGQUMNTREREREJCDU8RERERCQk1PEVEREQkJNTwFBEREZGQUMNTREREREJCDU8RERERCQk1PEVEREQkJNTwFJEjWr9+PcOHD8fpdGIYBu+8806T7n/Lli0YhsGLL77YpPuNZEOHDmXo0KFWhyEiEhRqeIqEuY0bN3LVVVfRrl074uPjSU1NZfDgwTz66KMcOHAgqO89btw4VqxYwb333svLL79Mv379gvp+oXTJJZdgGAapqamH/TuuX78ewzAwDIMHH3zQ7/3v3LmTKVOmsGzZsiaIVkTEHmKsDkBEjuw///kPF1xwAQ6Hg9///vd069aNuro6FixYwI033siqVat47rnngvLeBw4cYOHChdx2221MnDgxKO/Rpk0bDhw4QGxsbFD2fzQxMTHs37+ff//731x44YU+z73yyivEx8dTU1MT0L537tzJXXfdRX5+Pr169Trm13388ccBvZ+ISCRQw1MkTG3evJkxY8bQpk0b5s6dS05Ojve5CRMmsGHDBv7zn/8E7f337NkDQFpaWtDewzAM4uPjg7b/o3E4HAwePJhXX321UcNz5syZjBo1ijfffDMksezfv5/ExETi4uJC8n4iIlbQpXaRMDVt2jSqqqp4/vnnfRqdh3To0IFrr73W+7ihoYF77rmH9u3b43A4yM/P59Zbb6W2ttbndfn5+Zx99tksWLCAAQMGEB8fT7t27XjppZe820yZMoU2bdoAcOONN2IYBvn5+cDBS9SH/v1DU6ZMwTAMn3WzZ8/m5JNPJi0tjeTkZDp27Mitt97qff5IfTznzp3LkCFDSEpKIi0tjXPOOYdvv/32sO+3YcMGLrnkEtLS0nA6nYwfP579+/cf+Q/7IxdffDEffPAB5eXl3nWFhYWsX7+eiy++uNH2paWl3HDDDXTv3p3k5GRSU1MZOXIk33zzjXebTz/9lP79+wMwfvx47yX7Q3kOHTqUbt26sXjxYk455RQSExO9f5cf9/EcN24c8fHxjfIfMWIEzZo1Y+fOncecq4iI1dTwFAlT//73v2nXrh2DBg06pu0vv/xy7rjjDvr06cMjjzzCqaeeytSpUxkzZkyjbTds2MCvf/1rzjzzTB566CGaNWvGJZdcwqpVqwA477zzeOSRRwC46KKLePnll5k+fbpf8a9atYqzzz6b2tpa7r77bh566CF++ctf8vnnn//k6z755BNGjBhBSUkJU6ZMYdKkSXzxxRcMHjyYLVu2NNr+wgsvpLKykqlTp3LhhRfy4osvctdddx1znOeddx6GYfDWW295182cOZNOnTrRp0+fRttv2rSJd955h7PPPpuHH36YG2+8kRUrVnDqqad6G4GdO3fm7rvvBuDKK6/k5Zdf5uWXX+aUU07x7mffvn2MHDmSXr16MX36dE477bTDxvfoo4/SokULxo0bh9vtBuDZZ5/l448/5vHHHyc3N/eYcxURsZwpImGnoqLCBMxzzjnnmLZftmyZCZiXX365z/obbrjBBMy5c+d617Vp08YEzPnz53vXlZSUmA6Hw7z++uu96zZv3mwC5gMPPOCzz3Hjxplt2rRpFMOdd95p/vAr5ZFHHjEBc8+ePUeM+9B7vPDCC951vXr1MjMzM819+/Z5133zzTdmVFSU+fvf/77R+1166aU++/zVr35lZmRkHPE9f5hHUlKSaZqm+etf/9o844wzTNM0TbfbbWZnZ5t33XXXYf8GNTU1ptvtbpSHw+Ew7777bu+6wsLCRrkdcuqpp5qA+cwzzxz2uVNPPdVn3UcffWQC5l/+8hdz06ZNZnJysnnuueceNUcRkXCjiqdIGHK5XACkpKQc0/b//e9/AZg0aZLP+uuvvx6gUV/QLl26MGTIEO/jFi1a0LFjRzZt2hRwzD92qG/ou+++i8fjOabX7Nq1i2XLlnHJJZeQnp7uXd+jRw/OPPNMb54/9Ic//MHn8ZAhQ9i3b5/3b3gsLr74Yj799FOKi4uZO3cuxcXFh73MDgf7hUZFHfzqdLvd7Nu3z9uNYMmSJcf8ng6Hg/Hjxx/TtsOHD+eqq67i7rvv5rzzziM+Pp5nn332mN9LRCRcqOEpEoZSU1MBqKysPKbtt27dSlRUFB06dPBZn52dTVpaGlu3bvVZ37p160b7aNasGWVlZQFG3NhvfvMbBg8ezOWXX05WVhZjxozh9ddf/8lG6KE4O3bs2Oi5zp07s3fvXqqrq33W/ziXZs2aAfiVy1lnnUVKSgqvvfYar7zyCv3792/0tzzE4/HwyCOPcMIJJ+BwOGjevDktWrRg+fLlVFRUHPN7tmzZ0q+BRA8++CDp6eksW7aMxx57jMzMzGN+rYhIuFDDUyQMpaamkpuby8qVK/163Y8H9xxJdHT0Ydebphnwexzqf3hIQkIC8+fP55NPPuF3v/sdy5cv5ze/+Q1nnnlmo22Px/HkcojD4eC8885jxowZvP3220esdgLcd999TJo0iVNOOYV//vOffPTRR8yePZuuXbsec2UXDv59/LF06VJKSkoAWLFihV+vFREJF2p4ioSps88+m40bN7Jw4cKjbtumTRs8Hg/r16/3Wb97927Ky8u9I9SbQrNmzXxGgB/y46oqQFRUFGeccQYPP/wwq1ev5t5772Xu3Ln873//O+y+D8W5du3aRs+tWbOG5s2bk5SUdHwJHMHFF1/M0qVLqaysPOyArEPeeOMNTjvtNJ5//nnGjBnD8OHDGTZsWKO/ybH+CDgW1dXVjB8/ni5dunDllVcybdo0CgsLm2z/IiKhooanSJi66aabSEpK4vLLL2f37t2Nnt+4cSOPPvoocPBSMdBo5PnDDz8MwKhRo5osrvbt21NRUcHy5cu963bt2sXbb7/ts11paWmj1x6aSP3HUzwdkpOTQ69evZgxY4ZPQ27lypV8/PHH3jyD4bTTTuOee+7hiSeeIDs7+4jbRUdHN6qm/utf/2LHjh0+6w41kA/XSPfXzTffzLZt25gxYwYPP/ww+fn5jBs37oh/RxGRcKUJ5EXCVPv27Zk5cya/+c1v6Ny5s8+di7744gv+9a9/cckllwDQs2dPxo0bx3PPPUd5eTmnnnoqixYtYsaMGZx77rlHnKonEGPGjOHmm2/mV7/6Fddccw379+/n6aef5sQTT/QZXHP33Xczf/58Ro0aRZs2bSgpKeGpp56iVatWnHzyyUfc/wMPPMDIkSMpKCjgsssu48CBAzz++OM4nU6mTJnSZHn8WFRUFH/+85+Put3ZZ5/N3Xffzfjx4xk0aBArVqzglVdeoV27dj7btW/fnrS0NJ555hlSUlJISkpi4MCBtG3b1q+45s6dy1NPPcWdd97pnd7phRdeYOjQodx+++1MmzbNr/2JiFhJFU+RMPbLX/6S5cuX8+tf/5p3332XCRMmcMstt7BlyxYeeughHnvsMe+2f//737nrrrsoLCzkuuuuY+7cuUyePJlZs2Y1aUwZGRm8/fbbJCYmctNNNzFjxgymTp3K6NGjG8XeunVr/vGPfzBhwgSefPJJTjnlFObOnYvT6Tzi/ocNG8aHH35IRkYGd9xxBw8++CAnnXQSn3/+ud+NtmC49dZbuf766/noo4+49tprWbJkCf/5z3/Iy8vz2S42NpYZM2YQHR3NH/7wBy666CLmzZvn13tVVlZy6aWX0rt3b2677Tbv+iFDhnDttdfy0EMP8eWXXzZJXiIioWCY/vTAFxEREREJkCqeIiIiIhISaniKiIiISEio4SkiIiIiIaGGp4iIiIiEhBqeIiIiIhISaniKiIiISEhE9ATyHo+HnTt3kpKS0qS3pxMRERFrmaZJZWUlubm5REWFV52spqaGurq6oL5HXFwc8fHxQX0PK0R0w3Pnzp2NJm0WERER+ygqKqJVq1ZWh+FVU1ND2zbJFJe4g/o+2dnZbN682XaNz4hueKakpAAwfX4vEpKjLY7m+LzaJ3xOKhEREas1UM8C/uv9vz5c1NXVUVziZuvifFJTglOJdVV6aNN3C3V1dWp4hpNDl9cTkqNJSI7oVIgxYq0OQUREJHx8d1/FcO1Kl5xikJwSnNg8hGfOTSG8Ok2IiIiIiG1FdplQRERExAJu04PbDN6+7UoVTxEREREJCVU8RURERPzkwcRDcEqewdpvOFDFU0RERERCQhVPERERET958BCsnpjB27P1VPEUERERkZBQxVNERETET27TxG0Gpy9msPYbDlTxFBEREZGQUMVTRERExE8a1R4YVTxFREREJCRU8RQRERHxkwcTtyqeflPFU0RERERCQhVPERERET+pj2dgVPEUERERkZBQxVNERETET5rHMzCqeIqIiIhISPxsK54Hqty8+eh2Fs8uxbWvnjZdkvjtbW1o1yMZgN+f+NVhX/ebm/IYdXmuz7r6Og93/XoV29bs5553utGmS1LQ4/dXkbmBrayjjhqScdKR3jiNdKvD8kuZuYetrMNFGXXU0IMCMo2WVocVMDsck0PskItdPl92yQPsk8tmcw172EE1lUQRTRoZdKA7SUaK1aEFxA7ne1PwfLcEa992FRYVzyeffJL8/Hzi4+MZOHAgixYtCvp7Pn/bJlZ9XsFVD7Tnvvd70G2wk/svWUNpcR0Aj33e22e5fGo7DAP6D298cr02bRtpmbFBjzlQxWYR61hOO7owgGGkkMZSPqPOrLE6NL+4aSAZJ53obXUox80uxwTsk4tdPl92yQPsk0s5e2hFe/pzGn0YggcPS/kMt9lgdWh+s8v5LtaxvOH52muvMWnSJO68806WLFlCz549GTFiBCUlJUF7z7oaD19/XMpvbsyjU/9UstrEc941rchq42Duq7sBSGsR57Ms+aSMzgNTyWwd77Ovb+aVs2JBBRfd0jpo8R6vbayjJW3JNfJJNlLpRB+iiWYnW6wOzS/NjRw6GN0isuLxY3Y5JmCfXOzy+bJLHmCfXHobQ747P5ykGGl0pT817MdFmdWh+c0u53tTcH83j2ewFruyvOH58MMPc8UVVzB+/Hi6dOnCM888Q2JiIv/4xz+C9p7uBhOPG2IdvunHOqJYt7iy0fYVe+v5Zl45p1zQotH6f/x5E1c90J64+OigxXs8PKaHSspJJ9O7zjAM0sminH0WRvbzZadjYqdcREKlgXoAYomzOBL/6HyXpmBpw7Ouro7FixczbNgw77qoqCiGDRvGwoULg/a+CcnRdOidzLtP7aBsdx0et8nn7+5lw7IqyvfUN9p+wdt7iE+Kot8PLrObpsnfbt7I6Rdl0a57ctBiPV711GJiEodvpTYOB3Xo0ogV7HRM7JSLSCiYpsk6luEkg2TDaXU4ftH57sttBnexK0sbnnv37sXtdpOVleWzPisri+Li4kbb19bW4nK5fJZAXfVAe0wTrh2ylEu7LWL2S8UUnJ2BYTTedv4beygY3Zy4H1RIZ7+8mwPVbkZfldv4BSIiIoexhqVU4aI7A60ORcQSETWqferUqdx1111Nsq+s1vHc9koXave7OVDlJi0zjieuXU9mnu8vubWFLnZtrmHC9A4+61cvdLFhWRWXdvMdCHXn+SspGN2cq6a1b5I4j1csDgyMRr9G66ht9KtVQsNOx8ROuYgE2xpzKXvZRT+GEm8kWh2O33S++9Ko9sBYWvFs3rw50dHR7N6922f97t27yc7ObrT95MmTqaio8C5FRUXHHYMjMZq0zDiqKxpYuaCCPmc083l+3ht7yO+WROvOvlMk/fb2Ntz7Xnf+8u7B5fq/dQRgwvQTuGBSq+OOq6lEGVGkkEYp3w/WMk2TUkpII8PCyH6+7HRM7JSLSLCYpskacyl72EFfTiHBCL8p946FzndpCpZWPOPi4ujbty9z5szh3HPPBcDj8TBnzhwmTpzYaHuHw4HD4WiS917+WTmYkNM2nt3bapl1/zZy2sUz5Pzm3m0OVDWw6MNSLj7MiPXmub5xOBIPDi7KzHOQnt00MTaV1pzIagpJNZvhJJ1trMdNAznkWx2aXxrMBg5Q5X18gGoqzXJiiYu46oFdjgnYJxe7fL7skgfYJ5e1LKWYInoyiGhiqf1u6qEYYok2wnNg6pHY5XxvCh4M3Bymf14T7duuLL/UPmnSJMaNG0e/fv0YMGAA06dPp7q6mvHjxwf1fQ9UuvnXQ0WUFteRlBZD/+Hp/HpSK2Jivy8Cf/l+KZhw0tmR/Usu28ij3qxlE6uppYYUnPTmZBxGZF0acVHKEuZ7H69nOQA5tKEr/a0KKyB2OSZgn1zs8vmySx5gn1y2swmAxczzWd+FfuRGWIPNLue7WMcwTetvCPrEE0/wwAMPUFxcTK9evXjssccYOPDoHa9dLhdOp5Nnl/QlIdnyNvRxealjntUhiIiIhI0Gs55PeZeKigpSU1OtDsfrUNvj61VZJKcEp8diVaWHfl13h13uTSEsWmsTJ0487KV1EREREbGPsGh4ioiIiEQSdxD7eAZrv+HA8jsXiYiIiMjPgyqeIiIiIn5SxTMwqniKiIiISEio4ikiIiLiJ49p4DGDNI9nkPYbDlTxFBEREZGQUMVTRERExE/q4xkYVTxFREREJCRU8RQRERHxk5so3EGq37mDstfwoIqniIiIiISEKp4iIiIifjKDOKrd1Kh2EREREZHjo4qniIiIiJ80qj0wqniKiIiISEio4ikiIiLiJ7cZhdsM0qh2Myi7DQuqeIqIiIhISKjiKSIiIuInDwaeINXvPNi35KmKp4iIiIiEhC0qnq/2aUWMEWt1GMflo53LrA6hyYzI7WV1CCIiIkGlUe2BUcVTRERERELCFhVPERERkVAK7qh29fEUERERETkuaniKiIiI+OngqPbgLf6YP38+o0ePJjc3F8MweOedd3yeN02TO+64g5ycHBISEhg2bBjr16/32aa0tJSxY8eSmppKWloal112GVVVVT7bLF++nCFDhhAfH09eXh7Tpk3z+++mhqeIiIhIBKuurqZnz548+eSTh31+2rRpPPbYYzzzzDN89dVXJCUlMWLECGpqarzbjB07llWrVjF79mzef/995s+fz5VXXul93uVyMXz4cNq0acPixYt54IEHmDJlCs8995xfsaqPp4iIiIifPEThDpN5PEeOHMnIkSMP+5xpmkyfPp0///nPnHPOOQC89NJLZGVl8c477zBmzBi+/fZbPvzwQwoLC+nXrx8Ajz/+OGeddRYPPvggubm5vPLKK9TV1fGPf/yDuLg4unbtyrJly3j44Yd9GqhHo4qniIiIiJ8ODS4K1gIHq4w/XGpra/2Oc/PmzRQXFzNs2DDvOqfTycCBA1m4cCEACxcuJC0tzdvoBBg2bBhRUVF89dVX3m1OOeUU4uLivNuMGDGCtWvXUlZWdszxqOEpIiIiEoby8vJwOp3eZerUqX7vo7i4GICsrCyf9VlZWd7niouLyczM9Hk+JiaG9PR0n20Ot48fvsex0KV2ERERET95iAr6LTOLiopITU31rnc4HEF5v1BSxVNEREQkDKWmpvosgTQ8s7OzAdi9e7fP+t27d3ufy87OpqSkxOf5hoYGSktLfbY53D5++B7HQg1PERERET+5TSOoS1Np27Yt2dnZzJkzx7vO5XLx1VdfUVBQAEBBQQHl5eUsXrzYu83cuXPxeDwMHDjQu838+fOpr6/3bjN79mw6duxIs2bNjjkeNTxFREREIlhVVRXLli1j2bJlwMEBRcuWLWPbtm0YhsF1113HX/7yF9577z1WrFjB73//e3Jzczn33HMB6Ny5M7/4xS+44oorWLRoEZ9//jkTJ05kzJgx5ObmAnDxxRcTFxfHZZddxqpVq3jttdd49NFHmTRpkl+xqo+niIiIiJ/cQZxOye3ndEpff/01p512mvfxocbguHHjePHFF7npppuorq7myiuvpLy8nJNPPpkPP/yQ+Ph472teeeUVJk6cyBlnnEFUVBTnn38+jz32mPd5p9PJxx9/zIQJE+jbty/Nmzfnjjvu8GsqJQDDNCP3hqAulwun08lQziHGiLU6nOPy0c5lVofQZEbk9rI6BBERiXANZj2f8i4VFRU+A2ysdqjt8eLSniSmRAflPfZXurmk9zdhl3tTUMVTRERExE8eMwqPGaRR7ZFbEzwq9fEUERERkZBQxVNERETET+HUxzOSqOIpIiIiIiGhiqeIiIiInzzQpPNt/njfdqWG51EUmRvYyjrqqCEZJx3pjdNItyye+QsP8ODTZSxZXsuu3W7e/Ec2545M9j7/1n+qePalCpasqKW0zMPi2Xn06uZ7p4M/3FjCnM/2s3O3m+REg4L+Cfz1tgw6nRDn3Wbb9nr+eMsePv38AMlJUfz+whTuuzWDmJjgnGTHYrO5hj3soJpKoogmjQw60J0kI8WymAKx3dzIdjZxgGoAkkmlLZ1pbuRYHFngwu08CZRd8gD75KI8wo+dcpHQs/RS+/z58xk9ejS5ubkYhsE777xjZTiNFJtFrGM57ejCAIaRQhpL+Yw6s8aymKr3e+jZxcHj97U44vMnD0xg6m0ZR9xHnx4Onn8ki1XzW/PBq7mYpskvxuzE7T7Yp8TtNhn9u13U1Zss+HcrXng0kxmvubhzWmlQcjpW5eyhFe3pz2n0YQgePCzlM9xmg6Vx+ctBAh3oxkDOYABn0IxMvuELqswKq0MLSDieJ4GwSx5gn1yUR/ixUy7H69C92oO12JWlmVVXV9OzZ0+efPJJK8M4om2soyVtyTXySTZS6UQfoolmJ1ssi2nkGUncc0sGvzor+bDP/+6CVG6flM6wUxKPuI8rf+fklIIE8vNi6dMjnntuzqBoZwNbig424D7+dD+r19Xx8hNZ9OrmYOQZSdx1UwZPvVhBXZ11HZ57G0O+OxZOUow0utKfGvbjosyymALRwsiluZFDopFCkpFCB6Mb0cRQgbUN+0CF43kSCLvkAfbJRXmEHzvlItawtOE5cuRI/vKXv/CrX/3KyjAOy2N6qKScdDK96wzDIJ0sytlnYWRNq3q/hxdnuWjbOoa83IM9L75cXEP3znFktfi+J8aI0xJxVXpYtbbOqlAbaeDg/WJjiTvKluHLNE2KzSLcuHFy5Cp1uLLLeWKXPMA+uSiP8GOnXJqC24wK6mJX6uN5BPXUYmISR7zP+jgcVOOyKKqm8/SLFdx8z16q95t0bB/LR6+1JC7uYP/N4hI3mc1978aQ9d3j4pIGwPHj3YWcaZqsYxlOMkg2nFaH47cqs4JC5uLBQzQx9KSAZCPy7k5hl/PELnmAfXJRHuHHTrmIdSKqSV1bW4vL5fJZJDAXn5fM4tl5/O+tlpzQPpYxVxZTUxM54+jWsJQqXHRnoNWhBCSRFAZyJv05nVa0YxWFVJn6PIuIRAoPRlAXu4qohufUqVNxOp3eJS8vL2jvFYsDA4M6fDtM11Hb6NdeJHKmRnNCuzhOKUjgX3/LYc2GOt7+4OAo6+zMaEr2un223/3d4+xM64vka8yl7GUXfTmVeOPIfVnDWZQRRaKRTKrRjA5Gd1JIo4j1VoflN7ucJ3bJA+yTi/IIP3bKRawTUQ3PyZMnU1FR4V2KioqC9l5RRhQppFFKiXedaZqUUkJaBPbF+ymmeXCp/W7g0El941nxbR0le78fLT573n5SU6LocqJ1/SlN02SNuZQ97KAvp5BgJFkWS1MzMfFE4MxtdjlP7JIH2CcX5RF+7JRLU1Afz8BYX77yg8PhwOEIXf/C1pzIagpJNZvhJJ1trMdNAznkhyyGH6uq9rBhc7338ZZtDSxbWUt6WhStW8VSWuZm244Gdu4+2Ghcu/HgYKDszGiyM2PYtLWe19+t5MxTE2mREc32XQ3c/0QZCQkGZ51xsHo4fGgiXU6M4/cTd3P/7c0pLmngjvtL+eMlThwO68r/a1lKMUX0ZBDRxFL73fQdMcQSbUQf5dXhY4O5ggyyiScRNw0Us40y9tCbIVaHFpBwPE8CYZc8wD65KI/wY6dcxBqWNjyrqqrYsGGD9/HmzZtZtmwZ6enptG7d2sLIDso28qg3a9nEamqpIQUnvTkZh2HdJYWvv6nhjPN3eh9fP2UvAL+/MIUXHs3ivY+ruey673+NXvyH3QDccX0z7rwhg3iHwWdf1fDo3yooq3CT1SKGIQPjWfBeKzKbH/w4REcbvPdSDn+8ZQ+Dz95OUqLB7y9M5a6brJ0geDubAFjMPJ/1XehHbgR96dVRyyoKqaWGGGK/+1wNIcPIsjq0gITjeRIIu+QB9slFeYQfO+VyvIJ7r3b7VjwN0zQtm5jx008/5bTTTmu0fty4cbz44otHfb3L5cLpdDKUc4gxYoMQYeh8tHOZ1SE0mRG5vawOQUREIlyDWc+nvEtFRQWpqeEz68ehtseDX59MQnJw6ncHqhq4od+CsMu9KVha8Rw6dCgWtntFREREAuIxDTzBuld7kPYbDuxbyxURERGRsBJRg4tEREREwoEniH08da92EREREZHjpIqniIiIiJ88ZhSeIM23Gaz9hgP7ZiYiIiIiYUUVTxERERE/uTFwB+me6sHabzhQxVNEREREQkIVTxERERE/qY9nYOybmYiIiIiEFVU8RURERPzkJnh9Md1B2Wt4UMVTREREREJCFU8RERERP6mPZ2Dsm5mIiIiIhBVVPEVERET85DajcAepMhms/YYD+2YmIiIiImFFFU8RERERP5kYeII0qt3UnYtERERERI6PKp4iIiIiflIfz8DYNzMRERERCSuqeIaJEbm9rA6hydy2aZnVITSJe9v1sjoEEREJUx7TwGMGpy9msPYbDlTxFBEREZGQUMVTRERExE9uonAHqX4XrP2GA/tmJiIiIiJhRRVPERERET+pj2dgVPEUERERkZBQxVNERETETx6i8ASpfhes/YYD+2YmIiIiImFFFU8RERERP7lNA3eQ+mIGa7/hQBVPEREREQkJVTxFRERE/KRR7YFRxVNEREREQkIVTxERERE/mWYUHjM49TszSPsNB/bNTERERETCiiqeIiIiIn5yY+AmSKPag7TfcKCKp4iIiIiEhCqeIiIiIn7ymMEbfe4xg7LbsKCKp4iIiIiEhCqeIiIiIn7yBHFUe7D2Gw7U8DyCzeYa9rCDaiqJIpo0MuhAd5KMFKtDC0iRuYGtrKOOGpJx0pHeOI10y+L55qsDvPZcGetW1rKvxM09z2Zz8vBk7/OmafLCI6X8Z5aLKpeHbv3i+dM9LWjVNg6A4u31vPR4KUu/OEDpHjfNs6IZdm4Kv52QTmzcwUsf2zbW8cif97B1fR1VlR6aZ0Vzxi9TGHdtOjGx1nXc3m5uZDubOEA1AMmk0pbONDdyLIspUGXmHrayDhdl1FFDDwrINFpaHVbAwu08OR6RnoudvoN1noh8z75N6uNUzh5a0Z7+nEYfhuDBw1I+w202WB2a34rNItaxnHZ0YQDDSCGNpXxGnVljWUw1Bzy07+zg2rtbHPb5Wc+W89aLFfzpLy146u1WxCdEcdO4ndTVeoCDjUrTA5PuzeSFj1vzxz+34N+vuPj7A/u8+4iJNRh+XgrTXsrlpTmtmXB7C/4zy8UL00tDkuOROEigA90YyBkM4Ayakck3fEGVWWFpXIFw00AyTjrR2+pQjls4nieBskMudvoO1nliTx6MoC52ZWnFc+rUqbz11lusWbOGhIQEBg0axP3330/Hjh2tDAuA3sYQn8ddzf7M59+4KKMZh28shattrKMlbck18gHoZPZhL7vYyRby6WRJTAOHJjFwaNJhnzNNkzf+Uc7vJjbzVkEnP5TJef23sODjak4fncKAU5MYcOr3r89tHUvRpjTee6WCq29r7l2X2zrWu012q1iWfZXMikUHgpjZ0bUwcn0ed6Ab282NVFBKMk6LogpMcyOH5nxXqY3wzvDheJ4Eyg652Ok7WOeJPblNA3eQBhcFa7/hwNKK57x585gwYQJffvkls2fPpr6+nuHDh1NdXW1lWIfVQD0AscRZHIl/PKaHSspJJ9O7zjAM0sminH0/8Urr7CpqoHSPm74nJ3rXJadG07mXg1VLjvyrurrSQ0pa9BGf37GljsJ5++k5MKFJ4z0epmlSbBbhxo2TDKvD+dmKxPPkSOyUyw9F6newndj1syWhZWnF88MPP/R5/OKLL5KZmcnixYs55ZRTLIqqMdM0WccynGSQbERWRaqeWkxM4oj3WR+Hg2pcFkX100r3HLyU1qy5byOyWfMYSve4D/uaHVvqePulCv4wuXHjbeL521m3spb6OpOzL0pl/CTr+yJVmRUUMhcPHqKJoScFJBupVof1sxWJ58mR2CmXQyL5O9hO7PjZOh4aXBSYsBpcVFFxsI9bevrhGwa1tbXU1tZ6H7tcofmgr2EpVbjox9CQvJ/4Z09xAzddsotTRyZz9kWN/1O64/Es9lebbPy2lmem7uW152K56A/NLIj0e4mkMJAzaaCeErazikL6mkPV+BQ5DH0Hi9hH2DSpPR4P1113HYMHD6Zbt26H3Wbq1Kk4nU7vkpeXF/S41phL2csu+nIq8Ubi0V8QZmJxYGBQh+8l6jpqG/1qDRfpLQ7+Hirb61vdLNvbQHoL3yro3t0NTLpoB137xHP91MP3+8rMjSX/hDjO+GUKV97UnBmPluJ2W9vRKsqIItFIJtVoRgejOymkUcR6S2P6OYvE8+RI7JQLRP53sJ3Y7bN1vDwYeMwgLTYeXBQ2Dc8JEyawcuVKZs2adcRtJk+eTEVFhXcpKioKWjymabLGXMoedtCXU0gwDj8QJtxFGVGkkEYpJd51pmlSSglpYdqnMCcvhvQW0Sz5fL93XXWlh2+X1dK1z/dfbnuKG/jTmB2c2N3BzQ9kEhV19BPV4zFpaDAxPUEJPWAmJh7CLKifkUg8T47ELrnY5TvYTuzy2RJrhcWl9okTJ/L+++8zf/58WrVqdcTtHA4HDocjJDGtZSnFFNGTQUQTS+13U0XEEEu0ceQBLOGoNSeymkJSzWY4SWcb63HTQA75lsV0oNrDjq313se7ihrYsLqWFGcUWS1j+fWlabz8RBkt8+PIyYvhHw+X0jwrmpOHH/zPZ09xA3+6aAdZLWP4w63NqSj9vjp6qGI6+51KYmKgXScHsXEGa5fX8PcH9nHaqGRL5/HcYK4gg2ziScRNA8Vso4w99GbI0V8cZhrMBg5Q5X18gGoqzXJiiYu46lQ4nieBskMudvoO1nliT2YQpz0ybVzxtLThaZom//d//8fbb7/Np59+Stu2ba0Mx8d2NgGwmHk+67vQj9wIO8GyjTzqzVo2sZpaakjBSW9OxmFYd2lk7Yoa/nTRTu/jp/6yF4AR56dwy4NZjLkqjQP7PTx0awlVLg/d+8dz/4u5xDkOFukXL9jPji317NhSz4UFW3z2/b/NHQCIjoFXny1n++Y6TBOyWsZy7u+dXHBZWkhyPJI6allFIbXUEEPsd8djCBlGlqVxBcJFKUuY7328nuUA5NCGrvS3KqyAhON5Eig75GKn72CdJyLfM0zTtKyz2x//+EdmzpzJu+++6zN3p9PpJCHh6FPeuFwunE4nQzmHGCP2qNtLaNy2aZnVITSJe9v1sjoEEZGfrQaznk95l4qKClJTw2fg5aG2x/mfjCM2KTjTe9VX1/HmsBlhl3tTsLSP59NPP01FRQVDhw4lJyfHu7z22mtWhiUiIiIiQWD5pXYRERGRSKN5PANj38xEREREJKyExah2ERERkUhyaM7NYO3brlTxFBEREZGQUMVTRERExE+eIM7jqTsXiYiIiIgcJ1U8RURERPykPp6BUcVTREREREJCFU8RERERP6niGRhVPEVEREQkJFTxFBEREfGTKp6BUcVTREREJIK53W5uv/122rZtS0JCAu3bt+eee+7xuTW5aZrccccd5OTkkJCQwLBhw1i/fr3PfkpLSxk7diypqamkpaVx2WWXUVVV1aSxquEpIiIi4qdDFc9gLf64//77efrpp3niiSf49ttvuf/++5k2bRqPP/64d5tp06bx2GOP8cwzz/DVV1+RlJTEiBEjqKmp8W4zduxYVq1axezZs3n//feZP38+V155ZZP9zUCX2kVEREQi2hdffME555zDqFGjAMjPz+fVV19l0aJFwMFq5/Tp0/nzn//MOeecA8BLL71EVlYW77zzDmPGjOHbb7/lww8/pLCwkH79+gHw+OOPc9ZZZ/Hggw+Sm5vbJLGq4ikiIiLiJ5Pv717U1It51Hf3NWjQIObMmcO6desA+Oabb1iwYAEjR44EYPPmzRQXFzNs2DDva5xOJwMHDmThwoUALFy4kLS0NG+jE2DYsGFERUXx1VdfHdff6odU8RQREREJQy6Xy+exw+HA4XA02u6WW27B5XLRqVMnoqOjcbvd3HvvvYwdOxaA4uJiALKysnxel5WV5X2uuLiYzMxMn+djYmJIT0/3btMUVPEUERER8VMo+njm5eXhdDq9y9SpUw8by+uvv84rr7zCzJkzWbJkCTNmzODBBx9kxowZofyTHBNVPEVERETCUFFREampqd7Hh6t2Atx4443ccsstjBkzBoDu3buzdetWpk6dyrhx48jOzgZg9+7d5OTkeF+3e/duevXqBUB2djYlJSU++21oaKC0tNT7+qagiqeIiIiIn0JR8UxNTfVZjtTw3L9/P1FRvk266OhoPB4PAG3btiU7O5s5c+Z4n3e5XHz11VcUFBQAUFBQQHl5OYsXL/ZuM3fuXDweDwMHDmyyv5sqntLk7m3Xy+oQmsT1G1ZZHUKTeahDV6tDEBGRIBk9ejT33nsvrVu3pmvXrixdupSHH36YSy+9FADDMLjuuuv4y1/+wgknnEDbtm25/fbbyc3N5dxzzwWgc+fO/OIXv+CKK67gmWeeob6+nokTJzJmzJgmG9EOaniKiIiI+C2c7lz0+OOPc/vtt/PHP/6RkpIScnNzueqqq7jjjju829x0001UV1dz5ZVXUl5ezsknn8yHH35IfHy8d5tXXnmFiRMncsYZZxAVFcX555/PY4891mR5ARjmD6e1jzAulwun08lQziHGiLU6HLEZVTxFRKzTYNbzKe9SUVHh08/RaofaHqf8+4/EJB3+0vfxaqiuZf7op8Iu96agiqeIiIiIn8Kp4hlJNLhIREREREJCFU8RERERP5mmgRmkymSw9hsOVPEUERERkZBQxVNERETET4fuqx6sfduVKp4iIiIiEhKqeIqIiIj4SaPaA6OKp4iIiIiEhCqeIiIiIn7SqPbAqOIpIiIiIiGhiqeIiIiIn9THMzCqeIqIiIhISKjiKSIiIuIn9fEMjCqeIiIiIhISqniKiIiI+MkMYh9PVTxFRERERI6TKp4iIiIifjIB0wzevu1KDc+jKDI3sJV11FFDMk460hunkW51WH6zSx4QfrmsWLSfN/62jw0raygtaeD2p1sxaHiK93nTNHl5+l4+fK2MapeHLn0TmHh3Di3bxnm3efXJvRT+r4pN39YQE2vwxrKOjd6nZGc9T9y+i+Vf7ic+MYph5zkZf2Mm0THWX5IJt2MSKLvkAfbJRXmEHzvlIqFn6aX2p59+mh49epCamkpqaioFBQV88MEHVobko9gsYh3LaUcXBjCMFNJYymfUmTVWh+YXu+QB4ZlLzX4P7To5+OOUrMM+/6/n9vHejFL+754cpr+VT3xiFH8ev426Wo93m4Z6kyFnpTBqbLPD7sPtNrnzsiIa6uGhf+Vz/QO5zH6rgpen7wlKTv4Ix2MSCLvkAfbJRXmEHzvlcrw8GEFd7MrShmerVq3461//yuLFi/n66685/fTTOeecc1i1apWVYXltYx0taUuukU+ykUon+hBNNDvZYnVofrFLHhCeufQfmsy46zMZPCK10XOmafLOC6WMmdCcgjNTaNspnhsezGXf7ga++LjSu93vrmvBry7NIP9Ex2HfY8ln1WzbUMuND+fSvks8/Ycm8/s/teDfL5dRX2ftRZlwPCaBsEseYJ9clEf4sVMuYg1LG56jR4/mrLPO4oQTTuDEE0/k3nvvJTk5mS+//NLKsADwmB4qKSedTO86wzBIJ4ty9lkYmX/skgdEZi7FRfWU7XHTe3CSd11SSjQdeyWwZumBY97Pt0sPkN/RQbPm3/eO6Tskif1VHraur23SmP0RicfkcOySB9gnF+URfuyUS1M4NI9nsBa7CptR7W63m1mzZlFdXU1BQYHV4VBPLSYmccT7rI/DQR2Rc0nBLnlAZOZStqcBgGbNo33WN2se7X3uWPeT1ty3S/ahx/7sp6lF4jE5HLvkAfbJRXmEHzvlItaxfHDRihUrKCgooKamhuTkZN5++226dOly2G1ra2uprf2+uuNyuUIVpoiIiIiXxzQwdK92v1le8ezYsSPLli3jq6++4uqrr2bcuHGsXr36sNtOnToVp9PpXfLy8oIWVywODIxGv+LqqG30ay+c2SUPiMxcmrX4riq51+2zvmyv2/vcse6nfK9vZfPQY3/209Qi8Zgcjl3yAPvkojzCj51yEetY3vCMi4ujQ4cO9O3bl6lTp9KzZ08effTRw247efJkKioqvEtRUVHQ4ooyokghjVJKvOtM06SUEtLICNr7NjW75AGRmUt2XizNWkSz7Itq77rqSjdrlx2gU++EY95P594JbFlb69P4XLKgmsTkKFp3iPuJVwZXJB6Tw7FLHmCfXJRH+LFTLk3BNIO72JXll9p/zOPx+FxO/yGHw4HDcfhRv8HQmhNZTSGpZjOcpLON9bhpIIf8kMXQFOySB4RnLgeqPezcWud9vHt7HRtX15CSFk1mbiznjk9n1pN7aZkfR1ZeLC8/vIeMrBifuT5LdtZTWe6mZFc9Hg9sXH2wopDbJo6EpCj6DEmidQcHD9ywk8tuzqRsTwMvPbyH0b9rRpzD2t+P4XhMAmGXPMA+uSiP8GOnXMQaljY8J0+ezMiRI2ndujWVlZXMnDmTTz/9lI8++sjKsLyyjTzqzVo2sZpaakjBSW9OxmFE1iUFu+QB4ZnL+hUHuHnsNu/j5+49WA0Ydp6T6x/I5YIrM6jZb/LYbbuocnno2i+Be17I82kwvvzIHj55q8L7eOLozQDc/0prepyURHS0wZS/5/HE7buY9OstOBKjGPYrJ7+7rkWIsjyycDwmgbBLHmCfXJRH+LFTLscrmKPP7Tyq3TBN6wq6l112GXPmzGHXrl04nU569OjBzTffzJlnnnlMr3e5XDidToZyDjFGbJCjlZ+b6zeEx3yyTeGhDl2tDkFExC8NZj2f8i4VFRWkpjaeJ9kqh9oeXWbdRHRicK7CuvfXsnrMtLDLvSlYWvF8/vnnrXx7ERERkYCo4hkYywcXiYiIiMjPQ9gNLhIREREJd5rHMzCqeIqIiIhISKjiKSIiIuKnYM63aed5PFXxFBEREZGQUMVTRERExE8HK57BGtUelN2GBTU8RURERPyk6ZQCo0vtIiIiIhISqniKiIiI+Mn8bgnWvu1KFU8RERERCQlVPEVERET8pD6egVHFU0RERERCQhVPEREREX+pk2dAVPEUERERkZBQxVNERETEX0Hs44n6eIqIiIiIHB9VPEVERET8dPCWmcHbt12p4ikiIiIiIaGKp4iIiIifNI9nYNTwFDmChzp0tTqEJjN+7VarQ2gSL3RsY3UIIiJyHNTwFBEREfGXaQRv9LmNK57q4ykiIiIiIaGKp4iIiIifNKo9MKp4ioiIiEhIqOIpIiIi4i/dqz0gqniKiIiISEio4ikiIiLiJ83jGRhVPEVEREQkJFTxFBEREQmEjftiBosqniIiIiISEqp4ioiIiPhJfTwDo4qniIiIiISEKp4iIiIi/tI8ngFRxVNEREREQkIVTxERERG/Gd8twdq3PaniKSIiIiIhoYqniIiIiL/UxzMgqniKiIiISEio4nkUReYGtrKOOmpIxklHeuM00q0Oy292yKPM3MNW1uGijDpq6EEBmUZLq8MKWCQckwNVbt54dAeFs8tw7asnv0siv7utNe17JAMw9sTCw77uoptacfblOQDs2lzDzGlFrFtcRUO9h9YdE/n1dS3pelJqyPI4VpFwTI6VHXKx0zlvh+Ox2VzDHnZQTSVRRJNGBh3oTpKRYnVo1lDFMyCqeP6EYrOIdSynHV0YwDBSSGMpn1Fn1lgdml/skoebBpJx0oneVody3CLlmPztts2s+LyCqx9ox1/f70b3wU6mXrKO0uI6AJ78vJfPcuXUfAwDBgxv5t3Hg1etw9NgcttLHbn37a607pTIQ1etp3xPvVVpHVakHJNjYZdc7HLO2+V4lLOHVrSnP6fRhyF48LCUz3CbDVaHJhEkbBqef/3rXzEMg+uuu87qULy2sY6WtCXXyCfZSKUTfYgmmp1ssTo0v9glj+ZGDh2MbhFb8fihSDgmdTUeCj8u46Ib8+jcP4XsNvGcf01Lsto4+OTVEgDSWsT6LIs/KafLwBQyW8cDUFlaT/GWWkZfmUPrTolk58cz5oZW1B7wsH3dfivTayQSjsmxsksudjnn7XI8ehtDvsvBSYqRRlf6U8N+XJRZHZo1TCO4i02FRcOzsLCQZ599lh49elgdipfH9FBJOelketcZhkE6WZSzz8LI/GOXPOwkUo6Ju8HE44ZYh+/XRJwjinWLqxptX7G3nmXzKjj1ghbedcnNYshpG89n7+ylZr8bd4PJ3NdKSM2IoW23pKDncKwi5ZgcCzvlYgd2Ph4NHLxqEUucxZFIJLG84VlVVcXYsWP529/+RrNmzY7+ghCppxYTkzjifdbH4aCOyLk8Ypc87CRSjklCcjQn9E7inad2Ura7Do/bZMG7e1m/rOqwl8nnv72X+KQo+v/gMrthGEye0ZGt3+7n8t5LuKT71/z3hd3c/PyJJDnDp4t5pByTY2GnXOzArsfDNE3WsQwnGSQbTqvDsYRpBnexK8sbnhMmTGDUqFEMGzbsqNvW1tbicrl8FhEJnqsfaIdpwsQh3zCu29d89FIJg85OxzjMVaB5b+xl8OgM4n5QITVNkxfv2kpqeiy3z+zE3W90od+wNB68aj1lJXUhzEREmtIallKFi+4MtDoUiTCWlhxmzZrFkiVLKCw8/MjYH5s6dSp33XVXkKM6KBYHBkajX6R11Db65RrO7JKHnUTSMclqHc/tr3SiZr+bA1VummXG8di1G8jMc/hst6awkl2ba/i/6e191q9aWMnS/5Xz3Nd9SEyOBqBt1yRWfL6cz97exy+vyglZLj8lko7J0dgpFzuw4/FYYy5lL7vox1DijUSrw7GORrUHxLKKZ1FREddeey2vvPIK8fHHdvJNnjyZiooK71JUVBS0+KKMKFJIo5QS7zrTNCmlhDQygva+Tc0uedhJJB6T+MRommXGUV3RwIoFLvqekebz/Kdv7KFtt0TadPb9T6iuxgNA1I8qpFFRBqYnfL5ZI/GYHImdcrEDOx0P0zRZYy5lDzvoyykkGOHTT1sih2UVz8WLF1NSUkKfPn2869xuN/Pnz+eJJ56gtraW6Ohon9c4HA4cDsePdxU0rTmR1RSSajbDSTrbWI+bBnLID1kMTcEueTSYDRzg+0EtB6im0iwnlriI+9UdKcdk+WcVmCbktI1n97YaZt5fRE67eE45v7l3m/1VbhZ9WMbFt+Q1ev0JvZJISo3hmZs386sJucTFR/G/1/dQsr2WXkPTQpjJ0UXKMTkWdsnFLue8XY7HWpZSTBE9GUQ0sdR+Nx1UDLFEG9FHebUNBXP0uY1HtVvW8DzjjDNYsWKFz7rx48fTqVMnbr755kaNTitkG3nUm7VsYjW11JCCk96cjMOIrMsjdsnDRSlLmO99vJ7lAOTQhq70tyqsgETKMdlf6ea1h7ZTWlxHcloM/Yc348JJLYmJ/f5iyZfv78M0YdDZjSfDTkmP5ebnT+T1R7Zz37g1NNSbtDohgUlPdWhUHbVapByTY2GXXOxyztvleGxnEwCLmeezvgv9yI2wRrRYxzDN8Bk7NXToUHr16sX06dOPaXuXy4XT6WQo5xBjxAY3OJEINn7tVqtDaBIvdGxjdQgiEiINZj2f8i4VFRWkpobPnc4OtT3yHr2bqITg/HjwHKih6No7wi73pmD5qHYREREROT47duzgt7/9LRkZGSQkJNC9e3e+/vpr7/OmaXLHHXeQk5NDQkICw4YNY/369T77KC0tZezYsaSmppKWlsZll11GVVXjeZuPxzFdan/vvfeOeYe//OUvAw7m008/Dfi1IiIiIiETRqPay8rKGDx4MKeddhoffPABLVq0YP369T7zo0+bNo3HHnuMGTNm0LZtW26//XZGjBjB6tWrvYO8x44dy65du5g9ezb19fWMHz+eK6+8kpkzZzZZasfU8Dz33HOPaWeGYeB2u48nHhERERHxw/33309eXh4vvPCCd13btm29/zZNk+nTp/PnP/+Zc845B4CXXnqJrKws3nnnHcaMGcO3337Lhx9+SGFhIf369QPg8ccf56yzzuLBBx8kNze3SWI9pkvtHo/nmBY1OkVERORnIYzu1f7ee+/Rr18/LrjgAjIzM+nduzd/+9vfvM9v3ryZ4uJin5v1OJ1OBg4cyMKFCwFYuHAhaWlp3kYnwLBhw4iKiuKrr746zj/W99THU0RERCQM/fhujbW1tYfdbtOmTTz99NOccMIJfPTRR1x99dVcc801zJgxA4Di4mIAsrKyfF6XlZXlfa64uJjMzEyf52NiYkhPT/du0xQCmk6purqaefPmsW3bNurqfG97d8011zRJYCIiIiJhKwR9PPPyfOdHvvPOO5kyZUqjzT0eD/369eO+++4DoHfv3qxcuZJnnnmGcePGBSnIwPjd8Fy6dClnnXUW+/fvp7q6mvT0dPbu3UtiYiKZmZlqeIqIiIg0gaKiIp/plI50E52cnBy6dOnis65z5868+eabAGRnZwOwe/ducnK+v1Xx7t276dWrl3ebkpISn300NDRQWlrqfX1T8PtS+5/+9CdGjx5NWVkZCQkJfPnll2zdupW+ffvy4IMPNllgIiIiImHLDPICpKam+ixHangOHjyYtWvX+qxbt24dbdocnPu4bdu2ZGdnM2fOHO/zLpeLr776ioKCAgAKCgooLy9n8eLF3m3mzp2Lx+Nh4MCBAf2JDsfvhueyZcu4/vrriYqKIjo6mtraWvLy8pg2bRq33nprkwUmIiIiIkf3pz/9iS+//JL77ruPDRs2MHPmTJ577jkmTJgAHJx16LrrruMvf/kL7733HitWrOD3v/89ubm53pmLOnfuzC9+8QuuuOIKFi1axOeff87EiRMZM2ZMk41ohwAutcfGxhIVdbC9mpmZybZt2+jcuTNOp5OioqImC0xEREQkbIXRPJ79+/fn7bffZvLkydx99920bduW6dOnM3bsWO82N910E9XV1Vx55ZWUl5dz8skn8+GHH3rn8AR45ZVXmDhxImeccQZRUVGcf/75PPbYY02VFRBAw7N3794UFhZywgkncOqpp3LHHXewd+9eXn75Zbp169akwYmIiIjI0Z199tmcffbZR3zeMAzuvvtu7r777iNuk56e3qSTxR+O35fa77vvPm/H1HvvvZdmzZpx9dVXs2fPHp577rkmD1BEREQk7ITRPJ6RxO+K5w8nFs3MzOTDDz9s0oBERERExJ4CmsdTRERE5OfMMA8uwdq3Xfnd8Gzbti2GceQS8KZNm44rIBERERGxJ78bntddd53P4/r6epYuXcqHH37IjTfe2FRxiYiIiISvMBrVHkn8bnhee+21h13/5JNP8vXXXx93QCIiIiJiT36Paj+SkSNHem/NJCIiIiLyY03W8HzjjTdIT09vqt2JiIiIiM0ENIH8DwcXmaZJcXExe/bs4amnnmrS4ERERETCkUEQR7UHZ7dhwe+G5znnnOPT8IyKiqJFixYMHTqUTp06NWlwIpb6idkbIs0LHdtYHUKT6LzYPjPAfdu3weoQRERCzu9v8SlTpgQhDBEREZEIEsw7DNn4zkV+9/GMjo6mpKSk0fp9+/YRHR3dJEGJiIiIhDUzyItN+d3wNM3D/zVqa2uJi4s77oBERERExJ6O+VL7Y489BoBhGPz9738nOTnZ+5zb7Wb+/Pnq4ykiIiI/D5pAPiDH3PB85JFHgIMVz2eeecbnsnpcXBz5+fk888wzTR+hiIiIiNjCMTc8N2/eDMBpp53GW2+9RbNmzYIWlIiIiEg4M8wgTqekiuf3/ve//wUjDhERERGxOb8HF51//vncf//9jdZPmzaNCy64oEmCEhEREQlrGtUeEL8bnvPnz+ess85qtH7kyJHMnz+/SYISEREREfvx+1J7VVXVYadNio2NxeVyNUlQIiIiImFNo9oD4nfFs3v37rz22muN1s+aNYsuXbo0SVAiIiIiYj9+Vzxvv/12zjvvPDZu3Mjpp58OwJw5c5g5cyZvvPFGkwcoIiIiEm40qj0wfjc8R48ezTvvvMN9993HG2+8QUJCAj179mTu3Lmkp6cHI0YRERERsQG/G54Ao0aNYtSoUQC4XC5effVVbrjhBhYvXozb7W7SAEVERETCjmkcXIK1b5vyu4/nIfPnz2fcuHHk5uby0EMPcfrpp/Pll182ZWwiIiIiYiN+VTyLi4t58cUXef7553G5XFx44YXU1tbyzjvvaGCRiIiI/HxoVHtAjrniOXr0aDp27Mjy5cuZPn06O3fu5PHHHw9mbCIiIiJiI8dc8fzggw+45ppruPrqqznhhBOCGZOIiIhIWNOo9sAcc8VzwYIFVFZW0rdvXwYOHMgTTzzB3r17gxmbiIiIiNjIMVc8TzrpJE466SSmT5/Oa6+9xj/+8Q8mTZqEx+Nh9uzZ5OXlkZKSEsxYQ2qzuYY97KCaSqKIJo0MOtCdJCMycywyN7CVddRRQzJOOtIbpxGZ019Fei6mabLJXMUutlJHDQ4SyDHyaUtnDCMyRzKG8zHxuD0senYla/+7lep9NSS1iKfz6Lb0v7yr9+9dt7+eLx5bzqZPt1NTUUdqbhI9LzqR7r/u4N1PQ62bBQ8vZf3H23DXeWhdkM3Qyf1IzIi3KrXDKjP3sJV1uCijjhp6UECm0dLqsAIWzp+tY6VjYlPq4xkQv0e1JyUlcemll7JgwQJWrFjB9ddfz1//+lcyMzP55S9/6de+pkyZgmEYPkunTp38DSkoytlDK9rTn9PowxA8eFjKZ7jNBqtD81uxWcQ6ltOOLgxgGCmksZTPqDNrrA7Nb3bIZQtr2M5GOhl9KDB+QQejB1vNtRSxwerQAhLux2Txi9+y4o0NnHJzX3775kgGX9OLJTPWsHzWeu82Cx5ayrYvdjH8Lyfx2zdH0uviE5l3/2I2zdvh3eazh5ay+bOd/OL+wZz3t9Op3nOA/96wwIqUfpKbBpJx0oneVody3ML9s3WsdExEvhfwdEoAHTt2ZNq0aWzfvp1XX301oH107dqVXbt2eZcFC8Lji7y3MYRcI59kw0mKkUZX+lPDflyUWR2a37axjpa0/S6fVDrRh2ii2ckWq0Pzmx1yqTD30YJcmhs5JBhJZBmtSCcLl1lqdWgBCfdjUvzNPtqd2pK2Q3JJzU2mw7A8Wp+Uze6V+7zb7Fq+j06j82nVL4vU3GS6nd+B5iekebepraxj9TubGDKpN3kDssjsks4ZUway65u9FC8Pry5HzY0cOhjdIrqidki4f7aOlY6JTZnf9/Ns6kUVz6OIjo7m3HPP5b333vP7tTExMWRnZ3uX5s2bN0VITa6BegBiibM4Ev94TA+VlJNOpnedYRikk0U5+37ileHHLrk4jQxKKaHarASg0iyngr1kGNkWR+a/SDgm2T0zKFq0m7KtLgD2rCtj57I9tBmc490mp0cGm+ftpKpkP6Zpsr1wN+XbKml90sFjUvJtGZ4GD3kDs7yvSW+bSkp2IruWh0eedhMJn62fGx0TaQoB3bmoKa1fv57c3Fzi4+MpKChg6tSptG7d2uqwfJimyTqW4SSDZMNpdTh+qacWE5M4fPuhxeGgGpdFUQXGLrnk04kG6llofohhGpiYtDe6kWO0sTo0v0XCMek3vgt11Q3887z/EhVt4HGbFEzoQcez8r3bnHpzX+b+pZAXfvEeUTEGGAan396fln0P/ge7f98BomKjcKT4/vBMyIhn/74DoUznZyMSPls/NzomP6I+ngGxtOE5cOBAXnzxRTp27MiuXbu46667GDJkCCtXrjzsQKXa2lpqa2u9j12u0HzQ17CUKlz0Y2hI3k/sbTdFFLONbsZAknFSSTnrzGU4SCDXyLc6PNtZP3sb6z7Ywoj7Ckhv52Tv2jI+e2gpSS0S6Dy6LQDfzFpP8Yp9nP3IEFJyktixpIR5f11MUosEWg+MvEq0iEi4srThOXLkSO+/e/TowcCBA2nTpg2vv/46l112WaPtp06dyl133RXKEFljLmUvu+jHUOKNxJC+d1OIxYGBQR2+Hb/rqG30qzXc2SWX9eZy8o1OZBsHK/vJODlANVvMNRHX8IyEY/L59GX0vaQLJ444WFFufkIalcX7+fqF1XQe3ZaGmgYWPrGcsx46mbZDcg9uc2Iae9eVs/SlNbQemE1iRgKeeg+1lXU+Vc8D+2pIzEiwJC+7i4TP1s+NjsmPqOIZkCbp49lU0tLSOPHEE9mw4fCjeydPnkxFRYV3KSoqClospmmyxlzKHnbQl1NIMJKC9l7BFGVEkUIapZR415mmSSklpJFhYWT+s0suHtwY+E6bdPBx5H3TRMIxaahxY/zom86IMsBz8N+eBhNPg+ew25jmwWOS2bkZUTFRFC3a7X2+bIuLyuL95PQIjzztJhI+Wz83OibSFCzv4/lDVVVVbNy4kd/97neHfd7hcOBwOEISy1qWUkwRPRlENLHUfjdVRAyxRBvRIYmhqbTmRFZTSKrZDCfpbGM9bhrIId/q0Pxmh1yak8Nm81viSSSJVCopZ5u5jlzaWh1aQML9mOSfkkvh86tJzk4io30qe9aUs/Sfa+lyzsG/d1xyLC37tuDz6d8Q44gmJSeJnYtLWPOfLQyZ1AsAR0ocXc5tx4KHlhKfGkdcUizzpi0mu0cG2T3Ca0Bkg9nAAaq8jw9QTaVZTixxEXfVJtw/W8dKx8SedOeiwFja8LzhhhsYPXo0bdq0YefOndx5551ER0dz0UUXWRkWANvZBMBi5vms70I/ciPsBMs28qg3a9nEamqpIQUnvTkZhxF5l0bskEtHozcbzVWsMZd4J5BvSXvaGV2sDi0g4X5MTr2pL18+tYJ5U79mf1ktSS3i6XZ+ewZc2dW7zYipg1j4+HI+vu1Lalx1pOQkUjChO91+MIH8kOt7Yxjw3xs/x13npnVBDkMn97UipZ/kopQlzPc+Xs9yAHJoQ1f6WxVWQML9s3WsdExEvmeYh64lWWDMmDHMnz+fffv20aJFC04++WTuvfde2rdvf0yvd7lcOJ1OhnIOMUZskKOVn50IvYvQYVl3mjepzovD6iLNcfm2b+TdjEIklBrMej7lXSoqKkhNTbU6HK9DbY/2t95HdHxwGtzumho23ndr2OXeFCz9Fp81a5aVby8iIiIiIWSf8oGIiIhIqGhUe0DCalS7iIiIiNiXKp4iIiIiftKo9sCo4ikiIiIiIaGKp4iIiEggbFyZDBZVPEVEREQkJFTxFBEREfGXRrUHRBVPEREREQkJVTxFRERE/KRR7YFRxVNEREREQkIVTxERERF/qY9nQFTxFBEREZGQUMVTRERExE/q4xkYVTxFREREJCRU8RQRERHxl/p4BkQVTxEREREJCVU8RURERPylimdAVPEUERERkZBQxVNERETETxrVHhh7NDwN4+ASyUwbf8oilZ2OSaSfH9/5tm+D1SE0mWErK60OoUl80i3F6hBEJILYo+EpIiIiEkrq4xkQ9fEUERERkZBQxVNERETEX6p4BkQVTxEREREJCVU8RURERPykUe2BUcVTREREREJCFU8RERERf6mPZ0BU8RQRERGRkFDFU0RERMRP6uMZGFU8RURERCQkVPEUERER8Zf6eAZEFU8RERERCQlVPEVERET8pYpnQFTxFBEREZGQUMVTRERExE/Gd0uw9m1XqniKiIiISEio4ikiIiLiL/XxDIganiIiIiJ+0gTygVHD8wgWeP5DDfsbrW9FezpF9bEgoqaxxVzDBlaSRwc6Gr2sDicgReYGtrKOOmpIxklHeuM00q0Oy292yMM0TTaZq9jFVuqowUECOUY+bemMYURuL6VwPk9qqxv4+LH1rJpTQlVpHbmdUxl9SyfyujsBmP3kBpZ/UEx5cQ3RsQatuqQy/NoTaN0jzbuPuc9uZM38vexa4yI6NoopX55hUTZHZ4fz5IfC+bN1rOx2TOzmr3/9K5MnT+baa69l+vTpANTU1HD99dcza9YsamtrGTFiBE899RRZWVne123bto2rr76a//3vfyQnJzNu3DimTp1KTEzTNhXVx/MIBhjDGGKM9i69jVMAyDRaWRxZ4CrMUraziWScVocSsGKziHUspx1dGMAwUkhjKZ9RZ9ZYHZpf7JLHFtawnY10MvpQYPyCDkYPtpprKWKD1aEFLNzPkzfvWMX6hfu48K/due7tQZwwKIO/X/41FbsPfnZatEnkl7d15rq3B3H1ywNJa5nA81cspqq0zrsPd71J9+FZDPxNnlVpHBO7nCeHhPtn61jY7ZgcFzPISwAKCwt59tln6dGjh8/6P/3pT/z73//mX//6F/PmzWPnzp2cd9553ufdbjejRo2irq6OL774ghkzZvDiiy9yxx13BBbIT7C84bljxw5++9vfkpGRQUJCAt27d+frr7+2OiziDAcOI9677DV3kUASzWhhdWgBaTAbWMUiOtOXGGKtDidg21hHS9qSa+STbKTSiT5EE81Otlgdml/skkeFuY8W5NLcyCHBSCLLaEU6WbjMUqtDC0i4nyf1NW5Wzt7NWdd3pF2/dJq3SeLMCR1o3jqRL2cVAdDr7FxOKMggIy+RrA7JnH1TJ2qrGiheV+ndz5kTOzBkXD7ZJ6RYlcoxsct5AuH/2TpWdjomdlNVVcXYsWP529/+RrNmzbzrKyoqeP7553n44Yc5/fTT6du3Ly+88AJffPEFX375JQAff/wxq1ev5p///Ce9evVi5MiR3HPPPTz55JPU1dUd6S0DYmnDs6ysjMGDBxMbG8sHH3zA6tWreeihh3z+YOHAY3ooZiu5RtuIvXy4lqVkkE2GkXX0jcOUx/RQSTnpZHrXGYZBOlmUs8/CyPxjlzwAnEYGpZRQbR5s1FSa5VSwlwwj2+LIAhPu54nHbeJxm8Q4fL+6YxxRbFla1mj7hjoPi/5VRHxKDDkdw7uR+WN2Ok8g/D9bx8Jux6RJBLna6XK5fJba2tojhjJhwgRGjRrFsGHDfNYvXryY+vp6n/WdOnWidevWLFy4EICFCxfSvXt3n0vvI0aMwOVysWrVKv//Lj/B0j6e999/P3l5ebzwwgvedW3btrUwosPbww4aqCeXfKtDCUixWYSLMgYQvv24jkU9tZiYxBHvsz4OB9W4LIrKf3bJAyCfTjRQz0LzQwzTwMSkvdGNHKON1aH5LRLOE0dSDK17pTHnmY1ktksiOcPBN//dxbZvyslonejd7ttPS3j1huXU17hJaeHgsr/1I6lZnIWR+89O50kkfLaOhZ2OSaTIy/PtDnPnnXcyZcqURtvNmjWLJUuWUFhY2Oi54uJi4uLiSEtL81mflZVFcXGxd5sfNjoPPX/ouaZkacPzvffeY8SIEVxwwQXMmzePli1b8sc//pErrrjisNvX1tb6tPZdrtB80HeYm8kgG4eREJL3a0o15n7WsYzeDCHaiLY6HLGZ3RRRzDa6GQNJxkkl5awzl+EggVwj3+rwjlkknSe/mdqdN25fyX2nzSMq2iC3cwo9z8phx+rvvw/bD0jnmjcL2F9ez6I3tjPz+m+Y8OpAkjMcFkb+8xRJny3xTyhGtRcVFZGamupd73A0PoeLioq49tprmT17NvHx8Y2eDzeWNjw3bdrE008/zaRJk7j11lspLCzkmmuuIS4ujnHjxjXafurUqdx1110hjfGAWU0pu+lhDArp+zYVF2XUUcsi5njL9yYm5exlu7mR0zkvYroPxOLAwKAO307sddQ2+gUezuySB8B6czn5RieyjdYAJOPkANVsMddEVMMzks6TjNaJXDVjAHX7G6ipdpPawsHM678hvdX3P4zjEmNo3iYG2kDrnmk8MPIzCt/awWlXtLMwcv/Y5TyJpM/W0djlmESS1NRUn4bn4SxevJiSkhL69Pl+xh232838+fN54okn+Oijj6irq6O8vNyn6rl7926ysw92i8rOzmbRokU++929e7f3uaZkacPT4/HQr18/7rvvPgB69+7NypUreeaZZw7b8Jw8eTKTJk3yPna5XI3K0E1tp7mFOOJpTk5Q3ydY0snkJM70Wbear0kkhXw6RswXHkCUEUWKmUYpJWTSEjg4nU8pJeTR3uLojp1d8gDw4Mb40c3dDj6OrEnoIvE8iUuMIS4xhv0V9az7fC8jJ514xG1N06ShzhPC6I6fXc6TSPxsHYldjkmTCZMJ5M844wxWrFjhs278+PF06tSJm2++mby8PGJjY5kzZw7nn38+AGvXrmXbtm0UFBQAUFBQwL333ktJSQmZmQf78M6ePZvU1FS6dOnSNDl9x9KGZ05OTqOEOnfuzJtvvnnY7R0Ox2HLzMFimia72EIObYgyLJ8AICAxRmyjqTuizGhiiSPZiLwpPVpzIqspJNVshpN0trEeNw3kRFj/W7vk0ZwcNpvfEk8iSaRSSTnbzHXkEn59tX9KJJ0n6xbsxTRNWrRNYt+2/fz3wXW0aJtEv1+1pG5/A3Of20SX0zJJaeGguqyOha8W4dpdS48R31ctynceYH9FPeW7DuBxm+z89uBl+ozWiTiSwmd6ZzucJ5H02ToWdjgmdpOSkkK3bt181iUlJZGRkeFdf9lllzFp0iTS09NJTU3l//7v/ygoKOCkk04CYPjw4XTp0oXf/e53TJs2jeLiYv785z8zYcKEJm93WfoNM3jwYNauXeuzbt26dbRpEx4DE0rZTQ37yTUi6z9RO8s28qg3a9nEamqpIQUnvTkZhxFZl3nskkdHozcbzVWsMZd4J5BvSXvaGU37C1m+V1PVwIfT11FRXEOiM5ZuZ2Yx4toTiI6NwuMx2bO5mn++u4zqsjoS0+Jo1S2Vq14aQFaHZO8+Pn5iA0ve3el9/NivD45sveKF/rQfED4TgdvlPLETHZPvRdKdix555BGioqI4//zzfSaQPyQ6Opr333+fq6++moKCApKSkhg3bhx333130wYCGKZpWnZNrLCwkEGDBnHXXXdx4YUXsmjRIq644gqee+45xo4de9TXu1wunE4nQ41ziTEid140AKw7DPJzEEGX836Sjc6TYSsrj75RBPikW2RN0ySRo8Gs51PepaKi4qj9HEPpUNuj++X3ER0XnAa3u66GFX+/NexybwqWXj/u378/b7/9Nq+++irdunXjnnvuYfr06cfU6BQRERGxTBjeuSgSWN6Z5+yzz+bss8+2OgwRERERCTLLG54iIiIikSaS+niGk8gcqi0iIiIiEUcVTxERERF/hck8npFGFU8RERERCQlVPEVERET8pYpnQFTxFBEREZGQUMVTRERExE8a1R4YVTxFREREJCRU8RQRERHxl/p4BkQVTxEREREJCVU8RURERPxkmCaGGZzSZLD2Gw5U8RQRERGRkFDFU0RERMRf6uMZEFU8RURERCQkVPEUERER8ZPm8QyMKp4iIiIiEhKqeIqIiIj4S308A6KKp4iIiIiEhD0qnmYwf3aI3wzD6giahp3mUbNTLjbxSbcUq0NoEjdtXGF1CE1mWvvuVocgEUR9PAOjiqeIiIiIhIQ9Kp4iIiIioaQ+ngFRxVNEREREQkIVTxERERE/qY9nYFTxFBEREZGQUMVTRERExF/q4xkQVTxFREREJCRU8RQREREJgJ37YgaLKp4iIiIiEhKqeIqIiIj4yzSDd1c4G99tThVPEREREQkJVTxFRERE/KR5PAOjiqeIiIiIhIQqniIiIiL+0jyeAVHFU0RERERCQhVPERERET8ZnoNLsPZtV6p4ioiIiEhIqOIpIiIi4i/18QyIKp4iIiIiEhKqeB5FkbmBrayjjhqScdKR3jiNdKvD8kuZuYetrMNFGXXU0IMCMo2WVocVkBrzABvM5eyjGDcNJJBMV6M/qRF0TDaba9jDDqqpJIpo0sigA91JMlKsDs1vdvpsgT3O90PCKZfli/bzr+dKWb+yhtISN3c+k8vg4d9/3k3T5KXp+/hgVjlVLg9d+yZwzT1ZtGwb591m5pP7WPS/KjauriUm1uDtb0444vu5ytz8YdQW9hY38NayDiSnRgc1v6Oxy3my3dzIdjZxgGoAkkmlLZ1pbuRYHJk1NI9nYCyteObn52MYRqNlwoQJVoblVWwWsY7ltKMLAxhGCmks5TPqzBqrQ/OLmwaScdKJ3laHclzqzTq+NudiEEUvYwgFxi840ehJDHFHf3EYKWcPrWhPf06jD0Pw4GEpn+E2G6wOzW92+WyBfc53CL9cavZ7aNfZwcS7sg77/OvPlvLOi2Vc85csHnurNfGJUUy+ZDt1td+PsGioMxkyMoWzx6Yd9f0euqWYth0dTRX+cbPLeeIggQ50YyBnMIAzaEYm3/AFVWaF1aFJBLG04llYWIjb7fY+XrlyJWeeeSYXXHCBhVF9bxvraElbco18ADqZfdjLLnayhXw6WRucH5obOTTnu1+kEfwraou5hngS6RrV37sugSQLIwpMb2OIz+OuZn/m829clNGMFhZFFRi7fLbAPuc7hF8uA4YmM2Bo8mGfM02Tt18o4+KJGQw682AV9KYHs7lwwEY+/7iK00anAvD7PzUH4OM3frqR8+9/llHtcjP2/zIonFfdhFkEzi7nSQsj1+dxB7qx3dxIBaUk47QoKgvpXu0BsbTi2aJFC7Kzs73L+++/T/v27Tn11FOtDAsAj+mhknLSyfSuMwyDdLIoZ5+Fkf187WUnKTRjuWch8zzv8aVnNjvMTVaHddwaqAcgNsIqt3Zip/M90nIpLqqndI+bPoMTveuSUqPp1Cueb5ce8GtfW9fX8srj+7jpwRyiooymDlV+wDRNis0i3LhxkmF1OBJBwqaPZ11dHf/85z+ZNGkShmH9F0Y9tZiYxBHvsz4OB9W4LIrq5+0A1exgI605kXyjEy7KWGsuxSDKW9mJNKZpso5lOMkg2fgZVgzChJ3O90jLpXTPwateac19/ztq1jyGsj3uw73ksOpqPUy9dheXT25BZstYdhXVN2mcclCVWUEhc/HgIZoYelJAspFqdViWUB/PwIRNw/Odd96hvLycSy655Ijb1NbWUltb633scoXfl6gEj4lJKul0iOoOQCrNqDYr2GFujNiG5xqWUoWLfgy1OhSRiPaPB/aS1yGOYefqB1wwJZLCQM6kgXpK2M4qCulrDv3ZNj7Ff2EzndLzzz/PyJEjyc3NPeI2U6dOxel0epe8vLygxROLAwODOnw749dR26iSIKHhIIEkfL/ckoxUathvUUTHZ425lL3soi+nEm8kHv0FEjR2Ot8jLZf0FgdHnJfv9R1cV7a3gWYtjn00+rKF+/nsv5X84oS1/OKEtdz82yIAft13Ay89srfpAv6ZizKiSDSSSTWa0cHoTgppFLHe6rCsYQZ5samwqHhu3bqVTz75hLfeeusnt5s8eTKTJk3yPna5XEFrfEYZUaSYaZRSQiYHp70wTZNSSsijfVDeU36akwz2U+mzrtqsJD7CBhiZpslalrGHHfTlVBKMyIrfjux0vkdaLtl5saS3iGbpF/tp3+Vgw7i60s2aZTXHNIL9kDueyqW25vv/rdctr+Ghm4t5+LXW5LSObeqw5TsmJh5sfH9HaXJh0fB84YUXyMzMZNSoUT+5ncPhwOEI3RQZrTmR1RSSajbDSTrbWI+bBnLID1kMTaHBbOAAVd7HB6im0iwnlriIqrS1Nk7ka3Mum81vySIPF6XsYBOdjb5Wh+aXtSylmCJ6MohoYqn9boqbGGKJNqydb9BfdvlsgX3Odwi/XA5Ue9i5tc77uLiono2ra0hxRpPZMpZfjW/GzCf20TI/juxWsbz4yF4ysmIYPPz7kfAlO+qprHBTsrMej8dk4+qD501umzgSkqLIbeM7OM9VdrB/aOsOcZbP42mX82SDuYIMsoknETcNFLONMvbQmyFHf7ENqY9nYCxveHo8Hl544QXGjRtHTIzl4fjINvKoN2vZxGpqqSEFJ705GYcRfperfoqLUpYw3/t4PcsByKENXel/pJeFHaeRTg8GscFcwWZWE08SHY1e5BhtrA7NL9s5OBJ/MfN81nehH7kR1sixy2cL7HO+Q/jlsm5FDTdeXOR9/Oy9ewA48/xUbnwghwuvSqfmgMn0W4upcnno1i+B+15oRZzj+95gM6bvZfab3/frv/rsrQA8MDOPnieFd+PNLudJHbWsopBaaogh9rvP1RAyjMPPzypyOIZpWjtZ1Mcff8yIESNYu3YtJ554ol+vdblcOJ1OhnIOMYYupYSNMJiVoEnYeB41kaZy08YVVofQZKa17251CPIDDWY9n/IuFRUVpKaGz+ClQ22Pk866m5jY4PyYa6iv4cv/3hF2uTcFy0uMw4cPx+K2r4iIiIiEgOUNTxEREZFIoz6egQmb6ZRERERExN5U8RQRERHxVzDn27RxxVMNTxERERE/6VJ7YHSpXURERERCQhVPEREREX95zINLsPZtU6p4ioiIiEhIqOIpIiIi4i8NLgqIKp4iIiIiEhKqeIqIiIj4ySCIo9qDs9uwoIqniIiIiISEKp4iIiIi/jLNg0uw9m1TqniKiIiISEio4ikiIiLiJ925KDCqeIqIiIhISKjiKSIiIuIvzeMZEFU8RURERCQkVPEUERER8ZNhmhhBGn0erP2GA1U8RURERCQkVPGUpmfjX2oi4mta++5Wh9BkLvy22OoQmsTrnbOtDuHnwfPdEqx925QqniIiIiISEqp4ioiIiPhJfTwDo4qniIiIiISEKp4iIiIi/tI8ngFRxVNEREREQkIVTxERERF/mWbwZnFRH08RERERCUdTp06lf//+pKSkkJmZybnnnsvatWt9tqmpqWHChAlkZGSQnJzM+eefz+7du3222bZtG6NGjSIxMZHMzExuvPFGGhoamjRWNTxFRERE/GSYwV38MW/ePCZMmMCXX37J7Nmzqa+vZ/jw4VRXV3u3+dOf/sS///1v/vWvfzFv3jx27tzJeeed533e7XYzatQo6urq+OKLL5gxYwYvvvgid9xxR1P9yQBdahcRERGJaB9++KHP4xdffJHMzEwWL17MKaecQkVFBc8//zwzZ87k9NNPB+CFF16gc+fOfPnll5x00kl8/PHHrF69mk8++YSsrCx69erFPffcw80338yUKVOIi4trklhV8RQRERHx16E+nsFaAJfL5bPU1tYeU2gVFRUApKenA7B48WLq6+sZNmyYd5tOnTrRunVrFi5cCMDChQvp3r07WVlZ3m1GjBiBy+Vi1apVTfInAzU8RURERMJSXl4eTqfTu0ydOvWor/F4PFx33XUMHjyYbt26AVBcXExcXBxpaWk+22ZlZVFcXOzd5oeNzkPPH3quqehSu4iIiIifDM/BJVj7BigqKiI1NdW73uFwHPW1EyZMYOXKlSxYsCA4wR0nVTxFREREwlBqaqrPcrSG58SJE3n//ff53//+R6tWrbzrs7Ozqauro7y83Gf73bt3k52d7d3mx6PcDz0+tE1TUMNTRERExF8h6ON57KGYTJw4kbfffpu5c+fStm1bn+f79u1LbGwsc+bM8a5bu3Yt27Zto6CgAICCggJWrFhBSUmJd5vZs2eTmppKly5djuMP5UuX2kVEREQi2IQJE5g5cybvvvsuKSkp3j6ZTqeThIQEnE4nl112GZMmTSI9PZ3U1FT+7//+j4KCAk466SQAhg8fTpcuXfjd737HtGnTKC4u5s9//jMTJkw4pkv8x0oNTxERERF/hdG92p9++mkAhg4d6rP+hRde4JJLLgHgkUceISoqivPPP5/a2lpGjBjBU0895d02Ojqa999/n6uvvpqCggKSkpIYN24cd9999/Fk0oganiIiIiIRzDyGS/Px8fE8+eSTPPnkk0fcpk2bNvz3v/9tytAaUcNTRERExE+GaWIE6Z7qwdpvOFDD8wi2mxvZziYOcPB2U8mk0pbONDdyLI4sMEXmBrayjjpqSMZJR3rjNNKtDssvdjkmm8017GEH1VQSRTRpZNCB7iQZKVaH5rcycw9bWYeLMuqooQcFZBotrQ4rYHY4Tw6J9Fwi5TypqWrg3ce2sfSTfVTuqyevcxJjbmtHfvfv49y1cT9vPriFdYUVeNwmOe0T+cNjncjIjQdg/mvFLHq/hG2rq6mpdjN90Ukkpobff8+RckwkvGlU+xE4SKAD3RjIGQzgDJqRyTd8QZVZYXVofis2i1jHctrRhQEMI4U0lvIZdWaN1aH5xS7HpJw9tKI9/TmNPgzBg4elfIbbbLA6NL+5aSAZJ53obXUox80u5wnYI5dIOU9eun0Dq78o59L7T+TO93rTZXAaD49fSdnug3eYKdl2gGkXLye7XQI3vNSdO97tzag/5hHr+P6/37oaN12HNGPkVa2O9DZhIVKOSciE0aj2SGJpw9PtdnP77bfTtm1bEhISaN++Pffcc88x9VUIthZGLs2NHBKNFJKMFDoY3YgmhgpKrQ7Nb9tYR0vakmvkk2yk0ok+RBPNTrZYHZpf7HJMehtDvjsWTlKMNLrSnxr246LM6tD81tzIoYPRLaKrnIfY5TwBe+QSCedJXY2bJR/v5fwb8jmxv5PMNgn88v/akNk6nnmvHhxV/M70rXQ7tRm/vrEtrbskk9k6gV6nZ5Ca8f19r4eNa8nIK/No1zO8K4eRcEwk/Flay7///vt5+umnmTFjBl27duXrr79m/PjxOJ1OrrnmGitD82GaJrvZjhs3TjKsDscvHtNDJeXk08m7zjAM0s0sytlnYWTHJ5KPyY81UA9ALHFH2VKCxU7niZ1y+aFwPE88DSYeNz7VS4DY+Gg2LK7A4zFZ8WkZIy5vyfTLVlL0bTUZrRyMvDKP3sMi+3sLwvOYhJQJBOnORUEbLR8GLG14fvHFF5xzzjmMGjUKgPz8fF599VUWLVpkZVheVWYFhczFg4doYuhJAclG6tFfGEbqqcXEJI54n/VxOKjGZVFUgbPDMfkh0zRZxzKcZJBsOK0O52fLTueJnXI5JFzPk/jkGNr1SuE/T20jp10Cqc3jWPSfPWxa5iKzdQKV++qp3e/mw79t55xr23D+Dfms/KyMZ/7vWybN6E7HAeGTi7/C9ZhI+LP0UvugQYOYM2cO69atA+Cbb75hwYIFjBw58rDb19bW4nK5fJZgSiSFgZxJf06nFe1YRSFVZmR+cduF3Y7JGpZShYvuDLQ6FJGwFc7nyaXTTsQ04aZTC/ljj8+Z+/JOBoxqgREFpudg2arX6RmceUlL8jonM/LKPLoPTWf+rF0WR358wvmYhMqhUe3BWuzK0ornLbfcgsvlolOnTkRHR+N2u7n33nsZO3bsYbefOnUqd911V8jiizKiSCQZgFSa4TLLKGI9nekbshiOVywODAzq8B1UUEdto4pIJLDDMTlkjbmUveyiH0OJNxKtDudnzU7niZ1ygfA/TzJbJ3DjP3tQu9/NgSo3aZlxPPenNTTPiye5WSxRMQY5HRJ8XpPTPoENiyP4B3OYHxMJb5ZWPF9//XVeeeUVZs6cyZIlS5gxYwYPPvggM2bMOOz2kydPpqKiwrsUFRWFNF4TE0/QOnQER5QRRQpplPL9vVdN06SUEtIivG8kROYxMU2TNeZS9rCDvpxCgpFkdUg/e3Y6T+ySS6SdJ47EaNIy46iuaGDVgjJ6nZ5BTFwU+d2SKd58wGfb3VsOeKdSiiSRdkyCziSIo9qtTi54LK143njjjdxyyy2MGTMGgO7du7N161amTp3KuHHjGm3vcDia9H6hP2WDuYIMsoknETcNFLONMvbQmyEhef+m1JoTWU0hqWYznKSzjfW4aSCHfKtD84tdjslallJMET0ZRDSx1H43xU0MsUQb0RZH558Gs4EDVHkfH6CaSrOcWOIirhJil/ME7JFLpJwnqz4rwwSy2yZQsvUAbzywhex2iQw6LxOAEZe15LlJazmxn5OOA52s/KyM5f8r5fqXunv3UbGnDtfeOkq2Hcxxx7pq4pOiSc9xkJQWa0VahxUpx0TCm6UNz/379xMV5Vt0jY6OxuOxvoJVRy2rKKSWGmKIJQUnvRlChpFldWh+yzbyqDdr2cRqaqn5LpeTcRiR9YvbLsdkO5sAWMw8n/Vd6EduBDUMAFyUsoT53sfrWQ5ADm3oSn+rwgqIXc4TsEcukXKeHKhq4K2Ht1JeXEtiWgx9zmzOuX9qQ0zswf/bep/ZnLFTGvjwue3MuncTWW0T+MNjnTmh7/cDcubN2sX7T35/Be+B364A4JL7TmDQeeHz/RYpxyRkgjnfpo37eBqmhZNmXnLJJXzyySc8++yzdO3alaVLl3LllVdy6aWXcv/99x/19S6XC6fTyVDOIcYIn1+FIiISeS78ttjqEJrE652zrQ6hSTSY9XzKu1RUVJCaGj6zlxxqe5ze82ZiooNzFbbBXcvcb+4Pu9ybgqUVz8cff5zbb7+dP/7xj5SUlJCbm8tVV13FHXfcYWVYIiIiIj/NAxhB3LdNWdrwTElJYfr06UyfPt3KMEREREQkBCxteIqIiIhEomDOt2nneTwtnU5JRERERH4+VPEUERER8ZdGtQdEFU8RERERCQlVPEVERET8pYpnQFTxFBEREZGQUMVTRERExF+qeAZEFU8RERERCQlVPEVERET8pTsXBUQVTxEREREJCVU8RURERPykOxcFRhVPEREREQkJVTxFRERE/KVR7QFRxVNEREREQkIVTxERERF/eUwwglSZ9KjiKSIiIiJyXFTxFBEREfGX+ngGRA1PEREREb8FseGJGp4iEsmMYN1eI8RsXAWIWHb5bAGvd8mxOoQmcdm6TVaH0CT2V7r5tI/VUUhTU8NTRERExF+61B4QDS4SERERkZBQxVNERETEXx6ToPXF1HRKIiIiIiLHRxVPEREREX+ZnoNLsPZtU6p4ioiIiEhIqOIpIiIi4i+Nag+IKp4iIiIiEhKqeIqIiIj4S6PaA6KKp4iIiIiEhCqeIiIiIv5SH8+AqOIpIiIiIiGhiqeIiIiIv0yCWPEMzm7DgSqeIiIiIhISqniKiIiI+Et9PAOiiqeIiIiIhIQqniIiIiL+8niAIN1T3aN7tYuIiIiIHBdVPEVERET8pT6eAVHD8wg2m2vYww6qqSSKaNLIoAPdSTJSrA7NL9vNjWxnEweoBiCZVNrSmeZGjsWR+a/M3MNW1uGijDpq6EEBmUZLq8MKiJ1yqTEPsMFczj6KcdNAAsl0NfqTaqRbHVrAtphr2MBK8uhAR6OX1eEEpMjcwFbWUUcNyTjpSG+cEXZM7PLZ2uhZxWZW+6xLJIVBUb+wKKLDO1Dl5vXpO/l6djkV++rJ75LIuD/n0b5HEgA11W5efXAHX88up7K8gcxWDkb8PpMzL27h3cecWXv4/N+lbFm1nwPVHv6+uCdJqWpqyPcs/TRUVlZy++238/bbb1NSUkLv3r159NFH6d+/v5VhAVDOHlrRnlSaYWKygZUs5TMKzOFEG5FzEjlIoAPdSCQZE9jFVr7hCwaaw0g2nFaH5xc3DSTjJJd8lrPQ6nCOi11yqTfr+NqcSzMy6WUMIQ4H+6kkhjirQwtYhVnKdjaRTGSdHz9UbBaxjuV0pg+ppFPEepbyGYPMEcQZ8VaHd0zs9tlKIpU+xqnexwaGhdEc3nO3baVo3QH++EA+zbJiWfBuKfeOW8eDH3QlPTuOl6duZ9XCSiY81JYWLeNYvsDFP6Zso1lWLP3OSAOg9oCHnqc46XmKk1kP7rA2oWBTxTMglvbxvPzyy5k9ezYvv/wyK1asYPjw4QwbNowdO6z/sPY2hpBr5JNsOEkx0uhKf2rYj4syq0PzSwsjl+ZGDolGCklGCh2MbkQTQwWlVofmt+ZGDh2MbhFbGfwhu+SyxVxDPIl0jeqP00gnwUgiw8gm0Ui2OrSANJgNrGIRnelLDLFWhxOwbayjJW2/+w5LpRN9iCaanWyxOrRjZrfPloGBw4j3LnGGw+qQfNTVeFj0URkX39SKzgNSyG4Tz6+vySW7TTyzZ+4BYN2SKk75VQZdBqbQopWDM8a0oE2nRDZ+U+3dz1njszjnqmxO6JVkVSoS5ixreB44cIA333yTadOmccopp9ChQwemTJlChw4dePrpp60K64gaqAcgNkJ/bQOYpkmxWYQbN04yrA5HbGAvO0mhGcs9C5nneY8vPbPZYW6yOqyArWUpGWSTYWRZHUrAPKaHSspJJ9O7zjAM0sminH0WRuYfu3229lPFfM+/+dzzX1Z6vqLG3G91SD7cDSYeN8Q5fCuxcfEGaxdXAXBin2QWzy2ntLgO0zRZ9WUlu7bU0OPkVCtCtp7HDO5iU5ZdM25oaMDtdhMf73vZJyEhgQULFlgU1eGZpsk6luEkI+IuTwNUmRUUMhcPHqKJoScFJBs/0y8KaVIHqGYHG2nNieQbnXBRxlpzKQZR5Br5Vofnl2KzCBdlDOAMq0M5LvXUYmISh+93axwOqnFZFJX/7PTZchrpdKU/iaRQRw2bzNV8bf6PkxhOjBEelfWE5GhO6J3EW0/uIrd9PGnNY/n8/VLWLa0mu83B6uwlt+fxt9u3MmHICqJjDv6gueLeNnQeEFljH8RaljU8U1JSKCgo4J577qFz585kZWXx6quvsnDhQjp06HDY19TW1lJbW+t97HKF5kt0DUupwkU/hobk/ZpaIikM5EwaqKeE7ayikL7mUDU+5biZmKSSToeo7gCk0oxqs4Id5saIahzUmPtZxzJ6M4RoI9rqcAT7fLaARoM5U0lngfkfdrOdlrS1KKrGJjzQlmcmb2HCySuIioa2XRMZdHY6m1cerM5+9HIJG5ZVc8Mz7WneMo41hVW8cNc2mmXG0n3wz+//E9P0YJrBmW8zWPsNB5aOknn55Ze59NJLadmyJdHR0fTp04eLLrqIxYsXH3b7qVOnctddd4U0xjXmUvayi34MJd5IDOl7N5UoI4pEDvaLSqUZLrOMItbTmb4WRyaRzkECSfj+h5NkpFJibrcoosAcnF2glkXMge+ucJmYlLOX7eZGTuc8DCP8BoMcTiwODAzqqPFZX0dtoypoOLPLZ+twYo04kswUDphVhNMYo6w2Du6c2ZGa/W4OVHlolhnLo9duIjMvjroaD7Me3smkJ9vT57SDV/7adEpk67f7ef/53T/LhqcExtLBRe3bt2fevHlUVVVRVFTEokWLqK+vp127dofdfvLkyVRUVHiXoqKioMVmmiZrzKXsYQd9OYUEwz4dpU1MPMG624L8rDjJYD+VPuuqzUriiazzJZ1MTuJMBjLMu6TSjGxaM5BhEdPohIM/NFNIo5QS7zrTNCmlhLQI6tttl8/W4TSYDeynKmxnGIhPjKZZZixVFQ0s/8xFv2FpNNSbuOtNon7UaoiKMjBt3B/xJ5lB7N9p41HtYTEvUFJSEklJSZSVlfHRRx8xbdq0w27ncDhwOEIzEnAtSymmiJ4MIppYas2D1YMYYiPqUtwGcwUZZBNPIm4aKGYbZeyhN0OsDs1vDWYDB6jyPj5ANZVmObHERVw12i65tDZO5GtzLpvNb8kiDxel7GATnY3IqqbHGLGNpk+KMqOJJS4i+3W35kRWU0iq2Qwn6WxjPW4ayCHf6tCOmV0+WwDrPN/QwsglnkRqOcAmcxUGBtm0tjo0H998VoFpQm7beIq31jLz/u3ktovn1PObExNr0HlAMq/cv524+Cia58bx7aJK5r+zj99NzvPuo3xPPeV76ineerBbXNHaA8QnRdM8N47ktLBocojFLP0UfPTRR5imSceOHdmwYQM33ngjnTp1Yvz48VaGBcB2Do6eXMw8n/Vd6EduBH1511HLKgqppYYYYknBSW+GROSoXRelLGG+9/F6lgOQQxu6Yv3cr/6wSy5OI50eDGKDuYLNrCaeJDoavcgx2lgd2s9atpFHvVnLJlZTS8135/3JOMK0wnY4dvps1XKAFeaX1FNHHA7SaE5/44ywm1Jpf6WbWQ/uoLS4nuS0aAaMaMZvJrUkJvZgxf+a6e2Y9eAOnrh+M1XlDbRoGcdvJrVk2MXNvfv45NU9vPn4Lu/juy5eB8Af/tqGU89vjq2YJt6+OUHZtz0Zpmlddq+//jqTJ09m+/btpKenc/7553PvvffidB5bhcHlcuF0OhnKOWEzMlAkLEXQpeKfZOMv44hll8+WjVy2NnKnnfqh/ZVuLuuzjIqKClJTw6cP6aG2xxnO3xFjBGeKxQazjjkVL4dd7k3B0ornhRdeyIUXXmhlCCIiIiL+83jACNJ4CRuPard0cJGIiIiI/Hyop6+IiIiIv9THMyCqeIqIiIhISKjiKSIiIuIn0+PBDFIfTzvfuUgVTxEREREJCVU8RURERPylPp4BUcVTREREREJCFU8RERERf3lMMFTx9JcqniIiIiISEqp4ioiIiPjLNIFg3blIFU8RERERkeOiiqeIiIiIn0yPiRmkPp6mKp4iIiIiIsdHFU8RERERf5kegtfHU3cuEhEREZEw9uSTT5Kfn098fDwDBw5k0aJFVofUiBqeIiIiIn4yPWZQF3+99tprTJo0iTvvvJMlS5bQs2dPRowYQUlJSRCyD5waniIiIiIR7uGHH+aKK65g/PjxdOnShWeeeYbExET+8Y9/WB2aDzU8RURERPxleoK7+KGuro7FixczbNgw77qoqCiGDRvGwoULmzrz4xLRg4sOTTfQQD3Yd+YBkSZgWB1A07DxFCORyyafLRvZX+m2OoQmcaDqYB7hOrVQMNseDdQD4HK5fNY7HA4cDkej7ffu3Yvb7SYrK8tnfVZWFmvWrAlOkAGK6IZnZWUlAAv4r8WRiIS58PzeFjvQZyvsfNrH6giaVmVlJU6n0+owvOLi4sjOzmZBcXDbHsnJyeTl5fmsu/POO5kyZUpQ3zfYIrrhmZubS1FRESkpKRhGcH51u1wu8vLyKCoqIjU1NSjvESp2ycUueYB9crFLHmCfXJRH+LFLLqHKwzRNKisryc3NDdp7BCI+Pp7NmzdTV1cX1PcxTbNR2+Zw1U6A5s2bEx0dze7du33W7969m+zs7KDFGIiIbnhGRUXRqlWrkLxXampqRH9R/JBdcrFLHmCfXOySB9gnF+URfuySSyjyCKdK5w/Fx8cTHx9vdRhecXFx9O3blzlz5nDuuecC4PF4mDNnDhMnTrQ2uB+J6IaniIiIiMCkSZMYN24c/fr1Y8CAAUyfPp3q6mrGjx9vdWg+1PAUERERiXC/+c1v2LNnD3fccQfFxcX06tWLDz/8sNGAI6up4XkUDoeDO++884j9KiKJXXKxSx5gn1zskgfYJxflEX7skotd8rCjiRMnht2l9R8zzHCdp0BEREREbEUTyIuIiIhISKjhKSIiIiIhoYaniIiIiISEGp5H8eSTT5Kfn098fDwDBw5k0aJFVofkt/nz5zN69Ghyc3MxDIN33nnH6pACMnXqVPr3709KSgqZmZmce+65rF271uqw/Pb000/To0cP7xx4BQUFfPDBB1aHddz++te/YhgG1113ndWh+G3KlCkYhuGzdOrUyeqwArZjxw5++9vfkpGRQUJCAt27d+frr7+2Oiy/5OfnNzomhmEwYcIEq0Pzi9vt5vbbb6dt27YkJCTQvn177rnnnrC9DeTRVFZWct1119GmTRsSEhIYNGgQhYWFVoclEUQNz5/w2muvMWnSJO68806WLFlCz549GTFiBCUlJVaH5pfq6mp69uzJk08+aXUox2XevHlMmDCBL7/8ktmzZ1NfX8/w4cOprq62OjS/tGrVir/+9a8sXryYr7/+mtNPP51zzjmHVatWWR1awAoLC3n22Wfp0aOH1aEErGvXruzatcu7LFiwwOqQAlJWVsbgwYOJjY3lgw8+YPXq1Tz00EM0a9bM6tD8UlhY6HM8Zs+eDcAFF1xgcWT+uf/++3n66ad54okn+Pbbb7n//vuZNm0ajz/+uNWhBeTyyy9n9uzZvPzyy6xYsYLhw4czbNgwduzYYXVoEilMOaIBAwaYEyZM8D52u91mbm6uOXXqVAujOj6A+fbbb1sdRpMoKSkxAXPevHlWh3LcmjVrZv7973+3OoyAVFZWmieccII5e/Zs89RTTzWvvfZaq0Py25133mn27NnT6jCaxM0332yefPLJVofR5K699lqzffv2psfjsToUv4waNcq89NJLfdadd9555tixYy2KKHD79+83o6Ojzffff99nfZ8+fczbbrvNoqgk0qjieQR1dXUsXryYYcOGeddFRUUxbNgwFi5caGFkckhFRQUA6enpFkcSOLfbzaxZs6iurqagoMDqcAIyYcIERo0a5XOuRKL169eTm5tLu3btGDt2LNu2bbM6pIC899579OvXjwsuuIDMzEx69+7N3/72N6vDOi51dXX885//5NJLL2107+pwN2jQIObMmcO6desA+Oabb1iwYAEjR460ODL/NTQ04Ha7G90qMiEhIWKvEEjoaQL5I9i7dy9ut7vRjP9ZWVmsWbPGoqjkEI/Hw3XXXcfgwYPp1q2b1eH4bcWKFRQUFFBTU0NycjJvv/02Xbp0sTosv82aNYslS5ZEfB+vgQMH8uKLL9KxY0d27drFXXfdxZAhQ1i5ciUpKSlWh+eXTZs28fTTTzNp0iRuvfVWCgsLueaaa4iLi2PcuHFWhxeQd955h/Lyci655BKrQ/HbLbfcgsvlolOnTkRHR+N2u7n33nsZO3as1aH5LSUlhYKCAu655x46d+5MVlYWr776KgsXLqRDhw5WhycRQg1PiUgTJkxg5cqVEfsru2PHjixbtoyKigreeOMNxo0bx7x58yKq8VlUVMS1117L7NmzG1VAIs0Pq089evRg4MCBtGnThtdff53LLrvMwsj85/F46NevH/fddx8AvXv3ZuXKlTzzzDMR2/B8/vnnGTlyJLm5uVaH4rfXX3+dV155hZkzZ9K1a1eWLVvGddddR25ubkQej5dffplLL72Uli1bEh0dTZ8+fbjoootYvHix1aFJhFDD8wiaN29OdHQ0u3fv9lm/e/dusrOzLYpK4OAtwd5//33mz59Pq1atrA4nIHFxcd4KQd++fSksLOTRRx/l2WeftTiyY7d48WJKSkro06ePd53b7Wb+/Pk88cQT1NbWEh0dbWGEgUtLS+PEE09kw4YNVofit5ycnEY/YDp37sybb75pUUTHZ+vWrXzyySe89dZbVocSkBtvvJFbbrmFMWPGANC9e3e2bt3K1KlTI7Lh2b59e+bNm0d1dTUul4ucnBx+85vf0K5dO6tDkwihPp5HEBcXR9++fZkzZ453ncfjYc6cORHbFy/SmabJxIkTefvtt5k7dy5t27a1OqQm4/F4qK2ttToMv5xxxhmsWLGCZcuWeZd+/foxduxYli1bFrGNToCqqio2btxITk6O1aH4bfDgwY2mGVu3bh1t2rSxKKLj88ILL5CZmcmoUaOsDiUg+/fvJyrK97/a6OhoPB6PRRE1jaSkJHJycigrK+Ojjz7inHPOsTokiRCqeP6ESZMmMW7cOPr168eAAQOYPn061dXVjB8/3urQ/FJVVeVTudm8eTPLli0jPT2d1q1bWxiZfyZMmMDMmTN59913SUlJobi4GACn00lCQoLF0R27yZMnM3LkSFq3bk1lZSUzZ87k008/5aOPPrI6NL+kpKQ06l+blJRERkZGxPW7veGGGxg9ejRt2rRh586d3HnnnURHR3PRRRdZHZrf/vSnPzFo0CDuu+//27u3kKi+PQ7g3022p0knJkvzgjM5SWolkkXhSyZZ+SLWEArdRqygtDIrS4mgErUejMoHL1ij0QUlS0yFySRNAwsqI8KmRpIKfBDCwsLrrPPwpzlnTpfTmO7J0/cD8zB7r73Wb++H2V/W2jOTj6SkJDx69AhlZWUoKytzd2kus9vtMJvNMJlM8PCYmrerhIQE5OXlQafTYfHixXj69CnOnj2L1NRUd5c2LhaLBUIIhIaGwmazISsrC2FhYVPuvkhu5O6v1f/pioqKhE6nE7IsixUrVoiOjg53l+Sye/fuCQDfvEwmk7tLc8n3zgGAMJvN7i7NJampqUKv1wtZloWPj49Ys2aNuHPnjrvLmhBT9eeUkpOThb+/v5BlWQQGBork5GRhs9ncXda43b59WyxZskSoVCoRFhYmysrK3F3SuFgsFgFAWK1Wd5cybp8+fRIZGRlCp9OJGTNmCIPBII4dOyaGhobcXdq4VFVVCYPBIGRZFn5+fiI9PV309/e7uyyaQiQhpujfJxARERHRlMJnPImIiIhIEQyeRERERKQIBk8iIiIiUgSDJxEREREpgsGTiIiIiBTB4ElEREREimDwJCIiIiJFMHgSERERkSIYPInoj5SSkoINGzY43q9evRoHDhxQvI6WlhZIkoT+/n7FxyYi+n/D4ElELklJSYEkSZAkCbIsIyQkBKdOncLo6Oikjnvz5k3k5ub+UluGRSKiP5OHuwsgoqknPj4eZrMZQ0NDaGxsRHp6OqZPn46cnByndsPDw5BleULG9Pb2npB+iIjIfTjjSUQuU6lU8PPzg16vx549exAXF4e6ujrH8nheXh4CAgIQGhoKAHj37h2SkpKg1Wrh7e2NxMRE9PT0OPobGxvDwYMHodVqMWfOHBw5cgRCCKcx/3upfWhoCEePHkVQUBBUKhVCQkJw8eJF9PT0IDY2FgAwe/ZsSJKElJQUAIDdbkdBQQGCg4OhVqsRGRmJGzduOI3T2NiIhQsXQq1WIzY21qlOIiL6PQyeRPTb1Go1hoeHAQDNzc2wWq1oampCfX09RkZGsH79emg0GrS1teHBgwfw8vJCfHy845jCwkJUVFTg0qVLaG9vx4cPH3Dr1q2fjrl9+3Zcv34dFy5cQFdXF0pLS+Hl5YWgoCDU1NQAAKxWK3p7e3H+/HkAQEFBAS5fvoySkhK8ePECmZmZ2Lp1K1pbWwH8E5CNRiMSEhLQ2dmJnTt3Ijs7e7IuGxHRX4dL7UQ0bkIINDc3w2KxYN++fejr64OnpyfKy8sdS+xXrlyB3W5HeXk5JEkCAJjNZmi1WrS0tGDdunU4d+4ccnJyYDQaAQAlJSWwWCw/HPfVq1eorq5GU1MT4uLiAAAGg8Gx/+uyvK+vL7RaLYB/Zkjz8/Nx9+5dREdHO45pb29HaWkpYmJiUFxcjAULFqCwsBAAEBoaiufPn+PMmTMTeNWIiP5eDJ5E5LL6+np4eXlhZGQEdrsdmzdvxokTJ5Ceno6IiAin5zqfPXsGm80GjUbj1Mfg4CC6u7vx8eNH9Pb2YuXKlY59Hh4eWL58+TfL7V91dnZi2rRpiImJ+eWabTYbvnz5grVr1zptHx4extKlSwEAXV1dTnUAcIRUIiL6fQyeROSy2NhYFBcXQ5ZlBAQEwMPj3x8lnp6eTm0HBgawbNkyXL169Zt+fHx8xjW+Wq12+ZiBgQEAQENDAwIDA532qVSqcdVBRESuYfAkIpd5enoiJCTkl9pGRUWhqqoKvr6+mDVr1nfb+Pv74+HDh1i1ahUAYHR0FI8fP0ZUVNR320dERMBut6O1tdWx1P6fvs64jo2NObYtWrQIKpUKb9++/eFMaXh4OOrq6py2dXR0/O+TJCKiX8IvFxHRpNqyZQvmzp2LxMREtLW14c2bN2hpacH+/fvx/v17AEBGRgZOnz6N2tpavHz5EmlpaT/9Dc758+fDZDIhNTUVtbW1jj6rq6sBAHq9HpIkob6+Hn19fRgYGIBGo8Hhw4eRmZmJyspKdHd348mTJygqKkJlZSUAYPfu3Xj9+jWysrJgtVpx7do1VFRUTPYlIiL6azB4EtGkmjlzJu7fvw+dTgej0Yjw8HDs2LEDg4ODjhnQQ4cOYdu2bTCZTIiOjoZGo8HGjRt/2m9xcTE2bdqEtLQ0hIWFYdeuXfj8+TMAIDAwECdPnkR2djbmzZuHvXv3AgByc3Nx/PhxFBQUIDw8HPHx8WhoaEBwcDAAQKfToaamBrW1tYiMjERJSQny8/Mn8eoQEf1dJPGjp/eJiIiIiCYQZzyJiIiISBEMnkRERESkCAZPIiIiIlIEgycRERERKYLBk4iIiIgUweBJRERERIpg8CQiIiIiRTB4EhEREZEiGDyJiIiISBEMnkRERESkCAZPIiIiIlIEgycRERERKeJfbTVrmw4EVtsAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 800x600 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.metrics import confusion_matrix\n",
    "\n",
    "\n",
    "def plot_confusion_matrix(model, loader, device):\n",
    "    model.eval()\n",
    "    all_preds = []\n",
    "    all_targets = []\n",
    "\n",
    "    with torch.no_grad():\n",
    "        for images, labels in loader:\n",
    "            images = images.to(device)\n",
    "            labels = labels.to(device)\n",
    "\n",
    "            outputs = model(images)\n",
    "            _, preds = torch.max(outputs, 1)\n",
    "\n",
    "            all_preds.extend(preds.cpu().numpy())\n",
    "            all_targets.extend(labels.cpu().numpy())\n",
    "\n",
    "    cm = confusion_matrix(all_targets, all_preds)\n",
    "\n",
    "    plt.figure(figsize=(8, 6))\n",
    "    plt.imshow(cm)\n",
    "    plt.colorbar()\n",
    "\n",
    "    # Add numbers inside cells\n",
    "    num_classes = cm.shape[0]\n",
    "    for i in range(num_classes):\n",
    "        for j in range(num_classes):\n",
    "            plt.text(j, i, cm[i, j],\n",
    "                     ha=\"center\",\n",
    "                     va=\"center\")\n",
    "\n",
    "    plt.xlabel(\"Predicted\")\n",
    "    plt.ylabel(\"Actual\")\n",
    "    plt.xticks(np.arange(num_classes))\n",
    "    plt.yticks(np.arange(num_classes))\n",
    "    plt.title(\"Confusion Matrix\")\n",
    "    plt.tight_layout()\n",
    "    plt.show()\n",
    "\n",
    "\n",
    "plot_confusion_matrix(model, test_loader, device)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Yi4axcnTIl6p"
   },
   "source": [
    "### Inspecting Misclassifications\n",
    "\n",
    "This tool extracts the specific images where the model was wrong and displays them with their incorrect labels.\n",
    "\n",
    "* **How to use it for debugging:** Check if the errors are \"reasonable.\" If a human can't tell if the digit is a 5 or a 6 because the handwriting is poor, the model has likely reached its maximum potential.\n",
    "* If the images look perfectly clear to you but the model fails, the issue is architectural—your network might be too \"shallow\" to perceive those features.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 514
    },
    "id": "wbfmrgM7HQdc",
    "outputId": "ae748be6-d6cb-43b2-c958-34d1e3621fc8"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABKAAAAHxCAYAAABas8RJAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjUsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvWftoOwAAAAlwSFlzAAAPYQAAD2EBqD+naQAAS/BJREFUeJzt3Xmc3dP9P/AzkhBCbIlEikhiTa2hglpqLVJLhFZFW75KFa0WVVvR2qrKt5ZQpY0tbSgJUUup2NuvLaKIPZZqgiQSEiKyfH5/nMf8st4zk3vvmbkz83w+HvMYua97zufccd9z733PZ6kriqIIAAAAAJDJMs29AAAAAABaNw0oAAAAALLSgAIAAAAgKw0oAAAAALLSgAIAAAAgKw0oAAAAALLSgAIAAAAgKw0oAAAAALLSgAIAAAAgKw2o1uj660Ooq2veNRx+eAjrrtu8a4CWQs1Cy6JmoWV5+OFYs2+/3XxrOOec5v+9AS2F19lWSwOq3lVXxSd5//7lzzFhQnxxGTu2Omu65ZYQttsuhE6dQlhllRC23z6E0aOrM3e9urr5X8ssE0KPHiHsuWd8oW5uo0aF0K9fCB07hrDOOiGcfXYIc+Y096qoFbVWs8OHz3++du0awpFHhjB5cuXzLqpWa/bzz0O48MIQ+vYNYYUVQvjSl0I4+OAQXnqpeddF7ai1mv3HP0LYZZcQunSJr7HbbBPCTTdVPu+iarFmp0wJ4eKLQ9hpp/j7apVVQth22/i+A+rVUs2OGBHCt74VQu/e8TVmww1DOOmkEKZNq2zeJVl33YXrdo01QthxxxBGjqz+tso1bVpcV11dCLfd1tyroVbUUs0uWkcLfq2/fmVzL6oWX2dDCOGnP42fDVZbLf7e2njj+LOdMaN519XM2jf3AmrGsGGxUJ56KoQ33ghhvfWWfo4JE0L45S/jPFtsUdl6zjknhF/9KoSDDord19mzQ3jxxRD++9/K5l2SPfYI4bvfDaEoQnjrrfjLa9ddQ7j77hD23rv622uMe+8N4YADQvja10K44ooQXnghhPPOC+HDD0O4+urmWRO1pZZq9uqrQzj22BB22y2ESy8N4b33QrjsshCeeSaEJ5+MTalqqsWaHTw4No2POiq+2E6YEMKQIbGJ/sILIfTs2TzronbUUs2OGhVfY7bbbv5eCbfeGutq8uT4prGaaq1m//WvEM44I4R99gnhzDNDaN8+hNtvD+GQQ0IYNy7+jKGWavboo+OHysMOi3+UfOGFEK68MoR77glhzJgQll++/LmXZIstYoMrhPgYrrkmhAMPjK/3xxxT3W2V46yzQvjss+ZeBbWmlmr2d79bvNHyzjvxNWfPPcuft5Rae50NIYSnn47N6yOOiJ8FnnsuhF//Ov4B7NFHY7OsLSooivHjiyKEohgxoii6di2Kc84pb56nn47zDB1a2Xr+9a+iqKsriksvLW/80KFxHY0RQlEcd9zCt/373/H2PfcsPW7mzKKYO7d0/r3vFUXPno1bw5L07VsUm29eFLNnz7/tjDPiz+Xll8ufl9ahlmp21qyiWGWVothpp6KYN2/+7XfdFee+/PKG52jpNfvee3H7J5+88O2jR8fby/1dRutRSzVbFEWxxx5F0aNHUXz++fzbZs8uij59imKzzRoe39Jrdvz4onj77YVvmzevKHbdtSiWW64oZswob15aj1qr2YceWvy2G26Ic197bePGh1AUb73V8H179iyKAQMWvm3ixKLo1KkoNtig9LjZs+N7glLOPrvxvzdSXnihKNq3L4pf/SrO99e/Vj4nLV+t1eySnHtunPuJJxq+b0t/nS3lt7+N6/rXv6o7bwvSRttuixg2LIRVVw1hwIC4x9GwYUu+37Rp8a+i664bwnLLhbDWWvP/WvrwwyF85SvxfkccMX83wOuvj7d99lkIr7zSuENyfve7ELp3D+GEE2IXt6l309t003hIwltvxX/XHzc/fHjsWn/pS3E3wk8+ifkdd4SwySaxs7vJJqV3UZ44Mf4MZs9Ob3/cuPh19NHxr7L1jj02/jzsakwt1eyLL8btfOtbCx+r/o1vhLDiirFucmvump0+PX7v1m3h29dcM36v9l+maXlqqWZDiLWw6qpxG/Xat4911BTP1+au2V69Ft8rsa4u7hU2a1YI48dX8OBoFWqtZr/2tcVvGzgwfn/55aV6aGXp3j0ePlNfs2+/HR/Lb38b37f36RMf/7hxMX/88fjYO3aM2TXXLHneyZPjz2Bp9mY64YT42HfcsZJHRGtTazW7JH/+c3z92X778sYvjeZ+nS2l/pxSOQ4fbiE0oEKIBXrggSEsu2wI3/52CK+/HneZW9CMGfEX/RVXxN0GL7ss7oL7yivxcJuNN46HzIUQGyc33RS/dtop3vbUU/E+V17Z8HoefDAW/+WXx3MzrLRS/CDXmLHVMHVq/Fp99YVvP/fcuBvjySeHcMEF8ed1//0hDBoUC/rCC+Ob1yOOiIceLeq00+LPoKHDCJ97Ln7feuuFb+/RI/6SrM9pu2qpZmfNit+X9KF1+eXj83XevMoeb0Oau2b79Im1ecklIdx1V/z5PvVU/Hn36hUP66Ftq6WaDSF+mH3ppRB+8Yt4mMKbb8Z6eeaZEE45paoPfYmau2ZLef/9+L1Ll/LG03rUWs0uSVM+X2fPDuE//1m8ZocOjY//6KPja+Bqq8XDA/fcM5424pxzYr2effaSP9BeeWX8GTz1VOPW8de/hvDPf4bwm99U/JBoZWq9Zp97LjaLDz20ssfZWLXyOjtnTmzYTZgQt3PmmfGz/TbbVPwQW6zm3gWr2T3zTNwN7oEH4r/nzSuKtdYqihNOWPh+Z501f7fGRdUfdpPaZbF+19+zz06v56OP4v1WX70oVlyxKC6+uChuuaUo9tor3v773zf8mJZ2l8UjjyyKSZOK4sMPi+LJJ4tit93i7ZdcsvDae/cuis8+W3j8FlsUxZprFsW0afNvu//+eP9Fd1n83vcat/vzxRfH+7377uLZV75SFNtu27jHRutUazU7aVI8NPTIIxe+/ZVX4vgQimLy5PQcLb1miyKuo0+f+Y85hKLYaqt42AJtW63VbFHEQ8y++c1Yu/XP1xVWKIo77mjcY2oNNbuoKVOKYo01imLHHZd+LK1LLdbskhx5ZFG0a1cUr73W8H2X9hC8PfeMNTtpUlE8/3xRHHJIHP+jH8X7vPVW/HfnzrGuF3TAAUXRsWNRvPPO/NvGjYtrXfT3Rv1heUs6xHBRn31WFOusUxSnnbbwY3IIHi2hZk86KY4dN65x928tr7P/+tfC74033LBx9d6KaUD99KdF0a1bUcyZM/+2k05a/LYvfzmekyilGsfMvvvu/Cfo8OHzb587N54Xaa21Gp5jaQt20a+OHYvixBPnHxNbX7C//OXCYydMiLefeuri8/btW/4xs/XHtH/wweLZjjs2/P+B1q3WarYoiuJb34rnY/jtb4vizTeL4tFH47Y7dIjz/+c/6fEtvWaLIn4AGDQozn3HHfFnsfrqRbHDDvEYe9quWqzZ2bOL4swzi+Lgg4viL38piptvjudxW3HFxp2XoTXU7ILmzo1/6Fp22aIYO7Y6c9Jy1WLNLmrYsDjvKac07v5L24BatGbbtSuK73xn/gfX+gbUEUcsPHbOnKJYfvnYsFrUPvtUdg6os86KH5KnT1/4MWlAUes1O3duUXzpS0Wx5ZaNH9NaXmc//jg2Bu+4I/6+6tcvnie2DWvbh+DNnRuPA91ll3h86BtvxK/+/UP44IN4KFy9N9+Mx4PmVn8YT4cO8fjdesssE88x8957Ibz7bnW3uf/+ITzwQDwj/5NPxt0EL7lk8TPz9+q18L/feSd+X9KlNDfcsPz11P8M6g9tWtDnnzufTFtWizUbQjy3wz77xN15+/SJuypvumkI++4b8xVXrO72aq1mP/447tK93XZx1+X9949XD7r99ngejKFDy5+blq1Wa/b44+PhosOHx0NEBw+O9bTmmvH8KtVWazW7qB/9KIT77gvhuutC2Hzz6s1Ly1OrNbugxx4L4cgjQ/j610M4//w82+jff37N/vOfsWZvvHHx96CL1uykSSHMnFn9mn377RAuvjg+3mq/p6Blawk1+8gj8ZC1wYPzbaNWX2c7dw5h993j+i66KL4/3n//EJ5/vvK5W6j2Dd+lFRs9Op5IbPjwJZ8oeNiwPJeJTFlttXjys1VWCaFdu4WzNdaI36dOjZegrZa11oqF0ZCmavzUn7h44sQQ1l574WzixLZ9zGxbV4s1G0IIK68cwp13xubw22/Hk/v27BlPsti1a6znaqq1mr399vgmZ7/9Fr59553jC+8TT4Twwx82zVqoLbVYs198EcIf/xjP9bTgG9MOHeKlmq+8Mt5n2WWrt81aq9kF/fKX8XLVv/51CN/5TtNvn9pSizW7oOefj681m2wSL0rTPtNHmS5daqtmzzornjT5a1+L7zNCmH8OrEmT4m3rrNN2L+veltV6zdavYZll4rmpcqnl19kFHXhgfK0dPrzN/sGnbTeghg2LTZ0hQxbPRoyIJwv8/e/jE7VPn3i1q5QFr4BVrmWWCWGLLeJJ4xZ9AzxhQvzetWvl26mG+ivovP764tmrr5Y/7xZbxO/PPLNws2nChLgH2NFHlz83LVst1uyC1llnfnN42rQQnn02ntSwVuSq2Q8+iN/nzl349qKIt82ZU/7ctGy1WLNTpsTn5KLP1xDiiYbnzVty1hxy1Wy9IUPiSZJ/8pMQfv7zyuej5avFmq335psh7LVXXN8999TmnkBdu8afTbVr9t13414tvXsvnh17bPw+dWr1/+BF7avlmg0hHtFy++2xedqjR3Xnrobcr7OLmjUrvs/4+OPqz91CtN02+cyZsSi/8Y14qNuiX8cfHy8tPmpUvP+gQfGvLku6gkVRxO+dOsXvS7qs4tJctvJb34pvfm+4Yf5tn38ef8H07Vs7xbvmmrFZdMMNCxfRAw/Mvwztghp72covfzmEjTYK4Q9/WPhDwNVXx1+KCx6aSNtRyzW7JKedFj/k/vSn5Y3PIVfNbrBB/L7oX95GjQrh009D2HLLipZNC1WrNbvGGvFD2siR8Q899WbMiIflbbRR8/+FtF6umg0hhFtuCeHHP46HRFx6adWWTAtWqzUbQtzbZ8894x9q//732vlj7KLatYuHBt5xx8KnzHj55bjuRU2eHH8Gn32Wnve88+LPecGvc8+N2SmnxH/X/6xpO2q5Zuvdc0+cK+fhd5XI9To7bdqS73PddfH7old7b0ua+yRUzWb48HjCsVJXvJk7tyi6di2KffeN/54+PZ6IrF27ojjqqHg1ugsuiFdkqz9h5xdfFMUqq8Sz2193XTyx6fjxMVuaqwZ89lk8SVyHDkVx8slFcfnl8epv7doVxT33NDx+aU/adtxx6fukTnJ4771FscwyRbHJJkVx6aXxpK4rrxzXX8lVA+66K16daNddi+IPfyiKH/84bueooxr1sGiFarlmL7ywKAYPjrV61VXx6jkhFMV55zXusbX0mp01K46vqyuKww+PP+uTT44ngFxzzXhVEtqeWq7Z886L991yy6L43/+NJ83feON42803Nzy+pdfsk0/GE4537VoUf/pTUdx008Jfb77ZuMdG61LLNbv55vNPOr7o8/X++xsev7QnIR8wIH2f+pOQX3zx4tnzz8fXv3XWKYpf/zr+vunWrSg226yyq+AtyknIqeWarTdoUFEst9zCV5hrjJb+OjtyZFGsvXY8QfxVVxXF734XfxZ1dUWx9dbxvXMb1XYbUPvuG18cPv209H0OPzw2geovoT5lSlEcf3w8i/+yy8Yr0n3vewtfYv3OO2Nht2+/8BUElrZgP/ggzr3aarFo+/cvivvua9zYpizYoiiK22+Pb9yXWy4+9hEj4torvWzlyJHxspjLLRd/1meeGX8p0jbVcs3+7W9Fsc02RbHSSvFS7ttuWxS33tr4x9Yaavajj+KL7AYbxHm7dIlXAap/00LbU8s1WxTxKlrbbBPfaC+/fHydve22xo1t6TVbv/5SX9W+YhktQy3XbOr5uvPODY9vygZUURTFI48UxVZbxZ9J797xg359s2lBGlBUopZrtijiFeA6diyKAw9c+sfW0l9n33ijKL773Vj/yy8ffw5f/nL82c2Y0ZhH1WrVFUX9/na0GtdfH8IRR8zflRKobWoWWhY1Cy3Lww/Pv0rYuus292qAhnidbbXa7jmgAAAAAGgSGlAAAAAAZKUBBQAAAEBWzgEFAAAAQFb2gAIAAAAgKw0oAAAAALLSgAIAAAAgq/aNvWNdXV3OdUCbV+3TsalZyEvNQsuiZqFlUbPQsjSmZu0BBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBW7Zt7AQAAAJX605/+VDI74ogjkmPvuOOOZD5w4MBylgTAAuwBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBW7Zt7AQAALc28efOS+YgRI5J5XV1dMh83blzJ7Be/+EVyLLRVqbpsqGaLoqj2cgBYhD2gAAAAAMhKAwoAAACArDSgAAAAAMhKAwoAAACArDSgAAAAAMhKAwoAAACArDSgAAAAAMiqfXMvoC378Y9/nMwvv/zyJloJUOvOPPPMZP6rX/0qmdfV1ZXMJk2alBy76667JvMXX3wxmUNrVBRFMj/ggAOSeaomQwhh//33L5mNGTMmOXbkyJHJHIDq6tChQzJfffXVk/nGG29cMuvatWty7Fe+8pVkvtdeeyXzTp06lcxuu+225NiGXHrppcn8448/LpnNnDmzom1Tm+wBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZFVXNHQd4fo7NnC54LYqddnKX//618mx6667bjLfd999y1kSLVQjS7HR1GzTW2+99ZL5KaecUjL77ne/mxzbvn37ZJ7z//ebb76ZzDfYYINs265larZtO/rooysaf9555yXz1CW7x4wZkxzb0CW52yo12/pdd911JbMjjjgiOfbOO+9M5gceeGBZa6J8tVSzPXr0SOY//vGPk/nJJ59c9rYb0tDjqvbPcWk0tLYnnniiZHb22Wcnxz700ENlrYl8GvNcswcUAAAAAFlpQAEAAACQlQYUAAAAAFlpQAEAAACQlQYUAAAAAFlpQAEAAACQlQYUAAAAAFm1b+4FtHS9evUqmR177LHJsf3796/2coCMNthgg2T+s5/9LJn/z//8TzWXs5APPvggmX/66acls969eyfHNpQPGTIkmR933HHJHFqiP/zhDxWN79evXzL//ve/X9H8AFTPT37yk2R+4oknJvMpU6Yk8zFjxiztkv6/urq6ZN6pU6dkvt1225W97Uptv/32JbOLLrooOXa33XZL5tOnTy9rTeRlDygAAAAAstKAAgAAACArDSgAAAAAstKAAgAAACArDSgAAAAAstKAAgAAACArDSgAAAAAsmrf3Ato6X73u9+VzF588cXk2JkzZ1Z5NUBOw4cPT+abb7552XOPHDkymT/99NPJ/Pe//30yv+GGG0pmvXv3To6tq6tL5vvtt18yP+6445I5sLhU3T322GNNuBIA/vd//zeZ33jjjcl8xowZyfztt99e2iU12vLLL5/Md99995LZz372s+TY7bffvqw1NUa/fv2S+YABA5J5Q+/baR72gAIAAAAgKw0oAAAAALLSgAIAAAAgKw0oAAAAALLSgAIAAAAgKw0oAAAAALLSgAIAAAAgq/bNvYBat8ceeyTz9u1L/wg333zzai+nyfTp0yeZr7LKKiWzZ599Njl2l112SeZf/epXk3klnn/++WR+1113Zds2tW+99dZL5t27d69o/oceeqhkduihhybHfvHFFxVtO6eGah5Y3MCBA5N5URQlsxEjRlR7OQAkTJw4saK8Oc2cOTOZpz7//OMf/0iOvfXWW5P5Pvvsk8wrce211ybzjz/+OJnfe++91VwOjWQPKAAAAACy0oACAAAAICsNKAAAAACy0oACAAAAICsNKAAAAACy0oACAAAAIKv2zb2AWvf1r389mc+bN6+JVrK4Hj16lMzuuOOOiubu3LlzMl9uueVKZu+9915ybNeuXZP5+uuvn8wrMXny5GT+zjvvJPNtttmmmsuhxhx99NHJvFu3bsm8oef+ySefXDL74osvkmOb04wZM5L5pZde2kQrgdZj5MiRyfyoo44qmT3++OPVXg60CD179kzm66yzThOtBNqGmTNnJvN99903md99993JfK+99lrqNdVbYYUVkvnf/va3ZJ76vPvpp5+WtSYaZg8oAAAAALLSgAIAAAAgKw0oAAAAALLSgAIAAAAgKw0oAAAAALLSgAIAAAAgKw0oAAAAALJq39wLaG49evRI5ptvvnky//73v18y23rrrZNj33333WT+4YcfJvM//elPJbPOnTsnx9bV1SXz9dZbL5mn3HDDDcm8Xbt2yfz0008ve9sN6dKlSzJ/6qmnsm2b2nDQQQeVzE444YSK5h4zZkwyHzt2bNlzN/TcPeyww5L5rrvuWva2H3744WT+6KOPlj03tFRdu3ZN5qeddloyHzhwYDIfN27cUq8JWrvdd989me+2224ls9/85jfJsU8++WRZawJKO/LII5P5e++910QrWdyoUaNKZqnfJVTGHlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZNW+uRfQ3G666aZk/rWvfS2ZX3PNNSWzddZZJzl28ODByfzDDz9M5jNmzCiZHXTQQcmxyyyT7j1269Ytmac8+uijyXzttddO5g2tvVevXiWzDh06JMf+/e9/T+b/8z//k8xp+VZeeeWSWfv2lf1K7N69ezLff//9y5574403Tubnn39+2XOnfpeEEMKll15a9txQy3r27JnMJ02aVDI77LDDkmNPOOGEZP7ZZ58l84MPPjiZA0tn6NChyfy1115ropVA2zF16tRkPnr06JLZrrvuWu3lLGTrrbcumfXt2zc5dty4cdVeTpthDygAAAAAstKAAgAAACArDSgAAAAAstKAAgAAACArDSgAAAAAstKAAgAAACArDSgAAAAAsmrf3AvIrX///sn8K1/5SjJ/7rnnkvmpp55aMjvxxBOTYz/66KNk3pCDDjqoovHN5Y033kjm22yzTTIfMmRIyWzw4MHJsRMmTEjmkyZNSua0fO+9917J7LPPPkuOXWGFFZJ5Q8/dESNGJPPm8t///jeZP/LII020EmhaTz31VDJPvY6nXv9DCKEoimR+wQUXJPNXXnklmUNr9M1vfjOZX3vttcn8yCOPLJm99tprZa0JKN+sWbOS+ejRo0tmu+66a7WXs5B58+aVzD7//POs227L7AEFAAAAQFYaUAAAAABkpQEFAAAAQFYaUAAAAABkpQEFAAAAQFYaUAAAAABk1b65F5DbD37wg2TeqVOnZD5s2LBk/uyzz5bMBg8enBzLknXr1i2Z+7lSib///e8lsyeffDI5dpdddqn2cv6/8ePHJ/PevXtn23ZDl7WGlmrgwIHJvGvXrsn89NNPL3vsyy+/nMwvvPDCZA5t0ZAhQ5J5URRNtBKgKVxyySUls1VXXTU59qSTTqpo2507dy6Z3XDDDcmxO+64Y0XbbsvsAQUAAABAVhpQAAAAAGSlAQUAAABAVhpQAAAAAGSlAQUAAABAVhpQAAAAAGSlAQUAAABAVu2bewHVcNZZZ5XMDjvssOTYxx9/PJlfccUVZa2J0s4555xk/vOf/zyZX3bZZSWz008/PTl27ty5yZy2raHfF1dffXUy79evXzJ/5513SmYNPXdTz/sQQthiiy2Secp//vOfssdCbhtttFHJbNCgQcmxp556ajIviiKZ33777SWzvn37JscecMAByfyMM85I5ueff34yB4Ba171792S+9957l8x69eqVHFtXV1fWmuots0zpfXFWWWWV5Njlllsumc+aNaucJbUJ9oACAAAAICsNKAAAAACy0oACAAAAICsNKAAAAACy0oACAAAAICsNKAAAAACy0oACAAAAIKv2zb2AajjnnHNKZkVRJMfOmzcvmc+ZM6ecJbVpF1xwQTLfY489kvlFF12UzO+7776S2eeff54cCynvv/9+Mh84cGAyX2+99ZL5G2+8UTLr3LlzcuzcuXOTeUPeeuutktnjjz9e0dxQiZ49eybz888/v2TWUE0++uijyXzddddN5n/+859LZp06dUqOffnll5P5ueeem8zffvvtktmwYcOSY6FWrbXWWsm8Xbt22bbdvn36Y09Dr+GVuPrqq5P5Y489lsx/+MMflr3tjz/+OJlPnDgxmTf0O/qjjz4qmS277LLJsT/72c+SOU2jd+/eJbP+/fsnx+61117J/LDDDkvmDX1WzzU2hHQfYOONN06Oveuuu5L5qaeemszHjBmTzFsze0ABAAAAkJUGFAAAAABZaUABAAAAkJUGFAAAAABZaUABAAAAkJUGFAAAAABZaUABAAAAkFX75l5ANdTV1ZXMiqJIjl1ppZWSeffu3ZP5+++/n8xbqq233jqZH3PMMSWz7373u8mxEydOTOY33nhjMh8/fnwyh+byxhtvlD12wIAByXyrrbYqe+4Q0mtrrb/HaBka+p3/1a9+tWQ2adKk5NgTTzwxmb/77rvJfPLkySWzFVZYITn25ZdfTuYjR45M5qeffnrJ7LPPPqtobmguV155ZTJfeeWVK5o/9fuic+fOybGXXnppRdtOaeh31euvv57MK6npJ554IpnfcMMNyfy0005L5vfee2/JrFu3bsmx/fr1S+ZUR0Ofd2+55ZaS2ZZbblnt5bQKu+66azK/8847k/lmm21WMps6dWpZa2op7AEFAAAAQFYaUAAAAABkpQEFAAAAQFYaUAAAAABkpQEFAAAAQFYaUAAAAABkVVcURdGoO9bV5V5L2ebNm1cya+TDK+kf//hHMv/2t79dMvvoo48q2nYlUpd2DCGEgw8+OJmfcsopyfyee+4pmT311FPJsY8++mgyb+hysa1Vpc/VRdVyzea06qqrJvPf/va3JbOxY8cmx15xxRXlLKlRHnvssWS+/fbbVzT/wIEDS2ajRo2qaO62Ss02TteuXZP5Bx98kMxTrxlf+9rXyllSi/fhhx8m87322iuZjxkzpprLaTHUbNPYe++9S2ZXXXVVcuw666xT7eU02nPPPZfMhwwZUvbcDdXs3XffXfbcrZmarY6G6uraa68tme22224VbbuhunrppZfKnvsb3/hGMl9llVWSeer50NBzb+LEicn8oosuSuY33nhjyeyTTz5Jjq1ljalZe0ABAAAAkJUGFAAAAABZaUABAAAAkJUGFAAAAABZaUABAAAAkJUGFAAAAABZaUABAAAAkFX75l5ANbz55psls65duybHrrTSSsl89913T+bDhw8vmR177LHJsRdffHEyX2+99ZJ5SufOnZP55Zdfnsz79euXzCdOnFgy++ijj5JjIadDDjkkmR9++OEls2HDhlV5NQtbYYUVSmYdO3bMum1oLgMHDkzmRVEk85EjR1ZzOa3Cd77znWTet2/fZD5mzJhqLgcWsummm5bM1llnnazbnj59esls0KBBybEvv/xyMp8wYUJZa4Lm9u677ybz/fbbr2S28sorV7TtTz/9tKI85Qc/+EEyHzJkSNlzN+TFF19M5ldeeWW2bbd09oACAAAAICsNKAAAAACy0oACAAAAICsNKAAAAACy0oACAAAAICsNKAAAAACy0oACAAAAIKv2zb2Aalh//fVLZn/605+SY2fMmJHMt91222S+2267lcxeffXV5NhKzZkzp2R22WWXJcc++OCDyfyll14qa03QkjVU7927d0/mH3/8cTK/8sorS2b9+vVLjm3I7Nmzk/nnn39e0fxQrkmTJiXzKVOmJPOjjz66ZPbuu+8mx44cOTKZ17KBAweWzEaMGJEcO2/evGR+8803l7UmCCGEAw44IJlvvPHGZc99zz33JPOGXuuGDBlSMmvovS+0VbNmzSqZffjhh024kqXz+OOPJ/NPP/00mXfu3Llk1tDr6IYbbpjM11xzzWQ+ceLEZN6a2QMKAAAAgKw0oAAAAADISgMKAAAAgKw0oAAAAADISgMKAAAAgKw0oAAAAADISgMKAAAAgKzaN/cCcvv1r3+dzMePH5/M11lnnWR+5513lsy6d++eHFups846q2R29dVXZ9021KrPP/88mc+bN69k1qdPn+TYSy65JJmvuuqqyfzrX/96Mk+ZM2dOMn/kkUeS+f3331/2tqESI0eOTOY9e/ZM5kceeWTJ7MYbb0yOveCCC5L5hRdemMxzOuOMM5L5qaeeWjJL/R4LIYTzzz+/rDVBY4wYMSKZF0VRMrv++uuTY48//vhkPnPmzGQOtB0vvfRSMn/77beT+SabbFIyS/0eCyGE1VZbLZl36tQpmbdl9oACAAAAICsNKAAAAACy0oACAAAAICsNKAAAAACy0oACAAAAICsNKAAAAACyqisausZg/R3r6nKvBdq0RpZio6nZJRs/fnzJrKHLwTenhx56KJnvvvvuTbQS6qnZprHRRhuVzO67777k2K233jqZT548uaw1NcZNN92UzFOPK4QQunTpUjI78cQTk2NHjhyZzNsqNVsdP/jBD5L59ttvXzL76U9/mhz70UcflbUmWic1SyVOPvnkZH7RRReVzBp67o0ZMyaZX3311cl86NChybylakzN2gMKAAAAgKw0oAAAAADISgMKAAAAgKw0oAAAAADISgMKAAAAgKw0oAAAAADISgMKAAAAgKzqiqIoGnXHurrca4E2rZGl2GhqdsmOPPLIktkf/vCHJlzJwl599dVkvtdeeyXzd999t5rLoRHUbPPbYYcdkvkrr7ySzCdPnlz2tjfaaKNk/vTTTyfzCy64IJlfe+21JbNK1t2WqVloWdQslejatWsyHz16dMls4403rmjb06ZNS+YbbrhhyWzKlCkVbbs5NaZm7QEFAAAAQFYaUAAAAABkpQEFAAAAQFYaUAAAAABkpQEFAAAAQFYaUAAAAABkpQEFAAAAQFZ1RVEUjbpjXV3utUCb1shSbDQ1u2RbbLFFyWzAgAHJsSeccEIy/8tf/pLMP/zww5LZDTfckBz73nvvJXOanpqFlkXNQsuiZsnpG9/4RsnswAMPTI5dYYUVknnXrl2T+f77718ymzFjRnJsLWtMzdoDCgAAAICsNKAAAAAAyEoDCgAAAICsNKAAAAAAyEoDCgAAAICsNKAAAAAAyEoDCgAAAICs6oqiKBp1x7q63GuBNq2RpdhoahbyUrPQsqhZaFnULLQsjalZe0ABAAAAkJUGFAAAAABZaUABAAAAkJUGFAAAAABZaUABAAAAkJUGFAAAAABZaUABAAAAkJUGFAAAAABZaUABAAAAkJUGFAAAAABZaUABAAAAkJUGFAAAAABZaUABAAAAkJUGFAAAAABZaUABAAAAkJUGFAAAAABZaUABAAAAkJUGFAAAAABZaUABAAAAkJUGFAAAAABZaUABAAAAkJUGFAAAAABZ1RVFUTT3IgAAAABovewB1Rpdf30IdXXNu4bDDw9h3XWbdw3QUqhZaFnULLQsahZaFjXbamlA1bvqqvgk79+//DkmTAjhnHNCGDu2WquK9tgjru3446s7bwhx3vqvZZYJoUePEPbcM4SHH67+tpbGjBkh/OQnIay1VgjLLRfCxhuHcPXVzbsmakut1ezw4SH06xdCx44hdO0awpFHhjB5cuXzLqpWa/bzz0O48MIQ+vYNYYUVQvjSl0I4+OAQXnqpeddF7VCztVOzDz+88LoW/Tr//OZbG7Wj1mr2H/8IYZddQujSJYRVVglhm21CuOmmyuddVC3WbAjeG9OwWqvZEEK45ZYQttsuhE6dYt1uv30Io0dXZ+56tVqz66675NfYY45p3nU1s/bNvYCaMWxYfJI89VQIb7wRwnrrLf0cEyaE8Mtfxnm22KI66xoxIoR//as6c5Wyxx4hfPe7IRRFCG+9FX957bprCHffHcLee+fd9pLMnRvC178ewjPPhHDccSGsv34If/97CMceG8LUqSGcfnrTr4naU0s1e/XV8fm5224hXHppCO+9F8Jll8Xn8JNPxg+41VRrNRtCCIMHhzBqVAhHHRU/1E+YEMKQIfFNxwsvhNCzZ/Osi9qhZmunZjfeeMkf3G+6KYT7749v3KGWanbUqBAOOCC+ppxzTvwQd+utsa4mTw7hpz8tf+4lqbWa9d6Yxqilmg0h1uqvfhXCQQfFvYlmzw7hxRdD+O9/K5t3SWqtZuttsUUIJ5208G0bbNAsS6kZBUUxfnxRhFAUI0YURdeuRXHOOeXN8/TTcZ6hQ6uzrpkzi2LddYviV7+K8x53XOPGDR0a798YS5r33/+Ot++5Z3ptc+eWzr/3vaLo2bNxa1jUrbfG7f/xjwvfPmhQUXTsWBQffFDevLQetVSzs2YVxSqrFMVOOxXFvHnzb7/rrjj35Zc3PEdLr9n33ovbP/nkhW8fPTrefuml5c1L66FmF76tuWu2lPXWK4r116/unLRMtVSzRVEUe+xRFD16FMXnn8+/bfbsoujTpyg226zh8S29Zr03piG1VrP/+ldR1NWV/x6wpddsUcSxAwaUP76VcgheCLFbvOqqIQwYEDu0w4Yt+X7TpsW/sKy7btz1da215v/l5eGHQ/jKV+L9jjhi/i52118fb/vssxBeeWXpdu//zW9CmDcvhJNPLv+xlWPTTePuzW+9Ff9dv6v+8OEhnHlmPLRmhRVC+OSTmN9xRwibbBL/YrzJJiGMHLnkeSdOjD+D2bPT23/ssfj9kEMWvv2QQ+JhPnfeWe4jo7WopZp98cW4nW99a+Fj1b/xjRBWXDHWTW7NXbPTp8fv3botfPuaa8bvyy9fzqOiNVGzC2vuml2S+r+YDx5cziOitamlmg0h1sKqq8Zt1GvfPtZRU7zGNHfNem9MQ2qtZn/3uxC6dw/hhBPiXkkzZlT8EJdKc9fsgr74IoRPP63k0bQqGlAhxAI98MAQll02hG9/O4TXXw/h6acXvs+MGSHsuGMIV1wRd02/7LJ4/OYrr8Rd9zfeOO5iGEIIRx8dd2O/6aYQdtop3vbUU/E+V17ZuDW9+24Iv/51CBdd1PQf3qZOjV+rr77w7eeeG3djPPnkEC64IP687r8/hEGDYkFfeGHcPfqII+Iuwos67bT4M2hot8tZs0Jo1y7Ov6AVVojfn3227IdGK1FLNTtrVvy+pDpdfvkQnnsuNpJzau6a7dMnvoG55JIQ7ror/nyfeir+vHv1WvwNM22Pml1Yc9fsktR/WNGAIoTaqtkQQvja1+I5BX/xi9goffPNWC/PPBPCKadU9aEvUXPXrPfGNKTWavbBB2Mz6/LL43kWV1op/mGysZ+FK9XcNVtv9OhYpyuuGJt+l11W6SNr8ZwD6tlnY9FdcUX89w47xA9Sw4bN7wCHEMLFF8e/mo4YEcLAgfNvP/PM2NWtq4vHl551Vjw+/bDDKlvXSSeFsOWWTfPB7fPPYye7/pjZ00+Px5offPDi93vmmYXftP/853Gvh8cfD2HlleNtO+8cf6mVe86XDTeM2/+//4v/P+rV//Unx3HDtBy1VrPrrx/neuKJ+GJV79VXQ5g0Kf73kl4AK1FrNduhQwi33x7CoYeGsN9+82/faqsQ/vnPeNJJ2i41W3s1u6i5c+OJYrfZprxzhtC61FrNhhAbT2+9FU+Qf9558bYVVoivPfvvX/68pdRazXpvTEqt1ezUqbF+nngiNmDOPjuEddYJYejQEH70o/i+8Qc/KP/xLkmt1WwIIWy2Wfx/seGGIUyZEvck+8lP4nm2Lrqo/HlbuuY+BrDZ/fSnRdGtW1HMmTP/tpNOWvy2L3+5KDbfPD1XtY6ZHT06HjP71FPzb8t5DqhFvzp2LIoTT5x/TOxDD8Xbf/nLhcdOmBBvP/XUxeft27f8Y2YnTiyKlVeO56G4//6ieOutorjmmqLo3Dlub7fdypuX1qEWa/Zb3yqK9u2L4re/LYo33yyKRx+N2+7QIc7/n/+kx7f0mi2KonjttXguilNPLYo77og/i9VXL4oddojH2NN2qdnarNkF/f3vcTuXXVad+WjZarFmZ88uijPPLIqDDy6Kv/ylKG6+OZ7HbcUV47lmGtLSa9Z7Y1JqrWbffXd+7QwfPv/2uXNjHay1VsNztPSaXZJ584ri61+P7z8aep/RirXtQ/Dmzo3Hge6yS+yUvvFG/OrfP4QPPoi7DtZ78814PGhuc+aE8OMfh/Cd7yzcsc5p//1DeOCBeHnbJ5+M3eNLLomXsVxQr14L//udd+L39ddffM4NNyx/Pd27x6udzJoVO8+9eoXws5/N7+qvuGL5c9Oy1WLNhhDCNdeEsM8+cXfePn3irsqbbhrCvvvGvNrP2Vqr2Y8/jrt0b7dd3HV5//3jXpy33x7/mjR0aPlz07Kp2ajWanZRw4bFw3u+9a3qzUnLVKs1e/zx8RDv4cPj0QGDB8d6WnPNeI6Zaqu1mvXemFJqsWbr9y7q0CGej6reMsvE15n33ounm6mmWqvZJamri+ffmjMnnpOqjWrbh+CNHh1PJDZ8+JJPOjpsWNNfivjGG+NhANdcE8Lbby+cTZ8eb1tjjfnHfFfDWmuFsPvuDd+vKc9FtdNOIYwfHy/f/umnIWy+edxdMQSXrmzLarFmQ4i76955Z3wxffvtuLtuz54hbL99PO692oeg1VrN3n57fJOz4OF3IcTdlzt3jrtg//CHTbMWaouajWqtZhc0c2Y82eruuy9+IQHanlqs2S++COGPf4znelrww2SHDvFwoSuvjPdZ9PxIlajFmvXemCWpxZpdbbV4Mu9VVol/3FjQGmvE71OnxsPyqqUWa3ZJ1l47fv/oo+ZdRzNq2w2oYcNiEQwZsng2YkR8Q/b738cnap8+8ZjZlAWvplOud9+NZ9X/6lcXz268MX6NHBlPjtbc6o+Jff31xbNXX618/nbtQthii/n//sc/4vfG/HKhdarFml3QOuvMfzGdNi0ekz9oUHW3UYlcNfvBB/H73LkL314U8bY5c8qfm5ZNzVYm9+tsCHGviunTnXycqBZrdsqU+Dqy6GtMCPE987x5S86ag/fGNLVarNlllonP06efXrw5XN807dq18u1UQ1O8zi5o/Pj4vVYefzNou4fgzZwZi/Ib34i7Bi76dfzx8Q3ZqFHx/oMGhfD880u+JGNRxO+dOsXv06Ytfp/GXrbykEPiNhb9CiEeLjByZNylshasuWb85XLDDfEQnHoPPBDCuHGL37+Sy0NPmhRP1rbZZl5k26pardlSTjstvmH+6U/LG59Drpqt/8vron95GzUq/pV2yy0rWjYtlJqtXFO8zv75z3Gv6gVPSEvbVKs1u8YacU+KkSPjh9l6M2bEw/I22qj592qo570xTalWazaEeKjd3LmxFup9/nlsmPXtG0KPHkv1ULPJVbMffbR4Y3z27HiV+2WXjYdMtlXNfRKqZjN8eDzh2B13LDmfO7counYtin33jf+ePj2eiKxdu6I46qii+P3vi+KCC4pi222LYuzYeJ8vviiKVVYpig03LIrrrosnSRw/Pmb1Jz47++zy1pvzJOQNzVu/9r/+dfHs3nuLYpllimKTTYri0kvjCSJXXjme5G7Rk7Z973txnrfeanhdO+1UFD//eVFce21RnHtuUay9dlGsumpR/PvfjXpYtEK1XLMXXlgUgwcXxeWXF8VVVxXFnnvGseed17jH1tJrdtasOL6urigOPzz+rE8+OZ4Acs01i2LSpMY9NloXNRvVYs3WmzIlnnj9kEMad39at1qu2fPOi/fdcsui+N//jRcQ2HjjeNvNNzc8vjXUrPfGLKqWa/azz+JzvkOH+J7w8suL4itfidu+556Gx7f0mh06tCj69Ik1W/9z3mSTOPaCCxr3uFqptrsH1LBh8djUPfZYcr7MMiEMGBDCfffFXX9XXDFe6vSHPwzhnnviicKvuiqenGytteKYDh1i97RduxCOOSaEb387hEceabrH1Bz22iuEv/41dnhPOy124YcODWHrrSubd6ut4rzHHx/C5ZfHExw/80w8SSxtUy3X7Kabxl13zzgjntT4k09CuPXW+O9ak6Nml102/qx/8pMQ/vnPeELY66+Phwo/8UQIXbpUafG0KGq2OnK9zoYQ5509O4RDD618Llq+Wq7ZM86I6+vQIYRf/jKEX/winmPwtttq7/BR741pKrVcs8svH89PdeihIfzpT/Gk+cssE8Ldd8dzt9WSHDW76aZxT6+bb44/5wsuiHty3npr3EYbVlcU9fvb0Wpcf30IRxwxf1dKoLapWWhZ1Cy0LGoWWhY122q13T2gAAAAAGgSGlAAAAAAZKUBBQAAAEBWzgEFAAAAQFb2gAIAAAAgKw0oAAAAALLSgAIAAAAgq/aNvWNdXV3OdUCbV+3TsalZyEvNQsuiZqFlUbPQsjSmZu0BBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBW7Zt7AQAAACuttFIyP/7448uee88990zm2267bTK/9NJLy86nTJmSHAvQVtgDCgAAAICsNKAAAAAAyEoDCgAAAICsNKAAAAAAyEoDCgAAAICsNKAAAAAAyKquKIqiUXesq8u9FpbSyiuvXDL729/+VtHc119/fTL/4x//WNH8LK6Rpdhoarb6evfuncx33333sufedNNNyx4bQggvvPBCMv/zn/9cMpsxY0ZF226r1GzLt/rqqyfzq6++umQ2YMCA5NgDDjggmT/wwAPJnOpTs81vww03TOZPPfVUMu/UqVPZ227o/1elz4/PPvusZHbaaaclxw4ZMqSibbdWahZalsbUrD2gAAAAAMhKAwoAAACArDSgAAAAAMhKAwoAAACArDSgAAAAAMhKAwoAAACArDSgAAAAAMiqriiKolF3rKvLvZY2Z6WVVkrmu+22WzIfO3ZsyezNN99Mjm3o/+fs2bOT+ZZbblkyGzduXHIsS9bIUmw0NbtkHTt2LJmdccYZybGHHHJIMu/Tp09Za2oKf/nLX0pmxx13XHLstGnTqrya1kHN1r6jjz46mV944YXJ/L///W/J7IYbbkiOvfvuu5P5K6+8ksxTOnTokMz79u2bzN99991kPnXq1KVeU0ugZptGly5dSma33HJLcuzOO+9c7eX8fw39/6r282NBs2bNSuYPPvhgMt9vv/2quZwWQ83SXFZfffVkvsIKK1Q0f7du3UpmDf0enDJlSjIfNmxYMm/oc34lGlOz9oACAAAAICsNKAAAAACy0oACAAAAICsNKAAAAACy0oACAAAAICsNKAAAAACy0oACAAAAIKv2zb2A1uzLX/5yMh81alQy/9KXvpTMt9lmm5LZ//3f/yXHbrfddsm8ffv0U2OzzTYrmY0bNy45FprTqaeeWjI744wzsm77b3/7W8ns/vvvT449+uijk/kmm2ySzL/97W+XzJZffvnk2AMPPDCZQ0577bVXyWzNNddMjm3otW7YsGHJ/Nxzzy2ZTZo0KTk2px/84AfJ/LLLLkvmhx9+eDK/6aablnZJtCFrrLFGMk/V1c4771zt5bQIyy23XDLv0qVLE60EWo4ddtghmTf0WXunnXZK5qnPs2uvvXZybOfOnZN5c2rovdGFF17YRCtZMntAAQAAAJCVBhQAAAAAWWlAAQAAAJCVBhQAAAAAWWlAAQAAAJCVBhQAAAAAWWlAAQAAAJBV++ZeQEu36qqrlsz+/Oc/J8f26tUrmY8dOzaZ//vf/y6Z3Xnnncmx2223XTJvyKBBg0pmw4cPr2huqMQnn3ySzDt16lT23O+8804yHzhwYDJ/4YUXSmZz585Njr3mmmuS+TLLpP+e8Prrr5fMdtlll+TYhpxyyinJ/Fe/+lXJbPTo0cmxgwcPTuZTp05N5tS+gw8+OJl/97vfLZkdeuihybFDhw4ta021IPX+4uSTT27ClcDCGnqtq/Q1JWXOnDnJ/IwzziiZPfLII8mxBx10UDJXd7RFHTt2TOZf/epXk/kBBxyQzFN1t+KKKybHjhs3Lpk//PDDyfzGG28smT3//PPJse+//34yz2mLLbZI5s8991wyv/DCC6u4mqVnDygAAAAAstKAAgAAACArDSgAAAAAstKAAgAAACArDSgAAAAAstKAAgAAACCr9s29gFq3wQYbJPMhQ4aUzDbddNPk2BdffDGZN3RZy5TbbrstmV900UVlzx1CCD169KhoPJSrW7duyXy55ZZL5nV1dSWzSZMmJcc2dOnpsWPHJvNKzJ49u6Lxqcvcbrjhhsmxjz76aDLfeuutk/myyy5bMtt2222TY9dYY41kPnXq1GRO7evdu3cyf+aZZ0pm06dPr/Zyakbq8tINXYo+9XsOKnXEEUc027Zfe+21ZH7JJZeUPXf//v3LHgst2brrrlsy++1vf5scu++++ybzF154IZmfeuqpJbO77747OXby5MnJvLU69NBDk/l9993XRCspjz2gAAAAAMhKAwoAAACArDSgAAAAAMhKAwoAAACArDSgAAAAAMhKAwoAAACArDSgAAAAAMiqfXMvoLn1798/mV9++eXJfOutty6ZFUWRHDty5MhkPmfOnGSeMn78+GTe0NoayjfddNOS2QYbbJAc+9prryVzSDnzzDOTeYcOHZL5pEmTSmYDBgxIjh07dmwyr2Uff/xxyayhn+n6669f7eX8fzfffHMyf/XVV7Ntm6bRu3fvZH7qqacm89NOO62ay2kxUq/Dq622WnLsAw88kMxvu+22stYEub388svJfL/99su27cGDB2ebuyGp9ybQkJVWWimZn3LKKcn8xBNPLJkNGzYsOTb1mTAEn/vKcfjhhyfz3XbbLZnvtddeVVxN9dkDCgAAAICsNKAAAAAAyEoDCgAAAICsNKAAAAAAyEoDCgAAAICsNKAAAAAAyEoDCgAAAICs2jf3AprbSSedlMy33nrrbNv+xS9+kcx32WWXZH7rrbeWzG666aay1tRYnTp1Kpn17NkzOfa1116r9nJoRTbZZJNkfvTRR1c0/9SpU0tmzzzzTEVz17I777yzZLb++utXNPfkyZOT+d///veS2c9//vOKtk3tu+qqq5L5s88+m8yvu+66ai6nZuy9997JfPXVVy+ZjR49Ojn2m9/8ZjKfOXNmMofm8uabbybzt99+u+y5991332S+5ZZblj13pX73u98127apfZ07d07mo0aNSuZ9+vRJ5occckjJ7K677kqOpTw77LBDyWyfffZJjt12222T+ezZs8taU1OxBxQAAAAAWWlAAQAAAJCVBhQAAAAAWWlAAQAAAJCVBhQAAAAAWWlAAQAAAJCVBhQAAAAAWbVv7gU0t379+mWbe/z48cn8n//8ZzJfZZVVkvlll11WMvv5z3+eHAu1ql27dsm8Q4cOTbSS2rLGGmsk84MPPjiZ9+zZs5rLWchjjz2WzL/zne9k2zYt39lnn53M58yZ00Qrqa7evXsn87vvvjuZf/TRRyWzo48+Ojl2+vTpyRxSNt9882S+zjrrZNv2f/7zn2xzb7XVVsk85/uLV199NZm//vrr2bZNy9C9e/eS2ciRI5Njp02blswbqunU6w15/Pvf/y6ZHXfcccmxs2fPrvZympQ9oAAAAADISgMKAAAAgKw0oAAAAADISgMKAAAAgKw0oAAAAADISgMKAAAAgKzaN/cCmttOO+2UzHfbbbdkPm7cuJLZs88+W9aaGit1Odn77ruvornr6uoqGg/lev/995P5k08+mcz79++fzLt06VIyO/fcc5Njzz///GT++eefJ/OU0047LZkfccQRyXy99dYre9sNrbuhx3311VeXvW1av4ceeiiZv/XWW020kuraaKONkvmDDz6YzBt6nf3Rj35UMst5qXro3bt3Ml9jjTWybXvllVdO5h07dkzmv/jFL0pmp556anJsURTJvBLvvfdeRTmt35577lky69y5c3Ls3nvvncynTZtWzpIapaGaXX311ZP5+PHjq7mcFuOTTz5p7iU0G3tAAQAAAJCVBhQAAAAAWWlAAQAAAJCVBhQAAAAAWWlAAQAAAJCVBhQAAAAAWWlAAQAAAJBVXVEURaPuWFeXey1U0aqrrprMp0yZkswb+bRYor322iuZP/DAA2XP3ZpV8jNfktZasyuttFIynzx5cjLv0KFD2dtuaO5K/h926dIlmVf6/3PSpEklswcffDA59tBDD61o262Vmm2cU045JZkPGDAgmR9wwAEls6lTp5azpEZbfvnlS2Y77LBDcuxf//rXZP7yyy8n8913371k9umnnybHsmRqtjqefPLJZL711ls30UqWzjLLpP/uPm/evGzb7t+/fzJ/5plnsm27JWtLNXvXXXeVzMaMGZMce/bZZ1d7OY02atSoZL7zzjsn80cffbRkNmLEiOTYhvLp06cn85w131Y1pmbtAQUAAABAVhpQAAAAAGSlAQUAAABAVhpQAAAAAGSlAQUAAABAVhpQAAAAAGSlAQUAAABAVu2bewHkMXXq1GT+0ksvJfO+fftWczlQNdOnT0/mv/nNb5L5McccUzJbffXVk2O7dOmSzGvZkUceWTL729/+1oQroa1pqCYbcvjhh5fM3n777eTYPn36JPPVVlstmadeCx966KHk2IZccsklyfzTTz+taH7IpSiKivLmMm/evGReq+umbejevXvJbMCAAcmxZ599drWX02j77bdfMl977bWTeeqxpd6zhxDCH//4x2Q+atSoZP6jH/2oZPaf//wnOZby2QMKAAAAgKw0oAAAAADISgMKAAAAgKw0oAAAAADISgMKAAAAgKw0oAAAAADISgMKAAAAgKzqiqIoGnXHurrca6EJ3XDDDcn8sMMOK3vuY489Nplfc801Zc/dmjWyFBtNzS5Zly5dSmY//vGPK5p78ODBybxXr14VzZ/y+OOPJ/P99tuvZDZt2rQqr6ZtULNNY4MNNiiZNfR609Bze9SoUcl8zJgxJbNtt902OfaJJ55I5u3atUvmVJ+arY6BAwcm87/+9a9NtJKl09D/r2o/PxbU0O+LZ555Jtu2W7K2VLN9+/YtmY0dOzY59rrrrkvm55xzTjL/8MMPk3lzad++fTLfe++9k/lRRx2VzHfeeeeS2aBBg5Jj//GPfyTztqoxNWsPKAAAAACy0oACAAAAICsNKAAAAACy0oACAAAAICsNKAAAAACy0oACAAAAIKu6opHXt6zly1ay9I455phkftVVV5U990033ZTMv/e975U9d2vWli4121J17949md99993JfMsttyx720ceeWQyHzlyZDJv6HL0LD012/qttNJKJbOGLos9YcKEZL7jjjuWsyQqoGaro1OnTsl82223LZn98Ic/rGjbm222WTLv3bt3yayh/1/Vfn4s6JZbbknmgwcPzrbtlkzNRg3Vzfnnn5/MG/o53nbbbSWzhj7XPf7448m8ll188cUls4Zqsl+/fsn8/fffL2tNLV1jatYeUAAAAABkpQEFAAAAQFYaUAAAAABkpQEFAAAAQFYaUAAAAABkpQEFAAAAQFYaUAAAAABk1b65F0DzePzxx5P5zJkzk3nHjh1LZmPGjClrTVDrdtppp2S+5ZZblj33nDlzkvnEiROT+bRp08reNrBkW221Vcls3XXXTY79yU9+Ut3FQI349NNPk/mDDz5YVtYYG220UTJ/8cUXK5o/ly5dujT3EmjBrr766oryk046KZnvuuuuJbN77703OTb1mTCEymu+Et26dUvmm2++ecnsmmuuSY59//33y1oT9oACAAAAIDMNKAAAAACy0oACAAAAICsNKAAAAACy0oACAAAAICsNKAAAAACy0oACAAAAIKv2zb0AmkePHj2SeceOHcueu1evXmWPhbbq3nvvTeb33XdfE60EqHfSSSeVzKZMmZIc+/DDD1d5NcCECROaewll6devXzLfYostkvnYsWOrtxjanEsuuaTsvGvXrsmxPXv2TOabb755Mk/Zfvvtk/k///nPsucOIYQnn3yyZDZu3LiK5qY0e0ABAAAAkJUGFAAAAABZaUABAAAAkJUGFAAAAABZaUABAAAAkJUGFAAAAABZaUABAAAAkFVdURRFo+5YV5d7LTSh1VdfPZm/+uqryXzVVVctmT3xxBPJsTvttFMyb6saWYqNpmaXXp8+fZL5/fffn8x79eqVzN9+++2S2T777JMc+8orryRzmp6abfm22mqrZP7444+XzKZOnZoc26NHj7LWRD5qtuXr3LlzMv/oo49KZg39/6r282Np7LHHHsn8oYceaqKV1BY1Cy1LY2rWHlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBW7Zt7ATSPKVOmJPPZs2cn89RlTHfYYYey1gTN7ZhjjknmvXr1qmj+P/7xjyWzV155paK5gaW3ySabJPPllluuZHbxxRdXezkAAK2aPaAAAAAAyEoDCgAAAICsNKAAAAAAyEoDCgAAAICsNKAAAAAAyEoDCgAAAICsNKAAAAAAyKp9cy+A2lQURUU5ANS6QYMGJfOXXnqpZHbFFVdUezlAAxp6//nFF1+UzJZbbrlqL6fRXn311WT++uuvN9FKAJqXPaAAAAAAyEoDCgAAAICsNKAAAAAAyEoDCgAAAICsNKAAAAAAyEoDCgAAAICsNKAAAAAAyKquKIqiUXesq8u9FmrI5ZdfnsyPO+64sudu165d2WNbs0aWYqOp2aW35pprJvPXXnstmXfq1CmZf/DBByWzDz/8MDl2ypQpyXzXXXdN5lSfmq19X/7yl5P5Cy+8kMw/+eSTktmgQYOSYx988MFkTtNTs63fzjvvXDIbPXp0cmylz4+XXnqpZHbhhRcmxw4fPryibbdWahZalsbUrD2gAAAAAMhKAwoAAACArDSgAAAAAMhKAwoAAACArDSgAAAAAMhKAwoAAACArDSgAAAAAMiqfXMvgNp0xhlnJPP//ve/JbPvf//71V4ONImJEycm86FDhybz448/Ppl37969ZNatW7fk2N/85jfJHFhcv379Khp/7bXXlswefPDBiuYGqu+RRx4pmbVr164JVwLAktgDCgAAAICsNKAAAAAAyEoDCgAAAICsNKAAAAAAyEoDCgAAAICsNKAAAAAAyKquKIqiUXesq8u9FmjTGlmKjaZmIS81Cy2LmoWWRc1Cy9KYmrUHFAAAAABZaUABAAAAkJUGFAAAAABZaUABAAAAkJUGFAAAAABZaUABAAAAkJUGFAAAAABZaUABAAAAkJUGFAAAAABZaUABAAAAkJUGFAAAAABZaUABAAAAkJUGFAAAAABZaUABAAAAkJUGFAAAAABZ1RVFUTT3IgAAAABovewBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBWGlAAAAAAZKUBBQAAAEBW/w/XJuNz5f+NwAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 1200x500 with 10 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def plot_errors(model, loader):\n",
    "    model.eval()\n",
    "    images, labels, preds = [], [], []\n",
    "    with torch.no_grad():\n",
    "        for img, lbl in loader:\n",
    "            img, lbl = img.to(device), lbl.to(device)\n",
    "            output = model(img)\n",
    "            _, prd = torch.max(output, 1)\n",
    "\n",
    "            # Find indices where prediction is wrong\n",
    "            mask = prd != lbl\n",
    "            images.extend(img[mask].cpu())\n",
    "            labels.extend(lbl[mask].cpu())\n",
    "            preds.extend(prd[mask].cpu())\n",
    "            if len(images) > 10: break # Just get a few\n",
    "\n",
    "    plt.figure(figsize=(12, 5))\n",
    "    for i in range(min(10, len(images))):\n",
    "        plt.subplot(2, 5, i+1)\n",
    "        plt.imshow(images[i][0], cmap='gray')\n",
    "        plt.title(f\"Act: {labels[i]} | Prd: {preds[i]}\", color='red')\n",
    "        plt.axis('off')\n",
    "    plt.tight_layout()\n",
    "    plt.show()\n",
    "\n",
    "plot_errors(model, test_loader)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "J0QlVRLFIk2Q"
   },
   "source": [
    "### Feature Map Visualization\n",
    "\n",
    "Feature maps show us exactly what the convolutional filters are detecting in real-time.\n",
    "\n",
    "* If the feature maps are mostly blank (black) or look like static noise, the model's filters haven't \"activated.\"\n",
    "*  This usually indicates a **Vanishing Gradient** or a **Dead ReLU** problem. You may need to check your weight initialization or add Batch Normalization(BatchNorm fixes this by normalizing the activations of a layer so they have a mean of 0 and a standard deviation of 1 for every mini-batch.)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 135
    },
    "id": "3G_qK48xHWc0",
    "outputId": "799f7a12-683a-4663-9d75-5dd42545cea6"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA7YAAAB2CAYAAAD8+g+xAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjUsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvWftoOwAAAAlwSFlzAAAPYQAAD2EBqD+naQAAI4dJREFUeJzt3WmUXGd95/Haq7p6q967tXRrsyzZxvIm22BjhIkZsLMQMhCGAEkYMhMSQjiBzJyZkzM5vJkXsxAyJ8BMDhNmCCczw3LCEmbwBLyAY4xt2ZK8SJZkWbIsq7ulXqu79mXe3f/ze5iWZam7Srf9/bx6rv5V1aV67nPvfar+//tEm81mMwIAAAAAQEjF2v0GAAAAAAC4HExsAQAAAAChxsQWAAAAABBqTGwBAAAAAKHGxBYAAAAAEGpMbAEAAAAAocbEFgAAAAAQakxsAQAAAAChlrjYB173x38m29XuVX8vWEGtoynbL/7xH63q67/t3n8n24nl2qq+PlZW69Qh+PD//her9trXfUbHbC27ai+N11DP6Jg99q9Xd8zedZ+O2Xilsaqvj5VFG9q3D/zoX63q69/w+5+T7XhpVV8eFxLVzf1fXt1xu/U//UfZbsaaKzwSqy3a0M596ZOfXrXXftduPQZES+VVe21cWDObke0fPPdvV/X1t35ex2y8FF3hkVhtzbhuX+zch19sAQAAAAChxsQWAAAAABBqTGwBAAAAAKF20TW2qUWtBWnGyTNvlUb8tR9zOZKLFdn2a8iwdpqxix6Cr1vcK/Opda7Zn4JvjYdQek47N1qtr+0fRKCRWrsxG4lEIrUOPbfGyxyPWyW5vLafdbzgXzdxHdUqzTUcts20vnh0qbB2fwwiWqq89oMuQ6NLz62NDsbslY5fbAEAAAAAocbEFgAAAAAQakxsAQAAAAChxsQWAAAAABBqTGwBAAAAAKHGxBYAAAAAEGpMbAEAAAAAocbEFgAAAAAQakxsAQAAAAChxsQWAAAAABBqTGwBAAAAAKHGxBYAAAAAEGpMbAEAAAAAocbEFgAAAAAQakxsAQAAAAChxsQWAAAAABBqiVb/wUbc2rVsU2KpxWjQ7ny1oU+0UCRW1VBxwObnzbjG6h2X9DZxCcoD6aBdzep3Jqm89WfNi3VMlYN2PaMdmFi2zm7G9XnRurePYE1E69auZ3TM1lPWTi5FJdZIOrFlfU33ee7rRyKRSLPlR6U3rvj0QtBunDsvsejEpqBd6++UWCVnHRht6D6RKFmHRis6RmOV2qW/WbwuiaL1S7VTx2ZhzLZjFX1e3Q7jkcyM96JOV7uvH4lE5ByNtRVzhlHnGY0NHrSDbXxWD7yN3mzQzm/JSmxpk517Kz36mvW019dYE40B++AXr+6VWDln1z/+mO2YtWNux6va57HFYtCOFkqr8TZxCeI9di07MaIH1i1ds0F7JL0osUMLG4P21FK3xGZmuoJ2s+JNfmpv3AMyv9gCAAAAAEKNiS0AAAAAINSY2AIAAAAAQq3l1WzFDVYLsOGqcxKbf3A0aEe9ko7MjD0vXtKivMyczc8bCc0rL/V7eeerwXtviZL9Q9Qr+1was79f64qsa3M7rKjS779X77EPJtNXkFgpb0VdiXNJiXWftFq+eFlCkeTy6tf9xCv6mh1TVpPi1/hGmvbYWG391vtWeu3/uW/fIYk9NW31H3Mv9UksPWv7ftwr7WleoGa+moqsvguUmySWdNutDV7v9b71aecYfN0OiZ2/3im08z6/9KLt76lFPR7XkzZOEtW1HxeNtHZSYSwTtP3jkFvPHyt5O9464/7fZ2/Qfoj2WpHe+OisxM7MWG1fMaYfYOWs1Von87pTxIurX9Pl3zOjkbT30zGtfy+55LzXdV5eFivbf7DvqB5c48deCdrRDr3JSGzO6vd6jujzuvJ5e1yn1tRHU2twUI55ndSfC5oLNwxJaG6ncx3VuX7rfZe229h79Z06ZvfuPh60J7I6Zk8V+oP2c1OjEiufGAnamfPemF2Dklv/fN511s4P3S/MSSy64J1817GRAbufxVhW62j/+ciDQfvmlB704iN6zeU6ULbz2XRdJxinqwOX9D4vJN/IyPZj89uC9uFzIxJbnHGOIZXW/obKL7YAAAAAgFBjYgsAAAAACLWWJ9oNbrMUis9s/78S+1rn7UH7wOlNEovFnLSMk5omk3bSK7LTmqbScP6HtQ4vDaO8ckqLn1LcvEBqU+8xu7164rSmV8dv3By0Z67TNNv1xk3jjd6rtzP/rfFng/ZcTZcZmCxZyuPUZr2decetltdydlHXIOjK2m3si1X9bOMx7cCG04GVmqZ6dKctLW+5oilXZ5z02tzz+j1Q56T9jezZYmS9Sl9laTO/NfQTieVrvxC0fzat/dMctZTzhVlv3a24s6+U9HNtdljqUiytaa7N+gW+i/PyTps157ENHcAdp2x/6XlZ95VKlz12eeP6zmmcf9+NQXvqbl2KZ8/2E0F7clnH5ZmTlvqWPa1jJmUZjZFGPC2xmNud/jHWOxu5x1w/5pYlxLzygXranugfx+NF6/fMOk9Friftc9j+Da3jOPuHdsxLeMfKP3jTw/YaXk5vebd9flUvT/jB6Z1Buzvp1Y1cQMzrpHzV0t3iXqzg1CmcndHlUDofs/OKpCWvQ6URG0gv/zM9Rm4dts/BPw++dN7SVJvP65juf94e23PCW5+taGMl2vDS2kve2jOOnyvfSTnnae91IvWVyxsSRTcVecU/F3qdL1tqbtdR3b8nx+38urt7UmK/OfoPQfvOLZrum99rn+XzFS0XerZk16ddryMveameWTGW9nKRv3t2T9A+871xiW10pgDrPS156rnhoJ1/VdPFP5q4Jmg3vHNdacz6b+c1r0jsD8Z/FLT3pHS5vrdk7ERcbepY64rpebnhnIwLDe2/ZHTla657u54L2v+ja6/EvhWxfs+f1WPNWuMXWwAAAABAqDGxBQAAAACEGhNbAAAAAECotbzG9vw5qxOI7dS874G01XW8bdtxid3UcypoD92kt8o+V7PX/P7UmyS2VLFc8t19WpdwfFFvKe/WYS6VNQd9vMfqFm7JvSyx//VX7wjaY39xWGKpBTeXfn3X2LryS1pT+dc/vMs2vHrHiLOkRHNEa7NGB+0W6TGvXujWQdsnzhRzEhtJ6z7i1oOdL+tt0T+1wQo9vjF3q8S+vWh1AvW01gb7S1GsV5WjNr6+NHq3xH520JaISS7qB1LLWG1Ibkz7I520es7zc1p/US/YYam3V5eG8vcreV7V+57O2X7Tbh2zzxYngnb397W2p9pv9UPLG/U4sN4sj9pn1D+kfTRVsHES9eqX33StfZ4DN2k93kzZiuB6klq3tVRd+fPsS2tfx5x11dJxrf+9/4Xd9t68JWmyWTuG1J7QmrJ4Zf0uy+Xr+5/7g3Z8dFhi6R9Ybd3pTVob/6WKxcqD+nklN1hfj/Xp/jK9aPvLbEKPldn0ynWYDe8GFpOvWp/9yg0HJPanwz8O2l+Yu0liX5naF7T7n1nftfHpMRsrv7zjGYm9vduuQXIxHVPL41ajnLtF7wvhLuVxpqbjJhm18Tdf1yLXOa/o1a2ZzkR13Lo121889DaJdT5ir9PwVhcqDTljfB2XTzf3W83ipqN6Xjx//rqg/d3OMYl9O2mf5fJGHbN9u+y+Npt7tP52tmSf+WCHngf9Ovlqw7mGKmmf3zZwMmh/tv+ExN7XfSRo37PvYxJbPp4L2l1Pr+8a28y0nWtHH9VzZvyQM9+J6XVMw1mGK5rR2uYvTPxi0K6O6v6yPOoso1n27iPSqX/DPb379x5a2Gb9Xsl59zDaaseQ4X49HzQvdGOiNcYvtgAAAACAUGNiCwAAAAAItZanIqdP2M/jn1r+sMSazhIgsYLOuR8ctKUERoYWJHbniKU+ZOLeEg5OSstAUn/+X85qWlytaX8zFdPbzX9+y98G7fGEprL+dY+lIse2bJbY0uj6TmV05V60zz65rKlo6QX7PFMLmp6UH7fPqNKtqaYLXbYd91aQuL8xGLSr3hIA/vIgbvpSPaPpFB8Y3xq0/dTXWHzl1MWew/P2mt3rt58TeUspefzHuyWWdD6e9KymnlScZTuWk5pCM99n+0B8UTsr67zOXEPTJJOz+lh3NZB03vv7N1hq05+Of1di79//yaAdy2ufNwdWXspgvRk6YINq6dyAxBoXSPk7OWSf9fNj3hjZYOnHzbr2ibtyQGenpim/FOmX7XrDHlx4VY+5Eed177tjv4Su6pgO2l968j6JJecvfkmLsKu8/fqgnVjS82LUOb31HdaO7nrFPqNYWY/VFTdNf1T7JJGzPkmf031iaWjl79BjXpZyesTa3ylruvG+eyytcdxb3qLR7b7X9V32U37VTnjfLN0osW/FbNsvIejptmPd9j5dks9dQqY3rmnKvXE9RsrzMmdku+H8XjJZ1SVr7p+2fbI27Z3rndK0ZrfudxFnSbjk/PqtAYr3WQp4Y0mvV4cftM+5mfXOUdPWl9GEniOb/dYHs1u3SKyets/1fFlL8073rzw9qHqH46/u2Ri0S7fp2NueseNxsaSxbM8b57e10rDt32f26QVr6kYreYvVdMwmC7bdfUovglPHzwbtxKtTEss5S2vV53TOlE1611FRZ4m8zRsklijYdUEzrufz+Wm7zp/arPtkw+3qVGtLgN44exUAAAAAYF1iYgsAAAAACDUmtgAAAACAUGt5jW3CKXFKnHodtRKTlr+dj2ku9/eytpSBf5v4xLLlhL/QNy6xWG3l21HXRzWX/YGN24L24aLmoOeOW/54eULrxJZH1289iC+5aMVSucWVl3fw5Y5aPU0zrt+1VHotUf/nag/mrY8KG7Wm16/jbSasr2ev1p2k2mN/44M3PSGxLz93R9DuP+LVqS04t6dfxzW27tdf7njy1b2VeOIV57FefXS84NR/dGi/urXUyRk9RCUKK//90oC+zr07bNmLuLdGxMBBp6akrvUfi1u9g8g6lpqzA3L/3MXXn7qLgTSTeowr99lYiNX1c3dXAKh2a/20v4yHu+zAxIFXJPbSb9tyTZ8Yekhinzn5a0E7d0z7Nuq8H79eaL3Jb3KLnLS2zS29rHV4dVNX2fnVX/ohM2vFuTWvzC87ZZ91vKLPS895nevuB5369/uft9dZ3Krj/4EFq/H375mROe2O23W8JkzEOw4vX/w9AfIRO0g/HdOa+ie7bOm2n/v4nHNA06+X85bbcuvfU1Pe/RPOWqxDy28jjZucZU282uDKrFfUuU5Fs849RbLeCbVq1zRy7RGJRCJp5/qjrveHiTi1l53e86RWd3ZeYp29enxupu0Ycu7NgxKLOLvENw5pXfymUVtiqHlKa0u7Tr9x7nlQz9qHVBjXMbRyBbuavl2vTeIFuz+Mf68Cd34T9XaJuPexN5xhWrpG6+sbVRuLHSf0POIegpOLeu1ey9rzai2+pOIXWwAAAABAqDGxBQAAAACEWstTkVdD1MuESS5dXEpZdvLC83h3yZi921+SWMVZP+abD98usR3H7fd4PyW2ppt4DX5aaHq2vMIjI5FIzPo9e7a48uMikUh+wjqiNKRpTvfcfihoN5q6j8QPWQpUx/5jEquPD0dwkbwh6maa+enFNWccJor+E3XTTW3dcfPLEvvowCNB+/0/+x2JbT9ot7+vbNLygeLg+k5RXW3RquY5ZaYvLrEqff7C8djLtkxE+TpdRi2z15a3OFLRZSqee3pL0N7+iuZcNaP07etRT+vntTxm58G4l/rmpjT76c0+dwmLalYfm8rb/lS9UfelX8wdCNqfeOKDEus+u77Tj1ebfx2VWLzY3zku/Di3BKVjUvvELU3wz8ONRUunjS5reUOiyri9aHH97KKdnSs8MBKJFpzjY8ZLaS/rAK+NWO54cUj7o2PYxulYblFip6Ys5T13XP9E6rQuOYULc5dEjUQikVp3fYVHvj6ZQbt+HurUa+lzM91Bu9qlf79+gQq8eldrl/hx8YstAAAAACDUmNgCAAAAAEKNiS0AAAAAINRCWWO7WhreSjy1HZZbflffUYntz9vyEoP7tb6gkbIXKgx73xVQGtIW1R69v3h+s/VL4ypdJuKOHqud/Q9H7pHY0EFn2aChPon5y5xg7fm3rS9utSWY3jv2tMTmG7ZcQtfDWmcUm7Ia+vmbtkqs+YY+KrZPbMGrzW1ajc6pd2kxz9M3fTFof/DF90hs7CdWBxQr6BJdfJXbHv7yP25dbe9J7aMzd9kAfN/On0nsYMmW7Iu/4I3pGjW27eAfk1O2uksk4d36Ij/h3Bdj64LEEjFnmafFnD6Rrm25ZpfeIGZut20vb9HlFJM1O7Cm4xprzNu1WN8Rb52Zqj4WrRHt0EG7oc/G4tl5XeYp9aJdR6Xm9XVKg86SPp3e0n7x9g1aTvMAAAAAgFBjYgsAAAAACLU3dNJdJac/lf/KroNBOxPT9KiH/v6GoD2yqD/jz11tP9VXu8g9vhIUhnTXLmyyNKcPXfOkxI6WxoJ2+ZmcxLI/dNJbt41H0F7Vbh2zd1xraeT7sroc07u+/emgvfMxXYKgsMeWjykNMGavBNGiLu01//YdQXvvnUck9mwlGbSfe2qLxLa/xtJfaD1/maVE0cZxcVCP1fUNth+MpTRd9UuH3xq0M6+xXBRaI1b3jp/OzyXFYY1Vd1m5wUCHpqWeOW3LwiTLHJPbrTrUJdvLG5w08mEt5+rM2NJAR57Xpdn6DtkOkZzOr+ZbxCVKZXUpp+m89XVxWlPQu9wKIS+7ODNj+8RS55VTL8AvtgAAAACAUGNiCwAAAAAINSa2AAAAAIBQe0PX2DY3ay3Wbw48GrT/fPIXJDbyuNXVphb1FuULW+1jbLICTFs0Y1qTkx/X72xy260gK+mtT/DVJ98ctK/+ttZiRsc3Bu16T+ay3ydeJ69so9qrfferg08F7YcKV0lsg7PsS/y81urN7usN2nW6tS2idW95AG95ianbbEx/ZdPfSezXD/zToD1+v+4TsaJzfOar2/bwa7HmtI9S81bj9dJ7dAB+8uYHgvbhwpjEEo/aUhSJwpVT0/VG1kh4Szk59xmp9DYklkzafjA1q8uKJGacy1F9GlohpgfLSk6XTKz2WD8nGvrYuLNUU3JWY8M/tfWfokvekm5oDb8MPqZjdmnSamy7j3vTQue55QF9nrvETyNz5QxaTvsAAAAAgFBjYgsAAAAACDUmtgAAAACAUHtD1djW07p9y8TLsj1Z6w7aj/zoTRKbOG/1uAvbOyTW0FIEtMHyRq3TcutBIpFI5Opeq/N4bHarxHqetQ6MHTsqsfquidV6i7gEUa9sY3B8Xrbf2nE2aN/2fz4lsd2HrK56+Xqt1at2rsrbw2WInp6U7eLe7bJ9zd6TQfuJkq4hXX68P2hnJucl1oyzBma7pZb1+NtIaZ8UR+1knBjXNTGHErbW5Z8fuEdiAwvU1bZbvKJ9mZrXuHsLi+JGPYCnotZ/9Tm9IEvQtW1V29gv24tbdHpQ67F7F3Qk9T4zk6dsDeKRI9qRsVnWrm23zIDeTyjt9V+5aL9x+vcJcu9BEm3o2K936L0TrhT8YgsAAAAACDUmtgAAAACAUFv3qcgN52f18k79OX5b9rxsf/b4LwXt4Sc1haaRsu8AioOkul0Jyn2WyrS0QfMnqiMV2b6z/8Wg/YUDb5PYzgfnbWN0SP9IjL5uNTeVrbBd+/F3t/5Mtr+2YCUDow/qPhAtloP27K6k90cu803iksRnLC2t2dCUtTN36eno6+PfCdofevqjEht9wvaLZpTOvBIkStafhSH9zjx7Ts+n89st/ks7npXYQ/O7gnbPczpuo03yVdsh5mQudkxqH2TPa9/OXm3H4XS/XnNVytafibz3uwpd23LNnJXfnd+j9Tnz12m6aiRtJ+Z6XfsuM2nH7r6Ds94foWPbIdZVDdpbB2ckNpnv1sc65QUN71KpnnLKB7JeX16hp15+sQUAAAAAhBoTWwAAAABAqDGxBQAAAACE2rqrsW16Od/FMasL+LVrD0jsZGFAtud/PBq0N8xqbcjihN3zurnuPrVwaCT0e5iFbVYMUBjT3P/fu/VB2T68bMu99DymyzVFT1j9bf2aLZf7NvF6eWUblZz9w2/crDW1E6lzsv1H3/lI0N7p1fa4S/zUspf7JnEpolVdDqB+dipoz37gJon943f/g2zvLzlLbT3eK7H0lC3f1Uzw/Ww7+EtxNZ1uaHjnyGpWT8zpN1vN1w2duuzenzz03qA9uEh9XjtEvY89sWz91zGjY7rYr+OvtLNkj43rTlKZdpbW8/YftEBC70OxuDsXtOd3e7XTQ7oMl7tEzNwrejwePWqdGZtbvNx3iUsR0/7bODwftBfLuhzm/JzWU2eWbHw3vdOpu8RPPXtlLu/j44oAAAAAABBqTGwBAAAAAKG27pJqq936c/ytNx8L2os1/Tn+yZfHZXvsGUu1qHbpR8MSP+3hLuUxe42mENecbIo79+mSEZuTmpb6xf3vCNq7HvJuR+8v8YO111yhHYlEbnzL0aB9Z9dRif37k/9Itjc94KTG1L1lJ/wlftAaTjc0Dh2RUO3um4N294fPSOz3Bh6V7X2PfCJoTzxellgzxney7ZZe0PE2c52lOQ7vr0rs5Q9oCtvHthwI2t87v0diPc+745ZU5HaIF/R6p++o9V9Ts1kjszdq3/b1WQprfknP2Yklxm3LOddQhd2jEjp3g/VHfKQgse4OPeYuly2NvOOMXh/3HdTlZNB6Gya0DzoSdgx+ebZPYukXdS6UdLLOKz36urWu8NUMcJQBAAAAAIQaE1sAAAAAQKgxsQUAAAAAhNq6qLFtODUfXXs0z/zegUNB+7+cvEti3Q/pGiC1DqvnKXdrjQlL/LRHI2OdWxzWPkncYkt+fGLkRxL79LH3y/boT5zvcE5pbV/92q2X+zZxGRp78rL9iQ3Wlz9YuF5i81/fKNtDU7a0wML1unwXS/y0R6xUCdrRic0Se/E+q598YOffSOzPzunxOfeg1eelZuYlxhI/7RGz21BElke1D4afcmq67tPYyJDe16A/sRS0n3hip8by1NW2g7t8U1zLKyPFAevPvHe6HN0yLdsxZ62ghWmt7YuHr1wv9JoZq43Nb/YuZK+y4sqBHl3ep1LTYuriCSu+HDusddXReT2Ho0WSNqD8uwCdmrGxVzmry/vkzukxttax8nI/9Wz4Bi1XBwAAAACAUGNiCwAAAAAItVAm2Da939wrffaz+ke2PC2xqpNDPPcTvdV5b15/Yq9m7YUrOZb3aYdGWtNfZq9OB+3SkPbXx7Y9FbT/8/TbJTb98AbZ3vKDw0G7uVXTWdECXjZLPWNj9h1bj0nsLudO9L/7lTsltukZTZcqD1q66sI2bx0KtESsqEu71A9bf07+4Vsk9t9/9S+C9oHysMS+//3bZHvC6WuW92mPqGYcRuJlG7eJoqazLTppjpkRTU380MTjsv35Z+8O2j1H/b4lFbkVot7HnJq3a57stAbLzvVQ/01TEhvrXJTtp49NBO1kieuolkvpMneLe+w4u7RJ+6OybGnKmf4FiU2e0tKegSP23O5nNP0cLZLQcbnNKQM4v6TpxpEjXUGz/7SG6kndDxrOLlPJhS/12MfVAgAAAAAg1JjYAgAAAABCjYktAAAAACDUQlljW+3SPPOr9p4K2uerXRL7y4NWo7f5gNaCVTu1Jq/gLCfTpFyvLZY2pGW73G998pa9hyV2W+fxoP3lh/dJbPff6JI+kd7uoFnPpiJoraaW/US232ljdl+v9uu+Z98TtDc8UpJYtKljf3aX9WWDbm2Pc3OyGb3luqBduK0gsaF4MWj/zlMfkdjoT2uyHSvZdjNOrV47xCs63ha223fhI4/rmjBn99nlxMd3/VRi35vUZbuSj9vxOFGiprYdYlUdUx3nVu6H/BaruyvldR21ycmcbCfOOQf78JfrhU51LCfbs9fYxWxqjx6rO+NWRH/6nC7N1HVcpweDT1oNbrSg52W0RiZ38Z973blXSaXHW77U+0mzPGBjv5kK/6DlF1sAAAAAQKgxsQUAAAAAhFpoUpEbTmpwY3tRYu8ZPRC0z1ZyEuvab8uBdPz4GYlV7r1W/wapjG1R6bUPfmmjftdS27MUtL+25SGJXfPoh4L24H59XtO75X29T9OnsPbcpUKKExWJvX/syaAd95b3ePWpsaC94+nnJbbwzt2yXe2OoA3ic86yS95Ye/F91ikfv/5+iX1++h1BO7q/R2KpBV3KifTj9nDTj2sZ7YPBQ5YePnWrlo38t3d/MWgXGhr7y8ffKdu5WdKP28E9Jqfm/aA1CyPa78m8bTeOa7lX0h+mdG3LNXtsqZe5q/Vap7jBxuzdo7ruy3sH7Tz8ycf+icQGntPSvfi0pjGjNaJZ67+RXl1G7dW53qBdXtYJTMKp7Mnv1uuvaEFrLZuJ9TVo+cUWAAAAABBqTGwBAAAAAKHGxBYAAAAAEGqhqbGt5CwH/N07n5PYaGI+aH978gaJJfP2vEZe89P9+iG0R6nfdsPlibrE/s2eHwTtelNvQ16ct/uZDyxqrDbQKdvRxvqqIQiDao995nt3vSSxvRlb7udfnnyvxOJFG5f1+QWJVbq8McsQbo+6jdOFN2+W0NgNk0F7R3pSYj88tytoZ2a8MckQvSI0nNrmtHdcrXbZd+Hp22ckdpezvMR/XdClQ1KLFGJeCWJ1p1bWW4KtOGSx4rD2e2LZ+Q3E70q6tu1q/Xa9UxjVDhmZmA3aHx3+scTq7m9bXr+l53Q5L7RHIm3FsueX9Lq2tGR1tdFlnc5FnSEcy+h1daPhDdr6+rqQ4hdbAAAAAECoMbEFAAAAAIRaaFKRa0N26/EP9j8msWzMYi+8MiKxQecu17W7b5ZYtXN9/fweVuVe64eJXWclti01HbQ/dvptEot3WHpFZromsVhZt5tJvb051l6t23Jh9uZOrvi4w6+MyvbAKcuJSmzaKLG6riKCNqluGgja5/bo96MfHjsctH+5syCxP1mw5QkGT+gSBLGSjlm+dm2PzIIdV7ue13Tj4789HLTv3fiixD43uy1oT1V1KSfSU68Mtax1RHlA043dPkoMlSS0edjSWWcLHRJbONov27Ey11WtFq07fel9/Ft6re+eKm6V2LfO3Bi0m7N6co3WliJov2rB0o3rkzply8zaSbL7lB5k605pyHIjI7FKv479Znx9HaC5dAAAAAAAhBoTWwAAAABAqDGxBQAAAACEWmhqbKMxywGve0UEk7XuoN0oay1lfsIeW+7VGoJGKoIrQMpZkunMTK/Evtm3N2gfmNJ6y/6/t7qB2CM/lVjs2qtlu06Nbcu5dRvZmNZT/tXMHUG772Gt/+g8azXzc3fqUjLVbuq3rgSzu63ObuRWXdIn7dzzYO9T75dY/Ie2DExyiRquK1Ey7ywN0dTaqy5bpSvynf03Suwjtz0atL++/xaJ5RZX7/3h0tVyVsd+x3XHJPbZjX8XtLcnuyT26bM3Be2/PbJXYokqx+R2i79o9yYZLw5K7GDSllh7fNsWiaVO2rl35IjWXcYWi6v4DnGp0qdtojL6WFVinc+fCdqNXl0KKL/TrqXrab3+rfbomKXGFgAAAACAKwgTWwAAAABAqF10KnLDe2S09v9/3FpJnbI04o8f+g2J7eg/bxs1nat3TNtP7LUOL2XG+/W91f+nixWvrG2qTzPmpyW09vuOuJtdcULTKX50yNKeJr41LbH6C0768e3XS6waktTjaOO1H3Op/DHbatmX7Q187v77JNZM2eDrGNT9L1Gw5+UndF9shqNb1/wrw2i5rv/Q4q8oM3O24549oMs1ffHEUNAefFw7bOTQgm34+35IvmZtZNZ2J2wkve0Wj+PkrC31Uj92QmJd11qaY31YU8m/+lMrL8g9o286VtWTbbuPTSupp9f2XJtY1tePV1d44Bppxu2Df2Z6TGK/X/r1oP3CSY1lTlo6ZKeuBBQaa7nP1Xq0nCZZb21qZ3TWjqv1g4cltrW0I2hPv3VIYrGavc/cYa9eIKkfWDPXHbkiNdf2s451tHdiUKnbMWNxQvskNW9LbVVyWls5c62dp8pbyxKLxr2088t+l2ujUbm0c+2V+v8BAAAAAOCiMLEFAAAAAIQaE1sAAAAAQKhFm801TlAHAAAAAGAN8YstAAAAACDUmNgCAAAAAEKNiS0AAAAAINSY2AIAAAAAQo2JLQAAAAAg1JjYAgAAAABCjYktAAAAACDUmNgCAAAAAEKNiS0AAAAAINT+H2yLuUt5UxLjAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 1200x400 with 8 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def visualize_features(model, image):\n",
    "    model.eval()\n",
    "    with torch.no_grad():\n",
    "        # Pass the image through just the first conv layer\n",
    "        feature_maps = model.features[0](image.to(device))\n",
    "\n",
    "    plt.figure(figsize=(12, 4))\n",
    "    for i in range(feature_maps.shape[1]): # Loop through filters\n",
    "        plt.subplot(1, 8, i+1)\n",
    "        plt.imshow(feature_maps[0, i].cpu(), cmap='viridis')\n",
    "        plt.axis('off')\n",
    "    plt.show()\n",
    "\n",
    "# To call it using one image from the loader:\n",
    "sample_img, _ = next(iter(test_loader))\n",
    "visualize_features(model, sample_img[:1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 16. From MNIST to Anemoi\n",
    "\n",
    "The debugging procedure of this session also applies to Anemoi. The table shows where you find each problem in an Anemoi training.\n",
    "\n",
    "| Problem in this session | Equivalent in Anemoi |\n",
    "|---|---|\n",
    "| Shape mismatch between layers | The number of variables or grid points in the dataset does not agree with the model or the graph. |\n",
    "| Learning rate too high | The loss increases or becomes `NaN`. Anemoi sets the learning rate in the config at `training.lr.rate`. |\n",
    "| No normalization | Anemoi normalizes each variable with statistics from the dataset (for example, `mean-std`). A bad normalizer config gives very large losses. |\n",
    "| Overfit 10 samples | Train on a small number of batches (for example, with `dataloader.limit_batches`). Make sure that the loss decreases. |"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python (ML course)",
   "language": "python",
   "name": "ml-course"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
