main.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "main.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/EmperorTransisthor/DDoS_DetectionModel/blob/main/main.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gg4EQiMhq1Hp"
      },
      "source": [
        "# **Begin**\n",
        "In this section, we import libraries to our project, define datatypes, and download + unzip csv dataset"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "o3azcgYo597c"
      },
      "source": [
        "# here we write all libraries which we are using in project\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "import sklearn\n",
        "import gc\n",
        "import csv\n",
        "import os, glob\n",
        "from sklearn.utils import shuffle\n",
        "from sklearn.neighbors import KNeighborsClassifier\n",
        "from sklearn import linear_model, preprocessing\n",
        "from sklearn.metrics import confusion_matrix\n",
        "from sklearn.metrics import plot_confusion_matrix\n",
        "from sklearn.metrics import classification_report\n",
        "from sklearn.metrics import log_loss\n",
        "from sklearn.metrics import precision_score\n",
        "from sklearn.metrics import recall_score\n",
        "import pickle #for saving our model\n",
        "from sklearn.model_selection import StratifiedShuffleSplit\n",
        "\n",
        "#test\n",
        "from sklearn.decomposition import PCA\n",
        "import seaborn as sns\n",
        "from tqdm import tqdm"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zh9H6TuvNFwb"
      },
      "source": [
        "First of all, we have to define datatypes of our dataset"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3-ME1JDX5-ol"
      },
      "source": [
        "dtypes = {\n",
        "    'Src IP': 'category',\n",
        "    'Src Port': 'uint16',\n",
        "    'Dst IP': 'category',\n",
        "    'Dst Port': 'uint16',\n",
        "    'Protocol': 'category',\n",
        "    'Flow Duration': 'uint32',\n",
        "    'Tot Fwd Pkts': 'uint32',\n",
        "    'Tot Bwd Pkts': 'uint32',\n",
        "    'TotLen Fwd Pkts': 'float32',\n",
        "    'TotLen Bwd Pkts': 'float32',\n",
        "    'Fwd Pkt Len Max': 'float32',\n",
        "    'Fwd Pkt Len Min': 'float32',\n",
        "    'Fwd Pkt Len Mean': 'float32',\n",
        "    'Fwd Pkt Len Std': 'float32',\n",
        "    'Bwd Pkt Len Max': 'float32',\n",
        "    'Bwd Pkt Len Min': 'float32',\n",
        "    'Bwd Pkt Len Mean': 'float32',\n",
        "    'Bwd Pkt Len Std': 'float32',\n",
        "    'Flow Byts/s': 'float32',\n",
        "    'Flow Pkts/s': 'float32',\n",
        "    'Flow IAT Mean': 'float32',\n",
        "    'Flow IAT Std': 'float32',\n",
        "    'Flow IAT Max': 'float32',\n",
        "    'Flow IAT Min': 'float32',\n",
        "    'Fwd IAT Tot': 'float32',\n",
        "    'Fwd IAT Mean': 'float32',\n",
        "    'Fwd IAT Std': 'float32',\n",
        "    'Fwd IAT Max': 'float32',\n",
        "    'Fwd IAT Min': 'float32',\n",
        "    'Bwd IAT Tot': 'float32',\n",
        "    'Bwd IAT Mean': 'float32',\n",
        "    'Bwd IAT Std': 'float32',\n",
        "    'Bwd IAT Max': 'float32',\n",
        "    'Bwd IAT Min': 'float32',\n",
        "    'Fwd PSH Flags': 'category',\n",
        "    'Bwd PSH Flags': 'category',\n",
        "    'Fwd URG Flags': 'category',\n",
        "    'Bwd URG Flags': 'category',\n",
        "    'Fwd Header Len': 'uint32',\n",
        "    'Bwd Header Len': 'uint32',\n",
        "    'Fwd Pkts/s': 'float32',\n",
        "    'Bwd Pkts/s': 'float32',\n",
        "    'Pkt Len Min': 'float32',\n",
        "    'Pkt Len Max': 'float32',\n",
        "    'Pkt Len Mean': 'float32',\n",
        "    'Pkt Len Std': 'float32',\n",
        "    'Pkt Len Var': 'float32',\n",
        "    'FIN Flag Cnt': 'category',\n",
        "    'SYN Flag Cnt': 'category',\n",
        "    'RST Flag Cnt': 'category',\n",
        "    'PSH Flag Cnt': 'category',\n",
        "    'ACK Flag Cnt': 'category',\n",
        "    'URG Flag Cnt': 'category',\n",
        "    'CWE Flag Count': 'category',\n",
        "    'ECE Flag Cnt': 'category',\n",
        "    'Down/Up Ratio': 'float32',\n",
        "    'Pkt Size Avg': 'float32',\n",
        "    'Fwd Seg Size Avg': 'float32',\n",
        "    'Bwd Seg Size Avg': 'float32',\n",
        "    'Fwd Byts/b Avg': 'uint32',\n",
        "    'Fwd Pkts/b Avg': 'uint32',\n",
        "    'Fwd Blk Rate Avg': 'uint32',\n",
        "    'Bwd Byts/b Avg': 'uint32',\n",
        "    'Bwd Pkts/b Avg': 'uint32',\n",
        "    'Bwd Blk Rate Avg': 'uint32',\n",
        "    'Subflow Fwd Pkts': 'uint32',\n",
        "    'Subflow Fwd Byts': 'uint32',\n",
        "    'Subflow Bwd Pkts': 'uint32',\n",
        "    'Subflow Bwd Byts': 'uint32',\n",
        "    'Init Fwd Win Byts': 'uint32',\n",
        "    'Init Bwd Win Byts': 'uint32',\n",
        "    'Fwd Act Data Pkts': 'uint32',\n",
        "    'Fwd Seg Size Min': 'uint32',\n",
        "    'Active Mean': 'float32',\n",
        "    'Active Std': 'float32',\n",
        "    'Active Max': 'float32',\n",
        "    'Active Min': 'float32',\n",
        "    'Idle Mean': 'float32',\n",
        "    'Idle Std': 'float32',\n",
        "    'Idle Max': 'float32',\n",
        "    'Idle Min': 'float32',\n",
        "    'Label': 'category'\n",
        "}"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yN5Z2xecNV3A"
      },
      "source": [
        "Importing dataset from kaggle"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fwVD3rIgI5y4",
        "outputId": "008d5017-397f-4238-eba2-4666d33d6743"
      },
      "source": [
        "# importing dataset from kaggle\n",
        "os.system('chmod 600 /content/kaggle.json')               # Making kaggle.json invisible to other users of colab\n",
        "os.environ['KAGGLE_CONFIG_DIR'] =  '/content'             # setting path to folder by using environmental variable\n",
        "!kaggle datasets download -d devendra416/ddos-datasets    # here we download datasets"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading ddos-datasets.zip to /content\n",
            "100% 2.68G/2.68G [00:26<00:00, 57.5MB/s]\n",
            "100% 2.68G/2.68G [00:26<00:00, 109MB/s] \n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "j8Dk5onf1Z2Y",
        "outputId": "d2eec37b-f64c-44d9-a2ad-0398e3238ab1"
      },
      "source": [
        "!unzip \\*.zip && rm  *.zip                                # we are using these commands to unzip and then remove zip files"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Archive:  ddos-datasets.zip\n",
            "  inflating: ddos_balanced/final_dataset.csv  \n",
            "  inflating: ddos_imbalanced/unbalaced_20_80_dataset.csv  \n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N61_bfHegFh5"
      },
      "source": [
        "Here we load our dataset and assign datatypes. Since, we don't have lots of RAM, we have to set low_memory option to TRUE"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5zWjsXImZCzd",
        "outputId": "1d78c0e7-ea12-440e-ef14-0eee37858efb"
      },
      "source": [
        "#######   using panda to view data in our datasets\n",
        "df_combined = pd.read_csv('/content/ddos_balanced/final_dataset.csv', dtype=dtypes,low_memory=True)\n",
        "len(df_combined)\n",
        "\n",
        "#df_mem_prev = df_combined.memory_usage().sum() / 1024**2"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "12794627"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IF3svHI8SN6P"
      },
      "source": [
        "# **Data separation, in case we run out of RAM**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lLkUxtR8ZRUK"
      },
      "source": [
        "# SKIP\n",
        "# here we take 1/2 of our dataset from head and from tail\n",
        "df_combined.head(3198656).to_csv('dataset_ddos.csv')\n",
        "df_combined.tail(3198656).to_csv('dataset_benign.csv')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dmJaQI1umYQw"
      },
      "source": [
        "We delete full dataset, which is no longer needed and will only fill memory"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MEVzmB7mN2Wj",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "389daace-3963-4396-f565-b309327672c2"
      },
      "source": [
        "del df_combined\n",
        "gc.collect()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "50"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7jRJ8kIXOKVC"
      },
      "source": [
        "Here we merge it"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "O_16a6puZX4K"
      },
      "source": [
        "#find all csv files in the folder\n",
        "#use glob pattern matching -> extension = 'csv'\n",
        "#save result in list -> all_filenames\n",
        "extension = 'csv'\n",
        "all_filenames = [i for i in glob.glob('*.{}'.format(extension))]\n",
        "#print(all_filenames)\n",
        "\n",
        "#combine all files in the list\n",
        "combined_csv = pd.concat([pd.read_csv(f) for f in all_filenames ])\n",
        "#export to csv\n",
        "combined_csv.to_csv( \"combined_csv.csv\", index=False, encoding='utf-8-sig')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "es8kbGcBz_Z4",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "48627a91-8fd4-4f12-aae2-9cbef228363e"
      },
      "source": [
        "del combined_csv\n",
        "del all_filenames\n",
        "del extension\n",
        "gc.collect()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "50"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FQ0TLzSMOk2s"
      },
      "source": [
        "We are loading our cut dataset"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1wNFL9GH5Nv1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "e2e9ea85-7d74-423e-c8cf-e52d1ce1d37f"
      },
      "source": [
        "df_combined = pd.read_csv('/content/combined_csv.csv', dtype=dtypes,low_memory=True) \n",
        "df_mem_post = df_combined.memory_usage().sum() / 1024**2\n",
        "#print('final_dataset is using', df_mem_prev, ' MBs of RAM')\n",
        "#print('difference = ', df_mem_prev - df_mem_post)\n",
        "data = df_combined[['Flow Duration', 'Label']]\n",
        "data['Label'].value_counts() "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Benign    3198656\n",
              "ddos      3198656\n",
              "Name: Label, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KButtub5VkYK"
      },
      "source": [
        "**End of data separation**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9fcApEdgVs-w"
      },
      "source": [
        "# **Data processing**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lm5bh2WAWAUf"
      },
      "source": [
        "First we verify our data\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NfuBGp43U6HI",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d2324495-9feb-49e3-9c79-bb377205b6c6"
      },
      "source": [
        "df_combined.isna().sum().sum"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<bound method Series.sum of Unnamed: 0      0\n",
              "Unnamed: 0.1    0\n",
              "Flow ID         0\n",
              "Src IP          0\n",
              "Src Port        0\n",
              "               ..\n",
              "Idle Mean       0\n",
              "Idle Std        0\n",
              "Idle Max        0\n",
              "Idle Min        0\n",
              "Label           0\n",
              "Length: 86, dtype: int64>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WuHlzfqgnNfO",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5218ab7d-5c63-4992-fa08-5fa7be8b47bc"
      },
      "source": [
        "data = df_combined[['Flow Duration', 'Label']]\n",
        "data['Label'].value_counts() "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Benign    3198656\n",
              "ddos      3198656\n",
              "Name: Label, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wD5pgGIf9GkP",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "94b8d103-6ba9-4350-b832-4d9208760504"
      },
      "source": [
        "del data\n",
        "gc.collect()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "150"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RE9GH1n2O0WY"
      },
      "source": [
        "Dropping rows with values, which aren't unique"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HkxSd6Mioj_c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5dccc45c-3265-42ac-d310-40e6886f79f0"
      },
      "source": [
        "# dropping collums with one value\n",
        "colsToDrop = np.array(['Fwd Byts/b Avg', 'Fwd Pkts/b Avg', 'Fwd Blk Rate Avg', 'Bwd Byts/b Avg', 'Bwd Pkts/b Avg', 'Bwd Blk Rate Avg'])\n",
        "gc.collect()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "50"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9YcqZPLRoweO",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "73c038a2-a34f-45cf-c44a-a91e76cf0c4d"
      },
      "source": [
        "# counting unique values and checking for skewness in the data\n",
        "rowbuilder = lambda col: {'col': col, 'unique_values': df_combined[col].nunique(), 'most_frequent_value': df_combined[col].value_counts().index[0],'frequency': df_combined[col].value_counts(normalize=True).values[0]}\n",
        "frequency = [rowbuilder(col) for col in df_combined.select_dtypes(include=['category']).columns]\n",
        "skewed = pd.DataFrame(frequency)\n",
        "skewed = skewed[skewed['frequency'] >= 0.95]\n",
        "colsToDrop = np.union1d(colsToDrop, skewed['col'].values)\n",
        "colsToDrop\n",
        "del skewed\n",
        "del rowbuilder\n",
        "del frequency\n",
        "gc.collect()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WwlyMzYkPJxP"
      },
      "source": [
        "Dropping null values"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "REdnUx_Towx8",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "04d8c783-bd20-4819-f0e8-aac8cfefbfc2"
      },
      "source": [
        "missing = df_combined.isna().sum()\n",
        "missing = pd.DataFrame({'count': missing, '% of total': missing/len(df_combined)*100}, index=df_combined.columns)\n",
        "colsToDrop = np.union1d(colsToDrop, missing[missing['% of total'] >= 50].index.values)\n",
        "dropnaCols = missing[(missing['% of total'] > 0) & (missing['% of total'] <= 5)].index.values\n",
        "gc.collect()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "50"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7ncEF4zFPpgT"
      },
      "source": [
        "Replacing faulty data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ICHqixVvpEf_",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "73a0b8bd-430a-416c-cda1-08f01639afe9"
      },
      "source": [
        "df_combined['Flow Byts/s'].replace(np.inf, np.nan)\n",
        "df_combined['Flow Pkts/s'].replace(np.inf, np.nan)\n",
        "dropnaCols = np.union1d(dropnaCols, ['Flow Byts/s', 'Flow Pkts/s'])\n",
        "gc.collect()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "50"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YrHeYszZpJ3w",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "53561cf8-ea1b-4bdb-8910-6c0869e5e416"
      },
      "source": [
        "# perform actual drop\n",
        "df_combined.drop(columns=colsToDrop)\n",
        "gc.collect()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "53"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LpJzix2fP8ds"
      },
      "source": [
        "Regaining RAM in order not to drop the session"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "h4lHpr2-_DIi",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f52fc627-c1ef-46d1-a9d7-09dc2e6362c2"
      },
      "source": [
        "del colsToDrop, missing\n",
        "gc.collect()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "50"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zWDvVEUI--L5",
        "outputId": "0f94d95f-31b3-4ba8-bcf5-e5a8482d3df8"
      },
      "source": [
        "df_combined.dropna(subset=dropnaCols)\n",
        "gc.collect()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "63"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-cKdALBU_f4E",
        "outputId": "8cbebb48-01be-46d9-ec43-ec903a0306ec"
      },
      "source": [
        "del dropnaCols\n",
        "gc.collect()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "50"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "sn5wn0WtwDiG",
        "outputId": "7d8b75bd-fc28-40f1-8798-c10b0e319da2"
      },
      "source": [
        "data = df_combined[['Flow Duration', 'Label']]\n",
        "data['Label'].value_counts() "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Benign    3198656\n",
              "ddos      3198656\n",
              "Name: Label, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ACWi_JtQj9IC",
        "outputId": "59a56904-8583-42b2-8be6-762de5f797ab"
      },
      "source": [
        "del data\n",
        "gc.collect()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "100"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7X4fWrFiQLKE"
      },
      "source": [
        "Removing rows with negative values"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iSdHFMT_pPSn",
        "outputId": "91994082-1574-4099-9623-54b46e8703ac"
      },
      "source": [
        "negValCols = ['Flow Pkts/s', 'Flow IAT Mean', 'Flow IAT Max', 'Flow IAT Min', 'Bwd IAT Tot', 'Bwd IAT Mean', 'Bwd IAT Max', 'Bwd IAT Min']\n",
        "for col in negValCols:\n",
        "    df_combined = df_combined[df_combined[col] >= 0]\n",
        "gc.collect()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "68"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FZTqqG3q_24a",
        "outputId": "d97a0109-01d1-4aa1-b3c1-79bd8c510eed"
      },
      "source": [
        "data = df_combined[['Flow Duration', 'Label']]\n",
        "data['Label'].value_counts() "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "ddos      3198656\n",
              "Benign    3198454\n",
              "Name: Label, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9Qtb0HMF_4-s",
        "outputId": "1e195b6e-1c95-43f5-a4e3-b4bb00779efc"
      },
      "source": [
        "del data, negValCols\n",
        "gc.collect()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "150"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KBjMoLE5RXT_"
      },
      "source": [
        "Now, we have prepared dataset to train our model, but before we do anything, we have to shuffle dataset in order to prevent it from overfitting"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2MhODC7OAnsT"
      },
      "source": [
        "split = StratifiedShuffleSplit()\n",
        "for train_index, test_index in split.split(df_combined, df_combined[\"Label\"]):\n",
        "    traindf = df_combined.iloc[train_index]\n",
        "\n",
        "del df_combined, split\n",
        "gc.collect();"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Hy2aifqc1Llb"
      },
      "source": [
        "# **AI TRAINING**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8FGB_T9vQ1Lf"
      },
      "source": [
        "Before we can train our AI, we have to change format of our data to numerical type"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "z7slXoCEAx05"
      },
      "source": [
        "le = preprocessing.LabelEncoder()\n",
        "\n",
        "timestamp = le.fit_transform(list(traindf[\"Timestamp\"]))\n",
        "fl_dur = le.fit_transform(list(traindf[\"Flow Duration\"]))\n",
        "pk_size_avrg = le.fit_transform(list(traindf[\"Pkt Size Avg\"]))\n",
        "fwd_pkts = le.fit_transform(list(traindf[\"Fwd Pkts/s\"]))\n",
        "bwd_pkts = le.fit_transform(list(traindf[\"Bwd Pkts/s\"]))\n",
        "flow_byts = le.fit_transform(list(traindf[\"Flow Byts/s\"]))\n",
        "flow_pkts = le.fit_transform(list(traindf[\"Flow Pkts/s\"]))\n",
        "Label = le.fit_transform(list(traindf['Label']))\n",
        "\n",
        "#X - data on which machine is learning\n",
        "X = list(zip(timestamp, fl_dur, pk_size_avrg, fwd_pkts, bwd_pkts, flow_byts, flow_pkts)) \n",
        "#Y - data which says if data is normal or is anomaly \n",
        "Y = list(Label)            "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V0vZs_E7ROl0"
      },
      "source": [
        "Regaining RAM once again"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Frt1RiSvqm3L"
      },
      "source": [
        "# here we divide our data to 2 sets: training and test\n",
        "x_train, x_test, y_train, y_test = sklearn.model_selection.train_test_split(X, Y, test_size = 0.1)   # test_size = 0.1 draws 10% of dataset"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yI__F7kbLP8V",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "1b514a43-a0fa-4b18-a4dd-a1328da4f4c2"
      },
      "source": [
        "del X, Y, traindf\n",
        "#del x_test, y_test\n",
        "gc.collect()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X_dBkKc4RT3O"
      },
      "source": [
        "To train our our model, we have chosen KNN method. We have iteratively checked the most optimal value for n_neighbours = 9"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1hPeEUAe0M7y",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "cd55dda1-e5dc-479f-9dab-2ad3c1710348"
      },
      "source": [
        "#here we train our machine\n",
        "model = KNeighborsClassifier(n_neighbors=9)\n",
        "model.fit(x_train, y_train)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n",
              "                     metric_params=None, n_jobs=None, n_neighbors=9, p=2,\n",
              "                     weights='uniform')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PI1qZKMcK26K"
      },
      "source": [
        "# saving our model\n",
        "with open(\"ddosdetectormodel.pickle\",\"wb\") as ddos:\n",
        "  pickle.dump(model, ddos)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HOipSpxO2zbg"
      },
      "source": [
        "# Testing our model on balanced dataset"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ygdttj7IATSL",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "dd239898-522e-47d8-e972-eef8e790e96b"
      },
      "source": [
        "acc = model.score(x_test, y_test)\n",
        "print(\"Prediction accuracy: \", acc)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Prediction accuracy:  0.9981293639490048\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7cTF9DdhACVI"
      },
      "source": [
        "Confusion Matrix for data, where we have trained our AI"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ewTCod0_ABTm",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "2ce23db4-a078-4c3d-8c49-5b7ba27de537"
      },
      "source": [
        "y_pred = model.predict(x_test)\n",
        "conf_m = confusion_matrix(y_test, y_pred)\n",
        "print(conf_m)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[287580    689]\n",
            " [   388 287083]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GJFj2l8w9Azi",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 279
        },
        "outputId": "1d713845-6c2c-4b07-bbf6-9a2508204941"
      },
      "source": [
        "plot_confusion_matrix(model, x_test, y_test)\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUsAAAEGCAYAAADscbcsAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXhV1b3/8fc3IQkQwhCRQUgAJaLUKrVexTrUCRna5zrUATvIdawoivNVb6/2atvbinOr+NPKVTs42+uEItJW0VsQHJnEpCKEMBPmKdP398fe0ZOQnLMPySHD+byeZz85Z+2111onB75Za6+99zJ3R0RE4sto6QaIiLQFCpYiIhEoWIqIRKBgKSISgYKliEgEHVq6AbF65mf6wIKslm6GJOHzTzu3dBMkCTvZRoXvsqaUMfLEXF9fXh0p7wef7prm7qOaUl9r0aqC5cCCLN6fVtDSzZAkjNxvWEs3QZIw22c0uYz15dW8P60wUt7MvsU9m1xhK9GqgqWItH4O1FDT0s3Y6xQsRSQpjlPp0Ybh7YmCpYgkTT1LEZEEHKc6DW+TVrAUkaTVoGApIhKXA9UKliIiialnKSKSgAOVOmcpIhKf4xqGi4gk5FCdfrFSwVJEkhPcwZN+FCxFJElGNU16FkebpGApIkkJJngULEVE4gqus1SwFBFJqEY9SxGR+NSzFBGJwDGq03BFGgVLEUmahuEiIgk4RoVntnQz9joFSxFJSnBRuobhIiIJaYJHRCQBd6Pa1bMUEUmoRj1LEZH4ggme9Asd6deXFpEmqZ3gibLFY2YFZvY3M1toZgvMbGKY/nMzKzOzj8NtTMwxN5tZiZktNrORMemjwrQSM7spJn2Qmc0O058xs+wwPSd8XxLuH5jocytYikjSqt0ibQlUAde5+1BgOHCFmQ0N993r7sPCbSpAuG8s8A1gFPCQmWWaWSbwIDAaGAqcF1POb8KyBgMbgIvC9IuADWH6vWG+uBQsRSQptXfwRNniluO+0t0/DF9vARYB/eIcchrwtLvvcvclQAlwZLiVuPsX7l4BPA2cZmYGnAQ8Hx7/BHB6TFlPhK+fB04O8zdKwVJEklbjGZE2oKeZzY3ZLm2ovHAY/C1gdpg0wcw+NbMpZtYjTOsHlMYctjxMayx9H2Cju1fVS69TVrh/U5i/Uel3llZEmiR4kEbkftY6dz8iXgYz6wK8AFzt7pvNbDJwR1jVHcDdwIV73uLmoWApIklxjMpmut3RzLIIAuWf3P1FAHdfHbP/UeDV8G0ZUBBzeP8wjUbS1wPdzaxD2HuMzV9b1nIz6wB0C/M3SsNwEUmKO1R7RqQtnvAc4WPAIne/Jya9b0y2M4D54euXgbHhTPYgoAh4H5gDFIUz39kEk0Avu7sDfwPOCo8fB7wUU9a48PVZwF/D/I1Sz1JEkmTNdVH6McBPgHlm9nGYdgvBbPYwgmH4l8BPAdx9gZk9CywkmEm/wt2rAcxsAjANyASmuPuCsLx/B542s18AHxEEZ8KffzCzEqCcIMDGpWApIklxaJbbHd39XWgw6k6Nc8wvgV82kD61oePc/QuC2fL66TuBs5Npr4KliCRND/8VEUnAMT38V0QkkWAp3PQLHen3iUWkiUzPsxQRScSh9u6ctKJgKSJJU89SRCQBd1PPUkQkkWCCR6s7iogkoDV4REQSCiZ4dM5SRCQh3cEjIpKA7uAREYko0WJk7ZGCpYgkxR0qaxQsRUTiCobhCpYiIgnpDh5hTVkWkyYWsnFtFpgz5sfrOePidXXybNmYyT3XFrByaQ5ZOTVcd08pAw/a2aR6K3YZk64qpHheZ7r2qOKWh5fSp6CCVaXZXPLdg+i//y4ADvr2Nib+ZnmT6mqPcrtWc81dwffgDvdcW8CiD3K/2t+lWxXX3lNK3wEVVO4y7r62gKWLOzWpzqzsGm54YBlF39zB5g0d+NVlA1i9PJvDj9/ChbespEOWU1VpPHpHXz55L6+pH7HVSNdLh1LalzazUWa22MxKzOymVNbVXDI7OJfeuoJH3/6M+18t5pXHe7L085w6eZ5+oDcHfGMHD89YzA33L2PyrfGWOq5rVWk2N/xg8G7p057Kp0v3ah7/v0WceclaHvvF18uQ9B2wi8lvLWbyW4sVKBsx/vYy5v49j4uPP4jxpxzIsuKOdfaPvWoN/1zQifGnDGHSxELG374ictm9+1dw5/Mlu6WPPK+crRs7cMExB/Pioz256GdBmZvKM7l13CAuO3kIkyYWcOMDy5r24VodS2Yp3HYjZZ/GzDKBB4HRwFCCdTWGpqq+5rJP7yqKDt0BQOcuNRQM3sW6lVl18iwrzuGwY7cCUFi0i9Wl2WxYG3TSZ7zQgyvHFDH+lCHcf2N/qquj1fuPad0YcXY5AMd9fyMfv5tH/OWTpFbnvGq+OXwbb/w5H4Cqygy2ba57O15h0U4+ebcLAKUlHeldUEH3npUAnHTmBh547XMemr6Yq35TSkZGtF/80SM3Mf25YEnrma92Z9ixWwHnn/M7U746+DezdHFHcjo6Wdk1zfFRW42acB2eRFt7ksrQfyRQ4u5fuHsF8DRwWgrra3arSrP55/xOHHT49jrpg4bu5L2p3QD47KPOrF6ezbqVWSwrzuHtl7pz70vFTH5rMRmZ8NcXezRU9G7Wrcpi3/2C/7yZHYJh5eby4D/8qmXZXD7iQK4/czDzZufGKyYt9SmsYNP6TK67t5QH31zM1XeVktOp7l+pJQs7ccyYTQAMGbad3v0r6Nm3koLBO/nuaRu55rQiLh8xhJpq46QzN0Sqt2efKtauCIJiTbWxbXMmXfPr1nvs9zZRMr8TlRXtp5cVzIZnRtrak1Ses+wHlMa8Xw4cVT+TmV0KXApQ2K/1nELdsS2DOy4eyGW3l5GbV7dXcO6E1Uz+z36MP2UIgw7eweBDdpCRAR/NzKN4XmeuHD0EgIqdRvd9qgD4rwsHsmpZDlWVxpqyLMafEuQ5/eK1jBxb3mg78ntV8sc5C+maX03xp534+QWDeOTvn+3WpnSWmekM/uYOHvxZPxZ/lMtlt5dx7oQ1PDnp61MZz/yuF+PvKOOh6YtZsqgTJfM7UVNjfOu4rRR9czu/ff1zALI7OhvXB/8Ob31sCX0KK+iQ5fTqV8lD0xcD8L+/35c3n8lP2K4BB+7kov9YyS3n7Z+CT91ydFF6C3H3R4BHAI44rGOrGHhWVcIdFw/kpDM3cGzYG4mVm1fD9fcFfwfcYdxRQ+kzYBfzZ+cy4uxyLrxl5W7H3DblSyDord59dSGTXqh7Dqxnn0rWrgh6l9VVfNVLMYPsnKC3UnToDvYbWEHZFzkceNiOZv7Ubde6lVmsXZnF4o+CXve7r3bjnAlr6uTZvjWTu68pDN85T8xexKql2Rxy1FamP5fP//x3X+q7/aJBQHDO8rr7lnHjWXXPNa9b1YF996tk3cpsMjK9zmigZ98Kbn1sCZMmFrJyac5uZbd17W2IHUUqxwZlQEHM+/5hWqvmDvdcV0hB0S5+8NO1DebZuimTyorgH8vrf87nkOFbyc2rYdhxW5j5Wnc2rgv+Bm3ekMnq5VkNllHf8FM3M/25oLcy89XuHHbsFsxg4/rMr857rlyaTdmSbPoUVjTxU7YvG9ZmsW5FNv0PCK5IGHbc1t0meHK7VtMhK+iNj/5hOfNndWH71kw+npnHcd/bSLd9glMged2r6NUv2u931pvdGHF2MGQ/7vsbw3OiRm7Xau54cglTftWXhXPa32mT2tnwKFt7ksqe5RygyMwGEQTJscAPU1hfs1jwfi4zns9n0ME7vhoqX3DzCtaUZQPw/fPXs6w4h7uuLsSAAUN2cs3dQS9zwIG7GHfjSm4eewDuwcz6hF8tp3f/yoT1jjpvPXdeNYB/+87B5HWv4pbJSwGYN6sLT07qQ4cOkJHhXPXr5XTtEXHWKI08+LN+/PvvltEhy1m1LJu7ryngez8JLvl67Q89KSzayfX3LcMxli7uyL3X9QdgWXFHnrizD//99BeYQXWV8btb+n31fcfzxlP53PjAMv7nvUVs2ZjJr8YPAOBfL1jHfoMq+NG1q/nRtasBuHns/mxaH+0PZ1vQ3ma6ozBP4ZSrmY0B7gMygSnhAumNOuKwjv7+tIJ4WaSVGbnfsJZugiRhts9gs5c3qcvX46BeftKUsyLlffGYyR+4+xFNqa+1SOk5S3efCkxNZR0isve1tyF2FC0+wSMibUu63sGjYCkiSVOwFBFJIF2vs0y/KS0RabLmuN3RzArM7G9mttDMFpjZxDA938ymm1lx+LNHmG5m9kD4rIlPzezwmLLGhfmLzWxcTPq3zWxeeMwDZmbx6ohHwVJEkuIOVTUZkbYEqoDr3H0oMBy4Inx+xE3ADHcvAmaE7yF4zkRRuF0KTIYg8AG3EdwheCRwW0zwmwxcEnPcqDC9sToapWApIklrjovS3X2lu38Yvt4CLCK4Tfo04Ikw2xPA6eHr04AnPTAL6G5mfYGRwHR3L3f3DcB0YFS4r6u7z/LgGskn65XVUB2N0jlLEUlKkucse5rZ3Jj3j4S3ONdhZgOBbwGzgd7uXnvP8Cqgd/i6oedN9EuQvryBdOLU0SgFSxFJmkcPlusSXZRuZl2AF4Cr3X1zeFoxrMfdzFL6zIiodWgYLiJJa67nWZpZFkGg/JO7vxgmrw6H0IQ/a5+K0tjzJuKl928gPV4djVKwFJGkuDfPOctwZvoxYJG73xOz62WgdkZ7HPBSTPr54az4cGBTOJSeBpxqZj3CiZ1TgWnhvs1mNjys6/x6ZTVUR6M0DBeRJBnVzbMU7jHAT4B5ZvZxmHYL8GvgWTO7CFgKnBPumwqMAUqA7cAFAO5ebmZ3EDy8B+B2d699SOzlwONAJ+D1cCNOHY1SsBSRpCVxzjJOGf4uNDpWP7mB/A5c0UhZU4ApDaTPBQ5pIH19Q3XEo2ApIknRveEiIlE4abmYnoKliCQtHZeVULAUkaR4803wtCkKliKSNA3DRUQiaI7Z8LZGwVJEkuKuYCkiEokuHRIRiUDnLEVEEnCMGs2Gi4gkloYdSwVLEUmSJnhERCJKw66lgqWIJE09yxhm9lvi/P1w96tS0iIRadUcqKlRsIw1N84+EUlXDqhn+TV3fyL2vZl1dvftqW+SiLR26XidZcKLpczsaDNbCHwWvj/MzB5KectEpPXyiFs7EuXK0vsIFjFfD+DunwDHp7JRItKaGe7RtvYk0my4u5fGruULVKemOSLSJrSzXmMUUYJlqZl9B/Bwjd+JwKLUNktEWi0HT8PZ8CjD8MsIVlTrB6wAhtHICmsiki4s4tZ+JOxZuvs64Ed7oS0i0lak4TA8ymz4/mb2ipmtNbM1ZvaSme2/NxonIq2UZsMb9GfgWaAvsB/wHPBUKhslIq1Y7UXpUbZ2JEqw7Ozuf3D3qnD7I9Ax1Q0TkdbLPdrWnsS7Nzw/fPm6md0EPE3wN+VcYOpeaJuItFZpOBseb4LnA4LgWPtb+WnMPgduTlWjRKR1s3bWa4yi0WG4uw9y9/3Dn/U3TfCIpKuokzsRAqqZTQknjufHpP3czMrM7ONwGxOz72YzKzGzxWY2MiZ9VJhWEo6Ea9MHmdnsMP0ZM8sO03PC9yXh/oGJ2hppIQ0zO8TMzjGz82u3KMeJSHsUcXIn2gTP48CoBtLvdfdh4TYVwMyGAmOBb4THPGRmmWaWCTwIjAaGAueFeQF+E5Y1GNgAXBSmXwRsCNPvDfPFFeXSoduA34bbicCdwL8mOk5E2rFm6lm6+ztAecRaTwOedvdd7r4EKAGODLcSd//C3SsI5ldOs+Ae7ZOA58PjnwBOjymr9slqzwMnW717uuuL0rM8CzgZWOXuFwCHAd0ifjgRaY9qIm57boKZfRoO03uEaf2A0pg8y8O0xtL3ATa6e1W99Dplhfs3hfkbFSVY7nD3GqDKzLoCa4CCCMeJSHuU3HWWPc1sbsx2aYQaJgMHENxavRK4O3UfJrooD9KYa2bdgUcJZsi3Av9IaatEpFVLYjZ8nbsfkUzZ7r76q3rMHgVeDd+WUbej1j9Mo5H09UB3M+sQ9h5j89eWtdzMOhCMltfHa1fCnqW7X+7uG939YWAEMC4cjotIukrh7Y5m1jfm7RlA7Uz5y8DYcCZ7EFAEvA/MAYrCme9sgkmgl93dgb8RnEoEGAe8FFPWuPD1WcBfw/yNindR+uHx9rn7h/EKFhFJxMyeAk4gGK4vB24DTjCzYQTh9kvCa7zdfYGZPQssBKqAK9y9OixnAjANyASmuPuCsIp/B542s18AHwGPhemPAX8wsxKCCaaxCdvaWDA1s7/FOc7d/aREhSerq+X7UXZycxcrKTRtxcct3QRJwpEjS5n7yc4m3X6TU1jg/a6/OlLeJROv/yDZYXhrFW/BshP3ZkNEpI1wdLujiEgkaXi7o4KliCQtHe8NV7AUkeSlYbCMcrujmdmPzezW8H2hmR2Z+qaJSKulJ6U36CHgaOC88P0WgpvWRSQNmUff2pMow/Cj3P1wM/sIwN031D7mSETSlGbDG1QZPgLJAcxsX5p6i7yItGntrdcYRZRh+APAX4BeZvZL4F3gVyltlYi0bml4zjLKuuF/MrMPCB7TZsDp7r4o5S0TkdapHZ6PjCJhsDSzQmA78EpsmrsvS2XDRKQVU7Bs0Gt8vXBZR2AQsJjg0e4ikoYsDWctogzDvxn7Pnwa0eUpa5GISCuU9B087v6hmR2VisaISBuhYfjuzOzamLcZwOHAipS1SERaN03wNCov5nUVwTnMF1LTHBFpExQs6wovRs9z9+v3UntEpC1QsPxa7SI/ZnbM3myQiLRuhmbD63uf4Pzkx2b2MvAcsK12p7u/mOK2iUhrpHOWjepIsETkSXx9vaUDCpYi6UrBso5e4Uz4fL4OkrXS8FclIl9JwwgQL1hmAl2oGyRrpeGvSkRqaRhe10p3v32vtURE2g4FyzrS7+meIpKYaza8vpP3WitEpG1Rz/Jr7l6+NxsiIm2HzlmKiEShYCkikkA7XDIiCgVLEUmKkZ7D8CgLlomI1NFc64ab2RQzW2Nm82PS8s1supkVhz97hOlmZg+YWYmZfRo+iLz2mHFh/mIzGxeT/m0zmxce84CZWbw64lGwFJHkNd/qjo8Do+ql3QTMcPciYEb4HmA0UBRulwKTIQh8wG3AUcCRwG0xwW8ycEnMcaMS1NEoBUsRSV4zBUt3fweof+XNacAT4esngNNj0p/0wCygu5n1BUYC09293N03ANOBUeG+ru4+y90deLJeWQ3V0SidsxSR5CT31KGeZjY35v0j7v5IgmN6u/vK8PUqoHf4uh9QGpNveZgWL315A+nx6miUgqWIJC96sFzn7kfscTXubpba6aSodWgYLiJJs5po2x5aHQ6hCX+uCdPLgIKYfP3DtHjp/RtIj1dHoxQsRSRpzTUb3oiXgdoZ7XHASzHp54ez4sOBTeFQehpwqpn1CCd2TgWmhfs2m9nwcBb8/HplNVRHozQMF5HkNONF6Wb2FHACwbnN5QSz2r8GnjWzi4ClwDlh9qnAGKAE2A5cAMGt2WZ2BzAnzHd7zO3alxPMuHcCXg834tTRKAVLEUleMwVLdz+vkV27PcgnnNG+opFypgBTGkifCxzSQPr6huqIR8FSRJKSrnfwKFiKSNKsJv2ipYKliCRHD9IQEYlGw3ARkSgULEVEElPPUkQkCgVLEZEEtLqjiEhius5SRCQqT79oqWApIklTz1LIyqnh7hdLyMp2Mjs4M1/rzh/u6lMnT69+FVx7Tynd9qliy8ZM7ryykHUrs5tUb173Km55eCm9+1ewenk2v/zpALZu6sCJZ2zgnCvWYAY7tmXw25v688XCTk2qq71ZU5bFpImFbFybBeaM+fF6zrh4XZ08WzZmcs+1BaxcmkNWTg3X3VPKwIN2Nqneil3GpKsKKZ7Xma49gu+vT0EFq0qzueS7B9F//10AHPTtbUz8zfIEpbUhaXpResoe0dbQQkRtQeUu48azD2D8iCGMHzGEI07YwkGHb6uT55JbV/DW8z0Yf8oQ/nRvby64eVXk8g89eivX3btst/RzJqzho3e7cOGxB/PRu104d0LweL3Vpdnc8IMDuOzkoK6Jd7aj/3TNJLODc+mtK3j07c+4/9ViXnm8J0s/z6mT5+kHenPAN3bw8IzF3HD/Mibf2q+R0na3qjSbG34weLf0aU/l06V7NY//3yLOvGQtj/2i71f7+g7YxeS3FjP5rcXtK1CGUvw8y1Yplc+zfJzdFyJqA4yd2zMB6JDlZGb5bqdnBhy4k0/e6wLAJ+914eiRm77ad9b4NTww9XMmv7WYn1wfPYgePXIzbz2bD8Bbz+Zz9KjNACycm8vWTcEA4LMPO9Ozb8Uef7L2ap/eVRQdugOAzl1qKBi8i3Urs+rkWVacw2HHbgWgsGgXq0uz2bA2+L3OeKEHV44pYvwpQ7j/xv5UV0er9x/TujHi7OBJYMd9fyMfv5uXNqfyFCybUSMLEbUJGRnOQ9MX88ynC/jonS4s/ii3zv4vFnbimNFBgDxm9CZy82rI61HF4d/dQr9Bu7hqTBGXjziQom9u55Cjtkaqs0fPSsrXBP/By9d0oEfPyt3yjDqvnDl/69rET9e+rSrN5p/zO3HQ4dvrpA8aupP3pnYD4LOPOrN6eTbrVmaxrDiHt1/qzr0vFTP5rcVkZMJfX0y4KioA61Zlse9+wfeU2QFyu1azuTz4Q7tqWTaXjziQ688czLzZufGKaXucYIInytaOtPg5SzO7lGBZSzrSuYVbE6ipMS4fMYTcrtXc9tgSBgzZwdLFX58nfOT2/bjil2WMOLecebO6sHZFFjXVxre/u4XDv7uFh6Z/DkCnzjX0238X82d34f5Xi8nKqaFT5xryulfz0PTFADz2i7588Hb9AGi4W52Uw76zlZHnlXPt6bsPByWwY1sGd1w8kMtuLyM3r2635twJq5n8n/0Yf8oQBh28g8GH7CAjAz6amUfxvM5cOXoIABU7je77VAHwXxcOZNWyHKoqjTVlWYw/Jchz+sVrGTm28X5Afq9K/jhnIV3zqyn+tBM/v2AQj/z9s93a1JZpgqcFhCu9PQLQ1fJb1VewbXMmn/xfF/7lxC11gmX56izuuHggAB07V3PsmE1s25yJAc/8tjdT/7jPbmVN/H4REJyzHHFOOXdfU1hn/4Z1WeT3CnqX+b0q2bj+669m0ME7uPquUn724/3ZsqHFv7JWqaoS7rh4ICeduYFjx2zabX9uXg3X3xcsAOgO444aSp8Bu5g/O5cRZ5dz4S0rdzvmtilfAkFv9e6rC5n0Qkmd/T37VLJ2RdC7rK4K/r10za/GDLJzgrF80aE72G9gBWVf5HDgYTua+VO3oFb1P3Xv0Bo89XTLryK3a/APPbtjDYcfv5XSko518nTNr6J2MbixV67hzWeCc41z385j5NhyOnYOjt+nTyXd9tl9ON2QWW925ZRzgt7KKeeU849pQW9z334V3Pr7L5l0VSFlX+TEKyJtucM91xVSULSLH/x0bYN5tm7KpLIi6K2//ud8Dhm+ldy8GoYdt4WZr3Vn47rgj9DmDZmsXp7VYBn1DT91M9OfC777ma9257Bjt2AGG9dnfnXec+XSbMqWZNOnsP2ca669KD2Fa/C0Suqm1JPfu5Lr719GRgZkZMA7r3Rj9ltdOf+GVXz+SSdmvdmNQ4/eyoU3r8Qd5s3uwoO3BDOrH76dR+Hgndz3StAD2bEtgzuvLGTT+sT1PvO7XvzHw0sZNbacNWXBpUMAP7pmNXk9qpnw38GManWVceXoA1Pz4duoBe/nMuP5fAYdvOOrofIFN69gTVlwOdf3z1/PsuIc7rq6EAMGDNnJNXcHvcwBB+5i3I0ruXnsAbgHM+sTfrWc3v0T/5Ebdd567rxqAP/2nYODS78mLwVg3qwuPDmpDx06BOe/r/r1crr2iDhr1Ba4p+XDf81TdBI2diEiYDVwm7s/Fu+YrpbvR1lSy2JIC5u24uOWboIk4ciRpcz9ZKclztm4vO79/VvHT4yUd+YrN37QlHXDW5OU9SzjLEQkIm1cextiR6FhuIgkx4E0HIYrWIpI8tIvVipYikjyNAwXEYkgHWfDFSxFJDlp+tQhBUsRSUpwUXr6RUsFSxFJXvu5zT0yBUsRSVo69ix1b7iIJMeT2BIwsy/NbJ6ZfWxmc8O0fDObbmbF4c8eYbqZ2QNmVmJmn5rZ4THljAvzF5vZuJj0b4fll4TH7vHdSwqWIpKk4N7wKFtEJ7r7sJjbIm8CZrh7ETAjfA8wGigKt0uByRAEV+A24CjgSOC22gAb5rkk5rg9fiC5gqWIJC+1D/89DXgifP0EcHpM+pMemAV0N7O+wEhguruXu/sGYDowKtzX1d1nefAQjCdjykqagqWIJMeTWlaip5nNjdku3b003jSzD2L29Xb32geMrgJ6h6/7AaUxxy4P0+KlL28gfY9ogkdEkhe917guwVOHjnX3MjPrBUw3s8/qVuNu1jruF1LPUkSS10wTPO5eFv5cA/yF4Jzj6nAITfhzTZi9DCiIObx/mBYvvX8D6XtEwVJEkmY1NZG2uGWY5ZpZXu1r4FRgPvAyUDujPQ54KXz9MnB+OCs+HNgUDtenAaeaWY9wYudUYFq4b7OZDQ9nwc+PKStpGoaLSHKc5roovTfwl/Bqng7An939DTObAzxrZhcBS4FzwvxTgTFACbAduADA3cvN7A5gTpjvdnevXVHucoJluTsBr4fbHlGwFJGkGN4sF6W7+xfAYQ2krwd2WzIhnNG+opGypgBTGkifCxzS5MaiYCkieyIN7+BRsBSR5ClYiogk0HznLNsUBUsRSVqime72SMFSRJLUpFsZ2ywFSxFJjqNgKSISSfqNwhUsRSR56fjwXwVLEUmegqWISALuUJ1+43AFSxFJnnqWIiIRKFiKiCTgQPT1ddoNBUsRSZKD65yliEh8jiZ4REQi0TlLEZEIFCxFRBLRgzRERBJzQI9oExGJQD1LEZFEdLujiEhiDq7rLEVEItAdPCIiEeicpYhIAu6aDRcRiUQ9SxGRRByvrm7pRux1CpYikhw9ok1EJCxkXzsAAARzSURBVCJdOiQiEp8Drp6liEgCrof/iohEko4TPOat6BIAM1sLLG3pdqRAT2BdSzdCktJev7MB7r5vUwowszcIfj9RrHP3UU2pr7VoVcGyvTKzue5+REu3Q6LTdyb1ZbR0A0RE2gIFSxGRCBQs945HWroBkjR9Z1KHzlmKiESgnqWISAQKliIiEShYppCZjTKzxWZWYmY3tXR7JDEzm2Jma8xsfku3RVoXBcsUMbNM4EFgNDAUOM/MhrZsqySCx4F2cRG1NC8Fy9Q5Eihx9y/cvQJ4GjithdskCbj7O0B5S7dDWh8Fy9TpB5TGvF8epolIG6RgKSISgYJl6pQBBTHv+4dpItIGKVimzhygyMwGmVk2MBZ4uYXbJCJ7SMEyRdy9CpgATAMWAc+6+4KWbZUkYmZPAf8AhpjZcjO7qKXbJK2DbncUEYlAPUsRkQgULEVEIlCwFBGJQMFSRCQCBUsRkQgULNsQM6s2s4/NbL6ZPWdmnZtQ1uNmdlb4+vfxHvJhZieY2Xf2oI4vzWy3VQAbS6+XZ2uSdf3czK5Pto0iUSlYti073H2Yux8CVACXxe40sz1aB97dL3b3hXGynAAkHSxF2hMFy7ZrJjA47PXNNLOXgYVmlmlmk8xsjpl9amY/BbDA78Lna74F9KotyMz+bmZHhK9HmdmHZvaJmc0ws4EEQfmasFd7nJnta2YvhHXMMbNjwmP3MbM3zWyBmf0esEQfwsz+18w+CI+5tN6+e8P0GWa2b5h2gJm9ER4z08wOao5fpkgie9QTkZYV9iBHA2+ESYcDh7j7kjDgbHL3fzGzHOA9M3sT+BYwhODZmr2BhcCUeuXuCzwKHB+Wle/u5Wb2MLDV3e8K8/0ZuNfd3zWzQoK7lA4GbgPedffbzex7QJS7Xy4M6+gEzDGzF9x9PZALzHX3a8zs1rDsCQQLiV3m7sVmdhTwEHDSHvwaRZKiYNm2dDKzj8PXM4HHCIbH77v7kjD9VODQ2vORQDegCDgeeMrdq4EVZvbXBsofDrxTW5a7N/Zcx1OAoWZfdRy7mlmXsI4zw2NfM7MNET7TVWZ2Rvi6IGzreqAGeCZM/yPwYljHd4DnYurOiVCHSJMpWLYtO9x9WGxCGDS2xSYBV7r7tHr5xjRjOzKA4e6+s4G2RGZmJxAE3qPdfbuZ/R3o2Eh2D+vdWP93ILI36Jxl+zMNGG9mWQBmdqCZ5QLvAOeG5zT7Aic2cOws4HgzGxQemx+mbwHyYvK9CVxZ+8bMaoPXO8APw7TRQI8Ebe0GbAgD5UEEPdtaGUBt7/iHBMP7zcASMzs7rMPM7LAEdYg0CwXL9uf3BOcjPwwX3fp/BCOIvwDF4b4nCZ6sU4e7rwUuJRjyfsLXw+BXgDNqJ3iAq4AjwgmkhXw9K/9fBMF2AcFwfFmCtr4BdDCzRcCvCYJ1rW3AkeFnOAm4PUz/EXBR2L4FaKkO2Uv01CERkQjUsxQRiUDBUkQkAgVLEZEIFCxFRCJQsBQRiUDBUkQkAgVLEZEI/j/lpn9HItKkSAAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A631FCF1oqW7"
      },
      "source": [
        "Precision of our algorithm"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EgxYEpq_o96L",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c6db9193-e0c0-4d93-85b0-54ac20c5bda1"
      },
      "source": [
        "precision_m = precision_score(y_test, y_pred, average=None)\n",
        "print(\"Precision in benign traffic detection: \", precision_m[0])\n",
        "print(\"Precision in DDoS traffic detection:   \", precision_m[1])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Precision in benign traffic detection:  0.9986526280697855\n",
            "Precision in DDoS traffic detection:    0.9976057434357756\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wKJSrhoEr7TQ"
      },
      "source": [
        "Recall of our algorithm"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oTN7pBOnQORY",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "db6066fc-c631-4f09-de95-c884b0e9b9f8"
      },
      "source": [
        "recall_m = recall_score(y_test, y_pred, average=None)\n",
        "print(\"Recall of benign traffic: \", recall_m[0])\n",
        "print(\"Recall of DDoS traffic:   \", recall_m[1])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Recall of benign traffic:  0.9976098713354541\n",
            "Recall of DDoS traffic:    0.9986502986388193\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FCA1PJysTd5L"
      },
      "source": [
        "Loss"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mfVpeRxnTfJV",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "be8821e7-0857-42e6-e11d-d8bffabacd29"
      },
      "source": [
        "loss = log_loss(y_test, y_pred)\n",
        "print(\"Log loss is \", loss)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Log loss is  0.06461043717641432\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yQq1QRRkgfcz",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "1a0376b8-92a1-4d0c-8d74-3417c0f58ea7"
      },
      "source": [
        "# Regaining some RAM\n",
        "del x_test\n",
        "del y_test\n",
        "del x_train\n",
        "del y_train\n",
        "gc.collect()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3638"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 41
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zCI1rui03Xu8"
      },
      "source": [
        "Now, we consider our algorithm to be quite precise. Unfortunately, we can't make it even better, because colab machine has limted resources. With full dataset, we would 've made our algorithm even more precise with better Loss parameter. The problem is, that we have tested our algorithm on dataset, which is balanced. This is important, because in reality, DDoS attacks aren't occuring on a daily basis. Therefore, we have to test it on unbalanced dataset, which is much closer to the reality, than balanced."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RV8Henc8WlLG"
      },
      "source": [
        "# Unbalanced dataset test"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IrKFTbDe5Roe"
      },
      "source": [
        "To make the test legit, we have to test it on unbalanced dataset, because real environment is unbalanced. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lLGtnmPKWxMc"
      },
      "source": [
        "df_test = pd.read_csv('/content/ddos_imbalanced/unbalaced_20_80_dataset.csv', dtype=dtypes,low_memory=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HJG-qvPodfD6"
      },
      "source": [
        "# here we load imbalanced dataset\n",
        "# df_train = pd.read_csv('/content/ddos_imbalanced/unbalaced_20_80_dataset.csv', dtype=dtypes,low_memory=True, nrows = 1000000)  \n",
        "\n",
        "timestamp = le.fit_transform(list(df_test[\"Timestamp\"]))\n",
        "fl_dur = le.fit_transform(list(df_test[\"Flow Duration\"]))\n",
        "pk_size_avrg = le.fit_transform(list(df_test[\"Pkt Size Avg\"]))\n",
        "fwd_pkts = le.fit_transform(list(df_test[\"Fwd Pkts/s\"]))\n",
        "bwd_pkts = le.fit_transform(list(df_test[\"Bwd Pkts/s\"]))\n",
        "flow_byts = le.fit_transform(list(df_test[\"Flow Byts/s\"]))\n",
        "flow_pkts = le.fit_transform(list(df_test[\"Flow Pkts/s\"]))\n",
        "Label = le.fit_transform(list(df_test['Label']))\n",
        "\n",
        "#X - data on which AI is testing itself\n",
        "X_test = list(zip(timestamp, fl_dur, pk_size_avrg, fwd_pkts, bwd_pkts, flow_byts, flow_pkts)) \n",
        "#Y - data which says if data is normal or is anomaly \n",
        "Y_test = list(Label) \n",
        "\n",
        "x_train2, x_test, y_train2, y_test = sklearn.model_selection.train_test_split(X_test, Y_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HNOGvNRILZex",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ee1d9e60-e987-49d7-c00f-f110f90afa81"
      },
      "source": [
        "del X_test, Y_test, df_test, x_train2, y_train2\n",
        "gc.collect()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 44
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KEM_aWUMIJ31"
      },
      "source": [
        "# **Testing our model on Unbalanced Dataset**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "W1Z9oATxY4t_",
        "outputId": "52d95370-b6af-4d3d-c538-9b011d72ea16"
      },
      "source": [
        "acc = model.score(x_test, y_test)\n",
        "print(\"Prediction accuracy: \", acc)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Prediction accuracy:  0.8769830599623555\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fPIrepIvJZLo",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "17dbf59b-5d60-410f-c299-825d53bd40fc"
      },
      "source": [
        "y_pred = model.predict(x_test)\n",
        "confusion_matrix(y_test, y_pred)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[1535109,   45419],\n",
              "       [ 188821,  134779]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 46
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PjMJ-qR0az9C",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 290
        },
        "outputId": "42744573-05fc-43a3-b882-2e20e1537235"
      },
      "source": [
        "plot_confusion_matrix(model, x_test, y_test)\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAATUAAAERCAYAAAAaIjAkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXwV1f3/8dcnCTuBAAFBNlEBpSJKkUVaRUVF2692dalarVrrArW2amtttdparf5abSlqsVK6uXax1KJYtBatIIu4ABahaFnEBhIgYc32+f0xEwyB5M6Qe5Ob4f18PObhvTPnnnMuMZ+cZc4cc3dERJIip7krICKSTgpqIpIoCmoikigKaiKSKApqIpIoCmoikigKaiKyFzObZmZFZrYkYvpzzGyZmS01s0cyXb8G66L71ESkLjM7AdgK/Mbdj0qRdiDwBHCyu28ysx7uXtQU9dwXtdREZC/uPgcoqX3OzA4zs2fNbJGZvWRmR4SXvgxMcfdN4WebLaCBgpqIRDcVmOTuHwWuB+4Pzw8CBpnZv8xsnplNaLYaAnnNWbiItAxm1hE4HnjSzGpOtwn/mwcMBMYBfYA5ZjbU3Tc3dT1rKiMikkoOsNndj9nHtbXAq+5eAbxrZu8QBLkFTVnBGup+ikhK7l5KELA+D2CBYeHlpwhaaZhZIUF3dFVz1BMU1ERkH8zsUWAuMNjM1prZZcAFwGVm9gawFDg7TD4LKDazZcA/gBvcvbg56g26pUNEEkYtNRFJlKyaKCjsmuuH9G3V3NWQGN55s31zV0Fi2Mk2yn2XpU5Zv9NP6uDFJVWR0i56c9csd2/SWzyyKqgd0rcV82f1be5qSAynH7yvyTDJVq/6843Oo7ikivmz+kVKm9trRWGjC4wpq4KaiGQ/B6qpbu5q1EtBTURicZwKj9b9bA4KaiISm1pqIpIYjlOVxbeCKaiJSGzVKKiJSEI4UKWgJiJJopaaiCSGAxUaUxORpHBc3U8RSRCHquyNaQpqIhJPsKIgeymoiUhMRhWNWhOfUQpqIhJLMFGgoCYiCRHcp6agJiIJUp3FLTU9+VZEYqlpqUU5UjGzaWZWZGZLUqQ7zswqzexzqfJUUBORWByjipxIRwTTgQafjGtmucCPgOeiZKigJiKxVbtFOlJx9zlASYpkk4A/AkVR6qYxNRGJxTHKPTdq8kIzW1jr/VR3nxr1w2bWG/g0cBJwXJTPKKiJSCzBzbeRO3kb3X1EI4q7D/imu1ebRZucUFATkdia8JaOEcBjYUArBM40s0p3f6q+DyioiUgs7kaVN81wvLsPqHltZtOBpxsKaKCgJiL7oTpNLTUzexQYRzD2tha4FWgF4O4P7k+eCmoiEkswUZCe0OHu58dIe0mUdApqIhJLzImCJqegJiKxVWXxMikFNRGJpWZFQbZSUBOR2KqbaPZzfyioiUgswYJ2BTURSQjHqIi+TKrJKaiJSCzuNNnNt/tDQU1EYrK03XybCQpqIhKLo5aaiCSMJgpEJDGcaA+AbC4KaiISS7BFXvaGjuytmYhkKW1mLCIJ4mhFgYgkjFpqIpIY7qaWmogkRzBRoGVSIpIYTbdHwf5QUBORWIKJAo2piUiCZPOKguytmYhkpZoVBVGOVMxsmpkVmdmSeq5fYGZvmtlbZvaKmQ1LlaeCmojEVk1OpCOC6cCEBq6/C5zo7kOB7wNTU2Wo7qeIxOIOFdXpaQ+5+xwzO6SB66/UejsP6JMqTwU1EYkl6H5GDmqFZraw1vup7p6ytVWPy4BnUiVSUBOR2GKsKNjo7iMaW56ZnUQQ1D6WKq2CWh0/vq4vr87uREFhJVP/sXyv62+80pHvfWkAPfuWAzD2zM1c+PX/NarM8l3GPV/tx4q32tOpSyXffvC/u/NftawtP/tmX7aV5ZCTA5NnvkPrtt6o8pIqJ8eZ/Ow7FK9vxS0XH7rHtVPPKeHy775P8QetAJjxq0KefaRbo8rLLwh+Vgf1Ked/a1tzx1f6s3XLh79Sg4Zt576/ruCHV/Xn5b8VNKqsbNLUt3SY2dHAL4Ez3L04VfqMThSY2QQzW25mK83sW5ksK11OO7eEO36/qsE0R43aygOzl/PA7OWxAtoHa1pzw2cP3+v8rEe70rGgiumvvM1nvryBh3/QC4CqSrh7Un8m3bWGh15czj1/WEluKwW0+nzq8o2sWdG23utzZhRw9amDufrUwbEC2tFjtvKNe1fvdf6ciUUsfrkjl37sSBa/3JFzJxbtvpaT41x283oW/TM/3pdoEYLuZ5Sj0SWZ9QP+BFzk7u9E+UzGgpqZ5QJTgDOAIcD5ZjYkU+Wly9DR28jvUrVfn33+j12YdOZArho/mJ/e2IeqiNnMndWZUz9fAsDHP7mZ11/Oxx0W/TOfAUfu4LCP7ASgU9cqcrN3dUqzKuxVzshTSnnmka6xP/u5q4r42cx3eGD2ci66/oPInxtzeimznwjKm/1EV8ZMKN197exLN/LyzM5s3pjMzlB1uE9BqiMVM3sUmAsMNrO1ZnaZmV1pZleGSW4BugH3m9nrdcbn9imTLbWRwEp3X+Xu5cBjwNkZLK/JvL2oA1eOH8zNFxzKe8uDlsHqFW34518KuPcvK3hg9nJycuGFP3WJlN/GD1rR/eAKAHLzoEOnKkpLclm7qi1m8O3zD+Wa0wbxxJQeGftOLd2Vt73PL3/QC6+u/xdp7JlbeGD2cr4z9T26Hxx074efWEbvAbv46pkDufrUQQwcup2jRm2NVGaXwgpKioLubElRHl0Kg59ht54VHH/GFp7+deO6t9kqmP3MjXSkzsvPd/de7t7K3fu4+8Pu/qC7Pxhev9zdu7j7MeGRcnwuk39GegNrar1fC4yqm8jMrgCuAOjXO/v/qh0+dDu/nb+Mdh2qmf98PrddOoBf/ettFr+Uz4q32jPpjMEAlO80CrpVAnDbpYfwweo2VFYYRetacdX4IM2nLt/A6eeV1FtWVSUsmd+ByTPfoU27ar517uEMPHo7x3482i/dgWLU+FI2b8xj5VvtOXrMvv9t5v29Ey8+VUBFeQ5nXljM9fet4ZvnHMZHTyxj+Ill3P/3oGfTrn01vQ/dxZJXO/LTp1fQqk017dpXk19Qxf1/D8ZYH/5BLxb9s1OdEgwPx5muvG0dD9/Ra/f7pNHjvFMIp3enAowYlv0j4B3yq3e/HnlKGT+/ydhSnAsOp36+hEu/vX6vz9w67T0gGFP78df6cc8fV+5xvbBnBRveD1prVZWwrTSXTl2r6N6rgqGjt9G5W9CPPe7kUla+1U5BrY4hx21j9GmlHHfKMlq3cdrnV3Hj5P9y96T+u9OUbfrwf/VnH+nK5d95HwADHp98EDN/t3er6tpPDgSCMbVTzynhx9f12+P6po2t6NojaK117VHB5uKgjEHDdnDTA/8FoHPXKkaeUkZVlTH32c5p/d7NKZu3yMtk93Md0LfW+z7huRatpCgPD0Pvvxe3p7o6GOs65uNlvPS3gt1jKKWbcvnf2laR8hx9Wil/fzIYm3np6QKGfawMM/jouDLee7stO7cbVZXw5tyO9Bu0KyPfqyX71Z29uHDEEC4eNYQ7r+rPGy933COgAXTtUbH79ejTSlkdTigs/Gc+p59XQtv2wR+Obj0r6NytgijmPdeJ8ecELe3x55Qwd1bQert49JFcPCqoz0tPd2byTb0TFdBqZj/TsUwqEzLZUlsADDSzAQTB7DzgCxksLy3uvKo/b87tyJaSPC746BAu+sYHVFYGP5xPfrGYl54u4OnfdCM3D9q0reamB97DDPoP2sXFN67npvMOwx1y85yJP1zLQX1S/4JMOL+Yu7/an0uOPzK4TSD8K59fUMVnvrKBSWcOwgxGnlzKqPGlKXKTGl+84QPeeaMd857rzNmXbWTMaVuoqjTKNufy4+uCv7ev/TOffofv5L6/Bq3nHdtyuHtSP7akvHEAHv95D25+8L9MOK+EonXBLR0Himx+SKS5Z67HZ2ZnAvcBucA0d7+jofQjhrX1+bP6NpREsszpBx/T3FWQGF715yn1kkY1oboc0cNPnva5SGn/NPaBRem4+TaOjI6puftMYGYmyxCRpqeJAhFJDD0kUkQSR0FNRBJD96mJSOJk831qCmoiEos7VKbpIZGZoKAmIrGp+ykiiaExNRFJnGxerK+gJiKxaaJARBLDXWNqIpIoRpVmP0UkSTSmJiKJobWfIpIsDhl8YlmjZW/HWESyVhp3k5pmZkVmtqSe62ZmPwu32XzTzIanylNBTURi8XCiIMoRwXRgQgPXzwAGhscVwAOpMlRQE5HY3KMdqfPxOUD9W6oF22r+xgPzgAIz69VQnhpTE5HYYsx+FtbZgHhquINcVPvaarM3sPe2bSEFNRGJJWiFRQ5qGxO1R4GIJFMT3tIRe6tNjamJSGzpGlOLYAbwxXAWdDSwxd3r7XqCWmoiEpNjVKdpmZSZPQqMIxh7WwvcCrQCcPcHCXajOxNYCWwHvpQqTwU1EYktXffeuvv5Ka47cE2cPBXURCSeeBMFTU5BTUTiy+JlUgpqIhJbi2ypmdlkGojH7v7VjNRIRLKaA9XVLTCoAQsbuCYiByoHWmJLzd1/Xfu9mbV39+2Zr5KIZLsW/eghMxtjZsuAf4fvh5nZ/RmvmYhkL494NIMod9DdB5wOFAO4+xvACZmslIhkM8M92tEcIs1+uvsasz0qWJWZ6ohIi5DF3c8oQW2NmR0PuJm1Aq4F3s5stUQkazl4Fs9+Rul+XkmwTKE38D5wDDGXLYhI0ljEo+mlbKm5+0bggiaoi4i0FFnc/Ywy+3momf3VzDaEGyT8xcwObYrKiUiWauGzn48ATwC9gIOBJ4FHM1kpEcliNTffRjmaQZSg1t7df+vuleHxO6BtpismItmrCR8SGVtDaz+7hi+fMbNvAY8RxOhzCR7cJiIHqiye/WxoomARQRCrqf1Xal1z4KZMVUpEsptl8URBQ2s/BzRlRUSkhWjGSYAoIq0oMLOjgCHUGktz999kqlIiks2abxIgipRBzcxuJdgYYQjBWNoZwMuAgprIgSqLW2pRZj8/B5wCfODuXwKGAZ0zWisRyW7VEY8UzGyCmS03s5XhhGTd6/3M7B9mttjM3jSzM1PlGSWo7XD3aqDSzDoBRey5uaiIHEjSdJ+ameUCUwh6f0OA881sSJ1k3wGecPdjgfOAlI89izKmttDMCoCHCGZEtwJzI3xORBIqTbOfI4GV7r4KwMweA84GltVK40Cn8HVngvXnDYqy9vPq8OWDZvYs0Mnd34xRcRFJmvQEtd7Amlrv1wKj6qT5HvCcmU0COgDjU2Xa0M23wxu65u6vpcpcRA54hWZWe7+Tqe4+Ncbnzwemu/uPzWwM8FszOyocEtunhlpqP27gmgMnx6hYJCv+XcAnxp6d7mwlg3K7a9uKlsRK0rMrZozu50Z3H1HPtXXsOT7fJzxX22XABAB3n2tmbYFCgrH9fWro5tuTotRYRA4wTrqWSS0ABprZAIJgdh7whTppVhPcfTHdzI4kuFd2Q0OZajNjEYkvDWNq7l5pZhOBWUAuMM3dl5rZ7cBCd58BfAN4yMyuC0u9xL3hpfIKaiISW7rWfrr7TOo8IMPdb6n1ehkwNk6eCmoiEl9LXlFggQvN7JbwfT8zG5n5qolI1mrhT769HxhDMLUKUEZwF7CIHIDMox/NIUr3c5S7DzezxQDuvsnMWme4XiKSzVroQyJrVIRrtBzAzLoTaamqiCRVNj8kMkr382fAn4EeZnYHwWOHfpjRWolIdsviMbUoaz9/b2aLCG6AM+BT7q4d2kUOVM04XhZFlIdE9gO2A3+tfc7dV2eyYiKSxVpyUAP+xocbsLQFBgDLgY9ksF4iksUsi0fVo3Q/h9Z+Hz694+p6kouINKvYKwrc/TUzq/vMIxE5kLTk7qeZfb3W2xxgOBGePikiCdXSJwqA/FqvKwnG2P6YmeqISIvQUoNaeNNtvrtf30T1EZGWoCUGNTPLC593FOuxHyKSbEbLnf2cTzB+9rqZzQCeBLbVXHT3P2W4biKSjRIwptYWKCbYk6DmfjUHFNREDlQtNKj1CGc+l/BhMKuRxV9JRDIuiyNAQ0EtF+jInsGsRhZ/JRHJtJba/Vzv7rc3WU1EpOVooUEte58CJyLNx7N79rOh56md0mS1EJGWJU3PUzOzCWa23MxWmtm36klzjpktM7OlZvZIqjwb2sy4JHWVRORAlI4xtfDm/inAqcBaYIGZzQi3xatJMxC4CRgbbiXQI1W+UZ58KyKyp/S01EYCK919lbuXA48BZ9dJ82VgirtvAnD3olSZKqiJSDxRA1oQ1ArNbGGt44paOfUG1tR6vzY8V9sgYJCZ/cvM5pnZhFTV02bGIhKLEav7udHdRzSiuDxgIDAO6APMMbOh7r65vg+opSYisaVp3891QN9a7/uE52pbC8xw9wp3fxd4hyDI1UtBTUTiS8+Y2gJgoJkNCPcSPg+YUSfNUwStNMyskKA7uqqhTBXURCS+NAQ1d68EJgKzgLeBJ9x9qZndbmZnhclmAcVmtgz4B3CDuxc3lK/G1EQknjQ+pcPdZwIz65y7pdZrB74eHpEoqIlIfC10mZSIyD5l8zIpBTURia2lPqVDRGRvEdd1NhcFNRGJT0FNRJIi5oqCJqegJiKxWXX2RjUFNRGJR2NqIpI06n6KSLIoqIlIkqilJiLJoqAmIomR5btJKaiJSCy6T01EksezN6opqIlIbGqptSDX3rSYkWP/x+ZNbbjmopP2ut4xv5xrb3qdXr23UV6ey09/eAz/fbdTo8rMa1XFN767mMMHb6ZsS2vuumUERR+0p0fP7Tz4yAusW90RgH8v7cKUe4Y1qqwk+tptSxl5wkY2l7Tm6s+O2ev66HFFXHTNKqqrobrK+MU9g1m2uKBRZXbsVMFNd79Fj4N3UPR+O+68YShby1oxdEQJt9z3Bh+sawfAKy/04NFfHNqosrJOlt98m7HHeZvZNDMrMrMlmSojE2bP7MctXx9d7/VzvriCVSs6M/Hik/jJ94/liq9F/3o9em7nzsn/2uv86Z9czdayVnz53PE89fhhfOnq3Xu5sn5dByZdMo5Jl4xTQKvH7L8czHevOrbe66+/2pVrPj+KSeeO5t5bh3DtrcvqTVvX0BElXHf70r3On3Ppe7w+vytfPmssr8/vyucve2/3taWLuzDp3NFMOnd08gJayKqjHc0hk3sUTAdS7tGXbZa+0Y2y0tb1Xu93SBlvvlYIwNrV+RzUazsFXXYCcNJpa/jJQ3OYPP1FJt7wBjk50f6cjfr4Bzw/M9hU5+UXezHsoxvJ6j+FWWbJa10oK21V7/WdO/IIhrehbbuqPYaDPnvxe9z3+/lMeXIeF1z1n8hljj5pA7Nn9AJg9oxejDlpw37VvaU6IIOau88BSjKVf3NZtbIzx5+4HoBBR26ix0E7KOyxk779y/j4Ke9zw5UfY9Il46iuNsadtjZSnt2672RDUdBdqa7KYfu2PDp1LgegZ6/t/OxXL3LXz//FR4Y1uN+ENGDMyUX84qlXuO3nr3PfrUMAOHZMMQf328HXLjiOieeMYuCQMo4avilSfgVdy9m0sQ0Amza2pqBr+e5rRxy9hZ8/MY/bpyym32Fb0/9lmpsTTBREOZpBs4+phTs2XwHQNi+/mWuT2pO/PZyvfG0Jk6e/yHv/6cR/VnSmutoYNmIjhx+xmfsengNA6zZVbN4UtPhu/uF8eh68nby8aroftIPJ018E4C9PHMrsmf3qLaukuA2XfOZUykpbc/jgzXznzvlcdeFJ7Nhef6tE9m3uCz2Y+0IPjhq+iYuuWcXNXxnO8DHFDB9TzOTHXwWgXfsqDu6/nSWvdeHe380nr1U17dpXkd+5gsmPzwPgVz8dyGuvdKuTu+1uV698uxOXTBjLzh15jPjYRr577xt8+ayxTfdFm0i6JgrCHdd/CuQCv3T3u+pJ91ngD8Bx7r6woTybPai5+1RgKkDnNj2zvs+1Y3sr7vthzfiNM+0Ps1m/rj0fGVbM88/05dcPDtnrM3d8eyQQjKldd/Nibpq05//kxRva0r3HDoo3tCMnt5r2HSop3dIaMMoqcgFYubyA9es60LvfNlb+u3GD3AeyJa91oWefZXQqKMcMnph2CM/8oc9e6a67MPiZDR1Rwviz1nPvLR/Z4/rmktZ0KdzFpo1t6FK4iy0lwR+wHds+/JVa+HIh13z733QqKKd0c/1DGi1SGn5TzSwXmAKcSrBp8QIzm+Huy+qkyweuBV6Nkq/2/YypQ8cK8vKCwYLT/281S17vxo7trXh9YXfGjltP54JdQDBL2v2g7ZHyfPXlnpxy5hoAPjZuPW8uKgSMTgW7do/L9Tx4Gwf33cYH69qn/0slXK++26n5LTzsiFJata6mdHMrFr3SjdM+9T5t21UC0K3HTjrX6kY2ZN6L3Rl/VjAMMf6s9cz7R3cAunTbtbusQUdtwXKc0s3JalnX3Hybhh3aRwIr3X2Vu5cDjwFn7yPd94EfATuj1K/ZW2rZ5sbvLWLosRvpVFDOr//8HL9/eDC5ecFP55mnDqFv/zK+/p3FOLD63Xx+eucxAKx5L5/fPnQEP7hvLmZOVWUO9/9kKBv+lzoIPfd0P67/7ms89Phsykpbc/etHwXgqGOKufDy5VRVGtXVxpR7jmZrWcL+4qfBjXe9xdEjNtGpoILfPPcSv3vgUPLCn9nMJ/swdnwRp/zfeiorjPJdudx141DAWDy3G/0GbOMnv10AwI7tedzz7Y/sbnU15Mlp/bnpnrc47VPrKFof3NIBMPbUIj5xzlqqKo3yXTn86JtBWYniHuchkYVmVru7ODXsnQH0BtbUurYWGFX7w2Y2HOjr7n8zsxuiFGieocE8M3uUYLv4QuB/wK3u/nBDn+ncpqcf3+fCjNRHMsO3RmuNSnaYW/IHtlQUNSrK5hf08WNPuDZS2pf+euMidx+xr2tm9jlggrtfHr6/CBjl7hPD9znAC8Al7v6emb0IXN9sY2rufn6m8haR5pWmiYJ1QN9a7/uE52rkA0cBL5oZQE9ghpmd1VBgU/dTROJxID17FCwABprZAIJgdh7whd3FuG8h6OkBELWlpokCEYnPIx4NZeFeCUwEZgFvA0+4+1Izu93MztrfqqmlJiKxpes+NXefCcysc+6WetKOi5KngpqIxKYt8kQkObL8KR0KaiISS3DzbfZGNQU1EYlPexSISJKopSYiyaExNRFJllhrP5ucgpqIxKfup4gkhjYzFpHEUUtNRBIle2OagpqIxGfV2dv/VFATkXgc3XwrIslhuG6+FZGEUVATkURRUBORxNCYmogkjWY/RSRBXN1PEUkQJ6uDmnaTEpH4qiMeKZjZBDNbbmYrzexb+7j+dTNbZmZvmtnzZtY/VZ4KaiISm7lHOhrMwywXmAKcAQwBzjezIXWSLQZGuPvRwB+Au1PVTUFNROJzj3Y0bCSw0t1XuXs58Bhw9p7F+D/cfXv4dh7BLu4N0piaiMTjDlWRZz8Lzaz2jupT3X1q+Lo3sKbWtbXAqAbyugx4JlWBCmoiEl/0iYKN7j6iscWZ2YXACODEVGkV1EQkvvTMfq4D+tZ63yc8twczGw/cDJzo7rtSZaqgJiLxOJCePQoWAAPNbABBMDsP+ELtBGZ2LPALYIK7F0XJVEFNRGJy8MavKHD3SjObCMwCcoFp7r7UzG4HFrr7DOAeoCPwpJkBrHb3sxrKV0FNROJx4kwUNJyV+0xgZp1zt9R6PT5ungpqIhJfFq8oUFATkfgU1EQkObSgXUSSxAE9ekhEEkUtNRFJjljLpJqcgpqIxOPgabhPLVMU1EQkvvSsKMgIBTURiU9jaiKSGO6a/RSRhFFLTUSSw/GqquauRL0U1EQknvQ9eigjFNREJD7d0iEiSeGAq6UmIonh6XlIZKYoqIlIbNk8UWCeRVOzZrYB+G9z1yMDCoGNzV0JiSWpP7P+7t69MRmY2bME/z5RbHT3CY0pL66sCmpJZWYL07FNmDQd/cxaLu3QLiKJoqAmIomioNY0pjZ3BSQ2/cxaKI2piUiiqKUmIomioCYiiaKglkFmNsHMlpvZSjP7VnPXR1Izs2lmVmRmS5q7LrJ/FNQyxMxygSnAGcAQ4HwzG9K8tZIIpgNNerOopJeCWuaMBFa6+yp3LwceA85u5jpJCu4+Byhp7nrI/lNQy5zewJpa79eG50QkgxTURCRRFNQyZx3Qt9b7PuE5EckgBbXMWQAMNLMBZtYaOA+Y0cx1Ekk8BbUMcfdKYCIwC3gbeMLdlzZvrSQVM3sUmAsMNrO1ZnZZc9dJ4tEyKRFJFLXURCRRFNREJFEU1EQkURTURCRRFNREJFEU1FoQM6sys9fNbImZPWlm7RuR13Qz+1z4+pcNLbY3s3Fmdvx+lPGeme2161B95+uk2RqzrO+Z2fVx6yjJo6DWsuxw92Pc/SigHLiy9kUz2699XN39cndf1kCScUDsoCbSHBTUWq6XgMPDVtRLZjYDWGZmuWZ2j5ktMLM3zewrABb4efh8t9lAj5qMzOxFMxsRvp5gZq+Z2Rtm9ryZHUIQPK8LW4kfN7PuZvbHsIwFZjY2/Gw3M3vOzJaa2S8BS/UlzOwpM1sUfuaKOtfuDc8/b2bdw3OHmdmz4WdeMrMj0vGPKcmhHdpboLBFdgbwbHhqOHCUu78bBoYt7n6cmbUB/mVmzwHHAoMJnu12ELAMmFYn3+7AQ8AJYV5d3b3EzB4Etrr7/wvTPQLc6+4vm1k/glUTRwK3Ai+7++1m9gkgyt34l4ZltAMWmNkf3b0Y6AAsdPfrzOyWMO+JBBuiXOnuK8xsFHA/cPJ+/DNKQimotSztzOz18PVLwMME3cL57v5ueP404Oia8TKgMzAQOAF41N2rgPfN7IV95D8amFOTl7vX91yx8cAQs90NsU5m1jEs4zPhZ/9mZpsifKevmtmnw9d9w7oWA9XA4+H53wF/Css4HniyVtltIpQhBxAFtZZlh7sfU/tE+Mu9rfYpYJK7z6qT7sw01iMHGO3uO/dRl8jMbBxBgBzj7tvN7EWgbT3JPSx3c91/A5HaNKaWPLOAq62gGbwAAADySURBVMysFYCZDTKzDsAc4NxwzK0XcNI+PjsPOMHMBoSf7RqeLwPya6V7DphU88bMaoLMHOAL4bkzgC4p6toZ2BQGtCMIWoo1coCa1uYXCLq1pcC7Zvb5sAwzs2EpypADjIJa8vySYLzstXDzkF8QtMj/DKwIr/2G4EkUe3D3DcAVBF29N/iw+/dX4NM1EwXAV4ER4UTEMj6chb2NICguJeiGrk5R12eBPDN7G7iLIKjW2AaMDL/DycDt4fkLgMvC+i1Fj0iXOvSUDhFJFLXURCRRFNREJFEU1EQkURTURCRRFNREJFEU1EQkURTURCRR/j9uO2bkDqNolQAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nbXZEXqZZAIQ",
        "outputId": "4b178b6e-3ea9-4b47-c516-d45568e32728"
      },
      "source": [
        "precision_m = precision_score(y_test, y_pred, average=None)\n",
        "print(\"Precision in benign traffic detection: \", precision_m[0])\n",
        "print(\"Precision in DDoS traffic detection:   \", precision_m[1])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Precision in benign traffic detection:  0.8904706107556571\n",
            "Precision in DDoS traffic detection:    0.7479494777966459\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-XpZdZsdZEgc",
        "outputId": "538d806f-ed7b-41f0-9309-db2f459b41b6"
      },
      "source": [
        "recall_m = recall_score(y_test, y_pred, average=None)\n",
        "print(\"Recall of benign traffic: \", recall_m[0])\n",
        "print(\"Recall of DDoS traffic:   \", recall_m[1])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Recall of benign traffic:  0.971263400585121\n",
            "Recall of DDoS traffic:    0.41649876390605683\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "s7k5XuigLjAE",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "e0277acb-105e-4258-f998-65b063da3d4d"
      },
      "source": [
        "loss = log_loss(y_test, y_pred)\n",
        "print(loss)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "4.248873657474477\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Tiw08CfkeDLy",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 191
        },
        "outputId": "8c9d8f61-4bb1-422b-b936-55acf406e2b5"
      },
      "source": [
        "# if we don't want colab to disconnect from session\n",
        "while True:pass"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "error",
          "ename": "KeyboardInterrupt",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-43-984feba246c5>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# if we don't want colab to disconnect from session\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0;32mwhile\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;32mpass\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9K0NVuAwNRHU"
      },
      "source": [
        "#del model\n",
        "del traindf, split\n",
        "gc.collect()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MxH53bqoy4HV"
      },
      "source": [
        "# **Conclusions**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xibm0zXBy7vE"
      },
      "source": [
        "In unbalanced dataset, our algorithm is very precise in determining Benign traffic, but much less efficient in catching DDoS traffic. We are quite convinced, that further data processing would solve the problem, since we were training our model on wrongly balanced dataset."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dq-2Z4NgyjAR"
      },
      "source": [
        "Model saving"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a7KEJd-ohnxk"
      },
      "source": [
        "traindf.to_csv('traindf.csv')\n",
        "with open(\"ddosdetectormodel.pickle\",\"wb\") as ddos:\n",
        "  pickle.dump(model, ddos)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HFh1ObxHao_b"
      },
      "source": [
        "Model loading"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sUKghy9lL_3s"
      },
      "source": [
        "#wczytanie zapisanego przez nas modelu\n",
        "pickle_read = open(\"ddosdetectormodel.pickle\", \"rb\")\n",
        "model = pickle.load(pickle_read)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sv45fpKMY4ei"
      },
      "source": [
        "# **Future plans**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DrkhwSiaZywo"
      },
      "source": [
        "Next step of project would be to use created model, which would gather information about traffic in network and detect possible ddos attacks. When detected, it would inform us about attack. \n",
        "Futher implementation would be done on Raspberry Pi 4B.\n"
      ]
    }
  ]
}