home-loan-default-prediction-logistic-regression/home-loan-default-prediction-logistic-regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <b> Problem Statement </b>\n",
    "\n",
    "\n",
    "You work as a Business Analytics Consultant at the Bank of Corporate.The bank was witnessing slower than usual growth in its book of business for the most recent quarter in 2018. The bank provides financial services/products such as savings accounts, current accounts, debit cards, etc. to its customer. The data suggested that it was the home loan business of the bank that was hit by a major loss. Now, loans are the core business of banks. The main profit comes directly from the loan’s interest. The head of the Home Loan business asked the heads of the Sales, Operations, Risk and Analytics teams to investigate and identify the root causes for the slowing growth and solve the problem. A business like selling Home Loans can grow or shrink based on several factors like demand or supply side. Some of the reasons are:\n",
    "\n",
    "- <b> Demand Side: </b> Are interest rates high?\n",
    "- <b> Demand Side: </b> Are there any macro economic reasons, such as recession or low salary growth or inflation?\n",
    "- <b> Supply Side: </b> Are new and attractive housing projects not available in the markets being served?\n",
    "- <b> Supply Side: </b> Have real estate prices shot up making homes unaffordable, relatively speaking?\n",
    "- <b> Competitor Side: </b> Are we losing customers to our competition? Is our competition also facing lower growth?\n",
    "\n",
    "\n",
    "\n",
    "The team found out that the credit risk was in abnormal standards and the default loan rates were high. So what do you mean by default loans and credit risk?\n",
    "\n",
    "<b> Deafult loans : </b> <br>\n",
    "Default is the failure to repay a loan according to the terms agreed to in the promissory note. For most federal student loans, you will default if you have not made a payment in more than 270 days.\n",
    "\n",
    "<b> Credit risk : </b> <br>\n",
    "It is understood simply as the risk a bank takes while lending out money to borrowers. They might default and fail to repay the dues in time and these results in losses to the bank. \n",
    "\n",
    "## <b> So, what do banks do then? </b> <br>\n",
    "They need to manage their credit risks. The goal of credit risk management in banks is to maintain credit risk exposure within proper and acceptable parameters. It is the practice of mitigating losses by understanding the adequacy of a bank’s capital and loan loss reserves at any given time. For this, banks not only need to manage the entire portfolio but also individual credits.\n",
    "\n",
    "## <b> Measures taken </b> <br>\n",
    "So in 2019, the bank came up with a project to build a \"Credit risk estimate model\" for its home loan branch.The loan should be granted after an intensive process of verification and validation. The dataset (provided below) contains the information about all the customers who were contacted during this year and were provided loans based on various parameters. The \"Credit risk estimate model\" need to be cost-efficient so that the bank not only decreases their credit risk but also increase the total profit.\n",
    "\n",
    "\n",
    "## <b> Business objective </b> <br>\n",
    "\n",
    "Your aim is to build the \"Credit risk estimate model\" to classify new loans availed as \"Low Risk\", \"High Risk\" and \"Medium Risk\". This will help the bank to sanction loans to \"Low Risk\" customers, following up with the latest information/data for the \"Medium Risk\" customers and reject the loan approval for \"High Risk\" customers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <b> Read the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>loan_amnt</th>\n",
       "      <th>funded_amnt</th>\n",
       "      <th>int_rate</th>\n",
       "      <th>installment</th>\n",
       "      <th>emp_length</th>\n",
       "      <th>annual_inc</th>\n",
       "      <th>loan_status</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1077501</td>\n",
       "      <td>5000</td>\n",
       "      <td>5000</td>\n",
       "      <td>10.65</td>\n",
       "      <td>162.87</td>\n",
       "      <td>10</td>\n",
       "      <td>24000.0</td>\n",
       "      <td>Low Risk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1077430</td>\n",
       "      <td>2500</td>\n",
       "      <td>2500</td>\n",
       "      <td>15.27</td>\n",
       "      <td>59.83</td>\n",
       "      <td>1</td>\n",
       "      <td>30000.0</td>\n",
       "      <td>High Risk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1077175</td>\n",
       "      <td>2400</td>\n",
       "      <td>2400</td>\n",
       "      <td>15.96</td>\n",
       "      <td>84.33</td>\n",
       "      <td>10</td>\n",
       "      <td>12252.0</td>\n",
       "      <td>Low Risk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1076863</td>\n",
       "      <td>10000</td>\n",
       "      <td>10000</td>\n",
       "      <td>13.49</td>\n",
       "      <td>339.31</td>\n",
       "      <td>10</td>\n",
       "      <td>49200.0</td>\n",
       "      <td>Low Risk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1075358</td>\n",
       "      <td>3000</td>\n",
       "      <td>3000</td>\n",
       "      <td>12.69</td>\n",
       "      <td>67.79</td>\n",
       "      <td>1</td>\n",
       "      <td>80000.0</td>\n",
       "      <td>Medium Risk</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        id  loan_amnt  funded_amnt  int_rate  installment  emp_length  \\\n",
       "0  1077501       5000         5000     10.65       162.87          10   \n",
       "1  1077430       2500         2500     15.27        59.83           1   \n",
       "2  1077175       2400         2400     15.96        84.33          10   \n",
       "3  1076863      10000        10000     13.49       339.31          10   \n",
       "4  1075358       3000         3000     12.69        67.79           1   \n",
       "\n",
       "   annual_inc  loan_status  \n",
       "0     24000.0     Low Risk  \n",
       "1     30000.0    High Risk  \n",
       "2     12252.0     Low Risk  \n",
       "3     49200.0     Low Risk  \n",
       "4     80000.0  Medium Risk  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Import the libraries\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "#Load the loan dataset\n",
    "\n",
    "# URL for the dataset\n",
    "url = 'https://github.com/vmahawar/data-science-datasets-collection/raw/main/home-loan.csv'\n",
    "\n",
    "# Load the dataset\n",
    "loan_data = pd.read_csv(url)\n",
    "\n",
    "# Print the first 5 rows to verify\n",
    "loan_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 38642 entries, 0 to 38641\n",
      "Data columns (total 8 columns):\n",
      " #   Column       Non-Null Count  Dtype  \n",
      "---  ------       --------------  -----  \n",
      " 0   id           38642 non-null  int64  \n",
      " 1   loan_amnt    38642 non-null  int64  \n",
      " 2   funded_amnt  38642 non-null  int64  \n",
      " 3   int_rate     38642 non-null  float64\n",
      " 4   installment  38642 non-null  float64\n",
      " 5   emp_length   38642 non-null  int64  \n",
      " 6   annual_inc   38642 non-null  float64\n",
      " 7   loan_status  38642 non-null  object \n",
      "dtypes: float64(3), int64(4), object(1)\n",
      "memory usage: 2.4+ MB\n"
     ]
    }
   ],
   "source": [
    "#Details of the dataset\n",
    "loan_data.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The dataset has the following columns: </b>\n",
    "\n",
    "<b> id : </b>Transaction ID use to identify each transaction uniquely <br>\n",
    "<b> loan_amnt : </b> Loan amount that was requested by the customer <br>\n",
    "<b> funded_amnt : </b>Amount that was sanctioned by the bank <br>\n",
    "<b> int_rate : </b> Interest rate offered on the loan amount <br>\n",
    "<b> installment : </b>Amount of money paid during each installment <br>\n",
    "<b> emp_length :  </b> Work experience (employment length of the customer)  <br>\n",
    "<b> annual_inc : </b>What is the annual income of the customer<br>\n",
    "<b> loan_status : </b> Classified as whether it is \"High Risk\", \"Low Risk\" and \"Medium Risk\" <br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>loan_amnt</th>\n",
       "      <th>funded_amnt</th>\n",
       "      <th>int_rate</th>\n",
       "      <th>installment</th>\n",
       "      <th>emp_length</th>\n",
       "      <th>annual_inc</th>\n",
       "      <th>loan_status</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1077501</td>\n",
       "      <td>5000</td>\n",
       "      <td>5000</td>\n",
       "      <td>10.65</td>\n",
       "      <td>162.87</td>\n",
       "      <td>10</td>\n",
       "      <td>24000.0</td>\n",
       "      <td>Low Risk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1077430</td>\n",
       "      <td>2500</td>\n",
       "      <td>2500</td>\n",
       "      <td>15.27</td>\n",
       "      <td>59.83</td>\n",
       "      <td>1</td>\n",
       "      <td>30000.0</td>\n",
       "      <td>High Risk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1077175</td>\n",
       "      <td>2400</td>\n",
       "      <td>2400</td>\n",
       "      <td>15.96</td>\n",
       "      <td>84.33</td>\n",
       "      <td>10</td>\n",
       "      <td>12252.0</td>\n",
       "      <td>Low Risk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1076863</td>\n",
       "      <td>10000</td>\n",
       "      <td>10000</td>\n",
       "      <td>13.49</td>\n",
       "      <td>339.31</td>\n",
       "      <td>10</td>\n",
       "      <td>49200.0</td>\n",
       "      <td>Low Risk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1075358</td>\n",
       "      <td>3000</td>\n",
       "      <td>3000</td>\n",
       "      <td>12.69</td>\n",
       "      <td>67.79</td>\n",
       "      <td>1</td>\n",
       "      <td>80000.0</td>\n",
       "      <td>Medium Risk</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        id  loan_amnt  funded_amnt  int_rate  installment  emp_length  \\\n",
       "0  1077501       5000         5000     10.65       162.87          10   \n",
       "1  1077430       2500         2500     15.27        59.83           1   \n",
       "2  1077175       2400         2400     15.96        84.33          10   \n",
       "3  1076863      10000        10000     13.49       339.31          10   \n",
       "4  1075358       3000         3000     12.69        67.79           1   \n",
       "\n",
       "   annual_inc  loan_status  \n",
       "0     24000.0     Low Risk  \n",
       "1     30000.0    High Risk  \n",
       "2     12252.0     Low Risk  \n",
       "3     49200.0     Low Risk  \n",
       "4     80000.0  Medium Risk  "
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Check the details of the dataset\n",
    "loan_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "loan_status\n",
       "Low Risk       32145\n",
       "High Risk       5399\n",
       "Medium Risk     1098\n",
       "Name: count, dtype: int64"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Distribution of the target variable\n",
    "loan_data['loan_status'].value_counts()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <b> Data Cleaning </b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "id             0\n",
       "loan_amnt      0\n",
       "funded_amnt    0\n",
       "int_rate       0\n",
       "installment    0\n",
       "emp_length     0\n",
       "annual_inc     0\n",
       "loan_status    0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Check for null values\n",
    "loan_data.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Check for duplicate values\n",
    "loan_data.duplicated().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <b> Feature Creation </b>\n",
    "<b> funded_amnt: </b> Percentage of amount sanctioned compared to the total loan amount. Higher the value, it states that the bank is positive in lending the loan to the customer. <br>\n",
    "<b> incToloan_perc: </b> Percentage of annual income when compared to the loan amount. Higher the value it states that the customer is more likely to pay back without defaulting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Adding new variables  \n",
    "# fund_perc variable represents the ratio of funded amount wrt loan amount\n",
    "loan_data['fund_perc']=loan_data['funded_amnt']/loan_data['loan_amnt']\n",
    "\n",
    "#incToloan_perc variable represent the ratio of annula income wrt loan amount\n",
    "loan_data['incToloan_perc'] = loan_data['annual_inc']/loan_data['loan_amnt']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>25%</th>\n",
       "      <th>50%</th>\n",
       "      <th>75%</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <td>38642.0</td>\n",
       "      <td>681040.355391</td>\n",
       "      <td>211304.549803</td>\n",
       "      <td>54734.000000</td>\n",
       "      <td>513435.0000</td>\n",
       "      <td>662770.500000</td>\n",
       "      <td>836491.250000</td>\n",
       "      <td>1.077501e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>loan_amnt</th>\n",
       "      <td>38642.0</td>\n",
       "      <td>11291.615988</td>\n",
       "      <td>7462.136215</td>\n",
       "      <td>500.000000</td>\n",
       "      <td>5500.0000</td>\n",
       "      <td>10000.000000</td>\n",
       "      <td>15000.000000</td>\n",
       "      <td>3.500000e+04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>funded_amnt</th>\n",
       "      <td>38642.0</td>\n",
       "      <td>11017.101211</td>\n",
       "      <td>7193.038828</td>\n",
       "      <td>500.000000</td>\n",
       "      <td>5500.0000</td>\n",
       "      <td>9950.000000</td>\n",
       "      <td>15000.000000</td>\n",
       "      <td>3.500000e+04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>int_rate</th>\n",
       "      <td>38642.0</td>\n",
       "      <td>12.052427</td>\n",
       "      <td>3.716705</td>\n",
       "      <td>5.420000</td>\n",
       "      <td>9.3200</td>\n",
       "      <td>11.860000</td>\n",
       "      <td>14.590000</td>\n",
       "      <td>2.459000e+01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>installment</th>\n",
       "      <td>38642.0</td>\n",
       "      <td>326.760477</td>\n",
       "      <td>209.143908</td>\n",
       "      <td>15.690000</td>\n",
       "      <td>168.4425</td>\n",
       "      <td>282.830000</td>\n",
       "      <td>434.397500</td>\n",
       "      <td>1.305190e+03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>emp_length</th>\n",
       "      <td>38642.0</td>\n",
       "      <td>5.092050</td>\n",
       "      <td>3.408338</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>2.0000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>1.000000e+01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>annual_inc</th>\n",
       "      <td>38642.0</td>\n",
       "      <td>69608.277211</td>\n",
       "      <td>64253.202240</td>\n",
       "      <td>4000.000000</td>\n",
       "      <td>41400.0000</td>\n",
       "      <td>60000.000000</td>\n",
       "      <td>83199.990000</td>\n",
       "      <td>6.000000e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fund_perc</th>\n",
       "      <td>38642.0</td>\n",
       "      <td>0.985571</td>\n",
       "      <td>0.070317</td>\n",
       "      <td>0.101250</td>\n",
       "      <td>1.0000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>incToloan_perc</th>\n",
       "      <td>38642.0</td>\n",
       "      <td>8.915950</td>\n",
       "      <td>13.845454</td>\n",
       "      <td>1.204819</td>\n",
       "      <td>4.0000</td>\n",
       "      <td>6.066667</td>\n",
       "      <td>10.016699</td>\n",
       "      <td>1.266667e+03</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  count           mean            std           min  \\\n",
       "id              38642.0  681040.355391  211304.549803  54734.000000   \n",
       "loan_amnt       38642.0   11291.615988    7462.136215    500.000000   \n",
       "funded_amnt     38642.0   11017.101211    7193.038828    500.000000   \n",
       "int_rate        38642.0      12.052427       3.716705      5.420000   \n",
       "installment     38642.0     326.760477     209.143908     15.690000   \n",
       "emp_length      38642.0       5.092050       3.408338      1.000000   \n",
       "annual_inc      38642.0   69608.277211   64253.202240   4000.000000   \n",
       "fund_perc       38642.0       0.985571       0.070317      0.101250   \n",
       "incToloan_perc  38642.0       8.915950      13.845454      1.204819   \n",
       "\n",
       "                        25%            50%            75%           max  \n",
       "id              513435.0000  662770.500000  836491.250000  1.077501e+06  \n",
       "loan_amnt         5500.0000   10000.000000   15000.000000  3.500000e+04  \n",
       "funded_amnt       5500.0000    9950.000000   15000.000000  3.500000e+04  \n",
       "int_rate             9.3200      11.860000      14.590000  2.459000e+01  \n",
       "installment        168.4425     282.830000     434.397500  1.305190e+03  \n",
       "emp_length           2.0000       4.000000       9.000000  1.000000e+01  \n",
       "annual_inc       41400.0000   60000.000000   83199.990000  6.000000e+06  \n",
       "fund_perc            1.0000       1.000000       1.000000  1.000000e+00  \n",
       "incToloan_perc       4.0000       6.066667      10.016699  1.266667e+03  "
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Understanding distribution of all the numerical variables in dataset\n",
    "loan_data.describe().transpose()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['id', 'loan_amnt', 'funded_amnt', 'int_rate', 'installment',\n",
       "       'emp_length', 'annual_inc', 'loan_status', 'fund_perc',\n",
       "       'incToloan_perc'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#column names\n",
    "loan_data.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <b> Train-Test Split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "# choosing all the numerical variables as independent variables (classifier can only take numerical input)\n",
    "# dropping two variable funded_amnt as we have created new variable transformation based on it \n",
    "X = loan_data.select_dtypes(np.number).drop(['id','funded_amnt'],axis=1)\n",
    "\n",
    "#Dependent variable representing status of the loan\n",
    "Y = loan_data['loan_status']\n",
    "\n",
    "#splitting the dataset in train and test datasets using a split ratio of 70:30\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X,Y,test_size=0.3,random_state=10)\n",
    "\n",
    "# standardizing all the variables using standard scaler\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "scaler = StandardScaler()\n",
    "X_train_scaled = scaler.fit_transform(X_train)\n",
    "X_test_scaled = scaler.transform(X_test)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <b> Model Building"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression \n",
    "from sklearn.multiclass import OneVsRestClassifier, OneVsOneClassifier\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.metrics import accuracy_score, classification_report\n",
    "from sklearn.model_selection import train_test_split\n",
    "import warnings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-1 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: black;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-1 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-1 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-1 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: block;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-1 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-1 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-1 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-1 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-1 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 1ex;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-1 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>OneVsRestClassifier(estimator=LogisticRegression())</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" ><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;&nbsp;OneVsRestClassifier<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.multiclass.OneVsRestClassifier.html\">?<span>Documentation for OneVsRestClassifier</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></label><div class=\"sk-toggleable__content fitted\"><pre>OneVsRestClassifier(estimator=LogisticRegression())</pre></div> </div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" ><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">estimator: LogisticRegression</label><div class=\"sk-toggleable__content fitted\"><pre>LogisticRegression()</pre></div> </div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-3\" type=\"checkbox\" ><label for=\"sk-estimator-id-3\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;LogisticRegression<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.linear_model.LogisticRegression.html\">?<span>Documentation for LogisticRegression</span></a></label><div class=\"sk-toggleable__content fitted\"><pre>LogisticRegression()</pre></div> </div></div></div></div></div></div></div></div></div>"
      ],
      "text/plain": [
       "OneVsRestClassifier(estimator=LogisticRegression())"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Building a classification model using one vs rest method\n",
    "LR = LogisticRegression()\n",
    "oneVsrest = OneVsRestClassifier(LR)\n",
    "\n",
    "# Fitting the model with training data\n",
    "oneVsrest.fit(X_train_scaled,y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <b>Step 3 : </b>\n",
    "<b> Model Prediction </b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test set accuarcy: 82.88622444578625%\n",
      "\n",
      "\n",
      "classification_report:\n",
      "\n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "   High Risk       0.56      0.01      0.01      1654\n",
      "    Low Risk       0.83      1.00      0.91      9607\n",
      " Medium Risk       0.20      0.00      0.01       332\n",
      "\n",
      "    accuracy                           0.83     11593\n",
      "   macro avg       0.53      0.34      0.31     11593\n",
      "weighted avg       0.77      0.83      0.75     11593\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Making a prediction on the test set\n",
    "prediction_oneVsrest = oneVsrest.predict(X_test_scaled)\n",
    "   \n",
    "# Evaluating the model\n",
    "print(f\"Test set accuarcy: {accuracy_score(y_test,prediction_oneVsrest)*100}%\\n\\n\")\n",
    "print(f\"classification_report:\\n\\n {classification_report(y_test,prediction_oneVsrest)}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b> Accuracy : </b> Accuracy is the most intuitive performance measure and it is simply a ratio of correctly predicted observation to the total observations.The formula is given as: <br>\n",
    "<b> *Accuracy = True Positives + True Negatives/True Positives+False Positives+False Negatives+True Positives* </b> <br> <br>\n",
    "<b> Precision : </b> The quality of being exact and refers to how close two or more measurements are to each other, regardless of whether those measurements are accurate or not. The formula is : <br>\n",
    "<b> *Precision = True Positives / (True Positives + False Positives)* </b> <br> <br>\n",
    "<b> Recall : </b> It is calculated as the number of true positives divided by the total number of true positives and false negatives. The result is a value between 0.0 for no recall and 1.0 for full or perfect recall. The formula is : <br>\n",
    "<b> *Recall = True Positives / (True Positives + False Negatives)* </b> <br> <br>\n",
    "<b> F1 score : </b> F1 Score is the weighted average of Precision and Recall. Therefore, this score takes both false positives and false negatives into account.The highest possible value of an F-score is 1.0, indicating perfect precision and recall, and the lowest possible value is 0, if either the precision or the recall is zero. The formula is : <br>\n",
    "<b> *F1 score = 2\\*((precision\\*recall)/(precision+recall))* </b> <br> <br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <b>Analysing the probabilties and classification values </b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>loan_amnt</th>\n",
       "      <th>int_rate</th>\n",
       "      <th>installment</th>\n",
       "      <th>emp_length</th>\n",
       "      <th>annual_inc</th>\n",
       "      <th>fund_perc</th>\n",
       "      <th>incToloan_perc</th>\n",
       "      <th>Scaled_features</th>\n",
       "      <th>Actual</th>\n",
       "      <th>prediction_oneVsrest</th>\n",
       "      <th>prob_pred_oneVsrest</th>\n",
       "      <th>prob_pred_oneVsrest_highRisk</th>\n",
       "      <th>prob_pred_oneVsrest_lowRisk</th>\n",
       "      <th>prob_pred_oneVsrest_mediumRisk</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>17614</th>\n",
       "      <td>25000</td>\n",
       "      <td>16.77</td>\n",
       "      <td>888.46</td>\n",
       "      <td>1</td>\n",
       "      <td>104000.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4.160000</td>\n",
       "      <td>[1.8371293187309605, 1.2736784709856193, 2.678...</td>\n",
       "      <td>Low Risk</td>\n",
       "      <td>Low Risk</td>\n",
       "      <td>[0.13534193651914989, 0.8645659803810904, 9.20...</td>\n",
       "      <td>0.135342</td>\n",
       "      <td>0.864566</td>\n",
       "      <td>0.000092</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3728</th>\n",
       "      <td>3600</td>\n",
       "      <td>7.51</td>\n",
       "      <td>112.00</td>\n",
       "      <td>2</td>\n",
       "      <td>31800.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>8.833333</td>\n",
       "      <td>[-1.0230547720561989, -1.2180942746831072, -1....</td>\n",
       "      <td>Low Risk</td>\n",
       "      <td>Low Risk</td>\n",
       "      <td>[0.07647973946862915, 0.9198006937311205, 0.00...</td>\n",
       "      <td>0.076480</td>\n",
       "      <td>0.919801</td>\n",
       "      <td>0.003720</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37880</th>\n",
       "      <td>9000</td>\n",
       "      <td>11.97</td>\n",
       "      <td>298.80</td>\n",
       "      <td>2</td>\n",
       "      <td>57500.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>6.388889</td>\n",
       "      <td>[-0.30132607625009317, -0.017953189836182465, ...</td>\n",
       "      <td>Low Risk</td>\n",
       "      <td>Low Risk</td>\n",
       "      <td>[0.11707833369961512, 0.8793550460122722, 0.00...</td>\n",
       "      <td>0.117078</td>\n",
       "      <td>0.879355</td>\n",
       "      <td>0.003567</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33951</th>\n",
       "      <td>17000</td>\n",
       "      <td>12.18</td>\n",
       "      <td>566.10</td>\n",
       "      <td>10</td>\n",
       "      <td>63504.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>3.735529</td>\n",
       "      <td>[0.7679016212404337, 0.03855569532477113, 1.14...</td>\n",
       "      <td>High Risk</td>\n",
       "      <td>Low Risk</td>\n",
       "      <td>[0.12149736787933875, 0.8775032932056606, 0.00...</td>\n",
       "      <td>0.121497</td>\n",
       "      <td>0.877503</td>\n",
       "      <td>0.000999</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>982</th>\n",
       "      <td>5500</td>\n",
       "      <td>14.27</td>\n",
       "      <td>188.70</td>\n",
       "      <td>5</td>\n",
       "      <td>24000.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4.363636</td>\n",
       "      <td>[-0.7691131939021987, 0.6009536476409307, -0.6...</td>\n",
       "      <td>Low Risk</td>\n",
       "      <td>Low Risk</td>\n",
       "      <td>[0.1950791560703943, 0.7912356920418981, 0.013...</td>\n",
       "      <td>0.195079</td>\n",
       "      <td>0.791236</td>\n",
       "      <td>0.013685</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       loan_amnt  int_rate  installment  emp_length  annual_inc  fund_perc  \\\n",
       "17614      25000     16.77       888.46           1    104000.0        1.0   \n",
       "3728        3600      7.51       112.00           2     31800.0        1.0   \n",
       "37880       9000     11.97       298.80           2     57500.0        1.0   \n",
       "33951      17000     12.18       566.10          10     63504.0        1.0   \n",
       "982         5500     14.27       188.70           5     24000.0        1.0   \n",
       "\n",
       "       incToloan_perc                                    Scaled_features  \\\n",
       "17614        4.160000  [1.8371293187309605, 1.2736784709856193, 2.678...   \n",
       "3728         8.833333  [-1.0230547720561989, -1.2180942746831072, -1....   \n",
       "37880        6.388889  [-0.30132607625009317, -0.017953189836182465, ...   \n",
       "33951        3.735529  [0.7679016212404337, 0.03855569532477113, 1.14...   \n",
       "982          4.363636  [-0.7691131939021987, 0.6009536476409307, -0.6...   \n",
       "\n",
       "          Actual prediction_oneVsrest  \\\n",
       "17614   Low Risk             Low Risk   \n",
       "3728    Low Risk             Low Risk   \n",
       "37880   Low Risk             Low Risk   \n",
       "33951  High Risk             Low Risk   \n",
       "982     Low Risk             Low Risk   \n",
       "\n",
       "                                     prob_pred_oneVsrest  \\\n",
       "17614  [0.13534193651914989, 0.8645659803810904, 9.20...   \n",
       "3728   [0.07647973946862915, 0.9198006937311205, 0.00...   \n",
       "37880  [0.11707833369961512, 0.8793550460122722, 0.00...   \n",
       "33951  [0.12149736787933875, 0.8775032932056606, 0.00...   \n",
       "982    [0.1950791560703943, 0.7912356920418981, 0.013...   \n",
       "\n",
       "       prob_pred_oneVsrest_highRisk  prob_pred_oneVsrest_lowRisk  \\\n",
       "17614                      0.135342                     0.864566   \n",
       "3728                       0.076480                     0.919801   \n",
       "37880                      0.117078                     0.879355   \n",
       "33951                      0.121497                     0.877503   \n",
       "982                        0.195079                     0.791236   \n",
       "\n",
       "       prob_pred_oneVsrest_mediumRisk  \n",
       "17614                        0.000092  \n",
       "3728                         0.003720  \n",
       "37880                        0.003567  \n",
       "33951                        0.000999  \n",
       "982                          0.013685  "
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Adding followig variables to the test dataset\n",
    "\n",
    "#Scaled feature array\n",
    "X_test['Scaled_features'] = X_test_scaled.tolist()\n",
    "\n",
    "#Actual target variable\n",
    "X_test['Actual'] = y_test\n",
    "\n",
    "#OnevsRest target prediction\n",
    "X_test['prediction_oneVsrest'] = prediction_oneVsrest\n",
    "\n",
    "#OnevsRest probability prediction\n",
    "X_test['prob_pred_oneVsrest'] = oneVsrest.predict_proba(X_test_scaled).tolist()\n",
    "\n",
    "#OnevsRest individual class prediction probabilities\n",
    "X_test['prob_pred_oneVsrest_highRisk'] = oneVsrest.predict_proba(X_test_scaled)[:,0].tolist()\n",
    "X_test['prob_pred_oneVsrest_lowRisk'] = oneVsrest.predict_proba(X_test_scaled)[:,1].tolist()\n",
    "X_test['prob_pred_oneVsrest_mediumRisk'] = oneVsrest.predict_proba(X_test_scaled)[:,2].tolist()\n",
    "\n",
    "X_test.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <b>Display the coefficient and intercept values for each Logistic Regression model </b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classes for which individual models are created:\n",
      "['High Risk' 'Low Risk' 'Medium Risk']\n",
      "\n",
      "Coefficient matrix and intercept values for each model:\n",
      "\n",
      "Class: High Risk\n",
      "{'penalty': 'l2', 'dual': False, 'tol': 0.0001, 'C': 1.0, 'fit_intercept': True, 'intercept_scaling': 1, 'class_weight': None, 'random_state': None, 'solver': 'lbfgs', 'max_iter': 100, 'multi_class': 'deprecated', 'verbose': 0, 'warm_start': False, 'n_jobs': None, 'l1_ratio': None, 'n_features_in_': 7, 'classes_': array([0, 1], dtype=int32), 'n_iter_': array([16]), 'coef_': array([[ 0.43194235,  0.56154559, -0.41518276,  0.04873909, -0.37666534,\n",
      "         0.04237839, -0.00830218]]), 'intercept_': array([-1.9715301])}\n",
      "Shape of Coefficient Matrix: (1, 7)\n",
      "Coefficients: [ 0.43194235  0.56154559 -0.41518276  0.04873909 -0.37666534  0.04237839\n",
      " -0.00830218]\n",
      "Intercept: -1.97153009953857\n",
      "\n",
      "Class: Low Risk\n",
      "{'penalty': 'l2', 'dual': False, 'tol': 0.0001, 'C': 1.0, 'fit_intercept': True, 'intercept_scaling': 1, 'class_weight': None, 'random_state': None, 'solver': 'lbfgs', 'max_iter': 100, 'multi_class': 'deprecated', 'verbose': 0, 'warm_start': False, 'n_jobs': None, 'l1_ratio': None, 'n_features_in_': 7, 'classes_': array([0, 1], dtype=int32), 'n_iter_': array([12]), 'coef_': array([[-1.16342173, -0.61894016,  1.13406584, -0.07002787,  0.28239313,\n",
      "        -0.2107662 ,  0.08026597]]), 'intercept_': array([1.80459894])}\n",
      "Shape of Coefficient Matrix: (1, 7)\n",
      "Coefficients: [-1.16342173 -0.61894016  1.13406584 -0.07002787  0.28239313 -0.2107662\n",
      "  0.08026597]\n",
      "Intercept: 1.8045989398705697\n",
      "\n",
      "Class: Medium Risk\n",
      "{'penalty': 'l2', 'dual': False, 'tol': 0.0001, 'C': 1.0, 'fit_intercept': True, 'intercept_scaling': 1, 'class_weight': None, 'random_state': None, 'solver': 'lbfgs', 'max_iter': 100, 'multi_class': 'deprecated', 'verbose': 0, 'warm_start': False, 'n_jobs': None, 'l1_ratio': None, 'n_features_in_': 7, 'classes_': array([0, 1], dtype=int32), 'n_iter_': array([20]), 'coef_': array([[ 4.80593383,  0.98060216, -5.46911149,  0.15603937,  0.17847509,\n",
      "         1.28355812, -0.62272178]]), 'intercept_': array([-5.06916976])}\n",
      "Shape of Coefficient Matrix: (1, 7)\n",
      "Coefficients: [ 4.80593383  0.98060216 -5.46911149  0.15603937  0.17847509  1.28355812\n",
      " -0.62272178]\n",
      "Intercept: -5.069169762889693\n"
     ]
    }
   ],
   "source": [
    "# Classes for which individual models are created\n",
    "print(\"Classes for which individual models are created:\")\n",
    "print(oneVsrest.classes_)\n",
    "\n",
    "# print(\"\\n\\nThe estimators are:\")\n",
    "# print(oneVsrest.estimators_)\n",
    "\n",
    "# Coeff_array = oneVsrest.\n",
    "\n",
    "Coeff_array = []\n",
    "intercept_arr = []\n",
    "\n",
    "# Coefficient matrix and intercept values for all the models created\n",
    "print(\"\\nCoefficient matrix and intercept values for each model:\")\n",
    "for idx, model in enumerate(oneVsrest.estimators_):\n",
    "    print(f\"\\nClass: {oneVsrest.classes_[idx]}\")\n",
    "    print(model.__dict__)    \n",
    "    print(\"Shape of Coefficient Matrix:\", model.coef_.shape)\n",
    "    print(\"Coefficients:\", model.coef_[0])\n",
    "    Coeff_array.append(model.coef_[0])\n",
    "    print(\"Intercept:\", model.intercept_[0])\n",
    "    intercept_arr.append(model.intercept_[0])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <b>Analyse probability values for one test sample</b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.13534193651914989, 0.8645659803810904, 9.208309975965633e-05]\n"
     ]
    }
   ],
   "source": [
    "print(X_test.iloc[0]['prob_pred_oneVsrest'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <b>Understand the mathematics and calculations inside the Model </b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(0.13917355386605684, 0.8890423998351974, 9.469003159076782e-05)\n",
      "[1.35341937e-01 8.64565980e-01 9.20830998e-05]\n"
     ]
    }
   ],
   "source": [
    "#Below example demonstrate the calculation of prediction probability for a observation in the dataset\n",
    "#The demonstartion uses coefficient values of each model for the calculation\n",
    "\n",
    "# Choose the first observation\n",
    "arr = X_test.iloc[0]['Scaled_features']\n",
    "\n",
    "# Class calculates the log of odds value for a given class\n",
    "def y(class_cat, input_arr):\n",
    "    val = 0\n",
    "    for i in range(7):\n",
    "        val += Coeff_array[class_cat - 1][i]*input_arr[i]\n",
    "        \n",
    "    return val\n",
    "\n",
    "# Calculates the probability values given the log of odds\n",
    "def z(a):\n",
    "    return (1/(1+np.exp(-a)))\n",
    "\n",
    "#Non-normalized probability of all the classes\n",
    "Z_non_norm = z((intercept_arr[0] +y(1,arr))),z(intercept_arr[1] + y(2,arr)),z(intercept_arr[2] +y(3,arr))\n",
    "\n",
    "\n",
    "#Normalized probability of all the classes\n",
    "Z_norm = Z_non_norm/sum(Z_non_norm)\n",
    "\n",
    "print(Z_non_norm)\n",
    "print(Z_norm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <b> Building Logistic Regression Model and using it in One vs One Classifier </b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Classification using OnevsOne method\n",
    "LR1 = LogisticRegression()\n",
    "oneVsone = OneVsOneClassifier(LR1)\n",
    "\n",
    "# Fitting the model with training data\n",
    "oneVsone.fit(X_train_scaled,y_train)\n",
    "   \n",
    "# Making a prediction on the test set\n",
    "prediction_oneVsone = oneVsone.predict(X_test_scaled)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <b> Model Prediction </b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test set accuracy 82.86034676097646 %\n",
      "\n",
      " \n",
      "Classification accuracy report: \n",
      "\n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "   High Risk       0.75      0.00      0.01      1654\n",
      "    Low Risk       0.83      1.00      0.91      9607\n",
      " Medium Risk       0.15      0.01      0.01       332\n",
      "\n",
      "    accuracy                           0.83     11593\n",
      "   macro avg       0.58      0.34      0.31     11593\n",
      "weighted avg       0.80      0.83      0.75     11593\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Evaluating the model\n",
    "print(f\"Test set accuracy {accuracy_score(y_test, prediction_oneVsone) * 100} %\\n\\n \")\n",
    "print(f\"Classification accuracy report: \\n\\n {classification_report(y_test,prediction_oneVsone)}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <b>Analysing the probabilties and classification values </b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>loan_amnt</th>\n",
       "      <th>int_rate</th>\n",
       "      <th>installment</th>\n",
       "      <th>emp_length</th>\n",
       "      <th>annual_inc</th>\n",
       "      <th>fund_perc</th>\n",
       "      <th>incToloan_perc</th>\n",
       "      <th>Scaled_features</th>\n",
       "      <th>Actual</th>\n",
       "      <th>prediction_oneVsrest</th>\n",
       "      <th>prob_pred_oneVsrest</th>\n",
       "      <th>prob_pred_oneVsrest_highRisk</th>\n",
       "      <th>prob_pred_oneVsrest_lowRisk</th>\n",
       "      <th>prob_pred_oneVsrest_mediumRisk</th>\n",
       "      <th>prediction_oneVsone</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>17614</th>\n",
       "      <td>25000</td>\n",
       "      <td>16.77</td>\n",
       "      <td>888.46</td>\n",
       "      <td>1</td>\n",
       "      <td>104000.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4.160000</td>\n",
       "      <td>[1.8371293187309605, 1.2736784709856193, 2.678...</td>\n",
       "      <td>Low Risk</td>\n",
       "      <td>Low Risk</td>\n",
       "      <td>[0.13534193651914989, 0.8645659803810904, 9.20...</td>\n",
       "      <td>0.135342</td>\n",
       "      <td>0.864566</td>\n",
       "      <td>0.000092</td>\n",
       "      <td>Low Risk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3728</th>\n",
       "      <td>3600</td>\n",
       "      <td>7.51</td>\n",
       "      <td>112.00</td>\n",
       "      <td>2</td>\n",
       "      <td>31800.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>8.833333</td>\n",
       "      <td>[-1.0230547720561989, -1.2180942746831072, -1....</td>\n",
       "      <td>Low Risk</td>\n",
       "      <td>Low Risk</td>\n",
       "      <td>[0.07647973946862915, 0.9198006937311205, 0.00...</td>\n",
       "      <td>0.076480</td>\n",
       "      <td>0.919801</td>\n",
       "      <td>0.003720</td>\n",
       "      <td>Low Risk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37880</th>\n",
       "      <td>9000</td>\n",
       "      <td>11.97</td>\n",
       "      <td>298.80</td>\n",
       "      <td>2</td>\n",
       "      <td>57500.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>6.388889</td>\n",
       "      <td>[-0.30132607625009317, -0.017953189836182465, ...</td>\n",
       "      <td>Low Risk</td>\n",
       "      <td>Low Risk</td>\n",
       "      <td>[0.11707833369961512, 0.8793550460122722, 0.00...</td>\n",
       "      <td>0.117078</td>\n",
       "      <td>0.879355</td>\n",
       "      <td>0.003567</td>\n",
       "      <td>Low Risk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33951</th>\n",
       "      <td>17000</td>\n",
       "      <td>12.18</td>\n",
       "      <td>566.10</td>\n",
       "      <td>10</td>\n",
       "      <td>63504.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>3.735529</td>\n",
       "      <td>[0.7679016212404337, 0.03855569532477113, 1.14...</td>\n",
       "      <td>High Risk</td>\n",
       "      <td>Low Risk</td>\n",
       "      <td>[0.12149736787933875, 0.8775032932056606, 0.00...</td>\n",
       "      <td>0.121497</td>\n",
       "      <td>0.877503</td>\n",
       "      <td>0.000999</td>\n",
       "      <td>Low Risk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>982</th>\n",
       "      <td>5500</td>\n",
       "      <td>14.27</td>\n",
       "      <td>188.70</td>\n",
       "      <td>5</td>\n",
       "      <td>24000.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4.363636</td>\n",
       "      <td>[-0.7691131939021987, 0.6009536476409307, -0.6...</td>\n",
       "      <td>Low Risk</td>\n",
       "      <td>Low Risk</td>\n",
       "      <td>[0.1950791560703943, 0.7912356920418981, 0.013...</td>\n",
       "      <td>0.195079</td>\n",
       "      <td>0.791236</td>\n",
       "      <td>0.013685</td>\n",
       "      <td>Low Risk</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       loan_amnt  int_rate  installment  emp_length  annual_inc  fund_perc  \\\n",
       "17614      25000     16.77       888.46           1    104000.0        1.0   \n",
       "3728        3600      7.51       112.00           2     31800.0        1.0   \n",
       "37880       9000     11.97       298.80           2     57500.0        1.0   \n",
       "33951      17000     12.18       566.10          10     63504.0        1.0   \n",
       "982         5500     14.27       188.70           5     24000.0        1.0   \n",
       "\n",
       "       incToloan_perc                                    Scaled_features  \\\n",
       "17614        4.160000  [1.8371293187309605, 1.2736784709856193, 2.678...   \n",
       "3728         8.833333  [-1.0230547720561989, -1.2180942746831072, -1....   \n",
       "37880        6.388889  [-0.30132607625009317, -0.017953189836182465, ...   \n",
       "33951        3.735529  [0.7679016212404337, 0.03855569532477113, 1.14...   \n",
       "982          4.363636  [-0.7691131939021987, 0.6009536476409307, -0.6...   \n",
       "\n",
       "          Actual prediction_oneVsrest  \\\n",
       "17614   Low Risk             Low Risk   \n",
       "3728    Low Risk             Low Risk   \n",
       "37880   Low Risk             Low Risk   \n",
       "33951  High Risk             Low Risk   \n",
       "982     Low Risk             Low Risk   \n",
       "\n",
       "                                     prob_pred_oneVsrest  \\\n",
       "17614  [0.13534193651914989, 0.8645659803810904, 9.20...   \n",
       "3728   [0.07647973946862915, 0.9198006937311205, 0.00...   \n",
       "37880  [0.11707833369961512, 0.8793550460122722, 0.00...   \n",
       "33951  [0.12149736787933875, 0.8775032932056606, 0.00...   \n",
       "982    [0.1950791560703943, 0.7912356920418981, 0.013...   \n",
       "\n",
       "       prob_pred_oneVsrest_highRisk  prob_pred_oneVsrest_lowRisk  \\\n",
       "17614                      0.135342                     0.864566   \n",
       "3728                       0.076480                     0.919801   \n",
       "37880                      0.117078                     0.879355   \n",
       "33951                      0.121497                     0.877503   \n",
       "982                        0.195079                     0.791236   \n",
       "\n",
       "       prob_pred_oneVsrest_mediumRisk prediction_oneVsone  \n",
       "17614                        0.000092            Low Risk  \n",
       "3728                         0.003720            Low Risk  \n",
       "37880                        0.003567            Low Risk  \n",
       "33951                        0.000999            Low Risk  \n",
       "982                          0.013685            Low Risk  "
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Adding followig variables to the test dataset\n",
    "\n",
    "#OnevsOne target prediction\n",
    "X_test['prediction_oneVsone'] = prediction_oneVsone\n",
    "X_test.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <b>Display the parameters and coefficients for each Logistic Regression model </b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['High Risk', 'Low Risk', 'Medium Risk'], dtype=object)"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#OneVsOne.classes_\n",
    "oneVsone.classes_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(LogisticRegression(), LogisticRegression(), LogisticRegression())"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "oneVsone.estimators_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classes for which individual models are created:\n",
      "['High Risk' 'Low Risk' 'Medium Risk']\n",
      "(LogisticRegression(), LogisticRegression(), LogisticRegression())\n",
      "\n",
      "Coefficient matrix and intercept values for each model:\n",
      "\n",
      "Class: High Risk\n",
      "{'penalty': 'l2', 'dual': False, 'tol': 0.0001, 'C': 1.0, 'fit_intercept': True, 'intercept_scaling': 1, 'class_weight': None, 'random_state': None, 'solver': 'lbfgs', 'max_iter': 100, 'multi_class': 'deprecated', 'verbose': 0, 'warm_start': False, 'n_jobs': None, 'l1_ratio': None, 'n_features_in_': 7, 'classes_': array([0, 1]), 'n_iter_': array([14]), 'coef_': array([[-0.61168342, -0.59558283,  0.5796362 , -0.05686887,  0.37417748,\n",
      "        -0.0771769 ,  0.01222523]]), 'intercept_': array([1.93419542])}\n",
      "Shape of Coefficient Matrix: (1, 7)\n",
      "Coefficients: [-0.61168342 -0.59558283  0.5796362  -0.05686887  0.37417748 -0.0771769\n",
      "  0.01222523]\n",
      "Intercept: 1.9341954159943808\n",
      "\n",
      "Class: Low Risk\n",
      "{'penalty': 'l2', 'dual': False, 'tol': 0.0001, 'C': 1.0, 'fit_intercept': True, 'intercept_scaling': 1, 'class_weight': None, 'random_state': None, 'solver': 'lbfgs', 'max_iter': 100, 'multi_class': 'deprecated', 'verbose': 0, 'warm_start': False, 'n_jobs': None, 'l1_ratio': None, 'n_features_in_': 7, 'classes_': array([0, 1]), 'n_iter_': array([20]), 'coef_': array([[ 4.43634761,  0.58723427, -5.04702209,  0.10963985,  0.40801233,\n",
      "         1.11406325, -0.52365189]]), 'intercept_': array([-3.12729465])}\n",
      "Shape of Coefficient Matrix: (1, 7)\n",
      "Coefficients: [ 4.43634761  0.58723427 -5.04702209  0.10963985  0.40801233  1.11406325\n",
      " -0.52365189]\n",
      "Intercept: -3.1272946490988516\n",
      "\n",
      "Class: Medium Risk\n",
      "{'penalty': 'l2', 'dual': False, 'tol': 0.0001, 'C': 1.0, 'fit_intercept': True, 'intercept_scaling': 1, 'class_weight': None, 'random_state': None, 'solver': 'lbfgs', 'max_iter': 100, 'multi_class': 'deprecated', 'verbose': 0, 'warm_start': False, 'n_jobs': None, 'l1_ratio': None, 'n_features_in_': 7, 'classes_': array([0, 1]), 'n_iter_': array([20]), 'coef_': array([[ 4.88485316,  1.09452991, -5.53316995,  0.17442395,  0.0947159 ,\n",
      "         1.3001182 , -0.64723781]]), 'intercept_': array([-4.90749505])}\n",
      "Shape of Coefficient Matrix: (1, 7)\n",
      "Coefficients: [ 4.88485316  1.09452991 -5.53316995  0.17442395  0.0947159   1.3001182\n",
      " -0.64723781]\n",
      "Intercept: -4.907495049867593\n"
     ]
    }
   ],
   "source": [
    "# Classes for which individual models are created\n",
    "print(\"Classes for which individual models are created:\")\n",
    "print(oneVsone.classes_)\n",
    "\n",
    "\n",
    "print(oneVsone.estimators_)\n",
    "\n",
    "Coeff_array = []\n",
    "intercept_arr = []\n",
    "\n",
    "# Coefficient matrix and intercept values for all the models created\n",
    "print(\"\\nCoefficient matrix and intercept values for each model:\")\n",
    "for idx, model in enumerate(oneVsone.estimators_):\n",
    "    print(f\"\\nClass: {oneVsone.classes_[idx]}\")\n",
    "    print(model.__dict__)\n",
    "    print(\"Shape of Coefficient Matrix:\", model.coef_.shape)\n",
    "    print(\"Coefficients:\", model.coef_[0])\n",
    "    Coeff_array.append(model.coef_[0])\n",
    "    print(\"Intercept:\", model.intercept_[0])\n",
    "    intercept_arr.append(model.intercept_[0])\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".ds_venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}