data_pre_processing_project.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ecf4b28d-8b4b-45ac-af8e-a908fc3e3d66",
   "metadata": {},
   "source": [
    "# **Риск-скоринг заемщиков** <a id='начало'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84479247-4ebd-4293-a957-fbaaf618899c",
   "metadata": {},
   "source": [
    "Заказчик — кредитный отдел банка. Нужно разобраться, влияет ли семейное положение и количество детей клиента на факт погашения кредита в срок. Входные данные от банка — статистика о платёжеспособности клиентов.\n",
    "\n",
    "**Цель иследования:**\n",
    "* Есть ли зависимость между количеством детей и возвратом кредита в срок?\n",
    "* Есть ли зависимость между семейным положением и возвратом кредита в срок?\n",
    "* Есть ли зависимость между уровнем дохода и возвратом кредита в срок?\n",
    "* Как разные цели кредита влияют на его возврат в срок?\n",
    "\n",
    "**Ход иследования:**\n",
    "Полученны данные из data.csv, котоый хранит статистику о платёжеспособности кредитополучателей. Прежде чем ответить на вопросы поставленные Заказчиком нужно проверить качество полученных данных."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9127fcb7-832b-4d11-97d2-df2c86471e67",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "## **Обзор данных** <a id='обзор'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "991184b8-bdf0-4fa9-adc7-e57c4a283fc6",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "### Импорт библиотек"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "82ff1a39-5d23-4566-87f1-b6a7aceb2f6a",
   "metadata": {},
   "outputs": [],
   "source": [
    "#импортируем библиотеки \n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "268d1fc7-cec2-49db-a1a4-425b97c98e69",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "### Загрузка данных из data.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "da0f9d69-ab9b-41fe-837d-794eb503703b",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "#читаем файл и сохраняем в df\n",
    "df = pd.read_csv('C://Users//sahin//Desktop//datasets//data.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7dafe23-bfad-4a0c-98f7-58d79937df0f",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "### **Обзор данных**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "cd834644-4950-439f-a20a-9d71e6241907",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>children</th>\n",
       "      <th>days_employed</th>\n",
       "      <th>dob_years</th>\n",
       "      <th>education</th>\n",
       "      <th>education_id</th>\n",
       "      <th>family_status</th>\n",
       "      <th>family_status_id</th>\n",
       "      <th>gender</th>\n",
       "      <th>income_type</th>\n",
       "      <th>debt</th>\n",
       "      <th>total_income</th>\n",
       "      <th>purpose</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>-8437.673028</td>\n",
       "      <td>42</td>\n",
       "      <td>высшее</td>\n",
       "      <td>0</td>\n",
       "      <td>женат / замужем</td>\n",
       "      <td>0</td>\n",
       "      <td>F</td>\n",
       "      <td>сотрудник</td>\n",
       "      <td>0</td>\n",
       "      <td>253875.639453</td>\n",
       "      <td>покупка жилья</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>-4024.803754</td>\n",
       "      <td>36</td>\n",
       "      <td>среднее</td>\n",
       "      <td>1</td>\n",
       "      <td>женат / замужем</td>\n",
       "      <td>0</td>\n",
       "      <td>F</td>\n",
       "      <td>сотрудник</td>\n",
       "      <td>0</td>\n",
       "      <td>112080.014102</td>\n",
       "      <td>приобретение автомобиля</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>-5623.422610</td>\n",
       "      <td>33</td>\n",
       "      <td>Среднее</td>\n",
       "      <td>1</td>\n",
       "      <td>женат / замужем</td>\n",
       "      <td>0</td>\n",
       "      <td>M</td>\n",
       "      <td>сотрудник</td>\n",
       "      <td>0</td>\n",
       "      <td>145885.952297</td>\n",
       "      <td>покупка жилья</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>-4124.747207</td>\n",
       "      <td>32</td>\n",
       "      <td>среднее</td>\n",
       "      <td>1</td>\n",
       "      <td>женат / замужем</td>\n",
       "      <td>0</td>\n",
       "      <td>M</td>\n",
       "      <td>сотрудник</td>\n",
       "      <td>0</td>\n",
       "      <td>267628.550329</td>\n",
       "      <td>дополнительное образование</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>340266.072047</td>\n",
       "      <td>53</td>\n",
       "      <td>среднее</td>\n",
       "      <td>1</td>\n",
       "      <td>гражданский брак</td>\n",
       "      <td>1</td>\n",
       "      <td>F</td>\n",
       "      <td>пенсионер</td>\n",
       "      <td>0</td>\n",
       "      <td>158616.077870</td>\n",
       "      <td>сыграть свадьбу</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21520</th>\n",
       "      <td>1</td>\n",
       "      <td>-4529.316663</td>\n",
       "      <td>43</td>\n",
       "      <td>среднее</td>\n",
       "      <td>1</td>\n",
       "      <td>гражданский брак</td>\n",
       "      <td>1</td>\n",
       "      <td>F</td>\n",
       "      <td>компаньон</td>\n",
       "      <td>0</td>\n",
       "      <td>224791.862382</td>\n",
       "      <td>операции с жильем</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21521</th>\n",
       "      <td>0</td>\n",
       "      <td>343937.404131</td>\n",
       "      <td>67</td>\n",
       "      <td>среднее</td>\n",
       "      <td>1</td>\n",
       "      <td>женат / замужем</td>\n",
       "      <td>0</td>\n",
       "      <td>F</td>\n",
       "      <td>пенсионер</td>\n",
       "      <td>0</td>\n",
       "      <td>155999.806512</td>\n",
       "      <td>сделка с автомобилем</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21522</th>\n",
       "      <td>1</td>\n",
       "      <td>-2113.346888</td>\n",
       "      <td>38</td>\n",
       "      <td>среднее</td>\n",
       "      <td>1</td>\n",
       "      <td>гражданский брак</td>\n",
       "      <td>1</td>\n",
       "      <td>M</td>\n",
       "      <td>сотрудник</td>\n",
       "      <td>1</td>\n",
       "      <td>89672.561153</td>\n",
       "      <td>недвижимость</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21523</th>\n",
       "      <td>3</td>\n",
       "      <td>-3112.481705</td>\n",
       "      <td>38</td>\n",
       "      <td>среднее</td>\n",
       "      <td>1</td>\n",
       "      <td>женат / замужем</td>\n",
       "      <td>0</td>\n",
       "      <td>M</td>\n",
       "      <td>сотрудник</td>\n",
       "      <td>1</td>\n",
       "      <td>244093.050500</td>\n",
       "      <td>на покупку своего автомобиля</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21524</th>\n",
       "      <td>2</td>\n",
       "      <td>-1984.507589</td>\n",
       "      <td>40</td>\n",
       "      <td>среднее</td>\n",
       "      <td>1</td>\n",
       "      <td>женат / замужем</td>\n",
       "      <td>0</td>\n",
       "      <td>F</td>\n",
       "      <td>сотрудник</td>\n",
       "      <td>0</td>\n",
       "      <td>82047.418899</td>\n",
       "      <td>на покупку автомобиля</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>21525 rows × 12 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       children  days_employed  dob_years education  education_id  \\\n",
       "0             1   -8437.673028         42    высшее             0   \n",
       "1             1   -4024.803754         36   среднее             1   \n",
       "2             0   -5623.422610         33   Среднее             1   \n",
       "3             3   -4124.747207         32   среднее             1   \n",
       "4             0  340266.072047         53   среднее             1   \n",
       "...         ...            ...        ...       ...           ...   \n",
       "21520         1   -4529.316663         43   среднее             1   \n",
       "21521         0  343937.404131         67   среднее             1   \n",
       "21522         1   -2113.346888         38   среднее             1   \n",
       "21523         3   -3112.481705         38   среднее             1   \n",
       "21524         2   -1984.507589         40   среднее             1   \n",
       "\n",
       "          family_status  family_status_id gender income_type  debt  \\\n",
       "0       женат / замужем                 0      F   сотрудник     0   \n",
       "1       женат / замужем                 0      F   сотрудник     0   \n",
       "2       женат / замужем                 0      M   сотрудник     0   \n",
       "3       женат / замужем                 0      M   сотрудник     0   \n",
       "4      гражданский брак                 1      F   пенсионер     0   \n",
       "...                 ...               ...    ...         ...   ...   \n",
       "21520  гражданский брак                 1      F   компаньон     0   \n",
       "21521   женат / замужем                 0      F   пенсионер     0   \n",
       "21522  гражданский брак                 1      M   сотрудник     1   \n",
       "21523   женат / замужем                 0      M   сотрудник     1   \n",
       "21524   женат / замужем                 0      F   сотрудник     0   \n",
       "\n",
       "        total_income                       purpose  \n",
       "0      253875.639453                 покупка жилья  \n",
       "1      112080.014102       приобретение автомобиля  \n",
       "2      145885.952297                 покупка жилья  \n",
       "3      267628.550329    дополнительное образование  \n",
       "4      158616.077870               сыграть свадьбу  \n",
       "...              ...                           ...  \n",
       "21520  224791.862382             операции с жильем  \n",
       "21521  155999.806512          сделка с автомобилем  \n",
       "21522   89672.561153                  недвижимость  \n",
       "21523  244093.050500  на покупку своего автомобиля  \n",
       "21524   82047.418899         на покупку автомобиля  \n",
       "\n",
       "[21525 rows x 12 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#выводим df\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "a23dd87d-42a1-4ad5-a4b7-4745fa5a8aab",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 21525 entries, 0 to 21524\n",
      "Data columns (total 12 columns):\n",
      " #   Column            Non-Null Count  Dtype  \n",
      "---  ------            --------------  -----  \n",
      " 0   children          21525 non-null  int64  \n",
      " 1   days_employed     19351 non-null  float64\n",
      " 2   dob_years         21525 non-null  int64  \n",
      " 3   education         21525 non-null  object \n",
      " 4   education_id      21525 non-null  int64  \n",
      " 5   family_status     21525 non-null  object \n",
      " 6   family_status_id  21525 non-null  int64  \n",
      " 7   gender            21525 non-null  object \n",
      " 8   income_type       21525 non-null  object \n",
      " 9   debt              21525 non-null  int64  \n",
      " 10  total_income      19351 non-null  float64\n",
      " 11  purpose           21525 non-null  object \n",
      "dtypes: float64(2), int64(5), object(5)\n",
      "memory usage: 2.0+ MB\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "#выводим информацию о датафрейме\n",
    "print (df.info())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d81d4214-e1ac-44c1-9aea-28007c635ccb",
   "metadata": {},
   "source": [
    "## **Предобработка данных** <a id='предобработка'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b45102e8-1c71-42b0-8bce-cfe67de2f0c3",
   "metadata": {},
   "source": [
    "### **Обработка пропущенных значений**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9809a7c7-3473-4bd8-bf8d-c5bedc08db4a",
   "metadata": {},
   "source": [
    "Перед обработкой пропущенный значений, посчитаем их количество по каждому столбцу."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "ce8ef232-59b2-45fc-8803-d0e89d98b8ac",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "children               0\n",
      "days_employed       2174\n",
      "dob_years              0\n",
      "education              0\n",
      "education_id           0\n",
      "family_status          0\n",
      "family_status_id       0\n",
      "gender                 0\n",
      "income_type            0\n",
      "debt                   0\n",
      "total_income        2174\n",
      "purpose                0\n",
      "dtype: int64\n"
     ]
    }
   ],
   "source": [
    "#считаем пропуски в датафрейме по столбцам\n",
    "print (df.isna().sum())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73644e35-b7ed-4688-916f-11f9817e9a34",
   "metadata": {},
   "source": [
    "В датафрейме пропущенны значения в столбце `days_employed` и `total_income`, содержащая информацию о стаже заемщика и информацию о доходах.\n",
    "Посмотрим какую долю состаляют пропущенные значения в своих столбцах, в процентах."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "46e9c73e-507d-493b-a7da-db90eb75eac1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "В столбце total_income значения отсутсвуют в: 10.1%\n",
      "В столбце days_employed значения отсутсвуют в: 10.1%\n"
     ]
    }
   ],
   "source": [
    "#Считаем какую долю составляют пропущенные значения в каждом из столбцов с пропусками\n",
    "df_income_null = len(df[df['total_income'].isna()]) / len (df)\n",
    "print (f'В столбце total_income значения отсутсвуют в: {df_income_null:.1%}')\n",
    "\n",
    "df_employed_null = len(df[df['days_employed'].isna()]) / len (df)\n",
    "print (f'В столбце days_employed значения отсутсвуют в: {df_employed_null:.1%}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54ae661c-31a6-4e26-a785-b497973595e2",
   "metadata": {},
   "source": [
    "Можно предположить, что одной из причин отсувтсвия информации о доходах заемщика может быть из за начилия в Банке возможности выдачи кердита по одному документу."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "0fc45c6a-5f37-4be1-878c-5bc4fb3e9911",
   "metadata": {},
   "outputs": [],
   "source": [
    "#найдем медиану в столбце total_income и заменим пропуски в столбце на медианое значение\n",
    "total_income_median = df['total_income'].median()\n",
    "df['total_income'] = df['total_income'].fillna(total_income_median)\n",
    "\n",
    "#найдем медиану в столбце days_employed и зменим пропуски в столбце на медианое значение\n",
    "days_employed_median = df['days_employed'].median()\n",
    "df['days_employed'] = df['days_employed'].fillna(days_employed_median)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5536b591-b61c-45c1-b029-3ea1592194cb",
   "metadata": {},
   "source": [
    "Заполнив пропуски в столбце `total_income` и `days_emploed` мы усреднили значения, чтоюы избежать сильного искажения конечного реузльтата.\n",
    "Убедимся что в столбцах не осталось пропусков."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "28b4cc49-a42d-43c3-8299-842b66f51ce0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "children            0\n",
      "days_employed       0\n",
      "dob_years           0\n",
      "education           0\n",
      "education_id        0\n",
      "family_status       0\n",
      "family_status_id    0\n",
      "gender              0\n",
      "income_type         0\n",
      "debt                0\n",
      "total_income        0\n",
      "purpose             0\n",
      "dtype: int64\n"
     ]
    }
   ],
   "source": [
    "print (df.isna().sum())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70c12a0c-b938-4479-bc78-6622d3711545",
   "metadata": {},
   "source": [
    "### **Обработка аномалий**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e98abfe5-29a7-4e12-ac79-3d25ad12abe9",
   "metadata": {},
   "source": [
    "#### Поиск аномалий по столбцам"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8aa9adef-1c55-431d-b405-3f67cf463393",
   "metadata": {},
   "source": [
    "Для начала посмотрим какие столбцы мы можем выделить как аномальные с помощью функции `.describe()`.\n",
    "\n",
    "поиска аномалий будем обработывать по отедльности каждый столбец."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "eff74434-75a0-452f-9db5-118bd2ee18be",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>children</th>\n",
       "      <th>days_employed</th>\n",
       "      <th>dob_years</th>\n",
       "      <th>education_id</th>\n",
       "      <th>family_status_id</th>\n",
       "      <th>debt</th>\n",
       "      <th>total_income</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>21525.000000</td>\n",
       "      <td>21525.000000</td>\n",
       "      <td>21525.000000</td>\n",
       "      <td>21525.000000</td>\n",
       "      <td>21525.000000</td>\n",
       "      <td>21525.000000</td>\n",
       "      <td>2.152500e+04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.538908</td>\n",
       "      <td>56557.335698</td>\n",
       "      <td>43.293380</td>\n",
       "      <td>0.817236</td>\n",
       "      <td>0.972544</td>\n",
       "      <td>0.080883</td>\n",
       "      <td>1.651595e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1.381587</td>\n",
       "      <td>134922.319298</td>\n",
       "      <td>12.574584</td>\n",
       "      <td>0.548138</td>\n",
       "      <td>1.420324</td>\n",
       "      <td>0.272661</td>\n",
       "      <td>9.786607e+04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>-1.000000</td>\n",
       "      <td>-18388.949901</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.066726e+04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>-2518.168900</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.077982e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1203.369529</td>\n",
       "      <td>42.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.450179e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>-385.106616</td>\n",
       "      <td>53.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.955436e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>20.000000</td>\n",
       "      <td>401755.400475</td>\n",
       "      <td>75.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>2.265604e+06</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           children  days_employed     dob_years  education_id  \\\n",
       "count  21525.000000   21525.000000  21525.000000  21525.000000   \n",
       "mean       0.538908   56557.335698     43.293380      0.817236   \n",
       "std        1.381587  134922.319298     12.574584      0.548138   \n",
       "min       -1.000000  -18388.949901      0.000000      0.000000   \n",
       "25%        0.000000   -2518.168900     33.000000      1.000000   \n",
       "50%        0.000000   -1203.369529     42.000000      1.000000   \n",
       "75%        1.000000    -385.106616     53.000000      1.000000   \n",
       "max       20.000000  401755.400475     75.000000      4.000000   \n",
       "\n",
       "       family_status_id          debt  total_income  \n",
       "count      21525.000000  21525.000000  2.152500e+04  \n",
       "mean           0.972544      0.080883  1.651595e+05  \n",
       "std            1.420324      0.272661  9.786607e+04  \n",
       "min            0.000000      0.000000  2.066726e+04  \n",
       "25%            0.000000      0.000000  1.077982e+05  \n",
       "50%            0.000000      0.000000  1.450179e+05  \n",
       "75%            1.000000      0.000000  1.955436e+05  \n",
       "max            4.000000      1.000000  2.265604e+06  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91d741bc-f9bc-4d86-b1e3-3d958da838a7",
   "metadata": {},
   "source": [
    "Мы видим описание каждого столбца и стоить обратить внимание на следующие аномали:\n",
    "- В столбце **children** мы видим значния -1 и 20;\n",
    "- В столбце **days_emploed** мы видим отрицательные значения и слишком большой разрыв;\n",
    "- В столбце **dob_years** мы видим нулеове значние;\n",
    "\n",
    "Каждый столбец с аномалиями мы будем обрабатывать по отдельности."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11586fa7-53d4-4c9f-8512-44d8f97389d8",
   "metadata": {},
   "source": [
    "#### Столбец **days_emploed**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e7b1235-f1b3-4095-ba07-ff0b0737bd74",
   "metadata": {},
   "source": [
    "Обработку значений в столбце `days_epmloyed` мы начнем с изменнения отрицательных знаков вещественных чисел."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "369c9078-aacb-46e1-98aa-dc2b7983ea29",
   "metadata": {},
   "outputs": [],
   "source": [
    "#создадим функцию по замене отрицательных значений на положительные\n",
    "def reverse(value):\n",
    "    if value < 0:\n",
    "        value *= (-1)\n",
    "        return (value)\n",
    "    else:\n",
    "        return (value)\n",
    "\n",
    "#заменим отрицательные значения в столбце days_employed с помощью функции reverse\n",
    "df['days_employed'] = df['days_employed'].apply(reverse)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cca720a5-a1ef-4b07-a616-82ca6501882a",
   "metadata": {},
   "source": [
    "Выведим экстремальные значаения по столбце."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "d4415bee-6c41-4161-a0db-33d4d0970542",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count     21525.000000\n",
       "mean      60277.957929\n",
       "std      133301.583103\n",
       "min          24.141633\n",
       "25%        1025.608174\n",
       "50%        1808.053434\n",
       "75%        4779.587738\n",
       "max      401755.400475\n",
       "Name: days_employed, dtype: float64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#опишем значения в столбце\n",
    "df['days_employed'].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf9010f1-d30e-43ec-94af-b36634778ed3",
   "metadata": {},
   "source": [
    "Теперь разберемся с причиной слишком большого разрыва в значениях max - 401755 и min - 24.\n",
    "Стоит обратить внимание на то что значения в данном столбце считаются в днях, можно предполодить, что значния были записаны в часах.\n",
    "Минимальный возраст для работы по трудовому договору 16 лет, а возраст для выхода на пенсию 62.5, как среднее значние.\n",
    "Получается 62.5 - 16 = 46.5 лет трудового стажа или окургленно 17тыс дней (без окургления 16972 дня).\n",
    "Все что больше 17тыс дней будем считать, что записанно в часах."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "afc415a6-63d9-4d76-8b1f-f96a8faf8ba1",
   "metadata": {},
   "outputs": [],
   "source": [
    "#создадим функцию, переводящая часы в дни\n",
    "def employed(days):\n",
    "    if days > 17000: \n",
    "        days /= 24\n",
    "        return(days)\n",
    "    else:\n",
    "        return(days)\n",
    "\n",
    "#обработаем столбец с помощью функции employed\n",
    "df['days_employed'] = df['days_employed'].apply(employed)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9071c09f-a520-47cb-a8e8-9fea3d3c150a",
   "metadata": {},
   "source": [
    "Выведим экстремальные значения в столбце чтобы убедиться, что исправили аномалии."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "222c89b3-d115-4a52-8b94-73433a74d4e8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    21525.000000\n",
       "mean      4292.776745\n",
       "std       5181.318336\n",
       "min         24.141633\n",
       "25%       1025.441612\n",
       "50%       1807.397693\n",
       "75%       4779.202040\n",
       "max      16739.808353\n",
       "Name: days_employed, dtype: float64"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['days_employed'].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a6f687f-7430-4eef-a371-064cc32bf3ca",
   "metadata": {},
   "source": [
    "#### Столбец **children**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c08fd65-bdef-4b01-8f62-7cda890f57af",
   "metadata": {},
   "source": [
    "Посмотрим на уникальные значения в столбце."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "b96f3168-1d98-4df0-a787-aad5b971990f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 1  0  3  2 -1  4 20  5]\n"
     ]
    }
   ],
   "source": [
    "#найдем уникальные значения столбца children\n",
    "print(df['children'].unique())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10cb41d2-cc55-4892-bc78-c64f502572e7",
   "metadata": {},
   "source": [
    "Видим что в столбце есть аномальные значения -1, 20.\n",
    "Предположим, что вводя -1 предполагалсь 1, а при вводе 2 было случайно 0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "1771c9c6-aca5-4d1e-8eec-429c6c28dc3a",
   "metadata": {},
   "outputs": [],
   "source": [
    "#заменяем значения -1 на 1, в столбце children\n",
    "df['children'] = df['children'].replace(-1,1)\n",
    "#заменяем значения 20 на 2, в столбце children\n",
    "df['children'] = df['children'].replace(20,2)\n",
    "#найдем уникальные значения столбца children, чтобы убедиться, что аномальых значений нет\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f6e1ee3-2f4f-4437-aaac-32be9b3ca6ab",
   "metadata": {},
   "source": [
    "Выведим уникальные значения в столбце чтобы убедиться, что исправили аномалии."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "f5afb4cc-62f3-4d89-9857-0d25e7338a60",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1 0 3 2 4 5]\n"
     ]
    }
   ],
   "source": [
    "print(df['children'].unique())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7adab7b3-6823-4f8d-af97-0f01a6ba4643",
   "metadata": {},
   "source": [
    "#### Столбец **dob_years**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b4b3820c-8ca9-4afe-ba4c-7c05c5bc396a",
   "metadata": {},
   "source": [
    "Опишем значения в столбце."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "1c0cf485-cfea-4363-b7a3-7c5caa5aabdb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "count    21525.000000\n",
      "mean        43.293380\n",
      "std         12.574584\n",
      "min          0.000000\n",
      "25%         33.000000\n",
      "50%         42.000000\n",
      "75%         53.000000\n",
      "max         75.000000\n",
      "Name: dob_years, dtype: float64\n"
     ]
    }
   ],
   "source": [
    "print (df['dob_years'].describe())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82851240-dcda-45a5-8f82-a45f8229fc87",
   "metadata": {},
   "source": [
    "Видим что есть значения равные 0. \n",
    "Так в РФ кредитные обязательства можно брать с 18 лет выведим значения мменьше 18. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "c521c54d-29fc-49bd-8761-9884251d37ba",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]\n"
     ]
    }
   ],
   "source": [
    "year = []\n",
    "for years in df['dob_years']:\n",
    "    if years < 18:\n",
    "        year.append(years)\n",
    "print (year)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82a5fe5c-ca8c-4bce-9bbe-23d91ffce105",
   "metadata": {},
   "source": [
    "Можно увидеть, что все значения меньше 18 равны 0. Будем считать, что нули появились в следствии отсутствия кода.\n",
    "Заменим все значения равные нули на среднее значение отностильно типа занятости."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "0021e9fe-9cff-4320-90c9-7a9aac7c86bd",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\sahin\\AppData\\Local\\Temp\\ipykernel_8528\\1039318553.py:3: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value '39.82102707077975' has dtype incompatible with int64, please explicitly cast to a compatible dtype first.\n",
      "  df.loc[(df['income_type'] == type) & (df['dob_years'] == 0), 'dob_years'] = mean_years\n"
     ]
    }
   ],
   "source": [
    "for type in df['income_type'].unique():\n",
    "    mean_years =  df.loc[df['income_type'] == type, 'dob_years'].mean()\n",
    "    df.loc[(df['income_type'] == type) & (df['dob_years'] == 0), 'dob_years'] = mean_years"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b91dd174-19af-42aa-91c8-3bd20778d6db",
   "metadata": {},
   "source": [
    "#### Мы смогли найти несколько аномалий, обработва датафрейм мы исправили неточности в данных .  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0771c4b1-157d-4624-9eb0-c4e4b305db57",
   "metadata": {},
   "source": [
    "### **Замена типа данных** "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4031a246-2b51-4bf2-84f1-c2d8d0a517c2",
   "metadata": {},
   "source": [
    "В столбцах `total_income` и `days_employed` заменим тип данных, чтобы отбросить значения после запятой."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "58d0b335-66fd-488f-a5c8-fbb307075280",
   "metadata": {},
   "outputs": [],
   "source": [
    "#заменим вещественный тип данных на целочисленный в столбце total_income\n",
    "df['total_income'] = df['total_income'].astype(int)\n",
    "#заменим вещественный тип данных на целочисленный в столбце days_employed\n",
    "df['days_employed'] = df['days_employed'].astype(int)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f572ae7-4e5a-4fed-98c1-22c3f351d069",
   "metadata": {},
   "source": [
    "### **Обработка дубликатов**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "085fa507-376c-4002-a3b8-933545297ea8",
   "metadata": {},
   "source": [
    "Проверим регистр значений в столбцах типа данных `str`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "5f02ced5-af01-4e7b-98f4-d65055e097ec",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Уникальные значения в столбце education: \n",
      "['высшее' 'среднее' 'Среднее' 'СРЕДНЕЕ' 'ВЫСШЕЕ' 'неоконченное высшее'\n",
      " 'начальное' 'Высшее' 'НЕОКОНЧЕННОЕ ВЫСШЕЕ' 'Неоконченное высшее'\n",
      " 'НАЧАЛЬНОЕ' 'Начальное' 'Ученая степень' 'УЧЕНАЯ СТЕПЕНЬ'\n",
      " 'ученая степень']\n",
      "\n",
      "Уникальные значения в столбце family_status: \n",
      "['женат / замужем' 'гражданский брак' 'вдовец / вдова' 'в разводе'\n",
      " 'Не женат / не замужем']\n",
      "\n",
      "Уникальные значения в столбце gender: \n",
      "['F' 'M' 'XNA']\n",
      "\n",
      "Уникальные значения в столбце income_type: \n",
      "['сотрудник' 'пенсионер' 'компаньон' 'госслужащий' 'безработный'\n",
      " 'предприниматель' 'студент' 'в декрете']\n",
      "\n",
      "Уникальные значения в столбце purpose: \n",
      "['покупка жилья' 'приобретение автомобиля' 'дополнительное образование'\n",
      " 'сыграть свадьбу' 'операции с жильем' 'образование'\n",
      " 'на проведение свадьбы' 'покупка жилья для семьи' 'покупка недвижимости'\n",
      " 'покупка коммерческой недвижимости' 'покупка жилой недвижимости'\n",
      " 'строительство собственной недвижимости' 'недвижимость'\n",
      " 'строительство недвижимости' 'на покупку подержанного автомобиля'\n",
      " 'на покупку своего автомобиля' 'операции с коммерческой недвижимостью'\n",
      " 'строительство жилой недвижимости' 'жилье'\n",
      " 'операции со своей недвижимостью' 'автомобили' 'заняться образованием'\n",
      " 'сделка с подержанным автомобилем' 'получение образования' 'автомобиль'\n",
      " 'свадьба' 'получение дополнительного образования' 'покупка своего жилья'\n",
      " 'операции с недвижимостью' 'получение высшего образования'\n",
      " 'свой автомобиль' 'сделка с автомобилем' 'профильное образование'\n",
      " 'высшее образование' 'покупка жилья для сдачи' 'на покупку автомобиля'\n",
      " 'ремонт жилью' 'заняться высшим образованием']\n",
      "\n"
     ]
    }
   ],
   "source": [
    "#названия столбцов с типом данных str\n",
    "str_columns = ['education','family_status','gender','income_type','purpose']\n",
    "\n",
    "#выведим все уникальные значения из выбраных столбцов\n",
    "for columns in str_columns:\n",
    "    print (f'Уникальные значения в столбце {columns}: ')\n",
    "    print(df[columns].unique())\n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2fbad262-dc79-44ae-b2f2-83358c7d0072",
   "metadata": {},
   "source": [
    "Видно что в неокторых столбцах значения записаны разным регистром.     \n",
    "Приведем все значения к одному регистру с пмощью функции `.str.lower()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "980781ba-e57e-4dc4-a9d8-7dd36c445213",
   "metadata": {},
   "outputs": [],
   "source": [
    "#в столбцах с типом данных str, приведем все значения к одному регистру\n",
    "for columns_name in str_columns:\n",
    "    df[columns_name] = df[columns_name].str.lower()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9d6826b-5fea-4cbb-9101-9e153420c4ac",
   "metadata": {},
   "source": [
    "Теперь найдем дубликаты с помощью функции `.duplicated()` и посчитаем их количество."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "63a1ce60-20ff-4368-b898-749b4bf6174b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Найдено дубликатов: 71\n"
     ]
    }
   ],
   "source": [
    "#найдем дубликаты    \n",
    "print (f'Найдено дубликатов: {df.duplicated().sum()}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64c28a58-a3e2-4dc7-b284-f7052b5eb437",
   "metadata": {},
   "source": [
    "Удалим все обнаруженные дубликаты с помощью функции `.drop_duplicates()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "f648e7ee-8dad-4a19-b361-06cbf7fd19f6",
   "metadata": {},
   "outputs": [],
   "source": [
    "#удаляем все дубликаты\n",
    "df = df.drop_duplicates()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1103e3c9-0cb2-452d-9796-df66c92f73cd",
   "metadata": {},
   "source": [
    "Убедимся что удалили все дубликаты."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "26b78ce9-28b2-4a04-aa7f-e64103302880",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Найдено дубликатов: 0.\n"
     ]
    }
   ],
   "source": [
    "print (f'Найдено дубликатов: {df.duplicated().sum()}.')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "215862b1-c590-46fc-bf61-ee5635d42e25",
   "metadata": {},
   "source": [
    "## **Создание новых датафреймов**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "438bc55d-03b5-4d2c-921b-cd8a399deaa0",
   "metadata": {},
   "source": [
    "### Создание **df_education**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "763b31ed-0768-44a3-ac0c-9a15651d2364",
   "metadata": {},
   "source": [
    "Создадим `df_education` на основе столбцов `education_id` и `education` из `df`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "4c3b5697-c003-483f-9f77-f882d13c59d8",
   "metadata": {},
   "outputs": [],
   "source": [
    "#создаем новый даатафрейм хранящий в себе столбцы education_id и education из df\n",
    "df_education = df[['education_id','education']]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a469db19-0bb5-48a1-8bae-cb3883c29618",
   "metadata": {},
   "source": [
    "Выведем первые 10 строк `df_education`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "d22c37ea-fa64-492c-a014-a4466b57cb9c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   education_id education\n",
      "0             0    высшее\n",
      "1             1   среднее\n",
      "2             1   среднее\n",
      "3             1   среднее\n",
      "4             1   среднее\n",
      "5             0    высшее\n",
      "6             0    высшее\n",
      "7             1   среднее\n",
      "8             0    высшее\n",
      "9             1   среднее\n"
     ]
    }
   ],
   "source": [
    "print(df_education.head(10))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d968cc21-c291-4545-838f-ce9dc1614b66",
   "metadata": {},
   "source": [
    "Удалим столбец `education` из `df`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "84baecb1-069a-499c-afaa-02f4897b9338",
   "metadata": {},
   "outputs": [],
   "source": [
    "del df['education']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b24c29cf-6f87-4f24-976e-bf6e8905905d",
   "metadata": {},
   "source": [
    "### Создание **df_family_status**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf6eb6d8-4a08-415d-a318-06f2d0b272f4",
   "metadata": {},
   "source": [
    "Создадим `df_family_status` на основе столбцов `family_status_id` и `family_status` из `df`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "8f921859-2d7d-4fd2-9886-dcc868a163ed",
   "metadata": {},
   "outputs": [],
   "source": [
    "#создаем новый даатафрейм хранящий в себе столбцы family_status_id и family_status из df\n",
    "df_family_status = df[['family_status_id','family_status']]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e6a4165-a4f6-4ee5-a9bc-0e70b791c0ff",
   "metadata": {},
   "source": [
    "Выведем первые 10 строк `df_family_status`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "bdf2d031-10c0-492f-9aad-799193963cd5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   family_status_id     family_status\n",
      "0                 0   женат / замужем\n",
      "1                 0   женат / замужем\n",
      "2                 0   женат / замужем\n",
      "3                 0   женат / замужем\n",
      "4                 1  гражданский брак\n",
      "5                 1  гражданский брак\n",
      "6                 0   женат / замужем\n",
      "7                 0   женат / замужем\n",
      "8                 1  гражданский брак\n",
      "9                 0   женат / замужем\n"
     ]
    }
   ],
   "source": [
    "print(df_family_status.head(10))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "653ea339-1a38-41d3-9fdc-d13bba7209fa",
   "metadata": {},
   "source": [
    "Удалим столбец `family_status` из `df`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "970808b8-4836-4f62-8035-46e07e18ca50",
   "metadata": {},
   "outputs": [],
   "source": [
    "del df['family_status']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8abae18d-cf9c-4aef-bc0d-eee1c339ab55",
   "metadata": {},
   "source": [
    "## **Категоризация**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "766d5926-1606-4dce-aa54-5a34e1c1cda8",
   "metadata": {},
   "source": [
    "#### Создание столбца **total_income_category**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "022ba99b-aaf2-4a2f-a075-69339bf33daa",
   "metadata": {},
   "source": [
    "Категоризируем доходы кредитополучателей на основании диапазонов, указанных ниже:\n",
    "- 0–30000 — **E**;\n",
    "- 30001–50000 — **D**;\n",
    "- 50001–200000 — **C**;\n",
    "- 200001–1000000 — **B**;\n",
    "- 1000001 и выше — **A**;\n",
    "\n",
    "Создадим функцию `categorization_total_income` для категоризации значений."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "0d744e1b-4ba4-4625-a9e7-6ac6740e1e1c",
   "metadata": {},
   "outputs": [],
   "source": [
    "#создадим функцию для категоризации значений\n",
    "def categorization_total_income(category):\n",
    "    try:\n",
    "        if 30000 > category > 0:\n",
    "            return('E')\n",
    "        elif 50000 > category > 30001:\n",
    "            return('D')\n",
    "        elif 200000 > category > 50001:\n",
    "            return('C')\n",
    "        elif 1000000 > category > 200001:\n",
    "            return('B')\n",
    "        elif category > 1000001:\n",
    "            return('A')\n",
    "    except:\n",
    "        return('нет категории')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71f6c064-a39c-4a6a-bbdc-324f40617f43",
   "metadata": {},
   "source": [
    "Теперь сформируем столбец `total_income_category`,который категоризирует значения в столбце `total_income`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "d7880178-61b1-4d02-b3dc-45e6aef01bec",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\sahin\\AppData\\Local\\Temp\\ipykernel_8528\\559591598.py:3: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  df['total_income_category'] = df['total_income'].apply(categorization_total_income)\n"
     ]
    }
   ],
   "source": [
    "#обработаем столбец с помощью функции categorization_total_income, \n",
    "#c сохранением в новом столбце total_income_category\n",
    "df['total_income_category'] = df['total_income'].apply(categorization_total_income)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d7f9c0a-2e49-4e6a-bee6-d5e4d2cf922e",
   "metadata": {},
   "source": [
    "Посмторим как выглядит датафрейм с новым столбцом `total_income_category`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "d56e3c40-45a3-4e68-80da-7ab969a0aade",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>children</th>\n",
       "      <th>days_employed</th>\n",
       "      <th>dob_years</th>\n",
       "      <th>education_id</th>\n",
       "      <th>family_status_id</th>\n",
       "      <th>gender</th>\n",
       "      <th>income_type</th>\n",
       "      <th>debt</th>\n",
       "      <th>total_income</th>\n",
       "      <th>purpose</th>\n",
       "      <th>total_income_category</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>8437</td>\n",
       "      <td>42.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>f</td>\n",
       "      <td>сотрудник</td>\n",
       "      <td>0</td>\n",
       "      <td>253875</td>\n",
       "      <td>покупка жилья</td>\n",
       "      <td>B</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>4024</td>\n",
       "      <td>36.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>f</td>\n",
       "      <td>сотрудник</td>\n",
       "      <td>0</td>\n",
       "      <td>112080</td>\n",
       "      <td>приобретение автомобиля</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>5623</td>\n",
       "      <td>33.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>m</td>\n",
       "      <td>сотрудник</td>\n",
       "      <td>0</td>\n",
       "      <td>145885</td>\n",
       "      <td>покупка жилья</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>4124</td>\n",
       "      <td>32.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>m</td>\n",
       "      <td>сотрудник</td>\n",
       "      <td>0</td>\n",
       "      <td>267628</td>\n",
       "      <td>дополнительное образование</td>\n",
       "      <td>B</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>14177</td>\n",
       "      <td>53.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>f</td>\n",
       "      <td>пенсионер</td>\n",
       "      <td>0</td>\n",
       "      <td>158616</td>\n",
       "      <td>сыграть свадьбу</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0</td>\n",
       "      <td>926</td>\n",
       "      <td>27.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>m</td>\n",
       "      <td>компаньон</td>\n",
       "      <td>0</td>\n",
       "      <td>255763</td>\n",
       "      <td>покупка жилья</td>\n",
       "      <td>B</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0</td>\n",
       "      <td>2879</td>\n",
       "      <td>43.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>f</td>\n",
       "      <td>компаньон</td>\n",
       "      <td>0</td>\n",
       "      <td>240525</td>\n",
       "      <td>операции с жильем</td>\n",
       "      <td>B</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0</td>\n",
       "      <td>152</td>\n",
       "      <td>50.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>m</td>\n",
       "      <td>сотрудник</td>\n",
       "      <td>0</td>\n",
       "      <td>135823</td>\n",
       "      <td>образование</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2</td>\n",
       "      <td>6929</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>f</td>\n",
       "      <td>сотрудник</td>\n",
       "      <td>0</td>\n",
       "      <td>95856</td>\n",
       "      <td>на проведение свадьбы</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0</td>\n",
       "      <td>2188</td>\n",
       "      <td>41.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>m</td>\n",
       "      <td>сотрудник</td>\n",
       "      <td>0</td>\n",
       "      <td>144425</td>\n",
       "      <td>покупка жилья для семьи</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   children  days_employed  dob_years  education_id  family_status_id gender  \\\n",
       "0         1           8437       42.0             0                 0      f   \n",
       "1         1           4024       36.0             1                 0      f   \n",
       "2         0           5623       33.0             1                 0      m   \n",
       "3         3           4124       32.0             1                 0      m   \n",
       "4         0          14177       53.0             1                 1      f   \n",
       "5         0            926       27.0             0                 1      m   \n",
       "6         0           2879       43.0             0                 0      f   \n",
       "7         0            152       50.0             1                 0      m   \n",
       "8         2           6929       35.0             0                 1      f   \n",
       "9         0           2188       41.0             1                 0      m   \n",
       "\n",
       "  income_type  debt  total_income                     purpose  \\\n",
       "0   сотрудник     0        253875               покупка жилья   \n",
       "1   сотрудник     0        112080     приобретение автомобиля   \n",
       "2   сотрудник     0        145885               покупка жилья   \n",
       "3   сотрудник     0        267628  дополнительное образование   \n",
       "4   пенсионер     0        158616             сыграть свадьбу   \n",
       "5   компаньон     0        255763               покупка жилья   \n",
       "6   компаньон     0        240525           операции с жильем   \n",
       "7   сотрудник     0        135823                 образование   \n",
       "8   сотрудник     0         95856       на проведение свадьбы   \n",
       "9   сотрудник     0        144425     покупка жилья для семьи   \n",
       "\n",
       "  total_income_category  \n",
       "0                     B  \n",
       "1                     C  \n",
       "2                     C  \n",
       "3                     B  \n",
       "4                     C  \n",
       "5                     B  \n",
       "6                     B  \n",
       "7                     C  \n",
       "8                     C  \n",
       "9                     C  "
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e75be878-17de-4637-a6c9-687bc3361673",
   "metadata": {},
   "source": [
    "#### Создание столбца **purpose_category**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b50352fb-cf75-453f-9285-f4acf6555353",
   "metadata": {},
   "source": [
    "Выведим цели получения кредита, чтобы выделить основыне категории."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "4dd6220f-7380-4356-942a-b89157e3b3cb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Цели получения кредита:  ['покупка жилья' 'приобретение автомобиля' 'дополнительное образование'\n",
      " 'сыграть свадьбу' 'операции с жильем' 'образование'\n",
      " 'на проведение свадьбы' 'покупка жилья для семьи' 'покупка недвижимости'\n",
      " 'покупка коммерческой недвижимости' 'покупка жилой недвижимости'\n",
      " 'строительство собственной недвижимости' 'недвижимость'\n",
      " 'строительство недвижимости' 'на покупку подержанного автомобиля'\n",
      " 'на покупку своего автомобиля' 'операции с коммерческой недвижимостью'\n",
      " 'строительство жилой недвижимости' 'жилье'\n",
      " 'операции со своей недвижимостью' 'автомобили' 'заняться образованием'\n",
      " 'сделка с подержанным автомобилем' 'получение образования' 'автомобиль'\n",
      " 'свадьба' 'получение дополнительного образования' 'покупка своего жилья'\n",
      " 'операции с недвижимостью' 'получение высшего образования'\n",
      " 'свой автомобиль' 'сделка с автомобилем' 'профильное образование'\n",
      " 'высшее образование' 'покупка жилья для сдачи' 'на покупку автомобиля'\n",
      " 'ремонт жилью' 'заняться высшим образованием']\n"
     ]
    }
   ],
   "source": [
    "print ('Цели получения кредита: ', df['purpose'].unique())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "838f8ee5-cf4b-4947-a783-b3f4ea1b507e",
   "metadata": {},
   "source": [
    "Из целей получения кредита мы можем выделить следующие категории:\n",
    "- Операции с автомобилем;\n",
    "- Операции с недвижимостью;\n",
    "- Получение образования;\n",
    "- Проведение свадьбы;\n",
    "\n",
    "Создадим функцию `categorization_purpose` для категоризации значений."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "aa5472f9-8644-4fe9-aa5a-a21889fd84bc",
   "metadata": {},
   "outputs": [],
   "source": [
    "#создадим функцию для категоризации значений\n",
    "def categorization_purpose(category):\n",
    "    try:\n",
    "        if 'образов' in category:\n",
    "            return('получение образования')\n",
    "        elif 'авто' in category:\n",
    "            return('операции с автомобилем')\n",
    "        elif ('жил' in category) or ('недвиж' in category):\n",
    "            return('операции с недвижимостью')\n",
    "        elif 'свад' in category:\n",
    "            return('проведение свадьбы')\n",
    "    except:\n",
    "        return('нет категории')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df35cf94-d751-4400-be2a-8413e0586dae",
   "metadata": {},
   "source": [
    "Теперь сформируем новый столбец `purpose_category`,который категоризирует значения в столбце `purpose`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "18ac5584-e609-447c-ad09-146613b432db",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\sahin\\AppData\\Local\\Temp\\ipykernel_8528\\1666098698.py:3: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  df['purpose_category'] = df['purpose'].apply(categorization_purpose)\n"
     ]
    }
   ],
   "source": [
    "#обработаем столбец с помощью функции categorization, \n",
    "#c сохранением в новом столбце purpose_category\n",
    "df['purpose_category'] = df['purpose'].apply(categorization_purpose)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9dd0cd29-e42e-4785-b308-b968a4d9e0eb",
   "metadata": {},
   "source": [
    "Посмторим как выглядит датафрейм с новым столбцом `purpose_category`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "f08534bb-38e3-41a3-a252-f87619ce43a9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>children</th>\n",
       "      <th>days_employed</th>\n",
       "      <th>dob_years</th>\n",
       "      <th>education_id</th>\n",
       "      <th>family_status_id</th>\n",
       "      <th>gender</th>\n",
       "      <th>income_type</th>\n",
       "      <th>debt</th>\n",
       "      <th>total_income</th>\n",
       "      <th>purpose</th>\n",
       "      <th>total_income_category</th>\n",
       "      <th>purpose_category</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>8437</td>\n",
       "      <td>42.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>f</td>\n",
       "      <td>сотрудник</td>\n",
       "      <td>0</td>\n",
       "      <td>253875</td>\n",
       "      <td>покупка жилья</td>\n",
       "      <td>B</td>\n",
       "      <td>операции с недвижимостью</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>4024</td>\n",
       "      <td>36.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>f</td>\n",
       "      <td>сотрудник</td>\n",
       "      <td>0</td>\n",
       "      <td>112080</td>\n",
       "      <td>приобретение автомобиля</td>\n",
       "      <td>C</td>\n",
       "      <td>операции с автомобилем</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>5623</td>\n",
       "      <td>33.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>m</td>\n",
       "      <td>сотрудник</td>\n",
       "      <td>0</td>\n",
       "      <td>145885</td>\n",
       "      <td>покупка жилья</td>\n",
       "      <td>C</td>\n",
       "      <td>операции с недвижимостью</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>4124</td>\n",
       "      <td>32.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>m</td>\n",
       "      <td>сотрудник</td>\n",
       "      <td>0</td>\n",
       "      <td>267628</td>\n",
       "      <td>дополнительное образование</td>\n",
       "      <td>B</td>\n",
       "      <td>получение образования</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>14177</td>\n",
       "      <td>53.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>f</td>\n",
       "      <td>пенсионер</td>\n",
       "      <td>0</td>\n",
       "      <td>158616</td>\n",
       "      <td>сыграть свадьбу</td>\n",
       "      <td>C</td>\n",
       "      <td>проведение свадьбы</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0</td>\n",
       "      <td>926</td>\n",
       "      <td>27.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>m</td>\n",
       "      <td>компаньон</td>\n",
       "      <td>0</td>\n",
       "      <td>255763</td>\n",
       "      <td>покупка жилья</td>\n",
       "      <td>B</td>\n",
       "      <td>операции с недвижимостью</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0</td>\n",
       "      <td>2879</td>\n",
       "      <td>43.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>f</td>\n",
       "      <td>компаньон</td>\n",
       "      <td>0</td>\n",
       "      <td>240525</td>\n",
       "      <td>операции с жильем</td>\n",
       "      <td>B</td>\n",
       "      <td>операции с недвижимостью</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0</td>\n",
       "      <td>152</td>\n",
       "      <td>50.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>m</td>\n",
       "      <td>сотрудник</td>\n",
       "      <td>0</td>\n",
       "      <td>135823</td>\n",
       "      <td>образование</td>\n",
       "      <td>C</td>\n",
       "      <td>получение образования</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2</td>\n",
       "      <td>6929</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>f</td>\n",
       "      <td>сотрудник</td>\n",
       "      <td>0</td>\n",
       "      <td>95856</td>\n",
       "      <td>на проведение свадьбы</td>\n",
       "      <td>C</td>\n",
       "      <td>проведение свадьбы</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0</td>\n",
       "      <td>2188</td>\n",
       "      <td>41.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>m</td>\n",
       "      <td>сотрудник</td>\n",
       "      <td>0</td>\n",
       "      <td>144425</td>\n",
       "      <td>покупка жилья для семьи</td>\n",
       "      <td>C</td>\n",
       "      <td>операции с недвижимостью</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   children  days_employed  dob_years  education_id  family_status_id gender  \\\n",
       "0         1           8437       42.0             0                 0      f   \n",
       "1         1           4024       36.0             1                 0      f   \n",
       "2         0           5623       33.0             1                 0      m   \n",
       "3         3           4124       32.0             1                 0      m   \n",
       "4         0          14177       53.0             1                 1      f   \n",
       "5         0            926       27.0             0                 1      m   \n",
       "6         0           2879       43.0             0                 0      f   \n",
       "7         0            152       50.0             1                 0      m   \n",
       "8         2           6929       35.0             0                 1      f   \n",
       "9         0           2188       41.0             1                 0      m   \n",
       "\n",
       "  income_type  debt  total_income                     purpose  \\\n",
       "0   сотрудник     0        253875               покупка жилья   \n",
       "1   сотрудник     0        112080     приобретение автомобиля   \n",
       "2   сотрудник     0        145885               покупка жилья   \n",
       "3   сотрудник     0        267628  дополнительное образование   \n",
       "4   пенсионер     0        158616             сыграть свадьбу   \n",
       "5   компаньон     0        255763               покупка жилья   \n",
       "6   компаньон     0        240525           операции с жильем   \n",
       "7   сотрудник     0        135823                 образование   \n",
       "8   сотрудник     0         95856       на проведение свадьбы   \n",
       "9   сотрудник     0        144425     покупка жилья для семьи   \n",
       "\n",
       "  total_income_category          purpose_category  \n",
       "0                     B  операции с недвижимостью  \n",
       "1                     C    операции с автомобилем  \n",
       "2                     C  операции с недвижимостью  \n",
       "3                     B     получение образования  \n",
       "4                     C        проведение свадьбы  \n",
       "5                     B  операции с недвижимостью  \n",
       "6                     B  операции с недвижимостью  \n",
       "7                     C     получение образования  \n",
       "8                     C        проведение свадьбы  \n",
       "9                     C  операции с недвижимостью  "
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e207f4b-1549-40e2-8c39-e57084b93ea4",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "## **Ответы на вопросы**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0eb04277-4ce3-4acd-841c-050b8e635e63",
   "metadata": {},
   "source": [
    "### Есть ли зависимость между количеством детей и возвратом кредита в срок?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eaa561ef-2151-445d-90b6-8a9bf7171467",
   "metadata": {},
   "source": [
    "Для ответа на данный вопрос создадим таблицу c процентным соотношением."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "b24b7c6e-05f3-4cee-a3d9-06239a57b1fe",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>debt_sum</th>\n",
       "      <th>debt_count</th>\n",
       "      <th>debt</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>children</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>41</td>\n",
       "      <td>9.76</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>202</td>\n",
       "      <td>2128</td>\n",
       "      <td>9.49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>445</td>\n",
       "      <td>4855</td>\n",
       "      <td>9.17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>27</td>\n",
       "      <td>330</td>\n",
       "      <td>8.18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1063</td>\n",
       "      <td>14091</td>\n",
       "      <td>7.54</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          debt_sum  debt_count  debt\n",
       "children                            \n",
       "4                4          41  9.76\n",
       "2              202        2128  9.49\n",
       "1              445        4855  9.17\n",
       "3               27         330  8.18\n",
       "0             1063       14091  7.54\n",
       "5                0           9  0.00"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#создадим датафрейм\n",
    "children_debt = pd.DataFrame()\n",
    "#посчитаем отношение в процентах\n",
    "children_debt['debt_sum'] = df.groupby('children')['debt'].sum()\n",
    "children_debt['debt_count'] = df.groupby('children')['debt'].count()\n",
    "children_debt['debt'] = round((children_debt['debt_sum'] / children_debt['debt_count']) * 100, 2)\n",
    "children_debt.sort_values(by='debt', ascending = False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb294c25-9c55-44ca-a235-2b39e28c8cff",
   "metadata": {},
   "source": [
    "Из сформированной таблицы мы видим, что бездетные реже просрачивают оплату по кредиту. С ростом количества детей видно, что растет и вероятность просрочки оплаты по кредиту.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8528e07b-43eb-46dc-96be-43de44b3f085",
   "metadata": {},
   "source": [
    "### Есть ли зависимость между семейным положением и возвратом кредита в срок?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6aaa208f-95d6-4ff5-8b88-47ef99a8fcc2",
   "metadata": {},
   "source": [
    "Для ответа на данный вопрос создадим таблицу c процентным соотношением."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "d42eef7d-40df-4d66-b19e-1f3dace1b882",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\sahin\\AppData\\Local\\Temp\\ipykernel_8528\\2096640794.py:5: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  df_family_status['debt'] = df['debt']\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>debt_sum</th>\n",
       "      <th>debt_count</th>\n",
       "      <th>debt</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>family_status</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>не женат / не замужем</th>\n",
       "      <td>274</td>\n",
       "      <td>2810</td>\n",
       "      <td>9.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>гражданский брак</th>\n",
       "      <td>388</td>\n",
       "      <td>4151</td>\n",
       "      <td>9.35</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>женат / замужем</th>\n",
       "      <td>931</td>\n",
       "      <td>12339</td>\n",
       "      <td>7.55</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>в разводе</th>\n",
       "      <td>85</td>\n",
       "      <td>1195</td>\n",
       "      <td>7.11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>вдовец / вдова</th>\n",
       "      <td>63</td>\n",
       "      <td>959</td>\n",
       "      <td>6.57</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                       debt_sum  debt_count  debt\n",
       "family_status                                    \n",
       "не женат / не замужем       274        2810  9.75\n",
       "гражданский брак            388        4151  9.35\n",
       "женат / замужем             931       12339  7.55\n",
       "в разводе                    85        1195  7.11\n",
       "вдовец / вдова               63         959  6.57"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#создадим датафрейм\n",
    "family_status_debt = pd.DataFrame()\n",
    "\n",
    "#добавим к df_family_status столбец 'debt'\n",
    "df_family_status['debt'] = df['debt']\n",
    "\n",
    "#посчитаем отношение в процентах\n",
    "family_status_debt['debt_sum'] = df_family_status.groupby('family_status')['debt'].sum()\n",
    "family_status_debt['debt_count'] = df_family_status.groupby('family_status')['debt'].count()\n",
    "family_status_debt['debt'] = round((family_status_debt['debt_sum'] / family_status_debt['debt_count']) * 100, 2)\n",
    "family_status_debt.sort_values(by='debt', ascending = False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "13ea3c28-c38c-4867-8a19-bfbea287ac90",
   "metadata": {},
   "source": [
    "Из сформированной таблицы мы видим, что не женатые/не замужние просрачивают оплату по кредиту чаще, нежели женатые/замужние. Но, те кто состоял в браке чаще опалчивают кредит в срок, чем состоящие в браке."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e4b08be-d134-489c-a111-d165eaa05ec8",
   "metadata": {},
   "source": [
    "### Есть ли зависимость между уровнем дохода и возвратом кредита в срок?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "669beec2-a000-4341-bd91-b81f677a0b13",
   "metadata": {},
   "source": [
    "Для ответа на данный вопрос создадим таблицу c процентным соотношением."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "3133bfae-8c82-47fc-a878-9f6f6b41cb9b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>debt_sum</th>\n",
       "      <th>debt_count</th>\n",
       "      <th>debt</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>total_income_category</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>A</th>\n",
       "      <td>2</td>\n",
       "      <td>25</td>\n",
       "      <td>8.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>B</th>\n",
       "      <td>356</td>\n",
       "      <td>5040</td>\n",
       "      <td>7.06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>C</th>\n",
       "      <td>1360</td>\n",
       "      <td>16016</td>\n",
       "      <td>8.49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>D</th>\n",
       "      <td>21</td>\n",
       "      <td>350</td>\n",
       "      <td>6.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>E</th>\n",
       "      <td>2</td>\n",
       "      <td>22</td>\n",
       "      <td>9.09</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                       debt_sum  debt_count  debt\n",
       "total_income_category                            \n",
       "A                             2          25  8.00\n",
       "B                           356        5040  7.06\n",
       "C                          1360       16016  8.49\n",
       "D                            21         350  6.00\n",
       "E                             2          22  9.09"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#создадим датафрейм\n",
    "total_income_debt = pd.DataFrame()\n",
    "#посчитаем отношение в процентах\n",
    "total_income_debt['debt_sum'] = df.groupby('total_income_category')['debt'].sum()\n",
    "total_income_debt['debt_count'] = df.groupby('total_income_category')['debt'].count()\n",
    "total_income_debt['debt'] = round((total_income_debt['debt_sum'] / total_income_debt['debt_count']) * 100, 2)\n",
    "total_income_debt "
   ]
  },
  {
   "cell_type": "raw",
   "id": "9147fef5-d12f-4628-ac5e-4470fa40fcb9",
   "metadata": {},
   "source": [
    "Из сформированной нами таблицы мы видим, что зависимости между уровнем дохода и возврата кредита в срок нет.        \n",
    "Но можно выдлеить как группу риска категорию **E** с доходом до 30 тыс."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c3b1883-e898-4900-a16a-4d236ca52167",
   "metadata": {},
   "source": [
    "### Как разные цели кредита влияют на его возврат в срок?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1981c68a-f470-4993-85e4-d4e57539bbcf",
   "metadata": {},
   "source": [
    "Для ответа на данный вопрос создадим таблицу c процентным соотношением."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "4cf3659d-608a-455f-82e0-258537d6f5f0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>debt_sum</th>\n",
       "      <th>debt_count</th>\n",
       "      <th>debt</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>purpose_category</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>операции с автомобилем</th>\n",
       "      <td>403</td>\n",
       "      <td>4306</td>\n",
       "      <td>9.36</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>получение образования</th>\n",
       "      <td>370</td>\n",
       "      <td>4013</td>\n",
       "      <td>9.22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>проведение свадьбы</th>\n",
       "      <td>186</td>\n",
       "      <td>2324</td>\n",
       "      <td>8.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>операции с недвижимостью</th>\n",
       "      <td>782</td>\n",
       "      <td>10811</td>\n",
       "      <td>7.23</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                          debt_sum  debt_count  debt\n",
       "purpose_category                                    \n",
       "операции с автомобилем         403        4306  9.36\n",
       "получение образования          370        4013  9.22\n",
       "проведение свадьбы             186        2324  8.00\n",
       "операции с недвижимостью       782       10811  7.23"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#создадим датафрейм\n",
    "purpose_debt = pd.DataFrame()\n",
    "#посчитаем отношение в процентах\n",
    "purpose_debt['debt_sum'] = df.groupby('purpose_category')['debt'].sum()\n",
    "purpose_debt['debt_count'] = df.groupby('purpose_category')['debt'].count()\n",
    "purpose_debt['debt'] = round((purpose_debt['debt_sum'] / purpose_debt['debt_count']) * 100, 2)\n",
    "purpose_debt.sort_values(by='debt', ascending = False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cbd64244-dd0e-48d2-81de-c855631d03ef",
   "metadata": {},
   "source": [
    "Из сформированной нами таблицы мы видим, что чаще всего просрачивают опалту по кредиту на операции с автомобилем,  \n",
    "реже всего просрачивают опалту по кредиту на операции с недвижимостью."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f51d0a46-b15e-4a4d-bb62-3a9609daa8ac",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "## **Общий вывод**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6eb6a5dc-5101-48f1-bb25-48fc74b1985f",
   "metadata": {},
   "source": [
    "На основе проанализрованных нами данных можно сделать вывод, что существует зависимость между семейным положением и колличеством детей на факт погашения кредита, и зависимость между уровнем дохода и возвратом кредита в срок. \n",
    "Также можно выделить уровни дохода и категории трат кредита, которые можно выделить как группу риска. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}