notebooks/How_To_Train_CNN_Classifier_With_TF.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "cb3fbcc4",
   "metadata": {
    "colab_type": "text",
    "id": "view-in-github"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/ML-Bioinfo-CEITEC/genomic_benchmarks/blob/main/notebooks/How_To_Train_CNN_Classifier_With_TF.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a0f8e486",
   "metadata": {
    "id": "a0f8e486"
   },
   "source": [
    "# How To Train CNN Classifier With TF\n",
    "\n",
    "This notebook demonstrates how to use `genomic_benchmarks` to train a neural network classifier on one of its benchmark datasets [human_nontata_promoters](https://github.com/ML-Bioinfo-CEITEC/genomic_benchmarks/tree/main/docs/human_nontata_promoters)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "akPONjEhhr9m",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "akPONjEhhr9m",
    "outputId": "5b9c5ccc-5fe7-4c5f-d554-543deadededd"
   },
   "outputs": [],
   "source": [
    "#If you work in Google Colaboratory - uncomment the following line to install the package to your virtual machine  \n",
    "#!pip install -qq tensorflow_addons genomic-benchmarks"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a06a600",
   "metadata": {
    "id": "3a06a600"
   },
   "source": [
    "# Data download\n",
    "\n",
    "With the function `download_dataset` downloads, we can download full-sequence form of the benchmark, splitted into train and test sets, one folder for each class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "1a57e0d2",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "1a57e0d2",
    "outputId": "70c1d7ba-5995-4573-ec54-a2c22d20f09c"
   },
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "\n",
    "from genomic_benchmarks.loc2seq import download_dataset\n",
    "from genomic_benchmarks.data_check import is_downloaded, info\n",
    "from genomic_benchmarks.models.tf import vectorize_layer, binary_f1_score\n",
    "from genomic_benchmarks.models.tf import basic_cnn_model_v0 as model\n",
    "\n",
    "if not is_downloaded('human_nontata_promoters'):\n",
    "    download_dataset('human_nontata_promoters')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "ef313c21",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 196
    },
    "id": "ef313c21",
    "outputId": "059cb8ee-cd74-441e-a33b-1b10e4b687a4"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dataset `human_nontata_promoters` has 2 classes: negative, positive.\n",
      "\n",
      "All lenghts of genomic intervals equals 251.\n",
      "\n",
      "Totally 36131 sequences have been found, 27097 for training and 9034 for testing.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>train</th>\n",
       "      <th>test</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>negative</th>\n",
       "      <td>12355</td>\n",
       "      <td>4119</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>positive</th>\n",
       "      <td>14742</td>\n",
       "      <td>4915</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          train  test\n",
       "negative  12355  4119\n",
       "positive  14742  4915"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "info('human_nontata_promoters', 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f4ce24e0",
   "metadata": {
    "id": "f4ce24e0"
   },
   "source": [
    "## TF Dataset object\n",
    "\n",
    "To train the model with TensorFlow, we must create a TF Dataset. Because the directory structure of our benchmarks is ready for training, we can just call `tf.keras.preprocessing.text_dataset_from_directory` function as follows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "409118c7",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "409118c7",
    "outputId": "7ff1886f-a615-48b6-ead5-948d270722a0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 27097 files belonging to 2 classes.\n"
     ]
    }
   ],
   "source": [
    "BATCH_SIZE = 64\n",
    "SEQ_PATH = Path.home() / '.genomic_benchmarks' / 'human_nontata_promoters'\n",
    "CLASSES = [x.stem for x in (SEQ_PATH/'train').iterdir() if x.is_dir()]\n",
    "\n",
    "train_dset = tf.keras.preprocessing.text_dataset_from_directory(\n",
    "    SEQ_PATH / 'train',\n",
    "    batch_size=BATCH_SIZE,\n",
    "    class_names=CLASSES)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef9706c8",
   "metadata": {
    "id": "ef9706c8"
   },
   "source": [
    "## Text vectorization\n",
    "\n",
    "To convert the strings to tensors, we internally use TF `TextVectorization` layer and splitting to characters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "60ac5547",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "60ac5547",
    "outputId": "38251e46-7a0b-4e4a-d071-2e1eabea8e34"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['', '[UNK]', 'a', 'c', 't', 'g', 'n']"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vectorize_layer.adapt(train_dset.map(lambda x, y: x))\n",
    "#vectorize_layer.set_vocabulary(vocabulary=np.asarray(['a', 'c', 't', 'g', 'n']))\n",
    "vectorize_layer.get_vocabulary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "6a5c8a30",
   "metadata": {
    "id": "6a5c8a30"
   },
   "outputs": [],
   "source": [
    "def vectorize_text(text, label):\n",
    "  text = tf.expand_dims(text, -1)\n",
    "  return vectorize_layer(text)-2, label\n",
    "\n",
    "train_ds = train_dset.map(vectorize_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2216d2dd",
   "metadata": {
    "id": "2216d2dd"
   },
   "source": [
    "## Model training\n",
    "\n",
    "To get a baseline (other models can be compared to) we ship a package with [a simple CNN model](https://github.com/ML-Bioinfo-CEITEC/genomic_benchmarks/blob/main/src/genomic_benchmarks/models/tf.py). We have vectorized the dataset before training the model to speed up the process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "2fbf777e",
   "metadata": {
    "id": "2fbf777e"
   },
   "outputs": [],
   "source": [
    "model.compile(loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),\n",
    "              optimizer='adam',\n",
    "              metrics=[tf.metrics.BinaryAccuracy(threshold=0.0), binary_f1_score])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "57ae2ebd",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "57ae2ebd",
    "outputId": "95fb0586-76e7-4bee-e3ee-a8c775760a2d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "424/424 [==============================] - 10s 21ms/step - loss: 0.4457 - binary_accuracy: 0.7996 - f1_score: 0.7699\n",
      "Epoch 2/10\n",
      "424/424 [==============================] - 9s 21ms/step - loss: 0.3847 - binary_accuracy: 0.8284 - f1_score: 0.8186\n",
      "Epoch 3/10\n",
      "424/424 [==============================] - 8s 19ms/step - loss: 0.3551 - binary_accuracy: 0.8408 - f1_score: 0.8350\n",
      "Epoch 4/10\n",
      "424/424 [==============================] - 8s 19ms/step - loss: 0.3309 - binary_accuracy: 0.8577 - f1_score: 0.8509\n",
      "Epoch 5/10\n",
      "424/424 [==============================] - 8s 20ms/step - loss: 0.3091 - binary_accuracy: 0.8666 - f1_score: 0.8621\n",
      "Epoch 6/10\n",
      "424/424 [==============================] - 9s 20ms/step - loss: 0.2946 - binary_accuracy: 0.8739 - f1_score: 0.8690\n",
      "Epoch 7/10\n",
      "424/424 [==============================] - 8s 20ms/step - loss: 0.2800 - binary_accuracy: 0.8799 - f1_score: 0.8756\n",
      "Epoch 8/10\n",
      "424/424 [==============================] - 9s 21ms/step - loss: 0.2683 - binary_accuracy: 0.8867 - f1_score: 0.8843\n",
      "Epoch 9/10\n",
      "424/424 [==============================] - 10s 24ms/step - loss: 0.2559 - binary_accuracy: 0.8916 - f1_score: 0.8905\n",
      "Epoch 10/10\n",
      "424/424 [==============================] - 11s 26ms/step - loss: 0.2457 - binary_accuracy: 0.8978 - f1_score: 0.8961\n"
     ]
    }
   ],
   "source": [
    "EPOCHS = 10\n",
    "\n",
    "history = model.fit(\n",
    "    train_ds,\n",
    "    epochs=EPOCHS)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5bae33fa",
   "metadata": {
    "id": "5bae33fa"
   },
   "source": [
    "## Evaluation on the test set\n",
    "\n",
    "Finally, we can do the same pre-processing for the test set and evaluate the F1 score of our model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "38e009a1",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "38e009a1",
    "outputId": "60fc6e58-383f-4fc8-e2b7-66f18efbeaff"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 9034 files belonging to 2 classes.\n"
     ]
    }
   ],
   "source": [
    "test_dset = tf.keras.preprocessing.text_dataset_from_directory(\n",
    "    SEQ_PATH / 'test',\n",
    "    batch_size=BATCH_SIZE,\n",
    "    class_names=CLASSES,\n",
    "    shuffle=False)\n",
    "\n",
    "test_ds = test_dset.map(vectorize_text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "b7ec77f5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "142/142 [==============================] - 1s 6ms/step - loss: 0.3246 - binary_accuracy: 0.8581 - f1_score: 0.8706\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.3245792090892792, 0.8580916523933411, 0.8705631494522095]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.evaluate(test_ds)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1fcbc4a7",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "include_colab_link": true,
   "name": "How_To_Train_CNN_Classifier_With_TF.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}