Merge pull request #1351 from leoleojie/master

FEA: add sequential, context, knowledge quick start

Author: Sherry-XLL

docs/source/developer_guide/customize_trainers.rst

@@ -19,6 +19,8 @@ and revise :meth:`~recbole.trainer.trainer.Trainer.evaluate` or :meth:`~recbole.
 
 Example
 ----------------
+1. Alternative Optimization
+>>>>>>>>>>>>>>>>>>>>>>>>>
 Here we present a simple Trainer example, which is used for alternative optimization.
 We revise the :meth:`~recbole.trainer.trainer.Trainer._train_epoch` method.
 To begin with, we need to create a new class for
@@ -102,3 +104,107 @@ Complete Code
                     total_loss += loss.item()
             return total_loss
 
+2. Mixed precision training
+>>>>>>>>>>>>>>>>>>>>>>>>>
+Here we present a simple Trainer example, which is used for mixed
+precision training. Mixed precision training offers significant
+computational speedup by performing operations in half-precision
+format, while storing minimal information in single-precision to
+retain as much information as possible in critical parts of the
+network. Let's give an example based on torch ``torch.cuda.amp``. To
+begin with, we need to create a new class for ``NewTrainer`` based on
+``Trainer``.
+
+.. code:: python
+
+  from recbole.trainer import Trainer
+  import torch.cuda.amp as amp 
+  class NewTrainer(Trainer):
+      def __init__(self, config, model):
+          super(NewTrainer, self).__init__(config, model)
+          
+Then we revise ``_train_epoch()``.
+
+.. code:: python
+
+  def _train_epoch(self, train_data, epoch_idx):
+      self.model.train()
+      scaler = amp.GradScaler(enabled=self.enable_scaler)
+      for batch_idx, interaction in enumerate(iter_data):
+            interaction = interaction.to(self.device)
+            self.optimizer.zero_grad()
+            with amp.autocast(enabled=self.enable_amp):
+                losses = loss_func(interaction)
+            total_loss = losses.item() if total_loss is None else total_loss + losses.item()
+            scaler.scale(loss).backward()
+            scaler.step(self.optimizer)
+            scaler.update()
+            
+Complete Code
+^^^^^^^^^^^^^^^^
+.. code:: python
+
+  from recbole.trainer import Trainer
+  import torch.cuda.amp as amp 
+  class NewTrainer(Trainer):
+      def __init__(self, config, model):
+          super(NewTrainer, self).__init__(config, model)
+          
+  def _train_epoch(self, train_data, epoch_idx):
+      self.model.train()
+      scaler = amp.GradScaler(enabled=self.enable_scaler)
+      for batch_idx, interaction in enumerate(iter_data):
+            interaction = interaction.to(self.device)
+            self.optimizer.zero_grad()
+            with amp.autocast(enabled=self.enable_amp):
+                losses = loss_func(interaction)
+            total_loss = losses.item() if total_loss is None else total_loss + losses.item()
+            scaler.scale(loss).backward()
+            scaler.step(self.optimizer)
+            scaler.update()        
+
+3. Layer-specific learning rate
+>>>>>>>>>>>>>>>>>>>>>>>>>
+Here we present a simple Trainer example, which is used for setting
+layer-specific learning rate. For pretrained model, layers closer to
+the input layer are more likely to have learned more general
+features. On the other hand, later layers of the model learn the
+detailed features. In this case, we can set different learning rate
+for different layers. We can do this by modifying the optimizer.
+
+.. code:: python
+
+      def _build_optimizer(self, learner, learning_rate, weight_decay):
+          pretrained_params = list(map(id, self.model.pretrained_part.parameters())
+          base_params = filter(lambda p: id(p) not in pretrained_params, self.model.parameters())
+          if learner.lower() == 'adam':
+              optimizer = optim.Adam([
+                  {"params":base_params},
+                  {"pretrained_params":self.model.pretrained_part.parameters(),"lr":1e-5}],
+                  lr=learning_rate,weight_decay=weight_decay)
+          return optimizer             
+
+
+
+Complete Code
+^^^^^^^^^^^^^^^^
+.. code:: python 
+
+  from recbole.trainer import Trainer
+  class NewTrainer(Trainer):
+      def __init__(self, config, model):
+          super(NewTrainer, self).__init__(config, model)
+          self.optimizer = self._build_optimizer()
+          
+  def _train_epoch(self, train_data, epoch_idx):
+          self.model.train()
+          total_loss = 0.
+          for batch_idx, interaction in enumerate(train_data):
+          interaction = interaction.to(self.device)
+          self.optimizer.zero_grad()
+          loss = self.model.calculate_loss1(interaction)
+          self._check_nan(loss)
+          loss.backward()
+          self.optimizer.step()
+          total_loss += loss.item()
+          return total_loss

docs/source/get_started/quick_start.rst

@@ -1,184 +1,15 @@
 Quick Start
 ===============
-Here is a quick-start example for using RecBole. We will show you how to train and test **BPR** model on the **ml-100k** dataset from both **API**
+Here is a quick-start example for using RecBole. RecBole supports general recommendation, sequential recommendation, context-aware recommendation and knowledge-based recommendation. We will select a representative model from each type of recommendation to show you how to train and test on the **ml-100k** dataset from both **API**
 and **source code**.
 
+.. toctree::
+   :maxdepth: 1
 
-Quick-start From API
---------------------------
-
-1. Prepare your data:
->>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
-Before running a model, firstly you need to prepare and load data. To help users quickly get start, 
-RecBole has a build-in dataset **ml-100k** and you can directly use it. However, if you want to use other datasets, you can read
-:doc:`../user_guide/usage/running_new_dataset` for more information.
-
-Then, you need to set data config for data loading. You can create a `yaml` file called `test.yaml` and write the following settings:
-
-.. code:: yaml
-
-    # dataset config
-    USER_ID_FIELD: user_id
-    ITEM_ID_FIELD: item_id
-    load_col:
-        inter: [user_id, item_id]
-
-For more details of data config, please refer to :doc:`../user_guide/config/data_settings`.
-
-2. Choose a model:
->>>>>>>>>>>>>>>>>>>>>>>>>
-In RecBole, we implement 73 recommendation models covering general recommendation, sequential recommendation,
-context-aware recommendation and knowledge-based recommendation. You can choose a model from our :doc:`../user_guide/model_intro`.
-Here we choose BPR model to train and test. 
-
-Then, you need to set the parameter for BPR model. You can check the :doc:`../user_guide/model/general/bpr` and add the model settings into the `test.yaml`, like:
-
-.. code:: yaml
-
-    # model config
-    embedding_size: 64
-
-If you want to run different models, you can read :doc:`../user_guide/usage/running_different_models` for more information.
-
-3. Set training and evaluation config:
->>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
-In RecBole, we support multiple training and evaluation methods. You can choose how to train and test model by simply setting the config.
-
-Here we want to train and test the BPR model in training-validation-test method (optimize model parameters on the training set, do parameter selection according to the results on the validation set,
-and finally report the results on the test set) and evaluate the model performance by full ranking with all item candidates, 
-so we can add the following settings into the `test.yaml`.
-
-.. code:: yaml
-
-    # Training and evaluation config
-    epochs: 500
-    train_batch_size: 4096
-    eval_batch_size: 4096
-    neg_sampling:
-        uniform: 1
-    eval_args:
-        group_by: user
-        order: RO
-        split: {'RS': [0.8,0.1,0.1]}
-        mode: full
-    metrics: ['Recall', 'MRR', 'NDCG', 'Hit', 'Precision']
-    topk: 10 
-    valid_metric: MRR@10
-    metric_decimal_place: 4
-
-For more details of training and evaluation config, please refer to :doc:`../user_guide/config/training_settings` and :doc:`../user_guide/config/evaluation_settings`.
-
-4. Run the model and collect the result
->>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
-Now you have finished all the preparations, it's time to run the model!
-
-You can create a new python file (e.g., `run.py`), and write the following code:
-
-.. code:: python
-
-    from recbole.quick_start import run_recbole
-
-    run_recbole(model='BPR', dataset='ml-100k', config_file_list=['test.yaml'])
-
-
-Then run the following command:
-
-.. code:: bash
-
-    python run.py 
-
-And you will obtain the output like:
-
-.. code:: none
-
-    24 Aug 01:46    INFO  ml-100k
-    The number of users: 944
-    Average actions of users: 106.04453870625663
-    The number of items: 1683
-    Average actions of items: 59.45303210463734
-    The number of inters: 100000
-    The sparsity of the dataset: 93.70575143257098%
-    Remain Fields: ['user_id', 'item_id']
-    24 Aug 01:46    INFO  [Training]: train_batch_size = [4096] negative sampling: [{'uniform': 1}]
-    24 Aug 01:46    INFO  [Evaluation]: eval_batch_size = [4096] eval_args: [{'split': {'RS': [0.8, 0.1, 0.1]}, 'group_by': 'user', 'order': 'RO', 'mode': 'full'}]
-    24 Aug 01:46    INFO  BPR(
-    (user_embedding): Embedding(944, 64)
-    (item_embedding): Embedding(1683, 64)
-    (loss): BPRLoss()
-    )
-    Trainable parameters: 168128
-    Train     0: 100%|████████████████████████| 40/40 [00:00<00:00, 200.47it/s, GPU RAM: 0.01 G/11.91 G]
-    24 Aug 01:46    INFO  epoch 0 training [time: 0.21s, train loss: 27.7228]
-    Evaluate   : 100%|██████████████████████| 472/472 [00:00<00:00, 518.65it/s, GPU RAM: 0.01 G/11.91 G]
-    24 Aug 01:46    INFO  epoch 0 evaluating [time: 0.92s, valid_score: 0.020500]
-    ......
-    Train    96: 100%|████████████████████████| 40/40 [00:00<00:00, 229.26it/s, GPU RAM: 0.01 G/11.91 G]
-    24 Aug 01:47    INFO  epoch 96 training [time: 0.18s, train loss: 3.7170]
-    Evaluate   : 100%|██████████████████████| 472/472 [00:00<00:00, 857.00it/s, GPU RAM: 0.01 G/11.91 G]
-    24 Aug 01:47    INFO  epoch 96 evaluating [time: 0.56s, valid_score: 0.375200]
-    24 Aug 01:47    INFO  valid result: 
-    recall@10 : 0.2162    mrr@10 : 0.3752    ndcg@10 : 0.2284    hit@10 : 0.7508    precision@10 : 0.1602    
-    24 Aug 01:47    INFO  Finished training, best eval result in epoch 85
-    24 Aug 01:47    INFO  Loading model structure and parameters from saved/BPR-Aug-24-2021_01-46-43.pth
-    Evaluate   : 100%|██████████████████████| 472/472 [00:00<00:00, 866.53it/s, GPU RAM: 0.01 G/11.91 G]
-    24 Aug 01:47    INFO  best valid : {'recall@10': 0.2195, 'mrr@10': 0.3871, 'ndcg@10': 0.2344, 'hit@10': 0.7582, 'precision@10': 0.1627}
-    24 Aug 01:47    INFO  test result: {'recall@10': 0.2523, 'mrr@10': 0.4855, 'ndcg@10': 0.292, 'hit@10': 0.7953, 'precision@10': 0.1962}
-
-Finally you will get the model's performance on the test set and the model file will be saved under the `/saved`. Besides, 
-RecBole allows tracking and visualizing train loss and valid score with TensorBoard, please read the :doc:`../user_guide/usage/use_tensorboard` for more details.
-
-The above is the whole process of running a model in RecBole, and you can read other docs for depth usage. 
-
-
-Quick-start From Source
---------------------------
-Besides using API, you can also directly run the source code of `RecBole <https://github.com/RUCAIBox/RecBole>`_. 
-The whole process is similar to Quick-start From API. 
-You can create a `yaml` file called `test.yaml` and set all the config as follow:
-
-.. code:: yaml
-
-    # dataset config 
-    USER_ID_FIELD: user_id
-    ITEM_ID_FIELD: item_id
-    load_col:
-        inter: [user_id, item_id]
-    
-    # model config
-    embedding_size: 64
-
-    # Training and evaluation config
-    epochs: 500
-    train_batch_size: 4096
-    eval_batch_size: 4096
-    neg_sampling:
-        uniform: 1
-    eval_args:
-        group_by: user
-        order: RO
-        split: {'RS': [0.8,0.1,0.1]}
-        mode: full
-    metrics: ['Recall', 'MRR', 'NDCG', 'Hit', 'Precision']
-    topk: 10 
-    valid_metric: MRR@10
-    metric_decimal_place: 4
-
-Then run the following command:
-
-.. code:: bash
-
-    python run_recbole.py --model=BPR --dataset=ml-100k --config_files=test.yaml
-
-And you will get the output of running the BPR model on the ml-100k dataset.
-
-If you want to change the parameters, such as ``embedding_size``,
-just set the additional command parameters as you need:
-
-.. code:: bash
-
-    python run_recbole.py --model=BPR --dataset=ml-100k --config_files=test.yaml --embedding_size=0.0001 
-
-
+   started/general
+   started/sequential
+   started/context-aware
+   started/knowledge-based
 
 In-depth Usage
 -------------------