add training code for reward model #222
Conversation
.vscode/settings.json
Outdated
| @@ -1,4 +1,4 @@ | |||
| { | |||
| "python.formatting.provider": "black", | |||
There was a problem hiding this comment.
Sorry, but we require all contributors to use the same pre-commit rules.
There was a problem hiding this comment.
just updated, please revise
There was a problem hiding this comment.
I still see the provider as autopep8, when it should be black. do you maybe have a local commit that you didn't push yet?
| from rank_datasets import DataCollatorForPairRank, HFSummary, WebGPT | ||
| from torch.utils.data import DataLoader | ||
| from transformers import AutoTokenizer | ||
|
|
There was a problem hiding this comment.
very useful file, a short docstring at the beginning would be nice to explain how it is used during dev/purpose (e.g. dataloader test, batch-shape inspection)
.vscode/settings.json
Outdated
| @@ -1,4 +1,4 @@ | |||
| { | |||
| "python.formatting.provider": "black", | |||
There was a problem hiding this comment.
I still see the provider as autopep8, when it should be black. do you maybe have a local commit that you didn't push yet?
|
@yk yeah, it's my problem. just reset the format setting |
trainer code to train a single score reward model. Currently support webgpt and raw datasets from humanfeed back summary by openai. See readme and rank_datasets.py for more details.