From 864c3dd70b65c167ae6cf52c1d30fdde9698c25b Mon Sep 17 00:00:00 2001 From: "shengzhe.li" Date: Tue, 20 Aug 2024 11:28:08 +0900 Subject: [PATCH 1/5] Update README --- README.md | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/README.md b/README.md index 5a35967..35e3176 100644 --- a/README.md +++ b/README.md @@ -44,3 +44,33 @@ poetry run python -m jmteb \ > [!NOTE] > If you want to log model predictions to further analyze the performance of your model, you may want to use `--log_predictions true` to enable all evaluators to log predictions. It is also available to set whether to log in the config of evaluators. + +## Multi-GPU support + +There are two ways to enable multi-GPU evaluation. + +* New class `DPSentenceBertEmbedder` ([here](src/jmteb/embedders/data_parallel_sbert_embedder.py)). + +```bash +poetry run python -m jmteb \ + --evaluators "src/configs/tasks/jsts.jsonnet" \ + --embedder DPSentenceBertEmbedder \ + --embedder.model_name_or_path "" \ + --save_dir "output/" +``` + +* With `torchrun`, multi-GPU in [`TransformersEmbedder`](src/jmteb/embedders/transformers_embedder.py) is available. For example, +```bash +MODEL_NAME= +MODEL_KWARGS="\{\'torch_dtype\':\'torch.bfloat16\'\}" +torchrun \ + --nproc_per_node=$GPUS_PER_NODE --nnodes=1 \ + src/jmteb/__main__.py --embedder TransformersEmbedder \ + --embedder.model_name_or_path ${MODEL_NAME} \ + --embedder.pooling_mode cls \ + --embedder.batch_size 4096 \ + --embedder.model_kwargs ${MODEL_KWARGS} \ + --embedder.max_seq_length 512 \ + --save_dir "output/${MODEL_NAME}" \ + --evaluators src/jmteb/configs/jmteb.jsonnet +``` \ No newline at end of file From e9ba66f0d04c02f8633d6edeeccf5d8d102c7019 Mon Sep 17 00:00:00 2001 From: "shengzhe.li" Date: Tue, 20 Aug 2024 11:30:37 +0900 Subject: [PATCH 2/5] fix lint --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 35e3176..9e4b755 100644 --- a/README.md +++ b/README.md @@ -60,6 +60,7 @@ poetry run python -m jmteb \ ``` * With `torchrun`, multi-GPU in [`TransformersEmbedder`](src/jmteb/embedders/transformers_embedder.py) is available. For example, + ```bash MODEL_NAME= MODEL_KWARGS="\{\'torch_dtype\':\'torch.bfloat16\'\}" From b51faebdfcee1d625f949d32f37382a865ff335a Mon Sep 17 00:00:00 2001 From: "shengzhe.li" Date: Tue, 20 Aug 2024 11:31:59 +0900 Subject: [PATCH 3/5] fix lint --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 9e4b755..afac366 100644 --- a/README.md +++ b/README.md @@ -74,4 +74,4 @@ torchrun \ --embedder.max_seq_length 512 \ --save_dir "output/${MODEL_NAME}" \ --evaluators src/jmteb/configs/jmteb.jsonnet -``` \ No newline at end of file +``` From 0737287da0112e61ec65bfe085d1154fd670a3ae Mon Sep 17 00:00:00 2001 From: "shengzhe.li" Date: Tue, 20 Aug 2024 11:38:49 +0900 Subject: [PATCH 4/5] Add note for batch size --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index afac366..f02108b 100644 --- a/README.md +++ b/README.md @@ -75,3 +75,4 @@ torchrun \ --save_dir "output/${MODEL_NAME}" \ --evaluators src/jmteb/configs/jmteb.jsonnet ``` +Note that the batch size here is global batch size (`per_device_batch_size` × `n_gpu`). From a22291f396a8ab569952f5d0c3637dbd2c17a368 Mon Sep 17 00:00:00 2001 From: "shengzhe.li" Date: Tue, 20 Aug 2024 11:54:36 +0900 Subject: [PATCH 5/5] fix lint --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index f02108b..0fb5271 100644 --- a/README.md +++ b/README.md @@ -75,4 +75,5 @@ torchrun \ --save_dir "output/${MODEL_NAME}" \ --evaluators src/jmteb/configs/jmteb.jsonnet ``` + Note that the batch size here is global batch size (`per_device_batch_size` × `n_gpu`).