Feature/batched prompt generation

[sokovninn]

• Add batch prompt generation.
• Default batch_size_prompt is set to 64 (TinyLlama and Mistral-4bit should fit into an 8GB GPU).
• Replace the generate_prompt() function with generate_prompts_batch().

Output of the script datadreamer/examples/measure_batched_prompt_gen_speed.py:

Model           1               2               4               8               16              32              64              128             256             512
tinyllama       2.779           1.018           0.507           0.263           0.144           0.071           0.045           0.039           0.036           0.036
mistral_int4    3.096           3.774           1.902           0.963           1.146           0.608           0.341           0.205           0.138
mistral_fp16    1.851           1.172           0.497           0.308           0.309           0.2             0.135           0.1

(batch_size x time_per_prompt) - maximal batch size for each model that fits into an NVIDIA L4 24GB GPU.

TinyLlama looks like the best choice. Mistral-Int4 should be used only if GPU memory is less than 16GB.

[github-actions]

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines	Covered	Coverage	Threshold	Status
757	373	49%	0%	🟢

New Files

No new covered files...

Modified Files

File	Coverage	Status
datadreamer/pipelines/generate_dataset_from_scratch.py	43%	🟢
datadreamer/prompt_generation/lm_prompt_generator.py	58%	🟢
datadreamer/prompt_generation/prompt_generator.py	87%	🟢
datadreamer/prompt_generation/tinyllama_lm_prompt_generator.py	82%	🟢
TOTAL	67%	🟢

updated for commit: a0ae403 by action🐍

[github-actions]

Test Results

  6 files    6 suites   44m 23s ⏱️
 70 tests  28 ✅  42 💤 0 ❌
420 runs  168 ✅ 252 💤 0 ❌

Results for commit a0ae403.

[HonzaCuhel]

LGTM!