Multi-GPU Support

[yuzhenmao]

Hi, thank you for your fantastic work! I was wondering if the current codebase supports multi-GPU settings.

I have two GPUs on my machine, and when I run the following command: ./bench_batch_decode -a seqlen=4096 -a page_budget=[64,512], I encounter the following error:

[happierpig]

Hi @yuzhenmao ,

This seems to be an issue related to dependency library (NVIDIA/nvbench#113). You can simply test single GPU by export CUDA_VISIBLE_DEVICES=0.

[yuzhenmao]

Thanks!