Add on-device runner and v5e benchmark model configs

[raymondzouu]

Description

Adding on-device runner to benchmark_runner.py. This will run the model benchmark on the current device rather than launching it as an xpk workload. The benchmark script must be ran on all workers. This will allow our XLML perf tests to run our latest benchmark model configs.

This change introduces a new command-line flag that specifies which runner to use (xpk, on-device). Usage is as follows:

xpk runner:

python3 benchmarks/benchmark_runner.py xpk --project=<project> --zone=<zone> --device_type=v5litepod-256 --num_slices=1  --cluster_name=<cluster> --base_output_directory=gs://<your-bucket> --model_name="default_16b_v5e_256" --base_docker_image maxtext_base_image

on-device runner:

python3 benchmarks/benchmark_runner.py on-device --base_output_directory=gs://<your-bucket> --model_name="default_16b_v5e_256" --run_name=test-run

Tests

http://shortn/_grjxE2vPHD

Checklist

Before submitting this PR, please make sure (put X in square brackets):

• I have performed a self-review of my code.
• I have necessary comments in my code, particularly in hard-to-understand areas.
• I have run end-to-end tests tests and provided workload links above if applicable.
• I have made or will make corresponding changes to the doc if needed.

[Obliviour]

Really great improvements here! Adding v5e configs and a maxtext direct way of running these commands is awesome.

Can you add details on this way of directly running maxtext_xpk_runner.py in the readme file in the bnechmarks directory?