Data sets used in benchmark reported in the "Introducing Nova" blog

[DrJJ]

Where can I download all the data sets used in benchmarks reported in blog: https://blog.deepgram.com/nova-speech-to-text-whisper-api/?utm_campaign=2023-04-13-Nova-Announcement-Email.

Public benchmarks like these require the benchmark data to be made available so that the results can be reproduced, see, e.g. Google Terms of Service.

[natalierutgers]

Hi @DrJJ, glad you caught the release! For this release, we benchmarked on real-world customer and public data to see how each vendor would perform in the wild. For that reason it wouldn't be appropriate to publish the dataset publicly.

[DrJJ]

What about this from Google Terms of Service? You did benchmark Google Video STT.

3.2.4 Benchmarking. Customer may not publicly disclose directly or through a third party the results of any comparative or compatibility testing, benchmarking, or evaluation of the Services (each, a “Test”), unless the disclosure includes all information necessary for Google or a third party to replicate the Test. If Customer conducts, or directs a third party to conduct, a Test of the Services and publicly discloses the results directly or through a third party, then Google (or a Google directed third party) may conduct Tests of any publicly available cloud products or services provided by Customer and publicly disclose the results of any such Test (which disclosure will include all information necessary for Customer or a third party to replicate the Test).

AWS has a similar clause:

1.8. You may perform benchmarks or comparative tests or evaluations (each, a “Benchmark”) of the Services. If you perform or disclose, or direct or permit any third party to perform or disclose, any Benchmark of any of the Services, you (i) will include in any disclosure, and will disclose to us, all information necessary to replicate such Benchmark, and (ii) agree that we may perform and disclose the results of Benchmarks of your products or services, irrespective of any restrictions on Benchmarks in the terms governing your products or services.