API response time too high -- Please help

[pdns90]

Hi there,

https://developers.deepgram.com/reference/pre-recorded

I am using Prerecorded audio file and my file size is 200 MB and its 18 minutes and getting response in 8-10 minutes using the API.

Coding language is PHP and used the same code as show in https://developers.deepgram.com/reference/pre-recorded

How i can reduce the response time, please help.

[lukeocodes]

Hey @pdns90, can you provide a little more info about the file you're sending over?

What is the file type? Did you try a smaller file? Can you share the code, including any features/options you've included?

[pdns90]

Hi @lukeocodes, Thanks for your reply.

Our file type is mp3 and also we tried to use 20 MB file and its takes 2-3 minutes.

I am using below PHP code.

<?php
ini_set('max_execution_time', '0');
require_once('vendor/autoload.php');

$client = new \GuzzleHttp\Client();

$response = $client->request('POST', 'https://api.deepgram.com/v1/listen', [
  'body' => '{"url":"https://example.com/laravel/public/uploads/[email protected]"}',
  'headers' => [
    'Authorization' => 'Token xxxxxx',
    'accept' => 'application/json',
    'content-type' => 'application/json',
  ],
]);

echo $response->getBody();

your fast response is highly appreciated.

Thanks,

[lukeocodes]

@pdns90 are you able to share the file? If not, give me a little time to find a suitable alternative.

[lukeocodes]

@pdns90 so, I've tried a 26mb mp3 of a speech from Wikipedia. I made the request from my terminal, so not exactly the same. But here is my request... Please use this file from your server, and let me know if it's as slow for you.

If it is as slow from your CDN, it may be a bandwidth limit problem with where you are hosting your file. Here you'll see I uploaded this to our cloudinary CDN quickly. In this case, I would suggest reading the file from your server as a local file (if possible), or finding a host for your file without limited bandwidth.

Some untested PHP that reads a file from disk:

<?php
require_once 'vendor/autoload.php';

$client = new \GuzzleHttp\Client();

$audio = fopen('/path/to/audio.mp3', 'r');
$response = $client->request('POST', 'https://api.deepgram.com/v1/listen', [
  'body' => $audio,
  'headers' => [
    'Authorization' => 'Token xxxxxx',
    'accept' => 'application/json',
    'content-type' => 'audio/mpeg',
  ],
]);

echo $response->getBody();

If it is not as slow from your CDN, I would prefer to test it with your file to see if it is a problem with that file, or how we are receiving that file.

My tests:

curl -w "@curl-format.txt" --request POST \
     --url https://api.deepgram.com/v1/listen \
     --header 'Authorization: Token xxxxxx' \
     --header 'accept: application/json' \
     --header 'content-type: application/json' \
     --data '
{
  "url": "https://res.cloudinary.com/deepgram/video/upload/v1685106433/dg-audio/stanford_rjb4xa.mp3"
}
'

The file curl-format.txt allows me to time various aspects of the request.

     time_namelookup:  %{time_namelookup}s\n
        time_connect:  %{time_connect}s\n
     time_appconnect:  %{time_appconnect}s\n
    time_pretransfer:  %{time_pretransfer}s\n
       time_redirect:  %{time_redirect}s\n
  time_starttransfer:  %{time_starttransfer}s\n
                     ----------\n
          time_total:  %{time_total}s\n

The resulting request:

     time_namelookup:  0.119798s
        time_connect:  0.277871s
     time_appconnect:  0.611886s
    time_pretransfer:  0.611971s
       time_redirect:  0.000000s
  time_starttransfer:  0.000000s
                     ----------
          time_total:  4.115854s

[pdns90]

Hi, Thanks so much for your help.

I have tried below code and response time reduced 8 minutes to 3-4 minutes but I feel its its very high.

<?php
ini_set('max_execution_time', '0');
require_once('vendor/autoload.php');

$client = new \GuzzleHttp\Client();
$audio = fopen('https://example.com/uploads/xxx.mp3','r');
$response = $client->request('POST', 'https://api.deepgram.com/v1/listen', [
  'body' => $audio,
  'headers' => [
    'Authorization' => 'Token xxxxxxx',
    'accept' => 'application/json',
    'content-type' => 'audio/mpeg',
  ],
]);

echo $response->getBody();

Also, I am getting below error very frequently. --- 408 Request timeout.

Also, my client audio file which is 200 MB and response time is 3 minutes.

Please share your email so i can share you.

Thanks

[lukeocodes]

It would be interesting to see how it handled the file being read from a filepath, rather than a URL. I believe what we might be seeing is the result of one or more HTTP requests being restricted by available bandwidth.

You can email your file to [email protected] and I'll try it from my end. Thanks for sticking with it!

[pdns90]

Hi @lukeocodes, Thanks for your reply.

I sent an email to you my client audio file.

Once you test it. please delete it.

Thanks,

[lukeocodes]

@pdns90 when I downloaded this file, is it from the same server that you tried to host the file on for transcription?

[lukeocodes]

Hey @pdns90

I tried your file from my local machine. 212.1mb transcribed in 1 minute.

My request:

curl -w "@curl-format.txt" --request POST \
     --url https://api.deepgram.com/v1/listen \
     --header 'Authorization: Token xxxxx' \
     --header 'accept: application/json' \
     --header 'content-type: audio/mp3' \       
     --data-binary @/Users/lukeoliff/Downloads/xxx.mp3

This is the result:

     time_namelookup:  0.001613s
        time_connect:  0.126888s
     time_appconnect:  0.389840s
    time_pretransfer:  0.389899s
       time_redirect:  0.000000s
  time_starttransfer:  0.557873s
                     ----------
          time_total:  57.727225s

Going back to my original thought, I think it may be to do with where you are hosting that file. If you're using the server that I downloaded the file from, I am on a gigabit connection and I usually get around 500/mb down, but for your file download I got 200kb download.

If you want to run your file locally, you can use this code on your own machine:

<?php
require_once 'vendor/autoload.php';

$client = new \GuzzleHttp\Client();

$audio = fopen('/the/file/on/your/machine/xxx.mp3', 'r'); // make this the FILE PATH, not a URL
$response = $client->request('POST', 'https://api.deepgram.com/v1/listen', [
  'body' => $audio,
  'headers' => [
    'Authorization' => 'Token xxxxxx',
    'accept' => 'application/json',
    'content-type' => 'audio/mpeg',
  ],
]);

echo $response->getBody();

Important notes about making the most of slower connections:

1. If you give Deepgram 'body' => '{"url": "https://example.com/uploads/xxx.mp3"}', then Deepgram will download the file from that server, and it is limited by the upload speed of that server
2. If you give Deepgram 'body' => fopen('/the/file/on/your/machine/xxx.mp3', 'r'); then Deepgram will recieve the file from the server where your PHP runs, limited by the upload speed of THAT server. Assuming it's the same server, no difference to option 1.
3. But, if you give Deepgram 'body' => fopen('https://example.com/uploads/xxx.mp3', 'r');, then Deepgram will recieve the file from the server where your PHP runs, but only after PHP downloads the file from the speed of that server. This is almost always the wrong solution.

Conclusion

Running your 200mb file for me took 1 minute. I think Deepgram is taking a long time to get the file, but only because the server where the file is located is limited or having a problem with bandwidth. A screenshot of the download of the .zip file

[pdns90]

Hi, Thank you for your reply.

I will check and get back to you.

Thanks

[pdns90]

Hi @lukeocodes, I tried in my local server and I got result in 3 minutes and checked my internet speed is on https://www.speedtest.net and its showing 104.12 Mbps Download and 121 Mbps Upload speed.

Below is my php code.

<?php
ini_set('max_execution_time', '0');
require_once('vendor/autoload.php');

$client = new \GuzzleHttp\Client();
$audio = fopen('C:\Users\pc\Downloads\client_audio.mp3','r');
$response = $client->request('POST', 'https://api.deepgram.com/v1/listen', [
  'body' => $audio,
  'headers' => [
    'Authorization' => 'Token xxxxxxxxxxxxxxxxx',
    'accept' => 'application/json',
    'content-type' => 'audio/mpeg',
  ],
]);

echo $response->getBody();

I am thinking still is delay in result?

Thanks,

[lukeocodes]

No for a 200mb file I think that is an acceptable amount of time, especially from a local machine. I think the server you've used has a bandwidth limitation. You should consider using a CDN like cloudflare to host your file.