Keep getting closing frames from Live Transcription WebSocket

[yunxianggu99]

Hi! I've been developing live transcription feature with Deepgram's live streaming audio endpoint. Understand that it will get disconnected after eg I mute my microphone or switch audio input device. But when audio is recovered and I want to reconnect, I keep getting this closing frame(?):

{:ok, %{"channels" => 1, "created" => "2023-05-04T04:06:56.048Z", "duration" => 15.239938, "model_info" => %{"3312b52b-d08e-4072-bfff-124c410b770d" => %{"arch" => "nova", "name" => "general-nova", "version" => "2023-03-13.31000"}}, "models" => ["3312b52b-d08e-4072-bfff-124c410b770d"], "request_id" => "6d752087-22e7-4e48-a170-8c24a9b88b81", "sha256" => "a31281a98cb68a674cdb163eb7bb87c71476d9205d89edaef7484b60f6352011", "transaction_key" => "deprecated"}}

This happenes even if I start a new socket and connect.

[nikolawhallon]

This looks like the final metadata websocket text message that Deepgram will send back when a stream is ending (immediately before sending a close websocket message) - I imagine this is then coming from the previous websocket connection you are closing, and not the new websocket connection you are creating.

[nikolawhallon]

Seeing <unk><unk><unk><unk> sounds like an issue on our end, I'll take a look, but I looked at that request id (3af31498-9378-4fe0-8fea-24c6b9f6e047) internally and see:

Failed to transcode: Client error: failed to process audio: corrupt or unsupported data

this will occur when not sending raw audio (which requires specifying the encoding, sample_rate, and channels) and when the audio is not properly containerized - when I run into these issues, I usually fork/dump the audio in my pipeline to a file to inspect to make sure the format is as expected. Let me look a bit more.

[nikolawhallon]

(Do you happen to have some docs which might describe Twilio Video's media format more? I'm trying to look into that - I know their programmable voice media format is strictly raw mulaw.)

[nikolawhallon]

There's a cool example of running a mock Deepgram server and validating your audio here which can be quite useful: https://developers.deepgram.com/documentation/guides/streaming-test-suite/#3-streaming-other-audio - it has utilities for dumping the streamed audio for inspection

[yunxianggu99]

So if we dump things into an audio for transcription, then everything works. Also the disconnection seems really random. Sometimes it comes back, sometimes transcription just keeps terminating

[nikolawhallon]

So if we dump things into an audio for transcription, then everything works

do you mean if you take that audio to do a batch-mode request it will work?

If you are able to share some of the following, I can do some investigation:

• request id's where the disconnection occurs
• saved audio which worked or didn't work
• any code you are willing/able to share