Using aiortc and Deepgram together

[ekingedik]

Hello,

I have a Python server application that receives an audio stream from the browser using aiortc. I've been using a local ASR solution to transcribe this audio which I like to replace with Deepgram.

I successfully open a websocket using transcription.live. However, as soon as I send the audio received by aiortc in the async recv function, the websocket closes. In the usage dashboard, I see a 101 OK for a transcription of 0 seconds. I'm assuming this is happening because the send function is not in a loop. Is there any way to ensure the socket stays open?

Are there any guides or tips on how to use aiortc together with Deepgram? Any help will be greatly appreciated.

Best,
Ekin

[shirgoldbird]

Hey @ekingedik! Unfortunately, there aren't any docs on using aiortc with Deepgram. If you're able to connect to Deepgram but the connection closes right away, one possibility is that you're hitting our websocket timeout. This could be because you're waiting too long to start sending audio, or possibly because you're not sending audio at all due to something in the aiortc integration not working correctly.

We just released a new version of our streaming test suite, which is designed to help walk you through the process of integrating your audio with DG and debug any issues. I'd be especially curious what results you get if you try and send audio to the mock server included with the test suite (step 3, Streaming Other Audio). This can help narrow down where any potential bugs lie—in your audio source, in your audio integration code, in your Deepgram integration code, or with Deepgram itself.

If you give the test suite a try, please let me know how it goes!

[ekingedik]

Thanks a lot for the reply @shirgoldbird!

The streaming test suite has been really helpful. I integrated the client code into our application and the example wav file is being sent correctly and the client periodically receives updates from the server side. However, when I switch to sending our own audio received via aiortc, I only receive the updates after I send the CloseStream signal. The audio format I'm sending is the correct one and the server actually receives it, I can see the packages arriving periodically on the server side. If I switch to the Deepgram websocket, I receive the transcriptions correctly after the CloseStream signal. The only difference between the two versions is the first one sends the chunks of the example wav file in a loop while the second one fetches the raw audio chunks from a non-blocking queue and sends them in a loop. Receiving the transcriptions after the socket is closed might still work for our use case but it would be best to fix this if it's possible.

I also have an additional question regarding the timeout. Our use case includes a dyadic chat scenario where DeepGram should only transcribe the turns of one user. So, it should stop the transcription when its the turn of the other user and start again when the other users turn ends. Would it be best to establish websocket connections per turn?

Thanks a lot!