Skip to content

Commit

Permalink
Update doc about the new stt format
Browse files Browse the repository at this point in the history
  • Loading branch information
icywind committed Oct 30, 2024
1 parent 9899b8a commit a322eb3
Showing 1 changed file with 7 additions and 1 deletion.
8 changes: 7 additions & 1 deletion docs/GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,7 @@ The TEN agent uses the data stream from the RTC channel for the transcription of
```swift
open func rtcEngine(_ engine: AgoraRtcEngineKit, receiveStreamMessageFromUid uid: UInt, streamId: Int, data: Data) {
do {
let stt = try JSONDecoder().decode(STTStreamText.self, from: data)
let stt = try sttStreamDecoder.parseStream(data: data)
let msg = IChatItem(userId: uid, text: stt.text, time: stt.textTS, isFinal: stt.isFinal, isAgent: 0 == stt.streamID)
streamTextProcessor.addChatItem(item: msg)
} catch let error {
Expand All @@ -226,6 +226,12 @@ The transcript is separated into chunks and sent into sequential messages. Each
}
```
The struct is translated into a view model, IChatItem, that the stream text processor uses to make the transcript ready for display as a whole.
Note in version 0.5.0, the data stream message is wrapped in a format of
```
<id>|<idx>|<total>|<content part>
```
where the content part is a part of a long base64 encoded text. The number of parts is in <total\>, and the <idx\> is the part number. We use a STTStreamDecoder class to decode the string into STTStreamText struct.


**Stream Text Processor**
The stream text processor keeps track of the incoming transcript chunks and stores them in a list (sttWords). When addChatItem() is called, the processor processes the message with its time and the isFinal field to determine where to put the message in the list. The final result will update the observed object *messages* in AgoraManager. Later, ChatView uses *messages* to generate the transcription view. It is worth reading the code comments to understand the sorting logic here:
Expand Down

0 comments on commit a322eb3

Please sign in to comment.