NullReferenceException in AsyncStreamingUpdateEnumerator

[Nockiro]

Service

OpenAI

Describe the bug

We try to use the Assistant API for streaming and on some days (as they did today) requests via the .NET library fail because OpenAIs servers can't correctly respond to our requests.
The problem for us is that the library seems to crash if the server responds with an error during generating (or before actually starting) the response stream.

Since they are just instances of a NullReferenceException without further error messages, it's hard to debug where the crashes actually come from.
Looking at the stack trace via Visual Studio, the exception occurs here at _updates = enumerable.GetEnumerator();.
The variable _events.Current in that case contains the following data:

• Data (as JSON string):

{"error":{"message":"The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if you keep seeing this error. (Please include the request ID req_<redacted> in your email.)","type":"server_error","param":null,"code":null}}

• EventType: "error"

This leads to enumerable being null and therefore throwing the NullReferenceException with the following call stack:

Long call stack

> OpenAI.dll!OpenAI.Assistants.AsyncStreamingUpdateCollection.AsyncStreamingUpdateEnumerator.System.Collections.Generic.IAsyncEnumerator<OpenAI.Assistants.StreamingUpdate>.MoveNextAsync() Line 63	C#
[Resuming Async Method]	
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.InvokeMoveNext(object stateMachine) Line 50	C#
mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 515	C#
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 486	C#
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Line 36	C#
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.OutputAsyncCausalityEvents.AnonymousMethod__0() Line 142	C#
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke() Line 75	C#
System.Threading.Tasks.Extensions.dll!System.Runtime.CompilerServices.ValueTaskAwaiter..cctor.AnonymousMethod__9_0(object state)	Unknown
Microsoft.Bcl.AsyncInterfaces.dll!System.Threading.Tasks.Sources.ManualResetValueTaskSourceCore<bool>.InvokeContinuation()	Unknown
Microsoft.Bcl.AsyncInterfaces.dll!System.Threading.Tasks.Sources.ManualResetValueTaskSourceCore<bool>.SignalCompletion()	Unknown
Microsoft.Bcl.AsyncInterfaces.dll!System.Threading.Tasks.Sources.ManualResetValueTaskSourceCore<bool>.SetResult(bool result)	Unknown
OpenAI.dll!System.Net.ServerSentEvents.SseParser<byte[]>.EnumerateAsync(System.Threading.CancellationToken cancellationToken) Line 187	C#
[Resuming Async Method]	
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.InvokeMoveNext(object stateMachine) Line 50	C#
mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 515	C#
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 486	C#
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Line 36	C#
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.OutputAsyncCausalityEvents.AnonymousMethod__0() Line 142	C#
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke() Line 75	C#
mscorlib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__0() Line 158	C#
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke() Line 75	C#
mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask) Line 200	C#
mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Line 3145	C#
mscorlib.dll!System.Threading.Tasks.Task.FinishStageThree() Line 2442	C#
mscorlib.dll!System.Threading.Tasks.Task<int>.TrySetResult(int result) Line 304	C#
mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<int>.SetResult(int result) Line 277	C#
[Completed] OpenAI.dll!System.Net.ServerSentEvents.SseParser<byte[]>.FillLineBufferAsync(System.Threading.CancellationToken cancellationToken) Line 354	C#
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.InvokeMoveNext(object stateMachine) Line 50	C#
mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 515	C#
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 486	C#
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Line 36	C#
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.OutputAsyncCausalityEvents.AnonymousMethod__0() Line 142	C#
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke() Line 75	C#
mscorlib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__0() Line 158	C#
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke() Line 75	C#
mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask) Line 200	C#
mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Line 3145	C#
mscorlib.dll!System.Threading.Tasks.Task.FinishStageThree() Line 2442	C#
mscorlib.dll!System.Threading.Tasks.Task<int>.TrySetResult(int result) Line 304	C#
mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<int>.SetResult(int result) Line 277	C#
[Completed] System.ClientModel.dll!System.ClientModel.Internal.ReadTimeoutStream.ReadAsync(byte[] buffer, int offset, int count, System.Threading.CancellationToken cancellationToken)	Unknown
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.InvokeMoveNext(object stateMachine) Line 50	C#
mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 515	C#
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 486	C#
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Line 36	C#
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.OutputAsyncCausalityEvents.AnonymousMethod__0() Line 142	C#
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke() Line 75	C#
mscorlib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__0() Line 158	C#
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke() Line 75	C#
mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask) Line 200	C#
mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Line 3145	C#
mscorlib.dll!System.Threading.Tasks.Task.FinishStageThree() Line 2442	C#
mscorlib.dll!System.Threading.Tasks.Task<int>.TrySetResult(int result) Line 304	C#
mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<int>.SetResult(int result) Line 277	C#
[Completed] System.Net.Http.dll!System.Net.Http.HttpClientHandler.WebExceptionWrapperStream.ReadAsync(byte[] buffer, int offset, int count, System.Threading.CancellationToken cancellationToken)	Unknown
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.InvokeMoveNext(object stateMachine) Line 50	C#
mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 515	C#
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 486	C#
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Line 36	C#
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.OutputAsyncCausalityEvents.AnonymousMethod__0() Line 142	C#
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke() Line 75	C#
mscorlib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__0() Line 158	C#
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke() Line 75	C#
mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask) Line 200	C#
mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Line 3145	C#
mscorlib.dll!System.Threading.Tasks.Task.FinishStageThree() Line 2442	C#
mscorlib.dll!System.Threading.Tasks.Task<int>.TrySetResult(int result) Line 304	C#
mscorlib.dll!System.Threading.Tasks.TaskFactory<int>.FromAsyncTrimPromise<System.IO.Stream>.Complete(System.IO.Stream thisRef, System.Func<System.IO.Stream, System.IAsyncResult, int> endMethod, System.IAsyncResult asyncResult, bool requiresSynchronization) Line 60	C#
mscorlib.dll!System.Threading.Tasks.TaskFactory<int>.FromAsyncTrimPromise<System.IO.Stream>.CompleteFromAsyncResult(System.IAsyncResult asyncResult) Line 48	C#
System.dll!System.Net.LazyAsyncResult.Complete(System.IntPtr userToken)	Unknown
System.dll!System.Net.LazyAsyncResult.ProtectedInvokeCallback(object result, System.IntPtr userToken)	Unknown
System.dll!System.Net.ChunkParser.CompleteUserRead(object result)	Unknown
System.dll!System.Net.ChunkParser.CompletePayloadReadOperation(int bytesRead)	Unknown
System.dll!System.Net.ChunkParser.HandlePayload()	Unknown
System.dll!System.Net.ChunkParser.ProcessResponse()	Unknown
System.dll!System.Net.ChunkParser.ReadCallback(System.IAsyncResult ar)	Unknown
System.dll!System.Net.LazyAsyncResult.Complete(System.IntPtr userToken)	Unknown
System.dll!System.Net.LazyAsyncResult.ProtectedInvokeCallback(object result, System.IntPtr userToken)	Unknown
System.dll!System.Net.Security._SslStream.ProcessFrameBody(int readBytes, byte[] buffer, int offset, int count, System.Net.AsyncProtocolRequest asyncRequest)	Unknown
System.dll!System.Net.Security._SslStream.ReadFrameCallback(System.Net.AsyncProtocolRequest asyncRequest)	Unknown
System.dll!System.Net.AsyncProtocolRequest.CompleteRequest(int result)	Unknown
System.dll!System.Net.FixedSizeReader.CheckCompletionBeforeNextRead(int bytes)	Unknown
System.dll!System.Net.FixedSizeReader.ReadCallback(System.IAsyncResult transportResult)	Unknown
System.dll!System.Net.LazyAsyncResult.Complete(System.IntPtr userToken)	Unknown
System.dll!System.Net.ContextAwareResult.CompleteCallback(object state)	Unknown
mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 515	C#
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 486	C#
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Line 479	C#
System.dll!System.Net.ContextAwareResult.Complete(System.IntPtr userToken)	Unknown
System.dll!System.Net.LazyAsyncResult.ProtectedInvokeCallback(object result, System.IntPtr userToken)	Unknown
System.dll!System.Net.Sockets.BaseOverlappedAsyncResult.CompletionPortCallback(uint errorCode, uint numBytes, System.Threading.NativeOverlapped* nativeOverlapped)	Unknown
mscorlib.dll!System.Threading._IOCompletionCallback.PerformIOCompletionCallback(uint errorCode, uint numBytes, System.Threading.NativeOverlapped* pOVERLAP) Line 51	C#
[Native to Managed Transition]	
[Async Call Stack]	
[Async] OpenAI.dll!OpenAI.Assistants.AsyncStreamingUpdateCollection.GetValuesFromPageAsync(System.ClientModel.ClientResult page) Line 124	C#

It would be great if we could get at least kind of an OpenAIStreamingException or even just an Exception with a more detailed error message, if that's possible, so we can analyze what's going on without having a Debugger attached.

Steps to reproduce

We can't reproduce this often and our architecture is a bit more complex, but in general the steps are:

1. Call CreateRunStreaming:

AssistantClient client = 
  new OpenAIClient(new(apiKey), new() {ProjectId = projectId}).GetAssistantClient(); 
Assistant assistant =
  await client.GetAssistantAsync(options.AssistantId, cancellationToken).ConfigureAwait(false);
AssistantThread thread =
  await client.GetThreadAsync(threadId, cancellationToken).ConfigureAwait(false);

await client.CreateMessageAsync(
    thread.Id, MessageRole.User, [MessageContent.FromText(question)], null, cancellationToken)
  .ConfigureAwait(false);

ConfiguredCancelableAsyncEnumerable<StreamingUpdate> runStreamingEnumerable =
  client
    .CreateRunStreamingAsync(thread.Id, assistant.Id, cancellationToken: cancellationToken)
    .ConfigureAwait(false);

2. Iterate through StreamingUpdates:

await foreach (StreamingUpdate update in runStreamingEnumerable)
{
  if (update is MessageContentUpdate newMessage)
  {
      // do something with newMessage.Text;
  }
}

3. In some tries, it crashes at "do something"

Code snippets

(I have no further code snippets to provide)

OS

Windows

.NET version

.NET Framework 4.8

Library version

2.1.0 (c046c34)