Optimize Microsoft.Extensions.Caching.Memory

[pentp]

Remove some allocations and interface calls.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-extensions-caching
See info in area-owners.md if you want to be subscribed.

[mgravell]

The main win here is not eagerly allocating the options lists, is that right? Plus using the list-specific iterator instead of IEnumerable-T/IEnumerator-T. Definitely worthwhile - however, I'm not a huge fan of exposing the naked field, even as internal. I wonder if we could have a private field and an internal get that is the span via collectionsmarshal, and return the default/empty span if the list is absent. Alternatively, maybe an internal HasFoos => _foos is { Length :> 0 }; and just add an if (options.HasFoos) ? (although then we still need to think about the enumerator - the span approach avoids that entirely, and should be more direct than the custom list iterator, additionally skipping bounds checks)

[pentp]

Yes, the allocation savings come from on-demand allocation of options lists and avoiding an enumerator allocation each time when using options. And skipping options allocation entirely in MemoryDistributedCache.Set.

I'll try the Span based variant - it could actually generate less code compared to List<T>.Enumerator.
Nvm, CollectionsMarshal is only available on .NET 5+.

[mgravell]

Yes, the allocation savings come from on-demand allocation of options lists and avoiding an enumerator allocation each time when using options. And skipping options allocation entirely in MemoryDistributedCache.Set.

I'll try the Span based variant - it could actually generate less code compared to List<T>.Enumerator. Nvm, CollectionsMarshal is only available on .NET 5+.

Since this is an internal API and isn't being used by IVTA hackery except for the tests, we could actually cheat by having the property typed differently on different runtimes, but perhaps we can keep things simple

private List<Whatever>? _foos;
public IList<Whatever> Foos => _foos ??= []; // lazy alloc
internal List<Whatever>? FoosDirect => _foos; // note never allocs

This adds a line of code, but is IMO significantly preferable to exposing the naked field.

[pentp]

This should be ready for merging if there are no other concerns.