Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Null Reference due to missing sqlAliasManager due to double nullable optimisation #35507

Open
ocdi opened this issue Jan 22, 2025 · 2 comments
Open

Comments

@ocdi
Copy link

ocdi commented Jan 22, 2025

Bug description

I have a query that after upgrading to EFCore 9 stopped working. The query in the main system is performing paginated syncing in chunks.

The bug appears to be in how the optimiser is handling nullability, with an IN clause having both sides nullable.

Your code

This code demonstrates the bug. The master branch is .NET9, downgrading to EFCore 8 + .NET8 the code runs (net8 branch).

https://github.com/ocdi/ef9bug

Note the code includes a bunch of extra complexity that mirrors more of what we are syncing, turns out the bug is triggered with much less in the first projection. The complexity is in how we're doing the pagination perhaps. It was difficult to trigger this code path, if I make OwnerId/OwnerType required on the Item class, the bug does not happen.

Stack traces

This is from debugging with symbols loaded

   at Microsoft.EntityFrameworkCore.Query.SqlExpressions.SelectExpression.PushdownIntoSubqueryInternal(Boolean liftOrderings) in /_/src/EFCore.Relational/Query/SqlExpressions/SelectExpression.cs:line 3648
   at Microsoft.EntityFrameworkCore.Query.SqlExpressions.SelectExpression.ApplyPredicate(SqlExpression sqlExpression) in /_/src/EFCore.Relational/Query/SqlExpressions/SelectExpression.cs:line 1543
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.VisitIn(InExpression inExpression, Boolean allowOptimizedExpansion, Boolean& nullable) in /_/src/EFCore.Relational/Query/SqlNullabilityProcessor.cs:line 842
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SqlExpression sqlExpression, Boolean allowOptimizedExpansion, Boolean preserveColumnNullabilityInformation, Boolean& nullable) in /_/src/EFCore.Relational/Query/SqlNullabilityProcessor.cs:line 459
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.VisitSqlBinary(SqlBinaryExpression sqlBinaryExpression, Boolean allowOptimizedExpansion, Boolean& nullable) in /_/src/EFCore.Relational/Query/SqlNullabilityProcessor.cs:line 1274
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SqlExpression sqlExpression, Boolean allowOptimizedExpansion, Boolean preserveColumnNullabilityInformation, Boolean& nullable) in /_/src/EFCore.Relational/Query/SqlNullabilityProcessor.cs:line 469
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SqlExpression sqlExpression, Boolean allowOptimizedExpansion, Boolean& nullable) in /_/src/EFCore.Relational/Query/SqlNullabilityProcessor.cs:line 427
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SelectExpression selectExpression, Boolean visitProjection) in /_/src/EFCore.Relational/Query/SqlNullabilityProcessor.cs:line 345
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SelectExpression selectExpression) in /_/src/EFCore.Relational/Query/SqlNullabilityProcessor.cs:line 290
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Process(Expression queryExpression, IReadOnlyDictionary`2 parameterValues, Boolean& canCache) in /_/src/EFCore.Relational/Query/SqlNullabilityProcessor.cs:line 85
   at Microsoft.EntityFrameworkCore.SqlServer.Query.Internal.SqlServerParameterBasedSqlProcessor.ProcessSqlNullability(Expression selectExpression, IReadOnlyDictionary`2 parametersValues, Boolean& canCache)
   at Microsoft.EntityFrameworkCore.Query.RelationalParameterBasedSqlProcessor.Optimize(Expression queryExpression, IReadOnlyDictionary`2 parametersValues, Boolean& canCache) in /_/src/EFCore.Relational/Query/RelationalParameterBasedSqlProcessor.cs:line 56
   at Microsoft.EntityFrameworkCore.SqlServer.Query.Internal.SqlServerParameterBasedSqlProcessor.Optimize(Expression queryExpression, IReadOnlyDictionary`2 parametersValues, Boolean& canCache)
   at Microsoft.EntityFrameworkCore.Query.Internal.RelationalCommandCache.GetRelationalCommandTemplate(IReadOnlyDictionary`2 parameters) in /_/src/EFCore.Relational/Query/Internal/RelationalCommandCache.cs:line 73
   at Microsoft.EntityFrameworkCore.Internal.RelationalCommandResolverExtensions.RentAndPopulateRelationalCommand(RelationalCommandResolver relationalCommandResolver, RelationalQueryContext queryContext) in /_/src/EFCore.Relational/Extensions/Internal/RelationalCommandResolverExtensions.cs:line 27
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.InitializeReader(Enumerator enumerator) in /_/src/EFCore.Relational/Query/Internal/SingleQueryingEnumerable.cs:line 263
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.<>c.<MoveNext>b__21_0(DbContext _, Enumerator enumerator) in /_/src/EFCore.Relational/Query/Internal/SingleQueryingEnumerable.cs:line 199
   at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.MoveNext() in /_/src/EFCore.Relational/Query/Internal/SingleQueryingEnumerable.cs:line 257

This is the console output from the test project linked.

fail: Microsoft.EntityFrameworkCore.Query[10100]
      An exception occurred while iterating over the results of a query for context type 'EF9Bug.TestContext'.
      System.NullReferenceException: Object reference not set to an instance of an object.
         at Microsoft.EntityFrameworkCore.Query.SqlExpressions.SelectExpression.PushdownIntoSubqueryInternal(Boolean liftOrderings)
         at Microsoft.EntityFrameworkCore.Query.SqlExpressions.SelectExpression.ApplyPredicate(SqlExpression sqlExpression)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.VisitIn(InExpression inExpression, Boolean allowOptimizedExpansion, Boolean& nullable)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SqlExpression sqlExpression, Boolean allowOptimizedExpansion, Boolean preserveColumnNullabilityInformation, Boolean& nullable)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.VisitSqlBinary(SqlBinaryExpression sqlBinaryExpression, Boolean allowOptimizedExpansion, Boolean& nullable)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SqlExpression sqlExpression, Boolean allowOptimizedExpansion, Boolean preserveColumnNullabilityInformation, Boolean& nullable)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SqlExpression sqlExpression, Boolean allowOptimizedExpansion, Boolean& nullable)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SelectExpression selectExpression, Boolean visitProjection)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SelectExpression selectExpression)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Process(Expression queryExpression, IReadOnlyDictionary`2 parameterValues, Boolean& canCache)
         at Microsoft.EntityFrameworkCore.SqlServer.Query.Internal.SqlServerParameterBasedSqlProcessor.ProcessSqlNullability(Expression selectExpression, IReadOnlyDictionary`2 parametersValues, Boolean& canCache)
         at Microsoft.EntityFrameworkCore.Query.RelationalParameterBasedSqlProcessor.Optimize(Expression queryExpression, IReadOnlyDictionary`2 parametersValues, Boolean& canCache)
         at Microsoft.EntityFrameworkCore.SqlServer.Query.Internal.SqlServerParameterBasedSqlProcessor.Optimize(Expression queryExpression, IReadOnlyDictionary`2 parametersValues, Boolean& canCache)
         at Microsoft.EntityFrameworkCore.Query.Internal.RelationalCommandCache.GetRelationalCommandTemplate(IReadOnlyDictionary`2 parameters)
         at Microsoft.EntityFrameworkCore.Internal.RelationalCommandResolverExtensions.RentAndPopulateRelationalCommand(RelationalCommandResolver relationalCommandResolver, RelationalQueryContext queryContext)
         at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.InitializeReader(Enumerator enumerator)
         at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.<>c.<MoveNext>b__21_0(DbContext _, Enumerator enumerator)
         at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
         at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.MoveNext()
      System.NullReferenceException: Object reference not set to an instance of an object.
         at Microsoft.EntityFrameworkCore.Query.SqlExpressions.SelectExpression.PushdownIntoSubqueryInternal(Boolean liftOrderings)
         at Microsoft.EntityFrameworkCore.Query.SqlExpressions.SelectExpression.ApplyPredicate(SqlExpression sqlExpression)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.VisitIn(InExpression inExpression, Boolean allowOptimizedExpansion, Boolean& nullable)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SqlExpression sqlExpression, Boolean allowOptimizedExpansion, Boolean preserveColumnNullabilityInformation, Boolean& nullable)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.VisitSqlBinary(SqlBinaryExpression sqlBinaryExpression, Boolean allowOptimizedExpansion, Boolean& nullable)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SqlExpression sqlExpression, Boolean allowOptimizedExpansion, Boolean preserveColumnNullabilityInformation, Boolean& nullable)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SqlExpression sqlExpression, Boolean allowOptimizedExpansion, Boolean& nullable)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SelectExpression selectExpression, Boolean visitProjection)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SelectExpression selectExpression)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Process(Expression queryExpression, IReadOnlyDictionary`2 parameterValues, Boolean& canCache)
         at Microsoft.EntityFrameworkCore.SqlServer.Query.Internal.SqlServerParameterBasedSqlProcessor.ProcessSqlNullability(Expression selectExpression, IReadOnlyDictionary`2 parametersValues, Boolean& canCache)
         at Microsoft.EntityFrameworkCore.Query.RelationalParameterBasedSqlProcessor.Optimize(Expression queryExpression, IReadOnlyDictionary`2 parametersValues, Boolean& canCache)
         at Microsoft.EntityFrameworkCore.SqlServer.Query.Internal.SqlServerParameterBasedSqlProcessor.Optimize(Expression queryExpression, IReadOnlyDictionary`2 parametersValues, Boolean& canCache)
         at Microsoft.EntityFrameworkCore.Query.Internal.RelationalCommandCache.GetRelationalCommandTemplate(IReadOnlyDictionary`2 parameters)
         at Microsoft.EntityFrameworkCore.Internal.RelationalCommandResolverExtensions.RentAndPopulateRelationalCommand(RelationalCommandResolver relationalCommandResolver, RelationalQueryContext queryContext)
         at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.InitializeReader(Enumerator enumerator)
         at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.<>c.<MoveNext>b__21_0(DbContext _, Enumerator enumerator)
         at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
         at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.MoveNext()
Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.EntityFrameworkCore.Query.SqlExpressions.SelectExpression.PushdownIntoSubqueryInternal(Boolean liftOrderings)
   at Microsoft.EntityFrameworkCore.Query.SqlExpressions.SelectExpression.ApplyPredicate(SqlExpression sqlExpression)
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.VisitIn(InExpression inExpression, Boolean allowOptimizedExpansion, Boolean& nullable)
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SqlExpression sqlExpression, Boolean allowOptimizedExpansion, Boolean preserveColumnNullabilityInformation, Boolean& nullable)
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.VisitSqlBinary(SqlBinaryExpression sqlBinaryExpression, Boolean allowOptimizedExpansion, Boolean& nullable)
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SqlExpression sqlExpression, Boolean allowOptimizedExpansion, Boolean preserveColumnNullabilityInformation, Boolean& nullable)
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SqlExpression sqlExpression, Boolean allowOptimizedExpansion, Boolean& nullable)
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SelectExpression selectExpression, Boolean visitProjection)
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SelectExpression selectExpression)
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Process(Expression queryExpression, IReadOnlyDictionary`2 parameterValues, Boolean& canCache)
   at Microsoft.EntityFrameworkCore.SqlServer.Query.Internal.SqlServerParameterBasedSqlProcessor.ProcessSqlNullability(Expression selectExpression, IReadOnlyDictionary`2 parametersValues, Boolean& canCache)
   at Microsoft.EntityFrameworkCore.Query.RelationalParameterBasedSqlProcessor.Optimize(Expression queryExpression, IReadOnlyDictionary`2 parametersValues, Boolean& canCache)
   at Microsoft.EntityFrameworkCore.SqlServer.Query.Internal.SqlServerParameterBasedSqlProcessor.Optimize(Expression queryExpression, IReadOnlyDictionary`2 parametersValues, Boolean& canCache)
   at Microsoft.EntityFrameworkCore.Query.Internal.RelationalCommandCache.GetRelationalCommandTemplate(IReadOnlyDictionary`2 parameters)
   at Microsoft.EntityFrameworkCore.Internal.RelationalCommandResolverExtensions.RentAndPopulateRelationalCommand(RelationalCommandResolver relationalCommandResolver, RelationalQueryContext queryContext)
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.InitializeReader(Enumerator enumerator)
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.<>c.<MoveNext>b__21_0(DbContext _, Enumerator enumerator)
   at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.MoveNext()
   at System.Linq.Enumerable.<ToArray>g__EnumerableToArray|314_0[TSource](IEnumerable`1 source)
   at Program.<Main>$(String[] args) in D:\code\EF9Bug\Program.cs:line 53

Verbose output


EF Core version

9.0.1

Database provider

SqlServer

Target framework

.NET 9.0

Operating system

No response

IDE

No response

@roji
Copy link
Member

roji commented Jan 22, 2025

Confirmed, thanks for submitting @ocdi - see below for a minimal repro (it's always appreciated if the repro can be narrowed down as much as possible, like the below).

await using var context = new TestContext();
await context.Database.EnsureDeletedAsync();
await context.Database.EnsureCreatedAsync();

_ = context
    .Set<Item>()
    .Select(a => a.OwnerId!)
    .Where(a => context
        .Set<Item>()
        .Select(a => a.OwnerId)
        .Take(100)
        .Contains(a))
    .ToListAsync();

public class Item
{
    public int Id { get; set; }
    public string? OwnerId { get; set; }
}

public class TestContext : DbContext
{
    public DbSet<Item> Items { get; set; }

    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        => optionsBuilder
            .UseSqlServer("Server=localhost;Database=test;User=SA;Password=Abcd5678;Connect Timeout=60;ConnectRetryCount=0;Encrypt=false")
            .LogTo(Console.WriteLine, LogLevel.Information)
            .EnableSensitiveDataLogging();
}

@roji
Copy link
Member

roji commented Jan 22, 2025

Probably the same as #35192.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants