Once, on one of my projects, we decided to upgrade from .NET Core 2.1 to the latest LTS version (3.1 at the time of writing). Prior to this, I had several experiences updating small projects, both from classic .NET and older versions of .NET Core. However, most of these projects used Dapper instead of full-fledged ORMs. I had only updated EntityFrameworkCore directly a couple of times, and it’s worth noting that I didn’t encounter any significant problems during those updates. Until this moment.

So the process went like this: first an update to 2.2, then to 3.0, and finally to 3.1. After each update, a minimal check was performed: ensuring that the application starts and executes the most basic queries. Everything seemed to be going fine, but during the final testing of the application on version 3.1, I began to notice that the data retrieved using EF was throwing a NullReferenceException. After some time, I noticed that the issue occurred when there was a specific construct in the query, namely .Include(p => p.NavProperty).

Here is a simplified example of code that was throwing the error:

var profiles = await context.Profiles
    .Include(p => p.User)
    .ToArrayAsync();

foreach (var profile in profiles) {
    await SendEmail(profile.User.Email, CreateNotification(profile));
}

The error occurred when accessing the Email property, as User turned out to be null The possibility of such a situation was immediately ruled out, as User and Profile entities are created simultaneously and cannot be deleted from the application in principle. Additionally, a check of the database showed that all Profiles reference the corresponding Users.

The table schema represents a one-to-one relationship (in this example, all tables are simplified and only a couple of fields are kept for clarity):

The table schema

So first, let’s look at the Entity configuration. All that was found there is a correct setup of the one-to-one relationship with Id specified as the FK key:

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
    modelBuilder.Entity<User>()
        .HasOne(p => p.Profile)
        .WithOne(p => p.User)
        .HasForeignKey<Profile>(p => p.Id);

    modelBuilder.Entity<Profile>()
        .HasOne(p => p.User)
        .WithOne(p => p.Profile)
        .HasForeignKey<User>(p => p.Id);

    base.OnModelCreating(modelBuilder);
}

Следующее логичное предположение то, что генерируется некорректный SQL код. Но если посмотреть данный запрос под профайлером, мы внезапно обнаружим, что запрос полностью корректен (запрос был отформатирован для простоты чтения):

SELECT
    [p].[Id],
    [p].[FirstName],
    [p].[LastName],
    [u].[Id],
    [u].[Email]
FROM [Profiles] AS [p]
LEFT JOIN [Users] AS [u] ON [p].[Id] = [u].[Id]

The next logical assumption is that incorrect SQL code is being generated. However, if we look at the query under a profiler, we suddenly find that the query is completely correct (the query has been formatted for readability):

Result of the query execution

In the application, however, we get this (for clarity, serialized data will be presented in JSON format; all data is taken from the example at the end of the article in the file Cases/Case1.cs):

[
  { "Id": 1, "FirstName": "Ellen", "LastName": "Hunt", "User": null },
  { "Id": 2, "FirstName": "Lisa", "LastName": "Lilly", "User": null },
  { "Id": 3, "FirstName": "Brianna", "LastName": "Salter", "User": null },
  { "Id": 4, "FirstName": "Aidan", "LastName": "Synnot", "User": null },
  { "Id": 5, "FirstName": "Alice", "LastName": "Paten", "User": null }
]

The next attempt was to modify the query to select the NavigationProperty User into a separate field of the object. Although the correctly constructed query in the previous example ruled out the possibility of incorrect optimization, it was still desirable to see the result.

The query was rewritten as follows:

var profiles = await context.Profiles
    .Include(p => p.User)
    .Select(p => new
    {
        User = p.User,
        Profile = p
    })
    .ToArrayAsync();

The SQL query did not change at all, but the mapping result was completely different. Now, the framework was able to retrieve both User and Profile, and even populate the property that was previously null. However, for the sake of fairness, it should be noted that deeper Navigation Properties still remained null (this case can be found in the example Cases/Case2.cs).

[
  {
    "User": {
      "Id": 1,
      "Email": "user1@main.com",
      "Profile": {
        "Id": 1,
        "FirstName": "Ellen",
        "LastName": "Hunt"
      }
    },
    "Profile": {
      "Id": 1,
      "FirstName": "Ellen",
      "LastName": "Hunt",
      "User": {
        "Id": 1,
        "Email": "user1@main.com"
      }
    }
  }
]

Further experiments led to the conclusion that adding .AsNoTracking() to the query makes Entity Framework load the data without any issues (this case can be found in the example Cases/Case3.cs).

var profiles = await context.Profiles
    .Include(p => p.User)
    .AsNoTracking()
    .ToArrayAsync();

[
  {
    "Id": 1,
    "FirstName": "Ellen",
    "LastName": "Hunt",
    "User": { "Id": 1, "Email": "user1@main.com" }
  },
  {
    "Id": 2,
    "FirstName": "Lisa",
    "LastName": "Lilly",
    "User": { "Id": 2, "Email": "user2@main.com" }
  },
  {
    "Id": 3,
    "FirstName": "Brianna",
    "LastName": "Salter",
    "User": { "Id": 3, "Email": "user3@main.com" }
  },
  {
    "Id": 4,
    "FirstName": "Aidan",
    "LastName": "Synnot",
    "User": { "Id": 4, "Email": "user4@main.com" }
  },
  {
    "Id": 5,
    "FirstName": "Alice",
    "LastName": "Paten",
    "User": { "Id": 5, "Email": "user5@main.com" }
  }
]

And the last symptom of strange behavior that I found is that users already preloaded into the context are mapped correctly during the query (this example is Cases/Case4.cs):

var userIds = new[] {3, 5};

var preloadedUsers = await context.Users
    .Where(p => userIds.Contains(p.Id))
    .ToListAsync();

var items = await context.Profiles
    .Include(p => p.User)
    .ToArrayAsync();

And this code will produce the following result — for profiles with Id 3 and 5, the User was successfully loaded.

[
  { "Id": 1, "FirstName": "Ellen", "LastName": "Hunt", "User": null },
  { "Id": 2, "FirstName": "Lisa", "LastName": "Lilly", "User": null },
  { "Id": 3, "FirstName": "Brianna", "LastName": "Salter", "User": { "Id": 3, "Email": "user3@main.com" } },
  { "Id": 4, "FirstName": "Aidan", "LastName": "Synnot", "User": null },
  { "Id": 5, "FirstName": "Alice", "LastName": "Paten", "User": { "Id": 5, "Email": "user5@main.com" } }
]

It looks very strange and definitely behaves like a bug. However, the cause was equally surprising. The culprit was the model definition; find the error here:

public class User
{
    public User()
    {
        Profile = new Profile();
    }

    public int Id { get; set; }
    public string Email { get; set; }

    public Profile Profile { get; set; }
}

Of course, this is very noticeable in the test example, but in a real project, I didn’t pay attention to it, which cost me a few hundred nerve cells. Yes, indeed, the creation of an empty Profile was to blame. Simply removing the constructor completely resolves the issue. For some reason, Entity Framework Core in recent versions cannot correctly map data if something was already written to the property. Instead of retaining the field’s value, EF forcibly sets it to null, which, in my opinion, complicates understanding the problem. In EF version 2.1, it correctly handles such initialized properties. This can be seen in a small example.

This example involves three console applications. For the sake of the experiment, a separate copy of correctly and incorrectly configured contexts was created for each version of .NET. Additionally, each example includes all the query cases described in the article. Don’t forget to set the correct ConnectionStrings in appsettings.json.

  • DemoConsoleApp5 - Example behavior based on .NET Core 5.0 RC
  • DemoConsoleApp31 - Example behavior based on dotnet core 3.1
  • DemoConsoleApp21 - Example of all queries based on .NET Core 2.1, in which the loss of data loaded via .Include did not occur. This example is provided for clarity.