-
Notifications
You must be signed in to change notification settings - Fork 540
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Pomelo.EntityFrameworkCore.MySql component #1161
Conversation
We are adding support for (The next official release will be |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for another contribution, @bgrainger
I have added the 2 new nuget packages to the internal mirror. They should be finished mirroring in a few minutes.
...Components/Aspire.Pomelo.EntityFrameworkCore.MySql/PomeloEntityFrameworkCoreMySqlSettings.cs
Outdated
Show resolved
Hide resolved
src/Components/Aspire.Pomelo.EntityFrameworkCore.MySql/AspireEFMySqlExtensions.cs
Outdated
Show resolved
Hide resolved
src/Components/Aspire.Pomelo.EntityFrameworkCore.MySql/AspireEFMySqlExtensions.cs
Outdated
Show resolved
Hide resolved
src/Components/Aspire.Pomelo.EntityFrameworkCore.MySql/AspireEFMySqlExtensions.cs
Outdated
Show resolved
Hide resolved
@@ -0,0 +1,120 @@ | |||
// Licensed to the .NET Foundation under one or more agreements. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Once #1217 is merged, can you update the integration tests to also include this new component?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure I understand the ask here. #1217 seems to add tests for Aspire.Hosting. That project (and Aspire.Hosting.Tests) has no dependency (and should have no dependency) on Pomelo.EFCore.MySql (from this PR).
Are there more MySQL functional tests you would like to see added to Aspire.Hosting.Tests (that are logically more of a follow-up to #825)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
#1217 added tests that test the end-to-end of both the hosting and the runtime components. These new tests ensure that the very basic runtime component can connect to a Docker container created by the Hosting layer. They are "integration tests" that ensure both sides work correctly with each other.
We only have 1 MySQL "hosting" set of APIs that spins up the MySQL server when the App starts.
But now we have 2 MySQL runtime components - one for just pure MySQL and one for EF Core + MySQL. We should have tests that ensure both runtime components work with the hosting pieces.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for being dense, but I'm still not understanding what's needed here.
How is the MySQL scenario (MySqlConnector vs Pomelo.EntityFrameworkCore.MySql) different from PostgreSQL (Npgsql vs Npgsql.EntityFrameworkCore.PostgreSQL)? I don't see any EF-specific tests in Aspire.Hosting.Tests so it's still not clear to me what a test for the second runtime component would look like. Can you point to a Npgsql or SQL Server example that can be used as a template?
(Just as Npgsql.EntityFrameworkCore.PostgreSQL depends on Npgsql for its DB connectivity, Pomelo.EntityFrameworkCore.MySql depends on MySqlConnector, so if MySqlConnector can successfully connect to a test Docker container, Pomelo EFCore should be able to, too.)
This PR adds the Aspire.Pomelo.EntityFrameworkCore.MySql.Tests project, which I thought was sufficient for testing Aspire.Pomelo.EntityFrameworkCore.MySql.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How is the MySQL scenario (MySqlConnector vs Pomelo.EntityFrameworkCore.MySql) different from PostgreSQL (Npgsql vs Npgsql.EntityFrameworkCore.PostgreSQL)?
It isn't different.
I don't see any EF-specific tests in Aspire.Hosting.Tests
These tests are very new, and not yet complete. I've opened #1566 for them.
so it's still not clear to me what a test for the second runtime component would look like.
It would look the same as the non-EF version. Only instead of using the non-EF component, it would use the EF version. The "hosting" side of the test would still be the exact same - we only need 1 MySQL server/container running. But we want to verify that each components work end-to-end with the Hosting pieces. We've had some early bugs that would have been caught if we had these tests to start with.
This PR adds the Aspire.Pomelo.EntityFrameworkCore.MySql.Tests project, which I thought was sufficient for testing Aspire.Pomelo.EntityFrameworkCore.MySql.
They are a good start, but it doesn't verify that the Aspire.Pomelo.EntityFrameworkCore.MySql component works with the MySql hosting pieces.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the pointers; I think I finally understand what you're suggesting and have added dedicated integration tests for the Pomelo component in 817940b.
Conflicts: Aspire.sln
Co-authored-by: Eric Erhardt <[email protected]>
Co-authored-by: Eric Erhardt <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a few comments. Once they are addressed I think this is ready to be merged.
tests/Aspire.Pomelo.EntityFrameworkCore.MySql.Tests/TestDbContext.cs
Outdated
Show resolved
Hide resolved
src/Components/Aspire.Pomelo.EntityFrameworkCore.MySql/ConfigurationSchema.json
Show resolved
Hide resolved
src/Components/Aspire.Pomelo.EntityFrameworkCore.MySql/README.md
Outdated
Show resolved
Hide resolved
src/Components/Aspire.Pomelo.EntityFrameworkCore.MySql/AspireEFMySqlExtensions.cs
Outdated
Show resolved
Hide resolved
@@ -0,0 +1,120 @@ | |||
// Licensed to the .NET Foundation under one or more agreements. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
#1217 added tests that test the end-to-end of both the hosting and the runtime components. These new tests ensure that the very basic runtime component can connect to a Docker container created by the Hosting layer. They are "integration tests" that ensure both sides work correctly with each other.
We only have 1 MySQL "hosting" set of APIs that spins up the MySQL server when the App starts.
But now we have 2 MySQL runtime components - one for just pure MySQL and one for EF Core + MySQL. We should have tests that ensure both runtime components work with the hosting pieces.
Conflicts: Aspire.sln src/Components/Aspire_Components_Progress.md src/Components/Telemetry.md
Co-authored-by: Eric Erhardt <[email protected]>
using Aspire; | ||
using Aspire.Pomelo.EntityFrameworkCore.MySql; | ||
|
||
[assembly: ConfigurationSchema("Aspire:Pomelo:EntityFrameworkCore:MySql", typeof(PomeloEntityFrameworkCoreMySqlSettings))] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should also add all the logging categories that are in Telemetry.md. That way they light up in the appsettings.json file intellisense.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I shouldn't have duplicated the MySqlConnector logging categories into the Pomelo section in Telemetry.md. (I assume they will ultimately get pulled in through being a transitive dependency?) I'll remove them from Telemetry.md.
src/Components/Aspire.Pomelo.EntityFrameworkCore.MySql/AspireEFMySqlExtensions.cs
Outdated
Show resolved
Hide resolved
...nents/Aspire.Pomelo.EntityFrameworkCore.MySql/Aspire.Pomelo.EntityFrameworkCore.MySql.csproj
Show resolved
Hide resolved
@bgrainger - just checking in here. Do you have an idea on the timeline for this PR? We are planning on getting an Aspire preview3 out in the next few weeks. I was wondering if this would be ready in time for that. |
I think I have time to look at this more next week. Is there a hard cut-off date to make it into Preview 3? Did we land in a "good enough" place with Then the remaining big component is "tests that test the end-to-end of both the hosting and the runtime components" (plus a few other minor still-unresolved comments you left)? Thanks for all the help and feedback with this PR so far! |
Our plan is to snap the release branch on January 29.
I think what you have now is good enough for the first preview. I agree that the underlying Pomelo behavior could be enhanced, but it is out of scope for this PR.
👍 |
Are we getting this for preview3? |
Conflicts: Aspire.sln src/Components/Aspire_Components_Progress.md
Co-authored-by: Eric Erhardt <[email protected]>
This reverts commit f133452.
These were modeled on AddPostgresTests and AddRedisTests.
I've added what I think the Pomelo EFCore E2E tests should look like. Since these are the first EF E2E tests (AFAICT) I may have approached it differently from what you expected. Let me know if you want any changes there. This PR is ready for review. |
{ | ||
ThrowForMissingConnectionString(); | ||
} | ||
serverVersion = ServerVersion.AutoDetect(connectionString); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Playing with this change locally, I'm a bit concerned about this now. I'm trying to run the integration tests (which aren't run in CI yet - we are working on that). If the Pomelo test is run by itself, it fails because just trying to get the DbContext out of DI is trying to establish a connection. But the MySql docker container isn't up yet (it takes a while to get up) and this line is throwing because it can't make a connection.
I wonder if we should retry this up to the MaxRetryCount. It isn't a great experience to fail getting the DbContext because it happens as part of ASP.NET calling into the user's code. So there's no chance for them to retry.
Thoughts? Maybe this shouldn't block the initial check-in, since you can work around the issue by setting ServerVersion explicitly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if we should retry this up to the MaxRetryCount.
And then still fail with a fatal error if it can't make a connection? (As opposed to making up a value like "8.0.0"
or "5.7.28"
or something that might not actually be safe if the user is running MariaDB, depending on how much this changes Pomelo's behavior.)
That definitely seems like an improvement over the status quo to me. (Unless of course 100% of the time it still fails but now just takes longer.)
In my original PR (before 2623f93), this was a mandatory option that had to be supplied by the user. That was less user-friendly on one hand, but maybe more predictable?
Very random thought (and may be going completely down the wrong path): is there some "magic" we could do by detecting if there's a ContainerImageAnnotation
? E.g., set an appropriate default ServerVersion by faking a version string based on whether the user is using mysql:latest
or mariadb:11.0
? (That's not actually customisable in AddMySqlContainer
right now though maybe it should be.) That only works for the local Docker scenario, but maybe that's OK because for production deployment we can assume that the "real" database is up and serving requests?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's discuss the options in a new issue explicitly for this. Can you open it? or do you want me to?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Add log categories to ConfigurationSchema - Check the "API" boxes for the MySql components in the Progress doc - Issue a query in the functional test. Follow the same pattern as Oracle - Set the ServerVersion explicitly in the functional test, so the connection can be retried when the MySql server isn't running.
This is looking great, @bgrainger. Thanks for another great contribution. I had a few tweaks that I made in mysql-net#1 (I couldn't push to your branch directly). Once those changes are made, I think this will be ready to merge. We can log a follow up issue on my comment above. |
Well, not sure how I missed the Oracle ones, but now I see them plain as day! Thanks for pushing the fixes directly; pulled that into this branch. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks great. I'll merge this either tonight or tomorrow to give other people time to review.
* Change default port for dashboard applicationUrl (#1853) Windows allows systems to exclude specific port ranges, after which attempts to bind them will fail. The ranges will vary from machine to machine, and can be viewed with the command `netsh interface ipv4 show excludedportrange protocol=tcp`. We're seeing users hitting issues with port 63824. While the excluded ranges will vary from machine to machine, it seems like these higher numbers are more likely to be excluded. This change moves the ports lower, which should help alleviate this issue. * Add details column to trace view, remove status badge and put counter badge on resource name (#1788) * add details column to trace detail * Update xlfs, style button * remove error badge * Put unread error counter on name * update trace detail details column to absolute width * Remove unused resx strings, re-add titles for resource name * Make error counter badge text white in dark theme * fixed variable name --------- Co-authored-by: Adam Ratzman <[email protected]> * Fix horizontal overflow in metrics popup (#1838) * Remove unused "known property" getter methods (#1856) This code is no longer used, so remove it. * Update dashboard README (#1854) * Fix trace detail page's gap between trace ticks (#1861) * Fix SQL server resource version (#1870) * Add Pomelo.EntityFrameworkCore.MySql component (#1161) * Add Pomelo.EntityFrameworkCore.MySql component. * Add Pomelo.EntityFrameworkCore.MySql tests. * Update XML documentation. * Update XML documentation to current conventions. * Add documentation for Pomelo metrics. * Use configuration schema generator. * Allow ServerVersion setting to be optional. * Remove (now optional) ServerVersion setting from test. * Remove MySqlConnector logging categories from Pomelo section. * Add Microsoft.Extensions.Configuration.Binder. * Add basic MySQL Aspire.Hosting tests. These were modeled on AddPostgresTests and AddRedisTests. * Add Pomelo EFCore integration tests. * Incorporate project change from dff1467. * PR feedback - Add log categories to ConfigurationSchema - Check the "API" boxes for the MySql components in the Progress doc - Issue a query in the functional test. Follow the same pattern as Oracle - Set the ServerVersion explicitly in the functional test, so the connection can be retried when the MySql server isn't running. --------- Co-authored-by: Eric Erhardt <[email protected]> * Fix TryAddWillNotAddTheSameLifecycleHook test. (#1862) * Use the right health check properties in ServiceBus README. (#1876) Fix #1715 * Trace detail page - customize peer icons based on span attributes (#1865) * Update SqlServerBuilderExtensions.cs (#1883) Follow up to #1870 * Fix Storage Queue and Cosmos EF READMEs (#1884) * Fix Storage Queue README The ConnectionStrings setting had the wrong name, so it doesn't work with what is in the code section above it. Fix #1679 * Fix CosmosDB README The extension method takes 2 strings: connection name and database name. Fix #873 * remove link in name column (#1886) Co-authored-by: Adam Ratzman <[email protected]> * Orleans: enable distributed tracing by default (#1858) * add failedtostart error icon to state (#1887) * add failedtostart error icon to state * add starting icon --------- Co-authored-by: Adam Ratzman <[email protected]> * Fix grid layout issue (#1889) * Update dependencies from https://github.com/microsoft/usvc-apiserver build 0.1.49+5 (#1890) Microsoft.DeveloperControlPlane.darwin-amd64 , Microsoft.DeveloperControlPlane.darwin-arm64 , Microsoft.DeveloperControlPlane.linux-amd64 , Microsoft.DeveloperControlPlane.linux-arm64 , Microsoft.DeveloperControlPlane.windows-386 , Microsoft.DeveloperControlPlane.windows-amd64 , Microsoft.DeveloperControlPlane.windows-arm64 From Version 0.1.48 -> To Version 0.1.49 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> * Fix inconsistent trace detail spacing (#1863) * Fix inconsistent trace detail spacing * Clean up * Playground app for SQL Server resources (uses EF). (#1895) * End to end SQL playground. * Branding changes for preview 4 (#1892) * Fix dashboard not working with two applicationUrls specified. (#1901) * Fix dashboard not working with two applicationUrls specified. * PR feedback. Defensive code. * Remove unnecessary code from AppHost in Orleans playground (#1904) * remove unnecessary code from Orleans AppHost * format comment in Orleans AppHost * Added a mongo playground project (#1921) * Consistent resource type summaries (#1772) * Removed static web assets from aspire hosting (#1923) - We're no longer embedding the dashboard so this shouldn't be required. * Transform outgoing address to try to find a resource match (#1932) * Improve environment variable debugging in the app model (#1746) * Move shared code to shared directory (#1751) * Fixed build from shared code change (#1937) * Allow collapsing operations in the trace view (#1323) Co-authored-by: James Newton-King <[email protected]> * Update Extensions to preview 1 versions (#1891) * Update dependencies from https://github.com/dotnet/arcade build 20240125.5 (#1947) Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Installers , Microsoft.DotNet.Build.Tasks.Workloads , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.RemoteExecutor , Microsoft.DotNet.SharedFramework.Sdk , Microsoft.DotNet.XUnitExtensions From Version 8.0.0-beta.24060.4 -> To Version 8.0.0-beta.24075.5 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> * Update dependencies from https://github.com/dotnet/extensions build 20240126.7 (#1954) Microsoft.Extensions.Http.Resilience From Version 9.0.0-alpha.1.24076.5 -> To Version 9.0.0-preview.1.24076.7 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> * Fix dotnet watch. (#1933) * Fix dotnet watch. * Fix precondition in DcpHostService (#1943) * Fix precondition in DcpHostService Two lines are changed here, though it's the second (in `StopAsync`) that has a functional change. The first guard clause allows a null publisher, or a "dcp" publisher. The second clause would allow any non-null publisher, which looks like a bug to me. Specifically, the first half of the || excludes "dcp", then the second half allows it back in. This means that there's no point checking for "dcp" in the code as written. My interpretation of this code is that both clauses are intended to be the same, and this commit brings them in line with one another. * Deduplicate condition * Fix histogram chart calculation (#1968) Co-authored-by: Drew Noakes <[email protected]> * Fix exception when app host has no ASPNETCORE_URLS (#1970) If an app host project's launch profile doesn't specify a `launchUrl` or `applicationUrl` then the running app host won't have a `ASPNETCORE_URLS` environment variable. We want to pass this variable's value to the dashboard process, and currently if it's missing we throw. The dashboard executable has a fallback value when the `ASPNETCORE_URLS` variable is not present (of `http://localhost:18888`), so it's perfectly happy to run without this variable. This change causes the app host to fall back to that same default value for the dashboard's `ASPNETCORE_URLS` variable. While we could pass no value at all, and have the dashboard choose the default, it's more helpful to the user if we log the dashboard URL. This allows the user to find the dashboard, as the IDE will not be launching it automatically when in this state. * Revert state column badge changes (#1955) * revert state column badge changes * Slightly darken error in dark theme * Remove spurious VS-added itemgroups * Auto-close empty element --------- Co-authored-by: Adam Ratzman <[email protected]> * Update OpenTelemetry packages and use built-in metric methods (#1948) * Update OpenTelemetry packages - Update all the OpenTelemetry packages to their latest versions. - Use built-in methods to add .NET metrics. - Add suppression for new AoT warning on `AddSqlClientInstrumentation()`. * Resolve review feedback Mark as not AoT compatible due to dependency on OpenTelemetry.Instrumentation.SqlClient, which itself is not AoT compatible. * Prevent ANE when resource state is null (#1975) Ideally we'd make the field nullable on the proto definition, but that's a breaking change at this point. So instead, we pass an empty string. Other options are discussed in #1930. * Remove PublishBuildArtifacts (#1990) This already happens as part of the publish step in the build. Doing this over again (especially to the same container) will overwrite the previous results in some cases, and also unnecessarily duplicate files. It also will cause the wixpacks to get picked up for signing validation. * Updating some new projects to target net9.0 * Skip Pomelo tests failing due to Pomelo depending on EF 8.0 methods that were removed in 9.0 --------- Co-authored-by: Drew Noakes <[email protected]> Co-authored-by: Adam Ratzman <[email protected]> Co-authored-by: Adam Ratzman <[email protected]> Co-authored-by: James Newton-King <[email protected]> Co-authored-by: Mitch Denny <[email protected]> Co-authored-by: Bradley Grainger <[email protected]> Co-authored-by: Eric Erhardt <[email protected]> Co-authored-by: David Fowler <[email protected]> Co-authored-by: Reuben Bond <[email protected]> Co-authored-by: Tim Mulholland <[email protected]> Co-authored-by: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: Jose Perez Rodriguez <[email protected]> Co-authored-by: Vincent Hoogendoorn <[email protected]> Co-authored-by: Stephan Bauer <[email protected]> Co-authored-by: Martin Costello <[email protected]> Co-authored-by: Matt Mitchell <[email protected]>
Follow up to #825 that adds the related EF Core component for MySQL. Fixes #786.
This will need the following NuGet packages to be mirrored:
Possible areas of concern:
ServerVersion
passed in; this PR always uses astring
so it can be bound from configurationMySqlDataSource
so this configures the DbContext with a connection stringMySqlDataSource
isn't used, we have to configure logging through the "legacy" MySqlConnector APIDbDataSource
support PomeloFoundation/Pomelo.EntityFrameworkCore.MySql#1817. CC @lauxjpn