could not connect a container to the network #6639

megashrieks · 2024-11-09T15:56:00Z

Is there an existing issue for this?

I have searched the existing issues

Describe the bug

This issue occurs when I do dotnet run in the AppHost project, It does not affect the running of the applications however this error is repeated and is printed every second for all the containers

fail: Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler[0]
      could not connect a container to the network      {"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 11, "Container": "452393e9a799e8624d921d36b88fa2e187a220d89cc3f5c528b8a59b617a7b7f", "Network": "default-aspire-network-4vd3gfu7i8", "error": "podman command 'ConnectNetwork' returned with non-zero exit code 125: command output: Stdout: '' Stderr: 'Error: container 452393e9a799e8624d921d36b88fa2e187a220d89cc3f5c528b8a59b617a7b7f is already connected to network default-aspire-network-4vd3gfu7i8: network is already connected\n'"}
fail: Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler[0]
      could not connect a container to the network      {"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 11, "Container": "0c3bd72d02478e9e67e976fca7a8a11eca7dc5bbb493319dc689dda022c2ff5a", "Network": "default-aspire-network-4vd3gfu7i8", "error": "podman command 'ConnectNetwork' returned with non-zero exit code 125: command output: Stdout: '' Stderr: 'Error: container 0c3bd72d02478e9e67e976fca7a8a11eca7dc5bbb493319dc689dda022c2ff5a is already connected to network default-aspire-network-4vd3gfu7i8: network is already connected\n'"}
fail: Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler[0]
      could not connect a container to the network      {"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 11, "Container": "40933e042ba65600e082a0a5299a02500a136fdc9be916f987871669d4ce6909", "Network": "default-aspire-network-4vd3gfu7i8", "error": "podman command 'ConnectNetwork' returned with non-zero exit code 125: command output: Stdout: '' Stderr: 'Error: container 40933e042ba65600e082a0a5299a02500a136fdc9be916f987871669d4ce6909 is already connected to network default-aspire-network-4vd3gfu7i8: network is already connected\n'"}
fail: Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler[0]
      could not connect a container to the network      {"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 12, "Container": "452393e9a799e8624d921d36b88fa2e187a220d89cc3f5c528b8a59b617a7b7f", "Network": "default-aspire-network-4vd3gfu7i8", "error": "podman command 'ConnectNetwork' returned with non-zero exit code 125: command output: Stdout: '' Stderr: 'Error: container 452393e9a799e8624d921d36b88fa2e187a220d89cc3f5c528b8a59b617a7b7f is already connected to network default-aspire-network-4vd3gfu7i8: network is already connected\n'"}
fail: Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler[0]
      could not connect a container to the network      {"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 12, "Container": "0c3bd72d02478e9e67e976fca7a8a11eca7dc5bbb493319dc689dda022c2ff5a", "Network": "default-aspire-network-4vd3gfu7i8", "error": "podman command 'ConnectNetwork' returned with non-zero exit code 125: command output: Stdout: '' Stderr: 'Error: container 0c3bd72d02478e9e67e976fca7a8a11eca7dc5bbb493319dc689dda022c2ff5a is already connected to network default-aspire-network-4vd3gfu7i8: network is already connected\n'"}
fail: Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler[0]
      could not connect a container to the network      {"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 12, "Container": "40933e042ba65600e082a0a5299a02500a136fdc9be916f987871669d4ce6909", "Network": "default-aspire-network-4vd3gfu7i8", "error": "podman command 'ConnectNetwork' returned with non-zero exit code 125: command output: Stdout: '' Stderr: 'Error: container 40933e042ba65600e082a0a5299a02500a136fdc9be916f987871669d4ce6909 is already connected to network default-aspire-network-4vd3gfu7i8: network is already connected\n'"}

Here is my csproj file

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net9.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <IsAspireHost>true</IsAspireHost>
    <UserSecretsId>d0ee5701-532e-470f-ae99-a0d293698115</UserSecretsId>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Aspire.Hosting.AppHost" Version="8.2.2" />
    <PackageReference Include="Aspire.Hosting.Keycloak" Version="9.0.0-preview.4.24511.1" />
    <PackageReference Include="Aspire.Hosting.PostgreSQL" Version="8.2.2" />
  </ItemGroup>

  <ItemGroup>
 other project references here which I've omitted...
  </ItemGroup>

</Project>

I'm on linux ubuntu, podman version

> podman -v
podman version 4.9.3

Expected Behavior

No response

Steps To Reproduce

No response

Exceptions (if any)

No response

.NET Version info

dotnet --info
.NET SDK:
 Version:           9.0.100-rc.2.24474.11
 Commit:            315e1305db
 Workload version:  9.0.100-manifests.3424025a
 MSBuild version:   17.12.0-preview-24473-03+fea15fbd1

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  24.04
 OS Platform: Linux
 RID:         linux-x64
 Base Path:   /home/shrikanth/dotnet/sdk/9.0.100-rc.2.24474.11/

.NET workloads installed:
 [aspire]
   Installation Source: SDK 9.0.100-rc.2
   Manifest Version:    8.2.2/8.0.100
   Manifest Path:       /home/shrikanth/dotnet/sdk-manifests/8.0.100/microsoft.net.sdk.aspire/8.2.2/WorkloadManifest.json
   Install Type:        FileBased

Configured to use loose manifests when installing new manifests.

Host:
  Version:      9.0.0-rc.2.24473.5
  Architecture: x64
  Commit:       990ebf52fc

.NET SDKs installed:
  9.0.100-rc.2.24474.11 [/home/shrikanth/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 9.0.0-rc.2.24474.3 [/home/shrikanth/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 9.0.0-rc.2.24473.5 [/home/shrikanth/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
  None

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/downloa

Anything else?

My Program.cs file

var builder = DistributedApplication.CreateBuilder(args);

var postgres = builder.AddPostgres("postgres")
                .WithPgWeb();
var database = postgres.AddDatabase("conductor");

var identity = builder.AddKeycloak("idp", 8443);

var automations = builder.AddProject<Projects.Conductor_Automations>("automations")
    .WithReference(database)
    .WaitFor(database);

var webapi = builder.AddProject<Projects.Conductor_WebApi>("webapi")
    .WithExternalHttpEndpoints()
    .WithReference(identity)
    .WithReference(database)
    .WaitFor(database);

builder.Build().Run();

The text was updated successfully, but these errors were encountered:

kylejuliandev · 2024-11-10T16:07:07Z

I was having this issue initially. I was able to resolve them after I upgraded my project to .NET Aspire 9.0 (RC 1) - you can see the upgrade guide here

megashrieks · 2024-11-12T14:22:49Z

@kylejuliandev this seems to fix the frequency of the errors, but the error still persists.

joperezr · 2024-11-13T00:16:42Z

We released .NET Aspire 9.0.0 earlier today, would you be able to update to that stable version and see if the issue persists?

dovijoel · 2024-11-14T07:56:30Z

I updated my project to Aspire 9.0 and started encountering this issue as well. I am on Windows 11 using Docker Desktop.

AwkEng · 2024-11-14T18:58:06Z

I was getting the same error on my desktop machine, but worked fine on my laptop. Both are running Win11 and Docker Desktop. I ended up reinstalling Docker Desktop on my desktop and that seemed to fix my issue.

ITedInnovator · 2024-11-18T15:55:48Z

I had the same problem with Redis when following the learn-dotnet-aspire tutorial. I updated Docker Desktop and now it is working.

Also, it might have been restarting WSL fixed the issue.

tjementum · 2024-12-11T17:33:38Z

I found that this problem happens if I manually kill the Aspire Developer Control Pane (DCP) while it's cleaning up.

Specifically, when it does not clean up the Docker networks default-aspire-network-*. I ran this command to delete all my orphaned aspire networks, after which the problem was fixed.

docker network ls --filter name=default-aspire-network- -q | xargs -r docker network rm

megashrieks · 2024-12-19T19:58:53Z

@joperezr I updated to Aspire 9.0.0 and that seems to fix it for me Thanks

Kralizek · 2024-12-24T01:03:42Z

I'm using Aspire 9.0 and I am still getting some leftovers

docker network ls
NETWORK ID     NAME                                DRIVER    SCOPE
9ff856f8b76d   bridge                              bridge    local
1b16a57d5aeb   default-aspire-network-3npb8kn5cg   bridge    local
0479b3a747a6   default-aspire-network-3ta4tskc4k   bridge    local
316d4b988980   default-aspire-network-5dv4n11j2k   bridge    local
585b3f3bc88e   default-aspire-network-5i68h7ehmo   bridge    local
0c0266891988   default-aspire-network-6bu94e1uf8   bridge    local
e16b078fde2c   default-aspire-network-7b9dg21s74   bridge    local
9bd3ace9d40d   default-aspire-network-7che3bdn5g   bridge    local
357eaca5f375   default-aspire-network-8u4m7ve5rk   bridge    local
4cb99aea8402   default-aspire-network-69p9tcusek   bridge    local
4b9ea7a22fd8   default-aspire-network-98nlo3n4t8   bridge    local
9e300bbe8760   default-aspire-network-146qetl9ek   bridge    local
9f6ee5f61c9b   default-aspire-network-326s0iof5k   bridge    local
4e34bfe80c51   default-aspire-network-dgpthr89dk   bridge    local
a9471a88ec1e   default-aspire-network-dp776boi4s   bridge    local
f95eb40ad411   default-aspire-network-gfcjhevgbg   bridge    local
555d8e5a1279   default-aspire-network-hkghd3vq00   bridge    local
5b5f3022705d   default-aspire-network-kr8vcvbros   bridge    local
3e037830082e   default-aspire-network-m4tbhos38k   bridge    local
fc5b3c76abc7   default-aspire-network-m9eheimtvo   bridge    local
a96b16d64c16   default-aspire-network-oujsp0ahvc   bridge    local
abfa0e98dc72   default-aspire-network-pml3r7jrf0   bridge    local
a537b362e019   default-aspire-network-pvorkorl1c   bridge    local
fb17455ac1d8   default-aspire-network-sp46nf9j9o   bridge    local
90ae877297a2   default-aspire-network-t9hvmc1j30   bridge    local
8eb9398cfd3b   default-aspire-network-tbvhfbf6qk   bridge    local
7c7b124a011a   default-aspire-network-tshdr4g3hs   bridge    local
6143a58751c0   default-aspire-network-vjvvp46oek   bridge    local
4f70ecd21895   default-aspire-network-vuamdhl0l8   bridge    local
e05c49f91607   host                                host      local
809438683e2f   kind                                bridge    local
6bbf9dd48d98   none                                null      local

If you use powershell, you can use this script to kill all the networks:

docker network ls --filter "name=aspire" --format "{{.ID}}" | ForEach-Object { docker network rm $_ }

davidfowl · 2024-12-24T01:09:10Z

You can use docker network prune

karolz-ms · 2025-01-13T22:34:31Z

Should be fixed (Aspire main branch) by #7085.

tang2087 · 2025-01-18T09:35:17Z

Should be fixed (Aspire main branch) by #7085.

It will be nice to see this fixed. I am using podman on Ubuntu with Aspire and the issue still exists.

davidfowl · 2025-01-18T16:22:28Z

You can try out daily builds https://github.com/dotnet/aspire/blob/main/docs/using-latest-daily.md and tell us if it fixes your problem

tang2087 · 2025-01-19T05:44:03Z

You can try out daily builds https://github.com/dotnet/aspire/blob/main/docs/using-latest-daily.md and tell us if it fixes your problem

I used the following versions and still get the same error:

 <ItemGroup>
    <PackageReference Include="Aspire.Hosting.AppHost" Version="9.1.0-preview.1.25068.4" />
    <PackageReference Include="Aspire.Hosting.NodeJs" Version="9.1.0-preview.1.25068.4" />
    <PackageReference Include="Aspire.Hosting.Azure.Storage" Version="9.1.0-preview.1.25068.4" />
    <PackageReference Include="Aspire.Hosting.SqlServer" Version="9.1.0-preview.1.25068.4" />
  </ItemGroup>

Service defaults:


    <PackageReference Include="Microsoft.Extensions.Http.Resilience" Version="9.2.0-preview.1.25067.2" />
    <PackageReference Include="Microsoft.Extensions.ServiceDiscovery" Version="9.1.0-preview.1.25068.4" />

davidfowl · 2025-01-19T05:57:30Z

Did you clean up the existing networks? docker prune network -f?

Though I'm still seeing this problem in some cases too on our CI.

tang2087 · 2025-01-19T06:21:50Z

Did you clean up the existing networks? docker prune network -f?

Though I'm still seeing this problem in some cases too on our CI.

nope. i forgot to do that. since i have reverted back from daily feeds to the nuget.org one, I will give that a try when i get time. Right now i need to focus on my app development first.

karolz-ms · 2025-01-21T03:20:49Z

@tang2087 please double check what is the version of the Aspire.Hosting.Orchestration package that your application/test is using (e.g. do a ps aux and note where dcp and dcpctrl executables are loaded from). Then do dcp version using that path. You want to see 0.10.1 or newer, not 0.9.something. I have heard from at least one other developer that reference the newer version of the app host package but somehow ended up using the old application orchestrator (DCP).

If you are using the version 0.10.1 or newer but still see the issue, I would really appreciate sharing a way to reproduce it.

tang2087 · 2025-01-22T10:45:05Z

@tang2087 please double check what is the version of the Aspire.Hosting.Orchestration package that your application/test is using (e.g. do a ps aux and note where dcp and dcpctrl executables are loaded from). Then do dcp version using that path. You want to see 0.10.1 or newer, not 0.9.something. I have heard from at least one other developer that reference the newer version of the app host package but somehow ended up using the old application orchestrator (DCP).

If you are using the version 0.10.1 or newer but still see the issue, I would really appreciate sharing a way to reproduce it.

~/.nuget/packages/aspire.hosting.orchestration.linux-x64/9.1.0-preview.1.25068.4/tools$ ./dcp version
{"version":"0.10.2","commitHash":"e4668ec7e6f98197599a8ff06d5eafb0a84fb427","buildTimestamp":"2025-01-16T23:45:08Z"}

I also pruned networks.

I will try to publish a project somewhere to reproduce this.

tang2087 · 2025-01-22T21:09:51Z

@karolz-ms I've created one here https://github.com/tang2087/aspire-example/tree/main
However i am hitting a new issue that the sql server container cannot start. I will fix it later (after work). Despite of that, the error message already shows as well:

      could not connect a container to the network      {"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 8, "Container": "834e3b95b68976d8fd4b684061869303f7038cf9e64ef278091406ca33959458", "Network": "default-aspire-network-k6l8o0auf8", "error": "container 834e3b95b68976d8fd4b684061869303f7038cf9e64ef278091406ca33959458 is not connected to network 7957dba6640ba7dbc309e0de81499db467264999493602e0972c20392044e744"}
fail: Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler[0]
      could not connect a container to the network      {"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 9, "Container": "834e3b95b68976d8fd4b684061869303f7038cf9e64ef278091406ca33959458", "Network": "default-aspire-network-k6l8o0auf8", "error": "container 834e3b95b68976d8fd4b684061869303f7038cf9e64ef278091406ca33959458 is not connected to network 7957dba6640ba7dbc309e0de81499db467264999493602e0972c20392044e744"}
fail: Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler[0]
      could not connect a container to the network      {"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 10, "Container": "834e3b95b68976d8fd4b684061869303f7038cf9e64ef278091406ca33959458", "Network": "default-aspire-network-k6l8o0auf8", "error": "container 834e3b95b68976d8fd4b684061869303f7038cf9e64ef278091406ca33959458 is not connected to network 7957dba6640ba7dbc309e0de81499db467264999493602e0972c20392044e744"}
fail: Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler[0]
      could not connect a container to the network      {"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 11, "Container": "834e3b95b68976d8fd4b684061869303f7038cf9e64ef278091406ca33959458", "Network": "default-aspire-network-k6l8o0auf8", "error": "container 834e3b95b68976d8fd4b684061869303f7038cf9e64ef278091406ca33959458 is not connected to network 7957dba6640ba7dbc309e0de81499db467264999493602e0972c20392044e744"}

davidfowl · 2025-01-22T21:27:38Z

The dotnet watch issue is fixed in the next version.

Kralizek · 2025-01-23T09:04:28Z

The dotnet watch issue is fixed in the next version.

do you mean aspire 9.1 or .net 10 or next version of the .net sdk?

davidfowl · 2025-01-23T09:07:08Z

The next version of the .NET SDK 9.0.200, see dotnet/sdk#36971

karolz-ms · 2025-01-23T19:38:00Z

@tang2087 thank you for sharing the repro.

Like @davidfowl said, dotnet watch is effectively not supported with Aspire at the moment. I have run your project using dotnet run on my Ubuntu WSL VM 6 times in quick succession, have not seen any issues or leaked containers/networks (the SQL Server container persists because it is configured as such in the app host code).

tang2087 · 2025-01-25T11:02:15Z

@tang2087 thank you for sharing the repro.

Like @davidfowl said, dotnet watch is effectively not supported with Aspire at the moment. I have run your project using dotnet run on my Ubuntu WSL VM 6 times in quick succession, have not seen any issues or leaked containers/networks (the SQL Server container persists because it is configured as such in the app host code).

ok, look forward to the release in Feb (hope the milestone target date is still up to date)

joperezr added untriaged New issue has not been triaged area-orchestrator and removed untriaged New issue has not been triaged labels Nov 12, 2024

joperezr added the needs-author-action An issue or pull request that requires more info or actions from the author. label Nov 13, 2024

joperezr added this to the Backlog milestone Nov 13, 2024

karolz-ms mentioned this issue Dec 4, 2024

Aspire 9 fails to stop projects #6704

Closed

1 task

dotnet-policy-service bot added the no-recent-activity label Dec 6, 2024

dotnet-policy-service bot removed the no-recent-activity label Dec 11, 2024

dotnet-policy-service bot removed the needs-author-action An issue or pull request that requires more info or actions from the author. label Dec 19, 2024

dbreshears modified the milestones: Backlog, 9.1 Jan 8, 2025

dbreshears assigned karolz-ms Jan 8, 2025

karolz-ms closed this as completed Jan 13, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

could not connect a container to the network #6639

could not connect a container to the network #6639

megashrieks commented Nov 9, 2024 •

edited

Loading

kylejuliandev commented Nov 10, 2024

megashrieks commented Nov 12, 2024

joperezr commented Nov 13, 2024

dovijoel commented Nov 14, 2024

AwkEng commented Nov 14, 2024

ITedInnovator commented Nov 18, 2024

tjementum commented Dec 11, 2024

megashrieks commented Dec 19, 2024

Kralizek commented Dec 24, 2024

davidfowl commented Dec 24, 2024

karolz-ms commented Jan 13, 2025

tang2087 commented Jan 18, 2025

davidfowl commented Jan 18, 2025

tang2087 commented Jan 19, 2025 •

edited

Loading

davidfowl commented Jan 19, 2025

tang2087 commented Jan 19, 2025

karolz-ms commented Jan 21, 2025

tang2087 commented Jan 22, 2025

tang2087 commented Jan 22, 2025

davidfowl commented Jan 22, 2025

Kralizek commented Jan 23, 2025

davidfowl commented Jan 23, 2025

karolz-ms commented Jan 23, 2025

tang2087 commented Jan 25, 2025

could not connect a container to the network #6639

could not connect a container to the network #6639

Comments

megashrieks commented Nov 9, 2024 • edited Loading

Is there an existing issue for this?

Describe the bug

Expected Behavior

Steps To Reproduce

Exceptions (if any)

.NET Version info

Anything else?

kylejuliandev commented Nov 10, 2024

megashrieks commented Nov 12, 2024

joperezr commented Nov 13, 2024

dovijoel commented Nov 14, 2024

AwkEng commented Nov 14, 2024

ITedInnovator commented Nov 18, 2024

tjementum commented Dec 11, 2024

megashrieks commented Dec 19, 2024

Kralizek commented Dec 24, 2024

davidfowl commented Dec 24, 2024

karolz-ms commented Jan 13, 2025

tang2087 commented Jan 18, 2025

davidfowl commented Jan 18, 2025

tang2087 commented Jan 19, 2025 • edited Loading

davidfowl commented Jan 19, 2025

tang2087 commented Jan 19, 2025

karolz-ms commented Jan 21, 2025

tang2087 commented Jan 22, 2025

tang2087 commented Jan 22, 2025

davidfowl commented Jan 22, 2025

Kralizek commented Jan 23, 2025

davidfowl commented Jan 23, 2025

karolz-ms commented Jan 23, 2025

tang2087 commented Jan 25, 2025

megashrieks commented Nov 9, 2024 •

edited

Loading

tang2087 commented Jan 19, 2025 •

edited

Loading