Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bootstrap: More details for context deadline exceeded error #4422

Merged
merged 1 commit into from
Dec 8, 2023
Merged

bootstrap: More details for context deadline exceeded error #4422

merged 1 commit into from
Dec 8, 2023

Conversation

somtochiama
Copy link
Member

@somtochiama somtochiama commented Nov 22, 2023 •
edited
Loading

This PR updates the bootstrap health checks to include GitRepository resource, return the Ready condition message on timeout, and log the error at the end for better visibility.
Fixes: #4411

Before (showing only relevant output:

✔ reconciled sync configuration
◎ waiting for Kustomization "flux-system/flux-system" to be reconciled
✗ client rate limiter Wait returned an error: context deadline exceeded
► confirming components are healthy
✔ helm-controller: deployment ready
✔ kustomize-controller: deployment ready
✗ notification-controller: deployment not ready
✔ source-controller: deployment ready
✗ bootstrap failed with 2 health check failure(s)

After:

✔ reconciled sync configuration
◎ waiting for GitRepository "flux-system/flux-system" to be reconciled
✗ gitrepository 'flux-system/flux-system' not ready: 'failed to checkout and determine revision: unable to list remote for 'https://github.xcom/somtochiama/fleet-infra.git': Get "https://github.xcom/somtochiama/fleet-infra.git/info/refs?service=git-upload-pack": dial tcp: lookup github.xcom on 10.96.0.10:53: no such host'
◎ waiting for Kustomization "flux-system/flux-system" to be reconciled
✗ client rate limiter Wait returned an error: context deadline exceeded
► confirming components are healthy
✔ helm-controller: deployment ready
✔ kustomize-controller: deployment ready
✔ notification-controller: deployment ready
✔ source-controller: deployment ready
✔ all components are healthy
✗ bootstrap failed with 2 health check failure(s): [error while waiting for gitrepository to be ready: 'gitrepository 'flux-system/flux-system' not ready: 'failed to checkout and determine revision: unable to list remote for 'https://github.xcom/somtochiama/fleet-infra.git': Get "https://github.xcom/somtochiama/fleet-infra.git/info/refs?service=git-upload-pack": dial tcp: lookup github.xcom on 10.96.0.10:53: no such host'', error while waiting for Kustomization to be ready: 'client rate limiter Wait returned an error: context deadline exceeded']

@somtochiama somtochiama requested review from stefanprodan, hiddeco and darkowlzz and removed request for stefanprodan November 22, 2023 14:30
@stefanprodan stefanprodan added area/bootstrap Bootstrap related issues and pull requests area/ux In pursuit of a delightful user experience labels Nov 22, 2023
stefanprodan
stefanprodan approved these changes Nov 23, 2023
View reviewed changes
Copy link
Member

@stefanprodan stefanprodan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Thanks @somtochiama 🏅

@stefanprodan stefanprodan marked this pull request as ready for review November 23, 2023 08:25
darkowlzz
darkowlzz reviewed Nov 23, 2023
View reviewed changes
pkg/bootstrap/bootstrap.go Outdated Show resolved Hide resolved
pkg/bootstrap/bootstrap.go Outdated Show resolved Hide resolved
pkg/bootstrap/bootstrap.go Outdated Show resolved Hide resolved
pkg/bootstrap/bootstrap_plain_git.go Outdated Show resolved Hide resolved
pkg/bootstrap/bootstrap.go Outdated Show resolved Hide resolved
stefanprodan
stefanprodan requested changes Nov 24, 2023
View reviewed changes
pkg/bootstrap/bootstrap_test.go Outdated Show resolved Hide resolved
darkowlzz
darkowlzz reviewed Nov 24, 2023
View reviewed changes
pkg/bootstrap/bootstrap.go Outdated Show resolved Hide resolved
pkg/bootstrap/bootstrap_plain_git.go Outdated Show resolved Hide resolved
@darkowlzz darkowlzz force-pushed the fix-bootstrap-failed branch from 44e8b37 to 3a5f784 Compare December 8, 2023 13:22
@somtochiama @darkowlzz
bootstrap: provide better error message on timeout
2289c6c 
Check GitRepository and return Ready condition message as part of error
on timeout.

Signed-off-by: Somtochi Onyekwere <[email protected]>
@darkowlzz darkowlzz force-pushed the fix-bootstrap-failed branch from 3a5f784 to 2289c6c Compare December 8, 2023 13:30
@darkowlzz darkowlzz merged commit 8e3a809 into fluxcd:main Dec 8, 2023
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@stefanprodan stefanprodan stefanprodan approved these changes

@darkowlzz darkowlzz darkowlzz approved these changes

@hiddeco hiddeco Awaiting requested review from hiddeco

Assignees
No one assigned
Labels
area/bootstrap Bootstrap related issues and pull requests area/ux In pursuit of a delightful user experience
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

bootstrap: More details for flux-system context deadline exceeded
3 participants
@somtochiama @darkowlzz @stefanprodan