Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add deep health check for scheduler #3304

Merged
merged 39 commits into from
Feb 1, 2024
Merged

Conversation

Chethan-rao
Copy link
Contributor

@Chethan-rao Chethan-rao commented Jan 10, 2024

Type of Change

  • Bugfix
  • New feature
  • Enhancement
  • Refactoring
  • Dependency updates
  • Documentation
  • CI/CD

Description

This PR adds support for performing a deeper health check of the scheduler which includes:

Database connection check with read, write and delete queries (Added support for database transaction too)
Redis connection check with set, get and delete key (Set key with expiry of 30sec)

The deep health check API returns a 200 status code if all the components are in good health, but returns 500 if at least one component's health is down

Additional Changes

  • This PR modifies the API contract
  • This PR modifies the database schema
  • This PR modifies application configuration/environment variables

Motivation and Context

How did you test it?

  1. /health

image

  1. /health/ready

image

  1. 'health/ready with some db error

image

Cannot be tested on sandbox as the endpoint is not exposed

Checklist

  • I formatted the code cargo +nightly fmt --all
  • I addressed lints thrown by cargo clippy
  • I reviewed the submitted code
  • I added unit tests for my changes where possible
  • I added a CHANGELOG entry if applicable

@Chethan-rao Chethan-rao added A-process-tracker Area: Process tracker C-feature Category: Feature request or enhancement S-waiting-on-review Status: This PR has been implemented and needs to be reviewed labels Jan 10, 2024
@Chethan-rao Chethan-rao self-assigned this Jan 10, 2024
@Chethan-rao Chethan-rao requested a review from a team as a code owner January 10, 2024 10:08
@Chethan-rao Chethan-rao linked an issue Jan 10, 2024 that may be closed by this pull request
5 tasks
@Chethan-rao Chethan-rao force-pushed the scheduler-health-checks branch from 7d0cd6e to 14de558 Compare January 10, 2024 10:15
NishantJoshi00
NishantJoshi00 previously approved these changes Jan 12, 2024
SanchithHegde
SanchithHegde previously approved these changes Jan 12, 2024
@Chethan-rao Chethan-rao requested review from a team as code owners January 31, 2024 07:51
crates/router/src/bin/scheduler.rs Outdated Show resolved Hide resolved
crates/router/src/bin/scheduler.rs Show resolved Hide resolved
crates/router/src/bin/scheduler.rs Show resolved Hide resolved
crates/router/src/bin/scheduler.rs Outdated Show resolved Hide resolved
crates/router/src/bin/scheduler.rs Outdated Show resolved Hide resolved
dracarys18
dracarys18 previously approved these changes Jan 31, 2024
Copy link
Member

@dracarys18 dracarys18 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@SanchithHegde SanchithHegde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than that, looks good to me!

config/config.example.toml Outdated Show resolved Hide resolved
@Gnanasundari24 Gnanasundari24 added this pull request to the merge queue Feb 1, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Feb 1, 2024
@likhinbopanna likhinbopanna added this pull request to the merge queue Feb 1, 2024
Merged via the queue into main with commit 170e10c Feb 1, 2024
10 of 12 checks passed
@likhinbopanna likhinbopanna deleted the scheduler-health-checks branch February 1, 2024 11:32
@SanchithHegde SanchithHegde removed the S-waiting-on-review Status: This PR has been implemented and needs to be reviewed label Feb 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-process-tracker Area: Process tracker C-feature Category: Feature request or enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Deep health check for Hyperswitch Scheduler
5 participants