Fix compose.mssql.yml performance problem #67

mngoe · 2025-01-14T08:06:40Z

Performance problem in Cameroon.

Context: Error Encountered

When attempting to allocate a lot of (more than 30) cores to the database server, the following error occurred:

fork: retry: Resource temporarily unavailable

This error is typically caused by the system reaching its process or file descriptor limits, which can occur when the database server operates with a high number of cores or handles many concurrent connections. These limits are set by default in Docker containers and may be insufficient for resource-intensive applications like databases.

It's also important to check host configuration to be sure that those container value will not be blocked by host limits.

To resolve this issue and prevent similar errors in the future, the following adjustments were made:

1. init: true

Purpose: Ensures that an init system is used inside the container to handle process reaping and signal forwarding properly.
Why It's Important: In a typical containerized environment, zombie processes (defunct processes that are not properly cleaned up) can accumulate over time. Using init: true adds a lightweight init system to the container, which ensures: Proper termination of processes when the container stops. Management of orphaned or zombie processes.
Use Case: This is particularly useful in database containers where multiple processes may run in the background, and clean management of these processes is critical for stability and performance.

2. ulimits

Purpose: Sets user limits for processes and file descriptors within the container.
Details:
-- nproc: 65535: Increases the maximum number of processes a user can create within the container.
-- nofile:
--- Soft limit: 65535 (maximum number of open file descriptors a user process can have). -- Hard limit: 65535 (absolute maximum number of open file descriptors a user process can have).

- Why It's Important: Databases like SQL Server often require higher process and file descriptor limits, especially when handling a high number of concurrent connections or performing resource-intensive operations. Without these adjustments, the database container might encounter errors or performance degradation under heavy workloads.

We may have this kind of problems on postgresql servers as well

Performance problem in Cameroon. Context: Error Encountered When attempting to allocate a lot of (more than 30) cores to the database server, the following error occurred: fork: retry: Resource temporarily unavailable This error is typically caused by the system reaching its process or file descriptor limits, which can occur when the database server operates with a high number of cores or handles many concurrent connections. These limits are set by default in Docker containers and may be insufficient for resource-intensive applications like databases. It's also important to check host configuration to be sure that those container value will not be blocked by host limits. To resolve this issue and prevent similar errors in the future, the following adjustments were made: 1. init: true - Purpose: Ensures that an init system is used inside the container to handle process reaping and signal forwarding properly. - Why It's Important: In a typical containerized environment, zombie processes (defunct processes that are not properly cleaned up) can accumulate over time. Using init: true adds a lightweight init system to the container, which ensures: Proper termination of processes when the container stops. Management of orphaned or zombie processes. - Use Case: This is particularly useful in database containers where multiple processes may run in the background, and clean management of these processes is critical for stability and performance. 2. ulimits - Purpose: Sets user limits for processes and file descriptors within the container. - Details: -- nproc: 65535: Increases the maximum number of processes a user can create within the container. nofile: -- Soft limit: 65535 (maximum number of open file descriptors a user process can have). -- Hard limit: 65535 (absolute maximum number of open file descriptors a user process can have). - Why It's Important: Databases like SQL Server often require higher process and file descriptor limits, especially when handling a high number of concurrent connections or performing resource-intensive operations. Without these adjustments, the database container might encounter errors or performance degradation under heavy workloads. We may probably have this kind of problems on postgresql servers as well