Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bellon/getjerry customize branch test #2

Open
wants to merge 104 commits into
base: master
Choose a base branch
from

Conversation

bellondr
Copy link
Collaborator

@bellondr bellondr commented Dec 28, 2024

Summary by CodeRabbit

  • New Features

    • Introduced high-availability support and job scheduling improvements to enhance overall service reliability.
    • Added a new monitoring endpoint for real-time status and performance insights.
  • Chores

    • Refined security and request routing for more effective proxy handling.
    • Updated secure connection defaults by shifting to a new HTTPS port (changed from 8443 to 8843).
    • Enhanced dependency management by incorporating additional database-related requirements.
    • Simplified the input acceptance criteria for Redis host configuration.
    • Added a readiness check script to monitor service availability.
    • Implemented leader election for Kubernetes high availability.
    • Enhanced Nginx configuration for improved request handling and performance metrics.
    • Modified ModSecurity configurations for improved security and logging capabilities.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
src/scheduler/main.py (1)

999-1023: Consider making leader election timeouts configurable.

The leader election configuration uses hardcoded timeouts. Consider making these configurable through environment variables to allow fine-tuning in different environments.

Apply this diff to make the timeouts configurable:

-    election_config = electionconfig.Config(
-        ConfigMapLock(lock_name, namespace, pod_name),
-        lease_duration=60,
-        renew_deadline=50,
-        retry_period=20,
-        onstarted_leading=kubernetes_start,
-        onstopped_leading=onstopped_leading
-    )
+    election_config = electionconfig.Config(
+        ConfigMapLock(lock_name, namespace, pod_name),
+        lease_duration=int(getenv("LEASE_DURATION", "60")),
+        renew_deadline=int(getenv("RENEW_DEADLINE", "50")),
+        retry_period=int(getenv("RETRY_PERIOD", "20")),
+        onstarted_leading=kubernetes_start,
+        onstopped_leading=onstopped_leading
+    )
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5576ca5 and 3e6630d.

📒 Files selected for processing (1)
  • src/scheduler/main.py (4 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
src/scheduler/main.py (1)

Pattern **/*: Be concise and precise. Skip minor issues and focus on critical problems and potentially wrong code. Avoid raising multiple comments with similar or repetitive content and instead use one comment in the selected key place to clearly state common concerns.

🔇 Additional comments (2)
src/scheduler/main.py (2)

24-31: LGTM! Kubernetes imports and configuration variables are well-structured.

The imports and environment variables are correctly set up for Kubernetes leader election functionality.

Also applies to: 93-94


1025-1030: LGTM! Clean integration of Kubernetes HA mode.

The main execution block elegantly handles the routing between Kubernetes HA mode and normal start based on the integration type.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
src/scheduler/main.py (1)

997-1020: Consider adjusting leader election timeouts for better resilience.

The leader election configuration looks good, but consider these improvements:

  • The lease duration (60s) and renew deadline (50s) are quite close, leaving only a 10s buffer.
  • The retry period (20s) is relatively high, which could delay leadership transitions.

Consider adjusting the timeouts for better resilience:

-        lease_duration=60,
-        renew_deadline=50,
-        retry_period=20,
+        lease_duration=120,
+        renew_deadline=90,
+        retry_period=10,

This provides:

  • A larger buffer between lease duration and renew deadline
  • Faster leadership transitions with a shorter retry period
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 3e6630d and 36fa52f.

📒 Files selected for processing (4)
  • src/autoconf/IngressController.py (5 hunks)
  • src/bw/entrypoint.sh (1 hunks)
  • src/common/settings.json (1 hunks)
  • src/scheduler/main.py (4 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/autoconf/IngressController.py
🧰 Additional context used
📓 Path-based instructions (3)
src/common/settings.json (1)

Pattern **/*: Be concise and precise. Skip minor issues and focus on critical problems and potentially wrong code. Avoid raising multiple comments with similar or repetitive content and instead use one comment in the selected key place to clearly state common concerns.

src/bw/entrypoint.sh (1)

Pattern **/*: Be concise and precise. Skip minor issues and focus on critical problems and potentially wrong code. Avoid raising multiple comments with similar or repetitive content and instead use one comment in the selected key place to clearly state common concerns.

src/scheduler/main.py (1)

Pattern **/*: Be concise and precise. Skip minor issues and focus on critical problems and potentially wrong code. Avoid raising multiple comments with similar or repetitive content and instead use one comment in the selected key place to clearly state common concerns.

🪛 Ruff (0.8.2)
src/scheduler/main.py

1024-1024: Undefined name INTEGRATION

(F821)


1024-1024: Undefined name dotenv_env

(F821)

🔇 Additional comments (4)
src/bw/entrypoint.sh (1)

47-47: HTTPS Port Default Value Update
The configuration has been updated to set HTTPS_PORT to 8843—this change now matches the updated default value in the settings JSON file. Ensure that this new port is properly supported in your deployment pipelines and documentation.

src/common/settings.json (1)

29-37: HTTPS_PORT Default Value Modification
The default value for HTTPS_PORT has been updated from "8443" to "8843". This modification is consistent with the corresponding update in the entrypoint script, ensuring uniform configuration across components. Please verify that any associated documentation or external configurations referencing this setting are updated accordingly.

src/scheduler/main.py (2)

28-33: LGTM! Kubernetes HA mode dependencies and configuration.

The new imports and global variables are correctly set up to support Kubernetes leader election functionality.

Also applies to: 91-92


1023-1029: LGTM! Clean integration with Kubernetes HA mode.

The main execution block correctly handles the Kubernetes integration path.

🧰 Tools
🪛 Ruff (0.8.2)

1024-1024: Undefined name INTEGRATION

(F821)


1024-1024: Undefined name dotenv_env

(F821)

Comment on lines 1024 to 1026
SCHEDULER = JobScheduler(environ, logger, INTEGRATION, db=Database(logger, sqlalchemy_string=dotenv_env.get("DATABASE_URI", getenv("DATABASE_URI", None)))) # type: ignore
INTEGRATION = get_integration()
if INTEGRATION == "Kubernetes":
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix undefined variables in the main block.

The variables INTEGRATION and dotenv_env are used before they are defined, which could lead to runtime errors.

Apply this diff to fix the issue:

 if __name__ == "__main__":
+    # Parse arguments
+    parser = ArgumentParser(description="Job scheduler for BunkerWeb")
+    parser.add_argument("--variables", type=str, help="path to the file containing environment variables")
+    args = parser.parse_args()
+
+    INTEGRATION = get_integration()
+    tmp_variables_path = Path(args.variables or join(sep, "var", "tmp", "bunkerweb", "variables.env"))
+    dotenv_env = dotenv_values(str(tmp_variables_path))
+
     SCHEDULER = JobScheduler(environ, logger, INTEGRATION, db=Database(logger, sqlalchemy_string=dotenv_env.get("DATABASE_URI", getenv("DATABASE_URI", None))))  # type: ignore
-    INTEGRATION = get_integration()
     if INTEGRATION == "Kubernetes":
         run_on_kubernetes_ha_mode()
     else:
         start()
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
SCHEDULER = JobScheduler(environ, logger, INTEGRATION, db=Database(logger, sqlalchemy_string=dotenv_env.get("DATABASE_URI", getenv("DATABASE_URI", None)))) # type: ignore
INTEGRATION = get_integration()
if INTEGRATION == "Kubernetes":
if __name__ == "__main__":
# Parse arguments
parser = ArgumentParser(description="Job scheduler for BunkerWeb")
parser.add_argument("--variables", type=str, help="path to the file containing environment variables")
args = parser.parse_args()
INTEGRATION = get_integration()
tmp_variables_path = Path(args.variables or join(sep, "var", "tmp", "bunkerweb", "variables.env"))
dotenv_env = dotenv_values(str(tmp_variables_path))
SCHEDULER = JobScheduler(
environ,
logger,
INTEGRATION,
db=Database(logger, sqlalchemy_string=dotenv_env.get("DATABASE_URI", getenv("DATABASE_URI", None)))
) # type: ignore
if INTEGRATION == "Kubernetes":
run_on_kubernetes_ha_mode()
else:
start()
🧰 Tools
🪛 Ruff (0.8.2)

1024-1024: Undefined name INTEGRATION

(F821)


1024-1024: Undefined name dotenv_env

(F821)

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
src/scheduler/main.py (2)

989-1013: Add error handling for Kubernetes leader election.

While the leader election configuration is well-structured, consider adding error handling for potential Kubernetes API failures.

Apply this diff to improve error handling:

 def run_on_kubernetes_ha_mode():
-    config.load_incluster_config()
+    try:
+        config.load_incluster_config()
+    except Exception as e:
+        logger.error(f"Failed to load in-cluster configuration: {e}")
+        _exit(1)
+
     lock_name = "scheduler-election"
     election_config = electionconfig.Config(
         ConfigMapLock(lock_name, namespace, pod_name),
         lease_duration=60,
         renew_deadline=50,
         retry_period=20,
         onstarted_leading=kubernetes_start,
         onstopped_leading=onstopped_leading
     )
     logger.info(f'I am {pod_name} with {lock_name} in namespace {namespace}')
-    leaderelection.LeaderElection(election_config).run()
+    try:
+        leaderelection.LeaderElection(election_config).run()
+    except Exception as e:
+        logger.error(f"Leader election failed: {e}")
+        _exit(1)

989-1013: Add type hints for better code maintainability.

Consider adding type hints to the new functions for better code maintainability and IDE support.

Apply this diff to add type hints:

-def run_on_kubernetes_ha_mode():
+def run_on_kubernetes_ha_mode() -> None:
     config.load_incluster_config()
     # ...

-def kubernetes_start():
+def kubernetes_start() -> None:
     logger.info(f'{pod_name} is leader')
     start()

-def onstopped_leading():
+def onstopped_leading() -> None:
     logger.info(f'{pod_name} on stop leading, will exit 0')
     _exit(0)
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 36fa52f and 13db453.

📒 Files selected for processing (1)
  • src/scheduler/main.py (5 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
src/scheduler/main.py (1)

Pattern **/*: Be concise and precise. Skip minor issues and focus on critical problems and potentially wrong code. Avoid raising multiple comments with similar or repetitive content and instead use one comment in the selected key place to clearly state common concerns.

🔇 Additional comments (2)
src/scheduler/main.py (2)

28-33: LGTM! Well-structured Kubernetes imports and configuration.

The imports and global variables are correctly set up for Kubernetes leader election support. The pod name generation with UUID ensures uniqueness, and the namespace default aligns with Kubernetes conventions.

Also applies to: 91-95


1015-1031: LGTM! Fixed variable initialization order.

The changes correctly address the undefined variables issue from the past review by moving argument parsing and variable initialization before their usage. The integration check for Kubernetes mode is properly implemented.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
src/scheduler/main.py (1)

986-1010: Consider making leader election timeouts configurable.

The implementation looks good, but the lease duration, renew deadline, and retry period are hardcoded. Consider making these values configurable through environment variables to allow tuning based on specific deployment needs.

Apply this diff to make the timeouts configurable:

-    election_config = electionconfig.Config(
-        ConfigMapLock(lock_name, namespace, pod_name),
-        lease_duration=60,
-        renew_deadline=50,
-        retry_period=20,
-        onstarted_leading=kubernetes_start,
-        onstopped_leading=onstopped_leading
-    )
+    election_config = electionconfig.Config(
+        ConfigMapLock(lock_name, namespace, pod_name),
+        lease_duration=int(getenv('LEADER_LEASE_DURATION', '60')),
+        renew_deadline=int(getenv('LEADER_RENEW_DEADLINE', '50')),
+        retry_period=int(getenv('LEADER_RETRY_PERIOD', '20')),
+        onstarted_leading=kubernetes_start,
+        onstopped_leading=onstopped_leading
+    )
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 13db453 and 255ea1e.

📒 Files selected for processing (1)
  • src/scheduler/main.py (4 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
src/scheduler/main.py (1)

Pattern **/*: Be concise and precise. Skip minor issues and focus on critical problems and potentially wrong code. Avoid raising multiple comments with similar or repetitive content and instead use one comment in the selected key place to clearly state common concerns.

🔇 Additional comments (2)
src/scheduler/main.py (2)

28-33: LGTM! Kubernetes imports and configuration variables are well-structured.

The imports and global variables are correctly set up for Kubernetes leader election functionality.

Also applies to: 91-95


1012-1028: LGTM! Main block initialization is well-structured.

The changes properly address the previous review comment about undefined variables and correctly handle the Kubernetes integration mode.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 255ea1e and 76b61f4.

📒 Files selected for processing (2)
  • src/scheduler/entrypoint.sh (1 hunks)
  • src/scheduler/run.py (1 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
src/scheduler/entrypoint.sh (1)

Pattern **/*: Be concise and precise. Skip minor issues and focus on critical problems and potentially wrong code. Avoid raising multiple comments with similar or repetitive content and instead use one comment in the selected key place to clearly state common concerns.

src/scheduler/run.py (1)

Pattern **/*: Be concise and precise. Skip minor issues and focus on critical problems and potentially wrong code. Avoid raising multiple comments with similar or repetitive content and instead use one comment in the selected key place to clearly state common concerns.

🪛 Ruff (0.8.2)
src/scheduler/run.py

4-4: Undefined name join

(F821)


4-4: Undefined name sep

(F821)


5-5: Undefined name sys_path

(F821)


6-6: Undefined name sys_path

(F821)

🔇 Additional comments (1)
src/scheduler/entrypoint.sh (1)

50-50: Change to run.py looks good.

Switching from main.py to run.py reflects the updated Kubernetes HA-mode logic. This is consistent with the new scheduler script in src/scheduler/run.py.

parser = ArgumentParser(description="Job scheduler for BunkerWeb")
parser.add_argument("--variables", type=str, help="path to the file containing environment variables")
args = parser.parse_args()
variables = args.args.variables
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Use args.variables instead of args.args.variables.

Referencing args.args.variables will lead to an AttributeError. The correct usage is likely:

-variables = args.args.variables
+variables = args.variables
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
variables = args.args.variables
variables = args.variables

Comment on lines 36 to 41
def kubernetes_start():
print(f'{pod_name} is leader')
if variables:
subprocess.run(["/usr/share/bunkerweb/scheduler/main.py", "--variables", variables, "&"], capture_output=True, text=True)
else:
subprocess.run(["/usr/share/bunkerweb/scheduler/main.py", "&"], capture_output=True, text=True)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Remove the & argument or switch to a non-blocking call.

Passing & to subprocess.run has no effect unless the target script specifically handles it as a command-line argument. If you intend to run the process in the background, use subprocess.Popen (and omit the & argument), or if you want to wait for completion, remove &:

-if variables:
-    subprocess.run(["/usr/share/bunkerweb/scheduler/main.py", "--variables", variables, "&"], capture_output=True, text=True)
-else:
-    subprocess.run(["/usr/share/bunkerweb/scheduler/main.py", "&"], capture_output=True, text=True)
+if variables:
+    subprocess.Popen(["/usr/share/bunkerweb/scheduler/main.py", "--variables", variables], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
+else:
+    subprocess.Popen(["/usr/share/bunkerweb/scheduler/main.py"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
def kubernetes_start():
print(f'{pod_name} is leader')
if variables:
subprocess.run(["/usr/share/bunkerweb/scheduler/main.py", "--variables", variables, "&"], capture_output=True, text=True)
else:
subprocess.run(["/usr/share/bunkerweb/scheduler/main.py", "&"], capture_output=True, text=True)
def kubernetes_start():
print(f'{pod_name} is leader')
if variables:
subprocess.Popen(["/usr/share/bunkerweb/scheduler/main.py", "--variables", variables], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
else:
subprocess.Popen(["/usr/share/bunkerweb/scheduler/main.py"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)

src/scheduler/run.py Show resolved Hide resolved
if SCHEDULER.db.readonly and changes == old_changes:
continue
if isinstance(changes, str):
raise Exception(f"An error occurred when checking for changes in the database : {changes}")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Quality Violation

Exception is too generic (...read more)

Do not raise Exception and BaseException. These are too generic. Having generic exceptions makes it difficult to differentiate errors in a program. Use a specific exception, for example, ValueError, or create your own instead of using generic ones.

Learn More

View in Datadog  Leave us feedback  Documentation

Comment on lines 995 to 997
except:
logger.error(f"Exception while executing scheduler : {format_exc()}")
stop(1)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Code Quality Violation

no bare except (...read more)

Avoid bare except. Try to always use specialized exception names in except blocks.

View in Datadog  Leave us feedback  Documentation

Comment on lines 481 to 488
if (
INTEGRATION in ("Swarm", "Kubernetes", "Autoconf")
or not tmp_variables_path.exists()
or not nginx_variables_path.exists()
or (tmp_variables_path.read_text(encoding="utf-8") != nginx_variables_path.read_text(encoding="utf-8"))
or SCHEDULER.db.is_initialized()
and SCHEDULER.db.get_config() != dotenv_env
):

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Code Quality Violation

Found too many nested ifs within this condition (...read more)

Too many nested loops make the code hard to read and understand. Simplify your code by removing nesting levels and separate code in small units.

View in Datadog  Leave us feedback  Documentation

)
if proc.returncode != 0:
logger.error("Config saver failed, configuration will not work as expected...")
def main():

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Code Quality Violation

function exceeds 200 lines (...read more)

This rule stipulates that functions in Python should not exceed 200 lines of code. The primary reason for this rule is to promote readability and maintainability of the code. When functions are concise and focused, they are easier to understand, test, and debug.

Long functions often indicate that a single function is doing too much. Adhering to the Single Responsibility Principle (SRP) can help avoid this. SRP states that a function should have only one reason to change. If a function is doing more than one thing, it can usually be split into several smaller, more specific functions.

In practice, to adhere to this rule, you can often break up long functions into smaller helper functions. If a piece of code within a function is independent and can be isolated, it is a good candidate to be moved into a separate function. This also increases code reusability. For instance, if a function process_data() is too long, you can identify independent tasks within it - such as clean_data(), transform_data(), and save_data() - and create separate functions for them. This makes the code easier to reason about and test, and promotes good coding practices.

View in Datadog  Leave us feedback  Documentation

Comment on lines 574 to 575
if saving:
custom_configs.append({"value": content, "exploded": (service_id, config_type, file.stem)})

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Code Quality Violation

too many nesting levels (...read more)

Avoid to nest too many loops together. Having too many loops make your code harder to understand.
Prefer to organize your code in functions and unit of code you can clearly understand.

Learn More

View in Datadog  Leave us feedback  Documentation

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
src/scheduler/main.py (2)

37-42: Remove duplicate import for _exit and getenv.

Lines 42 and 10 both import _exit and getenv. This redefinition triggers a static analysis warning and can introduce confusion. Please remove or consolidate these imports to avoid overwriting or duplicating them.

Proposed fix:

- from os import _exit, getenv
+ # Duplicate import already defined on line 10; remove or consolidate
🧰 Tools
🪛 Ruff (0.8.2)

42-42: Redefinition of unused _exit from line 10

Remove definition: _exit

(F811)


42-42: Redefinition of unused getenv from line 10

Remove definition: getenv

(F811)


448-1023: Consider splitting the main() function for improved maintainability.

This large function includes extensive logic for scheduling, plugin updates, configuration generation, failover handling, and more. Breaking these responsibilities into smaller, dedicated functions (e.g., network config generation, plugin initialization, concurrency management) can improve readability, testability, and maintainability.

🧰 Tools
🪛 Ruff (0.8.2)

834-834: Do not use bare except

(E722)


995-995: Do not use bare except

(E722)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between fa771f0 and 43ca13d.

📒 Files selected for processing (1)
  • src/scheduler/main.py (2 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
src/scheduler/main.py (1)

Pattern **/*: Be concise and precise. Skip minor issues and focus on critical problems and potentially wrong code. Avoid raising multiple comments with similar or repetitive content and instead use one comment in the selected key place to clearly state common concerns.

🪛 Ruff (0.8.2)
src/scheduler/main.py

42-42: Redefinition of unused _exit from line 10

Remove definition: _exit

(F811)


42-42: Redefinition of unused getenv from line 10

Remove definition: getenv

(F811)


834-834: Do not use bare except

(E722)


995-995: Do not use bare except

(E722)

Comment on lines 834 to 835
except:
logger.error(f"Exception while reloading after running jobs once scheduling : {format_exc()}")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Avoid using a bare except:.

Catching all exceptions without specifying an exception type can mask serious issues and make debugging more difficult. Use a more specific exception type (e.g. Exception) or appropriate subclass.

Proposed fix:

--- a/src/scheduler/main.py
+++ b/src/scheduler/main.py
@@ -834,2 +834,2 @@
-    except:
+    except Exception as e:
     logger.error(f"Exception while reloading after running jobs once scheduling : {format_exc()}")

@@ -995,3 +995,3 @@
-    except:
+    except Exception as e:
     logger.error(f"Exception while executing scheduler : {format_exc()}")
     stop(1)

Also applies to: 995-997

🧰 Tools
🪛 Ruff (0.8.2)

834-834: Do not use bare except

(E722)

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 13ba5f4 and bed8b63.

📒 Files selected for processing (2)
  • src/autoconf/main.py (3 hunks)
  • src/scheduler/run.py (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/scheduler/run.py
🧰 Additional context used
📓 Path-based instructions (1)
`**/*`: Be concise and precise. Skip minor issues and focus ...

**/*: Be concise and precise. Skip minor issues and focus on critical problems and potentially wrong code. Avoid raising multiple comments with similar or repetitive content and instead use one comment in the selected key place to clearly state common concerns.

  • src/autoconf/main.py
🪛 Ruff (0.8.2)
src/autoconf/main.py

111-111: Do not use bare except

(E722)

🔇 Additional comments (4)
src/autoconf/main.py (4)

30-31: Add validation for the namespace environment variable.

The namespace value is used in leader election configuration but lacks validation. Empty or invalid namespace values could cause runtime errors.

-namespace = getenv("NAMESPACE", "default")
+namespace = getenv("NAMESPACE", "default")
+if not namespace:
+    logger.error("Invalid NAMESPACE value, must not be empty")
+    _exit(1)

55-62: Make leader election configuration values configurable.

The timing values are hardcoded which reduces flexibility. Consider making them configurable via environment variables.

+lease_duration = int(getenv('LEASE_DURATION', '17'))
+renew_deadline = int(getenv('RENEW_DEADLINE', '15'))
+retry_period = int(getenv('RETRY_PERIOD', '5'))
 election_config = electionconfig.Config(
     ConfigMapLock(lock_name, namespace, pod_name),
-    lease_duration=17,
-    renew_deadline=15,
-    retry_period=5,
+    lease_duration=lease_duration,
+    renew_deadline=renew_deadline,
+    retry_period=retry_period,
     onstarted_leading=kubernetes_start,
     onstopped_leading=onstopped_leading
 )

79-94: LGTM!

The start function correctly handles instance waiting, health checks, and event processing.


111-113: Improve exception handling.

The bare except clause makes it difficult to debug specific issues and could mask important errors.

-    except:
+    except Exception as e:
         logger.error(f"Exception while running autoconf :\n{format_exc()}")
+        logger.error(f"Error type: {type(e).__name__}")
         sys_exit(1)
🧰 Tools
🪛 Ruff (0.8.2)

111-111: Do not use bare except

(E722)

Comment on lines +73 to +76
def kubernetes_start():
logger.info(f'{pod_name} is leader')
controller = IngressController()
start(controller=controller)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Add error handling to the kubernetes_start callback.

The callback should handle exceptions to prevent leader election interruption.

 def kubernetes_start():
-    logger.info(f'{pod_name} is leader')
-    controller = IngressController()
-    start(controller=controller)
+    try:
+        logger.info(f'{pod_name} is leader')
+        controller = IngressController()
+        start(controller=controller)
+    except Exception as e:
+        logger.error(f"Error in leader callback: {e}")
+        logger.debug(format_exc())
+        raise  # Re-raise to trigger leader election failover
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
def kubernetes_start():
logger.info(f'{pod_name} is leader')
controller = IngressController()
start(controller=controller)
def kubernetes_start():
try:
logger.info(f'{pod_name} is leader')
controller = IngressController()
start(controller=controller)
except Exception as e:
logger.error(f"Error in leader callback: {e}")
logger.debug(format_exc())
raise # Re-raise to trigger leader election failover

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 99c0377 and d09fe19.

📒 Files selected for processing (1)
  • src/ui/main.py (1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
`**/*`: Be concise and precise. Skip minor issues and focus ...

**/*: Be concise and precise. Skip minor issues and focus on critical problems and potentially wrong code. Avoid raising multiple comments with similar or repetitive content and instead use one comment in the selected key place to clearly state common concerns.

  • src/ui/main.py

Comment on lines +168 to +169
#csrf = CSRFProtect()
#csrf.init_app(app)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Serious Security Concern: Reintroduce CSRF Protection.
Commenting out CSRF enforcement opens the application to cross-site request forgery attacks, allowing malicious websites to send requests on behalf of authenticated users.

Below is a suggested fix to restore CSRF protection:

-    #csrf = CSRFProtect()
-    #csrf.init_app(app)
+    csrf = CSRFProtect()
+    csrf.init_app(app)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
#csrf = CSRFProtect()
#csrf.init_app(app)
csrf = CSRFProtect()
csrf.init_app(app)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant