sidekiq with redmine_custom_workflow

[ashrafalzyoud]

I want use sidekiq with radis to increase efficiency and performance
For recalculate custom field

if Rails.env.production?
  config.active_job.queue_adapter = :sidekiq

  redis_namespace = Rails.root.join("..").basename.to_s
  redis_url = "redis://#{ENV["REDIS_HOST"] || "127.0.0.1"}:#{ENV["REDIS_PORT"] || 6379}/1"

  Sidekiq.configure_server do |config|
    config.redis = { url: redis_url, namespace: redis_namespace }
  end

  Sidekiq.configure_client do |config|
    config.redis = { url: redis_url, namespace: redis_namespace }
  end
end

Sidekiq.yml

:verbose: false
:pidfile: ../sidekiq.pid
:logfile: ./log/sidekiq.log
:concurrency: <%= ENV["SIDEKIQ_WORKERS"] || 2 %>
:queues:
  - critical
  - default
  - recalculate_custom_fields
  - easy_git
  - easy_actions
  - easy_mail_campaigns
  - easy_integrations
  - mailers
  - easy_rake_tasks
  - low

What I can do to put queues for Redmine custom work flow

[ashrafalzyoud]

@picman

[ashrafalzyoud]

I'm reading more about this topic how I can increase performance because I'm using your plugins to calculate custom field and change statues and assigne
And some time to execute code tack more than 15 second because I'm using to update sub task and parents

I hope if there option like using sidekiq and cache

This what I found

Improving performance for redmine_custom_workflows involves optimizing its custom workflows implementation within Redmine. Here are specific coding strategies and examples to enhance performance:

1. Optimize Database Queries:

   • Use ActiveRecord's includes method to eager load associations and avoid N+1 query issues. For example:

     @issue = Issue.includes(:custom_field_values).find(params[:id])

   • Optimize complex queries by leveraging database indexes. Ensure that fields used in where clauses or joins are indexed appropriately in your database schema.

2. Cache Frequently Accessed Data:

   • Implement caching for custom workflows or any data that is read frequently but doesn't change often. Example using Rails caching:

     Rails.cache.fetch("custom_workflows_#{project.id}") do
       CustomWorkflow.where(project_id: project.id).to_a
     end

   • Use cache expiration strategies (expires_in, touch: true) to invalidate caches when data changes.

3. Refactor and Optimize Code:

   • Review existing code for performance bottlenecks, such as loops or conditionals that can be optimized. Example:

     def calculate_due_date(issue)
       # Inefficient version
       due_date = issue.start_date + 7.days if issue.start_date.present?
       
       # Optimized version
       due_date = issue.start_date + 7.days if issue.start_date
     end

   • Avoid unnecessary method calls or object instantiations within performance-critical sections.

4. Background Jobs for Heavy Tasks:

   • Move intensive tasks (like sending emails or complex computations) to background jobs using tools like Sidekiq or Delayed::Job to free up resources and improve response times.

5. Reduce Rendering Times:

   • Optimize views and partials by minimizing database queries and rendering complex HTML structures efficiently. Use fragment caching for views that are expensive to render.

6. Monitor and Benchmark:

   • Use performance monitoring tools (rack-mini-profiler, New Relic) to identify slow parts of the application. Benchmark critical sections of code to measure improvements.

7. Handle Large Data Sets Efficiently:

   • Paginate or batch process large data sets to avoid memory and performance issues. Use find_each for ActiveRecord queries to process records in batches.

8. Database and Server Optimization:

   • Tune database settings (like PostgreSQL or MySQL configuration) based on workload characteristics and usage patterns.
   • Optimize web server (like Puma, Unicorn) settings for concurrency and request handling.

By applying these coding strategies and examples specific to redmine_custom_workflows, you can effectively enhance the performance of custom workflows within your Redmine application. Remember to profile and benchmark changes to validate performance improvements.

[picman]

A very interesting reading but I don't know how can I help here though.

[ashrafalzyoud]

@picman
In number 2,4

Are custom workflow using cache?!
Can I using sidekiq in custom workflow?!

[picman]

No cache is used in the plugin. The plugin is very simple in principle. I just run scripts entered by users. I don't see much what I can speed up. On the other hand you can do whatever you want in your scripts. If you think that some caching would help, give it a try.

[ashrafalzyoud]

Can I use in sidekiq.yml

:verbose: false
:pidfile: ../sidekiq.pid
:logfile: ./log/sidekiq.log
:concurrency: <%= ENV["SIDEKIQ_WORKERS"] || 2 %>
:queues:
  - custom_workflow

If I want use cache
Where I can configuration it inside the plugin

Rails.cache.fetch("custom_workflows_#{project.id}") do
  CustomWorkflow.where(project_id: project.id).to_a
end
Use cache