From 0cf64b572a89fc1d8f78ff8a3843a458b5e2ea90 Mon Sep 17 00:00:00 2001 From: Rosa Gutierrez Date: Sat, 24 Aug 2024 20:15:30 +0200 Subject: [PATCH] Set `enqueue_after_transaction_commit` to `true` and don't allow changing it See the discussion in https://github.com/rails/rails/pull/52659. Adapters should set their own behaviour without making users choose. If users want to choose, they can just use Active Job's configuration. Besides, we're changing the value of this to `true` to make it harder for people to run into trouble in the future when switching adapters. --- README.md | 12 ++++++------ lib/active_job/queue_adapters/solid_queue_adapter.rb | 2 +- lib/solid_queue.rb | 2 -- 3 files changed, 7 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index a1714eab..81260d11 100644 --- a/README.md +++ b/README.md @@ -228,7 +228,6 @@ There are several settings that control how Solid Queue works that you can set a - `preserve_finished_jobs`: whether to keep finished jobs in the `solid_queue_jobs` table—defaults to `true`. - `clear_finished_jobs_after`: period to keep finished jobs around, in case `preserve_finished_jobs` is true—defaults to 1 day. **Note:** Right now, there's no automatic cleanup of finished jobs. You'd need to do this by periodically invoking `SolidQueue::Job.clear_finished_in_batches`, but this will happen automatically in the near future. - `default_concurrency_control_period`: the value to be used as the default for the `duration` parameter in [concurrency controls](#concurrency-controls). It defaults to 3 minutes. -- `enqueue_after_transaction_commit`: whether the job queuing is deferred to after the current Active Record transaction is committed. The default is `false`. [Read more](https://github.com/rails/rails/pull/51426). ## Errors when enqueuing Solid Queue will raise a `SolidQueue::Job::EnqueueError` for any Active Record errors that happen when enqueuing a job. The reason for not raising `ActiveJob::EnqueueError` is that this one gets handled by Active Job, causing `perform_later` to return `false` and set `job.enqueue_error`, yielding the job to a block that you need to pass to `perform_later`. This works very well for your own jobs, but makes failure very hard to handle for jobs enqueued by Rails or other gems, such as `Turbo::Streams::BroadcastJob` or `ActiveStorage::AnalyzeJob`, because you don't control the call to `perform_later` in that cases. @@ -307,13 +306,14 @@ to your `puma.rb` configuration. ## Jobs and transactional integrity -:warning: Having your jobs in the same ACID-compliant database as your application data enables a powerful yet sharp tool: taking advantage of transactional integrity to ensure some action in your app is not committed unless your job is also committed. This can be very powerful and useful, but it can also backfire if you base some of your logic on this behaviour, and in the future, you move to another active job backend, or if you simply move Solid Queue to its own database, and suddenly the behaviour changes under you. +:warning: Having your jobs in the same ACID-compliant database as your application data enables a powerful yet sharp tool: taking advantage of transactional integrity to ensure some action in your app is not committed unless your job is also committed and viceversa, and ensuring that your job won't be enqueued until the transaction within which you're enqueing it is committed. This can be very powerful and useful, but it can also backfire if you base some of your logic on this behaviour, and in the future, you move to another active job backend, or if you simply move Solid Queue to its own database, and suddenly the behaviour changes under you. -By default, Solid Queue runs in the same DB as your app, and job enqueuing is _not_ deferred until any ongoing transaction is committed, which means that by default, you'll be taking advantage of this transactional integrity. +Because this can be quite tricky and many people shouldn't need to worry about it, by default **job enqueuing is deferred until any ongoing transaction is committed** thanks to Active Job's built-in capability to do this. This means that even if you run Solid Queue in the same DB as your app, you won't be taking advantage of this transactional integrity. -If you prefer not to rely on this, or avoid relying on it unintentionally, you should make sure that: -- You set [`config.active_job.enqueue_after_transaction_commit`](https://edgeguides.rubyonrails.org/configuring.html#config-active-job-enqueue-after-transaction-commit) to `always`, if you're using Rails 7.2+. -- Or, your jobs relying on specific records are always enqueued on [`after_commit` callbacks](https://guides.rubyonrails.org/active_record_callbacks.html#after-commit-and-after-rollback) or otherwise from a place where you're certain that whatever data the job will use has been committed to the database before the job is enqueued. +If you prefer to change this, you can set [`config.active_job.enqueue_after_transaction_commit`](https://edgeguides.rubyonrails.org/configuring.html#config-active-job-enqueue-after-transaction-commit) to `never`. You can also set this on a per-job basis. + +If you set that to `never` but still want to make sure you're not inadvertently on transactional integrity, you can make sure that: +- Your jobs relying on specific data are always enqueued on [`after_commit` callbacks](https://guides.rubyonrails.org/active_record_callbacks.html#after-commit-and-after-rollback) or otherwise from a place where you're certain that whatever data the job will use has been committed to the database before the job is enqueued. - Or, you configure a different database for Solid Queue, even if it's the same as your app, ensuring that a different connection on the thread handling requests or running jobs for your app will be used to enqueue jobs. For example: ```ruby diff --git a/lib/active_job/queue_adapters/solid_queue_adapter.rb b/lib/active_job/queue_adapters/solid_queue_adapter.rb index 937cff6f..d3042194 100644 --- a/lib/active_job/queue_adapters/solid_queue_adapter.rb +++ b/lib/active_job/queue_adapters/solid_queue_adapter.rb @@ -9,7 +9,7 @@ module QueueAdapters # Rails.application.config.active_job.queue_adapter = :solid_queue class SolidQueueAdapter def enqueue_after_transaction_commit? - SolidQueue.enqueue_after_transaction_commit + true end def enqueue(active_job) # :nodoc: diff --git a/lib/solid_queue.rb b/lib/solid_queue.rb index c1b727c7..491e6570 100644 --- a/lib/solid_queue.rb +++ b/lib/solid_queue.rb @@ -32,8 +32,6 @@ module SolidQueue mattr_accessor :shutdown_timeout, default: 5.seconds - mattr_accessor :enqueue_after_transaction_commit, default: false - mattr_accessor :silence_polling, default: true mattr_accessor :supervisor_pidfile