Rate limiting/job enabling for ActiveJob using distributed locks in Redis or Memcached.
Rate controls for your
ActiveJobs, powered by Suo, a distributed semaphore library backed by Redis or Memcached.
Add this line to your application's Gemfile:
gem 'activejob-traffic_control'
And then execute:
$ bundle
Or install it yourself as:
$ gem install activejob-traffic_control
ActiveJob::TrafficControladds three modules you can mixin to your job classes as needed, or to
ApplicationJobif you are using ActiveJob 5+ (or you have created a base job class yourself).
# to initialize the type of locking client (memcached vs. redis): ActiveJob::TrafficControl.client = ConnectionPool.new(size: 5, timeout: 5) { Redis.new } # set poolthresholds as needed # or, ActiveJob::TrafficControl.client = ConnectionPool.new(size: 5, timeout: 5) { Dalli::Client.new } # or if not multithreaded, ActiveJob::TrafficControl.client = Redis.new
Throttle
class CanThrottleJob < ActiveJob::Base throttle threshold: 2, period: 1.seconddef perform # no more than two of
CanThrottleJob
will run every second # if more than that attempt to run, they will be re-enqueued to run in a random time # ranging from 1 - 5x the period (so, 1-5 seconds in this case) end end
If you do not care about the job being re-enqueued (if it's scheduled to run otherwise, or dropping will have no ill effect), you can specify
drop: trueinstead. The
drop: trueflag also applies to
Concurrency, below.
class CanThrottleAndDropJob < ActiveJob::Base throttle threshold: 2, period: 1.second, drop: truedef perform # no more than two of
CanThrottleJob
will run every second # if more than that attempt to run, they will be dropped end end
Concurrency
class ConcurrencyTestJob < ActiveJob::Base concurrency 5, drop: falsedef perform # only five
ConcurrencyTestJob
will ever run simultaneously end end
Disable
For
Disable, you also need to configure the cache client:
ActiveJob::TrafficControl.cache_client = Rails.cache.dalli # if using :dalli_store # or ActiveJob::TrafficControl.cache_client = ActiveSupport::Cache.lookup_store(:dalli_store, "localhost:11211")
class CanDisableJob < ActiveJob::Base def perform # you can pause this job from running by executing `CanDisableJob.disable!` (which will cause the job to be re-enqueued), # or have it be dropped entirely via `CanDisableJob.disable!(drop: true)` # enable it again via `CanDisableJob.enable!` end end
After checking out the repo, run
bin/setupto install dependencies. Then, run
rake testto run the tests. You can also run
bin/consolefor an interactive prompt that will allow you to experiment.
To install this gem onto your local machine, run
bundle exec rake install. To release a new version, update the version number in
version.rb, and then run
bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the
.gemfile to rubygems.org.
Bug reports and pull requests are welcome on GitHub at https://github.com/nickelser/activejob-traffic_control. Please look at the
.rubocop.ymlfor the style guide.