ruby

Chaining Sidekiq Workers

I’m working on an Rails app which syncs data with a 3rd party.  It turned out though that when one change was made in my application, it unfortunately needed to make 3 different sync requests to the 3rd party application.  And, the 3rd party app is finicky.  The request need to be in order, one at a time.

I needed to chain these requests together, but really did not want to introduce any of the gems I could find that would help me.  The gems were just “too much” for what I needed.  I ended up writing a “Command” class to call the various workers.  The command object would include itself as a param to the Sidekiq worker and the worker would then call the command class as part of it’s last step.  It ended up looking like this:

The command class needs to be able to serialize and deserialize from JSON since that is how Sidekiq persists parameters.  I wasn’t sure why I needed to do the deserialization in the worker myself, but it “worked”.

One thing I like about this strategy better than having a worker directly call the next worker is that from this command class, I can see exactly what workers are going to be called and in what order.  I can also use the workers independently, without being chained.  And, if I have another command  that needs to call a worker but getting the worker param is different, that logic is encapsulated in the command class.  Not spread around who knows where.

I still consider myself relatively new to Ruby and I’m not sure of the merits of Object.const_get vs. some other method, so feedback is appreciated.

Version Info:

  • Rails 4.2.7
  • Ruby 2.2.4
  • Sidekiq 4.1.4

Rails, Ransack, Simple Form, Checkboxes and More

I’ve been working in Rails recently, with a search from using Ransack that has a bunch of checkboxes.  This is a quick summary of some “features” I figured out along the way, for future reference.

Sample Form

In HAML:

= search_form_for @q, :url => search_path, :html => { :method => :get } do |f|
  = f.input :custom_scope, label: false, as: :boolean, inline_label: "#{ t 'label.label_here'}",
      checked_value: true,
      input_html: {checked: @q.custom_scope == "true"}
  = f.input :contact_info_opt_in_true, as: :boolean, label: false, 
      inline_label: "#{t 'label.opt_in'}"
  = f.input :contact_info_birthday_notification_true, as: :boolean, 
      label: false, inline_label: "#{t 'label.birthday_notification'}"
  = f.input :contact_info_interests_in, label: "#{t 'label.interests'}",
      collection: Interest.all
  = f.submit t('search_btn'), :class => 'btn btn-success'
  = link_to t('label.search_clear'), request.path, class:"cancel-button", role: :button

Sample Controller

def index
 if params[:q].present?
   clear_boolean(params[:q], :contact_info_opt_in_true)
   clear_boolean(params[:q], :contact_info_birthday_notification_true)
   clear_boolean(params[:q], :custom_scope)
 end

 if (params[:q].present? && params[:q].reject { |k, v| v.is_a?(Array) ? ( v.length == 1 ): v.blank? }.present?)
   # At least one search criteria is required
   @q = ContactInfo.ransack(params[:q])

   # Find the records for the current page.  Will use the max pages if current page is beyond the limit.
   @contact_infos = @q.result(distinct: true)
                      .page(current_page).per(25)
 else
   # There are no search params
   @q = ContactInfo.ransack(nil)
   @contact_infos = Kaminari.paginate_array([]).page(current_page).per(25)
 end

end

def max_pages
 10
end

private

# Current page or the max page limit
def current_page
 [(params[:page] || 1).to_i, max_pages].min
end

def clear_boolean(q, condition)
 q.delete(condition) if q[condition] == "0"
end

Ignore Unchecked Checkbox

In my form, if the checkbox is checked I want to include that field in the search criteria, but if it is unselected I want to ignore it completely, not just find the “false” ones. In the controller, before processing the search, the checkbox params are cleared. You can see this in the controller with the call to “clear_boolean” before ransack is called.

Use a Checkbox to Search by a Scope

The checkbox with the name “custom_scope” will search by the custom scope if the boolean is checked. In order to get this to work I needed to:

  1. Clear the checkbox value if it is unchecked (see above)
  2. Setup the scope for ransack. In the model define the scope and then add the scope to the ransackable_scopes
      def self.ransackable_scopes(auth_object = nil)
        %i(custom_scope)
      end
    
  3. Make the checkbox checked value equal to true. The value true, not the string “true”. You can see the setting in the checked_value of the custom_scope input.
  4. Finally, to make the checkbox be selected when the page reloads after the search, I set the checked value equal to the search param value. You can see this in the input_html of the custom_scope input

Search Only When Some Criteria is Selected

One of the requirements was for the search page to not run the search by default, but required some search criteria. To enforce this, I checked if there were any search params before running the search. Since my search criteria included a multi-select box, it also had to inspect some array lengths. Example is in the controller code, when checking if params are present.

Max Pages

Finally, I had a requirement to only show a max number of pages, regardless of if the results exceeded that page limit. My app was using will_paginate, but I found that Kaminari supported this requirement better. The call to “current_page” will ensure the results do not go past that max. In addition, I had to “fake” the empty search results for the view to properly display. You can see code above with the call to Kaminari.paginate.array.

Lots of searching to pull this all together and unfortunately I no longer have the links to the various sources. Stack Overflow featured heavily.

Version Info:
Rails 4.2.6, Ruby 2.2.4
Simple Form 3.2.1
Ransack 1.7.0
Kaminari 0.17.0

Mocking Fog with CarrierWave and Minitest

I kept getting the  error

Expected(200) <=> Actual(404 Not Found)

while trying to use Fog.mock! in unit tests.  I followed the guidelines in Fog and Carrierwave, but no luck.  Here’s the trick:  Fog uses credentials to segregate mock data, so be sure the controllers and tests are using the same credentials.

Final setup:

carrierwave.rb

CarrierWave.configure do |config|
  s3_config = YAML::load(File.read(Rails.root.join('config','s3.yml')))[Rails.env]

  config.fog_credentials = {
    :provider => 'AWS',
    :aws_access_key_id => s3_config['access_key_id'],
    :aws_secret_access_key => s3_config['secret_access_key'],
  }
  config.fog_directory = s3_config['bucket_name']
  config.fog_public = false
  config.fog_authenticated_url_expiration = 1800 # (in seconds) => 30 minutes

  if Rails.env.test?
    config.enable_processing = false
  end
end

s3.yml

test:
  bucket_name: testbucket
  access_key_id: secretkeyid
  secret_access_key: secretaccesskey

test_helper.rb

Fog.mock!
service = Fog::Storage.new({
  :provider                 => 'AWS',
  :aws_access_key_id        => 'secretkeyid',
  :aws_secret_access_key    => 'secretaccesskey'
})
service.directories.create(:key => 'testbucket')

Helpful links:

Version Info:

  • Rails 4.2.6, Ruby 2.2.4
  • carrierwave 0.11.0
  • fog 1.38.0
  • fog-aws 0.9.2

 

Ruby 2.2.2, UTF-8 CSV FILES And Excel

Okay, I’ve taken a bit of a detour with work and find myself supporting a Ruby on Rails application. All brand new to me.

Recently I had to get a CSV file, generated in the Rails app to open in Excel with accents correctly displayed.  The post at Exporting data to CSV and Excel in your Rails apps from plataformatec was very helpful, but for an older version of Ruby.  Here’s what I did to get it working w/Ruby 2.2.2:

The important point here is that if the file is in UTF-16LE with a BOM, Excel will correctly interpret the file. I found various information about this around the web.  The platformatec posting summarizes the three points that got this working:

  1. Use tabulations, not commas.
  2. Fields must NOT contain newlines.
  3. Use UTF-16 Little Endian to send the file to the user. And include a Little Endian BOM manually.

The open mode indicates the data is UTF-8 encoded but should be written UTF-16LE encoded.

w+:UTF-16LE:UTF-8

Helpful Links:

Version Info:

  • Ruby 2.2.2