Chaining Sidekiq Workers

I’m working on an Rails app which syncs data with a 3rd party.  It turned out though that when one change was made in my application, it unfortunately needed to make 3 different sync requests to the 3rd party application.  And, the 3rd party app is finicky.  The request need to be in order, one at a time.

I needed to chain these requests together, but really did not want to introduce any of the gems I could find that would help me.  The gems were just “too much” for what I needed.  I ended up writing a “Command” class to call the various workers.  The command object would include itself as a param to the Sidekiq worker and the worker would then call the command class as part of it’s last step.  It ended up looking like this:

	require 'json'
	class AddIndividiualCmd
	attr_reader :roster_id, :current_step
	STEPS = %w(
	PhoneInsertWorker
	RosterLinkToSchoolWorker
	RosterLinkToSchoolAddressWorker
	)
	def initialize(roster_id, current_step = nil)
	@roster_id = roster_id
	@current_step = current_step || STEPS[0]
	end
	def run
	case @current_step
	when "PhoneInsertWorker"
	worker = Object.const_get(@current_step)
	roster = Roster.find(@roster_id)
	worker.perform_async(roster.phone.id, self.to_json)
	when "RosterLinkToSchoolWorker"
	worker = Object.const_get(@current_step)
	worker.perform_async(@roster_id, self.to_json)
	when "RosterLinkToSchoolAddressWorker"
	worker = Object.const_get(@current_step)
	worker.perform_async(@roster_id, self.to_json)
	else
	Rails.logger.info "Finished AddIndividiualCmd for roster [#{@roster_id}]"
	end
	end
	def next
	current_step_index = STEPS.index(@current_step)
	@current_step = current_step_index.nil? ? nil : STEPS[(current_step_index + 1)]
	run
	end
	def to_json(*a)
	{
	"json_class" => self.class.name,
	"data" => {'roster_id' => @roster_id, 'current_step' => @current_step}
	}.to_json(*a)
	end
	def self.json_create(o)
	new(o['data']['roster_id'], o['data']['current_step'])
	end
	end

view raw

AddIndividualCmd.rb

hosted with ❤ by GitHub

	class RosterLinkToSchoolWorker
	include Sidekiq::Worker
	include MyHelper
	def perform(roster_id, cmd_json = nil)
	roster = Roster.find(roster_id)
	link_roster_to_school(roster, cmd_json)
	end
	def link_roster_to_school(roster, cmd_json)
	my_helper_method(roster)
	if cmd_json.present?
	cmd = JSON.load(cmd_json)
	cmd.next()
	end
	end
	end

view raw

RosterLinkToSchoolWorker.rb

hosted with ❤ by GitHub

The command class needs to be able to serialize and deserialize from JSON since that is how Sidekiq persists parameters.  I wasn’t sure why I needed to do the deserialization in the worker myself, but it “worked”.

One thing I like about this strategy better than having a worker directly call the next worker is that from this command class, I can see exactly what workers are going to be called and in what order.  I can also use the workers independently, without being chained.  And, if I have another command  that needs to call a worker but getting the worker param is different, that logic is encapsulated in the command class.  Not spread around who knows where.

I still consider myself relatively new to Ruby and I’m not sure of the merits of Object.const_get vs. some other method, so feedback is appreciated.

Version Info:

• Rails 4.2.7
• Ruby 2.2.4
• Sidekiq 4.1.4

https://gist.github.com/aghuddleston/c3051a563117996a1e659fcef48812d6#file-addindividualcmd-rb

Rails, Ransack, Simple Form, Checkboxes and More

I’ve been working in Rails recently, with a search from using Ransack that has a bunch of checkboxes.  This is a quick summary of some “features” I figured out along the way, for future reference.

Sample Form

In HAML:

= search_form_for @q, :url => search_path, :html => { :method => :get } do |f|
  = f.input :custom_scope, label: false, as: :boolean, inline_label: "#{ t 'label.label_here'}",
      checked_value: true,
      input_html: {checked: @q.custom_scope == "true"}
  = f.input :contact_info_opt_in_true, as: :boolean, label: false, 
      inline_label: "#{t 'label.opt_in'}"
  = f.input :contact_info_birthday_notification_true, as: :boolean, 
      label: false, inline_label: "#{t 'label.birthday_notification'}"
  = f.input :contact_info_interests_in, label: "#{t 'label.interests'}",
      collection: Interest.all
  = f.submit t('search_btn'), :class => 'btn btn-success'
  = link_to t('label.search_clear'), request.path, class:"cancel-button", role: :button

Sample Controller

def index
 if params[:q].present?
   clear_boolean(params[:q], :contact_info_opt_in_true)
   clear_boolean(params[:q], :contact_info_birthday_notification_true)
   clear_boolean(params[:q], :custom_scope)
 end

 if (params[:q].present? && params[:q].reject { |k, v| v.is_a?(Array) ? ( v.length == 1 ): v.blank? }.present?)
   # At least one search criteria is required
   @q = ContactInfo.ransack(params[:q])

   # Find the records for the current page.  Will use the max pages if current page is beyond the limit.
   @contact_infos = @q.result(distinct: true)
                      .page(current_page).per(25)
 else
   # There are no search params
   @q = ContactInfo.ransack(nil)
   @contact_infos = Kaminari.paginate_array([]).page(current_page).per(25)
 end

end

def max_pages
 10
end

private

# Current page or the max page limit
def current_page
 [(params[:page] || 1).to_i, max_pages].min
end

def clear_boolean(q, condition)
 q.delete(condition) if q[condition] == "0"
end

Ignore Unchecked Checkbox

In my form, if the checkbox is checked I want to include that field in the search criteria, but if it is unselected I want to ignore it completely, not just find the “false” ones. In the controller, before processing the search, the checkbox params are cleared. You can see this in the controller with the call to “clear_boolean” before ransack is called.

Use a Checkbox to Search by a Scope

The checkbox with the name “custom_scope” will search by the custom scope if the boolean is checked. In order to get this to work I needed to:

1. Clear the checkbox value if it is unchecked (see above)
2. Setup the scope for ransack. In the model define the scope and then add the scope to the ransackable_scopes

     def self.ransackable_scopes(auth_object = nil)
       %i(custom_scope)
     end
   

3. Make the checkbox checked value equal to true. The value true, not the string “true”. You can see the setting in the checked_value of the custom_scope input.
4. Finally, to make the checkbox be selected when the page reloads after the search, I set the checked value equal to the search param value. You can see this in the input_html of the custom_scope input

Search Only When Some Criteria is Selected

One of the requirements was for the search page to not run the search by default, but required some search criteria. To enforce this, I checked if there were any search params before running the search. Since my search criteria included a multi-select box, it also had to inspect some array lengths. Example is in the controller code, when checking if params are present.

Max Pages

Finally, I had a requirement to only show a max number of pages, regardless of if the results exceeded that page limit. My app was using will_paginate, but I found that Kaminari supported this requirement better. The call to “current_page” will ensure the results do not go past that max. In addition, I had to “fake” the empty search results for the view to properly display. You can see code above with the call to Kaminari.paginate.array.

Lots of searching to pull this all together and unfortunately I no longer have the links to the various sources. Stack Overflow featured heavily.

Version Info:
Rails 4.2.6, Ruby 2.2.4
Simple Form 3.2.1
Ransack 1.7.0
Kaminari 0.17.0

Ruby 2.2.2, UTF-8 CSV FILES And Excel

Okay, I’ve taken a bit of a detour with work and find myself supporting a Ruby on Rails application. All brand new to me.

Recently I had to get a CSV file, generated in the Rails app to open in Excel with accents correctly displayed.  The post at Exporting data to CSV and Excel in your Rails apps from plataformatec was very helpful, but for an older version of Ruby.  Here’s what I did to get it working w/Ruby 2.2.2:

	require 'csv'
	module DownloadService
	OPEN_MODE = "w+:UTF-16LE:UTF-8"
	BOM = "\xEF\xBB\xBF" #Byte Order Mark
	def student_list
	File.open("#{file_name}.tsv", OPEN_MODE) do |f|
	csv_file = CSV.generate({:col_sep => "\t"}) do |csv|
	# header row
	csv << ['First_Name', 'Middle_Name', 'Last_Name']
	# put lots of rows in the csv file here
	end
	f.write BOM
	f.write(csv_file)
	end
	end
	end

view raw

csv_for_excel.rb

hosted with ❤ by GitHub

The important point here is that if the file is in UTF-16LE with a BOM, Excel will correctly interpret the file. I found various information about this around the web.  The platformatec posting summarizes the three points that got this working:

1. Use tabulations, not commas.
2. Fields must NOT contain newlines.
3. Use UTF-16 Little Endian to send the file to the user. And include a Little Endian BOM manually.

The open mode indicates the data is UTF-8 encoded but should be written UTF-16LE encoded.

w+:UTF-16LE:UTF-8

Helpful Links:

• Exporting data to CSV and Excel in your Rails apps
• Is it possible to force Excel recognize UTF-8 CSV files automatically? This stackoverflow answer pretty much sums up what I did.  I could not get UTF-8 w/a BOM to work, unfortunately.

Version Info:

• Ruby 2.2.2