API.md

RowBoat API

This is really more of a summary of what you can do with RowBoat::Base since you subclass it to do everything :)

Contents

• Basic Usage
• .import
• initialize
• import
• import_into
• csv_source
• column_mapping
• preprocess_row
• preprocess_rows
• options
• handle_failed_row
• handle_failed_rows
• value_converters
• rollback_transaction?

Basic Usage

Just subclass RowBoat::Base and define the import_into and column_mapping methods to get started (They're the only methods that you're required to implement).

class ImportProduct < RowBoat::Base
  def import_into
    Product
  end

  def column_mapping
    {
      downcased_csv_column_header: :model_attribute_name,
      another_downcased_csv_column_header: :another_model_attribute_name
    }
  end
end

.import

Description

Imports database records form the given CSV-like object. The CSV-like object can be anything that can be passed to SmarterCSV.process (string paths to files, files, tempfiles, instances of StringIO, etc).

It returns a hash containing

• :invalid_records - an array of all records that failed to import since they were invalid. If you've configured the :validate option to be false it will be an empty array.
• :total_inserts - the total number of database inserts that were run.
• :inserted_ids - an array of all of the ids of records inserted into the database.
• :skipped_rows - every row skipped by returning nil from preprocess_row.

If you want to pass additional information to help import CSVs, don't override this method. It just passes through to initialize so override that :)

Example

Basic Use

class ImportProduct < RowBoat::Base
  # required configuration omitted for brevity
end

ImportProduct.import("path/to/my.csv")

Advanced Use

class ImportProduct < RowBoat::Base
  # required configuration omitted for brevity
  def intitialize(csv_source, my_options)
    super(csv_source)
    @my_options = my_options
  end
end

ImportProduct.import("path/to/my.csv", foo: "bar")

initialize

Description

Makes a new instance with the given CSV-like object. See .import for more details around when and how to override this method.

import

Description

The instance method that actually parses and imports the CSV. Generally, you wouldn't call this directly and would instead call .import.

import_into

Description

It is required that you override this method to return whatever ActiveRecord class you want your CSV imported into.

Example

Basic Use

class ImportProduct < RowBoat::Base
  # other required configuration omitted for brevity
  def import_into
    Product
  end
end

Advanced Use

class ImportProduct < RowBoat::Base
  # other required configuration omitted for brevity
  def import_into
    if csv_source.is_a?(String) && csv_source.match(/category/i)
      ProductCategory
    else
      Product
    end
  end
end

ImportProduct.import("path/to/category.csv")
ImportProduct.import("path/to/product.csv")

csv_source

Description

Whatever you originally passed in as the CSV source.

Example

class ImportProduct < RowBoat::Base
  # other required configuration omitted for brevity
  def import_into
    # `csv_source` is available in any of our instance methods
    if csv_source.is_a?(String) && csv_source.match(/category/i)
      ProductCategory
    else
      Product
    end
  end
end

ImportProduct.import("path/to/category.csv")
ImportProduct.import("path/to/product.csv")

column_mapping

Description

It is required that you override this method with either a hash that maps columns in your CSV to their preferred names or an array of your preferred column names.

By default when using a hash

• CSV column names are downcased symbols of what they look like in the CSV.
• CSV columns that are not mapped are ignored when processing the CSV.

If you're familiar with SmarterCSV, this method essentially defines your :key_mapping with the :remove_unmapped_keys setting set to true when provided with a hash. When given an array it is the :user_provided_headers option.

You can change these defaults by overriding the options method.

Example

class ImportProduct < RowBoat::Base
  # other required configuration omitted for brevity
  def column_mapping
    {
      prdct_nm: :name,
      "price/cost_amnt": :price_in_cents
    }
  end
end

# or...

class ImportProduct < RowBoat::Base
  # other required configuration omitted for brevity
  def column_mapping
    [:name, :price_in_cents]
  end
end

preprocess_row

Description

Implement this method if you need to do some work on the row before the record is inserted/updated.

If you return nil from this method, the row will be skipped in the import. You can access these rows in the return value from .import (the :skipped_rows key).

You also have access to row_number here.

If the work you intend to do with the row only requires changing one attribute, it is recommended that you override value_converters instead of this.

Example

  class ImportProduct < RowBoat::Base
    # required configuration omitted for brevity
    def preprocess_row(row)
      { position: row_number }.merge(row)
    end
    # or...
    def preprocess_row(row)
      if row[:name] && row[:price]
        row
      else
        nil
      end
    end
  end

preprocess_rows

Description

Override this method if you need to do something with a chunk of rows (the chunk size is determined by the :chunk_size option in the options method).

If you need to filter particular rows, it's better to just implement preprocess_row and return nil for the rows you want to ignore.

Example

class ImportProduct < RowBoat::Base
  # required configuration omitted for brevity
  def preprocess_rows(rows)
    if skip_batch?(rows)
      super([])
    else
      super
    end
  end

  def skip_batch?(rows)
    # decide whether or not to skip the batch
  end
end

options

Description

Implement this to configure RowBoat, SmarterCSV, and activerecord-import.

Except for :wrap_in_transaction, all options pass through to SmarterCSV and activerecord-import.

:wrap_in_transaction simply tells RowBoat whether or not you want your whole import wrapped in a database transaction.

Whatever you define in this method will be merged into the defaults:

• :chunk_size - 500
• :key_mapping - column_mapping
• :recursive - false
• :remove_unmapped_keys - true
• :validate - true
• :value_converters - csv_value_converters
• :wrap_in_transaction - true

Don't provide value_converters or key_mapping options here. Implement the value_converters and column_mapping respectively.

Example

class ImportProduct < RowBoat::Base
  # required configuration omitted for brevity
  def options
    {
      chunk_size: 1000,
      validate: false,
      wrap_in_transaction: false
    }
  end
end

handle_failed_row

Description

Implement this to do some work with a row that has failed to import.

It's important to note that

• This happens after the import has completed.
• The given row is an instance of whatever class was returned by import_into.

These records are also available in the return value of .import.

Example

class ImportProduct < RowBoat::Base
  # required configuration omitted for brevity
  def handle_failed_row(row)
    puts row.errors.full_messages.join(", ")
  end
end

handle_failed_rows

Description

Override this method to do some work will all of the rows that failed to import.

Example

class ImportProduct < RowBoat::Base
  # required configuration omitted for brevity
  def handle_failed_rows(rows)
    puts "Failed to import #{rows.size} rows :("
    super
  end
end

value_converters

Description

Implement to specify how to translate values from the CSV into whatever sorts of objects you need.

Simply return a hash that has the mapped column name (ie, what you mapped it to in the column_mapping method) as a key pointing to either

• a method name as a symbol
• a proc or lambda
• an object that implements convert

Regardless of which one you choose, it takes a value and returns a converted value.

This is essentially a sugared up version of :value_converters option in SmarterCSV.

Example

class ImportProduct < RowBoat::Base
  # required configuration omitted for brevity
  def value_converters
    {
      sell_by: :convert_date,
      name: -> (value) { value.titlelize },
      price: proc { |value| value.to_i },
      description: DescriptionConverter
    }
  end

  def convert_date(value)
    Date.parse(value) rescue nil
  end
end

module DescriptionConverter
  def self.convert(value)
    value.present? ? value : "default description :("
  end
end

rollback_transaction?

Description

Implement this method if you'd like to rollback the transaction after it otherwise has completed.

Note: imports are only wrapped in a transaction if the wrap_in_transaction option is true. It defaults to true but this can be configured in options

Example

class ImportProduct < RowBoat::Base
  # required configuration omitted for brevity
  def rollback_transaction?
    CsvService.already_imported?(csv_source)
  end
end