Kaligo Hotel Merge

This is a RoR API project for task at https://gist.github.com/arnvald/0af20bdceb375c4517dd7c845607e2fb

Notes

IMPORTANT

Development-mode class reloading is disable to make concurrency work.

# config/environments/development.rb
# config/environments/test.rb

config.cache_classes = true # enabled to make concurrency work
config.eager_load = true # enabled to make concurrency work

Requirement and depedencies

Ruby version: ruby 2.4.1

Get started

check out source code
run bundle

Run test

run rails t

Start services

run rake db:migrate
run rake db:setup
run rails s

Services will be running at http://localhost:3000

API

GET check/live
GET check/health
GET hotel/list

Call with specific suppliers, invalid supplier will be ignored:

curl "http://localhost:3000/hotel/list?checkin=20170812&checkout=20170913&destination=singapore&guests=2&suppliers=supplier1,supplier2,supplier3,supplier4"

Call without specific suppliers,

curl "http://localhost:3000/hotel/list?checkin=20170812&checkout=20170913&destination=singapore&guests=2"

System design

Dynamically loaded supplier API parser

Supplier data is saved via app/models/supplier.rb

name: API name
url: API entry point
api_parser: api specific parser file in app/api_parsers folder.

The file name need to be in snake_case, with extention, without path. eg. supplier_1_parser.rb The parser class in the file need to be in CamelCase. It need to be a match to the file name. eg. Supplier1Parser The parser need to be extended from BaseParser in app/api_parser/base_parser.rb. If no parser is specified, BaseParser will be used. If parser is specificed, but cannot be found, BaseParser will be used.

expire_sec: data expiration in second.

Druing run time, new supplier can be added to the database.

Its parser can be added to app/api_parsers folder without restaring the services.

Cache

This is a generic cache implemented as key-value store in database.
Cache's caller needs to define how cache behaves.
Cache data is stored via app/models/data_record.rb.
Cacher interface is implemented in app/models/data_cacher.rb as a singleton object.
Cache is used for individual API calls. So the cache key is defined as "#{self.name}-#{checkin}-#{checkout}-#{destination}-#{guests}".downcase

Potential Improvement

Thread pool should be implemented in app/api_parser/api_worker.rb to avoid re-creation of thread objects.
Clean up logic or key reuse logic should be implemented in app/modles/data_cacher.rb ro improve database utilization.
If merging hotel data is the bottleneck, then this result should be cached as well.
For a highly concurrent web services, each supplier API client should be implemented as seperated service. Cache, log and database should be implemented as indvidual services as well.

Kiza / kaligo_hotel_merge