Giters

ckan / ckanext-xloader

Express Loader - quickly load data into DataStore. A replacement for DataPusher.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

High availability setup

clauded opened this issue · comments

commented

We're planning to deploy our CKAN setup with high availability (two CKAN servers, one db container, one solr container and one redis container). We would like to have xloader highly available as well.

  • Is there a problem if we install the extension on both CKAN servers?
  • Is there any possible conflicts doing so?
  • Any special configuration to do?
commented

HA XLoader is certainly possible. However, there is an increased risk of slowdowns and database transaction deadlocks when datasets have a high number of resources and different XLoader instances compete over them.

We have a setup that puts XLoader on its own server(s), and no special configuration is needed; it just needs the CKAN configuration file, and access to the database and Redis queue, just like CKAN would.