Giters

Netflix / dynomite

A generic dynamo implementation for different k-v storage engines

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

dynomite cluster is not replicating between two data centers

venkatasivamadala opened this issue · comments

commented

I have 10 nodes per rack and one rack per dc.

dc_east4 - rack1 - 10 nodes
dc_central1 - rack2 - 10 nodes

Replication is not happening between datacenters. Please tell if I am doing anything wrong with the configuration.
dyn_o_mite:
data_store: '0'
datacenter: dc_east4
dyn_listen: 0.0.0.0:8101
dyn_port: 8101
dyn_seed_provider: simple_provider
dyn_seeds:

  • 100.148.52.69:8101:rack1:dc_east4:3006477103
  • 100.148.52.67:8101:rack1:dc_east4:2147483645
  • 100.148.52.21:8101:rack1:dc_east4:429496729
  • 100.148.52.66:8101:rack1:dc_east4:1288490187
  • 100.148.32.64:8101:rack1:dc_east4:1717986916
  • 100.148.52.22:8101:rack1:dc_east4:858993458
  • 100.148.52.72:8101:rack1:dc_east4:4294967290
  • 100.148.52.70:8101:rack1:dc_east4:3435973832
  • 100.148.52.68:8101:rack1:dc_east4:2576980374
  • 100.149.39.220:8101:rack2:dc_central1:3435973832
  • 100.149.33.5:8101:rack2:dc_central1:429496729
  • 100.149.33.131:8101:rack2:dc_central1:1288490187
  • 100.149.33.128:8101:rack2:dc_central1:858993458
  • 100.149.39.219:8101:rack2:dc_central1:3006477103
  • 100.149.39.217:8101:rack2:dc_central1:2147483645
  • 100.149.39.207:8101:rack2:dc_central1:1717986916
  • 100.149.39.218:8101:rack2:dc_central1:2576980374
  • 100.149.39.222:8101:rack2:dc_central1:4294967290
  • 100.149.39.221:8101:rack2:dc_central1:3865470561
    listen: 0.0.0.0:8102
    pem_key_file: /redis/dynomite/conf/dynomite.pem
    rack: rack1
    read_consistency: DC_ONE
    secure_server_option: all
    servers:
  • 100.148.52.71:6379:1
    timeout: 150000
    tokens: 3865470561
    write_consistency: DC_ONE
commented

@ipapapa Can you help?
Working with two nodes per DC, but replication is not working with similar configuration with 10 nodes per DC.

commented

From what I see you are using 1 per datacenter. I do not see any major issues. A few things to keep an eye on:

  • You can change secure_server_option: all to secure_server_option: datacenter as it does not play any role.
  • You can update listen: 0.0.0.0:8102 to listen 127.0.0.1:8102

I would recommend increasing the verbosity level and see what are the printed errors that cause the issue.

commented

@ipapapa Thanks for the update. Replication was not working for few nodes. Found out the issue is with encryption, so taken out the pem_key_file and it worked.