codelibs / elasticsearch-configsync

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Sync file immediately

toanluu opened this issue · comments

Thanks for you plugin, it maybe useful to update files on ES nodes, e.g synonym files. i have few questions:

  1. Do I need to have some setting for java security policy to make it running? e.g.:

grant {
permission org.elasticsearch.script.ClassPermission "org.codelibs.elasticsearch.configsync.service.ConfigSyncService";
permission java.io.FilePermission "/etc/elasticsearch/config/test/*", "read,write";
};

  1. How can i make the file is sync immediately without waiting for schedule? I tried
    curl -XPOST localhost:9200/_configsync/file?path=test/user-dict.txt --data-binary @/tmp/user-dict.txt with new content in /tmp/user-dict.txt
    however in ES folder, i see test/user-dict.txt is not updated with new content.
  1. It's not needed. ConfigSync has read/write permission for all files.
  2. Send POST requst to /_configsync/flush

Thanks for you info. It seems the file content is not flushed in my setup. I use ES version 2.2.0 and install ConfigSync 2.2.0.
What i did:

  1. Create a file /tmp/user-dict.txt with content: "hello"
  2. curl -XPOST localhost:9200/configsync/file?path=test/user-dict.txt --data-binary @/tmp/user-dict.txt
  3. See that content in config/test/user-dict.txt is "hello"
  4. Change content in /tmp/user-dict.txt to "bonjour"
  5. curl -XPOST localhost:9200/configsync/file?path=test/user-dict.txt --data-binary @/tmp/user-dict.txt
    response: {"acknowledged":true}
  6. Check: curl -XGET localhost:9200/_configsync/file?path=test/user-dict.txt
    response: "bonjour"
  7. curl -XPOST http://localhost:9200/_configsync/flush
    response: {"acknowledged":true}
  8. Check content in config/test/user-dict.txt, it is still "hello"

In the ES log i just see a sequence of lines:
[2017-02-20 10:15:19,808][INFO ][org.codelibs.elasticsearch.configsync.service.ConfigSyncService] [Alex Wilder] Scheduled ConfigFileUpdater with 1m
[2017-02-20 10:16:19,812][INFO ][org.codelibs.elasticsearch.configsync.service.ConfigSyncService] [Alex Wilder] Scheduled ConfigFileUpdater with 1m
[2017-02-20 10:17:19,814][INFO ][org.codelibs.elasticsearch.configsync.service.ConfigSyncService] [Alex Wilder] Scheduled ConfigFileUpdater with 1m
....

Many thanks for looking at the issue.

Maybe there is issue in logic of checking timestamp to apply the file modification.
On my machine, Zurich time zone, when the file is saved, its timestamp is the same as time on the machine: 11:50

However after 20 minutes (time on the machine is 12:10) I post new content, the content in index is:
_source": {
"path": "test/user-dict.txt",
"content": "aGFsbG8K",
"@timestamp": "2017-02-20T11:10:43.566Z"
}
So 11:10 < 11:50 => content is not modified.

ok i see the line sdf.setTimeZone(TimeZone.getTimeZone("UTC")); is missing in the method getTimestamp compare to newest version. Could you please add that line for branch 2.2.x and make new build? Because of some constraints we cannot update our ES to version 5.x for now so we still have to use 2.2.0. Many thanks.