A simple Elastic REST client written in Elixir.
This library talks to the Elastic(search) server through the HTTP/REST/JSON API. Its methods almost always return a HTTPoison request object.
When needed, the payload can be provided as an Elixir Map, which is internally converted to JSON. The library does not assume anything else regarding the payload and also does not (and will never) provide a magic DSL to generate the payload. That way users can directly manipulate the API data, that is sent to the Elastic server.
Elastix has 5 main modules and one utility module, that can be used, if the call/feature you want is not implemented (yet). However – please open issues or provide pull requests so I can improve the software for everybody. The modules are:
- Elastix.Index corresponding to: this official API Documentation
- Elastix.Document corresponding to: this official API Documentation
- Elastix.Search corresponding to: this official API Documentation
- Elastix.Bulk corresponding to: this official API Documentation
- Elastix.Mapping corresponding to: this official API Documentation
- and Elastix.HTTP – a thin HTTPoison wrapper
I will try and provide documentation and examples for all of them with time, for now just consult the source code.
start elastix application dependencies (or define it as an application dependency in mix.exs
):
Elastix.start()
create the Elastic index
Elastix.Index.create("http://127.0.0.1:9200", "sample_index_name", %{})
assuming you have a model product
create a document, search, or delete
# Elastic Server URL
elastic_url = "http://127.0.0.1:9200"
# Elastic Index Name
index_name = "shop_api_production"
# Elastic Document Type
doc_type = "product"
index_data = %{
name: product.name,
item_number: product.item_number,
inserted_at: product.inserted_at,
updated_at: product.updated_at
}
# Add mapping
mapping = %{
properties: %{
name: %{type: "text"},
item_number: %{type: "integer"},
inserted_at: %{type: "date"},
updated_at: %{type: "date"}
}
}
# add some search params according to Elastic JSON API
search_payload = %{}
# which document types should be included in the search?
search_in = [doc_type]
Elastix.Mapping.put(elastic_url, index_name, doc_type, mapping)
Elastix.Document.index(elastic_url, index_name, doc_type, product.id, index_data)
Elastix.Search.search(elastic_url, index_name, search_in, search_payload)
Elastix.Document.delete(elastic_url, index_name, doc_type, product.id)
It is possible to execute bulk
requests with elastix.
Bulk requests take as parameters the list of lines to send to Elasticsearch. You can also optionally give them options. Available options are:
index
the index of the requesttype
the document type of the request. (you can't specifytype
without specifyingindex
)
Examples
lines = [
%{index: %{_id: "1"}},
%{field: "value1"},
%{index: %{_id: "2"}},
%{field: "value2"}
]
# Send bulk data
Elastix.Bulk.post elastic_url, lines, index: "my_index", type: "my_type"
# Send your lines by transforming them to iolist
Elastix.Bulk.post_to_iolist elastic_url, lines, index: "my_index", type: "my_type"
# Send raw data directly to the API
data = Enum.map(lines, fn line -> Poison.encode!(line) <> "\n" end)
Elastix.Bulk.post_raw elastic_url, data, index: "my_index", type: "my_type"
# Finally, you can specify the index or the type directly in you lines
lines = [
%{index: %{_id: "1", _index: "my_index", _type: "my_type"}},
%{field: "value1"},
%{index: %{_id: "2", _index: "my_other_index", _type: "my_other_type"}},
%{field: "value2"}
]
Elastix.Bulk.post elastic_url, lines
Currently we can
- pass options to the JSON decoder used by Elastix (poison)
- optionally use shield for authentication (shield)
- optionally pass along custom headers for every request made to the elasticsearch server(s)s
- optionally pass along options to HTTPoison
- optionally use a different JSON library
by setting the respective keys in your config/config.exs
config :elastix,
json_options: [keys: :atoms],
shield: true,
username: "username",
password: "password",
httpoison_options: [hackney: [pool: :elastix_pool]]
The above for example will
- lead to the HTTPoison responses being parsed into maps with atom keys instead of string keys (be careful as most of the time this is not a good idea as stated here: https://github.com/devinus/poison#parser).
- use shield for authentication
To add custom headers to a request you must pass in the custom_headers option.
For example:
config :elastix,
custom_headers: {MyModule, :add_aws_signature, ["us-east"]}
This must be a {Module, :function, [args]}
tuple. The request will be added
to the head of the args list. The args list may be empty. The request is a map
with the method
, headers
, url
, and body
keys.
The function you define should return the full set of headers you want to send, including any headers passed in. For example:
defmodule MyModule do
def add_aws_signature(request, region) do
[{"Authorization", generate_aws_signature(request, region)} | request.headers]
end
defp generate_aws_signature(request, region) do
# See: https://github.com/bryanjos/aws_auth or similar
end
end
To use a different JSON library you must pass in the json_codec option.
For example:
config :elastix,
json_codec: JiffyCodec,
json_options: [:return_maps]
This must be a module that implements the Elastix.JSON.Codec behavior. For example:
defmodule JiffyCodec do
@behaviour Elastix.JSON.Codec
def encode!(data), do: :jiffy.encode(data)
def decode(json, opts \\ []), do: {:ok, :jiffy.decode(json, opts)}
end
You need elastic search running locally on port 9200. A quick way of running any version of elastic, is via docker:
$ docker run -p 9200:9200 -it --rm elasticsearch:5.1.2
Then clone the repo and fetch its dependencies:
$ git clone git@github.com:werbitzky/elastix.git
$ cd elastix
$ mix deps.get
$ mix test
Copyright © 2017 El Werbitzky werbitzky@gmail.com This work is free. You can redistribute it and/or modify it under the terms of the Do What The Fuck You Want To Public License, Version 2, as published by Sam Hocevar. See http://www.wtfpl.net/ for more details.