Registry is a web-based platform that captures geo-spatial content using CSW-T protocol. Information is indexed into the Elasticsearch engine allowing fast searches.
Assuming ubuntu 14.04 OS.
-
Install and configure elasticsearch
wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add - echo "deb https://packages.elastic.co/elasticsearch/2.x/debian stable main" | sudo tee -a /etc/apt/sources.list.d/elasticsearch-2.x.list sudo apt-get update && sudo apt-get install elasticsearch sudo sed -i -e 's/#ES_HEAP_SIZE=2g/ES_HEAP_SIZE=1g/' /etc/default/elasticsearch sudo service elasticsearch start
-
Get registry source code
wget https://github.com/boundlessgeo/registry/archive/master.zip unzip master.zip cd registry-master/
Alternative using git
git clone https://github.com/boundlessgeo/registry.git cd registry
-
Installation of registry modules
pip install -r requirements.txt pip install -e .
-
Configure pycsw database
python registry.py pycsw -c setup_db
-
Database optimization (works only with PostgreSQL and MySQL)
python registry.py pycsw -c optimize_db
-
Run the test suite to verify everything is okay and install some dependencies like pytest for generating fake data.
sh python setup.py test
-
Run the server. The server will listen in port 8000
python registry.py runserver
-
List catalogs
curl http://localhost:8000/catalog
-
Create catalog using registry API
curl -XPUT http://localhost:8000/catalog/<catalog_slug>/csw
-
Add records into the database and search engine
- Using CSW transactions.
curl -XPOST -d @payload.xml http://localhost:8000/catalog/<catalog_slug>/csw
Note. You cannot records to Registry if catalog has not been created before.
- From command line.
python registry.py pycsw -c load_records -p /records/files/path/ -s <catalog_slug>
-
Search api endpoint.
-
For all records.
curl http://localhost:8000/api/
-
For a single catalog.
curl http://localhost:8000/catalog/<catalog_slug>/api/
-
-
Get record from csw.
curl -XGET http://localhost:8000/layer/<layer_uuid>.xml
-
Get mapproxy yaml configuration file.
curl -XGET http://localhost:8000/layer/<layer_uuid>.yml
-
Get mapproxy png.
curl -XGET http://localhost:8000/layer/<layer_uuid>.png
-
Re-index layers from pycsw database.
python registry.py pycsw -c reindex -s catalog_slug
-
Delete catalog.
- Removing records using a server request.
curl -XDELETE http://localhost:8000/catalog/<catalog_slug>/csw
- From command line.
python registry.py pycsw -c delete_records -s catalog_slug
You should see the indexed information. The a.matchDocs
value refers
to the number of layers returned by the search api.
Note. In registry, is possible to read all catalogs and layers. However, the catalog slug is necessary in order to add layers.
For development and testing of search api using a standalone swagger-ui server, please do the following.
-
Open Google Chrome without web-security.
open -a Google\ Chrome --args --disable-web-security --user-data-dir
-
Download and create a swagger instance specifying a different port.
git clone git@github.com:swagger-api/swagger-ui.git python -m SimpleHTTPServer 8001
-
Paste in the api selector, the endpoint for the registry swagger yml configuration file
http://localhost:8000/api/config
-
List layers uuid and save in a text file.
python registry.py pycsw -c list_layers > uuids.txt
-
Execute checking function for each layer uuid.
cat uuids.txt | python registry.py check_layers > checked_uuids.txt
-
Update Elasticsearch including reliability.
cat checked_uuids.txt | python registry.py reliability
-
Start elasticsearch
-
Run tests
python setup.py test
-
Record parsing failed: 'Csw' object has no attribute 'repository'
<ows:ExceptionText>Transaction (insert) failed: record parsing failed: 'Csw' object has no attribute 'repository'</ows:ExceptionText>
Reason 1: Elasticsearch is not running. This makes pycsw to silent the error using exception. Start elasticsearch service.
Reason 2: Database was not configured. Run in console
python registry.py pycsw -c setup_db
-
UNIQUE constraint failed: records.identifier.
<ows:ExceptionText>Transaction (insert) failed: ERROR: UNIQUE constraint failed: records.identifier.</ows:ExceptionText>
Reason: Records have been added previously into the database.
-
To debug mapproxy for a single layer.
-
Install mapproxy locally.
pip install MapProxy==1.9.0
-
Retreive from registry the yaml configuration file and copy.
curl http://localhost:8000/layer/<layer_uuid>.yml > layer.yml
-
Create mapproxy local server using the downloaded configuration file. The server will listen port 8080
mapproxy-util serve-develop layer.yml
-
Navigate through mapproxy web server and check the logs in terminal.
-
For arcgis servers, make sure the url path does not have the ?f=json. Also, verify that layer coordinates are in wgs84 projection.
-
- CSW-T support via pycsw
- Mirror information to Elasticsearch for faster searches
- OpenSearch based API to enable the use of facets on different fields (extending CSW standard).
- MapProxy support for easy TMS/WMTS access to any kind of resource
- Multi-catalog support.