jembi / openhim-mediator-mapping

An OpenHIM mediator for mapping incoming content to a different format

Openhim Mediator Mapping

The Mapping Mediator's function is to allow data validation, mapping, and orchestration between previously disparate systems. What makes it different from other existing mapping mediators is that it is generic. Users can define their own validation and mapping schemas for their specific use case. This is useful as it opens up Mapping Mediators to users without any coding knowledge.

Please see our documentation page for more details and our YouTube setup Video for a step by step guide.

Prerequisite

To store endpoint data:

• MongoDB >= 3.6 as we need dotted key support (e.g. { 'key.path': 'xyz' })
• Your MongoDB set up needs to be a replica set

Setup Mongo Replica Set

Yarn Set Up

This set up guide will make use of Docker to run the Mongo cluster and yarn to run the Mapping Mediator. The containers will connect with each other over a Docker Network.

The easiest way to setup the mongo containers is to use a docker-compose script. One is provided in the repo (docker-compose.mongo.yml)

Run the script from the project root directory with the following command:

docker-compose -f docker-compose.mongo.yml up -d

Next, setup the replica set. Once the containers have started up exec into one of the containers with the following command, docker exec -it mapper-mongo-1 mongo

Inside the shell enter the following:

config = {
  "_id": "mapper-mongo-set",
  "members": [
    {
      "_id": 0,
      "priority": 1,
      "host": "172.17.0.1:27017"
    },
    {
      "_id": 1,
      "priority": 0.5,
      "host": "172.17.0.1:27018"
    },
    {
      "_id": 2,
      "priority": 0.5,
      "host": "172.17.0.1:27019"
    }
  ]
}

rs.initiate(config)

NOTE: The IP used here is the local docker bridge IP. This IP is generally consistent, however different docker configurations could result in differences. You can confirm your IP with this terminal command: ifconfig docker0

WSL setup

There is no docker0 broadcasting channel when running in WSL, use eth0 inet IP instead. Replace the host values in the mongo replica set config above. You can confirm your IP with this terminal command: ifconfig eth0. You can confirm your current config in Mongo by using: docker exec -it mapper-mongo-1 mongo and then rs.conf().

With your replica set running, you can start up your Mapping Mediator. First install the Node dependencies by running yarn. Then include the following environment variable configs to connect your Mapping Mediator to the MongoDB replica set:

• MONGO_URL='mongodb://localhost:27017,localhost:27018,localhost:27019/mapping-mediator?replicaSet=mapper-mongo-set'
• OPENHIM_REGISTER=false

MONGO_URL='mongodb://localhost:27017,localhost:27018,localhost:27019/mapping-mediator?replicaSet=mapper-mongo-set' OPENHIM_REGISTER=false yarn start

Your mapper output logs should include the following:

Server listening on port 3003...
Connected to mongo on mongodb://localhost:27017,localhost:27018,localhost:27019/mapping-mediator?replicaSet=mapper-mongo-set
MongoDB Change Event Listeners Added

Docker Set Up

This set up guide will make use of Docker to run the Mongo cluster and the Mapping Mediator. The containers will connect with each other over a Docker Network.

The easiest way to setup the mongo containers is to use a docker-compose script. One is provided in the repo (docker-compose.mongo.yml)

Run the script with the following command:

docker-compose -f docker-compose.mongo.yml up -d

Once the containers have started up exec into one of the containers with the following command, docker exec -it mapper-mongo-1 mongo

Inside the shell enter the following:

config = {
  "_id": "mapper-mongo-set",
  "members": [
    {
      "_id": 0,
      "priority": 1,
      "host": "mapper-mongo-1:27017"
    },
    {
      "_id": 1,
      "priority": 0.5,
      "host": "mapper-mongo-2:27017"
    },
    {
      "_id": 2,
      "priority": 0.5,
      "host": "mapper-mongo-3:27017"
    }
  ]
}

rs.initiate(config)

Note: The Mongo replica set config between the docker and yarn setup instructions are not interchangeable.

With your replica set running, you can start up your Mapping Mediator with the following command:

docker run -it -p 3003:3003 -e OPENHIM_REGISTER=false -e MONGO_URL='mongodb://mapper-mongo-1:27017,mapper-mongo-2:27017,mapper-mongo:27017/mapping-mediator?replicaSet=mapper-mongo-set' --network mapper-cluster-network jembi/openhim-mediator-mapping:latest

The following parameters are specific to the docker start up process:

• -e OPENHIM_REGISTER=false

  This environment variable allows the Mapping Mediator to be started up without the OpenHIM. See the next section to start up the Mapper with the OpenHIM.

• -e MONGO_URL='mongodb://mapper-mongo-1:27017,mapper-mongo-2:27017,mapper-mongo-3:27017/mapping-mediator?replicaSet=mapper-mongo-set'

  The Mongo URL lists all members of the replica set as well as the name of the replica set

• --network mapper-cluster-network

  The Mapping Mediator Container needs to connect to the same Docker network on which the Mongo replica set communicates

Registering with the OpenHIM

The mapping mediator, like most other OpenHIM mediators, can register to an instance of the OpenHIM. This gives the user a graphical interface to interact with the mediator for status information and config options(if available).

Note: For multiple instances of the mediator to register to the same instance of the OpenHIM each mediator has to have a unique urn. You can set the urn using the environment variable MEDIATOR_URN.

To get an OpenHIM Core and Console instance setup, use the docker-compose.openhim.yml script here (or your preferred method). Run the docker compose command with the following:

docker-compose -f docker-compose.openhim.yml up -d

Open a browser and navigate to http://localhost:9000. Login with the default username [e-mail] and password openhim-password then by-pass the security warnings. For more information on how to do this see our OpenHIM set up tutorial. Finally, set your new openhim root user password.

Yarn Set Up

To register the mediator, you will need the following environment variables:

• MONGO_URL='mongodb://localhost:27017,localhost:27018,localhost:27019/mapping-mediator?replicaSet=mapper-mongo-set'
• OPENHIM_URL=https://localhost:8080
• OPENHIM_PASSWORD={openhim_password}

Substitute in your openhim password into the following command:

MONGO_URL='mongodb://localhost:27017,localhost:27018,localhost:27019/mapping-mediator?replicaSet=mapper-mongo-set' OPENHIM_URL=https://localhost:8080 OPENHIM_PASSWORD={openhim_password} yarn start

The output logs should contain the message: Successfully registered mediator!

Finally, go back to your browser and navigate to the Mediators section. Here you should see your mapping mediator instance registered on the OpenHIM with a recent heartbeat

Docker Set Up

To register as a mediator, you will need the following environment variables:

• OPENHIM_URL=https://openhim-core:8080
• OPENHIM_PASSWORD={openhim_password}
• MONGO_URL=mongodb://mapper-mongo-1:27017,mapper-mongo-2:27017,mapper-mongo-3:27017/mapping-mediator?replicaSet=mapper-mongo-set

docker run -e OPENHIM_URL=https://openhim-core:8080 -e OPENHIM_PASSWORD={openhim_password} -e MONGO_URL=mongodb://mapper-mongo-1:27017,mapper-mongo-2:27017,mapper-mongo-3:27017/mapping-mediator?replicaSet=mapper-mongo-set --network mapper-cluster-network --name mapper -d jembi/openhim-mediator-mapping:v3.0.0

Note: The Mapping Mediator is not exposed to your local machine in this case. All requests would need to go through the OpenHIM. To expose the Mapping Mediator port to your local machine include the following flag in your run command: -p 3003:3003

To see the output logs of the mapping mediator run the following command:

docker logs -f mapper

Here you should see the message: Successfully registered mediator!

Finally, go back to your browser and navigate to the Mediators section. Here you should see your mapping mediator instance registered on the OpenHIM with a recent heartbeat.

Kafka Integration

The mediator can consume and produce to kafka. The kafka broker will have to to set in the config file or via the env variable KAFKA_BROKERS (default is localhost:9092). The topic to consume from (kafkaConsumerTopic) or to produce to (kafkaProducerTopic) are specified in the endpoint schema as done below.

{
  "name": "External Request Test",
  "endpoint": {
    "pattern": "/externalRequestTest"
  },
  "transformation": {
    "input": "JSON",
    "output": "JSON"
  },
  "kafkaConsumerTopic": "2xx",
  "requests": {
    "response": [
      {
        "id": "fhirPatient",
         "kafkaProducerTopic": "3xx"
      }
    ]
  }
}

There are scripts for setting up a kafka instance locally. Run following script in the root folder

docker-compose -f docker-compose.kafka.yml up -d

# To create topics go to the root folder of the running container and run
./opt/bitnami/kafka/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --topic mytopic