UN-OCHA / ocha-api-site

OCHA Api

Todo

• Use array for provider on user
• Autofilter on provider
• Auto build aliases for routes
• Add base class for state providers
• Add command to add providers
• Add user command with providers
• Check read/write security
• batch endpoint
• extra fields as json blob
• add service name in header
• test version
• add plain post to KeyFigures?
• add permissions for country?

OPS

env

• N8N_WORKFLOW_ENDPOINT="http://192.168.3.20:5678/api/v1"
• N8N_WORKFLOW_API_KEY="n8n_api_cd9be...."

n8n

n8n needs the following environment variables defined, they will not be visible in the UI.

• N8N_TEMPLATES_HOST=http://api-test.docksal.site/api/v1/n8n
• OCHA_API_URL=http://api-test.docksal.site/api/v1
• ACAPS_USERNAME
• ACAPS_PASSWORD

Workflows

Workflows can be executed from files, no UI needed.

files is a mapped directory.

docker-compose exec -u node n8n n8n execute --file /files/IDPS.json
docker-compose exec -e FTS_YEAR=2019 -u node n8n n8n execute --file /files/FTS.json

docker-compose exec -e CERF_YEAR=2022 -u node n8n n8n execute --file /files/cerf_by_country.json
docker-compose exec -e CERF_YEAR=2021 -u node n8n n8n execute --file /files/cerf_by_country.json
docker-compose exec -e CERF_YEAR=2020 -u node n8n n8n execute --file /files/cerf_by_country.json
docker-compose exec -e CERF_YEAR=2019 -u node n8n n8n execute --file /files/cerf_by_country.json
docker-compose exec -e CERF_YEAR=2018 -u node n8n n8n execute --file /files/cerf_by_country.json
docker-compose exec -e CERF_YEAR=2017 -u node n8n n8n execute --file /files/cerf_by_country.json
docker-compose exec -e CERF_YEAR=2016 -u node n8n n8n execute --file /files/cerf_by_country.json

Add new key figure provider

console app:add-provider cerf "CERF" cerf key_figures
console app:add-user cerf [e-mail] cerf cerf cerf
console app:grant-access numbers cerf
console cache:clear

Reset data of a provider

console dbal:run-sql "delete from key_figures where provider = \"rw_crisis\""
console dbal:run-sql "select id, value from key_figures where year=2022 and iso3=\"afg\" and provider = \"rw_crisis\""
console dbal:run-sql "select * from key_figures where id = \"rw_crisis_AFG_2021_PeopleinNeed\""

console dbal:run-sql "update user set webhook = "https://numbers.reliefweb.int/webhook/listen\" where Username = "numbers"" console dbal:run-sql "update user set webhook = "http://numbers.docksal.site/webhook/listen\" where Username = "numbers""

Resources

• Export as CSV
• Hide entity from docs

Helpers

Entity

console make:entity -a InternallyDisplacedPersonsValues
console make:entity --api-resource InternallyDisplacedPersons
console make:migration
console doctrine:migrations:migrate
console app:add-user idps [e-mail] idps --idps

State provider

Use for custom endpoints.

console make:state-provider

Import command

Use for creating a basic command.

console make:command

Testing

fin console doctrine:database:drop --env=test --force
fin console doctrine:database:create --env=test --if-not-exists -n
fin console doctrine:schema:create --env=test -n
fin console netbrothers:version --env=test --drop-version
fin console netbrothers:version --env=test
fin console hautelook:fixtures:load --env=test -n
fin exec phpunit

Providers

console app:add-provider fts "FTS" fts key_figures
console app:add-provider idps "Internally displaced persons key figures" idps key_figures
console app:add-provider cbpf "Country-Based Pooled Funds" cbpf key_figures

Security

API-Key (active)

Admin have read/write access to all resources.

Add users with read permissions.

console app:add-user fts [e-mail] fts fts
console app:add-user rwcrisis [e-mail] rwcrisis rw_crisis
console app:add-user idps [e-mail] idps idps
console app:add-user admin [e-mail] admin --admin

Add users with read/write permissions.

console app:add-user idps [e-mail] idps idps idps
console app:add-user cbpf [e-mail] cbpf cbpf cbpf

JWT (not enabled)

Install using composer require lexik/jwt-authentication-bundle

console app:add-user username password [e-mail]
console lexik:jwt:generate-token [e-mail] --user-class="App\Entity\User"

Sources

FTS

Import data

console import:fts 2022
console import:fts 2021

API

• /api/fts/years
• /api/fts/countries
• /api/fts/country/{iso3}
• /api/fts/year/{year}

Conventional changelog

• composer run changelog to create changelog
• composer run release to create new release
• composer run release:patch to create new release and bump patch version
• composer run release:minor to create new release and bump minor version
• composer run release:major to create new release and bump major version

New release

On develop branch run the following

git fetch --all
today=$(date +%d-%m-%Y)
git checkout -b $today-prep-release
composer run release:patch
git add composer.json
git add CHANGELOG.md
git commit -m "chore: $today prep release"
git push origin $today-prep-release
gh_pr

• Merge to dev
• create PR to merge to main
• Merge to main
• Tag a new release
• Deploy

OCHA Presence

JSON

external_id has to be a string.

SELECT *
FROM key_figures k0_
WHERE k0_.provider = 'fts' AND k0_.year = '2023' AND JSON_EXTRACT(k0_.extra, '$.external_id') = '1116';

Distinct presence for provider

select distinct op.id, op.name, op.office_type from ocha_presence op inner join ocha_presence_external_id opei on op.id = opei.ocha_presence_id where provider_id = 'fts';

Distinct years for provider and presence

select distinct year from ocha_presence op inner join ocha_presence_external_id opei on op.id = opei.ocha_presence_id where provider_id = 'fts' and op.id = 'yem';

List of figures

select kf.*
from ocha_presence op
inner join ocha_presence_external_id opei on op.id = opei.ocha_presence_id
inner join ocha_presence_external_id_external_lookup mtm on opei.id = mtm.ocha_presence_external_id_id
inner join external_lookup el on el.id = mtm.external_lookup_id
inner join key_figures kf on el.external_id = JSON_UNQUOTE(JSON_EXTRACT(kf.extra, '$.external_id'))
where provider_id = 'fts' and op.id = 'yem';