This workshop aims to explain and exemplify the use of Postgraphile and PostgreSQL to generate a spatial GraphQL API.
- Create and restore a PostgreSQL database
- Using PostGraphile
- Pagination
- Filters
- Smart tags
- Extending the schema
6.1) Computed columns
6.2) Custom queries - CRUD Mutations
- Authentication
GraphQL is a query language for your API. GraphQL isn't tied to any specific database or storage engine and is instead backed by your existing code and data.
A GraphQL service is created by defining types and fields on those types, then providing functions for each field on each type.
If you are new to GraphQL it might be good to check the official documentation: https://graphql.org/learn/
In order to move forward make sure you have installed:
- PostgreSQL with PostGIS (you can use docker)
- NodeJS v14.x.x
- npm
- pgAdmin4 (recommended)
For install procedures for local postgreSQL and pgadmin: here
In order to start the workshop we will use an existing database. The ideia is to show how you can use one existing spatial database and generate a GraphQL API on top of it.
Using pgAdmin please create a new, empty database and then restore it using the following file initial_db.backup into the new recently created database.
After restoring the DB you will see 4 tables and 3 schemas:
- municipality, Spatial table with portuguese municipalities.
- population, Non-spatial table with portuguese population per municipality;
- parcels, Spatial table used to collect polygons during field campaign;
- landcover, Spatial (vector) table with landcover for Lisbon region from Corine 2018
- srtm, Spatial (raster) table with SRTM for Lisbon region.
As mentioned here
- app_public, Tables and functions to be exposed to GraphQL (or any other system) - it's your public interface. This is the main part of your database.
- app_private, No-one should be able to read this without a SECURITY DEFINER function letting them selectively do things. This is where you store passwords (bcrypted), access tokens (hopefully encrypted), etc.
- public, Should be empty, used only as a default location for PostgreSQL extensions.
In order to implement a spatial GraphQL API we will make use of PostGraphile (https://www.graphile.org). If you never used PostGraphile we recommend to check its documentation. We also recommend these cheatsheets: https://learn.graphile.org/
Part of this workshop was based on PostGraphile docs.
According to the documentation PostGraphile is formed of three forms of usage:
-
CLI, the most user-friendly;
-
Library, it gives more power than using the CLI, suitable for Node.js with Connect, Express or Koa applications;
-
Schema-only, deepest layer which contains all the types, fields and resolvers.
At this workshop we will use mainly the CLI. Eventually, if we have time, we'll show a very basic library usage with NodeJS and Express.
You can check the official docs for more information on how to use the CLI, https://www.graphile.org/postgraphile/usage-cli/
Install PostGraphile globally via npm:
npm install -g postgraphile
PostGraphile can be customized using plugins. You can find more info about this on GraphQL Schema Plugins.
We will make use of the following plugins:
- @graphile-contrib/pg-simplify-inflector, more info here
- @graphile/postgis - Adds postgis support to PostGraphile, more info here
- postgraphile-plugin-connection-filter - Adds a powerful filtering to PostGraphile, more info here
- postgraphile-plugin-connection-filter-postgis - Adds spatial filtering mechanisms into PostGraphile and the above plugin, more info here.
In order to install them we need to run:
npm install -g \
@graphile-contrib/pg-simplify-inflector \
@graphile/postgis \
postgraphile-plugin-connection-filter \
postgraphile-plugin-connection-filter-postgis
More info about plugins can be found on PostGraphile community plugins
Now that we have installed the CLI we will run it as following
postgraphile \
--subscriptions \
--watch \
--dynamic-json \
--no-setof-functions-contain-nulls \
--no-ignore-rbac \
--no-ignore-indexes \
--port 5000 \
--show-error-stack=json \
--extended-errors hint,detail,errcode \
--append-plugins @graphile-contrib/pg-simplify-inflector,@graphile/postgis,postgraphile-plugin-connection-filter,postgraphile-plugin-connection-filter-postgis \
--skip-plugins graphile-build:NodePlugin \
--simple-collections only \
--graphiql "/" \
--enhance-graphiql \
--allow-explain \
--enable-query-batching \
--legacy-relations omit \
--connection "postgres://postgres:postgis@localhost/workshop_graphql" \
--schema app_public
This will generate a minimal schema, since we are omitting the NodePlugin, with advanced filter mechanism and postgis support given by the added plugins from above.
Now that you run the CLI command, point your browser to http://localhost:5000 give it a first try. This interface is GraphiQL, a GraphQL IDE.
Now that we setup our inital API let's query it:
- Query municipality with
ID 153
{
municipality(id:153){
name
district
}
}
- Query municipality with
ID 153
and get its population. Don't forget population table is related to themunicipalities
using attributeDICO
.
{
municipality(id:153){
name
district
populationByDico{
households
femaleResidents
maleResidents
}
}
}
- Get all municipalities. Notice that plural connections are generated automatically
municipalitiesList
.
{
municipalitiesList{
name
district
}
}
- Get all municipalities and its population.
{
municipalitiesList{
name
district
populationByDico{
femaleResidents
maleResidents
households
}
}
}
- Get
first 10
municipalities and its population.
{
municipalitiesList(first:10){
name
district
populationByDico{
femaleResidents
maleResidents
households
}
}
}
- Get all parcels.
{
parcelsList{
name
createdBy
}
}
Its now time to go spatial!
- Get the geometry as geojson and the SRID from the first municipality.
{
municipalitiesList(first:1) {
name
district
geom{
srid
geojson
}
}
}
- Get the geometry as geojson and the SRID from the first parcel.
{
parcelsList(first:1){
name
createdBy
geom{
srid
geojson
}
}
}
PostGraphile automatically generates sub geometries, the next query shows how that can be achived out of the box. Parcels geom column is MultiPolygon data type, therefore we can generate all sub-geometries that compose MultiPolygon.
{
parcelsList(first:1){
name
createdBy
geom{
srid
geojson
polygons{
exterior{
geojson
points{
geojson
x
y
}
}
}
}
}
}
We will not focus on this workshop on pagination but it is a very important concept in GraphQL, we recommend reading https://graphql.org/learn/pagination/ to better understand how pagination can be handled in GraphQL.
As you might have noticed on the first queries we started querying one municipality and end up with plural connections which are part of the pagination concept.
- Using offset
{
municipalitiesList(first:10, offset:10){
name
district
populationByDico{
femaleResidents
maleResidents
households
}
}
}
- Using last
{
municipalitiesList(last:10){
name
district
populationByDico{
femaleResidents
maleResidents
households
}
}
}
In order to have some simplicity we deactivated cursor connections these type of connections come from the Cursor Connections Specification for more information you should read this specification since they can be quite useful. Cursor connections allows perform cursor-based pagination, and is seen as a GraphQL best practice.
We can control how PostGraphile CLI generates collections
using:
--simple-collections omit
"omit" - PostGraphile generates cursor connections only;
--simple-collections only
"only" - simple collections only (no cursor connections);
--simple-collections both
"both" - both cursor and simple connections.
You can try to activate both cursor and simple connections, and explore the schema, please check the differences as following:
postgraphile \
--subscriptions \
--watch \
--dynamic-json \
--no-setof-functions-contain-nulls \
--no-ignore-rbac \
--no-ignore-indexes \
--port 5000 \
--show-error-stack=json \
--extended-errors hint,detail,errcode \
--append-plugins @graphile-contrib/pg-simplify-inflector,@graphile/postgis,postgraphile-plugin-connection-filter,postgraphile-plugin-connection-filter-postgis \
--skip-plugins graphile-build:NodePlugin \
--simple-collections both \
--graphiql "/" \
--enhance-graphiql \
--allow-explain \
--enable-query-batching \
--legacy-relations omit \
--connection "postgres://postgres:postgis@localhost/workshop_graphql" \
--schema app_public
During this workshop we wont use relay connections anymore. You can remove them using --simple-collections only
or just copy the CLI command from the beginning.
If you, just like me, you prefer to use simple connections but you don't like List
suffix on the simple collections, you can remove it using {graphileBuildOptions: {pgOmitListSuffix: true}}
to the options passed to PostGraphile library.
PostGraphile supports rudimentary filtering on connections using a condition argument. This condition mechanism is very limited and does not support spatial filtering. Therefore we will use instead connection-filter plugin that we already installed and has advanced filter capabilities, including spatial filtering based on postgraphile-plugin-connection-filter-postgis.
- Query all parcels that have
benfica
🏟️ in its name.
{
parcelsList(filter: {name: {includesInsensitive: "benfica"}}) {
name
createdBy
}
}
- Query all parcels created by
user1
:
{
parcelsList(filter: {createdBy: {like: "user1"}}) {
name
createdBy
}
}
- Query all municipalities from Lisbon district:
{
municipalitiesList(filter: {district: {like: "Lisboa"}}) {
name
district
}
}
- Query all municipalities in a list of districts
{
municipalitiesList(filter: {district: {in: ["Lisboa","Porto"]}}) {
name
district
populationByDico {
households
}
}
}
Since we have postgraphile-plugin-connection-filter-postgis we can use spatial filters. Please take some time exploring available geometry filters in Graphiql IDE
{
municipalitiesList(
filter: {
geom: {
bboxIntersects2D: {
type: "Polygon"
coordinates: [
[
[-9.253921508789062, 38.70855351447061]
[-9.185256958007812, 38.70855351447061]
[-9.185256958007812, 38.74497964505743]
[-9.253921508789062, 38.74497964505743]
[-9.253921508789062, 38.70855351447061]
]
]
}
}
}
) {
name
district
}
}
We will now get all municipalities (from municipality table) that intersect Lisbon Airport. But at the same time we will show how to use variables.
- First lets get Lisbon Airport geojson
{
parcelsList(first: 1, filter: { name: { like: "Lisbon airport" } }) {
name
createdBy
geom {
geojson
}
}
}
You should get the following geojson result from the previous query. Please add it to https://geojson.io and confirm its location.
{
"type": "MultiPolygon",
"coordinates": [
[
[
[
-9.130609331,
38.801232389
],
[
-9.123779675,
38.799759326
],
[
-9.131948479,
38.784359123
],
[
-9.126725801,
38.766548453
],
[
-9.129805842,
38.763200583
],
[
-9.149223489,
38.765611049
],
[
-9.130609331,
38.801232389
]
]
]
]
}
- Next lets write a query with one input variable, in this case variable name is
input1
of type GeoJSON. On this particular example we name our queryquery1
.
query query1 ($input1: GeoJSON) {
municipalitiesList(filter: {geom: {intersects: $input1}}) {
name
district
}
}
Now we need to insert the variable, to achive that please insert below code into QUERY VARIABLES
{"input1": {
"type": "MultiPolygon",
"coordinates": [
[
[
[
-9.130609331,
38.801232389
],
[
-9.123779675,
38.799759326
],
[
-9.131948479,
38.784359123
],
[
-9.126725801,
38.766548453
],
[
-9.129805842,
38.763200583
],
[
-9.149223489,
38.765611049
],
[
-9.130609331,
38.801232389
]
]
]
]
} }
You should have something like the image below
As we can see Lisbon Airport is on 2 Municipalities: Lisbon and Loures.
Its possible to customise PostGraphile GraphQL schema by using tags on our database tables, columns, functions etc. These can rename, omit, etc from the GraphQL schema. In other words, it allow us to change the GraphQL schema without changing the database data model.
More information on Smart tags and how to use them can be found here: https://www.graphile.org/postgraphile/smart-tags/
Using PgAdmin lets run the following SQL code using PgAdmin. Check what happens on the GraphQL schema.
comment on table app_public.municipality is E'@omit';
As you realized all connections to municipality have been removed, although at the database level we only added one comment, nothing changed the DB.
We can remove the smart tag and revert its effect by simply remove the previous comment.
comment on table app_public.municipality is NULL;
Lets now omit SRTM from our schema because its a raster dataset. We'll access it using a different technique.
comment on table app_public.srtm is E'@omit';
In order to rename an object we can use @name. Please run the following to rename out table landcover
.
comment on table app_public.landcover is E'@name clc_landcover';
Notice that clc_landcover
was changed into clcLandcover
.
Columns can also be renamed.
comment on column app_public.landcover.label3 is E'@name label';
Moving forward on our schema simplification lets now rename a constrain (relationship) in order to have clear names. Please run the following example and check what happens in your schema, inside population
and municipality
.
comment on constraint population_dico_fkey on app_public.population_stat is
E'@foreignFieldName population\n@fieldName municipality\nDocumentation here.';
One of the most important capabilities of PostGraphile if the ability to extend GraphQL schema using functions. This gives us the ability to use the power of PostgreSQL & PostGIS to generate any processing algorithms.
From the docs: "Computed columns" add what appears to be an extra column (field) to the GraphQL table type, but, unlike an actual column, the value for this field is the result of calling a function defined in the PostgreSQL schema. This function will automatically be exposed to the resultant GraphQL schema as a field on the type; it can accept arguments that influence its result, and may return either a scalar, record, list or a set. computed-columns/)
In this example we will generate an extra field on the parcels connection which give us the area of that parcel.
create or replace function app_public.parcels_area(p app_public.parcels)
returns real as $$
select ST_Area(p.geom,true);
$$ language sql stable;
GraphQL query:
{
parcelsList(first: 2) {
name
area
}
}
Also works with filters Make sure you removed --no-ignore-indexes
from PostGraphile executable.
{
parcelsList(filter: {area: {greaterThan: 300000}}) {
name
area
srtmList {
min
max
mean
}
}
}
On the next example we will generate an extra field on the parcels connection which give us one array with all intersecting landcover types.
create or replace function app_public.parcels_clc_landcover(p app_public.parcels)
returns varchar[] as $$
SELECT array_agg(distinct l.label3)
FROM app_public.landcover AS l
WHERE ST_Intersects(p.geom,l.geom)
$$ language sql stable;
GraphQL query:
{
parcelsList(first: 2) {
name
area
clcLandcover
}
}
With filters, get all parcels that intersect Green urban areas
{
parcelsList(filter: {clcLandcover: {contains: "Green urban areas"}}) {
name
area
clcLandcover
}
}
On the next example we will generate an extra fields on the parcels connection which gives STRM raster statistics.
DROP TYPE IF EXISTS srtm_stats CASCADE;
CREATE TYPE srtm_stats AS (
"min" real,
"max" real,
"mean" real
);
create or replace function app_public.parcels_srtm(p app_public.parcels)
returns setof srtm_stats as $$
WITH t AS (
SELECT st_summarystats(ST_Union(ST_Clip(r.rast, ST_Transform(p.geom,3763),true))) as stats
FROM app_public.srtm AS r
WHERE ST_Intersects(ST_Transform(p.geom,3763),r.rast)
)
SELECT (stats).min,(stats).max,(stats).mean FROM t;
$$ language sql stable;
For more information on how to use raster data inside PostGIS you can check my workshop-postgis-raster.
GraphQL query:
{
parcelsList {
name
area
srtmList {
min
max
mean
}
}
}
With filters:
{
parcelsList(filter: { name: { includesInsensitive: "benfica" } }) {
name
area
srtmList {
min
max
mean
}
}
}
To discuss: What is the difference between a computed column and a PostgreSQL generated column?
While Computed columns generate one extra field on a specific connection, custom queries can add root-level Query fields to our GraphQL schema. This can be quite important while generating our API specially for processing algorithms.
On this example we are going to generate a custom query where the user can insert a GeoJSON with a geometry and a distance. It will return all landcover rows that intersect that geometry. If distance is specified the search radius will include that distance using a Buffer (ST_Buffer) around the specified geometry.
create or replace function app_public.get_landcover(geometry JSON, distance real DEFAULT NULL)
returns SETOF app_public.landcover as $$
declare
g_geom geometry;
BEGIN
IF distance IS NOT NULL AND distance > 0 THEN
IF distance > 10001 THEN
RAISE EXCEPTION 'Maximum allowed distance for this operation is 10 km.';
ELSE
g_geom=ST_SetSRID(st_buffer(ST_GeomFromGeoJSON(geometry)::geography,distance)::geometry,4326);
END IF;
ELSE
g_geom=ST_SetSRID(ST_GeomFromGeoJSON(geometry),4326);
END IF;
RETURN QUERY
SELECT *
FROM app_public.landcover AS l
WHERE ST_Intersects(g_geom,l.geom);
END;
$$ language plpgsql stable;
GraphQL query:
{
getLandcoverList(
geometry: { type: "Point", coordinates: [-9.1615, 38.7122] }
distance: 1000
) {
label
}
}
From the docs: CRUD stands for "Create, Read, Update, Delete", is a common paradigm in data manipulation APIs; "CRUD Mutations" refer to all but the "R". PostGraphile will automatically add CRUD mutations to the schema for each table; this behaviour can be disabled via the --disable-default-mutations
CLI setting.
According to GraphQL convention, any operation that cause change should be sent explicitly via a mutation. Mutations in GraphQL change data, like inserting data into a database or altering data already in a database.
In the current Parcels table we dont have any mandatory field apart from ID. In this case ID is automaticaly generated by the DB. Lets run the following code in PostgreSQL and check what happens on our GraphQL IDE.
--First lets remove any row without geometry
DELETE FROM app_public.parcels WHERE geom is NULL;
ALTER TABLE app_public.parcels ALTER COLUMN geom SET NOT NULL;
mutation {
createParcel(
input: { parcel: { createdBy: "userx", name: "My new parcel" } }
) {
parcel {
id
name
createdBy
geom{
geojson
}
}
}
}
As we can see PostGraphile automatically reads the DB constrains and transposes it into our GraphQL Schema mutations.
Since ID is automaticaly generated by the DB there's no need for it to appear as a mutation input. Lets omit it with a smart tag:
comment on column app_public.parcels.id is E'@omit create,update,delete';
As you can see, ID no longer appears as an option in create
, update
or delete
mutations.
Lets create a new parcel.
mutation {
createParcel(
input: {
parcel: {
createdBy: "userx"
name: "Campo grande garden"
geom: {
type: "MultiPolygon"
coordinates: [
[
[
[-9.155495166778564, 38.75901950184664]
[-9.155731201171875, 38.758768515539764]
[-9.148414134979248, 38.74879527866384]
[-9.15259838104248, 38.75674386038564]
[-9.155495166778564, 38.75901950184664]
]
]
]
}
}
}
) {
parcel {
id
name
createdBy
geom {
geojson
}
}
}
}
query {
parcelsList {
id
name
}
}
Note that we must use "MultiPolygon" because our datatype is "MultiPolygon".
To discuss: Is there a way to insert both "MultiPolygon" and "Polygon" GeoJSON?
In order to update we must provide a patch as an input. Lets change the name of our previous parcel to Garden in Lisbon
.
mutation {
updateParcel(input: { id: 10, patch: { name: "Garden in Lisbon" } }) {
parcel {
id
name
geom {
geojson
}
}
}
}
query {
parcelsList {
id
name
}
}
To delete the procedure is very similar. In this case we only need to provide as an input the ID of the parcel we like to delete.
mutation {
deleteParcel(input: { id: 10 }) {
parcel {
id
name
}
}
}
query {
parcelsList {
id
name
}
}