a demo dashboard that pulls data from a JSON api endpoint
in this instance the data was found in json pulled by pages for victoria park (json) and blackwall (json)
the api is undocumented, but relatively intuitive; a truncated sample is shown below, with the obs
and forecast
sections removed for brevity:
{
"dt": "610.568µs",
"rxs": {
"obs": [
{
"msg": {
"aqi": 25,
"idx": 14598,
"attributions": [
{
"url": "http://uk-air.defra.gov.uk/",
"name": "UK-AIR, air quality information resource - Defra, UK",
"logo": "UK-Department-for-environment-food-and-rural-affairs.png"
},
{
"url": "https://londonair.org.uk/",
"name": "London Air Quality Network - Environmental Research Group, King's College London",
"logo": "UK-London-Kings-College.png"
},
{
"url": "https://waqi.info/",
"name": "World Air Quality Index Project"
}
],
"city": {
"geo": [
51.5405194915241,
-0.033307388878
],
"name": "Tower Hamlets - Victoria Park, United Kingdom",
"url": "https://aqicn.org/city/united-kingdom/tower-hamlets-victoria-park"
},
"dominentpol": "pm25",
"iaqi": {
"h": {
"v": 60,
"t": "2023-09-15 13:00:00"
},
"no2": {
"v": 14.7,
"t": "2023-09-15 11:00:00"
},
"p": {
"v": 1014.1,
"t": "2023-09-15 13:00:00"
},
"pm10": {
"v": 12,
"t": "2023-09-15 11:00:00"
},
"pm25": {
"v": 25,
"t": "2023-09-15 11:00:00"
},
"t": {
"v": 22.7,
"t": "2023-09-15 13:00:00"
},
"w": {
"v": 2,
"t": "2023-09-15 13:00:00"
},
"wg": {
"v": 5,
"t": "2023-09-15 13:00:00"
}
},
"time": {
"s": "2023-09-15 11:00:00",
"tz": "+01:00",
"v": 1694775600,
"iso": "2023-09-15T11:00:00+01:00"
},
"obs": {...},
"forecast": {...},
"xsync": {
"gen": 1694779365
}
},
"status": "ok",
"cached": "2m39.770497357s"
}
],
"status": "ok",
"ver": "1"
}
}
adding these as data sources in pbi (Aug 2023 version) was quite easy, it automated most of the flattening process, turning the json blob into a wide table comprised of a single row, mainly making use of the Table.ExpandRecordColumn
function to break out json object properties into named columns.
the out-of-the-box powerquery was only modified in one place, to split a json array into named longitude and latitude columns via transform, split, and rename calls:
#"Extracted Values" = Table.TransformColumns(#"Expanded rxs.obs.msg.city", {"rxs.obs.msg.city.geo", each Text.Combine(List.Transform(_, Text.From), ","), type text}),
#"Split Column by Delimiter" = Table.SplitColumn(#"Extracted Values", "rxs.obs.msg.city.geo", Splitter.SplitTextByDelimiter(",", QuoteStyle.Csv), {"rxs.obs.msg.city.geo.1", "rxs.obs.msg.city.geo.2"}),
#"Renamed Columns" = Table.RenameColumns(#"Split Column by Delimiter",{{"rxs.obs.msg.city.geo.1", "latitude"}, {"rxs.obs.msg.city.geo.2", "longitude"}}),
the resulting datasets were then piped into "multi row card" and map visuals in the board itself.
this project uses pbi.tools to bring reliable versioning and documentation support.
a few of the more useful commands are explained below:
pbi-tools extract pbi-json-api.pbix
Extract the contents of a PBIX/PBIT file into a folder structure suitable for source control.
...you can then work on powerquery directly in a code editor, useful for managing complex products where you need to find/replace across multiple pages.
pbi-tools compile pbi-json-api -overwrite -format PBIT
re-compile a PBIT file from modified sources in the specified folder
...having fixed up your powerqueries, you can recompile these back into a PBIT, which in turn you can open in pbi, ready to save as a PBIX report...
pbi-tools launch-pbi pbi-json-api.pbit
open a pbit in pbi
...this file can then be saved as pbix format and published!