Hyper-Data Markup Language
HDML is a language for data virtualization. HTML has allowed us to compose texts and images freely and present them through web browser. So, why not for data?
Data for Everyone
This is a project for augmenting, subsetting, and aggregating data freely from multiple data sources such as MS Excel, DB, Parquet on Hadoop Distributed File System (HDFS) or AWS S3, and other binary formats such as netCDF. The development goal is to make HDML as the DNA of all data - simple, elegant, and universal.
HTML5 provides data tag. HDML expands it with src attribute.
Here are a few example data sources:
- src=”hdf4://AIRS.hdf/path/to/dset[0:2:4, 0:3:6]” You can replace hdf4 to hdf5/netcdf/geotiff.
- src=”excel://myexcel.xls/sheet1/A3:C10”
- src=”mysql://host:3306/user:password/SQLstatement”
- src=”hdfs://node:8020/user/hadoop/…”
- src=”s3://landsat-pds/L8/003/017/LC…” for Amazon S3
- src=”opendap://acdisc.sci.gsfc.nasa.gov/opendap/ncml/Aqua_AIRS_Level3/AIRH3ST8.005/2002/AIRS.2002.09.01.L3.RetStd_H008.v5.0.14.0.G07183213133.hdf.ncml.ascii?TotalCounts_A[0][3]”
- src="random" (Fill with random values)
- src="_FillValue" (Fill data with fill value)
- src="increasing" or "decreasing" (Fill data with monotonically increasing values that datatype allows. Repeat when overflow occurs)
- src="ipfs://..."?