A digital representation of Sikh Bani and other Panthic texts with a public logbook of sangat-sourced corrections.
- Shabad OS Database
- Table of Contents
- Getting Started
- Schema & File Structure
- Build
- Releases
- Benchmarks
- Viewers
- Licenses
Install the database with npm install @shabados/database. An objection.js object is returned, allowing for flexible and relational querying. To fetch a Shabad with first letters:
// const { Lines } = require('./index') // If using this repo
const { Lines } = require('@shabados/database') // If using npm module
// Fetch the line, with information about the shabad
Lines
.query() // Start a query on the lines table
.firstLetters('ਹਹਹਗ') // Search for the first letters, ordering the results sensibly
.first() // Use the first line that is returned
.then(line => line.$relatedQuery('shabad').eager('[lines, writer]')) // Return the shabad the line is from, with the lines and writer
.then(shabad => console.log(shabad))To search in the ascii equivalent, the API will automatically convert the search to unicode:
Lines.query().firstLetters('kkggAnj')Although column names are in
snake_case, the API returnscamelCasevia objection.js.
The schema can be modified in the migrations/schema.js file.
JSON files for Raags, Sources, Writers, and Line_Types can be found in the data folder. Changing a value here will be reflected everywhere else. The (array index) + 1 represents the id used for each relation in other tables.
Lines of JSON files are split by page or other sensible method via ./data/source/number.json.
Bani compilations can be added to the bani.json. To define the lines it contains, each bani should contain a list of objects with start_line and end_line, referring to the files in sources.
It is possible to make small changes and build a database from the JSON files and in reverse to make batch changes and build the JSON files from the database. Additionally, some SQLite files can be imported.
You can build an SQLite3 database with the following methods:
Node.js - Install node. Install dependencies and build with npm install && npm run build-sqlite.
Instead of making bulk changes to JSON files as a pull request, we highly urge you to submit an issue of the change first instead for visibility and security.
It is best practice to build the database, make changes to database.sqlite, and regenerate JSON files with the following methods:
Node.js - npm run build-json
You can run a Shabad OS MariaDB using Docker:
docker run -p 3306:3306 shabados/database
The username is shabados, and the database name is shabados.
If you'd like to build the image yourself, run: docker build -f docker/mariadb/Dockerfile -t shabados/database . .
It's possible to import other sqlite files. Run npm run import -- --help to see all options.
The importer will generate placeholder Sources, Translation Sources, and fill in -1 for Shabad sections and Writer IDs. These must be corrected in build/database.sqlite. (Refer to Database and JSON above).
npm run import -- nandlal.sqlite nandlal -o ID -s ShabadID -2 ShabadID -S SourceID -t English -t Punjabi -p PageNo -l LineNo -g GurmukhiBy making a pull request of changes to the data folder, others can submit spelling mistakes, grammar errors, and translation improvements. Format for errors should follow the guideline for git commits:
Subject: What for/in Line XXXX
Description:
From where did you source the information (Include title, year, and other pertinent information for 3rd parties).
(Optional) Why was the information incorrect in our db (e.g. mistake inherited from 2014 iGurbani db or mistake found in 2008 Sttm2 db).
We also have the Shabad OS Database viewer, which can be used to view the database and autofill issues!
The data folder is the source of truth for any builds.
- Source folders -
.jsonfiles contain the id, gurmukhi, pronunciation, and translations of each line of parent folder's source .jsonfiles - these files range from compiling banis, separating sections and sub-sections, to naming languages, translations, and writers
The builds for any of branches can be found on CircleCI.
If #Major, #Minor, #Patch are found in the latest commit messages, database assets will be compiled and released per semantic versioning.
Compiled databases are available via the release page, or via npm install @shabados/database.
Run benchmarks with npm run benchmark. Benchmarks depend on specs, but the following shows that between 0-200 results can be returned in a reasonable amount of time.
| Query | Results | Time |
|---|---|---|
| ਹਹ | 2748 | ~80ms |
| ਹਹਹ | 226 | ~50ms |
| ਹਹਹਹ | 50 | ~50ms |
| ਹਹਹਹਹ | 13 | ~50ms |
You can use an application like DBeaver to view the SQLite file.
Gurbani was written during a time and place without formal copyright laws. Therefore we identify it as being free of known restrictions. Instead we consider the Gurbani under the data folder and generated under the build folder, including the gurmukhi JSON and SQLite fields, to be in the Public Domain as a work of factual compilation with originality. Therefore any derogatory treatments, such as adding to, deleting from, altering or adapting the work in such a way that it amounts to a distortion or mutilation of the work, or is otherwise prejudicial to the honor or reputation of the work, are not allowed. The typographical corrections the Shabad OS team has made to the Gurbani honors the originality of Gurbani to the best of our knowledge.
Supporting text under the data folder and generated under the build folder, are subject to their respective source copyrights, some by other authors. Translations, transliterations, notes, compilations, or other items which are not Gurbani and which are created or uniquely organized by the Shabad OS team are subject to the Creative Commons Attribution-ShareAlike 4.0 International License.
All code and content resting outside of the data and build folders is licensed under the GNU General Public License v3.