Synthesizer API 🎹

🔥Synthesizer API Explorer!🔥
🔥Synthesizer Directory🔥 🔥Synthesizer Directory repo🔥

For all you gearslutz out there! This free API is the only one out there (since I checked). Explore over 800 synthesizers from past and present. Get insight in their manufacturers and specifications. The API builds upon the data available on the vintagesynthexplorer website. Future work will be aimed to broaden querying options and let you fellow synth lovers contribute to the collection.

For Usage:

Usage ⚡
Future Work 🔥

About this Project

The goal of this project is to build an public REST API 🎯 and thereby applying existing knowledge and explore new terrains. This was done independently by reading documentation, watching online videos, consulting StackExchange & sometime my coach.

Although this project aimed at making a public API, I obviously tried it out myself! Therefore I made a frontend that uses it. In development some features where added to the backend as well. Such as a route to suggest synthesizer and an admin login system.

Be sure to visit this deployed version as well: 🔥Synthesizer Directory🔥

This repo contains three main folders:

SCRAPER

The scraper folder contains DOM oriented programming. Before I could fill the database with synthesizers, manufacturers & specification I needed to scrape data. My source also had images, therefore I needed to download images from their respective URL's. After scraping the data it needed to be normalized to fit the SQL database structure. For this, some JS object/string manipulation was used.

SERVER

This folder is the heart of the project. It contains the server (with database). You will find endpoints (& queries) making use of validation middlewares (& Yup). People are able to suggest synths that can get approved through an admin login-system. Look around in authorization (& authorization in model) done through Express middlewares with JWT tokens. The development was mostly fuelled by end-to-end testing. Another thing that needed to be done was migrating images to Cloudinary db and save the image URL's to the main SQL database.

CLIENT

Here you can find the Synthesizer API explorer repo, done with ReactApp. Through the explorer an apikey can be sent to your email. It utilizes frontend validation(with React Hook Form) and some storage concepts(useContext) for the apikey. :fire:Go to API explorer!🔥

Leading topics:

ENVIRONMENTS

Throughout the project we have used: development, testing and production environments. Switching in/out sources and storing key's and secret's in .env and hosts. Wrapping my head around it was challenging at first. Now I feel comfortable to apply this in real world situations.

END-TO-END TESTING

All end-point have been thoroughly tested. The testing formed the basis of development Checking if correct responses were sent & testing all possible 'un-happy' paths was an endevour. But it pays out by making me feel safe to refactor code. Jest and Supertest were used in de backend, Jest & Playwright (like Puppeteer) in the frontend.

VALIDATION & ERROR HANDLING

Validation was done with Yup, used in the backend as well as the frontend. Together with middlewares to safeguarded the routes from user errors and send meaningful feedback to the client when they do occur. Also validated in the backend: multipart/form-data & attachments.

AUTHENTIFICATION & PASSWORD HASHING

To be able to control the flow of user suggested synthesizers I implemented a admin login-system with JWT token. Although this was not my first encounter with this, it was the first time that I implemented it inside of my model and validated the expiration of the token. Hashing/comparing of the user's password was another thing that was done, also inside the model directly.

Things I've done in the backend:


DOM manipulation	downloading images ⭐	managing multiple dsb’s	normalizing data
generating tables	relating tables	mocking seed data ⭐	using environments ⭐
managing secrets	deploying	writing npm scripts	big O notation ⭐
end-to-end testing ⭐	pagination	validation	sending emails ⭐
writing middlewares	routing middlewares	continuous Integration ⭐	error handling
CORS	handling FormData	HTTP methods (all)	Netlify
Heroku	JSDOM	Yup ⭐	Express
Cloudinary (db) ⭐	Sendgrid ⭐	Jest	Supertest ⭐
Sequelize ORM	ElephantSQL	SQL	JWT
custom Promises		Express	Node.js

Things I've done in the frontend:


React DOM routing	Bootstrap ⭐	Sass	caching ⭐
copy to clipboard	custom Hooks	UseContext ⭐	local storage
React children ⭐	validation	error handling	fetching data

⭐ New technologies learned in this project

User Stories

As a Developer that uses this API I want to be able to fetch manufacturers and synthesizer with their specifications, so I can use the API where it is meant for
As a Developer that uses this API I want to be able to fetch images of the synthesizers, so I can make something visual I my frontend
As a Developer that uses this API I want to be able to know what mistakes I made when I did something wrong in fetching the data,so I can adjust my query and still find the data I'm looking for
As a Developer that uses this API I want to be able to paginate and limit the results I'm getting, so I can easily built this in my frontend
As a Developer that uses this API I want to be able to query a specific synthesizer either by id or by name, so I'm not restricted to using two endpoint that share one purpose
As a Developer that uses this API I want to get the results sorted by year that the synthesizers where produced, so I can easily implement this behaviour when building the frontend
As a Developer that uses this API Explorer I want to be able to request apikey through email, so I can explore this API
As a Developer that uses this API Explorer I want to be able to view the JSON data retrieve from the URL, so I will see the data that is available
As a Developer that uses this API Explorer I want to be able to easily create the URL with route, params and query, so I will find the data that is available
As a Developer that uses this API Explorer I want to get suggestions of the endpoint, params and queries I'm aloud to enter, so I know what routes, params and queries are available
As a Developer that uses this API Explorer I want to be able to store the apikey across sessions, so I don't have to bother checking my email to find it
As a Developer that uses this API Explorer I want to copy the explorer URL to my clipboard so, I can quickly use it outside of the Synthesizer API Explorer

the following stories are derived form the Synthesizer Deirectory frontend build upon this API at later stage

As a User of the Synthesizer Directory I want to be able to suggest new synthesizers, so I can contribute to the collection
As Myself I want to be able to judge the suggested synthesizer, so I can decide which one's will be added to the collection

Project Board

You can find it here.

Wireframes API Explorer

Checkout the wireframes made for the API Explorer.

Datamodel

Checkout the datamodel used as base for data structure.

Git Version Control

...

Usage

API key

For each request send, you have to send your API key with it. It can be obtained through registering your email at API explorer. The API key can be located anywhere inside the query string. The following example requests with and without additional queries are all valid:

.../synths?key=<apiKey>

.../synths?yearProduced=1980&key=<apiKey>

.../synths?key=<apiKey>&yearProduced=1980

Pagination

Calling any API endpoint without a resource ID or name will return a paginated list of available resources for that API. By default, a "page" will contain up to 20 resources. If you would like to change this just add a 'limit' query parameter to the GET request, e.g. ?=60. You can use 'offset' to move to the next page, e.g. ?limit=60&offset=60.

Name	Result	Type
offset	Get all synths/manufacturers with offset	integer
limit	Get all synths/manufacturers with limit	integer

Endpoint: Synthesizers

GET /synths

Get a list of synth resources including their image, manufacturers and ten specifications

NOTE:
Some data is not normalized yet. Some specifications will have a null value.

Example of json resources:

{
    "count": 815,
    "synths": [
        {
            "id": 309,
            "name": "Native Instruments Pro-53",
            "img": "https://res.cloudinary.com/dgte68hwi/image/upload/v1613044108/Native_Instruments_Pro-53.jpg",
            "ManufacturerId": 50,
            "SpecificationId": 309,
            "synthType": null,
            "interfaceFeature": null,
            "format": null,
            "Specification": {
                "id": 309,
                "polyphony": "32 voices per Instance",
                "keyboard": null,
                "control": null,
                "yearProduced": 2002,
                "memory": "512 Programs (8 programs per bank, 8 banks per file, 8 files). Unlimited Files (512 programs) can be created and stored on your computer for virtually unlimited memory.",
                "oscillators": "2 Oscillators. Osc. A has sawtooth and pulse waveforms. Osc. B has sawtooth, triangle and pulse waveforms.",
                "filter": "24 dB/oct low-pass / high-pass filter with resonance, self oscillation, ADSR envelope",
                "lfo": "1 with sawtooth, square and pulse waveforms, MIDI sync, envelope triggering. Assignable to oscillators, filter, etc.",
                "effects": "Delay Section creates Multi-Echo, Chorus, Flange effects and is MIDI-syncable"
            },
            "Manufacturer": {
                "id": 50,
                "manufacturer": "Native Instruments"
            }
        },

GET /synths/1

GET /synths/Jupiter-8

Get one synth resource, by either giving it's ID or exact name.

NOTE:
All specifications and the manufacturer are of synth resources are delivered in a request by default. Specifying them specifically in a query is not necessary. Some data is not normalized yet. This makes it only possible to query with exact name matches. Look into future work to see development goals for querying.

Name	Result	Type
manufacturer	Get all synths by a given manufacturer	string
yearProduced	Get all synths by a given year they hit the market	integer
polyphony	Get all synths with description of amount of voices	string
keyboard	Get all synths with description of keyboard specifics	string
control	Get all synths with description of given control (e.g. MIDI, CV/GATE)	string
memory	Get all synth with description of memory	string
oscillators	Get all synths with description of oscillators	string
filter	Get all synths with description of filter	string
lfo	Get all synths with description of lfo	string
effects	Get all synths with description of effects	string

Endpoint: Manufacturers

GET /manufacturers

Get a list of manufacturer resources.

GET /manufacturers/1

GET /manufacturers/Roland

Get one manufacturer resource, by either giving it's ID or exact name.

Future Work

This API is a work in process. Future work aims to improve querying, removing and/or adding queriable synth specifications and creating a moderated POST endpoint, to make you able to add to this collection.

PLANNED

Name	Result	Type	Options
manufacturer	Get all synths produced by a given manufacturer	string	name or id
yearProduced	Get all synths that hit the market by a given year	integer	range, asc/dec
polyphony	Get all synths that have a given amount of voices	integer	range, asc/dec
keyboard	Get all synths that have or do not have a keyboard	ENUM	none/yes
control	Get all synths that are controlled with MIDI and/or CV/GATE	ENUM	MIDI, CV/GATE
oscillators	Get all synths that have a given amount of oscillators	integer	range, asc/dec
oscillatorType	Get all synths that are either VCO or DCO	ENUM	VCO, DCO
filter	Get all synths that contain one or a combination of lowpass, highpass, bandpass	ENUM	lowpass, highpass, bandpass
effects	Get all synths that have or do not have built in effects	ENUM	none/yes
type	Get all synths that are either analog, digital or hybrid	ENUM	analog, digital, hybrid

Updates

Adding sort by year, ascending and descending
Adding, merging specification that will be queriable
Make querying possible without strictly needing to equal the resource name
Sorting sorting by number functionalities used in more than yearProduced alone
And there will be more...

BernardDev / Synthesizer-API