TayPark / backend-1

A template repository for TypeScript backend server

Backend

1. Outline

1.1. Introduction

A template repository for Typescript Backend Server using those libraries:

• encrypted-nestjs: Encrypt NestJS for security
• nestia: Automatic SDK generator for NestJS
• safe-typeorm: Support the TypeORM to be much safer

1.2. Built-in Bullet System

This template repository has implemented a simple BBS (Built-in Bullet System) as an example project. This BBS is a typical subject for students who want to be a backend server developer. Therefore, I think this repository and the subject BBS would be the good reference for you.

Reading this README.md document, travel the source code. Also, for someone who wants to learn from the much complex example than this @samchon/backend, I've prepared much detailed backend projects like below:

• samchon/academic-backend: preparin...
• Send me PR (Pull Request) if you've implemented a good example

1.3. Directories

This template project has categorized directories like below:

• .github/workflows/build.yml: Configuration file of the Github Action
• .vscode/settings.json: Configuration for debugging
• designs/: ERD (Entity Relationship Diagram) drawed by Visio
• src/: TypeScript Source directory
  • src/api/: SDK that would be published to the @samchon/bbs-api
    • src/api/functional/: API functions generated by the nestia
    • src/api/structures/: Data structures for the API functions
  • src/controllers/: Controller classes of the Main Program
  • src/executable/: Executable programs
    • backend server itself
    • update program for the user
    • updator program in the server side
  • src/models/: ORM Models
  • src/providers/: Conversion between ORM Model and API Structure
  • src/test/: Test Automation Program
• package.json: NPM configuration
• tsconfig.api.json: TypeScript configuration for the SDK
• tsconfig.json: TypeScript configuration for the Main Program
• INFRASTRUCTURE.md: How to deploy the backend server on the cloud like AWS

2. Installation

2.1. NodeJS

This backend server has implemented through TypeScript and it runs on the NodeJS. Therefore, to mount this backend server on your local machine, you've to install the NodeJS.

• https://nodejs.org/en/

Also as you can see from the package.json file, this project requires the private npm module @samchon, provided from the Github. Therefore, to develop this backend server, you've configure the .npmrc file. Open the below link and complete the configuration.

• https://github.com/features/packages

2.2. MariaDB

This backend server has adopted MariaDB as principle DB. Therefore, to mount this backend server on your local machine, you've to install the MariaDB. Click the below link, download and install the MariaDB@10.5.

• https://downloads.mariadb.org/mariadb/10.5/

After the installation, open a terminal and configure account and password to be root.

sudo mysqladmin -u root password 'root'

After the account configuration, connect to the MariaDB and create a new DB schema bbs_test. Also, you should change the sql_mode options like below to enhance the strict grammer check for the safe development.

For a representatively, the ONLY_FULL_GROUP_BY condition occurs a syntax error when non-aggregated field has been appeared in the SELECT statement who is using the GROUP BY statement.

-- CREATE A NEW SCHEMA
CREATE SCHEMA bbs_test DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

-- STRICT MODE
SET GLOBAL sql_mode = 'ANSI,TRADITIONAL';

2.3. Repository

From now on, you can start the backend server development, right now.

Just download this project through the git clone command and install dependencies by the npm install command. After those preparations, you can start the development by typing the npm run dev command.

# CLONE REPOSITORY
git clone https://github.com/samchon/backend
cd backend

# INSTALL DEPENDENCIES
npm install

# START DEVELOPMENT (tsc --watch)
npm run dev

2.4. Data

When those installations are all completed, you can mount the basic data up or start the local backend server by typing below commands.

At first, npm run setup is a command seed the initial data. Range of the initial data means that minimum data that is required for running the local backend server. Therefore, to mount the backend server up, you've to run the npm run setup command, at least.

At second, npm run test is a command running the test automation program. The test automation program not only seeds the initial data, but also generates sample data during the testing. Also, you've to know that, whenever run the npm run test command, the local DB would be reset. Therefore, you've consider it carefully, whenever calling the npm run test command.

# Seed initial data
# minimum data to running the local backend server
npm run setup 

# Run test automation program
# seed not only initial data, but also sample data
# it resets the local DB
npm run test 

# Start the local backend server
npm run start local 

# Stop the local backend server
npm run stop 

3. Development

3.1. Definition

If you want to add a new feature or update ordinary thing in the API level, you should write the code down to the matched API controller, who is stored in the src/controllers directory as the Main Program.

However, @samchon does not recommend to writing code down into the Main Program first, without any consideration. Instead, @samchon recommends to declare the definition first and implement the Main Program later.

Therefore, if you want to add a new feature in the API level, define the matched data entity in the src/models and src/api/structures directories. After the data entity definition, declare function header in the matched API controller class in the src/controllers. Note that, it's only the declaration, header only, not meaning to implement the function body.

After those declarations, build the SDK through the npm run build:api command and implement the Test Automation Program using the SDK with use case scenarios. Development of the Main Program should be started after those preparations are all being ready. Of course, the Main Program can be verified with the pre-developed Test Automation Program in everytime.

• Declare data entity
• Declare API function header
• Build SDK
• Implement the Test Automation Program
• Develop the Main Program
• Validate the Main Program through the Test Automation Program
• Deploy to the Dev and Real servers.

3.2. SDK

@samchon/backend provides SDK (Software Development Kit) for convenience.

For the client developers who are connecting to this backend server, @samchon/backend provides not API documents like the Swagger, but provides the API interaction library, one of the typical SDK (Software Development Kit) for the convenience.

With the SDK, client developers never need to re-define the duplicated API interfaces. Just utilize the provided interfaces and asynchronous functions defined in the SDK. It would be much convenient than any other Rest API solutions.

To build the SDK, just type the npm run build:api command. The SDK would be generated by nestia, by analyzing source code of the controller classes in the compilation level, automatically. After the SDK building, you can publish the SDK through the npm publish command.

# BUILD SDK AND PUBLISH IT
npm run build:api
npm run package:api

When the SDK has been published, client programmers can interact with this backend server very easily. Just let them to install the SDK and call the SDK functions with the await symbol like below.

import api from "@samchon/bbs-api";

import { IBbsCitizen } from "@samchon/bbs-api/lib/structures/bbs/actors/IBbsCitizen";
import { IBbsQuestionArticle } from "@samchon/bbs-api/lib/structures/bbs/articles/IBbsQuestionArticle";
import { IBbsSection } from "@samchon/bbs-api/lib/api/structures/bbs/systematic/IBbsSection";

async function main(): Promise<void>
{
    //----
    // PREPARATIONS
    //----
    // CONNECTION INFO
    const connection: api.IConnection = {
        host: "http://127.0.0.1:37001",
        password: {
            key: "pJXhbHlYfzkC1CBK8R67faaBgJWB9Myu",
            iv: "IXJBt4MflFxvxKkn"
        }
    };

    // ISSUE A CUSTOMER ACCOUNT
    const customer: IBbsCustomer = await api.functional.bbs.customers.authenticate.issue
    (
        connection,
        {
            href: window.location.href,
            referrer: window.document.referrer
        }
    );

    // ACTIVATE THE CUSTOMER
    customer.citizen = await api.functional.bbs.customers.authenticate.activate
    (
        connection,
        {
            name: "Jeongho Nam",
            mobile: "821036270016"
        }
    );

    //----
    // WRITE A QUESTION ARTICLE
    //----
    // FIND TARGET SECTION
    const sectionList: IBbsSection[] = await api.functional.bbs.customers.systematic.sections.index
    (
        connection
    );
    const section: IBbsSection = sectionList.find(section => section.type === "qna")!;

    // PREPARE INPUT DATA
    const input: IBbsQuestionArticle.IStore = {
        title: "Some Question Title",
        body: "Some Question Body Content...",
        files: []
    };

    // DO WRITE
    const question: IBbsQuestionArticle = await api.functional.bbs.customers.articles.qna.store
    (
        connection, 
        section.code,
        input
    );
    console.log(question);
}

3.3. Test Automation Program

TDD (Test Driven Development)

After the Definition and SDK generation, you've to design the use-case scenario and implement a test automation program who represents the use-case scenario and guarantees the Main Program.

To add a new test function in the Test Automation Program, create a new ts file under the src/test/features directory following the below category and implement the test scenario function with representative function name and export symbol. I think many all of the ordinary files wrote in the src/test/features directory would be good sample for you. Therefore, I will not describe how the make the test function detaily.

• src/test/features/api
  • About the SDK that would be provided to the frontend developers,
  • Validate the matched API implemented in the Main Program
  • Use all of the API functions, through lots of scenarios
  • Most of the test functions are belonged to this category
• src/test/features/models
  • About the ORM Model classes
  • Validate tables, methods, and even materialized views
  • through lots of scenarios
• src/test/features/external
  • Open virtual external systems
  • Validate interactions with this backend server

Anyway, you've to remind that, the Test Automation Program resets the DB schema whenever being run. Therefore, you've to be careful if import data has been stored in the local (or dev) DB server. To avoid the resetting the DB, configure the skipReset option like below.

Also, the Test Automation Program runs all of the test functions placed into the src/test/features directory. However, those full testing may consume too much time. Therefore, if you want to reduce the testing time by specializing some test functions, use the include option like below.

• supported options
  • mode: mode of the target server
    • local
    • dev
    • real
  • include: test only restricted functions who is containing the special keyword.
  • exclude: exclude some functions who is containing the special keyword.
  • skipReset: do not reset the DB
  • count: repeating count of the test automation program.

# test in the dev server
npm run test -- --mode=dev

# test without db reset
npm run test -- --skipReset

# test only restricted functions whose name contain the "bbs" keyword
# do not reset db
npm run test -- --include=bbs --skipReset

3.4. Main Program

After Definition, SDK building and Test Automation Program are all prepared, finally you can develop the Main Program. Also, when you complete the Main Program implementation, it would better to validate the implementation through the pre-built SDK and Test Automation Program.

However, do not commit a mistake that writing source codes only in the controller classes. The API Controller must have a role that only intermediation. The main source code should be write down separately following the directory categorizing. For example, source code about DB I/O should be written into the src/providers directory.

4. Deploy

4.1. Non-distruptive Update System

If you committed a new version and pushed it into the repository, you can update the backend server without any distruption. By the npm run update command, you can let backend server to act those non-distruptive update processes.

• Pull new commit
• Build the new soure code
• Restart the backend server without distruption

To accomplish the non-distruptive update system, the server instance must run the updator program before mounting the backend server program up. If the target backend system is composed with multiple server instances like ELB (Elastic Load Balancer) and target instance is not a type of the master instance, you should write the npm run start:updator:slave command.

Otherwise, use the npm run start:updator:master command.

#----
# RUN UPDATOR PROGRAM
#----
# THE INSTANCE IS MASTER
npm run start:updator:master

# THE INSTANCE IS SLAVE
npm run start:updator:slave

#----
# MOUNT THE BACKEND SERVER UP
#----
npm run start real

4.2. Local Server

Sometimes, you may desire to mount the backend server on your local system, not for running the Test Automation Program, but for other reason like the Front-end Application Development.

In that case, you can mount the local backend server up and let it to keep the opening status, until the npm run stop command be executed, by typing the npm run start local command.

npm run start local
npm run stop

Also, if someone else committed a new version into the master branch, you can update your local backend server without distruption. It means that, non-distruptive update system even works in the local environment. To activate the non-distruptive update system, run the updator program before mounting the backend server up on your local machine.

# START THE LOCAL BACKEND SERVER WITH UPDATOR PROGRAM
npm run start updator:master
npm run start local

# UPDATE THE LOCAL SERVER WITHOUT DISTRUPTION
npm run update local

4.3. Dev Server

To update the dev server is very easy. Just commit a new code into the dev branch, and type the npm run update dev command on your local machine. By the command, the dev server will replace its code to the latest and the Non-distruptive Update System would be executed.

npm run update dev

Also, the dev server is designed to test and validate the newly commited source code before releasing to the Real Server. Therefore, even dev server may required to reset its DB like the Local Server who oftens run the Test Automation Program.

# MOVE TO THE PROJECT DIRECTORY
ssh ${dev_address}
cd ${project_directory}

# DO RESET
npm run reset:dev

# REFERENCE - COMMAND SET COMPOSING THE RESET:DEV
git pull
npm install
npm run build
npx pm2 delete all
npm run test -- --mode=dev
npm run start:updator:master
npm run start dev

4.4. Real Server

To update the dev server is very easy. Just commit a new code into the dev branch, and type the npm run update real command on your local machine. By the command, the dev server will replace its code to the latest and the Non-distruptive Update System would be executed.

npm run update real

5. Appendix

5.1. NPM Dev Dependencies

Running the npm publish command in this project means that, publishing the SDK.

Therefore, if you want to install any npm module required for the backend server, you've to install the module using the --save-dev option. Otherwise, there's an npm module that is required for the SDK, use the --save-dev tag.

# REQUIRED FOR BACKEND SERVER
npm install --save-dev nestia
npm install --save-dev nestia-helper
npm install --save-dev safe-typeorm

# REQUIRED FOR SDK
npm install --save node-fetch

5.2. NPM Run Commands

List of the run commands defined in the package.json are like below:

• build: Compile the source code
• dev: Incremental compilation using the --watch option.
• reset:dev: Restart the dev backend server with DB reset.
• sdk: Build SDK
• start: Start the backend server
  • npm run start local
  • npm run start dev
  • npm run start real
• start:updator:master: Start non-distruptive update system (master)
• start:updator:slave: Start non-distruptive update system (slave)
• start:reload: Restart the backend server
• stop: Stop the backend server
• stop:updator:master: Stop non-distruptive update system (master)
• stop:updator:salve: Stop non-distruptive update system (slave)
• update: Start the non-distruptive update
  • npm run update dev
  • npm run update real
• test: Start the Test Automation Program
• test:update: Test the non-distruptive update system

5.3. Github Action

This backend project utilizes the Github Action to run the cloud CI (Continuous Integration) test whenever a commit or PR event occurs. The CI test starts from installing the backend server program to a clean Ubuntu system that nothing has been installed yet.

You know what? All of CI processes, like installing required programs into the clean Ubuntu and compiling and running the Test Automation Program, are defined in the .github/workflows/build.yml script file. Also passing or failure expressed from the above badge represents outcome of the CI test.

5.4. Related Repositories

Write the related repositories down.