Enables TypeORM to be utilized in a distributed database environment.
Install
NPM
$ npm install typeorm-sharding-repository --save
Yarn
$ yarn add typeorm-sharding-repository
Usage
1. Prepare DataSources.
Instead of setting the TypeORM datasource, set the ShardingManager. Most of the settings are similar to TypeORM.
In this example, the User entity has a key of number type. It is recommended to set the ShardingManager according to the rules of a compatible key or range of keys.
constdbConf={host: 'localhost',database: 'playground',username: 'postgres',password: 'postgrespw',synchronize: true,logging: 'all',};constshardManager=awaitShardingManager.init({shardingType: ShardingType.RANGE,// The settings here apply to all database shards. type: 'postgres',entities: [User],shards: [// Settings for each database.// minKey and maxKey can also use various types. For multi-columns, a tuple format is also possible.// onInitialize(typeorm.DataSource) callback is optional.{ ...dbConf,port: 55000,minKey: 0,maxKey: 1000,onInitialize: (dataSource)=>{/* What to do after data source initialization */}},{ ...dbConf,port: 55001,minKey: 1000,maxKey: 2000,onInitialize: (dataSource)=>{/**/}},{ ...dbConf,port: 55002,minKey: 2000,maxKey: 3000,onInitialize: (dataSource)=>{/**/}},],});
2. Apply to Entity
Use @ShardingEntity instead of @Entity decorator.
Inherit ShardingBaseEntity instead of BaseEntity.
New data will be inserted into the added database shard.
@ShardingEntity<User,number>({// Modular sharding will be supported in the future.type: ShardingType.RANGE,// Decide in which shard the entity will be processed. To process one entity, it can be called as many as the number of shards.// If data is being inserted, the key may be empty.// If false is returned in all cases, the last shard is chosen.// For minKey and maxKey, the values defined in ShardingManager are delivered.// If you do not want to manage multiple ShardingManagers, you can adjust the minKey and maxKey keys according to the 1:n relationship of entities in this function.findShard: (entity: User,minKey,maxKey)=>entity.id&&minKey<=entity.id&&entity.id<maxKey,// Similar to findShard, but an ID value is passed instead of an entity.// The type of ID is ambiguous. In this example, passed as number.// Usually, ID values referenced in repository.findByIds() etc. are passed.// You need to determine which shard the id resides on.findShardById: (id: number,minKey,maxKey)=>id&&minKey<=id&&id<maxKey,})exportclassUserextendsShardingBaseEntity{
@PrimaryGeneratedColumn()id: number;
@Column()firstName: string;
@Column()lastName: string;
@Column()age: number;}
3. RepositoryService (Abstract repository for TypeORM.BaseEntity and ShardingBaseEntity)
RepositoryService compatible with TypeORM.BaseEntity and ShardingBaseEntity.
With this pattern, changing whether or not sharding is applied does not change the code.
// Provides almost the same functionality as BaseEntity.constentity=awaitCase1.save({firstName: 'Typeorm',lastName: 'Sharding',age: 10,});awaitentity.save();awaitentity.remove();awaitentity.softRemove();awaitentity.recover();awaitentity.reload();// _______.___________. ___ .___________. __ ______ // / | | / \ | || | / |// | (----`---| |----` / ^ \ `---| |----`| | | ,----'// \ \ | | / /_\ \ | | | | | | // .----) | | | / _____ \ | | | | | `----.// |_______/ |__| /__/ \__\ |__| |__| \______|// Create entity instanceCase1.create();Case1.create({firstName: 'Typeorm', ...});// Returns all entities from all shards. Sort order is not guaranteed.awaitCase1.find();// Returns all entities where the value of "firstName" column is 'Typeorm' from all shards. Sort order is not guaranteed.awaitCase1.find({where: {firstName: 'Typeorm'}});// Fetch one entity where the value of "firstName" column is 'Typeorm' from all shards.// If an entity is found on multiple shards, the entity from the oldest shard is returned.awaitCase1.findOneBy({firstName: 'Typeorm'});// Fetch one entity by ID from single shard.awaitCase1.findOneById(1);// Count entities where the value of "firstName" column is 'Typeorm' from all shards.awaitCase1.count({where: {firstName: 'Typeorm'}});// Update multiple entities.awaitCase1.update({firstName: 'Typeorm'},{lastName: 'Bob'});// Remove entity from single shard.awaitCase1.remove(entity);// __ ______ ____ __ ____ __ ___________ ____ _______ __ // | | / __ \ \ \ / \ / / | | | ____\ \ / / | ____|| | // | | | | | | \ \/ \/ / | | | |__ \ \/ / | |__ | | // | | | | | | \ / | | | __| \ / | __| | | // | `----.| `--' | \ /\ / | `----.| |____ \ / | |____ | `----.// |_______| \______/ \__/ \__/ |_______||_______| \__/ |_______||_______|// Returns all TypeORM DataSourcesCase1.getAllDataSource();// Returns TypeORM DataSource for specific entityCase1.getDataSource(entity);// Returns TypeORM DataSource for specific ID valueCase1.getDataSourceById(ID_VALUE);// Returns all TypeORM ReposigoriesCase1.getAllRepository<Case1>();// Returns TypeORM Reposigory for specific entityCase1.getRepository<Case1>(entity);// Returns TypeORM Manager for specific entityCase1.getManager<Case1>(entity);
Contributors
About
TypeORM Sharding Repository: Enables TypeORM to be utilized in a distributed database environment.