frost-beta / train-model-with-js

Train text generation model with JavaScript.

Train text generation model with JavaScript

This repo hosts some educational scripts for doing basic training on a decoder-only transformer, using node-mlx with Node.js.

Files:

• model.js - defines the model.
• input.txt - text file used for training the model.
• train.js - script for traning.
• generate.js - script for generating text using the trained model.

Platform

Only Macs with Apple Silicon are supported.

How to use

Download dependencies and run the training script, which generates tokenizer.json and weights.safetensors:

npm install
node train.js

Then use the generate script to actually generate some text from the weights:

node generate.js

What's next

After understanding the basics of model training, you can check the train-llama3-js repo on how to train a Llama3 model with large datasets.

License

Public domain.