aidarahim / Dolly

Databricks' Dolly

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Dolly

This fine-tunes the GPT-J 6B model on the Alpaca dataset using a Databricks notebook. Please note that while GPT-J 6B is Apache 2.0 licensed, the Alpaca dataset is licensed under Creative Commons NonCommercial (CC BY-NC 4.0).

To use:

  • Load all the files into a Databricks repo.
  • Start a 12.2 LTS ML (includes Apache Spark 3.3.2, GPU, Scala 2.12) single-node cluster with node type having 8 A100 GPUs (e.g. Standard_ND96asr_v4 or p4d.24xlarge).
  • Open the train_dolly notebook and run it.

About

Databricks' Dolly

License:Apache License 2.0


Languages

Language:Python 100.0%