This repo includes artifacts related to the Alexa Teacher Model initiative. Please check back for updates!
AlexaTM 20B is a 20B-Parameter sequence-to-sequence transformer model created by the Alexa Teacher Model (AlexaTM) team at Amazon. The model was trained on a mixture of Common Crawl (mC4) and Wikipedia data across 12 languages using denoising and Causal Language Modeling (CLM) tasks.
AlexaTM 20B can be used for in-context learning. "In-context learning," also known as "prompting," refers to a method for using NLP models in which no fine tuning is required per task. Training examples are provided to the model only as part of the prompt given as inference input, a paradigm known as "few-shot in-context learning." In some cases, the model can perform well without any training data at all, a paradigm known as "zero-shot in-context learning."
To learn more about the model, please read the Amazon Science blog post and the paper.
The model is currently available for noncommercial use via SageMaker JumpStart, as described in our AWS blog post. The model can be accessed using the following steps:
- Create an AWS account if needed.
- In your AWS account, search for
SageMakerin the search bar and click on it. - Once in the SageMaker experience, create a domain and a studio user if none yet exist. All of the default settings can be used.
- In the control panel, click
Launch appnext to the user you wish to use. Launch a studio instance. - Once in the studio, there will be a launcher showing JumpStart as one of the tiles. Click
Go to SageMaker Jumpstart. Alternatively, JumpStart can be accessed by 3-pointed orange symbol on the far left of the studio. - Once in JumpStart, click the
Notebooksbutton. - Browse or search for our example notebook entitled
In-context learning with AlexaTM 20B. - There will be a button at the top to copy the read-only version into your studio.
- Ensure that your kernel has started, and run the notebook.
Note: You can also find our example notebook here
See CONTRIBUTING for more information.
This project (the code) is licensed under the Apache-2.0 License. The AlexaTM 20B model weights are licensed under the Alexa Teacher Model License Agreement.