google / maxtext

A simple, performant and scalable Jax LLM!

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[request] bloom (alibi) model implementation

bzantium opened this issue · comments

I want to train bloom style model but hard to add.

Can you help me understand what you're looking for here? Are you looking for Alibi attention? Or Bloom model? Or both?

We generally help folks who add their models to MaxText get SOTA perf, but we don't typically write models from scratch for folks.