Giters

xenova / transformers.js

State-of-the-art Machine Learning for the web. Run 🤗 Transformers directly in your browser, with no need for a server!

Home Page:https://huggingface.co/docs/transformers.js

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Is it possible to run a music / sounds generation model?

flatsiedatsie opened this issue · comments

commented

Question

I'd love to create a browser-based music generation tool, or one that can turn text into sound effects. Is that supported?

I guess my more general question is: can Transformers.js run pretty much any .onnx I throw at it, or does each model require some level of implementation before it can be used?

commented

Well would you look at that plot thickening: https://huggingface.co/Xenova/musicgen-small

Except.. that the repo was created a day ago. Whoa.

commented

Yep! Somehow your feature request came at the perfect time! This is something I've been working on already for a few days. This is all now possible thanks to this PR by @fxmarty.

Stay tuned for updates! This might only be available in v3 due to WebGPU support (I think CPU/WASM-only will be too slow).

commented

Very very cool. Nice work @fxmarty. You wouldn't happen to have an early online test somewhere that I can tinker with?

commented

I did not implement it in the PR yet, I'll maybe do it with ONNX Runtime (patching some transformers methods to use ORT instead of torch)

commented

I have no idea what that means, except that I should be patient :-)