CommonJS
cogwirrel opened this issue · comments
Hey, it would be really awesome if we could have a commonjs version published to npm too, to avoid having to worry about battling cjs/esm issues :)
Hey, when I first published this package I tried to make it work with all module systems. Unfortunately I wasn't successful with that. I could of course release a separate package for CommonJS as you suggested, but I don't like the maintenance burden of that. So, I'm sorry but I won't be doing this.
If you need this in a project that uses CommonJS imports, I suggest that you copypaste this library as a single file into your project and modify the file to have the appropriate export definitions.
All good! I'll go for that approach for now :)
@cogwirrel This also worked for me:
const llamaTokenizer = await import('llama-tokenizer-js');
And thanks @belladoreai !