WASM file is not included in Next.js Node runtime API routes

[Aaron Friel]

Hey @dqbd, first of all thank you for this great library. I've tried to do as much due diligence here to make the issue reproducible, and I'm filing a corresponding issue on Next.js in case the issue lies on their side.

Issue

When using a Next.js route with the Node runtime and standalone output, using the @dqbd/tiktoken library throws "Error: Missing tiktoken_bg.wasm".

I'm not sure if is due to module.exports and the imported file not correctly tracing and importing the WASM bundle, or due to Next.js' standalone output mode incorrectly excluding this file. That issue is here: vercel/next.js#54395

Steps to reproduce

I've created a repository started from the with docker turborepo example, and applied a commit testing @dqbd/tiktoken in a server side component, a client side component (inside useeffect()), and an API route. You can find the repro-repo here:

https://github.com/aaronfriel/next-wasm-standalone-bug-report