Make mt3-docker compatible with (google colab local)

[Lucas Rodrigues]

Hello how are you?

I've managed to install everything right, but I don't know how to make the inference via POST, would it be possible for you to add the option to connect this docker to google colab? via jupyter notebook, while I can't use this docker, I'm using mt3-pytorch, but it's bad to have to do the commands to move files manually, so it would be interesting to make this docker compatible with google colab locally, if possible create this integration we appreciate it even more, thank you.

Suggestion link: https://research.google.com/colaboratory/local-runtimes.html

Another thing would be interesting to create a simple python cli ex: "python inference.py -i input.mp3 -o output.midi" would be very interesting :)

[Joseph Weidinger]

Sorry, I didn't see this until now! I've never done anything like that with colab -- not sure if it's really possible or was designed for that though... I just think of it as a sandbox in the cloud. But I don't really spend a lot of time on it.

Assuming you're running the docker container on your machine:

sudo docker run -p 5000:5000 --gpus all mt3

Try this out... does it run? Do you get "Hello Whirld"?

curl --request GET \
  --url http://127.0.0.1:5000/ \
  --header 'Content-Type: application/json'

That's just a health check basically.

Assuming the container is running, something like this will probably work. This is just random code I had somewhere that uses mt3-docker for other things...

# `requests` is https://pypi.org/project/requests/
# `mido` is https://github.com/mido/mido
# NOTE: samples is 16k
def transcribe(self, samples: np.ndarray) -> mido.MidiFile:
    base_url = "http://127.0.0.1:5000" # if it's running on the same computer as this script
    # base_url = "http://192.168.1.5:5000" # if it's on another computer on your LAN

    json = {"data": [float(s) for s in samples]}
    headers = {"Content-Type": "application/json"}
    res = requests.post(f"{base_url}/transcribe-anything-raw", json=json, headers=headers)
    return mido.MidiFile(file=io.BytesIO(base64.b64decode(res.json()["data"])))

You could write a cli.py that does something similar to the above but just reads/writes files

EDIT:
Uhh I just realized that the endpoint I made there was for something not in this repo. Basically you'd have this to your app.py (in this repo).

@app.route("/transcribe-anything-raw", methods=["POST"])
def transcribe_anything_raw():
    return {"data": get_transcription_b64(piano_model, request.get_json()["data"])}

NOTE: that app.py code is mostly bad code. It's just a quick proof of concept to show that it works.