How to play sound ?

Question

How to play sound ?

HectorRicardo opened this issue 4 years ago · comments

Co you have an example where there is actual sound output from the synthesizer?

If I am not wrong, the code in example/main.c produces an output file of float32output.pcm . I am guessing this is the output raw data. How can I play this sound ?

I am on Windows 10, and I compiled and ran the example/main.c with Cygwin.

Thanks

Hector Ricardo · Answer 1 · Sun Aug 16 2020 06:35:36 GMT+0800 (China Standard Time)

My question on SO: https://stackoverflow.com/questions/63431072/how-to-use-the-fluidlite-synthesizer

Robin Lobel · Answer 2 · Sun Aug 16 2020 15:55:43 GMT+0800 (China Standard Time)

Playing sound to the speakers is beyond the scope of FluidLite, as described in the README:
"FluidLite keeps very minimal functionnalities (settings and synth), therefore MIDI file reading, realtime MIDI events and audio output must be implemented externally."

You can use the RtAudio library to output sound for instance: https://github.com/thestk/rtaudio

Hector Ricardo · Answer 3 · Mon Aug 17 2020 01:46:58 GMT+0800 (China Standard Time)

Hey @divideconcept, thanks for your quick response.

Yes, I know that audio output is beyond the scope of this project. In fact, I didn't want to open an issue because this is not an issue, but I didn't find anywhere else to post the question.

May I do a suggestion for your project?

I am looking to play midi notes in an Android app. I have looked for Midi synthesizers around the web and found these projects:

https://github.com/billthefarmer/mididriver - this one works great, there's a sample app ready to go, but there's no way to change the default soundbank
https://github.com/KyoSherlock/MidiDriver - the soundfount can be changed on this one, but there's some performance and lagging issues. It is totally written in java and not in C, so performance is affected.
https://github.com/FluidSynth/fluidsynth - I think this one works, but I did not succeed in how to make it compile and work in Android
https://github.com/VolcanoMobile/fluidsynth-android - This one should be easier to compile, but I am still stuck.

I think that your project might do the job: a lite version of FluidSynth with ability to change soundbanks and without performance issues. However, I don't have experience in building and compiling C application, so I am kind of stuck.

If you could provide a sample Android app that plays a simple midi note (like in BillTheFramer's repo), I think more people would gravitate towards your repo, since it will offer user friendliness, relief to the developer of having to know how to compile, build and link C code, and flexibility to change soundbanks.

Just my two cents, I really thank you for having this project!

Gabriel · Answer 4 · Mon Aug 17 2020 02:45:22 GMT+0800 (China Standard Time)

There are ports of fluidsynth to Android, you probably want one of them as they are more feature complete (they support MIDI playing and audio output - Both stuff that is out of scope in fluidlite)

Audio output is highly platform dependent and a complex topic so I can understand why there are zero examples about this. For using Fludlite I recommend that you look at SDL2 Audio example code. In SDL_OpenAudioDevice register a callback function. This callback function is called when the audio buffer is exhausted. In that callback ask fluidlite to write audio in the buffer.

For just targeting Android without SDL2 you likely have to do something like exporting fluidlite API through JNI, copying the audio buffer to Java and playing it through the Android Audio API (no idea how to do this). This is also annoying and hard to get right.

Coder of Salvation / Leon van Kammen · Answer 5 · Sun May 16 2021 19:30:49 GMT+0800 (China Standard Time)

I succesfully got it to work using SOKOL (https://github.com/floooh/sokol) which can also compile to ANDROID:

  // INIT
  #define SAMPLE_SIZE sizeof(float)
  #define NUM_SAMPLES (64)  // FLUID_BUFSIZE is 64
  fluid_synth_t* synth;
  float* flbuffer;
  int audio_channels;

  saudio_setup(saudio_desc{0});
  audio_channels = saudio_channels();
  fluid_settings_t* settings = new_fluid_settings();
  synth = new_fluid_synth(settings);
  fluid_synth_sfload(synth, "C:\\Users\\You\\Desktop\\test.sf2", 1);
  flbuffer = (float *)calloc( SAMPLE_SIZE, NUM_SAMPLES );

  fluid_synth_noteon(synth, 0, 60, 127);
  printf("triggered note\n");

...

    // MAIN LOOP
    int num_frames  = saudio_expect();
      if( num_frames > 0  ){
        for( int i = 0; i < num_frames; i+= NUM_SAMPLES ){
          fluid_synth_write_float(synth, NUM_SAMPLES, flbuffer, 0, audio_channels, flbuffer, 0, audio_channels ); // optional: change to stereo
          saudio_push(flbuffer, NUM_SAMPLES );
        }
      }

My AHA moment was that there's many ways to forward audio inside C, but in the end it boils down to understanding the audiobuffer (sizes) of the parent process and the child process (fluid).