Giters

phoboslab / pl_mpeg

Single file C library for decoding MPEG1 Video and MP2 Audio

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Using mini_audio to play the audio?

SomeGuyDoinWork opened this issue · comments

commented

I noticed that the example uses SDL2, which lets you "push" audio, I don't think miniaudio supports this - any way to allow miniaudio to pull the required pcm frames from the audio as it needs?

commented

Hi, I'm currently working on the same thing, and this is what I have so far. The key to making it work good was the ring buffer. In this example, I'm using my own implementation, but MA implements one as well. I had been searching and searching, no help, no examples that I could find so I just pushed through until I got it working. Hope it can help you as well.

var
  ringBuffer: TRingBuffer<Single>;
  device: ma_device;
  plm: Pplm_t;
  dataReady: Boolean = False;
  CurrentVolume: Single = 0.5;

procedure ma_data_callback(pDevice: Pma_device; pOutput: Pointer; pInput: Pointer; frameCount: ma_uint32); cdecl;
var
  readPointer: PSingle;
  framesNeeded: Integer;
  i: Integer;
  output: PSingle;
begin
  framesNeeded := frameCount * 2;
  if ringbuffer.AvailableBytes >= framesNeeded then
    begin
      readPointer := ringBuffer.DirectReadPointer(framesNeeded);
      output := PSingle(pOutput);
      for i := 0 to framesNeeded - 1 do
      begin
        output^ := readPointer^ * UnitToScalarValue(CurrentVolume, 1);
        Inc(readPointer);
        Inc(output);
      end;
    end
  else
    begin
      FillChar(pOutput^, framesNeeded * SizeOf(Single), 0);
    end;
end;

procedure audio_decode_callback(plm: Pplm_t; samples: Pplm_samples_t; user: Pointer); cdecl;
begin
  ringBuffer.Write(samples^.interleaved, samples^.count*2);
end;

procedure Test05;
var
  lastTime,targetTime,currentTime,elapsedTime,remainingTime,lastFPSTime, endtime: double;
  frameCount: integer;
  framerate: integer;
  laudio: TAudio;
begin
  glfwInit;
  lastTime := glfwGetTime();
  lastFPSTime := lasttime;
  targetTime := 1.0 / 60;
  frameCount := 0;
  framerate :=0;
  endtime := 0;

  ringBuffer := TRingBuffer<Single>.Create(44100);


  var deviceConfig: ma_device_config := ma_device_config_init(ma_device_type_playback);
  deviceConfig.playback.format := ma_format_f32;
  deviceConfig.playback.channels := 2;
  deviceConfig.sampleRate := 44100;
  deviceConfig.dataCallback := @ma_data_callback;

  if ma_device_init(nil, @deviceConfig, @device) <> MA_SUCCESS then begin
    ringbuffer.free;
    exit;
  end;
  ma_device_start(@device);

  plm := plm_create_with_filename('arc/videos/GameKit.mpg');
  if plm = nil then begin
   ma_device_uninit(@device);
   ringbuffer.free;
   exit;
  end;

  plm_set_audio_decode_callback(plm, @audio_decode_callback, nil);

  while true do
  begin
    // start frame
    currentTime := glfwGetTime();
    elapsedTime := currentTime - lastTime;

    if AnyKeyPressed then Break;

    plm_decode(plm, targetTime);

    if plm_has_ended(plm) = MA_TRUE then
      break;

    // end frame
    Inc(frameCount);
    if (currentTime - lastFPSTime >= 1.0) then
    begin
      framerate := framecount;
      lastFPSTime := currentTime;
      frameCount := 0;
    end;
    lastTime := currentTime;
    remainingTime := targetTime - (currentTime - lastTime);
    if (remainingTime > 0) then
     begin
        endTime := currentTime + remainingTime;
        while glfwGetTime() < endTime do
        begin
          // Busy-wait for the remaining time
        end;
      end;
  end;

  plm_destroy(plm);
  ma_device_uninit(@device);
  laudio.Free;
  ringbuffer.Free;
  glfwTerminate;
end;
commented

Have a look at how the intro video for my wipeout-rewrite is handled: https://github.com/phoboslab/wipeout-rewrite/blob/master/src/wipeout/intro.c

There I'm using SDL2 with a pull callback for the audio. pl_mpeg writes audio data into a fixed size ring buffer (in audio_cb()) that has a read and write offset; both of which wrap around at the end of the buffer. audio_mix() in this file is called whenever SDL needs more audio data and reads from this ringbuffer.

Another solution would be to call plm_samples_t *plm_decode_audio(plm_t *self); yourself in the pull callback. Note that this will always give you 1152 samples (per channel), so you'd still need to buffer remaining ones that miniaudio can't immediately accept. You would then decode video with plm_decode_video() up the point of samples->time from the last audio decode, to keep everything synchronized.

commented

@jarroddavis68 Thanks, this was helpful. Why is your buffer 44100 bytes? That enough for all scenarios? And is your ring buffer implementation pretty standard? I was having trouble converting to mini_audios ring buffer implementation.

commented

@jarroddavis68 Thanks, this was helpful. Why is your buffer 44100 bytes? That enough for all scenarios?

No problem. Glad I could help.
I set the samplerate to 44100hz so it's big enough to play 1 sec of audio, which seems to work.

And is your ring buffer implementation pretty standard? I was having trouble converting to mini_audios ring buffer implementation.

Yea, I was not able to get MA's implementation working either. I think it is more due to my lack of understanding of how it is supposed to work. Implementing my own helped me to "see" how it works so at some point I may try to see if I can get it working. I still need to add support for making it threadsafe. Here is my current implementation:

type
  { TRingBuffer }
  TRingBuffer<T> = class
  private
    FBuffer: array of T;
    FReadIndex, FWriteIndex, FCapacity: Integer;
  public
    constructor Create(ACapacity: Integer);
    function Write(const AData: array of T; ACount: Integer): Integer;
    function Read(var AData: array of T; ACount: Integer): Integer;
    function DirectReadPointer(ACount: Integer): PSingle;
    function AvailableBytes: Integer;
  end;

constructor TRingBuffer<T>.Create(ACapacity: Integer);
begin
  SetLength(FBuffer, ACapacity);
  FReadIndex := 0;
  FWriteIndex := 0;
  FCapacity := ACapacity;
end;

function TRingBuffer<T>.Write(const AData: array of T; ACount: Integer): Integer;
var
  i, WritePos: Integer;
begin
  for i := 0 to ACount - 1 do
  begin
    WritePos := (FWriteIndex + i) mod FCapacity;
    FBuffer[WritePos] := AData[i];
  end;
  FWriteIndex := (FWriteIndex + ACount) mod FCapacity;
  Result := ACount;
end;

function TRingBuffer<T>.Read(var AData: array of T; ACount: Integer): Integer;
var
  i, ReadPos: Integer;
begin
  for i := 0 to ACount - 1 do
  begin
    ReadPos := (FReadIndex + i) mod FCapacity;
    AData[i] := FBuffer[ReadPos];
  end;
  FReadIndex := (FReadIndex + ACount) mod FCapacity;
  Result := ACount;
end;

function TRingBuffer<T>.DirectReadPointer(ACount: Integer): PSingle;
begin
  Result := @FBuffer[FReadIndex mod FCapacity];
  FReadIndex := (FReadIndex + ACount) mod FCapacity;
end;

function TRingBuffer<T>.AvailableBytes: Integer;
begin
  Result := (FCapacity + FWriteIndex - FReadIndex) mod FCapacity;
end;

Here is that code running, playing the audio from my intro video

ma_plmpeg.mp4
commented

@SomeGuyDoinWork
Hi, I managed to get something nice working now, using the structure outline above.

procedure Test09;
const
  CTitle = 'GameKit: Video Plackback from Zip file';
begin
  Gfx.OpenWindow(CTitle, SCREEN_WIDTH, SCREEN_HEIGHT);

  Video.Play(TZipFile.Open(CArchiveFilename, 'arc/videos/GameKit.mpg'), 1.0, True);

  while not Gfx.WindowShouldClose do
  begin

    if WasKeyPressed(VK_ESCAPE) then
      Break;

    Video.Update;

    Gfx.StartDrawing;
      Gfx.ClearWindow(DARKSLATEBROWN);

      Video.Draw(0, 0, 0.5);

    Gfx.EndDrawing;
    Gfx.SetWindowTitle(Format(CTitle+ ' (%d fps)', [Gfx.GetFrameRate]));
  end;

  Video.Stop;

  Gfx.CloseWindow;
end;
GameKit-Video.mp4
commented

That helps alot. I've got it working out for now. Thanks.