`AudioQueue::queue_audio()` unelegant cast for correct audio.

[Thomas Baugher]

Hello, I have a question concerning the usage of the AudioQueue::queue_audio() method in tandem with another Rust library, ffmpeg_next.

Background

I decided recently to start on working on understanding how to use the libav* libraries, and what better way to start than working with the abstracting layer of ffmpeg_next. Despite diving head first, I was able to get video decoding working with an example provided by that project, and was working on getting audio set up as well (with some help from this tutorial).

To cut it short, I decided to use the queuing feature of SDL2 instead of the callback feature, and I was able to get audio from what is considered the AUDIO_S16SYS/AUDIO_S16LSB format, albeit garbled and quiet. I decided to rewrite the portion of audio code in C++ (also using libav and SDL2) to get a more grounded understanding on what was happening, and after some time, was able to get the correct audio playing as intended!

Technical

So I had to track down the discrepancy between the C++ code and the Rust code and its abstracted libraries. I had a funny feeling about the AudioQueue struct, specifically with Channel generic type that was attached to it. In order to instantiate an AudioQueue, you need to provide a Channel type via a superfish. From my code:

let audio_device = audio.open_queue::<i16, _>(None, &desired_format).unwrap();

i16 correlates to AUDIO_S16SYS/AUDIO_S16LSB, and the format field for the SDL_AudioSpec struct. This generates a method something like this:

pub fn queue_audio(&self, data: &[i16]) -> Result<(), String>

...where i16 would be called Channel, but this is functionally the same.

I decode my packets from into a frame from the ffmpeg_next side of the code. My frame contains a method called Frame::data(), which requires an index argument, since there could be multiple data arrays, but since my audio data in my tests is packed, it should be only in the first (0) index. Basically,

let data: &[u8] = frame.data(0);

This data...has to go into a queue...that can only take &[i16]. Casting each element into i16 doesn't work, and is actually the source of my problem, since doing an as messes with how byte-length of the array is passed over. My solution was this:

let data = unsafe {
	std::slice::from_raw_parts(data.as_ptr() as *const i16, data.len() / 2)
};

which forces the slice into another type, preserving the byte-length of the pointer/array. On that last point, AudioQueue::queue_audio() uses the slice's length, and multiplies it by the size of the Channel type to get the byte-length of the data that it's receiving. data.len() / 2 is there to preserve that length that I received on the first definition of data. And in the end, it still sends the data (effectively) as u8!

Conclusion

The effort I put into this is so I can provide enough context for what I want to do, what I'm expecting, and what I experienced. I wouldn't be surprised if my understanding of Rust is lacking in some areas, and that there may be an easier solution that dipping into unsafe code, but I was curious if this is something that anyone has ran into when trying to queue audio, and if there was a more elegant solution beyond creating an unsafe block. The squarewave example doesn't seem to have a problem with the current implementation from what I have seen by the way.

Thanks for your time and help!

[Cobrand]

If you want to convert a &[u8] (bytes) into a &[i16] (array of integers) slice, I'd suggest you use the crate bytemuck: https://docs.rs/bytemuck/latest/bytemuck/fn.try_cast_slice.html

Keep in mind that if you look at the implementation, it uses from_raw_parts under the hood, so it's basically the same as your solution, but at least the unsafe is not in your code. https://docs.rs/bytemuck/latest/src/bytemuck/internal.rs.html#353

[Thomas Baugher]

Haha, I actually found this exact crate after looking on type casting slices. Nonetheless, thanks for the recommendation! I think it might be worthwhile to add a note in the documentation about casting slices in the case someone has incompatible data. I could go about doing that if you think a PR will be merged.