x265 with aac to mp4?
adminy opened this issue · comments
Is it possible to take h265/x265/hvec file and aac files to make up an mp4?
From my understanding the example only puts h264 into mp4 yes?
What will it take to make to make the h265 and aac files mux into an mp4 using this library?
Most of the muxers don't actually include h265 as a supported format in the muxer, any resources you can point me to on that please.
Thank you.
For h265 I think it's nearly same as h264. 1 nal - 1 MP4E__put_sample.
x265_encoder_encode(enc, &p_nal, &nal, input_frame, &out);
for (i = 0; i < nal; i++)
{
int is_intra = p_nal->type >= NAL_BLA_W_LP && p_nal->type <= NAL_CRA_NUT;
put_nal(p_nal->payload, p_nal->sizeBytes, out.pts, is_intra);
p_nal++;
}
But I'm not sure if mp4_h264_write_nal will work (even without sps/pps transconding), need to try.
For aac it's bit more trickier, because some parameters encoded out of band. Parameters block must be set using MP4E__set_dsi. I think it's compatible with aacEncInfo from fdk-aac, see encoding example here https://github.com/lieff/minirtmp/blob/master/minirtmp_test.c#L255 .
But again, may be I forgot something, need to write sample to be sure.
Thank you, I will do my best to attempt this. I will try to make a mp4_h265_write_nal, but the header is the least difficult bit yes? I guess I can focus on getting the video bit to work first. Thanks again!
Header handling should be easy, something like:
int mp4_h265_write_init(mp4_h265_writer_t *h, MP4E_mux_t *mux, int width, int height)
{
MP4E_track_t tr;
tr.track_media_kind = e_video;
tr.language[0] = 'u';
tr.language[1] = 'n';
tr.language[2] = 'd';
tr.language[3] = 0;
tr.object_type_indication = MP4_OBJECT_TYPE_HEVC;
tr.time_scale = 90000;
tr.default_duration = 0;
tr.u.v.width = width;
tr.u.v.height = height;
h->mux_track_id = MP4E__add_track(mux, &tr);
h->mux = mux;
.... possible h265 specific instead of sps/pps
return 1;
}
Most hard part - we need write something instead of BOX_avc1 and BOX_avcC https://github.com/lieff/minimp4/blob/master/minimp4.h#L1431
Is that because in muxing the hvec file it’s also needed to split it into chunks?
Any proper documentation on this?
Can we look at what ffmpeg is doing to get this working? It’s too complex for me to try and place a big picture of the code, But I’d love to learn more 😊
HEVC elementary stream split into NAL units, same as h264, but NALu types is different.
Here test app gets next nal: https://github.com/lieff/minimp4/blob/master/minimp4_test.c#L82
I've made first implementation step: a5a88b0
So mp4_h26x_write_nal helper now can add hevc nals and some output is produced.
But file can't be played because we need to encode BOX_hev1 instead of BOX_avc1 which I mention above.
(sample hevc elementary streams can be found here http://fate-suite.ffmpeg.org/hevc-conformance/ )
And here documentation I've found:
https://www.google.ru/url?sa=t&rct=j&q=&esrc=s&source=web&cd=3&cad=rja&uact=8&ved=2ahUKEwj5tNqxgJPgAhUKkiwKHcyZA7YQFjACegQICBAC&url=https%3A%2F%2Fwww.itscj.ipsj.or.jp%2Fsc29%2Fopen%2F29view%2F29n14452t.doc&usg=AOvVaw1IMGgBe2noGL2DXRd6h8HN
And here where things done in ffmpeg:
https://github.com/FFmpeg/FFmpeg/blob/master/libavformat/hevc.c#L1096
Probably start from aac will be easier, because aac support is ready while hevc is still needs BOX_hev1 implementation.
Amazing work already on writing the Nals to a file, that’s a great achievement!
I heard there is about 19 NALu types but is NALu what makes the moov atom to be closer up to the file? What exactly is NALu’s purpose? To restructure the mp4 packets in a different style more network friendly? I see having these example files makes it easy to know all the different types of hvec video structures, One of my favourite tools that allows me to see h265 files is http://www.solveigmm.com/en/products/zond/ , their demo used to have a couple of frames preview of the hvec file.
I didn’t understand much from the documentation, it seems like I’d need to learn everything about h264 first before I could understand hvec. ffmpeg’s implementation I understand a little bit.
I will start with aac if it’s fully supported already, maybe I’ll try to port aac to mp4 to js to see if I understand it well, currently minimp4, can it have more than 1 audio track? I’m assuming setting languages to the track is easy, just inserting the 3 letters in the header.
Спасибо 🙏
I compiled your program, Had to remove -flto -Wl,--gc-sections flags when building it (I'm on macOS), Then tried ./minimp4_x86 files/audio_en.aac out.mp4 which gave me: Segmentation fault: 11 but that makes sense, I mean the main only really takes in h264 and h265 encoded files. Then I tried encoding a h265 file and while the mp4 is playing, it is still not actually writing the h265 file but it has the notion of time, how long the video is supposed to be, playing in a black screen.
Can you add support for aac in the main?
Sure, here aac mux sample e80ffa3
To enable change #define ENABLE_AUDIO 0 to #define ENABLE_AUDIO 1 and add -lfdk-aac to link libraries. Installed fdk-aac library is required (libfdk-aac-dev in most distros).
It reads stream.pcm and assumes it 12khz mono, encoded and muxed into mp4 among with video. You can get sample raw stream here https://github.com/lieff/minirtmp
Puts the file toghether but file only 1275 bytes and it’s an empty container playing nothing. Also no console errors Just silently creates an empty container? I used the sample you provided along with the h264 video you have in this repo.
Also to add a second track, do I just call MP4E__set_dsi and MP4E__put_sample With an incremented track id ? Or there is more?
Hmm, I get following result: out.zip
and it plays fine.
Can you attach your modified minimp4_test.c ?
To add second track, you need to setup new MP4E_track_t structure, call
int second_audio_track_id = MP4E__add_track(mux, &second_tr);
and call
MP4E__set_dsi(mux, second_audio_track_id, info.confBuf, info.confSize);
on that new second_audio_track_id.
Here is my modified minimp4_test.c
It could be that the things you test with are different files from the one’s I’m testing, The files I Have are too large to upload here, I’ll see if I can put them on a public domain so you can try the testing too.
files/video.hvc files/audio1_eng.aac files/audio2_rus.aac
Probably you are trying to use encoded aac as input while sample uses uncompressed audio input and encodes it. Problem with .aac files is that there no aac elementary stream, this file probably in adts format or other container. So we need to detect format and demux aac input first. It will be more code and just using aac encoder is easier for small sample.
Also sample detects hevc by searching "hevc" and "265" in file name, so "video.hvc" will not work. Sample code or file name needs modification.
PS: You can attach starting few megabytes of files, it will be enough to test it.