Panic when dumping STPP Mdat

[diniboy]

Hi,

There is a streaming service that doesn't display subtitles within Kodi for some reason. I would like to know why and that's why I attempted to use mp4ff unsuccessfully to read the file and parse its MDAT box which has the XML content as-is.

I attached a sample here:

sub.zip

The first oddity happens with this code snippet:

package main

import "github.com/Eyevinn/mp4ff/mp4"

func main() {
	parsedMp4, err := mp4.ReadMP4File("sub.mp4")
	if err != nil {
		panic(err)
	}

	parsedMp4.Sidx.ReferenceID = 0
}

I get

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x4bef69]

Second happens with this:

package main

import (
	"log"
	"github.com/Eyevinn/mp4ff/mp4"
)

func main() {
	parsedMp4, err := mp4.ReadMP4File("sub.mp4")
	if err != nil {
		panic(err)
	}

	log.Default().Printf("Parsed MP4 Mdat: %v", parsedMp4.Mdat.Data)
}

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x4bf104]

Even calling Encode fails on Mdat which is strange. If I run Encode on the top level parsedMp4, it saves the file correctly.

I am not sure yet why is the subtitle broken, but ffmpeg complains with unknown track id 257 for example. If this would work, it would help me investigate a lot. I also thought of converting the stpp to webvtt on the fly, but the subs are usually base64 encoded images there...

[Torbjörn Einarson]

@Diniboy1123

The file is not broken, but you don't use mp4ff in the right way.

mp4ff is not a simple parser, but represents data in different ways depending on whether the file is "progressive" or "segmented".

Pointers likeSidx and Mdat are dangerous, in the sense that they may be nil, so one should better check these before using them. At the top level, they are only set for progressive files, while for progressive files, these boxes are mapped inside fragments inside segments. There is a method to ask whether the file is progressive or not.

In general, if you can parse the file with mp4ff-info, mp4ff should have a proper representation of the file structure, but what pointers are set depends on the type of file.

With mp4ff-info -l all:1 file.mp4 you should be able to get a good picture of the box content.

In this case it is parsedMp4.Segments[0].Fragments[0].Mdat that is the proper Mdat box.

For proper decoding you need to include the init segment that has the timescale for the segment and some more information like the trackID. You can also see here that the trackID is 257. That is very unusual, since you normally start at 1 and if you don't multiplex media, you can in fact use 1 for all different types of tracks.

From the mp4ff-info output you can see that mdat is the last 560 bytes, so you can easily extract its data with tail -c 552 sub.mp4 (removing 8 for the header size).

[3052]

note I ran into a similar problem, but it turns out the site I use also offer WebVTT subtitles. I would strongly recommend that approach, because as you may have discovered player support for TTML is awful. if you can name the site I can try to help see if WebVTT is offered

[Torbjörn Einarson]

In general, text-based subtitles are easier to handle. Most TTML subtitles are indeed text-based, but your sample is image based where the image is base64-encoded inside the XML.

The dash.js player has quite good support for TTML including both text and image subtitles in stpp (mp4) segments. I've implemented inclusion of binary images in stpp in dash.js, but there is also support for your variety (smpte-tt), so you may give it a try.

In any case, I don't see that there is a problem in the mp4ff code that is causing your problem, so I will close this ticket.