demuxer.NextData not returning PAT/PMT when seen
tmm1 opened this issue · comments
I have a mpegts segment which starts with three packets (SDT, PAT, PMT), followed by a ES data.
With astits-probe packets -i file.ts, I can confirm the first three packets:
2021/04/01 17:04:25 Fetching packets...
2021/04/01 17:04:25 PKT: 17
2021/04/01 17:04:25 Continuity Counter: 0
2021/04/01 17:04:25 Payload Unit Start Indicator: true
2021/04/01 17:04:25 Has Payload: true
2021/04/01 17:04:25 Has Adaptation Field: false
2021/04/01 17:04:25 Transport Error Indicator: false
2021/04/01 17:04:25 Transport Priority: false
2021/04/01 17:04:25 Transport Scrambling Control: 0
2021/04/01 17:04:25 PKT: 0
2021/04/01 17:04:25 Continuity Counter: 0
2021/04/01 17:04:25 Payload Unit Start Indicator: true
2021/04/01 17:04:25 Has Payload: true
2021/04/01 17:04:25 Has Adaptation Field: false
2021/04/01 17:04:25 Transport Error Indicator: false
2021/04/01 17:04:25 Transport Priority: false
2021/04/01 17:04:25 Transport Scrambling Control: 0
2021/04/01 17:04:25 PKT: 4096
2021/04/01 17:04:25 Continuity Counter: 0
2021/04/01 17:04:25 Payload Unit Start Indicator: true
2021/04/01 17:04:25 Has Payload: true
2021/04/01 17:04:25 Has Adaptation Field: false
2021/04/01 17:04:25 Transport Error Indicator: false
2021/04/01 17:04:25 Transport Priority: false
2021/04/01 17:04:25 Transport Scrambling Control: 0
2021/04/01 17:04:25 PKT: 256
2021/04/01 17:04:25 Continuity Counter: 0
2021/04/01 17:04:25 Payload Unit Start Indicator: true
2021/04/01 17:04:25 Has Payload: true
2021/04/01 17:04:25 Has Adaptation Field: true
2021/04/01 17:04:25 Transport Error Indicator: false
2021/04/01 17:04:25 Transport Priority: false
2021/04/01 17:04:25 Transport Scrambling Control: 0
2021/04/01 17:04:25 Adaptation Field: &{AdaptationExtensionField:<nil> DiscontinuityIndicator:false ElementaryStreamPriorityIndicator:false HasAdaptationExtensionField:false HasOPCR:false HasPCR:true HasTransportPrivateData:false HasSplicingCountdown:false Length:7 IsOneByteStuffing:false StuffingLength:0 OPCR:<nil> PCR:0xc000192070 RandomAccessIndicator:true SpliceCountdown:0 TransportPrivateDataLength:0 TransportPrivateData:[]}
However when I try to use the demuxer on the same file, the PAT/PMT are not returned to me first as I would expect.
for {
d, err := r.demux.NextData()
if err != nil {
panic(err)
}
if d.PAT != nil {
log.Printf("action=pat")
}
if d.PMT != nil {
log.Printf("action=pmt")
}
if d.PES == nil {
continue
}
pid := d.FirstPacket.Header.PID
log.Printf("action=pes pid=%v", pid)
}
I would expect to see action=pat and action=pmt printed as the first two lines. Instead I see:
...
2021/04/01 17:08:41 action=pes pid=256
2021/04/01 17:08:41 action=pes pid=256
2021/04/01 17:08:41 action=pes pid=256
2021/04/01 17:08:41 action=pes pid=256
2021/04/01 17:08:41 action=pes pid=256
2021/04/01 17:08:41 action=pes pid=257
2021/04/01 17:08:41 action=pes pid=256
2021/04/01 17:08:41 action=pes pid=256
2021/04/01 17:08:41 action=pes pid=256
2021/04/01 17:08:41 action=pes pid=256
2021/04/01 17:08:41 action=pes pid=256
2021/04/01 17:08:41 action=pes pid=256
2021/04/01 17:08:41 action=pes pid=256
2021/04/01 17:08:41 action=pes pid=257
2021/04/01 17:08:41 action=pes pid=256
2021/04/01 17:08:41 action=pes pid=257
2021/04/01 17:08:41 action=pat
2021/04/01 17:08:41 action=pes pid=256
2021/04/01 17:08:41 action=pes pid=257
2021/04/01 17:08:41 action=pmt
panic: astits: no more packets
Somehow the PAT and PMT are buffered until the entire file is done, and only returned at the end.
Is there a queue somewhere that's being read in the wrong order?
The same behavior can be seen with astits-probe data -d all -i <file.ts>, where the SDT, PAT and PMT are not printed out until the end.
...
2021/04/01 17:15:11 PES: 257
2021/04/01 17:15:11 Stream ID: 192
2021/04/01 17:15:11 Packet Length: 2824
2021/04/01 17:15:11 Optional Header: &{AdditionalCopyInfo:0 CRC:0 DataAlignmentIndicator:false DSMTrickMode:<nil> DTS:<nil> ESCR:<nil> ESRate:0 Extension2Data:[] Extension2Length:0 HasAdditionalCopyInfo:false HasCRC:false HasDSMTrickMode:false HasESCR:false HasESRate:false HasExtension:false HasExtension2:false HasOptionalFields:false HasPackHeaderField:false HasPrivateData:false HasProgramPacketSequenceCounter:false HasPSTDBuffer:false HeaderLength:5 IsCopyrighted:false IsOriginal:false MarkerBits:2 MPEG1OrMPEG2ID:0 OriginalStuffingLength:0 PacketSequenceCounter:0 PackField:0 Priority:false PrivateData:[] PSTDBufferScale:0 PSTDBufferSize:0 PTS:0xc0003bf450 PTSDTSIndicator:2 ScramblingControl:0}
2021/04/01 17:15:11 PAT: 0
2021/04/01 17:15:11 Transport Stream ID: 1
2021/04/01 17:15:11 Programs:
2021/04/01 17:15:11 &{ProgramMapID:4096 ProgramNumber:1}
2021/04/01 17:15:11 SDT: 17
2021/04/01 17:15:11 PES: 256
2021/04/01 17:15:11 Stream ID: 224
2021/04/01 17:15:11 Packet Length: 0
2021/04/01 17:15:11 Optional Header: &{AdditionalCopyInfo:0 CRC:0 DataAlignmentIndicator:false DSMTrickMode:<nil> DTS:0xc0003bf7c0 ESCR:<nil> ESRate:0 Extension2Data:[] Extension2Length:0 HasAdditionalCopyInfo:false HasCRC:false HasDSMTrickMode:false HasESCR:false HasESRate:false HasExtension:false HasExtension2:false HasOptionalFields:false HasPackHeaderField:false HasPrivateData:false HasProgramPacketSequenceCounter:false HasPSTDBuffer:false HeaderLength:10 IsCopyrighted:false IsOriginal:false MarkerBits:2 MPEG1OrMPEG2ID:0 OriginalStuffingLength:0 PacketSequenceCounter:0 PackField:0 Priority:false PrivateData:[] PSTDBufferScale:0 PSTDBufferSize:0 PTS:0xc0003bf7b0 PTSDTSIndicator:3 ScramblingControl:0}
2021/04/01 17:15:11 PES: 257
2021/04/01 17:15:11 Stream ID: 192
2021/04/01 17:15:11 Packet Length: 1544
2021/04/01 17:15:11 Optional Header: &{AdditionalCopyInfo:0 CRC:0 DataAlignmentIndicator:false DSMTrickMode:<nil> DTS:<nil> ESCR:<nil> ESRate:0 Extension2Data:[] Extension2Length:0 HasAdditionalCopyInfo:false HasCRC:false HasDSMTrickMode:false HasESCR:false HasESRate:false HasExtension:false HasExtension2:false HasOptionalFields:false HasPackHeaderField:false HasPrivateData:false HasProgramPacketSequenceCounter:false HasPSTDBuffer:false HeaderLength:5 IsCopyrighted:false IsOriginal:false MarkerBits:2 MPEG1OrMPEG2ID:0 OriginalStuffingLength:0 PacketSequenceCounter:0 PackField:0 Priority:false PrivateData:[] PSTDBufferScale:0 PSTDBufferSize:0 PTS:0xc0003bfa10 PTSDTSIndicator:2 ScramblingControl:0}
2021/04/01 17:15:11 PMT: 4096
2021/04/01 17:15:11 ProgramNumber: 1
2021/04/01 17:15:11 PCR PID: 256
2021/04/01 17:15:11 Elementary Streams:
2021/04/01 17:15:11 &{ElementaryPID:256 ElementaryStreamDescriptors:[] StreamType:H264 Video}
2021/04/01 17:15:11 &{ElementaryPID:257 ElementaryStreamDescriptors:[] StreamType:AAC Audio}
2021/04/01 17:15:11 Program Descriptors:
2021/04/01 17:15:11 astits: fetching data failed: astits: no more packets
So I think this bug is in (*packetPool).add(p *Packet), here:
Lines 61 to 65 in 077af8e
Basically we wait to flush out the PAT/PMT until the next packet shows up with a PayloadUnitStartIndicator set. Since my ts file only has a singular PAT/PMT at the front, this never triggers and its not until demuxer calls (*packetPool).dump() at the end until that data is flushed.
I guess one way to fix this would be to special case non-ES packets, and compare the size in the header with the accumulated data to decide when to flush out.
/cc @barbashov
Can you try to fix it and make a PR please?
Yes I will, I am trying to figure out what the best approach is first so any help would be appreciated.
I think maybe we can make a special case for when SectionSyntaxIndicator == true (PAT/PMT/CAT) and then compare if SectionLength is smaller than the amount of data accumulated inside the packetPool?
I think the same approach can also be taken for PES using the pes length (if present).
Does that make sense? It means packetPool would need to start parsing out the PSI and PES headers and cache them using the first packet.
Thing is, for SectionSyntaxIndicator doc says Set when a PES, PSI, or DVB-MIP packet begins immediately following the header.. Both PMT and PAT are PSI. Furthermore, SectionLength can't be trusted solely because a section could very well end right at the end of a payload and the next section start at the beginning of the next packet.
Unless I'm missing something, I don't really think there's a way to know when all data of PAT or PMT packets is available for parsing until the next data arrives.
Furthermore that shouldn't be a problem when using the lib : if you don't want to miss out on PES packets, you can buffer them until you get the PAT or PMT, can't you?
Furthermore that shouldn't be a problem when using the lib : if you don't want to miss out on PES packets, you can buffer them until you get the PAT or PMT, can't you?
In the current design that would mean buffering the entire file because the PAT/PMT aren't being flushed until a ErrNoMorePackets is encountered and dump() is called.
Lines 113 to 118 in 5caf397
Unless I'm missing something, I don't really think there's a way to know when all data of PAT or PMT packets is available for parsing until the next data arrives.
Yea I guess this is tricky. The only alternative would be to try to parse them iteratively as new data arrives, to see if the parser succeeds or runs out of data.
That approach also seems to be used by gots:
https://github.com/Comcast/gots/blob/1e4417348cff9f2167a720e496f6c3d259eec700/psi/pmt.go#L405
https://github.com/Comcast/gots/blob/1e4417348cff9f2167a720e496f6c3d259eec700/psi/pmt.go#L64-L68
Unless I'm missing something, I don't really think there's a way to know when all data of
PATorPMTpackets is available for parsing until the next data arrives.
Well, every PSI packet has section_number and last_section_number. From H.222 specification:
section_number – This 8-bit field gives the number of this section. The section_number of the first section in the Program Association Table shall be 0x00. It shall be incremented by 1 with each additional section in the program association table.
last_section_number – This 8-bit field specifies the number of the last section (that is, the section with the highest section_number) of the complete program association table.
So it is possible to know that you've got the whole PAT/PMT by comparing these two.
I thought about that a couple weeks ago.
To use this, one needs to partially parse PSI packets in packet pool. And have access to PAT (I mean the actual table, to determine PMT PIDs) from packet pool. Which is kinda dirty from architecture point of view. And I couldn't figure out the clean way, so haven't done anything there :)
Another idea of mine was to flush buffer on PID change, which is quite simple and clean to implement. But it doesn't work with some streams - some muxers tend to break PES sequence with other packets. This causes flush on PID change to return incomplete PES data.
Just in case, flush on PID change logic used to be like this: 69ec742#diff-bf6bc659050e8825df8f250e9e1db49f0d7e1cfcbe8f977f950046494dd90cb7
@barbashov this is really interesting ❤️
Even though it's the more complicated and weird way to solve this, your first proposition (partially parse PSI data in packet pool and store PAT information to retrieve PMT PIDs afterwards) seems like the best one.
In fact I think it makes sense to isolate the logic that determines whether all payloads of a data have been gathered and parseData must be called.
First I would share Demuxer.programMap with PacketPool so that it has the information about PMT PIDs.
Then I would partially parse packet payloads here to determine whether we can call parseData.
Does that sound like a good idea ?