How to get codec data suitable for mime type?

[ivanjaros]

I need to get the codec information so I can build proper mime type for video streams. I am using the TS container and I need to get this information for HLS streams https://wiki.whatwg.org/wiki/Video_type_parameters#Video_Codecs_3

In other words, I need to figure out how to get to this: video/mp2t; codecs="avc1.77.30, mp4a.40.2"

I can get to audio codec name, audio sample format, audio sample rate, audio channel layout, vudeo width, video height and video coded type name and I can calculate the bandwidth per each gop but I am not sure how I can figure out which codecs to output from this information.

See also:
https://developer.mozilla.org/en-US/docs/Web/API/MediaSource/addSourceBuffer#Example
https://en.wikipedia.org/wiki/Advanced_Video_Coding#Levels

PS: again, thanks for this library. I can't even begin to imagine having to write something like this from scratch.

[ivanjaros]

In addition to using stream.(av.AudioCodecData) and stream.(av.VideoCodecData) I can use the AAC and h264 parsers as stream.(aacparser.CodecData) and stream.(h264parser.CodecData), which gives me the following information:

aacparser.MPEG4AudioConfig{
  SampleRate: 44100,
  ChannelLayout: 6,
  ObjectType: 2,
  SampleRateIndex: 4,
  ChannelConfig: 2,
}

h264parser.SPSInfo{
  ProfileIdc: 100,
  LevelIdc: 31,
  MbWidth: 80,
  MbHeight: 45,
  CropLeft: 0,
  CropRight: 0,
  CropTop: 0,
  CropBottom: 0,
  Width: 1280,
  Height: 720,
}
h264parser.AVCDecoderConfRecord{
  AVCProfileIndication: 100,
  ProfileCompatibility: 0,
  AVCLevelIndication: 31,
  LengthSizeMinusOne: 3,
  SPS: [][]uint8{
    []uint8{
      39,
      100,
      0,
      31,
      172,
      43,
      96,
      40,
      2,
      221,
      128,
      136,
      0,
      0,
      3,
      0,
      8,
      0,
      0,
      3,
      0,
      247,
      2,
      0,
      7,
      161,
      32,
      0,
      122,
      18,
      222,
      247,
      193,
      218,
      28,
      50,
      224,
    },
  },
  PPS: [][]uint8{
    []uint8{
      40,
      238,
      60,
      176,
    },
  },
}

and I can see there are some additional useful information but still don't know how to figure out the codec string for the mime type.

[ivanjaros]

I see in the joy5 h264 version there are constants for the profiles defined and they correspond to the profiles section of the wiki i have linked to https://en.wikipedia.org/wiki/Advanced_Video_Coding#Profiles which means the profile indication fields(in my case 100) is what is important and I am on the right track. Yet I still don't see how to figure out the names/values I need in end of this.

For example, this from the wiki doen't give me much information:

Video Codecs
    H.264 Baseline: avc1.42E0xx, where xx is the AVC level
    H.264 Main: avc1.4D40xx, where xx is the AVC level
    H.264 High: avc1.6400xx, where xx is the AVC level
    MPEG-4 Visual Simple Profile Level 0: mp4v.20.9
    MPEG-4 Visual Advanced Simple Profile Level 0: mp4v.20.240
Audio Codecs
    Low-Complexity AAC: mp4a.40.2

[ivanjaros]

Luckily, I have found what I was looking for :) https://stackoverflow.com/a/16365526/11493911
The thing is that for the audio, there should be object type indication AND also object type, which is missing.

To quote:

As for mp4a.40.2, mp4a indicates MPEG-4 audio. It is followed by a dot and a hexadecimal ObjectTypeIndication (objectTypeId in mp4file output), which can be looked up on the MPEG4 registration site. If this hexadecimal value is 40 (ISO/IEC 14496-3 Audio), it is followed by another dot and an audio object type in decimal. These are listed in the ISO/IEC 14496-3 standard and on Wikipedia, and correspond to the first 5 bits of the DecoderSpecificInfo (decSpecificInfo) (unless these bits equal 31, in which case add 32 to the next 6 bits). mp4a.40.2 indicates AAC LC audio, which is what is usually used with H.264 HTML5 video.