Unhandled FormatField Exception
rpm5099 opened this issue · comments
This is very likely caused by corruption in an elf file, but I thought you might want to look into the possible cause. Unfortunately I do not have a sample I can provide. Here is the traceback:
---------------------------------------------------------------------------
FieldError Traceback (most recent call last)
File ~/miniconda3/envs/py3114/lib/python3.11/site-packages/elftools/construct/core.py:351, in FormatField._parse(self, stream, context)
350 try:
--> 351 return self.packer.unpack(_read_stream(stream, self.length))[0]
352 except Exception as ex:
File ~/miniconda3/envs/py3114/lib/python3.11/site-packages/elftools/construct/core.py:293, in _read_stream(stream, length)
292 if len(data) != length:
--> 293 raise FieldError("expected %d, found %d" % (length, len(data)))
294 return data
FieldError: expected 4, found 0
During handling of the above exception, another exception occurred:
FieldError Traceback (most recent call last)
File ~/miniconda3/envs/py3114/lib/python3.11/site-packages/elftools/common/utils.py:41, in struct_parse(struct, stream, stream_pos)
40 stream.seek(stream_pos)
---> 41 return struct.parse_stream(stream)
42 except ConstructError as e:
File ~/miniconda3/envs/py3114/lib/python3.11/site-packages/elftools/construct/core.py:190, in Construct.parse_stream(self, stream)
183 """
184 Parse a stream.
185
186 Files, pipes, sockets, and other streaming sources of data are handled
187 by this method.
188 """
--> 190 return self._parse(stream, Container())
File ~/miniconda3/envs/py3114/lib/python3.11/site-packages/elftools/construct/core.py:647, in Struct._parse(self, stream, context)
646 else:
--> 647 subobj = sc._parse(stream, context)
648 if sc.name is not None:
File ~/miniconda3/envs/py3114/lib/python3.11/site-packages/elftools/construct/core.py:353, in FormatField._parse(self, stream, context)
352 except Exception as ex:
--> 353 raise FieldError(ex)
FieldError: expected 4, found 0
During handling of the above exception, another exception occurred:
ELFParseError Traceback (most recent call last)
Cell In[34], line 2
1 eof = 0
----> 2 for i,seg in enumerate(elf.iter_segments()):
3 print(i, seg.header.p_offset, seg.header.p_filesz)
4 if seg.header.p_offset + seg.header.p_filesz > eof:
File ~/miniconda3/envs/py3114/lib/python3.11/site-packages/elftools/elf/elffile.py:207, in ELFFile.iter_segments(self, type)
200 """ Yield all the segments in the file. If the optional |type|
201 parameter is passed, this method will only yield segments of the
202 given type. The parameter value must be a string containing the
203 name of the type as defined in the ELF specification, e.g.
204 'PT_LOAD'.
205 """
206 for i in range(self.num_segments()):
--> 207 segment = self.get_segment(i)
208 if type is None or segment['p_type'] == type:
209 yield segment
File ~/miniconda3/envs/py3114/lib/python3.11/site-packages/elftools/elf/elffile.py:197, in ELFFile.get_segment(self, n)
194 """ Get the segment at index #n from the file (Segment object)
195 """
196 segment_header = self._get_segment_header(n)
--> 197 return self._make_segment(segment_header)
File ~/miniconda3/envs/py3114/lib/python3.11/site-packages/elftools/elf/elffile.py:604, in ELFFile._make_segment(self, segment_header)
602 return InterpSegment(segment_header, self.stream)
603 elif segtype == 'PT_DYNAMIC':
--> 604 return DynamicSegment(segment_header, self.stream, self)
605 elif segtype == 'PT_NOTE':
606 return NoteSegment(segment_header, self.stream, self)
File ~/miniconda3/envs/py3114/lib/python3.11/site-packages/elftools/elf/dynamic.py:247, in DynamicSegment.__init__(self, header, stream, elffile)
239 def __init__(self, header, stream, elffile):
240 # The string table section to be used to resolve string names in
241 # the dynamic tag array is the one pointed at by the sh_link field
(...)
244 # segment, we do so by searching for the dynamic section whose content
245 # is located at the same offset as the dynamic segment
246 stringtable = None
--> 247 for section in elffile.iter_sections():
248 if (isinstance(section, DynamicSection) and
249 section['sh_offset'] == header['p_offset']):
250 stringtable = elffile.get_section(section['sh_link'])
File ~/miniconda3/envs/py3114/lib/python3.11/site-packages/elftools/elf/elffile.py:174, in ELFFile.iter_sections(self, type)
167 """ Yield all the sections in the file. If the optional |type|
168 parameter is passed, this method will only yield sections of the
169 given type. The parameter value must be a string containing the
170 name of the type as defined in the ELF specification, e.g.
171 'SHT_SYMTAB'.
172 """
173 for i in range(self.num_sections()):
--> 174 section = self.get_section(i)
175 if type is None or section['sh_type'] == type:
176 yield section
File ~/miniconda3/envs/py3114/lib/python3.11/site-packages/elftools/elf/elffile.py:141, in ELFFile.get_section(self, n)
137 """ Get the section at index #n from the file (Section object or a
138 subclass)
139 """
140 section_header = self._get_section_header(n)
--> 141 return self._make_section(section_header)
File ~/miniconda3/envs/py3114/lib/python3.11/site-packages/elftools/elf/elffile.py:666, in ELFFile._make_section(self, section_header)
664 return ARMAttributesSection(section_header, name, self)
665 elif sectype == 'SHT_HASH':
--> 666 return self._make_elf_hash_section(section_header, name)
667 elif sectype == 'SHT_GNU_HASH':
668 return self._make_gnu_hash_section(section_header, name)
File ~/miniconda3/envs/py3114/lib/python3.11/site-packages/elftools/elf/elffile.py:740, in ELFFile._make_elf_hash_section(self, section_header, name)
738 linked_symtab_index = section_header['sh_link']
739 symtab_section = self.get_section(linked_symtab_index)
--> 740 return ELFHashSection(
741 section_header, name, self, symtab_section
742 )
File ~/miniconda3/envs/py3114/lib/python3.11/site-packages/elftools/elf/hash.py:80, in ELFHashSection.__init__(self, header, name, elffile, symboltable)
78 def __init__(self, header, name, elffile, symboltable):
79 Section.__init__(self, header, name, elffile)
---> 80 ELFHashTable.__init__(self, elffile, self['sh_offset'], symboltable)
File ~/miniconda3/envs/py3114/lib/python3.11/site-packages/elftools/elf/hash.py:33, in ELFHashTable.__init__(self, elffile, start_offset, symboltable)
31 self.elffile = elffile
32 self._symboltable = symboltable
---> 33 self.params = struct_parse(self.elffile.structs.Elf_Hash,
34 self.elffile.stream,
35 start_offset)
File ~/miniconda3/envs/py3114/lib/python3.11/site-packages/elftools/common/utils.py:43, in struct_parse(struct, stream, stream_pos)
41 return struct.parse_stream(stream)
42 except ConstructError as e:
---> 43 raise ELFParseError(str(e))
ELFParseError: expected 4, found 0
If you cannot provide the entire ELF file, can you maybe share the section header table (the output of readelf - S
) and the program headers (readelf - l
)? Additionally, can you check if the file has a dynamic symbol table but exports no symbols itself?
Thanks!
I don't have the data, but it was a fragment of an elf that was truncated well before the end of the program. The data after the truncation was essentially random as it was forensically carved based on the program headers. The traceback should get you close enough to find the code that did not handle the unexpected input well. If it was never intended to be able to safely parse potentially corrupt elf's that's understandable, but it happens and I figured I'd give you the option.
The root cause of the error is that the stream ended prematurely. It's consistent with data corruption. I don't see how this is an issue with pyelftools and not with the ELF file.
In general, our preferred rule of thumb is - can the GNU or the LLVM tools (e. g. readelf
) parse the binary with no errors? If they do, and pyelftools throws an error, it's issue with pyelftools. Else, it's the issue with the binary.
The root cause of the error is that the stream ended prematurely. I don't see how this is an issue with pyelftools and not with the ELF file.
The suggestion is to put in an exception indicating that the data is truncated or corrupt in place of the traceback above which is meaningless to users.
But you already got an exception. Maybe not a very descriptive one, but providing a separate exception type for every possible kind of data corruption out there is not feasible.
Here is a very similar discussion: #367
Looked at #367, he is reporting the same thing - readelf provides some information about what went wrong and how the elf is malformed. Without that information you're left wondering whether the elf was corrupt or the tool just failed on a valid elf. I agree that the goal should not be to parse the file the same as the Linux kernel loader does, which does not use section headers at all. Again, if it's not a feature that you want to support - don't.
Barring a clarification from @eliben, I'd say that providing diagnostics on the exact nature of file corruption is not the ambition of this effort.
I think this is a slightly different issue than #367, which was asking pyelftools to be forgiving and agree to parse a malformed file. That's a clear no.
This issue seems to be talking about producing a clearer error message in case this is simple; I don't have any fundamental objections to this, but the caveat is in the is simple.