Newest PolyTracker: PolyTrackerTrace.load() leads to KeyError

[BwRy]

Hello! I'm running Ubuntu20, python3.8 and ran into an issue attempting to load a tdag.

Steps to Reproduce

I cloned the newest version of polytracker from this repo, cd'd to it and installed via pip3 install ./.

I then attempted to load the tdag database, which led to a KeyError:

output$ python3
Python 3.8.10 (default, May 26 2023, 14:05:08) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from polytracker import PolyTrackerTrace, taint_dag, os
/usr/lib/python3/dist-packages/paramiko/transport.py:219: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'os' from 'polytracker' (/usr/local/lib/python3.8/dist-packages/polytracker/__init__.py)
>>> from polytracker import PolyTrackerTrace, taint_dag
>>> import os
>>> trace = PolyTrackerTrace.load("libpng-1.6.37.tdag")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/dist-packages/polytracker/taint_dag.py", line 649, in load
    return TDProgramTrace(open(tdpath, "rb"))
  File "/usr/local/lib/python3.8/dist-packages/polytracker/taint_dag.py", line 592, in __init__
    self.tdfile: TDFile = TDFile(file)
  File "/usr/local/lib/python3.8/dist-packages/polytracker/taint_dag.py", line 491, in __init__
    self.fd_headers: List[Tuple[Path, TDFDHeader]] = list(self.read_fd_headers())
  File "/usr/local/lib/python3.8/dist-packages/polytracker/taint_dag.py", line 498, in read_fd_headers
    sources = self.sections_by_type[TDSourceSection]
KeyError: <class 'polytracker.taint_dag.TDSourceSection'>

I verified this key does exist in the taint_dag class:

>>> taint_dag.TDSourceSection
<class 'polytracker.taint_dag.TDSourceSection'>

Overall, I want to use the github version of PolyTracker so that I can leverage the new code additions; namely, retrieving labels via polytracker.taint_dag.TDFile.input_labels and using those to pass off to additional analysis

[Henrik Brodin]

Thanks for using PolyTracker!
From the trace you added it seems like there is no TDSourceSection in the tdag-file. This situation deserves a better error message.
It looks like there might have been an issue with how the tdag-file was generated. Could you please share the steps used to generate the tdag?

[BwRy]

I had a version mismatch between the older docker image, tried building outside of it. After updating acropalypse's Dockerfile to also include the URL for libpng, I ran the pngtest, copied the tdag outside of the docker image and got it working. I'm going to close as this was a potato on my end.

By the way, thanks for all of the dev, tooling and support over the years!