Giters

nschloe / tuna

:fish: Python profile viewer

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

total execution time is wrong (handling of imports)

ellperro opened this issue · comments

commented

tuna helped me to pinpoint a performance issue in my program, thanks!
but I notced the total execution time is wrong.

I simplified it down tot this mini-program, essentially just a (big) import:
demo.py

import time

def b():
    import matplotlib
    time.sleep(.1)

b()

profiling:

time python3 -m cProfile -o prof ./demo.py

prof.gz

the profile and snakeviz show a total execution time of .281s
tuna shows 1.16s, i.e. x4
tuna is clearly wrong (time when generating the profile showed real:.382s)

commented

in an attempt to further simplify the case,
I replaced the matplotlob import with an import of my own module, which simply sleeps for one second.
i ended up with this profile: prof2.gz
which makes tuna crash:

raceback (most recent call last):
  File "/home/bart/.local/bin/tuna", line 8, in <module>
    sys.exit(main())
  File "/home/bart/.local/lib/python3.8/site-packages/tuna/cli.py", line 32, in main
    start_server(args.infile, args.browser, args.port)
  File "/home/bart/.local/lib/python3.8/site-packages/tuna/main.py", line 44, in start_server
    data = read(prof_filename)
  File "/home/bart/.local/lib/python3.8/site-packages/tuna/main.py", line 23, in read
    return read_runtime_profile(filename)
  File "/home/bart/.local/lib/python3.8/site-packages/tuna/_runtime_profile.py", line 88, in read_runtime_profile
    "children": [populate(root, None) for root in roots],
  File "/home/bart/.local/lib/python3.8/site-packages/tuna/_runtime_profile.py", line 88, in <listcomp>
    "children": [populate(root, None) for root in roots],
  File "/home/bart/.local/lib/python3.8/site-packages/tuna/_runtime_profile.py", line 47, in populate
    c = [populate(child, key) for child in children[key]]
  File "/home/bart/.local/lib/python3.8/site-packages/tuna/_runtime_profile.py", line 47, in <listcomp>
    c = [populate(child, key) for child in children[key]]
  File "/home/bart/.local/lib/python3.8/site-packages/tuna/_runtime_profile.py", line 47, in populate
    c = [populate(child, key) for child in children[key]]
  File "/home/bart/.local/lib/python3.8/site-packages/tuna/_runtime_profile.py", line 47, in <listcomp>
    c = [populate(child, key) for child in children[key]]
...
RecursionError: maximum recursion depth exceeded in comparison

snakeviz/pstats have no problem with the profile

commented

Please open a new issue for the last one and also provide Python code for that generates the profile.

commented

new issue: #91

commented

The longer runtime is correct. Note that snake gets the total time wrong:

snake

tuna

Python records the total CPU time, so if 3 cores are working 0.8 seconds, the total time will be 2.4 seconds.

python3 -m cProfile -o profile.prof k.py  0.72s user 0.63s system 283% cpu 0.477 total

(283% cpu)

commented

I've updated tune to drop the "root" element if possible. The plot looks cleaner now. #93

screenshot