markbates / pkger

Embed static files in Go binaries (replacement for gobuffalo/packr)

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

"Here" package caching does not work

xDuck opened this issue · comments

commented

Relates to #58 (maybe)

I have a medium sized angular application (13MB) that was taking way longer than it used to when it was a small project. I started digging through the code and there are many many repeated go list calls made with the same arguments and producing the same output.

Looks like these functions are ignoring the cache and running the go commands every time (slowing down execution times significantly)

https://github.com/gobuffalo/here/blob/master/pkg.go#L17
https://github.com/gobuffalo/here/blob/master/dir.go#L11
https://github.com/gobuffalo/here/blob/master/dir.go#L65

The way the gobuffalo/here module is used seems to create a new here doc everytime a call is made to the module, thus creating a new cache.

One of the public methods used by the pkger code a lot (about 10 times over 9 files):
https://github.com/gobuffalo/here/blob/master/dir.go#L61

gobuffalo/here New() method (creates a new cache)
https://github.com/gobuffalo/here/blob/master/here.go#L21

I'm not sure if this is an issue with gobuffalo or the integration of it within this project, but from looking over the code it seems the caching is local to each here object and there is a here object for each file - so the cache is per-file (i.e always empty). Implementing a very hacky static cache increased the performance significantly.

Although I could be totally off base on this, as I can't spend all afternoon digging into it. However, as amazing as this project is, it seems unfit for any projects with more than a handful of files / folders right now.

commented

I have made a couple changes to the code and got the time to build down significantly by increasing the cache hits. It appears the caching is working mostly as intended now, but I am still curious as to why there are so many calls to go list -json and go list json -m during the walk step? I think thats a much larger fix though.

Going to make a PR shortly for my changes for discussion.