zh217 / torch-dct

DCT (discrete cosine transform) functions for pytorch

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

numerical precision

Sunshine352 opened this issue · comments

assert (torch.abs(x - y)).sum() < 1e-10 # x == y within numerical tolerance
numerical tolerance error cannot satisfy 1e-10, far beyond this

Please provide a failing test. If the included pytest fails on your machine, please provide system and version information. Thanks.

OK I see that you are running the example on the README. That test used to pass, but it somehow fails with the recent pytorch build. Still, the difference can be considered "within numerical tolerance" on my machine (on the order of 1e-5). Unfortunately I don't have the time to investigate this issue further. If you have a fix you are welcome to submit a pull request.