chakravala / Grassmann.jl

The source code for this library has a very large number of functions marked @pure. My understanding is that the use of this macro is emphatically discouraged. In this issue JuliaLang/julia#20704, Keno said

@pure means I am @vtjnash and I certify that this function doesn't break any of the implicit assumptions I use in the compiler ;)

In this thread: https://discourse.julialang.org/t/pure-macro/3871/3, Matt says that

Improper @pure annotations can introduce bugs. The optimizations it enables rely on an extremely strict definition of pure. It really should be named something like @hyperpure . Some of the restrictions include:

It must always return exactly ( === ) the same result for a given input. Watch out for mutable types. I think constant globals are okay, though.

The function it’s used on cannot be further extended by other methods after it gets called.

It cannot recurse.

It’s undocumented and not exported (for good reason), but this means the complete list of preconditions is really only in a few people’s heads.

For instance,

Grassmann.jl/src/Grassmann.jl

Line 42 in f4e4f78

@pure getindex(a::Algebra,i::UnitRange{Int}) = [getindex(a,j) for j ∈ i]

is an invalid usage of @pure since an array is returned and pure functions must return outputs that are equal in the === sense.

Furthermore, adding methods to @pure functions is said to not be kosher and this repository is replete with @pure functions with methods added on.

In this thread https://discourse.julialang.org/t/can-pure-functions-throw-an-error/18459, Jameson advises that @pure functions throwing errors is "Probably, a very very bad idea", but I see that there are functions that throw in multivectors.jl and Grassmann.jl.

Given this, I think cutting back on most if not all uses of the uses of @pure in this package is probably a good idea.

Perhaps this deserves a separate issue, but I noticed that in utilities.jl there are struct definitions

struct Grade{G}
    @pure Grade{G}() where G = new{G}()
end

## Dimension{N}

struct Dimension{N}
    @pure Dimension{N}() where N = new{N}()
end

I believe the above inner constructors are no-ops that don't need to exist. For instance:

julia> struct MyStruct1{T} end

julia> struct MyStruct2{T}
           MyStruct2{T}() where {T} = new{T}()
       end

julia> @code_lowered MyStruct1{1}()
CodeInfo(
1 ─ %1 = (Core.apply_type)(Main.MyStruct1, $(Expr(:static_parameter, 1)))
│   %2 = %new(%1)
└──      return %2
)

julia> @code_lowered MyStruct2{1}()
CodeInfo(
1 ─ %1 = (Core.apply_type)(Main.MyStruct2, $(Expr(:static_parameter, 1)))
│   %2 = %new(%1)
└──      return %2
)

Thank you for your detailed report regarding the issue with @pure.

I am aware that there are certain issues associated with it; however, I do not intend on cutting back on the usage of it very much in this repository.

However, I would like to investigate my usage of it more and try to make it more stable and correct in its usage, because many things in this repository have rapidly changed.

This issue will be closed now, since I don't intend to cut back overall, but the discussion can remain open here for further comments since I do intend to resolve any critical issues of this type.

I see. For what it's worth, I forked the repo and removed all uses of @pure and that seems to have solved #7.

I am aware that there are certain issues associated with it; however, I do not intend on cutting back on the usage of it very much in this repository.

So it causes problems, but you want to keep doing it anyway? Why?

For an internal error as in #7 we could do something in the compiler to avoid the error, but it still indicates that there is a potentially invalid use of @pure.

If this package continues to misuse @pure it is very unlikely to ever work reliably. It's also unclear why you feel the need to use @pure annotations so heavily. Why do you feel that you need them?

Presumably its for performance reasons. I did some microbenchmarks against my fork where I removed all the @pures and various multivector multiplications seem to be about a factor of 10x faster (~100ns -> ~1μs). Surely there are better ways to eke out this extra speed than violating all the rules the compiler holds dear though.

The reason why I use it and made this project is because it's fun to try out and experiment with.

In fortran programming I had used pure methods also, and I see that in Julia it can lead to big performance boosts, but the rest of the design must assume extra constraints.

In this particular package, the entire design constraint is centered around passing around parametric type data ahead of compile time in order to make the generated code more efficient, yet also remain fully flexible for both numerical and symbolic computation with many different dimensional spin algebras.

That is why if you naively remove all usage of @pure you will have drastic performance costs.. in the JuliaLang discourse I have already explained why I used it, but without @pure you are not going to be able to get efficient code on many instances into the function ahead of compile time.

Yes, there is indeed very likely a bug or mistake with my usage, which I need to hunt down. I did not actually have time to double check my usage on every line yet.

At the beginning I was a lot more precise and careful with my usage, but as I had to redesign a lot of things in the package I may have introduced some bugs with the @pure usage.

In this particular package, the entire design constraint is centered around passing around parametric type data ahead of compile time in order to make the generated code more efficient, yet also remain fully flexible for both numerical and symbolic computation with many different dimensional spin algebras.

The complaint is that the way you do achieve relies on invalid assumptions that can at any point come up and bite a user in the ass unexpectedly. There are tried and true ways to perform operations at compile time which don't involve abusing @pure.

As discussed in #7 it was not the @pure macro that was really the issue in this repository. There were some other bugs which have been worked out, but overall the usage of it is relevant here.

Improper Usage of @pure