`ElementaryFunctions` should require arithmetic operators
NevinBR opened this issue · comments
In order to write generic mathematical functionality against the ElementaryFunctions protocol, it is necessary to be able to use basic arithmetic. This includes the infix operators (+, -, *, /) and the prefix operators (+, -).
Therefore, the ElementaryFunctions protocol should either refine SignedNumeric, or if it is intended that non-numeric types be able to conform, the arithmetic operators should be included in the protocol.
A protocol does not need to be implementable in terms of itself. It's reasonable to want to add this requirement, but not for this reason.
That said, I weakly don't think that we should add this requirement. In the most abstract sense, the mathematical requirement for elementary functions to make sense is that you can evaluate a power series, which means you have addition, subtraction, exponentiation, multiplication by rational constants, and some notion of convergence. It does not require that you have all the requirements of Numeric, so adding that conformance would be a mistake.
I'll add that SignedNumeric/Numeric isn't really applicable to non-scalar types, including SIMD vectors and tensors which should conform to ElementaryFunctions.
The
Numericprotocol provides a suitable basis for arithmetic on scalar values, such as integers and floating-point numbers.
AdditiveArithmetic conformance would be reasonable to add, though I'm not sure how beneficial it would be. I'd certainly be willing to add it if someone has a use case.
A protocol does not need to be implementable in terms of itself. It's reasonable to want to add this requirement, but not for this reason.
I am not talking about implementing the protocol, I am talking about using it. Writing generic code that will work for any type which conforms to ElementaryFunctions.
We don't need to give a name to every useful point in the protocol hierarchy. ElementaryFunctions & Numeric is a perfectly reasonable thing to use in a constraint. If it is frequently used, then it would merit a name. Simply making that point ElementaryFunctions isn't a good option for the reasons Richard and I mentioned up thread.
I think we are talking past each other.
The named points in a protocol hierarchy should be a subset of the useful points. In other words, if a point in the hierarchy is not useful, then it should not be named.
I posit that ElementaryFunctions is not useful in its current form. We cannot write any useful generic algorithms against ElementaryFunctions, because basic arithmetic is unavailable. Therefore, no protocol should exist with the exact set of requirements that ElementaryFunctions has.
Moreover, the definition of “elementary function”, as seen on Wikipedia, is:
In mathematics, an elementary function is a function of one variable which is the composition of a finite number of arithmetic operations (+ – × ÷), exponentials, logarithms, constants, and solutions of algebraic equations (a generalization of nth roots).
We are not currently modeling arbitrary roots of polynomials, although it would be reasonable to do so if and when we model polynomials at all. It is not reasonable to exclude the basic arithmetic operations, because they are in a very real sense the most elementary of all.
Wikipedia is describing a slightly different abstraction than the protocol does; mapping mathematical concepts to human language has some ambiguity. The elementary functions make perfect sense in a bunch of domains that should not conform to Numeric, so it doesn't make sense to add that constraint.
AdditiveArithmetic is fine. Something stronger than AdditiveArithmetic but weaker than Numeric would also be fine, but that intermediate protocol doesn't exist. If you throw up a PR adding AdditiveArithmetic, I'll merge it.
I like adding an AdditiveArithmetic refinement. Should this be discussed on Swift Evolution?
swift-numerics doesn't use evolution. This (and the PR) is the discussion =)
The elementary functions make perfect sense in a bunch of domains that should not conform to
Numeric, so it doesn't make sense to add that constraint.
Sure, that’s fine. The arithmetic operators are still necessary.
Also, note that exp and log turn addition into multiplication and vice-versa. Thus if we allow addition, then multiplication can be written as .exp(.log(x) + .log(y)), and if we allow subtraction, then division can be written as .exp(.log(x) - .log(y).
Since we agree on AdditiveArithmetic, it follows that the elementary functions necessarily support multiplication and division. Thus, ElementaryFunctions should include the * and / operators.
Also, note that
expandlogturn addition into multiplication and vice-versa
This is only true if multiplication is commutative. Also, it turns addition into a multiplication, but it's not necessarily the case that all types conforming to ElementaryFunctions want to bind that multiplication to the * operator, or even that they want to have a * operator at all; this is an apparently simple change with pretty far-reaching implications.
or even that they want to have a
*operator at all
Could you provide an example of a realistic type which wants to conform to ElementaryFunctions, but does not want to provide a * operator?
A real Matrix2D type should conform to ElementaryFunctions, but we do not necessarily want it to have a * operator.
A real
Matrix2Dtype should conform toElementaryFunctions, but we do not necessarily want it to have a*operator.
I strongly disagree. 2D matrices definitely have a multiplication, and should definitely use the established spelling for it.
and should definitely use the established spelling for it.
Clearly, the established spelling of matrix multiplication is not *.
The established spelling of multiplication is *.
Matrix multiplication is multiplication, therefore it should definitely use the established spelling for multiplication.
One thing to point out that is maybe not clear, that is causing you to talk past each other somewhat:
-
@rxwei (like most ML people) is talking about a matrix type to conform elementwise, so exp(a)_ij is exp(a_ij).
-
@NevinBR is talking about a matrix type that conforms as an operator, where exp() relates the Lie algebra to the Lie group. (This is a great example of a case where exp(a+b) is not exp(a)exp(b), btw.)
If this protocol implies operators, then componentwise conformance makes * the Hadamard product, rather than the usual matrix product (and operator conformance makes * the "usual" matrix product).
There are a bunch of ways in which we might resolve this in the long term. I don't want to lock us into any one of them yet, which is a big part of why I am not going to put * and / on this protocol (also the fact that these functions require only power associativity and a normed vector space structure to exist, which isn't strong enough to merit the * operator, in my opinion).
Nevin, you can continue to argue that it "should" be there if you like, but the potential downside to adding it (locking in API decisions that prevent future types from conforming) are significantly larger than the downside to holding it back for now (some generic code needs an extra constraint), which tilts the engineering balance clearly in favor of keeping the status quo. And as I said upthread, I am happy to take a PR adding AdditiveArithmetic, if you'd like to put one up.
Thank you for the detailed explanation Steve.