Minor inconsistency: is_extra_strong_lucas_pseudoprime(5) returns 0
trizen opened this issue · comments
Hi. There seems to be a minor inconsistency between:
Math::Prime::Util::is_extra_strong_lucas_pseudoprime(5): returns0Math::Prime::Util::GMP::is_extra_strong_lucas_pseudoprime(5): returns1
Example:
use 5.014;
use Math::Prime::Util;
use Math::Prime::Util::GMP;
say Math::Prime::Util::is_extra_strong_lucas_pseudoprime(5); #=> 0
say Math::Prime::Util::GMP::is_extra_strong_lucas_pseudoprime(5); #=> 1Probably related to:
say join ' ', Math::Prime::Util::lucas_sequence(5, 1, -1, 4); #=> 0 0 1
say join ' ', Math::Prime::Util::GMP::lucas_sequence(5, 1, -1, 4); #=> 3 2 1Also:
say join ' ', Math::Prime::Util::lucas_sequence(1001, -4, 4, 50); #=> 57 695 562
say Math::Prime::Util::lucasu(-4, 4, 50) % 1001; #=> 173
say Math::Prime::Util::lucasv(-4, 4, 50) % 1001; #=> 827
Both issues should be fixed with the latest push (but this isn't complete yet -- the PP code needs to be fixed as well as looking into the third issue).
The issue with is_extra_strong_lucas_pseudoprime(5) was interesting. The GMP front end is completely bypassing the check, but that's easy to override for testing, which showed the actual GMP code is fine. The C code was checking the jacobi symbol to see if it had a trivial factor, but it turns out for the case of 5, said trivial factor was 5 itself. So replaced the naive "return if jacobi(D,n) == 0" with a followup gcd check, so we only return 0 if (D|n)==0 AND 1 < gcd(D,n) < n.
The second issue is actually unrelated, and there were multiple issues, including a small one in the GMP code (it wasn't modding the return Q value if k=0). These were with k=0, k=1, n=5, and n even. The outputs match GMP now.
For the third issue, it look like there is more work to do. GMP dies with a D=0 error. This is not only undocumented, but the standard C code handles this. In addition, a simple check shows lots of examples where these aren't matching, not only the D=0 case (e.g. 13,3,-1,35).
In theory I should do a big refactor of the lucas code, as there are so many variations between the C, GMP, Perl code plus all the variants (modular with multiple optimizations, modular for even n, non-modular with some optimizations, etc.). They were written across many years. Go through each one noting the number of mulmods/muladds required.
Just in case it may be useful, here's some pseudocode for computing the Lucas sequences U_n(P,Q) and V_n(P,Q) modulo m, with the optimization of computing U_n(P,Q) from V_n(P,Q) when D = P^2 - 4*Q is coprime to m:
func _lucasVmod(P, Q, n, m) {
var (V1, V2) = (2, P)
var (Q1, Q2) = (1, 1)
for bit in (n.as_bin.chars) {
Q1 = (Q1*Q2)%m
if (bit) {
Q2 = (Q1*Q)
V1 = (V1*V2)
V2 = (V2*V2)%m
V1 -= P*Q1
V2 -= 2*Q2
V1 = V1%m
}
else {
Q2 = Q1
V2 = (V2*V1)
V1 = (V1*V1)%m
V2 -= P*Q1
V1 -= 2*Q2
V2 = V2%m
}
}
return (V1%m, V2%m, (Q1*Q2)%m)
}
func lucasUVmod(P, Q, n, m) {
if (n == 0) {
return (0, 2, 1)
}
if (n < 0) {
return (NaN, NaN, NaN)
}
var(U1 = 1)
var(V1 = 2, V2 = P)
var(Q1 = 1, Q2 = 1)
var D = (P*P - 4*Q)
# When D is coprime to m, compute U_n(P,Q) from V_n(P,Q)
if (gcd(D, m) == 1) {
#say "# Using Lucas V sequence...";
var (V1, V2, Q1) = _lucasVmod(P, Q, n, m)
V2 *= 2
V2 -= V1*P
V2 *= invmod(D, m)
V2 %= m
return (V2, V1, Q1)
}
var s = n.valuation(2)
n >>= s+1
for bit in (n.as_bin.chars) {
Q1 = (Q1 * Q2)%m
if (bit) {
Q2 = (Q1 * Q)%m
U1 = (U1 * V2)%m
V1 = (V2*V1 - P*Q1)%m
V2 = (V2*V2 - 2*Q2)%m
}
else {
Q2 = Q1
U1 = (U1*V1 - Q1)%m
V2 = (V2*V1 - P*Q1)%m
V1 = (V1*V1 - 2*Q2)%m
}
}
Q1 = (Q1 * Q2)%m
Q2 = (Q1 * Q)%m
U1 = (U1*V1 - Q1)%m
V1 = (V2*V1 - P*Q1)%m
Q1 = (Q1 * Q2)%m
s.times {
U1 = (U1 * V1)%m
V1 = (V1*V1 - 2*Q1)%m
Q1 = (Q1 * Q1)%m
}
return (U1, V1, Q1)
}
say [lucasUVmod(-4, 4, 50, 1001)] #=> [173, 827, 562]
say [lucasUVmod(-4, 7, 50, 1001)] #=> [87, 457, 595]
say [lucasUVmod(1, -1, 50, 1001)] #=> [330, 486, 1]
say [lucasUVmod(1, -1, 4, 5)] #=> [3, 2, 1]References:
- On Lucas Sequences Computation: https://file.scirp.org/pdf/IJCNS20101200011_90376712.pdf
- Efficient computation of full Lucas sequences: https://archive.is/2c0Vz (second link)
Aside:
The argument order I use for lucas_sequence is stupid and I wish I could change it. Alternately, since other than some primality tests, almost nobody cares about the Qk values, make:
u = lucasu(P,Q,k)
v = lucasv(P,Q,k)
u = lucasumod(P,Q,k,n)
v = lucasvmod(P,Q,k,n)
(u,v) = lucasuv(P,Q,k)
(u,v) = lucasuvmod(P,Q,k,n)
Albeit solving the issue by creating 4 new functions has its own problems. I suppose the last two could return Qk as well, though really just powmod(Q,k,n) would get the value if they need. Thoughts?
The GMP code's issues were with the Qk return value when k=0 and some unnecessary restrictions on the values. The latest commit matches everything I've tested. For the GMP module internals:
The function lucasuv in essentially identical to the Joye-Quisquater algorithm, which is what is used for the lucasu and lucasv calls in that module (they both call the same function and return whichever one was requested).
The alt_lucas_seq is the same but with modulos.
The main path used with odd n is more optimized with a couple special cases as well. I've got a todo to compare the different routines, including constructing U. The various primality tests have even more streamlined code . It looks like I wrote on my pseudoprimes web page about the shortcut for computing U_k being described in Crandall and Pomerance. It's implemented in the optimized lucas_seq code for Q=1, which is what most of the primality tests use, and it's substantially faster (2 mulmods per bit vs. 3-7 for the other paths).
Regarding paper 1 (Koval), the author doesn't mention that if you can't do a modular inverse to get U, you're out of luck. I assume because they always use a prime modulus since their focus is on cryptography. Your code takes that into account, as does the Q=1 path I use.
For the Math::Prime::Util code, it looks like there are some more issues.
lucasu in Math::Prime::Util is just the generic code like you have with overflow detection scattered throughout.
The lucasv function is a reduced version of lucasu. It's not identical to the Koval paper because they skip the final optimization step (the loops from s to 0 can be done in half the time), and of course we don't need U calculated at the end.
The general code in "alt_lucas_seq" is the basic algorithm (Joye-Quisquater), but with a fast path for P=1 Q=-1.
lucas_seq has fast paths and a different algorithm that requires n odd.
It looks like the actual Lucas computations in the C and C+GMP code were unchanged. The issues mostly were around modulos in the prefaces, especially with Qk. There was an issue around the D=0 case (the old GMP code bypassed it all by just asserting that that wasn't acceptable).
The PP code still needs to be bought into line, and I'll get the more comprehensive xt test checked in also.
Thanks! I confirm the fixes. Great work as usual! Thank you also for the detailed explanations.
I'm positive towards the idea of having two separate functions for the modular Lucas U and V:
u = lucasumod(P,Q,k,n)
v = lucasvmod(P,Q,k,n)
Additionally, if you plan to keep the lucas_sequence function around, then lucasuvmod would be superfluous. But, one solution is to remove the documentation for the lucas_sequence (without removing the function itself, as this would break a lot of existent code) and add lucasuvmod as the alternative with reordered parameters (that simply calls lucas_sequence under the hood), so new code will use this new function.
Returning 3 values from lucasuvmod is generally a good idea, especially if the third value (Q_k) is computed for free.
Regarding a lucasuv function, I don't really have an opinion about it. Maybe it can be added for completeness.
The latest commit seems to fix everything mentioned here.
I'll be adding the new functions to the GMP code soon.
After that, looking at indexing the different implementations, then benchmarking.