Verifying ratpoints in Lean: a blueprint

6 The choice of the moduli and the blocks

The sieve of one denominator (Chapter 9) removes a numerator when the row of one of its moduli does not admit it. This chapter says what the moduli are, how the search chooses them, as ratpoints does in sieving_info, and how it runs over the denominators in blocks, between which it may correct its choice, as adapt_primes does. A modulus is a prime of the table of ratpoints, a power of such a prime or a product of such powers; its rows come from tables computed from the curve (Chapter 7). The search ranks the candidates by a model of what each costs and what it says, computed in doubles.

None of this changes the points that the search finds or their order, whatever the values of the doubles: the theorems hold for every choice that the search makes (Section 6.7).

6.1 The table of a prime and the loop of the tables

Write \(f = \sum _{i \le d} c_i x^i\) and \(F(x, z) = \sum _{i \le d} c_i x^i z^{D - i}\) with \(D = 2 \lceil d/2 \rceil \) (Definition 2.1). For a point \((a, y, b)\) with \(\gcd (a, b) = 1\) and a prime \(p \nmid b\), \(F(a, b) \equiv b^D f(a b^{-1}) \bmod p\), and \(b^D\) is the square of a unit, so that \(f(a b^{-1})\) is a square modulo \(p\), zero included; for \(p \mid b\), \(p \nmid a\). These two facts are all that the rows of a prime rest on.

The primes of the sieve are the odd primes below \(1024\) in increasing order (primes.h:29-40), which decide checks; ratpoints chooses its moduli among the \(\mathrm{numPrimes}\) of them below \(2^{\mathrm{maxBitsInPrime}}\), the \(53\) primes up to \(251\) for the \(8\) bits of the default build, whose Makefile passes PRIME_SIZE \(= 8\) in place of the header’s own default \(7\) (rp-private.h:321-323, rp-private.h:325-361). The table of a prime \(p\) has \(p + 1\) entries: entry \(x {\lt} p\) says whether \(f(x)\) is a square modulo \(p\), and entry \(p\) whether there are points at infinity modulo \(p\), that is, whether \(d\) is odd or \(c_d\) is a square modulo \(p\); it is kept with \(p\) and the inverses modulo \(p\) (rp-private.h:542-555). The row of the denominator \(b\) admits the numerator \(a\) when, for \(r = b \bmod p\) and \(x = a \bmod p\), the entry at \(x r^{-1} \bmod p\) is true if \(r \ne 0\), and \(x \ne 0\) if \(r = 0\). This admission test is what the bit of \(a\) in the row of \(r\) says in ratpoints.

ratpoints computes this table in examine_prime, by Horner’s rule with Barrett’s reduction; that it is the table of Definition 6.1 is shown in Chapter 7.

The loop of sieving_info over the primes of the table while fewer than the number to look at, the normalized value of -p, have been looked at (find_points.c:2601-2605). The examination of a prime (find_points.c:2606-2626) gives its table, the coefficients modulo \(p\), its density (the share of the numerators that its rows let through) and a code: \(-1\), the exit, when the curve has no points modulo \(p\) (Chapter 4); \(0\) when the prime says nothing; \(2\) when it says only that numerator and denominator are coprime; \(1\) otherwise. A prime that says something becomes a candidate with its key for the first stage, and its table of rows goes into the store; while fewer than their bound have been found, the coefficients modulo \(p\) give the forbidden divisor of \(p\) (Chapter 8). When the caller left the number of primes to the library, the loop looks at one more prime, at most \(\mathrm{numPrimes}\) in all, while the first two stages would be short of candidates that say more than coprimality (find_points.c:2694-2705).

The contract: for options that look at no more than \(\mathrm{numPrimes}\) primes, an exit means that the curve has no points of any height; otherwise at most \(\mathrm{numPrimes}\) primes are to be looked at, each table that the loop keeps, that of a prime that says something, is that of Definition 6.1 for one of the first \(\mathrm{numPrimes}\) primes, every candidate is a prime of the table with the sieve entry of its table, the forbidden divisors are sound, and the store holds the tables of rows of the primes.

6.2 Prime powers and products

ratpoints also sieves with odd composite moduli from \(9\) up to \(\mathrm{compositeMax} = 64\), products of at most three powers of primes looked at. The tables of a power \(q = p^e\) of a prime, which examine_power computes (Chapter 7), test the numerator against \(f\) when \(p \nmid b\), and the denominator against the reversed polynomial \(\tilde f(t) = \sum _k c_k t^{D - k}\) when \(p \mid b\), since then \(p \nmid a\) and \(F(a, b) \equiv a^D \tilde f(b a^{-1}) \bmod q\). No Chinese remainder theorem is needed for a product: its entry only intersects the conditions of its factors, and each factor needs only that its modulus divides the product.

The tables of a prime power \(q = p^e\) are the bit arrays fsq, bit \(x {\lt} q\) set when \(f(x)\) is a square modulo \(q\), and gsq, bit \(t\) set when \(p \mid t\) and \(\tilde f(t)\) is a square modulo \(q\), with the inverses of the units (rp-private.h:531-534). Its row of \(b\) admits \(a\), for \(r = b \bmod q\) and \(x = a \bmod q\), when bit \(x r^{-1}\) of fsq is set if \(p \nmid r\), and when \(p \nmid x\) and bit \(r x^{-1}\) of gsq is set if \(p \mid r\). A factor is a prime with its table or a prime power with its tables. The sieve entry of a modulus \(m\) holds its factors, the reciprocal of \(m\), with which the sieve reduces, the fields of the packing (Chapter 9), the reference to its table of rows and the kind of its builder (Chapter 7), its density, and the count of its factors, \(0\) for a prime, \(1\) for a prime power and their number for a product (rp-private.h:542-555). Its row of \(b\) admits \(a\) when the row of every factor admits \(a \bmod m\) at \(b \bmod m\); a numerator survives a list of entries for \(b\) when the row of \(b\) of every entry admits it, the interface between the sieve and the check.

The composite moduli that the choice offers (Ratpoints.addModuli) are the odd \(m\) from \(9\) up to \(\mathrm{compositeMax} = 64\) and below \(2^{\mathrm{maxBitsInPrime}}\) whose table the run can pay for and that factor into at most three powers of primes looked at, each among the first \(64\) primes of the table and each saying something (find_points.c:2346-2349); the entry of one that the choice takes is made from the codes of its factors (Ratpoints.makeModulus), with the inverse of \(\mathrm{rbaLength}\) modulo \(m\) by the extended Euclidean algorithm (find_points.c:2414-2426, find_points.c:2434-2470).

Let \(b\) be a natural number and \(a\), \(y\) integers with \(\gcd (a, b) = 1\) and \(y^2 = F(a, b)\).

  1. (1)

    For a prime \(p\), the row of \(b\) in the table of \(p\) admits \(a\).

  2. (2)

    For a prime \(p\) and \(e \ge 1\) with \(p^e\) within the words of the bit arrays, the row of \(b\) in the tables of \(p^e\) admits \(a\).

  3. (3)

    A factor is sound for \(m\) when its row of \(b \bmod m\) admits \(a \bmod m\) for every such \((a, y, b)\), its inverses serve the scaling of the rows by a unit modulo \(m\), its builder has what it needs, and its tables are those of Definition 6.1 or of examine_power. A prime of the sieve that divides \(m\) is a sound factor for \(m\), and so is a power \(p^e \mid m\) of one, for \(e \ge 1\) and \(0 {\lt} m {\lt} 2^{\mathrm{maxBitsInPrime}}\).

  4. (4)

    A sieve entry is sound when its modulus is positive and its row of \(b\) admits \(a\) for every such \((a, y, b)\). An entry of an odd modulus \(1 {\lt} m {\lt} 2^{\mathrm{maxBitsInPrime}}\) with sound factors whose moduli multiply to \(m\), the reciprocal and the bias of \(m\) and an inverse \(x {\lt} m\) of \(\mathrm{rbaLength}\) modulo \(m\) is an entry of \(m\) as the choice makes it (\(\mathrm{IsModulusEntry}\)), whatever its density, count, reference and builder; such an entry is sound.

Proof ▶

The entry of the table of \(p\) at \(t {\lt} p\) is true whenever \(f(t)\) is a square modulo \(p\). For \(p \nmid b\), \(b\) is a unit modulo \(p^e\), and in any commutative ring \(F(k b, b) = b^D f(k)\), with \(b^D\) the square of the unit \(b^{D/2}\); so \(f(a b^{-1})\) is a square. For \(p \mid b\), \(p \nmid a\), which is all that the table of \(p\) asks; modulo \(p^e\), \(F(a, b) = a^D \tilde f(b a^{-1})\), by the same identity with \(a\) and \(b\) exchanged. Zero counts as a square, so a high power of \(p\) dividing \(F(a, b)\) needs no case of its own. A factor whose modulus divides \(m\) reads \(b \bmod m\) and \(a \bmod m\) as it reads \(b\) and \(a\), and an entry admits \(a\) when every factor does.

6.3 The cost model

ratpoints ranks the candidate moduli by what each costs per word of numerators swept, divided by what it says: a modulus of density \(r\) says \(-\log r\).

A candidate modulus carries its density, key and cost, its mask of primes, its sieve entry and the codes of its factors (find_points.c:147-149). Its cost per word swept, in units of an and of the first stage, is a part for every word, the cost of its entry in the list of the moduli of a denominator and, in the first two stages, the costs of its set-up and, for a positive cost of a row, of its table of rows, spread over the \(U\) words and the \(V\) denominators of the run (find_points.c:1220-1235); its key is that cost over \(-\log r\) (find_points.c:1241-1247), with a variant for the first stage (find_points.c:1256-1270). The rule of the first stage always takes the first candidate and then the next while the survivors per word that it removes, weighted by what a surviving word costs further on, exceed a target times its cost (find_points.c:1583-1589); the second stage adds as many moduli as an offset, \(11\) by default, scaled by \(1/(1 + u_0/U)\) (find_points.c:1619-1626). The run is estimated from \(64\) samples of each segment of the loop over the denominators (find_points.c:1939-1944), and the cost of an exact check from the bit lengths of the coefficients and of \(H\) (find_points.c:1285-1318). Every expression is computed in Float, in the order of the operations of the C code; there is no contract.

Remark 6.6 Why no proof reads a double
#

The doubles are values of Lean core’s Float, IEEE 754 binary64 with the arithmetic of the platform, as the C code, compiled without fused multiply-adds and without -ffast-math, computes them; log and pow are the functions of the platform’s C library. No contract of the choice, the blocks or the correction reads the value of a Float: each states only what the natural numbers of the C code do, the cut points staying within the pool and the entries from the end of the second stage on counting no factor, which holds whatever the doubles are, NaN included. So the theorems hold for every choice that the cost model may make, and nothing is proved about the quality of the choice. Since glibc’s log and pow are not correctly rounded and glibc chooses their variant by the processor, the choice of ratpoints depends on the machine; the model calls the same routines of the same library (on glibc an older symbol version of them, with the same values on every argument that occurs here), and the differential tests compare the choices of the two programs on one machine (Chapter 11).

6.4 The choice of the moduli and of the cut points

In sieving_info (find_points.c:2522-3103) the first stage takes candidates in the order of their keys, and so does the second, after the rest is keyed again by the cost of a word that survives the first; each drops every candidate that shares a prime with one taken, so that the moduli of the first two stages are pairwise coprime. The third stage takes the prime of least density left while each pays for itself; the primes left after it are the reserve of the correction (Section 6.6).

The choice for the curve, with the options of the sieve as the search normalizes them (Definition 2.11), takes these steps:

  1. (1)

    the tuning, from -r, -R, -U and -C or the defaults (find_points.c:2569-2575);

  2. (2)

    the first estimate, the loop of the tables with its exit, which returns the prime, and the further forbidden divisors (find_points.c:2522-2532);

  3. (3)

    the estimate and the keys again, the composite moduli with their entries (add_moduli, Section 6.2) and the sort (find_points.c:2808-2812, find_points.c:2816-2823, find_points.c:2826-2826);

  4. (4)

    the first stage, the value of -n or as many as its rule wants, and the second, the value of -N or the offset, each taking candidates in their order (find_points.c:2490-2508, find_points.c:2843-2846, find_points.c:2864-2871);

  5. (5)

    the third stage, up to the value of -P more or while each prime pays for itself, looking at further primes, with the exit, when no prime left pays (find_points.c:2973-2975, find_points.c:3009-3016);

  6. (6)

    the end: the third stage and the reserve sorted by density from \(s_2\) on, the entries of all the candidates, their reciprocals and the store with the tables of rows of every entry (find_points.c:3024-3034).

The result holds the cut points \(s_1 \le s_2 \le s_3\), the ends of the three stages, the number of primes looked at and the estimates of the run (ratpoints.h:395-397), with the sieve entries of all the moduli, the reserve included, their reciprocals, the forbidden divisors, the store of the tables of rows and the switch of the checked loop.

For options that look at no more than \(\mathrm{numPrimes}\) primes, the checked loop only for even degree, and any values of the doubles: an exit of the choice means that the curve has no points of any height; otherwise the result is what the run needs:

  1. (1)

    every sieve entry is sound (Lemma 6.4), with the tables of inverses of its factors and the fields that the packing reads;

  2. (2)

    every entry that counts no factor has a single factor, a table of its modulus \(1 {\lt} p \le 2^{32}\) whose table of inverses gives every nonzero residue below \(p\) a nonzero residue below \(p\), and the reciprocal of \(p\): the shape that the third stage of the sieve needs;

  3. (3)

    the reciprocals are those of the moduli of the entries, in their order, and the store fits all the entries, the reserve included;

  4. (4)

    the first decision, the ends \(s_2\) and \(s_3\) from block \(0\) on, is well formed: \(s_1 \le s_2 \le s_3 \le \) the number of the entries, and every entry from \(s_2\) on counts no factor;

  5. (5)

    the forbidden divisors are sound (Chapter 8), and the checked loop is on only for even degree.

Proof ▶

The loop of the tables gives the state of Definition 6.2. The loops of add_moduli keep a pool (\(\mathrm{ChoicePoolWF}\)) of primes with their entries and of composites whose codes name tables that the choice holds and whose factors multiply to the modulus, so that the entry made from such codes at a position is an entry of its modulus, with the reference and the builder of the position (\(\mathrm{IsChoiceEntry}\), Lemma 6.4). The stages keep every property of the candidates that does not read their key and cost and leave the cut points within the pool; the third stage exits only at a prime modulo which the curve has no points. The doubles enter only through values that no conjunct reads.

So the result of the choice needs no well-formedness of its own. The search normalizes the options of the sieve before the choice (Ratpoints.SieveOptions.normalize), which bounds the number of primes looked at by \(\mathrm{numPrimes}\), so that the theorems need no hypothesis on the options.

6.5 The blocks of the run

The run cuts the steps of the loop of the shape (Chapter 8) into about \(1024\) blocks of consecutive steps and sifts each block with the moduli of the decision of the correction in force for it; the model sifts the blocks in turn, as ratpoints does without threads. No proof depends on the values of the constants of the blocks and of the correction but for the positivity of a block length that the build fixes (Ratpoints.blockLen_pos).

The steps of a shape form segments (find_points.c:3812-3837): the \(k\) of the squares \(k^2\), for each divisor \(s\) of the divisor shape the \(k\) of \(s k^2\), both within \([b_{\mathrm{low}}, b_{\mathrm{high}}]\), or the words of \(64\) denominators, or the denominators. The blocks have length \(\lfloor \mathrm{steps}/1024 \rfloor + 1\), at least \(16\), or the length that the build fixes (find_points.c:3838-3842), and cut each segment into consecutive pieces (find_points.c:3843-3850). A decision of the correction is the first block that it is for, with the ends \(s_2\) and \(s_3\) from that block on; the decision in force for block \(i\) is the last whose first block is at most \(i\) (find_points.c:3908-3910). Block \(i\) (find_points.c:3889-3898) hands each denominator of its steps that passes the tests of the shape to the sieve of one denominator, with the sieve entries cut by the decision in force into the first two stages, the first ending at \(s_1\), and the third; it returns when the callback stops the search, and reports its counters with its decision.

The plan of the blocks is well formed, with blocks of at least one step, and the denominators of the blocks, concatenated in order, are those of the shape from \(b_{\mathrm{low}}\) to \(b_{\mathrm{high}}\) in the order of its loop. When the first decision is for block \(0\), the decision found for block \(i\) is the one in force. Let the callback not have stopped the search, the pieces have at least one bit array, the sieve entries be fit for the run (of the shape that the third stage needs where they count no factor, with the tables of inverses of their factors, their reciprocals in order and the row offsets of the classes; soundness is not needed here), the word of the classes have bit \(j\) set exactly when class \(j\) admits a numerator, the decision in force be well formed, the forbidden divisors be as the shape of the words needs them, \(i\) be below the number of the blocks, \(1 \le b_{\mathrm{low}}\), \(b_{\mathrm{high}} {\lt} 2^{63}\), \(H {\lt} 2^{63}\), and the store fit all the entries. Then block \(i\) keeps the store so and delivers, in order, over the denominators of its steps that pass the tests of the shape, the points that the exact check finds among the numerators that survive the first \(s_3\) entries of its decision, for every choice of the words of the classes and the ranges of the numerators that the packing and the bounds give; and when the callback has not stopped the search within it, its report is a closed form in the arguments of the run, which reads neither the callback nor the flags of the output.

Proof ▶

The plan is arithmetic of the quotients by the length, which is positive. The loop of each shape, for a step with a contract on each denominator, delivers the concatenation of their lists in order; a stop of the callback ends it with the delivery of a prefix, which is that of the whole list (Lemma 10.2). The step is the sieve of one denominator (Definition 9.11) with the entries \(E_{:s_2}\) and \((E_{s_2:})_{:s_3 - s_2}\), which concatenate to \(E_{:s_3}\). Building rows never removes one, so that the store keeps fitting the entries.

6.6 The correction of the moduli

The sieving thread counts, for each block, the words that the first stage sweeps, the bit arrays that survive it, the numerators extracted, those prime to the denominator, those that pass the third stage and the calls of the sieve, all before the exact check, so that the counts depend neither on the flags of the output nor, in a block in which the callback does not stop the search, on the callback. From the rates that the run has shown, adapt_primes fits the survivors per word as a linear function of the product of the densities; with -A 2 or more (not by default) it walks the end \(s_2\) of the second stage up through the entries beyond it, or down, above \(s_1\) and not past a composite modulus; and without -P it computes the end \(s_3\) of the third stage again from the new \(s_2\). A decision made at the fold of block \(j\) is for the blocks from \(j + \mathrm{adaptLag} + 1\) on, with the lag \(\mathrm{adaptLag} = 32\) of RP_ADAPT_LAG (find_points.c:1379-1381).

The counters of the run, the sums of the counters of the blocks folded (Definition 9.1), those from the extracted numerators on restarted when a block sifted with another \(s_2\) is folded (find_points.c:1363-1367). The correction (find_points.c:1383-1388), when the run has swept enough words, sets the next one at twice the words swept and computes the rates, the walk of \(s_2\) under -A 2 or more and the end of the third stage, every double in the order of the C code. The fold of block \(j\) (find_points.c:4114-4127) adds its counters to the state of the correction and appends a decision for the blocks from \(j + \mathrm{adaptLag} + 1\) on when the correction changes an end. The run starts from the decision of the choice for block \(0\), sifts each block with the decisions made before it and folds its report, returns after the block in which the callback stopped the search, and gives the ends \(s_2\) and \(s_3\) of the last block folded, the moduli used (find_points.c:5152-5160).

For any values of the doubles: when the latest decision and the decision with \(s_3 = s_2\) for the end \(s_2\) under which the counters of the run were counted are well formed, so is the decision that the correction makes. A state of the correction is well formed when its first decision is for block \(0\) and its decisions, the latest one and the one for the end of the counters are well formed; the fold of a block whose decision is one of the state keeps it so, adds one block folded and appends decisions only for the blocks from \(\mathrm{folded} + \mathrm{adaptLag} + 1\) on. The state after the folds of the first \(k\) blocks, each with its report in closed form (Lemma 6.10), and the decision of each block are functions of the arguments of the run that read neither the callback nor the flags of the output, well formed when the decision of the choice is.

Proof ▶

The walk up ends at most at the number of the entries and passes only entries from the old \(s_2\) on; the walk down ends at least at \(s_1\) and passes only entries that count no factor; the third stage ends between the new \(s_2\) and the number of the entries. The contracts of these loops state only what the natural numbers do, also where the walks compare costs that may be NaN. The closed form follows by induction on the number of folds.

6.7 Why the choice does not change the output

By the block lemma (Theorem 3.13) and Lemma 6.12, the run delivers, over each denominator \(b\), the points that the exact check finds among the numerators that survive the first \(s_3(b)\) entries, with \(s_3(b)\) independent of the callback and the flags (Ratpoints.Sieve.exists_run_sieveRun_eq_deliver); every entry is sound (Lemma 6.8), so every point survives, whatever the choice and the corrections. The choice changes only which candidates reach the exact check, and so the output under RATPOINTS_NO_CHECK, the numbers of the moduli used and the report under -v (Chapter 10). The statement under RATPOINTS_NO_CHECK that compares its output with that of the exact check (Theorem 3.8) compares two searches with the same options of the sieve and the same region, which differ only in the flags of the output (the second has the exact check on and NO_Y set); neither the choice nor the decisions read these flags or the callback.

With threads (-t) ratpoints sifts the blocks apart in a pool, which the model leaves out (Chapter 11). The lag and thread lemmas (Theorems 3.14 and 3.15) prove that blocks sifted apart, block \(i\) once the blocks before block \(i - \mathrm{adaptLag}\) are folded and each with a store of its own, and delivered in their order give what the run delivers; that the pool of the C code works so is read off the C code, not proved.