The Generalized Gauss Reduction Algorithm

Michael Kaib^{∗} and Claus P. Schnorr

Fachbereich Mathematik / Informatik, Universität Frankfurt Postfach 11 19 32, 60054 Frankfurt a.M., Germany

January 28, 1994

Abstract

We generalize the Gauss algorithm for the reduction of two–dimensional lattices from the l_{2}-norm to arbitrary norms and extend Vallée’s analysis [J. Algorithms 12 (1991), 556-572] to the generalized algorithm.

1 Introduction

Gauss [Ga1801] gave, in the language of qaudratic forms, an algorithm which reduces a basis a, b of a two–dimensional lattice and finds the two successive minima of the lattice. Vallée [Va91] shows that the Gauss reduction algorithm performs at most log_{1+\sqrt{2}}(\frac{2\sqrt{2}}{3} ∥ a ∥_{2}^{2} + ∥ b ∥_{2}^{2}) + 2 many iterations. This bound is optimal up to an additive constant. Vallée also characterizes for the lattice ZZ^{2} the minimal size input bases for which the Gauss algorithm performs exactly k iterations. The bit complexity of the Gauss algorithm has been studied by Schönhage [Sch91] in the language of quadratic forms.

While these results are all for the l_{2}-norm, other norms are important, too. The l_{∞}-norm is the natural norm for integer programming problems. Schnorr [Sch93] reduces the problem of factoring integers to a closest lattice vector problem in the l_{1}-norm. Lovász and Scarf [LS92] propose a generalized basis reduction algorithm that extends the L^{3}-algorithm of A.K. Lenstra, H.W. Lenstra Jr. and L. Lovasz [LLL82] to an arbitrary norm.

∗e-mail: kaib@informatik.uni-frankfurt.de

Our results. We extend the Gauss reduction algorithm from the l_{2}-norm to an arbitrary norm. This generalized Gauss algorithm (gGA) essentially coincides with the Lovász–Scarf algorithm for two–dimensional lattice bases. The gGA finds for any norm the two successive minima of the lattice. Given a reduced basis we exhibit minimal size input bases requiring a given number of iterations. These minimal size input bases represent the worst case inputs. They are universally worst case for all norms for which the given output basis is reduced. They satisfy the same recursion which holds true for the worst case inputs of the centered Euclidean algorithm according to Dupré [Du1846].

We show that the generalized Gauss algorithm terminates for any norm after at most log_{1+\sqrt{2}} (2\sqrt{2} B/λ_{2}) + o(1) many iterations, where B is the maximum of the norms of the two input vectors and λ_{2} is the second successive minimum of the lattice with respect to the given norm.

The paper is organized as follows. In Section 2 we introduce reduced lattice bases. In section 3 we present the generalized Gauss algorithm and its analysis. Section 4 gives complexity bounds for the RAM–model. A preliminary version of this paper has been published by Kaib [Ka91].

2 Geometrical preliminaries

We generalize the concept of reduced lattice bases for lattices of rank 2 to an arbitrary norm ∥ . ∥ on IR^{n}. We use the following three elementary lemmata:

Lemma 1. Let a, b ∈ IR^{n}, a ≠ 0, let F : IR → IR^{n} : ξ ↦ ξa + b describe a line in IR^{n} and f(ξ) = ∥ F (ξ) ∥ . Then f is a convex function.

Lemma 2. Let F : IR → IR^{n} be a line in IR^{n} and ξ_{1}, ξ_{2}, η_{1}, η_{2} be four reals with ξ_{1} < ξ_{2}, η_{1} < η_{2}, ξ_{1} ≤ η_{1}, ξ_{2} ≤ η_{2} . Then ∥ F (ξ_{1}) ∥ ≤ ∥ F (ξ_{2}) ∥ implies ∥ F (η_{1}) ∥ ≤ ∥ F (η_{2}) ∥, and ∥ F (ξ_{1}) ∥ < ∥ F (ξ_{2}) ∥ implies ∥ F (η_{1}) ∥ < ∥ F (η_{2}) ∥.

We will usually apply Lemma 2 in the case ξ_{1} = 0 and ξ_{2} = 1.

Lemma 3. Let M be a closed set in IR^{n} and 0 ∉ M. Then every point in M with minimal norm lies on the boundary of M.

Throughout the paper let (a, b) ∈ IR^{n} × IR^{n} be basis of the two–dimensional lattice L = ZZa + ZZb . We define reduced and well–ordered lattice bases. The reduction algorithm in the next section recurs on well–ordered bases until a reduced basis is found.

Definition. A lattice basis (a, b) is called

reduced if ∥ a ∥ , ∥ b ∥ ≤ ∥ a − b ∥ ≤ ∥ a + b ∥ and

well–ordered if ∥ a ∥ ≤ ∥ a − b ∥ < ∥ b ∥ .

By Lemma 2 ∥ a − b ∥ < ∥ b ∥ implies

∥ b ∥ < ∥ ηa + b ∥ ∀η > 0 .  (1)

Thus (a, b) is well–ordered iff ∥ a ∥ ≤ ∥ a − b ∥ < ∥ b ∥ < ∥ a + b ∥ .

The i–th successive minimum λ_{i} of a lattice L with respect to the norm ∥ . ∥ is defined as the minimal real ρ such that there are at least i linearly independent lattice vectors of norm at most ρ.

Theorem 4. If (a, b) is a reduced basis then ∥ a ∥, ∥ b ∥ are the two successive minima of the lattice L = ZZa + ZZb.

Proof. W.l.o.g. let ∥ a ∥ ≤ ∥ b ∥ . The theorem claims the following:

∥ a ∥ ≤ ∥ ra + sb ∥ for all (r, s) ∈ ZZ^{2} − {(0, 0)} ,

∥ b ∥ ≤ ∥ ra + sb ∥ for all r ∈ ZZ, s ∈ ZZ − {0} .

These inequalities follow from the inequalities

∥ a ∥ ≤ ∥ b ∥ ,
∥ a ∥ ≤ ∥ ra ∥ for all r ∈ ZZ − {0} ,
∥ b ∥ ≤ ∥ ξa + ηb ∥ for all ξ, η ∈ IR with |ξ|, |η| ≥ 1 .  (2)

It is therefore sufficient to prove Inequality 2. For this we show the following

Claim. Consider the four dotted areas in Figure 1. The norm takes its minimum in each of the four dotted areas in the points ±a ± b.

[figure: .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; a; b; a + b; b − a; 0]

Figure 1: Reduced basis (a, b)

Inequality 2 is an immediate consequence of the claim and the reduction conditions ∥ b ∥ ≤ ∥ a ± b ∥ .

Proof of the claim. Each dashed line in the figure contains three lattice points where the middle point has minimal norm, i.e. we have

∥ ±a − b ∥ ≥ ∥ ±a ∥ ≤ ∥ ±a + b ∥ ,
∥ −a ± b ∥ ≥ ∥ ±b ∥ ≤ ∥ a ± b ∥ .

Lemma 2 yields

∥ ±a ± ξb ∥ ≥ ∥ ±a ± b ∥ ≥ ∥ ±a ∥ ,
∥ ±ξa ± b ∥ ≥ ∥ ±a ± b ∥ ≥ ∥ ±b ∥

for ξ ≥ 1. This proves that the points ±a ± b have minimal norm for the dotted lines. By Lemma 3 the norm takes its minimum for each of the four dotted areas on the boundary, i.e. on the dotted lines. This proves the claim. □

3 Analysis of the generalized Gauss algorithm

We extend the Gauss basis reduction algorithm from the l_{2}-norm to an arbitrary norm. We choose the sign of the basis vectors in the algorithm so that the algorithm recurs on well–ordered bases. As a consequence all occurring integral reduction coefficients μ are positive.

The generalized Gauss algorithm (gGA).

INPUT a well–ordered lattice basis (a, b) .

WHILE ∥ b ∥ > ∥ a − b ∥ DO

1. b := b − μa, where the integer μ is chosen to minimize the norm ∥ b − μa ∥ .

2. IF ∥ a + b ∥ < ∥ a − b ∥ THEN b := −b.

3. Swap a and b.

END WHILE

OUTPUT (a, b).

Comments.

1. The exchange in Step 3 produces either a well–ordered or a reduced basis. The algorithm traverses, upon exit of Step 3 (resp. entry of Step 1), a sequence of well–ordered bases until a reduced basis is produced.

2. The algorithm terminates after finitely many steps because the norm of the basis vectors decreases in every, except the last, iteration.

3. To have a well defined algorithm we require to choose in Step 1 the smallest μ that minimizes ∥ b − μa ∥ .

We associate with an input basis the sequence of lattice bases occurring in the algorithm upon exit of Step 3. The bases of this sequence are all well–ordered, except that the final basis is reduced. If (b, c), (a, b) are two consecutive bases in any of these sequences we call (a, b) the successor basis of (b, c) and (b, c) a predecessor basis of (a, b). A well–ordered basis has at most one successor basis but may have infinitely many predecessor bases corresponding to runs of the algorithm with various input bases. If (b, c) is a predecessor basis of (a, b) we call the vector c a predecessor of (a, b). The transition of a well–ordered basis (b, c) via Steps 1-3 to its successor basis (a, b) is of the form

(a, b) := (b, c) ( \begin{matrix}1 & −μ \\ 0 & 1 \end{matrix} ) ( \begin{matrix}1 & 0 \\ 0 & ε \end{matrix} ) ( \begin{matrix}0 & 1 \\ 1 & 0 \end{matrix} )
= (b, c) ( \begin{matrix}−εμ & 1 \\ ε & 0 \end{matrix} ) = ( ε(c − μb), b)  (3)

where ε = ±1 denotes the possible change of sign in Step 2. We see that the predecessor c is of the form c = εa + μb. The following lemma characterizes the predecessors of a well–ordered basis (a, b). It generalizes Lemma 1 of Vallée [Va91].

Lemma 5. If (a, b) is a well–ordered lattice basis then a vector c = εa + μb is a predecessor of (a, b) if and only if either ε = 1, μ ≥ 2 or ε = −1, μ ≥ 3.

Lemma 5 shows that the set of predecessor bases of (a, b) does not depend on the norm, i.e. if (a, b) is well–ordered for two distinct norms then its two sets of predecessors coincide.

Proof. Since (a, b) is well–ordered we have

∥ a ∥ ≤ ∥ a − b ∥ < ∥ b ∥ .  (4)

The predecessor basis (b, c) is well–ordered iff ∥ b ∥ ≤ ∥ b − c ∥ < ∥ c ∥ . We consider the lines

F (ξ) = (1 − ξ) (b − a) + ξ  b
G(ξ) = (1 − ξ)  a  + ξ  b
H^{+}(ξ) = (1 − ξ)  a  + ξ (a + b)
H^{−}(ξ) = (1 − ξ) −a + ξ (b − a)

.

[figure: .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; .; 0; a; b; −a; 2b; a + b; a + 2b; a + 3b; −a + b; −a + 3b; −a + 4b; F; H^{+}; G; H^{−}]

Figure 2: Well-ordered basis (a, b)

Inequality 4 implies:

\begin{aligned}∥ F (0) ∥ & = & ∥ b − a ∥ & < & ∥ b ∥ & = & ∥ F (1) ∥ \\ ∥ G(0) ∥ & = & ∥ a ∥ & < & ∥ b ∥ & = & ∥ G(1) ∥ \\ ∥ H^{−}(0) ∥ & = & ∥ a ∥ & ≤ & ∥ b − a ∥ & = & ∥ H^{−}(1) ∥ . \end{aligned}

Thus by Lemma 2 ∥ F (ξ) ∥ and ∥ G(ξ) ∥ is strictly increasing and ∥ H^{−}(ξ) ∥ is increasing (i.e. non–decreasing) for ξ ≥ 1. This yields a corresponding inequality for H^{+}:

∥ H^{+}(0) ∥ = ∥ G(0) ∥
< ∥ G(1) ∥ = ∥ F (1) ∥
< ∥ F (2) ∥ = ∥ H^{+}(1) ∥ .

We decide for all possible cases of μ and ε = ±1 whether (b, c) is well–ordered.

ε = 1, μ ≤ −1: Then ∥ b − c ∥ = ∥ (1 − μ)b − a ∥ = ∥ H^{−}(1 − μ) ∥ ≥ ∥ H^{−}(2) ∥ = ∥ 2b − a ∥ = ∥ G(2) ∥ > ∥ G(1) ∥ = ∥ b ∥ . Thus (b, c) is not well–ordered.

ε = −1, μ ≤ 0: Then ∥ b − c ∥ = ∥ (1 − μ)b + a ∥ = ∥ H^{+}(1 − μ) ∥ ≥ ∥ H^{+}(1) ∥ = ∥ F (2) ∥ > ∥ F (1) ∥ = ∥ b ∥ . Thus (b, c) is not well–ordered.

μ = 0: Then ∥ c ∥ = ∥ a ∥ < ∥ b ∥ and (b, c) is not well–ordered.

μ = 1: Then ∥ b − c ∥ = ∥ a ∥ < ∥ b ∥ and (b, c) is not well–ordered.

ε = −1, μ = 2: Then ∥ b − c ∥ = ∥ a − b ∥ < ∥ b ∥ and (b, c) is not well–ordered.

ε = 1, μ ≥ 2: Then ∥ c ∥ = ∥ a + μb ∥ = ∥ H^{+}(μ) ∥ > ∥ H^{+}(μ − 1) ∥ = ∥ a + (μ − 1)b ∥ = ∥ b − c ∥ ≥ ∥ H^{+}(1) ∥ = ∥ F (2) ∥ > ∥ F (1) ∥ = ∥ b ∥ . Thus (b, c) is well–ordered.

ε = −1, μ ≥ 3: Then ∥ c ∥ = ∥ −a + μb ∥ = ∥ H^{−}(μ) ∥ ≥ ∥ H^{−}(μ − 1) ∥ = ∥ b − c ∥ ≥ ∥ H^{−}(2) ∥ = ∥ G(2) ∥ > ∥ G(1) ∥ = ∥ b ∥ . Thus (b, c) is well–ordered. □

For the analysis of the algorithm we consider the sequence of well–ordered bases that is traversed upon exit of Step 3. We index this sequence in reverse order (b_{0}, b_{1}), …, (b_{k}, b_{k+1}) so that (b_{k}, b_{k+1}) is the input basis, (b_{0}, b_{1}) is the last well– ordered basis and (b_{−1}, b_{0}) is the reduced output basis. Let ε_{i}, μ_{i} be the coefficients which, according to Equation 3, transform (b_{i}, b_{i+1}) into the successor basis (b_{i−1}, b_{i}). We have

(b_{i−1}, b_{i}) = (b_{i}, b_{i+1}) ( \begin{matrix}−ε_{i}μ_{i} & 1 \\ ε_{i} & 0 \end{matrix} )

(b_{k}, b_{k+1}) = (b_{0}, b_{1}) ( \begin{matrix}0 & ε_{1} \\ 1 & μ_{1} \end{matrix} ) ( \begin{matrix}0 & ε_{2} \\ 1 & μ_{2} \end{matrix} ) ⋯ ( \begin{matrix}0 & ε_{k} \\ 1 & μ_{k} \end{matrix} )

The latter matrix product can be expressed by the generalized continuants which Rieger [R78] has introduced for the analysis of the centered Euclidean algorithm. These polynomials

\left[{x_{2} … x_{n} \atop y_{1} … y_{n}}\right]_{n} ∈ ZZ[x_{2}, …, x_{n}, y_{1}, …, y_{n}]

are recursively defined as

[ ]_{−1} = 0, ^{[ ]}_{0} = 1, _{[y_{1} ]}_{1} = y_{1},

\left[{x_{2} … x_{n} \atop y_{1} … y_{n}}\right]_{n} = y_{1} \left[{x_{3} … x_{n} \atop y_{2} … y_{n}}\right]_{n−1} + x_{2} \left[{x_{4} … x_{n} \atop y_{3} … y_{n}}\right]_{n−2} .  (5)

An easy induction shows that

( \begin{matrix}0 & ε_{1} \\ 1 & μ_{1} \end{matrix} ) ⋯ ( \begin{matrix}0 & ε_{k} \\ 1 & μ_{k} \end{matrix} ) = ( ε_{1} \left[{ε_{3}...ε_{k−1} \atop μ_{2} ... μ_{k−1}}\right]_{k−2} ε_{1} \left[{ε_{3}...ε_{k} \atop μ_{2} ... μ_{k}}\right]_{k−1}_{\left[{ε_{2}...ε_{k−1} \atop μ_{1} ... μ_{k−1}}\right]k−1 \left[{ε_{2}...ε_{k} \atop μ_{1} ... μ_{k}}\right]k} ) .

Hence

b_{k+1} = ε_{1} \left[{ε_{3} … ε_{k} \atop μ_{2} … μ_{k}}\right]_{k−1} b_{0} + \left[{ε_{2} … ε_{k} \atop μ_{1} … μ_{k}}\right]_{k} b_{1} .  (6)

There is a simple formula for the continuants with μ_{j} = 2, ε_{j} = 1:

T_{i} := \left[{1 … 1 \atop 2 … 2}\right]_{i} = \frac{1}{2\sqrt{2}} [(1 + \sqrt{2})^{i+1} − (1 − \sqrt{2})^{i+1}] .  (7)

Simultaneous induction on i, via Equation 5, yields the following inequalities:

Lemma 6. Let μ_{j} ≥ 2 for ε_{j} = 1, and μ_{j} ≥ 3 for ε_{j} = −1. Then

\left[{ε_{2} … ε_{i} \atop μ_{1} … μ_{i}}\right]_{i} ≥ T_{i} , \left[{ε_{2} … ε_{i} \atop μ_{1} … μ_{i}}\right]_{i} ≥ 2 \left[{ε_{3} … ε_{i} \atop μ_{2} … μ_{i}}\right]_{i−1} .

Lemma 7. Every sequence of successive well–ordered bases (b_{0}, b_{1}), …, (b_{k}, b_{k+1}) satisfies ∥ b_{k+1} ∥ ≥ T_{k} ∥ b_{1} ∥ .

Proof. We see from Lemma 6 that the coefficient of b_{1} in Equation 6 is positive and the coefficient of b_{0} has sign ε_{1}. We distinguish two cases:

Case 1. ε_{1} = 1. We have

∥ b_{k+1} ∥ ≥ \left[{ε_{2} … ε_{k} \atop μ_{1} … μ_{k}}\right]_{k} ∥ b_{1} ∥ ≥ T_{k} ∥ b_{1} ∥ .

The first inequality follows from Equation 6 by Inequality 1 since (b_{0}, b_{1}) is well– ordered. The second inequality holds by Lemma 6.

Case 2. ε_{1} = −1. We have ∥ b_{0} ∥ < ∥ b_{1} ∥ since (b_{0}, b_{1}) is well–ordered. Therefore Equation 6 and the triangular inequality yields

∥ b_{k+1} ∥ ≥ (\left[{ε_{2} … ε_{k} \atop μ_{1} … μ_{k}}\right]_{k} − \left[{ε_{3} … ε_{k} \atop μ_{2} … μ_{k}}\right]_{k−1}) ∥ b_{1} ∥ .

The right-hand factor can be simplified to

\left[{ε_{2} … ε_{k} \atop μ_{1} … μ_{k}}\right]_{k} − \left[{ε_{3} … ε_{k} \atop μ_{2} … μ_{k}}\right]_{k−1} = \left[{ε_{2} … ε_{k} \atop (μ_{1} − 1) … μ_{k}}\right]_{k} .

To verify this equation develop its first and last term via Equation 5. Finally the claim follows from Lemma 6 since μ_{1} ≥ 3 . □

We consider the number of iterations of the gGA or equivalently the number of traversed well–ordered bases. We bound this number as a function of B := ∥ b ∥/λ_{2} where b is the largest input vector.

Theorem 8. The gGA performs on input (a, b), for B = ∥ b ∥/λ_{2} → ∞, at most log_{1+\sqrt{2}} (2\sqrt{2} B) + o(1) many iterations where λ_{2} is the second successive minimum of the lattice.

Remark. If the input basis is not well–ordered there may be an extra iteration. The o–term is at most 2 − log_{1+\sqrt{2}}(4\sqrt{2}) ≈ 0.0339 where the maximum occurs in case of a single iteration.

Proof. Let (b_{k}, b_{k+1}) be a well–ordered input basis and (b_{−1}, b_{0}) the output basis. There are k + 1 iterations. Lemma 7 tells us that

T_{k} ≤ \frac{∥ b_{k+1} ∥}{∥ b_{1} ∥} ≤ B .

Equation 7 implies

\frac{1}{2\sqrt{2}} [(1 + \sqrt{2})^{k+1} − (1 − \sqrt{2})^{k+1}] ≤ B ,

and thus we have

k + 1 ≤ log_{1+\sqrt{2}} (2\sqrt{2} B) + o(1)

for B → ∞. □

The bound of Theorem 8 is optimal for all norms:

Theorem 9. Let (b_{−1}, b_{0}) be a reduced basis with ∥ b_{−1} ∥ ≤ ∥ b_{0} ∥ and b_{i+1} = b_{i−1} + 2b_{i} for i = 0, …, k. Then the gGA performs on input (b_{k}, b_{k+1}) exactly k + 1 iterations where k + 1 ≥ log_{1+\sqrt{2}} (2 B) − 1 + o(1) and B = ∥ b_{k+1} ∥/λ_{2}.

Remarks.

• The difference of the upper and lower bound in the above two theorems is 1 + log_{1+\sqrt{2}} (\sqrt{2}) + o(1) ≈ 1.393 + o(1) .

• In the particular case that b_{−1}, b_{0} are the integers b_{−1} = 0, b_{0} = 1 the recursion b_{i+1} = b_{i−1} + 2b_{i} for i = 0, …, k yields, according to Dupré [Du1846], the minimal integers b_{k}, b_{k+1} for which the centered Euclidean algorithm performs exactly k + 1 divisions. Vallée [Va91] has extended this minimality, in the case of the l_{2}-norm, to bases of the lattice ZZ^{2}.

• The novelty in Theorem 9 is that the recursion b_{i+1} = b_{i−1}+2b_{i} for 0 = 1, …, k is valid for all norms, all lattices and all reduced output bases (b_{−1}, b_{0}).

For the proof we characterize the well–ordered predecessor bases (b, c), c = εa + μb , of reduced bases (a, b). This extends Lemma 5.

Lemma 10. Let (a, b) be a reduced basis and c = εa+μb where ε = ±1 and μ ∈ ZZ .

1. If ∥ a ∥ ≤ ∥ b ∥ and (ε, μ) ≠ (−1, 2) then (b, c) is well–ordered iff μ ≥ 2.

2. If ∥ b ∥ ≤ ∥ a ∥ then (b, c) is not well–ordered for μ ≤ 0 and well–ordered for ε = 1, μ > \frac{2λ_{2}}{λ_{1}} − 1 and for ε = −1, μ > \frac{2λ_{2}}{λ_{1}} where λ_{i} = λ_{i}(ZZa + ZZb) .

Proof of Theorem 9. The bases (b_{i}, b_{i+1}) for i = 0, …, k are all well–ordered. This holds by Lemma 10 for i = 0 and by Lemma 5 for i > 0. Hence the gGA performs on input b_{k}, b_{k+1} exactly k + 1 iterations with all reduction coefficients equal to 2, and then finds the reduced basis (b_{−1}, b_{0}). Equation 6 implies

b_{k+1} = T_{k−1}b_{0} + T_{k}b_{1} .

Thus we see from ∥ b_{0} ∥ = λ_{2}, ∥ b_{1} ∥ ≤ 3λ_{2} and Equation 7 that

∥ b_{k+1} ∥ ≤ (T_{k−1} + 3T_{k})λ_{2}
= (1 + \sqrt{2})^{k} \frac{1 + 3(1 + \sqrt{2})}{2\sqrt{2}} λ_{2} (1 + o(1)) ,

hence k ≥ log_{1+\sqrt{2}} (2 B) − 2 + o(1) . □

Proof of Lemma 10. We collect facts that cover all the claims. The basis (b, c) is well–ordered iff

∥ b ∥ ≤ ∥ εa + (μ − 1)b ∥ < ∥ εa + μb ∥ .  (8)

If ∥ a ∥ ≤ ∥ b ∥ the left–hand inequality holds by Theorem 4 iff μ ≠ 1. This inequality is trivial for ∥ b ∥ ≤ ∥ a ∥ .

For the right–hand inequality we consider the line H(ξ) = εa + ξb, assuming that (a, b) is reduced. We have ∥ H(−1) ∥ = ∥ εa − b ∥ ≥ ∥ a ∥ = ∥ H(0) ∥ ≤ ∥ εa + b ∥ = ∥ H(1) ∥ . For μ ≤ 0 Lemma 2 implies

∥ εa + (μ − 1)b ∥ = H(μ − 1) ≥ H(μ) = ∥ εa + μb ∥ .

Hence (b, c) is not well–ordered. This proves all claims in the case μ ≤ 0.

Now let μ ≥ 1. We have

∥ εa + (μ − 1)b ∥ = H(μ − 1) ≤ H(μ) = ∥ εa + μb ∥ .  (9)

If ε = −1 we have ∥ a ∥ ≤ λ_{2} ≤ ∥ c ∥ which shows that

∥ b ∥ = \frac{1}{μ} ∥ a + c ∥ ≤ \frac{2}{μ} ∥ c ∥ .  (10)

If ε = 1 the inequality ∥ b − a ∥ ≤ ∥ a + b ∥ = H(1) ≤ ∥ c ∥ implies

∥ b ∥ = \frac{1}{μ + 1} ∥ (b − a) + c ∥ ≤ \frac{2}{μ + 1} ∥ c ∥ .  (11)

Now assume that the left–hand Inequality 8 holds but (b, c) is not well–ordered. In this case equality must hold in Inequality 9. This implies that (b, c) is reduced and thus ∥ b ∥ = λ_{1}, ∥ c ∥ = λ_{2}. Moreover if ∥ a ∥ ≤ ∥ b ∥ , i.e. the right– hand Inequality 8 does not hold, we have ∥ b ∥ = λ_{2} = ∥ c ∥ , and thus the inequalities above imply μ ≤ 1 for ε = 1 and μ ≤ 2 for ε = −1. On the other hand, if ∥ b ∥ ≤ ∥ a ∥ , Inequality 10 yields μ ≤ 2\frac{λ_{2}}{λ_{1}} and Inequality 11 yields μ ≤ 2\frac{λ_{2}}{λ_{1}} −1 .

The claims in case μ ≥ 1. For those ε, μ where it is claimed that (b, c) is well–ordered we have shown, assuming that (b, c) is not well–ordered, an inequality excluding this μ. Moreover, in the case that ∥ a ∥ ≤ ∥ b ∥ , we have shown that (b, c) is not well–ordered if μ = 1. □

Remark. In the indefinite cases of Lemma 10, i.e.,

• ∥ a ∥ ≤ ∥ b ∥ , ε = −1, μ = 2 ,

• ∥ b ∥ ≤ ∥ a ∥ , ε = −1, 1 ≤ μ ≤ 2\frac{λ_{2}}{λ_{1}} ,

• ∥ b ∥ ≤ ∥ a ∥ , ε = 1, 1 ≤ μ ≤ 2\frac{λ_{2}}{λ_{1}} − 1 ,

the above proof shows that (b, c) is either reduced or well–ordered. Both reduced and well–ordered bases do actually occur, as the norm and the lattice vary, in each of the three cases. However there is only one indefinite case for the Euclidean norm, namely ∥ b ∥ ≤ ∥ a ∥ and ε = −1, μ = 1 .

4 Time bounds

The generalized Gauss algorithm described in the last section needs acces to a norm oracle which for given a ∈ IR^{n} outputs ∥ a ∥ . We give time bounds for the RAM model with the arithmetic operations multiplication, division, addition, subtraction, comparison and next integer computation at unit costs. We count for steps arithmetic steps and oracle calls. In this section we prove the following

Theorem 11. Given an oracle for an arbitrary norm ∥ . ∥ there is an algorithm which ∥ . ∥-reduces a given basis a, b ∈ IR^{n} using O(n log (n+λ_{2}/λ_{1}) + log B) many steps where B = max(∥ a ∥, ∥ b ∥)/λ_{2} .

Efficient ∥ . ∥-reduction. For an efficient reduction of a basis a, b ∈ IR^{n} in an arbitrary norm ∥ . ∥ we first reduce a, b in the norm corresponding to a suitable inner product <, > and we subsequently reduce the resulting basis in the ∥ . ∥-norm. We initially perform a <, >-reduction since it only costs O(1) arithmetic steps per iteration.

The inner product <, > is chosen so that {x ∈ IR^{n}| ∥ x ∥ ≤ 1} is spherical in the sense that max_{x,y∈IR^{n}} \frac{∥x∥ <y,y>^{1/2}}{∥y∥ <x,x>^{1/2}} = O(n^{1.5}) . The existence of <, > follows from [Le83], see construction of τ in Section 2, pp. 542 ff. We assume that the inner product is given, we do not count the steps for producing it. The constant B_{<,>} = max (< a, a >^{1/2}, < b, b >^{1/2})/λ_{2,<,>} satisfies B_{<,>} = O(n^{1.5} B) .

The initial <, >-reduction in O(n + log B) arithmetic steps. In each iteration

we transform the Gram matrix G = ( \begin{matrix}< a, a > & < a, b > \\ < b, a > & < b, b > \end{matrix} ) and the transformation

matrix H ∈ GL_{n}(ZZ) satisfying (a_{current}, b_{current}) = (a_{input}, b_{input})H as G := S^{⊤}GS,

H := HS where S = ( \begin{matrix}−εμ & 1 \\ ε & 0 \end{matrix} ) and μ is the integer closest to \frac{<a,b>}{<a,a>} . Each

iteration requires 6 multiplications, 6 subtractions, 1 division and 1 next integer computation. The initial (resp. final) transformations of (a, b) into G (resp. back from H into (a, b) ) require 7n multiplications and 5n − 3 additions. According to Theorem 8 the entire <, >-reduction of a, b is done in O (log B_{<,>}) = O (log (B +n)) iterations.

Computing μ in O(1) resp. O(log λ_{2}/λ_{1}) oracle steps. Let μ : IR^{n} × IR^{n} ⟶ ZZ denote the function that minimizes ∥ b − μ(a, b) a ∥ . For a given well–ordered basis a, b we have to compute μ = ∥ b ± x ∥/∥ a ∥ where (x, a) is the successor basis of (a, b). By the inequality |μ − ∥ b ∥/∥ a ∥| ≤ ∥ x ∥/∥ a ∥ we can compute μ, via the bisection method, using O(log ∥ x ∥/∥ a ∥) oracle steps. Except for the final iteration we always have ∥ x ∥ < ∥ a ∥ and the step bound is O(1). For the final iteration we have ∥ x ∥/∥ a ∥ ≤ λ_{2}/λ_{1}.

The final ∥ . ∥-reduction in O(n log (n+λ_{2}/λ_{1})) steps. It follows from Theorem 8 that the final ∥ . ∥-reduction requires at most log_{1+\sqrt{2}} (2\sqrt{2} max_{x,y∈IR^{n}} \frac{∥x∥ <y,y>^{1/2}}{∥y∥ <x,x>^{1/2}} ) + 1 + o(1) = O(log n) many iterations. Every iteration, except the final one, requires O(1) norm computations and O(n) arithmetic steps. The final iteration costs O(log λ_{2}/λ_{1}) norm computations and O(n log λ_{2}/λ_{1}) arithmetic steps.

The case of the l_{1}(l_{∞})-norm. There are particularly efficient algorithms to compute μ for the l_{1}- and l_{∞}-norm. For the l_{1}-norm the real t minimizing ∥ b − ta ∥_{1} is the generalized median, with weights |a_{i}|, of the component fractions b_{i}/a_{i} for i = 1, …, n which can be computed using O(n) arithmetic steps. For the l_{∞}-norm the graph of the function ∥ b − ta ∥_{∞} with real indeterminate t is the maximum polygon of the 2n lines ±(b_{i}−ta_{i}) . We sort, using O(n log n) arithmetic steps, these lines in order of descending gradient. A subsequent scan of the lines computes, using O(n) arithmetic steps, the vertices of the polygon and in particular its minimal point which yields the real t that minimizes ∥ b − t a ∥_{∞}. Details can be found in [KS93].

Hence the l_{1}(l_{∞})-norm reduction of a, b takes at most O(log B + n log n) arithmetic steps where B = max (∥ a ∥, ∥ b ∥)/λ_{2} and ∥ . ∥ is the l_{1}(l_{∞})-norm.

Acknowledgments

We are grateful to Joos Heintz for encouragement and critical comments.

References

[Da93] H. Daudé: Des fractions continues a la réduction des réseaux: Analyse en moyenne. Thèse de doctorat, Université de Caen 1993.

[Du1846] A. Dupré: Sur le nombre de divisions à effectuer pour obtenir le plus grand commun diviseur entre deux nombres entiers. J. de Math., vol. 11 (1846), pp. 41-64.

[Ga1801] C.F. Gauss: Disquisitiones Arithmeticae. Leipzig 1801. German translation: Untersuchungen über die höhere Arithmetik. Springer, Berlin 1889. (reprint: Chelsea, New York, 1981.)

[Ka91] M. Kaib: The Gauß Lattice Basis Reduction Algorithm Succeeds With Any Norm. Proceedings of the FCT’91, Springer Lecture Notes on Computer Science, vol. 529 (1991), pp. 275-286.

[KS93] M. Kaib, C.P. Schnorr: The Generalized Gauss Reduction Algorithm. Technical Report. Universität Frankfurt 1993.

[Le83] H.W. Lenstra, Jr: Integer programming with a fixed number of variables. Mathematics of Operations Research, Vol. 8, No. 4, (1983), pp. 538-548.

[LLL82] A.K. Lenstra, H.W. Lenstra, Jr. and L. Lovász: Factoring polynomials with rational coefficients. Math. Annalen 261 (1982), pp. 515-534.

[LS92] L. Lovász, H. Scarf: The Generalized Basis Reduction Algorithm. Mathematics of Operations Research, vol. 17, No. 3 (1992), pp. 754-764.

[R78] G.J. Rieger: Über die mittlere Schrittzahl bei Divisionsalgorithmen. Math. Nachr. 82 (1978), pp. 157-180.

[Sch93] C.P. Schnorr: Factoring Integers and Computing Discrete Logarithms via Diophantine Approximation. In: Advances in Computational Complexity, Ed. Jim–Yi Cai, DIMACS Series in Discrete Mathematics and Theoretical Computer Science, AMS (1993), pp. 171-181.

[Sch91] A. Schönhage: Fast Reduction and Composition of Binary Quadratic Forms. In: Proc. ISSAC 1991, Ed. S.M. Watt, ACM 1991, pp. 128-133.

[Va91] B. Vallée: Gauss’ Algorithm Revisited. Journal of Algorithms 12 (1991), pp. 556-572.

[VF90] B. Vallée, Ph. Flajolet: The Lattice Reduction Algorithm of Gauss: An Average Case Analysis. Proc. 31st IEEE Symposium on Foundations of Computer Science, 1990, pp. 830-842.