Smooth transfer of Kloosterman integrals

SMOOTH TRANSFER OF
KLOOSTERMAN INTEGRALS
HERVÉ JACQUET
Abstract
We establish the existence of smooth transfer between absolute Kloosterman integrals
and Kloosterman integrals relative to a quadratic extension.
Contents
1. Introduction . . . . . . . . . . . . . .
2. Orbital integrals in M(m × m, F) . . .
3. Weil formula: The split case . . . . . .
4. Inversion formula for the orbital integrals
5. Orbital integrals in H (n × n, E/F) . .
6. Weil formula: The twisted case . . . . .
7. Inversion formula in the Hermitian case
8. Smooth matching . . . . . . . . . . .
9. Application to the fundamental lemma .
10. Concluding remarks . . . . . . . . . .
References . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . .
121
123
128
130
138
141
144
146
148
150
151
1. Introduction
Let E/F be a quadratic extension of local non-Archimedean fields, let η : F × →
{±1} be the corresponding quadratic character, and let ψ : F → C× be a nontrivial character. Let Nm be the subgroup of upper triangular matrices in GL(m) with
unit diagonal, and let Am be the group of diagonal matrices. We define a character
P
θ : Nm (F) → C× by θ(n) = ψ
n i,i+1 . The group Nm (F) × Nm (F) operates
on GL(m, F) by g 7→ t n 1 gn 2 , and the orbits that intersect Am (F) form a dense
open subset. We define the diagonal orbital integrals of a smooth function of compact
support 8 on GL(m, F):
Z
(8, ψ : a) :=
8( t n 1 an 2 )θ(n 1 n 2 ) dn 1 dn 2 .
(1)
Nm (F)×Nm (F)
DUKE MATHEMATICAL JOURNAL
c 2003
Vol. 120, No. 1, Received 28 May 2002. Revision received 5 November 2002.
2000 Mathematics Subject Classification. Primary 11F70; Secondary 11F72, 11L05.
Author’s work partially supported by National Science Foundation grant number DMS-9619766.
121
122
HERV É JACQUET
Likewise, we let H (m × m, E/F) be the space of Hermitian matrices m × m and
set Sm (F) = H (m × m, E/F) ∩ GL(m, E). We define a character n 7→ θ (nn) of
P
Nm (E) by θ (nn) = ψ
(n i,i+1 + n i,i+1 ) . The group Nm (E) operates on Sm (F) by
s 7→ t nsn, and the orbits that intersect Am (F) form a dense open subset of Sm (F).
We define the diagonal orbital integrals of a smooth function of compact support 9
on Sm (F):
Z
(9, E/F, ψ : a) :=
Nm (E)
8( t nan)θ(nn) dn.
(2)
We say that 8 matches 9 for ψ if for every a ∈ Am (F),
(8, ψ : a) = γ (a, ψ)(9, E/F, ψ : a),
where
γ (a, ψ) := η(a1 )η(a1 a2 ) · · · η(a1 a2 , . . . , am−1 ) if a = diag(a1 , a2 , . . . , am ).
In this paper we prove that for every 8 there is a matching 9, and conversely.
Suppose that E/F is unramified, and suppose that the conductor of ψ is O F .
Let 80 be the characteristic function of the set of matrices with integral entries in
M(m × m, F). Similarly, let 90 be the characteristic function of the set of matrices
with integral entries in H (m × m, E/F). The fundamental lemma asserts that the
restriction of 80 to GL(m, F) matches the restriction of 90 to Sm (F). This has been
established by Ngô [10] in the case of positive characteristic.
In general, if 8 matches 9, then there are similar relations between the other
orbital integrals (see [3], [4]).
Within the context of the relative trace formula, the problem at hand is the analogue of the transfer problem for the ordinary trace formula. Indeed, when E/F is
an extension of function fields, the result presented in this paper, together with the
fundamental lemma of Ngô [9], [10], implies the existence of global identities of the
form
Z
X
θ(n 1 n 2 ) dn 1 dn 2
8( t n 1 ξ n 2 )
Nm (F)×Nm (F)\Nm (FA )×Nm (FA )
Z
=
Nm (E)\Nm (E A )
ξ ∈Gl(n,F)
θ(nn) dn
X
9( t nξ n)θ(nn) dn .
ξ ∈Sm (F)
In turn, the identities are a crucial step in extending the results of [5] to GL(m).
Our method is first to linearize the problem by replacing GL(m, F) and Sm (F)
by the vector spaces M(m × m, F) and H (m × m, E/F), respectively. We then establish a simple relation between the orbital integrals of a function on either space
and its Fourier transform. It follows that if 8 matches 9, then the Fourier transform
SMOOTH TRANSFER OF KLOOSTERMAN INTEGRALS
123
of 8 matches the Fourier transform of 9. This proves the existence of many pairs of
matching functions and, in turn, is used to prove the existence of the transfer in a very
simple way.
Our method is suggested by the work of J. Waldspurger [13]. Waldspurger considers orbital integrals in the context of the Lie algebras and derives the existence of
the transfer from the fundamental lemma (assumed to hold) and global identities. He
uses the Fourier transform on the Lie algebras in an essential way. It is reasonable to
ask whether, conversely, the results of the paper can be used to prove the fundamental
lemma in the context of the relative trace formula. We answer this question in the
affirmative for the case of m = 2, 3 in Section 9. In a forthcoming paper (see [7]), we
use the previous results to prove the fundamental lemma for all m.
2. Orbital integrals in M(m × m, F)
We now consider the action of Nm (F) × Nm (F) on M(m × m, F) given by r 7→
t n r n . We denote by θ the algebraic character θ : N (F) → F defined by
1 2
0
0
θ0 (n) =
i=m−1
X
n i,i+1 .
i=1
We say that an element x ∈ M(m × m, F) (or its orbit) is relevant if the character
(n 1 , n 2 ) 7→ θ0 (n 1 n 2 ) is trivial on the stabilizer of x in N (F) × N (F).
We let 1i (x), 1 ≤ i ≤ m, be the determinant of the matrix obtained by removing
the last m − i rows and the last m − i columns of a matrix x. In particular, 1m (x) =
det x. The functions 1i are invariants of the action of Nm (F) × Nm (F). For 1 ≤ i ≤
m − 1, we denote by Oi the set defined by 1i 6= 0. We denote by Bm the group of
upper triangular matrices in GL(m), by Am the group of diagonal matrices, and by
α1 , α2 , . . . , αm−1 the simple roots of Am corresponding to Bm . We denote by Nα the
root group corresponding to a root α. We identify the Weyl group of Am with the
group Wm of permutation matrices, and we let Am be the set of diagonal matrices in
M(m × m, F). We often drop the index m from the notation.
PROPOSITION 1
In M(m × m, F) the set defined by 1m = 0, 1m−1 = 0 contains no relevant orbit.
Proof
We begin with an elementary lemma.
LEMMA 1
Any r ∈ M(m × m, F) can be written (in possibly several ways) in the form
r = t nwb
124
HERV É JACQUET
with w ∈ Wm , n ∈ Nm (F), and b an upper triangular matrix.
Proof
We let
F0 = {V0 ⊂ V1 ⊂ V2 ⊂ · · · ⊂ Vn }
be the canonical flag. Thus Vi is the space spanned by the first i vectors of the canonical basis. Let r ∈ M(m × m, F) be given. Consider the sequence of subspaces
r V0 ⊆ r V1 ⊆ r V2 ⊆ · · · ⊆ r Vm .
It need not be a flag, but, inductively, it is easy to show that there is a flag
F = {U0 ⊂ U1 ⊂ U2 ⊂ · · · ⊂ Um }
such that
r Vi ⊆ Ui ,
0 ≤ i ≤ n.
By the standard Bruhat decomposition,
F = t nw F0
for suitable w and n ∈ Nm (F). Thus
(t nw)−1r Vi ⊆ Vi ,
0 ≤ i ≤ n.
Since (t nw)−1r stabilizes F0 , it is an upper triangular matrix. This proves the lemma.
To prove our assertion, it suffices then to consider the case of a relevant element of
the form
r = wb
with w ∈ Wm and b an upper triangular matrix such that det b = 0. Our task is then
to show that 1m−1 (wb) 6= 0. We remark that if the column of index i, i < m, of the
matrix r is zero, then r is irrelevant because then r n αi = r for all n αi ∈ Nαi (F). In
particular, the first diagonal entry b1,1 of b is nonzero. Then at the cost of multiplying
b by a suitable element of Nα1 on the right, we may assume that b1,2 = 0. Again,
since the second column of b cannot be zero, we must have b2,2 6= 0. Inductively, we
see that we may assume b to have the form
0
b 0
b=
,
0 0
SMOOTH TRANSFER OF KLOOSTERMAN INTEGRALS
125
where b0 is an invertible diagonal matrix of size (m − 1) × (m − 1). Thus the last row
of b is zero. Since wb is relevant, the first m − 1 rows of wb cannot be zero. This
forces w to have the form
0
w 0
w=
,
0 1
where w0 is a permutation matrix in GL(m − 1). Now
1m−1 (wb) = det b0 det w0 .
Thus 1m−1 (wb) 6= 0, as claimed.
Now it is elementary that every element x such that det x = 0 but 1n−1 (x) 6= 0 is in
the orbit of an element of the form
0
x 0
.
0 0
Moreover, x is relevant if and only if x 0 is a relevant element in GL(m − 1, F). Also,
a = diag(a1 , a2 , . . . , am ) ∈ Am
is relevant if and only if a1 a2 · · · am−1 6= 0.
Now we recall the classification of the relevant orbits in GL(m, F) (cf. [1], [12]).
Every orbit has a unique representative of the form wa with w ∈ W and a ∈ Am (F).
If this element is relevant, then for every pair of positive roots (α1 , α2 ) such that
wα2 = −α1 , for n αi ∈ Nαi (F) we have
t
n α1 wan α2 = wa ⇒ θ0 (n α1 n α2 ) = 0.
(3)
This condition implies that if α1 is simple, then α2 is simple, and conversely. Thus w
and its inverse have the property that if they change a simple root to a negative one,
then they change it to the opposite of a simple root. Let S be the set of simple roots α
such that wα is negative. Then S is also the set of simple roots α such that w−1 α is
negative and wS = −S. Let M be the standard Levi subgroup determined by S. Thus
S is the set of simple roots of M for the torus A, w is the longest element of W ∩ M,
and w2 = 1. This being so, if α2 is simple, then condition (3) implies α2 (a) = 1.
Thus a is in the center of M.
Let wm be the m × m permutation matrix with antidiagonal entries equal to 1. We
see that elements of the form


wm 1 a1
0
···
0
 0
wm 2 a 2 · · ·
0 
,
g=
 ···

0
0
···
wm r ar
126
HERV É JACQUET
with m 1 + m 2 + · · · + m r = m, a1 , a2 , . . . , ar ∈ F × , form a set of representatives
for the relevant orbits in GL(m, F). A set of representatives for the relevant orbits in
M(n × n, F) is formed of the same elements, except that ar = 0 is allowed when
m r = 1 (and wm r = 1).
We set θ = ψ ◦ θ0 . We define the orbital integral of a relevant element x:
Z
 [8, ψ : x] = 8(t n 1 xn 2 )θ(n 1 n 2 ) dn 1 dn 2 ;
the integral is over the quotient of Nm (F) × Nm (F) by the stabilizer of x. The normalization of the measure is described below.
It is convenient to introduce the intermediate orbital integrals. Let m, n be two
integers greater than zero. For 8 ∈ S (M((m + n) × (m + n), F)), An ∈ GL(n, F),
Bm ∈ M(m × m, F), we set
Z
An 0
1n 0
An 0
1n X
nm 8, ψ :
:= 8
0 Bm
Y 1m
0 Bm
0 1m
m
m
(4)
× ψ Tr n X + Tr Y ˜n d X dY.
Here = nm is the matrix with m rows and n columns whose first row is the row
matrix of size n,
n = (0, 0, . . . , 0, 1),
and all other rows are zero. Likewise, ˜ = ˜nm is the matrix with n rows and m columns
whose first column is the column matrix
 
0
0
 
 
˜n = . . . ,
 
0
1
and all other columns are zero. If X = (xi, j ), then the measure d X is the tensor
product of the self-dual Haar measure d xi, j .
The above integral can we written more explicitly as
Z
An
An X
8
ψ Tr( X ) + Tr(Y ˜ ) d X dY,
Y An Bm + Y An X
or, after a change of variables,
Z
An 0
An
X
−2m
n
m 8, ψ :
= | det An |
8
0 Bm
Y Bm + Y A−1
n X
−1
× ψ Tr( A−1
n X ) + Tr(Y An ˜ ) d X dY.
SMOOTH TRANSFER OF KLOOSTERMAN INTEGRALS
127
The previous form shows that the integral converges and defines a smooth function
on GL(n, F) × M(m × m, F). More precisely, the following result is easily obtained.
LEMMA 2
If 8 is supported on On , then nm [8, ψ : •] is a smooth function of compact support
on GL(n, F) × M(m × m, F). Conversely, any smooth function of compact support
on GL(n, F) × M(m × m, F) is equal to nm [8, ψ : •] for a suitable function 8
supported on On .
We also introduce the normalized intermediate orbital integrals
An 0
An
n
m
n
˜
m 8, ψ :
:= | det An | × m 8, ψ :
0 Bm
0
0
Bm
.
(5)
If 9 is a smooth function of compact support on GL(n, F) × M(m × m, F),
we can define its orbital integrals relative to the action (Nn × Nn ) × (Nm × Nm ) on
GL(n, F) × M(m × m, F). They are denoted by (9, ψ : xn , xm ), where xn is
relevant in GL(n, F) and xm relevant in M(m × m, F). In particular, consider the
case of the function
An 0
9(An , Bm ) = nm 8, ψ :
0 Bm
and a relevant element x of M((n + m) × (n + m), F) of the form
x
0
x= n
,
0 xm
where xn is relevant in GL(n, F) and xm is relevant in M(m × m, F). We then have
the following reduction formula:
 [8, ψ : x] =  [9, ψ : xn , xm ] .
(6)
This formula reduces the computation of the orbital integrals (and the normalization
of the measures) to the case of an element of the form wn a, where a is a scalar matrix.
Then


0
0
0
···
0
a
 0

0
0
···
a
ax2,n 


Z


···
···
···
···
· · · 
· · ·
[8, ψ : wn a] = 8 

 0
0
a
· · · axn−2,n−1 axn−2,n 


 0
a
axn−1,3 · · · axn−1,n−1 axn−1,n 
a axn,2
axn,3
···
axn,n−1
axn,n
×ψ
i=n
X
i=2
xi,n−i+2 ⊗ d xi, j ,
128
HERV É JACQUET
where the measures d xi, j are self-dual.
We also denote by (8, ψ : a1 , a2 , . . . , an ) the orbital integral (8, ψ : a),
where a = diag(a1 , a2 , . . . , an ). We introduce normalized diagonal orbital integrals
as follows. For a ∈ Am (F) we set
σn (a) := 11 (a)12 (a) · · · 1n−1 (a).
Thus
σn (a) = a1n−1 a2n−2 · · · an−1
and a is relevant if σn (a) 6= 0. For a relevant, we set
˜
(8,
ψ : a) := |σn (a)|(8, ψ : a).
(7)
We have a reduction formula for these normalized orbital integrals. In a precise
way, for 8 ∈ S (M(n + m) × (n + m), F), set
˜ nm 8, ψ : An 0
.
9(An , Bm ) = 
0 Bm
Consider a diagonal matrix
a
a= n
0
0
am
with an ∈ An (F) and am relevant in Am (F). The normalized orbital integral of 9 on
the pair (an , am ) is defined to be
˜ [9, ψ : an , am ] = |σn (an )| |σm (am )| [9, ψ : an , am ] .

It follows from the relation
(det an )m σn (an )σ (am ) = σn+m (a)
˜
that it is equal to (8,
ψ : a).
3. Weil formula: The split case
Our goal is to obtain a simple relation between the orbital integrals of a function and
the orbital integrals of its Fourier transform. Our main tool is the Weil formula for
the Fourier transform of a character of second order. We state the form of the Weil
formula that we are using. We consider the direct sum
M(n × m, F) ⊕ M(m × n, F),
the first index being the number of rows. A function 8 on that space is written as
0
X
8 n,n
.
Y
0m,m
SMOOTH TRANSFER OF KLOOSTERMAN INTEGRALS
129
Thus the matrix written is a square matrix of size (n +m)×(n +m) with zero diagonal
blocks, X ∈ M(n × m, F), and Y ∈ M(m × n, F). The Fourier transform of such a
function is the function defined by
Z
0 U
0 X
cc0 U
0n,n
X
ψ − Tr
dU d V
:= 8
8̂
Y 0
V
0
Y
0m,m
V 0
Z
0 U
= 8
ψ − Tr(Y U ) − Tr(X V ) dU d V.
(8)
V 0
2
For A ∈ GL(n, F), B ∈ GL(m, F), and any Schwartz function 8 on M(n × m, F) ⊕
M(m × n, F), we have
Z
0 X
ψ (Tr BY AX ) d X dY
8
Y 0
Z
0 X
= | det A|−m | det B|−n 8̂
ψ(− Tr B −1 Y A−1 X ) d X dY.
Y 0
PROPOSITION
Proof
Assume first that A and B are unit matrices. Set
Z
0 V
81 (U, V ) = 8
ψ − Tr(Y U ) dU.
Y 0
Then the left-hand side of the identity reduces to
Z
81 (−X, X ) d X.
On the other hand, using the (partial) Fourier inversion formula, we see that the righthand side is equal to
Z
81 (Y, −Y ) dY.
Our assertion then follows. In general, we remark that Tr(BY AX ) = Tr Y AX B. We
change X to X B −1 and Y to Y A−1 in the left-hand side, and we change Y to BY and
X to AX in the right-hand side. We see that we are reduced to the case of A = 1,
B = 1 applied to the function
0
X B −1
(X, Y ) 7→ | det A|−m |B|−n 8
,
Y A−1
0
the Fourier transform of which is the function
0
(X, Y ) 7→ 8̂
BY
AX
.
0
130
HERV É JACQUET
Our assertion follows.
We record a simple consequence of this formula. For P ∈ M(m × n, F) and Q ∈
M(n × m, F),
Z
0 X
8
ψ Tr(P X ) + Tr(Y Q) + Tr(BY AX ) d X dY
Y 0
= | det A|−m | det B|−n
Z
0 X
× 8̂
ψ − Tr B −1 (Y + P)A−1 (X + Q) d X dY.
Y 0
(9)
4. Inversion formula for the orbital integrals
For 8 ∈ S (M(n × n, F)), we define the Fourier transform of 8 to be
Z
8̂(X ) = 8(Y )ψ[− Tr(X Y )] dY.
The measure is self-dual. Recall that wn ∈ Wn is the permutation matrix with unit
antidiagonal. We set
8̌(X ) = 8̂(wn X wn ).
Our goal in this section is to obtain a relation between the orbital integrals of 8 and
the orbital integrals of 8̌. We first establish such a relation for the intermediate orbital
integrals.
3
Let n ≥ 1, m ≥ 1 be two integers. Let 8 ∈ S (M((m + n) × (m + n), F)). Then
Z Z
˜ nm 8, ψ : An 0
ψ [− Tr Bm wm Cm wm ] d Bm

0 Bm
−1
m
× ψ Tr(wm Cm wm nm A−1
n ˜n ) ψ − Tr(An wn Dn wn ) d An
Cm 0
˜m
=
8̌,
ψ
:
.
n
0 Dn
PROPOSITION
The integrals are for Bm ∈ M(m × m, F) and An ∈ M(n × n, F).
Proof
We consider the partial Fourier transform 2 (with respect to Bm ) of the normalized
intermediate orbital integral of 8:
Z
˜ nm 8, ψ : An 0
2 (An , Cm ) := 
ψ [− Tr Bm Cm ] d Bm .
0 Bm
SMOOTH TRANSFER OF KLOOSTERMAN INTEGRALS
131
If we change Cm to wm Cm wm and Dn to wn Dn wn , we see that the identity of the
proposition amounts to
Z
m
2(An , Cm )ψ[Tr Cm−1 nm A−1
n ˜n ]ψ[− Tr An Dn ] d An
Z
D n + wn Y wm C m wm X wn wn Y wm C m
= | det Cm |n 8̂
C m wm X wn
Cm
× ψ[− Tr mn X − Tr Y ˜mn ] d X dY.
(10)
Changing X to wm Y wn and Y to wn X wm and noting the relations
wn mn wm = ˜nm ,
wm ˜mn wn = nm ,
we see that the right-hand side of the last formula can also be written
Z
Dn + XCm Y XCm
8̂
ψ[− Tr Y ˜nm − Tr nm X ] d X dY.
Cm Y
Cm
(11)
From now on, let us write for nm and ˜ for ˜nm . After a change of variables, we find
that 2 is also equal to
Z
An
An X
| det An |m 8
Y An Bm
× ψ[Tr( X ) + Tr(Y ˜ ) + Tr(Cm Y An X )] d X dY
× ψ[− Tr(Cm Bm )] d Bm .
After a new change of variables, this becomes
Z
An X
−m
| det An |
8
Y Bm
−1
−1
× ψ Tr( A−1
n X ) + Tr(Y An ˜ ) + Tr(C m Y An X ) d X dY
× ψ − Tr(Cm Bm ) d Bm .
We now introduce a partial Fourier transform of 8:
81
An
V
U
Cm
Z
:=
An
8
Y
X
0
ψ − Tr
Bm
Y
X
Bm
cc0 U
d X dY d Bm .
V Cm
By the Weil formula, the previous expression is then
Z
An X
−1
ψ − Tr Cm−1 (Y + A−1
| det Cm |−n 81
n )An (X + An ˜ ) d X dY.
Y Cm
132
HERV É JACQUET
Expanding ψ, we find
| det Cm |−n ψ − Tr(Cm−1 A−1
n ˜ )
Z
An X
× 81
Y Cm
× ψ − Tr(Cm−1 Y An X ) − Tr(Cm−1 Y ˜ ) − Tr(Cm−1 X ) d X dY.
Changing variables once more, we obtain, at last,
2(An , Cm ) = | det Cm |n ψ − Tr(Cm−1 A−1
n ˜ )
Z
An
XCm
× 81
Cm Y
Cm
× ψ − Tr Cm Y An X − Tr(Y ˜ ) − Tr( X ) d X dY
To obtain (10), we must multiply by
ψ Tr(Cn−1 A−1
n ˜ )
and take the Fourier transform of the result with respect to An . The Fourier transform
is evaluated at Dn . We indeed obtain (10). The proof shows that for Cm ∈ GL(m, F),
the integral converges as an iterated integral.
Our main result in this section is the following inversion formula for diagonal orbital
integrals.
1
For 8 ∈ S (M(n × n, F)),
THEOREM
˜ 8̌,ψ : a1 , a2 , . . . , an )
(
Z
˜
= (8,
ψ : p1 , p2 , · · · , pn )
i=n
i=n−1
X
X
×ψ −
pi an+1−i +
i=1
i=1
1 d pn d pn−1 · · · d p1 ,
pi an−i
where the multiple integral is only an iterated integral.
Proof
For n = 1 the formula is just the definition of the Fourier transform. Thus we may
assume that n > 1 and that our assertion is true for n − 1. We prove it for n. We apply
the formula of Proposition 3 to the pair of integers (n − 1, 1). We obtain, for fixed
SMOOTH TRANSFER OF KLOOSTERMAN INTEGRALS
a1 ∈ F × ,
Z Z
˜ n−1

1
8, ψ :
An−1
0
0
pn
133
ψ [− pn a1 ] dpn
× ψ Tr(a1−1 n−1 A−1
n−1 ˜n−1 ) ψ − Tr(An−1 wn−1 Dn−1 wn−1 ) d An−1
0
˜ 1n−1 8̌, ψ : a1
=
.
0 Dn−1
This formula shows that the function 81 (An−1 ) defined by
81 (An−1 ) := ψ Tr(a1−1 n−1 A−1
n−1 ˜n−1 )
Z
A
0
n−1
n−1
˜
ψ [− pn a1 ] d pn ,
× 
8, ψ :
1
0
pn
which a priori is defined only on GL(n − 1, F), extends in fact to a Schwartz-Bruhat
function on M((n − 1) × (n − 1), F), still denoted by 81 such that 8̌1 = 2, where
2 is the Schwartz function on M((n − 1) × (n − 1), F) defined by
0
˜ 1n−1 8̌, ψ : a1
2(Dn−1 ) = 
.
0 Dn−1
The induction hypothesis applied to 81 gives
˜
(2,ψ
: a2 , a3 , . . . , an )
Z
˜ 1 , ψ : p1 , p2 , . . . , pn−1 )
= (8
i=n−2
i=n−1
X
X
×ψ −
pi an+1−i +
i=1
i=1
1 dpn−1 · · · d p1 .
pi an−i
(12)
On the other hand, with
a = diag(a1 , a2 , . . . , am ),
a 0 = diag(a2 , a3 , . . . , an ),
we have
˜
(2,ψ
: a2 , a3 , . . . , an )
= |σn−1 (a 0 )|(2, ψ : a 0 )
= |a1 |n−1 |σn−1 (a 0 )|
Z
a
1
×
n−1 8̌, ψ : 1
0
Nn−1 (F)×Nn−1 (F)
= |σn (a)|(8̌, ψ : a)
˜ 8̌, ψ : a1 , a2 , . . . , an ).
= (
0
t u a0u
2
1
θ(u 1 u 2 ) du 1 du 2
134
HERV É JACQUET
Hence we get
˜ 8̌,ψ : a1 , a2 , . . . , an )
(
Z
˜ 1 , ψ : p1 , p2 , . . . , pn−1 )
= (8
i=n−2
i=n−1
X
X
×ψ −
pi an+1−i +
i=1
i=1
1 d pn−1 · · · d p1 .
pi an−i
(13)
To complete the proof, we have to compute the orbital integral of the function
An−1 7→ 81 (An−1 ) on the matrix p 0 = diag( p1 , p2 , . . . , pn−1 ). Note that the factor
ψ Tr(a1−1 n−1 A−1
n−1 ˜n−1 ) ,
which appears in the definition of 81 , is constant on the orbits of Nn−1 (F)× Nn−1 (F)
and, in particular, takes the value ψ[1/(a1 pn−1 )] on the orbit of p 0 . Observe, furthermore, that if φ is the characteristic function of a compact open set in F × , then the
function
˜ n−1 8, ψ : An−1 0
(An−1 , pn ) 7→ φ(det An−1 )
1
0
pn
is a smooth function of compact support on GL(n − 1, F) × F. In particular,
t
Z Z
u 1 xu 2 0
n−1
˜
1
8, ψ :
θ(u 1 u 2 )ψ[− pn a1 ] d pn du 1 du 2
0
pn
is an absolutely convergent integral where the order of integration can be reversed.
Moreover, for a given pn ,
|σn−1 ( p 0 )|
Z
t
0
˜ n−1 8, ψ : u 1 p u 2

1
0
0
pn
θ (u 1 u 2 ) du 1 du 2
˜ [8, ψ : p1 , p2 , . . . , pn ] .
=
SMOOTH TRANSFER OF KLOOSTERMAN INTEGRALS
135
It follows that
˜ 1 ,ψ : p 0 ]
[8
h 1 i
|σn−1 ( p 0 )|
=ψ
a1 pn−1
t
Z Z
0
˜ n−1 8, ψ : u 1 p u 2 0
×

θ(u 1 u 2 )ψ[− pn a1 ] du 1 du 2 d pn
1
0
pn
h 1 i
=ψ
|σn−1 ( p 0 )|| det p 0 |
a1 pn−1
t
Z u 1 p0 u 2 0
n−1
×
1
8, ψ :
θ(u 1 u 2 )ψ[− pn a1 ] du 1 du 2 d pn
0
pn
Z
h 1 i
=ψ
|σn ( p)| [8, ψ : p1 , p2 , . . . , pn ]ψ[− pn a1 ] d pn
a1 pn−1
h 1 iZ
˜
=ψ
[8,
ψ : p1 , p2 , . . . , pn ]ψ[− pn a1 ] d pn .
a1 pn−1
Upon inserting this result in the right-hand side of (13), we find the identity of the
theorem.
There is another more elementary formula that comes directly from the Fourier inversion formula.
PROPOSITION 4
For 8 ∈ S (M(n × n, F)), n > 1, the function
φ(a) =  [8, ψ : wn a]
is a smooth function of compact support on F × . Furthermore,
Z
2
−wn−1 a −1 0
φ(a) = |a|−n +1  8̌, ψ :
db.
0
b
136
HERV É JACQUET
Proof
After a change of variables, we find

0
 0


Z

· · ·
(n−n 2 )/2
φ(a) = |a|
8 
 0

 0
a
0
0
···
0
a
xn,2
0
0
···
a
xn−1,3
xn,3
···
···
···
···
···
···
0
a
···
xn−2,n−1
xn−1,n−1
xn,n−1
a

x2,n
···





xn−2,n 

xn−1,n 
xn,n
i=n
X
× ψ a −1
xi,n−i+2 ⊗ d xi, j .
i=2
Since |1n | is bounded above on the support of 8, the above integral vanishes for
|a| sufficiently large. On the other hand, since the integrand is a smooth function of
the xi, j , the integral vanishes for |a| small enough.
We pass to the second assertion. In terms of 8̌ we find that φ is equal to


0
0
0
···
−a −1
x1,n

 · · ·
···
···
···
···
··· 


Z



0
−a −1 · · · xn−3,n−1 xn−3,n 
 0
8̌ 


 0
−a −1 xn−2,3 · · · xn−2,n−1 xn−2,n 

 −1
−a
xn−1,2 xn−1,3 · · · xn−1,n−1 xn−1,n 
xn,1
xn,2
xn,3
···
xn,n−1
xn,n
× |a|
(n−n 2 )/2
i=n
X
ψ a
xi,n−i+1 ⊗ d xi, j .
i=1
We define a function 81 on GL(n − 1, F) by
Z
An−1
n−1
81 (An−1 ) = 1
8̌, ψ :
0
0
b
db.
We remark that if f is a smooth function of compact support on F × , then the product
f (det An−1 )81 (An−1 ) is a smooth function of compact support on GL(n − 1, F). It
easily follows that
Z
−wn−1 a −1 0
 8̌, ψ :
db = (81 ; ψ : −wn−1 a −1 ).
0
b
SMOOTH TRANSFER OF KLOOSTERMAN INTEGRALS
137
Now
81 (An−1 ) =
Z
8̌
An−1
Y An−1
An−1 X
b + Y An−1 X
= | det An−1 |−2
Z
An−1
× 8̌
Y
X
b
ψ(−n−1 X − Y ˜n−1 ) d X dY db
−1
ψ(−n−1 A−1
n−1 X − Y An−1 ˜n−1 ) d X dY db.
Now
[81 , ψ : −wn−1 a −1 ] =
Z
i=n−1
X
81 (An−1 )ψ −
xi,n−i+1 d xi, j ,
i=2
where An−1 is the matrix

0
0
 ···
·
··


0
 0

 0
−a −1
−a −1 −a −1 xn−1,2
0
···
−a −1
−a −1 xn−2,3
−a −1 xn−1,3
···
···
···
···
···

−a −1

···


−a −1 xn−3,n−1  .

−a −1 xn−2,n−1 
−a −1 xn−1,n−1
After a change of variables, we find
|a|
((n−1)2 −(n−1))/2
Z
i=n−1
X
81 (An−1 )ψ a
xi,n−i+1 d xi, j ,
i=2
where now An−1 is the matrix

0
0
 ···
···


0
 0

 0
−a −1
−1
−a
xn−1,2
0
···
−a −1
xn−2,3
xn−1,3
···
···
···
···
···

−a −1
··· 


xn−3,n−1  .

xn−2,n−1 
xn−1,n−1
Combining with the previous formula for 81 , we find that its orbital integral on
−wn−1 a −1 is
|a|((n−1)
2 −(n−1))/2+2(n−1)
Z
i=n−1
X
8̌(An )ψ a
xi,n−i+1 d xi, j ,
i=1
138
where An is now the matrix

0
0
 ···
·
··


0
 0

 0
−a −1
 −1
−a
xn−1,2
xn,1
xn,2
HERV É JACQUET
0
···
−a −1
xn−2,3
xn−1,3
xn,3
···
···
···
···
···
···
−a −1
···
xn−3,n−1
xn−2,n−1
xn−1,n−1
xn,n−1

x1,n
··· 


xn−3,n 
.
xn−2,n 

xn−1,n 
xn,n
Comparing with the last expression for φ(a), we obtain our assertion.
5. Orbital integrals in H (n × n, E/F)
We let E/F be a quadratic extension of F and denote by H (n × n, E/F) the space of
n ×n Hermitian matrices, that is, the matrices m such that t m = m. The group Nn (E)
operates on H (n × n, E/F) by m 7→ t umu. Viewed as functions on H (n × n, E/F),
the functions 1i are invariants of this action. For 1 ≤ i < n, we let Oi0 be the subset of
H (n × n, E/F) defined by 1i 6= 0. We define an algebraic character θ1 : N (E) → F
P
by θ1 (u) = i=n−1
u i,i+1 . Often we write θ1 (u) = θ0 (uu). We say that an element
i=1
x (or its orbit) is relevant if θ1 is trivial on the stabilizer of x in Nn (E).
5
In H (n × n, E/F) the set defined by 1n−1 = 0, 1n = 0 contains no relevant orbit.
PROPOSITION
Proof
Every s ∈ H (n × n, E/F) has the form s = t nwb with n ∈ Nn (E), w ∈ Wn , and
b upper triangular. Suppose that det s = 0, and suppose that s is relevant. We have
to show that 1n−1 (s) 6= 0. Now, if a column of s with index i < n is zero, then the
row with index i is also zero because s is Hermitian, and then s is irrelevant since it is
fixed by the group Nαi (E). In particular, b1,1 6= 0. As before, at the cost of replacing
s by t n 1 sn 1 , we may assume that s = t nwb, where b has the form
0
b 0
b=
,
0 0
where b0 is an invertible diagonal matrix in GL(n − 1, E). In particular, the last row
of b is zero. We may replace s by the element wbn −1 , and the last row of bn −1 is
again zero. Since the rows of wbn −1 with index less than n cannot be zero, w must
have the form
0
w 0
w=
0 0
with w0 ∈ Wn−1 . Then 1n−1 (wbn −1 ) = det b det w0 6= 0, as claimed.
SMOOTH TRANSFER OF KLOOSTERMAN INTEGRALS
139
As before, an element x such that det x = 0 but 1n−1 (x) 6= 0 is in the same orbit as
an element of the form
0
x 0
.
0 0
Moreover, x is relevant if and only if x 0 is a relevant element in GL(m − 1, E) ∩
H ((m − 1) × (m − 1), E/F).
Now we recall the classification of the relevant orbits in GL(n − 1, E) ∩ H ((n −
1) × (n − 1), E/F). Every orbit has a unique representative of the form wa with
w ∈ W , w2 = 1, a ∈ Am (E), and waw = a (cf. [11]). Suppose that α is a simple
root such that wα = −β where β is positive. For n α ∈ Nα , define n β ∈ Nβ by
t
n β wan α = wa.
t
n α wan β = wa.
Then
There exists an element n α+β ∈ Nα+β (i.e., n α+β = 1 if α + β is not a root) such that
n = n α n β n α+β satisfies
t
nwan = wa.
If wa is relevant, this relation implies
θ0 (n α n α n β n β ) = 0.
If β is not simple, this leads to a contradiction. Thus β is simple. Since w2 = 1, we
see that, as before, there is a standard Levi subgroup M such that w is the longest
element in Wm ∩ M. The above relation also implies that a is in the center of M and
in A(F).
We conclude that the set of representatives for the relevant orbits of Nm (F) ×
Nm (F) in M(m × m, F) given in Section 2 is also a set of representatives for the
relevant orbits of Nm (E) in H (m × m, E/F).
We set θ(uu) = ψ(θ0 (uu)). The orbital integrals
Z
[8, ψ, E/F : x] = 8(t uxu)θ(uu) du
of a relevant element x are defined as before: the integral is over the quotient of
Nm (E) ∩ M by the stabilizer of x. Our goal is to study these orbital integrals in
complete analogy with the previous discussion.
It is convenient to introduce the intermediate orbital integrals. Let m, n be two
integers greater than zero. For 8 ∈ S (H ((m + n) × (m + n), E/F)), An ∈ Sn (F),
140
HERV É JACQUET
Bm ∈ H (m × m, E/F), we set
An 0
n
m 8, E/F, ψ :
0 Bm
Z
1n
0
An 0
1n
:= 8
tX 1
0
B
0
m
m
× ψ Tr( X ) + Tr( t X ˜ ) d X.
X
1m
(14)
Here and ˜ are as above. Note that ˜ = t . If X = (xi, j ), then the measure d X
is the tensor product of the self-dual Haar measures d xi, j . The integral converges
and defines a smooth function on Sn (F) × M(m × m, F). In particular, we have the
following easy result.
3
If 8 is supported on On0 , then the function nm (8, ψ : •) is a smooth function of
compact support on Sn (F) × M(m × m, F). Conversely, every smooth function of
compact support on that space is of this form for a suitable 8 supported on On0 .
LEMMA
After a change of variables, we find, keeping in mind that det An ∈ F × ,
An 0
nm 8, E/F, ψ :
0 Bm
Z
An
X
= | det An |−2m
8
tX
F
Bm + t X A−1
n X
−1
t
−1
× ψ Tr( An X ) + Tr( X An ˜ ) d X.
We also introduce the normalized intermediate orbital integrals
˜ nm 8, E/F, ψ : An

0
0
Bm
:= η(det An ) | det
m
An |mF
× 8nm
8, E/F, ψ :
An
0
0
Bm
. (15)
One can define the orbital integrals of a smooth function of compact support 9 on
Sn (F) × H (m × m, E/F) relative to the action of (Nn (E)) × (Nm (E)) on Sn (F) ×
H (m × m, E/F). They are denoted by (9, ψ : xn , xm ), where xn is relevant in
Sn (F) and xm is relevant in H (m × m, E/F). In particular, consider the case of the
function
An 0
n
9(An , Bm ) = m 8, ψ, E/F :
0 Bm
SMOOTH TRANSFER OF KLOOSTERMAN INTEGRALS
141
and a relevant element x of H (m × m, E/F) of the form
x
0
x= n
,
0 xm
where xn is relevant in Sn (F) and xm is relevant in H (m × m, E/F). We then have
the reduction formula
 [8, E/F, ψ : x] =  [9, E/F, ψ : xn , xm ] .
(16)
This formula reduces the computation of the orbital integrals (and the normalization
of the measures) to the case of an element of the form wn a, where a ∈ F × is a scalar
matrix. Then
[9, E/F,ψ : wn a]

0
 0

Z

· · ·
= 8 
 0

 0
a
×ψ
i=n
X
0
0
···
0
a
axn,2
0
0
···
a
axn−1,3
axn,3
···
···
···
···
···
···
0
a
···
axn−2,n−1
axn−1,n−1
axn,n−1

a

ax2,n 


· · · 

axn−2,n 

axn−1,n 
axn,n
xi,n−i+2 ⊗ d xi, j ,
i=2
where xi,i ∈ F, xi, j ∈ E, xi, j = x j,i , where the measures d xi, j , i < j, on E and the
measures d xi,i on F are self-dual.
We also denote by (8, E/F, ψ : a1 , a2 , . . . , an ) the orbital integral
(8, E/F, ψ : a), where a = diag(a1 , a2 , . . . , an ) is relevant. We introduce normalized diagonal orbital integrals as follows:
˜
(8,
E/F,ψ : a1 , a2 , . . . , an )
:= η σn (a) |σn (a)|(8, E/F, ψ : a1 , a2 , . . . , an ).
(17)
As before, there is a reduction formula for the normalized diagonal orbital integrals.
6. Weil formula: The twisted case
As before, our goal is to obtain a simple relation between the orbital integrals of a
function and the orbital integrals of its Fourier transform. Again, our main tool is the
Weil formula for the Fourier transform of a character of second order. We first recall
the one-variable case. Define the Fourier transform of 8 ∈ S (E) by
Z
8̂(z) =
8(u)ψ(−uz − uz) du.
E
142
HERV É JACQUET
Then, for a ∈ F × ,
Z
Z
zz 8̂(z)ψ(azz) dz = |a|−1
η(a)c(E/F,
ψ)
8(z)ψ
−
dz.
F
a
We may take this formula as a definition of the constant c(E/F, ψ). We also have
Z
Z
zz −1
8(z)ψ(azz) dz = |a| F η(a)c(E/F, ψ) 8̂(z)ψ −
dz.
a
Applying the formula twice, we get the relation
c(E/F, ψ)c(E/F, ψ) = 1.
More generally, if 8 is a Schwartz function on the space of column vectors of size
n with entries in E, we define its Fourier transform, a function on the space of row
vectors of size n, by
Z
8̂(X ) = 8(U )ψ[−ZU − ZU ] dU.
6
For every matrix A ∈ Sn (F), we have
PROPOSITION
Z
En
8̂(Z )ψ[Z A t Z ] d Z
= | det
A|−1
F η(det
A)c(E/F, ψ)
n
Z
En
8(X )ψ[− t X A−1 X ] d X.
Proof
If A is a diagonal matrix (with diagonal entries in F × ), the formula follows at once
from the case of n = 1. In general, we may write
A = Ma t M,
with M ∈ GL(n, E), a diagonal. Then the left-hand side is
Z
−1
| det M| E
8̂(Z M −1 )ψ[Z a t Z ] d Z .
−1
Since Z 7→ | det M|−1
E 8̂(Z M ) is the Fourier transform of X 7→ 8(M X ), this is
equal to
Z
η(det a)| det a|−1 c(E/F, ψ)n 8(M X )ψ(− t Xa −1 X ) d X.
SMOOTH TRANSFER OF KLOOSTERMAN INTEGRALS
143
After a change of variables, this becomes
η(det a)| det
−1
n
M|−1
E | det a| c(E/F, ψ)
Z
8(X )ψ(− t X A−1 X ) d X.
Since | det A| F = | det a| F | det M| E , our assertion follows.
On the space H (n × n, E/F), we define the Fourier transform by
Z
9̂(X ) = 9(Y )ψ − Tr(X Y ) dY,
where the measure is self-dual. Thus if Y = (yi, j ), then yi,i ∈ F, y j,i = y i, j for
i < j, and the measure dY is the tensor product of the self-dual Haar measures dyi,i
and dyi, j , i < j. Note that Tr(X Y ) is indeed in F. In H ((n + m) × (n + m), E/F)
we consider the subspace of matrices with zero diagonal n × n and m × m blocks.
The Fourier transform of a function in that space is then defined by
Z
tX
0
0
V
ψ − Tr(X V + X V ) d V.
8̂ n,n
= 8 tn,n
X 0m,m
V 0m,m
This being so, the form of Weil formula that we are using is as given in the next
proposition.
7
Let A, B be Hermitian matrices in GL(n, E) and GL(m, E), respectively. Then
PROPOSITION
Z
0
8̂ n,n
Z
tZ
0m,m
ψ Tr(B Z A t Z ) d Z
−n
n
mn
= η(det A)m | det A|−m
F η(det B) | det B| F c(E/F, ψ)
Z
0
X
ψ − Tr(B −1 t X A−1 X ) d X.
× 8 tn,n
X 0m,m
Proof
Let us observe that Tr(B Z A t Z ) is indeed in F. As before, we may reduce the computation to the case where B is the diagonal matrix diag(b1 , b2 , . . . , bm ). We may
regard Z as a matrix
 
Z1
 Z2 

Z =
· · · ,
Zm
where each Z i is a row of size n. Then the formula follows from the previous formula
applied to the matrices b1 A, b2 A, . . . , bm A.
144
HERV É JACQUET
7. Inversion formula in the Hermitian case
As before, we set
9̌(X ) = 9̂(wm X wm ).
We then have the following result.
PROPOSITION 8
Let n ≥ 1, m ≥ 1 be two integers. Let 9 ∈ S H ((m + n) × (m + n), E/F) . Then
we have
Z Z
An 0
n
˜
m 9, E/F, ψ :
ψ[− Tr Bm wm Cm wm ] d Bm
0 Bm
× ψ Tr(wm Cm−1 wm A−1
n ˜ ) ψ − Tr(An wn Dn wn ) d An
Cm 0
mn ˜ m
.
= c(E/F, ψ) n 9̌, E/F, ψ :
0 Dn
Proof
The proof follows step by step the proof of the corresponding result for M((n + m) ×
(n + m), F). We consider the partial Fourier transform 2 (with respect to Bn ) of the
normalized intermediate orbital integral of 9:
Z
A
0
n
n
˜ m 9, E/F, ψ :
2 (An , Cm ) := 
ψ [− Tr Bm Cm ] d Bm .
0 Bm
The critical step is as follows. We get for 2:
Z
An X
m
−m
η(det An ) | det An |
9
tX
Bm
−1
t
t
−1
× ψ Tr( An X ) + Tr( X A−1
n ˜ ) + Tr(C m X An X ) d X
× ψ − Tr(Cm Bm ) d Bm .
We now introduce a partial Fourier transform of 9:
Z
An U
An X
0
:= 9 t
ψ − Tr t
91 t
U Cm
X Bm
X
X
Bm
0
tU
U
Cm
d X d Bm .
By the Weil formula, the previous expression is then
| det Cm |−n η(det Cm )n c(E/F, ψ)mn
Z
An X
−1
× 91 t
ψ − Tr Cm−1 (t X + A−1
n )An (X + An ˜ ) d X.
X Cm
The rest of the proof is identical to the proof of Proposition 3.
SMOOTH TRANSFER OF KLOOSTERMAN INTEGRALS
145
Our main result in this section is the following inversion formula.
2
For 8 ∈ S (H (n × n, E/F)),
THEOREM
˜ 8̌, ψ, E/F : a1 , a2 , . . . , an )
c(E/F,ψ)(n(n−1))/2 (
Z
˜
= (8,
ψ, E/F : p1 , p2 , . . . , pn )
i=n
i=n−1
X
X
×ψ −
pi an+1−i +
i=1
i=1
1 d pn dpn−1 · · · dp1 ,
pi an−i
where the integral is only an iterated integral.
Proof
Again the formula is trivial for n = 1, so we may assume that n > 1 and that our
assertion is true for n − 1. We prove it for n. We apply the formula of Proposition 8
to the pair of integers (n − 1, 1). We obtain
Z Z
An−1 0
n−1
˜
1
9, E/F, ψ :
ψ[− pn a1 ] d pn
0
pn
× ψ Tr(a1−1 n−1 A−1
n−1 ˜n−1 ) ψ − Tr(An−1 wn−1 Dn−1 wn−1 ) d An−1
a1
0
n−1 ˜ 1
= c(E/F, ψ) n−1 9̌, E/F, ψ :
.
0 Dn−1
We set
91 (An−1 ) := ψ Tr(a1−1 n−1 A−1
n−1 ˜n−1 )
Z
˜ n−1 9, E/F, ψ : An−1
× 
1
0
0
pn
[− pn a1 ] dpn .
Then 9̌1 (Dn−1 ) = 2(Dn−1 ), where
˜ 1n−1 9̌, E/F, ψ : a1
2(Dn−1 ) = c(E/F, ψ)n−1 
0
0
Dn−1
.
The induction hypothesis applied to 91 gives
˜
c(E/F,ψ)(n−1)(n−2)/2 (2,
E/F, ψ : a2 , a3 , . . . , an )
Z
˜ 1 , E/F, ψ : p1 , p2 , . . . , pn−1 )
= (9
i=n−2
i=n−1
X
X
×ψ −
pi an+1−i +
i=1
i=1
1 d pn−1 · · · d p1 .
pi an−i
(18)
146
HERV É JACQUET
The rest of the proof is the same as before once we observe that
c(E/F, ψ)n−1 c(E/F, ψ)(n−1)(n−2)/2 = c(E/F, ψ)n(n−1)/2 .
We leave it to the reader to formulate and prove the analogue of Proposition 4 in the
present situation.
8. Smooth matching
Definition 1
We say that a function 8 ∈ S (M(n ×n, F)) and a function 9 ∈ S (H (n ×n, E/F))
ψ
have matching orbital integrals for ψ, and we write 8 ←→ 9 if for every diagonal
matrix a ∈ An with 1n−1 (a) 6= 0,
˜
˜
(8,
ψ; a) = (9,
E/F, ψ : a).
From the inversion formula for the orbital integrals, we have at once the following
result.
PROPOSITION
ψ
9
ψ
If 8 ←→ 9, then 8̌ ←→ c(E/F, ψ)n(n−1)/2 9̌, and conversely.
Now we can formulate our main result.
THEOREM 3
Given 8 ∈ S (M(n × n, F)), there is 9 ∈ S (H (n × n, E/F)) with matching orbital
integrals for ψ, and conversely.
Proof
We treat the case of 8. The case of 9 is similar. The case of n = 1 being trivial, we
may assume that n > 1 and that our assertion is proved for n 0 < n. For i < n we can
consider smooth functions of compact support on GL(i, F) × M((n − i) × (n − i), F)
and Si (F) × H ((n −i) × (n −i), E/F), respectively, and their orbital integrals for the
action of the groups Ni (F) × Ni (F) × Nn−i (F) × Nn−i (F) and Ni (E) × Nn−i (E),
respectively. It follows from the induction hypothesis that any function on the first
space matches a function on the second space for ψ. Now let Oi (resp., Oi0 ) be the
open set of M(n × n, F) (resp., H (n × n, E/F)) defined by 1i 6= 0. If 8 is supported
on Oi , then its normalized intermediate orbital integral
˜ in−i [8, ψ : gi , m n−i ]

SMOOTH TRANSFER OF KLOOSTERMAN INTEGRALS
147
is a smooth function of compact support 8i,n−i on GL(i, F)× M((n −i)×(n −i), F);
it matches a function of compact support 9i,n−i on Si (F)× H ((n −i)×(n −i), E/F),
which in turn is the normalized intermediate orbital integral of a function 9 supported
on Oi0 . Now consider diagonal matrices ai ∈ Ai , an−i ∈ M((n − i) × (n − i), F),
with 1n−i−1 (an−i ) 6= 0. Then we have
0
˜ 8, ψ : ai
˜ 8i,n−i , ψ : ai , an−i .

=
0 an−i
ψ
There is a similar formula for 9. Thus, in fact, 8 ←→ 9. Hence our assertion is true
for 8 supported on Oi , i < n.
Next, suppose that 8 is such that
(8, ψ : wn a) = 0
for all a ∈ F × . Our classification of relevant orbits shows then that all the integrals
of 8 over relevant orbits contained in the closed set Q defined by
11 = 12 = · · · = 1n−1 = 0
vanish. Hence 8 has the same orbital integrals as a function supported on the complement of Q. We may assume that the support of 8 is contained in the complement
of Q. Using a partition of unity, we see that we can write
8=
i=n−1
X
8i ,
i=1
ψ
where 8 is supported on Oi . Then 8i ←→ 9i , where the function 9i is supported
on Oi0 and
i=n−1
X
ψ
8 ←→
9i .
i=1
Now let 8 be an arbitrary function. Then
φ(a) := |a|(n+1)(n−1) [8, ψ : wn a]
is a smooth function of compact support on F × . Let 81 be a function of compact
support contained in On−1 . Then
gn−1 0
1
n−1 81 , ψ :
0
b
148
HERV É JACQUET
is an arbitrary smooth function of compact support on GL(n −1, F)× F. In particular,
we can choose 81 so that
−1
Z
−a wn−1 0
φ(a) =  81 , ψ :
dp.
0
p
Let 82 be the function such that 8̌2 = 81 . Since 81 matches a function for ψ, it
ψ
follows from Proposition 9 that 82 ←→ 92 for a suitable function 92 . On the other
hand, by Proposition 4 applied to 82 ,
(8, ψ : wn a) = (82 , ψ : wn a),
so that
(8 − 82 , ψ : wn a) = 0.
ψ
By the previous case, it follows that 8−82 ←→ 93 for a suitable function 93 . Then
ψ
8 ←→ 92 + 83 and our assertion follows.
9. Application to the fundamental lemma
Suppose that E/F is unramified, and suppose that the conductor of ψ is O F . Let
80 be the characteristic function of the set of matrices with integral entries in
M(m × m, F). Similarly, let 90 be the characteristic function of the set of matrices with integral entries in H (m × m, E/F). The fundamental lemma asserts
that 80 matches 90 . We prove this for m = 2 and m = 3. The case where
| det a| = 1 is already known, but the proof presented below is much simpler. First
˜ 0 , ψ : a) = (8
˜ 0 , ψ : a), and likewise for 90 . It follows that the difference
(8
˜ 0 , ψ : a) − (9
˜ 0 , E/F, ψ : a)
ω(a) := (8
satisfies the functional equation
Z
ω(a1 , a2 , . . . , am ) = ω( p1 , p2 , . . . , pm )
i=m−1
i=m
X
X
×ψ −
pi am+1−i +
i=1
i=1
1 d pm dpm−1 · · · dp1 .
pi am−i
(19)
Moreover, it is supported on the set |a1 a2 · · · am | ≤ 1. The fundamental lemma asserts
that ω = 0.
For m = 2 it is easily checked that
Z
(80 , ψ : a1 , a2 ) da2
SMOOTH TRANSFER OF KLOOSTERMAN INTEGRALS
149
is 1 if |a1 | = 1 and 0 otherwise, and likewise for
Z
(80 , E/F, ψ : a1 , a2 ) da2 .
Thus
Z
If we set
µ(a1 , b1 ) :=
ω(a1 , a2 )da2 = 0.
Z
ω(a1 , p2 )ψ(− p2 b1 ) dp2 ,
then (19) is equivalent to the relation
µ(a1 , −b1 ) = µ(b1 , a1 )ψ
1 .
a 1 b1
(20)
Moreover, µ(a1 , b1 + t) = µ(a1 , b1 ) for |t| ≤ |a1 |. Since µ(a1 , 0) = 0, we see that
µ(a1 , b1 ) 6= 0 implies |b1 | > |a1 |. By (20), µ(a1 , b1 ) 6= 0 also implies |b1 | > |a1 |.
Thus µ = 0 or, equivalently, ω = 0, and we are done.
For m = 3 we first establish the following lemma.
LEMMA
4
If
Z
ω(a1 , a2 , a3 ) da3 = 0,
then ω = 0.
Proof
Indeed, if we set
µ(a1 , a2 , b1 ) :=
Z
ω(a1 , a2 , a3 )ψ(−a3 b1 ) da3
and
σ (a1 , a2 , b1 ) := µ(a1 , a2 , b1 )ψ
1 ,
a2 b1
then (19) is equivalent to
σ (a1 , a2 , −b1 ) =
Z
σ (b1 , p2 , a1 )ψ( p2 a2 ) dp2 .
(21)
The condition on the support of ω implies that µ(a1 , a2 , b1 + t) = µ(a1 , a2 , b1 ) for
|t| ≤ |a1 a2 |. Under the assumption of the lemma, we thus have µ(a1 , a2 , b1 ) = 0 for
|b1 | ≤ |a1 a2 |. Equivalently, σ (a1 , a2 , b1 ) 6= 0 implies |a2 | ≤ |$ b1 a1−1 |. From (21)
we have σ (a1 , a2 + t, b1 ) = σ (a1 , a2 , b1 ) for |t| ≤ |$ −1 b1 a1−1 |. Thus σ = 0 and
ω = 0.
150
HERV É JACQUET
Thus it suffices to prove the relation
Z
Z
(80 , ψ : a1 , a2 , a3 ) da3 = η(a2 ) (90 , E/F, ψ : a1 , a2 , a3 ) da3 .
R
Now 21 (80 , ψ : g, a3 ) da3 is | det g|−2
F times the characteristic function 81 of the
set of g ∈ M(2 × 2, F) such that
k(0, 1)g −1 k ≤ 1,
kgk ≤ 1,
kg −1 t (0, 1)k ≤ 1.
R
Likewise, 21 (90 , E/F, ψ : g, a3 ) da3 is | det g|−2
F times the characteristic function
91 of the set of g ∈ H (2 × 2, E/F) such that
kgk ≤ 1,
k(0, 1)g −1 k ≤ 1.
Thus it suffices to show that
(81 , ψ : a1 , a2 ) = η(a2 )(91 , E/F, ψ : a1 a2 ).
(22)
If |a1 a2 | = 1, then this relation is equivalent to
(80 , ψ : a1 , a2 ) = η(a1 )(9 0 , E/F, ψ : a1 a2 ),
where 80 is the characteristic function of GL(2, O F ) and 9 0 is the characteristic function of GL(2, O E ) ∩ H (2 × 2, E/F); in turn, this relation is a special case of the
fundamental lemma for m = 2.
Now
Z
(81 , ψ : a1 , a2 ) = ψ(x1 + x2 ) d x1 d x2
over the set
|a2 | ≥ 1,
|a1 xi | ≤ 1,
|xi | ≤ |a1 |−1 ,
|a2 + a1 x1 x2 | ≤ 1.
If |a2 | = 1, then this is 1. If |a2 | > 1, then this is zero unless |a1 a2 | = 1. A similar
remark applies to (90 , E/F, ψ : a1 , a2 ). The relation (22) follows. We are done.
10. Concluding remarks
More generally, one can use Propositions 3 and 4 to prove inductively the theorem of
density established in [3]; in a precise way, if 8 is such that (8, ψ : a) = 0 for all
diagonal matrices a, then (8̌, ψ : a) = 0, and (8, ψ : g) = 0, (8̌, ψ : g) = 0
ψ
for all relevant g. Similarly, one can prove that if 8 ←→ 9, then for every g in the
common set of representatives for the two sets of orbits,
(8, ψ : g) = γ (g, ψ)(9, E/F, ψ : g),
SMOOTH TRANSFER OF KLOOSTERMAN INTEGRALS
151
where the constant γ (g, ψ) (the transfer factor) does not depend on the functions,
a result that also follows from [3]. This gives another way to compute the transfer
factors.
Since the determinant is an invariant of the two actions, it follows from the theorem that for any smooth function of compact support on GL(m, F) there is a function
ψ
of compact support 9 on Sm (F) such that 8 ←→ 9.
Except for the last section, the previous discussion applies directly to the extension C/R.
Suppose that E/F is unramified, and suppose that the conductor of ψ is O F .
Then c(E/F, ψ) = 1. If E/F is a quadratic extension of global fields and ψ is a
nontrivial character of FA /F, then the product of the constants c(E v /Fv , ψv ) over all
places v of F inert in E is 1. Likewise, for g relevant in GL(n, F)∩ Sn (F) the product
of the local transfer factors is 1.
References
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
D. BUMP, S. FRIEDBERG, and D. GOLDFELD, “Poincaré series and Kloosterman sums”
in The Selberg Trace Formula and Related Topics (Brunswick, Maine, 1984),
Contemp. Math. 53, Amer. Math. Soc., Providence, 1986, 39 – 49. MR 87j:11043
125
H. JACQUET, The continuous spectrum of the relative trace formula for GL(3) over a
quadratic extension, Israel J. Math. 89 (1995), 1 – 59. MR 96a:22029
, A theorem of density for Kloosterman integrals, Asian J. Math. 2 (1998),
759 – 778. MR 2001e:11054 122, 150, 151
H. JACQUET and Y. YE, Relative Kloosterman integrals for GL(3), Bull. Soc. Math.
France 120 (1992), 263 – 295. MR 94c:11047 122
, Distinguished representations and quadratic base change for GL(3), Trans.
Amer. Math. Soc. 348 (1996), 913 – 939. MR 96h:11041 122
, Germs of Kloosterman integrals for GL(3), Trans. Amer. Math. Soc. 351
(1999), 1227 – 1255. MR 99j:11053
, Kloosterman integrals over a quadratic extension, to appear in Ann. of Math.
(2). 122
E. LAPID and J. ROGAWSKI, Periods of Eisenstein series: The Galois case, Duke Math.
J. 120 (2003), 153 – 226.
B. C. NGÔ, Faisceaux pervers, homomorphisme de changement de base et lemme
fondamental de Jacquet et Ye, Ann. Sci. École Norm. Sup. (4) 32 (1999),
619 – 679. MR 2001g:11076 122
, Le lemme fondamental de Jacquet et Ye en caractéristique positive, Duke
Math. J. 96 (1999), 473 – 520. MR 2000f:11059 122
T. A. SPRINGER, “Some results on algebraic groups with involutions” in Algebraic
Groups and Related Topics (Kyoto/Nagoya, 1983), Adv. Stud. Pure Math. 6,
North-Holland, Amsterdam, 1985, 525 – 543. MR 86m:20050 139
152
HERV É JACQUET
[12]
G. STEVENS, Poincaré series on GL(r ) and Kloostermann sums, Math. Ann. 277
[13]
J. L. WALDSPURGER, Le lemme fondamental implique le transfert, Compositio Math.
[14]
Y. YE, The fundamental lemma of a relative trace formula for GL(3), Compositio
(1987), 25 – 51. MR 88m:11031 125
105 (1997), 153 – 236. MR 98h:22023 122
[15]
Math. 89 (1993), 121 – 162. MR 95b:22023
, An integral transform and its applications, Math. Ann. 30 (1994), 405 – 417.
MR 95j:11045
Department of Mathematics, Columbia University, Mail Code 4408, New York, New York
10027, USA; [email protected]