In one of the recent posts by T. Tao on the progress of the Polymath8 project, the question arose of whether such functions as

defined for and a rational function , are trace functions, and more importantly, what is their conductor (see this, and the following, comments). In particular, if is obtained by reduction modulo primes of a fixed rational function in , one could expect that the answer is “Yes”, with a bound for the conductor independent of .

This is in fact a question that É. Fouvry, Ph. Michel and I had considered in special cases (or variants) in some of our papers. In fact, it is natural to consider more generally the linear map sending a function to the function

(which is an analogue of an integral operator) and to ask whether this linear map sends trace functions to trace functions, and if yes, if the conductor of is bounded in terms of the conductor of and the degree of the numerator and denominator of .

The most important case, which is crucial in all our series of papers, is the Fourier transform, which corresponds simply to . We proved the desired property (which we view, rather naturally I think, as a form of “continuity” of the Fourier transform, in an algebraic sense) in that case using Deligne’s definition and Laumon’s study of the sheaf-theoretic Fourier transform. Most importantly, in order to estimate the conductor of the Fourier transform of a trace function, we used Laumon’s theory of the local Fourier transform, which is rather deep.

It is by no means clear (and this is a rather interesting question of algebraic geometry!) that such a local theory exists, and leads to the continuity property, for arbitrary “kernels” . However, we figured out a way to prove the conductor bound that bypasses these local results. It applies to the Fourier transform, and although it is then much less precise than what is obtained from Laumon’s theory, it gives a (possibly) more accessible proof of the continuity property.

We have just put here our preprint with this result. It is not yet submitted to arXiv because we are considering various possibilities for either extensions of applications, but the proof of the main result is complete.

The paper was rather interesting to write. On the one hand, it turns out that it is not needed for the original question from Polymath8: Philippe found an elementary argument that reduces the specific example of the problem which reduces it, essentially, to a special case of the Fourier transform (this is written down in the Deligne section of the Polymath8 paper). On the other hand, although we had thought a bit about this question beyond the Fourier transform, we had not made progress. The reason, in retrospect, is that in order to treat the general transform

(as defined above), we begin by treating the special case

and without the motivation from the Polymath8 project, we had not thought of this first step.

The reason that things work this way is, as all other main ideas in this paper, very easy to explain by writing down and manipulating sums, and assuming that those behave always as the best Riemann Hypothesis over finite fields suggest. But the actual arguments are purely algebraico-geometric, and we end up using quite a bit of the general formalism of étale cohomology, but not the Riemann Hypothesis (which is morally as things should be.)

I will give here the informal sketch: first of all, the function

is indeed a trace function if is one, in almost all cases: precisely, the existence of higher-direct image sheaves with compact support, the proper base change theorem, and the Grothendieck trace formula, show that

where is the trace function of the sheaf

where are the two projections and , the sheaf is the one with trace function , and the sheaf is the Artin-Schreier sheaf on the plane with trace function

for . In many cases, both and vanish, and then is (minus) the trace function of the sheaf

This essentially answers the first question in full generality: the “transform” maps trace functions to trace functions.

Now consider the conductor of the sheaf above. We defined it as the sum of three terms. Two are relatively accessible: they are the (generic) rank of the sheaf, and the number of singularities. We can basically determine these if we know the dimension of all fibers: the maximum dimension gives an upper bound for the rank, and a result of Deligne says (roughly) that the singularities are the points where the fiber has smaller than maximal dimension. So let us assume that we can bound these quantities for the transform sheaf . Then there only remains to estimate the third term, which is the sum of the Swan conductors at the singularities. This is rather delicate, at least for people for whom — and this applies to us… — the Swan conductor remains a rather mysterious and subtle data.

The first idea is that, assuming the other two pieces of the conductor are under control, the sum of the Swan conductors is bounded by a global invariant that may be accessible in applications. Namely, if a sheaf is lisse on a dense open subset of the affine line then the Euler-Poincaré characteristic formula (of Néron, Ogg, Shafarevitch) easily proves that

So we have to deal with the dimension of the cohomology group . The point is that, in good circumstances, and especially if is of weight , we can expect that

where is the sum of the trace function of over the points of . Estimating this becomes a problem of analytic number theory, and we may hope to succeed.

For instance, if we apply this principle to

with the Artin-Schreier sheaf associated to a rational function as before, the sum is simply

were is the trace from to .

In good circumstances, we know square-root cancellation for this two-variable character sum, and we obtain a bound for the limsup of , which depends only on the degree of the numerator and denominator of , using Bombieri’s bounds for sums of Betti numbers for such exponential sums (or the generalizations of Adolphson-Sperber, or those of Katz.)

This deals (optimistically) with the transform with kernel when the input sheaf is the trivial sheaf, which we note is the case in the Polymath8 case. Now for the second idea: assume we consider

now, and try to use the same principle. With denoting the trace function of , the sum now satisfies

In impeccable style, we exchange the two sums of course. We get

where

But, *by the first step*, applied to instead of , the function is a trace function with conductor bounded in terms of the degrees of , or equivalently of . Thus is the inner-product, over , of the trace functions of two sheaves with bounded conductor, and we can expect both to have weight . We can then expect quasi-orthogonality from the Riemann Hypothesis, and a resulting bound for the limsup that depends only on the conductors of these two sheaves, i.e., on the conductor of (for ) and on the degrees of the numerator of denominator of (for ). This is the desired conclusion.

This sketch explains why we can prove the results in our paper. In many cases, it is certainly a valid reasoning, but it is not easy to make it rigorous in great generality. The basic problems are that it depends on the sums having square-root cancellation (which for the transform of the trivial sheaf is a non-trivial assumption), and also on detecting all of the cohomology space, and not just the part of weight : by Deligne’s Riemann Hypothesis, the eigenvalues of Frobenius on are of weight if has weight , but the limsup only gives the number of eigenvalues of weight , and having too many smaller eigenvalues would create problem.

We work around these possible difficulties by dropping the diophantine motivation, and going straight at the dimension of . To do this, we need algebraic analogues of the two fundamental analytic steps we used:

(1) expressing the sum for the trivial sheaf as a two-variable character sum;

(2) exchanging the order of the two sums when inserting a general input sheaf .

Both of these are replaced by (very elementary) arguments with Leray spectral sequences. This is a relatively well-known idea (it is part of the “dictionary” in Deligne’s survey on *Sommes trigonométriques* in SGA 4 ½ and quite a few concrete examples are found in papers and books of Katz), but it is the first time we use it ourselves. I will survey and explain this in a later post, since it seems that a good concrete example of the use of spectral sequences in analytic number theory might be a useful thing to have somewhere…

The reader who opens the PDF file of our preprint might be surprised to see that the paper in more than thirty pages long, in comparison with the rather simple-looking discussion above. The length is justified partly by the two motivating discussions we have included (the diophantine argument with , and a self-contained algebraic treatment of the important case of the Fourier transform). But it also turns out that taking care of the “easy” parts of the conductor requires somewhat lengthy elementary arguments with rational functions. Most importantly maybe, we must take into account the fact that, in contrast with our previous works, we now have to handle general constructible -adic sheaves, and not only middle-extension sheaves: there is no reason for our transformed sheaves to be so-well behaved in general. This requires adding a further component to the conductor (roughly, the support and dimension of the fibers of the “punctual part” of the sheaf, e.g., the conductor of a sheaf supported at with fiber of dimension must increase with ), and we also need to control it before applying the previous ideas. We also prove, both as a useful too and as a by-product, the analogue of the Bombieri bounds for a general input sheaf : the Betti numbers

are bounded in terms of the conductor of and the degree of the numerator and denominator of .