Small gaps between consecutive primes

Recent work of D. Goldston and C. Yildirim

        What are the shortest intervals between consecutive prime numbers? The twin prime conjecture, which asserts that $p_{n+1}-p_n = 2$ infinitely often is one of the oldest problems; it is difficult to trace its origins.

        In the 1960's and 1970's sieve methods developed to the point where the great Chinese mathematician Chen was able to prove that for infinitely many primes $p$ the number $p+2$ is either prime or a product of two primes. However the well-known ``parity problem'' in sieve theory prevents further progress.

        What can actually be proven about small gaps between consecutive primes? A restatement of the prime number theorem is that the average size of $p_{n+1}-p_n$ is $\log p_n$ where $p_n$ denotes the $n$th prime. A consequence is that

$$\Delta:=\liminf_{n\to \infty} \frac{(p_{n+1}-p_n)}{\log p_n}\leq 1.$$

In 1926, Hardy and Littlewood, using their ``circle method'' proved that the Generalized Riemann Hypothesis (that neither the Riemann zeta-function nor any Dirichlet L-function has a zero with real part larger than 1/2) implies that $\Delta\le 2/3$. Rankin improved this (still assuming GRH) to $\Delta\le 3/5$. In 1940 Erdös, using sieve methods, gave the first unconditional proof that $\Delta<1$. In 1966 Bombieri and Davenport, using the newly developed theory of the large sieve (in the form of the Bombieri - Vinogradov theorem) in conjunction with the Hardy - Littlewood approach, proved $\Delta \leq 1/2$ unconditionally, and then using the Erdös method they obtained $\Delta \leq(2+\surd 3)/8=0.46650\ldots$. In 1977, Huxley combined the Erdös method and the Hardy - Littlewood, Bombieri - Davenport method to obtain $\Delta\leq 0.44254\dots$. Then, in 1986, Maier used his discovery that certain intervals contain a factor of $e^{\gamma}$ more primes than average intervals. Working in these intervals and combining all of the above methods, he proved that $\Delta\leq 0.2486\dots$, which was the best result until now.

        Dan Goldston and Cem Yildirim have a manuscript which advances the theory of small gaps between primes by a quantum leap. First of all, they show that $\Delta=0$. Moreover, they can prove that for infinitely many $n$ the inequality

$$p_{n+1}-p_n < (\log p_n)^{8/9}$$

holds.

        Goldston's and Yildirim's approach begins with the methods of Hardy-Littlewood and Bombieri - Davenport. They have discovered an extraordinary way to approximate, on average, sums over prime $k$-tuples. We believe, after work of Gallagher using the Hardy-Littlewood conjectures for the distribution of prime $k$-tuples, that the prime numbers in a short interval $[N, N+\lambda \log N]$ are distributed like a Poisson random variable with parameter $\lambda$. Goldston and Yildirim exploit this model in choosing approximations. They ultimately use the theory of orthogonal polynomials to express the optimal approximation in terms of the classical Laguerre polynomials. Hardy and Littlewood could have proven this theorem under the assumption of the Generalized Riemann Hypothesis; the Bombieri - Vinogradov theorem allows for the unconditional treatment.

        This new approach opens the door for much further work. It is clear from the manuscript that the savings of an exponent of 1/9 in the power of $\log p_n$ is not the best that the method will allow. There are (at least) two possible refinements. One is in the examination of lower order terms that arise in his method. Can they be used to enhance the argument? The other is in the error term Gallagher found in summing the ``singular series'' arising from the Hardy-Littlewood $k$-tuple conjecture. There is reason to believe that this error term can be improved, possibly using ideas in recent work of Montgomery and Soundararajan (``Beyond Pair - Correlation''.)

        It is not clear just how far this method can be pushed and what other problems might be attacked using his new ideas; at this point we can't rule out developments that would even approach the centuries old twin prime problem. What is clear is that a monumental barrier which has impeded progress for at least the last 80 years has been broken down.

        Note: Actually this work is astonishing in another regard. They have actually proven that for any fixed number $r$ the inequality

$$p_{n+r}-p_n < (\log p_n)^{(8r)/(8r+1)}$$

holds for infinitely many $n$. Those familiar with work on large gaps between primes will recall that in 1977 Helmut Maier burst onto the analytic number scene with his tour de force proof that the largest gaps known to hold for two consecutive primes could be proven for each gap of $r$ consecutive gaps for any fixed $r$. Goldston and Yildirim have achieved a similar sort of result for $r$ consecutive small gaps at the same time that they have demolished all previous records for one small gap.