Naturally occurring populations of bacteria and archaea are vital to life on the earth and are of enormous practical significance in medicine, engineering and agriculture. However, the rules governing the formation of such communities are still poorly understood, and there is a need for a usable mathematical description of this process. Typically, microbial community structure is thought to be shaped mainly by deterministic factors such as competition and niche differentiation. Here we show, for a wide range of prokaryotic communities, that the relative abundance and frequency with which different taxa are observed in samples can be explained by a neutral community model (NCM). The NCM, which is a stochastic, birth-death immigration process, does not explicitly represent the deterministic factors and therefore cannot be a complete or literal description of community assembly. However, its success suggests that chance and immigration are important forces in shaping the patterns seen in prokaryotic communities.