Bayesian statisticians quantify their belief that the true parameter is ϑ0 by its posterior probability. The starting question of this paper is whether the posterior at ϑ0 increases when the data are generated under ϑ0, and how it behaves when the data come from ϑ ≠ ϑ0. Can it decrease and then increase, and thus additional data may mislead Bayesian statisticians?

For data arriving sequentially, we consider monotonicity properties of the posterior probabilities as a function of the sample size with respect to certain stochastic orders, specifically starting with likelihood ratio dominance.
When the data is generated by ϑ ≠ ϑ0 , Doob’s consistency theorem says that the posterior at ϑ0 converges a.s. to zero and therefore its expectation converges to zero. We obtain precise asymptotic rates of the latter convergence for observations from an exponential family and show that the expectation of the ϑ0 -posterior under ϑ ≠ ϑ0 is eventually strictly decreasing. Finally, we show that in a number of interesting cases this expectation is a log-concave function of the sample size, and thus unimodal. In the Bernoulli case we obtain this result by developing an inequality that is related to Turán’s inequality for Legendre polynomials.