Oracle posterior rates in the White Noise Model

Abstract

All the results about posterior rates obtained until now are related to the optimal (minimax) rates for the estimation problem over the corresponding nonparametric smoothness classes, i.e. of a global nature. In the meantime, a new local approach to optimality has been developed within the estimation framework, namely, the oracle approach. The main goal of this thesis is to develop and study the oracle approach to the posterior concentration performance in the Gaussian sequence model. Therefore, a new benchmark for the posterior concentration rate is proposed, the so called posterior oracle rate, which is the smallest possible rate over a family of posterior rates corresponding to an appropriately chosen family of priors. We start with the notion of oracle projection convergence rate for the problem of estimation over the corresponding nonparametric smoothness class. Further, we propose an appropriate hierarchical prior, which also allows us to address the problem of Bayesian model selection, and establish that the posterior concentrates around the true parameter with the oracle projection convergence rate, which is stronger than the posterior convergence with the minimax rate over the nonparametric class if our family of projection estimators contains a minimax estimator over that class. We also construct a Bayes estimator based on the posterior and show that it satisfies an oracle inequality. Besides, we complement the upper bound results on the posterior concentration rate by a lower bound result for the oracle rate. It turns out that the rates of the upper and lower bounds coincide with the oracle projection convergence rate. This implies that the oracle posterior rate is sharp and all of the posterior mass concentrates in some annulus around the true parameter value. All these results are nonasymptotic and uniform over `2. Further, we study the posterior distributions in the many normal means model using numerical simulations and compare the results with those previously obtained theoretically in the Gaussian sequence model. To illustrate our findings, we construct credible bands for the signal function in the equivalent white noise model formulation. In this thesis we also consider the problem of Bayes estimation of a linear functional under the assumption that the unknown signal is from a Sobolev smoothness class. It turns out that the constructed Bayes estimator of the linear functional attains the minimax rate over the Sobolev smoothness class from the both frequentist and Bayesian perspectives. Using this, we establish the result on the convergence rate of the posterior distribution of the linear functional, which turns out to be optimal in the minimax sense over the Sobolev class. Under less informative prior we consider a Bayesian version of the adaptive estimation of the functional, which is in fact an adaptive filtering problem. We propose the Bayesian oracle and the oracle Bayes risk and show that our adaptive estimator of the functional is asymptotically sharp, i.e. its risk coincides asymptotically with the oracle Bayes risk. In other words, we show that the resulting adaptive estimator of the functional mimics the Bayesian oracle