Joyee Ghosh

Efficient Bayesian Computation and Model Search in Linear Hierarchical Models

Merlise A. Clyde, Advisor

Abstract

Computational advances have made it possible to routinely consider a long list of competing models. This can be easily done in the Bayesian framework by treating models as unknown parameters. The posterior probabilities of models, which are key quantities for inference and prediction are often difficult to compute. This thesis considers different model selection scenarios in which posterior probabilities of models cannot be calculated in closed form, since they are analytically and/or computationally intractable. A common problem in this type of setting is to identify high probability models. Thus model search and techniques to estimate model probabilities are an integral part of these endeavors.

The first part of this thesis develops new sampling without replacement algorithms for model search, in linear regression with Gaussian errors. These are first compared to Markov chain Monte Carlo algorithms based on simulated data for an enumerable model space. Then comparisons are made based on two real datasets where enumeration is not possible.

In the second part the aim is to exploit some attractive properties of orthogonal design matrices in the context of Bayesian model selection and Bayesian model averaging for linear models. This is accomplished by augmenting the given non-orthogonal design matrix with new rows to form a "complete" orthogonal design. New model search techniques for linear regression and probit regression are proposed that enable one to construct Rao-Blackwellized estimates of model probabilities. The methods are demonstrated through illustrative simulations using two real datasets.

In the final part a class of heavy-tailed prior distributions is developed for Bayesian factor analytic models, which can be used as a default in the absence of substantive prior knowledge. The new class of prior distributions is induced through a parameter expansion approach. Good mixing properties of the proposed prior distribution are demonstrated through simulation studies and epidemiology and toxicology applications. An approach to deal with uncertainty regarding the number of factors is also outlined.

This thesis is available in PDF.