Econometrics Exercises 7
BAYESIAN ECONOMETRIC MODELS

Gary Koop, Dale J. Poirier and Justin L. Tobias

Preface

Bayesian econometrics has enjoyed an increasing popularity in many fields. This popularity has been evidenced through the recent publication of several textbooks at the advanced undergraduate and graduate levels, including those of Poirier (1995), Bauwens, Lubrano, and Richard (1999), Koop (2003), Lancaster (2004), and Geweke (2005). The purpose of the present volume is to provide a wide range of exercises and solutions suitable for students interested in Bayesian econometrics at the level of these textbooks.

The Bayesian researcher should know the basic ideas underlying Bayesian methodology (i.e., Bayesian theory) and the computational tools used in modern Bayesian econometrics (i.e., Bayesian computation). The Bayesian should also be able to put the theory and computational tools together in the context of substantive empirical problems. We have writen this book with these three activities--theory, computation, and empirical modeling--in mind. We have tried to construct a wide range of exercises on all of these aspects. Loosely speaking, Chapters 1 through 9 focus on Bayesian theory, whereas Chapter 11 focuses primarily on recent developments in Bayesian computation. The remaining chapters focus on particular models (usually regression based). Inevitably, these chapters combine theory and computation in the context of particular models. Although we have tried to be reasonably complete in terms of covering the basic ideas of Bayesian theory and the computational tools most commonly used by the Bayesian, there is no way we can cover all the classes of models used in econometrics. Accordingly, we have selected a few popular classes of models (e.g, regression models with extensions and panel data models) to illustrate how the Bayesian paradigm works in practice. Particularly in Chapters 12 through 18 we have included substantive empirical exercises--some of them based closely on journal articles. We hope that the student who works through these chapters will have a good feeling for how serious Bayesian empirical work is done and will be well placed to write a Ph.D. dissertation or a journal article using Bayesian methods.

For the student with limited time, we highlight that a division in this book occurs between the largely theoretical material of Chapters 1 through 9 and the largely regression-based material in Chapters 10 through 18. A student taking a course on Bayesian statistical theory could focus on Chapters 1 through 9, whereas the student taking a Bayesian econometrics course (or interested solely in empirical work) could focus more on Chapters 10 thorugh 18 (skimming through the more methodologically oriented material in the early chapters).

Although there have been some attempts to create specifically Bayesian software (e.g., BUGS, which is available at http://www.mrc-bsu.cam.ac.uk/bugs, or BACC, which is available at http://www2.cirano.qc.ca/~bacc), in our estimation, most Bayesians still prefer to create their own programs using software such as Matlab, OX, or GAUSS. We have used Matlab to create answers to the empirical problems in this book. Our Matlab code is provided on the Web site associated with this book:

http://www.econ.iastate.edu/faculty/tobias/Bayesian_exercises.html

A few notational conventions are applied throughout the book, and it is worthwhile to review some of these prioer to diving into the exercises. In regression-based models, which constitute a majority of the exercises in the later chapters, lowercase letters such as y and x_i are reserved to denote scalar or vector quantities whereas capitals such as X or X_j are used to denote matrices. In cases in which the distinction between vectors ans scalars is critical, this will be made clear within the exercise. In the regression-based problems, y is assumed to denote the n x 1 vector of stacked responses for the dependent variable, y_i the i th element of that vector, x_i a k vector of covariate data, and X the n x k matrix obtained from stacking the x_i over i. Latent variables, which are often utilized in the computational chapters of the book, are typically designated with a "*" superscript, such as y_i^*. In Chapters 1 through 9, many exercises are presented that are not directly related to linear regression models or models that can be viewed as linear on suitably defined latent data. In these exercises, the distinction between random variables and realizations of those variables is sometimes important. In such cases, we strive to use capital letters to denote random variables, which are unknown ex ante, and lowercase letters to denote their realizations, which are known ex post. So, in the context of discussing a posterior distribution (which conditions on the data), we will use overline y, but if we are interested in discussing the sampling properties of the sample mean, overline Y would be the appropriate notation. Finally, "x" is used to denote multiplication in multiline derivations, and specific parameterizations of various densities are provided in the Appendix associated with this book.

On the issue of parameterization, the reader who is somewhat familiar with the Bayesian literature may realize that researchers often employ different parameterizations for the same model, with no particular choice being "correct" or "ideal". A leading example is the linear regression model, in which the researcher can choose to parameterize this model in terms of the error variance or the error precision (the reciprocal of the variance). In this book, we try and remain consistent in terms of parameterization within individual chapters, though some departures from this trend do exist, particularly in Chapters 11 and 16. These differences arise from our own individual tastes and styles towards approaching these models, and they are superficial rather than substantive. In our view it is quite valuable to expose the student ot the use of different parameterizations, since this is the reality that he or she will face when exploring the Bayesian literature in more detail. In all cases, the parameterization employed is clearly delineated within each exercise.

We would like to thank the editors of the Econometrics Exercises series--Karim Abadir, Jan Magnus, and Peter Phillips--for their helpful comments and support during the planning and writing of this book. Hoa Jia, Babatunde Abidoye, and Jingtao Wu deserve special recognition for reviewing numerous exercises and helping to reduce the number of typographical errors. The list of other colleagues and students who have helped us--is too long to enumerate here. We would, however, like to thank students at the University of California, Irvine; Leicester University; University of Toronto; and the Institute for Advanced Studies and CIDE (Italy) for their participation, wise insights, and enthusiasm.

Contents

List of exercises
Preface to the series
Preface
Ix
xv
xix
1 The subjective interpretation of probability
1
2 Bayesian inference
11
3 Point estimation
29
4 Frequentist properties of Bayesian estimators
37
5 Interval estimation
51
6 Hypothesis testing
59
7 Prediction
71
8 Choice of prior
79
9 Asymptotic Bayes
91
10 The linear regression model
107
11 Basics of Bayesian computation
      11.1 Monte Carlo integration
      11.2 Importance sampling
      11.3 Gibbs sampling and the Metropolis-Hastings algorithm
      11.4 Other (noniterative) methods for generating random variates
117
119
124
128
157
12 Hierarchical models
169
13 The linear regression model with general covariance matrix
191
14 Latent variable models
203
15 Mixture models
      15.1 Some scale mixture of normals models
      15.2 Other continuous and finite-mixture models
253
254
260
16 Bayesian model averaging and selection
      16.1 Bayesian model avaraging
      16.2 Bayesian variable selection and marginal likelihood calculation
281
282
287
17 Some stationary time series models
203
18 Some nonstationary time series models
203
Appendix