Subjective “vs.” Objective Bayes

It is popular to divide Bayesians into two main categories, “objective” and “subjective” Bayesians. The divide is sometimes made formal, there are conferences labelled as one but not the other, for example.

A caricature of subjective Bayes is that all probabilities are just opinion, and the best we can do with an opinion is make sure it isn’t self contradictory, and satisfying the rules of probability is a way of ensuring that. A caricature of objective Bayes is that there exists a correct probability for every hypothesis given certain information, and that different people with the same information should make exactly the same probability judgments.

My view is that the distinction between subjective and objective Bayes is not helpful, and that something in between is usually desirable. The purpose of Bayesian inference (and ME updating) is to study how an ideal perfect reasoner would respond to certain pieces of information. When I choose a hypothesis space and a prior distribution, I’m not writing down my prior beliefs. That would be impossible, since probabilities aren’t actually a thing that exist in my brain. I am writing down a hypothetical state of prior beliefs that may or may not be a good approximation to what I (or some expert) think. But whatever I use in a calculation is always an idealisation, just like all models in physics.

If I want the posterior distribution to capture pretty much all of the information available about a question (for example, because we need to make an important decision), then I will do things that resemble subjective Bayes, for example, deriving various consequences of the prior probability assignment and making sure I (or someone who knows more about the topic than me) agree with them. On the other hand, if I want to understand the process of learning from a certain kind of data, then I will use various types of flat priors in an attempt to keep things simple so I can see what is going on.

I would like to finish with this (long) quote from physicist Ariel Caticha, that says similar things but in a different way.

“Different individuals may hold different beliefs and it is certainly important to figure out what those beliefs might be — perhaps by observing their gambling behavior — but this is not our present concern. Our objective is neither to assess nor to describe the subjective beliefs of any particular individual. Instead we deal with the altogether different but very common problem that arises when we are confused and we want some guidance about what we are supposed to believe. Our concern here is not so much with beliefs as they actually are, but rather, with beliefs as they ought to be. Rational beliefs are constrained beliefs. Indeed, the essence of rationality lies precisely in the existence of some constraints. The problem, of course, is to figure out what those constraints might be. We need to identify normative criteria of rationality. It must be stressed that the beliefs discussed here are meant to be those held by an idealized rational individual who is not subject to practical human limitations. We are concerned with those ideal standards of rationality that we ought to strive to attain at least when discussing scientific matters. Here is our first criterion of rationality: whatever guidelines we pick they must be of general applicability—otherwise they fail when most needed, namely, when not much is known about a problem. Different rational individuals can reason about different topics, or about the same subject but on the basis of different information, and therefore they could hold different beliefs, but they must agree to follow the same rules.”

About Brendon J. Brewer

I am a senior lecturer in the Department of Statistics at The University of Auckland. Any opinions expressed here are mine and are not endorsed by my employer.
This entry was posted in Uncategorized. Bookmark the permalink.

1 Response to Subjective “vs.” Objective Bayes

  1. Entsophy says:

    I don’t find the distinction helpful within my own thinking either. Suppose we’re trying to estimate the weight of an Elephant. I could use a prior distribution which says basically “it’s some were between 15,000-20,000 lbs”.

    I could have arrived at this by knowing definite upper and lower bounds which had been measured in some way. For example I may know that the elephant walked over a bridge that could only hold at most 20,000 lbs, which places an upper bound on it’s weight.

    Or I could have asked a biologist who specializes in elephants to guess the weight.

    The former I suppose would be called an “objective” bayes while the later would be “subjective”. On the other hand, if the true weight is 17,000lbs their both objectively right, while if the true weight is 7,000lbs they’re both objectively wrong. So in that sense perhaps I can think of both as “objective bayes”, which is how I tend to think of it.

    Having said that, using the distinction between subjective and objective bayes does help when talking to Frequentist. Otherwise, it’s just about impossible to carry on an intelligent conversation with them.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s