Saturday, July 23, 2011

The difference between polls and forecasts

Update(07/25):  " I realized that polls and forecasts are close. They take coarse signals, and try to derive the "state of the world'' from them." says basically nothing. Sorry

I am going to write a couple of posts on polls and forecasts, because this is one of the topics I am working on right now. Basically, I want to understand what information  we can get through polls. The motivation was quite simple:

  • Polls are used and have predictive power(Nate Silver relied on polls for the 2008 election)
  • Polls are often wrong(The French election of 2002 did not exactly go as expected. The Dewey/Truman contest in 1948 did not go as expected either)
The conclusion of that was that polls are partially informative, and it could be interesting to try to understand how much information is actually conveyed. So this led me to think about polls. Then I realized that polls and forecasts are close. They take coarse signals, and try to derive the "state of the world'' from them.

Let me give just two quick examples of what I mean by polls and forecasts. Polls are all the number you see that tells you the share of the vote Obama get in 2012 or how many people want to raise the debt ceiling. Forecasts try to predict what GDP growth will be in 4 months.

Polls and forecasts provide various incentives. The initial discussion on rational expectations equilibrium, by Grunberg and Modigliani(1954), started with the question on whether there exists an accurate forecast. The issue is similar to the Heisenberg principle: once the poll or the forecast is out, it changes the information set. This means that even if it was "accurate" in the first place, the very act of its revelation will probably make it inaccurate. If the polls are saying that the National Front is at 30%, some people might consider turning out at the polls while they would have stayed home otherwise.

Is that a problem? My feeling is that it depends on the goals of polls and forecasts.  Polls and forecasts seem to differ in their objective.

In theory, we want the poll to provide accurate information, in my opinion. The reasons polls and forecasts are made is in order to provide information. In forecasts, this seems obvious. Decisions are based on forecasts, and we can assume that the better the information, the better the decision. One issue here is that if a forecast changes decisions, it might change some assumptions on which the forecast is based, thus making the forecast inaccurate, and the decision inefficient. Therefore, the forecast is more about rational expectations: it has to take into account the impact it will have on the very assumptions it is based on.

In polls, things looks different. Why do we have polls in the first place? Polls measure the state of the world at a moment in time. From these, decisions are made. For instance, political candidates might change their platform. The decision to take a vote on an issue might change. A company might change its motto or its product. This means that the poll should provide the ex-ante accurate information, and should not care about its impact on what it polled. Agents will react to the poll, and in a perfect and infinite world, we could probably assume that this back and forth between agents and polls will end up in an equilibrium. The issue is that this infinite world does not exist: we do not have an infinite time for polling, and the agents are not infinitely rational.

Therefore, forecasts aim at being accurate ex-post while the polls only aim at being accurate ex-ante. Those are the criteria on which they should be evaluated.

Why is this relevant? I started thinking about rating agencies, given the fuss about their behavior in the European debt crisis and the raise of the US debt ceiling. Here is a primer. Rating agencies are not a real problem. The incentives are bad, of course, when they are paid by the agent they rate. However, the main issue is that their rating are taken into account by law. For instance, one big issue of the last couple of weeks was that the ECB wouldn't accept Greek bonds as collateral if Greece was considered in default. Some pension funds can only invest in tripe A bonds, from what I understand. 

This is the real problem. Even with the negative outlook on US debt by Moody's and S&P, the interest rate on US Treasuries did not move. If there are no rules to force them to invest depending on ratings, investors might not care about the rating agencies, if they think their rating is not just. My gut feeling is that rating agencies are "polls", and should be treated as such: they provide information at a date in time, and agents might respond to it. But there should be no strict constraint in how people act on this information.

No comments:

Post a Comment