Background
Our goal for the project is to produce a system that is accurate, finegrained, and timely. The primary measures of forecast accuracy are for season start and peak; the magnitude of cases is generally a secondarily concern. In addition to the general public, an important target audience for GFT has been public health officials, who can benefit from reliable daily
estimates and often make far-reaching decisions based on predicted flu incidence (such as how to stock and distribute vaccine, and the content of public health messaging). During the development of GFT we met regularly with a variety of health officials, and we convened with more than a dozen leaders from around the world in 2010.
The original GFT model was created in 2008 and released in multiple countries. The country selection was limited by availability of "ground truth" data in the form of incidence reports of ILI, typically provided by a national or international public health agency. The flu surveillance data itself was publicly available or acquired via a partnership license.
Since the initial model’s release, there has been one update in response to slightly underestimating 2009 H1N1 swine flu (PloS 2011). From the launch in 2008 until the 2012-13 season, the highest estimation error for national flu incidence was 1.13 percentage points (week starting Jan. 1, 2012: CDC data 1.74%, GFT estimate 2.86%), and the mean absolute error during this period across all weekly estimates was 0.30 percentage points. However, in the 2012-13 season, the overestimation peaked at 6.04 percentage points, an estimate more than twice the CDCreported incidence (week starting Jan. 13: CDC data 4.52%, GFT estimate 10.56%). (Also see Nature 2/13/13, When Google got Flu Wrong for an
external report.) This paper addresses several questions related to our model’s recent performance: Why were this
season’s predictions so high? Is our model too simple? Were there unforeseen side effects from the 2009 update? Does this reveal a phenomenon not captured in incidence data provided by the US Centers for Disease Control and Prevention (CDC)?