Polling Is Too Hard—for Humans

In 1936, after polling its readers, the Literary Digest famously predicted a landslide victory for Alf Landon. On 2 November 1948, based on widespread polling that all pointed in one direction, the Chicago Tribune famously headlined its early edition, “Dewey Defeats Truman.”

Polls have been making mistakes ever since, and it’s always, fundamentally, the same mistake. They’re based on a representative sample of the electorate that isn’t sufficiently representative. 

After the election of 2016, in which the polling was not only wrong but itself might have inspired decisions that affected the outcome—where the Clinton campaign shepherded its resources; whether James Comey would hold a press conference—pollsters looked inward, re-weighted various variables, assured us that the errors of 2016 had been identified and addressed, and then proceeded to systematically mis-predict the 2020 presidential election much as they had four years earlier.

After a century of often-wrong results, it would be reasonable to conclude that polling is just too difficult for humans to get right. 

But what about software? Amazon, Netflix, and Google do a remarkable job of predicting consumer sentiment, preferences, and behavior. Could artificial intelligence predict voter sentiment, preferences, and behavior? 

Well, it’s not as if they haven’t tried. And results in 2020 were mixed. One system predicted Biden’s lead in the popular vote to be large, but his electoral college margin small—not quite the actual outcome. Another system was even further from the mark, giving Biden wins in Florida, Texas, and Ohio—adding up to a wildly off-base electoral college margin.

One system, though, did remarkably well. As a headline in Fortune magazine put it the morning of election day, “The polls are wrong. The U.S. presidential race is a near dead heat, this AI ‘sentiment analysis’ tool says.” The AI tool predicted the popular vote almost perfectly.

That AI company is called Expert.ai, and its Chief Technology Officer, Marco Varone, is our guest today.