‘It’s just a story’ – in a business or decision-making setting – is a contemptuous put-down. “We’ve all got a story to tell,” people say, which of course means: “Yes, but where’s the evidence?” (Sometimes this is framed with a harder edge as what does the science say?)

The evolution of language reveals this direction of travel.  ‘Anecdotalism’ has recently taken on a new, more cynical definition. It used to mean a tendency to tell amusing anecdotes. Now, it’s a failure of a methodology. Academic discourse here is filtering into the mainstream – in part because behavioural economics portrays storytelling as a classic flaw in human judgement. First, we attach ourselves to the story (usually a self-serving one), then we illustrate the tale with cherry-picked facts.

Big data, in theory, was supposed to ride to our rescue, myth-busting the antique crutch of storytelling. As soon as we could simply run the numbers, surely hard facts and numerical rigour would relegate storytelling to the status of campfire curiosity?

Well, it hasn’t worked out that way.

First, the proliferation of data ended up bewildering decision-makers as often as informing them. Choose any question, any data set, any algorithmic tool: what’s not to like? But infinite choice fatigues people – as Spotify users will know – so they often end up doing what they’ve always done (or nothing at all), as though their options were narrowing rather than expanding.

Second, from the perspective of management rather than strategy, saying “the data supports your selection for the team” doesn’t turn out to be a great way to inspire human beings in the real world. Data-heavy coaches can underestimate how teams gel, preferring to focus on assembling under-valued players (like an investment portfolio).

Data gurus Billy Beane (baseball) and Daryl Morey (basketball) – despite their insights and revolutionary impact on their sports – have been less effective at exploiting the non-quantifiable levers that also forge champion teams. Intangibles aren’t necessarily myths just because they are elusive to measure.

Paradoxically, the proliferation of data (and the advance of AI) puts effective storytelling front and centre again. Data might identify the right players, but data can’t make them feel loved. The USP of England cricket coach Brendon McCullum is creating a narrative of freedom and self-expression. Intriguingly, it’s proved robust in downturns as well as booms: despite recently losing the Test series against India 4-1, there has been little internal or external scrutiny of his position. Win or lose, the story of a team seeking risk and adventure rolls on.

Narrative buy-in, a sense of mission and belonging that transcends personal ambition, mitigates anxiety and discontent. Workers at Fiat, even in hard times, had a saying about the legendary owner Gianni Agnelli: “Agnelli is Fiat; Fiat is Turin; and Turin is Italy”. The question that ranks all others is: What are we about?

By now, you will have anticipated the obvious rejoinder about the role of storytelling. What about when a gifted storyteller loses track of underlying realities (or never worried about it in the first place)?

At the problematic end of this spectrum stands Adam Neumann and the debacle of WeWork. “Through a combination of egomaniacal glamour and millennial mysticism, the Neumanns sold WeWork not merely as a real estate play,” concluded the business author Gabriel Sherman. “It was a movement, complete with its own catechisms (What is your superpower? was one). Adam said WeWork existed to ‘elevate the world’s consciousness’.”

Neumann would have made a fascinating extra profile in Feet of Clay, Anthony Storr’s classic study of charismatic leadership and gurus. Storr’s thesis is that gurus, wherever they stand on a moral axis, share the ability to galvanise followers behind a revelatory story or vision.

But WeWork failed because the story was a lie. And that’s what defines a storyteller’s long-term legacy. It’s not enough to have the confidence and the technique to make stories look right, the stories have to be right, too. And time will eventually sort that out, with no mitigation or mercy.

There’s another paradox within modern attitudes to storytelling. As social science decries anecdotalism, the real masters of truth-seeking via storytelling – artists – find themselves pushed to the margins of cultural life. We’ve lost confidence in truths that can’t be verified, which constitutes a large chunk of the important ones. George Eliot’s Middlemarch is made up, but it’s still importantly “true”.

(As I write that paragraph, I’m conscious of not having evolved much in 27 years. My first published article, written as an undergraduate caught between History and English, began: Can novelists tell us more about the past than historians? Can fiction offer us deeper, different truths? And I agree with the implied answer even more strongly now.)

Conversely, much academic history consists of ‘true facts’ assembled into an unconvincing whole. One of history’s wrong turns was trying to turn itself into a quasi-science. But there is no verifiable “method” for history beyond sound and expert judgement. Historical interpretation necessarily relies not only on the curation of facts but also their imaginative narration.

There’s an analogy here between the study of the past and the demands of present judgements. Piling up an un-sieved heap of facts doesn’t advance historical insight; nor can a mountain of data tell us what decision to make next.

The master of the algorithm, Jeff Bezos at Amazon, always understood this point. “The thing I have noticed,” he’s reflected, “is that when the anecdotes and the data disagree, the anecdotes are usually right. There’s something wrong with the way you’re measuring.”

A data guy who gets the value of stories – an example of uniting insights from two apparently contradictory sources of authority. Instead of picking sides, you have to get the best out of both.

Ed Smith is director of the Institute of Sports Humanities and the author of Making Decisions