If we could have bought shares in the concept of ‘process,’ we’d be rich today.

Having leapt far beyond its sensible origins in the law – where ‘due process’ protects against hasty errors – process has invaded all aspects of life. Grief is framed as a process. Psychoanalysis insists people commit to the process.

Recruitment is process-led (a rule of thumb: bad recruiters congratulate themselves on having followed ‘a really good process’). Popular non-fiction sells us ‘the creative process’ at £6.99 a book. Academia agonises over its grading process: an essay must be marked against often byzantine ‘Intended Learning Outcomes.’ After ‘teaching to the test,’ we now have ‘writing to the rubric.’

Decision-making is now expected to follow a definable process that is laid out in advance and according to tenets, as though all aspects of life must be shuffled towards the scientific method. This creeping scientism is bound up with a loss of confidence in the humanities: process feasts on the insecure longings of disciplines and domains that lack a ‘methodology.’ In the absence of real science, let’s at least posit a process. Behavioural economics has put the boot in too, recasting the concept of good judgement as merely ‘bias’ repackaged as self-serving story-telling.

Process is the first cousin to another booming stock: transparency (we’ll come to that one another time). The surest way to overturn an outcome is not establishing misjudgement, but opening up a fault line according to ‘a failure of process.’ It’s a matter of time until the concept of process itself must prostrate itself to further process, doubtless in the pursuit of ‘best practice.’

Managerialism has given process the greatest leg-up of all. As middle management has expanded exponentially, process is an endlessly useful ally. It creates work and fills hours, while also levelling the playing field against more talented rivals by boring them into submission. Process is the unimaginative bureaucrat’s best friend.

Safely hidden in the comforting thickets of management waffle and compliance, careerists can survive for decades while making consistently poor decisions that emerged from processes which looked good on paper. As the baseball sage Yogi Berra quipped, “In theory, there is no difference between theory and practice. In practice there is.”

What follows is not an attack on process in all contexts (far from it) but the workings out of a curious and sometimes puzzled decision-maker. ‘Running a process’ fills time and appeases line managers. But does it really pay its way?

At the outset, here is what I am not saying. I’m not arguing that processes can’t help. In some contexts, in particular judicial and quasi-judicial domains, a thorough process is essential to civilised values.

And process can clearly catch sloppiness and filter out errors. When I was the chief selector for England cricket, we had a rigorous process for picking England squads. If I am honest, however, I’d say that process was most useful at helping us to identify avoidable errors and make helpful tweaks. It protected us against acting hastily, emotionally or being reactive. But process alone didn’t unearth major solutions. Yes, there is value in re-examining, looking again, trying another perspective, going over one more time, filtering, cleaning. But you also need to mine the gold first.

That’s the first part of my argument: process, even when it’s helpful, cannot ever be sufficient. At best, it’s only one part of real-world problem-solving.

Further, we need to know when to guard against process as well as when to attend to it. When I began writing Making Decisions, I revisited three years’ worth of notebooks from my time selecting England teams. All taken together, one trend troubled me.

At the beginning, ideas and solutions dominated. By the end, reminding myself not to forget internal procedures had crept into the picture – checklists of things, none of which advanced the England team at all, but which might cause professional hassle for me if I forgot to do them. I was still perceived as ‘leftfield,’ but I’d been dragged progressively into the world of process. Which is where big corporations like you to be. For their sake. But not for yours.

Pushing back is only one part of it. There is also the question of protecting yourself from anti-creative environments – and people (even the nice ones). There is a scene in the Italian television series Inspector Montalbano, where the hero-detective is challenged by a diligent colleague for keeping him at arm’s distance. Montalbano answers that he cannot develop solutions and ideas – which often begin as half-formed hunches which he cannot explain – while being constantly scrutinized by a hyper-rational mind.

How to escape the process trap? Owner-entrepreneurs stay creative because they have the privilege of setting things up around their own needs. The designer Paul Smith argued recently that the key to his success was not having an email account. An envious tip of the hat.

The way we live and work makes creativity harder. Life is simultaneously more observed – thanks to the dread pincer movement of open-plan offices and interminable video-conferencing – and also more atomised. We are more visible than ever – through the glass door, over the glass screen – and yet more likely to be half-present at best, checking something else on another device, carving up our inattention to least displease everyone. This is the liminal space of uncreative compromise: neither socially together nor usefully alone.

With advancing AI, we know that human beings’ best chance of adding enduring value is to stay good at the most human bits – analogy, surprise connections, joining dots that aren’t obviously linked. Creativity.

I distrust guides to ‘the creative process’ because a genuinely creative process is significantly unknowable. But I hope we can improve the conditions where it might, with luck, break free.

It takes bravery (often the quiet, unnoticed kind), but we should trust a little less in process and more in solitude. Think more, indulge our curiosity, naively follow the thread, protect freshness. Which is another way of saying that to arrive at better decisions, we might have to take on the hardest challenge of all: to live a bit better.

Ed Smith is director of the Institute of Sports Humanities and the author of Making Decisions