Non-Tech CEO › Book notes › The thinking behind them
Every book note here asks whether a book is true — and, separately, whether it works. Those are different questions, the answers disagree more often than not, and almost nobody reports both.
Is it true?
Does the evidence hold when you go and check it? Did anybody count? Is there a real argument against it?
Does it work?
Do people who actually use it report that something changed? At what scale, and for how long?
Multiply the two and you get one number — and a book with no evidence scores zero however much it helps people. That is the mistake, written as arithmetic.
Four readings, kept apart. The verdict is a set of coordinates, not a score — there is no honest way to collapse it, so it isn’t collapsed.
Take almost any business or self-help book from the last twenty years. Check its claims properly and a good proportion fall over — a study that never replicated, a figure that traces back to nothing, an expert misquoted by the first person to repeat him.
Then look at the readers. Many will tell you, plainly and with no reason to lie, that the book changed how they work.
Both of those are true at the same time. A single verdict has to throw one away.
That is the whole reason for separate scores. A review saying the evidence is thin, therefore this is worthless has quietly told a few million people they were fools. One saying people love it, therefore it’s good has helped a bad idea travel further. Neither is honest, and both are avoidable by asking more than one question.
The disagreement is not new, and serious people have sat at both ends of it.
Ayn Rand is the clearest modern statement of the position that truth is objective, independent of anyone’s convenience, and not up for negotiation. It is a bracing position with an obvious strength: it makes it very hard to talk yourself into something because you want it.
These notes do not take that position, and it is worth saying why. Not because it is wrong about reality, but because it answers a question nobody is stuck on. Almost nobody reading a business book is confused about whether facts exist. They are stuck on something else — this has no evidence behind it and it seems to be working for people, so what do I do with that? A strictly objective account has nothing to say there except ignore it, which is not usable advice.
A century ago William James put the opposite case, in the language of a businessman:
“What, in short, is the truth’s cash-value in experiential terms?”
William James, Pragmatism, Lecture VI, 1907
His argument, roughly: an idea’s truth is not a badge it wears. It is what the idea does — whether it can be tested, leant on, and made to pay its way in experience.
Taken to its limit that goes too far, and it is where the pragmatists get into trouble: something can be useful to believe and still be flatly untrue. But the instinct is right, and it is the half most reviewing throws away.
The most useful version comes from a statistician rather than a philosopher.
“All models are wrong but some are useful.”
George E. P. Box, section heading in ‘Robustness in the strategy of scientific model building’, 1979
A live example of why this matters
That line is nearly always dated to Box’s 1976 paper. It isn’t in it. The 1976 paper says something quieter — “Since all models are wrong the scientist cannot obtain a ‘correct’ one by excessive elaboration” — and the famous wording appears three years later, as a section heading.
It took about four minutes to check. It is a small thing, and it is exactly the kind of small thing that is wrong in most of what you read — not invented, just repeated until nobody looks.
Two things changed, and they compound.
Checking got harder while claiming got easier. Producing a confident, fluent, well-organised paragraph containing a false citation now costs nothing and takes a second. Verifying that same paragraph costs what it always did. That gap has never been this wide, and it widens every year.
And the evidence itself turned out to be less solid than advertised. A great deal of published research in psychology and the social sciences — the seam most business books mine — has failed to reproduce when people went back and tried. Some of the most-quoted findings in the genre are among them.
So “is there a study?” stopped being a good enough question. Sometimes there is a study and it means very little.
Which leaves a reader unable to trust the citation and unable to trust the enthusiasm either. The answer is not to pick one. It is to report both, separately, and let you decide.
Every note is scored in four layers. You get the shape in about fifteen seconds, before you read a word of the review.
What I hold myself to
Every book read in full. Never from a summary, never assembled from other people’s reviews.
A real, named critic in every note. Not a devil’s advocate invented to look balanced — someone who actually disagreed, in public, with their name on it. Where nobody has, the note says so, because that is a finding too.
Both halves reported, always. Where a book has nothing behind it and helps people anyway, you get both sentences rather than the one that makes a tidier verdict.
Scores are judgements, not measurements. They are mine, they are argued for in the note, and you are meant to disagree with them where you do.
The reason I built this for books is that I needed it for something else first. Working out which AI tools were worth a small company’s time meant reading a great deal of confident material with nothing underneath it, and noticing the two questions had come apart there too. Plenty of tools have no evidence and clearly help. Plenty have impressive numbers and change nothing.
Books turned out to be the honest place to practise, because you can check a book. The claims sit still, the sources can be chased, and if I get it wrong somebody can tell me.