The Optimal Stopping Problem

Today I have been thinking about the optimal stopping problem. I first heard about it in Algorithms to Live By, which is a book of very interesting sciencey life strategies.

It’s a puzzle from the field of probability in mathematics: put very simply, how long should I keep searching for a new house/job/partner/t-shirt, and how do I know I’ve found the right one?

The general consensus is that you should stop your search 37% of the way through the period of time you expect to spend on your quest.

Once you are 37% of the way through, you should snap up the next option that is equal to, or better than, the best of what you have seen so far. On probability, that’s an excellent option, and if you keep searching, it’s gone forever.

For example, if you plan to give yourself 3 months to find a new job, you should search the market for the first month or so — just looking at what’s out there, not acting yet, getting a sense of what you want and what’s possible for you to get — then apply for the role(s) that look killer from that point on. Same for buying a house, or a car, or any other big search.

Spend about a third of your time just being curious and observant, and then act.

(Several of the mathematicians that developed this 37% number applied it to finding a spouse; one could not stop himself at the optimal stopping point, carried on through to about 110%, and then went back to the lady at whom the puzzle said he should have stopped. They lived happily ever after.)

I like this puzzle. It is a deeply satisfying solution to the overthinking brain.

It gives me a framework to wrangle masses of information into a workable strategy.

It also helps me identify when I’m becoming paralysed by indecision: if I’ve already spent a lot of time on something, then I can trust that actually, I probably know what I want by now, and any more time spent is just procrastination.

Optimal stopping might not be perfect.

I could miss out on something marginally better. But it saves me from the tyranny of wasted time and rumination and keeps my life moving forward in a dynamic and exciting way. It gives me permission to stop fretting.

Asking yourself about the optimal stopping point for a particular problem is a great writing prompt to start digging down into any restlessness and dissatisfaction that might be lurking beneath the surface. It shows you how far along your quest you are, and what you’ve learned so far. It shows you what you want, and don’t, and why.

And if you don’t want to stop at the optimal stopping point, that opens up a whole new array of questions to chew on:

  • Why not?
  • Is there some reason this doesn’t apply, and if so, is that true?
  • What do I think is out there — what am I holding out for?
  • Am I willing to risk missing the next best option in pursuit of something better, even if that something might be vanishingly rare?

Funnily enough, coming from a field of pure theory, the optimal stopping problem is very much about action.

It’s about spending the time to understand yourself — your needs and desires — and then trusting your own judgement enough to make a move.

And that’s the whole point of living, isn’t it?