This thought experiment could have many different outcomes. When you started the project, you guessed at the right direction, but the benefit of hindsight may suggest a better way. Parts of the project may be kept around only because of the sunk cost fallacy. Maybe you’ve learned more about what the requirements really are. Maybe it’s time to kill this project and start a new one. Maybe the project is too long or too complicated and should be broken down into several smaller projects. Maybe your plans for the rest of the project need to change. Or maybe you can have more complete confidence in your current approach. Most likely you will find at least a few small things to tweak.
Let me give you a hypothetical example. Suppose that you have a software system with two threads: one that produces data objects and one that consumes them. You do extensive performance profiling and find that both threads run too slowly, so you spend some time speeding up both threads. Eventually, you speed up the consumer thread until it only takes 12 CPU instructions per incoming data object. If you were under time pressure to deliver a critical patch, then you’d be happy with your optimizations and you would ship what you had. But if you thought through what you would do if you were to start over, then you would probably decide that using a consumer-producer multithreading architecture is the wrong strategy entirely because a single thread is good enough.
Around the start of 2013, when I was working on my PhD, I was wandering in circles around a particular problem: I wanted to discover the conditions required for a pair of Q-learning agents to learn to take turns in a simulated context. I parameterized the agents’ reward functions and tried to discover a pattern that would compactly describe how to incentivize turn taking. I went down a few blind alleys: genetic algorithms, linear decompositions and staring blankly at a simple visualization of the problem. Eventually, I realized that game theory is the correct approach for analyzing this kind of multi-agent scenario. At that point, I revised the entire plan for my PhD so that the next part was focused almost entirely on game theory.
Most of your time must go to detail work, so when is it time to step back and look at the big picture? Unfortunately, you probably won’t reach an epiphany moment where you know its time to rethink the project, like I did with my PhD. No tree will scream “this is the wrong part of the forest.” At the start of this essay, I suggested that you imagine starting over "every once in a while,” but a definite schedule may be helpful. How about 1 January each year? Or the first day of each month? What projects are you working on now? Do you have one where you’ve focused exclusively on details for a long time? Maybe now is the time to imagine starting that project over
