I don’t understand the cognitive function of sleep and dreams. The following is based on my anecdotal experiences, and should not be mistaken for the output of rigorous academic research.

The most powerful problem solving technique I employ is to use the last few hours before sleep to cognitively prepare for the problems I have to solve the next day. This preparation generally has 3 stages:

1. Deepening my knowledge of the specific problem – I try to absorb everything I can about the problem I have to solve. This generally takes the form of studying the existing code, related systems orchestrated by that code, the specific and implied requirements I have to meet, and the constraints I have to operate within.
2. Deepening my knowledge of the general problem – This is research that by-and-large is reading articles. Ultimately, I am trying to gather as many options as possible for solving the problem.
3. Previsualized Trial and Error – I imagine different solutions to the problem, and try to predict their outcomes. This is pure imagination, away from a computer. I run thought experiments where I control the variables, run the experiment, and theorize as to the potential results. These results are not specific, but are meant to clarify a path forward. You can think of it as me building a tree of experiments with each result in a fork in the road, and each fork leading to other forks. Over the years, I’ve gotten better at constructing these experimental branches without losing my place.

In the course of an evening, a rough time sequence for a non-trivial problem might be:

• 6pm – 7pm – Understand my problem
• 7pm – 8pm – Understand potential solutions
• 8pm – 9pm – Understand all the possible ways the problem can be solved

The last step is done in the shower, to the point where the shower’s purpose is primarily to think. The tougher the problem, the longer the shower. There have been times where I’ve thought so deeply about a problem, that I’ve exited the shower not clear if I had actually performed the traditional tasks of a shower, and then have to take a 2nd actual shower in case I was just standing under hot water for half an hour doing nothing but thinking.

If I am lucky, the output of the shower’s previsualized trial-and-error experimental tree is a concise checklist of solutions to try. The process of writing this checklist takes loose, abstract, fuzzy approaches and converts them into simple concrete steps to follow the next day. Crafting this list is normally a quick exercise, but often requires iteration to determine the right phrasing and sequence of what to try.

I then watch movies or TV shows on my iPad, and fall asleep generally around 11pm. The next morning I wake up naturally (no alarm) around 8am. I rarely if every refer to the list of concrete steps I made the night before.

That next day, I work in state of confident joyfulness:

• I’m generally in a good mood; not scared or apprehensive, but upbeat and positive.
• I have a sense that I already have solved the problem.
• I am very resilient to things not working out as planned, and can abandon an approach and move on very quickly to try another approach.
• I often re-frame the problem into a different problem that’s easier to solve
• I am working purely off of instinct, and almost never Google anything beyond refreshing my memory on syntax or APIs.
• I never compile, run, or test the code I write, I just write the code for hours while in a state of deep concentration.

The net result is that I iterate through dozens of potential solutions very quickly, one of which typically is a solution to the original problem, or a solution to another problem that indirectly solves the original problem.

This process evolved gradually over many years as I took on lead designer and implementer roles for sprawling software architectural problems. Solving problems without clear definitions demanded a style of abstract problem solving you don’t encounter with purely algorithmic deterministic problems where a known input should produce a known output.

Please let me know in the comments below if this process works for anyone else. If its success is based fundamental rules of human cognition, it should work for anyone. Additionally, if anyone can help explain this phenomenon in terms of peer reviewed academic research, please post links in the comments below.