Why it has to be learnt
A key component of programming expertise is to be able to imagine how a computer would solve the given problem. In other words, it is the ability to exactly specify a procedure that should be blindly followed by an agent (the computer) to get a solution. The procedure has to be in terms of certain basic operations and a few ways to combine them which are the only things the agent “understands”, or rather, can execute. It turns out to be one of the harder crafts to learn for novice programmers, compared to learning the syntax, or learning about the basic operations and the ways to combine them.
I think the reason for this difficulty is very fundamental. The thinking patterns required when we solve a problem ourselves, are drastically different from those required when we want to program a computer to solve it.
The human approach to problem solving, be it the prehistoric hunting, or a modern mathematical problem, is to act dynamically according to the current state of affairs in order to progress towards a solution. We do have a plan, whether it is formally written down, or in our minds, based on our knowledge and previous experience. But the purpose of the plan is to serve only as an overall direction for our actions. We judge from time to time whether our current course of actions is taking us closer to the outcome picture that we have in mind, and evaluate the plan itself in the process. The plan is nowhere close to a detailed step-by-step instruction sequence to be followed exactly. (In fact there are situations where such plans are required. Following a such detailed step-by-step plan requires discipline and training.)
Further, we do not, and need not, anticipate all the situations that will upset our progress. Our plans do have a certain amount of contingency considerations, but in many situations we revise and adapt the plans.
We have evolved to be solution creators, since the environment in which we are supposed to survive is extremely complex, and any detailed plan will eventually go for a toss. We create new solutions or find new ways if the unpredictable environment upsets our original plans (and it will). We are comfortable with vague plans, and confident that the details will appear in due time. In short, our way of problem-solving makes the road appear as we walk, rather than choosing a path from an already existing road network.
On the other hand, when we program, we are developing a precise, detailed, and fixed plan of action, i. e. we are solving the problem in an apparently unnatural way. We have the responsibility to imagine and plan for all sorts of situations that the world (within the computer) could be in, with clarity. We are no more in our natural comfort zone of only deciding the overall direction in the beginning and let the course of actions adaptively fill in more details. To novice programmers, this required clarity at all steps and levels at once is new and could be overwhelming.
Overcoming this barrier requires practice, like any other skill. Part 2 of this series will present some thoughts in this direction.