Why is it so hard to find out exactly what is needed when designing changes to a system?

When adding functionality to databases I always like to dive in and start adding extra columns and new tables. But at some point you need to find out what is needed.

The process of gathering requirements that I have followed goes something like this.

My boss tells me that we need a system to solve x problem. This is the broad overview of what is needed.

Next I need to find out specifically how the system is being used and what things need adding. Discussing the specifics with my boss often results in a high level of confusion. I always try to speak with the users that will be using this system on a daily basis. If I create something that isn’t liked people will avoid using it and it won’t solve the original problem so I always think it is very important to speak to the day to day user.

After I have an idea of what is needed, I build something. Once I have something that sort of works, I will try and demonstrate this to the user to gather feedback. This feedback is invaluable as it will often reveal if I am going in the right direction and reveal missing requirements that need to be incorporated in the finished solution.

I will often repeat the last stage a few times especially if there is a big change needed, once I am happy I will demonstrate to my boss before deploying the solution.

After deployment there is often a period of fixing issues and gathering feedback before I can consider the project finished.

This whole process is very time consuming as there is a period of changes being tweaked back and forth following feedback. A better way is to gather a detailed specification of what is needed and work towards delivering that.