One of my standard wisecracks is that software developers are the most optimistic people ever born — which is why their estimates to completion are so unreliable

.software developers optimistic

Well, it turns out there is a psychological measure for this effect — the Dunning-Kruger Effect. A good, quick overview can be found here: http://www.sw-engineering-candies.com/blog-1/how-good-i-really-am-as-developer–the-dunning-kruger-effect-in-software-engineering.

In summary, the Dunning-Kruger Effect says that less-talented individuals significantly overestimate their abilities as measured on quantitative tests (bottom two quartiles), while the more talented individuals somewhat underestimate their abilities (top quartile). The third quartile self-assesses fairly accurately, however.

In other words, the less I know about a process, the less reliable my estimates will be and the more hubris influences my behavior — while the more skilled I become, the more humble I become, as well.

Among other conclusions, the Dunning-Kruger Effect suggests that agile development teams should always strive to have at least one senior, proven developer on-board to keep from experiencing — and worse, communicating – over-optimism.