Ben HalpernLots of people will get in touch with you about improvements to your developer-oriented product or library. This feedback is great, but take it with a grain of salt, there is a much quieter group with much bigger struggles.
The people who are getting in touch with you are the ones who aren't having the biggest problems. They understand your docs enough not to fear embarrassment of asking the wrong questions. They overcame missing context themselves, only to be able to ask questions which represent their group.
Survivorship bias or survival bias is the logical error of concentrating on entities that passed a selection process while overlooking those that did not. This can lead to incorrect conclusions because of incomplete data.
They did not get tripped up by implied dependency installation steps. They did not get lost due to assumptions of typical CLI behavior that you did not document. They survived to give you feedback.
During World War II, the statistician Abraham Wald took survivorship bias into his calculations when considering how to minimize bomber losses to enemy fire. The Statistical Research Group (SRG) at Columbia University, which Wald was a part of, examined the damage done to aircraft that had returned from missions and recommended adding armor to the areas that showed the least damage, based on his reasoning. This contradicted the US military's conclusions that the most-hit areas of the plane needed additional armor. Wald noted that the military only considered the aircraft that had survived their missions; any bombers that had been shot down or otherwise lost had logically also been rendered unavailable for assessment. The bullet holes in the returning aircraft, then, represented areas where a bomber could take damage and still fly well enough to return safely to base. Thus, Wald proposed that the Navy reinforce areas where the returning aircraft were unscathed, inferring that planes hit in those areas were lost. His work is considered seminal in the then-nascent discipline of operational research.
These are the planes which came home for you. Please find a way to account for the ones which never made it.
