People who actually study history and try to do it objectively realize this. There were many, many bad things done to people in the name of colonialism. The Belgian Congo is a huge case in point.
My parents are from India, and while British colonialism did provide some benefits it also created much damage.
In Ireland, British colonialism also led to both some good things as well as lots of bad things.
It's the nature of the beast. Love of money drove colonialism. Greed does not bring out the best in human nature. Even if colonial powers, as a whole, lost money on the deal, individuals became very wealthy and powerful. And that was their focus, not the suffering of their own or native peoples.