As with most stories about artificial intelligence, this is the key:
“The primary difference between Facebook’s and Instagram’s filtering tools appear to be that Instagram’s AI system was specifically “trained” with the spam and offensive comments that specifically appeared on its own service, the Wired report shows. Instagram had to hire contractors to comb through spam and gross comments to teach the machine-learning system to recognize patterns and identify whether potential comments are junk or distasteful.”
Someone had to tell the AI system what was junk and what was inappropriate speech. So it will be interesting how the inherent bias plays out. What is distasteful has already been decided by a group of humans, the AI will simply carry out their definitions automatically. This is the danger of censorship. Instagram isn’t the government, they are free to censor content on their network, but push it too far, or get it wrong, and it could destroy the trust between the network, and the people who use it. We’re seeing that struggle play out all across social media.