This is just one example of why algorythms should not be in charge of everything! For now, in many cases, there is a human “review” option, but in an increasingly AI-run world, the goal is to do away with that. It will not end well when/if that goal is reached. –STL
In a recent article, the New York Times tells the story of a father who attempted to seek telemedicine treatment for his son amidst the coronavirus pandemic, sending photos of his son to the doctor for inspection at the request of the medical office. Google tagged the images as child abuse material, disabled his account, and reported him to the police.
Source: A Dad Took Pictures of His Son at a Doctor’s Request, then Google Labeled Him a Sex Offender