Twitter is using AI to intelligently crop image previews

Twitter added the ability to upload images back in 2011, and while many people take advantage of that feature, one of its big drawbacks is crappy cropping. As Twitter engineers explained in a recent post, the platform automatically crops image previews for the sake of consistency, but these crops usually focus on the center of the image... often at the expense of the photo's subject.

A poorly cropped image may hide the most interesting aspect of the photo—instead presenting a glimpse of a wall, empty sky, or something else similarly boring. And that adorable photo of Fido is a lot less adorable when it's cropped right through the center of his head.

According to Twitter engineers Zehan Wang and Lucas Theis, the company at one point used facial recognition to somewhat solve this issue. With that, the system would identify the most prominent face in an image and base the crop around it. The system wasn't perfect, though, nor relevant to images without faces.

A better system, the researchers explain, is one that focuses on saliency—that is, on the parts of the image that are prominent and mostly likely to be noticed. In other words: the most 'eye-catching' part of the photo.

"In general, people tend to pay more attention to faces, text, animals, but also other objects and regions of high contrast," the duo explain. While a neural network can be trained to identify the salient parts of an image, it presents its own issue: it is too slow to put into production.

However, the team found a solution to that problem—one that enables Twitter's platform to immediately detect the most 'eye-catching' part of an image and then crop with that at its center. The end result are image previews more likely to contain interesting elements that, as demonstrated in the screenshots above.

Twitter began rolling out its improved image preview cropping earlier this week, which means all of those really pretty landscape photos and product shots that pop up on the DPReview Twitter should be cropped a lot better in short order.

[Read More ...]
Twitter is using AI to intelligently crop image previews was originally posted by proton T2a