AI & Phot. part I: Computer vision, human senses, and language of art

"What is the most important reason for using Computer Vision methods in humanities research? In this article, I argue that the use of numerical representation and data analysis methods offers a new language for describing cultural artifacts, experiences and dynamics. The human languages such as English or Russian that developed rather recently in human evolution are not good at capturing analog properties of human sensorial and cultural experiences. These limitations become particularly worrying if we want to compare thousands, millions or billions of artifacts—i.e. to study contemporary media and cultures at their new twenty-first century scale. When we instead use numerical measurements of image properties standard in Computer Vision, we can better capture details of a single artifact as well as visual differences between a number of artifacts–even if they are very small. The examples of visual dimensions that numbers can capture better then languages include color, shape, texture, contours, composition, and visual characteristics of represented faces, bodies and objects. The methods of finding structures and relationships in large numerical datasets developed in statistics and machine learning allow us to extend this analysis to very big datasets of cultural objects. Equally importantly, numerical image features used in Computer Vision also give us a new language to represent gradual and continuous temporal changes—something which natural languages are also bad at. This applies to both single artworks such as a film or a dance piece (describing movement and rhythm) and also to changes in visual characteristics in millions of artifacts over decades or centuries".
Manovich, Lev (2020). Computer vision, human senses, and language of art. AI & Society. doi.org/10.1007/s00146-020-01094-9

manovich.net

tags: Computer vision, Digital humanities, Cultural analytics, Language of art, artificial intelligence, lev manevich

In the original vision of artificial intelligence (AI) in 1950s, the goal was to teach computer to perform a range of cognitive tasks. They included playing chess, solving mathematical problems, understanding written and spoken language, recognizing content of images, and so on. Today, AI (especially in the form of supervised machine learning) has become a key instrument of modern economies employed to make them more efficient and secure: making decisions on consumer loans, filtering job applications, detecting fraud, and so on.

What has been less obvious is that AI now plays an equally important role in our cultural lives, increasingly automating the realm of the aesthetic. Consider, for example, image culture. Instagram Explore screen recommends images and videos based on what we liked in the past. Artsy.net recommends the artworks similar to the one you are currently viewing on the site. All image apps can automatically modify captured photos according to the norms of "good photography." Other apps "beatify" selfies. Still other apps automatically edit your raw video to create short films in the range of styles. The App The Roll from EyeEm automatically rates aesthetic quality of you photos (...)

Does such automation leads to decrease in cultural diversity over time? For example, does automatic edits being applied to user photos leads to standardization of “photo imagination”? As opposed to guessing or just following our often un-grounded intuitions, can we use AI methods and large samples of cultural data to measure quantitatively diversity and variability in contemporary culture, and track how they are changing over time?

manovich.net/index.php/projects/automating-aesthetics-artificial-intelligence-and-image-culture