Art & Beauty

Self-cannibalizing AI

Artistic Strategies to expose generative text-to-image models
What occurs when machines learn from one another and engage in self-cannibalism within the generative process? Can an image model identify the happiest person or determine ethnicity from a random image? Most state-of-the-art text-to-image implementations rely on a number of limited datasets, models, and algorithms. These models, initially appearing as black boxes, reveal complex pipelines involving multiple linked models and algorithms upon closer examination. We engage artistic strategies like feedback, misuse, and hacking to crack the inner workings of image-generation models. This includes recursively confronting models with their output, deconstructing text-to-image pipelines, labelling images, and discovering unexpected correlations. During the talk, we will share our experiments on investigating Stable-Diffusion pipelines, manipulating aesthetic scoring in extensive public text-to-image datasets, revealing NSFW classification, and utilizing Contrastive Language-Image Pre-training (CLIP) to reveal biases and problematic correlations inherent in the daily use of these models.
The talk will be conducted by sharing various experiments we've done under the umbrella of generative AI models. We will begin with a general idea of how we, as artists/programmers, perceive these models and our research on the workflow of these constructs. Then, we will further elaborate on our exploration of the Stable Diffusion pipeline and datasets. Throughout our investigation, we discovered that some essential parts are all based on the same few datasets, models, and algorithms. This causes us to think that if we investigate deeper into some specific mechanisms, we might be able to reflect on the bigger picture of some political discourses surrounding generative AI models. We deconstructed the models into three steps essential to understanding how they worked: dataset, embedding, and diffusions. Our examples are primarily based on Stable-Diffusion, but some concepts are interchangeable in other generative models. As datasets and machine-learning models grow in scale and complexity, understanding their nuances becomes challenging. Large datasets, like the one for training Stable Diffusion, are filtered using algorithms often employing machine learning. To "enhance" image generation, LAION's extensive dataset underwent filtering with an aesthetic prediction algorithm that uses machine learning to score the aesthetics of an image with a strong bias towards water-color and oil paintings. Besides the aesthetic scoring of images, images are also scored with a not safe-for-work classifier that outputs a probability of an image containing explicit content . This algorithm comes with its own discriminatory tendencies that we explore in the talk and furthermore asks how and by whom we want our datasets to be filtered and constructed. Many generative models are built upon Contrastive Language-Image Pre-training (CLIP) and its open-source version, Open-CLIP, which stochastically relates images and texts. These models connect images and text, digitize text, and calculate distances between words and images. However, they heavily rely on a large number of text-image pairs during training, potentially introducing biases into the database. We conducted experiments involving various "false labelling" scenarios and identified correlations. For instance, we used faces from ThisPersonDoesNotExist to determine "happiness" faces, explored ethnicities and occupations on different looks, and analyzed stock images of culturally diverse food. The results often align with human predictions, but does that mean anything? In the third part, we take a closer look at the image generation process, focusing on the Stable Diffusion pipeline. Generative AI models, like Stable Diffusion, have the ability not only to generate images from text descriptions but also to process existing images. Depending on the settings, they can reproduce input images with great accuracy. However, errors accumulate with each iteration when this AI reproduction is recursively used as input. We observed that images gradually transform into purple patterns or a limited set of mundane concepts depending on the parameters and settings. This raises questions about the models' tendencies to default to learned patterns.

Additional information

Live Stream https://streaming.media.ccc.de/37c3/granville
Type lecture
Language English

More sessions

12/27/23
Art & Beauty
Saal Zuse
Wir machen Sound Grafitti mit Echokammern, produzieren Protest-Jingles und Sprachwerkstätten. Mit diesem Beitrag möchten wir euch zwei Vorgehensweisen zu dieser Art von „Protest mit Sound als Audio Intervention" vorstellen, sowie die künstlerischen, kreativ-technischen Prozesse näher bringen und euch einladen, diese kritisch zu beäugen und zu hören, sich unserer Ideen und Verfahren zu ermächtigen und die dargestellten Formen und Ansätze nach eurem Belieben weiterzuentwickeln und ...
12/27/23
Art & Beauty
Paglen (he/him)
Saal 1
How the history of military and government PSYOPS involving mind-control, UFOs, magic, and remote-control zombies, explains the future of AI and generative media. Along the way, talk attendees will be given an enrollment code to join a specialized CTF/ARG game called CYCLOPS that explores these themes and that will run the duration of Congress.
12/27/23
Art & Beauty
Saal Zuse
OPENCOIL and the fine art of appropriating micro-mobility services for fun and debate.
12/27/23
Art & Beauty
Scott Beibin
Saal Zuse
Date/Time: 27 December 2023 - Wednesday @ 21:10 CET +++ Simulating the Acoustics of Mars for a Concert of Martian Music by Scott Beibin (aka Ptelepathetique) +++ During Mission 286 in November 2023 at the Mars Desert Research Station (MDRS), Analog Astronaut and crew artist Scott Beibin performed several concerts of original live musical compositions during a two week immersive astronaut training. +++ The concerts were played through a custom audio filter based on data gathered by the NASA Mars ...
12/28/23
Art & Beauty
Saal Zuse
What does it take to create a "wild animal"? While one might think "wildness" implies the absence of humans, in the age of the anthropocene and rapid climate change, the opposite is the case. It requires the development of an extensive, more-than-human-infrastructure. Our talk is based on artistic research into the ongoing rewilding project of the Northern bald ibis (Waldrapp), a large migratory bird, that has become extinct north of the alps in 1621 and are being released into the wild since ...
12/28/23
Art & Beauty
Robert Seidel
Saal Zuse
Exploring the transfer of Seidel's experimental films into physical spaces reveals challenges that are intensifying with advances in machine learning, dissolving the lines between original and imitation. In this more or less silent restructuring of society, artists become templates for a digitally assembled future, challenging traditional hierarchies as history collapses into the present.
12/28/23
Art & Beauty
LordSpreadpointAmiga
Saal 1
The demoscene is an underground computer art culture. The Speaker is a member of the Demoscene since the 1980ies and gives insights how it is now and how it was back in the days and how you can participate!