Transhuman Expression

The painting robot developed at the University of Konstanz in southwestern Germany is a pioneer project in this field and is presently the only one with a visual feedback system. Much more than just a printer capable of reproducing a flat image, the e-David creates unique works through the application of paint strokes that are irreproducible in terms of their colour blend and the materiality of their layering. The possibility of visual feedback brings up many questions within the contemporary discourse on deep learning, artificial intelligence and robotic creativity. The Collaborating explored further possibilities to exploit the painting robot creatively, and reflected on ideas about the ways in which these could be implemented in the form of software and hardware. A number of questions of wider impact arose as the result of our collaboration: When and why would a semantic method of defining the object in the image be used? Is it an advantage or a disadvantage to paint semantic objects without having a pre-existing cognitive understanding of them? How could I use abstract forms, grammatical structures or mathematic models to achieve more complex surfaces? How would computer language be used to express the intentions of a composition? When and why would different painting styles be used? Further, on a technical level, we had to take into consideration how different materials would react with one another. For example, how could different colours be mixed on the canvas or on the palette? How should the size of the brush be set, and when is it necessary to add glaze? We would have to develop a range of distinct, individual brushstrokes (controlling the velocity and the z-axis) whose characteristics are analogous to those made by human painters in the “real world”, in order to be able to pre-define when, in which order and for which tasks each stroke is to be used. In doing so, we are basically defining and categorizing singular parameters within a library of painterly “acts” and “perceptions”, in order to create a grammatical structure for the “language” of robotic painting. Constructing e-David has provided many challenges in the realms of engineering, robotics, computer graphics and computer vision. We discuss how we arrived at our current painting set up and highlight the strengths and weaknesses found so far, as well as plans for future extensions. On the robotics side, the main issue is working with machines designed for repetitive, precise tasks and applying them for approximate and dynamic processes. Supervision of a painting process using vision or other sensors is challenging, as well as finding methods to process the data in a meaningful way. Seemingly simple tools like brushes are highly complex in their behaviour and using them precisely is difficult. In the talk, we elaborate on current problems and what future plans are to deal with these issues.