Two S+T+ARTS AIR Residencies Begin at HLRS

Taking the concept of air as a metaphorically rich starting point, the S+T+ARTS AIR open call invited proposals addressing two themes: resilient interspecies urban ecosystems, and human-AI ecosystems. In total, 127 artists and artist groups submitted proposals, with S+T+ARTS AIR awarding 10 applicants with residencies. Two of the winning projects will be conducted at HLRS, while the supercomputing center will also support other S+T+ARTS AIR projects in other ways. In mid-January, artists Hiền Hoàng and the Uncharted Limbo Collective held kickoff meetings with the HLRS Visualization Department, and their collaborations will run until September.

An experiment in human-AI collaboration

Uncharted Limbo Collective is a team of creative coders and visual artists based in Athens and London who share a background in architecture but have expanded to develop creative projects using state-of-the-art digital tools. In their project Monolith, they will explore methods for facilitating communication and collaboration between humans and what they characterize as “digital beings.”

At HLRS, Uncharted Limbo will conduct experiments with dancers and movement artists. An algorithm they are developing will generate an audiovisual environment in HLRS’s CAVE visualization facility to prompt the dancers’ motions. At the same time, they will use sensors to capture data characterizing the dancers’ movements and interior states such as heart rate and neural oscillations. These data will then be analyzed using deep neural networks, and the results will be fed back into the algorithm to train it to recognize and respond to patterns in the dancers’ movements. During the development of this method, Uncharted Limbo also plans to work with S+T+ARTS AIR partner Sony Computer Science Laboratories, who conduct research on augmented creativity. The team expects that by gathering and analyzing many hours of data, the algorithm should learn to anticipate, react to, and encourage human movements and emotions in an increasingly confident and deliberate way.

At the conclusion of the project, the team will stage a performance involving equal cooperation between one or more dancers and the resulting digital being — humans will react to audiovisual impulses while the algorithm responds to their movements, creating a performative dialogue between humans and a machine. In addition to testing new interactive technologies, Uncharted Limbo will use the project as a way to engage the public in thinking about issues related to artificial agency, digital presence, data ownership, and the omnipresence of digital technology. The results will also be available to be exhibited elsewhere in the future.

“S+T+ARTS Air offers us a unique opportunity to develop a project we’ve been discussing for a while — an urge to address the important questions of agency and emergent sentience of digital entities,” commented Eleana Polychronaki on behalf of Uncharted Limbo. “As artists with significant experience in coding and art-directing real-time visuals for dance performances and interactive installations, we found ourselves wondering how we can break the norm of the typical ‘accompanying visuals’ and ‘one-way reactivity’ seen in most New Media performances. We hope that by collaborating with the HLRS team and taking advantage of their expertise in innovative visualization and access to supercomputing power we will be able to contribute to the immense field of research that explores human-machine interaction and the ever-evolving dynamics of that relationship.”

The inner lives of city trees

Based in Hamburg, photographer and multimedia artist Hiền Hoàng is interested in how humans perceive and interact with the world around them. In her project Garden of Entanglement, she will develop an immersive virtual environment that encourages participants to explore and gain a better understanding of how human activities affect trees in cities.

Hoàng has already begun working with scientists at the TU Harburg to begin studying the physiology of trees and their biomechanics, including performing 3D and CT scans of trees to gain a dataset that will be the basis for the project. Working with HLRS and the Association for Culture and Education PiNA in Slovenia, she will develop an immersive multimedia installation that will combine such scanning data with other technologies such as data sonification and visualization in virtual reality.

Once completed, Hoàng intends Garden of Entanglement to offer viewers an emotionally resonant, immersive experience of moving through the inside of a tree as it is buffeted by the urban environment surrounding it. She suggests that this should promote environmental consciousness by providing a more holistic understanding of the relationship between urban development and the natural world.

“In the past, conversations with scientists have helped me to understand the biology of trees, but working as a fellow at HLRS is something special because it offers the chance to collaborate directly with experts in visualization and high-performance computing in realizing a project together,” Hoàng said during her first visit to the center. “I am looking forward to seeing how this develops and how it might help to develop my future artistic practice.”

— Christopher Williams