The idea behind the project emerged at the Innovation Day in 2019, our very own fair at the Ackerstraße in Düsseldorf. Among the numerous guests were also representatives from the Goethe Museum. Curator Damian Mallepree came to us and requested a digital version of Goethe’s only existing life mask.
To make this possible, we used photogrammetry and laser scanning and were able to digitally recreate the poet’s mask in detail, which was produced in 1807. The 3D model created this way was already exhibited at the museum in 2019. Via an AR app, it offered the visitors a particularly close impression due to its high resolution. While the original piece can only be looked at behind glass due to its fragile nature, the digital image offers a detailed view from all sides.
From this first project, a deeper collaboration emerged. The vision of a complete avatar of Goethe was born, and we were able to achieve this for the Düsseldorfer Nacht der Museen (Night of the Museums) in June 2022.
Avatar
We developed the avatar iteratively in cooperation with the Goethe Museum. The avatar’s appearance is inspired by the Goethe Schiller memorial in Weimar, while its clothing is based on reference images from the museum’s archive. Parts of the avatar were created with Meta Humans, a system of the Unreal Engine. This software enables users to configure realistic virtual images of human beings and to display them in the game engine. Afterwards, we used a wrapping procedure to adjust the 3D model to a bust of the poet modeled by Carl Gottlieb Weisser. This bust is based on Goethe’s life mask and is considered to be the depiction most similar to the poet’s actual appearance.
Finally, we used 3D sculpting to make the face look more like the idealized depiction of the famous Goethe Schiller memorial to improve the avatar’s recognizability. Its hair is based on a bust that also resembles the memorial.
Animations
The avatar of Goethe is primarily a model that does not move. However, it can be animated due to the skeletal structure it is built upon. We used two different approaches to generate the required animations for the avatar’s face and body and to transfer them to the model. For the facial animations, we used the iPhone app Live Link Face, which can recognize a user’s face and identify its features. The facial movements deform these features and an overlying 3D model. Next, these deformations are sent to the Unreal Engine via a wireless connection and mapped onto the Meta Human’s face. For the body animations, we used the motion capture system XSens. A suit equipped with sensors tracks the performer’s movements and transfers them wirelessly to the Unreal Engine.
Selfie station
Taking a selfie with Goethe – this will be possible at the Goethe Museum. For this purpose, the avatar created by Mirevi will be embedded into an installation and be able to interactively react to the visitors.
Utilizing a depth camera and the digital image of Goethe, the museum invites visitors to pose with the famous poet. Pictures taken this way can be printed out or received via mail, ready for use on social media.