Overlook

Let’s get right to it, XRs have been around for a while now. In the last few editions of this daily paper, we discussed with experts of digital image (Simone Arcagni, Bruno Di Marino, Lev Manovich) and I think it’s about time we let those in the know speak. We contacted two of the best creative professionals, both very European, who love to explore what images can do when unrestricted by the frame and, possibly, by the very rules of cinema.
Max Sacker and Ioulia Isserlis are creatives associated with Berlin-based Another World VR. They know Venice very well. In 2018, their film Kobold passed at the Venice Film Festival, followed by Pagan Peak VR, inspired by the Sky Original series of the same name. Max Sacker is also a tutor at the Biennale College, which means he enjoys an all-around vision of the field.

Max and Ioulia, are XRs cinema, or are they something else?
Max Sacker_The sense of presence in a spatial environment, the freedom of choice and movement put us in a different place viz. cinema. The story is literally walked through—lived, not merely observed. This requires the involvement of further areas of our brain. It is up to us creatives to understand how strong, even overwhelming, this sense of immersion is. We must make sure we are able to bring some humanity when we use this technology. We must give it some soul.
Ioulia Isserlis_XR narration is more physical, more engaging. As an author, what I love about it is the blending of real and virtual elements, and especially the ability to integrate some ‘magical realism’ in our real-life experiences.

When producing XR, what expressive techniques work best?
II_The designs of levels, sound, and light are essential, given that developers create a whole new world the audience will step in. The level of immersion is up to directors, according to their sensibilities and to their approach to narration.
MS_I love to begin by hooking the audience in using photorealism. It must look like actual reality. This means that I often use photogrammetry to capture different real-world environments, objects, or characters. I also pick soft lighting and I am careful with saturation. First impressions are paramount to decide whether to go on with the experience or not.

How can you guide the attention of the audience through the virtual environment without compromising freedom of exploration?
II_Interactions and game mechanics must respect intuition, because the perception of players is initially overwhelmed by the amount of freedom they have.
MS_There’s no magic trick, but I do believe there are a few constants: to show, not tell; to invite, not force; to be ready for deviation; to account for different events; to compensate exploration. We want something unexpected to happen, something interesting, fun, scary, engaging, even philosophical. We want to remind audiences that it’s probably a good idea to deviate from the established path thanks to countless creative options like interface, dialogue, inner monologue, cause and effect…

How does time control influence the audience’s immersion?
MS_Like in traditional cinema, in literature, or in music, XRs do offer the ability to compress or expand time using precise creative techniques. Only, in XR we can also manipulate time to influence interaction and even physical motion. Think about bullet time: one thing is watching an actor on screen, contorting their body in slow motion to dodge a bullet—doing it yourself is a different thing altogether! Time and the laws oh physics are precious creative material for us.

Let’s talk editing.
MS_In XRs, we have to be extremely careful with it. Every time we make cuts or switch ‘framing’, we are pulling spectators away from their perspectives with a jerk, and shoving them into a new one. Editing must be modulated with classical techniques like fade-ins, which guide the user’s attention in a soft, almost subliminal way. As far as sound goes, spatialization is of the utmost importance.
Cinema has a long tradition behind it. XR – we are now building a new language that partly continues cinema’s legacy, though also looks for new solutions for challenges we never know existed.