NYE VIRTUAL CONCERT WITH JEAN-MICHEL JARRE
As one may expect, the circumstances for this year’s NYE celebration was slightly different than a full-blown warehouse party, laced with smoke, laser beams, and a couple of hundred people out to see the sunrise.
This year, I set out to discover a digital reincarnation of the Notre-Dame Cathedral in VRChat, the venue for Welcome to the Other Side, a virtual concert by electronic music legend Jean-Michel Jarre.
Arriving at the Venue
Jumping into VRChat with my Quest 2, I searched under worlds for Welcome to the Other Side. It was top listed, so I jumped straight into the venue. When I arrived I was required to select the levels of animation desired based on my VR headset and PC’s capabilities. With one trigger pull, I was in.
I arrived at the back of the Notre-Dame Cathedral between giant gothic columns that guided my eyes to the impossibly high ceiling above. I inched forward as the audio kicked in, running into a crowd of enthused NPCs (non-player characters) with automated cheers and audience roars typical of sitcom effects.
Quickly, I located Jean-Michel Jarre, a glowing red avatar with his hands poised above a range of synthesisers, symbolically placed in lieu of an altar. It appeared that the show was only just beginning. Jarre addressed the audience in French and English, and I waved my arms around to support the atmosphere as a few attendees arrived in the same instance as myself.
We stood near the front of the stage, teleporting back and forth to explore the limitations of the venue as Jarre began to play. Incredible lighting synced with the crescendo of notes that enveloped the Cathedral, where in real life, such a concert would have been impossible with its acoustics.
Above, the stained glass windows glowed with an unearthly luminescence. It seemed only natural to shine laser beams and blasts of colorful pixels. Once I’d figured out the front of the venue, I dodged between the columns towards the back, looking for hidden nuggets, finding a menu that would beam me to the upper floor.
Up I went, gaining a new perspective on the centre aisle that focused all attention to the avatar on stage. The NPCs were wild with cheers and flailing limbs, and a couple audience members hung around. The atmosphere was beautifully constructed, but after enjoying the light and music, taking lots of photos and sussing out the audience members for their reactions, I jumped out, wishing I’d been able to interact more with the experience, but still impressed with the team’s progress since their first virtual concert Alone Together.
Above: my screenshots showing how many people were attending the concert during the event, and after when the After Party world opened.
The After-Party
Opening up the menu to leave, I was delighted to find an after-party for the concert listed on my world menu. I clicked on in, this time, arriving at a square in front of the Notre-Dame. Assuming the after-party was inside, I scurried my avatar towards the lights, and the concert began to replay. This is when I realised I had only experienced a recording of the performance. You will find out the technical reasons for this shortly.
Undeterred, I ventured outside. I was surprised by the design option to enclose the square with buildings, as in Paris, the Cathedral sits on Île de la Cité, enclosed by the River Seine. However, it was charming, and there were a couple avatars in a little group having a lovely conversation about their headsets, motion tracking, the concert and virtual reality.
Next, I set off to check out big lights illuminating out of manholes that lined the square with labels. I maneuvered my avatar to drop through the drainpipe that revealed another replay of the Jarre concert. Dropping down each drainpipe, I discovered more and more angles or segments of the Jarre concert replay.
A little frustrated from hearing and seeing the same thing multiple times, I went to the next light source, a giant spotlight searching the square, beaming down from a Hindenburg-style airship. As the light captured my avatar, I was beamed up to the ship, where the Jarre concert started to play again. Inside there was a replication of the synths that Jarre had used during the concert, and I was amused to watch a hot dog with a young French voice pointing his breadstick fingers at the keys in an attempt to play, but the instruments were there just for show.
After a couple more back-and forths between the recorded concert and the after-party to see if more people would arrive, I jumped out for a final time.
Speaking with the Designers and Developers
As a part of my mission to better understand how virtual concerts are developed and influenced by technology and the platforms they are created in, I spoke with Antony Vitillo, lead developer on the Jarre concerts taking place in VRChat, and Louis Cacciuttolo, founder and CEO of VRrOOm, responsible for the design of the concert.
With Vitillo, I get a better sense of the challenges of creating a virtual concert.
“The real problem isn’t the quality of the experience, it’s the number of people,” he says, highlighting the limitations of how many people can attend a virtual concert simultaneously.
Essentially, unlimited people can attend virtual events, however, only so many can attend in one instance before putting too much strain on the system (crashing the party with data overload). This is when sharding occurs.
Caption: Sharding is similar to the concept of multiple universes as per the quantum physics many-worlds theory, however in this case, in each new world created the same event occurs. Image in public domain.
For readers new to the ins-and-outs of virtual worlds, you can think of sharding as duplicates of the event that branch off every time the capacity of one room is maxed out. The real-world equivalent would be when a club of 400 people fills up, instead of person 401-800 having to wait in line for the club to empty or people to leave, another identical club magically appears where the same event is taking place.
In the case of VRChat, the platform used for the Jean-Michel Jarre concerts, the maximum per-instance (room) capacity is 40 individuals. This means that if you are the 41st person to arrive at the concert, it will be a new instance, a copy like the one I entered into.
After 39 more people arrive, the 81st person will be bumped into a new instance, and the replication continues.
This brings us to the challenge of automation, and how to ensure that every audience member in each instance has the same quality of experience when a virtual concert is taking place.
For those in the first instance of the virtual concert, the avatar stream is live on stage. As you can see, the avatar is different from the ones in the pre-recorded versions of the concert I experienced. Image Credit: Antony Vitillo.
Stitching Realities
“We can make a great experience for 40 people, but more is difficult,” says Vitillo.
“This is the big limitation, the number of people and synchronizing it. In a real concert, there are thousands, tens of thousands of people. That’s what makes a big concert amazing.”
“We didn’t know how to bring Jean-Michel Jarre into the public rooms, we liked the idea of taking the video streaming and creating a hologram from the video, and making the background disappear. We also tried to add a depth effect,” says Vitillo, speaking of his first concert with Jarre, Alone Together.
By public rooms, Vitillo is referring to all instances of the virtual concert created after the first instance reaches capacity.
Above: Jean-Michel Jarre getting suited up for motion capture for the concert on Dec 22nd. Image credit: @louisadrienleblay
“Everything is a big compromise,” he says. “There’s a lot of data to work with, you can’t put in everything you want otherwise it crashes.”
“I got many critics of the last concert, but a lot of people thanked me for the blog piece I wrote. If I made some errors, and other people can avoid them… In part it was very successful, in part, it was a failure,” he says.
Vitillo is brave to speak about this. As well as leading the development of this concert and many other prominent events such as Venice VR Expanded, Vitillo is a well-known figure in the virtual community for his blog The Ghost Howls, renowned for his honesty and fearlessness in talking about the positive and negative aspects of experiences and technology.
“This technology is at an early stage, it’s not possible to do everything you want to do, but in the end, it’s already possible to do amazing things. This is important for people to know,” says Vitillo.
Jarre’s first virtual concert in VRChat “Alone Together” Image credit: Vincent Masson
Choosing a Platform
Choosing a platform is an incredibly important decision because it will determine the development and aesthetic parameters of a virtual concert.
VRrOOm CEO and founder Cacciuttolo says he got a lot of push back for choosing VRChat as a platform for the VR concerts, but when I ask him about his reasons for this choice, he is quick to respond:
“I deeply believe that it’s the most interesting social VR platform in the way that it engages with the users,” he says. “VRChat also allows UGC [user-generated content], with access to its backend and creative tools.”
When I ask Cacciuttolo if he would like to build his own platform, he says he’s not so interested because the technology isn’t there yet.
“I want to go to the bottom of it, and learn more and more,” he says. “I want to refine what we are already doing, and reach a level of excellence so that our partners can be safe, and so that we can deliver it in the best way.”
When Will Artists Have the Tools They Need for Virtual Concerts?
When it comes to the millions of artists around the world, entire generations of musicians who don’t have a stage to perform, the budgets to produce, or the tools to reach their fans in the emerging virtual concert industry, Cacciuttolo isn’t so sure about when they will have the tools they need to enter the virtual concert industry, but for the Jarre concert on NYE, he is working on monetization models for this to be a plausible endeavor for artists.
“Anyone who wants to go back in [the venue] will be able to relive the show by pressing a button with the JMJ avatar performing. Right now VRChat doesn’t have a monetization model like Sansar, or other platforms. They say they will implement a ticketing system next year. We want to show that since we have this replayability, people will be willing to pay for it.”
Cacciuttolo makes an interesting point that even though there are platforms where virtual concerts can take place, monetization has not been the focus for creators using these platforms. It means that creating compelling virtual concerts for a global audience today must be done primarily with public or major label funding to cover the costs where in-world monetization isn’t yet available. This presents an impasse to up-and-coming artists looking to reach their audience virtually.
Wishlist
In order to overcome the hurdles of developing virtual concerts, there are a few things that Vitillo speaks of when I ask him about what he’d put on his wishlist for developing future concerts.
“We have to find a way with 5G, we need optimization of data so that we can see what we want to see, and hear what we want to hear. We need time,” he says.
Vitillo is also frustrated by the limitations of the Quest that sacrifices high-quality VR for consumer viability in terms of price and ease of use as a tetherless headset (not attached to a PC).
“It’s really a pain in the ass because there are so many limitations like 50MB for the world, frame rate, etc. Whatever you do is too much for the device. When you finally manage to publish the world, you see that there are so many limitations,” he says.
He also lists more development control on the platform as something that would greatly improve the design of the concerts. Although VRChat is one of the most accommodating platforms for developers, it still does not offer the full capabilities of Unity, the game engine that the platform is developed with.
“Doing something from scratch is really difficult and time-consuming. This is why we use VRChat, because they have already done the heavy lifting, but if I had the full power, I could do much more, especially with the interactions of the user. I want this to be able to go beyond that,” he says.
Pioneers
“We achieved something, we made a point, that these types of things can happen in VR, and it was also transmitted to other channels, giving VR a lot of visibility,” says Vitillo.
While there are many challenges to overcome, such as creating a live experience for everyone who attends, this will take time as the technology itself must evolve. There were some frustrating design points such as endless replays of the same concert that deterred rather than rewarded my sense of exploration, but still this initial step towards digital rewards was good to see.
In the future I hope to see additional content, unlockables, and much more digital art from a variety of artists to decorate the venue or after-party area as there are so many up-and-coming artists who would benefit from and also intrigue an audience at a cultural event such as this.
Regardless, I have singled out Alone Together and Welcome to the Other Side as two of the most significant virtual concerts of 2020 because they are a truly groundbreaking exploration of the potential of VR to create a new type of concert experience.
One day, all artists will have the tools to create and perform in ways previously unimaginable, unlimited by time, space, budget and physics.
If you want to learn more about what a virtual concert is, and Jarre’s hopes for the future, you can check out my interview with him here.
Article originally posted with VRScout.
