It is now more than 25 years ago that the internet became an attractive and lucrative public good for commercial providers, and subsequently spread en masse. Just as the number of internet users increased by leaps and bounds during this period, so did the number of applications. There were hardly any limits to the imagination. Anything seemed possible at the time.
When the first digital learning and teaching platforms and the first teleconferencing systems came onto the market in the 1990s, this also generated interest among the members of the higher music education community. But the number of those who dealt with such systems as part of their professional activities, whether as developers or users, remained very small for a long time. That changed all of a sudden when the COVID pandemic broke out in 2020 and triggered a veritable boom in demand for such systems among musicians, music students and teachers at higher music education institutions (HMEIs).
Teleconferencing technology was no longer a brand-new thing at this point, and there was already extensive experience of using it in the field of music. Just to remind: The company Polycom, one of the pioneers in internet-based teleconferencing, came onto the market in 1994 with the first suitable product. Almost 10 years later, a small team of software developers from Estonia launched a simple and user-friendly software that made teleconferencing accessible for everyone: Skype. And as early as 2005, the Conservatorio di Musica Giuseppe Tartini Trieste (Italy) started a project in collaboration with GARR, the Italian Research and Academic Network, to develop a teleconferencing system designed specifically for remote musical interaction, which later became known under the name LoLa (Low Latency Audio Video Streaming System)
The LoLa System
Unlike Skype and later Zoom, LoLa was from the start not meant to be a home-to-home system, but only worked from institution to institution. On the one hand, because the commercial internet providers did not and still do not provide the necessary data throughput (at least not in the upload) that is needed for low-latency musical ensemble playing. On the other hand, LoLa requires special hardware and usually cannot be operated without technical assistance. In short, LoLa can only be used from a studio connected to a pan-European high speed university network such as GÉANT, but to which AEC member institutions usually have access.
When the pandemic started, around 50 AEC member institutions had their own LoLa studio, but in many cases, there were only very few digital nerds in these institutions who really used it. The desire to use this LoLa studio arose among many HMEI members only at the very moment when these facilities were no longer accessible due to the Corona lockdown. Many even found out only then that there was such an option provided by their home institution.
Music teleconferencing on the commercial internet
With the outbreak of the pandemic, there was suddenly a huge demand for teleconferencing systems available on the commercial internet. This drew attention to software providers such as Zoom, Skype and Microsoft Teams. However, all of these systems were optimised for verbal communication and extremely unsuitable for the transmission of music, in particular when it came about real-time musical interaction.
A lot has happened in the meantime. Commercial providers have diversified their products, as e.g. Zoom did by developing a high-fidelity mode specifically optimised for streaming music. In addition, projects emerged from the community of users with the aim of creating non-profit and technically accessible virtual workspaces for artists from the fields of music and theatre such as Digital Stage.
European projects to investigate and promote remote musical interaction
The AEC has been closely following the development of the above-mentioned systems and their application in HMEIs for a long time and has been involved in several pilot projects for the development and testing of these systems, including EU-funded projects, such as SWING and INTERMUSIC. In January 2020, the AEC, together with its members RSC Glasgow and NMH Oslo, organised a doctoral colloquium in which technicians, musicians, researchers, users and doctoral students from all over Europe met to discuss the state of the art and future perspectives of exchange music-teleconferencing systems. The meeting clearly showed that some developments were already quite mature, but the application of such systems was still a small niche in the HME system. But none of the participants could have imagined at the time that Europe would vanish in a complete lockdown just two months later.
A recent survey on the state and future perspectives
Three and a half years and one pandemic later, the AEC Artemis Digitisation working group decided to take the opportunity to ask participants of this 2020 Doctoral Colloquium on ‘Low latency streaming systems – learning and teaching perspectives from higher music education’, but also other people from the AEC community, who are dealing with the topic in one way or another, what has changed in the application of these systems since then and how they assess the future of these systems.
The question which systems best meet the requirements of the post-COVID era was answered almost unanimously as follows: When asked about performance quality, LoLa is still unrivalled at the top, with a latency of less than 5 milliseconds, an almost realistic multichannel audio system and a well synchronised and flexible video system (even though video is not considered the strongest feature of LoLa). However, in recent years new systems have emerged or been further developed that are no longer that far behind LoLa in terms of quality and might even have individual components that can easily compete with LoLa. Mentioned are the flexibility to use different hardware (MVTP) and the option of bringing a larger number of musicians together (JackTrip). However, the most important point that stands in the way of a wider spread of LoLa is the fact that LoLa can hardly be used without technical assistance.
What possibilities does the conventional internet offer?
While systems such as LoLa, MVTP, Jacktrip and Ultragrid either cannot be used on the conventional commercial internet or do not unfold their full potential there, music lessons based on commercial software were given on the commercial internet on a large scale during the pandemic. According to the experts surveyed, among the many commercial software that has been used during the pandemic, Zoom has proven perform above-average due to its comparatively good sound quality in the medium and low frequency ranges (provided external microphones and loudspeakers are used), its user-friendliness and the low costs. However, the high latency of Zoom does not allow musicians to play together at the same time. In other words: Zoom is ranked by many higher than any other available commercial software if it’s about home-to-home online music teaching, as long as the users abstain from expecting certain features to properly work, such as playing music together and the reproduction of a spatial sound which is close to reality.
The only alternative to Zoom mentioned several times in the questionnaires, is Jacktrip, which offers significantly better latency than Zoom (at least on distances less than 100 km) and is reported to provide good audio support if there is a good internet connection. However, JackTrip is considered to be challenging to use, so it seems more suited for communication among musicians with high affinity for the internet and digital means than for being used in lessons at a music school.
The Digital Stage project, which is still in the development phase, tries to reduce the latency in the transmission of audio signals on the commercial internet, among other things by reducing the number of stations that these signals pass through. Sonobus follows a different path, trying to reduce latency by focusing on the transmission of only the most elementary audio signals. All in all, however, it seems obvious that real-time interaction is still not possible on the conventional, publicly accessible commercial internet.
Simultaneous music making on the Internet
Even though the provision of high-speed internet access is steadily improving, most experts assume that simultaneous music-making in real time will not lead to satisfactory results on the commercial Internet in the foreseeable future. However, it is not first of all technical limitations that stand in the way of improvements, but economic considerations and security issues. On the one hand, it is not profitable for commercial providers to make high speed upload available which is a must for real-time musical interaction. On the other hand, publicly accessible networks will always be protected by firewalls or rely on the use of administrative procedures such as NAT (Network Address Translation), both of which inevitably contribute to slowing down data transmission.
Low-latency solutions, provided on non-public network connections such as GÉANT, still allow to do both performances and rehearsals with remote participants. In particular remote rehearsals are considered by many to be quite helpful and attractive. These allow, for example, a string quartet composed of musicians from different cities or countries to start rehearsing for a new concert programme over the network and to meet up in person only for the final rehearsal phase and the concerts. This can save a lot of time, travel cost and carbon emissions’, as one of the survey respondents states. However, it is also pointed out that such remote rehearsals work quite satisfactorily in the field of classical music, but not with music based on a common steady beat (as e.g. rock, jazz).
What will the future bring?
All respondents agree that teleconferencing will keep on gradually improving also for musical applications. However, there is also agreement that on the long run only tailor-made systems will be suitable to serve the needs of music and music education. Several respondents point out that it is not enough just to improve the transmission of data on the network. At least as important is the development of good microphonics and high-end systems, which then allow to reproduce the data received via the Internet in good sound quality. It is noted with regret that this aspect seems even less interesting commercially than the provision of fast network connections: “Unfortunately, the pandemic has not contributed to push the commercial services to provide better solutions” in this area.
Last but not least, some of the participants of the survey also pointed to the artistic potential of music-related teleconferencing and remote musical interaction. Instead of going into exhaustive detail on the topic, let’s conclude with two quotes from the survey:
– “Latency can also” be seen to be “part of the musical concept”, in order to generate “art that won’t suffer from the transmission or physical limitations but use them as a means to create something new conceived to exist in a displaced artistic environment.”
– “In the long term …. musicians, composers, engineers, producers, distributors etc. will develop their competences knowing the existence, affordances and limitations of network-based musical interaction … to take advantage of this ‘virtual dimension’, offering an additional layer of interaction/collaboration rather than replacing something that exists.”
