At the beginning of the year, Blooloop.com, a professional resource providing information about museums, amusement parks and other attractions, named six technology trends in leisure activities for 2020. This list includes virtual and augmented reality (VR/AR), artificial intelligence (AI), interactivity, as well as robots and e-sports. The latter replaced the bots and voice control that were among the trends for 2019. Although the forecast for this year had been made before the world faced the coronavirus pandemic, it describes a further shift in the development of cultural entertainment and tourism to online.

ICT.Moscow asked the developers of digital solutions for cultural institutions and representatives of museums about the specific features of their interaction and asked how they evaluate digital platforms in this area. The experts spoke about the changing demand for digital solutions in tourism and cultural leisure activities during the pandemic. Moreover, the material contains practices and examples of the application of emerging technologies in this area in Moscow and abroad.

Culture clients

Vladimir Opredelenov, Deputy Director for IT at the Pushkin State Museum of Fine Arts and Head of the Department of Information Technologies in Culture at the Higher School of Economics, highlighted an important issue in terms of interaction of the museum as a customer with IT developers - understanding what each party needs. The situation is complicated due to the fact that cultural institutions rarely have clearly defined business processes.

The developer usually does not understand our organizational structure and how the internal business process works. In addition, business processes are defined in the museum not quite clearly. The most difficult thing is to first pave the way in the museum so that any technological product can be introduced.

Vladimir Opredelenov

Deputy Director for IT at the Pushkin State Museum of Fine Arts

Opredelenov emphasizes that after the terms of reference are created, it is necessary to prepare a large amount of time for its debugging, not to skip the stages of testing, commissioning and training of employees.

The experience of the “Future History” studio - the creator of the immersive theater application “Mobile Art Theater” - can be an example of successful interaction and complete understanding of aims and objectives. Maria Shashayeva, COO of “Future History”, claims that usually the company completely comes up with what the project should look like and how it should work, then it hires programmers or a team to implement these ideas, and then the company employees become engaged in filling the project with content and its further development.

Moscow museums often initiate new digital projects on their own, especially when it comes to solutions that are close to the user, says Sergey Shakryl, curator of IT projects in the field of culture and business of the Moscow Department of Information Technologies. The agency initiates solutions affecting the entire industry. The DIT of Moscow notes that there is a large number of requests from cultural institutions for revision and improvement of the current functionality; it is attributed to the active implementation of IT solutions.

Maxim Panin, head of the project office of digital transformation of the Ministry of Culture of Russia, claims that cultural institutions make different requests. According to him, when considering such applications, the department identifies “common problems and tasks which the business may not be so interested in, but at the same time there is a need to automate and digitalize a number of processes”.

Egor Yakovlev, co-founder and development director of izi.TRAVEL, a street and museum audio guide platform, says that when it comes to upgrading this application, he most often hears about VR/AR and iBeacon. However, these requests are not so common and much more often “stem from the fantasies of museums, not from the real requests of visitors”. The expert believes that it is more important for museums to deal with the quality of content, for example, focus on making storytelling more interesting.

Platforms in culture

In the field of culture, as in many other areas, digital platforms are popular, which allows partners - in this case cultural institutions - to use ready-made solutions, providing their customers with convenient services. Such examples include Google Arts and Culture or Artefact.

Maxim Panin, head of the project office of digital transformation of the Ministry of Culture of Russia, which is the project customer of Artefact, recalls that the department’s strategy includes the creation of platform solutions. He calls Artefact the UGC platform, where User is a museum that can create a multimedia guide for free with the help of a WYSIWYG editor, using only a computer and a camera.

Maxim Yakovlev from Datastack (developer of the Artefact application) believes that, starting from a certain level of the task, a digital platform can be its solution. He claims that “it depends on the scale and tasks, everything always starts with individual developments and only then becomes a platform solution”.

If we talk about mass products, then the future belongs to platform solutions. When cultural institutions use ready-made platforms, they do not have to worry about expensive development, technical support of the system and project development.

Maxim Yakovlev

CEO of Datastack

The platforms allow to combine various functionalities, including universal solutions that are not directly related to culture. For example, street and museum audio guides platform izi.TRAVEL provides its museum content via API to third-party applications with other technologies: Yandex.Alice and the Atrefact application using image recognition and AR technologies. Egor Yakovlev from izi.TRAVEL considers integration with voice assistants one of the most important strategic tasks of his service.

We are convinced that in the future people will listen to audio guides not using separate audio guide applications, but in the applications they use every day: maps, navigators and, of course, voice assistants.

Egor Yakovlev

Сo-founder and development director of izi.TRAVEL

Sergey Shakryl from the DIT of Moscow agrees that platform solutions are suitable for meeting some mass requests. However, he is convinced that such solutions will not be able to replace individual developments, since in the field of culture there will always be unique projects, which require their own unique solutions.

Alexey Steblev from RDI.Digital agrees with Sergey Shakryl. He believes that platforms cannot replace individual IT developments in the field of culture. He claims that “every cultural object is unique, and its creators or keepers are not always ready to fit it into a common, even the best, interface”.

The influence of COVID-19 on development

The  pandemic has also changed the relationship between IT companies and cultural institutions. Alexey Steblev, CEO of RDI.Digital Russia, notes that demand is growing, but not booming.

The pandemic brought us several projects, including the creation of a marketplace for the Vienna contemporary art fair Viennacontemporary and an educational project for the Pushkin Museum of Fine Arts.

The pandemic brought us several projects, including the creation of a marketplace for the Vienna contemporary art fair and an educational project for the Pushkin Museum of Fine Arts.

Alexey Steblev

CEO RDI.Digital Russia

Sergey Shakryl from the DIT of Moscow says that the introduction of technologies in certain areas was put on hold because of the pandemic, but at the same time the pandemic contributed to the increase in the number users of existing online cultural sites. He notes that the number of virtual tours and exhibitions has increased significantly.

The Pushkin State Museum of Fine Arts is trying to bring all the products that the museum creates to online. The museum has difficulties with funding, since the off-budget income has drastically decreased, Vladimir Opredelenov says. This made the museum switch to more rational use of existing resources and think over new business processes. For example, Opredelenov says that during the pandemic, it has become clear that the system of 3D modeling and design of future expositions of the Smart Museum can be used for full-fledged virtual exhibitions.

Impact of COVID-19 on consumption

Along with the introduced restrictions, the demand for online projects has  also increased, including in the field of cultural entertainment. Egor Yakovlev from izi.TRAVEL says that during the times of severe restrictions people started to listen to virtual tours of museums more actively. He attributes this to the fact that museums quickly began to redirect offline visitors to online. He adds that when the restrictions were lifted, the demand for local travel and audio guides to all places that could be reached by car on weekends increased sharply. He also said that in August 2020 izi.TRAVEL had reached an absolute record for the number of stories people listened to - more than 2 million.

The head of the company that developed Artefact platform claims there was a three-fold increase in the attendance of digital resources of the platform in the first months of quarantine. However, after a couple of months, the indicators began to decline, but still became higher than before the start of the pandemic.

Sergey Shakryl, curator of IT projects in the field of culture and business of the DIT of Moscow, also notes there was an increase in demand for museum online projects, although there was no sharp growth, and the user’s behavioral model remained the same.

There is a lot of content in digital form on the Internet, and only if the company has an unusually unique and monopoly product, its use can increase significantly. In the field of culture, there are many world museums, and there are resources that bring all this together.

Sergey Shakryl

curator of IT projects in the field of culture and business of the DIT of Moscow

The fact that the growth in demand during the coronavirus pandemic is associated with the uniqueness of the content is confirmed by the data of the Pushkin State Museum of Fine Arts (46th most visited in the world in 2019): in the first month of the pandemic, in the interest in VR expositions and excursions increased by 1500%.

Users actively watch virtual museum tours online, but are not yet ready to pay for it, Vladimir Opredelenov stated. The museum notes that with the introduction of fees for VR tours after the quarantine was canceled, their popularity has declined.

There is a lot of content now, even too much content to some extent. It is more convenient for people to choose what is free. 

Vladimir Opredelenov

Deputy Director for IT at the Pushkin State Museum of Fine Arts

Individual consumption technologies

The interviewed experts agree that IT technologies make cultural entertainment and tourism head towards individualization.

According to Alexey Steblev from RDI.Digital, the introduction of technologies will dramatically expand the audience of the cultural sphere, and in tourism it will provide great opportunities for individualizing user experience, leading to an the expansion of both audience and the geographic scope.

Maxim Panin, head of the project office of digital transformation of the Ministry of Culture of Russia also claims that the further development of individual tourism is a likely scenario.

Now we buy plane tickets and book hotels. We want to get personalized services - in museums, libraries, etc. We also need to develop new opportunities for interaction with art objects, we need new experience and new formats of interaction with cultural values. The introduction of technology will make museums more open for individual visits.

Maxim Panin

Head of the project office of digital transformation of the Ministry of Culture of Russia

Museums you can visit virtually

Access to museum collections in digital format was already available even before the spread of the Internet, when such image collections were stored on CD-ROMs. The Museum of Computer Art (MOCA), founded in 1993, can be considered the first online project that is not a digital version of a traditional cultural institution. Today, according to the results of the VCIOM (Russia Public Opinion Research Center) poll, among Russians with Internet access (83% of the population), one in five people visited a virtual museum (19%). Among them, 57% took a virtual tour during the self-isolation regime.

Practically all residents of Moscow (94%) are aware of the possibilities of virtual museums. In the capital a single resource called Museums of Moscow online was launched in 2017, it has more than 50 thousand photographs of exhibit items. During the period of self-isolation due to the spread of COVID-19, the project Culture of Moscow online was also opened, it contains video versions of performances, concerts, lectures, etc.

Virtual reality

Virtual exhibitions and interactive tours are of much greater interest in terms of technology applications. The most famous global project in this field is Google Arts and Culture - a platform with panoramic views of halls and images of exhibits in high resolution. There you can find more than 20 collections from Moscow, including, for example, the Museum of Cosmonautics or the Pushkin State Museum of Fine Arts (Pushkin Museum also has its own large VR project). In total, the Google project includes over 2500 museums and galleries from different countries. From 2016 onwards, the application supports VR format and is compatible with virtual reality headsets.

Yandex has a similar service based on the Yandex.Maps functionality - about 15 Moscow museums are available in it for the moment.

In Moscow in 2016 the Pushkin Museum launched its own virtual tours. The experience of the Tretyakov Gallery in using immersive technology is also interesting: in 2018, an interactive exhibition in virtual reality “In Three Dimensions: Goncharova and Malevich” was held there, it was organized jointly with the VRTech Group of Companies.

Some museums in their VR projects do not repeat the existing exposition, but offer original content and experiences. An example is the panoramic VR film “Journey to the Past”, presented in 2017, which became part of the permanent exhibition of the Jewish Museum and Tolerance Center. And in the “Smart City” pavilion at VDNH, you can take a virtual trip in an unmanned vehicle.
The Consumer Technology Association predicts that VR and AR in 2020 will return to double-digit growth for the first time in four years. The opinions of Russian experts on the prospects of this field can be found in the review material of ICT.Moscow.

This year, due to the pandemic, the organizers of the Venice Film Festival have also used VR technology. Moscow was one of 15 cities where special platforms were opened (in the Russian capital it is the Flacon design factory) for viewing immersive works selected by the curators of the Biennale.

Augmented reality

Another immersive technology popular in the area of cultural recreation and tourism is augmented reality, for which it is enough to have a smartphone. Many museums in the capital, including the Tretyakov Gallery, the Pushkin State Museum of Fine Arts, the Museum-panorama the Borodino Battle, the Darwin and Historical Museums, use the Artefact application for this. Museums are free to join this interactive exhibition guide. The solution was developed by the Moscow company Datastack at the request of the Ministry of Culture of Russia.

An example of a separate AR-development is the mobile application of the Museum of Cosmonautics - a virtual guide that introduces the main exhibit items in game form.

In addition to navigating inside exhibition spaces with virtual annotations to exhibits, AR technology is also used to guide tourists between attractions in city streets. In the capital, the application “Discover Moscow” has been developed for these purposes. In addition to geographical points with descriptions of monuments, the application contains a 3D-model of the Ostankino TV tower: being near this tourist attraction, users can get acquainted with its history and internal structure in AR mode.

Here is another scenario of using AR for tourists: the possibility to take a picture with a virtual character (similar to catching Pokémon in a mobile game which was popular a few years ago). Such functionality in Moscow is now available in the Histars application, but it is planned to be transferred to the “Discover Moscow” app as well.

Artificial intelligence

In May 2019, the Moscow Samsung Artificial Intelligence Center presented an algorithm that helps to create “living” portraits. A moving Mona Lisa was shown as an example. You can read more about the prototype created by Russian developers here.

At the end of 2019, the Metropolitan Museum of Art, Massachusetts Institute of Technology and Microsoft company held a joint hackathon to find new ways of using artificial intelligence to present the collection of the New York Museum to the visitors. As a result, several prototypes were developed. Among them there was the Artwork of the Day service, which selects a piece of art, one way or another corresponds the current information agenda in the world, taking into account the preferences and characteristics of each user. Another interesting solution is Storyteller. It selects suitable images from the museum’s collections that can serve as illustrations for what the user is talking about. For this purpose, speech recognition algorithms in 60 languages are used.

The Salvador Dali Museum in Saint Petersburg (Florida, USA) in 2019 used artificial intelligence to create the image of a revived artist. The algorithm was trained on the basis of archival records, interviews, quotes, to recreate the appearance and behavior of Dali. His virtual image helps visitors to get acquainted with the collection of the museum.

Artificial intelligence was also used in the Tretyakov Gallery’s VR-exhibition “In Three Dimensions: Goncharova and Malevich”, which has already been mentioned above. Visitors could compose their own still-life painting from virtually proposed objects. AI helped them, processing the compositions created by visitors in real time. The neural network was trained using a dataset of art objects images presented in the museum.

Another example of using AI in a museum is Viktor Vasnetsov’s “revived” paintings as part of a multimedia exhibition in the same Tretyakov Gallery. To do this, two machine learning algorithms trained on the artist’s paintings were used.

Computer vision algorithms were used in another cultural project. With the help of the face recognition technology Vision which was developed by Mail.ru Group, archival photographs of the Great Patriotic War were processed, and users could find images of their relatives.

Artificial intelligence was also an independent creator of visual art works. For example, last spring, Yandex opened the Gallery of neural network art. The neural network was trained on works of different painting styles. In total, there were about 40 thousand paintings in the training sample. On their basis, AI generated 4 thousand new images collected in an online gallery.

Chatbots and voice assistants

Traditional guides and rental devices for listening to recorded excursions are gradually being replaced by smartphones through the use of chatbots and voice assistants.

In 2019, the Hermitage launched “Liza” chatbot. Visitors of the museum can communicate with it via messages in social network “Vkontakte”.

Alice, a voice assistant from Yandex, can act as a guide to the Pushkin Museum exposition. It is able to recognize a picture from a photograph and tell about it.

Chatbots are also used to help tourists in urban space. For example, for the 2018 FIFA World Cup in the Russian capital, “Boris” chatbot was created.

Blockchain

Distributed ledger technologies are used not only in corporations: in 2019, a charity project of the Tretyakov Gallery with a blockchain was announced, it was supposed to be implemented by the Russian company RDI.Digital Russia and the Austrian Riddle & Code. However, later it was decided not to use blockchain, and  Riddle & Code left the project.

The Pushkin State Museum of Fine Arts also had plans to implement blockchain. In 2018, the director of the museum spoke about the possible use of this technology in the processes of buying and collecting art objects.

Previously, the blockchain has already been used by Christie’s auction house in New York during the sale of art projects.

5G communications

Analysts from the Consumer Technology Association predict that fifth-generation communications will provide significant impetus to the use of VR and AR, which will also affect the areas of tourism and cultural recreation. Among the many demonstrations of 5G capabilities there are a plenty of such examples.

DEUS, a Moscow manufacturer of virtual reality headsets, agrees that new communication opportunities will also affect the VR field.

New approaches in cultural entertainment

In a very traditional area of cultural recreation, new ways of contacting visitors with exhibits are appearing, and technological solutions themselves become units of cultural value. Tourism which has always been associated with movement in space turns into a remote multimedia experience.

For example, social networks are used in the field of tourism and recreation not only as platforms for chatbots or information and reference resources, but also as part of independent cultural and technological projects. At the end of 2018, the Russian Museum, together with the “Odnoklassniki” social network, launched “the Masterpieces of the Russian museum” virtual project. It is a separate site with audio guides of ten works from the museum collection, voiced by various Russian celebrities. The site also offers a virtual tour of the museum in interactive panoramic mode.

Screenshot rusmuseum.ok.ru

The coronavirus pandemic and mobility restrictions have forced people to use virtual tourism opportunities. Among other online excursions, the Faroe Islands initiative should be highlighted. The local Ministry of Trade and Industry has launched an online tour, the specific feature of which is that the user sees a panorama from a camera carried by a guide. At the same time, the virtual tourist can give the guide instructions in real time where to turn or where to go.

You can see Moscow and its architectural monuments without leaving your home using street panoramas on Yandex.Maps or Google.Maps. The self-isolation mode allowed Yandex to make panoramas of empty streets of the Russian capital with the opportunity to compare them with filming of previous years.

An interesting technological project is the Moscow gigarama - panoramas of the city from a bird’s eye view and “infinite” zoom, created with gigapixel photography technology.   

New technologies for old formats

VOMA (Virtual Online Museum of Art) has been opened in September. The VR space is available free of charge from anywhere in the world, it displays digital copies of classical art objects from various collections (for example, the Musée d'Orsay and the New York Metropolitan Museum of Art), as well as works by contemporary authors. The project uses modern computer graphics for video games and distributed computing.

The coronavirus pandemic seems to have encouraged a return to a fully virtual environment that simulates an exhibition space. It is curious that such an immersive experience (on a more primitive technical level) was offered by Apple almost 30 years ago by releasing a CD with a three-dimensional model of a non-existent science museum.

Image amazon.com