Yandex has switched its Internet search engine to a new text analysis technology based on transforming neural networks. The company notes that this is the most significant event for this product over the past 10 years.
YATI technology (Yet Another Transformer With Improvements) allowed Yandex search engine to better assess the semantic relationship between user requests and the content of documents on the Internet.  According to the developers, the difficulty that arises on the path to training a transformer is the computational complexity of the task. The new models scale well in quality, but at the same time they are millions of times more complex than those that were used before. Therefore, it is necessary to launch a specialized computing cluster and distributed training on it.

Now the model is simultaneously trained on about 100 accelerators, which are physically located in different servers and communicate with each other via the network.

More information about the technology is available on the company's blog (in Russian)

Original (in Russian)