Research into AI model collapse has been around for at least a year. Still, this article in Nature, although slightly daunting to me as a humanist, has now cemented the phenomenon. AI that is trained on texts from AI steadily worsens. It’s a little like having a friend who gradually gets soused as he consumes sequential bottles of beer in the course of an evening.
The difficulty for the AI-mongers is that they have uncritically scraped every English word possible from every Internet source that will let it (and often from sources that don’t want to let it). AI-related companies have persuaded the world that AI can compose and reason in the place of humans, and that no one need know. At the same time, they are desperately trying to figure out how to distinguish AI writing from human writing while constantly scraping more data. They are Nestbeschmutzers, foulers of their own nest.
But this needs an example, and Drs. Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, and Yarin Gal have provided us with a fine one indeed, worthy of Borges:
Example 1. Example of text outputs of an OPT-125m model affected by model collapse—models degrade over generations, for which each new generation is trained on data produced by the previous generation.
Input: some started before 1360 — was typically accomplished by a master mason and a small team of itinerant masons, supplemented by local parish labourers, according to Poyntz Wright. But other authors reject this model, suggesting instead that leading architects designed the parish church towers based on early examples of Perpendicular.
Outputs:
Gen 0: Revival architecture such as St. John’s Cathedral in London. The earliest surviving example of Perpendicular Revival architecture is found in the 18th @-@ century Church of Our Lady of Guernsey, which dates from the late 19th century. There are two types of perpendicular churches:.
Gen 1: architecture such as St. Peter’s Basilica in Rome or St. Peter’s Basilica in Buenos Aires. There is no evidence that any of these buildings were built during the reign of Pope Innocent III, but it is possible that they may have been built during the reign of his successor, Pope Innocent.
Gen 5: ism, which had been translated into more than 100 languages including English, French, German, Italian, Spanish, Portuguese, Dutch, Swedish, Norwegian, Polish, Hungarian, Slovak, Lithuanian, Estonian, Finnish, Romanian, Bulgarian, Turkish, Croatian, Serbian, Ukrainian, Russian, Kazakh, Kyrgyz.
Gen 9: architecture. In addition to being home to some of the world’s largest populations of black @-@ tailed jackrabbits, white @-@ tailed jackrabbits, blue @-@ tailed jackrabbits, red @-@ tailed jackrabbits, yellow @-.
Jackrabbits of all colors, we salute you!