AI scraping hits heritage

Marcin Wilkowski / https://betterimagesofai.org / https://creativecommons.org/licenses/by/4.0/

Cultural heritage institutions are struggling with a surge of commercial AI scraping. Research from the ekip Engine is now feeding into the European policy debate through two responses submitted by the University of Edinburgh ekip team. Both responses focus on a fast-moving challenge for cultural heritage institutions: the impact of commercial AI scraping on galleries, libraries, archives and museums.

From emerging signals to policy action

It is a concrete example of how ekip turns research, stakeholder insight and foresight into policy recommendations that can travel. From interviews and workshops to evidence submitted to the European Commission. From emerging signals to policy action. “Our research shows that industrial-scale data extraction by for-profit AI companies is overwhelming cultural heritage institutions’ websites,” says Morgan Currie, Senior Lecturer at Edinburgh University “This is crashing servers, raising engineering costs, reducing visits to their websites and decontextualising valuable cultural material when it is presented through GenAI models.” The team argues that current regulatory frameworks do not yet balance the needs of the cultural heritage sector with those of the AI industry. That imbalance matters. Much of the infrastructure affected by scraping is publicly funded. Yet the value extracted from it often flows to private companies, without consent, attribution or remuneration.

Infrastructural issue

The two responses address connected policy questions. One focuses on the impact of AI scraping bots on the gallery, library, archives and museum (GLAM) sector. The other looks at the copyright environment for cultural heritage institutions in relation to scraping by AI companies. Both draw on original ekip research. This includes 49 interviews and three workshops with 66 stakeholders from cultural heritage, GLAM, open culture and academic sectors in the UK, the US and the EU.

“In dozens of research interviews with galleries, libraries, archives and museums, we have been hearing the same thing,” says Caitlin McDonald Research Associate at Edinburgh University “Digital archives on the open web have become vulnerable to a flood of bot traffic coming from AI companies.”

The findings point beyond copyright alone. They show that AI scraping is also an infrastructural issue. It affects servers, staff time, visitor access and the everyday capacity of cultural heritage institutions to keep collections available online.

Heritage institutions are responding in different ways, including by blocking certain IP addresses, restricting automated access or, in more extreme cases, temporarily taking services offline. But, as Caitlin McDonald notes, “the open web is being reshaped by their mitigation strategies”. Commercial bot-blocking services are increasingly becoming part of how institutions distinguish legitimate use from misuse.

Spotting problems early

For ekip, this is where the Engine demonstrates its strength. It brings together evidence, trends and stakeholder knowledge, making visible problems that may already be entrenched in practice but remain under-recognised in policy debate. It then translates these insights into recommendations that can inform European action. The University of Edinburgh submissions show how foresight-oriented research can create a bridge between the lived realities of the cultural and creative sectors and the policy processes shaping Europe’s digital future.

The reuse of this article is authorised under the Creative Commons Attribution 4.0 International (CC BY 4.0) licence (https://creativecommons.org/licenses/by/4.0/). This means that reuse is allowed, provided that appropriate credit is given and any changes are indicated.