What happens if we continue automating the systems that make up our societies, taking humans out of the loop? That was one of the main themes of this year’s Ars Electronica.

Not surprisingly, the parts of the festival I attended focused on one such loop: our democratic system itself. This is a system based on, among other things, pluralism, open debates, principles of accountability, and opportunities for contestation. All of it built on the underlying principles of human rights, and an implied pursuit of human dignity for all.

What happens, then, when all of those principles dissolve into the code of agents, automated decision-making systems, perhaps even Artificial General Intelligence, whatever that may be?

Mechanical Kurds: precarious refugees as humans in the loop, made complicit in their own surveillance

At the Contested Futures Conference and the Tech-Bro-Topia symposium, artist Hito Steyerl took an interesting stab at those questions through her Mechanical Kurds installation, honored with the S+T+ARTS Grand Prize for Artistic Exploration at the festival. In it, she shows how Syrian refugees stuck in a refugee camp in northern Iraq are employed as data workers, labeling images, so autonomous vehicles and drones can learn to recognize pedestrians, tuk-tuks, and other potential traffic hazards - or targets.

So here, the humans who do still play a role in this automated system are exploited data workers living under precarious conditions in refugee camps. Worse still: they're made complicit in their own surveillance, and possibly their own deaths. Above the camp, drones trained on datasets like the ones its inhabitants have been labeling monitor the conditions below. In a dramatic turn, the installation shows footage of a car erroneously attacked by a Turkish drone, killing all but one of its passengers.

The futurist-inspired aesthetics and ethics of defense tech

Such dissolution of accountability, debate, contestation, and ethics is not an accident. Artist Simon Denny explained how this can be related to an accelerationist ideology propagated by Silicon Valley tech-bros such as Peter Thiel, Marc Andreessen, and Alex Karp. Building upon a Futurist-inspired aesthetics and poetics, they claim that all that deliberation and democracy is slowing us down. No time for asking permission or discussion, or as defense minister Peter Hegseth would have it: we should aim for ‘maximum lethality, not tepid legality’.

Part of that ideology rests on the idea that our atomic age is over. AI is now the determining force. Because states can't deliver on their own, the argument goes, they must align with companies like Palantir, which build them the automated systems they need to dominate this new age. That comes wrapped in a strong nationalist reveille, aimed at turning MAGA USA into the first "technological republic." Imagery borrowed heavily from Futurist and even Nazi propaganda underwrites these claims on a symbolic level. The homo acceleratio is one who acts firmly. No more time for thinking, or for indulgent theatrical debates. Our time requires (I’m quoting from the Palantir Manifesto now) hard power. ‘And hard power in this century will be built on software.’

In Germany, the same defense-industrial turn is being sold, not coincidentally, as compensation for jobs lost to the declining auto industry: "Deutschland braucht ein Update" is an advertising slogan for the defense industry that would have been unimaginable in public a few years ago. Denny called this whole aesthetic package ‘a negative sublime’, and a ‘pop art of dread.’

Long-termism: exploitation, surveillance and extraction as moral goods for a greater good

But it is not their poppy aesthetics we should fear most. In another talk, Rainer Mühlhoff (author of the book AI and the New Fascism) highlighted the fascist logic behind these tendencies. For him, fascism isn't so much a coherent ideology or a fixed system as a process of radicalization that drives societies into barbarism, characterized by three elements: anti-democratic politics; a reliance on violence and a "right of the strongest"; and a close relationship with technology as an instrument of power for surveillance and oppression.

In the work of the aforementioned Palantir CEO Alex Karp, AI is proposed as a force of nature, like a thunderstorm on the horizon. You can prepare for it, but you can't avoid it. It connects to a strong social Darwinist streak: only the strongest, with the best genes, will survive, and only one AI-based state will make it through. It’s a claim that just last week was echoed in Donald Trump’s handling of the debates on AI. ‘WHOEVER WINS AI, WINS!’, he tweeted in his typical all-caps style. Adding that ‘the only control or “guardrails” that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT.’

Such ideas are related to extreme visions of "long-termism": the goal is to save civilization over the long term, on an evolutionary timescale of generations, possibly millennia. And that long-term goal should take precedence over the interests of people alive today.

In their playbook, this means that growth-based, exploitative economies, militarization, and state surveillance become moral goods in service of this highly speculative future. Democracy, by contrast, is an inefficient obstacle. Instead, we should build neo-monarchic states run by a CEO.

This can be achieved by folding big tech companies into the state. Or by experimenting with its mirror image: the network state, built on a terra nova (Greenland, private islands off the coast of Honduras) where regulation is absent and companies have free rein. In both cases, states become dependent on companies' private models for decision-making in the military, the police, or the everyday running of services for citizens, to the point where they can no longer regulate them.

Big tech companies replacing the nation state and the authoritarian stack

‘Move fast and break things’ has been the unofficial mantra of Silicon Valley for the last two decades. In the past decade, this claim of creative destruction has grown much deeper. Digital platforms started off as pursuing new business models within neoliberal democracies, with social media algorithms promoting anti-democratic tendencies mostly as a bug: collateral damage, an unwelcome outcome of their business strategy. Something that could perhaps be designed or regulated away. But now, a series of new books and commentaries (The Nerd Reich, End Times Fascism) show that some of these features that undermine democracies are not just a bug, they have become a feature. A number of Big Tech Companies want to become states themselves, it seems, and their governance models are far from democratic.

At Ars, Francesca Bria showed us the roadmap of these companies, building an ‘authoritarian stack’. From her report, analyzing an emerging bloc of Palantir, defense tech Anduril, Elon Musk, and Donald Trump Jr., among others:

<< Under the banner of "patriotic tech", this new bloc is building the infrastructure of control—clouds, AI, finance, drones, satellites—an integrated system we call the Authoritarian Stack. It is faster, ideological, and fully privatized: a regime where corporate boards, not public law, set the rules.>>

The stack they are building consists of an operating system for the state built by Palantir, the autonomous warfare products of Anduril at the military layer, all the way up to the building of a new monetary system built with crypto.

That's a bleak picture. Is there a way out?

It was not difficult to become disheartened after these talks, if only because of the sheer political, military and financial power of the main actors involved in what Naomi Klein has started to call ‘end time fascism.’

So, what can we do?

For Mühlhoff, it starts with the insight that these positions aren't just an extreme position within the political debate. They're de facto incompatible with democracy itself, and should be treated as such. As a starting point, democratic governments should refuse to do business with companies like Palantir that openly subscribe to these visions.

Francesca Bria made the case again for the Euro Stack: a deliberate EU investment to build an alternative to the Pax Silica of the US and China's Digital Silk Road. Rather than copying that logic, she argues, we should design it around democratic values. For instance, by developing AI models purpose-built for application in specialized sectors such as hospitals, newsrooms, and communities, rather than for scale. She also finds inspiration in the Bauhaus mentality, where artists and craftspeople explored human- and society-centered applications for the industrializing society of their time.

In two talks during the conference, Hito Steyerl picked up that role: the artist offering a symbol of hope, and a reminder that we've long neglected the public domain as a central tenet of democratic societies.

She found hope in a war memorial in former Yugoslavia, near the current border of Croatia and Bosnia: the Kordun Monument, designed by Vojin Bakić. Bakić's mirrored tower,  resembling the layers of a stack, honors the Partisans who resisted the fascist occupation of the country during the Second World War.

Can we resist them again? Perhaps by taking up Yugoslavia's former position as a non-aligned country, refusing to choose between the Cold War power blocs of East and West, and continuing the principles of the 1955 Bandung Conference, where 29 Asian and African states came together to seek a way out of colonialism.

The Kordun Monument, ironically, finds itself at a dire moment in history: the landscape nearby was recently fenced off, awaiting construction of a large data center, leaving the monument for now mostly as an urgent call to action

Strengthening the public domain

I found direction in another experiment by Steyerl. With colleagues, she's been building an alternative AI for image generation called "Lossy," trained only on public-domain material.

The pictures it's generated so far, though (including a portrait of the artist herself), are, by her own description, still rather ugly. The people depicted look, to quote Steyerl, "like they cannot afford proper health insurance and are in urgent need of a visit to the dentist.

She doesn’t see this as a failure of the system she built. Rather, it’s an accurate picture of what the public domain has become: dilapidated and disinvested. So far, nobody has funded it as shared infrastructure for a democratic society, leaving our public sphere to be managed by private companies instead.

For me, this is an important hook. Democratic societies are in dire need of a strong public infrastructure, and a digital public space that is built on democratic values, managed by organizations that are deeply committed to these values. The state also has an important role to play here as an enabler and regulator.

And so do we, albeit modestly, as researchers at a university of applied sciences. That’s why, in our projects, we keep on exploring alternative directions, looking for ways to design and maintain digital public spaces that do contribute to democracy.

And as always, we are looking forward to collaborating on that with many of you.

---

This is edition #9 of my irregular newsletter in which I reflect on Civic Interaction Design. With our AUAS - Civic Interaction Design Research Group at the Amsterdam University of Applied Sciences, we explore how design & technology can contribute to civic life: the numerous and varied interactions through which people in a society come together to strive towards collective well-being. My own interests include the design and societal implications of digital media platforms, hybrid public spaces & urban culture, XR storytelling, media architecture, and democracy.

I would like to move this newsletter to the open web. Would be great if you could follow me there: www.martijndewaal.nl.