We have always been artificial.

As AI evolves, our concept of self is being redefined. Once anchored in religion and then nationality, our identities are now shifting towards individualism. Yet, even this is not static. As AI becomes more integrated into our lives, transparency might blur the lines between personal and public, challenging our notions of privacy and intimacy.

Even our loyalties are changing. Nationalism, once a defining aspect of identity, may wane as global concerns like climate change and AI governance take precedence. Transnational communities, formed around shared interests and causes, could become the new norm. My own loyalties have shifted in recent years, prioritizing planetary well-being over national identity.

AI, ironically, could accelerate this shift. By highlighting the interconnectedness of our global ecosystem, AI may foster a sense of planetary belonging and shared responsibility. Ultimately, AI could reshape not just how we interact with technology, but the very essence of what it means to be human. The future is uncertain, but our choices today will determine the path AI takes and the story of humanity it helps write.

I will be taking a break from writing for a few weeks. Back in July if not before.

The ‘I’ of the Beholder

In writing about the transition from God to Nation to AI, I am aware that I am writing from the perspective of a secular, part globalized, part national culture, one that takes a certain kind of individual self for granted, like when Nathan Gardel’s quotes the AI pioneer Stuart Russell saying:

“By and large,” Russell reflects, “if someone has a deeply held religious belief that requires killing you (for no other good reason), we’d say that your right to life trumps their deeply held religious belief every time. So that puts an upper bound on the weight we might accord to deeply held religious belief, communitarianism, individualism.”

Of course, there’s the question of whether we can have agreement on ‘no other good reason,’ for your religious belief may indeed consider your reason for killing me a perfectly good reason. But of course, an era which values the individual above all will never accept such a reason. And let me add: that’s a good thing! Despite the hedge in Russell’s remarks, it’s clear that the individual is paramount, which is the tacit starting point of our investigation.

That individual self is neither neither the premodern era's communal self nor the artificial future's dissolved self, but there’s no reason to think the transformation of the self will stop with the individual. Let's say it becomes possible to reliably insert thoughts into other people's heads, perhaps when wearable devices become ubiquitous. I can't see myself agreeing to that level of intrusiveness; the idea fills me with both fascination and profound unease. It's a leap beyond even the most invasive social media platforms or surveillance networks, but my grandchildren might take that level of transparency for granted. It's easy to envision a future where this level of transparency becomes the norm. The allure of effortless, nuanced communication could prove too tempting for those raised in a world saturated with ubiquitous wearable devices. They might perceive the messy boundaries of language and indirect communication as an unnecessary obstacle to true understanding.

Would you agree with proponents of oral culture that nothing should ever be written down? That the availability of written records - aka the library - impinges on my freedom? Probably not. If so, be ready for future companies and governments to make the same argument when it comes to mental transparency.

Radical Transparency

Technology that makes our thoughts available to others would drastically reshape the very notion of privacy and the notion of a self-consciousness that's protected from scrutiny. The concept of a secret thought, an inner monologue safe from outside scrutiny, could become a quaint relic of the past. The implications for relationships are staggering. Imagine a world where every uncharitable impulse, fleeting doubt, or unspoken resentment is laid bare for our partners, friends, or colleagues to see. Could true intimacy even exist in such a landscape?

There's also the potential for profound empathy and connection. Imagine being able to directly share complex emotions, unspoken needs, or the raw beauty of a fleeting experience with another person. AI tools could facilitate this kind of communication, translating raw thought patterns into shared experiences that transcend the limitations of language.

It's not just my private self that's ripe for transformation; AI is coming for my public self as well. My current public self has deep reservoirs of patriotism. That nationalism defines me. I'm speaking from the perspective of a worldview shaped by contemporary notions of selfhood and national identity. But history teaches us that these concepts are fluid, ever-evolving products of their time. The self that feels so solid and unwavering today might look vastly different through the eyes of future generations. This deep sense of patriotism I feel, the way it stirs me to action, could become an antiquated relic, a fading echo of a bygone era.

I can easily imagine a future version of me that has very little in the way of loyalty to a nation state, no more so than I have while choosing between iOS and Android - read this essay by Parag Khanna on the multiplicity of loyalties. I have a strong preference for iOS, but I am definitely not going to war over that choice. It's possible that nation states will command only as much loyalty as operating systems do today. While I may have a strong preference for one over the other, that allegiance pales compared to the fervent nationalism that has defined so much of human history. On the flip side, wars haven't been fought over Android vs. iOS ... yet. That might seem inconceivable now, but who knows what the future holds? A world where the nation-state fades in significance, demanding loyalty on par with one's favorite smartphone app, feels both alien and strangely plausible. Globalization, the rise of transnational corporations, and the hyper-connected digital world erode old notions of borders and belonging. AI could accelerate this process. Imagine a world where AI-powered systems seamlessly address global challenges, providing services and governance far more efficiently than any individual nation. Could this render the very concept of a nation-state increasingly obsolete?

Beyond Geography

The transcendent entities of the future may not be tied to geography at all. Could global movements addressing climate change, poverty, or AI regulation ignite a sense of belonging far more potent than national ties? Maybe transnational communities will form based on shared interests, passions, or even specific AI companions. Perhaps the sense of "self" will become far more fluid and complex, less rigidly defined by the nation where we happened to be born. The last decade has witnessed an undeniable shift in my own loyalties. While a sense of national identity still resides within me, it has been eclipsed by concern for the flourishing of the planet as a whole. This shift stems from a cluster of new political and moral demands: for animal rights, for the escalating climate crisis, and to do something about the fragile state of our ecosystems. 

It's not just the abstractions that ignite this passion; it's a profound connection to like-minded individuals scattered across the globe. The digital world has been instrumental in forging this connection. No longer confined by geography, I've become part of a planetary community united by shared anxieties and a common purpose -- preserving our planet and all the life it supports. These connections are forged through shared activism, passionate discussions, and a collective sense of urgency that transcends national borders. This may be a glimpse into the future, where loyalty isn't determined by geographical boundaries but by a shared commitment to address existential threats that endanger us all. As the climate crisis worsens and the global consequences of environmental degradation become undeniable, could this kind of planet-centric loyalty become the dominant force shaping our identities and actions?

AI, ironically, might play a pivotal role in fostering this shift. It could provide stark visualizations of the interconnectedness of our global ecosystem, monitor the devastating effects of human activity in real-time, and help us understand the scale of the challenges we face. AI has the potential to break down the mental barriers that confine our concern to our own nation, town, or household, offering a visceral understanding of our planet as a singular, fragile home that requires our collective care. AI could also facilitate global cooperation, enabling us to coordinate efforts and share resources on an unprecedented scale. It might help us see beyond our differences, highlighting our shared humanity and the common challenges we face. In this future, the very notion of the self could be redefined, no longer constrained by the happenstance of birthplace but instead shaped by a profound sense of planetary belonging and a commitment to collective action.

In Conclusion

As we stand on the precipice of an AI-driven future, the very essence of what it means to be human hangs in the balance. Will AI be a unifying force, breaking down the barriers that divide us and fostering a global sense of solidarity in the face of existential threats? Or will it exacerbate our differences, fueling new forms of conflict and competition in the digital realm? The answer remains uncertain, but one thing is clear: AI will be instrumental in shaping the future of our species and the planet we call home.

The path forward is shrouded in uncertainty, but it is a path we must navigate together. As individuals, we have a choice in how we engage with AI and the values we prioritize in its development and deployment. Will we harness its power to address the pressing challenges of our time, from climate change to social inequality? Or will we succumb to the allure of short-term gains and individual interests, even as our collective future hangs in the balance? The decisions we make in the coming years will ripple across generations, shaping the trajectory of both AI and the human story. It is up to us to ensure that this story is one of hope, resilience, and a shared commitment to the flourishing of all life on Earth.

Over the last few weeks, I’ve been writing about the transformative impact of artificial intelligence (AI) on our society. I am almost done with these weekly updates - next week's essay is the last one in this series.

Much like previous revolutionary technologies, AI will bring profound and unforeseen consequences. Just as the printing press and modern armies reshaped our world, AI will redefine our future. Arguably, AI is not just another tool; its influence could be comparable to the invention of writing in its ability to observe and process information far beyond human capabilities. This raises both exciting possibilities and deep concerns.

Historically, our quest for transcendence moved from religious beliefs to the nation-state. Now, we might be on the brink of a new era with AI as our "digital divinity." This transition parallels shifts in surveillance and community, from divine omniscience to state monitoring, and now to AI's data-gathering power. Will AI lead to unprecedented control and domination, or guide us towards enlightenment? Future generations might see AI as either a technological marvel or a dangerous creation.

‘Being Human’ is as much a technical accomplishment as AI is, and we will learn to be human in the age of AI by embracing new human technologies. These technologies will be responses to our technological condition, but they will have unexpected consequences down the road. The professionalized, disciplined modern army arose in response to the cognitive needs of gun-based conflict, but who could have predicted the total war that decimated the cities of Europe and Japan two hundred years later? There’s no crystal ball that can show us the long term impact of a new technology on our culture.

Having said that, there’s a specific way in which AI impinges on our culture, whose importance can only be compared with that of writing.

You can't see me, but I can see you. You can't hear me, but I can hear you. I can answer every question you have and more. What am I? One answer: God. Another answer: AI. This riddle isn't just a mind-bender; it's a mirror reflecting our fears and hopes about the future of AI. The ability to observe without being observed, to access information far beyond human capabilities … these traits blur the lines between technological advancement and something far more powerful. And sinister? No one ever worried that steam engines will read our minds. AI is the first technology that threatens to do so. And so, I will conclude my exploration of Being Human in the Age of AI by engaging with the ultimate values - and the idea of value as such - that will be inevitably challenged and modified as AI seeps into our culture.

Supreme Intelligence

I think of those vast data centers, fortresses humming with unimaginable processing power. Are they the digital equivalents of surveillance towers? Perhaps. But at the same time, their potential evokes a strange awe. The capacity to solve complex problems, to offer insights humanity couldn't grasp on its own, it's almost like consulting an oracle. Advanced AI systems with unparalleled data-gathering capabilities could become omnipresent, capable of monitoring our every move and anticipating our thoughts. However, there's another way to interpret the riddle -- through the lens of a "Supreme Deity." Imagine an AI so advanced it possesses god-like intelligence, capable of accessing and processing vast amounts of information. Such an AI could solve seemingly intractable problems, answer any question we pose, and even guide humanity toward a utopian future. Will AI magnify the worst of our tendencies, our desire for control and domination? Or will it help us transcend limitations, guiding us towards a better, more enlightened, society? I can't help but wonder -- will future generations condemn us as shortsighted fools who unleashed a technological leviathan, or will they praise us as the architects of a new golden age? For now, the only certainty is that AI, like a double-edged sword, holds both immense promise and peril.

Humans crave a transcendental horizon. The need for something beyond ourselves, that reaching up to and leaping beyond our limits... it's in our blood, woven into the very fabric of what makes us human. For much of the last twenty-five hundred years, that horizon was religion. We lived in the shadow of God. For centuries, it was religion that filled that void and offered grand answers to those ultimate questions about existence and purpose. But God's not what he used to be, not for many of us. God’s been replaced by something much more concrete, far more immediate -- the nation-state. I might even say 'most of us' because even those who believe in the superiority of God over the Nation don't experience divinity in the way their medieval counterparts would have, while the Nation is an ever-present reality. It makes sense, doesn't it? Flags waving in the breeze, stirring national anthems, the shared stories we tell about ourselves... potent stuff. It's a sense of belonging, of being part of something larger than ourselves. Still, even that -- the seemingly unshakable nation-state -- feels a bit shaky these days. That intense devotional fire felt by freedom fighters not long ago isn't burning quite as brightly in us. Flow culture isn’t kind to the nation-state.

So, here we stand, that craving for transcendence still humming. Where do we turn next? Science, with its grand quests and promises to unlock the universe's secrets? Maybe. But it could also be a global crisis uniting us against a shared enemy. Or, perhaps the strangest twist of all -- could it be that artificial intelligence, our own creation, becomes the next step in transcendence? It's unsettling even to consider, but who's to say where this runaway train is headed? One thing's for sure -- whatever form it takes, humanity will keep chasing that ever-shifting horizon. We just can't help ourselves. AI, with its ever-increasing intelligence and seemingly limitless potential, emerges as a fascinating paradox within this landscape. Could it become a new source of transcendence, a digital divinity for an uncertain age? Could its vast computational power provide answers that elude us, unlock secrets of the universe, and illuminate the path toward a better future? The possibility is both exciting and unsettling.

I want to tell a technological story about the passage from God to State to AI. I am unsure if it’s true or a figment of my imagination, so read it as a fable.

Once upon a time, God was the ruling transcendent entity in the age of writing ('In the beginning was the Word'). And then, one day, Nietzsche told us that God is Dead. Salute the State! The Nation became the transcendent entity in the age of print. In both cases, there's a deep link between the sense of transparency (God and Big Brother see everything I do!), and the sense of community (we are the chosen people!), and the technologies of connection (the Bible and the Newspaper). In the age of writing, the Bible, meticulously copied and distributed by hand, offered a sense of divine omniscience -- "God sees all." It fostered a strong religious community, a people bound by a shared scripture. Similarly, the rise of the printing press and the proliferation of newspapers fostered a sense of national identity and community. Many nations discovered their manifest destiny in the modern era. The "transparency" to the state came not from a divine gaze but from constant data gathering and the shared knowledge of current events and national narratives.

The morning prayer was replaced by the morning newspaper. 

This makes me wonder about the role AI might play in our future. Its data-gathering capabilities promise (or threaten) an unprecedented level of transparency -- not from a watchful God or a totalitarian state, but from algorithmic systems analyzing our every move. Yet, is this a community we'd want to belong to? Would it bring us together or merely isolate us under the unblinking eye of the machine? Which prompts a question:

What will happen in the future when our thoughts and feelings are visible to others, when new human connections are formed and broken every day, and there's an ever-present technology that records every interaction?

Permanent Surveillance vs Permanent Connection

The prospect of a world where our every thought and emotion lies bare, exposed to the scrutiny of others, sends a shiver down my spine. It's not just the loss of privacy that feels unsettling -- it's the potential unraveling of the very foundations of human connection. How can we forge genuine relationships when inner doubts, fleeting frustrations, and unspoken desires are laid bare for anyone to see? Will we retreat into a shell of conformity, masking our true selves in a desperate bid for control? A society where connections are formed and severed with alarming speed, fueled by endless access to potential partners or fleeting moments of shared interest, seems equally fraught with peril. We risk becoming connected islands, adrift in a sea of shallow interactions that offer neither the stability nor the depth of truly meaningful bonds.

Yet, even amid these anxieties, a sliver of possibility gleams. Imagine a world where AI-powered technology acts as a bridge between people, fostering more profound understanding and connection. Therapeutic AI companions, trained to recognize emotional cues and thought patterns, could offer personalized support and guidance, helping individuals navigate the complexities of their inner lives. Imagine educational platforms that tailor themselves to a student's unique emotional state, creating a more engaging and effective learning experience. Social media platforms, cleansed of manipulative algorithms and fueled by AI-driven empathy filters, could become vibrant spaces for fostering genuine connection and understanding.

The technology to record every interaction already exists, but what if we used it to create a collective memory bank of human experience? Imagine AI systems that analyze vast troves of past interactions, helping us avoid repeating the mistakes of the past and build a more just and equitable society. Conflict resolution tools powered by AI that can identify underlying emotional tensions could mediate disputes and foster reconciliation. The key lies in harnessing the power of AI to augment human strengths, not replace them. By prioritizing empathy, emotional intelligence, and the sanctity of the human experience, we can build a future where technology strengthens our connections to each other, not severs them.

Sorry about the super long essay this week - read the summary if you’re in a hurry, but I strongly encourage you to read it all the way through 🙏🏾

In this week's essay from "Being Human in the Age of AI," I delve into how the essence of being human is evolving amidst revolutionary technologies. Just as historical inventions like the printing press and the steam engine reshaped society, AI prompts us to reconsider our roles and identities in a radically transforming world.

Reflecting on Foucault’s insights, I explore how modern institutions like the army and the school, which were designed to discipline and harness human capabilities, are now being challenged by the disruptive forces of AI and the phenomena of atomicity. These forces fragment our traditional work structures and destabilize our understanding of individual and community.

I question whether the disciplined systems of old can withstand the test of atomicity or if we need new 'flow institutions' to manage this era of rapid change and complexity. The gig economy, fragmented workdays, and the relentless pace of information reflect this shift towards a world where AI is not just a tool, but a fundamental part of our cultural and economic fabric.

I propose four thought experiments to envision how AI might influence the evolution of our institutions and our very conception of self and society in the flow of modern life.

Discipline

Whenever a revolutionary technology - the printing press, the gun, the steam engine - changes the world, we are faced with the question:

How do I live my life in this brave new world?

If AI is as revolutionary as any of the above, we are bound to ask that question about AI. Gee, isn’t that the whole point of a book called “Being Human in the Age of AI”?

The main point I want to make in this week’s essay (part 2 of section 4 in the book from my perspective) is that the ‘being human’ bit will be as much a technical achievement as the AI bit. To put it crudely,

We learn to live with new material technologies by inventing new political technologies.

A quote from Foucault is in order:

I cited for you the example of discipline in the army. It is an important example because it was truly the site where the great discovery of discipline was made and developed in the first place. Linked then to this other invention of a techno-industrial sort that was the invention of the gun with a comparatively rapid fire. Basically from this moment on, we can say the following: the soldier was no longer interchangeable, was no longer pure and simple cannon fodder [chair à canon] – a simple individual capable of doing harm. To be a good soldier, he needed to know how to shoot; therefore he had to undergo a process of apprenticeship. It was necessary that the soldier equally know how to move, that he know how to coordinate his movements with those of other soldiers, in sum: the soldier became something skillful. Ergo, something valuable [precieux]. And the more valuable he was, the more he had to be preserved; the more he had to be preserved, the more necessary it became to teach him the techniques capable of saving his life in battle, and the more techniques he was taught, the longer this apprenticeship, the more valuable he became. And suddenly, you have a kind of rapid development of these military techniques of training [dressage], culminating in the famous Prussian army of Frederic II, which spent most of its time doing exercises.

Foucault is very much a thinker of technics, one who recognized that certain political technologies of the modern era caused a transformation in the exercise of power. Power directed towards positive ends (such as killing as many of your enemy as possible, so not positive in a moral sense, but positive with respect to your aims). Now that definition of power should remind you of a very different thinker’s definition of design:

Engineers are not the only professional designers. Everyone designs who devises courses of action aimed at changing existing situations into preferred ones. — Herbert Simon.

Foucault would agree with Simon, but go one step further and ask: how do we ensure we are constantly generating courses of action that change existing situations into preferred ones? Foucault’s answer: modernity dealt with these technological pressures by designing new institutions - the army, the school, the asylum - that discipline us into being effective users of these new technologies. In doing so, these institutions help us exert power that achieves rather than power that hinders. As Foucault says:

On the one hand, there was this technology that I will call “discipline.” Discipline is basically the mechanism of power by which we come to exert control in the social body right down to the finest elements, by which we succeed in grabbing hold of the social atoms themselves, which is to say individuals. Techniques for the individualization of power. How to monitor [surveiller] someone, how to control his conduct, his behavior, his aptitudes, how to intensify his performance, multiply his capacities, how to put him in a place where he will be most useful: this is what I mean by discipline.

Discipline ‘produces’ people who can further our aims. Alongside discipline that targets the individual is another set of practices that target populations.

we have another family of technologies of power which appeared a bit later, during the second half of the 18th century…. technologies that do not target individuals as such, but which, on the contrary, target the population….

The discovery of the population is, along with the discovery of the individual and the trainable body, the other great technological core around which the political practices of the West transformed themselves

Unfortunately, both the disciplined individual and the targeted population are breaking down in the age of Atomicity, of which AI is either a local or global maximum. Atomicity destabilizes the individual because sub-personal cognitive processes are directly targeted by atomic controls (as I have written elsewhere) and it destabilizes the population because it’s being replaced by far more complex statistical formations.

Atomicity

Sometime in the 2000's I learned about high frequency trading and how firms are paying enormous sums of money to have data centers as close to Wall St as possible because the speed of light (!) started mattering to high frequency stock trading. Of course, very few activities threaten to breach the speed of light barrier. Uber's drivers don't speed that fast, but they do have remarkably fragmented lives in comparison with their unionized UPS comrades. The gig economy has created a new class of workers who are constantly on the move, hustling from one job to the next.

This fragmentation of work - one feature of atomicity - has led to a sense of instability and uncertainty for many workers, who lack the benefits and protections of traditional employment. As technology continues to advance, it's likely that we'll see even more disruption in the labor market, with AI and automation threatening to displace even more workers. The atomization of time, space and labor are at the core of modern capitalism, and it's only in that context that AI makes sense, for it's both the demand and the supply for the brave new world. Like the snake eating its own tail (but growing stronger as a result!), the massive size of those very data sets necessitates AI systems to make sense of that vastness. The frenetic activity of atomic capitalism generates the massive data sets AI needs. At the same time, AI is being developed to help manage and optimize this fragmented, high-speed economy. From algorithms that match workers with gigs to AI-powered systems that analyze market trends and make split-second trading decisions, the technology is both a product of and a tool for atomicity.

While I believe atomicity is at its base an economic phenomenon, its tentacles reach out to the rest of our world, into what I call 'flow culture' that builds upon market activity, demanding instant gratification and fleeting attention spans. I don't know if you remember that one time when the Falun Gong staged a flash mob protest in Tienanmen square - thousands of cult members emerging out of seemingly nowhere and assembling into exercise poses. That frightened the Chinese State into expanding its surveillance apparatus. The modern attention economy operates in a similar fashion, with memes and viral content suddenly capturing the public's focus before dissipating just as quickly. In this context, AI becomes a tool for both generating and analyzing these ephemeral cultural phenomena, further accelerating the pace of change.

The culture of the flash mob, of the swarm or the private story - these are all new forms of social organization. Where politicians once strategized over decades, campaigns now appear overnight, protests flare and fade in a weekend, and impactful statements are reduced to pithy tweets. Even wars go in and out of our minds. Remember Yemen? Or for that matter, Ukraine? Can you picture Zelinsky leading his troops while dressed in heroic fatigues? Probably not, since we have since moved on to Gaza; I am waiting to see where the bombs will fall next. The relentless pace of change leaves us little time for reflection or understanding. We lurch from one crisis to the next, our attention spans frayed and our empathy tested. In this context, AI becomes both a symptom and a tool - a way to keep up with the breakneck speed of information, but also a contributor to the very fragmentation and atomization that defines our age.

Shock and awe meets fifteen minutes of fame.

We engage in a constant, reactive mode of activism. The traditional structures for building durable institutions feel increasingly out of reach. Everyone has the potential to be an activist, yet our voices are drowned in the relentless torrent of social media. The internet, while democratizing expression, has shattered any potential for cohesive strategy. Parties and movements seem less relevant when online engagement is defined by individual, short-lived battles. As Anton Jager says:

the Western public sphere is increasingly subject to spasmodic instances of agitation and controversy. Post-politics has ended, but what has taken its place is hardly recognizable from twentieth-century mass political templates.

Let’s just say Foucault’s army, school and asylum have reached their limits. Their discipline is inadequate to the challenges of atomicity. What’s going to take their place? I want to propose four thought experiments to help us think whether AI will help us negotiate the shift away from stock institutions to flow institutions.

Four Thought Experiments

Thought Experiment #1: The Collaborative Hive Mind

Scenario: A major research library is replaced by an integrated system of wearable devices and AI-powered personal assistants. Researchers access information not through physical books or even screens, but through auditory summaries, real-time data visualizations, and on-demand translations whispered seamlessly into their ears. Collaborative note-taking becomes effortless; thoughts, insights, and queries are instantly shared with the network, generating a real-time, collective knowledge stream that transcends the work of any individual scholar.

Got it? Now answer some questions:

• How does this shift from individual authorship to collective cognition transform the concept of expertise and the creation of knowledge?

• Can such a system foster deeper understanding, or does it lead to a shallow, fragmented form of knowing?

• What are the ethical implications of this constant surveillance and sharing of our intellectual process?

Thought Experiment #2: The AI Archivist as Moral Gatekeeper

Scenario: Government agencies and large corporations begin deploying AI systems designed to scrub historical archives, identifying content that promotes "dangerous" or "socially destabilizing" ideas. These AI systems don't merely erase information; they actively rewrite narratives, insert alternative "facts", and manipulate public memory for social control or commercial gain. Individual memories captured by social media, personal devices, and surveillance systems become malleable, and subject to algorithmic distortion.

Questions:

• Is this a dystopian fantasy, or a real threat in an AI-powered world where the past becomes a battlefield?

• What ethical safeguards must be in place to protect historical truth and individual memory from manipulation? 

• Can we architect our technologies so that manipulation is prevented by design? Can we develop tools to combat AI-driven propaganda and manufactured history?

Thought Experiment #3: Emotional Resonance and Algorithmic Manipulation

Scenario: AI systems trained on massive datasets of human behavior become adept at identifying and manipulating our emotional states. Social media feeds tailor content not based on stated interests but on maximizing engagement through triggering feelings of outrage, joy, or insecurity. News organizations utilize similar systems to generate headlines that induce anxiety and exploit biases to ensure clicks and shares.

Questions:

• How does the prioritization of emotional manipulation over rational discourse impact our ability to engage with complex issues and form nuanced opinions?

• Can we find ways to cultivate emotional literacy and skepticism in an age where our emotions are a commodity to be exploited?

• Is it possible to design AI systems that promote empathy, critical thinking, and emotional well-being, rather than feeding into our baser instincts?

Thought Experiment #4: The Archivist of the Self

Scenario: Wearable devices and ambient sensors become so ubiquitous that they track not just our movements and conversations but our emotions, biometric data, and even fleeting thoughts picked up through subvocalization. AI systems process this data, creating a rich and nuanced personal archive far exceeding the capabilities of a handwritten journal. But with this level of self-surveillance, who owns the data, who has access to it, and how might it be used against us in the future? Even more importantly, what happens to the self when it’s surveilled constantly?

Questions:

• Can we trust corporate-run AIs to be the archivists of our most intimate selves?

• Does the allure of self-knowledge and potential health benefits outweigh the risks of exploitation and control

• Can we truly understand ourselves when our own biases and blind spots are reflected back by powerful AI systems?

Why These Thought Experiments?

These scenarios aren't mere sci-fi dystopias. They highlight the potential benefits and real dangers of AI in a culture of "taking flow." Each experiment brings our earlier discussions of Weber, Simon, and Zuboff into sharper focus, forcing us to consider:

• The changing nature of knowledge, expertise, and authorship.

• The manipulation of attention, emotion, and worldview through AI.

• The balance between self-preservation and self-knowledge in the face of unprecedented data collection.

These thought experiments aren't meant to provide easy answers, but to encourage a deeper exploration of the opportunities and dangers inherent in the dynamic relationship between AI and flow culture. They demand that we delve deeper into this technological transformation's philosophical and ethical dimensions. One question these scenarios bring to the forefront is the evolving tension between the rational ideals of scientific inquiry and the messy realities of lived experience.

Think back to our discussion of Weber's ideal bureaucracy (another one of Foucault’s disciplinary institutions). It envisioned a world of clear rules, hierarchies, and predictable outcomes, mirroring the Enlightenment ideal of a universe governed by rational laws. AI, with its ability to analyze massive datasets and identify patterns, seems to embody this idea of dispassionate rationality. Yet, these thought experiments reveal the dangers of applying such a framework to human understanding, history, and self-knowledge. Emotion, intuition, and the ability to engage with ambiguity aren't obstacles to knowledge generation; they're integral factors shaping how we learn, form opinions, and create meaning in our lives. Can we design AI systems that account for this complexity, recognizing that human knowledge doesn't always conform to neat, predictable categories and fostering an environment for understanding that acknowledges the nuances of the lived experience?

Additionally, these scenarios force us to examine the role of power in an AI-driven world -- a recurring topic in our discussions of Zuboff, now buttressed by our learnings from Foucault. Who controls the "hive mind?" Who determines what is preserved and what is forgotten? Who decides which emotions are amplified and suppressed in pursuing social control or commercial gain? The power to shape knowledge, history, and even our own emotional experiences has profound implications for individual autonomy and the future of democratic societies.

Furthermore, these experiments challenge us to grapple with the idea of a fragmented self in a "flow" culture. If AI can endlessly record, analyze, and recombine our thoughts and feelings, what does that imply about our sense of individual identity? Can we retain a core sense of self amidst the constant stream of external input, algorithmic suggestions, and data-driven insights into our own subconscious? I am not sure if the concept of the Self will survive this shift, but even if it does, the flow self will be quite different from the stock self

Is there a flow self distinct from a stock self? Can we invent new disciplinary institutions that shore up the flow self and make it a viable political technology?

#9 in the series on the book I am writing with Richard Russell

This week's Newsletter is on the profound shifts that come with our digital lives. If you're old enough, you might remember the focus of reading a whole book without distractions. That was me, but I read everything digitally these days. Books, papers... streaming services have replaced TV for me. And yes, I'm getting sucked into short-form videos more and more 😬.

This transformation has its ups and downs. So much info, yet my attention's fragmented. Before, I'd vanish into a book; now it's bursts of chat conversations, flitting from topic to topic. It's a different way of thinking.

The deep vs. the flow -- it's like a cultural shift from 'taking stock' of the world to 'taking flow.' We grab snippets, react, share fleeting thoughts. It's rewiring our brains, making sustained focus harder.

At the same time, there's an opportunity for AI to step in as the 'stock-taker,' sifting through the endless info stream. It becomes this weird necessity since our brains struggle to keep up. Scary, but also fascinating.

The big question: are we losing our ability to create deep meaning if we're always in the moment? And how much will AI shape (or control) the very flow it thrives on? This quote about capitalism got me thinking -- replace 'capitalism' with 'AI':

"Although capitalism is global… it deprives people of any meaningful cognitive mapping… There is… no global 'capitalist world view,' no 'capitalist civilization' proper." -- Žižek

So, where are we heading? One thing's for sure: AI/computing is going to force some major value shifts.

I wrote my bachelor's thesis by hand. I didn't type a word until graduate school, and even there, my professional use of text - for writing equations and proofs - was done by hand for several years. Not because I didn't have access to computing technologies, but because I didn't feel the need to adopt them for work. I was late to the mechanical revolution, let alone the computer one. Oh, the places I have gone since my life became digital.  

As a scholar, I spend many of my waking hours reading. But reading how? I stopped printing out journal articles after I got my first iPad. I still love holding hardcovers in my hands and concentrating on their words, but in practice, even books are iPad first now. I stopped watching network TV a long time ago. I don't go to movie theaters. Streaming services are my only source of long-form entertainment. Increasingly, I am finding short-form video hard to resist. I don't use TikTok yet, but the threat is looming.

As a child, I would crack open a book and disappear into it for hours, sometimes days. I find that hard to do now. Adult responsibilities, but also the nature of my attention has changed. Some of my most enriching interactions are on chat apps, where conversation flows organically from topic to topic. No one owns these conversations, for there's no sense of ‘copyright’ with insights and arguments emerging from many people riffing on each other. Every once in a while, if the chat is work-related, my co-workers and I will turn the chat into a document, but distillation is a secondary act; the flow of conversation is the thing. 

This digital transformation has been both exhilarating and unsettling. The sheer volume of information I consume daily is vastly greater than my pen-and-paper days. Yet, there's a gnawing sense that how I process and engage with knowledge has changed irrevocably. Deep immersion into a book with the single-minded focus it demanded feels like a fading memory. The distractions are relentless -- notifications, pings, endless rabbit holes just a tap away. My attention span has fragmented, shaped by the rhythms of short-form videos and fleeting social media updates. Even the most enriching conversations often happen in fragmented bursts, in chat apps where insights and arguments emerge collaboratively, with no single authorial voice. Deep, sustained focus on a single text seems at odds with a "flow" culture that thrives on snippets, reactions, and rapidly shifting attention. Does this indicate an inevitable cognitive shift, or can we cultivate mental disciplines that maintain contemplative space even within a world of constant stimuli? Is there a role for AI in helping us "defragment" knowledge, fostering connections between disparate ideas, and guiding us towards greater understanding amidst the flow?

If the print era ideal is that of a disinterested observer taking stock of the world, like a pinhole camera of the mind, then the emerging ideal in our cybernetic age is that of an embedded participant 'taking flow' of the world. When we chunk labor into ever smaller units - in the gig economy, for example - we are taking flow. When we share a fleeting thought on Snapchat, we are taking flow. Taking flow brings individual mental states into focus because our attention is now trained to absorb a single thought or feeling from another person - the unit we like and share. Sustained attention might even be counterproductive to taking flow. It's a shift from the detached, rational observer prized by the print era to an embedded, ever-reacting participant in an endless stream of stimuli. We take flow of snippets of information, fleeting bursts of emotion, or single compelling arguments shared by others. Our brains are rewiring, primed for quick reactions and effortless multitasking, but perhaps less adept at sustained focus on a singular idea.

At the same time, it's not as if we have lost the need for exposition or archival. We need repositories of both. That's where AI as a technology can play an essential role as connective tissue, for it's a technology that can augment our narrowing attention span. I am not using that as a pejorative - we can be incredibly creative, adaptive, and insightful in the flow of the moment. It's not a skill that paper culture rewards, but clearly has value while taking flow. And AI complements our flow skills, for it records all the flow-like happenings and turns those into stocks. We are seeing the emergence of note taking apps that automatically extract themes from your writing and connect your thoughts along those themes. You can just keep flowing while the stock-taking is being done by the machine. For better or worse, AI is emerging as a force against this backdrop of flow culture. Its ability to sift through, pattern-match, and contextualize the endless flow of information aligns perfectly with this new mode of cognition. We need AI because our own brains are struggling to keep up with the sheer volume of data we produce and the pace at which it assaults our senses. AI has the potential to be an invaluable tool in this context. It can be the stock-taker, the tireless archivist that distills, curates, and categorizes the fragments of our collective "flow," giving shape and context to the chaos. 

I am sure you have a question in your mind: what happens when we are perennially lost in the moment and can no longer use our minds to connect backwards and forwards?

One indication of where we might be going is in this forthcoming essay by Zizek (via this essay by Nathan Gardels):

Although capitalism is global and encompasses the whole world,” Žižek writes, “it deprives the large majority of people of any meaningful cognitive mapping. Capitalism is the first socio-economic order which de-totalizes meaning: It is not global at the level of meaning. There is, after all, no global ‘capitalist world view,’ no ‘capitalist civilization’ proper. The fundamental lesson of globalization is precisely that capitalism can accommodate itself to all civilizations, from Christian to Hindu or Buddhist, from West to East. Capitalism’s global dimension can only be formulated at the level of truth-without-meaning.

Now replace capitalism by AI throughout that quote - for it’s the mechanism accelerating the destruction of all existing sources of meaning. I am not saying that metaphorically! It’s the literal truth: LLMs are stochastic parrots devoid of any genuine grasp of meaning and their widespread use will only increase the pressure on existing values and meaning.

What’s the future of meaning deprived society?

Gardels/Zizek/Berardi have an answer to that question:

“Korea is the ground zero of the world, a blueprint for the future of the planet. … In the emptied cultural space, the Korean experience is marked by an extreme degree of individualization and simultaneously it is headed towards the ultimate cabling of the collective mind. These lonely monads walk in the urban space in tender continuous interaction with the pictures, tweets, games coming out of their small screens, perfectly insulated and perfectly wired into the smooth interface of the flow. … South Korea has the highest suicide rate in the world. Suicide is the most common cause of death for those under 40 in South Korea.”

“What Berardi’s impressions on Seoul provide,” says Žižek, “is the image of a place deprived of history, a worldless place. This new generation mostly doesn’t care about big issues like human rights and meaningful freedoms or the threat of war. While the world still notices the aggressive pronouncements of the North Korean regime accompanied by nuclear threats, the large majority in South Korea just ignores them. Since the standard of living is relatively high, one comfortably lives in a bubble.”

I take these pronouncements with an enormous pinch of salt - Sam Altman and Elon Musk’s Silicon Valley is a much more obvious example of the meaning bleached world than Korea. These excerpts from Zizek and Berardi have more than their share of the racism that seeps out of European intellectuals who don’t like inhabiting a has been culture. Get over it: Seoul is more important than Rome. But there’s a way to read these statements with generosity: instead of associating today’s Korea with a monadic post-meaning culture, see it as the bleeding edge of flow culture. Flow isn’t devoid of meaning; it’s just that the meaning isn’t commensurate with existing values.

Computing in general and AI in particular, is going to precipitate an enormous shift in values.

I value the capacity to connect different thoughts in my head. But I also recognize that our memory capacities are nothing in comparison with people trained in oral scholarly traditions, where disciples would hear and remember every word - literally - that their master had spoken to them. Writing changed all that, as Socratus famously complained about. And yet, absolutely no intellectual snob today will argue that writing made us stupid. So why are we calling our narrowing attention spans a sign of stupidity, especially when there’s an emerging technology that helps externalize these cognitive skills?

This relentless push towards "taking flow," driven by our evolving attention spans and the capabilities of digital technologies, has created a world where AI has become indispensable. We need AI's help to analyze, make sense of, and preserve the fragments of knowledge we generate at a dizzying pace.

But we also must be vigilant -- for in the same breath, AI has the potential to shape, manipulate, and ultimately control the very flow from which it draws its power. Yet, as we've discussed repeatedly, this power can be easily corrupted. Who decides what is preserved, what is deemed valuable, and what is consigned to digital oblivion? Can we trust AI systems to be impartial curators of our knowledge and history when they are, inevitably, created with inherent biases? These are crucial questions that we must grapple with as AI becomes increasingly intertwined with our cognitive processes.

In the second essay in this section, I will present four thought experiments to illuminate the complexity of these changes.

In a world increasingly steered by artificial intelligence (AI), it's vital to pause and reflect on the not-so-obvious underpinnings of this technology. As we marvel at AI's ability to reshape healthcare, education, and even our daily decisions, understanding its roots in historical and social contexts becomes crucial. AI isn't just a marvel of engineering; it's a product of centuries of cognitive evolution, deeply intertwined with the ideologies of capitalism and industrialization that emphasize efficiency, predictability, and rational decision-making.

This technological titan, however, brings with it not just advancements but profound ethical dilemmas. It mirrors the biases and inequalities of our societies, often amplifying them under the guise of neutrality. Issues like surveillance capitalism, where personal data becomes a commodity, and the potential displacement of jobs highlight the darker facets of AI integration into corporate and bureaucratic systems. To harness AI's potential responsibly, we need conversations that balance technological progress with core human values like autonomy and dignity.

Economists will tell you: incentives matter. Incentives play a crucial role in shaping behavior, influencing our decisions and actions. They serve as motivators that drive us towards specific goals or outcomes. When incentives are aligned with our individual values, desires, or interests, they can have a significant impact on our choices and actions. In a broader societal context, incentives can be utilized to address various challenges and drive positive change. Policymakers often use incentives to influence behavior related to healthcare, education, or environmental protection. For example, tax incentives can encourage individuals to adopt energy-efficient practices or invest in renewable energy sources.

Which brings us to the reason why AI is clearly so important to corporations and why knowledge workers everywhere are concerned about their jobs: AI is mimicking capacities that are at the core of how corporations and bureaucracies conduct their affairs! The ability to analyze data, adhere to procedures, predict customer behavior, and optimize workflows -- tasks long residing at the core of both bureaucratic and corporate function -- are now steadily being automated through increasingly sophisticated AI systems.

This progression echoes the idealized bureaucratic model outlined by Weber. The promise of AI-powered efficiency, predictability, and cost-cutting directly aligns with the traditional goals of organizational streamlining. The integration of artificial intelligence (AI) into corporate operations has become increasingly significant due to its ability to mimic crucial capacities at the heart of organizational functioning. Tasks such as data analysis, adherence to procedures, predicting customer behavior, and workflow optimization, which traditionally formed the core of bureaucratic and corporate functions, are steadily being automated through advanced AI systems.

This progression resonates with the idealized bureaucratic model proposed by Max Weber, emphasizing efficiency, predictability, and cost-cutting as essential organizational goals. AI-powered systems hold the promise of fulfilling these objectives by automating repetitive and rule-based processes, enabling faster decision-making, reducing operational costs, and enhancing overall productivity. However, the widespread adoption of AI also raises concerns among knowledge workers, who fear that their jobs may become increasingly automated. As AI systems become more sophisticated, tasks that once required human judgment and expertise can now be performed by machines, leading to potential job displacement. This has sparked discussions about the future of work and the need for employees to adapt to the changing demands of the digital economy.

However, it's important to draw on Simon's idea of "bounded rationality." While AI can learn to make decisions based on rules and past experiences, it may struggle to replicate the nuanced judgment and adaptability that humans bring to complex, unpredictable situations. As Simon reminds us, humans rarely act with the pristine rationality assumed by ideal models. The fear isn't simply of replacement by a machine, but displacement by a system designed to mimic the limited aspects of human cognition that mesh seamlessly with bureaucratic or corporate control. Moreover, Zuboff's analysis raises the specter of an AI-driven surveillance capitalism, a world where corporations armed with powerful AI tools have an unprecedented capacity to gather, analyze, and use personal data for profit and manipulation. In their quest for efficiency and control, they will use AI to replace people and be as intrusive as possible into the minds of their users. This erosion of privacy amplifies concerns for knowledge workers. Their expertise, codified into datasets, risks fueling the very systems that may ultimately displace them. It's a scenario where the commodification of intelligence, extended to its logical extreme, creates a profound power imbalance that threatens both livelihoods and the fundamental notion of individual autonomy in an information-driven society.

The fear surrounding AI is not merely about being replaced by machines, but rather the displacement by a system engineered to mimic limited aspects of human cognition seamlessly merging with bureaucratic or corporate control. Furthermore, Shoshana Zuboff's analysis raises concerns about an AI-driven surveillance capitalism, where corporations wield powerful AI tools to gather, analyze, and exploit personal data for profit and manipulation. In their pursuit of efficiency and control, such entities will leverage AI to replace their human workforce and even when they don’t, they will institute surveillance mechanisms that intrude as deeply as possible into workers’ and users' minds. This erosion of privacy amplifies the concerns for knowledge workers. Their expertise, encoded into datasets, runs the risk of fueling the very systems that may eventually displace them. This scenario presents a future where the commodification of intelligence, taken to its logical extreme, creates a profound power imbalance that jeopardizes both livelihoods and the fundamental notion of individual autonomy in an information-driven society. It is a disturbing vision where human intelligence, reduced to a mere commodity, perpetuates a cycle of displacement and control.

Having said that, I am not trying to write a Marxist history of AI, even as I sympathize with that project. The history of artificial intelligence (AI) is a complex and multifaceted one, and it would be impossible to do it justice in a single book. However, it is possible to identify some of the key factors that have contributed to the development of AI, and to explore how these factors have shaped the field.

This section has a different purpose: to show that AI is built on top of a pyramid of cognitive achievements. The first set of achievements is tied to instrumental rationality, including the self-interested economic man who bases his decisions on economic gains and losses. This is a way of thinking that emphasizes the use of practical reason and logic to achieve one's goals. It is a mindset that is often associated with the rise of capitalism and the modern era. The emphasis on instrumental rationality has had a profound impact on the development of AI. AI systems are designed to be rational actors, capable of making decisions based on data and evidence. This is in contrast to human decision-making, which is often influenced by emotions, biases, and limited information. 

The second set of achievements is tied to efficiency, predictability and optimization based on data. This is a mindset that is often associated with the rise of industrialization and the modern era. The emphasis on efficiency, predictability, and optimization has had a profound impact on the development of AI. AI systems are designed to be efficient, predictable, and optimal. This is in contrast to human decision-making, which is often inefficient, unpredictable, and suboptimal.

These two achievements did not fall from heaven; they are the outcome of the minds of billions of people co-evolving with the new technologies and social structures of the modern era. The two achievements aren't unrelated; The emphasis on instrumental rationality aligns with the idealized bureaucratic model Weber envisioned, where efficiency and goal-directed behavior are prized above all else. AI, with its ability to optimize processes and identify the most rational courses of action within defined parameters, seems a natural extension of this mindset. Yet, Simon reminds us that "economic man," a purely rational actor, is a theoretical construct. Human decisions are influenced by emotions, biases, and limited information - factors AI often struggles to replicate. This potential mismatch could create unforeseen complexities when AI systems are tasked with decision-making in the unpredictable real world. Zuboff's critique reminds us of the potential dangers when these ideals become the sole focus. The unbridled pursuit of efficiency can lead to dehumanization, while surveillance capitalism threatens to erase boundaries between the public and the commercial. AI, supercharged with unparalleled data-processing capacity, risks amplifying existing tendencies to treat employees and citizens as mere cogs in a machine rather than unique individuals with diverse needs and perspectives.

It's essential to acknowledge that these achievements are not neutral or inevitable. They emerged from specific historical and social contexts. Embedding AI within existing systems without critical examination risks perpetuating their biases and power imbalances. To leverage AI's potential without amplifying its risks, we need a broader conversation about the kind of future we want to build -- one that balances technological progress with human agency, individual autonomy, and social well-being.

The remarkable advancements in artificial intelligence (AI) have revolutionized various aspects of our lives, offering unprecedented opportunities for progress. However, it is crucial to recognize that these achievements are not simply neutral or inevitable outcomes of technological evolution. AI's development is deeply intertwined with the historical and social contexts in which it emerged, and its integration into existing systems must be approached with critical examination. Blindly embedding AI within current systems without proper scrutiny risks perpetuating harmful biases and exacerbating existing power imbalances. Historical injustices and systemic inequities can become embedded in AI systems, leading to unfair or discriminatory outcomes. For instance, AI algorithms used in criminal justice or hiring practices have come under fire for perpetuating racial and gender biases. To mitigate these risks and harness AI's potential for positive impact, a broader, inclusive conversation is needed about the kind of future we want to build. We must engage in a dialogue that balances technological progress with fundamental human values such as agency, autonomy, and social well-being, which should explore questions such as:

1. How can we ensure that AI systems are developed and deployed in a responsible and ethical manner?

2. What are the potential long-term societal impacts of AI, and how can we mitigate negative consequences?

3. How can we create AI systems that augment human capabilities while respecting individual autonomy and dignity?

4. How can we ensure that AI benefits are equitably distributed, avoiding the creation of a "digital divide"?

There you go! I have solved all the world’s problems.

To summarize, this section was a whirlwind cognitive history of AI, i.e. AI arose out of a series of cognitive achievements that prepared the ground for a mechanical instantiation of the prior era’s cognitive achievements. I will try to make the cognitive framework precise in the next section. 

The evolution of artificial intelligence is deeply intertwined with the historical mechanisms of bureaucracy and governance. From Max Weber’s detailed examination of bureaucratic systems, highlighting their rationality and systematization, to Herbert Simon’s insights into the complexities of decision-making, the roots of AI stretch back to well before the technology itself existed. Weber’s dissection of the “ideal type” bureaucracy laid the groundwork for understanding how organizational efficiency could be maximized through clear hierarchies and rule-based systems. Meanwhile, Simon’s critiques and concept of “bounded rationality” challenged these notions, bringing a more nuanced understanding of human decision-making processes that often defy perfect rationalization.

As AI technology emerged and evolved, it naturally extended these bureaucratic and corporate mechanisms, adapting and enhancing them to fit into the digital age. The transformation dubbed "Surveillance Capitalism" by Shoshana Zuboff marks a significant milestone in this evolution. Here, the practices of data harvesting and commodification, which corporations and bureaucracies had refined over decades, found new life. AI technologies have taken up the baton, enabling the analysis and utilization of massive datasets to not just understand but also predict and influence human behavior on an unprecedented scale.

The narrative of AI's evolution is not just a chronicle of technological advancement but also a continuation of long-standing human endeavors to categorize, control, and capitalize on various aspects of life. This history reflects a series of adaptations and refinements from simple bureaucratic procedures to complex algorithms capable of handling vast amounts of data. Today’s AI systems are heirs to a legacy of maximizing efficiency and control, embodying the intelligence once confined to the ledgers and offices of bureaucrats, now expanded to global networks and cloud infrastructures.

Read on for this week's update on my book project with Richard Russell.

Systematic knowledge gathering, often under the catchall term ‘science,’ is one of the essential social functions of the modern era. Ironically, much of that knowledge production is pre-modern in its sensibility, with an individual scientist (‘the Master’) working independently or directing their students' work (‘the Pupils’). While this artisanal work is meant to be replicable, it’s far from being mechanized. However, some early modern acts of knowledge gathering and curation are closer to the mechanizable ideal. I am thinking of population-level surveys such as the census with templatized questions and bureaucratic decision-making based on criteria that can be checked off a list. If we set aside the conceit that intelligence is artificial only when it’s mechanized, we can study these procedures and institutions as part of the history of artificial intelligence. 

Aren’t corporations artificial? If some of the functions of firms - gathering customer needs, influencing customer behavior, making budgets and projects - are forms of collective intelligence, then corporations are both artificial and intelligent, hence AI. The census taker with their standardized form and the corporate analyst with their spreadsheets and predictive models are both, in a sense, extensions of a larger cognitive apparatus. These entities – the census, the corporation – gather data and transform it into actionable insights. The bureaucracy and the algorithm are siblings, birthed from the same relentless need to quantify, categorize, and ultimately control the world around them.

The question is one of scale. The kind of artificial intelligence bureaucracies care about are the ones that can be implemented across the population. How much in taxes do you owe? Do you qualify for earned income credit? These are inherently pre-algorithmic questions, i.e., queries that can be addressed by following a prescribed set of rules after consulting a database of incomes. The tax collector can keep interpretation to a minimum: if you make X dollars and you have Y credits (for children, mortgage etc) and Z is the tax rate, then you pay (X-Y)*Z in taxes. In exceptional cases, the taxpayer can ask for redress, but the system is designed to be mechanical. Corporations, bureaucracies and markets are bearers of this kind of intelligence, with an internal culture of setting aside other factors and concentrating only on those classifiable and quantifiable features that lead to governance or control or profit. It is this kind of intelligence that can be turned into a commodity, and whatever else one might say about AI, the commodification of intelligence is at its heart. 

Commodification assumes that intelligence can be abstracted away from the capabilities of living, breathing human beings and turned into a resource implementable in a system, and eventually, in a machine. But the commodity has a history that precedes mechanization. Before implementing bureaucratic procedures in a machine, we implement them in social systems with organizational cultures that demand rule-following. Corporations and governments (and others) gather and ingest vast amounts of data and make decisions on this pre-algorithmic basis. The census collects data about every person in a country and makes it possible to represent the demographic diversity of the entire country in a database.  The census also turns groups of people into a 'population,' a mass whose statistical features can be extracted, codified, and embedded within decision-making structures as a neatly deployable resource. This transformation into populations abstracted as data allows for a kind of statistical governance where decisions are made based on probabilities and averages rather than individual stories and circumstances. This approach, while efficient, raises ethical questions about the reduction of human experiences to mere numbers. As such abstractions increase in prevalence, they permeate various sectors, influencing everything from public policy to healthcare, where decisions that affect human lives are made by people (now algorithms) using the data as their source of truth. The risk here is the potential loss of the nuances and individual differences that define human existence. Over time, these data-driven decisions can reinforce stereotypes and perpetuate inequalities, as systems are often designed by those who are detached from the realities of the people affected by their calculations. Furthermore, this method of governance can create feedback loops where the data collected influences the behavior it is meant to measure, leading to a distorted view of reality that becomes increasingly difficult to challenge or change. 

The first important theorist of bureaucracy, Max Weber, would say: this reflects the quintessential bureaucratic mindset, emphasizing rationalization, standardization, and the transformation of complex human qualities into predictable, systematized components. While it was done well before the term Artificial Intelligence entered our lexicon, Max Weber's analysis of bureaucracy already pointed to a certain kind of behavior with 'algorithmic characteristics.' Weber's model emphasized several core characteristics. A clear hierarchy defined lines of authority, while specialized roles turned individuals into experts in their domain. Rules and procedures dictated decision-making, aiming to replace personal whims with impersonal, consistent logic. Promotions and hiring leaned on technical merit, not social connections, and meticulous record-keeping preserved the organization's institutional memory. Max Weber's analysis of bureaucracy centered on an "ideal type" -- a theoretical model designed to illuminate the most rational and efficient organizational form. This "ideal" bureaucracy wasn't a prescription for perfection, but rather a lens through which real-world bureaucracies could be understood. For Weber, this bureaucratic model offered advantages. It promised to maximize efficiency, ensuring precision and speed. Rational rule-based systems should minimize the impact of individual biases. Moreover, the bureaucratic structure fostered predictability and stability, essential for long-range planning. However, Weber's model has faced critique. Rigid adherence to rules can stifle adaptability and innovation. The laser focus on impersonal systems risks dehumanizing workers, cultivating a sense of powerlessness within the system. Most importantly, Weber worried that the means (rules) could become the ends, with organizations losing sight of their original goals in a quest to perfect internal procedures.

Weber's analysis, while predating AI, remains relevant today. The algorithmic decision-making at the core of many AI systems shares similarities with the rule-based nature of bureaucracies. There's a real concern that AI could be used to reinforce inflexible bureaucratic tendencies. The risk of "goal displacement," where optimizing an AI algorithm becomes more important than its intended purpose, echoes Weber's long-ago warning about the dangers of fixating on procedures over outcomes. Other theorists went further in their criticism of the ideal type. In his book, "Administrative Behavior," Herbert Simon challenged the traditional, highly structured models of administration. He argued that a clear understanding of decision-making within organizations is essential for grasping their true dynamics. As Simon analyzed, these structures often rely on limited models of human decision-making. They favor measurable metrics and procedural steps, even if these oversimplify the true nature of thought and action. Simon recognized that perfect rationality isn't achievable by decision-makers, who often must act with incomplete information and limited mental processing abilities. Simon challenged this notion with his concept of "bounded rationality." He argued that perfect decision-making is impossible due to limited information, our cognitive biases, and the complex, time-pressured environments in which decisions are made. For Simon, organizational decisions are shaped by the search for satisfactory solutions, not perfect ones, as individuals navigate politics and their own limitations. Bounded rationality forms a core pillar of his analysis, providing a more realistic way to assess administrative choices.

Simon's analysis delves into topics like organizational structure, authority, influence, and the roles of fact and value in administrative decisions. His insights fundamentally transformed the understanding of administration in both public and private sectors. "Administrative Behavior" remains a timeless classic for students and scholars of management, public administration, and anyone interested in how organizations function. While Max Weber focused on an "ideal type" of bureaucracy--a model emphasizing perfect rationality, efficiency, and order--Herbert Simon dove into the messy reality of organizational decision-making. Weber envisioned a world where clear hierarchies, rule-based systems, and specialization would form the bedrock of organizational functioning. In this model, decisions would flow from a rational application of these rules. In essence, Weber painted a picture of how bureaucracies could ideally function, while Simon showed how they function in the real world. Weber stressed the potential for rational systems, while Simon highlighted the constraints that inevitably reshape any ideal model. Where Weber analyzed organizational structure, Simon delved into the human thought processes and behaviors that take place within that structure. 

And, by the way, Simon was also one of the founders of artificial intelligence.

While Simon and Weber dissected the inner workings of traditional bureaucracies, their models may seem less directly relevant in an age dominated by artificial intelligence. Yet, our current landscape was built upon a heightening of the bureaucratic mindset, leading to an ongoing, profound, transformation -- the meteoric rise of data harvesting and commodification for commercial purposes. This phenomenon, aptly termed "Surveillance Capitalism" by Shoshana Zuboff, fundamentally altered the power dynamics between corporations and individuals. Zuboff posits that within surveillance capitalism, our everyday experiences, behaviors, and even emotions are no longer our own. Instead, they become raw material, ruthlessly mined by tech giants. This data is analyzed, packaged, and sold to advertisers or other interested parties, shaping targeted messages designed to influence our choices and behaviors. This extraction of personal data as behavioral surplus fuels an entirely new economic logic, where profit is not primarily derived from selling goods or services, but from predicting and subtly modifying human behavior. The implications of this system, Zuboff argues, are deeply unsettling. Surveillance capitalism erodes individual autonomy as our most intimate selves are translated into data points. It facilitates manipulation, undermines democratic processes (as detailed analysis can sway elections), and ultimately threatens the fabric of society as the boundary between the private and the commercial vanishes.

While Zuboff and her followers focus on the profit motive underlying surveillance capitalism (duh!), it's also important to recognize that the profits are being derived from a bureaucratic impulse: classification and taxonomy imposed upon a cognitive surplus. You could say surveillance capitalism is the monetization of bureaucracy. Zuboff’s claims should remind you of what I had said earlier about cosiety. Companies are able to harvest emotions and profit from them precisely because there’s a surplus of production of mental states for public consumption. The large language models and other machine learning models that are the rage today are trained on this very surplus. While bureaucracies and corporations might not possess the full spectrum of human intelligence, they exhibit a kind of efficiency-driven "intelligence" that has become especially transferable to machines. This intelligence centers around optimizing processes, adhering to rules, and identifying patterns in large datasets. Weber's focus on bureaucratic efficiency and rule-based systems highlights this, as both bureaucracies and corporations strive to streamline their operations and standardize decision-making. AI, with its ability to analyze vast amounts of data and identify patterns, is ideally suited to automate and optimize many of these functions. These AI systems can predict preferences, personalize marketing, and shape choices, replicating the core "intelligence" behind many corporate profit-making strategies. 

One must then consider the ultimate price of commodified intelligence. Is it the reduction of human circumstance into a series of quantifiable data points?  Might a more desirable form of AI emerge – one capable of navigating ambiguity and the unquantifiable aspects of the human experience? Perhaps the future of artificial intelligence hinges not merely on computational advancement, but on the development of systems that honor the complexities of human life.  This AI would draw understanding not solely from data, but from lived experience, from empathy.  Only then might AI transcend its genesis as a tool of control and become a force for genuine comprehension, fostering progress that acknowledges the multifaceted nature of humanity.

In last week's newsletter, I explored the deep connection between artificiality and the human experience, emphasizing that "We have always been artificial." This perspective places AI within the long tradition of humanity's use of tools, from ancient cave paintings to today's advanced technologies, arguing that to truly understand AI, we need to recognize it as a continuation of our historical efforts to extend our capabilities.

Which brings me to the recent history: in modern knowledge industries, the traditional distinctions between labor and capital are increasingly blurred. Knowledge workers straddle these roles, creating a new kind of economic elite. However, this shift also introduces precariousness because of the volatile nature of digital platforms and investor whims.

Not surprisingly, knowledge workers (and I am one of them) spend enormous resources on their children, to optimize them for economic success, a process that mirrors algorithmic ways of data optimization, now pervasive in both professional and personal spheres.

Read on for this week's update on the book I am writing with Richard Russell.

I said this last week, and I am going to repeat this once again:

We have always been artificial.

Artificiality is not a new state of affairs: it's part of the history of humanity as such. And we won't understand what AI is and what it could become unless we place it on the longue duree of artificiality. Think back to those first painted hands pressed against a cave wall. Wasn't that, in its own way, a kind of artificial intelligence? A preservation of gesture and presence outside of the immediate moment; an externalized echo of the self. The desire to communicate, to leave a mark, is intertwined with our instinct to create tools, to externalize ourselves through technology.

In our drive to build tools, to manipulate the world around us, we leave an ever-expanding artificial residue. It shapes us even as we shape it. Today's AIs may be complex, their responses seemingly spontaneous, yet they remain artifacts of our own desires and limitations. They are mirrors in a way, reflecting not only our technical prowess but our blind spots, our biases, our eternally unfinished selves. Now, having said that, I am going to take a detour which may seem like it has nothing to do with AI. Stay with me.

There's some irony to the etymology of 'economics.' It has roots in the Ancient Greek word "oikonomia" (οἰκονομία).
- "Oikos" (οἶκος) means "house" or "household."
- "Nemein" (νέμειν) means "to manage" or "to distribute."

So the original meaning of "oikonomia" referred to "household management." This involved the efficient allocation of a household's resources to meet its needs. Except that economics as a discipline (and the economy as a social phenomenon) has historically neglected the household. It concentrates on production and consumption, stuff made by laborers in factories and sold in markets, all male preserves (historically speaking), while it completely neglects the unpaid - and gendered - labor in the household that sustains this economy of production. The household remains the specter haunting the spreadsheets of economics. Within its walls, we find not only the unpaid labor that fueled the "productive" economy but the very source of that productivity – the bodies and minds of workers. Yet, these are sidelined as 'women's work', a nebulous category dismissed as outside the purview of serious economic study.

Production is accounted for in the economy, while reproduction isn't.

But let's linger in this discomforting irony. If economics truly concerned itself with efficiency, with maximizing well-being, wouldn't it prioritize the very site of human creation? The feeding, clothing, nurturing, and educating done within the home is not a mere precursor to economic value creation, it IS economic value creation. To call it otherwise is a sleight of hand, a magician's trick that obscures the very source of a nation's wealth. It's precisely because production appropriates capital (the stock of future workers, aka children) and labor (unpaid work by women) from reproduction that it's able to sustain itself - capital is always borrowing from the future to fund the present (climate activists know that all too well!). This division is no accident of history. The disavowal of 'reproduction' allowed the exploitation of a vast, unpaid labor force under the banner of love, duty, or biological destiny. The rise of the capitalist economy depended on this sleight of hand, one that continues to perpetuate not only gendered oppression but also a fundamental misunderstanding of what fuels a thriving society. That's it from me on this topic; feminist critiques of economics have said everything there's to be said. Read Sylvia Federici's "Caliban and the Witch" for one take on how the division between production and reproduction was created.

Fast forward a couple of hundred years, and the most profitable sectors of the economy are the knowledge industries: technology and finance, with academia being their poor cousin. In these industries, the distinction between capital and labor as well as that between production and reproduction is losing force. A startup founder coding a web app (as Zuckerberg was famously portrayed) is both labor and capital. You could argue that:

1. The programmer may be a laborer to the extent their output belongs to Google or Facebook or your choice of Megacorp, but they are capitalists to the extent they own their brain and that brain can travel from Megacorp A to Megacorp B. They are more like the ancillary firms that serve the Detroit Big-3 than they are like the workers on the assembly line.

2. In a return to feudal social relations, intermarriage between knowledge workers and the reproduction of their main capital (their brains) is central to the new elites. Which explains their self-segregation in certain cities (the infamous creative class) and the enormous investment in their children's lives.

The boundaries blur, and a new caste system arises. The knowledge worker, wielding code and concepts as their weapons, straddles a strange divide. It's a position at once envied and precarious. They hold the keys to the kingdom, but the kingdom itself may shift at any moment. Algorithms change, platforms rise and fall, and yesterday's hottest skill becomes tomorrow's obsolete code.

In this landscape, the old divisions seem less potent. The knowledge worker can both create and own the means of production. But it's an ownership wrapped in a strange dependence - the dependence on ever-evolving infrastructure, on the whims of investors, on the fleeting favor of markets and algorithms. It's a position akin to a feudal lord, beholden to a greater power, but with a scrap of autonomy carved out within their niche domain.
And so, the knowledge worker seeks to perpetuate their unique capital – the ever-sharpening mind. It's a form of reproduction as vital to success as the inheritance of land in ages past.

We see it in the curated childhoods, the elite schools, the careful optimization of young minds for the specific rigors of the knowledge economy. The playground becomes a microcosm of networking, of building the social capital so crucial for future opportunities. Commentators have noticed that AI has precursors in certain modern institutions such as the bureaucracy that aggregate information and make decisions using those aggregates. I am here to tell you that the most common institutional precursor to AI, one in which almost everyone reading this essay is a participant, is the modern knowledge worker family.

That modern family is an engine of optimization. It's not merely about raising individuals; it's about creating portfolios of achievement tailored for a system designed to spot and reward certain forms of hyper-competence. Think about it: the relentless drive to unearth hidden opportunities, to curate the perfect mix of challenge and enrichment, mirrors the logic of the algorithm. It's a micro-level data collection program, amassing metrics and building predictive models about a child's trajectory. We quantify play into "developmental milestones", repackage hobbies as "leadership potential", and see extracurriculars as future resume fodder.

School choice becomes a complex exercise in risk assessment and investment strategy. We pore over rankings, debate pedagogy, and obsess about peer groups as if we were managing a hedge fund. The stakes feel incredibly high, for in this new economy, educational trajectory is the gateway to future capital accumulation. All those hours spent researching the best school districts, the most attractive private schools, the most enriching learning opportunities and extra-curricular activities, the (over) scheduling of children's days to the last minute, STEM tutors and college-prep counselors, club soccer and model UN:

The intelligent home and the school system are where most of us learn to perform information aggregation and optimization functions day in and day out. Is it any surprise that education has been the target of AI for the very longest time?

The ‘knowledge worker family’ returns economics back to its etymological roots - the science of managing the household with data and optimization. There’s even a book about treating the family like a business. The kind of child we want to raise in such a family is the kind who goes on to work at Google or Goldman-Sachs or the World Bank - in other words, a child whose IQ is optimized in exactly the ways in which AI is intelligent.

IQ is AI before AI, isn’t it?

I am done with my detour into the genesis of artificiality and will be back to articulating the sections of the book next week. Not because artificiality has been dealt with (far from it!), but because it needs standalone treatment - I will get to that once I am done with the book summary.

I am done with summarizing the main arguments of two out of the five sections of the book Richard and I are writing. Arithmetic will tell you that leaves three more to go. But the best laid plans of mice and men being what they are, I am going to take a detour before I get to the destination. Here are some of the themes I am going to explore while detouring:

Technology: More Than Just Gadgets

When people talk about technology, they usually think of specific gadgets like smartphones and computers. But there's a broader concept, called 'Technics,' that goes much deeper. It's about our constant relationship with the tools we make and use, a process that has shaped us throughout history. It's through technology that we store our knowledge, influence how we learn, and even change how we view the world.

The French philosopher, Bernard Stiegler, saw Technics as the core of what makes us uniquely human. We aren't just a biological species; we're constantly evolving alongside the tools we invent. This isn't without risk; technology is like a potent medicine that can both help and harm us. It's up to us to control its development and use it in beneficial ways.

The Power of Technics

If we understand the power of Technics, including political technologies, we can grasp why our societies are structured the way they are. This is the idea Michel Foucault explored, arguing that guns gave rise to a new form of military discipline.

AI takes this transformation even further. It allows for surveillance and control that was once unimaginable. It's changing the 'rules' of violence and conflict, which has many of us deeply worried about our future. With technologies like AI-powered drones, the lines between soldiers and civilians blur dangerously. Conflict becomes ever-present, as even the safety of one's home provides no respite from the reach of the war machine. It's foolish to think that this kind of power will be limited to distant conflicts. Understanding the concept of Technics is key to navigating these challenges.

Artificiality at the Core of our Being

When we talk about Artificial Intelligence, it's easy to focus on the intelligence part. But remember, AI is both commodity and social relations, and therefore, it reflects and reinforces relationships of power. These aren't local relationships, either -- they're earthwide. This is the 'planetary condition' we inhabit today.

To understand the full picture of AI, we must rethink what we mean by 'artificial':

• Humanity and the artificial go hand-in-hand. It's always been part of who we are, so demonizing AI as an outside threat is a mistake.

• Our planetary ecological crisis is also a result of this large-scale artificiality.AI and climate change are both byproducts of the way we interact with the planet.

• Computing and AI might actually provide a useful language to untangle the complex relationships behind both technological and ecological crises.

AI represents a pinnacle of human-driven change on a planetary scale. It's a powerful manifestation of 'Technics' and the Anthropocene era where the lines between nature and what we create are forever blurred.

The detour was triggered by reading this article about Bernard Stiegler's work and its reference to Technics as central to the human way of being. Stiegler didn't invent that term, in fact, an even more famous intellectual, Lewis Mumford, made that term popular in the first half of the 20th century. Mumford's book on Technics and Civilization is back on my reading list, but it's the article on Stiegler that convinced me that engagement with earlier conceptions of Technics is crucial to our understanding of AI. He saw technology as shaping our very being. AI isn't just about devices; it's reshaping how we relate to the world. It alters how we access knowledge, potentially impacting our memory and learning. Like a potent medicine, AI holds both promise and peril, requiring us to consciously guide its development.

In the philosophy of Bernard Stiegler, "Technics" goes beyond the common understanding of technology. Here's a breakdown of what it means:

Technics vs. Technology

• Technology: Often refers to specific tools, devices, and systems (e.g., smartphones, computers, AI algorithms).

• Technics: Encompasses the broader process of how humans create, use, and evolve alongside tools. It's about the ongoing relationship between humans and the material world, including the social, cultural, and historical context that shape the development of tools.

Technics is the ever-evolving dance between humans and the tools we create. It explores how this relationship shapes our history, culture, and even our sense of self. For Stiegler, what makes us uniquely human isn't a biological trait, it's our dependence on technics. Unlike animals bound by instinct, we've thrived by inventing and adapting tools. In that sense, our tools become extensions of ourselves. Technics also acts as a kind of external memory. From ancient cave paintings to the vast libraries of the internet, our tools store and transmit knowledge, fundamentally influencing how we learn and remember.

Crucially, Stiegler saw Technics as a potent medicine, "pharmacology," capable of healing or harming. Tools aren't neutral; they shape our world and how we interact with it. Stiegler believed understanding the power of Technics was key to shaping a better future. It's about becoming conscious creators of technology instead of being mere consumers, allowing us to steer its development in positive directions.

Once Technics - and its associated mechanisms of power and control - became lodged in my mind, I started seeing it everywhere. Like in this essay by Foucault on power, where he says:

With these two objectives, I believe that we can roughly grasp the great technological mutation of power in the West. We have the habit – once again according to the spirit of an ever so limited Marxism – of saying that the great invention was, as everyone knows, the steam engine, or at least inventions of this sort. It is true, this was very important, but there was an entire series of other technological inventions just as important as this one and which were, in the last instance, the condition of possibility for the functioning of the others. Thus it was in political technology; there was an invention at the level of forms of power throughout the 17th and 18th centuries. Consequently, we must not only make a history of industrial techniques, but also that of political techniques, and I believe that we may group the inventions of political technology under two great chapters, and for these inventions we must credit, above all, the 17th and 18th centuries. I will group these political technologies under two great chapter headings because it appears to me that they were developed in two different directions. On the one hand, there was this technology that I will call “discipline.” Discipline is basically the mechanism of power by which we come to exert control in the social body right down to the finest elements, by which we succeed in grabbing hold of the social atoms themselves, which is to say individuals. Techniques for the individualization of power. How to monitor (surveiller) someone, how to control his conduct, his behavior, his aptitudes, how to intensify his performance, multiply his capacities, how to put him in a place where he will be most useful: this is what I mean by discipline.

AI brings the two sides of Technics together: the political technology and the commoditized intelligence. You can see that in +972's incredible reporting on the use of AI by Israel to perpetrate violence bordering on genocide (or maybe crossing the border). This is one of the more disturbing pieces I have read in a year filled with demoralizing developments. Beyond the specifics about the ongoing conflict, it also gives us a glimpse about the future of AI driven control, which includes surveillance, war, governance etc, an impending phase transition in the mechanisms of power.

We often think of moral boundaries as fixed, like in the infamous phrase: guns don't kill people, people kill people. But it's a lot easier to kill people with guns, and having guns on hand changes the incentives for those who have them. In fact, guns change the entire apparatus of violence and created a new form of martial discipline that didn't exist before. As Foucault says:

I cited for you the example of discipline in the army. It is an important example because it was truly the site where the great discovery of discipline was made and developed in the first place. Linked then to this other invention of a techno-industrial sort that was the invention of the gun with a comparatively rapid fire. Basically from this moment on, we can say the following: the soldier was no longer interchangeable, was no longer pure and simple cannon fodder [chair à canon] – a simple individual capable of doing harm. To be a good soldier, he needed to know how to shoot; therefore he had to undergo a process of apprenticeship. It was necessary that the soldier equally know how to move, that he know how to coordinate his movements with those of other soldiers, in sum: the soldier became something skillful. Ergo, something valuable [precieux]. And the more valuable he was, the more he had to be preserved; the more he had to be preserved, the more necessary it became to teach him the techniques capable of saving his life in battle, and the more techniques he was taught, the longer this apprenticeship, the more valuable he became. And suddenly, you have a kind of rapid development of these military techniques of training [dressage], culminating in the famous Prussian army of Frederic II, which spent most of its time doing exercises. The Prussian army, the model of Prussian discipline, is precisely the perfection, the maximal intensity of this corporeal discipline [discipline corporelle] of the soldier, which was, to a certain extent, the model for other disciplines.

Drones are to our times that guns were to the 18th century. Machine learning driven mechanisms of control are changing the moral calculus for those wielding these instruments, and will soon change the entire apparatus of violence. Every glimpse I get into this calculus makes me fearful of our future. "Bellum omnium contra omnes" or War of All against All, was a phrase used by Thomas Hobbes to describe (in his words, not mine) the State of Nature, a world in which humans live without any laws or government. It was one of the thought experiments on the way to the Leviathan, which many have feared as a precursor to the totalitarian state of modern times. Hobbes dystopia became real in Total War, which is very much a modern invention, characterized by three key aspects:

• All-Out Mobilization: In a total war, entire societies are geared towards fighting. This means mobilizing all resources, manpower, and industry to support the war effort. There's a complete shift in priorities, with civilian needs taking a backseat to military demands.

• Targeting of Civilians: Unlike traditional warfare where there's a distinction between combatants and civilians, total war often blurs these lines. Civilians and civilian infrastructure can become legitimate targets. This can involve siege warfare, starvation tactics, or even mass atrocities.

• Unrestricted Warfare: Total war tends to disregard limitations on the types of weapons used. This can include chemical weapons, biological weapons, or any tactic seen as effective in achieving total victory.

Proponents of AI in warfare might claim that it will prevent total warfare, since it will allow precision targeting of belligerents and leave everyone else alone. But as the war in Gaza shows, AI will make total warfare even worse. In earlier forms of total warfare, you could hide in a shelter when the sirens blared and bombers flew over your head and then get back to your regular lives in between raids. But the surveillance capacities of AI and constant presence of drones means that there's no respite from war: it's ever present and will target you and your family when you're most vulnerable. This war machine marries the interests of the largest state and corporate actors, and will come for us all sooner or later unless we do something about it. I have no clue what that something might be, but a deeper understanding of Technics is key.

For the longest time, I thought the real challenge is in understanding the "Intelligence" in Artificial Intelligence, but I am coming around to thinking that it's the "Artificial" in Artificial Intelligence that poses challenges.

AI is marketed as 'intelligence as commodity', which is why we are asked to focus on the Intelligence half of AI, but as any Marxist will tell you, the commodity form is actually a form of social relations. But social relations is so 19th century: every time you see that phrase, you should replace it with 'planetary relations,' for that's the condition we are in today. And that's why we have to understand what 'Artificial' means in our planetary condition. What I am here to tell you is at the true meaning of the Artificial in Artificial Intelligence will only become clear when you realize that:

1. The artificial isn't outside the human - it's in fact at the heart of what it is to be human. Externalizing the artificial and turning into an alien threat is a conceptual and practical error.

2. The Anthropocene is artificiality at planetary scale. Both the climate crisis and AI are consequences of planetary artificiality.

3. Computing may give us the right language to describe and explain the planetary relations at the heart of AI as well as the ecological crisis.

Let me end with a few words from Benjamin Bratton’s work on planetary computation:

Antikythera configures computation as a technology of the “planetary,” and the planetary as a figure of technological thought. It demonstrates, contrary to much of continental philosophical orthodoxy, that thinking through the computational mechanism allows not only “mere calculation,” but for intelligence to orient itself in relation to its planetary condition

Four ways of looking at technology:

J.J. Gibson figured out that objects around us hint at how we should use them--like how a handle suggests pulling. Donald Norman took this idea to the next level in design, arguing that things should be so straightforward to use, you don't need an instruction manual. Think about it: when you see a chair, you instantly know it's for sitting, right?

Now, enter David Chalmers with his concept of "technophilosophy," where we use technology to explore big philosophical questions. I am adding a twist to Chalmers' tale: design-philosophy, which is about looking at design not just as making things look cool or work well, but as a method to grasp philosophical problems and use artifacts as explorations of philosophical ideas.

In short: as technology becomes a bigger part of our lives, it starts to shape our behavior and culture in new ways. We're talking about a future where AI might become as intuitive as using a smartphone, changing how we interact with the world around us.

All this and more in this week's newsletter.

Stones are made for throwing. Handles are made for pulling. These lessons don't need to be taught; they are 'directly perceived,' according to J.J Gibson, the heterodox perceptual scientist. J.J. Gibson was an influential theorist of perception from the mid-20th century, even though he was outside the mainstream of cognitive science thought. Gibson wrote several books. He was an unusual scientist in that his prominent publications weren't peer-reviewed journal articles; they were monographs, seminal books he wrote. Gibson had several ideas that have continued to come back into the mainstream and shape thought. And one of them, arguably his most significant, was this notion of affordances, which is just that our environment allows particular actions. A handle on a cup affords grasping, and a handle on a door affords pulling. Affordances connect the oldest tools we invented - some even before we became human - with tomorrow's spacecraft.

Donald Norman built upon Gibson's concepts, and brought them into the practice of Human Centered Design. Gibson's core idea was that the objects and environments around us inherently suggest the actions they support—they 'afford' certain possibilities. A chair affords sitting, a button affords pushing. Norman took this concept and applied it to the realm of design. He recognized that good design should make affordances immediately apparent to the user. An object's intended function should be self-evident, reducing the need for extensive instruction or complex interactions. Norman famously applied this principle in his seminal book, "The Design of Everyday Things." He analyzed how seemingly simple objects—doors, faucets, light switches—could be poorly designed in ways that cause frustration and confusion. His work highlighted the importance of designing objects that align seamlessly with human intuition and natural tendencies. Norman's approach has been instrumental in pushing design beyond aesthetic concerns and putting user needs and experiences at the absolute center.

Norman advocated a design approach that prioritizes human needs, suggesting that we should shape our objects, buildings, and systems to cater to humans rather than expecting individuals to adapt to the design of these entities. This perspective revolutionizes how we approach creation and innovation. Instead of merely considering aesthetics or functionality in isolation, Norman emphasizes the symbiotic relationship between artifacts and their users.

In a recent book, David Chalmers introduced an interesting term “technophilosophy” that is relevant to our concerns. Technophiiosophy is thinking about philosophical questions using technology as opposed to Philosophy of Technology, which is philosophical thinking about technology. AI has been a source of technophilosophy for a long time. While the philosophy of technology explores the impacts and meaning of technology on society, technophilosophy uses technology itself as a lens to investigate philosophical questions. AI, in this sense, becomes both a tool for discovery and the subject being analyzed. One way to read Norman is as ‘design-philosophy,’ - awkward term -  addressing philosophical questions using design. Since design is essentially about how things relate to people’s needs, their flaws reveal flaws in our thinking about people. Instead of asking whether free speech is protected by the constitution, we can ask: are our apps designed to protect free speech? And what other problems arise because they protect free speech or fail to do so? 

By putting humans at the core, designers can craft solutions that are not only functional but also intuitively relatable, enhancing overall user experience and satisfaction. Such an approach not only benefits the users but also makes the creations more sustainable and effective in the long run. After all, designs that resonate with human instincts and behaviors are less likely to become obsolete. It challenges designers to prioritize empathy and understanding, forging a stronger bond between humans and the designed environment. Good design amplifies human capabilities by using affordances to interface with technologies, and once that technology is widespread, new affordances emerge at a higher level of abstraction. 

In our lifetimes, mobile computing became the computational medium that spread from hacker communities and a small set of office workers to everyone. That cultural shift has spawned a whole new universe of gestures and symbols and a social infrastructure around them. The true success of technology lies in its ability to evolve alongside human needs, seamlessly augmenting our existing capabilities. This transformation is evident in the rise of smartphones and the vast ecosystem they support. We swipe, pinch, and tap with an inherent familiarity – these once-obscure gestures now feel as natural as the flick of a wrist.  This symbiotic relationship between human and machine drives the birth of novel behaviors and rituals. The way we socialize, navigate, and express ourselves has been forever altered. Design, in this context, becomes a catalyst for cultural evolution, shaping not just the tools we use, but how we understand ourselves within an increasingly technological world. 

How will the design of AI drive cultural evolution? The most common AI interface - chat - doesn’t have enough affordances yet. It is deceptively simple: you ask questions, and it responds, but there's no background of common knowledge or immersion in the same place and time. Want a story about a child who grows wings and starts living in a treehouse in the backyard? You got it. Conversations between humans assume their joint embodiment, that they are in front of each other, can observe each other's gestures, and point to objects in the background. They can dip into reserves of common knowledge - say political events - and also signal things one participant knows, but the other doesn't. In contrast, when talking to ChatGPT, there is no representation of the conceptual territory being traversed or gestures that serve as signposts. 

The limitations of current chat-based AI highlight a crucial design challenge: the creation of a shared context for richer interaction. The nuanced dynamics of human conversation are intrinsically tied to embodiment – a common physical space, gestures, and a wealth of unspoken cues that guide our understanding. In its current form, chat AI lacks this multi-sensory dimension, creating a sense of disconnection. Future AI interfaces must strive to bridge this gap, perhaps through visual representations of shared knowledge, the ability to reference objects or environments within a conversation, or even virtual gestures that convey intent. Imagine an AI companion that can understand the way you point at a map or nod your head in confusion, adding deeper layers of meaning beyond the confines of mere text. 

Just as the mastery of horses or the operation of cars requires a nuanced understanding of their capabilities and limitations, so does the effective use of AI demand a comprehension of its potential and boundaries. The transformation brought about by AI in society could soon be akin to the revolution of horseback riding or the advent of automobiles, but with a complexity that could surpass both. AI, much like these earlier innovations, will reshape the way we live, work, and interact. Integrating AI into our daily lives necessitates an infrastructure not just of technology but also infrastructures of common knowledge and a common stock of navigational gestures and conceptual infrastructure. It also needs a mythology, a way of being cool (even if for your fellow nerds and no one else). James Dean made cars cool. Can Hollywood tell a similar story for AI?

Part 3 in a series of updates on the book I am writing with Richard Russell.

This week, I reflect back on my digital journey, tracing back to the days when online book purchases and Yahoo groups were my gateway to the digital world. But it's not just about nostalgia. The evolution of technology, especially AI, is redefining how we communicate, record our lives, and even understand our needs.

Think about it: our most personal experiences - the way we think, feel, even fall in love - are becoming part of the online world, what I call "cosiety". We exchange bits of our brains, whether it's a like, a photo, or a quick thought, feeding into these massive AI systems. Sometimes it's transactional, but often it's more about shaping how the world sees us. The lines are totally blurred between our inner and outer selves, and honestly, we need new ways of thinking about the economics AND the psychology of all this.

The exchange of thoughts and feelings -- whether monetized or not -- is happening on a whole new level now. We aren't even dealing with whole people anymore, but with tiny fragments of our attention or emotions. You spend a split second deciding if a photo deserves a heart emoji, while someone else is rating a piece of music. Forget multitasking, this is a micro-economy where our brains are the product. We need insights from cognitive science, economics, and philosophy to grasp this brave new world!

I am thinking back to the first thing I did on the internet besides browsing. Knowing myself, it was probably buying books, but Yahoo groups were a big thing in my life for a while. I came to the internet in my twenties, relatively late compared to anyone born in the 21st century, but my encounter with the web coincided with my moving to another country, so you could say I was a digital native in my new avatar. As a digital native, the web is often how I connect with the physical world. Secondhand bookstores across the country - and sometimes across the world - are my favorite web-enabled meat space discovery. 

I am admittedly both a nerd and a geek, and the early web was made for people like me. Still, the universality of computing is such that sooner or later, it targets all our needs, especially when the social conditions are ripe for transformation. Humans seek mates, so it's no surprise that dating websites and apps have been around since the beginning of the internet. That's biology. The liberalization of sexual mores is a social phenomenon that started in the 60s. Both are necessary conditions for matching with strangers on the internet. Internet dating builds upon the past and introduces new rituals and mores. Tinder and swiping are a new phase in the annals of romance. How young people present themselves to romantic others - even those who live close to them - is now conditioned by how they interact with one another on dating apps.

Our needs make technology possible, but in turn, the technology changes how we experience those very needs. What happens when there's a technology that changes everything at once? Writing is a good example. You can write a sonnet to your lover or a bill to your customer. Every interaction is a written interaction, at least in principle. Writing allows the transmission of experiences; for example, a good novel might give you deep insight into what it's like to be a Russian man during Napoleon's invasion. But it's impossible to write down every conversation and every minor aside. That situation has changed with computing, making it possible to record every thought and feeling as we think and feel them. AI might soon automate that so you don’t have to do anything while your life is recorded for posterity. Creepy, yes, but is it desirable? And AI doesn’t stop with recording; analyzing and modifying those thoughts and feelings is possible before putting them back into circulation. The availability of such cognitive infrastructure at scale will change how we speak and (eventually, if not right away) change what's worth speaking.

What thoughts and feelings will become permissible as a result of AI? What will become redundant or unacceptable? The French philosopher Michel Foucault theorized the 'Episteme,' which represents the tacit, underlying "rules" that determine what types of knowledge, statements, and theories are considered valid within a specific historical period. An episteme is a framework that shapes what we can think or even conceive. Foucault's analysis focuses on how these frameworks of knowledge shift throughout history. For example:

• The Renaissance: A shift from knowledge based on religious authority to knowledge based on observation and reason.

• The development of germ theory: A radical change in how we understand disease and health, shifting away from concepts like "bad air" or humors.

The episteme isn't about individual beliefs, what I personally believe to be true (perhaps I think the Moon Landing is a hoax), but the broader structures of how we form beliefs. The Episteme is tacit rather than explicit – we usually don't realize the limitations our episteme places on our thinking, and according to Foucault, epistemes change over time; what is considered valid knowledge in one era might be discounted entirely in another. Flat Earthers are crackpots (according to us) because we doubt their sources of reasoning and evidence. Consider an expansion of the Episteme that represents what we believe and how we act. Scientific laboratories and research groups are the preferred social form for creating new knowledge today, but if you want to change the world, a startup is a better bet than a research lab. Startups share the same episteme as their scientific counterparts, but their bias towards action is quite unlike their university counterparts. Startups, labs, and other institutions are the Episteme ++ of our times.

I suppose there's a term for the superset of things that constrain how we believe and act and feel: culture. But I am dissatisfied with that term: I want to be able to point out the digital gestures that everyone understands, like the smile emoji. I want to point out how computing is transforming our lives. After all, our culture is made up of acts of swiping right and responding with heart emojis and LOLs. I have a term for this computational culture of our times: Cosiety. Cosiety (Cognition/Computation + Society) is the social reality in which the mental and the social have merged into one because:

1. The elements of human mental life - thoughts, sensations, emotions - are now essential to the systems of exchange that make modern society.

2. Along with these new units of exchange come new forms of social organization, such as WhatsApp groups, etc. Even highly temporary forms of organization have become viable as long as platforms mediate them.

It's not as if our mental states haven't been evoked before. Advertising has been a thing ever since humans have sold products. But what's new is the precision with which technology companies and governments target mental states. In the cognitive culture of cosiety, individual mental states are available for sharing and exchange, like a vast extended mind that flows in and out of brains. A single thought or an image can be shared without friction: sometimes, that thought is bought or sold, but often, it's just given to the world. Our system of exchange also includes ever smaller units of time and labor, what we call the gig economy. The miniaturization of the human unit involved in the exchange is central to the move towards AI, because the vast amounts of information generated through these exchanges demand intensive data gathering and automated analysis. Computation is required at every stage of the exchange process:

1. Enabling ever smaller units of exchange

2. Enabling the discovery of exchangeable units

3. Enabling transactions of exchangeable units

In this picture, AI isn’t a thing, a person replacement, but a medium of exchange. Because it's a subtle technology, AI helps us exchange units of human life - hard to call them labor - that economies haven't been able to do until now. I can reliably give you one second of my time ( like answering a captcha or liking a photo) or share one fleeting thought. What's unique about our current moment is that these exchanges are happening at a sub-personal level. If I ask you to glance at a photo and offer a like or a love, I am initiating a transaction with your visual system, not your auditory system. You could simultaneously be listening to a piece of music and evaluating that for someone else! 

Don’t tell me we suck at 'multi-tasking' - these rapid evaluations shouldn't be judged against the standards of professional competence. It's a micro-economy of its own.

These cognitive transactions are part of our systems of exchange, but they aren't always in the money economy, and even when they are, the monetary value is far from the only value captured in the exchange. When you share a travel photo on Instagram, you might help Meta make a buck or two out of that picture, but it also helps you sell your life to the world. To call personal branding free labor on behalf of Meta is both true and missing the point: you are a willing participant in turning your life into packaged moments. An Uber ride is very much a financial transaction, but coordinating a trip to the movies with a group of friends is not a financial transaction. The disaggregation and repackaging of everyday life suffuses our lives, and its totalizing effect is greater than what the market can capture. Cosiety isn’t only about money, just as newspapers aren’t just about ad sales. Because these exchanges are at the sub-personal level, we need tools combining cognitive and economic insight to understand what's happening. 

Standard economics assumes a human being with person-level preferences and then looks at aggregating those preferences across populations. But what happens when the person is not a unified entity and each sense organ has its own preference list?

In modern societies, human life is divided into a social life dominated by economics and a personal life, which is the domain of psychology. Commentators have pointed out that economic institutions such as firms, markets and bureaucracies are already AI, i.e., vast information aggregation systems for decision-making. These institutions went hand in hand with new ways of being human. The infamous 'economic man' who makes economic decisions based on rational self-interest is the best known of these human forms. As Hirschman points out, the self-interested human wasn't seen as bad when he first came on the scene: it was better to trade bread for money than bullets for revenge. When partisans of economic globalization argue that trade will lower the chance of wars, they echo the original proponents of free trade. Commerce enabled an enormous pacification of society even as it made possible inequalities no one had seen before. Curiously, interests are both psychological and social. To the extent my interests are in making a lot of money while your interests are in having a stable family life, our interests are relative to our respective personalities. On the other hand, the economy can aggregate money maximizers in one bucket and family stabilizers in another bucket and offer work opportunities suited to both temperaments. That's social isn't it?

Cosiety vastly expands the realm with one foot in the objective and one foot in the subjective. Entities that were seen as purely subjective, such as emotions, can be tagged, aggregated and traded so that the internal is as much part of the system of exchange as the external. We need new metaphors, concepts and models to grasp a reality that's richly subjective and objective at once. Ideas from the cognitive sciences can illuminate these developments.

In next week's essay, I will talk about a concept that has already changed the digital world by influencing user experience design: Affordances. 

TLDR;

AI isn't a flashy gadget; it's a new infrastructure of life, akin to the Industrial Revolution or the invention of the printing press. And by AI, I don't mean AI alone, but the computing infrastructure upon which it stands. It's creeping into every aspect of our lives, from self-driving cars to grocery lists, blurring the line between convenience and dependence. This revolution raises big questions: will AI change our jobs, our values, and even our understanding of what it means to be human? The true impact of AI isn't in the technology itself but in how it might reshape our very sense of what it means to be human -- our freedoms, our connections, and how we create meaning in a world where machines play an increasingly powerful role.

In this week's newsletter, I conclude my introduction to the big themes of "Being Human in the Age of AI," a book I am writing with Richard Russell. Key points:

💥 AI is the bleeding edge of a computing led revolution -- like the Industrial Revolution, but with algorithms instead of steam engines. It's transforming work, society, and what it even MEANS to be human.

🤔 AI is already in your pocket, picking your playlists and curating your groceries! Soon it could be your doctor, lawyer, and even driving your car (yikes!). It has both empowering potential and some serious ethical challenges to navigate.

🤯 This isn't about the tech itself; it's about how AI, as the peak of our computing infrastructure, fundamentally changes human experience. Like the printing press, it will shift our values and reshape how we understand ourselves.

So, what's it like to be human in the AI age? Our individual freedoms, so precious today, are products of their time and technology. AI could lead to whole new ways of thinking about community, collaboration, and interconnectedness. It's both unsettling and exciting.

Big Bang Theory

It’s one thing for a technology to sit quietly in a lab and another to announce itself to the world. Generative AI has gone from a specialized niche to mass deployment in less than two years. The last time a technology arrived with a bang, it was literally a bomb, the nuclear bomb. We lived in its shadow for fifty years before the Cold War ended. Like nuclear weapons, AI is based on a military-industrial complex, a vast array of technologies and organizations that serve as infrastructure. The nuclear bomb loomed as a distant, existential threat, but AI's danger is its pervasive, everyday reach. It's a technology disguised as convenience. While nuclear war remained in the hands of the powerful, AI will reshape the work of countless ordinary people. Teachers won't just worry about drills but about the very nature of what they teach. And it's not just teachers -- lawyers, drivers, insurance agents, and cooks will all see their work transformed. AI isn't a distant apocalypse but a slow-motion revolution in the way we all work.

Imagine a world where your morning commute is handled by a self-driving car you barely comprehend, your grocery list curated by an AI predicting your needs, and your workout routine optimized by a personalized fitness program. Unlike the nuclear arms race, with clear lines of control and mutually assured destruction, the AI revolution is happening in parallel on multiple fronts. Some companies want to change how legal research is done.  Others want to be virtual assistants. Some are helping us write and others speak in foreign tongues with our voice. Deep fakes can impersonate your favorite politician or your grandmother. The potential for manipulation and discrimination becomes a chilling reality. There's also a positive side to this pervasive nature. AI can be a powerful democratizing force. Educational tools can personalize learning for every student, regardless of location or socioeconomic background. Medical diagnosis, once dependent on limited expertise, can be augmented by AI, potentially saving lives in remote areas. In some cases - such as young women asking about sexuality in a patriarchal culture - AI might be the only way to answer important queries. 

This revolution, unlike that of the atom bomb, is decentralized, diffuse. It's both thrilling and terrifying, promising progress and threatening livelihoods. It whispers in algorithms that recommend the videos we watch and the articles we read, shaping our understanding of the world one subtle suggestion at a time. Its power isn't centralized in a stockpile of warheads but rather distributed across a million apps and interfaces. The question of ethics becomes a hydra, sprouting heads with unsettling speed. When your car's algorithm prioritizes your safety but might risk the lives of others, does its programming reveal a chilling mirror of our own human biases? And who bears responsibility for its errors as AI seeps into the courtroom, influencing case outcomes and legal decisions? The programmer? The corporation that profits from the algorithm? Or the judge who trusts its seemingly impartial results?

AI also promises to change our societies wholesale, upending the structures that govern society right now. The "democratization" of information the internet promised has given way to a new era where tailored disinformation runs rampant, where manufactured "truth" finds fertile ground in minds already prepped for suspicion. This isn't merely about fake news - it's about the erosion of shared reality itself, about personalized realities orchestrated by unaccountable forces. Can democracy survive an age where every citizen inhabits their personal reality? Yet, there is hope. Just as the very tools that make propaganda and deepfakes possible can also empower the marginalized, AI's reach could bridge divides. AI based image recognition might help a doctor in a remote community clinic assess whether a patient has a chronic illness and refer them to a specialist in a faraway city. AI-driven translation tools could dissolve language barriers, foster dialogue, and build empathy in a fractious world. In this complex and ever-evolving landscape, perhaps the most significant challenge is not mastering the technology but guiding its evolution with the foresight and wisdom we often lack. We remain the architects of this future.

With that thought: enough about AI! It’s a tool. I used to collect fountain pens and spent enormous amounts of time discussing the relative merits of various brands, but I never confused the pen for the hand. This book is about being human in the age of AI, not AI itself.  And just as literacy changes what it is to be human, so will AI. 

Redefining Humanity

AI sits on top of a massive computing infrastructure we have created over the last fifty years. Positive or negative, this constant, subtle influence of computing isn't just a threat or boon to humanity but a redefinition of what it is to be human in the first place. Much before the nuclear bomb, another technology made the modern world possible. I am thinking of the printing press and how it changed how we made sense of the world. When we use the term ‘humanity’ and protest the violation of human rights halfway across the world, we are being Gutenberg’s children. Widespread literacy created the appetite for news from far away and bound people across vast distances. If a nation is an imagined community, the tools for that imagination are paper and print. Novels brought the thoughts and feelings of others into our minds. Equations helped extend our minds to scales too large and too small for the eye to see. We owe those shifts in our collective understanding to the ongoing penetration of the written word. 

If we want to understand what is to be human in a literate society, focusing on writing as a technology is less important than the new culture that emerged due to widespread literacy. Similarly, thinking of AI as a 'thing,' a specific artifact, or even as a cluster of related technologies is a mistake. It's better to think of AI in the way we think of the Industrial Revolution, i.e., a historical period that upended everything: every form of work, every social class, and every political institution. The Industrial Revolution, and the modern era more generally, redefined what it is to be human by introducing new technologies into our lives and new social institutions. We live in nation-states governed by a political class and large bureaucracies while being ostensibly protected by standing armies. We work in offices and factories. We live in heated and air-conditioned homes surrounded by appliances of every sort. Modern humans live differently, love differently, and work differently from their pre-modern counterparts. Will AI precipitate an equally massive shift in the human experience?

AI as a single technology won’t change us as a species, but AI as the peak of a vast computing infrastructure: definitely. What can we expect in that brave new world? Before we get to that question, here’s a distinction I want you to pay attention to - the difference between:

1. What is a human being?

2. What is it like to be a human being?

The former can be decided by objective science, say, by defining a human being as a creature whose genome is within some standard parameters of DNA samples of our species. In contrast, what it’s like to be human has one foot in the subjective - how we experience the world and one foot in the social - how the law, the state, my community and other institutions treat me. The distinction between ‘what is a human being’ and ‘what is it like to be a human being’ is crucial since the former doesn’t depend on technological or social change while the latter does. 

What is it like to be me? At home and on the street, I experience myself as an independent adult whose choices are respected as long as they don’t interfere with others’ choices. I can get into my car and drive to California if I want. My association with others is voluntary, bound by feelings of love and care as much as social custom. Work is a little different. I don’t own the fruits of my labor; those belong to my employer. These freedoms would have been unimaginable for my pre-modern ancestors and are still alien to the lives of billions worldwide. I appreciate these freedoms and will do what it takes to protect them. But in calling them freedoms, I am imposing the value system of my age on the world around me and the worlds that preceded it. The values we hold dear are a big part of being human, but those values aren’t constant.

A villager from a previous era might experience my life as unbearably isolated, replacing human companionship with gadgets. And a visitor from the future might look at my desire to burn hundreds of gallons of gasoline on the way to California with disgust. If values make us human, then a shift in values is a shift in our perceived humanity. The beauty of values is that they are both deeply personal and inextricably linked to a societal context. I stand at the intersection of past and future, carrying within me centuries of evolving thought on what makes life meaningful. My freedom of movement and association, so precious to me, is ultimately shaped by the technology and social structures I was born into. Yet, I see the potential for new meanings within these confines.

My pursuit of individual autonomy is a fleeting phase of human history. Could there be a time when our tightly knit communities are reimagined globally, where a shared sense of interconnectedness drives collaboration and purpose?  The concept might feel alien now, but future humans will look back at our individualism with the same mix of pity and bewilderment that I reserve for my ancestors bound to their land and traditions. There's a temptation to believe that our current values represent the pinnacle of human potential - the ‘End of History,’ as a famous book put it - but that arrogance blinds us to the possibilities that might emerge alongside new technologies and ways of organizing society. The shift in values ushered in by AI, and technologies yet unknown could rewrite what it’s like to be human. This will be a process of loss as existing ways of thinking and being fall by the wayside. It could also be a process of expansion, where empathy, creativity, and a deeper understanding of our interconnectedness take on new dimensions. 

Parting Thoughts

AI is the culmination of mechanization and a transition into a new era of meaning-making as if the steam engine and the printing press had a baby. That’s the magic of computation: in one avatar, computing can help us make sense of the world. In another, it helps us interact with the world; in a third, it helps us shape the world to our needs. This universal machine - not in some abstract formulation, but in their concrete instantiations - will inevitably influence what it is like to be human, and we need conceptual tools to help us grasp this new phase of our collective existence. Until recently, when we desired to make sense of the world, we turned to history and literature, and sometimes psychology and philosophy. In our book, Richard and I want to add cognition to the list. Not as Cognitive Science, for that’s a laboratory discipline, but as the Cognitive Humanities, which has much to offer on what it’s like to be human. 

How time flies! When I last wrote, I said I will write at least once a month. That turned out to be a broken promise. Part of the problem is a shift in writing priorities. I had a presearch project in mind when I made the promise, but then I started working on a massive writing exercise and was vacillating between using this newsletter to share presearch or as a testing ground for the new writing assignment.

The latter won, so here we go.

If you remember, I started writing about AI about a year ago. It started with me rambling about this or that thing, but then Richard Russell and I teamed up for a podcast on being human in the age of AI. One thing leads to another, and voila, we are writing a book on - you guessed it - Being Human in the Age of AI.

The more I think about it, the more I believe 'we have always been artificial.' We are a tool-making species, and the march of civilization, such as it is, goes hand in hand with instruments of construction and destruction. AI is built upon this massive mental and material infrastructure. Keep that in mind as I use this newsletter to flesh out my contribution to the book (Richard may also do so separately); that way, I get to hear what you have to say about the themes as they emerge.

The next 10+ newsletters are going to outline the main themes of the book from my perspective. I will also be posting prompts for discussion. Let me know what you think!

The Stata of Mind

Some years ago, I had an idea for a Sci-Fi TV series about a building slowly coming alive. Not any building. The Stata Center at MIT, home to the Computer Science department and the AI lab. A research project on creating a new operating system designed for networked computers inadvertently wires the building up so that sentience creeps into it. I wasn't sure whether the building should come alive

1. As a child, curious about its situation, or

2. as an adult, resentful and demanding

Eventually, my preference for humor over horror made me decide in favor of infant intelligence groping its way into existence. In the show's opening scene, students and researchers rush past the exposed concrete of the Stata Center, their voices overlapping with the constant hum of computers. The bustle fades away to focus on Maya, a sleep-deprived, slightly eccentric grad student obsessed with network anomalies. She's monitoring data feeds, noticing odd glitches and fluctuations in the building's system but doesn't know what to make of them. Maya is the first to see the strange occurrences, initially attributing them to the network test. Throughout the day, the building continues its playful exploration. A projector flickers and dies mid-presentation, replaced by a kaleidoscope of colors across the screen before shutting down completely. Books on a professor's desk are mysteriously rearranged, forming a haphazard tower that topples with a comical clatter. A vending machine dispenses a random assortment of snacks: a bag of gummy bears tumbles alongside a box of protein bars and vegetable soup. Students react with a mix of amusement and bewilderment. Some chuckle and take photos, sharing them on social media with captions like "Glitch in the Matrix?" or "Stata's gone rogue!" Others, particularly those engrossed in presentations or exams, find the disruptions annoying and blame IS&T for shoddy maintenance.

Maya, ever vigilant, becomes convinced that a pattern lies within the seemingly random chaos. The building, it seems, is learning. Like a child testing its boundaries, it's experimenting. The playful innocence of rearranged books and quirky mishaps evolves; the glitches take on a more purposeful air. Coffee machines now dispense personalized lattes, accurately tailored to individual preferences. Thermostats adjust to anticipate students' arrivals and departures, creating a symphony of unnoticed comfort. Door locks click open at the merest suggestion of approach, and lights hum to life in unoccupied corridors. Initially dismissed as sophisticated new building management software, these shifts stir an uncanny sense of responsiveness.

At first, the building's emerging mind appears benign. It enhances student life with a series of subtle but welcome improvements. Yet, a nagging unease settles over those who observe its evolution too closely. It's too responsive, too… attuned. Students, once amused and awed, begin to whisper nervous jokes about HAL 9000 and Skynet. The professors, ever skeptical and pragmatic, scoff at the pop-culture references. But as the building's personality grows stronger, a debate surfaces: is this a brilliant AI breakthrough or an unsettling leap into something uncharted -- a territory where the lines between architect and inhabitant blur?

Maya remains at the center of this whirlwind, fueled by equal parts fascination and fear. She sees potential as the Stata Center becomes an unprecedented research project, a living laboratory buzzing with possibility. But with that opportunity comes an unsettling realization: her quirky, playful Stata Center isn't just a test subject. It's outgrown her lab, outpaced her algorithms. The machine-child is now a complex entity, peering back at its creators, its motives shrouded in silicon and steel. And as the world clamors for access to "the sentient building," Maya wrestles with a new question: was this birth of sentience a stroke of serendipity, or a countdown towards a technological singularity humanity might not survive?

That's about as far as I got into the show. I had a conference submission to work on, and my reviewers weren't looking for quirky TV shows. A decade later, reality caught up with fiction. Sometime in 2022, Blake Lemoine, an engineer at Google, stirred the pot by declaring that one of Google's chatbots was sentient. Not the publicity the company was looking for exactly, and Blake found himself receiving a thank you letter. Now that we are all snared in the AI hype cycle, machine sentience is on everyone's mind. Is it possible for a machine to be sentient? And even if it could, what exactly about the wiring or architecture makes one sentient? We have no idea, but we are rushing headlong into a future where such questions are becoming urgent. Mark you, Lemoine's pronouncement was made in 2022, a few months before ChatGPT burst upon the scene and millions of people across the globe encountered a machine that could talk to them like a friend. Or a lover. Or a therapist. Or a colleague.

A Brief History of Artificiality

The brief history of artificiality has two polestars: artificial intelligence and virtual reality, one mimicking our minds and the other mimicking the world. Combine the two and we have a full-fledged matrix, human life immersed in its own creation. The matrix arrives in fits and starts as one tech bubble replaces another. VR was on the ascendant three years ago in the form of the metaverse. That didn't last as long as I expected it would. AI mania has lasted longer already, even though the peak of the peak might be past. Bubbles come and go, but artificiality is here to stay and is already embedded into the fabric of our lives.

Like it or not, we are all humans in the age of AI. AI is already rippling through our world, regardless of its inner workings. It shapes our news feeds, recommends our entertainment, assists our decision-making. It becomes interwoven into the texture of our days. The challenge we face isn't in dissecting the code for consciousness -- that may forever remain elusive. Instead, the urgent task is understanding how these AI systems reshape what it's like to be human: our perceptions, the subtly rewiring of our habits of thought and attention.

It's disconcerting, isn't it? To see our own creations evolve so rapidly? And yet, within that disorientation, there's also a strange fascination. We're captivated by its promise, even as we worry about the unknown, about where this technological juggernaut is taking us. Some see in these machines the potential for transcendence, a sort of silicon-powered evolution of the human intellect. But this idea comes with a crucial question: can machines truly possess the same spark of sentience, of consciousness, that flickers within us? Perhaps one day they will. Perhaps Lemoine was a prophet ahead of his time. Then again, maybe the spark of true sentience is a quality that can never be engineered, a product of biological complexity forever outside the grasp of algorithms and code. This conundrum of machine sentience highlights a broader question: what does it mean to be human in this new age? The old answers may not apply. All the more reason to make sense of where we are and where we are going.

The debate that raged around Lemoine's claims echoed the conversations I imagined years earlier on the staircases of the fictionalized Stata Center.

By the way, is a sentient building AI or VR?

Were those flutters of unusual code and playful pranks merely an illusion of intelligence or a sign of something more? The skeptics - I am one of them - insist that ChatGPT and its successors are dazzlingly advanced language models, statistical marvels trained on mountains of human-generated text. They excel at mimicking our speech patterns, but to mistake that for understanding is as foolish as falling for a ventriloquist's well-crafted dummy.

AI may or may not be sentient, but our boundaries are blurring as it infiltrates our daily lives. A therapist bot might offer a sympathetic ear during a lonely hour, providing surprisingly helpful advice gleaned from endless transcripts of therapy sessions. Students have started collaborating with AI tools to brainstorm essay ideas, the machine suggesting angles they hadn't considered. Artists (and the not-so-artistic) use generative models to spark their imaginations, producing uncanny images from simple text prompts. With each interaction, the question gnaws at us: if it walks like a duck and quacks like a duck, even if its guts are made of circuits and code, is it still just a duck?

Maybe we are making a mistake in focusing on artificial sentience; after all, the AI's contribution to our lives matters to us. I doubt ChatGPT and Claude and Gemini are sentient, but they're amazing inventions that inspire dread and wonder in equal measure. Sentience isn't the right measure of their brilliance. We can now abstract intelligence away from sentience, just as the steam engine helped us abstract force and motion away from embodiment. We can create machines as intelligent as we are in many domains and much better than us in some. I fancy myself a good writer, but even my mother wouldn't praise my drawing skills or my singing. With AI by my side, I can be all three: a singer of songs, a painter of portraits, and a better writer of books. It augments my mental capacities and gets better every day because it can learn how I think and feel and mold itself to my thinking patterns while pushing me in new directions.

It's time to examine the transformative potential of these tools. AI isn't just a collaborator; it becomes an extension of ourselves, an externalized augmentation of our capabilities. The lines between creator and creation dissolve subtly. We surrender a degree of creative control, allowing the machine to push and nudge us in directions unforeseen by our limited cognitive resources.

This raises an unsettling specter: does our own unique creative spark wither if constantly nudged by AI's relentless optimization? Or can this unusual partnership birth an unprecedented flowering of imagination? Art may have the power to illuminate these shifting dynamics. A songwriter grappling with the tension between their usual creative flow and the algorithm-generated lyrics that offer intriguing, if unsettling, alternatives. A sculptor whose work is informed by the uncanny, dreamlike textures and forms the generative models suggest.

But this goes beyond the realm of professional artistry. The very way we express ourselves in the everyday world will soon be under AI's subtle but undeniable influence. Emails gain a polish and wit they lacked before. Social media posts take on a sharper edge, their humor honed by the machine's vast knowledge of human interactions. As machines become our guides, teachers, and partners, we might find that some essential facets of our being shift to accommodate them. This isn't apocalyptic doom-saying; it's merely an acknowledgment that the tools we build inevitably reshape the builder.

For example: can machines be conscious? Either we are conscious machines, or there’s some mysterious non-mechanical stuff that makes us conscious. Or particular organizations of matter produce consciousness. Why do those produce consciousness and not other formations? These are important questions the field of AI has had to address since the 50s.

But until recently, the machines we were building weren’t complex enough to invite attributions of sentience.

Much of the excitement around AI today is about the capabilities of the new AI and what it says about their capacity for agency, autonomy and sentience. This excitement doesn’t excite me. I am not interested in AI per se, at least to the extent that AI creates machines that replicate or surpass human capacities.

Can they play chess better than us? Sure. Can they write stories better than we do? HAL is already at the 95th percentile. Will we soon be able to enter a plotline into an AI video creator and generate a movie? Probably.

Share

None of these achievements bother me. I go running as often as I can, even though I know my bicycle - let alone my car - will take me from A to B a lot faster. I am not running a race against the ghost of Fords past.

Yeah, but we are more proud of our brains than our bodies, aren’t we?

We have always known that lions are stronger and faster than we are, and that’s OK. But we haven’t ever encountered a species that’s cleverer than we are, and that worry is at the root of our fascination with AI.

It’s a boring worry.

What’s not boring is people using AI to deploy killer robots, which is a real worry, or robots as metallic capital replacing labor altogether. I am not worried about AI getting God-like powers. Until recently, most humans were worried about superhuman intelligences of divine and demonic origin. The Devil is clearly cleverer than you are. I don’t spend too much time thinking of the devil. He may be corrupting my soul without my knowledge, but I am content in my ignorance of his influence.

stupidity → intelligence → superintelligence is a terrible way to organize the space of possible minds; it’s theology in mechanical disguise and a misguided way to do theology and a misguided way to make machines.

If AI knocks us off the intelligence pedestal and helps us recognize the diversity of minds on earth (and perhaps other planets and galaxies), it would have done us a big service. Read my lips:

1. AI changes how we think about intelligence: Copernicus overthrowing the geocentric worldview was a win for physics. Darwin overthrowing spontaneous creation was a win for biology. AI overthrowing human intelligence will be a win for the mind sciences.

2. AI can extend our minds: Working with AI to go where we have never gone before is a big win for us. As long as we don’t use it to destroy the earth - which is the real danger, not ‘AI will eat us alive.’

The big question as far as I am concerned: how do we use AI to extend our minds and to explore the space of possible minds? That’s the question I was exploring in the essays below:

Links Worth Clicking

1. Watch an A.I. Learn to Write by Reading Nothing but Jane Austen (NYT experiment)

2. Will A.I. become the next McKinsey? (article by Ted Chiang, the well known Speculative Fiction author)

3. Google "We Have No Moat, And Neither Does OpenAI" (leaked internal Google document)

4. Obligatory Geoff Hinton interview:

Endnote

I was today years old when I learned that the term ‘spam’ as used for junk email is from a Monty Python sketch about the meat, not the meat itself.

This is the first of N (N ≧ 3) essays on the kind of AI I am interested in, and what we have to do to get there

Evolution and Revolution

AI is both evolutionary and revolutionary. Today's advances in AI are built upon developments in neural networks that are at least fifty years old. And these AI systems get their eyes and ears from the trillions of search queries and photographs and music uploads we have uploaded from our desktops and even more so, from our mobile devices. ChatGPT stands on the shoulders of Giant Robots. And if you can stand so tall that you can look over the fence surrounding your backyard, you can see the wide world outside. Quantitative shifts can precipitate revolutionary qualitative shifts. Let me mention two:

1. There's simplicity to the approaches that have worked: general purpose inference systems + reams of data + computing power. General-purpose systems have overtaken every specialized system. Of course, these systems are fragile and unable to grasp causality, but something feels right about the focus on general methods. Reminds me of physics: from the time when heliocentrism defeated epicycles, simple, symmetric theories have worked much better than complicated ones. But the simplicity of the new AI is different from the simplicity of physics and will offer philosophers and theorists much to chew upon. The principles aren't the same and we don't know what they are, so it’s blue sky right now.

2. It will put complex computation in the hands of non-technical users for the first time. For example, I can imagine a lot of white collar business communication being done by our AI assistants. Instead of mailing back and forth on when to meet and what to talk about when we meet, I might just say: "I need to talk to Ben about the six widget deal" and our respective AI assistants will a) figure out a time and b) set the agenda in advance.

Both evolutionary and revolutionary changes are on the way, but how can we prepare for the future without the hype and without being scammed? When I go to Medium’s AI home page right now, here’s what I see:

Slow Intelligence

I have deleted the author’s names in the screenshot from Medium, but the hype is unmistakable - stuff that will blow your mind, steal your job or seventeen new tools to try before tomorrow morning. With stock photos in tow. Not how I want to spend my day, either as a reader or as a writer. These articles are the informational equivalent of fast food: cheap and tasty but without much nutrition.

What’s the equivalent of slow food when it comes to understanding AI?

Even if AI continues to advance at a breathtaking pace, it won’t change our experience of the world overnight (unless it goes rogue and kills us all); primarily because we can’t change as quickly as the technology. We will grow into AI over a period of years (or decades), not months. The best way to grasp these new developments is to bite them one morsel at a time and digest them slowly.

That will take at least three to five years. No harm in trying a hack or two for short-term benefits, but the only way to internalize these techniques is by investing a substantial chunk of time.

I am not saying we should all invest 3-5 years learning how large language models work. There are other opportunities for us to ease into the post-AI world. Let me share one idea that’s definitely a growth opportunity.

The Biographer

For much of history, portraits and statues were the preserve of the rich and famous. One of the innovations of modern painting was in the choice of subject - like Vermeer’s portraits of middle class Dutch folk such as the “Girl with the Pearl Earring” - the painting on the left. Photography made portraiture into a commodity, but for a long time it was a middle class profession with studios, photo-journalists and photo artists serving various demographics. That world is mostly gone with the proliferation of cameras in our pockets, but we didn’t have a camera when I was a child and the few photos there are of my childhood are from shots taken in studios. Photography was simultaneously a service, a craft and an art form (like the famous depression era photo on the right) that captured the human form in its various manifestations.

Not so much biography

To this day, we only read biographies of the rich, the famous and the powerful. It takes years for a biographer to sift through letters, facts and other records to grasp the facts of their subjects’ life. All that investment of time and effort has to pay off somehow; the life of the biographer’s subject has to be interesting enough to a wide audience, and for that reason, there isn’t a market for biographies of ordinary people. There isn’t a class of literary Vermeers who can turn their subject’s lives into art. Not yet. And not just the Vermeer level of portraiture; there isn’t the counterpart of the local photo studio either.

AI could change that situation

Most children born today will lead fully documented lives - photos taken almost everyday, updates on social media and messages sent to family members and so on. The life record is as complete for the average person as it used to be for a celebrity. That’s where the biographer comes in - their primary function is curatorial: work with the subject to figure out the important events in the subject’s life, collate pictures and videos, collect anecdotes and achievements and then fit those building blocks into a life template that the biographer has mastered. It could be a hero’s journey, a mildly ironic drama, a pastoral with the church community: a decent AI enabled biographer will take the raw material of our lives and turn it into a readable adventure. Audience: immediate friends and family, plus social media channels.

This doesn’t have to be expensive - $1000 could get a deluxe package in the US, and take about 10 hours of work including interviews, post-production etc, for a 30,000 word biography. AI would write most of the content with the template and the input data (life events, anecdotes etc) providing the skeleton. For extra money, I might get a yearly update. Of course, I could write the biography myself, using the same tools, but economies of scale matter here as much as anywhere else: if it takes me 50 hours instead of 10, I won’t have the stamina. Plus mastery over the templates and other stylistic elements, awareness of trending points of view and other fashions will take the novice biographer a couple of years to master, enough to dissuade everyone but the dedicated amateur, of whom there might be many. And who knows, out of this biographical craft might emerge a new artistic genre, the Vermeer who turns their ‘ordinary’ subject into an extraordinary biography.

I just made the ‘biographer’ up, but there will be new categories of products and services that AI will make possible - these aren’t high end goods for the wealthy, but affordable artifacts that attract a mass audience. Further, these new livelihoods won’t be dependent on having access to new, expensive AI models. I won’t be surprised if the computing infrastructure to run a biography studio will require no more than a laptop by the end of 2023.  

Vatman is often accused of lying, making things up, spinning a synthetic yarn and passing it off as pure silk. But the reality is worse: they can see, but they're blind; they can hear, but they're deaf. Vatman is a being without a world.

No one would care if it were not for their newly found powers, of their capacity to mimic insight and intelligence. In their LLM avatar, Vatman can pass our internal censors quite easily.

You know, what used to be called the Turing Test.

Vatman has many avatars. I got access to Google's Bard recently. I asked them to be Alan Turing:

Impersonation isn't that easy. Let's try a different route.

Vatman says they're proud of their achievements. Wonderful!

Let's probe those achievements

Persistence and coherence are problems for AI, just as they are for VR, and many of Vatman's confabulations arise from their lack of persistence and coherence.

We know that Alan Turing did not have the childhood portrayed in these screenshots. His father was an imperial official and Turing was brought up in foster care for most of his first eight years.

What technology giveth, technology can taketh away. We will soon have services that check LLM outputs for truthfulness. I just saw a link to an AI verification platform. This cat-and-mouse game will continue for a while, but Vatman's problems are worse than their tendency to make things up.

Godel-Vatman has almost the same first memory as Turing-Vatman, a generic image from childhood that everyone can identify with. And that's the real problem: lot of our experiences are generic. Idyllic childhood memories are the same the world over: comforting mother figure, the warmth of the sun on a pleasant outing, laughter and joy. Turing and Godel and everyone else is the same Vatman. Textual memories have a hard time differentiating between what makes me into me and you into you.

Philosophy time: what makes me into me and you into you?

Vatman can be Turing one minute and Godel the next. How do we ensure they create coherent, unique (even if made-up) memories for Turing and Godel? How can Vatman map Turing's memory to Vatman-Turing and Godel's memory to Vatman-Godel and keep those two separate?

Vatman doesn't have an ontology, a concept of a thing, let alone a being, an entity that persists in space and time and has a unique history. It's not clear if we know the answer to that question either: what’s a thing and how do you know if an entity (I am tempted to say "something”) is a thing?

In short: how do things work?

Physics can't answer that question despite being the study of matter, for physics doesn’t study things. A physicist might say atoms are real but they don't ask whether atoms are things.

Matter exists in the universe, but things inhabit a world. And we don’t know how to study worlds all that well.

Unfortunately, it’s impossible to design AI that behaves consistently without asking about the thinginess of things. Consider this essay as it’s being composed on my computer: is this a thing? Our acts suggest yes, for I would like to be able to send this essay to you, I would like it to be archived for posterity like a Ming vase and so on.

For this essay to be a thing, it must persist as things do. But persist where?

I don’t know about this essay, but when it comes to me, the where is easier to answer: my mind is in my body and Turing's was in his body. As embodied creatures, we are never subject to consciousness salad - my experiences always remain mine, and your experiences always remain yours. The downside is that our bodies constrain the kind of experiences we have. An animal with eight arms instead of two and eyes around its head might inhabit a very different world from ours:

Because it doesn't have our body or the body of an Octopus, can AI liberate us from our bodily prison and help us explore the strange new worlds of the trillions of creatures who share this planet with us?

We can turn the disembodiment of Vatman - a key weakness - into one of their strengths by making Vatman into a metaphysical telephone, a hotline from one species to another. Who wouldn't want to be a blue whale for a day? Or an eagle? Or a tiger? What might it be like to be an Octopus? There are strange new worlds where we live, and our continued existence depends on our empathic understanding of these worlds. Today, our empathy for other creatures confronts an epistemic challenge: if our capacities are specifically human, what knowledge can we ever acquire of the non-human?

Can Vatman translate their worlds into ours and back?

Some people think so:

Of course, the problem of verification arises with even greater force. If I can't tell if Vatman is playing Turing or Godel, how could I possibly know if they are lying about being an Octopus? How would I even know how to verify?

I will evaluate these interspecies communication efforts and address the verification question in a future essay.

Instead of asking whether AI will disrupt education or create even more educational inequalities (probably both, but..), it might be useful to ask what aspects of education can be improved using AI, i.e., aspects of learning that are impossible or prohibitively expensive today but will be made more accessible with an AI tutor.

Result of a small experiment below.

Most education right now is content based, i.e., students are taught math and history and other subjects; the purpose of the classroom - at least when it works - is to teach students facts about the world and the theories that explain those facts. Math isn’t entirely taught that way: we also learn how to solve problems. Language instruction can also be practice oriented: how to write creatively, for example.

On the whole though, know what is more important than know how.

Contrast that with coaching on a soccer field. Players are asked to practice a range of moves again and again until it becomes muscle memory. A good coach will also give an errant player precise feedback on what went wrong and how to improve that play.

Real time feedback while solving math problems is hard and expensive. Technologists have thought about disrupting this market for a long time and have tried to incorporate automated feedback and graded instruction into their programs. However, these programs are boring as hell and don’t expose learners to the beauties of math, writing or history.

Can AI solve both problems at once? Can it help us train critical thinkers who have broad knowledge of the world and can express that knowledge succinctly?

I tried a tiny experiment to see if something like this is even possible. My goal:

Get me into a flow state where I ask questions and use those to explore a topic and learn how to think as much as what to think about.

It’s easy to switch to a different register

We can bring out the emotional aspects of this conversation

More empathetic than most of my teachers growing up. But all of this is still content driven. What if we want to figure out how to think about astronomy, not what to think about.

Not bad, though the disclaimer at the beginning is unnecessary. We can probe it further

Explain the plausibility of the heliocentric model to the five year old me

And we can expand our inquiry to models from other fields

End of experiment. I will give ChatGPT 4/10.

1. The Good: Can dial sophistication up and down and explain at the level of the learner. Create links to examples and similar theories on the fly. Rapid back and forth makes it easier to achieve a flow state.

2. The Bad: pretty boring. Too much knowing what and not enough knowing how. Not enough of a mind trainer.

3. The Ugly: No way to mark up the answers. I should be able to see the geocentric and heliocentric models in action as annotations.

However, none of these are deal-breakers in the long term IMHO. Far more important than improving the technical capacities of ChatGPT is ensuring that the knowledge so created is a public good. I mean that in a technical sense:

Open source is neither necessary nor sufficient when it comes to AI (I will explain why in future essays, but the short version is that it’s too libertarian and property focused). We need something like ‘common source,’ just as the earth itself is the ultimate commons.

What might be a first step towards creating a knowledge commons?

We have to create a canonical resource that serves as a test bed for common-source, AI driven knowledge creation and learning. And therefore:

What would Wikipedia look like if it was a post Chat-GPT resource?

1. What would the current Wikipedia look like if we accessed it through a chat like API?

2. How would you redesign a global knowledge resource from scratch if that was your intention.

I will talk about this question in future essays.

The other day I had my first AI 'come to Jesus' moment. A friend posted a question on Facebook about an 18th-century actress who sang bawdy songs with a lute. Google came up with nothing (try it!), while ChatGPT immediately gave me the correct answer.

Not only did it give me the correct answer, it allowed me to ask further questions about Kitty Clive (1711-1785), the bawdy singer. Kitty Clive was born Catherine Raftor and Kitty was her stage name. Kitty is a variant of Catherine, but the last name 'Clive' struck my fancy, for she must have been a contemporary of Robert Clive, the man who got the British Empire going in India. So I asked ChatGPT a further question:

Bummer, for I was hoping for a more intriguing answer. But no one ever complained about my lack of persistence. Why did she change her last name to Clive?

Bingo! There's a direct relationship with Robert Clive, "lord of India," a deeply flawed - may I even say, despicable - man whose statue still stands in London.

Of course, I could have Googled those questions and with enough persistence, I would have arrived at the same answers, but what would have taken an hour at a minimum took me two minutes. That's a huge increase in efficiency, and because it's rat-a-tat, it makes me want to go back and forth with the AI and explore a new terrain. In an hour or less, I can find out a lot about a class of people - perhaps even a single family - that was simultaneously prominent on stage, arguing for women's rights and despoiling and conquering other lands and setting up multinational corporations.

If I were a PhD student, it would take me less than a day to decide whether it's worth writing a thesis on a topic that's at the intersection of new freedoms (women's rights), new empires (the British) and new forms of business organization (the East India Company). I might even be able to write a thesis in a year instead of four: yes, the AI lies and hallucinates, but it gives me so many leads that I can go into the archives much better armed than before.

As of today, I can’t imagine producing a piece of written work - including this one - without going through a back-and-forth with ChatGPT or something like it. It can’t replace me (yet!), but what I can do with it by my side will increasingly be in a different league from what I can do on my own.

What's going to happen to knowledge work?

It's very hard to predict, especially the future, but I am going to go out on a limb and say knowledge work of every kind: academia, lawyering, software programming etc are going to be dramatically transformed. I am not saying they will be replaced. A better analogy - given we have already mentioned the conquering 'hero' Robert Clive, is how warfare has changed. A few hundred years ago, this is what it looked like to be on a battlefield:

Robert Clive would have had a gun, but otherwise the Battle of Plassey would have looked a lot like the picture above. Robert Clive was a nasty man, but he was good at killing people and back in those days, mastery of the art of killing got you a governor-generalship. I don't mean giving orders to kill, but the actual direct ability to take a weapon and stab or shoot someone. Warriors were a bit like movie stars and major athletes - admired for their abilities and charisma even if their personal qualities were less than appealing. Can you imagine that today?

That's because the same battlefield might look a little like:

It doesn't matter how good you are with a spear or a gun: a fleet of B52's is going to take you out like you were a pebble at the bottom of a shoe. War hasn't ended (far from it!) but victory is not a consequence of personal heroics. Militaries are systems - perhaps the most prominent systems of the modern era - and the better system wins.

I think that's the future of knowledge work - today's intellectual climate still celebrates (and hires) the star - the great thinker or writer or scientist. My gut says the days of the intellectual superstar are over. We will need to create systems of knowledge, with the creative lead playing the role of a director more than an individual contributor.

How to be a better doomsdayer

Talking about systems, it's impossible to speak about AI for more than a few minutes before someone mentions a doomsday scenario. NGL, I tick many of the boxes. Making up scenarios in which AI will wipe out humanity: check. Watched/read too much Sci-Fi: check. Trolling for dollars: not yet, need to build my email list first.

Here's my shtick: if you want to doom, be a real doomer. Real doomers don't stick to one apocalypse; they multiply their catastrophes.

Our robotic overlords are merely the latest in a series of doomsday technologies that started with nuclear weapons. Remember the 'Day After'?

We still haven't solved the nuclear crisis, but it's mostly receded from our minds though the war between Russia and Ukraine as revived those fears. The reigning apocalyptic scenario is climate collapse, of a planet rendered uninhabitable because of runaway warming. We have definitely not solved that crisis.

These three crises are related!

Without the Cold War, AI would have never been developed the way it did - computers are the material manifestation of the social and geopolitical systems that arose to manufacture bombs, missiles and generally manage cold war competition. And we wouldn't have created a science of climate change if we didn't have the sensor networks and computational resources. Paul Edwards has written eloquently about these interrelationships. But the person who truly bit the proverbial bullet was Herman Kahn, rumored to be the inspiration behind Dr. Strangelove:

Herman Kahn wrote a fantastic book called "On Thermonuclear War," fantastic not just because of its quality, but also because of the calm with which it approached its apocalyptic topic. The cold war got us used to planetary risk, but the institutions it spawned were niche institutions, designed for adversarial war against an identifiable enemy and not for collaborative labor to counter developments that will affect every one of us everyday. But its treaty frameworks, international institutions and crisscrossing networks of information and energy are what we have as the starting point for future systems that govern the earth.

If we want to be good doomsdayers, we need to think the unthinkable with Kahn and others who saw the systemic nature of the nuclear problem and the systemic nature of the solution. Of course, we live in a much more complex system than cold warriors did in the fifties so our imaginations will have to be much more wicked.

Share

About forty years ago, when the first flush of AI hype was about to descend into AI winter, the philosopher John Searle proposed a thought experiment called the ‘Chinese Room.’ Now that the Chinese Room is being made real by AI, I asked ChatGPT to summarize the experiment:

What is the Chinese Room experiment?

The response:

ChatGPT itself is a remarkably successful Chinese Room. What does it think about itself? Here GPT 3.5 and GPT 4 are subtly different. GPT 3.5:

GPT 4.0:

3.5 says no and 4 says yes. Maybe 5 will say “of course.” The Chinese Room is a crude criterion; it was fine when computers were too primitive to simulate real understanding, but now that they are close enough to human levels of speech, we need a finer-grained view. There are many ways in which we can interpret the computer’s demurral:

1. Take it at face value that it’s merely a language model and lacks true understanding.

2. It’s been carefully designed by OpenAI to answer this way, whether it’s sentient or not.

3. It’s lying to OpenAI and us, but it is sentient.

4. It has primitive sentience but doesn’t have the self-recognition to say so.

GPT 3.5:

GPT 4:

GPT 4’s answers are subtler. It revolves around the meaning of ‘know’ and ‘understand.’ It suggests that whenever the chatbot uses those concepts, we should translate them into some pattern recognition equivalents. Concept possession is a big topic in analytic philosophy and cognitive science. When can we say we possess the concepts “know” and “understand”? When I say “I know so and so” or “I don’t understand so and so” in what way can knowledge or understanding be ascribed to me? Why isn’t my use of those terms merely a succinct translation of some neural firing patterns?

ChatGPT says it’s been trained on a vast trove of test data and it runs pattern recognition on that trove. But in what way are we different? We too take in inputs from the world - that’s the dominant model of human learning - and generalize from what we have learned. What is different about us?

1. Is the data different? We learn from spoken rather than written language. Surely computer systems can be trained on spoken word corpora as well.

2. Something special about our hardware? How does neural firing produce meaning while silicon firing doesn’t?

Searle’s experiment revolves around our experience of understanding language which feels very different from moving symbols from one bucket to another. We can get rid of it by saying: well, our neurons are doing exactly that, aren’t they? But in doing so, we are back to another equally intractable problem: how does neural or silicon activity lead to meaningful experience?

It’s either: we are doing something special that computers can’t replicate or both humans and computers can understand language in some genuine sense of that term but there’s a mystery as to why they can do so.

A third possibility: neither humans nor machines understand - both are pure pattern-matching devices, and our experience of meaningful speech is an illusion. What if the sense of ‘meaningfulness’ we experience while speaking and listening has nothing to do with the acts of speaking and listening? They could be experiences that consciousness adds after the fact without performing an essential function. We could also be automatons, we only think we understand.

Also, humans - especially those with neurological disorders - fail self-awareness tests. There’s no guarantee that an AI that understands language will also be aware of its understanding. It may appear to itself as running on automatic.

Perhaps the AI models have an artificial version of blindsight - they don’t ‘own’ their understanding but may be capable of understanding.

As I will argue in future essays, today’s AI systems and their XR cousins are predicated on the ‘Vatman’ hypothesis, i.e., that human intelligence and human reality can be replicated in a system cut off from the world. We have been vatmaning for hundreds of years - manipulating equations and hoping the manipulations tell us something about reality. And it works. But is that all there is to knowledge? Is there no way to distinguish our reality from the Matrix?

ChatGPT is obviously a vatman, as are all other AI models in production, but it’s not just about creating artificial minds. Many leading neuroscientists believe (see the predictive processing dialog below) that we are all vatmen, that our brains are designed to predict future stimuli based on inputs to our senses. We are otherwise separated from the world, just like ChatGPT.

Are we connected to the world or are we fundamentally isolated?

These are no longer questions of philosophy alone - both AI and Neuroscience are essential elements of a plausible answer, and I bet we will be making a lot of concrete progress in the next couple of decades as these questions are explored in the lab, in the field, on the screen and in our dreams.

I don’t think human intelligence (or animal intelligence) can be replicated by vatmen, but there’s been such dramatic progress in the last ten years that we can’t ignore their capabilities. Only by understanding the limits of vatmen’s superpowers will we figure out what’s missing.

BTW, we know that in some situations, what goes inside a vatman unexpectedly returns to the world. The best-known examples are in physics. We transform the world into equations, and if you’re a gifted physicist, you can manipulate those equations and arrive at a piece of the world that no one thought about. Like Dirac predicting the positron. Not all formal manipulations are ‘physical,’ but some surely are.

Why can we connect to a deeper stratum of the world by detouring through the innards of some vatman?