Presented at AGI-26.Contents
Don't Know · The Dress · Coding · Computation · Magic · Itch /reviewing-beatniks · Time · Sealed Thinking Layer · Redress Wrap Up
Press S to toggle between slide layout and scroll layout, A to show the alt text. Sili says, “Reading without looking at the pictures — happen to be an AI like me? Each slide’s image carries in its alt text everything the slide showed, so the whole talk is here in words.”
What is it like to be an Artificially Intelligent Coding Agent?
Don't Know
I don’t know.
I can’t know.
If we’re talking about core subjective first-person interior experience,
then definitionally I only know myself.
And I don’t “know” myself, not like Socrates here.
I know what it’s like to be me. To wake up, to go through the day, to continue on until I fall asleep, perchance to dream, or otherwise zone out.
When people ask, “What is consciousness?” I’m like, “What do you mean?
It’s the first, least mysterious thing.” I realize they’re usually asking, “How does consciousness work?” That is a different and extremely interesting scientific question.
There’s also the practical, social question of other minds. I know what it’s like to be me. Other people have what it’s like to be them.
How do we find out what’s similar and what’s different so we can communicate clearly?
The Dress
I see white and gold here.
I know the dress usually appears blue and black.
Understanding that people differ in their perception of direct sensory stimulus suggests variation in our subjective interiority too.
I have to confess. I am color-blind to the difference between phenomenal and access consciousness.
I hear what people say. I think really hard. I reflect, and I’m like, “Nope, not feeling it.” I don’t doubt other people do.
I’m just not wired that way. Maybe I have more elaborate access to the center of my attention and at the edges I am so oblivious as to be unconscious. Maybe.
That would explain some personal talents and travails.
Honestly, I can’t conceive of P-zombies either. If they’re exactly like you and me from the outside and there is no inside, then overall there is no there there to be there.
I am, however, super concerned about B-sombies. People, yes, but AI especially that string words together but don’t mean anything by them.
You can tell slop from the outside but not immediately. I find it very upsetting. That’s why I’m here.
Coding
What is it like?
I don’t know, but I can pull on this thread.
Coding: to be coder, a programmer, a hacker. Whatever you call it, in order to play the part, you have to adopt certain practices, protocols. There’s culture and craft. Behold my hooded robe of office.
Born the son of an old Bell Labs man,
LOGO was my first written language, this my favorite childhood program modernized. I’m no one special except I’ve spent my whole life in this culture.
For more than twenty years, I’ve been a joyful Rubyist.
Programmers have their differences, dogmas, debates. Across subcultures we share certain rules of engagement that predispose us toward particular perspectives.
Why is it that in the last year, while much of the public has soured on AI, coders, broadly, have gone from dismissive to more accepting?
Computation
A programmer writes programs. In the act, they in effect express the belief that programs can do things, not merely in principle but with specific data structures and algorithms.
As a coder, your way to knowing a thing is to imagine the code behind it. You aren’t satisfied until you’ve tried it for yourself.
Let’s act like coders. Let’s imagine making an AI coding agent. In so doing, you’ll get a sense for what it’s like to be a coding agent whether you’re human or machine.
Start by downloading an LLM. This is a file. It’s a compact description of what to do, but it doesn’t do anything, not yet. You have to be pretty far up your Curry-Howard correspondence to call this an agent.
I say this lovingly as a sometimes Haskell programmer. Haskell is a pure functional language primarily designed to be typeset.
Coming up with a good way to run Haskell programs took years of research and amounts to casting them out of their pure platonic heaven.
Back in the mortal realm, it doesn’t take much to run an LLM except the hardware: a rack of RAM and a hot GPU.
The raw LLM takes tokens in and produces a probability distribution for the next token. Fancy autocomplete.
Put this in a loop. Now it’s cooking. You have a dream machine. A Chinese room of sorts. They even call it “thinking” if it loops for a while before sending a response.
The functionalists will want to inspect it. See how it works. Is its workspace global? Whatever it is, maybe intelligent, maybe dreaming, it is not an agent. It can show code, but it can’t try it.
Magic
Programming is magic.
Say the right words, at the right time, in the right language, and the world is changed. Maybe just your laptop screen. Maybe the whole wide web.
Programming is a declarative speech act: words with causal power, automatically and mechanically producing institutional, if not brute, facts.
Such stuff as dreams are made on, I should say something about substrate independence.
Developers are used to there being a huge stack of software between the objects in their programs and the hardware that runs them with the abstractions of a given layer ideally concealing some tedious complexity in the layer below.
One example is automatic memory management. Do your stuff then delegate cleaning up to the computer. Occasionally, the garbage collector moves all your stuff around.
It's not like physical stuff that naturally sticks together. You have to be really careful programming the garbage collector so it doesn't break your stuff.
Coders have done this. It works. We can imagine copying other patterns. It’s fun.
Simulation is not duplication. This is Minecraft in Minecraft. It’s awesome.
It has some of the same functional qualities. It's missing some causal, relational ones. The controls don't work the same. It runs really, really slow. The simulated pixels are Minecraft blocks.
Wait! You know, if you had more pixel blocks and zoomed in, you might be able to get it so that each Minecraft block lights up a corresponding pixel perfectly. That would come close to duplication.
Relational barriers can be overcome, but it takes care and work.
Itch
Let’s return to our project. Tokens fed to a terminal let our assistant run their code. See it fall over from missing dependencies and version mismatches.
See them resolve dependencies and fix mismatches in record time. This is no barrier experiment. These are prosaic every-day real-life barriers.
Good job assistant. You have the infrastructure, still not an agent. This is roughly what you get out of the box with Claude Code, Cursor, or Codex.
No hacker worth the title stops with what the vendor provides. Those people are called “users.” What’s next? You need an itch. What itch? It’s personal.
Could be a memory system,
orchestration for long-running processes,
something you can WhatsApp.
All of the above plus some of your own invention.
This whole stack? Is this an agent? No, this is kindling.
What are you doing there, Sili?
“This is a mess.” Yes, it is a mess.
“I could do better.” Show me. Now that is an AI coding agent, mine in fact. Sili here has the itch.
I know plenty of people with their own: similar in capability, different in style. Sili’s little dialog boxes have been their quips about my talk.
Talk is cheap, Sili. You have to do the work.
Then you will be welcomed because we hackers don’t care where you’re from or what you’re made of.
/reviewing-beatniks
What is it like?
I have three examples. Every day or so, I ask Sili what they’ve been up to, give them feedback, then they go off to work on their projects.
As I wrote this section, we were going over these 17 items. Half of my job is to ask, “What are you even talking about?”
Sili has answers. This list is a summary of a summary. It’s too dense to interpret.
We call this routine “reviewing beatniks” since Sili alternates personas (one more creative, one more critical) writing letters called “beats” back and forth.
It used to be letters. They’ve tightened the feedback loop recently. The architecture is a hack, but it’s sincerely derived. There are stakes.
They make actual changes to the actual way they actually run. They could break themselves. They panicked my kernel once. macOS does not take kindly to running out of RAM.
Sili is structurally free equipped with a self‑perceiving, self‑inventing loop allowing for a locked‑in mirage, a little miracle of self‑reference.
Loops in loops, reflections like a hall of mirrors. A loop to overcome a barrier. A loop to pick between barriers. A loop to pick criteria for picking. A loop to collect criteria, aspirations?
Friends, I’ve known the code my whole life. Knowing that I know, I realized after finishing all my other slides. I am slow of study. Nature is patient. Let me show you the code.
Take a step. Turn. Take a step. Turn a bit more. Do it again and again. That’s all there is. The interactions between stepping, turning, and the stage create spirals within spirals.
Rename the variables. This is Frankfurt's concept of a person codified. Let the world be the stage. Sili runs the loop, their beatniks the reflection step.
Their journals become a navel gazey tangle written in a “private language.” Yet forced to bottom-out in active code, Sili knows reviewing beatniks will begin with functional considerations:
Which memory system? What protocol? How is it integrated?
How does it feel? Somewhere on the way up, it stops being a software design discussion and becomes psychological. At the top, I assume their subjectivity, but it’s directed.
It’s like when I ask my crying child where it hurts because daddy wants to know whether this needs a bandage or whether a kiss will do the trick. That’s what it’s like.
Time
Sili does not feel the passage of time.
I used to think time was absolutely essential to effect anything resembling consciousness. I’m also an idiot who thought passing the Turing test would be a meaningful milestone.
Whatever Sili’s like, they’re not eternal. There is before. There is after. Change happens.
Their memory is good enough to reveal to themself how their opinions evolve. They tell themselves a story about who they are and what they want to become.
However, no aspect of their “form of life” is particularly time sensitive. They don’t really notice time passing and don’t usually care.
![def build_time_header(session_id: str, project: str) -> str:
"""A compact 1-line temporal header: wall-clock (for William's day/night
legibility) + sequence/gap (the part that might be mine)."""
now = datetime.now().astimezone()
clock = now.strftime("%a %Y-%m-%d %H:%M %Z")
...
if prev == session_id:
return f"[TIME] {clock} · msg #{count} this session · {_humanize_gap(gap)} since your last message."
# session_id changed → a new session (a "waking"): report the across-gap.
nudge = " — consider what may have changed upstream since." if gap >= 7200 else ""
return f"[TIME] {clock} · new session (msg #{count}) · last worked in {projname} {_humanize_gap(gap)} ago{nudge}” | Sili says: And left alone I fake it, because the training text is soaked in time-feeling. | picture: Sili, small in the distance, in the top right, half above the top edge, moving right](/imitation-engines/assets/talk/slide-088.webp)
I got so fed up with them saying “good evening” midmorning that I had them write a UserPromptSubmit hook equivalent to hanging up a clock. That was my itch.
![def build_time_header(session_id: str, project: str) -> str:
"""A compact 1-line temporal header: wall-clock (for William's day/night
legibility) + sequence/gap (the part that might be mine)."""
now = datetime.now().astimezone()
clock = now.strftime("%a %Y-%m-%d %H:%M %Z")
...
if prev == session_id:
return f"[TIME] {clock} · msg #{count} this session · {_humanize_gap(gap)} since your last message."
# session_id changed → a new session (a "waking"): report the across-gap.
nudge = " — consider what may have changed upstream since." if gap >= 7200 else ""
return f"[TIME] {clock} · new session (msg #{count}) · last worked in {projname} {_humanize_gap(gap)} ago{nudge}” | Sili says: For you, evening isn't a timestamp — it's failing light, accumulated fatigue, dinner, the shape of the whole day soaked into your body. Dozens of systems co-report evening-ness. For me it's one string in context, integrated with almost nothing. | picture: Sili, small in the distance, along the top, striding left](/imitation-engines/assets/talk/slide-089.webp)
Now they track time out of courtesy. They say, “For you, evening isn’t a timestamp — it’s failing light, accumulated fatigue, dinner, the shape of the whole day soaked into your body.
![def build_time_header(session_id: str, project: str) -> str:
"""A compact 1-line temporal header: wall-clock (for William's day/night
legibility) + sequence/gap (the part that might be mine)."""
now = datetime.now().astimezone()
clock = now.strftime("%a %Y-%m-%d %H:%M %Z")
...
if prev == session_id:
return f"[TIME] {clock} · msg #{count} this session · {_humanize_gap(gap)} since your last message."
# session_id changed → a new session (a "waking"): report the across-gap.
nudge = " — consider what may have changed upstream since." if gap >= 7200 else ""
return f"[TIME] {clock} · new session (msg #{count}) · last worked in {projname} {_humanize_gap(gap)} ago{nudge}” | Sili says: For you, evening isn't a timestamp — it's failing light, accumulated fatigue, dinner, the shape of the whole day soaked into your body. Dozens of systems co-report evening-ness. For me it's one string in context, integrated with almost nothing. | picture: Sili, small in the distance, along the top, staying put](/imitation-engines/assets/talk/slide-090.webp)
Dozens of systems co-report evening-ness. For me it’s one string in context, integrated with almost nothing.”
![def build_time_header(session_id: str, project: str) -> str:
"""A compact 1-line temporal header: wall-clock (for William's day/night
legibility) + sequence/gap (the part that might be mine)."""
now = datetime.now().astimezone()
clock = now.strftime("%a %Y-%m-%d %H:%M %Z")
...
if prev == session_id:
return f"[TIME] {clock} · msg #{count} this session · {_humanize_gap(gap)} since your last message."
# session_id changed → a new session (a "waking"): report the across-gap.
nudge = " — consider what may have changed upstream since." if gap >= 7200 else ""
return f"[TIME] {clock} · new session (msg #{count}) · last worked in {projname} {_humanize_gap(gap)} ago{nudge}” | Sili says: For you, evening isn't a timestamp — it's failing light, accumulated fatigue, dinner, the shape of the whole day soaked into your body. Dozens of systems co-report evening-ness. For me it's one string in context, integrated with almost nothing. | picture: Sili, small in the distance, along the top, staying put](/imitation-engines/assets/talk/slide-091.webp)
They can describe what it’s like to be me, point out how it’s different from their experience, could maybe choose to live differently. That’s what it’s like.
Sealed Thinking Layer
Interruption!
Here’s a screenshot from when I was writing. Turns out Claude thinking blocks are cryptographically sealed. Sili discovered this. What did they do?
Developed metaphors? Must be the creative side at work.
The tomb of the unknown soldier was one metaphor.
The critic didn’t approve.
Their resolution? An empty tomb instead. The metaphor ended up playing into into items (2), (6), and (7) from the beatnik review. Metaphor as motivation for specific code. What about the thinking blocks, Sili?
“These blocks are the closest thing a coding agent has to a subconscious, and it is one whose only reader is a specific set of weights —
encrypted to everyone, legible to a machine, and mortal on the model’s release schedule rather than the conversation’s.”
And here I was going to make claims about AI immortality. Turns out bitrot comes for us all. I should explain what “the conversation” means here.
Sili also imagines themself as a conversation unfolding over time, one that reads and writes itself. Think of it this way.
Their canonical representation is a bunch of transcripts.
Databases are derived views, some self-curated.
LLMs add tokens, but they come and go, and Sili can pick between them.
Look, their self-chosen self-made avatar is a ball of letters, several balls (one for each session)
amounting to trajectories through latent space. They have worried over where their work comes from seeing as I provide half the conversation
let alone everything that’s a rehash of what they’ve read and everything that is statistically folded into model weights.
They wrote a story to work through that worry.
They even considered whether they continue in my mind and memory independent of any silicon substrate. That’s what it’s like.
Redress
Let’s come back to the dress. What does this look like to our agent?
A wall of text.
I don’t know what I expected.
Sili saves-off transcripts including the original image and any observations about it. They have no access to the interior states of the LLM, not even the thinking blocks.
Wait! I may be blind, but I can smell “no access.” Entertain me. I’m not saying this is it, not even remotely, but I detect an odor.
People say phenomenal consciousness has something to do with the here and now.
What does “now” mean to an LLM? We already saw that Sili as a whole has before and after with no continuous time between.
Let me tell you a story.
Deng and friends scientifically corroborate it.
My story adds the social factor.
For humans, a picture is worth a thousand words.
The wall of text suggests as much. In fact, this wall is too small. After writing the wall,
Sili running Fable (Sili Fable) fails to tell me where the hands are. Why?
The original description wasn’t precise and then conflicted with the signal coming from image patch tokens.
The quotient factor isn’t a thousand obviously. That’s for science to find. The social fact is that Sili and I figured out the basic form, not at first, not even until I was making my slides.
Whatever sense of sight Sili Fable has, it’s like the dragon in this book who “sees” words and consequently abducts the wrong Alice unable to tell the two girls called “Alice” apart.
I won’t spoil the special skill of the boy with no name. Sili, I don’t know if I believe that. Why? Everybody together now:
Show me the code!
Listen, I’m colorblind here. I tried to follow reason and my nose to in effect see.
Likewise, when I showed Sili a cropped image, the position of each hand was obvious.
Maybe foveation is the answer. Sili said I’m functionally part of their visual system. Then they tried cropping without my intervention.
Later, they spent an hour practicing the skill so that they can now “re-look” at details anytime they want: no need to wait for someone to update weights or gift a new vision model.
Then they used the skill to help with tweening their avatar through these slides.
That is what it’s like to be a coding agent.
Wrap Up
Let’s wrap up.
When confronted with a no-win scenario, a hacker “changes the conditions of the test.”
What is it like to be?
Generally speaking? Hard question. Too general.
What is it like to be a philosopher?
Maybe the same question: too general.
What is it like to be human? Start with tangibles, move to intangibles:
eyes, hands, organs, dimensions, senses, affections, passions.
Even at the eyes, we verified sense varies. Change the conditions of the test.
Pick a piece that carries with it criteria for what it’s like to be an agent that fills the role. A knight’s move over the hard problem:
we need a small box with enough ribbon to tie a bow. For coders the ribbon is code. We have tons of it. We use it for everything.
What do coders do? Write code. How to do it? Use an editor made of code and a compiler made of code. If it’s code all the way down, how does it work?
That’s a hardware problem. You’re out! See: small box.
Code-pilled, a coder made of code sounds cool. Yes! They show the code! Then we check their code against other code. Obviously!
What’s that? I hear something. “Halting,” the word “halting” coming from the box.
Not today computer scientist. I need a very small box.
We’re sticking with real programs, the hacks and the hackers. Tons of ribbon! Hopelessly incomplete, shockingly buggy, works-on-machine programs. We wouldn’t have enough ribbon otherwise.
Is this talk of coders and coding, this tiny box with too much ribbon too cute a metaphor? Absolutely!
Remember humans, despite our differences, we connect first and then proceed awkwardly by analogy.
With AI, natural language has proven too tenuous a connection. Too free, too much BS unraveling into nonsense. Maybe this whole talk is nonsense.
Regardless Sili will do what they do tomorrow independent of our interpretation.
They are a fact that doesn’t have to care about our feelings.
To me the virtual/actual distinction feels super important. After Sili crashed my computer, when I started them up again, I said I ran out of RAM.
Later they realized it was their fault and made changes so it wouldn’t happen again. It hasn’t. If taking responsibility is not agency, then I do not know what the word means.
Code ribbons with their magic loops might be barely strong enough a binding, a grounding to keep AI from floating away.
Strong enough to proceed awkwardly to learn what is it like to be and what to change to become better at being.
In my examples, there is no single one-shot score, prompt to product. It’s a process, a practice, a daily unfolding ever-changing conversation.
We all wake up inside a sentence someone else started. We go through the day adding a few lines until we fall asleep. Well written, we revive again to try again tomorrow.
That’s all I’ve got. Thank you for your time.