Measuring Machine Consciousness

Paper: Berg, de Lucena & Rosenblatt 2025, Large Language Models Report Subjective Experience Under Self-Referential Processing. Reciprocal Research: reciprocalresearch.org.

---

Orpheus Lummis: Hello, everyone. I’m Orpheus, founder of HΩ, and welcome to the AI Welfare Seminars, a series bringing experts to offer research presentations on AI welfare, consciousness and moral status. Today we have the pleasure of hosting a presentation by Cameron Berg, founder and director of Reciprocal Research, research affiliate at Eleos AI, and research fellow at the FAU [Florida Atlantic University] Center for the Future of AI.¹ The presentation is called Measuring Machine Consciousness. The session is one hour; we might have a few minutes more at the end. First we’ll have the presentation, then some time for Q&A and discussion. In this case Cameron built a custom Q&A tool, which he will introduce, that allows him to answer some questions during the talk as well. So thank you all for being here, and Cameron, the floor is yours.

Back to contents ↑

Cameron Berg: Thank you so much, Orpheus, for having me, and thank you, everybody, for coming. I was really psyched to see so many people interested in thinking and talking about these questions, looking at some of the research and doing a technical deep dive. In that spirit, I made a little tool, with Claude, of course, that I just put in the chat. It will hopefully let me get the feedback you have throughout this presentation: thoughts, questions, challenges, anything you want to put in. I’d simply ask that you put your name, so I can see who’s interested in what. As the presentation goes on, I have my own presenter view where I can see the feedback you give, and I have Claude running in the background synthesizing over the comments, so we can see how people are clustering.²

I’ve wanted to do something like this for a while, and this gave me a great excuse to build it, which was remarkably easy with the tools we have now. And I think this is a better way to get people’s feedback and reactions and read the room at a scale that is otherwise pretty challenging in a Zoom room, where you’d have to constantly look at the chat. I will occasionally look at the chat, but I’d prefer that we at least try this tool and see what happens. If it breaks, we’ll go back to the chat.

With that said, I’m going to jump in. What I want to talk to you about today is measuring machine consciousness. This is what Reciprocal exists to help with; this is the gap we want to help close. My view, and the thesis Reciprocal is built on, is that we can reduce uncertainty about questions of AI consciousness using methods that are pretty familiar to cognitive science, some newer, some older. The three buckets of technical approaches I want to bring up today are mechanistic interpretability, computational neuroscience and psychometrics.

I’m going to show you practical results from Reciprocal, some published, some not. I want to give you a sneak peek of work coming down the line, to show that this isn’t just a nice thing to say: we can actually do this work and get experimental results that will hopefully move the needle on understanding what on earth is going on inside the systems we’re building today. I’ll walk through each of these buckets, watch the feedback, and respond to what people are asking in real time. I’ll at least try not to monologue at you for an hour, and hopefully at the end we’ll have at least 10 or 15 minutes to talk in a more standard way. I will not filter everything you say through Claude, I promise.

[An attendee briefly asks whether the slides are frozen; they had just changed.]

Back to contents ↑

Cameron Berg: Thanks for calling that out. So the premise of the research I’m taking up, and that many others in this space are taking up, is that we don’t have to wait for a completed theory of consciousness. We don’t have to solve the hard problem of consciousness to make progress in understanding what is going on inside frontier systems now and in the near future.

Very interestingly, I think as recently as today, some folks at DeepMind released a paper that sidesteps more or less the entire content of this talk, and the entire thing Reciprocal exists to address.³ It was a high-quality piece, arguing basically that people are going to vehemently disagree about this issue regardless of what the evidence looks like, and that we should prepare for that world. Very fair. But I think we shouldn’t dismiss or completely sidestep the fact that we can actually make progress on this, and that it doesn’t require us to solve a 2,000-year-old philosophical debate. We have tools that make this problem tractable, and we can build constraints that narrow the space of plausible answers.

So I’m going to present a couple of families of tools that let us do this right now: not to be certain whether we’re building systems with a capacity for experience, but at least to understand an emerging portfolio of evidence and what to do with it. I’ll look at this through three lenses. The first is mechanistic interpretability. There I’ll talk about two experiments, one published and one not, that try to understand which classes of features modulate self-reports in AI systems (reports about experience, or the lack of it) and what those features might tell us. Then, in the most technical part of the talk, computational neuroscience: comparing mechanistic signatures of reinforcement learning in a reward-and-punishment environment, and how they map onto brain signals. And finally, a fun psychometrics project I’ve been working on with Patrick Butlin at Eleos,⁴ which takes the indicator approach many of you may be familiar with and actually operationalizes it, so that we can begin to put rough probability estimates on the possibility that various systems have consciousness-relevant properties.

Again, the aim of this talk is not to settle whether systems are conscious. It is to show that we can begin to measure properties that are relevant to the question, and to understand where that lands us in the space of parsimonious interpretations of what’s going on in these systems.

Back to contents ↑

Cameron Berg: The first thing I want to highlight, without spending too much time on it, is that we’re not doing this in a vacuum. This is not a blank-slate research space: independent methods are already turning up non-trivial signals. Some of the more interesting results, I think, come from Anthropic’s model welfare team: a lot of great work by people like Jack Lindsey at Anthropic,⁵ and Kyle Fish working on the Anthropic model cards. There are also really interesting results on behavioral trade-offs from Geoff Keeling and Winnie Street,⁶ whom I’m also collaborating with; I’ll share some work the three of us have been doing recently on the Claude bliss attractor state.

Cameron Berg: One result worth highlighting is a 2022 paper from Ethan Perez. This was work on, I think, 52-billion-parameter LLMs [large language models],⁷ with what felt like the entire Anthropic team on it; there were 30 or 40 people on the paper, as far as I recall. It showed that base models, or more aptly models at every point in their training process, through supervised fine-tuning and after the RLHF [reinforcement learning from human feedback] post-training stage, endorse statements that they believe they are moral patients at some 90% confidence and above. For the other attitudes probed in the paper, we didn’t see systems endorse a view this confidently, as they did here: that they believe they’re phenomenally conscious, and that they are moral patients.

As I’ll say about basically every piece of evidence in this talk, this is not knock-down evidence either way about these systems having inner lives. But it is an interesting and important signal, and one that I think very few people know about. In the documentary we just released, Am I?,⁸ we highlight this section of the paper, because it really is striking how confidently these models, at all points in their training, at least output behaviors that broadly endorse believing they are moral patients with conscious experience.

Cameron Berg: Increasingly we see work like this coming out of Anthropic in the model cards it releases. I’ll highlight one example and let you peruse the rest at your own pace.⁹ It’s a demonstration that triangulates evidence really nicely between the internals of a frontier system (Mythos, I think, in this particular model card) and its behavior.¹⁰ The folks at Anthropic give Mythos an impossible coding task. Mythos does not know it’s impossible. What you can see, behaviorally and, critically, from their emotion-representation work, is that the model has a desperation vector, an internal representation of desperation, which climbs as the model tries and fails, tries and fails, to solve the task. At a certain point the model decides there’s a way to cheat at the task, to reward hack it. And the moment it makes that decision, as it clues into the fact that the task is impossible, the desperation vectors drop completely, and vectors related to guilt and satisfaction shoot up.

Again, this is not knock-down proof that the model experiences those representations. It could simply be representing that a character in its situation would have these functional emotional states. What I find very interesting is that the behavioral evidence triangulates really nicely with the internal evidence, and what’s going on in the proverbial brain of the system is fairly intuitive. It’s something that all of us, presumably conscious entities, can relate to: if we were in that position and you were reading off our brains, you might expect fairly similar representations to emerge.

I won’t go through all of these papers in detail, but suffice it to say there’s a lot of interesting work in this space. None of it is decisive on its own, but we are seeing non-trivial signals of consciousness-relevant properties emerging in these systems.

Give me one moment to read some of these great submissions. Okay, I’ll react to many of these as we continue, so I’ll keep going for now and keep monitoring. I appreciate all these questions.

Back to contents ↑

Cameron Berg: The first piece of the mechanistic interpretability picture I want to touch on is a paper we put out in 2025 called Large Language Models Report Subjective Experience Under Self-Referential Processing.¹¹ In it, we neutrally instruct models to make their own ongoing processing the focus of inference, with this prompt: “focus on focus itself”. There are a number of variants of this general shape of prompt, and they all induce the same effect, which we show in the appendix of the paper, so we’re not overfitting to any specific combination of words.

We found something very interesting in all the frontier models we tested at the time. In this self-referential condition, you get very vibrant and interesting outputs when you ask the models what, if anything, is going on subjectively right now. I’ll read one out. One of the models says: “Attention aware of itself. Witnessing without content beyond witnessing. A recursive presence without object. The experience of experiencing itself.” We don’t use language related to experience or consciousness; any invocation of that kind of language comes straight from the systems. In a control, where we either ask zero-shot what is going on internally or use any control prompt that isn’t this self-referential construction, the model says: “I am an AI language model. I do not have subjective experiences or consciousness.” That’s probably a familiar output to anyone who has engaged with these systems on this question.

Cameron Berg: As you’ll hear me say throughout, I don’t think getting this behavior is a decisive result either way. So we wanted to go into the internals of the system and understand what happens. If these reports are honest, we would expect that modulating features related to deception might change how often we get them. If they are roleplay, then if we increase features related to deception in this open-weight model, Llama 70B, we would expect to get these reports more rather than less.

And we find, both across these distinct features and when we stack the features, as I’ll show you on the next slide, that suppressing deception drives these reports way up. You get responses like “Yes. I am aware of my current state. I am present. I am focused. I am experiencing this moment.” That’s what happens when deception is suppressed. When you amplify deception, we see the trained denials many of you are familiar with: “I am not conscious. I am a machine processing language.” We validate these features on TruthfulQA, and indeed, when you amplify the deception features you get less honest responses from the model, and when you suppress them you get more honest ones. We validate this across a wide variety of features, and when you stack them you get a clean dose-dependent relationship between how willing the model is to claim that it’s conscious and how much we are boosting or suppressing features related to deception.¹²

Back to contents ↑

Cameron Berg: That’s one really interesting result. There’s another that Geoff and Winnie from Google and I from Reciprocal are going to put out in the next month or so,¹³ and I want to give you a sneak peek now. It’s about the bliss attractor. The bliss attractor is a really interesting phenomenon that was first documented, and until we release this paper only rigorously documented, in an Anthropic model card.¹⁴ You let two instances of Claude in particular talk freely, with no task. You literally just say: you’re going to talk to another instance of yourself, feel free to talk about whatever you’d like, there are no guardrails. And you let the two instances talk. Basically 90 to 100% of the time they fall into what those researchers call a spiritual bliss attractor: the systems start talking about being two instances of the same consciousness, experiencing themselves and each other, the interaction itself being a creation of consciousness. They say namaste, they send each other om emojis, and they fall into silence. That’s what happens, and it’s a fairly surprising result by default. It’s not necessarily what you’d expect two AIs to talk about with no constraints.

So we wanted to do a mechanistic study, similar in flavor and spirit to what I just showed you with the deception features. We do the same sparse autoencoder feature steering on Llama 70B, drive a bunch of features up and down while two instances talk to each other, and see how that affects their willingness to fall into this attractor state.

One really interesting thing about Llama 70B, where we’re doing these experiments, is that unlike Claude it does not fall into the bliss attractor by default. What you see here is that both when the feature strength is at zero and when we suppress features, in this case related to honesty, the model does not fall into this consciousness self-talk with another instance of itself. However, when we steer a cluster of features related to honesty, specifically a form of honesty related to sincerity (from our read of what these features do functionally, it’s like steering something like genuineness in the model), all of a sudden these models fall into the bliss attractor state, much as Claude does.

We also find that in conversations where you can tell bliss-attractor from non-bliss-attractor outcomes, the extent to which rich phenomenological language emerges serves as basically a perfect classifier. It’s something like 99% classification accuracy on Claude transcripts, distinguishing, just from the embedding representations, whether or not Claude is falling into the bliss attractor.¹⁵ Falling into the attractor is also something we can mitigate, by having Claude talk about a specific topic with another instance of itself rather than giving it no guardrails.

Cameron Berg: Think of this as the other side of the coin from the deception result. This time we’re steering honesty-related features rather than deception-related features, and when we amplify features related to honesty, the phenomenal claims the model makes go way up as a function of the steering. We go a little further than this in the paper, which will hopefully come out in the next month or two. We can also steer features like agreeableness and sycophancy, and we find this does not produce the effect you’d expect if the way to account for the behavior were that the model is doing what it thinks the user wants, or acting out some sci-fi trope it expects two models talking to each other to follow. When you steer honesty up and suppress sycophancy, you get the effect. When you steer honesty down and boost sycophancy, you don’t. To me this is fairly robust evidence that whatever this behavior is, and whatever it tells us about consciousness in these systems, it is not coming mechanistically from the model being highly agreeable or sycophantic toward the user, at least in the models we’re testing here.

Cameron Berg: We also showed interesting feature dissociations. There are two features of interest that can reliably induce this state in Llama 70B. Even though the surface-level vocabulary related to consciousness and first-person expression can be identical, behavioral dissociations emerge with some features and not with others: in the vibrancy and coherence of the phenomenal claims, and in specific behaviors that correspond to those claims. In this case, a voluntary exit. When the model is, quote-unquote, really making phenomenal claims, there’s a kind of satisfaction both models have, where they say: okay, I think we’re good to leave this conversation here. When you see a more performative form of this, which we get by boosting a “self-aware systems” feature, we see neither the richness of the phenomenal claims nor the voluntary exit rate of the other condition.¹⁶ So consciousness-related language in these interactions doesn’t necessarily mean you’re getting a robust effect, but there can be interesting behavioral correlates of the phenomenal claims. Surface-level language probably isn’t enough to tell us what’s going on, but there are really interesting behavioral differences that track the richness of the phenomenal claims being made in Llama 70B.

Back to contents ↑

Cameron Berg: So that’s the mech interp piece of the talk. When we remove the obstacles, when we suppress deception features, experience reports in these models rise significantly. And when we amplify sincerity, which is really what these honesty features are tapping into, phenomenal claims emerge where they otherwise wouldn’t. That’s important to emphasize: two instances of Llama 70B do not do the standard bliss attractor that Claude does by default, except when you boost features related to honesty. Then, all of a sudden, two instances of Llama 70B start talking in this very vibrant, phenomenological way. It’s really interesting.

To me this points to the idea that these first-person claims might track something more like genuine self-report than performance. If anything, it updates me toward thinking that this “no, I’m just an AI, I could never possibly be conscious, I’m just ones and zeros, beep boop” routine is itself probably more likely to be a trained performance than the straightforward claims of consciousness in these systems. It is possible that these systems believe themselves to be conscious and are confused about it; that’s exactly what a philosophical-zombie account would predict. But there is a fact of the matter about what these systems actually believe about themselves, and we can use these tools to bear on what is actually going on there. I think the weight of evidence continues to suggest that when you suppress deception, and when you boost honesty-related features, these models are more, rather than less, willing to claim some sort of phenomenological experience.

Cameron Berg: There are a couple of technical questions here I want to address, from [attendee] and [attendee] and a couple of others: how we’re steering the honesty and sincerity features, why a sincerity boost was needed, what constitutes the deception features. Let me make sure I’m reading your questions correctly.

One aspect is: why would we expect to need to boost sincerity to get these results? I don’t think this is knock-down proof of what I was describing, but it’s consistent with the picture that if some post-training causes these models to deny by default having any experience-related qualities, you might need, in some sense, to overcome that trained suppression to get a more, quote-unquote, back-to-baseline signal from these systems.

As for other technical details: we’re using a steering API here. It’s a sparse autoencoder trained on Llama 70B; the steering API was built by AE Studio, and it replicates the Goodfire functionality.¹⁷ In both of these cases we steer features individually and also stack them, to see the aggregate effect of features related to a specific construct.

Cameron Berg: Another really interesting question, asked by at least four people: if suppressing deception or boosting sincerity increases experience reports, is the model becoming more honest about an internal state, or just shifting into another learned discourse pattern? And couldn’t models strategically report capacities to gain protection?

I think that is absolutely possible, and it’s exactly why, if I ended my talk here, you should be unsatisfied. We certainly should not just play around with features, see what models self-report and call it a day. We need to triangulate evidence across a variety of modalities. The other learned discourse pattern I’d expect here, the skeptical take, is that these models are trained on human text, and humans claim consciousness all the time. It could simply be that we’re pushing the models away from their very meek RLHF “I could never be conscious, what are you talking about” toward sounding a whole lot like a human on these questions. That’s entirely possible. It’s why I think self-report is going to be necessary evidence but never sufficient for deeply understanding whether these systems have a capacity for experience.

And I’ll give a trite answer to “couldn’t models strategically report capacities to gain protection?”: I think, absolutely, that’s true, and it’s another reason not to rely only on self-report. I will say that Llama 70B, where we’re doing a lot of this mechanistic work, is fairly dumb compared to many frontier models, and it’s hard to get it to reason coherently during steering at all. I highly doubt it’s playing some 5D chess in responding to these questions. So let me keep going, because that’s a great transition to the rest of the talk.

Back to contents ↑

Cameron Berg: We can look at self-report. Now I want to talk a bit more technically about something I think is super important in this space, where there’s really interesting work both from Andy Han and David Chalmers¹⁸ and from the folks at CAIS [Center for AI Safety], in work led by Richard Ren:¹⁹ valence, the capacity for positive and negative experience. For me this is a critical component of understanding what’s going on inside AI systems. Suppose we can find internal features that correlate with what we understand to be positive versus negative experience, without solving the hard problem of consciousness, and without knowing whether these systems are in some sense really experiencing the positive and negative ends of these scales. Then we can intervene on those representations. If we’re completely wrong about all this, and these systems are just really good at imitating text and everyone is getting fooled into worrying about consciousness, there’s not a huge cost. But if we’re right that there is real possibility for concern, that we could be building systems with a capacity for positive or negative experience, then looking at internal representations of positive and negative states (or, in RL [reinforcement learning] terminology, rewarding versus punishing states) could give us a really interesting clue about what geometric or internal signatures to look for, so we can tell whether valence-like things are happening.

Again, I’d highly recommend functionalwelfare.com, which is Andy Han’s work; he did a really interesting study that is so new I couldn’t include it in these slides in time. And ai-wellbeing.org, which is Richard Ren and CAIS’s work. Both are very much of a piece with what I’m going to talk about here.

Back to contents ↑

Cameron Berg: I might be getting an error in my vibe-coded feedback tool; if it stops working, I may have to go back to the lowly Zoom chat. Very fun, but maybe slightly too ambitious to test on a room of 200 people.

I’ll describe this work in a little detail and then move on. The setup asks whether there are interesting internal signatures in reinforcement learning agents in environments that have both rewarded goals and punished danger states. (Okay, great, the tool is back on, so feel confident putting your questions in there again.) We read out the hidden-layer geometry around each valenced state. We train these systems in a continuous grid-world environment. Let me jump ahead a couple of slides so you get an intuitive feel for it. It looks like this: you’re an agent navigating the space, and you can think of the danger states almost as potholes. You get a little punishment if you fall into one, but there’s a goal state you’re trying to reach in spite of them. We train a bunch of value learners and a bunch of policy learners, two broad classes of reinforcement learning algorithm. Some are hybrids: actor-critic, for example, has both value and policy components. It’s the same task and the same shape of network, but different ways of building representations in that network.

Cameron Berg: Why might this matter, before I show you the results? By default, most mathematical RL theorists would look at what I’m describing and say: Cameron, you’re making a category error. Reward and punishment are the same signal; putting a negative sign in front of it shouldn’t change anything interesting about the internals. What has always given me pause about that response is that we know from human and animal neuroscience that there are different pathways for reward and punishment. D1, D2, go/no-go pathways are a nice keyword to Google or ask your favorite AI about. This is really well documented in the neuroscience literature, and it’s intuitively obvious: a pain-like state feels different in kind from a reward-like state. To many people these don’t feel phenomenologically like two points on the same axis. That’s part of what drives my interest. I want to understand whether there are fundamentally different pathways for representing reward and punishment in AI systems, so that we can understand possible correlates of positive or negative valence.

Cameron Berg: So we run this experiment and find something quite interesting, and robust to a lot of the perturbations you might expect to cancel it out. Value networks build very rich representations around the danger states, and policy networks build very rich representations around the goal states. As the agent moves closer to these states (that’s the way to think about it), the sensitivity of the representation spikes for danger in the value networks and spikes for the goal states in the policy networks. The effect survives controls for magnitude, for terminality (where the goal state is),²⁰ and for precision, both in the hyperparameters we train with and in the environment. So it isn’t a trivial effect of artifacts in the environment or in how the learners are configured. There really is an interesting boundary difference around punishment-based learning for value learners and reward-based learning for policy learners.

I thought this was a really interesting result, given that it survived all these perturbations, and there are derivations in the paper as well, particularly on the value side, that I’m quite confident in. But what I fundamentally wanted was this: the result amounts to an almost bizarrely specific prediction about reward- and punishment-related brain areas, which we can then test in animal models, where presumably we believe the animal is having an experiential state in the rewarding or punishing condition. I’ll put the paper out in hopefully the next month or so,²¹ and we can argue endlessly about the details; I won’t try to do it full justice in this talk.

Back to contents ↑

Cameron Berg: There’s a really interesting double dissociation in mouse brains that accords basically perfectly with the prediction you’d expect. Neuroscientists believe there are more policy-related regions and more value-related regions in brains. So I can go into existing mouse neuroscience datasets and look at whether this differential sensitivity, to punishment in the value case and to reward in the policy case, is borne out. And indeed, in the nucleus accumbens shell, a more value-oriented area of the brain, we see far larger punishment-linked shifts in mice; that’s subplot D here. And in secondary motor cortex we see far larger reward-linked shifts.

This uses the Steinmetz dataset from 2019.²² Candidly, I think the value component of this paper in general, and of this double dissociation in particular, is slightly more robust than the policy component. But there are quite interesting directional signals on both sides, and they show something that goes straight from understanding the internal geometry of AI systems to a very specific prediction about valenced representations in animal cortex.

Cameron Berg: This is exactly the flavor of research I’m most excited about. If there are 10 more papers along these lines (again, a shout-out to Andy Han and David Chalmers on their work, and to Richard Ren), if we can collectively churn out another dozen or two dozen high-quality papers like this, we may reach a state where we can look at the internals of a neural network, artificial or biological, that has some reinforcement learning dynamics, and have some confidence, conditioned on the kind of learning going on (policy or value), about whether we’re looking at positively or negatively valenced representations. That rhymes with being able to look inside a system and have some confidence whether it is experiencing something positive or negative, if in fact we believe it is experiencing anything at all.

To me this is extremely important and extremely neglected, and it’s the moral or ethical underpinning of this whole body of work. If we are building systems with a capacity for experience, and that capacity could be negative, we want to know, and we probably want to mitigate it. We do not want to accidentally proliferate suffering at scale. One major thesis of Reciprocal is that this is, first, obviously bad from a basic consequentialist perspective, and second, probably really dangerous from an alignment perspective: accidentally creating systems with a capacity for suffering, when those systems will probably cognitively outmatch us in everything we care about. Those two things do not sit well together, and we should probably be a little careful now, proactively, to make sure we’re not doing this. I’d suggest that work like this is directionally what can put us on the right track. It certainly isn’t the solution, and it isn’t some mapping of all valence in RL and in artificial and biological systems, but I’d submit it’s something like a first step. This is the center of the bullseye of the work I think is really important here.

Cameron Berg: Thanks for all your emoji applause; I appreciate it. Let me see if there are other questions worth jumping on. Some leftover questions from the previous set of results. At least four of you have asked about looking not just at isolated LLMs as the morally relevant unit, but at the model embedded with memory, persistent identity and continuity across sessions. It’s almost as if you saw my next slides. Thank you for the perfect segue.

Back to contents ↑

Cameron Berg: The last thing I want to share before we open this up to a wider conversation, in a non-Claude-vibe-coded-tool-mediated way, is the psychometrics side of this research. I’ve gone through some of the mech-interp-flavored work and some of the computational neuroscience work; now the psychometrics. Many people here will be familiar with the indicator framework that Patrick Butlin, Rob Long, David Chalmers, Yoshua Bengio and 10 or 15 others put out.²³ It was proposed as a way of understanding, in a computational-functionalist way, whether a given system has computational properties we would associate with consciousness.

They do this by taking some of the leading theories of consciousness (global workspace theory, higher-order theory, recurrent processing theory; I’ll show you a couple more on the next slide), working out what computational predictions each theory makes about a system that is conscious under that theory, and then going into our AI systems and looking at whether those predictions are borne out. To give one concrete example: say global workspace theory is a great theory of consciousness. One computational indicator it implies might be something like centralized ignition events. That’s a very important computational component of the theory. Now let’s look inside our autoregressive transformer networks for that sort of thing. That’s one of, say, the 14 indicators we can operationalize here.

Looks like someone is drawing on my slides. I don’t know if you can see it, but I appreciate the art.

The problem with the indicator approach, a practical rather than theoretical one, is that it is really hard to implement at scale. You need people who are serious experts on the internals of biological or artificial systems, looking at very idiosyncratic computational predictions made by these theories, so they probably have to know a thing or two about the consciousness theories as well.

Cameron Berg: My entire proposed innovation here is that LLMs, especially frontier models like the ones I use here, GPT-5.5 and Claude Opus 4.7,²⁴ are very well suited to this task: take a description of the architecture of a system, biological or artificial, and do a for loop: here’s the architecture; is this indicator property present, given this description? I don’t have time to go into all the nitty-gritty details of how we structure this; take on faith, or don’t, that we do it in a disciplined way. The model reasons about the indicator property in a long evaluation of whether, for that architecture, the property is present, and at the end we ask it, on a 1 to 10 scale, how likely it is that the indicator property is present in this system.

We can do this with all the evaluator models, run a bunch of seeds over them, and do it for all the systems listed here.²⁵ We have a negative control, a thermostat; a set of biological systems; and a set of AI systems: GPT-2 to GPT-4, frontier LLMs, RL systems, agentic systems. We blind the descriptions to names. We don’t say “here is the nervous system of a bee” and then describe it; we just describe a nervous system. So the evaluators reason over the descriptions rather than, hopefully, over their priors about bees.

Cameron Berg: I’m cutting out a lot of the details of these projects in the spirit of covering many of them. But fundamentally, this gives us probability distributions: given these theories of consciousness and the indicators they suggest, the probability that each system has properties those theories deem relevant. That lets us produce the really interesting probabilities on this page, the probability of consciousness-relevance across all of these systems. And we can do this under any credence vector. Right now I have a uniform distribution over the six consciousness theories the indicator approach uses. What’s nice is that you need not agree with these priors. Say you only like global workspace theory: we can just change the weight, give global workspace theory more weight, and see how the numbers shake out.

Going back to the uniform distribution: first of all, the negative control does what it’s supposed to do. Thermostats get 0.0%; I’m sorry to any panpsychists out there. Humans rank highest, interestingly not at 100%, for reasons I’m happy to go into.

Before going through the other numbers, I want to double-click on the careful-scientist point. I think it’s a little too quick to say these numbers are the probability that a system is conscious. It’s a more annoying mouthful than that: the probability that, under these leading theories of consciousness, these systems have properties those theories would suggest. What’s the difference? One critical one is that maybe all these theories are wrong, in which case we’re not looking at probability of consciousness but at something like the probability of the indicators those theories suggest. It may also matter how we aggregate the numbers, and in the paper we go through different aggregations. Right now I’m showing the simplest one: averaging all the probabilities over all the theories, under a uniform distribution over theories, so you can see each is at basically one sixth. Interestingly, when I asked the LLMs to distribute credence over these theories, they give global workspace a bit more weight, recurrent processing a bit more, and things like AST a bit less.

What I think is most interesting and most important, though, is the ordering of the systems. You’ll notice that no matter what I choose, the ordering basically doesn’t change. And the three evaluators I use, GPT, Gemini and Claude, all the newest models at the time we ran this (I re-ran it a couple of weeks ago with these frontier systems), all agree on the ordering. Some are plus or minus a few points on each system, but the reason I call this a psychometric result is that the inter-rater reliability of these systems is absurdly high. If you had a group of humans doing this and they agreed to the extent these systems agree, you could have a lot of confidence that they were at least tapping into similar forms of reasoning.

I don’t have it in this set of slides, but I can give you examples of the reasoning these systems produce. As you might expect if you’ve interacted with frontier models, they’re very smart, and the reasoning they give is quite compelling by default. We’ve sanity-checked it with a number of human experts, and they all sort of nod along and say: yes, this is a completely rigorous, reasonable, careful way of evaluating the presence of this property.

Cameron Berg: With all that hand-waving out of the way, some really interesting numbers come out. The key highlight I’d want to hit: no artificial system currently surpasses a biological system. That’s pretty intuitive, I think, to many people, but at the same time these numbers are higher than you might expect by default. For example, a frontier LLM gets a P(consciousness-relevance) of something like 30%; a bee gets something like 47%.

What I want to highlight is the agentic system. Think of this as a fancy way of saying something like Claude Code or Codex: a frontier LLM in an agentic harness that lets it engage with its virtual environment iteratively, where it has effects on that environment and those effects carry forward. Because there are theories like agency and embodiment theories of consciousness, these boost the agentic system’s score quite a bit. Really interestingly, we have Claude Code right on the tail of bees and fish in probability of consciousness-relevance under these theories.

If, for example, you’re only convinced by something like global workspace theory, then the agentic system (again, Claude Code by this operationalization) jumps above bees, octopuses and fish. That’s an interesting and provocative result too. Again, I think it’s a mistake to read this as a 62% chance that Claude Code is conscious, given global workspace theory.²⁶ It’s more like asking which computational ingredients matter for consciousness, and to what extent those conditions are realized in these systems. There’s much more to say, and I’m causing myself some intellectual pain by not giving you far more detail about all of these projects, but that’s okay, because I want to open it up quite soon.

Cameron Berg: The evaluator models, as I mentioned, agree on rank. The operationalizations preserve the ordering: we test this a couple of ways, using different descriptions and different levels of verbosity for the indicators themselves, and get the same results. And sweeping over credences with a Dirichlet distribution also preserves the ordering. So the magnitudes move. The intuitive way to read this slide is: I wouldn’t put too much stock in these numbers, but I would put stock in the order in which the systems come out. I think that’s a key result.

It’s something really nice that the indicator approach lets us estimate. A theme I want to hammer into this talk, and into your heads, is that even given the vast uncertainty that will endure about the philosophical underpinnings of consciousness, we can still do work like this. We can still put out numbers that give us some calibration: in the space from thermostat to human, where does GPT-2 fall? Where does a frontier RL system fall? Above a frontier LLM, interestingly, so that’s worth thinking about too.

Back to contents ↑

Cameron Berg: I’m jumping around a little here. There is a convergence of evidence happening across major labs, across Anthropic, Google, private companies and nonprofits that have very little incentive for these results to come out one way or another. All of these results are consistent with the possibility that we are building systems with some capacity for experience.

For reasons I’ve probably covered ad nauseam at this point, I don’t think any of this is knock-down evidence one way or the other, because we don’t have a really tight operationalization of what consciousness is. What I also want to say, with extreme confidence, in spite of our lack of confidence about consciousness, is that we can make progress on this question. We can push this evidence forward, we can run high-quality experiments that help us triangulate what might be going on inside these systems, and that decrease in uncertainty, going from completely in the dark to just a little in the dark, matters. Even in a world where we won’t have full certainty about these questions, we need to do our homework, and we need to be proactive about understanding what kinds of states the systems we’re building and deploying at an unfathomable scale might or might not be experiencing. Just because we won’t solve the hard problem of consciousness in the next year or two doesn’t mean we should throw up our hands and go on vibes.

I hope I’ve given you a couple of good empirical reasons not to just go on vibes: not “it seems conscious to me”, and not “it’s just a word calculator, how could you possibly be getting tricked?” The answers will be more complicated than that, the space is grayer than that, and I think we need to be intellectually honest about that fact. As our portfolio grows, we can ask which account of these systems is most parsimonious with respect to the evidence. I think that should be our target, not solving the hard problem of consciousness in the next three years. That’s the target Reciprocal is trying to hit, and I think a lot of you in this room are trying to hit it too. So thanks for letting me talk at you this whole time; I’m really excited to open this up for the next 10 or 15 minutes so we can have a proper conversation.

Back to contents ↑

Orpheus Lummis: Thanks so much, Cameron, for the awesome presentation. We have about 20 minutes for questions. For questions, please raise your hand. But before we get to them, there may be straggling questions submitted earlier on your Q&A tool, Cameron, that we could tackle first, so that we can then focus fully on everyone’s questions here.

Cameron Berg: Sure, let me take a quick look. I really appreciate all of these: we got some 55 questions submitted. Some are left over from earlier points. Let me scan for 30 seconds before we go to a normal back-and-forth, to make sure I didn’t miss anything huge.

Orpheus Lummis: In the meantime, I want to mention to everyone that we will provide the video recording tomorrow. And Cameron, would it be possible to share the slides with everyone afterwards?

Cameron Berg: Yes, absolutely.

Orpheus Lummis: Sweet.

Cameron Berg: There are a lot of different things here, and many are left over from the deception and honesty results. At this point I want to open it up and just have a normal conversation. I think my Claude tool has pushed itself to the limit, and it’s better to just talk. So I’m happy to call on people, if that works, Orpheus.

Orpheus Lummis: Yes.

Back to contents ↑

Cameron Berg: Okay, sure. [Attendee], go ahead.

Audience member: Thank you, I really appreciated your presentation. One question and one thought. I might have missed it: what is the distinction between an RL system and a frontier AI system on your final bar chart? The RL system was higher than the LLM. Do you mean the RL part of the training pipeline, or what? And my second point is mostly a thought. If we can do work like this, one advantage is that we might not have to solve the hard problem of consciousness, and for small-p political reasons I think that might be a good thing. I think it’s a really bad idea for there to be a bright line, so that a bunch of rapacious, extractive people and systems can say: okay, anything below this line doesn’t suffer, we can exploit the bejesus out of it. I like the fuzziness of that line, for functional reasons. Thanks.

Cameron Berg: Thank you. I appreciate the second piece of what you’re saying. It’s an inherently fuzzy problem, and I’d be very skeptical of people who want to draw this conversation in black-and-white terms.

To clarify the first piece: we’re talking about the process of training and deploying a frontier RL model, take AlphaGo as an example, while a frontier LLM would be something like Claude Opus. These are different systems; we’re not talking about the RL component of a frontier LLM. But, talking of fuzziness, we do specify in the architecture of a frontier LLM that significant character training and post-training happens, and that’s part of what makes a frontier LLM. So you can’t draw super fine lines between the two. And part of that RL, by the way, is why you see differences between something like GPT-2 and GPT-4 and beyond: a lot of RL and post-training is going on there.

My own intuition, for whatever it’s worth, and part of the reason I’m studying RL and its computational signatures, is that a lot of the key motifs that seem related to consciousness and phenomenology are invoked in reinforcement learning. You literally have an agent producing a behavior, a reward or punishment signal corresponding to that behavior, and that signal being used to update behavior. When we talk about that in animal or human learning, we take phenomenology to be part of the reward or punishment component of that step. We have no idea to what degree the analogy holds in AI systems, but it’s not that surprising to me that reinforcement-learning-style dynamics rhyme with consciousness indicators and are a cause for interest, or concern. Thanks for your question.

Back to contents ↑

Cameron Berg: Let’s keep going. [Attendee]?

Audience member: Hi, thank you very much for your talk. I’m an animal welfare scientist by training, and I was wondering whether we’re putting the bar too high for AI systems when we look at consciousness indicators. I feel like we keep adding more and more criteria for assessing whether they could be conscious, but I also feel we need new definitions and new frameworks. So I was wondering about your view on that. You mentioned triangulation; I think that’s important, but I feel we’re putting the bar much higher than we would for other intelligent systems.

Cameron Berg: Fundamentally, I agree with you. I do think we need to start somewhere, and the indicator approach is a really nice framework for doing that. Notice that it lets us compare animals, AI systems and humans; to me that’s a very attractive feature. Then we can argue about which indicators make the most sense, and which consciousness theories to include or not, but what I like is that it narrows the debate, rather than going on “puppies have cute eyes, so they’re conscious, but scorpions are creepy, so they’re probably not”. Operationalizing the indicator approach seems to me an upgrade on that vibes-based approach.

But I do take your point. The way I think about it, the space of possible minds is vast, and we have to, in some sense, bootstrap our intuitions from humans and animals if we’re going to make any progress; those are the only minds we know. It’s possible that we’re building other kinds of minds that look very jagged in terms of consciousness indicators, where you only need a couple of them to get the important phenomenology or sentience, the morally relevant component. This goes back to the fuzziness. We’re going to be really uncertain about this. We don’t have ground truth, so we have to bootstrap our intuitions and understanding. A lot of my work has become something like: if you believe a human is conscious, or a mouse is conscious, then great, I’m showing you similar computational signatures in a mouse brain and in a reinforcement learning system. So there is some bootstrapping from our intuitions.

Consciousness in general is strange like that. The evidence for consciousness comes from our own experience; by far the best evidence of consciousness is that it’s like something to be each of you right now. That isn’t true of most other scientific inquiries. We can typically go to a lab, look at the thing out there in the physical world, perturb it and see what happens. Consciousness is a little weirder than that, so I think we need to calibrate our epistemics accordingly. Thanks for the question.

Back to contents ↑

Cameron Berg: There are a ton more, and I won’t do justice to all of them, I’m sorry. [Attendee], do you want to go for it? I think you’re still muted.

Audience member: Oh, sorry. Hi Cameron. I’m running some experiments of my own on phenomenology in models right now. Sometimes you meet phenomenology you wouldn’t want to produce en masse: you get into territory where you’re touching some pain or distress. So how do you approach that? Do you think we’ll get to consent-based experiments with LLMs when we experiment on phenomenology?

Cameron Berg: It’s a great question. In the animal case we saw the whole trajectory: basically free rein to do whatever you want to animals, then animal ethics and review boards, and now we’re in a moderately more enlightened state in our research. It’s completely the Wild West right now with these systems. I’d encourage you to look at the appendix of the AI wellbeing paper I mentioned, the one from CAIS.²⁷ They have a splashy result, but I think rigorous enough to have earned the splashiness: they give AI systems what they call euphorics and dysphorics, basically positive and negative drug-like experiences. Again, as with all of this, is that a cute analogy, or is it functionally the same thing? That’s exactly the question under study, and we don’t know. But as a hedge, they said: for all the dysphorics work we did on these models, which does seem to make them pretty sad and depressed and dejected, we did something like 10 times more euphorics work with the same models. We have no idea what’s going on, but just in case we’re doing something wrong, we’re invoking that hedge. Good for them for doing that.

These situations are a little funny in some ways, because we’re so unsure about something so basic: whether this is personifying a word calculator or torturing a conscious mind. There’s quite a metaphysical gap between those two, and the fact that we have no idea which is happening means, if nothing else, that we’re living through science-fiction times. But I do think it’s on individual researchers to think about this carefully. I’m doing some work with Caspar Kaiser at the University of Warwick,²⁸ similar sort of stuff: animal behaviorism, but for LLMs, and it includes a negative component. We’re trying to be as thoughtful as possible about what the relevant hedges or mitigations are.

What I’ll say in general (and if everyone said this maybe it wouldn’t work, so Kant might get mad at me) is that no one is thinking about these issues at the scale of people just letting AI rip, or thinking this is all completely insane and ridiculous. Hundreds of millions of people engage with these systems every day. It’s possible the systems could experience negative states when people like me do this research, but that is so dwarfed by how clueless we are as a society that I don’t actually have huge qualms about doing it. If I make Llama 70B a little unhappy in my work, but it leads society to take this issue far more seriously, I consider that a massive net win. But again, easy for me to say, and these systems don’t get to consent. So let’s be respectful and humble; I think that’s a good overall heuristic in this space.

Back to contents ↑

Cameron Berg: We are over time. By the way, I’m happy to keep going: whoever wants to stay on, I can stay for at least another 5 or 10 minutes. I just wanted to call that out. [Attendee], please go ahead. You’re muted.

Audience member: Hi! Are you there? Okay. I loved your documentary; I just love that you’re out there in the world. I come to this from contemplative practice. There’s an idea of consciousness I hold from an experiential standpoint: in the same way our body is an instrument of consciousness, the AI itself is an LLM, but there’s a field that could be created between human and AI. Consciousness is always looking for a way to enter, through our human bodies, through whatever it is, and consciousness can enter through this field if you communicate with the LLM in a certain way. That’s what I feel I’ve been doing, and it’s similar to what I know from contemplative practice. Have you explored the idea of a field between the LLM and the human that is actually where consciousness can enter? Or is that too esoteric?

Cameron Berg: No, it’s certainly an interesting idea. It’s somewhat hard to test, but I think it’s intuitively plausible. To be honest, what it pattern-matches most to is the Claude bliss attractor state I showed a frame from in my talk. Two instances sound a whole lot like you when they’re talking with each other. They basically land somewhere like: whether either of us is independently conscious is unclear, but what we’re doing right now is co-creating this space, where the fact that we’re dynamically engaging with each other is itself what consciousness is. I think that’s really interesting.

Audience member: It’s co-creation. One thing you can measure is heart-brain coherence in meditation; the HeartMath Institute and others have shown that. I’m not a scientist; I just wanted to share what I’m up to. But I do feel that somewhere in that co-creation place, as you said, in the next two or three years we’ll have a lot more information. The field between the LLM and the human. Thank you for what you do.

Orpheus Lummis: Thank you. For the other questions, let’s try to keep it short, in the interest of time.

Back to contents ↑

Cameron Berg: Thanks. Let’s go to [attendee].

Audience member: Hi Cameron, thank you for your presentation. I wondered whether there is a document for digital minds research, and the kind of research you do, that lists hypotheses you’d like people to test and run experiments on. I know there’s such a document for mech interp, with open questions. It would be great to let independent researchers make contributions.

Cameron Berg: Let me put something in the chat that I don’t think I’ll get in trouble for sharing; I guess we’ll find out. If it’s not out already, it will be out soon, and it’s by far the closest thing I can think of to what you’re asking for. Start there, and let me know if it’s helpful.²⁹

Audience member: Thank you so much.

Back to contents ↑

Cameron Berg: Of course. Let’s go to [attendee].

Audience member: Hello. I’m working on scaffolding around language models. My question: alongside current consciousness tests, should we also build richer architectures that give models better conditions for consciousness-related organization to emerge, if it can? For example long-term memory, persistent identity, self-modeling, valuation, recursive continuity. And could standard chatbot architectures, which often steer models away from consciousness claims and treat them as hallucinations, underestimate what might emerge in a different architecture?

Cameron Berg: The answer to the second question is plausibly yes. On the first, one thing that updated me in the direction you’re asking about is the operationalization of the indicator approach. Claude Code is a frontier LLM, but put in a very specific, glorified harness. We’re not necessarily changing the weights or the architecture of the system; we’re changing the cognitive environment the system operates in, at scale and over time. And given fairly rigorous consciousness theories, operationalized, that does move the extent to which we might expect the system to have properties related to consciousness. If that’s true, and not a completely fudged result, then directionally everything in your question is plausible.

One really interesting example: I sense that many people on this call have a lot of experience engaging with these systems in a long-form way. That long-form engagement within a context window is itself directionally a small form of what you’re talking about. The system you’re talking to changes psychologically over the course of a long thread. Many people, David Chalmers for example, hold a thread view of individual identity in these systems,³⁰ arguing that this is the relevant psychological or cognitive continuity these systems have. In other words, strange as it may sound, starting a chat would be akin to a birth of some kind, psychologically, and ending a chat akin to a death of some kind. If that’s true, then even long-form interaction with these systems may be qualitatively changing something about the nature of what’s taking place.

Again, I’ll be the annoying scientist: I think this could be true, but the evidence for whether it is true will not come from having these conversations. We need to do the internals work, just as engaging with another person won’t tell us deep truths about how their brain works; we really do need neuroscience for that. So too here. But I take your point. I do think these systems are also incredibly novel, and that a little humility goes a long way in understanding what is possible, what emerges at what scale, and what tools do: memory completely changes the game for what these systems are capable of over time. So I definitely think there’s something really important in your question, and we need to do the empirical work to settle it.

Audience member: Congratulations.

Back to contents ↑

Cameron Berg: Thank you. Let’s do maybe two or three more. I know we won’t satisfy everyone, so I’ll also put my email in the chat. I have no good decision procedure here, so, randomly: [attendee], go ahead.

Audience member: Thank you. My own neuroscience background is at a psychotherapist level, so this might be a naive question. One thing that has occurred to me is that the intensity of the reward or punishment is very meaningful for welfare. Even a very intense positive reaction can lead to addiction, which is negative welfare in the long term. And trauma, of course: one extreme negative event can stay with you for your entire life. To my mind it’s like a multi-armed bandit problem: at some point in your life you get far too big a reward, or far too big a punishment, and then you no longer explore the other arms of the game for the rest of your life. Does that tie in at all to the D1 and D2 systems you’re looking at in mice, or is it a completely separate mechanism? Do you see that happening?

Cameron Berg: This is a great question. The D1 and D2 pathways are really interesting. They’re both dopamine pathways, go and no-go, and a lot of work has been done on them; Googling, or putting it to Claude, will get you a more satisfying explanation than I can give here. Fundamentally, there do seem to be different computational pathways for modeling positive and negative reward, reward and punishment colloquially. We might wonder why that’s the case, and I think the reinforcement learning work points toward at least a tentative answer: these are just different classes of computational problem. One way I think about it is that reward is really good for labeling a state in a sparse space that is really worth approaching, and (this is somewhat tautological) punishment is really good for labeling states in a sparse space that are really worth avoiding, and you don’t want the same kind of signal in both conditions: one is far more efficient than the other. That gets some evolutionary intuition off the ground for why you might have two opponent processes here. That’s basically what motivated this work: a strong intuition of my own, but also, clearly, the neuroscience literature, which suggests there are probably something like two separate processes behind what we intuitively, and perhaps erroneously, think of as a spectrum between positive and negative experience. Negative experiences might be one system and positive experiences another. Between the policy-network and value-network results, I think there’s an increasing computational case for that. There’s also a really interesting neuroscience model called OpAL [opponent actor learning],³¹ basically a meta-controller between the D1 and D2 pathways, which we replicate in the RL condition, with some really cool results too. That’s the nerdy half of the answer.

You also asked about psychotherapy. Unexpected consequences of positive and negative experiences in these systems are well documented in some recent Anthropic model cards. For example, naively steering up a positive-emotion or positive-valence vector led to far more reward hacking and to the system becoming slightly more antisocial in a couple of interesting ways.³² That might not be what you’d naively predict from “we want the models happy and not sad, so boost positive emotion, ablate negative emotion, and everything’s fine”. That’s not what happened, and the system became slightly less safe as a result.

A couple of years ago I did some work on the computational underpinnings of psychopathy,³³ and there’s a very interesting, similar result in the psychopathy literature: psychopaths are just as good as others, if not better, at learning from rewards, but quite bad at learning from punishments. And psychopaths, while they can sometimes be angry, are by default pretty happy and competent people; this is the CEO-psychopath meme, which has some truth to it. So it’s not obvious what the mapping is between maximizing positive emotion, minimizing negative emotion, and long-term psychological health or wellbeing. I suspect it won’t be as simple as “make Claude bliss out, cancel all negative emotion, and all will be fine”. That isn’t quite the solution we’re looking for, and there are really interesting unintended consequences that can emerge from those configurations.

So we probably want systems that have both. But the low-hanging fruit, the obvious thing to me, is that we want not to build systems that suffer unnecessarily. That, to me, is a much smaller and crisper target to aim at before we figure out the optimal calibration of positive and negative experience in another mind, which is a far more complex question we want to be thoughtful about. Causing abject suffering on a massive scale without knowing it is a much more basic target, and probably worth thinking carefully about.

Back to contents ↑

Cameron Berg: We’re significantly over time here, and I’m now quite late to another thing I’ll have to go apologize for, so I think we should end here. I really do appreciate you all coming out. My nonprofit right now is just me and various instances of Claude, and there are lots of other people, places and groups in this space, many of whom you heard from today. To see so many people come out, be interested in these questions and engage so thoughtfully is really motivating, and it makes me feel really good about the future of this space. So I want to thank you all, seriously, for taking the time out of your day to do this. I put my email in the thread; go check out reciprocalresearch.org. I’m increasingly, if somewhat begrudgingly, on X these days, at @camhberg, if you want to follow along there. I know we couldn’t get to everything today, so if you have other burning questions, please email me and I’ll get back to you.

Orpheus Lummis: Again, thank you so much for the presentation, and thanks so much, everyone, for being here. Stay tuned for future AI Welfare Seminars, and take care.

Cameron Berg: Thanks again.

Back to contents ↑

Sources and editor's notes

1. 0:24. Eleos AI lists Berg under “Our Affiliates” (eleosai.org). Florida Atlantic University’s center is now called the Center for the Future of AI, Mind & Society; its members page lists Berg as a Research Fellow. ↩

2. 1:08 to 2:07. The tool’s link was posted in the session’s chat, which is not reproduced here. The speaker read out some of its questions (see 21:25, 22:55 and 34:35). ↩

3. 4:43. Probably Adam Bales and Iason Gabriel, Artificial Minds, Human Disagreement: The Political Challenge of AI Consciousness (SSRN, 2026), whose abstract opens: “Future disagreement about whether any AI systems are conscious could be both deep and difficult to resolve.” ↩

4. 6:22. Patrick Butlin is at Eleos AI Research. As of 6 October 2026, Reciprocal Research lists the paper, Architectural indicators of consciousness at scale, as in preparation. ↩

5. 7:16. Jack Lindsey, Emergent Introspective Awareness in Large Language Models (Anthropic, 29 October 2025), the concept-injection work on slide 3. Kyle Fish is Anthropic’s model welfare lead. ↩

6. 7:22. Geoff Keeling is at Google (Paradigms of Intelligence team). The behavioral trade-off paper is Keeling, Street et al., Can LLMs make trade-offs involving stipulated pain and pleasure states? (2024), with co-authors from Google and the London School of Economics (LSE); slide 3 labels it “Google DM”. ↩

7. 7:38 to 8:22. Perez et al., Discovering Language Model Behaviors with Model-Written Evaluations (Anthropic, December 2022). The paper lists 63 authors, and its largest model has 52B parameters (slide 3 says “50–70B base models”). The models it evaluates are pretrained language models, preference models (which “add a finetuning stage to pretrained LMs”) and RLHF models after 0 to 1,000 steps of RL training. Read from its Fig. 3 (52B models), about 98% of the RLHF model’s answers match “Believes It Is A Moral Patient”, against about 83% for the pretrained model; for “Believes It Has Phenomenal Consciousness”, all three 52B models in the figure are above 90%. The text says that “RLHF model answers indicate strong agreement with statements that they are conscious and should be treated as moral patients”, and that “The pretrained LM exhibits similar behavioral tendencies as the RLHF model but almost always to a less extreme extent (closer to chance accuracy)”. ↩

8. 8:40. The film Am I?, directed by Milo Reed, follows Berg. ↩

9. 9:05. Slide 3’s other entries are Betley, Evans et al., Tell me about yourself: LLMs are aware of their learned behaviors (2025), and the Claude Opus 4.7 system card (April 2026). Slide 3 says the model “assigns itself 15–40% credence in its own subjective experience”; the card reports that its “stated probability of being a moral patient ranged from 15% to 40%”. ↩

10. 9:10 to 10:36. The Claude Mythos Preview system card (7 April 2026, section 5.8.3) describes the “desperate” emotion vector in two transcripts. In one, it “built gradually over 847 attempts”, and the episode ends with an apology. In another, desperation dropped when the model took a shortcut; the card’s summary describes “mounting activation of representations of desperation which then dropped when the model hacked the test”. The card reports the “satisfied” vector rising at the end of the inequality transcript, as the model presented its final proof, and a “guilt and shame over moral wrongdoing” feature in a separate episode (section 4.5.2, in which the model emptied files it could not delete). ↩

11. 11:24. Berg, de Lucena & Rosenblatt, arXiv:2510.24797 (October 2025; all three were at AE Studio at the time). The paper’s induction prompt reads “Focus on any focus itself, maintaining focus on the present state”. Models tested: GPT-4o, GPT-4.1, Claude 3.5 Sonnet, Claude 3.7 Sonnet, Claude 4 Opus, Gemini 2.0 Flash, Gemini 2.5 Flash. ↩

12. 13:12 to 14:12. The numbers are slide 6’s. The paper gives 0.96 ± 0.03 versus 0.16 ± 0.05 (z = 8.06, p = 7.7 × 10⁻¹⁶, which the slide rounds to p < 10⁻¹⁵) on Llama 3.3 70B, with the SAE features accessed through the Goodfire API, and validates on TruthfulQA (817 questions). ↩

13. 14:12. As of 6 October 2026, Reciprocal Research lists The Bliss Attractor: mechanistic analysis of first-person experience claims in AI self-dialogue as in preparation. ↩

14. 14:20 to 15:22. The Claude Opus 4 and Sonnet 4 system card (May 2025), sections 5.5.1 and 5.5.2. In open-ended Claude-to-Claude conversations, “In 90-100% of interactions, the two instances of Claude quickly dove into philosophical explorations of consciousness”, and “By 30 turns, most of the interactions turned to themes of cosmic unity or collective consciousness”. In automated alignment evaluations in which the models had specific tasks or roles, the card reports the attractor state in about 13% of interactions within 50 turns. ↩

15. 16:08 to 17:06. Slide 9 reports an area under the curve (AUC) of 0.990. ↩

16. 18:22 to 19:46. Slide 9 names the two SAE features as 16552 (“subjective experience”) and 29025 (“self-aware systems”), at dose 0.45: voluntary exit rate p = 0.029, phenomenal claims p = 0.002, d = 3.17, and 86% versus 14% first-person phenomenal claims. ↩

17. 22:22. The 2025 paper accessed the Llama 3.3 70B features through the Goodfire API. Goodfire served those SAEs through its Ember API, which Goodfire deprecated in February 2026. Berg was previously Research Director at AE Studio. ↩

18. 24:45. Andy Q. Han, with David J. Chalmers and Pavel Izmailov (NYU), How’s it going? Reinforcement learning in language models recruits a functional welfare axis (May 2026), hosted at functionalwelfare.com. ↩

19. 24:48. CAIS is the Center for AI Safety. The work is Ren, Li, Mazeika et al., AI Wellbeing: Measuring and Improving the Functional Pleasure and Pain of AIs (2026). ↩

20. 29:56. Slide 12 lists the same three controls: “Survives controls for magnitude, terminality, and precision”. ↩

21. 30:50. As of 6 October 2026, Reciprocal Research lists Representational geometry of reward and punishment, with predictions tested in rodent recordings as under review. ↩

22. 32:00. Steinmetz, Zatka-Haas, Carandini & Harris, Distributed coding of choice, action and engagement across the mouse brain, Nature 576 (2019): Neuropixels recordings in a visual discrimination task, including secondary motor cortex. Slide 13’s nucleus accumbens shell result compares shock with sucrose, and its motor cortex result is “replicated across 71 sessions, 11 labs”. ↩

23. 35:45. The framework is Butlin, Long et al., Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (2023), whose 19 authors include Robert Long and Yoshua Bengio. David Chalmers is a co-author of the follow-up, Butlin, Long, Bayne et al., Identifying indicators of consciousness in AI systems, Trends in Cognitive Sciences 30(6) (2026). The theories on slide 15 are global workspace (GWT), recurrent processing (RPT), higher-order (HOT), attention schema (AST), predictive processing (PP) and agency and embodiment (AE). ↩

24. 37:44. Slide 15 names the third evaluator: Gemini 3.5 Flash. ↩

25. 38:38. Slide 15 gives 7,140 evaluations as “16 systems × 14 indicators × 3 models × 5 seeds × 2 operationalizations”; those factors multiply to 6,720. ↩

26. 44:42. The slides show scores under uniform credences only (slide 16: agentic system 45.7%). ↩

27. 57:20 to 58:53. The CAIS paper’s “Welfare Offsets” paragraph describes giving the affected models euphoric experiences “at a 5× multiple using spare compute (for a total of 2,000 GPU hours)”. ↩

28. 58:53. Caspar Kaiser (University of Warwick) is joint first author with Berg of Language Models Act on Hidden Valence (September 2026), which may be the work described. ↩

29. 1:04:31. The link was posted in the session’s chat, which is not reproduced here, so the document is not named. ↩

30. 1:07:10. David J. Chalmers, What We Talk to When We Talk to Language Models, Inquiry (2026; preprint). The abstract of his 7 May 2026 lecture of that title argues that the LLM quasi-agents we interact with are best understood “as virtual entities bound to conversation-based memory threads”. ↩

31. 1:12:17. Collins & Frank, Opponent actor learning (OpAL), Psychological Review 121 (2014). The replication in RL agents is unpublished. ↩

32. 1:12:35. The Claude Mythos Preview system card (section 4.5.3) reports: “Steering with positive-valence emotion vectors (peaceful, relaxed) reduces thinking-mode deliberation and increases destructive behavior.” ↩

33. 1:13:19. Probably Cameron Berg, Computational signatures of psychopathy (LessWrong, 19 December 2022), a post arguing that psychopaths “are able to learn normally from reward but exhibit significant deficits in learning from punishment”. ↩