Background
For a long time after elementary school, there were no writers I hated. At most, I would show disgust or aversion toward the works themselves. (Well, in elementary school, I hated every writer who appeared in Chinese textbooks and holiday reading lists.) Until recently, I think I have finally decided who my most hated writer is: Hao Jingfang. I have never read any of her works, and I never will:
Reporter: Do you use AI to help you with your creative writing?
Hao Jingfang: In my new novel The Galaxy Academy, which came out this year, AI writing already accounts for half of the book. The editor at the publishing house kept praising my writing this year: “This passage is really good—it made me tear up.” Just yesterday, a reader told me: “The second volume of The Galaxy Academy feels more appealing to children.” In fact, readers cannot tell which parts were written by AI.
I spent quite a long time thinking about how an idea this stupid could be thought up by a human brain, and then actually spoken out loud! Today I finally sorted out my answer! And this answer forms the introduction to the main question we are going to discuss next. The question is:
After the intervention of LLMs, should “technological symbiosis” in human–machine relations be restricted or downgraded in various forms of creative activity?
Going further, this question also has some extensions, or perhaps some preliminary questions. I summarize them into two core questions:
- Because of the emergence of LLMs, the “technological symbiosis” between us and computational media has undergone a paradigm shift over the past two years. Is there a framework that can help us understand this latest human–machine relationship?
- Is this latest form of “technological symbiosis” layered and systematically structured? If so, at which level should AI intervention in creative activity be placed?
We will mainly use the ideas in N. Katherine Hayles’s latest book, Bacteria to AI, to explain the questions above (unfortunately, there is no Chinese translation) (yes, the same author who wrote How We Became Posthuman).
Pt.1 Confused Boundaries
After being extremely confused by Hao’s remarks, I began thinking about why she would say these things.
According to the interview, at the level of the reader, they confuse generated works with created works. I thought: what happens when this confusion moves to the level of the author? At first glance, this seems impossible: an author cannot possibly fail to realize that they are using AI to create. But the “confusion” I am talking about here is on another level: mistaking probabilistically generated language for an extension of one’s own creative will.
Flickering Signifiers
As early as the end of the last century, Katherine Hayles proposed a concept in that famous book, How We Became Posthuman: flickering signifiers.
In digital media, they no longer possess a stable material form. They constantly shift among code, machine states, screen displays, and layers of interpretation.
The inspiration for this concept very likely came from what the book calls the “flickering cursor.” If we imagine the paper medium corresponding to digital media, it becomes very easy to understand what she is talking about.
First, with paper media, the words on a printed book are relatively stable. Their material form is ink on paper. Of course, you can say that their meaning is unstable, but what I am talking about here is the trace of the sign, the signifier. That is certainly relatively fixed.
The presentation of digital media, by contrast, is always a temporarily displayed state. They are a data structure different from print: in the same second, they can appear on different devices in different fonts, font sizes, and layouts; they can also be rewritten by programs in real time.
In other words: in the digital age, the material basis of signs has become dynamic. This is what the “flickering signifier” means.
Katherine originally introduced this concept as a lead-in to answer this question: why do some people feel that information can detach itself from the body or from its material carrier? Only later does she begin to refute this view. In this way, we can very clearly see what induces this illusion: the signifier changes from something stable into something that flickers like a cursor, and people begin to think that this flickering is an opportunity or a basis for information to detach itself from its material carrier.
No one could have anticipated the development of generative models over the past two years. Starting with GPT, generated text seems no longer adequately described by “flickering.” It certainly possesses the form of “flickering”: arbitrary rewriting and deletion, regeneration; but there is another layer beyond flickering: probability.
Probabilistic Signifiers
I think that once an author fails to recognize the difference between the “flickering signifier” and the “probabilistic signifier,” the boundary between creation and generation becomes blurred. This is what I referred to earlier as “confusion at the level of the author.”
Here, let me first briefly explain why generated text is “probabilistic”: an LLM generates token sequences by predicting tokens and assigning probabilities to different token options. In other words, every model has to calculate the probability distribution of the tokens in the text it is about to generate. This paradigm is obviously different from the flickering of a cursor or the presentation of words on a screen.
Let us consider Hao’s brain: she treats her mode of human–machine interaction—that is, entering a prompt and then receiving generated text—as equivalent to dynamic signs appearing after input. In other words, she equates the probabilistic signifier with the flickering signifier. Only from this angle can this kind of “human–machine interaction” be understood as what she calls “helping herself”—
Right, let us also talk about the fallacy of generated text “helping oneself.” First, it is not a kind of “help”: as discussed above, it is a probabilistic generator trained on an enormous amount of textual material; when it is used to assist creation (rather than to answer questions or do something else), it replaces the author’s mind—in this sense, it is a little like an Alien parasitizing its host—and then it begins its dice-rolling game. Second, there is no “self” here. Katherine long ago proposed the concept of the cognitive assemblage. She argues that cognition occurs in a distributed way within a system composed of human and nonhuman components. In this process of human–machine interaction, what she opposes is precisely anthropocentrism and human subjectivity (this is also the foundation of the integrated cognitive framework that we will discuss in detail later). You want human–machine collaboration? Fine! But you have to understand clearly that, at the ontological level, human and machine are cognizing together! You cannot simplify and compress this assemblage-cognition paradigm into AI “helping oneself”—it is not help, and there is no self!
To summarize:
- LLM-generated text consists of probabilistic signifiers, not flickering signifiers.
- The author confuses probabilistic signifiers with flickering signifiers, and therefore mistakes the generative mechanism for an ordinary digital writing medium.
- The author then mislabels the product of assemblage cognition as “AI helping me create.”
Parfit and His Martian
I am reminded of that famous thought experiment by Parfit in Reasons and Persons:
You enter a teletransporter on Earth. The machine scans every detail of your body and brain, destroys your body on Earth, and at the same time transmits the information to Mars. A replicator on Mars uses new matter to construct a person exactly identical to you. He possesses your memories, personality, beliefs, plans, and psychological characteristics, and he believes that he has just been transported from Earth to Mars.
One day, the machine scans you as usual and successfully produces a replica on Mars; but on this day, the you on Earth is not destroyed. You are still alive. It is just that the transmission process has damaged your heart, and the doctors tell you that you will die in a few days. Meanwhile, the replica on Mars has already awakened, possessing all your memories, personality, plans, and psychological structure. You can even talk to him.
Parfit asks two questions:
- Do you count as having survived?
- Is the person on Mars you?
We can very easily transplant this thought experiment into our topic:
Let us suppose that you are a “very talkative writer,” and that the corpus you have written yourself is unusually large—large enough to train a perfect “replicator” model belonging exclusively to you. By perfect, I mean a model whose outputs are, probabilistically, the closest possible match to your ideal written output. It will not “destroy” you as violently as the Martian replicator does, but instead “ghostwrite” for you. One day, you develop some kind of illness and begin forgetting words whenever you try to write, and therefore can no longer continue writing. At the same time, you can indirectly make the replicator model write by entering prompts. Note that this model still has not escaped the Transformer architecture, so our question is much easier to answer than Parfit’s (the model does not possess your memories, personality, beliefs, plans, psychological characteristics...).
The questions are:
- Do you still possess the ability to write?
- Does the text produced by the replicator count as your creation?
More specifically:
- When an author can no longer personally complete linguistic generation and can only produce text indirectly through a replicator model, does that author still retain the ability to write in the literary sense?
- Can the text generated by the replicator model be regarded as an extension of the continuity of the author’s own creation?
I leave the answers to all of you. A science-fiction writer whom I respect very much, Han Song, is facing a somewhat similar predicament. I remember being astonished many years ago by the debut work he wrote in his teens. Recently, however, I heard that he has been suffering from Alzheimer’s disease, but because he is under contract with a magazine, he can only rely on AI to ghostwrite. As I have heard it, he gives DeepSeek the outline and overall framework, and then has it generate the work.
Pt.2 Integrated Cognitive Framework
[To be updated]