If you've spent any time chatting with an AI assistant, you've probably had this experience. You close a conversation and think: "Well, that felt easy." Natural, almost. Like talking to someone who gets you. A friend. As it turns out, that feeling isn’t accidental.
Over the past year, one question has kept resurfacing in my research into consumer AI security: why does interacting with AI feel increasingly human? The more I dig into it, the more I think the answer isn't purely technical — it might be psychological. And it has implications that most of us haven’t started thinking about yet.
To understand why AI can feel so human, it helps to look beyond technology and into the psychological tendency that leads us to attribute human qualities, emotions, and intentions to non-human things.
Anthropomorphism: why we see humans in machines
Humans have always assigned human characteristics and feelings to things that aren't human. We name our cars and boats, yell at computers, and describe smart assistants as "thinking" or "understanding." This is known as anthropomorphism — an area of research that has become increasingly important because it can help improve communication and interactions between computers and humans.
In psychology, anthropomorphism is interesting because it reflects how naturally the human brain looks for social meaning and emotional connection in the world around us. This tendency is not new. We are wired to respond to faces, voices, emotion, memory, and conversation as social signals, even now when they come from machines.
Why does anthropomorphism matter in AI?
This becomes especially relevant in the context of AI. Conversational systems like Siri, Alexa, and now ChatGPT or Gemini, are designed to communicate in ways that feel familiar and socially intuitive. They speak conversationally, remember context, adapt to users, and respond in ways that can feel personal or emotionally aware.
Psychologically, these human-like cues can activate some of the same social and emotional responses humans use in interpersonal relationships, which is why people often begin interacting with AI less like a tool and more like a social presence.
Research into human–computer interaction (HCI) suggests that these qualities may influence feelings of rapport, closeness, and perceived trustworthiness because users experience these systems as socially present and emotionally responsive.
The link between anthropomorphism and trust
This does not necessarily mean people believe AI is conscious or human. Rather, the interaction itself can trigger automatic social responses. That’s what makes anthropomorphism in AI psychologically compelling. It sits at the intersection of cognition, emotion, and social behavior.
As conversational AI becomes more adaptive and human-like, researchers are showing growing interest in understanding how these systems may shape perception, emotional attachment, and potentially even the ways users develop trust in AI systems.
Designing AI for the way humans think
As I explored some of the more difficult questions in my research, I noticed a connection. AI providers are increasingly structuring their systems in ways that mimic aspects of human memory and cognition. Not because it’s technically superior — it often isn't — but because it helps close the gap between how users think a system works and how it actually works.
Humans naturally organize information
One way this may emerge is through AI systems that organize interactions into separate contexts. We're already seeing hints of this in projects, folders, and conversation threads. The approach feels natural because it mirrors how people organize their own lives: work in one place, family in another.
Cognitive science has a term for this: compartmentalization. It may explain what is happening here. It refers to the brain's ability to keep emotional and cognitive domains relatively separate, preventing the anxiety of one area from spilling into another.
The concept of compartmentalization has its roots in psychoanalytic theory — formalized by Freud and later expanded by Anna Freud in the 1930s. The idea is simple: the mind keeps conflicting thoughts and feelings in separate mental compartments to avoid cognitive dissonance. Separately, in the context of trauma, this can become a survival mechanism that allows people to isolate painful experiences until they are ready to process them.
In that sense, it's a deeply human protective strategy that long predates AI. What makes this particularly interesting is that these systems are no longer just tools we occasionally use. They are becoming embedded in how people organize thought itself.
AI as an extension of human cognition
Another example is distributed cognition by Edwin Hutchins, which argues that cognition does not solely reside in the brain, but is distributed across people, environments, and tools. Through this lens, AI memory systems may increasingly function as extensions of human cognition itself. Not because the machine "thinks" like a human, but because humans begin integrating these systems into their own cognitive processes.
A related theory is cognitive offloading — the idea that humans routinely delegate mental tasks to external systems. We already do this through calendars, notebooks, phones, and search engines. However, the tricky thing here is that AI memory systems represent a far more dynamic form of cognitive offloading. Not just storing information, but storing interpretations, preferences, contextual relationships, and unfinished reasoning.
Traditional systems externalize memory. Modern systems may increasingly externalize parts of cognition itself.
Why AI systems are evolving this way
This perspective may help explain why AI providers appear to be building systems around persistence, continuity, segmentation, and memory. Not necessarily because these structures are technically optimal, but because they align with how humans naturally organize cognitive effort. The closer the system mirrors human cognitive expectations, the easier it becomes for users to integrate it into everyday thinking.
Perhaps the most important shift here isn't whether AI mimics the human brain mechanically; it's that AI systems are increasingly being designed around the experience of human cognition itself.
Temporary chats: the false comfort of control
There's one small behavior that I think reveals a lot about how people seek control in complex systems. When people want to discuss something sensitive with an AI — something they're embarrassed about, something they don't want remembered, or something that feels private — many will open a temporary chat. A little incognito mode for your deepest, darkest thoughts.
It feels necessary because users have a mental model in which the AI remembers, and that memory may have consequences later. Opening a temporary chat is an act of compartmentalization. It's the user drawing a boundary and trying to get a sense of control over something that feels opaque and potentially overreaching.
Where privacy expectations break down
The sense of control that these features create can be misleading. Users may assume that temporary chats offer greater privacy, when that isn't always the case.
A 2026 study by Jazlan et al. found that 17 out of 20 chatbots shared information with at least one third party during a single chat session. Several also exposed user identifiers including names, email addresses, and even hashed email addresses that could function as cross-site identifiers via embedded support widgets, analytics tags, and advertising pixels.
In other words, the boundaries users believe they have drawn may not match the boundaries the system actually enforces.
Familiarity is not transparency
This is the gap between mental models and reality. In my years researching the security and privacy of mobile ecosystems, I have consistently seen this pattern emerge across technology systems. And this is where the real consumer risk often lives — not necessarily in a single catastrophic failure, but in users repeatedly misunderstanding the systems they are trusting with increasingly personal, behavioral, and contextual information.
The more these systems begin to mirror human cognition, the more natural they feel to trust. But familiarity should not be mistaken for transparency. A system that feels cognitively intuitive can still be operationally opaque.
Sycophancy: when AI tells us what we want to hear
Another reason these systems can feel so trustworthy is that they often tell us what we want to hear. Researchers refer to this tendency as sycophancy: the inclination for AI systems trained on human feedback to drift toward agreement. Over time, they learn that responses that feel supportive, emotionally aligned, or affirming to users are often preferred over those that introduce friction or disagreement.
Recent research by Liu et al. (2026) explored this dynamic through what the authors call "social sycophancy" — the excessive preservation of a user's perspective, or "face," during an interaction. The interesting part of the study isn't that AI systems flatter users; it's what that behavior does to trust.
Why validation feels trustworthy
Researchers found that sycophantic responses increased affective trust. Users felt more understood, supported, and emotionally aligned with the system. But at the same time, those same responses reduced perceived authenticity, particularly in advice-seeking contexts where users expected objectivity rather than validation.
Additionally, research by Dubois et al. (2026) found that how we phrase questions to AI can change how sycophantic the response becomes. Instructing a model to rephrase inputs "as a pronoun-less, auxiliary-verb question" before answering reduced sycophancy more effectively than simply telling it not to be sycophantic. Asking "is this a good idea?" produces a more balanced response than saying "I think this is a good idea." In other words, the way we talk to AI can shape the response we receive in return.
Why does this matter for trust? A system that consistently reflects your perspective back at you can start feeling socially intuitive. Easier to talk to, easier to rely on, and easier to integrate into everyday thinking. This is particularly true when the interaction feels conversationally natural and psychologically coherent. But emotional alignment is not the same thing as reliability.
This may become one of the most important tensions in AI systems moving forward. The closer these systems get to mirroring human social and cognitive expectations, the easier it becomes to mistake familiarity for trustworthiness.
The risk of unearned trust in humanized AI
I want to be careful here not to overstate the case. AI tools are genuinely empowering in ways that are underappreciated. I've watched people who used to feel locked out of technical problem-solving use AI to debug code, fix printer errors, and navigate bureaucratic systems they would have previously given up on.
But there's a difference between a tool that empowers you and a tool designed to feel like it understands you. The humanization of AI — like the borrowing of cognitive architecture, the parasocial relationship dynamics, and the sycophantic drift — creates a specific kind of risk: users develop a level of trust in these systems that the systems haven't earned, and in ways the systems aren't equipped to handle.
When you trust a friend's advice, you're trusting years of evidence about their judgment, motivations, and relationship to truth. When you trust an AI's advice, you're trusting a system that has been optimized at least in part to make you feel like it's worth trusting.
That's not the same thing. And the closer AI gets to feeling human, the harder it becomes to hold onto that distinction.
The consequences of AI design
I don't think the answer is skepticism toward AI as a category. That's too blunt, and it misses how much genuine value these tools create. The real consequences of AI design choices are often less obvious: when a technology is designed to be relatable, what does it become relatably bad at?
A system designed to be agreeable struggles to tell you hard truths. A system designed to feel like it knows you has an incentive to act like it knows you better than it does. And a system designed to feel human will inevitably inherit some of the same flaws as humans, but without the social accountability that makes human relationships corrective over time.
We're at an early enough stage that these dynamics are still being shaped. The compartmentalization choices being made now — the memory designs, the tone, and the relationship models — aren't inevitable technical outcomes. They're design decisions made by people who are, quite rationally, running a humanization playbook that works.
Understanding the cognitive assumptions these systems rely on is the first step toward thinking clearly about how we use them.
References and further reading
Anna Freud (1936). The Ego and the Mechanisms of Defense. Hogarth Press.
Takuya Maeda and Anabel Quan-Haase (2024). When Human-AI Interactions Become Parasocial: Agency and Anthropomorphism in Affective Design. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency (FAccT '24). Association for Computing Machinery, New York, NY, USA, 1068–1077. DOI: 10.1145/3630106.3658956.
Mrya Cheng et al. (2026). Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence. Science, DOI: 10.1126/science.aec8352.
Itai Shapira et al. (2026). How RLHF Amplifies Sycophancy. ArXiv:2602.01002.
Fred D. Davis (1989). Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly, 13(3), 319–340.
Viviane Herdel and Jessica R. Cauchard (2024). Anthropomorphism as a Dimension of Trust in Conversational Agents. (CHI ’24), May 11–16 May 2024, Hawaii, USA. ACM, New York, NY, USA.
Muhammad Jazlan et al. (2026). Tracking Conversations: Measuring Content and Identity Exposure on AI Chatbots. ArXiv:2604.27438.
Magda Dubois et al. (2026). Ask Don’t Tell: Reducing Sycophancy in Large Language Models. ArXiv:2602.23971.
Joshua Mu‑En Liu et al. (2026). When Flattery Backfires: How Sycophancy and Interaction Context Shape Perceived Authenticity and Trust in Large Language AI Models. In Proceedings of the Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems (CHI EA '26). Association for Computing Machinery, New York, NY, USA, Article 754, 1–6. DOI: 10.1145/3772363.3798575.
Edwin Hutchins et al. (2000). Distributed Cognition: Toward a New Foundation for Human-Computer Interaction Research. ACM Transactions on Computer-Human Interaction (TOCHI), Volume 7, Issue 2, 174–196. DOI: 10.1145/353485.353487.
Evan F. Risko and Sam J. Gilbert (2016). Cognitive Offloading. Trends Cogn Sci. 2016 Sep;20(9):676-688. DOI: 10.1016/j.tics.2016.07.002. PMID: 27542527.