Why memory is a problem in the first place

A model works in a loop: gets text, gives a reply, forgets. Between your messages it keeps nothing, not your name, not yesterday's conversation.

The sense of continuity comes from the service. Before every reply it assembles a brief: who you are, what you talked about, where you landed, and sends it to the model along with your message. How much a character "remembers" depends entirely on what made it into that brief.

And the brief can't be infinite. A model has a limit on how much text it can take in at once, and every word in it costs money (more in what are tokens). Sending six months of chat every time just isn't possible. So memory is always a choice: what to take and what to leave behind.

What memory is made of on our site

On my-idol.ru the brief is built from four layers, and each has its own job.

  1. Recent messagesThe model sees the last fifteen to twenty or so messages word for word. That keeps things coherent: the answer to a question from three messages ago, a "what about that thing," a joke you haven't wrapped up yet.