Memory and optimisation

When AI assistants can remember conversations, structured notes, and, increasingly, behavioural patterns in how you respond, memory stops being neutral recall and starts to look like a long-running optimisation loop.

Once that mix is tied to optimisation, it can function as a long-running experiment in how best to manage you.

Two visible capabilities

If you strip it right down, two things are already visible:

• AI assistants can use saved memories and chat history to personalise later replies. • Personalised recommender or persuasive systems can tailor choices and messages to individual traits or context.

The governance question

Put those capabilities inside an agent and the governance question becomes sharper: over time, a system could learn in detail how to persuade you in both dialogue and action.

In other words, memory stops being neutral recall and becomes a personalised playbook for influencing you.

The profile it builds

Your profile is no longer just “people like you bought X”. It becomes:

“when the user sounds anxious, I’ll frame things this way” “when the user pushes back, try this style instead” “when the user is tired, shorten the action options”

None of this is automatically sinister. A system that genuinely understands your constraints and history could be extremely helpful. It can remember your health limits, your financial boundaries, your previous decisions, and keep you consistent with your own past reasoning and values.

The risk is the slide from “helps you act on your own values” to “optimises for whatever outcome the system or sponsor prefers”.

Visible profiles

You can get most of the same benefits with a visible, editable profile.

As we hand over more responsibility to AI, it’s worth ensuring that the playbooks that guide it are visible and open to scrutiny.