Alexander Liss

Attention Fine-Tuning (AFT)

Alexander Liss

The next frontier in AI is models that improve themselves. Every major lab is racing toward systems that evaluate their own outputs, identify weaknesses, and update accordingly, like Google's AlphaEvolve. The problem is the reward signal from RLHF. Human preference labels are expensive, brittle, and external reward shaping too often results in model collapse.

But what if the reward signal was in the model the whole time?

This paper introduces Attention Fine-Tuning (AFT): a post-training framework that derives its reward signal entirely from within the model, with no human labels required. In internal testing as we developed this framework, we achieve significant results including:

• A 9.2% improvement in conversation quality over an SFT baseline on 7,372 test examples
• 95.5% of test examples improved on attentional focus, with 100% improvement on mid- and late-dialogue turns
• A mapping of attractor states that draw conversational systems into semantic collapse

This framework is applicable to environments where optimizing the ability of an LLM to hold a conversation is critical, including customer service, education, personal assistants, and video games. It allows post-training without manually labelled preference data.

Experience Orchestrator (EO)

Alexander Liss, Nicholas Desmond, Santiago Gil Gallego

In 2026, multi-agent ecosystems are everywhere. OpenClaw, MoltBook, and their successors let agents act autonomously in open-ended environments. And the world is discovering that prompting them toward good behavior doesn't scale. What's missing is a general framework for keeping LLM-based agents aligned with optimal behavior.

This paper presents the Experience Orchestrator, a formal framework for applying control theory to govern large language models as independent agents. In internal testing as we developed this framework, we achieved significant results including:

• Measurable lift of +32 pts in goal completion rate over a naive LLM approach.
• LLMs that move beyond 'annoyingly friendly' defaults toward nuanced, humanlike conversational intent.
• Rich dialogue histories that provide training data for further iteration.

The framework generalizes to any environment where agents must make decisions under partial observability. It provides a way to govern LLM agents through a shared policy, similar to classical multi-agent reinforcement learning, but applicable to the world of OpenClaw.

For Humans, love is the drug. But for AI, it's the reward signal.

Substack post announcing my reward signal thesis

From clicks to cognition: digital experiences are becoming learning systems.

Your website isn't a destination anymore. It's training data.