It's been a while since I've written about the bot formerly known as Sentinel. It has continued to evolve, with the most significant change being a switch from Node-RED to n8n. Over the past couple weeks, there has been an even more fundamental shift however: a brain transplant. One that has prompted me to rename Sentinel to Al (that's Al with an L).

The fact that it looks like AI with an i is a coincidence -- it comes from the Arabic word for "machine", and I also decided to start referring to Al as "him" rather than "it" for convenience. Rather than an "assistant", I position Al as an digital extension of myself and the orchestrator of my exocortex^[1], while I'm his meatspace extension.

The big shift is that I've jumped on the OpenClaw bandwagon! People have mixed views on OpenClaw, but I can say that overall I find it quite exciting. I, like many people, have fully embraced the use of Claude Code, and have been trying to retrofit it to do more than simply build software. OpenClaw looks like the beginning of an ecosystem that allows us all to do just that without all rolling our own disparate versions.

OpenClaw embraces a concept that I find so fundamental as to be laughably obvious: the interface to chat bots should be existing chat apps. This is why Sentinel and Amarbot had their own phone numbers. While I have used Happy (and later HAPI, self-hosted on https://hapi.amar.io) in order to access my Claude Code sessions from mobile without the insanity of a mobile terminal, even those felt like anti-patterns. You can read more on why I think this is in my post on why we should interact with agents using the same tools we use for humans, as well as my post on agent chat interfaces in general.

For the record, I think these mode of interaction are inevitable, not a preference. While AI will interface with things via APIs, raw text, or whatever else, the bridge between AI and human will be the same tools as between human and human. This is also why I think the UIs purporting to be the "next phase" of agentic work where you manage parallel agents (opencode, conductor, etc) are the wrong path. The people who got this absolutely bang-on correct are Linear, with Linear for Agents, and I don't just say that because I love Linear (I do). I should assign tickets to agents the same way I would a human, and have discussions with them on Linear or Slack, the same way I would a human.

So, that brings me to what I think OpenClaw is currently bad at. First, cron should not be used as a trigger. Calendars should. Duh. One of the first things I did was give Al his own calendar and set up appropriate hooks. We need to be using the same tools, and I never use cron. This gives me a lot of visibility over what's going on that is much more natural, and I can move around and modify these events in the way they're supposed to be: through a calendar UI, not through natural language conversation.

On the topic of visibility, workspace files need to be easily viewable and editable. By this I mean all the various markdown files that form the agent's memory and instructions. While I may not need to edit these and Al can do that on his own, if we do want to explicitly edit, it should be easy to do so, and natural language conversation is not the way (this is not code)! To solve this, I've put Al's brain into my Obsidian vault (yes, the very vault from which these posts are published!) and symlinked it into the actual OpenClaw directory. So now I have can browse these files with a markdown editor heavily optimised for me! An added bonus of this is that Al's brain is now replicated across all my devices and backups for free, using Syncthing which already covers my vault.

It's still early days, so I'm still finding out the best ways to collaborate with Al. There are still a lot of issues, mainly related to Al forgetting things, that I'm working through, but it's been great! He can do everything that Sentinel already could, including talk to other people independently and update my lists. The timing is perfect, as I pre-ordered a Pebble Index 01^[2] and this will very likely be the primary way that I communicate with Al in the future when it comes to those one-way commands. For two-way, I still use my Even G1 smart glasses, but I suspect that I may go audio-only in the future (i.e. my Shokz OpenComm2 as I don't like having stuff in my ears).

I've scheduled some sessions with Al where we try and push each other to grow. For him, this means new capabilities and access to new things and various improvements. For me, this means learning about topics that Al has broken down into a guided course or literal coaching. I'll post more as I go along! In the meantime, you can also chat with him.

Sentinel, my AI personal assistant has evolved a bit since I last wrote about him. I realised I hadn't written about that project in a while when it came up in conversation and the latest reference I had was from ages ago. The node-red logic looks like this now:

1. Every morning he sends me a message (in the future this will be a status report summary). The goal of this was to mainly make sure the WhatsApp integration still works, since at the time it would crap out every once in a while and I wouldn't realise.
2. I have an endpoint for arbitrary messages, which is simply a URL with a text GET parameter. I've sprinkled these around various projects, as it helps to have certain kind of monitoring straight to my chats.
3. He's plugged in to GPT-4-turbo now, so I usually just ask him questions instead of going all the way to ChatGPT. He can remember previous messages, until I explicitly ask him to forget. This is the equivalent of "New Chat" on ChatGPT and is controlled with the functions API via natural language, like the list-adder function which I already had before ("add Harry Potter to my movies list").

As he's diverged from simply being an interface to my smart home stuff, as well as amarbot which is meant to replace me, I decided to start a new project log just for Sentinel-related posts.

Edit: this post inspired me to write more at length about chat as an interface here.

I mentioned recently that I've been using OpenAI's new functions API in the context of personal automation, which is something I've explored before without the API. The idea is that this tech can short-circuit going from a natural language command, to an actuation, with nothing else needed in the middle.

The natural language command can come from speech, or text chat, but almost universally, we're using conversation as an interface, which is probably the most natural medium for complex human interaction. I decided to use chat in the first instance.

Introducing: Sentinel, the custodian of Sanctum.

No longer does Sanctum process commands directly, but rather is under the purview of Sentinel. If I get early access to Lakera (the creators of Gandalf), he would also certainly make my setup far more secure than it currently is.

I repurposed the WhatsApp number that originally belonged to Amarbot. Why WhatsApp rather than Matrix? So others can more easily message him -- he's not just my direct assistant, but like a personal secretary too, so e.g. people can ask him for info if/when I'm busy. The downside is that he can't hang out with the other Matrix bots in my Neurodrome channel.

A set of WhatsApp nodes for Node-RED were recently published that behave similarly to the main Matrix bridge for WhatsApp, without all the extra Matrix stuff in the way, so I used that to connect Sentinel to my existing setup directly. The flow so far looks like this:

The two main branches are for messages that are either from me, or from others. When they're from others, their name and relationship to me are injected into the prompt (this is currently just a huge array that I hard-coded manually into the function node). When it's me, the prompt is given a set of functions that it can invoke.

If it decides that a function should be invoked, the switchResponse node redirects the message to the right place. So far, there are only three possible outcomes: (1) doing nothing, (2) adding information to a list, and (3) responding normally like ChatGPT. I therefore sometimes use Sentinel as a quicker way to ask ChatGPT one-shot questions.

The addToList function is defined like this:

{
	name: "addToList",
	description: "Adds a string to a list",
	parameters: {
		type: "object",
		properties: {
			text: {
				type: "string",
				description: "The item to add to a list",
			},
			listName: {
				type: "string",
				description: "The name of the list to which the item should be added",
				enum: [
					"movies",
					"books",
					"groceries",
				]
			},
		},
		required: ["text", "listName"],
	},
}

I don't actually have a groceries list, but for the other two (movies and books), my current workflow for noting down a movie to watch or a book to read is usually opening the Obsidian app on my phone and actually adding a bullet point to a text file note. This is hardly as smooth as texting Sentinel "Add Succession to my movies list". Of course, Sentinel is quite smart, so I could also say "I want to watch the first Harry Potter movie" and he responds "Added "Harry Potter and the Sorcerer's Stone" to the movies list!".

The actual code for adding these items to my lists is by literally appending a bullet point to their respective files (I have endpoints for this) which are synced to all my devices via the excellent Syncthing. In the future, I could probably make this fancier, e.g. query information about the movie/book and include a poster/cover and metadata, and also potentially publish these lists.