Skip to main content
BenchLM
Data

We measure the session. They sell the week.

Grok Bot, Hermes, and OpenClaw keep work going across sessions. Their hosting and controls differ; a session benchmark does not measure week-long reliability.

A short ink dash beside seven thin ticks on steel ledger paper, cobalt blot on the dash
In this article6 sections

SpaceXAI announced Grok Bot on August 11, 2026, as a managed agent product.

As of September 30, 2026, Cursor's billing docs include access with paid individual Cursor plans, self-serve Teams, and eligible linked individual accounts.

The product provides the computer, tools, and persistent state around a model. Our model scores do not rank Grok Bot's full system.

Hermes Agent and OpenClaw also retain state, memory, and tools across sessions, but their hosting and model controls differ. You can host either runtime yourself. Hermes also offers optional cloud hosting. These are documented choices, not evidence that the products complete the same work equally well.

Our agentic board still times a session.

The personal AI agents comparison separates hosting, model control, billing, and permissions across these products, Dots, Claude, and Manus. Those are configuration choices, not a product performance ranking.

What SpaceXAI actually shipped

The table reflects provider documentation read September 30, 2026: Grok Bot, Hermes Agent, and OpenClaw. It records configuration choices. We have not tested week-long reliability across these products.

Table 1
Grok Bot Hermes OpenClaw
Who ships it SpaceXAI / Cursor service Nous Research OpenClaw Foundation / community
Runtime Account-shared computer in Cursor cloud Self-hosted runtime; optional hosted service Gateway on your machine or server
Computer use Browser, filesystem, terminal; apps without a clean API Tools, skills, scheduled work Tools, skills, heartbeat, local files
Memory Named bots keep files, sessions, preferences Bounded memory across sessions; skills from completed work Local state and long-running process
How you talk to it Desktop and mobile apps Desktop, CLI, and supported messaging integrations Supported chat apps and web interfaces
Access today Paid individual Cursor plans and self-serve Teams; eligible account links; Enterprise through account team Open-source runtime; optional Portal and cloud services Open-source runtime
Model selection Cursor selects serving models; no customer picker Configure a supported provider and model Configure supported providers and per-agent routing
The trade Low setup, vendor computer, less visibility Control, you harden it Control, broader system access

Two details in the Grok Bot docs matter more than the teammate language.

All of your bots share one computer. Logins, files, and browser sessions are account-scoped, not bot-scoped. The computer documentation says to treat a login placed on that machine as available to every bot you create. That is a product feature for handoffs. It is also a single blast radius.

Second, the product refuses to begin with a workflow builder. You message a named bot, grant access, and it works in the real tool. If the job is real, the failure modes are also real: a wrong CRM write, a follow-up sent to the wrong thread, a credential sitting on a machine you do not inspect.

The billing page lists individual SuperGrok, SuperGrok Plus, SuperGrok Heavy, and X Premium+ links. Included usage resets weekly. Extra usage is billed through Cursor when on-demand is enabled. A running job can finish past its monthly limit. Cursor and linked-account grants do not stack. Check the account's allowance rather than treating a subscription as unlimited work.

Cursor manages model selection, and the serving mix can change. A model ranked on our agentic board is a candidate only where your agent lets you configure it. Grok Bot has no customer-facing model picker.

We have not run Grok Bot. The table above is the public architecture, not a ranking.

The open-source version was already the product

Hermes is explicit about the loop. It keeps curated memory across sessions, writes skills from completed work, and can sit on a cheap VPS while you reach it from a messaging app. Hermes is selling accumulated competence, not a better first answer.

OpenClaw is the high-adoption expression of the same idea: a local-first assistant that lives in the channels you already use, runs with a heartbeat, and can act without a fresh prompt every time. Flexibility and surface area arrive together. An agent with broader system access can finish more jobs. It can also touch more things you did not intend.

There are other shapes in the same category: collaborative workspaces, coding agents with computer use, enterprise managed agents. Across them, the pattern is stable. Chat optimized for looking intelligent in a turn. These systems optimize for a finished change in someone else's software while you are elsewhere.

If you still evaluate them by reading a chat transcript, you are grading the wrong artifact.

The model was not the missing piece

Tool calling, long context, and multi-step planning got better. That helped. It was not sufficient.

A persistent harness is now the decisive layer: state, a computer that can navigate real interfaces, memory that does not evaporate, scheduled execution, and explicit approval gates. Without those, a frontier model is still a very smart autocomplete that forgets the project between sessions.

Most "agentic" features bolted onto chat products add a few tool calls and stop there. Real agency looks different. The system keeps a running picture of the job. It notices a follow-up is due. It can recover from a partial failure without starting over. It escalates only the decisions that need a human.

That is why launch-day demos often disappoint in week two. Next-token quality is no longer the hard problem. Reliability over long horizons is. So is a website that changed its layout, a credential that expired, and a human who did not notice the agent was quietly doing the wrong thing.

We already see a smaller version of this on coding-agent products. In AI coding agents need receipts we refused to publish a ranking from vendor lists, because the lists rank their authors and the one public cross-agent table still cannot tell you what a failed Tuesday afternoon costs. Persistent general agents are that problem with email, CRM, and billing attached.

What a Terminal-Bench score can still tell you

The agentic board combines sourced model evidence under the current scoring contract. Terminal-Bench 2.0, BrowseComp, and OSWorld-Verified are examples of bounded task evaluations, not the complete current scoring basket. As of November 2026, the current agentic leader is Claude Fable 5.2 (90.7, Estimated).

Read that sentence for what it is. Those tests ask whether a model can finish a bounded, instrumented task: a terminal workflow, a web-research question, a computer-use job. They are the right shortlist for the brain you put inside a harness. They are a weak score for the harness itself.

Table 2
What the public tests measure What Grok Bot, Hermes, and OpenClaw sell
One task, one environment, a clock Work that continues after you close the laptop
A hidden acceptance check at the end A CRM row, inbox, or ticket that has to stay right overnight
The model, sometimes with a standard harness Memory, credentials, schedules, multi-bot handoff
Failures you can see in the log Failures that look like ordinary business activity

High Terminal-Bench scores are still useful. Pair them with SWE-bench if the work is software, and with the latency note if the loop has many calls. Then stop. Do not promote an agentic model rank into a verdict on Grok Bot versus Hermes versus OpenClaw. That comparison needs the same jobs, the same success criteria, counted interventions, and published silent failures.

We do not have that fixture for general persistent agents. Until we do, there is no honest product podium.

Who should run the computer

Grok Bot optimizes for low friction and always-on continuity. You do not provision the VM. You do not patch the runtime. Parallel bots keep working on a machine that does not sleep when you do. You trade visibility, pricing, and control of the shared credential boundary for that convenience.

Hermes and OpenClaw optimize for ownership. You can host the runtime on your hardware or VPS, select supported model providers, and inspect or extend the agent. OpenClaw's local-only option requires local models. Remote model calls and connected services still send data outside the runtime. Hermes setup likewise asks you to configure a provider and can connect to Nous Portal for hosted models and tools. The cost is setup, maintenance, and security work that the vendor would otherwise absorb.

Neither side has a permanent advantage. Split them by the work, not by tribe.

Use a managed computer when the work is high-volume, the data is not the crown jewels, and you will actually review the exceptions. Use a self-hosted runtime when you need to operate the computer or change the harness. If logins or task data cannot leave your machine, review inference and tool connections separately before relying on that setup. Many people will end up with both: a managed bot for the always-on grind, a local agent for the sensitive lane.

What matters more than the brand is whether the system can own an outcome across multiple steps without constant supervision, and whether you can see and correct the failures. Black-box "it finished the job" is not a control if the job can move money or customers.

One workflow, two weeks of review

Pick one repetitive job that already costs you real time. Write down what "done" looks like, what must never happen, and how you will check the work. Give a single agent that job and nothing else.

Review every output for two weeks. Instrument the failures: where it got stuck, what it misread, what context was missing. Do not start with five agents and a vague ambition. Compounding starts after the first workflow is trustworthy.

Demand a trajectory you can inspect: tool calls, intermediate state, and the points where a human had to judge. If the product cannot show you that, you are not managing an agent. You are hoping.

Model rankings still help you choose the brain. They do not tell you whether the full system stays reliable over a week of real work. That evaluation is still mostly manual. It will not stay that way, but pretending otherwise is how you ship a wrong Salesforce update and call it agency.

Chatbot windows made sense when models were primarily good at generating text. They no longer match the product people are buying.

The next number worth publishing is a week-horizon eval: the same jobs, counted interventions, silent wrong writes, and a credential scope you can state in one sentence. Until someone runs that across products, we will keep ranking models and refuse to rank the week.

→ Agentic leaderboard · Terminal-Bench 2.0 · OSWorld-Verified · Coding agents need receipts

Frequently asked questions

01What is Grok Bot?

Grok Bot is SpaceXAI's managed agent product, announced August 11, 2026. Bots work on a persistent computer in Cursor's cloud, using a browser, files, terminal, and connectors. Cursor selects the serving models. Background work can continue after you close the app, subject to access, usage, and approval requirements.

02What is the difference between Grok Bot, Hermes, and OpenClaw?

Grok Bot runs in Cursor's cloud with service-selected models. Hermes Agent and OpenClaw let you host the runtime and configure supported model providers. Hermes also offers optional cloud hosting. Hosting the runtime yourself does not keep inference local automatically: prompts can still go to remote providers or connected tools.

03Can BenchLM rank persistent agents?

Not yet. The agentic board scores model configurations using sourced task evaluations, including terminal, browsing, and computer-use tests. Those tests do not measure overnight reliability, silent wrong writes, or credential scope. We will not publish a product ranking of Grok Bot, Hermes, or OpenClaw until the same jobs are run across them.

04Do agentic benchmarks measure always-on agents?

They measure whether a model can finish a bounded, instrumented task with tools. That is necessary for a persistent agent and not sufficient. A high Terminal-Bench score does not tell you whether the system notices a missed follow-up, recovers after a layout change, or updates the wrong Salesforce record at 2 a.m.

05Should I use a managed or self-hosted persistent agent?

Choose managed hosting when you want the provider to operate the computer. Self-host Hermes or OpenClaw when you need control over the runtime. If data must stay local, also check the model endpoint and each connected tool. Both approaches need a narrowly defined workflow and review of its outputs.

Reader questions

External sources linked in this article

7
  1. 01Grok Bot
  2. 02Cursor's billing docs
  3. 03Hermes Agent
  4. 04OpenClaw
  5. 05Grok Bot
  6. 06Cursor manages model selection
  7. 07Hermes setup
7 external sources

Share or save

Share on XShare on LinkedIn

Keep readingAll research

Choose the right model before an expensive mistake. One weekly recommendation: what to choose, what costs less, and what is not worth switching to.

Join 7,000+ subscribers.