Constrain the pilot: knowledge first, then autonomy
The first pilot should deliberately limit two things: What knowledge you give the agent and what you let it do with that knowledge.
Start with one knowledge domain
Do not open by trying to make your whole content estate AI-ready.
Pick one knowledge area that matters commercially but is small enough to hold in your head. One product. One product line. One service. One customer proposition. One part of the business.
Ideally, it should:
- have clear business value;
-
already have enough good content and assets to work with;
-
be small enough that the team can inspect and understand the complete knowledge set;
-
involve a limited number of content items and associated assets — perhaps 3–5 core items for an initial experiment;
-
depend on only one or two key systems, for example CMS + PIM or CMS + DAM;
-
sit predominantly within one business function, limiting cross-department dependencies during the pilot.
The objective isn't to build the perfect semantic knowledge architecture.
It's to take one bounded area and ask: what would have to be true about this knowledge for an agent to use it reliably?
Which source is the authoritative one? Is the information consistent across places? What metadata does the agent need? Which relationships between pieces of information have to be spelled out? And what do our people know in their heads that has never been written down anywhere?
With five content items, humans can sit down and answer that. With fifty thousand, you have accidentally started a transformation programme.
Then constrain autonomy
Once you know what knowledge the agent is working with, decide what it may do.
Can it retrieve information? Summarise it? Draft something? Change existing content? Pick an asset? Make a recommendation? Set something off in another system? And where does a human need to look at it, approve it, or make the call? And where does a human need to review, approve, or make the final decision?
Don't start by asking:
"How autonomous can our CMS become?"
Ask instead:
"What useful decision or action are we comfortable delegating to an agent today?"
This is what we did
We put the same constraints on ourselves.
We upgraded a test environment to Optimizely CMS 13, connected it to Optimizely's AI Platform, and then narrowed the knowledge right down. From the whole website, to our case studies, to our Optimizely case studies specifically.
One principle guided everything after that. Automate the laundry and the cleaning. Not the thinking.
From there we tried three things, each harder than the last. First, a GEO agent that audits one selected CMS page and recommends how to make it more visible in AI-driven search. Second, a content quality workflow to see how far the AI Platform and the CMS could get working together on their own. Third, and more ambitious, a scenario joining the AI Platform, the CMS, and Salesforce, built deliberately on things NoA Ignite can already do.
For each one, our commercial, business and technical people agreed a minimum viable scenario first, then added complexity and iteration at a time.
Having something real to poke at did something we had not planned for. It turned AI FOMO and general unease into practical conversations about what an agent should and should not be allowed to do.
The answer changes by task. For controlled work like tidying up language and content structure, we can give the agent more room. Where it applies brand standards more creatively, or uses Salesforce inputs to recommend personalised [Szymon He1] content, we keep it at the recommendation stage. An editor still decides what gets published.
Which brings back the other constraint. Earlier I said a pilot should limit both the knowledge you hand the agent and what the agent may do with it. The Salesforce/CMS connection raised the challenge of preventing personal data from travelling between systems.
None of those lines are permanent. As we learn, tighten the instructions and the guardrails, and get more confident in what comes back, we can move them.
