Optimizing a website for AI search: what we changed on koana-labs.com

Why we looked at this

Until September 2026 our site was a single-page app built with Vite and React. In a browser it looked fine. To anything that doesn't execute JavaScript, the body of every page was this:

<body>
  <div id="root"></div>
  <script type="module" src="/src/main.tsx"></script>
</body>

That's it. No headline, no services, no contact details. Googlebot renders JavaScript, so Google got to see the real page eventually. The crawlers behind ChatGPT, Claude and Perplexity don't. Vercel and MERJ looked at crawler traffic across Vercel's network in late 2024 and found that none of the major AI crawlers rendered JavaScript. They download the script files and never run them. So when an assistant was deciding whom to recommend for custom software or AI consulting, our site had nothing to say.

The second problem was more embarrassing. The canonical tag in the old index.html pointed at koanalabs.com, without the hyphen. Every page told search engines that the real version lived on a domain we don't use.

What we changed

1. Complete HTML at build time

We rebuilt the site on Next.js with the App Router. Pages are server components, rendered to static HTML when the site is built. The header and the contact form are the only parts that need JavaScript in the browser. A crawler that fetches /services now gets the full text of the page in the first response.

If you only do one thing from this list, do this one. The rest only counts once the content is actually in the HTML.

2. One canonical URL, one H1, metadata per page

Each page exports its own title, description and canonical URL on koana-labs.com, and each has exactly one H1. That's standard SEO hygiene. AI systems use the same signals to work out what a page is about and which URL to cite, so it pays twice.

3. AI crawlers named in robots.txt

The old robots.txt allowed everything and mentioned Googlebot, Bingbot and two social crawlers. Allowing * is technically enough. In practice some CDNs block AI bots by default, and we wanted it written down. robots.txt is now generated from code and lists GPTBot, OAI-SearchBot, ChatGPT-User, PerplexityBot, ClaudeBot, Claude-SearchBot, Google-Extended and Applebot by name, plus the sitemap. /api/ stays off limits.

The OpenAI bots are easy to mix up. GPTBot collects training data. OAI-SearchBot decides whether your site can appear in ChatGPT's search answers. ChatGPT-User fetches a page when someone asks ChatGPT about it. If you block GPTBot, you can still show up in ChatGPT search. If you block OAI-SearchBot, you can't.

4. A sitemap built from the routes

The sitemap is generated from the app's routes, so a new page shows up in it without anyone remembering to add it. Blog articles are listed as they're published.

5. Structured data that matches the page

The home page carries Organization JSON-LD with our legal name, URL, logo, contact email and a one-line description that repeats what the page says. Blog articles get Article markup, and FAQPage markup when they have an FAQ.

We use schema to remove ambiguity. It tells a machine that "Koana Labs LLC" belongs to this website and this email address. We don't expect it to lift rankings. Google says outright that you don't need new machine-readable files, AI text files or special schema.org markup to appear in AI Overviews or AI Mode, and that normal SEO practice applies.

6. Articles that answer first

Every article on this blog opens with two or three sentences that answer the title's question before any background. AI assistants quote passages, and a passage that stands on its own is easier to quote than one that leans on the three paragraphs above it. The rest follows the same idea: descriptive H2s, sections that make sense when read alone, and a source next to every claim we didn't make up ourselves.

7. A blog that software can publish to

Our articles come from our own content engine and land in the site's repository as one JSON file each. The build checks every file against a schema and fails if one is broken, so a malformed article can't reach production. Each article is static HTML with its FAQ and JSON-LD, and there is an RSS feed at /blog/feed.xml.

How to check your own site

The most important test takes a minute. Fetch a page the way an AI crawler does and read what comes back:

curl -s -A "GPTBot" https://your-domain.com/ | sed 's/<[^>]*>/ /g' | tr -s ' ' | head -c 1000

If you see your headline and your services, the crawler sees them too. If you see a few menu items and little else, your content is built in the browser and AI assistants are reading an empty page.

After that, look at three things:

1. The canonical tag on a few pages. It should point at your real domain. Ours didn't.
2. robots.txt and your CDN's bot settings. Make sure OAI-SearchBot, PerplexityBot and Claude-SearchBot aren't blocked by accident.
3. Your structured data. Run the home page through Google's Rich Results Test or the Schema Markup Validator and compare the result with what the page shows.

What to expect

Fixing all of this won't get you recommended by itself. Citations come from pages that answer real questions better than the alternatives, and from what other sites say about you. What the fixes do is take away the reason an assistant couldn't consider you in the first place. For us that reason was an empty <div> and a typo in a domain name.

If you'd like us to look at your site, that curl command is where we'd start.

Sources

• The rise of the AI crawler (Vercel, with MERJ)
• AI features and your website (Google Search Central)
• Overview of OpenAI crawlers

Further reading

• Custom Software Development
• AI Phone Assistant