Your website has had bots visiting it for years.
What’s changing is what some of those bots are trying to do. They’re not all just building a traditional search index anymore. Some are helping AI systems find, quote and answer questions from what’s on your site.
That has produced a small industry around things like llms.txt. Worth knowing about, but it isn’t magic. Google says it doesn’t need special AI text files for its AI search features. OpenAI’s advice is much simpler: make sure its search crawler isn’t blocked.
So there’s a very boring but worthwhile job here. Check your robots.txt. Make sure the useful parts of the website can actually be crawled. And make sure the answers you want found are actually written on the page.
The strange bit is that your next website visitor may never see the website.