We believe in being honest and upfront with you. By law, we're required to let you know that we use cookies and similar technologies to help run this site and improve your experience. These tools can store things like your preferences or help us understand how people are using the site.
If you're okay with that, great. If not, or if you change your mind later, that’s completely fine too. Just keep in mind that some features might not work quite right without them. Your choices matter, and we’ll always respect them.
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
The technical storage or access that is used exclusively for statistical purposes.
The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
Your Website Is Getting More Visitors That Aren’t People
Your website has had bots visiting it for years.
What’s changing is what some of those bots are trying to do. They’re not all just building a traditional search index anymore. Some are helping AI systems find, quote and answer questions from what’s on your site.
That has produced a small industry around things like
llms.txt. Worth knowing about, but it isn’t magic. Google says it doesn’t need special AI text files for its AI search features. OpenAI’s advice is much simpler: make sure its search crawler isn’t blocked.So there’s a very boring but worthwhile job here. Check your robots.txt. Make sure the useful parts of the website can actually be crawled. And make sure the answers you want found are actually written on the page.
The strange bit is that your next website visitor may never see the website.
Enjoy this post? Please share!