How We Handle AI Crawlers on SiteGround Servers
Artificial Intelligence (AI) companies rely on automated crawlers (bots) to gather information from websites. These crawlers can be used for different purposes, such as powering live chat sessions where users ask questions, or collecting large amounts of data to train new AI models.
At SiteGround, we make a clear distinction between these two uses.
Which Crawlers We Allow by Default
We allow crawlers from major AI providers that are used in user chat sessions.
What this means:
- Your website can be accessed and referenced in real-time answers that users receive when using these AI chat tools.
- This can increase exposure and reach, since your content may surface directly in AI chat sessions when relevant to user queries.
- The crawlers only read and display information — they should not be storing or reusing your data.
Check our dedicated article for a full list of the allowed chat session AI crawlers.
Which Crawlers We Block by Default
We block crawlers intended for AI model training to protect our clients’ website data and intellectual property and save resources.
These bots attempt to collect large volumes of information to feed into training datasets, which could then be reused to build or improve AI systems without your consent.
Blocking them means your content remains under your control and is not repurposed for activities beyond its original intent.
If you have a specific reason to allow such crawlers to access your site, you have to submit an inquiry through our Help Desk > Other > Other.
Our support team will review your request and apply the change if appropriate.
