Research, privacy and legal, each with its own inbox.
Get in touch
Questions about the crawler, the data, or the project. We read and answer everything.
Research & crawler
SmetBot behavior, the datasets, methodology, collaboration, or press.
Legal & licensing
Feed attribution, data licensing, takedown or abuse reports. See the legal page first.
Want our crawler to skip your site?
You don't need to email us. Add these two lines to your robots.txt and it's honored automatically. Details on the bot page.
robots.txt
User-agent: SmetBot Disallow: /