Skip to content

DealmeteoBot

Last updated 25 Oct 2026.

DealmeteoBot/0.1 (+https://dealmeteo.com/bot) is the crawler of Dealmeteo, an independent research project on e-commerce promotions in Europe. This page explains what it collects and how to ask it to stop.

What we collect

  • Facts about promotions: event name, start and end dates, advertised discount, category, retailer, country, a link to the public page.
  • Prices of a sample of products shown on public promotion pages, with the reference price the retailer displays.
  • News signals: headline, publisher, date and link from news and trade sources, used to anticipate events.

What we never collect or do

  1. No login, account creation, cart, checkout, wishlist or personalised pages.
  2. No CAPTCHA solving, no bypassing anti-bot systems, no rotating proxies to evade a block. A block is an answer.
  3. No personal data: no reviews, reviewer names, private sellers, Q&A, or anything about a natural person.
  4. No product images, logos or long product descriptions copied to our site. We link to the retailer instead.
  5. No full catalogue copies: a small sample per event only.
  6. No reproduction of news articles: we summarise in our own words and link out.
  7. No use of collected pages to train AI models.

How we crawl

Rule Value
robots.txt Obeyed for DealmeteoBot and *, refreshed weekly
Rate At most 1 request every 10 seconds per domain and 200 pages per domain per day; a higher Crawl-delay is honoured
Hours Normal runs at 05:00 CET. During live shopping events, at most one refresh per hour
Errors We back off on 429, 403 and 503, and stop for 24 hours after three in a row
Storage Raw page snapshots kept 30 days for fact-checking, then deleted. Extracted facts are kept

Opt out

To ask us to stop crawling your site, write to hello@dealmeteo.com from your domain. We stop within 48 hours. You can also disallow DealmeteoBot in your robots.txt.