Bot & Scraping Policy

This page describes how our automated scraping pipeline collects data from public marketplaces, and the principles that govern its behaviour.

1. What we scrape

Our crawler visits the following public marketplaces on a regular schedule and collects only the information that is publicly visible on listing pages:

• Hispasonic.com — Spanish music gear marketplace
• Soundsmarket.com — Spanish music gear marketplace
• Audiofanzine.com — European music gear classifieds
• Noiz.gr — Greek music gear marketplace

We collect: listing title, asking price, city, country, listing date, instrument condition (where visible), and listing URL (for deduplication only — not published). We do not collect seller names, usernames, contact details, or any other seller identifiers.

2. How we behave

• robots.txt compliance: we check robots.txt before every scraping run. If a site disallows our crawler, we stop immediately and remove that source from the schedule.
• Rate limiting: requests are spaced at least 1.5 seconds apart per source. We do not parallelise requests to the same domain.
• User agent: we identify ourselves as SynthObservatoryBot/2.0 in the HTTP User-Agent header.
• 72-hour embargo: newly scraped listings are held for 72 hours before they can appear in any public output. This gives sellers time to remove their listings before they become part of our dataset.
• No login, no CAPTCHA bypass: we only access publicly available pages. We never attempt to access content that requires authentication or circumvent bot-detection systems.

3. eBay

We use the eBay Browse API under the eBay API License Agreement. In compliance with §3.1.b, §9.5, and §9.10 of that agreement, eBay data is only shown in real time and is never stored in our database or included in the open dataset.

4. Opting out

If you operate a marketplace and wish to exclude our crawler:

1. Add the following to your robots.txt:
   User-agent: SynthObservatoryBot
Disallow: /
   We check this automatically before each run.
2. Or email contact@intellisynthprices.com and we will manually disable scraping of your domain within 5 business days.

5. Telegram bot

The @SynthObservatoryBot is a separate service from the scraping pipeline. It allows users to set up price alerts. See the Privacy Policy §2b for details of what data it stores and how to delete it.

Bot source code is available in our public repository.