PulsarBot
PulsarBot is the crawler identity for tech.pulsar trend discovery. It collects public signals used for technology search, trend summaries, and source attribution on pulsar.ivan.digital.
Identity
Crawler Policy
- Checks robots.txt before crawling public web pages with the PulsarBot identity.
- Respects disallowed paths and crawl delays for explicit PulsarBot fetches.
- Prefers official APIs, RSS feeds, and documented data access paths over HTML scraping.
- Uses per-origin rate limits and backs off on errors or rate-limit responses.
- Does not use login-only, paywalled, private, or personal account content.
- Does not bypass access controls or anti-automation challenges.
Opt Out
To block the PulsarBot identity, add a robots.txt rule for this user agent. Site owners can also contact us for source removal or rate-limit changes.
User-agent: PulsarBot Disallow: /
Data Use
tech.pulsar displays short trend summaries with links back to original sources. Source pages remain the canonical location for the underlying content.