Substack Harvesting

By August Isley (c) 2025

Substack content is actively scraped and copied by various AI-driven data-gathering tools. Several web scraping platforms and open-source projects specifically target Substack posts, making them available for analysis, training, or republishing.

🔍 How AI and scraping tools interact with Substack

• Open-source scrapers: Tools like Substack2Markdown allow users to scrape both free and premium Substack posts, saving them as Markdown or HTML files for offline use Github.
• Commercial scraping services: Platforms such as Scrapelead and Apify offer automated Substack scrapers that extract article metadata (titles, authors, dates, body text, podcasts, reactions) for content analysis or competitive intelligence scrapelead.io Apify.
• AI-powered monitoring: Services like Browse AI provide robots that continuously scrape and monitor Substack for brand tracking, lead generation, or even feeding large language models (LLMs) with structured content Browse.
• General AI scraping tools: Broader AI-driven web scraping frameworks (e.g., Oxylabs and others) list Substack among the sites they can target, using machine learning to adapt to changing layouts and extract text efficiently DEV Community.

⚠️ Risks and implications

• Content copying: Writers on Substack may find their work duplicated or repurposed without consent, especially if scraped by AI systems for training datasets.
• Intellectual property concerns: While scraping public posts may be technically legal in some jurisdictions, it raises ethical and copyright issues when used for AI training or redistribution.
• Privacy issues: Premium or subscriber-only posts being scraped could violate terms of service and undermine trust between creators and readers.
• AI dataset feeding: Many LLMs rely on scraped web content. Substack, being a rich source of independent writing, is attractive for inclusion in these datasets — though companies rarely disclose exact sources.

✅ Key takeaway

Substack is indeed used by AI scraping and copying endeavors, both through open-source projects and commercial services. If you’re a Substack writer, it’s worth being aware that your content may be harvested for analysis or even AI training, often without explicit permission.