scraping
scraping on Beyond Market Intelligence: a running collection of 3 stories we have gathered and hand-picked because they are worth your time. Every post here touches on scraping in some way — the news, the analysis, the deep dives, and the occasional surprise find. Acme AI is the next-generation, AI-powered spreadsheet platform built to replace Excel and redefine how analysts, data scientists, and enterprise teams work with data. New stories are added to this page as we find them, so check back if you want to keep up with what is happening around scraping, or subscribe to the RSS feed to get them as soon as they are published. Browse the collection below, or head back to the homepage to see everything Beyond Market Intelligence is covering right now.
I scraped 5.94 billion TikTok videos and 3.23 billion profiles in 3 weeks. Uploaded full dataset to Hugging Face for free. Step by step tutorial and code below. [P]
A significant advancement in accessible data research has arrived. A developer has released a comprehensive dataset of 5.94 billion TikTok videos and 3.23 billion profiles, collected over three weeks and now freely available on Hugging Face. This unprecedented scale of data, alongside associated code and a detailed write-up, offers researchers a unique opportunity to explore TikTok’s ecosystem. For those interested in alternative machine learning approaches, consider “Deepity,” a C++ library demonstrating Predictive Coding Networks’ capabilities. Explore the full dataset and resources here: [https://huggingface.co/datasets/kuben-developer/tiktok-videos-4b](https://hugging
X sends cease-and-desist to open source project Nitter over alleged scraping
X has initiated legal action against Nitter, an open-source project providing privacy-focused alternatives to the X platform. The social media company issued cease-and-desist letters demanding the removal of Nitter’s instances and code repository, citing alleged scraping activities. This move underscores escalating tensions around data access and usage within the evolving social media landscape. For a broader perspective on the shifting dynamics of online platforms, explore our recent article on Andy Dunn’s startup, Pie.
CloudFlare Previews Automatic WebMCP Support for Web Pages
Cloudflare’s latest preview empowers developers to seamlessly integrate Web Model Context Protocol (WebMCP) support, enabling AI agents to interact directly with websites. With a single dashboard switch, any site can unlock this capability, moving beyond unreliable scraping methods. This innovation preserves user traffic and control while offering AI unprecedented access through structured tools. Explore how this shift transforms web interaction—and for a deeper dive into related AI tools, see our guide on “How to Install Claude Code.”