Querying the entire internet with DuckDB and Common Crawl ·
Common Crawl publishes petabytes of web crawl data on S3. With DuckDB you can query it directly, no cluster and no download, and measure how fast the vibe-coded web is really growing.
Common Crawl publishes petabytes of web crawl data on S3. With DuckDB you can query it directly, no cluster and no download, and measure how fast the vibe-coded web is really growing.
Measuring the rate of software output growth in the AI era, across Show HN, GitHub, package registries, and research.