@commoncrawl.bsky.social
Common Crawl is a non-profit foundation dedicated to the Open Web.
The latest Web Graphs from the November and December 2025 and January 2026 crawls are now available, comprising 279.4 million host-level nodes with 13.4 billion edges, and 122.3 million domain-level nodes with 6.1 billion edges.
www.commoncrawl.org/blog/host--a...
We are pleased to announce the release of the January 2026 crawl archive, containing 2.3 billion web pages, or 398 TiB of uncompressed content.
www.commoncrawl.org/blog/january...
Recently, a two-day Bristol datathon used Common Crawl web archives to analyse UK industries and policy, strengthening social science research through hands-on, team-based work.
www.commoncrawl.org/blog/web-arc...
As SEOs grapple with the shift from traditional Search Engine Optimization to AI visibility, they're discovering a resource that's been powering AI training for years: Common Crawl's Web Graph.
commoncrawl.org/blog/how-seo...
GneissWeb Annotations Examples
A new Common Crawl index annotation has been added to Hugging Face and our S3 bucket.
commoncrawl.org/blog/gneissw...
From the 6th to the 10th of November 2025, Pedro Ortiz Suarez attended Mozfest in Barcelona, as well as some satellite events.
www.commoncrawl.org/blog/common-...
We are pleased to announce a new release of host-level and domain-level web graphs based on the crawls of October, November, and December 2025.
commoncrawl.org/blog/host--a...
The crawl archive for December 2025 is now available, consisting of 2.16 billion web pages (or 364 TiB of uncompressed content).
commoncrawl.org/blog/decembe...
As another year here at Common Crawl comes to a close, we present a dozen papers from 2025 that demonstrate the range of topics and areas of study for which Common Crawl’s datasets are used and referenced.
commoncrawl.org/blog/a-sampl...
Laurie Burchell at a lectern presenting her Turing Seminar talk
Laurie Burchell at a lectern, with a blackboard behind her, presenting her Turing Seminar talk
A huge thank you to @very-laurie.bsky.social for delivering a fantastic UoB Turing seminar. Her talk was entitled “Common Crawl: open web data for everybody.”
In this talk, she introduced the @commoncrawl.bsky.social and the data products they offer.
We are pleased to announce the release of the web graphs based on the crawls of September, October, and November of 2025, consisting of 235.7 million nodes and 9.5 billion edges at the host level, and 100.7 million nodes and 6.6 billion edges at the domain level.
commoncrawl.org/blog/host--a...
We are pleased to announce that the crawl archive for November 2025 is now available, containing 2.29 billion web pages or 378 TiB of uncompressed content.
commoncrawl.org/blog/novembe...
Banner for the World Digital Preservation Day, 6th of November 2025
Common Crawl celebrates World Digital Preservation Day Nov. 6, which invites the community to unite in answering a powerful question: Why Preserve?
commoncrawl.org/blog/common-...
Setting the Record Straight
A recent article in The Atlantic makes several false and misleading claims about the Common Crawl Foundation, including the accusation that our organization has “lied to publishers” about our activities.
commoncrawl.org/blog/setting...
Check out our newsletter for October/November 2025, with updates on what we've been up to
commoncrawl.org/blog/october...
The Common Crawl team presented a seminar at Stanford HAI entitled “Preserving Humanity's Knowledge and Making it Accessible: Addressing Challenges of Public Web Data”.
commoncrawl.org/blog/common-...
We are pleased to announce a new release of host-level and domain-level web graphs based on the crawls of August, September, and October 2025, consisting of of 468.4 million nodes and 8.0 billion edges at the host level, and 97.7 million nodes and 6.0 billion edges at the domain level.
29.10.2025 18:45 — 👍 1 🔁 0 💬 0 📌 0
We are pleased to announce the release of the October 2025 crawl, containing 2.61 billion web pages or 468 TiB of uncompressed content.
commoncrawl.org/blog/october...
The Common Crawl team attended the 2nd Conference on Language Modeling in Montréal, organizing a workshop, giving invited talks, and strengthening links with the research community.
commoncrawl.org/blog/common-...
If you were able to join us, let us know about your experience: docs.google.com/forms/d/e/1F...
10.10.2025 20:52 — 👍 4 🔁 4 💬 0 📌 0
Thank you everyone for coming to WMDQS (pronounced "whim ducks")!
10.10.2025 20:50 — 👍 3 🔁 2 💬 1 📌 0
After lunch, @sebnagel.bsky.social gave a keynote about the data collected by @commoncrawl.bsky.social!
10.10.2025 20:46 — 👍 2 🔁 1 💬 1 📌 0
WMDQS is underway! Come join us in Room 520A at @colmweb.org! #COLM2025
10.10.2025 16:17 — 👍 2 🔁 3 💬 1 📌 0Looking forward to tomorrow's #COLM2025 workshop on multilingual data quality! 🤩
09.10.2025 23:16 — 👍 6 🔁 3 💬 0 📌 0
In collaboration with @commoncrawl.bsky.social, MLCommons, and @eleutherai.bsky.social, the first edition of WMDQS at @colmweb.org starts tomorrow in Room 520A! We have an updated schedule on our website, including a list of all accepted papers.
09.10.2025 20:17 — 👍 3 🔁 3 💬 1 📌 1
Common Crawl has added IBM’s GneissWeb quality and category annotations to its web dataset, enabling users to filter high-quality content and explore topics like medical, education, and technology.
commoncrawl.org/blog/announc...
Common Crawl’s Web Languages initiative has had many contributions since its introduction. We’re calling for native speakers of certain languages to review language contributions, to ensure that links we’re adding to our seed crawl are of good quality.
commoncrawl.org/blog/web-lan...
We are pleased to announce a new release of host-level and domain-level web graphs based on the crawls of July, August, and September 2025. The host-level graph consists of 628.7 million nodes and 6.9 billion edges, and the domain-level graph consists of 184.6 million nodes and 5.4 billion edges.
02.10.2025 22:04 — 👍 2 🔁 1 💬 0 📌 0
The era of traditional search engine optimization is rapidly evolving into "AIO" (AI optimization), where businesses must ensure their content exists in AI training datasets to remain discoverable as users increasingly turn to AI assistants for answers.
commoncrawl.org/blog/from-se...
We are pleased to announce the release of our September 2025 crawl, containing 2.39 billion web pages, or 421 TiB of uncompressed content.
www.commoncrawl.org/blog/septemb...
