Harvesting Data Together


Harvesting Data From the Centralized Web

In the context of Data Together, one prevalent pattern is the activity of communities Harvesting Data from centralized locations to other locations, often moving to decentralized systems.

This kind of harvesting is especially hard because the data are often arranged and exposed with the assumption that they will only ever be stored in a central location and will only ever be linked using location addresses.

Examples of Communities Harvesting Data

Please suggest additions for this list by submitting issues on github

  • Internet Archive, Archive Team & Archive Lab
  • EOT Harvest
  • Data Rescue hackathons, Stanford, Code for Science and Archive Team backing up data.gov
  • Kiwix

Tools for Harvesting Data

Please suggest additions for this list by submitting issues on github