Guides: Government Information & Data Rescue: Tools for Data Rescues

Data Rescue Activist Tools

These data activist organizations focus on rescuing and preserving data:

ArchiveBox has archived datasets from data.gov, CIBP, USCIS, NOAA, NASA, NSIDC
Archive Team is focusing on archiving datasets from the U.S. Government
DIY Web Archiving shows why everyone should participate in preserving the things on the web they care about, and how they can do so
GitHub: Awesome Datahoarding provides lists of tools for web harvesting
GovDiff shows side-by-side comparisons of government website changes
MIT Libraries: Data Management Checklist provides a checklist for curating data rescue efforts
r/Data Hoarder is a subreddit of data preservation activists
Safeguarding Research Discourse Group is a preservation group hosted outside of the US
WebRecorder has archived 8TB+ of government sites

#RStats
Package from @ropensci.org: gitcellar downloads and archives all repos, issues, and PRs from a GitHub organization in one shot: docs.ropensci.org/gitcellar/
ArchiveBox
ArchiveBox is an open source tool that lets organizations & individuals archive both public & private web content while retaining control over their data. It can be used to save copies of bookmarks, preserve evidence for legal cases, backup photos from FB/Insta/Flickr or media from YT/Soundcloud/etc., save research papers, and more.
Archive Team Warriors
This website is intended to be an offloading point and information depot for a number of archiving projects, all related to saving websites or data that is in danger of being lost. Besides serving as a hub for team-based pulling down and mirroring of data, this site will provide advice on managing your own data and rescuing it from the brink of destruction.
Awesome Web Archiving
Curated list of web archiving tools.
DIY Web Archiving
A 23-page standard-size, full-color zine co-created by Quinn Dombrowski, Tessa Walsh, Anna Kijas, Ilya Kreymer, and Amanda Wyatt Visconti. DIY Web Archiving shows you why everyone should participate in preserving the things on the web they care about, and how anyone can do so (no special expertise required!). Based on the 11/25/2024 virtual workshop co-sponsored by WebRecorder, SUCHO, and ACH; taught by Walsh, Kreymer, Dombrowski, & Kijas; with additional text/resources from & zineified by Amanda Wyatt Visconti.
GitHub: Awesome DataHoarding
Open-source list of tools related to archiving.
GovDiff
Tool built by Ben Borgers and Jerome Paulos to compare government websites before and after inauguration.
Inter-University Consortium for Political and Social Research (ICPSR)
MIT Libraries: Data Management Checklist
This checklist provides steps you can take to ensure the government data you use in your research remains accessible to you and others.
r/DataHoarder
This is a subreddit of data preservation activists that aims at bringing data hoarders together to share their passion with like minded people.
Safeguarding Research Discourse Group
Organizing to safeguard as much publicly available research, including GLAM (Gallery, Library, Archive, and Museum)-collections, as possible. Led by Henrik Schoenemann and hosted in Germany.
WebRecorder
The Webrecorder team has been developing open source web archiving tools for over 10 years. We strive to make web archiving tools accessible to anyone who needs to collect and preserve content that is meaningful to them. WebRecorder developed the ability to replace“.gov” with “.govarchive.us” on any of the supported government sites to seamlessly browse archived versions, including sites like USAID that are now gone.

Government Information & Data Rescue: Tools for Data Rescues

Data Rescue Activist Tools

Contact a Librarian