public_data.md

Public Data Sources

• Open data catalogs from various governments and NGOs:
  • NYC Open Data
  • DC Open Data Catalog / OpenDataDC
  • DataLA
  • data.gov (see also: Project Open Data Dashboard)
  • data.gov.uk
  • US Census Bureau
  • World Bank Open Data
  • Humanitarian Data Exchange
  • Sunlight Foundation: government-focused data
  • ProPublica Data Store
• Datasets hosted by academic institutions:
  • UC Irvine Machine Learning Repository: datasets specifically designed for machine learning
  • Stanford Large Network Dataset Collection: graph data
  • Inter-university Consortium for Political and Social Research
  • Pittsburgh Science of Learning Center's DataShop
  • Academic Torrents: distributed network for sharing large research datasets
• Datasets hosted by private companies:
  • Quandl: over 10 million financial, economic, and social datasets
  • Amazon Web Services Public Data Sets
  • Kaggle provides datasets with their challenges, but each competition has its own rules as to whether the data can be used outside of the scope of the competition.
• Big lists of datasets:
  • Rdatasets: collection of 700+ datasets originally distributed with R packages
  • RDataMining.com
  • KDnuggets
  • inside-R
  • 100+ Interesting Data Sets for Statistics
  • 20 Free Big Data Sources
• APIs:
  • Apigee: explore dozens of popular APIs
  • Python APIs: Python wrappers for many APIs
• Other interesting datasets:
  • FiveThirtyEight: data and code related to their articles
  • Donors Choose: data related to their projects
  • 200,000+ Jeopardy questions
• Other resources:
  • Datasets subreddit: ask for help finding a specific data set, or post your own
  • Center for Data Innovation: blog posts about interesting, recently-released data sets.

This is just the tip of the iceberg; there's a lot of data out there!