Guides: Data & Statistics: Large Datasets

What are Large Datasets?

For the purposes of this guide, these are sets of data that may be from large surveys or studies and contain raw data, microdata (information on individual respondents), or all variables for export and manipulation.

Macrodata vs. Microdata

Macrodata

In economics and statistics, macrodata are data at the aggregate or summary level. Macrodata are composed of a combination of several measurements or observations.

Examples of macrodata include:

The average education level for a census tract, county, or state
The overall inflation rate for a country
The CO2 emissions for a metropolitan area

Microdata

As opposed to macrodata, microdata are the data in their raw form, at the level of individual observations or measurements. In this type of data, each row represents an individual observation or measurement.

Examples of microdata include:

The General Social Survey
The raw output from an emissions sensor
Public-use microdata samples from the United States Census

Sources for Finding Large Datasets

There are numerous datasets available from government agencies, organizations, and individual researchers. Try these sources to find them:

A Guide to International and US Statistics Sources
List of major sources for datasets with descriptions and links. Page from the CISER Data Archive at Cornell Institute for Social and Economic Research.
Data.gov
'Find, download, and use datasets that are generated and held by the Federal Government.'
HealthData.gov
U.S. government website with links to health-related datasets from a variety of health agencies.

ICPSR (Inter-university Consortium for Political and Social Research)
ICPSR provides access to a large archive of social science data files for use with statistical software, such as SAS or SPSS. Includes some public opinion polls. In order to download data, you must create a MyData account with ICPSR, using a middlebury.edu email address. Tutorials are available on YouTube.

Harvard Dataverse
"Open for all researchers worldwide from all disciplines to deposit data."

iPoll (Roper Center)
Public opinion survey data from the Roper Center at the University of Connecticut. iPoll is organized at the question level, providing the tools to sift through nearly a half million questions asked on national public opinion surveys, 1935 to present. Includes RoperExpress (offers downloads of over 20,000 datasets from over 100 countries to use with statistical software to conduct bivariate and multivariate analysis) and Roper Explorer (online analysis of several hundred studies allowing cross-tabulations without specialized statistical software). To download datasets, register to create an account and agree to their terms about confidentiality, etc. Roper provides educational material for using their tools and learning the basics about polling and analysis.

IPUMS (Integrated Public Use Microdata Series)
'Census microdata for social and economic research' from the Minnesota Population Center at the University of Minnesota. Provides access to: IPUMS-USA (Census and ACS microdata samples from 1850-2013+ and complete count microdata for 1880 and 1940), IPUMS-International (Harmonized data for 1960 forward, covering 258 censuses from around the world), and IPUMS-CPS (Current Population Survey - monthly data 1989 - 2013+).
World Bank Microdata Catalog
"Facilitates access to data collected through sample surveys of households, business establishments or other facilities. These 'microdata' sets may also originate from population, housing or agricultural censuses or through an administrative data collection processes. "
European Union Open Data Portal
Free data produced by the institutions and other bodies of the European Union. Includes datasets from Eurostat, European Environment Agency, EuropeAid, European Banking Authority, and many more.
Datahub
Provides free access to datasets that use CKAN (data management system), including datasets from many government agencies and international organizations. Search for data, register published datasets, create and manage groups of datasets.
MacroData Guide
Guide to social science datasets from Norwegian Social Science Data Services. From the wayback machine- content is no longer updated
Responsible Datasets in Context
From the team: "Understanding the social and historical context of data is essential for all responsible data work. We host datasets that are paired with rich documentation, data essays, and teaching resources, all of which draw on context and humanities perspectives and methods. We provide models for responsible data curation, documentation, story-telling, and analysis."