Photo by NASA.
Aaron R. Williams and Claire McKay Bowen
One of our favorite data sources is weekly unemployment insurance (UI) claims from the Department of Labor. The data demonstrate the value of administrative data and involve coordinating across all fifty states, the District of Columbia, and U.S. territories every week to provide the real-time evidence decisionmakers need to understand and support the economy.
America’s data infrastructure has had a tough year as funding cuts have strained the statistical system, the federal government has shed data and statistical expertise, statistical agencies have been politicized, and several data sets have disappeared. Given these changes, data wonks have emphasized the need for states to better coordinate and collaborate on data. Although the use of state data is promising, maximizing the value of these data requires leadership and resources.
State Administrative Data Are Valuable
State administrative data offer tremendous value to policymakers. Weekly UI claims data, for example, have been used to understand the rapid deterioration of the economy during the COVID-19 pandemic, trigger extended UI benefits in periods of high unemployment, and assess underreporting in federal household surveys.
Why do states have so much valuable administrative data? Administrative data are collected by government agencies while administering programs such as education, UI, Medicaid, SNAP, and TANF. Because many of these programs are administered at the state level, states collect and store large volumes of data. Although these data were not collected to support evidence-based decisionmaking, they are often essential for it. In the case of weekly UI claims, federal leadership means the data are coordinated and harmonized across different state agencies in a way that is broadly useful to inform policy.
In most cases, however, these state-level data are not only siloed across states but also within states, where one state agency’s data are often inaccessible other agencies in the same state. As a result, states lack useful comparisons because other states either don’t report the same data or use different data collection processes or standards—if they have standardized processes at all. The lack of comparison means states often struggle to understand changes in their own data, whether due to cross-state migration or because many people work or learn across the border from where they live. Without coordinated state administrative data, federal agencies would struggle to produce useful nationwide data and statistics that are useful for evidence-based decisionmaking.
Examples of State Coordination
We highlighted three types of state data silos: within‑state data sharing, between‑state data sharing, and state‑to‑federal data sharing. Although these challenges are far from resolved across all states, some are beginning to lead the way.
Within-state data sharing
More states are working to link their internal data sources to support better decision‑making. Indiana offers a strong example: its Department of Workforce Development created Pivot, an AI‑enabled tool that connects data from the Department of Education, the Commission for Higher Education, and the state’s longitudinal data system. Pivot uses wage records and labor‑market trends to predict future sector demand and match job seekers to personalized career pathways. The tool can compare individuals with similar backgrounds, identify skills aligned with higher‑wage opportunities, and recommend relevant training programs.
Between-state data sharing
For state-to-state data sharing, one of our favorite examples of an effort led by state governments is the Multi-State Postsecondary Report and multi-state data collaborative for state longitudinal data systems (SLDSs). The ongoing Multi-State Postsecondary Report is a collaboration between New Jersey, Tennessee, Virginia, Kentucky, and Ohio to explore post-secondary completion and cross-state employment outcomes. The states used the Administrative Data Research Facility to combine SLDS data and report the metrics through an interactive dashboard. However, five states are far from covering all fifty states, the District of Columbia, and the U.S. territories.
State-to-federal data sharing
There are several federal efforts that coordinate and harmonize state data and then regularly report public data that is useful for evidence.
Table 1: A list of data sources that contribute to the weekly unemployment insurance (UI) claims from the Department of Labor.
| Dataset | Domain | Agency | Update Frequency |
| Initial Unemployment Insurance Claims | Employment | DOL ETA | Weekly |
| Current Employment Statistics | Employment | BLS | Monthly |
| Local Area Unemployment Statistics | Employment | BLS | Monthly/Annual |
| Quarterly Census of Employment & Wages | Employment | BLS | Quarterly |
| National Vital Statistics System Births/Deaths | Health | CDC/NCHS | Annual |
| Behavioral Risk Factor Surveillance System | Health | CDC | Continuous collection; Annual data files |
| Common Core of Data | Education | NCES | Annual |
| Integrated Postsecondary Education Data System | Higher Education | NCES | Tri-annual collection; Annual releases |
| NIBRS via Crime Data Explorer | Crime & Justice | FBI | Monthly/Annual |
| Fatality Analysis Reporting System | Transportation | NHTSA | Annual |
| SNAP Participation & Benefits | Social Programs | USDA FNS | Monthly/Annual |
| Medicaid & CHIP Enrollment | Health | CMS | Monthly/Quarterly |
Although we could not find any comparable example of efforts led by states, nonprofits, or philanthropy, such efforts may exist but are not widely known. Many states have All-Payer Claims Databases (APCD) that collect and aggregate health care claims, eligibility, and provider data from various public and private insurers. The APCD Council works to coordinate state efforts including data harmonization – but there is no example of centralized national reporting.
The National Association of Boards of Pharmacy’s Prescription Monitoring Program InterConnect facilitates state sharing of data from Prescription Drug Monitoring Programs (PDMP). Some individual states, like Pennsylvania, report PDMP data – but there are no national reports or data series that leverage these data.
Maximizing the Value of State Data
It isn’t surprising that the federal government has many more examples of coordinating states for data collaboration given the “carrots” and “sticks” at their disposal. The federal government can award grants and provide technical support to states while setting and maintaining these collaborations. They can also mandate or set regulations for state participation, or they can leverage legal levers to bring together sensitive microdata from multiple states. Any individual state or nonprofit, by contrast, would need to navigate a complex web of various state laws and regulations to achieve this coordination.
We strongly believe that coordinating states for data collaboration is a worthwhile endeavor – but doing this successfully requires resources and strong leadership at the state level. To cultivate and empower that potential state-level leadership, a few things could help.
- Efforts can build on existing infrastructure. For example, the SLDS multi-state data collaborative builds on federal investments in these systems and the legwork that states already do to meet administrative reporting requirements.
- Efforts can identify carrots, including building shared infrastructure to reduce costs for states or defining areas where access to data from other states can improve policy outcomes.
- Efforts can build broad buy-in from multiple partners, including philanthropic funders, state offices, and data users.
States have valuable, but underused, administrative data that can improve our understanding of key social and economic policy areas and improve evidence. If the federal government continues to be less involved in establishing and maintaining state data collaborations, new leadership will be needed to maximize the value of state administrative data.