Profile image
By Sunlight Foundation Blog (Reporter)
Contributor profile | More stories
Story Views

Last Hour:
Last 24 Hours:

Sharing sensitive data within government

Wednesday, February 11, 2015 16:26
% of readers think this story is Fact. Add your two cents.

(Before It's News)

By The Sunlight Foundation

Graphic showing globally connected online devices

(Image credit: Free Press/Flickr)

While institutional research projects depend on review by Institutional Review Boards (IRBs), data-sharing that occurs within governments is authorized in a different way. Privacy laws do often restrict the sharing of identified data to others within the agency that created it, but there are also generally exceptions to this rule for the purpose of evaluating public-funded services. Agencies can sign Memorandums of Understanding (MoUs) to describe the specific, legally-compliant rationale for sharing data, to ensure that the privacy and confidentiality of the data will be maintained, and to describe the roles and responsibilities of the agencies in the data-sharing process.

The frequency with which MoUs are used to coordinate interagency data-sharing varies by policy sector, state and city. Juvenile justice is a popular area for data-sharing, although a Juvenile Justice GPS survey found that only 27 states coordinate data-sharing across their child welfare and juvenile justice systems, with 19 of those states using MoUs as the formal basis for the collaboration. Health and corrections systems are another space for increased cooperation through data-sharing, with states like Texas and Connecticut being good examples of states accomplishing data-sharing for the purpose of better supporting inmates’ health through the implementation of MoUs. However, there is substantial room for increasing data-sharing between health and corrections agencies, and the Bureau of Justice Assistance provides helpful resources for government agencies that have yet to negotiate these agreements.

Where state and local agencies do not (or cannot) implement MoUs, state and federal privacy laws can be modified by state and local legislation. If a state agency is interested in having access to data from another agency, for example, and that sharing is potentially prohibited by a data privacy law, the agencies can make their case to the state legislature and get a bill passed to make data-sharing possible. (Minnesota, for example, recently enabled limited interagency county-level data-sharing through legislation.)

Legislation also represented an important early approach to achieving interagency data sharing before these arrangements were more commonplace. Two different criminal justice initiatives were leaders in interagency data-sharing but, perhaps because of their novelty, both heavily used legislation as a means of ensuring that agencies could legally share personally identifiable information (PII). Because of the specific, rehabilitative aims of juvenile justice, juvenile justice-related agencies (courts, detention, probation and social service) served as an early site for interagency information sharing. Statewide juvenile justice information sharing initiatives emerged in the early 1990s, with roots as far back as Maricopa County’s 1979 development of a countywide juvenile justice data-sharing initiative. Through the mid-1990s, 35 states passed laws to permit data-sharing or to seek data-based evidence in order to improve outcomes connected to juvenile justice. (The federal Office of Juvenile Justice and Delinquency Prevention has for some time offered guidelines for the development of juvenile justice information sharing programs.) Similarly, State Justice Information Sharing initiatives, which are efforts to link the databases of state law enforcement, prosecutors, courts and corrections agencies, have been under development since the 1990s and were mainly created through state law. Data-sharing between agencies can also be mandated by court order or achieved informally, but with the experience of several decades of interagency data-sharing, as well as the present array of existing resources and models, one would hope that attaining formal agreements or statutory change is within reach for most potential interagency initiatives.

Where interagency data-sharing relationships are established, governments can produce highly useful, policy-relevant analyses with the data they produce. For example, the Washington State Department of Corrections uses integrated criminal justice data, workforce data and some health and social service data to perform research (as well as make data available for external research.) As a result, the department is able to investigate such questions as the relationship between education programs and employment outcomes for their inmates.

Cooperation between governments and research institutions

By putting these formal and informal governance elements together, governments and external research organizations are able to find legal ways to work together and obtain valuable benefits from data integration and analysis. Research institutions, vetted by their IRBs and sometimes working as a research consortium, are eligible to respond to formal governmental RFPs connected to data analysis or to propose their own projects using sensitive government data. Governments can then sign data-sharing MoUs with research institutions that let them legally share sensitive data and allow them to get more effective and efficient use out of their data sets, since these external research partners are more likely to have the necessary expertise to get the most from governmental administrative data. Meanwhile, although research consortia are potentially attractive partners for governmental data-holders, they can also create complications legally, since MoUs and data-sharing agreements can be challenging to write as multi-party documents.

Again, the role of mutual trust appears to play an important role in the development of productive public-private partnerships around data integration and analysis. MoUs are often written with specific language to both limit the potential for releasing PII as well as to ensure that the government won’t be faced with additional liabilities for sharing data with an external group; for example, agencies can require that organizations submit potential publications for pre-approval to make sure the agency is comfortable with releasing them. Public-private collaborations can scale up over a series of projects as external groups demonstrate their utility and reliability with the data. Especially if an agency will need to advocate for a legislative measure in order to be permitted to share data, it seems to be useful to have an established relationship that lets the agency answer questions about the external group’s trustworthiness.

As governments seek to get the most benefit from the data they hold, they often need external partners to help them merge data sets together across departmental lines. These projects can be known as “longitudinal data systems” (which tends to refer to an education-focused merged data project) or “integrated data systems” (which carries the connotation of broader cross-agency involvement.) Getting underway chiefly within the last decade, these projects aim to merge separate data sets effectively by: using best available practices for matching records across data sets; cleaning data and assuring data quality; ensuring the continued protection of PII within merged data; and using the appropriate statistical methods to gain policy-relevant insights from the data. Because this work occurs between, and not within, individual agencies, and because of the specific expertise required to perform those tasks, many governments are choosing to create data-sharing relationships with individual external research organizations or consortiums in order to achieve the best outcome.

The example of California’s partnership with research organizations around county correctional data provides a good example of how collaboration with an external partner can substantially improve data collection and use. As a result of California’s passage of realignment legislation in 2011, the state diverted tens of thousands of inmates from the state to county correctional systems. A major focus of the legislation, in addition to reducing the overcrowding in California prisons, was to identify ways to reduce recidivism. However, in order both to track how the policy change has affected the total state correctional population, as well as to identify any potential interventions that improve recidivism rates, the counties need to implement new methods of data collection and storage. The Public Policy Institute of California has partnered with a number of California counties to help them standardize their correctional data collection and storage, thereby enabling the state to achieve its stated goals.

While the data-sharing that enables these collaborations is not open to the public, it is nonetheless very important to be aware of it because of the benefits that these integrations can achieve for creating policy-relevant information. In addition, this trend suggests that a research institution can serve as a kind of data intermediary which can eventually provide open access to data that’s transformed or aggregated in a way that both retains useful qualities and protects individual anonymity. For example, the Ohio Education Research Center supports the creation of Ohio’s state longitudinal data system and also stores the data as the state longitudinal data archive, a data source to which researchers can apply for access to data. Because the center has expertise in managing data confidentiality, they are trusted to ensure that the data they release to researchers is appropriately secured. They do this in two ways: First, they require researchers to apply for access to the data, including requiring a legal data-sharing agreement and review by the researcher’s IRB. Second, the data the center releases is deidentified and stripped of any variables that could allow individual reidentification.

The National Opinion Research Center (NORC) at the University of Chicago also serves an intermediary function for researchers, although much more broadly and ambitiously. Recognizing both the value of increasing researcher access to individual-level data and the legal concerns about individual identification, the institute has created a “data enclave” that allows researchers who have applied and been accepted for access to use individual-level data from NORC’s extensive collection of government-sponsored survey and administrative data. Researchers access the data through a virtual terminal that keeps all of the data physically located on NORC’s servers. In this way, without having direct access to the files, researchers are nonetheless able to query the data and receive statistical results, after results have been checked to ensure that releasing them does not violate individual confidentiality.

Research organizations can also serve as intermediaries not just between researchers and identified government data, but also between identified government data and the general policy audience. For example, the Providence Plan, which has signed data-sharing agreements with Rhode Island agencies in order to support their data integration efforts, also performs analyses that they then turn into “data stories.” Data stories, such as this exploration of factors predicting youth involvement in the juvenile justice system, use locally gathered data to answer policy questions in an interactive and easily shared format. With external sharing of data, even in an aggregated and transformed state, organizations need to be especially cautious to not inadvertently release any data that could lead to individuals’ identification, so organizations like the Providence Plan have a series of controls on the data they publish, including conservative cell-size restrictions and qualitative checks, that specifically look to eradicate any potential for reidentification.

The Sunlight Foundation is a non-profit, nonpartisan organization that uses the power of the Internet to catalyze greater government openness and transparency, and provides new tools and resources for media and citizens, alike.


Report abuse


Your Comments
Question   Razz  Sad   Evil  Exclaim  Smile  Redface  Biggrin  Surprised  Eek   Confused   Cool  LOL   Mad   Twisted  Rolleyes   Wink  Idea  Arrow  Neutral  Cry   Mr. Green

Top Stories
Recent Stories



Email this story
Email this story

If you really want to ban this commenter, please write down the reason:

If you really want to disable all recommended stories, click on OK button. After that, you will be redirect to your options page.