Establishing the role of honest broker: bridging the gap between protecting personal health data and clinical research efficiency

Existing literature on honest brokers

Three researchers (2 honest brokers and 1 medical informaticist) performed a literature review to locate relevant evidence on successful processes or systems that use honest brokers to protect patient private data in clinical research. All literatures were independently reviewed and all data in Table 1 were extracted by the aforementioned researchers. Discrepancies were resolved through discussion. We examined articles from a wide variety of academic fields, including medical informatics, ethics, statistics, and computer science. The 44 articles, 6 SOPs, and 8 presentations identified were evaluated for inclusion in this study. Ultimately, 5 SOPs, 4 presentations, and 6 articles, all of which were originally published as full papers, met the inclusion criteria for this present research (Fig. 1).

Task force activities for protecting patient privacy data

To prepare for the amended 2012 Korean Bioethics and Safety Act, the AMC developed a Task Force (TF) in 2012–2013 to guarantee legal clinical research. The TF consisted of 15 stakeholders in clinical research (including the departments of biomedical informatics, laboratory medicine, and clinical departments (pulmonology and gastroenterology, and pathology), in addition to the bio-bank, information protection subcommittee, relevant IRBs, and human research protection center). All authors of this paper participated in the TF. The first authors, Ms. Choi and Ms. Lee, who are honest brokers for clinical data, presented the agenda on the necessity, roles, and responsibilities of honest brokers, and with co-author Mr. Lyu, who is working to implement the honest broker system, provided a discussion of the roles and tasks for the system.

The first authors, Ms. Choi and Ms. Lee, who are honest brokers for clinical data, presented the agenda on the necessity, roles, and responsibilities of honest brokers, and the co-author, Mr. Lyu, who is working to implement the honest broker system, provided a discussion of the roles and tasks for the system. Co-authors Dr. Choi, Dr. Lee, and Dr. Shin, who are from the Office of Clinical Research Information at the AMC and who use clinical data as researchers, have accordingly provided counterarguments on the types of data that need to be de-identified for secondary use as well as reasons why this is necessary. Corresponding authors Dr. Park and Dr. Yoo, who are specialists in healthcare informatics and biomedical law and ethics, respectively, have provided a discussion of measures for de-identification of patients’ information, and its legal feasibility.

Since the amended Korean Bioethics and Safety Act requires that (1) any research that uses information that can directly or indirectly identify personal information must receive an IRB review, and (2) consent must be obtained from all research subjects even in the case of retrospective research that does not correspond to the consent waiver condition stated in the Act, the TF conducted research on ways to satisfy the consent waiver conditions while also protecting patient privacy. The following conditions were agreed upon by the TF: (1) establishment of 21 items of institutional personal health information (PHIs) that need to be de-identified when AMC researchers share and use individual patient data for health research purposes; (2) confirmation of the needs, roles, and work scope of the people who act as honest brokers; (3) confirmation of the needs, roles, and work scope of the honest broker system.

Review of the literature on honest brokers

Ten organizations were selected from our literature review to evaluate the commonalities and differences in honest broker systems that are currently used in biomedical research (Table 1). Most of these organizations (9 out of 10) built systems to de-identify clinical data automatically. Only one organization (the University of Pittsburgh) maintains 33 certified persons to validate de-identified clinical data. The University of Pittsburgh operates an honest broker certification process that is composed of IRB-mandated education modules, such as Research Integrity, Human Subjects Research in Biomedical Sciences, and HIPAA Researchers Privacy Requirements (Dhir et al., 2008). In particular, the University of Pittsburgh also developed a unique cross-divisional and collaborative broker service for performing multisite clinical research. The basic aim of the honest broker in biomedical research is to de-identify private patient data, and thus almost all organizations target and de-identify the patient PHIs included in clinical data and maintain a master patient index. Of the 10 organizations we reviewed, four target human biomaterials data, such as those included in tissue banks. Most notably, the Duke Comprehensive Cancer Center targets molecular data, such as those gathered by genomic experiments. Interestingly, only six organizations are explicitly compliant with HIPAA privacy regulations, while the other four organizations do not disclose information regarding compliance.

Institutional definition of PHI

The first step to de-identifying clinical data is to define the list of prohibited identifiers that need to be removed. In the case of HIPAA, the “safe harbor” option can be used as an alternative to full de-identification. Under safe harbor, data are considered de-identified if 17 listed types of identifiers are removed (US Government Publishing Office, 2015a). However, in recognition of the restrictive nature of HIPAA safe harbor practices in clinical research (Clause et al., 2004), HIPAA makes a concession to research organizations by allowing data custodians to release “limited data sets (LDS)” (US Government Publishing Office, 2015b), from which direct identifiers (e.g., name, address, social security numbers, biometric identifiers) are removed but not indirect identifiers (e.g., date of birth, date of death, and treatment dates, which are included on the safe harbor list). The TF at the AMC adopted the safe harbor option from HIPAA, but added new identifiers (resident and foreigner registration numbers and biometric identifiers) as prohibited PHIs in order to comply with domestic laws (Table 2). A more detailed description of the 21 PHIs determined by the AMC are described elsewhere (Shin et al., 2015; Shin et al., 2013).

Specified roles of the honest broker system

At the AMC, ABLE (Asan BiomedicaL research Environment) has been used as the honest broker system (anonymized clinical research data warehouse) since March 2014. The role of ABLE is to support research and to allow for searching and viewing of clinical data after the subject identification information has been anonymized. The ABLE includes three main tools: the cohort tool, chart review tool, and extraction tool. First, prior to establishing research plans, the cohort tool identifies cohorts of research participants who correspond to the requested conditions. Using the cohort tool, the researchers only obtain the number of patients within the search conditions. Second, the chart review tool provides more detailed anonymized information. Using the provided research IDs, the user (i.e., researcher) can access de-identified patient charts (including diagnosis, medication, and examination results, prescriptions, radiology reports, operation reports, admission notes, and discharge summaries). Lastly, the “extraction of anonymized clinical data tool” can download the identified results in Excel format. At that time, the downloaded results must link with the corresponding research that approved by the AMC IRB. According to the AMC policy, the ABLE restricts tool access only to our institution’s researchers who are qualified in GCP and ABLE user training. The ABLE has also recorded all data access logs in the above three tools. Based on these logs, honest brokers have monitored investigators’ violations.

Qualification, Status, and Obligations of honest brokers at Asan Medical Center

The AMC’s first honest broker was introduced in 2005 with the establishment of the Bio-Resource Center (BRC) for managing biomaterials and associated data. The first honest broker in charge of managing clinical data was recruited in May 2010. Following the introduction of the honest broker system and resulting task force for protecting patient privacy data, additional honest brokers were recruited in February 2014.

Honest brokers at the AMC are subdivided into handlers of clinical data or human biomaterials; this is because the characteristics of the two types of de-identifiable data differ, and thus it is advisable for a specialist who is knowledgeable of the characteristics of each type of data to be responsible for de-identification. Subsequently, the requirements for the position are divided into common essential requirements and individual essential requirements. The AMC has concluded that in the case of clinical data, the honest broker must be proficient in IT and capable of conducting validation through the honest broker system (ABLE) with at least 5 years of experience working in a hospital, whereas the honest broker for human biomaterials and associated data would be required to have knowledge of remaining tissues, fluids, and blood, with a license in clinical pathology or a degree in biotechnology.

It is also a mandatory requirement for all AMC honest brokers to complete the Good Clinical Practice (GCP) course, Health Information Privacy and Security (HIPS) course, and IT method for clinical data extraction course before starting their tasks. They are also required to retake these courses every two years for the continuation of their employment. The AMC IRB reviews whether the honest broker has completed the mandatory training courses. In addition, both types of honest brokers must disclose their present and potential conflicts of interest to the AMC IRB before their appointment.

In the case of the honest broker for clinical data, the Office of Clinical Research Information reviews the candidate’s qualifications, after which the President of the AMC appoints the candidate as an honest broker. In the case of the honest broker for biomaterials, the BRC reviews the candidate’s qualifications, after which the President of the AMC makes the appointment. All honest brokers at the AMC function solely as honest brokers, a full-time occupation.

All honest brokers at the AMC have the duty of maintaining the confidentiality of the type of data provided to researchers. This duty comes into effect when an honest broker signs an “Information Protection Confidentiality Agreement.” This Agreement stipulates that the honest broker must take all responsibility for the leakage of any information managed by the honest broker under the AMC SOP and the Korean Personal Information Protection Act. An honest broker who fails to maintain the confidentiality of the data risks being penalized according the SOP and the Act.

Specified roles of honest brokers at Asan Medical Center

The honest broker for clinical data must perform the following operations when using ABLE: (1) monitor the anonymity verifications of the honest broker system; (2) provide user training; (3) oversee investigator violations such as mismatches between extracted clinical data and IRB-approved research; (4) consult with investigators regarding the method of ABLE use; (5) provide de-identified data when requested (e.g., data collected by government organizations such as the Ministry of Government Administration and Home Affairs, Statistics Korea, Health Insurance & Assessment Services, and National Insurance Services); and (6) report monitoring/inspection results regarding ABLE to the IRB twice yearly.

The honest broker for human biomaterials performs the following functions: (1) provide coded IDs instead of registration numbers for patients who provided biomaterials, and (2) provide de-identified human biomaterials to the corresponding investigators when requested. The honest broker in charge of human biomaterials handles around 250 cases annually. Table 3 summarizes the functions and features of honest brokers and the honest broker system (ABLE).

At the AMC, honest brokers have monitored the anonymity verifications of the ABLE system since December 2013. Monitoring of the anonymity verifications of the system by honest brokers should be conducted four times per year. Oversight of investigator violations is performed on 1,800 requests for clinical data extractions per year. Based on the log of the ABLE, the honest broker retrospectively monitors about 650 requests per month. Figure 2 shows the anonymity verification results of the AMC ABLE system. Over 15 months, anonymity verifications were conducted five times.

The performance of the anonymizing function of the ABLE system has dramatically declined since September 2014 (precision = 69.93; recall = 71.81; F0.5 score = 70.3). We have selected 1,275 clinical documents (701, 471, and 103 documents for inpatient, outpatient, and emergency room, respectively) for a second anonymity verification (Table S1). Of 195 identifiers in 1,275 clinical notes, 107 identifiers were correctly masked, 46 non-identifiers were incorrectly masked, and 42 identifiers were not masked. All 88 of the incorrectly identified were found in narrative text in clinical documents, new patterns for describing PHI that are hard to detect by a de-identification algorithm without updating PHI detection rules. After the first honest broker system update, the anonymizing function of the third anonymity verification results (precision = 99.87; recall = 96.25; F0.5 score = 99.12) demonstrated similar accuracy to the first anonymity verification result (precision = 98.66; recall = 99.33; F0.5 score = 98.8). Three months later, the performance of the ABLE system had declined slightly by the fourth anonymity verification (precision = 99.33; recall = 97.7; F0.5 score = 99). After the second honest broker system update, the anonymizing function demonstrated the highest accuracy (precision = 100; recall = 99.05; F0.5 score = 99.81) among all anonymity verifications. A detailed validation data set and the PHIs used for anonymity verification are provided in Table S1.

Honest brokering processes and publication rules at Asan Medical Center

Honest brokering at AMC requires collaboration between humans and systems (Fig. 3). Figure 3A details the honest brokering processes specified by the AMC Honest Broker SOP for both the ABLE system and humans, and for clinical and human biomaterials data. Regardless of the type of study, it is a precondition at the AMC that the investigator obtain AMC IRB approval before conducting a secondary study on existing clinical or human biomaterials data without consent. Accordingly, the AMC IRB must first verify the ethical/legal suitability of the request.

The AMC IRB reviews whether the research protocol of using data for secondary research without the consent of the patient is legal and ethically appropriate. The honest broker provides consultation to the IRB prior to their deliberation as to whether the de-identification of the relevant data is practically or technically feasible, or examines this afterwards. For example, some clinical images are difficult to de-identify due to technical limitations. In this case, even after approval by the IRB, if the request is practically infeasible for de-identification the honest broker reports the results to the corresponding investigator and the IRB.

Figure 3B shows the processing details for data collected by public institutions. If the investigator requests public data (e.g., death or other data about a patient) and clinical data available at the AMC, initially the honest broker for clinical data and the IRB verify the suitability of the request. Second, if the request is appropriate, then the honest broker identifies and obtains the patient lists that meet the requested conditions from the honest broker system. Third, when the honest broker requests public data to public institutions with patient lists, then the public data provider provides anonymized public data that the investigator requested. Finally, the honest broker provides both the anonymized clinical data from the AMC and the public data to the corresponding investigator.

As above, data that has been de-identified for the purpose of conducting studies without the consent of the patient must be guaranteed to remain de-identified when the study results are made public. The researcher receives approval from the AMC IRB on the condition that he complies with the above. Thus, researchers are unable to provide raw data or original data to the public when publishing or presenting the results of the study. This is because the purpose of de-identification is to protect the personal information of the subject whose data is used without consent, whereas the release of raw data to the public is a form of re-identification, which means that the subjects’ confidentiality is not fully guaranteed.

Establishment of conditions for optimal roles of the honest broker

The roles of honest brokers operating at 10 other representative institutions, including both human and system (computer-based), were analyzed for comparison with the AMC in order to extract optimal roles of honest brokers. Optimal roles were determined for the following functions: (1) sufficient de-identification of clinical data (Boyd et al., 2007); (2) sufficient independence from any form of study team (Rothstein, 2010); (3) thorough examination of secondary use of clinical data for purposes restricted to those approved by the IRB (Benitez & Malin, 2010); and (4) efficient provision of de-identified data to researchers (Patel et al., 2007).

The first condition means that the honest brokers’ role is to be able to sufficiently de-identify PHIs or other clinical identifiers and so ensure regulatory compliance before the release of information to investigators. Furthermore, he or she should monitor the anonymity verification periodically. The second condition means that honest brokers should be an independent party completely separated from researchers, and the only entity able to track the codes linked to clinical identifiers. The third condition means that they should prohibit the use of clinical data for purposes not approved by IRB and report to the IRB in cases when anonymization will not be possible. The fourth condition emphasizes the need for efficient delivery of de-identified data to researchers in a timely and accurate manner.

One condition can be distinct from another, but there is no need for these conditions to be independent from each other. For example, “sufficient de-identification of clinical data” and “efficient provision of de-identified data to researchers” are tasks that can be interconnected. When a researcher wishes to conduct a retrospective study through chart reviews without the consent of a patient, the honest broker de-identifies the data to the extent possible and then delivers this data to the researcher. Hence, the four conditions may affect each other, can be interconnected, and can also facilitate the execution of each other’s roles.

All of these conditions determine the actual success or failure of honest brokers. Here, “sufficient,” “thorough,” or “efficient” does not mean “greatest” or “maximum,” but denotes “optimal” practices that honest broker systems need to adopt in order to protect information and promote clinical research with limited resources.

Evaluations of roles of honest brokers at AMC according to conditions

In the case of the AMC, the introduction of the ABLE somewhat paradoxically emphasizes the role of the human component of this system. One of the main functions of the honest brokers is system monitoring for anonymity verification. At the AMC, an enormous amount of anonymity validations were conducted by honest brokers before the ABLE was introduced (Shin et al., 2013). Since most clinical data consist of narrative text, new patterns often emerge. In order to operate a robust and consistent honest broker system, validation strategies need to be established and periodic checkups performed (Fig. 2). At the AMC, after honest broker monitoring, the accuracy of the anonymizing function of ABLE highly increased, although it was still not 100% (i.e., partial satisfaction of condition 1).

In addition, even though the anonymizing function had improved, there still remains an important and unique role for an honest broker in the overall system: that of providing education and consultations to researchers in order to improve the acceptance and use of the ABLE, auditing the authorizations to access the system, and reporting monitoring/auditing results to the IRB (i.e., satisfying condition 3).

The aforementioned roles of the honest broker have wide applicability outside of Korea. There is, however, a role that must be specifically applied to honest brokers in Korea due to various legal requirements. In Korea, government organizations must authorize the release of patient death information. In this case, personal identifiable information collected by research institutions (i.e., the resident registration number (RRN)) must be provided to the corresponding government organization. The honest brokers at the AMC send RRNs to these agencies when there are requests by researchers, who then receive this information from those organizations in a pseudonymized form. In other words, the honest broker at AMC fulfills the role of an “honest” intermediary between government organizations and researchers, such as blocking any possible re-identification by researchers (i.e., satisfying condition 2).

The honest broker frameworks, both system and human, will need to be enhanced in the near future, as reflected by their limitations at AMC and other institutions. In the case of the AMC, the ABLE cannot de-identify some clinical images, such as endoscopic and ultrasonic photographic images that do not comply with DICOM standards (Pianykh, 2009) (i.e., therefore not satisfying condition 1). The variety of honest broker systems, as shown in Table 1 (such as the anonymized image viewer used by the University of Chicago), highlight the need to improve the function of the honest broker process at the AMC. In addition, the University of Pittsburg’s cross-divisional and collective brokering service features an honest broker (both human and system) that is a part of the facility, provides de-identified data to departments/divisions, aggregates and provides specimens and data, and designs software tools for these purposes. The policy that the honest broker system encompasses several separate departments and divisions reduces possible conflicts of interest. Given the fact that fostering an enhanced research output is one of the essential conditions for the honest broker, cooperation between individual honest brokers working in different departments should be actively considered (which is needed to satisfy condition 4).