DICOM for quantitative imaging biomarker development: a standards based approach to sharing clinical data and structured PET/CT analysis results in head and neck cancer research

Background. Imaging biomarkers hold tremendous promise for precision medicine clinical applications. Development of such biomarkers relies heavily on image post-processing tools for automated image quantitation. Their deployment in the context of clinical research necessitates interoperability with the clinical systems. Comparison with the established outcomes and evaluation tasks motivate integration of the clinical and imaging data, and the use of standardized approaches to support annotation and sharing of the analysis results and semantics. We developed the methodology and tools to support these tasks in Positron Emission Tomography and Computed Tomography (PET/CT) quantitative imaging (QI) biomarker development applied to head and neck cancer (HNC) treatment response assessment, using the Digital Imaging and Communications in Medicine (DICOM®) international standard and free open-source software. Methods. Quantitative analysis of PET/CT imaging data collected on patients undergoing treatment for HNC was conducted. Processing steps included Standardized Uptake Value (SUV) normalization of the images, segmentation of the tumor using manual and semi-automatic approaches, automatic segmentation of the reference regions, and extraction of the volumetric segmentation-based measurements. Suitable components of the DICOM standard were identified to model the various types of data produced by the analysis. A developer toolkit of conversion routines and an Application Programming Interface (API) were contributed and applied to create a standards-based representation of the data. Results. DICOM Real World Value Mapping, Segmentation and Structured Reporting objects were utilized for standards-compliant representation of the PET/CT QI analysis results and relevant clinical data. A number of correction proposals to the standard were developed. The open-source DICOM toolkit (DCMTK) was improved to simplify the task of DICOM encoding by introducing new API abstractions. Conversion and visualization tools utilizing this toolkit were developed. The encoded objects were validated for consistency and interoperability. The resulting dataset was deposited in the QIN-HEADNECK collection of The Cancer Imaging Archive (TCIA). Supporting tools for data analysis and DICOM conversion were made available as free open-source software. Discussion. We presented a detailed investigation of the development and application of the DICOM model, as well as the supporting open-source tools and toolkits, to accommodate representation of the research data in QI biomarker development. We demonstrated that the DICOM standard can be used to represent the types of data relevant in HNC QI biomarker development, and encode their complex relationships. The resulting annotated objects are amenable to data mining applications, and are interoperable with a variety of systems that support the DICOM standard.


Introduction
Imaging has enormous untapped potential to improve clinical cancer treatment decision making.To harness this potential, research exploring the utility of image analysis to extract and process morphometric and functional biomarkers is essential.In the era of noncytotoxic treatment agents, multimodality imageguided therapies and rapidly evolving computational resources, quantitative imaging software performing such analyses can be transformative for precision medicine in enabling minimally invasive, objective and reproducible evaluation of imagebased cancer treatment targeting and response.Postprocessing algorithms are integral to highthroughput analysis and finegrained differentiation of multiple molecular targets.Software tools used for such analyses must be robust and validated across a range of datasets collected for multiple subjects, acquisition devices, timepoints and institutions.Ensuring the validity of this software requires unambiguous specification of analysis protocols, documentation of the analysis results, and clear guidelines for their interpretation.Yet cancer research imaging data often does not exist in consistent formats that facilitate advancement of quantitative analysis.There is lack of an infrastructure to support common data exchange and method sharing.These issues hinder development, validation and comparison of new approaches and comparison of results across sites.
Recent initiatives such as the Quantitative Imaging Network (QIN) and Informatics Technology for Cancer Research (ITCR) of the National Cancer Institute (NCI), and Quantitative Imaging Biomarker Alliance (QIBA) of the Radiological Society of North America (RSNA) focus on a spectrum of issues related to quantitative imaging (QI) biomarker development, including both the validation and deployment of promising QI tools, and the development of the supporting infrastructure.Quantitative Image Informatics for Cancer Research (QIICR) is one of the projects of the ITCR consortium ( http://qiicr.org, U01 CA190819).The overarching mission of QIICR is to provide Free and Open Source Software (FOSS) QI analysis tools accompanied by the imaging data and analysis results stored in a standardscompliant structured fashion to support imaging biomarker development, and facilitate both the reuse of the shared research data and the acceleration of the translation of those usecases into clinical practice.QIICR is a collaboration with three sites of the NCI QIN (namely, Brigham and Women's Hospital, University of Iowa, and Massachusetts General Hospital), each of which is focused on different diseases, and uses different imaging technologies and analysis methods.The research projects of interest at these three sites serve as use cases and testbeds for driving the requirements, testing and dissemination of the imaging informatics technology being developed by QIICR.In this paper, we focus on one of the QIICR use cases -PET/CT QI biomarker development for treatment response in head and neck cancer (HNC) -to demonstrate how the use of the Digital Imaging and Communications in Medicine (DICOM ® ) international standard (National Electrical Manufacturers Association 1 (NEMA)), in conjunction with FOSS tools, can enable interoperable sharing of the quantitative imaging analysis results.The contributions of this work are twofold.First, we propose a DICOMbased approach to data sharing in QI research, and present the resulting dataset of clinical information and analysis results generated by a clinical research study investigating QI biomarkers in Positron Emission Tomography and Computed Tomography (PET/CT) imaging for predicting therapy outcome in the patients undergoing treatment for HNC.Second, we develop a suite of FOSS tools to facilitate encoding of the analysis results using the DICOM standard.
The research study that generated the data described in this work investigated the use of quantitation of the [18F]fluorodeoxyglucose (FDG) tracer uptake in PET/CT images (CT is combined with PET for attenuation compensation as well as spatial localization).FDG PET/CT imaging is a modality commonly used for localization, characterization and qualitative assessment of therapy response in a variety of malignancies (Larson et al., 1999; Weber, 2006).Quantitative assessment of tumor burden using FDG PET/CT relies on a number of analysis steps, and can be sensitive to the processing technique and definition of the volumetric Region of Interest (ROI) (Boellaard, 2009; Vanderhoek, Perlman & Jeraj, 2012).It was a goal of the study that generated the data to investigate the process of PET quantitation to propose improved ROI segmentation tools and a reproducible PET/CT analysis workflow.Steps involved in the analysis of the PET/CT images included normalization of the PET image data using Standardized Uptake Value (SUV) body weight factor, segmentation of the tumor and involved lymph nodes using both manual and automated segmentation tools (performed by three users, twice for each patient), and quantitation of the various statistics related to the tracer uptake from the segmented ROIs.The various processing steps and their interactions are shown in Fig. 1, and are detailed in the Methods section.
Most of the methods used for QI analysis that led to the data presented in this paper are accompanied by FOSS tools developed as part of the QIICR project.However, the main objective of this paper is not to discuss these analysis methods in detail, or to validate the tools implementing those analysis methods.Instead, we focus on the use of the DICOM standard to enable interoperable communication of the analysis results produced by those tools.Our goal is to facilitate access to the data and analysis results so other research groups can perform similar validation and/or compare the results to different methods or apply new tools to the imaging data.
Interoperability can be defined as "a property of a product or system, whose interfaces are completely understood, to work with other products or systems, present or future, without any restricted access or implementation" (Interoperability Working Group (AFUL), 2015).Interoperability implies the use of a common standard -ideally, an "open standard" -to engage the broad community of various stakeholders in the industry and academia.We chose DICOM as that standard due to its broad and inclusive community of contributors, its ubiquitous adoption in the medical imaging community and the suitability of its data model to accommodate the requirements of the use case.For rapidly evolving research applications like imaging biomarker development, it is also important to note that DICOM is a standard that is being continuously refined to address new community demands and technologies, while maintaining backwards compatibility with the existing user base.This process is enabled via the mechanism of Correction Proposals (CPs) and Supplements that can be submitted for consideration and review by the DICOM community, and are integrated into the standard through the formal process of discussion, refinement and voting.
DICOM is primarily used to support interoperability between clinical systems and image interchange (Haak et al., 2015).Consumption of the DICOM images produced by preclinical and clinical acquisition systems is widely supported in research tools, enabling the availability of an evergrowing stream of imaging data to researchers.Sharing of results between different groups is widely regarded as a priority (Stodden, 2010; Walport & Brest, 2011; Boulton et al., 2011; Piwowar & Vision, 2013), with the failure to adopt standards for encoding results flagged as a critical barrier (Chan et al., 2014).We argue (and demonstrate by example in this paper) that DICOM provides the means to support interchange of not only acquired images but also clinical data and various types of analysis results, with a goal to enable their sharing and reuse.We recognize that output of the analysis results using DICOM in current research tools is severely limited or nonexistent.Instead, research tools often default to using local or domainspecific formats (Kindlmann, 2004; Ibanez & Schroeder, 2005; Schroeder & Martin, 2005; NIfTI Data Format Working Group, 2005; MRC Cognition and Brain Sciences Unit, 2013) that cover narrow use cases in restricted domains, ultimately compromising consistency, dissemination and reuse of the analysis results by fellow researchers.One of our objectives is to remedy this situation and provide the missing support of DICOM for QI research applications in tools and toolkits.
Interoperable communication of analysis results between research and clinical systems is another critical consideration for validation and translation of QI precision medicine tools.The development and evaluation of research applications, data, and software historically proceeds independently from clinical care and in distinct systems.Yet the extent to which data and software are interoperable between research and clinical environments directly impacts the ability to use clinical data for research, to use research applications in experimental clinical care, and to then translate research developments into clinical practice.Many barriers to such "translational" scenarios have been identified, among them being failure to use standard models and encoding formats in research and clinical environments (Katzan & Rudick, 2012; Chan et al., 2014) and failure to use standard codes (McDonald, Vreeman & Abhyankar, 2013).
The arguments presented above for the benefits of open standards such as DICOM are widely accepted, however adoption of such standards is not without effort.The DICOM standard is widely and fairly regarded in the research community as being nontrivial in complexity, while its documentation is immense and difficult to navigate.Support of DICOM in toolkits is widespread, but mostly limited to the lowerlevel abstractions and more commonly oriented towards consuming rather than producing DICOM objects .It is not uncommon for 2 reference implementations and sample datasets illustrating the application of the certain parts of the standard to be nonexistent.As with any complex endeavour, the DICOM standard itself is not without errors and may contain internal contradictions.In addition, the standard does not have (and does not claim to have) all of the features that are needed to support new or uncommon research use cases.These are some of the real obstacles for adoption of DICOM for communicating analysis results, both between the manufacturers of the commercial imaging workstations and within the QI research community.
In this contribution we take a number of steps to rectify this situation.We demonstrate the application of the DICOM standard to model and share a real example of a complex research dataset.We accompany this demonstration with the resulting dataset, conversion tools, developer toolkit and Application Program Interface (API), and integrated userlevel analysis and visualization tools, all available as FOSS.We provide detailed explanation of, and motivation for using specific parts of the standard.Finally, we demonstrate how the standard itself can be improved via the community review process, to address errors and limitations, which can best be identified and solved by applying the standard to a real use case.

Patient cohort selection
The primary data was extracted from HNC patients with squamous cell carcinoma, all treated according to the standard of care at the University of Iowa Hospital and Clinics.Clinical practice was to obtain a FDG PET/CT for staging (prior to treatment) and then a second FDG PET/CT scan for response assessment at approximately three months following the completion of the initial therapy.The imaging studies were acquired between 2004 and 2013.Patients that had a baseline and at least one posttherapy followup PET/CT were included in the research study.Patients were followed clinically and outcomes were available with a minimum of 2 years of followup.Patients may have had additional imaging studies following the three month response assessment FDG PET/CT based on clinical judgement and findings.
Clinical metadata for each patient was manually extracted from the electronic health records and included sex, age, smoking, and drinking history as well as pathology, stage, primary site location, and detailed location of involved nodal sites.Treatment details (e.g., radiation dose, technique, surgical intervention and chemotherapy delivery) and disease status and recurrences were recorded.All clinical metadata was deidentified and stored in a Postgres relational database locally.Measurements made on images that were used for clinical purposes and stored in the clinical records were not used during the conversion process, since new measurements were to be made, and homogeneity and accuracy of the clinical measurements could not be easily verified.
A total of 156 patients were identified as eligible for the study, with at least one PET/CT scan and related clinical data available for study (avg.3.05 studies/patient collected during a total of 472 visits).For a subset of 59 patients with the primary cancer affecting the pharynx region, including tonsil, oropharynx, base of tongue, pyriform sinus, hypopharynx, and nasopharynx, the pretreatment PET/CT was segmented.For one patient the first posttreatment PET/CT was segmented as well.Segmentation was performed for the primary cancer site and involved lymph nodes, and three normal Reference Regions (RRs) located in the liver, aortic arch and cerebellum of the patient, respectively.

Image acquisition
Patients fasted for a minimum of 6 hours prior to imaging.Injected dose of [F18]FDG was weightbased with a maximum injected dose of 555 MBq (15 mCi).Uptake time was 90 minutes ± 10 minutes between injection and start of imaging.Pertinent details related to image acquisition such as reconstruction procedure, image resolution and specific injected dose are encoded in the DICOM metadata.After initial deidentification, the image data was stored at an eXtensible Neuroimaging Archive Toolkit (XNAT) (Marcus et al., 2007) research PACS instance installed internally at the University of Iowa and not available for public access.

Image processing
First, all lesions, including the primary tumor and the involved lymph nodes (based on the presence of increased tracer uptake) and the RRs (based on the anatomical location and the expected appearance of those regions in the imaging), were segmented.Second, quantitative SUV measurements were extracted from those regions.The results of these processing steps were encoded in DICOM as described in the subsequent sections.
FDG is taken up by cells that exhibit high glucose uptake, which includes cancer cells due to the high rate of the relatively inefficient process of anaerobic glycolysis, even in the presence of oxygen (Warburg, Wind & Negelein, 1927).The accumulation of FDG may be seen and measured by PET imaging.FDG PET studies are commonly utilized in oncology to determine the presence, location, and aggressiveness of patient's cancer (Weber, 2006).FDG PET imaging is also a promising imaging modality for assessment of response to treatment and outcome prediction (Larson et al., 1999).A quantitative characterization of lesions and their response to treatment requires the assessment of FDG uptake in volumetric Regions of Interest (ROIs).Since the arterial input function for detailed pharmacokinetic modeling of the dynamic timecourse of the metabolic activity is not available for most routine clinical investigations, the SUV is utilized for a simple semiquantitative analysis (Lucignani, Paganelli & Bombardieri, 2004).SUV Body Weight (SUVbw) is commonly defined as the ratio of activity in tissue divided by the decaycorrected activity injected to the patient, normalized by body weight: SUVbw=(tissue activity)/(injected activity/weight).Several alternatives to SUWbw approach have been investigated including body surface area corrected (SUVbsa) and lean body mass corrected (SUVlbm) (Graham, Peterson & Hayward, 2000), but SUVbw remains the most commonly utilized quantity.
There are several underlying assumptions in using FDG SUVs for measuring metabolic activity in lesions, such as accurate measurement of injected dose and accurate decay correction of all measurements (Graham, Peterson & Hayward, 2000).The failure of one or more of these assumptions can introduce variability in calculated SUVs.To mitigate this problem, an SUV ratio (SUVr) can be used, which represents the ratio of the SUV from a lesion to the SUV of a RR defined in the same image.Normal tissue or organs such as the liver can be used as RRs.
The primary cancer site and all involved lymph nodes were segmented separately to allow quantification of SUV for either the primary cancer site alone, total tumor burden, or on a perregion basis.Segmentation of the primary tumor and lymph nodes was done using two interactive segmentation tools within 3D Slicer (Fedorov et al., 2012).The first tool is a manual contouring tool, requiring the user to draw the boundary of a lesion on every slice using the Editor module of 3D Slicer.The second tool is highly automated, performing segmentation in 3D using a specialized algorithm for segmenting HNC in FDG PET images.This semiautomated segmentation method treats the segmentation task as a graphbased optimization problem based on the ideas introduced by Yin et al. (Yin et al., 2010).Starting with a userprovided approximate lesion centerpoint, a graph structure is constructed in a local neighborhood, and a suitable cost function is derived based on local image statistics.For optimization, a maximum flow algorithm is utilized, and the resulting segmentation is converted back from a graphbased representation to a labeled volume.To handle frequently occurring situations that are ambiguous, several segmentation modes are introduced to adapt the behavior of the base algorithm accordingly.In addition, "just enough interaction" based approaches are provided to enable the user to efficiently perform local and/or global refinement of initial segmentations.This semiautomated segmentation method is implemented in the PET Tumor Segmentation extension of 3D Slicer (QIICR, 2015a).
Since both manual and semiautomatic methods for HNC segmentation depend on expert user input, results are expected to be subject to intra and interoperator variation.To allow assessment of the impact of such variation on subsequent processing steps, each data set was reviewed and segmented using both methods by three readers, who were experts in HNC PET/CT image interpretation.Images were presented to the readers in random order.For each combination of the segmentation tool and reader, this process was performed twice, resulting in twelve segmentation sessions per patient.
RRs in liver, cerebellum, and aortic arch were segmented automatically using the approach we presented earlier (Bauer et al., 2012).The cerebellum RR was segmented by fitting its active shape model (Cootes et al., 1995) to the general area of the brain in the patient data.The liver RR was segmented by first identifying the general liver area by SUV thresholding and fitting a hyperellipsoid inside the identified area.To segment the aortic arch, tubular structures were first identified in the CT images using tube detection filter, followed by the localization of the tubular structure corresponding to the aortic arch based on the anatomical information.
Given the segmentations of the primary tumor and lymph nodes, a total of 22 quantitative indices were extracted from each of these regions using the PETIndiC extension of 3D Slicer (QIICR, 2015b).The calculated quantitative indices consist of commonly utilized PETspecific indices such as maximum, mean and peak SUV (Wahl et al., 2009) and Total Lesion Glycolysis (TLG), as well as common summary statistics measures that included median, variance and Root Mean Square (RMS) of SUV, and segmentation volume.
To provide flexibility for imagebased FDG PET biomarker research, we also generated SUVbw summary statistics for all three RRs segmented by our algorithms: mean, max, min, standard deviation, median, and first and third quartiles.Thus, both SUVbw and SUVr can be calculated from the resulting data.

Data modeling and conversion into DICOM representation
Recent publications demonstrate that there is an increased recognition of the value of at least exporting images that are the result of research processing applications in DICOM format, so that they can be used to support various activities essential for imaging biomarker development.Such activities include consistently "tagging" analysis results to compare analyses done at different centers on different cohorts using different analysis tools (Waterton & Pylkkanen, 2012), supporting archival and distribution of the analysis results in a manner that enables indexing and secondary analysis (Chan et al., 2014) and transfer to and visualization of the analysis results in clinical systems in which metadata for patient identification and study management is required (Clark et al., 2013; Moore et al., 2015).(i.e., common composite context, e.g., at the Patient and Study level).
The DICOM standard provides a variety of objects that can be used to communicate information derived from the images.Independent of the specific object type, DICOM requires that all of the objects maintain the socalled "composite context".At the patient level, the composite context includes such attributes as patient name, ID and sex.The study context includes the date and time that the imaging study started, unique identification of the study and other information common to all series in the study.The composite context enables consistent indexing and crossreferencing of the various objects.In addition to common composite context, derived DICOM objects typically contain explicit references to the "source" objects from which they were derived, which supports both recording of the provenance of the object derivation and application functionality such as superimposition during rendering.Various relationships among the objects utilized in this study are illustrated in Fig. 2.
The purpose of QI research processing applications is not to produce more images, and in particular not just "pretty pictures" (screenshots), but rather to create quantitative or categorical results that can be used in the context of clinical research studies or clinical trials evaluating the prospective imaging biomarker in question (Krishnaraj et al., 2014).The type of data produced by such quantitative analysis routines dictates the choice of DICOM object that is suitable for encoding it, necessitating the use of various types of objects to represent the use case in its entirety.The object types that are needed to support quantitative imaging research typically fall into two major categories, those for: 1. quantitative or categorical information that is imagelike, i.e., encoded on a pervoxel basis, for which DICOM Segmentations (SEG) and Parametric Maps are appropriate, and those for 2. quantitative or categorical information that is Region of Interest (ROI) specific, and is not imagelike, for which DICOM Structured Reports (SR) are appropriate.
Other intermediate output may also be encoded and reused, and examples will be discussed later (e.g., DICOM Real World Value Maps (RWVMs)).Given that DICOM SR is used in this project for communicating measurements performed on the segmented regions, clinical data SR templates have also been defined for recording and sharing clinical data about patients in a research cohort and used in this project, as discussed below.
DICOM relies on coded terminology (Bidgood, 1997), both from standard external lexicons (such as the Systematized Nomenclature of Medicine ( SNOMED ® ) (Bidgood, 1998) as well as from the DICOM lexicon types of tumor, etc. (in the same manner as the "semantic web" ("Semantic Web W3C," 2015)).The extent to which coded terminology is used by the individual objects was an important consideration for the choice of the encoding approach.
In this section we discuss the motivation for the choices made for the use of specific DICOM objects.Our description of the various objects used starts with the PET and CT objects, since they were produced by the imaging equipment and underwent only minor editing for deidentification.Then we describe the objects containing patient clinical data (clinical history and outcomes), since they follow rather simple SR templates and are not referencing any other derived objects.Following that, we cover the derived imaging objects following a progression from the most simple to more difficult objects: • DICOM RealWorld Value Mapping (RWVM) objects encode mapping of the imagespecific SUV factor that is needed for normalization of the images and subsequent processing.• DICOM Segmentation (SEG) objects encode labeling of the PET and CT image voxels into anatomical structures relevant in this project, such as primary tumor and liver ROI.• DICOM Structured Reporting (SR) objects encode various measurements performed on the segmentationsdefined regions from the normalized PET image volumes.
While describing each of the derived objects, we follow the general pattern of first discussing the scope and capabilities of the object at the high level, followed by the design decisions we adopted to meet the requirements of our use case.Corrections to the standard that resulted directly from the process of data conversion are also listed.A separate section covers the implementation details of converting research representations into the DICOM format, and referencing the tools we developed for this purpose.

PET/CT Image Data: DICOM PT/CT
PET and CT image data is stored using DICOM format following Positron Emission Tomography Image object type (National Electrical Manufacturers Association (NEMA), 2015b) and Computed Tomography Image object type (National Electrical Manufacturers Association (NEMA), 2015c), respectively.The image data obtained from the modality was deidentified using a modified version of the Basic Attribute Confidentiality Profile defined by the DICOM in PS3.15 Appendix E.2 (National Electrical Manufacturers Association (NEMA), 2015d).Image deidentification was performed following the standard operational procedures established by the Cancer Imaging Archive (TCIA) (Clark et al., 2013; Moore et al., 2015).Research identifiers of the form QINHEADNECK01nnnn were assigned in place of the patient names and medical record numbers.Dates were shifted by the same fixed offset across all the datasets to maintain temporal relationships of the datasets.The deidentified images were then used for the remainder of the project (i.e., to make the measurements and convert them into derived DICOM objects) in order to mitigate the risk of leakage of patient identifiers into the publicly accessible analysis results.
Because a combined PET/CT scanner was used, no spatial registration between the PET and CT images from the modality was performed.The assumption was made that the patients did not move during acquisition.The DICOM images acquired for the PET and CT series already shared the same Frame of Reference Unique Identifier (UID) establishing the spatial correspondence between PET and CT imaging datasets.No DICOM Registration (REG) objects were created or used in this project.

Clinical Information: DICOM SR
Relevant clinical information available for the subjects enrolled in the study included clinical history (such as the diagnosis and pathology, surgery and radiotherapy administration, and demographics) and outcomes (followup date and status, and the date of death, when applicable).This information is important for the interpretation and secondary reuse of the image and quantitative data set, since it provides nonimaging predictors that can be used for machine learning, and ultimately the clinically relevant endpoints for the evaluation of the biomarker performance.The clinical information was extracted from the operational Postgres research database used for the project, and retrospectively encoded as DICOM SR objects, one per patient.
To the best of our knowledge, there is no consensus in the medical imaging research community on a standard mechanism for sharing the clinical information about the patient cohorts used in quantitative imaging research studies.Most widely used are spreadsheet formats, such as CSV (CommaSeparated Values) or Excel (Microsoft, Redmond, WA).As an example, the Cancer Genome AtlasLung Squamous Cell Carcinoma (TCGALUSC) data collection is accompanied by an archive of seven spacedelimited text files storing information about patient demographics, radiation therapy and drug administration (The Cancer Imaging Archive (TCIA), 2015a).Although this data is linked to the imaging data of the collection via common patient identifiers, it cannot be queried in a manner linked to querying of the imaging data, there are no tools to validate those text files, and the lack of a common standard or template used to encode them can lead to difficulties integrating clinical data from different sources.
On the clinical side, various standards exist for communicating clinical data.These include Clinical Document Architecture (CDA ® ) (Dolin et al., 2006), Health Level Seven (HL7) version 2 messages (Mwogi, Biondich & Grannis, 2014), HL7 version 3 messages (Goossen & Langford, 2014) and Fast Healthcare Interoperability Resources (FHIR) (Bender, Duane & Kamran, 2013; Health Level Seven® International, 2015).A nascent effort is the concept of "Electronic Health Records (EHR) based phenotyping" (Rasmussen et al., 2015), but EHR integration is beyond the scope of this project, if for no other reason an EHR was not the direct integrated source of the clinical information, as opposed to a dedicated research database into which the information had been transcribed.
Given the lack of a consensus about standard mechanism for sharing clinical information, and the expectation that the primary reuse of the image and quantitative result data sets would be in an imageaware context (i.e., within image display and analysis software), it was deemed expedient to develop a template in which to encode the clinical information as DICOM SR.This permits the use of standard codes, hierarchical organization, and reuse of parts of existing templates in the DICOM standard.
The choice of DICOM SR to encode clinical information was motivated by the following factors: 1. Harmonization.By using DICOM consistently, we can maintain composite patient context across the objects, enabling consistent linking of the clinical, imaging and quantitative derived information.
Recognizing that the source PET/CT data will always be in DICOM, this choice allows a consistent data format across all information types and allows us to focus on only one set of standardized conventions for data encoding.2. Interoperability .The resulting DICOM objects can be stored sidebyside with the images and annotations within readily available DICOM server infrastructure, enabling storage, search, and retrieval using such existing infrastructure deployed for managing medical image data.3. Accessibility.A variety of robust commercial and freely available tools and toolkits exist to interact with DICOM data, which can be applied in uniform manner to both clinical and imaging information.This makes it relatively straightforward to convert DICOM representation into other commonly used forms, such as PDF, HTML, XML, JSON, or delimited text.As an example, conversion of DICOM to XML and HTML is supported by DCMTK, and conversion of XML representation to delimited text can be performed using XSLT.There are even standard mechanisms defined for transcoding some components and/or templates of DICOM SR into HL7 CDA (National Electrical Manufacturers Association (NEMA), 2015e).4. Validation.Existing tools can be used to verify the validity of the resulting objects at different levels.
Specifically, dciodvfy tool (Clunie, 2015a) can be used to check encoding of the DICOM data elements, and com.pixelmed.validate.DicomSRValidator (Clunie, 2015b) can confirm compliance of the resulting object with a specific template, once the template has been added to the tool.
The choice of DICOM SR for this project was motivated not only by the best match of the capabilities of the mechanism to the requirements of the project, but also by the widespread clinical use of SR for similar applications.In those clinical applications for which imagederived results have achieved acceptance, such as echocardiography and obstetric ultrasound, and mammography Computer Aided Detection (CAD) (Brent J. Liu, Anh Le, H.K. Huang, 2015), both manual and automated processes routinely use the DICOM SR mechanism (Dean Bidgood, 1998; Clunie, 2000; Loef & Truyen, 2005) (Talati et al., 2013).Though a less common application for it, DICOM SR can and has been used for encoding humangenerated reports (i.e., those dictated by a radiologist) (Noumeir, 2006), and as a consequence is supported in common offtheshelf image viewing tools, such as OsiriX (Rosset, Spadola & Ratib, 2004).DICOM SR has also been used for more exotic applications such as organizing and sharing the results of functional MRI (Rosset, Spadola & Ratib, 2004; Soares & Alves, 2009), exchange of implantation plans (Treichel et al., 2010), interchange of ECG reports (von Wangenheim et al., 2013) as well as for clinical trial applications in neurology (Jacobs et al., 2010) and oncology (Clunie, 2007).
DICOM SR objects (sometimes referred to as SR "documents") contain information organized as a content "tree" consisting of a hierarchical structure of content items (tree nodes) (Clunie, 2007).These content items include, for example, containers, textual information, codes describing concept names (we will use "term" and "concept" interchangeably in this document) and values (where appropriate), references to images, and numeric values (National Electrical Manufacturers Association (NEMA), 2015f).The standard also defines the types of relationships between the parent and child content items, such as CONTAINS and HAS PROPERTIES.DICOM SR provides standard templates to constrain this intentionally general infrastructure for specific use cases (National Electrical Manufacturers Association (NEMA), 2015g).
Each content item, except for those that are containers, can be thought of as a "namevalue pair" (or alternatively, as a "question" and an "answer").Containers can be considered "section headings", and are often explicitly used as such when rendered in humanreadable form.The top level (root node) of the content tree is always a container, and its name (concept) is often referred to as the "document title".The concept name of a container or namevalue pair (mandatory in most cases) is always coded using a code from a controlled terminology.The value may or may not be coded depending on the value type.
The use of controlled terminology is fundamental to DICOM SR.DICOM SR codes are defined as triplets of code value, coding scheme designator and code meaning (e.g., (F02573, SRT, "Alcohol consumption") triplet, where "SRT" is the DICOM designation for the SNOMED coding scheme).While DICOM allows for reuse of the codes defined in other terminologies, such as SNOMED, as well as those defined in the DICOM standard itself, so called "private" codes can also be defined by the creator of the object, when no standard codes are available.Such private codes are distinguished by a coding scheme designator that start with the "99" prefix.The use of predefined codes provides not only semantic information about the reported entities, but also simplifies validation of the resulting objects.The codes that are allowed are constrained by the specific template.The constraints for values may be defined in the template itself, or in a "value set", called in DICOM a Context Group (and labeled with a Context Group ID (CID)).
Each template is assigned a Template Identifier (TID).Templates may define the entire content of an object (i.e., be a "root" template) or may be a reusable common pattern of nested content to be included by higher level templates (i.e., be a "subordinate" template).Though the DICOM standard contains templates for clinical data related to a few specific applications (e.g., cardiovascular (National Electrical Manufacturers Association (NEMA), 2015h), breast (National Electrical Manufacturers Association (NEMA), 2015i)), it does not define a template to represent all the items of interest in our HNC QI research use case.Given the lack of a suitable standard template to represent this data, we developed our own set of custom templates for communicating the clinical information.In DICOM, such custom templates are referred to as "private templates", even though they may be publicly shared and are required to be documented in the DICOM conformance statement of the product.The resulting SR objects encoded according to these templates include information about biopsy, treatment and other relevant data.The relationships among the various private templates are shown in Fig. 3, and detailed documentation of them is provided in Appendix 1 .These templates attempt to follow the patterns of the existing standard templates with the intent that they might form the basis for future enhancements of the standard.The private template documentation describes both the root template and private subordinate templates, as well as the private and standard context groups used (e.g., the value set of the racial group in the root TID QIICR_2000 is constrained to codes in CID QIICR_2001).
No structured terminology was used for defining the data elements at the time of initial clinical data collection, so terms with codes were selected retrospectively at the time of conversion of the data to the DICOM SR format.Our approach for selecting codes leveraged SNOMED (Cornet & de Keizer, 2008) and UMLS (Bodenreider, 2004) terminology as much as possible.It was not a priority of this project to survey all possible terminologies.As an example, with few exceptions, we did not use NCI Enterprise Vocabulary Services (EVS) Common Data Elements (CDE) (Warzel et al., 2003), nor was CDISC SDTM (Kuchinke et al., 2009) searched.Our choice of SNOMED was based on its extensive use by DICOM and the existence of a formal agreement between DICOM and IHTSDO (the owners of the SNOMED intellectual property) to preferentially use SNOMED in return for the right to use the codes in DICOM without a specific license or license fee.The few concepts that could not be located in the existing terminologies were added to the PixelMed private coding scheme designated by the code "99PMP", with the intention that if the template is proposed to DICOM and accepted, standard codes will be defined to replace the private codes (either through addition to SNOMED, to some other suitable standard lexicon, or to the DICOM terminology itself).
Translation of unconstrained freetext concepts into a term from a controlled terminology is fraught with difficulty and may depend on context that is not explicitly communicated, as well as creative interpretation in the absence of formal definitions.In certain instances, codes were reused that were deemed to be sufficiently close to the freetext counterpart instead of introducing a new code.As an example, to describe alcohol consumption, the SNOMED code (F02573, SRT, "Alcohol consumption") was used as the concept name.The values assigned were constrained to the following SNOMED codes as a replacement for the noncoded terms originally used in the research database: • (R40775, SRT, "None"); • (F60019, SRT, "Alcohol intake within recommended sensible limits") in place of "social drinker"; • (F60018, SRT, "Alcohol intake above recommended sensible limits") in place of "significant drinker".
To ensure that the final selection of codes was consistent with the interpretation of the original terms used by the study designers, the mapping to codes from the unconstrained freetext clinical descriptors was reviewed by the clinical experts of our team (JB and RB) and the suitability of the chosen terms was confirmed.
Throughout the DICOM standard, the "old" SNOMEDRT form of identifier (SNOMED ID) is used rather than the "new" SNOMEDCT (Concept ID) numeric form, even though the current version of SNOMED used in DICOM and our project is SNOMEDCT.This is a consequence of the early adoption by DICOM of SNOMED, prior to the merger with the UK Read Codes (Harding & StuartButtle, 1998).Accordingly, the SNOMEDRT form used in the current DICOM standard and in the installed base of systems, software and archived DICOM data, is also used for our project.Translation into the numeric Concept ID form would likely be required for any transformation into HL7 CDA.

Standardized Uptake Value: DICOM RWVM
Normalization of the activityconcentration PET image voxel data values to compensate for differences in volume of distribution and injected dose of radiopharmaceutical was performed by calculating SUV in the conventional manner.Although the calculation of SUV factor is a straightforward algebraic operation, it can be error prone, due to differences in how PET imaging equipment vendors encode radioactivity and information needed for decay correction in the DICOM image attributes, the dependence on private attribute values in some cases (e.g., for residual dose in the syringe), and the variation in operator behavior in terms of entering the information at the modality console (Quantitative Imaging Biomarker Alliance (QIBA), 2012).Care must also be taken to reduce ambiguity when encoding the resulting factor, since there are different approaches to normalization (i.e., using actual body weight as opposed to lean body mass, the latter accounting for the differences in tracer uptake in the adipose tissue).In this project, SUV Body Weight (SUVbw) normalization was used, which is calculated at each voxel as the ratio of the radioactivity concentration and the injected dose of radioactivity per kilogram of the patient's body weight (Sugawara et al., 1999).
The DICOM Real World Value Mapping (RWVM) object provides a mechanism to describe the calculation that was used (and can be reused) to create "real world values" (such as SUV) from stored pixel data values.The DICOM RWVM can be embedded within other DICOM objects (such as acquired or derived images), or it can be encoded as a standalone object (National Electrical Manufacturers Association (NEMA), 2015j), which in turn can either be referenced from other objects, such as SRs, or recognized as relevant from the commonality of patient and study identifiers.
Several DICOMcompliant approaches to communicating SUVbw are possible: 1.The addition of the RVWM macro to the existing PET images.The deficiency of this approach is that it requires modification of the image data, since SUVbw was not calculated by the imaging equipment, and the imaging data was archived prior to the production of derived data.In operational systems, though DICOM permits some modification of existing objects when interpretation is not affected, this may also be beyond the scope of what is allowed and to be strictly compliant would require the creation of new derived images with different identifiers than the originals.This approach would double the amount of storage occupied by the distributed data set, if the "original" images were also shared.2. The creation of new derived PET images that contain stored pixel values that have actually been scaled and require no further transformation, or have appropriate modified values of Rescale Slope, using either the PET object or the Parametric Map object (National Electrical Manufacturers Association (NEMA), 2015k).The latter object allows for floating point values obviating the need for rescaling of integer values.Either approach would effectively double the storage, since PET pixel data would need to be duplicated.
3. The creation of a standalone RWVM object.This approach does not require modifying or making derived copies of the original PET images with replicated pixel data, and, based on our experience, is consistent with the practice adopted by the vendors of quantitative image analysis workstations, such as Siemens syngo.via.
We chose to create a standalone RWVM object to encode SUVbw factor and leave the original (deidentified) activityconcentration images unchanged.The RWVM object encodes the actual scale factor, the range of stored pixel values to which it applies, and standard codes specifying the quantity that the scaled (real world) value represents (in this case, the SUV), the measurement method (the SUV body weight calculation method) and the measurement units ({SUVbw}g/ml).The DCM coding scheme is used for the quantity and the measurement method, and, as is the case throughout DICOM, the Unified Code for Units of Measure (UCUM) system (Schadow et al., 1999) is used for the units.The RWVM object also includes references to all of the PET image objects to which it applies.
The following corrections to the standard were proposed to remedy the errors or limitations of the standard identified while developing DICOM representation of the SUVbw factors for this project: 1. CP 1387 : Addition of Quantity Descriptors to Real World Value Maps (standard in 2014b) While 4 developing support for RWVM encoding of SUV for this project, we realized that it is necessary to distinguish the physical quantity that a real world value represents, from the units that are used for it.As a case in point, different types of SUV may share the same units even though they are computed in a different way, and this affects their interpretation.The original definition of the RWVM in DICOM only defined the encoding of measurement units.Accordingly, we proposed an improvement to the standard to include the definition of quantity in the RWVM encoding.2. CP 1392: Addition of Quantity Descriptors and Measurements for PET (standard in 2014b) This CP added new concepts related to encoding of the PET measurements.

Image Segmentation: DICOM SEG
Segmentations produced by the project consist of binary labeling of the PET image voxels to define ROIs that were used subsequently in the derivation of quantitative measurements.The segmented structures include RRs in the liver, aortic arch and cerebellum, which were used for further normalization of the measurements in the tumor area, primary tumor (when found) and the "hot" lymph nodes showing tracer uptake.Anatomical locations of the primary tumors were recorded by the clinical experts at the time segmentation was performed.The imaging time point was defined as an ordinal number corresponding to the imaging study performed for the patient in the course of management of the specific condition, with time point 1 corresponding to the baseline/staging study.For each such time point, RRs were segmented automatically using the algorithms described earlier.Segmentation of the primary tumor and "hot" lymph nodes was done using two different approaches: by contouring the regions manually, and by using a semiautomated segmentation algorithm "seeded" by the operator.The segmentation procedure described earlier resulted in 12 sets of segmentations for each imaging time point, whenever primary tumor and involved lymph nodes were identified in the PET image collected at that time point.
DICOM provides different mechanisms for encoding the results of image segmentation.The choice of the most suitable mechanism depends on the use case.
1.The Segmentation (SEG) object supports encoding of segmentations that are labelled rasterized use of coded terminology (Bidgood, 1997), both from standard external lexicons (such as SNOMED (Bidgood, 1998) as well as from the DICOM lexicon (National Electrical Manufacturers Association (NEMA), 2015a) when no suitable external terms are available (Bidgood et al., 1997).2. The Surface Segmentation object represents the boundary of the segmented region as a surface mesh.3. The Radiotherapy (RT) Structure Set (RTSS) object represents the boundary as a set of isocontours of 3D coordinates, but does not have any standard extensible coded mechanism for associating derived information with each ROI. 4. The SR object can also be used to encode 2D or 3D isocontours.5.The Softcopy Presentation State family of objects are intended for communicating information about how a particular image should be rendered for display (e.g., window and zoom level), and although they also permit text and vector graphic annotations, and the latter can be used to render contours as 2D (imagerelative) coordinates, these are unsuitable for reuse for further analysis or measurements (i.e., they do not convey the semantics of an ROI).
Since the native representation of the tools used in this project for segmentation was labeling of the individual voxels rather than a surface mesh or isocontours, we selected the SEG object as the most appropriate one for encoding the ROIs.
We made the following design decisions about the organization of the resulting SEG objects, to be consistent with the pattern that would likely be used by tools that created such objects prospectively rather than retrospectively: • Each of the RRs is stored as a separate object, since each of the RRs was segmented using a distinct automatic method, using data from different modalities, i.e., the aortic arch was segmented on the CT images, and the cerebellum and liver ROI were segmented on the PET images.• The primary tumor and involved lymph nodes segmented for each combination of operator/segmentation method/session are stored together as different segments in a single object, since both the tumor and nodes were segmented during the same session, with the segmentation of one structure being identified while considering the neighboring structures.• The identifier of the operator (reader) for the manual and semiautomated segmentation results is stored in the ContentCreatorName attribute.

6
• The identifier of the imaging time point was encoded as an integer number starting from 1, stored in the ClinicalTrialTimePointID attribute.• The identifier of the segmentation session for primary tumor and lymph nodes was encoded in the ClinicalTrialSeriesID attribute.• The type of algorithm that was used is encoded in the SegmentAlgorithmType attribute as MANUAL, SEMIAUTOMATIC or AUTOMATIC as appropriate.• The suggested color for each of the segmented structures is encoded in the RecommendedDisplayCIELabValue attribute.
The semantics of the segments are communicated using the standard AnatomicRegion (and its modifier in AnatomicRegionModifier sequence, if necessary), SegmentedPropertyType and SegmentedPropertyCategory sequences.The DICOM standard allows the anatomy to be encoded in the SegmentedPropertyType when the segmentation is purely anatomical (e.g., for anatomical atlases), or to encode both the type of material segmented with its category, and encode the anatomy separately (e.g., for tumors of different types or composition, in different anatomical locations).This distinction was clarified by the authors in an earlier DICOM correction proposal CP 1258.In this project, we used the latter approach of encoding both the nature (category and type) of the segmented area and its anatomic location.
DICOM defines a relatively small set of segmentation property categories, listed in CID 7150 (National Electrical Manufacturers Association (NEMA), 2015l), and a considerably larger set of segmentation property types in CID 7151 (National Electrical Manufacturers Association (NEMA), 2015m).There is no direct relationship specified in the standard between category and type, and the choice of an appropriate category is left to the discretion of the implementer (arguably the standard could be improved by grouping the types and assigning them to, and requiring them for, specific categories).
As an example, the primary tumor can be encoded as follows: Segmented Property Category = (M-01000, SRT, "Morphologically Altered Structure") Segmented Property Type = (M-80003, SRT, "Neoplasm, Primary") Anatomic Region = (T-53131, SRT, "base of tongue") Lymph nodes are encoded similarly, but with only the general region (head and neck) recorded rather than a precise code for the lymph node group, because of the lack of the detailed information about the specific lymph node name in the original dataset due to practical difficulties in assigning such a precise name when segmentation was performed: Segmented Property Category = (M-01000, SRT, "Morphologically Altered Structure") Segmented Property Type = (M-80006, SRT, "Neoplasm, Secondary") Anatomic Region = (T-C4004, SRT, "lymph node of head and neck") Semantics of the RR segmentations are communicated using the "spatial relationship concept" category: Segmented Property Category = (R-42018, SRT, "Spatial and Relational Concept") Segmented Property Type = (C94970, NCIt, "Reference Region") Anatomic Region = (T-62000, SRT, "Liver") Binary segmentations are encoded in the PixelData attribute of the SEG object, and are represented as a contiguous array of bits with one bit per voxel for each frame.There are separate frames for each slice of the volume, though all are encoded in a single multiframe object.When multiple segments (i.e., primary tumor and lymph nodes) are produced by the operator during a single session using a single segmentation tool, they are stored in a single SEG object, with each segment for each slice in a separate frame.Empty frames that do not contain any voxels of the segmentation are elided in order to reduce the size of the encoded object size.The matrix size (rows and columns) is not abbreviated to a rectangular bounding box enclosing the region of interest, which would be a further possible size optimization (i.e., each frame has the dimensions of the original image).
The SEG objects also include a recommendation for the colors in which to render the segmentations when superimposed on displayed images, encoded in the RecommendedDisplayCIELabValue attribute.DICOM requires the use of CIELab color values rather than RGB values in order to achieve greater color consistency.
Like the RWVM objects, SEG objects include references to the SOP Instance UIDs of the images (slices) that were segmented.
The process of mapping the internal research format of the segmentation result into the DICOM representation led to the development of the following correction proposals: 1. CP 1406: Add codes for tumor sites (standard in 2014c) The uncoded (plain text) labels of all the tumor regions used in this project were analyzed to identify common terms, and these were then mapped to SNOMED concepts.The resulting terms were introduced into the DICOM standard in the form of new context groups for lymph nodes (CID 7600) and HNC anatomic sites (CID 7601).A distinction between concepts for primary and secondary neoplasms was introduced in the same proposal.

CP 1426: Correct condition in Pixel Measures, Plane Position and Orientation Functional Groups for
Segmentation (standard in 2015a) Prior to this correction, the presence of the essential attributes that are needed for volumetric reconstruction of the segmentation image volumes was conditioned on attributes that were optional or not defined in segmentation objects.
3. CP 1464: Add reference region segmentation property type (standard in 2015c) This correction added the codes needed to describe RRs, using the NCI Thesaurus terminology.

CP 1496: Add Tracking Identifier and UID to Segmentation Instances (letter ballot) Use of a common
Tracking UID allows the establishment of a correspondence between segments encoded in various segmentation objects that represent the same region being segmented (i.e., across different time points, modalities, operators).Tracking UIDs were already present in the SR measurements objects, which can reference segmentation objects, but were not encoded directly in the segmentation objects themselves.

Quantitative Measurements: DICOM SR
So far, we have described RWVM objects, which encode the conversion of PET voxels into quantitative values, and SEG objects, which categorize those voxels that are included in specific ROIs.It remains to record the measurements made on those ROIs as numeric values, appropriately identified, described, categorized and linked to the corresponding SEG, RWVM, and PET and CT image objects.These volumetric measurements are encoded in DICOM SR objects using standard templates.
PET volumetric measurements were extracted for each of the segmented ROIs using the corresponding SUVnormalized PET image.The specific measurements of interest were defined by the research protocol.As a prerequisite to the conversion of the measurements into DICOM format, we specified the terminology that defines the measurement quantities, modifiers and units for each measurement of interest needed.The vocabulary required is not specified in any single standard context group.
Concepts from various standard context groups were leveraged as appropriate.Our overall strategy to find a suitable term was to first consult those already included in DICOM, then search for the related concepts in UMLS, SNOMED, and the NCI Thesaurus.If no concept could be located in those sources, we introduced a new code, which, when available, was accompanied by a relevant PubMedindexed publication defining the term.If no standard concepts were found, new terms were defined using the private 99PMP coding scheme.All of the terms used are described in Appendix 2 .
The basic quantity measured is SUVbw, and an appropriate code is present in DICOM for the concept name that is used for the namevalue pair that encodes the numeric value.This SUVbw concept, (126401,DCM,"SUVbw"), is precoordinated to encode both the concept of SUV and the measurement method of body weight.It is then "modified" (by child content items that have a HAS CONCEPT MOD relationship with the NUM measurement content item), i.e., postcoordinated.For example, this approach can be used to show how the measurement was "derived", such as whether it is mean, minimum or maximum value, using standard SNOMED codes from an existing DICOM context group.The units (e.g., {SUVbw}g/ml,UCUM,"Standardized Uptake Value body weight") are coded in the same manner as in the RWVM objects, using the UCUM coding scheme.For example (using the shorthand description produced by the dcsrdump tool from the dicom3tools toolkit (Clunie, 2009), in which the ">" symbols indicate successive levels of nesting of subordinate content items in the tree): NUM: (126401,DCM,"SUVbw") = 4.1944 ({SUVbw}g/ml,UCUM,"Standardized Uptake Value body weight") > HAS CONCEPT MOD: CODE: (121401,DCM,"Derivation") = (R-00317,SRT,"Mean") In choosing the codes for communicating quantitative measurements, there is a tradeoff between a "postcoordinated" approach, defining a generic concept and then modifying it extensively, as opposed to a "precoordinated" approach, defining a single, specific code for every permutation and combination of base measurement and modifier, such as "maximum SUVbw".Since the goal of the project was to explore novel derived measurements from SUVbw, the decision was made to precoordinate the base measurement as SUVbw, and to postcoordinate the derivation modifier.This exposes the method of derivation clearly, since this is the subject of the investigation, and separates it from the underlying numeric measurement.
The permissible patterns of postcoordination are described in the SR templates, which define not only the encoding of individual measurements, but their additional descriptors (such as the anatomical location of the ROI), their relationship to one another, to the ROIs from which they were derived, and the overall structure of a "measurement document".To avoid excessive repetition, the measurement templates also allow commonality to be factored out and encoded at a higher level in the tree, rather than repeated with every measurement (an enhancement suggested by our work and implemented in CP 1386 discussed below).The measurement documents follow the standard root template TID 1500 defined in PS3.16 (National Electrical Manufacturers Association (NEMA), 2015n), which makes use of other subordinate templates, as shown in Fig. 4. The template contains a preamble that describes general characteristics relevant to the measurement, such as the Image Library container (National Electrical Manufacturers Association (NEMA), 2015o), which lists the UIDs of the images in the original image series, radiopharmaceutical agent, and other items related to the acquisition protocol that may be relevant during interpretation.The Imaging Measurements container (section heading) includes the following attributes, which we discuss in detail since they have special meaning in the context of our use case: • Activity Session : a positive integer number, which encodes the segmentation session by the operator.
• Tracking Identifier : a humanreadable identifier of the finding, which is not required to be unique.In our project, RRs have tracking identifiers coded as " referenceRegionName reference region", where referenceRegionName was one of "liver", "cerebellum" or "aortic arch".The primary tumor identifier is always set to "primary tumor", individual lymph nodes are identified as "lymph node nodeID ", where nodeID is an integer number starting from 1.As mentioned earlier, lymph nodes are not tracked (i.e., their nodeID cannot be expected to identify the specific lymph node across time points or reading sessions).• Tracking Unique Identifier : a DICOM standard UUIDderived (random) identifier that starts with a "2.25."prefix (National Electrical Manufacturers Association (NEMA), 2015p): a primary lesion unique identifier can be used to track the lesion and reference regions across time points.
• Time Point : a positive integer number, which encodes ordinal number of the imaging study within the course of management of the given patient.• Referenced Segment and Source series for image segmentation refer to the segment within the segmentation object containing the segmentation of the structure reported on in the measurement group, and the UID of the series being segmented.• Finding Site : the coded anatomical location of the finding.
Related groups of measurements are encoded as a list, preceded by the codes of one or more findings following the structure defined by TID 1411 Volumetric ROI Measurements (National Electrical Manufacturers Association (NEMA), 2015q), which in turn invokes TID 1419 ROI Measurements (National Electrical Manufacturers Association (NEMA), 2015r), as summarized in Fig. 4.Each group of measurements is derived from the ROIs that apply to the voxels of a single reconstruction of a PET acquisition (image series).One SR measurement object is created for each SEG object.Voxels in the ROI used for the derivation of the measurements are encoded as a segment of a SEG object.Both the SEG image objects and the segment number used by the derivation are referenced for each measurement group in the SR.
The groups of measurements associated with the ROIs are assigned both a humanreadable identifier and a globally unique identifier, which, for example allows tracking of the same tumor region identified in more than one time point.
The following DICOM standard corrections were contributed while developing the conversion methodology: 1. CP 1366: Correction of Relationships in Planar and Volumetric ROI Templates (standard in 2014b) In the process of data encoding, we identified errors in the definition of the relationships in some templates.
2. CP 1386: Addition of Measurement Report Root Template for Planar and Volumetric ROIs (standard in 2014b) Before the introduction of this root template, measurement templates could only be used to construct subordinate objects included in other templates, but not to encode standalone measurement objects.This CP also added some of the codes needed for this project, and allowed common content items to be factored out of individual measurements to the group level.

CP 1388: Add Real World Value Map Reference to Measurements (standard in 2014b)
This CP added an explicit reference to the RWVM instance that was used to calculate the measurements to the measurements SR object template.

CP 1389: Factor Common Descriptions Out of Image Library Entries (standard in 2014b)
We introduced simplifications to the structure of the measurements SR object by allowing a group of images to share common image library attributes, greatly reducing the size and improving the readability of the object in cases when measurements were derived from many single frame images.

CP 1465: Add type of finding to measurement SR templates (standard in 2015c)
The measurement template was amended to include the type of finding, which is distinct from its anatomical location.

CP 1466: Add session to measurements group (standard in 2015c)
An extra item was added to the measurement template to enable encoding of the session identifier to support experiments where the measurement of the same finding is performed several times in order to evaluate its repeatability.

Implementation of the conversion to DICOM format
Our overall strategy for data conversion was developed to accommodate the organization of the data at the site conducting the study.As discussed earlier and illustrated in Fig. 1, a relational database (RDB) was used to support project data management.The database served the two main purposes.First, it linked the segmentation images stored in NRRD format with the information about the operator, session, algorithm type and semantic information about the region being segmented.
This section describes the customized routines developed to perform conversion of the individual components of the data stored in the internal databases.SUV normalization and quantitative measurements were calculated using the FOSS tools developed as part of this project, while segmentations were converted from the results obtained before the open source implementation of the semiautomatic segmentation tools was released.The toplevel script that was used to perform the conversion of a complete dataset by invoking conversion routines for the individual data types is available in the Iowa2DICOM code repository (QIICR, 2015c).

Clinical Information: DICOM SR
Clinical data was exported from the internal SQL database as a tabdelimited text file.An XSLT script was used to convert the tabdelimited representation into XML form, followed by another XSLT transformation that produced an XML representation of an object that follows DICOM SR template TID QIICR_2000 documented in Appendix 1 .Finally, the resulting XML representation was converted into DICOM format using standard functionality of the PixelMed toolkit (Clunie, 2015c).The conversion scripts are available in a public source code repository (QIICR, 2015d).The DICOM series containing the clinical data DICOM SR were assigned to a study separate from the one for the imaging and derived data, with both the StudyDescription and SeriesDescription attribute set to "Clinical Data".

Standardized Uptake Value: DICOM RWVM
RWVM objects were generated in batch mode using the SUV calculation plugin of 3D Slicer (QIICR, 2015e).The plugin operates on the list of files corresponding to the PET series DICOM objects, calculates SUVbw factor and produces a single RWVM object.Injected dose, patient weight, radionuclide halflife and injection time are obtained from the DICOM PET image header.

Image Segmentation: DICOM SEG
The process of converting segmentation results stored in the NIfTI format into DICOM representation was facilitated by the FOSS DICOM software library implementation available in DCMTK (DICOM Toolkit) and maintained by OFFIS in Germany (Eichelberg et al., 2004).At the time our project commenced, the toolkit was oriented primarily to reading DICOM objects and DICOM networking.Only low level routines were available in order to create and modify such objects, and new objects had to be put together by manipulating individual DICOM attributes (PatientName, PixelData, Rows, Columns, etc.).DICOM defines various rules to be followed for each object type in order to select the correct set of attributes from the several thousand in the data dictionary.Interpretation of the standard and implementation of the standard guidelines requires detailed knowledge of the subject, and can be errorprone in the absence of supportive tools.
To simplify the task of creating SEG objects for this project and other similar efforts, we extended DCMTK with three new libraries: dcmiod , dcmfg and dcmseg .dcmiod eases user access to nearly all kinds of DICOM image objects ("iod" in dcmiod stands for the DICOM "Information Object Definition" nomenclature) by providing dedicated methods to get, set or modify attributes through a specific API (e.g., setStudyDate () can be used to set the StudyDate DICOM attribute to a specific value).When a newly created DICOM dataset is serialized, dcmiod performs checks to enforce DICOM rules, such as the existence of an attribute in the appropriate DICOM module (related groups of attributes), and, if applicable, validates its value.As a result, the library helps to prevent the user from saving DICOM objects that do not conform to the standard.
The second library, dcmfg , adds constructs to support the characteristics of the "Enhanced Multiframe" family of DICOM objects, particularly the "Functional Groups".Traditional (now called "legacy") DICOM image objects are still the most commonly encountered in the output of the image acquisition systems, which store each image slice of a series as a separate object.As a result, most of the attributes and modules (e. g. patient, study and image information) are repeated across all of the objects that belong to the same series.The Enhanced Multiframe image objects rectify this issue by representing an entire acquisition with multiple reconstructed slices as a single object, addressing redundancy and duplication as well as integrity of the entire set.Being more recently defined, the Enhanced Multiframe objects also offer a much richer set of standard attributes and values to describe the parameters pertinent to contemporary image acquisition technology.
Functional Groups contain related attributes for each slice within an object that is either information common to all frames ("shared") and which can be factored out and encoded once, or specific to each frame ("per frame") and which needs to be specified for each frame separately.Enhanced Multiframe objects also support organization of the frames and annotation of their semantics beyond the descriptive attributes.For example, the "Multiframe Dimension" module can be used to impose a meaningful ordering of frames by giving them a position according to a meaningful set of dimensions, such as position in space and time.The dcmfg library also performs several checks before writing such objects in order to enable creation of only valid DICOM objects, to the extent possible.
The dcmseg library of DCMTK utilizes both dcmiod and dcmfg to support operations on SEG objects, which belong to the specific type of the Enhanced Multiframe objects.It provides an API that makes it clear to the user which data input is required and finally, whether the provided data input conforms to the DICOM standard.
All three libraries ( dcmiod , dcmfg and dcmseg ) have been integrated into the official DCMTK source code branch and thus are available without restrictions to the broader community of the DCMTK users and adopters, both academic and commercial.

Volumetric Measurements: DICOM SR
The process of calculation and encoding of the ROI measurements was implemented in 2 steps.First, measurements of interest were calculated in batch mode using the QuantitativeIndicesCLI tool available within PETIndiC extension of 3D Slicer (QIICR, 2015b).The tool accepts the SUVnormalized image volume and the segmentation label saved using a domainspecific format, such as NRRD or NIfTI, and produces a text file encoding the measurements as keyvalue pairs.The keys of the output correspond to the research labels assigned to the measurement classes.We note that not all of the measurements were generated for each of the ROIs.Specifically, calculation of a meaningful value for SUV peak (Wahl et al., 2009) is not possible when the ROI is too small.In the cases when the measurement was not generated by the tool, it was omitted from the DICOM SR measurements object.
Next, we used a converter EncodeMeasurementsSR available within the Iowa2DICOM repository (QIICR, 2015c) to generate DICOM SR objects containing the calculated measurements.This converter accepts as input the list of DICOM PET object file names, the SEG object file name, and the text measurements, and produces the final DICOM SR object.The conversion is done using the dcmsr library of DCMTK that provides interfaces to create and iterate through a tree of DICOM SR object content.

Validation of DICOM encoded objects
DICOM is a complicated standard that requires specialized experience to properly implement.Validation of the DICOM encoded data to ensure its consistency and conformance to the standard is important and requires special attention.We approached validation of the encoded data at different levels using FOSS DICOM validation tools discussed below.
The dciodvfy tool (Clunie, 2015a) was used to ensure that an object complies with the basic DICOM encoding rules and contains the appropriate required attributes for the images, SEG, RWVM, and SR objects.This tool does not validate compliance with specific SR templates, only that valid combinations of content items and relationships are present.
The dcentfvy tool was used to validate that a set of DICOM objects contain the correct values for all attributes for the same entity level in the DICOM Information Model (i.e., that all patient attributes are the same for the objects with the same PatientID value, that all study attributes are the same for objects with the same StudyInstanceUID value, etc.).This tool is particularly helpful when objects have been created along different paths or using different tools than the original images, and/or uploaded to the distribution archive on separate occasions.
The com.pixelmed.validate.DicomSRValidator tool (Clunie, 2015b) was applied to validate compliance with the subset of SR templates that are supported by the tool, which includes the TID 1500 root template and the subordinate templates used in this project.The validation consists of checking that the required content items are present at the correct level in the content tree, that conditional content items are present when specified conditions are satisfied, that correct concepts and required values from specified context groups are used, that concepts are encoded with the expected code meanings, and produce warnings when unrecognized content items are detected (which often signals that a content item has been misplaced in the tree).Since TID 1500 is a general purpose template and is not specific to this project, it does not contain the constraints that only the subordinate templates and codes for this project be used.This is an interesting limitation, in that earlier experiments using a private QIICRspecific template with a smaller defined set of valid subtemplates, concepts and values representing only projectspecific context actually performed a more thorough automated check.In the absence of a DICOM standard process or method for defining "specialized" templates based on existing standard templates, some toolspecific mechanism, such as providing a "profile" precondition option to the validator might be useful in future (in passing, dciodvfy already contains such a profile mechanism, which is used for validating IHEspecific constraints).
In addition to the image processing tools listed above, we provide source code of the FOSS tools used to create DICOM representations of the analysis results in the Iowa2DICOM repository: https://github.com/QIICR/Iowa2DICOM .

Results
Clinical data and the analysis results for the total of 60 PET/CT imaging studies were encoded in the DICOM format using the procedures described.One patient had a repeat imaging study.The remainder had only the baseline study augmented with the clinical data and quantitative analysis results DICOM objects.
One RWVM object, 15 SEG objects (3 RRs and tumor/lymph nodes segmentations by 3 readers using 2 tools during 2 reading sessions), and 15 volumetric measurement SR objects (one per SEG) were produced for each imaging study.
The DICOM objects were added to the QINHEADNECK collection of TCIA (The Cancer Imaging Archive (TCIA), 2015b) and are available for public access .TCIA was selected for archival of the resulting data since it 7 is the QINrecommended data sharing platform, and the analysis generating the encoded data was done as part of the QIN activities at the University of Iowa.
Standalone validation and consistency checks were conducted as described above.In addition, interoperability testing was performed as described in the remainder of this section to confirm that the objects can be ingested and used by commonly available tools and toolkits: DCMTK (OFFIS, 2014), GDCM (Malaterre, 2015), dicom3tools (Clunie, 2009) and PixelMed (Clunie, 2015c).
The traditional DICOM encoding format is binary, and data stored in that form is most easily visualized after transformation into a humanreadable text format, for which different options exist.One commonly used approach is to look at a socalled "dump", which lists each attribute with its tag, type (value representation), name and value (with hierarchical nesting of sequences shown as required).The following publicly available tools were tested and able to successfully dump the objects we created: It is also possible to convert the DICOM format into XML or JSON representations, either according to schemas recently defined by the DICOM standard for this purpose (National Electrical Manufacturers Association (NEMA), 2015s), or using nonstandard schemas.These representations make the data amenable for consumption by the variety of established tools such as various NoSQL databases, XML query and transformation engines, etc., and are also nominally "humanreadable".We tested the following tools to confirm they can perform conversion of the objects we generated into an XML representation: • DCMTK dcm2xml ( xml2dcm for reverse conversion) • GDCM gdcmxml • PixelMed com.pixelmed.dicom.XMLRepresentationOfDicomObjectFactory DICOM SR objects can also be interpreted at a higher level of abstraction, which describes the content items of the content tree instead of the individual attributes that compose each content item.Such SR content tree "dumps" are more amenable to human interpretation than the attribute level dumps.The following tools were tested to produce SR tree dumps of the objects we generated: DICOM SR objects can also be converted into an XML representation according to a schema defined at the level of abstraction of the SR content tree rather than the individual attribute level.Such representations are very suitable for integration of the DICOM data with a variety of XMLoriented tools.A caveat is that DICOM has not yet established a standard schema for such a conversion, so the XML representation is dependent on the schema implemented by the specific tool.The following tools were tested and found to be capable of generating XML representations of the DICOM SR content for the objects we generated: • DCMTK dsr2xml ( xml2dsr for reverse conversion) • PixelMed com.pixelmed.dicom.XMLRepresentationOfStructuredReportObjectFactory Finally, the DCMTK dsr2html tool can be used to generate an HTML representation of the SR content tree that can be rendered in a humanreadable form in any HTML viewer.The dsr2html tool was tested and found to be able to visualize the SR objects that we generated.
All of the tools discussed above are command line tools.Interactive applications that wrap those command line tools are also available.The dcmjs dump (CommonTK, 2015) tool provides a web interface to DCMTK dcmdump , with the data processing done fully on the client side.The dicomdump package (QIICR, 2015f) of the FOSS Atom editor wraps both dcmdump and dsrdump tools of DCMTK, and can be used to interactively invoke those tools on the DICOM objects opened in the Atom editor.A rendered view of a section of the HTML representation of the same object as produced by dsr2html is shown in Fig. 7.
The foregoing checks did not serve to test more complex applicationlevel interoperability.Additional tests were performed for the SEG objects.Since regions of interest encoded as segmentations may be visualized in relation to the images from which they were segmented, we investigated the interoperability of several imaging workstations with respect to their ability to correctly render segmentations superimposed on the PET images.The following software was tested:

Discussion and Future work
We have presented a detailed investigation of the development and application of the DICOM standard and supporting FOSS tools to encode research data for quantitative imaging biomarker development.Using the reallife research scenario of HNC PET/CT quantitative image analysis we demonstrated that the DICOM standard is capable of representing various types of analysis results and their interrelationships.The resulting standard format data objects are interoperable with the variety of tools readily available to the researcher, as well as commercial clinical imaging and analysis systems (which universally support many aspects of the DICOM standard).
Realistic quantitative imaging research scenarios necessitate the use of a variety of data sources and processing routines; the results of such analyses are inherently complex.Coupled with the complexity of the DICOM standard, the length of this descriptive document may be intimidating.The purpose is to provide a complete and reproducible description of the process, both from the data modeling and implementation perspective.This document is intended to serve as a reference guideline for adopting the DICOM standard for quantitative imaging research and interoperable communication of the results of such research.
A key strategy for mitigation of complexity is the provision of appropriate tools.We hope that the burden of complexity on the individual researcher can be minimized, whilst reusability and interoperability can be maximized, by leveraging and improving existing DICOM FOSS tools and toolkits, instrumenting widely used research applications with DICOM capability, and providing a clear path selecting and linking an appropriate, relevant, and sufficient subset of DICOM capabilities for the research use case.
We believe this work is first of its kind to demonstrate, by example, the utility of the DICOM standard for interoperable quantitative result encoding in the quantitative imaging research domain, complete with the publicly available FOSS implementing the conversion and interpretation/visualization tools, encoded objects and documentation describing the specialized templates used for data encoding.Furthermore, we intentionally delve into the details of the various correction proposals that were contributed to the standard as part of our work, to demonstrate that DICOM is an evolving standard that is open to improvements that are needed to support research use cases.The improvements to the standard contributed by this project have wider applicability and, as we hope, will greatly simplify the task for adopters of the DICOMbased encoding approach.
Data conversion, as implemented and described in this paper, was performed retrospectively.We did not use DICOM as the operational data format, but instead adopted it to enable archival and sharing of the final analysis results, since the purpose was to reuse data already acquired for a research study to test the hypothesis, rather than wait until improved tools were fully deployed for prospective data acquisition.We are not arguing that retrospective conversion is preferred, quite the contrary.It is practical though, since historical analysis pipelines often contain tools developed using different toolkits and languages that may not yet have support for the various DICOM objects we utilized.The installed base of research tools may also not yet contain sufficient mechanisms for maintaining and propagating the patient and study level information (the composite context).Our project demonstrates how in situations like the one encountered in this project composite context can be recovered and merged into the shared results retrospectively, to reassociate images, derived results and clinical data.Addressing this key barrier to interoperability with the clinical environment should be a high priority for the research community, particularly since scalability to large experiments as well as the conduct of clinical trials, especially those spread across multiple sites or using multiple tools, inherently requires a solution to management of data identity and provenance.That said, the questions of the format for interoperable exchange, and that for internal operational use, can remain distinct to the extent deemed appropriate for any particular research scenario.
The work presented in this paper is a step towards improving support of quantitative imaging research use cases in DICOM, and improving support of the relevant parts of the DICOM standard in both FOSS and commercial tools and toolkits.We are actively engaged in improved integration of DCMTK with 3D Slicer to provide streamlined user interfaces that empower the end users to store the results of their work as appropriate DICOM objects with minimum extra effort.Although the specific use case described in this paper investigates PET/CT QI research, the approach has broad applicability for interoperable communication of the segmentation and quantitative analysis results independent of the imaging modality.At the level of developer toolkits, we are developing an API in DCMTK to support abstractions related to generation of volumetric measurement SR objects (TID 1500).We are also in the process of extending the DCMTK API to support the creation of Enhanced Multiframe objects for Magnetic Resonance (MR) imaging.We are planning to use that functionality in other QI biomarker use cases being developed by QIICR that focus on the use of multiparametric MRI in glioblastoma and prostate cancer characterization.In support of those use cases that involve analyses that generate derived functional maps of tissue properties, QIICR has also contributed to the development of the Parametric Map object in DICOM (National Electrical Manufacturers Association (NEMA), 2015k), now part of the standard, which supports encoding of floating point pixel data without being restricted to rescaling of integer values, finally resolving a longstanding perceived weakness of DICOM for research applications.
Another area of QIICR focus is development of the tools to ease the process of interacting with the standard and exploring the content of the DICOM data.In this area, we have developed the initial version of a DICOM search index that provides fast and, in our view, a more intuitive interface to the DICOM standard (QIICR, 2015g), and contributed dicomdump package for the popular Atom editor discussed earlier (QIICR, 2015f).These multipronged activities are intended to assist a diverse variety of groups, which include academic QI researchers (both technical and clinical), software developers implementing QI analysis tools, clinical end users, and developers of the commercial tools deploying QI biomarkers.Our goal is to make it easier for the interested parties to explore, evaluate and implement DICOM standard capabilities relevant for QI research.We hope this will contribute to the technical solution of the overarching problem of standardized and meaningful sharing of reproducible research results, as well as improve the integration of the research tools with the clinical systems to facilitate the translation of QI biomarker clinical trials and clinical research studies into clinical practice.
The work presented is a direct result of two years of activities performed by the QIICR project, but it builds upon the foundation established by the various research groups, communities and FOSS projects, such as 3D Slicer and DCMTK, decades before QIICR was established.We are committed to continue working with those groups and communities, as well as other stakeholders and adopters interested in remedying the status quo of very limited sharing of the quantitative analysis results in the imaging community.

Figure 1 :
Figure 1: Diagram of the interaction among the various data sources and processing steps that result in the dataset described in this paper.Components of the dataset represented in DICOM are released publicly within the TCIA QINHEADNECK collection.FOSS tools corresponding to the processing steps other than Reference Region (RR) segmentation (processing steps with the dashed outline) are available.

Figure 2 :
Figure 2: An illustration of the relationships among the DICOM objects discussed in this manuscript.DICOM PET/CT is the original dataset obtained by the imaging equipment and is modified only by the deidentification procedure.DICOM RWVM, SEG, and measurement SR are derived objects.DICOM SR with the clinical information encodes the information about the patient originally stored in the relational database.Solid lines denote explicit reference of the object instances by the derived objects (referenced instance is pointed to by the arrow).Dashed bidirectional arrows denote commonality of identifiers(i.e., common composite context, e.g., at  the Patient and Study level).

Figure 3 :
Figure 3: Relationships of the private templates used for encoding of the clinical information.

Figure 4 :
Figure 4: The family of DICOM SR templates used for communicating the PET measurements.

Figure 5 :
Figure 5: An attributelevel dump corresponding to the section of the DICOM SR measurements object encoding SUVbw peak value for subject QINHEADNECK010024, series "tumor measurements User1 Manual trial 1", as displayed in the Atom editor using dicomdump package.

Figure 6 :
Figure 6: An XML representation of the attributelevel SUVbw peak measurement encoding for subject QINHEADNECK010024, series "tumor measurements User1 Manual trial 1".
(BusinessWire, 2013)ability.This interoperability transcends modality and Picture Archiving and Communication System (PACS) boundaries to include downstream systems, such as those used for speech recognition driven human reporting systems, which are now also capable of ingesting DICOM SR measurements and codes (i.e., in the PACSgear ModLink (Imaging Technology News, 2013), Nuance PowerScribe 360(BusinessWire, 2013)and MModal Fluency for Image Reporting (MModal, 2014)) products.The recent trend towards the widespread implementation of Radiation Dose SRs (RDSRs) has the potential to greatly extend the breadth of the clinical infrastructure that supports DICOM SR