Assessment of methods used for 3-dimensional superimposition of craniofacial skeletal structures: a systematic review

Background So far, several techniques have been recommended for the assessment of craniofacial changes through skeletal tissue superimposition, but the evidence that supports them remains unexplored. The purpose of the present study is to assess the available literature on skeletal-tissue superimpositions of serial craniofacial CT or CBCT images used to detect morphological changes. Materials and Methods Medline (via Pubmed), EMBASE, Google Scholar, Cochrane Library, Open Grey and Grey Literature Report were searched (last search: 17.11.2019) using specific terms that fulfilled the requirements of each database in the context of the study aim. Hand searches were also performed. The outcomes of interest were the accuracy, precision, or agreement between skeletal-tissue superimposition techniques to assess changes in the morphology of craniofacial structures. Studies of any design with sample size ≥3 were assessed by two authors independently. The study protocol was registered in PROSPERO (ID: CRD42019143356). Results Out of 832 studies, fifteen met the eligibility criteria. From the 15 included studies, 12 have shown high total risk of bias, one low risk of bias, and two studies have shown unclear risk of bias. Thirteen out of the 15 studies showed high applicability concerns, two unclear and no study had low applicability concerns. There was high heterogeneity among studies regarding the type of participants, sample size, growth status, machines, acquisition parameters, superimposition techniques, assessment techniques and outcomes measured. Fourteen of them were performed on Cone Beam Computed Tomography (CBCT) and one on Computed Tomography (CT) derived 3D models. Most of the studies (eleven) used voxel-based registration, one landmark-based registration and three studies compared different registration techniques, which include the surface-based registration. Concerning the area of interest, nine studies focused on the anterior cranial base and certain facial structures, four on maxillary structures and four on mandibular structures. Non-growing participants were included in six studies, growing in eight, whereas one study had both. Conclusion Most of the available studies had methodological shortcomings and high applicability concerns. At the moment, certain voxel-based and surface-based superimpositions seem to work properly and to be superior to landmark-based superimposition. However, further research in the field is required to develop and properly validate these techniques on different samples, through high quality studies with low applicability concerns.


INTRODUCTION
Page 4-5 Rationale 3 Describe the rationale for the review in the context of what is already known.

5
Objectives 4 Provide an explicit statement of questions being addressed with reference to participants, interventions, comparisons, outcomes, and study design (PICOS).
Page 6

METHODS
Page 5-8 Protocol and registration 5 Indicate if a review protocol exists, if and where it can be accessed (e.g., Web address), and, if available, provide registration information including registration number.

5
Eligibility criteria 6 Specify study characteristics (e.g., PICOS, length of follow-up) and report characteristics (e.g., years considered, language, publication status) used as criteria for eligibility, giving rationale.

5
Information sources 7 Describe all information sources (e.g., databases with dates of coverage, contact with study authors to identify additional studies) in the search and date last searched.

5, 6
Search 8 Present full electronic search strategy for at least one database, including any limits used, such that it could be repeated.

Appendix 1
Study selection 9 State the process for selecting studies (i.e., screening, eligibility, included in systematic review, and, if applicable, included in the meta-analysis).

6, 7
Data collection process 10 Describe method of data extraction from reports (e.g., piloted forms, independently, in duplicate) and any processes for obtaining and confirming data from investigators.

7
Data items 11 List and define all variables for which data were sought (e.g., PICOS, funding sources) and any assumptions and simplifications made.

6, 7
Risk of bias in individual studies 12 Describe methods used for assessing risk of bias of individual studies (including specification of whether this was done at the study or outcome level), and how this information is to be used in any data synthesis.

Tables 2, 3, 4
Risk of bias within studies 19 Present data on risk of bias of each study and, if available, any outcome level assessment (see item 12).

Table 1
Results of individual studies 20 For all outcomes considered (benefits or harms), present, for each study: (a) simple summary data for each intervention group (b) effect estimates and confidence intervals, ideally with a forest plot.

Page 14-17
Summary of evidence 24 Summarize the main findings including the strength of evidence for each main outcome; consider their relevance to key groups (e.g., healthcare providers, users, and policy makers).

15-17
Limitations 25 Discuss limitations at study and outcome level (e.g., risk of bias), and at review-level (e.g., incomplete retrieval of identified research, reporting bias).

Page 22
Funding 27 Describe sources of funding for the systematic review and other support (e.g., supply of data); role of funders for the systematic review.

PRISMA item # 4 and #6
Page 5 • Study design: Any study design, including prospective, and retrospective studies of any type.
• Study sample: Studies with sample size ≥ 3.
• Index test: 3D skeletal-tissue superimposition techniques to assess any change in the morphology of the craniofacial complex.
• Types of participants: Serial craniofacial CT or CBCT images of individuals or skulls who have received any kind of actual or simulated treatment, or whose craniofacial morphology is expected to be altered due to growth or pathology.
• Type of intervention: 3D skeletal-tissue superimposition to assess any morphological change in the craniofacial complex.
• Primary outcome: Superimposition accuracy or precision of a technique, or agreement between techniques measured in terms of angles or distances between specific skeletal or facial landmarks or area distances between corresponding models. Volume differences measured following 3D superimposition were also considered. Studies that evaluated any of the above parameters as a secondary outcome were also included.
• Comparator/control group: Studies that compared different superimposition techniques, direct measurements, or repeated measurements were selected.
• Unit of analysis: The measured distance, angle, or volume.
• Follow-up: Any observation period between subsequent models.

PRISMA item # 5
The protocol was registered in PROSPERO prior to the study implementation (ID: CRD42019143356).

PRISMA item # 7
The following databases were searched for eligible studies: Medline (via Pubmed), EMBASE, Google Scholar, Cochrane Library, OpenGrey and GreyLiteratureReport. The last search was performed on 17.11.2019, without time restriction. Unpublished literature was searched through the National Research Register, Pro-Quest Dissertation Abstracts and Thesis database, additional hand searches of all relevant studies were also performed. The specific search strategies applied for each database are provided as supplementary information (Appendix 1).

PRISMA item # 9
Page 6, 7 Following the search strategy, the selected databases were screened by two authors of the review (Daniel Dinh-Phuc Mai and Sven Stucki). There was no blinding concerning the authors' names and affiliations, or the outcomes of the included studies. Titles and abstracts were evaluated first, if necessary the full text was read to evaluate the eligibility. The same authors read all eligible studies again in full text, independently, whereas non-eligible studies were excluded. Thereafter the eligibility was discussed between all team members until a consensus was reached, under the guidance of the last author (Nikolaos Gkantidis). A record of all decisions made during this process was retained.

PRISMA item # 10 and 11
Page 7 The first and the last author performed data extraction independently and in duplicate, aiming to extract from the eligible studies the following information: • Methods: Author, title, year, objectives, and design of study.
• Materials: 3D model acquisition method and time between serial models.
• Superimposition method: Type of superimposition reference areas or points and software with specific settings used.
• Comparison/control group: Type and characteristics.
• Outcome: Type of outcome(s) and method of outcome assessment.
If necessary, the authors were contacted by email to request missing data. If the relevant information was not provided, only the available information was used.
The quality of the selected studies was evaluated using the QUADAS-2 tool (Whiting, 2011). This is a widely used tool to evaluate the diagnostic accuracy of methods in systematic reviews. Using the QUADAS-2 tool the patient selection, the index test, the reference standard and the flow and timing are evaluated regarding their risk of bias and applicability concerns. Usually, gradings are shown in a table using happy (low risk) or sad smiles (high risk). In case an evaluation is not possible, e.g. because of missing data, an interrogation mark is shown (unclear risk). The total risk of bias or applicability concerns of each study correspond to the worst rating given in the individual items assessed each time.
The quality assessment of all studies was performed by two authors (Daniel Dinh-Phuc Mai and Nikolaos Gkantidis) independently. If there was a disagreement, a consensus was reached through discussion among all authors. Studies graded with a high risk of bias were not to be included in a meta-analysis.

PRISMA item # 15
Page 7 We conducted an accurate, but also broad enough search of multiple sources, including on-going studies, to minimize potential reporting biases, such as publication bias and duplicate reports.

PRISMA item # 16
Page 7, 8 Results will also be tested for the following factors, if possible: • CBCT vs. CT data.
• Superimposition on the anterior cranial base vs. superimposition on maxillary structures vs. superimposition on mandibular structures The search results are shown in Figure 1. After searching various databases, 2'540 studies were found. Seven additional studies were identified through hand searches. After removing the duplicates, 832 studies remained. These studies were screened by reading the titles and abstracts.
Full-text reading of 24 studies was performed to evaluate the eligibility. Nine studies did not match the review question and thus, they were excluded as irrelevant to the study topic. Following the selection process 15 studies were included in this review.

PRISMA item # 22
Page 9 The quality assessment of the included studies is provided in Table 1.
From the 15 included studies, 12 of these have shown a high total risk of bias, one a low risk of bias, and 2 studies have shown an unclear risk of bias. Regarding the individual items 4 studies have high, 7 low, and 4 unclear risk of bias in the patient selection. Regarding the index test, 8 studies have high, 6 low, and one unclear risk of bias. The reference standard of 9 studies shows a high risk of bias, of 2 low, and of 4 unclear. The flow and timing of 2 studies has high, of 11 low, and of 2 unclear risk of bias.
Thirteen out of the 15 studies showed high total applicability concerns, 2 unclear and no study had low applicability concerns. Concerning the individual items, 6 studies had high, 6 unclear and 3 low applicability concerns in the patient selection. Regarding the index test, 9 studies had high, one unclear and 5 low applicability concerns. The reference standard of 9 studies showed high, of 3 unclear and of 3 low applicability concerns.

PRISMA item # 25
Page 17 Though a significant amount of studies was identified, a limitation of the present study is that the heterogeneity of the included studies is high, and the quality of the available evidence is limited. This can be attributed to the fact that the field has been developed in the last few years and gained much attention only recently.

PRISMA item # 26
Page 17 The fast evolution of 3D superimposition techniques has provided a key element in the toolkit of relevant fields to evaluate craniofacial changes following growth or treatment. Due the high heterogeneity and the moderate to low quality of the included studies, few valid conclusions can be drawn. Most of the available studies had methodological shortcomings and high applicability concerns. Therefore, no clear recommendation could be given at present for proper methods used for 3D-superimposition of craniofacial skeletal structures. At the moment, certain voxel-based and surface-based superimpositions seem to work properly and to be superior compared to landmark-based superimposition. However, further research is necessary to develop and properly validate these techniques on different samples, through studies with high quality and low applicability concerns.