Exploring a modelbased analysis of patient derived xenograft studies in oncology drug development
 Published
 Accepted
 Received
 Academic Editor
 Matteo Lambrughi
 Subject Areas
 Drugs and Devices, Computational Science, Data Science, Pharmacology, Oncology
 Keywords
 Patient derived xenograft, Tumour growth inhibition, Mathematical modelling, Statistical modelling
 Copyright
 © 2021 Dickinson et al.
 Licence
 This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. For attribution, the original author(s), title, publication source (PeerJ) and either DOI or URL of the article must be cited.
 Cite this article
 2021. Exploring a modelbased analysis of patient derived xenograft studies in oncology drug development. PeerJ 9:e10681 https://doi.org/10.7717/peerj.10681
Abstract
Purpose
To assess whether a modelbased analysis increased statistical power over an analysis of final day volumes and provide insights into more efficient patient derived xenograft (PDX) study designs.
Methods
Tumour xenograft timeseries data was extracted from a public PDX drug treatment database. For all 2arm studies the percent tumour growth inhibition (TGI) at day 14, 21 and 28 was calculated. Treatment effect was analysed using an unpaired, twotailed ttest (empirical) and a modelbased analysis, likelihood ratiotest (LRT). In addition, a simulation study was performed to assess the difference in power between the two dataanalysis approaches for PDX or standard cellline derived xenografts (CDX).
Results
The modelbased analysis had greater statistical power than the empirical approach within the PDX dataset. The modelbased approach was able to detect TGI values as low as 25% whereas the empirical approach required at least 50% TGI. The simulation study confirmed the findings and highlighted that CDX studies require fewer animals than PDX studies which show the equivalent level of TGI.
Conclusions
The study conducted adds to the growing literature which has shown that a modelbased analysis of xenograft data improves statistical power over the common empirical approach. The analysis conducted showed that a modelbased approach, based on the first mathematical model of tumour growth, was able to detect smaller size of effect compared to the empirical approach which is common of such studies. A modelbased analysis should allow studies to reduce animal use and experiment length providing effective insights into compound antitumour activity.
Introduction
Preclinical Oncology drug development is heavily reliant on xenograft studies to assess the antitumour effect of new compounds (Jung, 2014). These studies can vary in duration, as short as 14 days (Hather et al., 2014), to 4 weeks (Xu et al., 2018), and even longer depending on the control growth rate. They represent the first opportunity, during development, to assess how the kinetics of drug disposition affects the kinetics of tumour growth (Plowman et al., 1997). The xenograft study starts by grafting a human cellculture into the flank of an immunocompromised mouse. Digital callipers are then used to measure the length and width or the length, width and height to calculate tumour volume at regular time intervals. Once the grafted tumour has reached a certain prespecified volume each xenograft is randomised into one of the treatment arms or the untreated arm (control arm within the study). Thus, across all study arms the volume at randomisation is comparable. The treatment effect is then calculated by measuring the difference in mean tumour volumes between the treated and control group at the end of the study (Corbett et al., 2004). This metric is typically referred to as the Tumour Growth Inhibition (TGI) value and is calculated as follows: $\text{\%}\mathrm{TGI}\left(T=t\right)=\frac{V\left(\mathrm{control},T=t\right)V\left(\mathrm{treated},T=t\right)}{V\left(\mathrm{control},T=t\right)V\left(\mathrm{control},T=0\right)}$
where V (treated, T = t) and V (control, T = t) are the mean volumes of the treated and control group at time T = t, the end of the study and V (control, T = 0) is the mean volume of the control group at the time of randomisation. To assess whether the volumes at time t = T are significantly different between the control and treated arms of the study an unpaired, twotailed ttest is performed and the resultant pvalue is usually reported together with the TGI value.
The above approach to analysis of TGI data clearly ignores the timeseries that is generated up until the TGI value is recorded. Furthermore, the TGI value can become biased if mice have dropped out at a timepoint before the TGI value is calculated. This is common in the control arm due to the volume exceeding a predefined animal welfare limit. Thus, performing an unpaired, twotailed ttest on the final day of the study, results in an underprediction of the mean control volume and hence the efficacy of the treatment is underestimated. This issue, however, can be resolved by calculating TGI at an earlier timepoint where no dropouts exist or a joint longitudinal dropout model could also be used (Martin, Aarons & Yates, 2016).
An alternative to performing an unpaired, twotailed ttest on the final study day is to perform a modelbased regression analysis. The key advantage of performing a regression analysis over the approach discussed is that all data points in the time series are used. This will lead to an increase in statistical power, hence reduce the number of animals used and thus reduce the cost of a xenograft study.
A previous study, by Hather et al. (2014), has shown that a modelbased regression approach is likely to improve the power of xenograft studies over doing an unpaired, twotailed ttest. However, the Hather et al. study did not consider the application of such an approach to patient derived xenografts (PDX), which are known to have higher variance than the standard cellline xenograft and are becoming more popular within preclinical development. Furthermore, the approach by Hather et al. did not consider the use of mixedeffects/hierarchical modelling approach which may likely further increase the statistical power of such studies.
In this study we build on the work by Hather et al. by assessing the increase in power obtained by using a modelbased mixedeffects regression analysis and a naïve pooled approach over the typical unpaired, twotailed ttest analysis of final volumes for a large open PDX database (Gao et al., 2015). In addition, we also analysed a traditional standard cellline xenograft study (Knutson et al., 2016a) for comparison to the patientderived xenograft analysis and perform a brief simulation study highlighting the merits of a modelbased mixedeffects regression analysis.
There are numerous mathematical models that can be used within a modelbased analysis of tumour growth data (Ribba et al., 2014). Within this study we have chosen to use the first tumour growth model developed by Mayneord in 1932 (Mistry, Orrell & Eftimie, 2018). This model has been derived from first principles and recently been shown to provide a good description of both preclinical and clinical tumour size timeseries data (Mistry, Orrell & Eftimie, 2018; Orrell & Mistry, 2019).
Methods
Real data study
Xenograft data
Data from a study by Gao et al. (2015) which analysed 1000 PDX models across 59 treatments and controls was collected. The data was then grouped creating 59 two arm studies involving a control and treatment arm (hence a ‘study’ refers to a different treatment where, within that study, there was 2 arms, a treatment and control arm). This data was then truncated to either 14, 21 or 28 days, to mimic typical lengths of xenograft studies, for analysis.
Empirical approach: unpaired, twotailed Ttest
On the final day of the study, day 14, 21 or 28, an unpaired, twotailed ttest between the control and treated volumes was conducted with a pvalue and %TGI reported.
Modelbased approach: likelihood ratiotest
The timeseries from the animals used within the empirical approach were used for the following modelbased analysis. By assuming that the tumour is spherical we first converted tumour volume to radius using: $R={\left(\frac{3V}{4\pi}\right)}^{\frac{1}{3}}$
where V is the volume of the tumour and R the radius.
Next, we fitted the following model, whose mechanistic derivation can be found in Mistry, Orrell & Eftimie (2018), Orrell & Mistry (2019) and Mayneord (1932) and in the Supplemental Information, to the combined two arm dataset:
${R}_{ij}=\left({a}_{i}+{b}_{i}{t}_{ij}\right)\left(1+{e}_{ij}\right)$ $log\left({a}_{i}\right)\sim N\left({\mu}_{1},{\sigma}_{1}^{2}\right)$ $log\left({b}_{i}\right)\sim N\left({\mu}_{2},{\sigma}_{2}^{2}\right)$ (1)${e}_{ij}\sim N\left(0,{\sigma}_{3}^{2}\right).$
where R_{ij} is the observed radius of xenograft i at time j, a_{i} is the value of the radius at time 0, time of randomisation, for xenograft i, b_{i} is the rate of growth of the radius for xenograft i, t_{ij} is the timepoint for xenograft i at time j at which the observation was recorded and e_{ij} is the residual error for xenograft i at time j. Note, that a proportional error model was used as the variability in tumour size grew over time i.e., we have heteroscedastic variance (see Supplemental Information for more details). We then modified the distribution of b_{i} and introduced a population treatment effect parameter c to account for difference between control and treated growth rates in the following way
${b}_{i}\sim N\left({\mu}_{2}+c{T}_{i},{\sigma}_{2}^{2}\right)$ (2)${T}_{i}=\left\{\begin{array}{c}0\phantom{\rule{1em}{0ex}}\text{if control}\phantom{\rule{10.00002pt}{0ex}}\hfill \\ 1\phantom{\rule{1em}{0ex}}\text{if treated}\phantom{\rule{10.00002pt}{0ex}}\hfill \end{array}\right.$
and refitted the model to the data. Since the models considered are nested the likelihood ratiotest (LRT) was used to assess if adding a treatment effect improved model fit to the data over a model with no treatment effect.
In addition to regressing against radius an analysis regressing against volume was also considered. The model used for the volume analysis was as follows:
${V}_{ij}=\frac{4\pi {\left({a}_{i}+{b}_{i}{t}_{ij}\right)}^{3}}{3}\left(1+{e}_{ij}\right)$ $log\left({a}_{i}\right)\sim N\left({\mu}_{1},{\sigma}_{1}^{2}\right)$ $log\left({b}_{i}\right)\sim N\left({\mu}_{2},{\sigma}_{2}^{2}\right)$ (3)${e}_{ij}\sim N\left(0,{\sigma}_{3}^{2}\right).$
where V_{ij} is the observed radius of xenograft i at time j. The assessment of treatment effect was conducted in a similar way as stated above.
In addition to using mixedeffects we also conducted a naïve pooled data modelling approach to assess treatment effect i.e., using just fixedeffects. This analysis was conducted using the radius model only.
All parameter estimation for the mixedeffect models was conducted using the saemix library in R. Parameter estimation for the pooled data approach was done using the minpack.LM library in R.
Simulated data study: power calculation
A 14, 21 and 28 day 2arm simulation study, using model (1) stated above, was conducted in the following way. All parameter values except for the treatment effect c were based on fitting model (1) to all control data up to day 14, 21 or 28. The treatment effect parameter c was chosen to give either a 50% or 100% TGI at 14, 21 or 28 days. The sample size for each arm of the simulated study was set at 5, 8, 10, 12 and 15. When simulating the tumour volumes, we assumed that the measurements were taken on the same days as in the Gao et al. study, so every 3 to 4 days. In addition to using the Gao et al. study to generate model simulation parameters we also utilised a standard cellline (CDX) study by Knutson et al., which investigated the effect of paclitaxel and paclitaxel conjugated to αCEA680PTX antibody in a BxPC3 model cell line (Knutson et al., 2016a), data was taken from datadryad.org (Knutson et al., 2016b).
For each simulated study, control and treatment arm, the empirical and modelbased approaches were applied with pvalues recorded. This was done 1000 times for a given simulation setup with the proportion of pvalues <0.05 recorded and visualised.
Results
Real data study
The results of the 14, 21 and 28 day 2arm, treatment versus control, study analysis of the PDX data can be seen in Fig. 1. The Figure shows the cumulative fraction of p < 0.05 studies as a function of TGI for both the modelbased and empirical approach. The key result is that regardless of the duration of the study the modelbased approach gives greater statistical power over the empirical approach. Furthermore, this increase in power leads to the modelbased approach being able to detect smaller TGI values (25%) compared to the empirical approach (50%).
Similar results were seen when regressing against volume rather than radius (see Supplemental Information). If the analysis was done using a naïve pooled approach for radius, a reduction in power was seen when the study endpoint was 14 days compared to the mixedeffects model. However, this difference was no longer apparent for longer studies. (see Supplemental Information).
Simulated data study
The results of the simulation study using parameters derived from analysing the controls of the PDX study can be seen in Table 1. It shows that the power of the modelbased approach is greater than that using the empirical approach. We can see that the power does decrease as we decrease the number of mice, as expected. A further simulation power analysis was done using parameters derived from a CDX study, details on the parameter values can be found in Supplemental Information. The result of this analysis in comparison to the PDX analysis can be seen in Table 2. The results show that if the variability is low and a 100% TGI is sought, then the power of using an empirical versus modelbased approach is similar. However, in the other scenarios explored we found that the modelbased approach has greater power over the empirical approach. The table further highlights the difference in power between CDX and PDX studies; this is due to the increased variability in timeseries seen in PDXs, see Supplemental Information for parameter estimates.
Percentage of pvalues <0.05  

Empirical  ModelBased  Empirical  ModelBased  Empirical  ModelBased  
TGI (%)  N  Day 14  Day 14  Day 21  Day 21  Day 28  Day 28 
100  15  61  71.1  65.3  85.6  68.6  100 
12  50.3  70  53.5  85.5  53.9  100  
10  48.8  70.3  47.4  83.1  45.8  99.7  
8  42.6  69.4  40.1  83.3  39  99.7  
5  25.8  66  23.5  78  17  98.5  
50  15  16  54.4  15.2  77.7  6.7  99.2 
12  10.6  46.9  10.7  71.9  5  96.9  
10  10  47.9  10.7  63.5  3.9  95  
8  8.1  43.7  8.8  59.4  3.5  93.7  
5  3.1  37.1  2.8  51.7  1.7  78.1 
Notes:
 TGI

tumour growth inhibition
Percentage of pvalues <0.05  

Empirical  ModelBased  
TGI (%)  N  Day  CDX  PDX  CDX  PDX 
100  10  14  71.3  48.8  71.4  70.3 
50  39.5  10  54  47.9  
100  21  85.2  47.4  85.7  83.1  
50  48.1  10.7  73.2  63.6  
100  28  99.6  45.8  100  99.7  
50  50.3  3.9  98.1  95 
Notes:
 TGI

tumour growth inhibition
 CV

coefficient of variation
 CDX

cellline derived xenograft
 PDX

patient derived xenograft
Discussion
It is well known that cellline derived xenografts (CDX) that have been established over the decades poorly mimic human disease (Williams, 2018). Thus, there has been a longstanding interest in developing new animal models that better mimic patient tumours and their microenvironments. One approach that has been gaining favour in recent years is the use of PDXs. These models show more variability in their timeseries and treatment response than their cellline derived counterparts mainly due to the increased heterogeneity within the sample used (Day, Merlino & Van Dyke, 2015). Given the increased cost of using PDXs versus CDXs more importance should be placed on how these studies are analysed than is presently done.
In this study we explored the typical empirical based analysis methods using the final volumes to modelbased approaches that use the timeseries up to a chosen endpoint across 59 2 arm trials taken from a publicly available PDX database (Gao et al., 2015). The empirical approach consisted of applying an unpaired, twotailed ttest to the final volumes of the control versus treated arms of a study. The modelbased approach, however, involved using a parametric model to describe the timeseries and the LRT to assess if including a treatment effect parameter improved model fit. Thus, the modelbased approach used all the data whereas the empirical based approach did not. Regarding the modelbased approach we explored 3 options: (1) using radius within a mixedeffects framework; (2) using volume within a mixedeffects framework and (3) using radius within a naïve pooled approach. It must be noted that in many xenograft experiments tumour volume is not measured directly but calculated from two length measurements. Errors in measurement are therefore made on the length scale not on the volume scale. Thus, regressing against a length may be more appropriate.
The results showed that the modelbased approaches had more statistical power than the empirical based approach. Regarding the type of modelling approach, a modest difference was seen between the mixedeffects approach and the naïve pooled approach and no difference seen between regressing against volume versus radius. The key result is consistent with larger studies involving CDXs that have been previously reported (Hather et al., 2014). We found that the modelbased approach could identify TGI values as low as 25% whereas >50% TGI is required for the empirical based method to detect a difference. Given that we would expect modest TGI with PDXs compared to CDXs, due to increased heterogeneity (Gao et al., 2015), this highlights the importance of using a modelbased approach for such analyses.
A second key result was that due to lower variability in controls between CDXs compared to PDXs, no difference in statistical power between the modelbased and empirical based approach was found when TGI was 100%. This observation supports the common practice of using the empirical based approach to analyse CDX data when high TGI are expected. This was not the case when moving to PDXs, where an increase in power was observed using a modelbased over empirical based approach indicating that modelbased approaches should become the common practice for analysing PDX data.
In summary, the results have shown that detecting modest TGI values such as 50% a modelbased approach will lead to increased power for both CDX and PDX studies. Wong et al. (2012) have shown that clinical exposure targets that relate to at least 60% TGI in CDX studies are required to see clinical efficacy. Thus, based on the power analysis conducted here a modelbased analysis would be the most appropriate approach to detect such modest efficacy in both PDX and CDX studies.
Unlike previous studies conducted within this field we chose to use the same endpoint, percent TGI, when comparing analysis methods. It must be noted that the modelbased approach does not calculate TGI directly from the data but the rate of change in tumour size. If a model based analysis is performed one option is to report a model estimated TGI, which can be calculated via simulation. This is an important factor to highlight as by performing this simulation, as part of the modelbased analysis, the experimental community can continue to use this familiar measure of efficacy which we hope will encourage the community to explore modelbased xenograft analysis approaches as we enter the age of more sophisticated animal models in cancer.