Confirmation bias leads to overestimation of losses of woody plant foliage to insect herbivores in tropical regions

General approach

We compared the point data (i.e., measured at a single point in time) on damage of woody plant foliage by insects that were collected in five different ways (Table 1). The first data set (DS1 hereafter) was compiled from publications reporting this damage on the basis of field studies conducted in Brazil (Appendix S1). We presumed that researchers who conducted these studies possessed extensive knowledge on both plants and insects of the study region, and therefore their selection of study objects, study sites and timing of measurements may have been affected by both this knowledge and by their expectations based on the widespread opinion that plant losses to herbivores are high in the tropics. The second data set (DS2 hereafter) was collected in Brazil by one of the authors (MVK) from several abundant plant species (Appendix S2). The third data set (DS3 hereafter) consists of photographs taken for identification of plant species, foliar damage of which was assessed by MVK (DS2). Plant losses estimated from these images (DS3) were compared with the results of field measurements (DS2) to check the reliability of our method of image analysis. These three data sets (DS1–DS3) were all based on non-blind data collection. However, we presumed that the selection of study species by local scientists was more influenced by prejudice than was the selection of study species by MVK due to his limited knowledge of local flora and fauna. Additionally, selection of study sites and time of sampling by MVK was based on factors not related to the topic of the present study. The fourth data set (DS4 hereafter) consisted of nature, wildlife and portrait photographs taken by VZ and ELZ with a Canon 60D camera for non-scientific purposes (i.e., holiday snaps) in the same habitats, where MVK conducted the field measurements of insect folivory (Appendix S3). Both VZ and ELZ were not aware of the intention of MVK to use their photographs for the present study until the end of the visit to Brazil, and therefore this data set (DS4) was presumed to be collected in a blind way. The fifth data set (DS5 hereafter) consisted of nature and wildlife photographs found in the WWW. It differs from DS4 by the wider coverage of localities in Brazil and of the timing of photographing.

Collection of published data

To be included in DS1, a published study had to be conducted in a way that assures the comparability of its results with the results of field measurements conducted by MVK, i.e., had to fit the following criteria:

1. Losses of plant foliage, in terms of the proportion of damaged leaves, leaf area or leaf biomass consumed (or modified, as in the case of gallers) by herbivorous insects from three feeding guilds (defoliators, miners and gallers) were measured from individual leaves of woody plants and were reported in a form that allowed calculation of the mean values of these losses (i.e., median values and indices that could not be transformed to the percentages were excluded);

2. The leaves for measurements of herbivory were selected in such a way that they were representative of the plant-wide level of damage (i.e., studies separately reporting the levels of damage of young and mature leaves were excluded), and the measured plant individuals were haphazardly selected from the population (i.e., studies separately reporting the levels of damage of healthy and damaged trees, resistant and susceptible trees or trees foraged and not foraged by ants were excluded);

3. The study was conducted in a natural forest or savanna (cerrado) ecosystem (i.e., data collected from mangroves, orchards, plantations and urban plantings were excluded);

4. The losses were measured in plant species that were native to the study region;

5. The losses were measured for adult plants (i.e., seedlings and saplings were excluded);

6. The losses to insect feeding were not combined with losses to fungal damage or to other damaging agents;

7. The losses were attributable to the entire community of herbivorous insects (i.e., data on plant damage caused by a single insect species were excluded) at the background level of their density (i.e., losses caused by rare events of mass occurrence of a certain herbivore were excluded);

8. The researchers did not mention specifically that losses were measured on young leaves soon after their flush.

We searched for publications that met these criteria in the ISI Web of Science database and Google Scholar using several conbinations of keywords (“Brazil” or “cerrado”, “folivor*” or “herbivor*”, “tree*” or “shrub*” or “woody”, “insect” and “damag*” or “loss*” or “consum*”) and further examined the reference lists of the identified papers. The search was completed on 31 October 2014.

When extracting the data from multiyear studies, we averaged values for all study years. If the study involved some manipulations, we used the data from non-treated (control) plants.

Field measurements of foliar damage

To create DS2, MVK measured foliar damage in mid-September of 2013 in four areas of Brazil: around Ilhéus on the Atlantic coast of Bahia (14°48′S, 39°04′W), in Vale do Capão, Chapada Diamantina National Park in the inland region of Bahia (12°37′S, 41°30′W), near Novo Airão on the banks of the Rio Negro (2°38′S, 60°57′W) and in about the middle of the Trans-Pantanal road (16°25′S, 56°40′W). In each study area, MVK selected one to four abundant plant species and took their photographs for future identification (Appendix S2), which served the basis for DS3. He then haphazardly selected one branch (with about 100 leaves, from 1 to 3 m above ground) from each of two conspecific plant individuals, pointing at the selected branch from 5–10 m distance (i.e., from a distance that did not allow recognition of foliar damage imposed by insects).

Following widely used methodology (e.g., Southwood, Moran & Kennedy, 1982; Fox & Morrow, 1983; Alliende, 1989), each leaf from the selected branches was attributed to one of the following damage classes according to the proportion of the area of leaf lamina that was consumed or damaged by insects: intact leaves, 0.01–1, 1.01–5, 5.01–25, 25.01–50, 50.01–75 and 75.01–100 percent. This attribution was conducted four times for each leaf: separately for defoliators, miners and gallers, and for all these guilds combined. The numbers of leaves in each damage class were recorded separately for each feeding guild and for the total damage. From each individual plant, we calculated from these data (a) the proportions of leaves damaged by defoliators, miners, gallers and by all herbivores together (as the ratio between the number of damaged leaves and the total number of surveyed leaves, multiplied by 100%) and (b) the proportions of leaf area lost to (or damaged by) insects from each of these feeding guilds. The latter value was calculated as follows: the number of leaves in each damage class was multiplied by the respective median value of the damaged leaf area (i.e., 0 for intact leaves, 0.5% for the damage class 0.01–1%, 3% for the damage class 1.01–5%, etc.); the obtained values were summed across all damage classes and divided by the total number of leaves (including undamaged ones) within a sample (Alliende, 1989). Data obtained from two plant individuals were averaged prior the further analyses.

Selection and processing of photographs

To be included in DS4, a photograph showing leaves of woody plants had to fit the following criteria:

1. the photo was taken for the purposes other that studying of insect herbivory (i.e., holiday snaps);

2. the photo was taken in natural or semi-natural forest or cerrado habitat;

3. the branches of woody plants seen on the photo were not more than 3 m above the ground;

4. the images were of sufficient quality to allow an accurate estimate of the proportion of leaf area consumed by defoliating insects.

The photographs that formed DS5 were searched in the WWW on 7 November 2014 using keywords “Brazil” and “nature” or “forest” or “birds” or “animals” in both English and Russian. To create a sample of about the same size as DS4 (that contains 29 photographs), we selected the first 30 photographs satisfying the criteria 1, 2 and 4 (coined above) and containing images of at least 10 leaves (or leaflets) that were sharp enough for the analysis of their damage from about 2,500 surveyed photographs. We rejected photographs showing leaves of palms, because palms were absent in DS1–DS4.

In all photographs, we visually examined images of all leaves in which at least a half of the leaf lamina was clearly visible, attributed each of these leaves (or leaf parts) to one of the damage classes, and calculated the proportion of leaves damaged by defoliators and the proportion of leaf area lost to these insects as described above, i.e., exactly in the same way as in field measurements of foliar damage. If only a part of the leaf lamina was visible, then the percent of damage was estimated relative to the area of the visible part of the leaf, not to the area of the entire leaf. The estimation of foliar damage in DS4 was independently conducted by three persons, two of which were not informed of the hypothesis being tested.

To exclude the possibility that DS4 was biased in an opposite direction, i.e., towards lower plant damage due to selection of aesthetically pleasing objects or background for photographing, we compared the total losses of leaf area among photographs showing individual plants (selected for photographing for their beauty), persons (who may be asked to move to another position if the background contains unsightly objects), landscapes and animals (photographed immediately after they were seen), as well as between photographs focused on plants and having plants as a part of the background.

Data analysis

Distributions of both characteristics of herbivory (i.e., of proportions of damaged leaves and consumed leaf area) were greatly skewed and therefore were analysed by non-parametric methods. The reliability of the photograph-based estimates of foliar losses was checked by comparing (a) the results obtained by three different persons who evaluated the same set of photographs (within DS4), and (b) by comparing the measurements taken from plants (DS2) and from photographs of these plants (DS3). These comparisons were performed using signed rank test (SAS UNIVARIATE procedure; SAS Institute, 2009). The analysis of differences in foliar losses between different groups of photographs were evaluated by the Kruskal–Wallis test (SAS NPAR1WAY procedure; SAS Institute, 2009).

The prediction (1) that non-blindly collected data tends to overestimate the level of herbivory was tested by contrasting (with the Kruskal–Wallis test) these data (DS1 combined with DS2) with blindly collected data (DS4 combined with DS5 after the Kruskal–Wallis tests revealed no differences between them). The prediction (2) that the foliar losses calculated from published studies will be higher than those measured by one of the authors of the present study was tested by contrasting (with the Kruskal–Wallis test) the published data (DS1) and the original data (DS2).

The reporting and/or publication bias was searched for by calculating a Kendall τ_B correlation coefficient between sample size and the reported loss to insects (SAS CORR procedure; SAS Institute, 2009). Following Jennions et al. (2013), we presumed that if such a bias acts against small-sample studies that demonstrated unexpectedly low levels of herbivory, then the reported foliar damage should decrease with the increase in sample size.

Analysis of published data

We identified 42 papers (published from 1990 to 2014) fitting our search criteria. These papers (DS1) reported 94 values of the proportion of leaf area consumed or damaged by insects, and 23 values of the proportion of leaves damaged by insects (Appendix S1). Only two of the 42 papers separately reported foliar losses for all feeding guilds of insects (defoliators, miners and gallers) considered in our analysis. The researchers who studied 1–3 species of woody plants reported a two-fold larger consumption of their foliage by insects than did researchers who studied 10 or more plant species (mean ± S.E.: 8.30 ± 1.03% based on 34 studies and 3.84 ± 0.72% based on 7 studies, respectively; χ² = 4.47, df = 1, P = 0.03).

The proportion of leaf area lost to all feeding guilds did not differ from the proportion of leaf area lost to externally feeding defoliators alone (Kruskal–Wallis test: χ² = 0.07, df = 1, P = 0.80), justifying the combination of these data for the search of publication bias and demonstrating that miners and gallers usually contribute only a minor fraction to the total plant damage by herbivorous insects.

We did not find evidence for the existence of publication and/or reporting bias for the total values of foliar losses, combined with losses to defoliators: these values were independent of sample size (number of study trees: τ_B = 0.07, n = 83, P = 0.38; number of measured leaves: τ_B = 0.01, n = 66, P = 0.99). The shortage of published data did not allow performing a similar analysis for miners and gallers.

Validation of methodology

For the ten plant species we studied, the foliar losses to defoliators estimated from non-blindly taken photographs (DS3) did not differ from the results of the field measurements (DS2; signed rank test, proportion of damaged leaves: S = 9.5, df = 8, P = 0.30; proportion of consumed leaf area: S = 11.5, df = 8, P = 0.20), thus confirming that the analysis of photographs provides reliable estimates of folivory.

The measurements made by three different persons from the same set of 29 original photographs of blindly selected plants (DS4; Appendix S3) yielded similar values for both the proportion of damaged leaves and the proportion of leaf area lost to defoliators (signed rank tests, all P > 0.20); therefore, we averaged the results of three measurements for each photograph prior to further analysis.

Absence of differences in losses of leaf area among photographs taken from individual plants, persons, landscapes and animals (Kruskal–Wallis test; χ² = 0.76, df = 3, P = 0.86), as well as between photographs focused on plants and showing plants as a part of background (χ² = 0.05, df = 1, P = 0.82) suggests that the level of foliar damage did not influence selection of the objects for photographing, i.e., DS4 can be considered as an unbiased sample in relation to foliar losses of woody plants to insects.

Comparison between data sets

The results of our non-blind field measurements (DS2) did not differ from published data (DS1) in terms of both total losses of woody plant foliage to insects and losses to externally feeding defoliators (Figs. 1A and 1C). However, we found significantly (15 to 1,600 times) lower damage by miners and gallers than the damage reported in published studies (Figs. 1B and 1D).

We found no differences between our original photographs (DS4) and photographs found in the WWW (DS5) in either proportion of damaged leaves (Fig. 1A; χ² = 0.05, df = 1, P = 0.82) or losses of leaf area (Fig. 1C; χ² = 2.75, df = 1, P = 0.10). Therefore the blindly collected DS4 and DS5 were combined for the further analysis and contrasted to non-blindly collected DS1 and DS2. The blindly collected data on losses of leaf area of woody plants to externally feeding defoliators (measured from photographs) were, on average, one-fifth of the losses obtained by means of non-blind field measurements (Fig. 1C; χ² = 25.5, df = 1, P < 0.0001), and blindly collected data on the proportion of damaged leaves were, on average, one-third of values obtained by means of non-blind field measurements (Fig. 1A; χ² = 10.5, df = 1, P = 0.0012).