Conceived and designed the study: SL MC. Collected the data: FB AF-L MKG KK TU SB GV PL YC CLC ADA CJS AB LD SG SB. Performed the statistical analysis: SL. Interpreted the data: SL. Drafted the manuscript: SL. Critically revised the manuscript for important intellectual content: SL FB AF-L MKG KK TU SB GV PL YC CLC ADA CJS AB LD SG SB DG GB MC. Brought expertise in Epidemiology and Pediatric Infecious Diseses, respectively: GB DG. Obtained funding: SL.
Professors Gendrel and Chalumeau have received unrestricted educational grants for other studies, for a total amount lower than €5000, from Brahms AG, the manufacturer of procalcitonin, in 2007. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. This does not alter the authors' adherence to all the PLoS ONE policies on sharing data and materials.
Predicting vesico-ureteral reflux (VUR) ≥3 at the time of the first urinary tract infection (UTI) would make it possible to restrict cystography to high-risk children. We previously derived the following clinical decision rule for that purpose: cystography should be performed in cases with ureteral dilation and a serum procalcitonin level ≥0.17 ng/mL, or without ureteral dilatation when the serum procalcitonin level ≥0.63 ng/mL. The rule yielded a 86% sensitivity with a 46% specificity. We aimed to test its reproducibility.
A secondary analysis of prospective series of children with a first UTI. The rule was applied, and predictive ability was calculated.
The study included 413 patients (157 boys, VUR ≥3 in 11%) from eight centers in five countries. The rule offered a 46% specificity (95% CI, 41–52), not different from the one in the derivation study. However, the sensitivity significantly decreased to 64% (95%CI, 50–76), leading to a difference of 20% (95%CI, 17–36). In all, 16 (34%) patients among the 47 with VUR ≥3 were misdiagnosed by the rule. This lack of reproducibility might result primarily from a difference between derivation and validation populations regarding inflammatory parameters (CRP, PCT); the validation set samples may have been collected earlier than for the derivation one.
The rule built to predict VUR ≥3 had a stable specificity (ie. 46%), but a decreased sensitivity (ie. 64%) because of the time variability of PCT measurement. Some refinement may be warranted.
During the past decade, many European and American pediatric societies have recommended that all young children undergo a cystography after a first febrile UTI
We believed, then, that there is scope for an evidence-based strategy, one that offered a moderate alternative to two diametrically opposed policies (cystography for all or no children). We derived a predictive tool for moderate and high-grade VUR (grade ≥3), aiming to avoid
Eight centres were included
Centre |
Urine collection techniques (threshold of the positive bacteriuria) |
n | Male n (%) | Age median (IQR) | All-grade VUR n (%) | Grade ≥3 VUR n (%) | CRP Median (IQR) |
|
|||||||
Alex. | SA (any), UC (104), CVM (105) | 40 | 9 (23) | 10.5 (6.5–12.5) | 12 (30) | 8 (20) | 57.0 (14.5–91.0) |
Athens | SA (103), UC (104), CVM (105) | 52 | 26 (50) | 6.6 (3.0–9.8) | 10 (19) | 0 (0) | 42.4 (6.1–108) |
Barcelona | SA (any), UC (5.104), CVM (105) | 55 | 22 (41) | 6.0 (3.0–9.0) | 13 (24) | 4 (7) | 44.6 (14.1–76.7) |
Elazig | UC (103), CVM (105) | 52 | 25 (48) | 6.0 (6.0–36.0) | 3 (6) | 0 (0) | 12.5 (4–40) |
|
|||||||
Lille | SB (105) | 23 | 7 (29) | 8.5 (4.0–19.0) | 14 (58) | 3 (13) | 52.5 (19–83) |
Padova | SB (105) | 47 | 17 (38) | 6.4 (3.1–11.4) | 9 (19) | 2 (4) | 61.0 (37–120) |
Paris | SB (105) | 52 | 23 (44) | 7.6 (2.8–12.8) | 13 (25) | 5 (10) | 85.0 (53.6–117) |
Toulouse | SB (105), CVM (105) | 91 | 28 (31) | 9.2 (5.3–17.7) | 35 (38) | 17 (19) | 75.5 (33.0–117) |
|
413 | 157 (38) | 8.5 (4.0–16.0) | 109 (26) | 47 (11) | 53.8 (20–98) |
*Classified according to the urine collection technique in non-toilet-trained children.
In colony-forming units/mL.
Abbreviations: Alex for Alexandroupolis; CRP, C-reactive protein; CVM, Clean-voided midstream; IQR, Interquartile range; SA, Suprapubic aspiration; SB, Sterile bag; UC, Urethral catheterization; VUR, Vesicoureteral reflux.
Of the 530 who met the inclusion criteria, 417 were finally included in the analysis (
Abbreviations: PCT, Procalcitonin; VUR, Vesico-ureteral reflux.
When applying the rule to the validation population, 16 (34%) of the 47 patients with VUR ≥3 were misdiagnosed because they presented with either ureteral dilation and PCT <0.17 ng/mL (for one patient – 2%) or without ureteral dilation and PCT <0.63 ng/mL (for the 15–32% - other patients -
The horizontal lines are the dichotomization threshold in each group.
Derivation |
Validation (n = 413, prevalence of VUR ≥3: 11%) | Difference |
|
|
|||
aOR |
5.2 (2.4–11.3) | 1.5 (0.7–3.4) | |
Sensitivity | 86 (74–93) | 64 (50–76) | 22 (5 to 38) |
Specificity | 47 (42–51) | 46 (41–52) | 0 (−7 to 7) |
PPV | 17 (13–22) | 13 (10–18) | 4 (−3 to 10) |
NPV | 96 (93–98) | 91 (86–94) | 5 (1 to 11) |
|
|||
aOR |
6.8 (0.9–50.0) | 1.3 (0.6–3.2) | |
Sensitivity | 86 (76–94) | 62 (47–74) | 26 (9 to 41) |
Specificity | 44 (40–49) | 45 (40–50) | 1 (−6 to 8) |
PPV | 17 (13–21) | 13 (9–18) | 4 (−2 to 10) |
NPV | 97 (93–98) | 90 (85–94) | 6 (1 to 12) |
|
|||
aOR |
4.9 (2.3–10.6) | 1.3 (0.5–3.4) | |
Sensitivity | 86 (74–93) | 60 (45–72) | 28 (11 to 44) |
Specificity | 45 (40–50) | 46 (41–51) | 1 (−6 to 8) |
PPV | 17 (13–21) | 12 (9–17) | 4 (−2 to 10) |
NPV | 96 (93–98) | 90 (85–93) | 7 (2 to 12) |
Values are expressed as values or % (95% CI).
Discriminative values were compared using a χ2 test for unpaired sample.
*Data in the column come from the previously published derivation of the decision rule
**Adjusted OR were calculated with the multi-level logistic regression models.
Differences are rounded to the closer integer.
Abbreviations: NPV, Negative predictive value; OR, Odd ratio; PCT, Procalcitonin; PPV, Positive predictive value.
When applying the rounded rule and the rule based on PCT alone to the validation population, we also found a non-significant relationship between VUR ≥3 and the rule, as well as a significant decrease of sensitivity and negative predictive value (
In the subgroup of children for whom urine specimens were collected using suprapubic aspiration or urethral catheterization for non-toilet trained children, the relationships between VUR ≥3 and the rules were not significant (
Whole population | Subgroup |
Difference | |
|
|||
aOR |
1.5 (0.7–3.4) | 2.0 (0.4–11.6) | |
Sensitivity | 64 (50–76) | 60 (39–78) | 4 (−20 to 28) |
Specificity | 46 (41–52) | 58 (51–65) | 11 (2 to 20) |
PPV | 13 (10–18) | 14 (8–22) | 1 (−1 to 7) |
NPV | 91 (86–94) | 93 (86–96) | 2 (−1 to 8) |
|
|||
aOR |
1.3 (0.6–3.2) | 1.9 (0.3–11.1) | |
Sensitivity | 62 (47–74) | 60 (39–78) | 2 (−20 to 26) |
Specificity | 45 (40–50) | 55 (48–62) | 10 (1 to 19) |
PPV | 13 (9–18) | 13 (8–21) | 0 (−1 to 7) |
NPV | 90 (85–94) | 93 (86–96) | 2 (−5 to 9) |
|
|||
aOR |
1.3 (0.5–3.4) | 1.9 (0.3–11.1) | |
Sensitivity | 60 (45–72) | 60 (39–78) | 0 (−24 to 25) |
Specificity | 46 (41–51) | 56 (49–63) | 10 (1 to 18) |
PPV | 12 (9–17) | 13 (8–22) | −1 (−10 to 7) |
NPV | 90 (85–93) | 93 (86–96) | 3 (−5 to 9) |
Values are expressed as values or % (95% CI).
Discriminative values were compared using a χ2 test for unpaired sample.
*Subgroup of children for who urines were collected using suprapubic aspiration or urethral catheterization.
**Adjusted OR were calculated with the multi-level logistic regression models.
Abbreviations: NPV, Negative predictive value; OR, Odd ratio; PCT, Procalcitonin; PPV, Positive predictive value.
Because we did not find the similar results between derivation and validation populations, we compared their characteristics. There was no significant difference regarding gender, prevalence of all-grade or high-grade VUR, distribution of age (
Variables | Derivation |
Validation (n = 413, prevalence of VUR ≥3: 11%) | P-value |
Use of sterile bags | 238 (48) | 214 (52) | 0.3 |
Male gender | 197 (40) | 157 (38) | 0.6 |
All grade VUR | 126 (26) | 109 (29) | 0.8 |
High-grade VUR | 56 (11) | 47 (11) | 1.0 |
Age (months) | 12.1 (±11.2); 8.0 (4.0–17.0) | 11.9 (±10.7); 8.5 (4.0–16.0) | 0.7 |
CRP (mg/mL) | 94.6 (±74.0); 77.5 (38.0–140.0) | 68.4 (±64.1); 53.8 (20.0–98.0) | <0.0001 |
PCT (ng/mL) | 4.2 (±19.3); 0.9 (0.3–2.8) | 3.5 (±8.6); 0.8 (0.3–3.1) | 0.5 |
in children with VUR <3 | 3.6 (±19.3); 0.7 (0.3–2.4) | 3.4 (±8.8); 0.8 (0.3–2.5) | 1.0 |
in children with VUR ≥3 | 8.3 (±18.5); 2.9 (1.2–6.8) | 4.4 (±6.5); 1.5 (0.3–6.1) | 0.06 |
Ureteral dilation | 25 (5) | 33 (8) | 0.8 |
in children with VUR <3 | 16 (4) | 23 (6) | 0.08 |
in children with VUR ≥3 | 10 (18) | 20 (21) | 0.7 |
Values are expressed as n (%) for binary variables (gender, All grade and high-grade VUR, Ureteral dilation), and as: mean (±Standard deviation); median (inter-quartile range) for continuous variables (age, CPR, PCT).
*Data in the column come from the previously published derivation of the decision rule
**Binary variables were compared using a χ2 test, and continuous variables were compared using the non-parametric Mann-Whitney test.
Abbreviations: CRP, C-reactive protein; PCT, Procalcitonin; U dilation, Ureteral dilation; VUR, Vesico-ureteral reflux.
We report the first attempt to evaluate the reproducibility of the decision rule based on PCT and ureteral dilation proposed by our group
The first issue to be addressed to investigate the decreases of predictive ability of a rule is the difference in the derivation and validation populations, which would explain why a decision rule could not be transferred across those sets of patients
The second issue concerning the validation's difficulties to reproduce derivation results is the limitations of the external validation study. The validation was a secondary analysis of previously published prospective cohort studies, as was the derivation study. Because our group performed a systematic review and meta-analysis on PCT in UTI in children, we gathered the worldwide published data of children with PCT and VUR
The use of sterile bags for urine collection for the non-toilet trained children in half of the centres might have introduced a selection bias, because this technique is less specific than suprapubic aspiration or urethral catheterization
The validation of the decision rule based on PCT and ureteral dilation on renal US confirmed the rule specificity, but showed a loss of its sensitivity, which led to a misdiagnosis of 34% of children with VUR ≥3. The fact that the rule performed better in the derivation population than in the validation one was predictable, according to the Evidence Based Working Group
We conducted a secondary analysis of already published data on children with UTI, PCT and VUR. Because we had conducted a systematic review and meta-analysis on individual patient data on early and late renal scarring and PCT in children with UTI
All patients had a voiding cystography, the gold standard examination for the diagnosis and classification of VUR. Cystographies were read according to the “International System of Radiological Grading of Vesicoureteric Reflux”
Each child's serum PCT was prospectively measured at admission for febrile UTI (i.e. when children arrived at the Emergency Department), with the LUMItest PCT immunoluminometric assay (BRAHMS, Hennigsdorf, Germany). The other potential predictive variables came from the findings of renal US performed at the time of UTI diagnosis by an experienced senior pediatric radiologist blinded to the PCT measurement, and before cystography was performed: ureteral dilation (defined by ureter visibility during renal US). The imaging studies were reviewed blinded to cystography results, and data were extracted from the radiologist's record.
We first described the study population's general characteristics. We then applied the rule to every patient, classifying each one as cystography recommended or not. We also applied the rounded rule (i.e. a cystography should performed in case of ureteral dilation and PCT ≥0.2 ng/mL, or if PCT ≥0.6 ng/mL in absence of ureteral dilation), and the rule based on PCT alone (a cystography should be prescribed if PCT ≥0.6 ng/mL) to each child. We evaluated the relationship between VUR ≥3 and the decision rules with an adjusted OR using a multi-level logistic regression model (where centers were considered as the group level variable). We then calculated for the clinical decision rule, the rounded rule, and the rule based on PCT alone, the sensitivity, specificity, positive and negative predictive values. We ran the same calculation in the subgroup of children for whom urine specimens were collected using “recommended” techniques (i.e. suprapubic aspiration or urethral catheterization for non-toilet trained children and clean-voided midstream for toilet trained children). Discriminative values were compared with those of the derivation population
The authors thank Dr Deftereos (Department of Radiology, Democritus University of Thrace and University General Hospital of Alexandroupolis Thrace, Greece) Dr Bacchetta (Department of Pediatric Nephrology - Reference Centre for Rare Renal Diseases, Femme Mère Enfant Hospital, University of Lyon, Lyon, France) and Dr Tamara Giles-Vernicke for the Epidemiology Unit of Emerging Diseases in Institut Pasteur, Paris, for helpful discussions.