Figures
Abstract
Background
Neonatal trials remain difficult to conduct for several reasons: in particular the need for study sites to have an existing infrastructure in place, with trained investigators and validated quality procedures to ensure good clinical, laboratory practices and a respect for high ethical standards. The objective of this work was to identify the major criteria considered necessary for selecting neonatal intensive care units that are able to perform drug evaluations competently.
Methodology and Main Findings
This Delphi process was conducted with an international multidisciplinary panel of 25 experts from 13 countries, selected to be part of two committees (a scientific committee and an expert committee), in order to validate criteria required to perform drug evaluation in neonates. Eighty six items were initially selected and classified under 7 headings: “NICUs description - Level of care” (21), “Ability to perform drug trials: NICU organization and processes (15), “Research Experience” (12), “Scientific competencies and area of expertise” (8), “Quality Management” (16), “Training and educational capacity” (8) and “Public involvement” (6). Sixty-one items were retained and headings were rearranged after the first round, 34 were selected after the second round. A third round was required to validate 13 additional items. The final set includes 47 items divided under 5 headings.
Conclusion
A set of 47 relevant criteria will help to NICUs that want to implement, conduct or participate in drug trials within a neonatal network identify important issues to be aware of.
Summary Points
1) Neonatal trials remain difficult to conduct for several reasons: in particular the need for study sites to have an existing infrastructure in place, with trained investigators and validated quality procedures to ensure good clinical, laboratory practices and a respect for high ethical standards. 2) The present Delphi study was conducted with an international multidisciplinary panel of 25 experts from 13 countries and aims to identify the major criteria considered necessary for selecting neonatal intensive care units (NICUs) that are able to perform drug evaluations competently. 3) Of the 86 items initially selected and classified under 7 headings - “NICUs description - Level of care” (21), “Ability to perform drug trials: NICU organization and processes (15), “Research Experience” (12), “Scientific competencies and area of expertise” (8), “Quality Management” (16), “Training and educational capacity” (8) and “Public involvement” (6) - 47 items were selected following a three rounds Delphi process. 4) The present consensus will help NICUs to implement, conduct or participate in drug trials within a neonatal network.
Citation: Legrand F, Boulkedid R, Elie V, Leroux S, Valls E, Valls-i-Soler A, et al. (2014) A Delphi Process to Optimize Quality and Performance of Drug Evaluation in Neonates. PLoS ONE 9(9): e104976. https://doi.org/10.1371/journal.pone.0104976
Editor: Fausto Baldanti, Fondazione IRCCS Policlinico San Matteo, Italy
Received: March 18, 2014; Accepted: May 16, 2014; Published: September 11, 2014
Copyright: © 2014 Legrand et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: The research leading to these results has received funding from the European Union Seventh Framework Programme FP7/2007-2013 under grant agreement no. 261060 (Global Research in Paediatrics Network - GRiP). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
In the neonatal population, more than 90% of products are used unauthorised or off-label, especially when neonates are treated in Neonatal Intensive Care Units (NICUs). New treatments are often introduced in the neonatal therapeutic arena without specific evaluation, on the basis that they have proven efficacy in adults and older children. Altogether, many drugs, even when authorised for use in neonates, would benefit from further validated data and a consensus among neonatologists to ensure the most optimal use. In order to improve this situation, a few initiatives were undertaken starting in the USA with the Food and Drug Administration Modernization Act and Best Pharmaceuticals for Children Act respectively in 1997 and 2002, and the American Paediatric Research Equity Act and Newborn Drug Initiative respectively in 2003 and 2006. Similarly in Europe, the European Paediatric Regulation entered in force in June 2007 to increase the development of medicines for all paediatric age groups, including neonates.
There are many major and already well-known practical and ethical issues in conducting drug evaluation in neonates [1], [2]. Although neonates represent only a small part of the population, they have specific diseases and high variations in disease presentations with a major risk of unfavorable long-term outcome. Evaluation of drug pharmacokinetics, efficacy and safety are required in the different neonatal age groups from 24 to 44 weeks' gestational age, characterised by differences in physiological and pharmacological maturation affecting drug disposition and effects [3], [4]. Suitable approaches, adapted to neonates are recommended but need to be more widely used: population pharmacokinetic and bridging studies [5], [6], adapted designs and other methodologies recognised as pertinent to evaluate efficacy when randomized controlled trials are not possible [7]–[9].
Many of these issues can be solved by bringing together scattered expertise within a neonatal network dedicated to drug evaluation in neonates, in collaboration with the European Society of Developmental, Perinatal and Paediatric Pharmacology (ESDPPP). According to the request of the European Commission and following the EnprEMA network initiative (European Network of Paediatric Research at the European Medicines Agency), we conducted a Delphi process [10]–[14] to identify the criteria that would help neonatologists to organise a NICU research infrastructure in order to conduct drug evaluation trials and be part of a European network for drug evaluation in neonates.
Methods
Ethics Statement
The study did not need institutional review board approval as it did not affect patient care, and the information that it generated was used for consensual quality criteria only.
The objectives of the Delphi study were presented to all participants, their agreement and availability to participate were obtained (their written consent consisted of replying positively by email to the invitation sent by the organisers), and their independency verified.
Organization of the Delphi Study
The objective of the Delphi study was to obtain a consensus on the prerequisites that NICUs need to fulfil in order to perform clinical drug trials. Two experts groups conducted the Delphi study by email between February 2012 and February 2013.
Composition of the Scientific and Expert Committees
Two expert groups conducted the Delphi study: the Scientific Committee consisted of 10 experts responsible for organising the Delphi device, selecting and/or approving the members of the Expert Committee, drafting the successive versions of the questionnaire, and analysing the data after each round (questions and answers, results). All members of the Expert Committee (the panellists) agreed to answer the questionnaire during each round and to criticize the answers if necessary.
All the participants (Table 1) were initially selected by the organisers (FL & EJA) in order to ensure that they will represent all potential differences in background, occupational environment, clinical approaches or practices. They were contacted because of their recognised expertise in the different areas of knowledge required for drug evaluation in neonates (neonatology, pharmacology, clinical research), after evaluation of teaching functions, publications, participations to international scientific societies, scientific and ethic boards or networks). The composition of the Scientific and Expert committees included clinical investigators (paediatricians or neonatologists), pharmacologists or pharmacists, researchers, regulators, and employed by industry, academia or regulatory agencies and in most cases members of scientific societies and clinical networks. They were from various countries/continents, had a broad range of ages and different levels of expertise. All the members of the Scientific committee agreed on the composition of the groups, and accepted to participate. The expert selection was then submitted for approval to the Scientific Committee, and finally two additional members were added upon their suggestions.
Questionnaire preparation
The Scientific Committee drafted the first version of the questionnaire, composed of questions and simple items. Each member proposed recognition criteria and a total of 105 items were listed. They were then invited to confirm and/or refine all the items to allow them to be rated during the Delphi process. The resulting questionnaire was submitted a third time by the organisers, for validation by each member of the Scientific Committee. Finally, the elaborated questionnaire included 86 items divided over 7 headings: (Table 2, column A): H1: “NICUs description - Level of care” (21 items), H2: “Ability to perform drug trials: NICU organization and processes” (15 items), H3: “Research experience of the NICU” (12 items), H4: “Scientific competencies and capacity to provide expert advice” (8 items), H5: “Quality management” (16 items), H6: “Training and educational capacity” (8 items), and H7: “Public involvement” (6 items).
Rounds
As required in a Delphi study, each item is required to be assessed twice, using a 1st and a 2nd level of consensus to be finally selected.
First round
The first round was performed from February to July 2012. The panellists (Expert Committee) received the first questionnaire by electronic mail and they were invited to rate their agreement about each item on a 9-point scale, where 1 meant definitely not agree and 9 definitely agree. They were also invited to comment on each item using a dedicated “comment box”, and/or to add items considered as important. Items were included in the second round if a consensus was reached based on two selection criteria: a median score in the top tertile (7–9) and at least 65% of panel ratings in the top tertile (1st level of consensus). At the end of the first round, the questionnaire was slightly modified by the scientific committee and a few items were added to take into account comments and suggestions of the panellists.
Second round/Third round
The second round was conducted between November and December 2012. All panellists who had participated in the first round were sent the second-round questionnaire by email, with the results of the first round including median panel rating, frequency distribution rating as well as their individual ratings from the first round. They were asked to re-score each item based on their own opinion, and the panel responses obtained during the first round. The second round also included a limited number of “new” items, not evaluated in the first round and that had to send out from a second evaluation during what has been called a “third round” in order to evaluate each item twice. This third round took place between January and February 2013. To be included in the final list, the items were selected by the level of median rating in the top tertile (7–9) and a 75% agreement among panellists that the rating was in the top tertile (7–9) (2nd level of consensus).
Results
The characteristics of the two experts groups (n = 25) are presented Table 1 and they are all listed in alphabetic order in the “Acknowledgment” section. They were selected because they had at least 10 years' experience and a well-known international recognition in their field of expertise.
The members of the Scientific Committee were selected and contacted by the organisers, and all of them agreed (10/10, 100%) to participate.
Of the twenty panellists appointed by the Scientific Committee to participate to the Delphi rounds, 15 (79%) accepted: 9 were neonatologists, 2 were pharmacologists, 2 were institutional and industrial project managers and 1 was a neonatologist/scientific officer in a regulatory authority. All of them participated in the three rounds of the process.
First round
The 7 headings comprising a total of 86 initial items tested during the first round are listed in Table 2, column A (Figure 1).
Fifty-seven items reached the first level of consensus (57/86, 66%) (Table 2, column A), while twenty-nine items were discarded (29/86, 34%): 16 (19%) because of a median rating lower than 7, and 13 (15%) because less than 65% of panellists gave rating in the top tertile [7]–[9].
Adjustments at the end of the first round.
Adjustments were made prior to the second round, taking into account the suggestions and comments of the Expert Committee and the Scientific Committee review (Table 2, column B): rearrangements of headings and redefinition of a few items were made, such as H3: “Research Experience” and H6: “Training and educational capacity” that were re-organized and included under the first heading H1: “NICUs Description”; a few items were better defined (H1: i6; and i40; H2: i33 to i35; H3: I49; H4: I65, i66 and i72), while others were compiled into new items (H1: i6′and i75′; H2: i26′; H4: i60′; and H5: i82′); and a new “Access to electronic record” was added (H2: i36′).
After these adjustments, the questionnaire included 61 items under 5 headings (Table 2, column B) as: from 57 items selected in this first round, 53 items remained; similarly, among the 29 discarded items in the previous round, 7 items were maintained to be re-assessed in the second round; and one additional item was added.
Second round
At this step of the process, 61 items were therefore evaluated, including modified, retained or additional items (27 items), which were considered as “new items” and were rated for the first time through this second round. Results are presented Table 2.
47 items were selected (47/61, 77%). Among them, 34 items were definitively retained based on the second level of consensus, and 13 items required to be re-assessed (second assessment) through a third round (Figure 1). The remaining 14 items were, for their part, definitively discarded.
Third round
The previous 13 items that were re-assessed reached the second level of consensus and were definitively selected.
Discussion
Drug trials should always be conducted in sites that can guaranty quality, performance and high ethical standards and this is obviously of even greater importance in neonatal drug evaluations. In this context, the present Delphi study was conducted to define criteria that neonatologists should consider to optimise organisation of the NICU, trial management and conduct. Starting from 86 items divided in 7 headings, a consensus among 15 multidisciplinary experts with a wide range of experience in neonatology, experimental or regulatory practices identified 47 items in 5 re-organised headings (Figure 1). These criteria should help to set up a network of NICUs specialised in drug evaluation in neonates.
As previously stated, drug research in neonates is difficult and, consequently, the number of neonatal drug trials is limited and sometimes of poor quality [7]. Therefore, collaborative, multicenter and multinational studies are essential to recruit neonates with similar diseases from various regions or countries in order to obtain a sample size of sufficient magnitude and to conduct scientific sound studies. In addition to increasing recruitment capacities, such specialised centers will “combine and share” competences in order to build a network to guaranty trial quality and performances, but also to develop investigator and nurse training. This approach is in agreement with the request of the European Commission and is currently supported by the European Medical Agency that developed the EnprEMA network, “a network of networks” [15].
Most of the paediatric networks that we are aware of are so called “Paediatric improvement networks”, collaborating to reduce the gaps and disparities in health care quality and improve outcomes by accelerating the translation of evidence into practice [16]–[18]. They are mostly organised in paediatric subspecialties such as the Children's Oncology Group (COG) in the USA, the Paediatric Rheumatology INternational Trials Organization (Pinto) and the Paediatric European Network for Treatment of AIDS (Penta) in Europe [15]. Some of them are dedicated to perinatal or neonatal care such as the California Perinatal Quality Care Collaborative (CPQCC) [16], [19] but many other national or regional initiatives do exist [17]. All of them underline the importance of a close link with research although limited networks of NICUs were set up for research purposes. In the USA, the Paediatric Pharmacology Research Units network (PPRU) was a cooperation of clinical centers participating in the cooperative agreement with NICHD and represented academic institutions with experience in multi-center clinical research. They agreed to abide by the study protocols and have comparable staff, facilities, and equipment.
In Europe, the European Neonatal Network (EuroNeoNet) primarily aims to give European neonatologists a tool to perform their own quality assurance and benchmarking. Additional neonatal networks do exist at the national level in most countries, including the German Neonatal Network (GNN), the National Institute for Health Research Medecines for clinical research network (NIHR) in the UK, and the Paediatric Clinical Investigation Centers in France (CIC). However, the criteria that might facilitate initial adhesion and follow-up of the NICUs that are members of a neonatal network are not easily identified. The present Delphi study was conducted in order to combine opinions into group consensus on this topic and is reported here according to published recommendations [20].
The first heading “NICUs description” includes subdivisions focussing on “Level of care, Research experience and Training - Education capacities” with description of the size, organization, and teaching hospital status. All the items related to “highly specialised care technics” (such ECMO, or cooling) were put together as the centers providing them are highly specialised and always identified at the regional or national level. Therefore, the corresponding NICUs will be identified within a network and contacted at the initial step of study feasibility if such technics are required by the protocol.
The term “feasibility” is currently used but may cover different issues: for industry, the question behind feasibility is: do the patients corresponding to the inclusion criteria really exist and in addition, where do we find them? Indeed, when industry-driven, the protocol is developed according to regulatory guidelines for drug evaluation in neonates [3] and follows a Paediatric Investigation Plan that is binding. There are concerns that these may not always feasible in the neonatal population however, as per current experience with TINN (Treat Infections iN Neonates) projects (TINN1 and TINN2) - very few indications of the drug limiting inclusions for a RCT; registry considered as non-informative. In contrast, for clinicians, feasibility primarily means: is this acceptable for the patient and his parents? Do we have time and staff for this? Indeed, analysis of feasibility by a local Scientific Review Board with the medical and nurse staff is essential to evaluate recruitment capacities within predefined calendars and in our experience, the best response may sometimes be NO [21], [22].
Research activities will be reported annually in centers already conducting drug trials, underlining the major importance to be given to performances evaluated in terms of adherence, number of patients included, queries. Continuous re-evaluations of quality and performances are now currently performed in neonatal intensive care and with significant results in terms of long-term quality-adjusted survival [23]. They are also required to evaluate research activities, ensure that all organisational and training efforts are maintained, allowing to reach positive results in terms of recruitment, adherence to timelines, quality and Ethics. Although discussed, additional criteria such as success in grant applications and/or publications were thought to reflect more networking activities than individual NICU activities and performances.
Three headings “NICUs organisation, Scientific competencies, Quality management” include all the items required to guaranty quality and performances of clinical research. Among key items, a clear identification of the research staff, including senior doctors and nurses are key factors to insure quality. The role of nurses in improving neonatal care and outcome has been demonstrated [24]. Similarly, their role in clinical research should be better acknowledged and recognised. As neonates hospitalised in intensive care are the most nurse-intensive patients, conducting clinical drug research requires additional dedicated research nurses. In such context, the required neonate - research/nurse ratio in a given NICU obviously depends upon research workload related to the trials and should also include time for training care nurses, availability for parents' information and all additional tasks related to quality and ethics [25].
Although research in neonates is almost always preceded by research in adults and children, diseases in neonates are different in terms of clinical presentation, evolution and risks. It is therefore essential for researchers and pharmaceutical industry people to get advice from experts in the NICU and its surrounding [26], as involvement in pharmacokinetic and pharmacodynamics drug evaluation, trial designs adapted to low numbers, is expected. The quality management is based on Standard Operating Procedures, i.e documents with detailed instructions, written to describe steps to follow in all activities under defined conditions. They are derived from knowledge of Good Clinical and Laboratory Practices from International Conference on Harmonisation (ICH guideline Q11); and Good Laboratory Practice (GLP) from the Organisation for Economic Cooperation and Development (OECD). Such knowledge is required for all health professionals participating in a clinical trial. In particular, the roles, obligations and responsibilities of the sponsor and all health professionals, respect of ethical standards focussing on parental information and consent are clearly defined. The major “Quality SOPs” available in any research center and that have been used and adapted to the TINN2 multicenter European drug trial in neonates are listed Table 4.
Among them, “Trial Pharmacy Procedures” including procedures related to “preparation and administration of the IMPs” are essential as dilutions, sometimes multiple, of a concentrated product, low volumes of infusion, potential physico-chemical interactions are key issues [27], [28]. In addition, in this heading, adherence to trial dependent - standard operating procedures should be evaluated by monitors on a regular basis, and documentation and report of all adverse drug reactions.
The last heading is related to “Public involvement”. The philosophy of “Family-centered care” referring to a partnership between parents and the medical staff [29] focuses on involving them in all ethical and medical decisions related to their newborn's care [30]. Similarly, decisions regarding neonatal research enrollment need to be made conjointly with parents and health care professionals and many studies are now available to improve the consent process [31]–[33]. Researchers are also concerned by involving parents in an earlier phase, in order to help drafting the information and consent documents and make them more accessible to parents, as reflected in the selected items kept in the heading. One item “Involvement of parents and their representatives or organisations in trial design” was not selected, probably because “Trial design” refers to specific methodologies to optimize drug trials taking into account neonatal specificities. In contrast, trials dealing with safety evaluation or long-term follow up are to be discussed extensively with parents in order to evaluate their view on how the child and the whole family will be involve, on how to face open questions on outcome and potential sequelae or neurodevelopmental disorders.
In conclusion, this is to our knowledge, the first consensus obtained by experts and aiming to list the items that should be considered to organize a neonatal network of specialised intensive care units dedicated to drug evaluation. Some of these items may not be required in all units but should be identified to optimize trial design, conduct highly specialised evaluations and train health professionals to trial conduct for the benefit of neonates and their parents.
Acknowledgments
This work is part of the GRIP project (Global Research in Paediatrics) which aims to implement an infrastructure matrix to stimulate and facilitate the development and safe use of medicine in children. GRiP objectives are to create consensus on international standards, methodologies and tools for paediatric research.
This study is dedicated to Adolfo Valls-i-Soler who sadly passed away prematurely on December 2013.
We are grateful for the contribution of the expert and scientific panel members.
Expert Committee
Amparo Alemany (TFS - Trial Form Support, Madrid, Spain), Yannick Aujard (Hôpital Robert Debré, Paris, France), Ralph Bax (EMA, London, UK), Andrew Davidson (Royal Children's Hospital, Victoria, Australia), Laura Fabbri (Chiesi Farmaceutici, Parma, Italy), Henry Halliday (Queen's University, Belfast, Northern Ireland), Egbert Herting (Klinik für Kinder und Jugendmedizin, Lübeck, Germany), Shinya Hirano (Osaka Medical Center and Research Institute for Maternal and child Health, Osaka, Japan), Christine Kubiak (ECRIN, Paris, France), Florian Lagler (Salzburg University Childrens Hospital, Salzburg, Austria), Jean-Paul Langhendries (CHC - NICU Site St Vincent, Liege-Rocourt, Belgium), Emmanuel Lopez (NICU Port Royal, Cochin Hospital, Paris, France), Michael Rieder (University of Western Ontario, London, Canada), Bo Sun (Children's Hospital of Fudan University, Shanghai, China), Anton van Kaam (Emma Children's Hospital, Academic Medical Center, Amsterdam, Netherlands).
Scientific Committee
Noel Cranswick (Royal Children's Hospital, Melbourne, Australia), Elin Haf Davies (EMA, London, UK), Wolfgang Göpel (Klinik für Kinder- und Jugendmedizin, Lübeck, Germany), Carlo Giaquinto & Ursula Trafojer (Paediatric European Network Treatment AIDS (PENTA), Padova, Italy), Evelyne Jacqz-Aigrain (Hôpital Robert Debré, INSERM, Paris, France), Gregory Kearns (Children's Mercy, Kansas city, Missouri (USA), Thierry Lacaze-Masmonteil (Children Hospital of Eastern Ontario, Ottawa, Canada), Mark Turner (Liverpool Women's NHS Foundation Trust, Liverpool, UK), Adolf Valls-i-Soler & Elizabeth Valls (Cruces University Hospital, Bilbao, Spain), John Van Den Anker (Children's National Medical Center, Washington, USA & Sophia Children's Hospital, Rotterdam, Netherlands).
Author Contributions
Conceived and designed the experiments: FL EV AVS JVA EJA. Performed the experiments: FL RB SL VE EJA. Analyzed the data: FL EV JVA AVS EJA. Contributed reagents/materials/analysis tools: FL RB EJA. Wrote the paper: FL EJA.
References
- 1. Jacqz-Aigrain E, Kaguelidou F, van den Anker JN (2012) How to optimize the evaluation and use of antibiotics in neonates. Pediatr Clin North Am 59: 1117–1128
- 2. Ellsbury DL, Ursprung R (2012) A quality improvement approach to optimizing medication use in the neonatal intensive care unit. Clin Perinatol 39: 1–10
- 3.
CHMP & PDCO - “Guideline on the investigation of medicinal products in the term and preterm neonate” (2009) Guideline on the investigation of medicinal products in the term and preterm neonate. Available: http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500003750.pdf.
- 4. Kearns GL, Abdel-Rahman SM, Alander SW, Blowey DL, Leeder JS, et al. (2003) Developmental pharmacology–drug disposition, action, and therapy in infants and children. N Engl J Med 349: 1157–1167
- 5. Tod M, Jullien V, Pons G (2008) Facilitation of drug evaluation in children by population methods and modelling. Clin Pharmacokinet 47: 231–243
- 6. Cella M, Zhao W, Jacqz-Aigrain E, Burger D, Danhof M, et al. (2011) Paediatric drug development: are population models predictive of pharmacokinetics across paediatric populations? Br J Clin Pharmacol 72: 454–464
- 7. Kaguelidou F, Amiel P, Blachier A, Iliescu C, Rozé J-C, et al. (2013) Recruitment in pediatric clinical research was influenced by study characteristics and pediatricians' perceptions: a multicenter survey. J Clin Epidemiol 66: 1151–1157
- 8. Whitehead J, Stratton I (1983) Group sequential clinical trials with triangular continuation regions. Biometrics 39: 227–236.
- 9. Moran C, Smith PB, Cohen-Wolkowiez M, Benjamin DK Jr (2009) Clinical trial design in neonatal pharmacology: effect of center differences, with lessons from the Pediatric Oncology Cooperative Research experience. Clin Pharmacol Ther 86: 589–591
- 10. Jairath N, Weinstein J (1994) The Delphi methodology (Part one): A useful administrative approach. Can J Nurs Adm 7: 29–42.
- 11. Powell C (2003) The Delphi technique: myths and realities. J Adv Nurs 41: 376–382.
- 12. Green B, Jones M, Hughes D, Williams A (1999) Applying the Delphi technique in a study of GPs' information requirements. Health Soc Care Community 7: 198–205.
- 13. Keeney S, Hasson F, McKenna H (2006) Consulting the oracle: ten lessons from using the Delphi technique in nursing research. J Adv Nurs 53: 205–212
- 14. Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C (2011) Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PloS One 6: e20476
- 15. Ruperto N, Eichler I, Herold R, Vassal G, Giaquinto C, et al. (2012) A European Network of Paediatric Research at the European Medicines Agency (Enpr-EMA). Arch Dis Child 97: 185–188
- 16. Horbar JD, Soll RF, Edwards WH (2010) The Vermont Oxford Network: a community of practice. Clin Perinatol 37: 29–47
- 17. Lannon CM, Peterson LE (2013) Pediatric collaborative improvement networks: background and overview. Pediatrics 131 Suppl 4S189–195
- 18. Billett AL, Colletti RB, Mandel KE, Miller M, Muething SE, et al. (2013) Exemplar pediatric collaborative improvement networks: achieving results. Pediatrics 131 Suppl 4S196–203
- 19. Horbar JD (1999) The Vermont Oxford Network: evidence-based quality improvement for neonatology. Pediatrics 103: 350–359.
- 20. Hasson F, Keeney S, McKenna H (2000) Research guidelines for the Delphi survey technique. J Adv Nurs 32: 1008–1015.
- 21.
Campbell MK, Snowdon C, Francis D, Elbourne D, McDonald AM, et al.. (2007) Recruitment to randomised trials: strategies for trial enrollment and participation study. The STEPS study. Health Technol Assess Winch Engl 11: iii, ix–105.
- 22. Shilling V, Williamson PR, Hickey H, Sowden E, Smyth RL, et al. (2011) Processes in recruitment to randomised controlled trials of medicines for children (RECRUIT): a qualitative study. Health Technol Assess Winch Engl 15: 1–116
- 23. Doyle LW (2006) Evaluation of neonatal intensive care for extremely-low-birth-weight infants. Semin Fetal Neonatal Med 11: 139–145
- 24. Lake ET, Staiger D, Horbar J, Cheung R, Kenny MJ, et al. (2012) Association between hospital recognition for nursing excellence and outcomes of very low-birth-weight infants. JAMA J Am Med Assoc 307: 1709–1716
- 25. Franck LS (2005) Research with newborn participants: doing the right research and doing it right. J Perinat Neonatal Nurs 19: 177–186.
- 26. Bush A (2006) Clinical trials research in pediatrics: strategies for effective collaboration between investigator sites and the pharmaceutical industry. Paediatr Drugs 8: 271–277.
- 27. Richey RH, Shah UU, Peak M, Craig JV, Ford JL, et al. (2013) Manipulation of drugs to achieve the required dose is intrinsic to paediatric practice but is not supported by guidelines or evidence. BMC Pediatr 13: 81
- 28. Nahata MC, Allen LV Jr (2008) Extemporaneous drug formulations. Clin Ther 30: 2112–2119
- 29. Griffin T (2006) Family-centered care in the NICU. J Perinat Neonatal Nurs 20: 98–102.
- 30. Ward FR (2010) Parents' views of involvement in concurrent research with their neonates. J Empir Res Hum Res Ethics JERHRE 5: 47–55
- 31. McKechnie L, Gill AB (2006) Consent for neonatal research. Arch Dis Child Fetal Neonatal Ed 91: F374–376
- 32. Ménoni V, Lucas N, Leforestier J-F, Doz F, Chatellier G, et al. (2011) Readability of the Written Study Information in Pediatric Research in France. PLoS ONE 6: e18484
- 33. Laventhal N, Tarini BA, Lantos J (2012) Ethical issues in neonatal and pediatric clinical trials. Pediatr Clin North Am 59: 1205–1220