Advertisement

What Is a Pilot Study?

Published:February 04, 2019DOI:https://doi.org/10.1016/j.jogn.2019.01.005
      JOGNN’s editorial office occasionally receives manuscripts for publication consideration that are labeled in the title or in the abstract as pilot studies. However, on review of the objectives or aims of the study, it is clear that the project was not designed as a pilot study. Sometimes it seems that the project was retrospectively labeled a pilot study when some aspect of the methods was not achieved, particularly the proposed sample size. It is inappropriate to label a study a pilot after the fact because the objectives or aims of a pilot study should be specific to its purpose.
      A pilot study is a small feasibility study designed to test various aspects of the methods planned for a larger, more rigorous, or confirmatory investigation (
      • Arain M.
      • Campbell M.J.
      • Cooper C.L.
      • Lancaster G.A.
      What is a pilot or feasibility study? A review of current practice and editorial policy.
      ). The primary purpose of a pilot study is not to answer specific research questions but to prevent researchers from launching a large-scale study without adequate knowledge of the methods proposed; in essence, a pilot study is conducted to prevent the occurrence of a fatal flaw in a study that is costly in time and money (
      • Polit D.F.
      • Beck C.T.
      Nursing research: Generating and assessing evidence for nursing practice.
      ).
      In general, researchers use pilot studies to evaluate the adequacy of their planned methods and procedures (
      • Polit D.F.
      • Beck C.T.
      Nursing research: Generating and assessing evidence for nursing practice.
      ). This may begin with an evaluation of the likelihood of success of planned participant recruitment and retention strategies. For example, can a sufficient number of participants be recruited from the target population and retained in the study in a reasonable amount of time and at a reasonable cost? Another feasibility purpose may include testing the planned data collection instruments/methods for all types of data for quality and appropriateness. If a qualitative study is planned, does the interview guide or the training of the interviewer(s) produce the type and depth of data needed to answer the study question(s)? If an observational or psychometric instrument is to be used, is it appropriate (validity and reliability) for use in the target population under the proposed circumstances? If a specimen is needed to measure a biological parameter, what is the feasibility of obtaining the specimen, what methods are needed to protect the specimen from deterioration during transport, and what resources are required for its analysis? If health care record data are needed, are the elements of the data accessible in the record, and what are the necessary steps to reliably abstract and record the data? This may include inter-abstractor reliability.
      A well-planned and executed pilot study also may help researchers identify potential confounding variables that were not previously known and evaluate the strength of relationships among key variables to aid in the calculation of sample size (
      • Polit D.F.
      • Beck C.T.
      Nursing research: Generating and assessing evidence for nursing practice.
      ). In addition, researchers may use pilot studies to refine training strategies for research personnel and determine whether preliminary findings support a larger, more rigorous investigation. This latter point is critical in applications for external research funding, particularly applications to large national/international foundations and governmental agencies such as the National Institutes of Health.
      Research to test interventions involves another level of potential objectives to be addressed in a pilot or feasibility study, and
      • Arain M.
      • Campbell M.J.
      • Cooper C.L.
      • Lancaster G.A.
      What is a pilot or feasibility study? A review of current practice and editorial policy.
      suggested that a more formal distinction be made between a pilot study and a feasibility study for intervention research. The researchers categorized a pilot study as a study that is conducted with more rigorous methods to estimate sample size, randomization, and control group selection and a feasibility study as one that is more focused on the methods of intervention implementation or the components of the intervention itself, such as in behavioral or screening programs. A key element of pilot studies of interventions is that they are underpowered to test hypotheses regarding the effectiveness or efficacy of the intervention (
      • Chan C.L.
      • Leyrat C.
      • Eldridge S.M.
      Quality of reporting of pilot and feasibility cluster randomized trials: A systematic review.
      ).
      • Polit D.F.
      • Beck C.T.
      Nursing research: Generating and assessing evidence for nursing practice.
      discussed the use of pilot studies in nursing intervention research to evaluate the acceptability of an intervention to its intended audience, the comprehensiveness and clarity of the intervention protocol, and the effective dosage of the intervention. A pilot study also can be used to estimate the fidelity of implementation and the retention of participants in the intervention. Finally, the safety of the intervention and the occurrence of unanticipated side effects can be assessed.
      Nursing interventions are frequently complex and include several interacting components. Some of the elements that distinguish complex from simple interventions are the number of interacting components, the number and difficulty of required behaviors for the interventionist or the participants, the number of organizational groups or levels involved in the intervention, the number and variability of the intervention’s outcomes, and the degree of tailoring that is permitted in the intervention (
      • Craig P.
      • Dieppe P.
      • Macintyre S.
      • Michie S.
      • Nazareth I.
      • Petticrew M.
      Developing and evaluating complex interventions: The new Medical Research Council guidance.
      ). The United Kingdom’s
      Medical Research Council
      Developing and evaluating complex interventions: New guidance.
      recently released the updated guideline Developing and Evaluating Complex Interventions: New Guidance; nurse researchers may find this guideline useful for the development and evaluation of nursing interventions. Feasibility and pilot testing make up one of four key components of the process.
      • Polit D.F.
      • Beck C.T.
      Nursing research: Generating and assessing evidence for nursing practice.
      identified the features of an ideal nursing intervention as salient, efficacious, safe, conceptually sound, cost effective, feasible, developmentally appropriate, culturally sensitive, accessible, acceptable, adaptable, and readily disseminated. In this issue
      • Walker L.O.
      • Czajkowski S.M.
      Designing interventions to improve the health of women using the ORBIT model.
      discuss the use of feasibility studies during Phase II of intervention development.
      While all research generally leaves investigators with more questions than answers, pilot and feasibility studies are purposed to generate lessons to inform subsequent studies. They are designed to answer specific methods questions rather than research questions per se. In their discussion of the misuse of the pilot study label in clinical and translation research,
      • Moore C.G.
      • Carter R.E.
      • Nietert P.J.
      • Stewart P.W.
      Recommendations for planning pilot studies in clinical and translational research.
      defined pilot studies “as preparatory studies designed to test the performance characteristics and capabilities of study designs, measures, procedures, recruitment criteria, and operational strategies that are under consideration for use in a subsequent, often larger, study” (p. 332). Although this definition is broad, it identifies the purposeful nature of pilot or feasibility studies in contrast to a label often used for unfunded, vague or poorly developed, or underpowered research. Researchers may need to conduct a series of pilot studies to be successful in their applications for research funding. Finally, manuscripts that report well-planned and well-executed pilot studies are often publishable and can inform clinicians and researchers about lessons learned.

      References

        • Arain M.
        • Campbell M.J.
        • Cooper C.L.
        • Lancaster G.A.
        What is a pilot or feasibility study? A review of current practice and editorial policy.
        BMC Medical Research Methodology. 2010; 10: 67https://doi.org/10.1186/1471-2288/10/67
        • Chan C.L.
        • Leyrat C.
        • Eldridge S.M.
        Quality of reporting of pilot and feasibility cluster randomized trials: A systematic review.
        BMJ Open. 2017; 7e016970https://doi.org/10.1136/bmjopen-2017-016970
        • Craig P.
        • Dieppe P.
        • Macintyre S.
        • Michie S.
        • Nazareth I.
        • Petticrew M.
        Developing and evaluating complex interventions: The new Medical Research Council guidance.
        British Medical Journal. 2008; 337: a1655https://doi.org/10.1136/bmj.a1655
        • Medical Research Council
        Developing and evaluating complex interventions: New guidance.
        (Retrieved from)
        • Moore C.G.
        • Carter R.E.
        • Nietert P.J.
        • Stewart P.W.
        Recommendations for planning pilot studies in clinical and translational research.
        Clinical and Translational Science. 2011; 4: 332-337https://doi.org/10.1111/j.1752-8062.2011.00347.x
        • Polit D.F.
        • Beck C.T.
        Nursing research: Generating and assessing evidence for nursing practice.
        10th ed. Wolters Kluwer/Lippincott Williams & Wilkins, Philadelphia, PA2017
        • Walker L.O.
        • Czajkowski S.M.
        Designing interventions to improve the health of women using the ORBIT model.
        Journal of Obstetric, Gynecologic, & Neonatal Nursing. 2019; 48: 197-204https://doi.org/10.1016/j.jogn.2019.02.002

      Linked Article