Key points

Introduction

Craniofacial and pediatric plastic surgery encompass a wide variety of clinical presentations and conditions. Although the scope of practice of these two subspecialties is broad, there are concepts regarding the measurement of outcomes that pose similar challenges for craniofacial and general pediatric plastic surgeons. The management of clinical conditions such as cleft lip and/or palate (CLP) and craniosynostosis often begins in infancy and extends into adulthood. Treatment protocols vary widely and there may be procedures with competing goals within the same protocol. For example, it has been difficult to define a value for early palate repairs for optimal speech while taking into account the detrimental effects on facial growth. Determining the overall quality of care delivered depends on the seemingly straightforward but challenging task of describing clinically meaningful outcomes.

Unlike many conditions in which the primary goal of treatment may be the resection of a tumor or treatment of an infection that may be evaluated more directly, the goals of treating congenital conditions such as CLP or craniosynostosis are to optimize function, aesthetic outcome, and health-related quality of life (HR-QL). The evaluation of outcomes is challenging because there is no established baseline to be restored for a patient with a congenital condition and surrogate outcomes, or measurements that are used as substitutes for a clinically meaningful endpoints, are frequently used to describe final results. In addition, many of the conditions treated affect facial appearance. Having an appearance that “feels normal” has a significant influence on an individual’s interpersonal interactions. Correcting bony and soft tissue facial features toward the norm is a cornerstone of craniofacial surgery; however, evaluating the success of this goal is also an important challenge. The patient perspective is a key component of defining outcomes in congenital conditions. Patient-reported outcomes (PROs) reflect the final endpoint of treatment, but have proven to be the most difficult to evaluate.

Careful consideration of the methods by which outcomes are measured is also increasingly important as the allocation of scarce resources becomes more dependent on the clinician’s ability to prove that a treatment has had a beneficial effect. The management of many congenital conditions can be costly because of the large number of interventions required over a lengthy course of treatment, as in CLP, or because of the expense associated with innovative technology in surgical management, as in craniosynostosis. From a broader perspective, the consideration of cost of care in concert with the associated outcomes is consistent across models of health care delivered around the world. Advocating for the best possible care for patients with congenital conditions in cost-focused environments requires clinicians to demonstrate measurable effects of treatment on patients from the functional, aesthetic, and HR-QL perspectives.

The aim of this article is to describe and justify the need for clinically meaningful and scientifically sound measurement of PROs. Challenges in the measurement of outcomes in craniofacial and pediatric plastic surgery are discussed first, with a focus on CLP and craniosynostosis. The role of meaningful measurements in the development of evidence-based guidelines and the definition of goals of treatment of these two conditions are then addressed, with a specific focus on CLP and the development of the CLEFT-Q, a new PRO instrument.

Introduction

Craniofacial and pediatric plastic surgery encompass a wide variety of clinical presentations and conditions. Although the scope of practice of these two subspecialties is broad, there are concepts regarding the measurement of outcomes that pose similar challenges for craniofacial and general pediatric plastic surgeons. The management of clinical conditions such as cleft lip and/or palate (CLP) and craniosynostosis often begins in infancy and extends into adulthood. Treatment protocols vary widely and there may be procedures with competing goals within the same protocol. For example, it has been difficult to define a value for early palate repairs for optimal speech while taking into account the detrimental effects on facial growth. Determining the overall quality of care delivered depends on the seemingly straightforward but challenging task of describing clinically meaningful outcomes.

Unlike many conditions in which the primary goal of treatment may be the resection of a tumor or treatment of an infection that may be evaluated more directly, the goals of treating congenital conditions such as CLP or craniosynostosis are to optimize function, aesthetic outcome, and health-related quality of life (HR-QL). The evaluation of outcomes is challenging because there is no established baseline to be restored for a patient with a congenital condition and surrogate outcomes, or measurements that are used as substitutes for a clinically meaningful endpoints, are frequently used to describe final results. In addition, many of the conditions treated affect facial appearance. Having an appearance that “feels normal” has a significant influence on an individual’s interpersonal interactions. Correcting bony and soft tissue facial features toward the norm is a cornerstone of craniofacial surgery; however, evaluating the success of this goal is also an important challenge. The patient perspective is a key component of defining outcomes in congenital conditions. Patient-reported outcomes (PROs) reflect the final endpoint of treatment, but have proven to be the most difficult to evaluate.

Careful consideration of the methods by which outcomes are measured is also increasingly important as the allocation of scarce resources becomes more dependent on the clinician’s ability to prove that a treatment has had a beneficial effect. The management of many congenital conditions can be costly because of the large number of interventions required over a lengthy course of treatment, as in CLP, or because of the expense associated with innovative technology in surgical management, as in craniosynostosis. From a broader perspective, the consideration of cost of care in concert with the associated outcomes is consistent across models of health care delivered around the world. Advocating for the best possible care for patients with congenital conditions in cost-focused environments requires clinicians to demonstrate measurable effects of treatment on patients from the functional, aesthetic, and HR-QL perspectives.

The aim of this article is to describe and justify the need for clinically meaningful and scientifically sound measurement of PROs. Challenges in the measurement of outcomes in craniofacial and pediatric plastic surgery are discussed first, with a focus on CLP and craniosynostosis. The role of meaningful measurements in the development of evidence-based guidelines and the definition of goals of treatment of these two conditions are then addressed, with a specific focus on CLP and the development of the CLEFT-Q, a new PRO instrument.

Challenges in measuring outcomes in craniofacial and pediatric plastic surgery

There are unique challenges in measuring outcomes in craniofacial and pediatric plastic surgery patients. The most challenging aspects of surgical decision-making are frequently the difficulty presented by the options offered by different interactions between treatments and the complexity of measuring benefits associated with each. It is common to attribute such difficulties to the following:

• 1.
  

  The results of procedures performed in infancy do not become clear for several years and these results are not often reported in a standardized fashion

  
  
• 2.
  

  One of the primary goals of treatment is to improve aesthetic outcome and the measurement of this concept is challenging

  
  
• 3.
  

  A child represents a moving target with respect to measuring their patient perspective.

Ideally, a measurable outcome is a variable that can be assessed at the time of diagnosis or consultation to guide treatment decisions as well as evaluate surgical outcomes. Unfortunately, there are several domains of outcomes for which this is not possible in the pediatric congenital population. For example, a functional outcome such as speech is not measurable at the time of a primary cleft palate repair because the child has yet to learn to speak. The evaluation of such outcomes then necessarily occurs long after the initial procedure as a posttreatment audit tool alone and the measurement of change is not possible. Although this type of reporting of functional outcomes requires lengthy follow-up, outcomes such as speech, vision, or increased intracranial pressure are still more straightforward to evaluate than aesthetic or HR-QL outcomes; however, measuring a single functional outcome may not provide a complete reflection of the impact of the procedure. The reporting of functional outcomes in the literature is also highly variable due to the range of instruments used, making meta-analysis of study results difficult or impossible. Multicentered studies, such as Eurocleft and Americleft, have begun to introduce more standardized techniques of reporting to address this difficulty. It is unlikely that a single outcome measure could ever be developed that would reflect accurately the results of a complete package of interventions. Therefore, coordinated efforts to adopt more comprehensive reporting techniques have been undertaken to address this challenge in complex conditions such as craniosynostosis.

Defining goals of treatment

As the scientific community engaged in improving cleft and craniofacial care continues to develop bodies of evidence to guide treatment decisions and improve quality of care, the measures adopted to assess PROs become even more important in the evaluation of the whole gamut of interventions delivered in a child’s journey of care through to adulthood. Defining the goals of treatment in some conditions can be straightforward (eg, the restoration of occlusion following a fracture of the mandible); however, it is often more complex (eg, reshaping the cranial vault in craniosynostosis where improving HR-QL through functional and aesthetic changes is the goal). In complex cases, the completely objective isolated measurement of change of a functional or aesthetic variable may not reflect the overall goal of improving HR-QL. There is value to these individual measurements, but focusing on one or two specific outcomes may not provide a complete description of the affect of a surgical intervention on the patient.

The importance of defining clinically meaningful goals and then measuring these goals is highlighted using craniosynostosis as an example. For a single condition to be corrected, a family may be faced with options such as minimally invasive endoscopic approaches, open cranioplasty, distraction osteogenesis, and spring-assisted cranioplasty. The advantages and disadvantages of each technique may manifest in different types of outcomes. Most studies reported to date are case series or retrospective case reviews and there is great value in these reports from experts despite the apparent low level of evidence. The challenge lies in comparing these reports to each other, especially when the benefits of one procedure compared with another may be found in a different outcome, such as decreased operative time or decreased burden of care. In addition, surgical success has often been defined in terms of lower complication or reoperation rates and, although these variables are important surrogate outcomes, the impact of surgery on patients is not well represented. For example, the Whitaker scale provides an indication of the need for further surgery and the extent of subsequent treatment, and it is valuable for these reasons. However, this scale represents a surrogate, subjective measurement made from variable perspectives (the patient, the surgeon, and the family) that may not consistently show the impact of a cranioplasty on the patients themselves. Szpalski and colleagues have outlined a comprehensive set of outcomes commonly used in the management of craniosynostosis that includes a range of functional and aesthetic concerns. The complexity of the condition and its treatment is reflected in the wide variety of outcome categories, which include radiographic studies, otolaryngology, ophthalmology, dentistry and orthodontics, neurologic development, and quality of life assessments. The aesthetic outcome from procedures is often analyzed using craniometric measurements and, although these variables provide important insight into surgical techniques, these measures may not reflect the actual aesthetic impact of the procedure. These investigators also identify the need for the development of comprehensive PRO instruments to complement the array of specific physically determined metrics, to put these outcomes into a practical and useful context for treatment decision-making.

As the surgical techniques used in the management of craniosynostosis evolve, measuring a comprehensive set of outcomes that include the patient perspective should help to provide a more value-based assessment of the advantages and disadvantages of each technique. For example, a recent study showed that children undergoing a longer surgery with increased exposure to inhaled anesthetics had lower neurodevelopmental scores. This finding could have implications on the type of surgery selected for a patient if the association is proven to be causal. However, there may be other variables that explain this outcome, such as the severity of the initial cranial deformity and the impact of that deformity on the brain preoperatively. If the benefit of the aesthetic outcome of these procedures is not found to outweigh the risk of neurodevelopmental delay, surgical decision-making might follow a different path. Establishing the balance between different outcomes is complex. First, proving that the association is causal in this case would require a randomized controlled trial, controlling for other confounding variables that would be identified in a comprehensive assessment of outcomes. Second, comparing the benefit of the aesthetic outcome to the risk of neurodevelopmental delay would require well-defined outcome measures of aesthetic results beyond craniometric measurements alone.