Key points

Overview

The assessment of outcomes has taken on new importance in many areas of plastic and reconstructive surgery in the last decade, and this point has been discussed in previous articles. Several driving factors such as the public’s demand for more accountability, growing cost of health care, and emphasis on the value as well as impact of health services have culminated in an increased need for outcomes research. To address these broad issues, outcomes research differs from other medical research in that it is more inclusive of what is considered an intervention. Although most medical research examines the effects of a medication or surgical intervention, outcome research may examine the effects of counseling or the way care is delivered (process of care). As a result, outcome research is generally undertaken to address 1 of the following issues: (1) to help health care consumers make market decisions; (2) to assess the quality of medical care; or (3) to improve the knowledge base of medicine.

The most striking shift in outcomes research has been its incorporation into mainstream clinical research, including its incorporation into clinical trials and observational studies. In the past, the key features that distinguished clinical research from outcomes research was the emphasis on efficacy (effect of an intervention measured under the controlled circumstance of a clinical trial) as opposed to effectiveness (effect in the real world). However, with the infiltration of outcomes research into the mainstream clinical research, outcomes research studies now cover a broad range of topics such as quality of care, process of care, access, decision making, prediction rules, and effectiveness. Although most people still equate outcomes research with observational epidemiologic study designs only, outcome studies today and in the future include randomized controlled trials (RCT). Furthermore, outcomes research uses a wide variety of analytical methods such as analysis of administrative databases, decision analysis, propensity scoring as a way of minimizing bias in observational studies, and the use of instrumental variables in access to care research. Also, in addition to the traditional primary outcomes such as disease status end points, outcomes research now assesses a broad range of end points such as health-related quality of life (Hr-QOL), patient satisfaction, and cost. Fig. 1 shows a conceptualization of the relationship between study questions, research design, end points, and applications that define outcomes research that has been adapted from Lee and colleagues.

Conceptual framework for the intersection between clinical research and outcomes research.

( Modified from Lee SJ, Earle CC, Weeks JC. Outcomes research in oncology: history, conceptual framework, and trends in the literature. J Natl Cancer Inst 2000;92(3):195; with permission.)

Overview

The assessment of outcomes has taken on new importance in many areas of plastic and reconstructive surgery in the last decade, and this point has been discussed in previous articles. Several driving factors such as the public’s demand for more accountability, growing cost of health care, and emphasis on the value as well as impact of health services have culminated in an increased need for outcomes research. To address these broad issues, outcomes research differs from other medical research in that it is more inclusive of what is considered an intervention. Although most medical research examines the effects of a medication or surgical intervention, outcome research may examine the effects of counseling or the way care is delivered (process of care). As a result, outcome research is generally undertaken to address 1 of the following issues: (1) to help health care consumers make market decisions; (2) to assess the quality of medical care; or (3) to improve the knowledge base of medicine.

The most striking shift in outcomes research has been its incorporation into mainstream clinical research, including its incorporation into clinical trials and observational studies. In the past, the key features that distinguished clinical research from outcomes research was the emphasis on efficacy (effect of an intervention measured under the controlled circumstance of a clinical trial) as opposed to effectiveness (effect in the real world). However, with the infiltration of outcomes research into the mainstream clinical research, outcomes research studies now cover a broad range of topics such as quality of care, process of care, access, decision making, prediction rules, and effectiveness. Although most people still equate outcomes research with observational epidemiologic study designs only, outcome studies today and in the future include randomized controlled trials (RCT). Furthermore, outcomes research uses a wide variety of analytical methods such as analysis of administrative databases, decision analysis, propensity scoring as a way of minimizing bias in observational studies, and the use of instrumental variables in access to care research. Also, in addition to the traditional primary outcomes such as disease status end points, outcomes research now assesses a broad range of end points such as health-related quality of life (Hr-QOL), patient satisfaction, and cost. Fig. 1 shows a conceptualization of the relationship between study questions, research design, end points, and applications that define outcomes research that has been adapted from Lee and colleagues.

Conceptual framework for the intersection between clinical research and outcomes research.

( Modified from Lee SJ, Earle CC, Weeks JC. Outcomes research in oncology: history, conceptual framework, and trends in the literature. J Natl Cancer Inst 2000;92(3):195; with permission.)

Clinical trials

Outcomes research serves to complement, rather than compete with, clinical trials, because it attempts to better understand how treatment in the real world affects a wide range of outcomes outside the controlled trial setting. Because Hr-QOL, patient satisfaction, and cost are usually considered end points for outcomes research, a classically designed phase III clinical trial would be under the umbrella term of clinical outcomes research when it uses 1 of these as its primary end point (see Fig. 1 ). Thus, outcomes research, regardless of its design methodology, is designed to approximate what health care is intended to achieve: improvements in functional status and Hr-QOL. Thus, the intersection between outcomes research and clinical trials is a natural point of convergence in plastic surgery research in which the goal of treatment is to restore function and improve Hr-QOL.

In a review by McCarthy and colleagues, who examined all level I evidence publications in 5 leading plastic surgery journals from 1978 to 2009, although the number of RCTs and meta-analyses were increasing, they were still limited in number. In addition, most of the studies did not appropriately use blinding, randomization, power analyses, or consider cost. Other investigators also independently found that well-design RCTs are uncommon in the plastic surgery literature, likely because of the numerous challenges associated with designing a surgical trial. First, it is often difficult to motivate surgeons to participate in a randomized surgical trial because most surgeons tend to have a preferred surgical technique and may not be willing to perform the experimental technique. Thus, the lack of clinical equipoise would preclude the recruitment of participating surgeons. Patient accrual may similarly be hindered if there are strong patient preferences or if the surgical intervention in question becomes too widespread. Second, blinding and concealment can be challenging because the surgeon can rarely be blinded to the procedure being performed, and it may also be impossible for the patient to be blinded to the treatment if the result is physically visible. Third, and perhaps the greatest impediment to conducting an RCT, are the required resources (monetary, physical, and human). As a result, RCTs often involve only a small number of academic centers with high volumes of patients and access to full-time dedicated study personnel.

Well-conceived and executed clinical trials should be at the frontier of clinical outcomes research in plastic surgery. To achieve this goal, plastic surgeons, especially those in academic centers, must first resist the powerful temptation to prematurely draw anecdotally based conclusions and become more open to participating in surgical trials. Second, if blinding the surgeon or the patient is not ethical or feasible, then a blinded third-party assessor may be used to evaluate the outcome. Third, collective research efforts need to be fostered internationally to produce results that are adequately powered and can be generalized in the setting of multicenter trials as well as shared network research databases. Fourth, the future ability of plastic surgeons to conduct meaningful, high-quality clinical trials will rely heavily on plastic surgery foundations, academic associations, and research agencies to provide funding, mentoring, and capacity building in an area in which there is a gap in knowledge and expertise. Future improvement in the quality of RCTs in plastic surgery will result in more published studies in prestigious journals that will adhere to the standards for conducting and reporting of RCTs using the Consolidated Standards of Reporting Trials (CONSORT) statement.

Observational studies

Although the RCT study design is often considered to be the epitome of clinical research, the only basic difference between an RCT and a well-conducted prospective observational study is the allocation of patients. In an RCT, the allocation is random; in an observation study, there is always the possibility of selection bias on the part of the provider or patient. In recent years, biostatisticians have promoted propensity scores as a method of reducing bias and improving the quality of results generated from observational studies. Propensity scores provide a creative solution to dealing with the large number of confounding variables that are inherent in observational studies. In concept, a propensity score is found by using the confounding variables as predictors of the group to which a subject belongs, a step that is generally accomplished by using logistic regression. For example, in a cohort study, because the outcome is known for the subjects in the cohort, the confounding variables are used to develop a logistic regression equation to predict whether or not a patient has the outcome. This prediction, based on a combination of the confounding variables, is calculated for all subjects and then used as the confounding variable in subsequent analyses. Proponents of this technique maintain that a propensity score is a superior approach to controlling for confounding variables, compared with multiple regressions or analysis of covariance, especially in analyzing large datasets. Because the area of health services research using epidemiologic or population-based data will continue to flourish in plastic surgery, sophisticated and novel methods of analysis, such as propensity score, will be more widely applied.

Outcome measure development

In recent years, clinical researchers in plastic surgery have increasingly sought to improve understanding of the patient perspective on outcomes. This improvement has been facilitated by the development of a new generation of patient-reported outcome (PRO) instruments that can be used to reliably quantify subjective outcomes such as satisfaction and quality of life. In the past, most of the plastic surgery literature was populated with ad-hoc patient questionnaires that did not adhere to an acceptable level of validity or reliability. As a result, findings from such studies could only be interpreted with great caution. Other studies have relied on the use of generic PRO instruments, such as the Short Form 36. Although such instruments may be valid and reliable, they have low sensitivity to change and thus may fail to adequately measure the impact of plastic surgery procedures and changes in outcomes over time.

Newer generation, plastic surgery–specific PRO instruments such as the BREAST-Q and FACE-Q, represent an important advancement in this area. Developed and validated according to rigorous international standards, such condition-specific PRO instruments can provide clinically meaningful data to guide surgical advancements, patient selection, and clinical care. This emphasis on clinical meaning is a key attribute of new-generation PRO instruments. When developed using strong qualitative methodology in combination with modern psychometric analyses (Rasch), such instruments can offer valid, reliable, and meaningful quantification of the patient experience and perception of outcome.