Evaluation and Design of Cosmetic Research Studies

Amit G. Pandya

In making decisions regarding the diagnosis and treatment of skin diseases, clinicians are faced with a confusing array of data from various sources to help them make correct choices. A variety of questions may come to mind during this process. What procedures and tests need to be performed to make the right diagnosis? Is a biopsy necessary before embarking on a treatment course? Is there good evidence that preoperative labs and medications are useful for a particular procedure? What is the evidence that a particular treatment or cosmetic procedure is effective? What is my own experience in this situation? Can I rely on the data that is being presented? Am I giving the correct choices and advice to my patient?

As always, the first rule of professionalism is paramount: The needs of the patient supersede the needs of the professional. We want to do what is best for the patient, but how do we evaluate the large volumes of data that we are given at meetings, in journals, by colleagues, and by the lay press and advertisements? Fortunately, there is a substantial amount of material available on the use of evidence-based medicine for clinical practice.

Evidence-based medicine is the use of the best current evidence in making decisions about the care of individual patients.¹ The strength of the evidence is based on a hierarchy of evidence, which consists of, in descending order, results of systematic reviews of well-designed studies, results of one or more well-designed studies, results of large case series, expert opinion, and personal experience. There is a large body of evidence available for the care of patients with skin disease. However, there is a dearth of evidence-based data on aesthetics and cosmetic surgical procedures in darker racial ethnic groups. It is important to be familiar with the evidence that does exist for a diagnostic or therapeutic intervention that one might be considering. The clinician must be equipped to critically review such data to do what is best for the patient. By reviewing this information and remembering some guidelines, one can evaluate data critically and determine the correct diagnostic and treatment options for any clinical situation. Use of MEDLINE and other databases can be useful in locating the best evidence for a particular clinical circumstance.

In general, well-designed, randomized controlled trials give the best evidence for the efficacy of a particular therapeutic intervention. Trials of agents and procedures for cosmetic indications have often been performed without controls, randomization, or blinding. Bias can occur in poorly designed trials, which can mislead clinicians into making the wrong decisions and cause unnecessary treatment for patients.² Recently, the CONSORT (Consolidated Standards of Reporting Trials) recommendations on items to include when reporting a randomized clinical trial have been published (Table 5-1).³ These guidelines are helpful in evaluating any study to determine its validity.

The CONSORT guidelines were developed in 1996 to improve the suboptimal reporting of randomized control trials. The goal of the CONSORT guidelines is to assist health-care providers in making informed decisions about the validity of the studies on which they base their clinical practice. Eleven key methodological factors important in reporting a randomized control trial were initially identified. Several journals accepted these recommendations, with the exception of the New England Journal of Medicine. The second edition of CONSORT established a 22-item checklist, which emphasized the need for the word “randomized” to appear in the title to allow literature searches to identify the paper.³ The CONSORT statement is an important research tool that takes an evidence-based approach to improve the quality of reports of randomized trials and is available in several languages. One can assume a certain degree of validity when reading trials that have been conducted using these guidelines.

Although the randomized controlled trial provides medicine with its main source of evidence for supporting the use of a particular therapy or medical practice, there may be problems in the conduct and reporting of such trials.⁴ Many clinical situations, such as cardiac arrest and

pain relief, do not lend themselves to randomization. In addition, trials seldom study the effects seen in different subgroups of patients, nor can the results from a small group of study patients always be extrapolated to larger populations. Clinical trials performed in limited groups of patients of the same age, race, education, socioeconomic background, or skin type may not apply to a broader population. Short-term outcomes do not always translate into long-term outcomes for any particular treatment, necessitating specific clinical trials conducted over long periods of time.

Table 5-1 CONSORT guidelines on items to include when reporting a randomized clinical trial

Manuscript SECTION and topic	Item	Description
TITLE and ABSTRACT	1	How participants were allocated to interventions (e.g., “random allocation,” “randomized,” or “randomly assigned”)
INTRODUCTION Background	2	Scientific background and explanation of rationale
METHODS Participants	3	Eligibility criteria for participants and the settings and locations where the data were collected
Interventions	4	Precise details of the interventions intended for each group and how and when they were actually administered
Objectives	5	Specific objectives and hypotheses
Outcomes	6	Clearly defined primary and secondary outcome measures and, when applicable, any methods used to enhance the quality of measurements (e.g., multiple observations, training of assessors)
Sample size	7	How sample size was determined and, when applicable, explanation of any interim analyses and stopping rules
Randomization: Sequence generation	8	Method used to implement the random allocation sequence (e.g., numbered containers or central telephone), clarifying whether the sequence was concealed until interventions were assigned
Randomization: Allocation concealment	9	Method used to implement the random allocation sequence (e.g., numbered containers or central telephone), clarifying whether the sequence was concealed until interventions were assigned
Randomization: Implementation	10	Who generated the allocation sequence, who enrolled participants, and who assigned participants to their groups
Blinding (masking)	11	Whether or not participants, those administering the interventions, and those assessing the outcomes were blinded to group assignment. When relevant, how the success of blinding was evaluated
Statistical methods	12	Statistical methods used to compare groups for primary outcome(s); methods for additional analyses, such as subgroup analyses and adjusted analyses
RESULTS Participant flow	13	Flow of participants through each stage (a diagram is strongly recommended). Specifically, for each group report the numbers of participants randomly assigned, receiving intended treatment, completing the study protocol, and analyzed for the primary outcome. Describe protocol deviations from study as planned, together with reasons
Recruitment	14	Dates defining the periods of recruitment and follow-up
Baseline data	15	Baseline demographic and clinical characteristics of each group
Numbers analyzed	16	Number of participants (denominator) in each group included in each analysis and whether the analysis was by “intention-to-treat.” State the results in absolute numbers when feasible (e.g., 10/20, not 50%)
Outcomes and estimation	17	For each primary and secondary outcome, a summary of results for each group and the estimated effect size and its precision (e.g., 95% confidence interval)
Ancillary analyses	18	Address multiplicity by reporting any other analyses performed, including subgroup analyses and adjusted analyses, indicating those prespecified and those exploratory
Adverse events	19	All important adverse events or side effects in each intervention group
DISCUSSION Interpretation	20	Interpretation of the results, taking into account study hypotheses, sources of potential bias or imprecision, and the dangers associated with multiplicity of analyses and outcomes
Generalizability	21	Generalizability (external validity) of the trial findings
Overall evidence	22	General interpretation of the results in the context of current evidence
From Altman DG, Schulz KF, Moher D, et al., for the CONSORT Group. The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med 2001;134:663–694.