Composite indicators of quality combine multiple metrics of quality into an aggregate score. Table 1 (adapted from Nardo 15 ) summarizes the advantages and disadvantages of using composite indicators, regardless of field or purpose. We will discuss the advantages and disadvantages of composite indicators focusing on their two probable uses, benchmarking and quality improvement.

Benchmarking of providers based on only one or a few indicators of quality may be problematic for several reasons. First, benchmarking based upon a few indicators infers a strong correlation of performance across all dimensions of quality, whether measured or not. However, this has not been found in the extant literature. Several articles have highlighted weak correlations among metrics of quality 16 17 . In other words, performance in one aspect of care quality is not necessarily informative about performance in others. It is possible that composite indicators may be better suited to reflect an overall construct of quality.

A second benefit of composite indicators of quality is that they are communicable to diverse stakeholders and may be leveraged to induce competition on quality. Payers of healthcare increasingly employ these measurements to inform and direct patients' choice of providers through selective contracting. Patients may gain from transparent provider competition for quality and through the ability to make informed healthcare choices. While to date there is little evidence that benchmarking information affects patient choice of provider 18 , consumer attitudes may change as the quality and dissemination formats of quality information improve. However, any benefit to patients is dependent on the accuracy of classifying providers as superior or inferior. Variation in methods and quality of existing composites may lead to significant misclassification of providers as outliers 19 .

Composite indicators are a simplified representation of the underlying quality of care construct. In fact, simplification is their main appeal. There is a danger, however, that overly simplistic policy messages derived from composites may be misleading or misused to support narrow agendas. If the providers being measured perceive the indicators to lack scientific soundness, transparency, or content validity, they are unlikely to produce desired improvements in patient health status. In addition, a summary score may inaccurately suggest that providers are average if good scores on one metric compensate for poor performance on other metrics. In fact, 'average' providers may be 'poor' providers for patients whose needs are within the low scoring performance areas. Some of these dangers can be countered by using dissemination formats that convey results accurately while avoiding oversimplification (such as the ability to 'drill down' into individual components of the composite), and by making the process of indicator development explicit and transparent to all stakeholders. In addition, statistical techniques such as multi-criterion analysis mitigate the problem of performance averaging 15 . Nevertheless, it is likely that composites used for benchmarking will be subject to methodological and political challenge from providers disagreeing with results.

Composite indicators might support quality improvement in various ways. They may help providers translate a bewildering wealth of information into action and track effects throughout the care delivery system. To illustrate, the Vermont Oxford Network tracks the quality of healthcare delivery of over 800 neonatal intensive care units worldwide, with clinically rich information available for many processes and care outcomes 20 . It may be difficult for neonatal intensive care providers to translate large volumes of data into effective quality improvement efforts.

A multi-dimensional approach to quality measurement via composite indicators may support such a multi-dimensional approach to quality improvement. Composite indicators and their individual components may identify specific areas for attention, for which specific evidence-based interventions are then developed. The success of improvement can then be cross-checked with the comprehensive measure set to ensure that this focus has not worsened quality of care in another area. However, targeting individual quality metrics may lead to piecemeal rather than system-based efforts in quality improvement. Potentially, larger leaps in improvement may result from systems-based interventions that affect multiple areas of care simultaneously and have the potential to spread 21 throughout the care service and the institution. Improving safety attitudes among staff is an example of a system-based intervention that may improve outcomes and propagate throughout an institution 22 . Whether composites are used to track improvement targeting individual or multiple metrics will depend on local resources, support systems, expertise, and institutional capacity. In either application, composites would allow tracking of overall improvement and their sub-components could alert users to potential concordant or discordant effects of improvement activities on other measures of quality.

Thus, using composite indicators does not imply replacing the measurement of individual metrics of quality. Rather, composites merely summarize the information contained in the individual metrics and make that information more digestible. A synergistic approach of using both composites and individual metrics may permit harnessing the advantages of both.

Recognizing that there are numerous editorial choices in the development of composite indicators, and that quality of care can be defined in overly simplistic ways, we propose a composite-based approach to measuring pediatric care quality by combining the JRC composite development methodology 15 and Profit et al.'s quality measurement framework 12 .

As do other organizations, the JRC has significant institutional expertise in developing, applying, and evaluating composite indicators; it has, in fact, published guidelines for composite indicator development 15 23 24 . These guidelines have begun to be used in other settings of healthcare 25 . What differentiates the JRC's approach from that of other organizations is its highly explicit, transparent, and evaluative approach to composite indicator development. Proposed methods promote internal and external statistical and methodological consistency and offer users choices of building blocks at each step in composite indicator construction, tailored to the task at hand.

Table 2 shows the JRC's ten step approach to composite indicators development 15 . We present here a brief summary of this approach along with a theoretical example of composite score development for pediatric intensive care unit (PICU) quality. We refer readers to the JRC handbook 15 for additional detail.

The two basic methods used to arrive at metric weights are statistical (e.g., principal component analysis, factor analysis, multivariate techniques) and participatory methods (variations on eliciting expert opinion). Note that equal weighting does not imply an absence of weights: under this approach each metric is given a weight of one. An equal weighting scheme may introduce an element of double counting if two metrics prove to be highly correlated (e.g., VAP rates and VAP prevention practices).

Benefits of the statistical approach to weighting include its relative fairness and its freedom from bias. In contrast to the participatory approach, its primary disadvantage is that resultant weights may lack face validity.

Equal weighting has the benefit of simplicity and has been found to result in comparable performance assessment when compared to differential weighting schemes unless differences in weights are very large 38 . This is especially true if the number of metrics included in the composite is large. Because weighting schemes are inherently controversial, they are likely subject to opposition. One approach to addressing such concerns involves the use of data envelopment analysis, which allows each hospital to vary the weights to individual metrics such that the hospital can achieve its optimal position among its peers 39 .

In this phase the metrics are combined to form the composite indicator. The primary decision involved in choosing an aggregation method hinges on whether providers should be allowed to compensate for poor performance in one metric with superior performance in another. There are three principal choices: full compensation (additive), partial compensation (multiplicative), and no compensation (non-compensatory).

Because of its simplicity, the additive aggregation technique is used widely. However, developers need to be cognizant that additive aggregation implies full compensability between metrics and may therefore result in a biased composite indicator, with an error of dimension and direction not easily determined.

Multiplicative aggregation allows for partial compensability, which makes it more difficult to offset a bad indicator with a good one. This is in line with our concept of quality in which a quality performance metric is intended to foster superior quality throughout domains of care and not promote trade-offs between areas of strength and weakness.

Non-compensatory methods, such as multi-criterion analysis, demand achieving excellence in all metrics of quality or at least achieving minimum standards of quality, thereby promoting multi-dimensional improvement efforts. We believe that developers of pediatric composite indicators should seriously consider the use of non-compensatory aggregation methods, so that quality of care in one aspect cannot be traded off another, since negative consequences of poor quality of care in any area of healthcare may have long-term consequences for a child's health and social well being. At the least, we recommend this aggregation method be explored as a variant of indicator construction in uncertainty analysis (see step seven). One variant of non-compensatory methods, the 'all-or-none measurement' approach, has been recently propagated as a means to foster excellence in quality 40 . However, it has been argued that this particular approach is likely imprecise and may provide perverse incentives, such as promoting treatment irrespective of how small the potential benefit and how great the patient burden or risk 41 .

Developing a composite indicator of quality for pediatric care faces several challenges, including paucity of interactions with the healthcare system, paucity of critical health outcomes, and availability of quality of life and prevention metrics. These factors have various implications for measurement that, when taken together, present unique challenges to composite development for pediatric care.

The number of yearly admissions for pediatric patients is smaller than that for adults, making sample size a significant issue 45 . Metric development may therefore require ongoing data collection over several years and across multiple institutions. The aggregation of several metrics into a composite indicator may alleviate this problem, in that information from multiple quality metrics can be combined and thereby increase the power to detect a quality signal; however, this is an empirical question and needs to be addressed in future research.

As death is an uncommon outcome in children, mortality in isolation is a poor discriminator of care quality 43 . Moreover, mortality does not always represent poor care quality but may reflect appropriate decisions by providers and parents to provide comfort care for children with irreversible and debilitating conditions. Attitudes towards comfort care are likely to vary among providers, regions, and parental caregivers, which further undermines the ability of mortality to discriminate hospital quality of care 46 . Nevertheless, mortality is an important balancing measure, which ensures that hospitals do not receive undue credit for measures that are sensitive to mortality (e.g., length of stay). We therefore recommend including mortality in composite indicators measuring the quality of acute care settings. However, its effect on provider performance should be subject to sensitivity analysis, as should be its weighting.

Health-related quality of life is an important outcome of care quality, but it is difficult to measure in children. Because children under the age of five are typically unable to reliably answer quality of life questions, caregiver proxy assessment has been used as a reasonable substitute 47 48 . However, because parental rating of their children's quality of life may be positively biased 49 , health-related quality of life ratings may need to be obtained from health professionals or the general public. Recommendations for cost-effectiveness analysis favour the general public's perspective 50 ; yet such ratings are strongly influenced by responder personal experience with health status 51 and may also reflect the availability and quality of chronic care management and the degree of health system integration. In addition, studies by Saigal and colleagues suggest that patient utilities may not be stable over a patient's life, even in light of stable chronic disease 52 53 54 55 . This suggests that the effect of patient preferences on provider performance on a composite indicator of quality should be assessed by allowing preferences to vary over a reasonable range in sensitivity analyses. Future research should try to address these important methodological gaps that remain in the measurement of health-related quality of life of young children 56 . Until such research is conducted, the uncertainty in quality of life ratings should be reflected in lower relative weightings, so as to not threaten the external validity of the composite indicator.

Much of the job of pediatric health professionals is to prevent illness or illness exacerbation. Therefore, metrics of primary prevention should be given particular consideration during the metric selection process. Childhood illness may potentially lead to long-lasting, even devastating, adverse outcomes, permanently altering children's developmental trajectories 57 . Thankfully, high quality rehabilitation and educational services can support children's unique adaptation to injury, enabling them to reach full potential 58 . This implies that measurement of healthcare quality should emphasize longitudinal linkages to health outcomes over time, which will provide an opportunity for validation of the composite indicator and offer opportunities for further linkage to additional social well being outcomes to help assess the quality of larger societal systems, including social support and educational systems. Currently, few such metrics exist, and much research will be needed to develop them.

The importance of preventive care services in pediatrics does not necessarily imply that this aspect of care should be attributed higher relative importance compared to measures of acute care in a composite indicator of pediatric healthcare quality. Measure developers will have to make decisions on weighting with regard to the purpose of the indicator, the underlying data, and clinical applicability. For example, measures of preventive care are likely to feature less prominently in a composite of pediatric intensive care than in a composite of ambulatory care. In addition, developers may choose to apply differential weights among preventive care measures based on their value to public health in a given society (e.g., the prevention of obesity may be of greater value than administration of polio vaccine).