Chun Lok K. Li

Correspondence: Chun Lok K. Li krislichunlok@hotmail.com

**Author Affiliations**

Melbourne Institute of Applied Economic and Social Research, The University of Melbourne, Victoria, Australia.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

**Background:** Policymakers in many countries are confronted with the question of whether, and how much, resources should
be committed to improving hospital quality. This is because there is little consensus about the relationship between hospital
quality and cost, despite the extensive existing literature. This makes it difficult for policyholders to commit resources to make
improvements. The diversity of methods used also makes cross-comparison difficult. To address this, two specific methodological
issues that are commonly observed will be investigated. The first is the choice of metric used for assessing hospital quality. The
second is the way the distribution of the measure is specified.

**Methods:** An empirical example will be setup that resembles a typical study on this research topic. The purpose is to demonstrate
that a change in the metric used for measuring hospital quality, or a change in its distributional assumptions, will lead to a
different result, even when the same data is used. For simplicity, the two most general metrics of patient mortality and unplanned
readmissions will be used. The measuring statistic is the odds ratio to allow for patient risk-adjustment. A bootstrap-adjusted
regression modified from Lindley and Smith (1972) will be used to account for the distribution of the quality statistic. Hospital cost
data is derived from financial statements on a subset of Victorian public hospitals from 2002/03 to 2004/05. Patient data is sourced
from encoded records in the Victorian Admitted Episodes Database (VAED) for the six years from 1999/00 to 2004/05, which are
used for patient risk-adjustment. Hospitals and patients are anonymously linked.

**Results:** The relationship between quality and cost is negative at the 95% significance levels when the bootstrap adjustment is
applied, for both adjusted mortality and readmissions. When the adjustment was removed, the relationship became positive with
mortality and close to zero for readmissions, both with much wider confidence intervals. A hypothetical static comparison exercise
resulted in a difference in estimated costs of around several million AUD a year in operating costs of public hospitals in the state of
Victoria.

**Conclusion:** When determining the effectiveness of hospital policies, more than one operative measure of quality should be used as
a robustness check. The dispersion of quality measures must be explicitly accounted for.
MeSH codes: N03.219.262 (Hospital Economics); N05.300.375.500 (Costs, Hospital); N02.278.421.510 (Hospitals, Public);
N04.452.871.715.800 (Risk Adjustment); E02.760.400.620 (Patient Readmission); E05.318.308.985.550.400 (Hospital Mortality);
E05.318.740.750 (Regression Analysis).

**Keywords:** Hospital economics, risk adjustment, costs, hospital, regression analysis, hospitals, public, hospital mortality

The relationship between hospital quality and cost is an established research question. For example, in the US, hospital dummy variables were used as a measure of quality in a study that investigated the revealed preference of pneumonia patients in Los Angeles [1]. In Hong Kong, a medical study used the "30-day odds-ratio" to a selection of acute illnesses that are representative of the dataset [2]. And in Australia, the accuracy of adverse incidence rates and its impact on measuring hospital performance was investigated, with a strong emphasis on patient risk-adjustment [3]. The interested reader can find many more.

However, despite the abundance of existing studies investigating this research question, there is still a lack of a clear consensus as to whether the relationship between hospital cost and quality is positive, negative or not related. Without a representative conclusion, there is no benchmark to use for empirically validating new theories claiming to explain the phenomenon, and policymakers in many countries are still unsure whether suggested policies to improve hospitals will be effective. Some of these differences can be attributed to differences in data used, in which case a diversity of results could simply reflect a strong geographical influence. But the problem is that, given the differences in measures used for assessing hospital quality statistics, and the strong assumptions used for the specification of said measures, it is difficult to tell either way. A wide variety of metrics can be used to measure patient out- comes [4]. Some examples that are directly related to hospital studies are different types of ambulatory care [5] or various hospital-acquired infections. There is little consensus on which type of incidence to count, except that more than one measure should be used if the data and research question allow [6].

As for proper specification of hospital quality statistics, data limitations are the most commonly cited issue. The method of first resort is usually instrumental variables, solving the data shortage issue and reducing endogeneity at the same time. But finding a good instrument is often challenging because it needs to be closely related to the variable in question, and at the same time uncorrelated with the other variables. Some well-known examples include the patient's geographical distance from the nearest hospital [7] or even time differences [8]. Most studies do not have such clean solutions.

This paper provides an empirical reference to the literature that highlights the necessity of testing results using more than one quality measure, and the need to capture the distribution of said measures. By using the same data and only varying the estimation method in these two ways, any differences in results will provide further support that future studies will also need to introduce similar variations. The aim is to expedite progress towards this research question by enabling cross-comparison between studies, thereby reducing the uncertainty facing policymakers in this regard. The next section will explain the empirical setup.

This section is organised as follows. A model for capturing the distribution of a hospital-level Odds Ratio (OR) will be derived. An illustrative example, that of a typical empirical analysis between hospital quality and operational cost, will be described. The estimation strategy will be explained.

**Modelling the hospital odds ratio**

The OR is a generic measure that can be applied to most hospital
incidents. This study will measure hospital quality using
two incidence types, mortality and unplanned readmissions
(henceforth referred to as 'readmissions'). Mortality is clearly
defined, while one readmissions count is defined in the data
as an unplanned revisit within 28 days from the same patient
for the same problem.

There are H hospitals in the data, indexed *h* = 1,...,*H*. The
patients that each hospital admits are indexed *p* = 1,...,*P*_{h}.
The mortality status of each patient is modeled as a binary
variable 0 or 1, denoted as D_{hp} with probability of occurrence
E(D_{hp}). The number of previous readmissions of each patient
is a count variable Rhp with expected value E(R_{hp}).

Each hospital h has a specific number of mortality incidents
per year D_{h} and readmissions R_{h}. Their estimates, denoted as

The number of hospital incidents D_{h} and R_{h} are random
variables, with corresponding expected values of E(D_{h}) and
E(R_{h}). They are computed as follows:

The OR of these two incidence types, written as Z_{D} and Z_{R} is
then the ratio of observed to expected number of incidents:

We want to capture the stochastic processes of ZD_{h} and ZR_{h}.
By construction, they contain four random variables D_{h} R_{h} E(D_{h}) and E(R_{h}) each following its own distribution.

D_{h} is the sum of binary variables D_{hp} it follows a binomial
distribution with probability of incidence equal to D_{h}/P_{h}.

R_{h} is the sum of count variables R_{hp} which follows the Poisson
distribution with expected count rate R_{h}.

E(D_{hp}) is estimated using regression because probabilities
are not directly observed. Let δ_{hp} be the vector of dependent
variables associated with each patient *p* in hospital *h*. Each
element in that vector has a corresponding parameter
estimate that captures their marginal effect on incidence
probability. Denote the vector of parameters for mortality
using λ_{D}. Mortality is a binary dependent variable, estimated
using Logistic regression:

E(R_{hp}) is a count variable, estimated using Poisson regression
[9]. Let λ_{R} be the vector of parameters. The estimation is as
follows:

Both Logistic and Poisson regressions are estimated using standard maximum-likelihood estimation (MLE). This study used STATA v12; it is also available in most major off-the-shelf statistical software.

Computing point estimates of Z_{Dh} and Z_{Rh} is straight forward.
However, when they are included in a cost function
for estimation, its error structure needs to be accounted for
in order to reduce unobserved bias. Standard computed regressor
methods exist for this purpose, but they only work
for variables with Gaussian distributions or some other simple
functions. This is not the case for Z_{Dh} and Z_{Rh} where each is
formed as the ratio of two random variables. Their algebraic
structure does not lend itself to clean, analytical solutions either.

Approximating their distributions using parametric simulation
can get around this problem. Let ε_{D} and ε_{R} be the
vector of standard errors for λ_{D} and λ_{R} respectively. Index the
patient variables used for risk-adjustment from *u* = 1,...,*U* and,
accordingly, each element in the given vectors.

The distributions of Z_{Dh} and Z_{Rh} can be simulated based on
the regression assumption that each estimated parameter
follows the normal distribution, given as follows:

By generating a large number of simulated risk-adjustment
parameters and re-computing Z_{Dh} and Z_{Rh} we produce an
EDF of the estimated quality of each hospital. For this study,
parametric simulation is repeated for a total of N=1000 values
for each Z_{Dh} and Z_{Rh}. Results are denoted as Z_{Dh1} and Z_{DhN} and
Z_{Rh1} to Z_{RhN} respectively. The use of these simulated quality
values are explained later.

**The empirical setting**

The relationship between hospital quality and cost is an
established research topic without a consensus opinion. The
focus here is the differences in estimation results when the
distribution of the quality variable is controlled for. To do this,
we construct a hospital cost function with input prices, output
units and risk-adjusted quality measures. Throughout this
study, individual hospitals cannot be identified and summary
statistics are presented only in aggregate.

*Input*

Hospital input prices are available for public hospitals in
Victoria, Australia from the years 2002/03 to 2004/05 (Table 1).
They are extracted from financial accounting statements, with
years chosen based on similarity in reporting requirements.
To be included in this study, a hospital must have enough
entries in their financial statement to compute total expenses,
capital prices (K), labour prices (L), and materials prices (M).

Table 1 : **Selected hospitals in the VAED, 2002/03 to 2004/05: an
overview.**

*Output*

The unit of output is the number of patient-days per episode.
They are available in the Victorian Admitted Episodes database
(VAED) for the six years from 1999/00 to 2004/05 (Table 2). It
contains basic information such as demographics, disease diagnostics and how the patient was discharged. All six years
of VAED patient data are used for patient risk-adjustment,
while the last three are included into the cost function through
data-linkage with input prices.

Table 2 : **Summary of patient mortality rates in VAED by year and Major Diagnostic Categories.**

*Risk-adjusted quality*

Regression variables included for risk-adjustment include:
age, gender [10], and the ICD-10-AM Charlson comorbidity
index [11]. Additionally, the risk-factors for patients in each
of the 23 Major Diagnostic Categories (MDC) as listed in the
VEAD are separately estimated. The advantage over the more
common fixed-effects approach is it allows the same variable
to exert a different marginal effect depending on the MCD.

Note that, since patient data for this study begins at the year 2002/03 and ends at the year 2004/05, the readmissions rate for episodes that occurred near the first and last 28 days of the observation period will be under-represented.

**Estimation Strategy**

The hospital cost function used in this study contains the total
operating cost of each hospital in a given financial year as the
subject variable and all the explanatory factors contributing
to the total cost,

where **w** is the vector of input prices, **y** is the number of
patient-days and **z** are the risk-adjusted mortality and readmissions
measures.

We approximate the hospital cost structure using a translog
function [12]. Let *i* and *j* be indices for input prices. The reduced
form is given as follows:

The cost function is estimated using seemingly-unrelated regressions (SUR) [13], which improves estimation accuracy by imposing linear homogeneity and reducing standard error. It is now available in most commercial software; see Christensen et al., (1973) for the technical derivations.

The statistical uncertainty in the quality measures Z_{D} and Z_{R} also needs to be accounted for. We do this using a bootstrap
regression [14]. Recall that each hospital has two observed
quality measures Z_{Dh} and Z_{Rh}, each associated with a set of
N=1000 simulated quality values Z_{Dhn} and Z_{Rhn}. If each of
the observed hospital quality measures is replaced with a
simulated one, then the resulting parameter estimates will
be different as well.

For each bootstrap *n* = 1,...,*N*, denote the resulting parameters
as

The weighted-average estimates of β follows:

Point estimates from both the unadjusted and adjusted versions will be presented for comparison.

Parameter estimates of the cost function estimated with
adjustments for the distribution of the quality measure
are listed in Table 3. The risk-adjusted mortality and readmissions
parameters

Table 3 : **Parameter estimates by bootstrap regression.**

When the same cost function is estimated without the
adjustment as in Table 4,

Table 4 : **Parameter estimates without bootstrapping.**

A comparison between estimates generated with and without adjustment serves to highlight the importance of controlling for the statistical properties of hospital quality. However, the practical implications of these differences are not easily understood just by inspecting these numbers in isolation.

To put these differences in policy context, a static comparison
exercise can be used to evaluate the magnitude of the possible
financial miscalculations from an adjustment in cost function
estimation methods. The setup is a hypothetical policy
intervention whereby hospitals below a certain industry
standard for patient outcome are given resources to bring
their operations up to a minimum target level:

Here the symbols

Dollar values for the static comparison results are displayed in Table 5. In the case of the sampled Victorian hospitals, the magnitude is around 1 million AUD per year for the 1.0 "industry standard" scenario. This number increases to around 10 million for the 0.5 "major improvement" scenario. The difference in both cases is a change in sign.

Table 5 : **Static comparison: estimated cost of intervention.**

Note also that mortality and unplanned readmissions exert a different effect on hospital expenses. Both are measures of hospital quality, suggesting that the financial impact of quality improvements does depend on which variables are used to measure quality when evaluating policy options for improving hospital quality.

When the cost function was estimated with an adjustment to the distribution of the quality variables, there is a negative and statistically-significant relationship between risk-adjusted incidence rate and hospital cost. In contrast, when the adjust ment is removed, the correlation became positive and not significant. This comparison provides a reference to the literature supporting the notion that the distribution of quality measures needs to be captured.

As an example, consider an authoritative study done in Japan on its hospital spending and quality [16]. Their main research question is why Japan is able to have superior quality for lower costs as compared to other OECDs. The core analysis consists of a log-log constant elasticity function where hospital quality is the dependent variable and hospital characteristics are the explanatory variables. One of their central measures of quality that is similar to OR, namely the Standardised Mortality Ratio, is an estimated measure with standard errors that are not accounted for. This may distort the evaluation of the quality of individual hospitals in the analysis, possibly resulting in less accurate parameters for determinants of quality.

We have also reinforced the importance of the choice of variable used to measure hospital quality by comparing between two separate variables, mortality and unplanned readmissions. The two variables had different marginal effects on hospital cost, even under controlled experiment settings, where the same dataset, cost function and estimation strategy were used.

The weakness of aggregating quality measures into a single index can be further illustrated using a study on US public hospitals in several states [17]. Their main message is that the negative impact of poor quality on hospital efficiency is underestimated in existing literature. The basis for their argument is that they found a stronger correlation using a multiple-indicator measure as compared to a more popular single-indicator measure, the Acute Myocardial Infraction Indicator (AMI). However, since their analysis did not include estimating the impact of individual components in the aggregated measure, there is a chance that their result was driven by a couple of outlier components rather than a general underlying relationship.

We recommend that the stochastic properties of quality measures should be explicitly captured for future empirical studies. Also, as demonstrated in this study, more than one quality measure must be used for any assessment. We used Australian hospitals as an example but a similar notion applies to hospitals from other countries. Additionally, this study's approach can be used as a starting basis for exploring the stochastic properties of other commonly-used quality measures. It can also be used to explore other, more effective methods for capturing the multiple dimensions of quality in an empirical setting. Both are ongoing research areas.

The author declares that he has no competing interests.

The author acknowledges the Melbourne Institute of Applied Economic and Social Research for providing the data and research funding for this study.

Editor: Abdulbari Bener, Weill Cornell Medical College, Qatar.

EIC: Jimmy Efird, East Carolina University, USA.

Received: 16-Mar-2014 Final Revised: 29-Apr-2014

Accepted: 05-May-2014 Published: 19-May-2014

- Romley JA and Goldman DP.
**How costly is hospital quality? A revealed-preference approach**.*J Ind Econ.*2011;**59**:578-608. | Article | PubMed Abstract | PubMed Full Text - Wong EL, Cheung AW, Leung MC, Yam CH, Chan FW, Wong FY and Yeoh EK.
**Unplanned readmission rates, length of hospital stay, mortality, and medical costs of ten common medical conditions: a retrospective analysis of Hong Kong hospital data**.*BMC Health Serv Res.*2011;**11**:149. | Article | PubMed Abstract | PubMed Full Text - Hauck K, Zhao X and Jackson T.
**Adverse event rates as measures of hospital performance**.*Health Policy.*2012;**104**:146-54. | Article | PubMed - Donabedian A.
**Evaluating the quality of medical care. 1966**.*Milbank Q.*2005;**83**:691-729. | Article | PubMed Abstract | PubMed Full Text - Finegan MS, Gao J, Pasquale D and Campbell J.
**Trends and geographic variation of potentially avoidable hospitalizations in the veterans health-care system**.*Health Serv Manage Res.*2010;**23**:66-75. | Article | PubMed - Iezzoni LI.
**The risks of risk adjustment**.*JAMA.*1997;**278**:1600-7. | Article | PubMed - Gowrisankaran G and Town RJ.
**Estimating the quality of care in hospitals using instrumental variables**.*J Health Econ.*1999;**18**:747-67. | Article | PubMed - Carey K and Burgess JF, Jr.
**On measuring the hospital cost/quality trade-off**.*Health Econ.*1999;**8**:509-20. | Article | PubMed - Halfon P, Eggli Y, van Melle G, Chevalier J, Wasserfallen JB and Burnand B.
**Measuring potentially avoidable hospital readmissions**.*J Clin Epidemiol.*2002;**55**:573-87. | Article | PubMed - Leng GC, Walsh D, Fowkes FG and Swainson CP.
**Is the emergency readmission rate a valid outcome indicator?***Qual Health Care.*1999;**8**:234-8. | Article | PubMed Abstract | PubMed Full Text - Charlson ME, Pompei P, Ales KL and MacKenzie CR.
**A new method of classifying prognostic comorbidity in longitudinal studies: development and validation**.*J Chronic Dis.*1987;**40**:373-83. | Article | PubMed - Christensen LR, Jorgenson DW and Lau LJ.
**Transcendental logarithmic production frontiers**.*The Review of Economics and Statistics*. 1973;**55**:28–45. | Pdf - Zellner A.
**An efficient method of estimating seemingly unrelated regressions and tests for aggregation bias**.*Journal of the American Statistical Association*. 1962;**57**:348-68. | Pdf - Freedman D.
**Bootstrapping regression models**.*The Annals of Statistics*. 1981;**9**:1218–28. | Article - Lindley D and Smith A.
**Bayes estimates for the linear model**.*Journalof the Royal Statistical Society Series B (Methodological)*. 1972;**34**:1–41. | Article - Hashimoto H, Ikegami N, Shibuya K, Izumida N, Noguchi H, Yasunaga H, Miyata H, Acuin JM and Reich MR.
**Cost containment and quality of care in Japan: is there a trade-off?***Lancet.*2011;**378**:1174-82. | Article | PubMed - Clement JP, Valdmanis VG, Bazzoli GJ, Zhao M and Chukmaitov A.
**Is more better? An analysis of hospital outcomes and efficiency with a DEA model of output congestion**.*Health Care Manag Sci.*2008;**11**:67-77. | Article | PubMed

Volume 2

Li CLK. **Hospital quality measure specification and their
estimated relationship with hospital cost.** *J Med Stat
Inform*. 2014; **2**:4.
http://dx.doi.org/10.7243/2053-7662-2-4

View Metrics

Copyright © 2015 Herbert Publications Limited. All rights reserved.

Post Comment|View Comments