CHICAGO – A study in The Journal of the American Medical Association found that hospitals scoring well on the 2006 Leapfrog Safe Practice Survey were not "significantly associated with risk-adjusted inpatient mortality."

The Leapfrog survey assesses hospital performance based on four quality and safety practices: computerized physician order entry, intensive care unit physician staffing, evidence-based hospital referrals for high-mortality surgeries, and the Safe Practice Survey, which allows hospitals to report efforts toward implementing the National Quality Forum's Safe Practices for Better Healthcare.

The results of the Safe Practice Survey are posted online and open to the public, but the study says it remains unclear how well "quality as assessed by the Safe Practices Survey correlates with outcomes of interest to patients and policy makers, such as mortality."

Safe Practice Survey scores are ranked by quartiles, which suggests hospitals in the highest quartile provide safer care than those in lower quartiles.

A recent study examining the first three initiatives found some positive associations between survey performance and reduced mortality, but little research is available about the Safe Practice Survey, which is known to take the most time to complete, the study points out.

The study used mortality data from the most recent version (2005, available in 2007) of the Nationwide Inpatient Sample (NIS), a database of hospital inpatient stays which is used by researchers and policymakers to identify, track and analyze national trends in healthcare use, access, charges, quality and outcomes.

The study found that of the 155 hospitals in the NIS that participated in the Safe Practice Survey, 34 percent were teaching hospitals and 66 percent were nonteaching hospitals.

Within the NIS, hospitals that completed the survey were more likely to have higher volumes of discharges than those that did not complete the survey.

In 2005 there were 1,772,064 discharges from the 155 hospitals, of which 37,033 resulted in an inpatient death. According to the study, quartiles of the survey were not a significant predictor of mortality, regardless of whether adjusted for expected mortality risk and certain hospital characteristics.

The study authors say the way the survey was designed "gives hospitals credit for creating systems that could eventually support full implementation of a given safe practice. However, awarding survey points for hospital administrative structures raises the possibility that the survey is capturing excessive noise, which may be overwhelming an important signal."

An alternative "action-based" scoring method for the survey assigns all points for a safe practice based on whether the hospital said it had applied the key action for the practice. According to the study, the "action-based" scoring method "slightly improved the ability of the survey ranking to predict in-hospital mortality, although the association was not statistically significant. Focusing on actions in the future may improve the survey's ability to discriminate between high-quality and low-quality hospitals."

The Safe Practice survey has been condensed from 27 to 13 safe practices, which the study says indicate "scores based on the 13 retained practices are unlikely to be significantly associated with inpatient mortality, even if scoring is limited to actions taken."

Authors raise the question of whether the Safe Practice survey is really measuring the right things, but is rather giving credit to hospitals for good intentions.

"Our findings should not be interpreted, however, as indicating that the safe practices are not important or that they cannot be measured in an informative and valid way," the authors said.