Evaluating the quality of antimicrobial prescribing: Is standardisation possible?

Abstract

The quality of antimicrobial prescribing refers to the optimal way to use antibiotics in regard to their benefits, safety (e.g., resistance generation and toxicity) and cost. Evaluating the quality of antimicrobial prescribing in a way that focuses not only on reducing antimicrobial consumption but also on using them in a more optimal way allows us to understand patterns of use and to identify targets for intervention. The lack of standardisation is the primary problem to be addressed when planning an evaluation of antimicrobial prescribing. There is little information specifically describing an evaluation methodology. Information related to prescription evaluation can be obtained from the guidelines of Antimicrobial Stewardship Programs (ASPs) and from local and international experience. The criteria used to evaluate the quality of prescription should include the indication for antimicrobial therapy, the timeliness of initiation, the correct antibiotic choice (according to local guidelines), the dosing, the duration, the route of administration and the time at which to switch to oral administration. A locally developed guideline on antimicrobial therapy should preferably be the gold standard by which to evaluate the appropriatenes of prescriptions.

Various approaches used to carry out the evaluations have been described in the literature. Repeated point-prevalence surveys (PPS) have been proven to be effective in identifying targets for quality improvement. Continuous prospective monitoring allows the identification of more precise intervention points at different times during prescription. The design of the study chosen to perform the evaluation should be adapted according to the resources available in each centre. Evaluating the quality of antimicrobial prescribing should be the first step to designing ASPs, as well as to evaluating their impact and the changes in prescribing trends over time.

Resumen

La calidad de la prescripción de antibióticos se refiere a la forma óptima de usar antibióticos en relación con sus beneficios, seguridad (p. ej., generación de resistencia y toxicidad) y coste. La evaluación de la calidad de la prescripción de antibióticos de forma que se centre no solo en la reducción del consumo de antibióticos, sino también en su uso de una forma más óptima nos permite comprender los patrones de uso e identificar las dianas para la intervención. La ausencia de estandarización es el principal problema que debe ser resuelto cuando se planifica una evaluación de la prescripción de antibióticos. Existe poca información que describa específicamente una metodología de evaluación. La información relacionada con la evaluación de la prescripción puede obtenerse de las guías de los programas de optimización de uso de antibióticos (PROA) y de la experiencia local e internacional. Los criterios utilizados para evaluar la calidad de la prescripción deberían incluir la indicación para terapia antimicrobiana, el momento del inicio, la selección correcta del antibiótico (de acuerdo con las guías locales), la dosis, la duración, la vía de administración y el momento en el que se cambia a administración oral. Es preferible una guía sobre el tratamiento antibiótico desarrollada localmente para ser utilizada como gold standard en la evaluación de lo apropiado de las prescripciones.

Han sido descritos en la literatura varios planteamientos utilizados para llevar a cabo las evaluaciones. Las encuestas de prevalencia puntual (EPP) repetidas han demostrado ser eficaces para identificar las dianas de la mejora de la calidad. El control prospectivo continuo permite la identificación de puntos de intervención más precisos en diferentes momentos durante la prescripción. El diseño del estudio seleccionado para realizar la evaluación debe ser adaptado de acuerdo con los recursos disponibles en cada centro. La evaluación de la calidad de la prescripción de antibióticos debe ser el primer paso para diseñar PROA, así como evaluar su impacto y los cambios en las tendencias de prescripción a lo largo del tiempo.

Why evaluate the quality of prescribing?

The quality of antimicrobial prescribing refers to the optimal way to use antibiotics for the treatment and prevention of infectious diseases in humans, considering their benefits, safety and cost.¹ Antimicrobial use has been associated with clinical benefits in cases where the patients present with severe sepsis or shock, and it has been repeatedly shown that inadequate antimicrobial treatment is a risk factor for hospital mortality.^2,3 Better outcomes have been associated not only with antimicrobial use, but with their optimal prescription. For instance, certain β-lactams have been found to be more effective when they are administered in an extended infusion.⁴ It is also remarkable the importance of the early administration of antimicrobials in the case of sepsis, where the duration of hypotension before its initiation is a critical determinant of survival.⁵ In a recent Cochrane review, a variety of interventions were effective in reducing antibiotic consumption; however, only studies using antibiotic optimisation, not those that simply targeted a decrease in their use, showed an improvement in clinical endpoints.¹

Considering the optimal use of antimicrobials is important not only because of its health benefits but also because of its secondary effects. A temporal relationship between antimicrobial use and resistance has already been demonstrated.⁶ Factors related to antimicrobial resistance are multiple and often very difficult to individualise. There are horizontal plasmid transmissions of resistance mechanisms, but the importance of antimicrobial consumption to the selection of the resistant microorganism is clear. The use of certain antimicrobials is also associated with higher levels of resistance, e.g., quinolones are more prone to generating resistance compared with β-lactams.⁷ Not only the specific drug used, but also the dose or the duration of the treatment may contribute to resistance emergence in various ways.⁸ Until now, the lack of antimicrobial susceptibility has been solved with the generation of new drugs, however, not many more are expected to be developed in the next decade, which means we will be required to treat infections by resistant microorganisms without having any effective treatment available.⁹ This problem has been recognised by multiple international health organisations that are promoting interventions in order to control resistance.¹⁰ These interventions are conducted to support chemical companies in the development of new drugs, to optimise infection control programs in hospitals and to improve antimicrobial use through the development of antimicrobial stewardship programs (ASPs). Recognising the target of ASP interventions is one of the primary reasons to evaluate the quality of antimicrobial prescribing.

In addition to resistance, there are many adverse events created by antimicrobial use. Antimicrobials are one of the most commonly used drugs in hospital settings. From 25%–60% of hospitalised patients receive an antimicrobial course of treatment during their stay, and up to 60% receive at least a single dose. These treatments are often related to such adverse events as pharmacological toxicities or colitis due to Clostridium difficile; a frequent cause of admissions to the emergency department.^11–13

In addition to resistance and toxicity, antimicrobial treatments incur a high percentage of hospital costs, not only due to the direct cost of the drugs but also because their use often implies a longer patient stay.¹⁴ In addition, secondary costs due to antimicrobial adverse events or treatment for multiresistant microorganisms could be avoided with the optimisation of antimicrobial use.

These reasons highlight the importance of monitoring antimicrobial prescriptions, even at the patient level, particularly when studies show that antimicrobials are misused in 30%–50% of cases.¹⁵

Until the turn of the century, there were limited reliable data on antimicrobial use in hospitals. Most reports on the impact of ASPs have focused on the reduction of antibiotic consumption achieved after the implementation of different interventions.^1,16 This approach is poor if used alone. Assessing the quality of antimicrobial prescription (not focused on using fewer antibiotics, but on using them optimally) should be the aim of any ASP, because as mentioned earlier, the appropriate use of antimicrobials (not only their reduction) has been related to improved clinical outcomes and fewer adverse events.¹⁷ Evaluating the quality of antimicrobial prescription allows us to understand patterns of use and to identify targets for quality improvement for particular clinical departments. This should be the first step in designing ASPs. Continuous evaluation is also required to evaluate their impact and to adapt the ASPs to changes in prescribing trends over time.¹⁸

Methodology to evaluate the quality of antimicrobial prescribing

Most published studies have focused on the amount of antibiotic consumption, but little information is available about the methodology used to evaluate the quality of prescribing.¹⁹ In the text below we summarise the information extracted from the various ASPs described in the literature, the problems with evaluation and the discrepancies found.

Criteria for appropriateness

The criteria to evaluate the appropriateness of prescriptions should preferably be obtained from locally developed guidelines on antimicrobial therapy, as we will detail in the next section of this document.

The items used to evaluate the quality of the prescription should include the indication for antimicrobial therapy, the timeliness of initiation, the correct antibiotic choice (according to local guidelines), the dosing, the duration, the route of administration, and the time at which to switch to oral administration. Ideally, this information should be linked to clinical data (indication, source and severity of infection, etc.).^1,15,18,20 Assessing the correct collection of microbiological samples previous to the initiation of treatment is also important, since it is a key element in antibiotic de-escalation.

Evaluating the quality of a prescription exclusively according to its correspondence to a fixed list of pre-approved indications may be time-saving but inaccurate, because it lacks the consideration of the specific clinical factors of an individual patient.²¹ Assessing the inappropriateness of treatment solely on the basis of the susceptibility of the isolated microorganisms will frequently overlook the problem of patients over-treated with excessively broad-spectrum antibiotics or treated for too prolonged duration.²²

Evaluation target

All types of antibiotic treatment can be objects of evaluation. In this sense, evaluated treatments should be classified as prophylactic (prescribed to prevent the development of infections), empirical (prescribed before the etiologic agent is known) or targeted treatments (prescribed after the causative microorganism is known).¹⁵

Most published ASPs have targeted their interventions towards the use of select antimicrobials, typically those that are considered more likely to contribute to the development of resistance (e.g., cephalosporins, quinolones), those with the broadest spectrum or those that are particularly expensive (e.g., carbapenems, antifungals).^1,16 This may be an efficient approach when program resources are limited. However, when considering the target of the evaluation, limiting the use of some antimicrobials could induce an increase in the use of alternative agents, sometimes with significant ecological impact.²³ In this sense, positive results have been reported for programs that assessed antimicrobial treatments selected by alternative criteria: prolonged duration, combination therapy, the source of administration²⁴ and treatments of selected infectious diseases (ID) identified by positive microbiological tests,²⁰ as well as for educational programs evaluating randomly-selected treatments with all-class antibiotics.²⁵

Sources of information

The collaboration of the pharmacy department and an antibiotic census are essential for the identification of the patients to be evaluated. Electronic systems using computer surveillance of antimicrobial use, when available, can make recording the clinical data necessary for the assessment of the quality of prescriptions significantly easier. Otherwise, a review of patients’ charts and notes may be required).^15,20

Periodicity for evaluation

Evaluating the quality of prescriptions is essential for the successful development of any ASP, but can be somewhat time-consuming. Thus, it should be scheduled according to the resources available in each centre. Different approaches have been proven to be effective:

Continuous prospective monitoring

Centres that have implemented electronic prescribing systems may easily assess antimicrobial prescriptions in a prospective way.²⁶ However, these systems are not yet available in most hospitals. The continuous assessment of the appropriateness of antimicrobial prescriptions on a sequential basis is a factor of the interventions consisting of audit with feedback. This approach to surveillance will provide the most useful and accurate information regarding antibiotic use and will provide precise targets for interventions.¹⁵ Moreover, it implies an excellent opportunity for education when providing feedback, which may encourage acceptance of the program among clinicians.²⁷ In centres with a stable group of ID experts who are involved in the implementation of the ASP, this strategy has proven to be less time-consuming and more efficient,²⁵ even in small hospitals.²⁸

Point-prevalence surveys (PPSs)

Performing repeated PPSs is a resource-efficient strategy that has proven to be useful in assessing the quality of prescriptions and identifying the causes of the inappropriate use of antibiotics when time and resources do not allow continuous surveillance.^29,30 Point prevalence or ‘snapshot’ surveys are usually performed on a single day at a single place, and are repeated periodically (e.g., once a semester, once a year). PPSs will provide targets in the centre that may benefit from specific interventions, and will allow for the evaluation of their results. A successful international initiative to standardise the assessment of prescriptions through PPSs resulted in the European Surveillance of Antimicrobial Consumption (ESAC), a novel initiative established in 2000 that gathers standardised information from hospitals all over Europe.^29,31 Information from participant centres is recorded on a web-based system that facilitates the analysis and interpretation of data and provides much more complete information than previously reported European studies.³¹

Who can evaluate

Cooperation between ID clinicians, clinical pharmacists specifically trained in this area and microbiologists is essential to perform an integral assessment of the quality of prescriptions, focusing not only on the economic issues, but also on the clinical aspects of the prescriptions.^15,20 The evaluators should be aware of the guidelines used as the gold standard and should ideally be ASP team collaborators. They should present enough clinical knowledge to be able to evaluate antimicrobial prescription; for example, ID pharmacists, ID physicians and microbiologists integrated in ASP teams can lead periodic prevalence surveys.²⁰ In audit programs with feedback, the pharmacist and microbiologist can act as a triage for cases requiring assessment by an ID clinician.²⁰

The importance of gold standards (local protocols and guidelines) for antimicrobial quality evaluation

Any evaluation of antimicrobial prescribing quality should be preceded by a gold standard development. The evaluation process requires an objective tool as a comparator, available to prescribers and evaluators beyond their personal knowledge. Scientific societies recommend the use of guidelines as a quality comparator to avoid the arbitrary nature of evaluation that occurs when prescriptions are based on expert recommendations.¹⁵

The evidence-based guidelines have been developed to assist physicians in the selection of antibiotics and to reduce the variability in clinical care; the development of guidelines are an important strategy in ASPs. Poorer outcomes have been observed when implementing international guidelines without adapting them to local microbiological data.^32,33 International and national evidence-based guidelines should preferably be adapted to local circumstances that incorporate the results of local bacterial resistance patterns. ASP teams should welcome collaboration with physician prescribers when elaborating local guidelines. This collaboration may help not only to complete specific areas of knowledge, but also to involve prescribers in the antibiotic optimization process. The broad dissemination of guidelines in easily accessible forms (print and/or electronic) and active educational methods are required to increase the adoption of recommendations in clinical practice.³⁴ The guidelines should be revised and periodically updated to cover all treatable conditions, to change resistance patterns and to include new anti-infectious agents. The ASP members and all the quality prescribing evaluators should participate in the revisions in order to be updated about possible changes and also to incorporate the feedback information observed during the evaluations.

Guidelines are key in the antimicrobial optimizing process, not only as a comparator to evaluate the quality of antimicrobial prescribing, but also because their compliance leads to quality improvement. Some studies have shown that adherence to guidelines results in shorter stays and lower mortality. In several studies on community-acquired pneumonia, adherence to the guidelines was an independent protective factor against treatment failure and death.^35,36 There are other studies of hospital-acquired pneumonia, ventilator-associated pneumonia, meningitis and sepsis showing similar results: a reduction in antimicrobial use and costs, an increase of initial administration of adequate antimicrobial therapy, a shorten of therapy duration and a decrease of mortality.^37–39

However, the mere publication of guidelines is insufficient to significantly impact antimicrobial prescribing patterns because voluntary compliance with the recommendations is only moderate.⁴⁰ In a study from the Netherlands, the impact of the dissemination of national consensus guidelines for the treatment of bacterial meningitis via a printed handbook was studied; after the introduction of the guidelines, only 33% of cases were treated in accordance with the recommendations.⁴¹

There is little information regarding factors that influence adherence to guidelines. In a study evaluating guideline compliance, prescriptions were compared with guideline recommendations for specific indications; compliance was much higher for lower respiratory tract infections than for sepsis and urinary tract infections, but patient characteristics have only a limited impact on compliance. Predisposing illnesses and active malignancies were associated with more compliant prescriptions, whereas alcohol/drug abuse and serum creatinine levels were associated with less compliance. In the majority of noncompliant prescriptions, excessively broad-spectrum agents were used for the expected pathogens.⁴²

To improve guideline compliance, it is necessary to identify and understand the factors that influence adherence: the hospital department, patient characteristics (e.g., comorbid condition, initial severity) and ID. When evaluating the quality of antimicrobial prescribing, including the item “guidance compliance” as included in the last ESAC PPS²⁹ is recommended. Such information would facilitate recognising potential factors related to guideline adherence and may be conducive to planning strategies to improve compliance.

Evaluation problems and discrepancies

The lack of standardisation is the primary evaluation problem. Although there are many publications about how to monitor antibiotic consumption,¹⁹ information related to evaluating the quality of prescribing should be extracted from ASP guidelines and publications containing local and international experience.^15,20,43,44

First, a consensus in defining the quality of prescribing and its measure is needed. Performance measurement is an integral part of the quality improvement cycle, and a number of indicators for appropriate antimicrobial prescribing have been reported in the literature. These are predominantly process indicators such as the rates of adherence to the guidelines, the appropriateness and timeliness of therapy for a given infection, the advice acceptance rates and the rates of concordance with susceptibility reporting.¹⁷ In most studies, after the evaluation of a prescription, the treatment is classified into categories according to their appropriateness. Sometimes only the suitability of the treatment in regard to the microbiological results is considered. There is an established classification by Kunin et al.⁴⁵ that has recently been adapted by Apisarnthanarak et al.⁴⁶ Categories of judgement of antibiotics are shown in Table 1. Categories I and II indicate appropriate therapy; categories III–V indicate that there was some major deficiency in the choice or use of antibiotics by the physician managing the problem and that antibiotic use was inappropriate. Even between these two authors, the categories of judgement were not strictly the same (e.g., the item “duration” was incorporated into category IV by Apisarnthanarak et al.) As mentioned in the methodologies section, the most appropriate method of antimicrobial prescribing should imply that the indication and, if needed, the choice of drug, the route, the dosage, the frequency and the duration of administration have been rigorously determined.¹ All these elements should be specifically required and evaluated regarding the reference comparator (the local guideline when available) to make a judgement of appropriateness (some authors prefer the term “adequacy” when all these elements are considered). Evaluating the quality of antimicrobial prescribing considering all these elements may be the way to evaluate the effect of interventions on the different aspects of antibiotic prescribing (e.g., the decision to prescribe an antibiotic for prophylaxis or treatment, the antibiotic regimen considering the drug, dose and interval, the duration of prophylaxis or the treatment and timing). It may also be useful in comparing the effectiveness of combinations of interventions with those of a single intervention, and may help to evaluate the effect of increasing appropriate, evidence-based antibiotic prescribing on patient outcomes and healthcare costs.¹

Table 1.

Categories of judgement of antibiotics^45,46

I. Agree with the use of antimicrobial therapy/prophylaxis; the prescription is appropriate

II. Agree with the use of antimicrobial therapy/prophylaxis. A potentially fatal bacterial infection cannot be ruled out, or prophylaxis is probably appropriate, although advantages derived remained controversial

III. Agree with the use of antimicrobial therapy/prophylaxis, but a different (usually less expensive or less toxic) antimicrobial is preferred

IV. Agree with the use of antimicrobial therapy/prophylaxis, but a modified dose or duration is recommended (duration incorporated by Apisarnthanarak et al.)

V. Disagree with the use of antimicrobial therapy/prophylaxis; administration is unjustified

Second, there is a major problem arising from the lack of standardisation of studies designed to evaluate the quality of antibiotic prescribing. Often, parameters of varying quality are evaluated, complicating the comparison of results between studies. The primary result variable when evaluating the quality of prescribing should be the prescription adequacy rate, considering all the elements mentioned above. Clinical and microbiological data should also be collected, as they are needed to compare the prescription with the chosen gold standard.⁴⁷ As explained in the methodologies section, the measurement of these rates may occur as an intermittent audit (PPSs) or as ongoing continuous surveillance. Both designs present advantages and disadvantages. In the first case, PPSs can assess the percentage of appropriate or inappropriate prescriptions based on the various criteria used. These studies are simple to perform and analyse, require fewer resources and can be performed repeatedly. In the second case, an ongoing continuous surveillance allows for the measurement of days of inadequate or unnecessary treatment; these studies are more complex and require more resources.¹⁵ The biggest example of the standardisation of the evaluation of prescribing is the ESAC project, in which all 27 EU member states and another seven countries participated. ESAC carried out three hospital PPSs on antimicrobial use. As mentioned previously, they developed a standardised methodology for data collection and online data submission with feedback capability using a dedicated web-based tool. The objectives of the ESAC hospital PPSs were to determine the feasibility of a pan-European survey and identify targets for quality improvement.¹⁸ The number of participating hospitals increased from 20 to 172 between 2006 and 2009. The items assessed varied over time because of the increasing number of participant hospitals. The information available increased as more hospitals were included, which made it more difficult to compare the data between them. Guideline compliance (51%) was only introduced in the 2009 PPS, replacing “sample for culture and sensitivity” (<50% in 2006 and 2008) since samples were either not taken or no information was available for the majority (>50%) of patients.²⁹ This experience allowed for the identification of global targets for intervention, but the more countries that participated, the more patient information was lost.

Third, there is little information about the feasibility of the evaluation of antimicrobial prescribing to evaluate the impact of ASPs. As is frequent in healthcare epidemiology research, many interventions have been evaluated over varying time periods with various outcomes. A Cochrane review including only 10% of published ASP studies highlighted the difficulty in assessing studies with interventions that alone, or in combination, were effective in improving antibiotic prescribing to hospital inpatients. The authors suggested that future studies should be carefully designed to minimise the issues encountered in their evaluation.¹ A limiting of indicators to evaluate interventions to the detriment of others may facilitate a comparison among studies. Although randomised controlled trials, especially those involving multiple sites, are considered the highest level of evidence to evaluate any intervention, resource and logistical restraints limit the use of this design. Interrupted time series as repeated PPSs have become a more common, practical approach, but may require 2 years of data before and after the intervention to prevent external secular trends unrelated to the intervention from biasing the results.⁴⁸ This design provides two other salient pieces of information not found with controlled trials, assuming a sufficient number of data points: trends before the intervention and the timing and persistence of post-intervention trends.

Conclusion

The evaluation of the quality of antimicrobial prescribing should be the first step in identifying the target for quality improvement, which may be coordinated and performed in the context of ASPs. To evaluate the quality of antimicrobial prescribing, the suitability rate (adequacy) should be estimated considering the indication, the timing, the correct choice of drug, the dosing, the duration, the route of administration and the time at which to switch to oral therapy. These items may be evaluated in comparison with a gold standard, (a local adapted guideline) when possible, considering clinical and microbiological data. The evaluation should be developed as a continuous surveillance or as repeated PPSs according to the resources available to each centre. More multicentre studies using a standardised approach to evaluate the quality of antimicrobial prescribing, considering the items and designs outlined previously, are needed to plan and evaluate the best practices for ASPs and to reduce the confounding and bias seen in most reports.

Conflicts of interest

The authors declare that they have no conflicts of interest.