Study of the agreement between clinical opinion of rheumatologists regarding the need for immunosuppression and the result of renal biopsy in adult patients with lupus nephritis

Abstract

Introduction

In patients with lupus nephritis it is necessary to define the need for immunosuppressive therapy according to histological class observed in renal biopsy.

Objective

To evaluate the agreement between the opinions of six independent clinical rheumatologists regarding the need for immunosuppression and the result of renal biopsy in patients with lupus nephritis.

Materials and methods

A cross-sectional study on the agreement between a diagnostic test in adult patients with systemic lupus erythematosus. Each rheumatologist predicted the outcome of the biopsy. In order to evaluate the agreement, a dichotomous qualitative outcome was defined and was considered zero if it was not necessary to add a cytotoxic (classes I, II and VI), and otherwise was (classes III, IV, V or combinations). The percentage agreement and kappa statistics with a confidence interval of 95% was measured.

Results

Information was collected on 34 patients, with a total of 204 predictions made by 6 rheumatology interns. Rheumatologists were correct in their clinical impression in 180 cases (88.2% concordance rate, overall kappa of 0.62 (95% CI = 0.48–0.76)). Of the 204 scenarios generated, 162 corresponded to proliferative forms of lupus nephritis, for which the rheumatologists anticipated the need for immunosuppression in 153 and failed to treat in 9 cases (5.5%, or about 1 in 18).

Conclusions

The clinical opinion of rheumatologist is quite successful in defining the need for immunosuppression. In general, expert opinion could eventually be offered as an alternative choice to renal biopsy for the patient.

Resumen

Introducción

En pacientes con nefritis lúpica es necesario definir la necesidad de inmunosupresores de acuerdo con la clase histológica observada en la biopsia renal.

Objetivo

Evaluar la concordancia entre la opinión clínica de 6 reumatólogos independientes con respecto a la necesidad de inmunosupresión y el resultado de la biopsia renal en pacientes con nefritis lúpica.

Materiales y métodos

Estudio de corte transversal de concordancia de una prueba diagnóstica. Se incluyeron pacientes adultos con lupus eritematoso sistémico. Cada reumatólogo intentaba predecir el resultado de la biopsia. Para evaluar la concordancia se consideró un desenlace cualitativo dicotómico así: cero si no era necesario adicionar citotóxico (clases I, II o VI), y uno en caso contrario (clases III, IV, V o combinaciones). Se midió el porcentaje de acuerdo y el estadístico kappa con intervalo de confianza del 95%.

Resultados

Se recolectó información de 34 pacientes. Participaron 6 internistas reumatólogos para un total de 204 predicciones. Los reumatólogos acertaron en su impresión clínica en 180 ocasiones (porcentaje de concordancia 88.2%, kappa global de 0.62 (IC95% = 0.48-0.76)). De los 204 escenarios generados 162 correspondían a formas proliferativas, de los cuales los reumatólogos anticiparon la necesidad de inmunosupresión en 153 y dejaron de tratar en 9 ocasiones (5.5%, o uno de cada 18, aproximadamente).

Conclusiones

La opinión clínica del reumatólogo es bastante acertada para definir la necesidad de inmunosupresión. En general, la opinión del experto podría llegar a ofrecerse como una alternativa a la biopsia renal para el paciente que así lo escoja.

Introduction

Lupus nephritis (LN) is one of the most frequent and serious complications of systemic lupus erythematosus (SLE).¹ One-third of the adults with SLE have LN at the time of the diagnosis of their disease, and up to two-thirds of patients may have this complication during the course thereof.² In a European series of 1000 patients followed-up during 10 years LN was demonstrated in 279 patients (28%).³ In the cohort of the Latin American Group for the Study of Lupus – GLADEL, 51% of patients had LN.⁴ In a Colombian multicenter cross-sectional study which included 467 patients, 51% of them had LN.⁵

Currently the international guidelines recommend to perform a renal biopsy in all patients with suspected LN,^6–8 however, there has always been controversy about the true role of the renal biopsy to guide the treatment or to define a prognosis.^9–13 The poor reliability, the costs and the complications of this invasive procedure should also be taken into account.^14–19 The critical point when choosing the treatment for a patient with LN is to determine whether or not it is a proliferative form that indicates the addition of immunosuppression with cytotoxic agents. Both the American and the European guidelines suggest the same treatment scheme for the forms of LN classes III, IV, III + V and IV + V, which can be done with cyclophosphamide or mycophenolate, or the change to the other in case that there is not response with the first one.^7,8 For the pure form V there is some preference for starting with mycophenolate; however, cyclophosphamide is also an option in this scenario and actually this class of LN is rare in its pure form since most of the times is found combined with a class III or IV proliferative form.²⁰

Although the experts also recommend the renal biopsy to determine vascular and interstitial changes or histological activity and chronicity scores, recommendations regarding this information are not found in the guidelines for treatment. With respect to the prognosis in LN, several studies have shown that the contribution of the biopsy appears to be marginal since the main determinants of the renal outcomes are the proteinuria and the renal function at the time of the onset.^11,21–23 In our daily practice we have also observed that adherence, often affected by the supply of drugs by the insurance companies, appears to be one of the most critical factors that define the prognosis of the individual patient.

Since the main information of the renal biopsy consists in differentiating the classes of LN that need cytotoxic agents from those that do not in order to guide the treatment, we wonder if, with the clinical and laboratory information available before the biopsy, the rheumatologist would be able to obtain the same information and to approach the need for immunosuppression of the patient.

Our objectives were to determine the agreement between the clinical opinion of the rheumatologists and the final decision for immunosuppression based on the result of the renal biopsy in patients with LN, to determine the agreement between the histological class of LN suspected by the rheumatologists and the one finally observed in the renal biopsy considering 3 scenarios of clinical interest (classes I or II versus classes III, IV, III + V or IV + V versus class V), and finally to quantify the degree of empirical approximation of the rheumatologists to the activity and chronicity scores reported in the renal biopsy.

Materials and methods

Cross-sectional study on the agreement of a diagnostic test. We included adult patients, treated during the year 2014 at the San Vicente Foundation University Hospital of the city of Medellin, Colombia, with SLE²⁴ and suspected LN to whom a renal biopsy was requested. In this hospital every patient with SLE and possible LN is evaluated and treated by the Service of Rheumatology, which determines if a renal biopsy is required. At the time of requesting the biopsy was sent via email a brief clinical summary of each patient which included the age, gender, race, time of evolution of the SLE and symptoms attributable to the LN, previous treatments, presence of edemas, need for dialysis, blood pressure, BUN, creatinine, potassium, blood count, albumin, proteinuria in 24 h, urinalysis with sediment, anti-DNA, antiphospholipid profile, complement C3 and C4, SLEDAI score and the most important extrarenal manifestations. Each rheumatologist was requested to inform what he thought it would be the result of the renal biopsy regarding the class and the activity and chronicity scores; and assuming that the renal biopsy could not be done which would be the treatment that he would choose for this patient according with the renal manifestations. For convenience were included the 5 internist rheumatologists who make up the group of research in rheumatology of the University of Antioquia, in addition it was included a sixth internist rheumatologist who was trained and works in another different city to assess the consistency of the results.

The replies of the rheumatologists were independent, and they had to be available before knowing the final result of the biopsy. One of the authors received the mails with the answers and tabulated the information in an Excel spreadsheet. For the description of the population the medians and interquartile range were reported in the case of quantitative variables and the absolute frequency and percentage in the case of qualitative variables. To evaluate the agreement between the initial opinion of the rheumatologist and the final decision for cytotoxic treatment, based on the result of the biopsy, a dichotomous qualitative outcome was considered as follows: zero if it was not necessary to add a cytotoxic agent (cyclophosphamide or mycophenolate) because it was a LN class I, II and VI, and otherwise, one

(LN classes III, IV, III + V, IV + V, or V). We measured the percentage of agreement and Cohen's kappa statistic globally and for each of the participant rheumatologists. The agreement was also analyzed considering the class V or pure membranous as an outcome apart from the group of the proliferative (LN classes III, IV, III + V, IV + V). Finally, we measured the percentage of agreement between the activity and chronicity indexes predicted by the rheumatologists and those reported later in the biopsies, for which we accepted as accurate the answers approximated in 2 points more or less in the activity index and in one point more or less in the chronicity index. The results are shown with their respective 95% confidence intervals.

The study did not imply any direct contact with the patient and did not cause any modification of the usual treatment carried out by the treating team based on local protocols and international guidelines, so it was considered that it did not represent any risk for the included patients and there was no need to request informed consent. The information conveyed by mail and stored in the database was handled confidentially respecting the patients’ privacy. The main author generated the original idea, wrote the first draft, managed the database and was in charge of the analyses; all authors participated in the generation of data, recollection of the information, discussion of the results and review and approval of the final text.

Results

Information from 34 patients older than 18 years with SLE and clinical diagnosis of LN was collected. All the invited rheumatologists accepted to participate. The 6 specialists (5 from Medellin and one from Bogota) work in university hospitals. Four of them were trained in the University of Antioquia in Medellin and the other 2 in the National University of Colombia in Bogota. Three of the rheumatologists have less than 5 years of experience, 2 of them between 5 and 10 years, and the sixth more than 20 years. The response rate of the participant rheumatologists was 100%. The clinical, laboratory and histological characteristics of the included patients are shown in Table 1.

Table 1.

Description of the population.

Variable	All patients (n = 34)	No need for cytotoxic agents (n = 7)	Need for cytotoxic agents (n = 27)	p	p
Years of age	22 (18.5–30)	32 (22–35.75)	22 (18.25–27.75)	0.333	0.333
Female gender	28 (82%)	6 (86%)	22 (81.5%)	0.793	0.793
Mixed race	30 (88%)	6 (86%)	24 (89%)	0.816	0.816
Time of evolution of SLE months	6 (2–14)	6 (1–6)	5 (2–25.75)	0.448	0.448
Time of evolution of LN months	2 (1–4)	6 (1–6)	2 (1–2.75)	0.347	0.347
Creatinine mg/dl	0.8 (0.7–1.1)	0.75 (0.67–0.84)	0.81 (0.77–1.1)	0.281	0.281
Proteinuria mg/24 h	2421 (1155–4338)	1073 (667–1160)	2576 (1248–5041)	0.053	0.053
Complement C3 mg/dl	39 (31.5–67)	57 (47–96)	36 (29.5–65)	0.076	0.076
Complement C4 mg/dl	5.6 (3.6–9.4)	8 (6–15)	5 (3–8)	0.092	0.092
Anti-DNA titer	80 (20–240)	40 (0–130)	80 (25–320)	0.230	0.230
Hematuria (>5 HPF)	20 (59%)	1 (14%)	19 (70%)	0.007	0.007
SLEDAI	18 (12–21.5)	12 (12–17.25)	18.5 (12.5–22)	0.226	0.226
Histological class
I	0	II: 7	III: 4
II	7 (20.6%)		III + V:4
III	4 (11.8%)		IV: 14
III + V	4 (11.8%)		IV + V:1
IV	14 (41.2%)		V:4
IV + V	1 (3%)
V	4 (11.8%)
VI	0
Activity index (0–24)		NA	6 (4–10)
Chronicity index (0–12)		NA	0 (0–1)

HPF: high power field; SLE: systemic lupus erythematosus; NA: not applicable; LN: lupus nephritis; SLEDAI: Systemic Lupus Erythematosus Disease Activity Index.

Each rheumatologist analyzed the 34 different cases independently, for a total of 204 clinical predictions. The rheumatologists were right in their clinical impression regarding whether it was necessary or not to treat the patient with cytotoxic agents in 180 of the 204 predictions, corresponding to a percentage of agreement of 88.2% and a global kappa index of 0.62 (95% CI = 0.48–0.76), which is interpreted as a considerable concordance.

Of the 204 scenarios generated, 162 corresponded to proliferative forms of which the rheumatologists anticipated the need for immunosuppression in 153 and did not treat on 9 occasions (5.5%, or approximately one out of every 18). Of the remaining 42 scenarios that would not require addition of cytotoxic agents, according to the final report of the biopsy, the clinicians were wrong on 15 occasions suggesting that it was necessary to use immunosuppression (35.7%, or one out of every 3, approximately). Assuming the renal biopsy as a reference standard to determine whether or not is required to treat with cytotoxic agents, the clinical global impression of this group of rheumatologists, based on routine laboratory and clinical data, would reach the same information with a sensitivity of 94.4% (95% CI = 90.6–98.3), specificity of 64.3% (95% CI = 48.6–80), positive predictive value of 91.1% (86.5–95.7) and negative predictive value of 75% (61–89) (Table 2).

Table 2.

Table of 2 × 2 between the prediction of the rheumatologists and the result of the renal biopsy regarding the need for using cytotoxic agents.

Requires cytotoxic agent according to the renal biopsy	Does not require cytotoxic agent according to the biopsy
Require cytotoxic agent according to clinical prediction
153	15
Do not require cytotoxic agent according to clinical prediction
9	27
162	42

In Table 3 are shown the results grouped and discriminated for each of the participating rheumatologists. When considering in isolation the class V or pure membranous nephritis and evaluating the agreement according to 3 possible scenarios: classes I, II or VI, versus classes III, IV, III + V or IV + V, versus class V, the global kappa index was 0.625 (95% CI = 0.44–0.81), also showing a considerable concordance.

Table 3.

Agreement between the clinical opinion of the rheumatologists and the renal biopsy regarding the need for immunosuppression.

Rheumatologist	Percentage of agreement	Kappa index
A1	94	0.8
A2	88	0.6
A3	88	0.64
A4	88	0.64
A5	82	0.4
A6	88	0.64
Global	88.2	0.62 (0.48–0.76)

For the proliferative forms (classes III, IV, III + V or IV + V) the prediction of each rheumatologist regarding the indexes of activity and chronicity was also compared with those informed finally in the biopsy. It was considered an accurate response up to 2 points of difference in the activity index (the scales ranges from 0 to 24) and up to one point of difference in the chronicity index (the scale ranges from 0 to 12). The rheumatologists were right in 45.5% of the times regarding the activity score of the proliferative forms with a maximum error of 2 points, and in 50% of the times in the chronicity score with a maximum error of one point.

Discussion

In our study we show that 6 adult rheumatologists, using only clinical and laboratory information, were able to accurately predict the need or not for immunosuppression with cytotoxic agents defined by the result of the renal biopsy in patients with SLE and suspected LN in 88.2% of the occasions. We observed a high sensitivity of the clinical judgment to determine the necessity for using cytotoxic agents in patients with proliferative forms of LN (only one of every 18 patients who need immunosuppression would be left untreated by mistake). On the other hand, the specificity of the clinical judgment is only just moderate (64.3%) and for this reason approximately one out of 3 patients with non-proliferative nephritis would receive cytotoxic agents unnecessarily.

There have been reported previously at least 2 works evaluating the response to treatment of patients with LN based on clinical and laboratory information without performing renal biopsy.^12,13 In both cases the patients without a renal biopsy who were treated based on the clinical suspicion of a proliferative form of LN showed a clinical behavior similar to those treated in accordance with the biopsy. Other works have attempted to quantify the possible clinical-pathological correlation in LN trying to predict the type of nephritis and guide the treatment when a renal biopsy is not available.²⁵

Some problems of our study are that it was conducted in a high complexity hospital, which generates a reference bias, so that the proportion of patients with severe forms of SLE and nephritis was high and possibly magnifies the diagnostic performance of the clinical impression regarding the need for immunosuppression. The participants were internist rheumatologists with at least 2 years of experience and the cases came from a third level center, so unless there is more information our data cannot be extrapolated to primary care centers, pediatric patients or scenarios led exclusively by internists or general practitioners. It is noteworthy, however, that in our study there were not differences in the results obtained between rheumatologists from different training centers, practice centers or with different years of clinical experience.

An additional problem would be the possible unnecessary use of immunosuppressant agents when they are not indicated. However, this scenario is less frequent because in all series, including our patients, the proliferative forms always predominate between 80% and 90%. It would be required an evaluation of the clinical impact and the additional costs of the overtreatment of this small group of patients. Interestingly, in 2 cases in which the clinicians were apparently wrong predicting the use of cytotoxic agents for patients with presumed class II nephritis, we realized that the final clinical decision to use immunosuppression dismissing the result of the biopsy and the evolution of the renal function suggested that the initial predictions of the rheumatologists could be more accurate that the renal biopsy itself, which is a diagnostic aid of modest reliability.^14–16 A gold standard more appropriate than the result of the renal biopsy could be the final decision of an expert rheumatologist coupled with the long-term clinical evolution.

Although the current guidelines for treatment of LN indicate the performance of a renal biopsy for all patients with suspicion thereof, we believe that the clinical opinion of a rheumatologist is quite accurate to define the need or not for immunosuppression, especially in contexts where proliferative forms that require the addition of cytotoxic agents predominate. Although more studies that corroborate our findings are required, the treatment based on the clinical judgment of the rheumatologist could be a valid option in periods of the year of in places where there is no availability of a renal biopsy, in situations in which the risks of the biopsy seem to be too high, or to support the early initiation of a treatment while the renal biopsy is obtained. In general, the opinion of the expert could eventually be offered as an alternative to renal biopsy for the patient who chooses it.

Ethical disclosures

Protection of human and animals subjects

The authors declare that no experiments were performed on human beings or animals for this research.

Confidentiality of data

The authors state that patient data do not appear in this article.

Right to privacy and informed consent

The authors state that patient data do not appear in this article.

Funding

There was no funding to carry out this work.

Conflict of interest

None of the authors reported conflict of interest relevant for the execution and publication of this work.