Introduction

Top Abstract Introduction Methods Results Discussion References

Patients with arteriosclerosis, particularly those with recent carotid territory stroke or transient ischaemic attack who might benefit from carotid endarterectomy, may be investigated with conventional angiography, ultrasonography, or magnetic resonance angiography. ^{1 2} Magnetic resonance angiography was introduced in the 1980s. The technology continues to evolve rapidly, and three main approaches are currently used: two dimensional time of flight methods, three dimensional time of flight methods, and three dimensional contrast enhanced techniques. Time of flight (or "inflow") methods are non-invasive and emphasise flowing blood because a different signal is given by material that has moved into the imaging volume. Contrast enhanced methods use gadolinium based contrast materials that are better tolerated than the iodinated media used for conventional angiography. Contrast enhancement overcomes some of the drawbacks of time of flight techniques, especially signal loss near stenoses. A greater volume of the body can be imaged in a shorter time, allowing evaluation of the vasculature from the aortic arch to the circle of Willis.

The use of magnetic resonance angiography and ultrasonography has expanded rapidly in recent years, and the use of conventional angiography has declined. Although conventional catheter angiography remains the definitive imaging technique, it is an invasive procedure for which the patient must be admitted to hospital, it involves the use of ionising radiation, and when used in the carotid circulation it is accompanied by serious complications, including a 0.5-2% risk of stroke.³ Magnetic resonance angiography may be more acceptable to patients and may be of particular use in patients not suitable for conventional angiographyfor example, patients with an allergy to iodinated contrast medium, frail and elderly patients, and patients with severe peripheral vascular disease. These potential benefits may be offset by poor performance as a diagnostic test.

This systematic review examines the evidence on the performance of magnetic resonance angiography in evaluating patients with recently symptomatic internal carotid artery stenosis. The North American symptomatic carotid endarterectomy trial (NASCET) and the European carotid surgery trial (ECST) found a clear benefit of surgery in patients with recently symptomatic stenoses of 70-99% as measured by conventional angiography with the NASCET criteria. ^{4 5} We therefore evaluated the evidence on the diagnostic performance of magnetic resonance angiography in comparison with conventional angiography at this threshold. NASCET and ECST found a smaller benefit of surgery in patients with symptomatic 50-99% stenosis; ECST also showed a clear downward trend in the benefit of surgery for stenoses less than 70%.⁵ The benefit in this group also depended on the age and sex of the patients. However, a recent Cochrane review concluded that surgery was beneficial for patients with 50-69% stenosis,⁶ so we also evaluated the evidence on the diagnostic performance of magnetic resonance angiography when these more moderate stenoses were included (that is, for patients with 50-99% stenosis).

For each diagnostic threshold, we aimed to answer the question "What are the sensitivity and specificity of magnetic resonance angiography, in comparison with the gold standard of conventional angiography, in distinguishing severely stenosed arteries suitable for surgery, from either occluded or minimally stenosed arteries?"

	Methods

Top Abstract Introduction Methods Results Discussion References

We sought evaluations of the performance and effectiveness of magnetic resonance angiography in carotid artery stenosis published between January 1990 and December 1999. This review was part of a larger one, in which we also sought articles discussing the use of magnetic resonance angiography in peripheral vessels. We searched the electronic databases Medline, Embase, HealthSTAR, Science Citation Index, Index to Scientific and Technical Proceedings, Cochrane Library, Inside (British Library), and Online Computer Library Centre by using the keywords magnetic resonance angiography and MRA (or accepted synonyms and abbreviations).⁷ We also conducted a hand search of 10 key journals in the fields of imaging and vascular disease. We examined the reference lists of all articles retrieved from the above sources.

We compiled a list of unique articles by eliminating duplicates identified from more than one database. We applied preliminary exclusion criteria within the bibliographic databases, and one reviewer then applied these and a second list of exclusion criteria (table 1). We retrieved the remaining articles, and two reviewers applied final inclusion criteria (table 2) independently, in a hierarchical manner from A to H. Inclusion criteria A-C and F identified articles that described robust primary research comparing the diagnostic performance of magnetic resonance angiography against conventional angiography (A) in terms of sensitivity and specificity (B) and at the specific thresholds of stenosis described in our introduction (C, F). Criterion D avoids results from the same study being included twice. The remaining criteria are indicators of the validity of the results. The results from articles that did not satisfy criterion E may have had verification bias; if the study group included asymptomatic participants (G) then patient cohort bias may have been present; and disease progression bias could result from a large time period between magnetic resonance angiography and the gold standard (H).⁸ Checklists covering study design, study group characteristics, technical details, and potential biases in executing the study were completed for each article satisfying the inclusion criteria A to D. 

We included articles satisfying criteria A-D in the quantitative meta-analysis. Primary data (true positive, true negative, false positive, and false negative values) were extracted independently by two reviewers, and agreed by consensus, for the following clinical decision thresholds measured by using the NASCET criteria or similar⁴: 70-99% stenosed vessel (suitable for carotid endarterectomy) versus 0-69% stenosed or 100% occluded vessel (not suitable for carotid endarterectomy); 50-99% stenosed vessel (suitable for carotid endarterectomy) versus 0-49% stenosed or 100% occluded vessel (not suitable for carotid endarterectomy). Results derived with the very different ECST criteria were excluded from the meta-analysis.⁵

We plotted results from the independent studies on sensitivity versus 1specificity axes to illustrate the scatter of results. We then combined the sensitivity and specificity results for independent studies into a summary receiver operating characteristic curve by using the method recommended by the Cochrane Screening and Diagnostic Tests Methods Working Group for meta-analysis of diagnostic test results.^9-12 We used a logistic function to transform the sensitivity and specificity data. To prevent undefined values on logistic transformation, we applied a contingency correction of 0.01 to all true positive, false negative, true negative, and false positive values if any one was zero. We fitted a straight line to the transformed results by using an unweighted least squares fit, performed an inverse transformation on the fitted line, and plotted the result as a summary receiver operating characteristic curve.

The summary receiver operating characteristic curve is an excellent graphical summary, but for comparison purposes we calculated a further statisticQ* and its 95% confidence interval.⁹ Q* is the point on the summary receiver operating characteristic curve where sensitivity and specificity have the same value; it represents the maximal joint sensitivity and specificity. Q* is a good summary value in this application as there is no particular disadvantage to sensitivity and specificity being equal: patients with false positive results needlessly undergo the risks of surgery, but patients with false negative results are denied the benefits of surgery. Q* would not be a good choice for describing a screening test, where the aim is to have no false negatives but a few false positives are acceptable.

We then performed a multiple linear regression analysis at the line fitting phase of the summary receiver operating characteristic analysis, to determine if any of five covariates had a significant effect, at the 95% level, on the fitted summary receiver operating characteristic curve.¹³ The five covariates were technique of magnetic resonance angiography, inclusion of articles that did not satisfy the inclusion criteria E-H, the risk of test or diagnostic review bias, the risk of verification bias, and the risk of withdrawal bias.¹⁴ As five variables were tested, the P value corresponding to 95% significance was 0.01.¹⁵ Statistical analyses were performed with SPSS for Windows (release 9.0.0).

	Results

Top Abstract Introduction Methods Results Discussion References

We identified 16 185 articles with the initial broad search strategy. After we had removed duplicates, 7183 unique articles remained. The exclusion criteria reduced the number to 206 (table 1), and of 126 candidate articles on carotid artery stenosis 26 satisfied the inclusion criteria A to D.^16-41 Only eight articles satisfied all the inclusion criteria A to H. ^{17 20 22 23 25 35 36 39} Six of these eight papers included results for the 70-99% threshold, ^{17 22 23 25 35 36} and two included results for the 50-99% threshold. ^{20 39}

View larger version (23K): [in this window] [in a new window]   Fig 1.   Summary receiver operating characteristic curve for magnetic resonance angiography: 70-99% stenosis is a positive result, and 0-69% stenosis and 100% occlusion are negative results. Straight line shows 95% confidence interval of Q*. Enlargement of region of sensitivity and specificity 90-100% shown for clarity

For the diagnosis of 70-99% stenosis (fig 1 ), four sets of results obtained by using contrast enhanced techniques were included, ^{26 32 34 36} together with 11 sets of results (from nine articles ^{19 23 25 27 29 34 35 37 41} ) obtained by using three dimensional time of flight and 10 sets of results^16-38 obtained by using two dimensional time of flight techniques. Q* was 99% (95% confidence interval 98% to 100%). None of the variables tested in the multiple linear regression, including magnetic resonance angiography technique, was significant at the 95% level.

For the diagnosis of 50-99% stenosis (fig 2), no results obtained by using contrast enhanced techniques were included. Results from four studies using three dimensional time of flight techniques were included, ^{21 29 39 40} together with six sets of results (from five articles ^{20 24 28 31 33} ) obtained by using two dimensional time of flight. Q* was 90% (81% to 99%). None of the variables tested in the multiple linear regression was significant at the 95% level.

View larger version (19K): [in this window] [in a new window]   Fig 2.   Summary receiver operating characteristic curve for magnetic resonance angiography: 50-99% stenosis is a positive result, and 0-49% stenosis and 100% occlusion are negative results. Straight line shows 95% confidence interval of Q*

The patient populations of studies included in the quantitative meta-analysis were heterogeneous. Patient numbers ranged from 11 to 101 (mean=40). In all articles where sex distribution was reported most patients were men; in these articles the proportion of men ranged from 55% to 100% (mean=69%). Six articles did not report sex distribution. ^{17 22 26 29 30 35} The lower limit for age of patients ranged from 18 to 63 years (mean=43 years), and the upper limit ranged from 73 to 87 years (mean=80 years). Six articles did not report age range. ^{25 27 29 34 35 38} Eight articles stated that asymptomatic patients were included, ^{16 18 27-29 31 32 40} and 18 articles gave no information about symptoms. ^{16 17 19-26 28-30 33-37}

	Discussion

Top Abstract Introduction Methods Results Discussion References

Available evidence
Although many articles have been written about magnetic resonance angiography and carotid disease, little rigorous research has been conducted on the performance of magnetic resonance angiography in evaluating carotid artery stenosis. Small numbers of participants and inadequate details of study design mean that many studies included in this review have a potential for bias, but none of the factors tested in the multiple linear regression analysis had a significant effect on the results. Further sources of heterogeneity in patient populations (age, sex distribution, presenting symptoms) may influence patients' suitability for carotid endarterectomy, ^{5 42} and differences existed between the study groups included in the review, even among those that satisfied all the inclusion criteria.

Potential for bias in this review arose from the increasingly common practice of screening candidates for magnetic resonance angiography by using ultrasonography and proceeding only if this shows a stenosis greater than a predetermined value. Some of the results used in this review were probably obtained in groups of patients preselected in this way, but we were not able to determine this from the articles.

Ultimately, to show effectiveness, we need evidence of the impact of magnetic resonance angiography on clinical decision making and on outcomes in patients. ^{43 44} Comparative studies, with patients randomised to magnetic resonance angiography or conventional angiography, could be used to gather evidence of the impact on decision making. In this review, however, we found no studies that compared magnetic resonance angiography with conventional angiography for surgical decision making or outcomes. Although high quality research is needed, especially full cost effectiveness studies and evaluation of new magnetic resonance angiography techniques, further large trials involving conventional angiography are unlikely to be undertaken. Modelling, using existing evidence, may be the way forward.

Degree of stenosis
To be able to determine whether a patient is a suitable candidate for carotid endarterectomy, a diagnostic test must distinguish severely (>70%) stenosed or moderately (>50%) stenosed arteries (which are suitable for carotid endarterectomy) from both minimally stenosed (0-69% or 0-49%) and occluded arteries (100%), which are not suitable for carotid endarterectomy. ^{4 5} Candidate articles often failed to assess magnetic resonance angiography in these terms. We excluded 20 articles from the review because they did not classify operable carotid artery stenosis as 70-99% or 50-99%. Of these, seven articles also failed to report the diagnostic performance of magnetic resonance angiography in separating severely stenosed from occluded arteries. This can be an important distinction in selection for surgery, especially if ultrasonography has indicated a small amount of flow.

Some authors have recommended that the magnitude of the likely benefit of carotid endarterectomy in cases of moderate to severe stenosis should be assessed by using more restricted bands of stenosis.⁴⁵ Future trials with restricted bands might show which stenosis thresholds, as determined by magnetic resonance angiography, are associated with surgical benefit, in addition to determining the reliability of magnetic resonance angiography in detecting patients suitable for surgery. This research might be an ideal testing ground for tracker studies that evaluate fast changing technologies.⁴⁶

Conclusion
Our review does not support the use of magnetic resonance angiography to select surgical candidates with 50-99% stenosis. The 95% confidence interval for Q* extended from 81% to 99%, and only two of the articles whose results were included in the meta-analysis satisfied the inclusion criteria related to validity. It would be advisable for users of magnetic resonance angiography to ensure that rigorous training and audit are in place, including feedback from surgeons and continuing quality control comparisons with ultrasonography.

Our results indicate that magnetic resonance angiography is very effective for detecting 70-99% stenosis as defined by conventional angiography. Q* was 99% (98% to 100%). Although there is a promising trend towards better performance from contrast enhanced methods, further research is essential as only four articles were included in this review and no significant difference was found between the results obtained by using the three main techniques.