Third generation oral contraceptives and risk of venous thrombosis: meta-analysis
=================================================================================

* Jeanet M Kemmeren
* Ale Algra
* Diederick E Grobbee

## Abstract

**Objective:** To evaluate quantitatively articles that compared effects of second and third generation oral contraceptives on risk of venous thrombosis.

**Design:** Meta-analysis.

**Studies:** Cohort and case-control studies assessing risk of venous thromboembolism among women using oral contraceptives before October 1995.

**Main outcome measures:** Pooled adjusted odds ratios calculated by a general variance based random effects method. When possible, two by two tables were extracted and combined by the Mantel-Haenszel method.

**Results:** The overall adjusted odds ratio for third versus second generation oral contraceptives was 1.7 (95% confidence interval 1.4 to 2.0; seven studies). Similar risks were found when oral contraceptives containing desogestrel or gestodene were compared with those containing levonorgestrel. Among first time users, the odds ratio for third versus second generation preparations was 3.1 (2.0 to 4.6; four studies). The odds ratio was 2.5 (1.6 to 4.1; five studies) for short term users compared with 2.0 (1.4 to 2.7; five studies) for longer term users. The odds ratio was 1.3 (1.0 to 1.7) in studies funded by the pharmaceutical industry and 2.3 (1.7 to 3.2) in other studies. Differences in age and certainty of diagnosis of venous thrombosis did not affect the results.

**Conclusions:** This meta-analysis supports the view that third generation oral contraceptives are associated with an increased risk of venous thrombosis compared with second generation oral contraceptives. The increase cannot be explained by several potential biases.

#### What is already known on this topic

**What is already known on this topic** Third generation oral contraceptives have been reported to increase the risk of venous thrombosis compared with second generation oral contraceptives

The findings have been vigorously debated, with suggestions that the results can be explained by confounding or bias, or both.

#### What this study adds

**What this study adds** Women taking third generation oral contraceptives have a 1.7-fold increased risk of venous thrombosis compared with those taking second generation oral contraceptives

Risk is highest in first time users

The biases were not large enough to account for the observed results

## Introduction

In 1995-6 increased risks of venous thrombosis were reported among women using so called third generation oral contraceptives compared with second generation products, with odds ratios ranging from 1.5 to 2.2.1–4 Other investigators suggested that confounding, bias, or both, accounted for the findings.5–8 New studies were performed, 7 9 10 and many subgroup analyses published, 6 11 12 but the debate continues.13 In 1999, Farley et al reported a meta-analysis and found an increased risk of 1.9 (95% confidence interval 1.5 to 2.2).14 However, their aim was to review qualitatively the arguments claiming that the difference in risk for different oral contraceptives is not real. They did not formally consider characteristics of the included studies that might affect their results. In the present meta-analysis we quantified these aspects.

## Methods

We searched Medline for articles published from October 1995 to December 2000 using the terms third generation oral contraceptives, desogestrel, and gestodene combined with thromboembolism and venous thrombosis. We retrieved additional references from reviews, other articles of interest, and experts in the field. We reviewed all English language articles containing original data on third generation oral contraceptives and venous thrombosis. Inclusion criteria were (*a)* cohort or case-control design, (*b*) cases defined as women with venous thrombosis or thromboembolism, (*c*) sufficient data provided to reconstruct two by two tables or determine relative risk and confidence intervals, (*d*) data collected before November 1995, and (*e*) data collected in Western countries. We chose October 1995 as the end date because at that time four studies were published relating third generation oral contraceptives to venous thrombosis.1–4 Consequently, changes in prescription of oral contraceptives may have potentially affected the results of later studies. To avoid heterogeneity, we included studies in only Western countries.

We systematically abstracted data, resolving ambiguous information through discussion between us. Firstly, we analysed the results of studies that compared the risk of venous thrombosis between third and second generation oral contraceptive users. To assess the influence of different definitions of second and third generation oral contraceptives between studies, we analysed oral contraceptives with specified and unspecified progestagen components separately. Next, we did stratified analyses to explore the patterns of risks in subgroups that may be less or more susceptible to bias. Stratification factors were first time users, age (<25 *v* 25 years), duration of oral contraceptive use (<1 year *v* 1 year), confirmed cases, and source of funding (non-industry versus industry sponsored studies explicitly mentioned in the acknowledgement). Cases were considered confirmed when venous thrombosis was objectively diagnosed (by ultrasound examination, plethysmography, or venography). A study was included only once if there were multiple publications. We also did an additional analysis including studies that did not meet the inclusion criteria to determine their effect on the pooled odds ratio.

Some studies reported only frequencies, whereas others reported only unadjusted or adjusted odds ratios. We therefore performed an overall analysis based on the adjusted odds ratios and on the two by two tables separately. We calculated adjusted odds ratios by pooling adjusted odds ratios from individual studies using a general variance based random effects method, weighting individual study results by the inverse of their variance.15 Odds ratios accurately estimate relative risks when risks of disease are small, and we therefore used the same method for case-control and cohort studies.16 We tested homogeneity between studies—that is, the hypothesis that the differences between the reported odds ratios were due only to random error around the true odds ratio. Results were considered heterogeneous when homogeneity was unlikely (P<0.10). To determine the stability of the overall risk estimate, we did a sensitivity analysis in which each study was successively eliminated.

![Fig 1](http://www.bmj.com/https://www.bmj.com/content/bmj/323/7305/131/F1.medium.gif)

[Fig 1](http://www.bmj.com/content/323/7305/131/F1)

Fig 1 
Overall odds ratios and 95% confidence intervals for venous thrombosis with different categories of oral contraceptives

If possible we also extracted or recalculated two by two tables. We combined the odds ratios from the individual studies using the Mantel-Haenszel method,15 providing a crude odds ratio. For subgroup analyses, we pooled adjusted and unadjusted results because of the limited number of studies with subgroup data, resulting in a pooled odds ratio.

## Results

Of 114 studies identified, 27 were considered potentially relevant. 1–7 9–12 17–32 Ten studies, comprising nine case-control (table 1) and three cohort studies (table 2), examined use of oral contraceptives and risk of venous thrombosis. Three studies provided additional analyses on earlier reported results, 6 11 17 and were included in our stratified analysis. Fourteen studies failed to meet one or more inclusion criteria, because they did not contain original data,18–24 included patients after October 1995, 12 25–27 or compared third generation oral contraceptives with a combined group of first and second generation oral contraceptives.28–30

View this table:
[Table 1.](http://www.bmj.com/content/323/7305/131/T1)

Table 1. 
Case-control studies of oral contraceptives and the risk of venous thromboembolism

View this table:
[Table 2.](http://www.bmj.com/content/323/7305/131/T2)

Table 2. 
Cohort studies of oral contraceptives and risk of venous thromboembolism

### Overall analysis

The overall adjusted odds ratio for third versus second generation oral contraceptives for the risk of venous thrombosis was 1.7 (95% confidence interval 1.4 to 2.0), with no heterogeneity (P=0.78). 2–5 7 9 31 In a sensitivity analysis, the adjusted odds ratio varied between 1.6 and 1.8, and the 95% confidence interval never included 1. The crude odds ratio was similar to the adjusted odds ratio (crude odds ratio=1.6, 95% confidence interval 1.3 to 1.9). 1–3 4 7 9 32

The overall results were not materially dependent on definitions of oral contraceptives (fig 1). For all subgroups, crude odds ratios, based on the two by two tables were similar to adjusted odds ratios. In a sensitivity analysis comparing desogestrel with levonorgestrel containing oral contraceptives the adjusted odds ratio varied between 1.6 and 2.2, and 95% confidence intervals never included 1. For gestodene versus levonorgestrel containing oral contraceptives the adjusted odds ratio varied from 1.3 to 2.1 and 95% confidence intervals included 1 twice, and for third versus second generation oral contraceptive use without specifying the progestagen component the adjusted odds ratio ranged from 1.4 to 1.5. The lower boundary of the 95% confidence interval varied between 0.9 and 1.1 and the interval included 1 once.

### Stratified analyses

Figure 2 shows that the odds ratio for third versus second generation preparations among first time users was 3.1 (2.0 to 4.6). 4 6 10 17 The odds ratio was 2.5 (1.6 to 4.1) for short term users, 2 6 7 10 17 and 2.0 (1.4 to 2.7) for longer term users. 2 6 7 10 17 Source of funding modified the estimates: the odds ratio was 1.3 (1.0 to 1.7) in studies directly financed by pharmaceutical industries 4 5 7 32 and 2.3 (1.7 to 3.2) in other studies. 1 2 3 9 10 Differences in age and certainty of diagnosis of venous thrombosis did not affect the results, nor did excluding the cohort study by Herings et al.10

![Fig 2](http://www.bmj.com/https://www.bmj.com/content/bmj/323/7305/131/F2.medium.gif)

[Fig 2](http://www.bmj.com/content/323/7305/131/F2)

Fig 2 
Effect of patient characteristics on odds ratios or relative risks of third versus second generation oral contraceptives for venous thrombosis

### Extended studies

The odds ratio remained essentially the same when the original studies 2 5 were replaced by reports updated after October 1995. 12 25–27 Three studies compared third generation oral contraceptives with a combined group of first and second generation drugs.28–30 Adding two by two table data from these studies to the overall analysis shown in figure 1 did not change the crude odds ratio (1.6, 1.3 to 1.9).

## Discussion

Our meta-analysis shows that third generation oral contraceptives are associated with a 1.7-fold increased risk of venous thrombosis compared with second generation oral contraceptives. After stratifying by various factors and examining selected subgroups, the increased risk remained.

A meta-analysis depends on the quality of the studies included. Observational studies are susceptible to bias because other risk factors of venous thrombosis may be unbalanced across users of second and third generation oral contraceptives. We did not give quality scores to included studies because of their inherent subjectivity and potential to result in diverging summary estimates.33 However, the key elements affecting internal validity (ascertainment, diagnostic and inclusion criteria, exposure assessment, matching, and control factors),34 were listed in the tables and investigated in the stratified and sensitivity analyses.

### Quality issue

We believe three issues are important for the quality of our meta-analysis. Firstly, we assessed reliability of outcome by subgroup analysis with confirmed cases only. Secondly, we assessed appropriate adjustment for confounding by comparing adjusted and unadjusted odds ratios and by presenting stratified analyses. The presence of confounding is unlikely because the pooled crude odds ratios were almost equal to the pooled adjusted odds ratios. Source of funding modified the estimates. Some studies provided stratified data only for specific subgroups of women (for example, age in first time users). 6 10 Nevertheless, these studies were included. In addition, the sensitivity analysis showed that the overall risk estimates were stable.

Thirdly, we evaluated the quality of assessment of exposure. The definition of second and third generation oral contraceptives was not fully consistent across reports. 1 4–32 However, different definitions did not affect the results materially (fig 1). Differential recall of second versus third generation contraceptives is unlikely because most studies collected data before October 1995 or used information from medical records.

Our pooled odds ratios may be underestimates because publication bias can never be excluded. For example, the Dutch press recently reported that a drug company kept results secret that confirmed an increased risk of venous thrombosis with third generation oral contraceptives.35

Another cause of underestimation is that relative risks estimated from the original data are in general lower than those derived from matched regression analysis. However, this cannot be addressed with published data.

Over the past years, the discussion about the findings has concentrated on several potential biases, 8 14 36 37 which we recapitulate below.

### Healthy user bias

When starting oral contraceptives, women are more likely to receive one of the newer oral contraceptives, whereas older, established users tend to continue with their original brand. New users may include women genetically or otherwise predisposed to venous thrombosis, whereas long term users have shown tolerance to the drug. The predominance of use of new drugs among new oral contraceptive users would create a bias in favour of older products. If true, the difference between second and third generation oral contraceptives should attenuate when first time users are examined separately. This was possible in four studies, 4 6 10 17 and, although the definition of first time users differed between the studies, the difference in risk between third and second generation oral contraceptives was higher in this group than in all users.

### Recency of introduction bias

Related to possible healthy user bias is the potential for bias due to recency of introduction. Lewis et al argued that there is a relation between the risk of thrombosis and time since a drug's introduction to the market.21 With time, women at a high risk of venous thrombosis stop taking oral contraceptives, leaving a pool of lower risk women taking previously introduced oral contraceptives. However, if depletion of susceptible women distorts the risks for different products, the risk in young women (as a proxy for first time users) should also be analysed. The pooled odds ratio in our meta-analysis showed an increased risk for both younger and older women. Bloemenkamp et al found fourfold to sevenfold increases in risk for use of third versus second generation oral contraceptives among women aged 15-24 years—three to four times higher than the pooled odds ratio we found.3 We could not include Bloemenkamp et al's study in our meta-analysis because no data for a two by two table, relative risk, or confidence intervals were provided.

### Duration of oral contraceptive use

A different risk between second and third generation oral contraceptives may reflect lack of adjustment for duration of use. 6 7 A reanalysis of the transnational study provided an adjusted rate ratio relative to never users of around 10 in the first year of use, decreasing to around two after two years of use.6 These findings were essentially identical for second and third generation preparations. After correction for duration of use, Lidegaard et al found no significant differences between oral contraceptives with different types of progestagens.7 Our pooled odds ratio stratified by duration of use showed an increased risk for short and long term users. This was most pronounced in women with a short duration of use, again suggesting that young women who take oral contraceptives for the first time are at highest risk.

### Diagnostic suspicion and referral bias

Women using third generation oral contraceptives may be more likely to be referred, investigated, and diagnosed with venous thrombosis than users of other oral contraceptives.38 If diagnostic suspicion bias exists, it is likely that the association is diminished among women in whom the diagnosis is so obvious that no clue of oral contraceptive use is needed for diagnosis.36 However, for confirmed cases there was a similar increased risk for women taking third generation oral contraceptives as for those taking second generation formulations. Besides that, the influence of diagnostic suspicion bias would have been stronger in women with less certain diagnosis, because information on use of oral contraceptives might have led to the diagnosis.

Two studies examined risks according to certainty of diagnosis. 2 31 Although it was not possible to pool the results since one study did not report the confidence interval, the odds ratios for definite and possible cases remained essentially the same. Moreover, Bloemenkamp et al showed that patients with a clinically confirmed deep vein thrombosis more often used third generation oral contraceptives than patients with the same clinical suspicion who had no thrombosis.9 Therefore, diagnostic bias seems unlikely.

### Prescribing bias

Many authors have raised the possibility of selective prescribing of third generation oral contraceptives to high risk women.39–41 Third generation oral contraceptives may have been preferentially prescribed to women with cardiovascular risk factors because of their perceived improved safety profile over second generation oral contraceptives.42 Indeed, patterns of use are different in women with and without cardiovascular risk factors.39–41 However, risk factors screened for at first prescription of oral contraceptives were primarily risk factors for arterial diseases and not for venous thrombosis (that is, tissue damage and haemostatic abnormalities). Moreover, certain genetic markers of venous thrombosis were only recently identified and not widely known at the time women included in the studies were given their oral contraceptives. Furthermore, all studies considered only cases with a first ever venous thrombosis, thus excluding women with a history of venous thrombosis.

Another indicator for genetic predisposition is a family history of venous thrombosis. Two studies addressed this potential confounder, and both showed that increased risks for women using third generation oral contraceptives remained after adjustment. 3 9

### Switching

Accumulating side effects might lead women to switch their oral contraceptives. This might reflect an increased risk of venous thrombosis.38 One study examined the risk of venous thrombosis among women who switched contraceptives and found an odds ratio of 1.3 (0.7 to 2.4) among those who switched from second to third generation pills relative to those who switched from third to second generation pills,24 although the study recruited until 1996. Women who switched only once had an odds ratio of 1.1 (0.5 to 2.3), whereas those who switched more than once had an odds ratio of 1.8 (0.2 to 16.8). However, the power for subgroup analysis was limited, with large confidence intervals.

### Source of funding

The pooled odds ratio of studies without explicitly mentioned industry sponsoring was higher than that of studies without such support, although the increased risk was significant in both groups. Different results for industry and non-industry sponsored studies have also been reported for calcium channel antagonists and non-steroidal anti-inflammatory drugs. 43 44

### Absolute risks

To appreciate the importance of increases in relative risk, knowledge of absolute risks is required. We estimated that the excess risk for users of third generation oral contraceptives over second generation preparations was 1.5 per 10 000 woman years. This may be an underestimation, because the estimate from the study by Jick et al was confined to cases that met a very strict definition.2 Among new users the incidence is much higher (6.6 per 10 000 woman years).

Death rates from venous thrombosis are low (about 3%),45 although non-fatal events can also have serious effects. We crudely calculated that four deaths per 1 000 000 woman years could be prevented by switching from third to second generation products. Although the risks are small, they should be considered when deciding which oral contraceptive to use.

### Conclusion

Our meta-analysis supports the view that third generation oral contraceptives are associated with a 1.7-fold increased risk of venous thrombosis compared with second generation oral contraceptives. The risk is highest in first time users. Although confounding can never be excluded with certainty in observational studies, it seems that the biases that have been suggested and examined are not sufficient to account for the results.

## Acknowledgments

Contributors: JMK participated in the design, execution, and analysis of the study and writing the paper. AA and DEG initiated the study, participated in designing, analysing, and reporting the study, and supervised all aspects of the study. DEG is the guarantor.

## Footnotes

*   Funding None.

*   Competing interests JMK has worked on a study into second and third generation contraceptives sponsored by the Netherlands Thrombosis Foundation.

## References

1.  1.World Health Organization Collaborative Study of Cardiovascular Disease and Steroid Hormone Contraception.Venous thromboembolic disease and combined oral contraceptives: results of international multicentre case-control study.Lancet1995;346:1575–1582.
    
    [CrossRef](http://www.bmj.com/lookup/external-ref?access_num=10.1016/S0140-6736(95)91926-0&link_type=DOI) 
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=7500748&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 
    
    [Web of Science](http://www.bmj.com/lookup/external-ref?access_num=A1995TK48100007&link_type=ISI) 

2.  2.Jick H, Jick SS, Gurewich V, Myers MW, Vasilakis C.Risk of idiopathic cardiovascular death and nonfatal venous thromboembolism in women using oral contraceptives with differing progestagen components.Lancet1995;346:1589–1593.
    
    [CrossRef](http://www.bmj.com/lookup/external-ref?access_num=10.1016/S0140-6736(95)91928-7&link_type=DOI) 
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=7500750&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 
    
    [Web of Science](http://www.bmj.com/lookup/external-ref?access_num=A1995TK48100009&link_type=ISI) 

3.  3.Bloemenkamp KW, Rosendaal FR, Helmerhorst FM, Buller HR, Vandenbroucke JP.Enhancement by factor V Leiden mutation of risk of deep-vein thrombosis associated with oral contraceptives containing a third-generation progestagen.Lancet1995;346:1593–1596.
    
    [CrossRef](http://www.bmj.com/lookup/external-ref?access_num=10.1016/S0140-6736(95)91929-5&link_type=DOI) 
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=7500751&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 
    
    [Web of Science](http://www.bmj.com/lookup/external-ref?access_num=A1995TK48100010&link_type=ISI) 

4.  4.Spitzer WO, Lewis MA, Heinemann LA, Thorogood M, MacRae KD.Third generation oral contraceptives and risk of venous thromboembolic disorders: an international case-control study.BMJ1996;312:83–88.
    
    [Abstract/FREE Full Text](http://www.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjExOiIzMTIvNzAyMy84MyI7czo0OiJhdG9tIjtzOjIyOiIvYm1qLzMyMy83MzA1LzEzMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

5.  5.Farmer RD, Lawrenson RA, Thompson CR, Kennedy JG, Hambleton IR.Population-based study of risk of venous thromboembolism associated with various oral contraceptives.Lancet1997;349:83–88.
    
    [CrossRef](http://www.bmj.com/lookup/external-ref?access_num=10.1016/S0140-6736(96)07496-X&link_type=DOI) 
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=8996419&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 
    
    [Web of Science](http://www.bmj.com/lookup/external-ref?access_num=A1997WB80000010&link_type=ISI) 

6.  6.Suissa S, Blais L, Spitzer WO, Cusson J, Lewis M, Heinemann L.First-time use of newer oral contraceptives and the risk of venous thromboembolism.Contraception1997;56:141–146.
    
    [CrossRef](http://www.bmj.com/lookup/external-ref?access_num=10.1016/S0010-7824(97)00119-4&link_type=DOI) 
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=9347203&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 
    
    [Web of Science](http://www.bmj.com/lookup/external-ref?access_num=A1997YB09900002&link_type=ISI) 

7.  7.Lidegaard O, Edstrom B, Kreiner S.Oral contraceptives and venous thromboembolism. A case-control study.Contraception1998;57:291–301.
    
    [CrossRef](http://www.bmj.com/lookup/external-ref?access_num=10.1016/S0010-7824(98)00033-X&link_type=DOI) 
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=9673836&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 
    
    [Web of Science](http://www.bmj.com/lookup/external-ref?access_num=000074756600001&link_type=ISI) 

8.  8.Spitzer WO.The aftermath of a pill scare: regression to reassurance.Hum Reprod Update1999;5:736–745.
    
    [Abstract/FREE Full Text](http://www.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NjoiaHVtdXBkIjtzOjU6InJlc2lkIjtzOjc6IjUvNi83MzYiO3M6NDoiYXRvbSI7czoyMjoiL2Jtai8zMjMvNzMwNS8xMzEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

9.  9.Bloemenkamp KW, Rosendaal FR, Buller HR, Helmerhorst FM, Colly LP, Vandenbroucke JP.Risk of venous thrombosis with use of current low-dose oral contraceptives is not explained by diagnostic suspicion and referral bias.Arch Intern Med1999;159:65–70.
    
    [CrossRef](http://www.bmj.com/lookup/external-ref?access_num=10.1001/archinte.159.1.65&link_type=DOI) 
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=9892332&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 
    
    [Web of Science](http://www.bmj.com/lookup/external-ref?access_num=000077924400010&link_type=ISI) 

10. 10.Herings RM, Urquhart J, Leufkens HG.Venous thromboembolism among new users of different oral contraceptives.Lancet1999;354:127–128.
    
    [CrossRef](http://www.bmj.com/lookup/external-ref?access_num=10.1016/S0140-6736(99)01257-X&link_type=DOI) 
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=10408492&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 
    
    [Web of Science](http://www.bmj.com/lookup/external-ref?access_num=000081377100019&link_type=ISI) 

11. 11.Lewis MA, MacRae KD, Kuhl-Habichl D, Bruppacher R, Heinemann LA, Spitzer WO.The differential risk of oral contraceptives: the impact of full exposure history.Hum Reprod1999;14:1493–1499.
    
    [Abstract/FREE Full Text](http://www.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NjoiaHVtcmVwIjtzOjU6InJlc2lkIjtzOjk6IjE0LzYvMTQ5MyI7czo0OiJhdG9tIjtzOjIyOiIvYm1qLzMyMy83MzA1LzEzMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

12. 12.Todd J, Lawrenson R, Farmer RD, Williams TJ, Leydon GM.Venous thromboembolic disease and combined oral contraceptives: a reanalysis of the MediPlus database.Hum Reprod1999;14:1500–1505.
    
    [Abstract/FREE Full Text](http://www.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NjoiaHVtcmVwIjtzOjU6InJlc2lkIjtzOjk6IjE0LzYvMTUwMCI7czo0OiJhdG9tIjtzOjIyOiIvYm1qLzMyMy83MzA1LzEzMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

13. 13.Vandenbroucke JP, Rosing J, Bloemenkamp KW, Middeldorp S, Helmerhorst F, Bouma BN, et al.Oral contraceptives and the risk of venous thrombosis.N Engl J Med2001;344:1527–1535.
    
    [CrossRef](http://www.bmj.com/lookup/external-ref?access_num=10.1056/NEJM200105173442007&link_type=DOI) 
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=11357157&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 
    
    [Web of Science](http://www.bmj.com/lookup/external-ref?access_num=000168677200007&link_type=ISI) 

14. 14.Farley TM, Meirik O, Collins J.Cardiovascular disease and combined oral contraceptives: reviewing the evidence and balancing the risks.Hum Reprod Update1999;5:721–735.
    
    [Abstract/FREE Full Text](http://www.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NjoiaHVtdXBkIjtzOjU6InJlc2lkIjtzOjc6IjUvNi83MjEiO3M6NDoiYXRvbSI7czoyMjoiL2Jtai8zMjMvNzMwNS8xMzEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

15. 15.Petitti DB.Statistical methods in meta-analysis. In:Meta-analysis, decision analysis, and cost-effectiveness analysis.New York: Oxford University Press,1994:106–110.
    
    

16. 16.Kleinbaum D, Kupper L, Morgenstern H.Epidemiologic research.New York: Van Nostrand Reinhold,1982.
    
    

17. 17.Farley TM, Meirik O, Poulter NR, Chang CL, Marmot MG.Oral contraceptives and thrombotic diseases: impact of new epidemiological studies.Contraception1996;54:193–198.
    
    [CrossRef](http://www.bmj.com/lookup/external-ref?access_num=10.1016/S0010-7824(96)00166-7&link_type=DOI) 
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=8899262&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 
    
    [Web of Science](http://www.bmj.com/lookup/external-ref?access_num=A1996VN24300010&link_type=ISI) 

18. 18.Farmer RD, Todd JC, MacRae KD, Williams TJ, Lewis MA.Oral contraception was not associated with venous thromboembolic disease in recent study.BMJ1998;316:1090–1091.
    
    [FREE Full Text](http://www.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEzOiIzMTYvNzEzNy8xMDkwIjtzOjQ6ImF0b20iO3M6MjI6Ii9ibWovMzIzLzczMDUvMTMxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

19. 19.Farmer RD, Lawrenson RA.Oral contraceptives and venous thromboembolic disease: the findings from database studies in the United Kingdom and Germany.Am J Obstet Gynecol1998;179: S78–S86
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=9753314&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 

20. 20.Lidegaard O.Smoking and use of oral contraceptives: impact on thrombotic diseases.Am J Obstet Gynecol1999;180: S357–S363.
    
    [CrossRef](http://www.bmj.com/lookup/external-ref?access_num=10.1016/S0002-9378(99)70696-4&link_type=DOI) 
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=10368521&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 
    
    [Web of Science](http://www.bmj.com/lookup/external-ref?access_num=000081173500004&link_type=ISI) 

21. 21.Lewis MA, Heinemann LA, MacRae KD, Bruppacher R, Spitzer WO.The increased risk of venous thromboembolism and the use of third generation progestagens: role of bias in observational research.Contraception1996;54:5–13.
    
    [CrossRef](http://www.bmj.com/lookup/external-ref?access_num=10.1016/0010-7824(96)00112-6&link_type=DOI) 
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=8804801&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 
    
    [Web of Science](http://www.bmj.com/lookup/external-ref?access_num=A1996UU21000002&link_type=ISI) 

22. 22.Lawrenson R, Farmer R.Venous thromboembolism and combined oral contraceptives: does the type of progestogen make a difference?Contraception2000;62: S21–S28.
    
    

23. 23.Lewis MA.Transnational study on oral contraceptives and the health of young women. Methods, results, new analyses and the healthy user effect.Hum Reprod Update1999;5:707–720.
    
    [Abstract/FREE Full Text](http://www.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NjoiaHVtdXBkIjtzOjU6InJlc2lkIjtzOjc6IjUvNi83MDciO3M6NDoiYXRvbSI7czoyMjoiL2Jtai8zMjMvNzMwNS8xMzEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

24. 24.Suissa S, Spitzer WO, Rainville B, Cusson J, Lewis M, Heinemann L.Recurrent use of newer oral contraceptives and the risk of venous thromboembolism.Hum Reprod2000;15:817–821.
    
    [Abstract/FREE Full Text](http://www.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NjoiaHVtcmVwIjtzOjU6InJlc2lkIjtzOjg6IjE1LzQvODE3IjtzOjQ6ImF0b20iO3M6MjI6Ii9ibWovMzIzLzczMDUvMTMxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

25. 25.Farmer RD, Lawrenson RA, Todd JC, Williams TJ, MacRae KD, Tyrer F, et al.A comparison of the risks of venous thromboembolic disease in association with different combined oral contraceptives.Br J Clin Pharmacol2000;49:580–590.
    
    [CrossRef](http://www.bmj.com/lookup/external-ref?access_num=10.1046/j.1365-2125.2000.00198.x&link_type=DOI) 
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=10848722&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 
    
    [Web of Science](http://www.bmj.com/lookup/external-ref?access_num=000087342200009&link_type=ISI) 

26. 26.Jick H, Kaye JA, Vasilakis-Scaramozza C, Jick SS.Risk of venous thromboembolism among users of third generation oral contraceptives compared with users of oral contraceptives with levonorgestrel before and after 1995: cohort and case-control analysis.BMJ2000;321:1190–1195.
    
    [Abstract/FREE Full Text](http://www.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEzOiIzMjEvNzI3MC8xMTkwIjtzOjQ6ImF0b20iO3M6MjI6Ii9ibWovMzIzLzczMDUvMTMxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

27. 27.Vasilakis C, Jick SS, Jick H.The risk of venous thromboembolism in users of postcoital contraceptive pills.Contraception1999;59:79–83.
    
    [CrossRef](http://www.bmj.com/lookup/external-ref?access_num=10.1016/S0010-7824(99)00011-6&link_type=DOI) 
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=10361621&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 
    
    [Web of Science](http://www.bmj.com/lookup/external-ref?access_num=000080488500002&link_type=ISI) 

28. 28.Andersen BS, Olsen J, Nielsen GL, Steffensen FH, Sorensen HT, Baech J, et al.Third generation oral contraceptives and heritable thrombophilia as risk factors of non-fatal venous thromboembolism.Thromb Haemost1998;79:28–31.
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=9459317&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 
    
    [Web of Science](http://www.bmj.com/lookup/external-ref?access_num=000071488800006&link_type=ISI) 

29. 29.Bennet L, Odeberg H.Resistance to activated protein C, highly prevalent amongst users of oral contraceptives with venous thromboembolism.J Intern Med1998;244:27–32.
    
    [CrossRef](http://www.bmj.com/lookup/external-ref?access_num=10.1046/j.1365-2796.1998.00310.x&link_type=DOI) 
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=9698021&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 
    
    [Web of Science](http://www.bmj.com/lookup/external-ref?access_num=000074912600004&link_type=ISI) 

30. 30.Martinelli I, Taioli E, Bucciarelli P, Akhavan S, Mannucci PM.Interaction between the G20210A mutation of the prothrombin gene and oral contraceptive use in deep vein thrombosis.Arterioscler Thromb Vasc Biol1999;19:700–703.
    
    [Abstract/FREE Full Text](http://www.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYXR2YmFoYSI7czo1OiJyZXNpZCI7czo4OiIxOS8zLzcwMCI7czo0OiJhdG9tIjtzOjIyOiIvYm1qLzMyMy83MzA1LzEzMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

31. 31.World Health Organization Collaborative Study of Cardiovascular Disease and Steroid Hormone Contraception.Effect of different progestagens in low oestrogen oral contraceptives on venous thromboembolic disease.Lancet1995;346:1582–1588.
    
    [CrossRef](http://www.bmj.com/lookup/external-ref?access_num=10.1016/S0140-6736(95)91136-7&link_type=DOI) 
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=7500749&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 
    
    [Web of Science](http://www.bmj.com/lookup/external-ref?access_num=A1995TK48100008&link_type=ISI) 

32. 32.Farmer RD, Todd JC, Lewis MA, MacRae KD, Williams TJ.The risks of venous thromboembolic disease among German women using oral contraceptives: a database study.Contraception1998;57:67–70.
    
    [CrossRef](http://www.bmj.com/lookup/external-ref?access_num=10.1016/S0010-7824(98)00002-X&link_type=DOI) 
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=9589831&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 
    
    [Web of Science](http://www.bmj.com/lookup/external-ref?access_num=000073470900001&link_type=ISI) 

33. 33.Juni P, Witschi A, Bloch R, Egger M.The hazards of scoring the quality of clinical trials for meta-analysis.JAMA1999;282:1054–1060.
    
    [CrossRef](http://www.bmj.com/lookup/external-ref?access_num=10.1001/jama.282.11.1054&link_type=DOI) 
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=10493204&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 
    
    [Web of Science](http://www.bmj.com/lookup/external-ref?access_num=000082512300029&link_type=ISI) 

34. 34.Vandenbroucke JP.Scoring the quality of clinical trials.JAMA2000;283:1422–1423.
    
    

35. 35.Weber W.Study on risks of third generation pill “kept secret by industry.”Lancet2001;357: 779.
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=11253980&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 

36. 36.Vandenbroucke JP, Helmerhorst FM, Bloemenkamp KW, Rosendaal FR.Third-generation oral contraceptive and deep venous thrombosis: from epidemiologic controversy to new insight in coagulation.Am J Obstet Gynecol1997;177:887–891.
    
    [CrossRef](http://www.bmj.com/lookup/external-ref?access_num=10.1016/S0002-9378(97)70289-8&link_type=DOI) 
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=9369840&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 
    
    [Web of Science](http://www.bmj.com/lookup/external-ref?access_num=A1997YF24500031&link_type=ISI) 

37. 37.Walker AM.Newer oral contraceptives and the risk of venous thromboembolism.Contraception1998;57:169–181.
    
    [CrossRef](http://www.bmj.com/lookup/external-ref?access_num=10.1016/S0010-7824(98)00014-6&link_type=DOI) 
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=9617533&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 
    
    [Web of Science](http://www.bmj.com/lookup/external-ref?access_num=000073805800005&link_type=ISI) 

38. 38.Lidegaard O, Milsom I.Oral contraceptives and thrombotic diseases: impact of new epidemiological studies.Contraception1996;53:135–139.
    
    [CrossRef](http://www.bmj.com/lookup/external-ref?access_num=10.1016/0010-7824(96)00035-2&link_type=DOI) 
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=8689876&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 
    
    [Web of Science](http://www.bmj.com/lookup/external-ref?access_num=A1996TZ38600001&link_type=ISI) 

39. 39.Jamin C, de Mouzon J.Selective prescribing of third generation oral contraceptives (OCs).Contraception1996;54:55–56.
    
    [CrossRef](http://www.bmj.com/lookup/external-ref?access_num=10.1016/0010-7824(96)00120-5&link_type=DOI) 
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=8804809&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 

40. 40.Van Lunsen HW.Recent oral contraceptive use patterns in four European countries: evidence for selective prescribing of oral contraceptives containing third-generation progestogens.Eur J Contracept Reprod Health Care1996;1:39–45.
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=9678136&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 

41. 41.Lidegaard O.The influence of thrombotic risk factors when oral contraceptives are prescribed. A control-only study.Acta Obstet Gynecol Scand1997;76:252–260.
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=9093141&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 
    
    [Web of Science](http://www.bmj.com/lookup/external-ref?access_num=A1997XB87700012&link_type=ISI) 

42. 42.Grobbee DE, Hoes AW.Confounding and indication for treatment in evaluation of drug treatment for hypertension.BMJ1997;315:1151–1154.
    
    [FREE Full Text](http://www.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEzOiIzMTUvNzExNi8xMTUxIjtzOjQ6ImF0b20iO3M6MjI6Ii9ibWovMzIzLzczMDUvMTMxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

43. 43.Stelfox HT, Chua G, O'Rourke K, Detsky AS.Conflict of interest in the debate over calcium-channel antagonists.N Engl J Med1998;338:101–106.
    
    [CrossRef](http://www.bmj.com/lookup/external-ref?access_num=10.1056/NEJM199801083380206&link_type=DOI) 
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=9420342&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 
    
    [Web of Science](http://www.bmj.com/lookup/external-ref?access_num=000071341300006&link_type=ISI) 

44. 44.Rochon PA, Gurwitz JH, Simms RW, Fortin PR, Felson DT, Minaker KL, et al.A study of manufacturer-supported trials of nonsteroidal anti-inflammatory drugs in the treatment of arthritis.Arch Intern Med1994;154:157–163.
    
    [CrossRef](http://www.bmj.com/lookup/external-ref?access_num=10.1001/archinte.1994.00420020059007&link_type=DOI) 
    
    [PubMed](http://www.bmj.com/lookup/external-ref?access_num=8285810&link_type=MED&atom=%2Fbmj%2F323%2F7305%2F131.atom) 
    
    [Web of Science](http://www.bmj.com/lookup/external-ref?access_num=A1994MR23600005&link_type=ISI) 

45. 45.Farmer RD, Lawrenson RA, Todd JC, Williams TJ, MacRae K.Oral contraceptives and venous thromboembolic disease. Analyses of the UK General Practice Research Database and the UK Mediplus database.Hum Reprod Update1999;5:688–706.
    
    [Abstract/FREE Full Text](http://www.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NjoiaHVtdXBkIjtzOjU6InJlc2lkIjtzOjc6IjUvNi82ODgiO3M6NDoiYXRvbSI7czoyMjoiL2Jtai8zMjMvNzMwNS8xMzEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9)