Assessing the outcome of compulsory psychiatric treatment in the community: epidemiological study in Western Australia

Selection of subjects

We selected subjects on the basis of when they were placed on a community treatment order. We refer to the date of first placement on an order from either an authorised hospital or the community as the “index date.” We selected patients placed on orders between 13 November 1997 (the date of implementation of the Mental Health Act 1996) and 31 November 1998. Of the 313 patients who were preliminarily selected, seven died during the one year observation period after their index dates and 32 had not had a full year's contact with the mental health registry before their index dates, leaving 274 subjects.

Selection of matched controls

We selected matched controls as follows. For each subject we selected a group of candidates of the same sex and Aboriginal ethnicity and with an age difference of less than 2 years. We weighted clinical and demographic characteristics of subjects and controls in terms of their relative importance in the matching process. We gave diagnosis at index date the greatest relative importance in the selection algorithm, followed by length of stay, number of hospital admissions and occasions of service use (>80% and <120% of subject's length of stay, hospital admissions, and occasions of service use), and involuntary status in the year before the index date. We created a subject-control matching score that summed all the relative importance scores for the above clinical and demographic variables for each candidate. We then chose a matched control with the greatest matching score for the subject as the final matched control for that subject. Of 7881 matched controls who were preliminarily selected in the first stage of the matching process, 266 were determined as the final matched controls. Suitable matched controls could not be identified for 38 subjects, leaving 228 patients in each group. Subjects who died during the study were dropped from the study along with their matched controls to ensure that all subjects and controls had equal observation periods before and after the index date.

Analysis

We used the paired samples t test to test for differences between subjects and controls in admissions, bed days, and outpatient contacts before the index date and subsequent changes in bed days and outpatient contacts within each group. We performed each analysis on the logarithmically transformed dependent variables.

To further adjust for possible differences between the two groups, we generated standard multiple regression models to examine the contribution of patient characteristics and use of services before the index date to subsequent inpatient admissions, inpatient bed days, and outpatient contacts. Because of the non-normal distribution (skewness for admissions, inpatient bed days, and outpatient contacts after the index date were 2.55, 3.85, and 2.66), we performed logarithmic transformation on each dependent variable and inpatient admissions, bed days, and outpatient contacts before the index date. We examined predictor variables for each regression model by entering service use, including inpatient admissions, bed days, and outpatient contacts, before the index date and data on patient characteristics such as age, sex, Aboriginal ethnicity, and whether subject or control. We examined the contribution of diagnosis by creating dichotomous dummy variables for schizophrenia, mood disorders, and disorders of substance use. We kept personality disorders as the reference group for diagnosis. We examined social relationship variables by creating dichotomous dummy variables for marital status, with classifications of single versus once coupled and divorced or separated versus single or married. We reclassified occupational data into dichotomous dummy variables of professional occupations, trade and clerical workers, and labourers. We treated the home duty, student, or pensioned classification as the reference group for occupational status. We examined regional contributions to the dependent variables by classifying metropolitan versus rural versus remote regional areas. We tested for multivariate outliers on the predicted variables by using Mahalanobis distance and multicolinearity among independent variables on the predicted values before performing the regression analysis.

We also used the multiple linear regression model proposed by Hopper and Seeman to further assess within pair differences in use of health services before and after the index date, with adjustment for possible confounders.15

Limitations of the study

Although the matching process allowed selection of controls with similar characteristics to the patients placed on community treatment orders, significant differences in prior outpatient contacts and inpatient bed days existed. This meant that these variables had to be included in the regression models to control for their effects. This has further limitations, as controlling for variables may not be sufficient and additional confounders may be present. For example, it was not possible to control for variables such as social disability or dangerousness with the available database. Patients placed on an order may have greater degrees of dangerousness or social disability. These factors, in turn, may effect subsequent use of health services.

As this was an epidemiological study, we could not study clinical outcomes. However, the size of the study and the use of an epidemiological sampling frame that spanned the introduction of community treatment orders and covered all patients placed on such an order meant that the sample was representative of patients placed on an order and not subject to referral or selection bias. The design also permitted before and after comparisons of patients placed on an order and those not placed on an order for a greater number of subjects than has been previously possible. 6 10 12 The regression modelling allowed for multivariate examination of a wide range of predictors of subsequent use of health services for both subject and control groups.

Both groups showed a reduction in inpatient bed days, suggesting a regression to the mean in that patients with high rates of use will have lower rates of subsequent use. The finding that this effect was seen equally in both groups may indicate that the groups were similar. The regression results for within pair differences in inpatient admissions or bed days, and their covariates, also showed the similarity of the two groups.

Predictors of service use

Each of the three regression models showed that previous use of services, including number of admissions and outpatient contacts, contributed to subsequent use of health services. Other predictors, including education, occupational status, and diagnosis, did not contribute to subsequent use of services. Neither did being placed on a community treatment order contribute to subsequent bed days or admissions, indicating similar outcomes between subject and control groups. Being placed on an order did, however, predict higher subsequent outpatient contacts. Such results indicate that history of inpatient treatment was a greater predictor of subsequent use of services than being placed on a community treatment order. These results show that although placement on an order leads to a greater likelihood of increased outpatient contacts, this increase does not contribute to a greater reduction in inpatient bed days than is seen in the patients who were not placed on an order.

Although patients who were placed on a community treatment order showed reductions in inpatient admissions and length of inpatient stay one year after placement of the order, these improvements were no greater than those seen in controls who had not been placed on an order, after adjustment for diagnosis, sex, and previous use of health services. It is unlikely that this can be attributed to selection bias between the two groups, as service use and patient characteristics were controlled for in the regression models.

Are community treatment orders effective?

So the question still remains—are community treatment orders effective? This study provides mixed results. Although orders reduce admission rates and bed days, the effect is no greater than that seen in a group of patients who are not on such an order, after adjustment for possible confounders. The study therefore raises questions about the effectiveness of such an invasive procedure as enforcing treatment in the community. If efficacy is in part defined by reduced hospital admissions and length of stay, our results suggest that the policy is no more effective than not enforcing community treatment. Our results are consistent with previous studies of community treatment orders, which showed no significant improvements in outcome compared with matched controls, 11 12 but not with findings from England and Massachusetts, where patients on “extended leave” or involuntary outpatient treatment spent less time in hospital than did matched controls. 6 12 Possible explanations might be differing selection criteria and the fact that only age, sex, and diagnosis were controlled for in the design of both these studies. On the other hand, the British study included an assessment of dangerousness that was not recorded in the Western Australia Mental Health Information System.

This study examines only whether community treatment orders reduce immediate use of health services, as this has been the main reason for their introduction and hence the focus of previous research into their effectiveness.6 Such orders may reduce use of health services in the longer term or produce benefits in other areas, such as psychosocial functioning.

Some researchers have called for randomised assignment of patients to community treatment orders as a way of measuring the efficacy within a quasi-experimental paradigm,14 but this can be logistically difficult if the order is tied to statutory legislation. An alternative would be to compare patients from jurisdictions with and without community treatment orders and matched on sociodemographic, clinical, and health service characteristics. Such comparisons would be desirable before the introduction of similar measures in other jurisdictions and would ensure that policy in this area is evidence based. This would be consistent with the increased interest in evidence based public policy in several countries.16

This study shows that legislative solutions such as community treatment orders may not always offer a solution to the need to provide appropriate services for psychiatric patients within limited resources. It is important to examine what role such orders have in providing effective mental health treatment and whether therapeutic gains could be better delivered by enhancing the quality and assertiveness of community treatment for high risk patients.