| Non-independence and precision of confidence intervals [message #357] |
Mon, 03 July 2006 03:54 |
 Jonathon Little
Messages: 2
Registered: July 2004
Location: Office of Mental Health, ... |
Junior Member |
|
|
Now that the Socceroos have returned home after their gallant attempt at worldwide stardom, I find myself returning to the problem of non-independence of observations. As we are all probably aware NOCC data is nested data, which in the WA context will have at least 4-6 levels, described thus:
Patient behaviour is nested within clinician conceptualisations of psychopathology, rating clinicians are nested within collection reasons and occasions, which are nested within treatment settings, nested within mental health organisations, nested within larger organisational structures and so on all the way up to an Australia wide level. Clearly NOCC data is hierarchically structured, that is, full of intra-class correlations. Can we be confident in the robustness of confidence intervals for effect sizes, given these innumerable violations to the assumption of independence?
Is AMHOCN considering alternative ways of addressing these problems when conducting Australia wide analysis? I have played with the idea of implementing a method to resolve at least part of the non-independence problem for WA. This would involve undertaking separate analyses for each cluster. For example, calculating the effect at the mental health organisation or program level (WA’s lowest level at which organisational units are observed) and then using a fixed (random?) effects meta-analysis to estimate the precision and magnitude of the effect at a higher level of aggregation. I concede that a multi-level analysis would be a more attractive proposition, however implementing this in an automated environment is frankly beyond me, whereas the math for a meta-analysis is readily available and not too overwhelming. I would be interested in entering into a dialogue with anyone wishing to grapple with this problem.
I should add that this approach would not resolve dependence problems arising from having a small number of clinicians rating a large number of patients. That is, dependence arising from systematic observational errors (within and across collection occasions), or dependence arising from clinicians abandoning the glossary and imposing their own model – I see this as inevitable to at least some degree. Therefore, the approach would be restricted to self-rating instruments, namely the K10+.
Thoughts anybody?
Jonathon Little
Office of Mental Health
WA Department of Health
Tel:08 9222 4089
|
|
|
Home » NOCC Standard Reports » NOCC Standard Reports » Non-independence and precision of confidence intervals
Members
Search
Help
Register
Login
Home
E-mail to friend 
