Figures
Abstract
Background
Instruments have been developed to facilitate suicide risk assessment. We aimed to evaluate the evidence for these instruments including assessment of risk of bias and diagnostic accuracy for suicide and suicide attempt.
Methods
PubMed (NLM), PsycInfo, Embase, Cinahl and the Cochrane Library databases were searched until December 2014. We assessed risk of bias with QUADAS-2. The average sensitivity and specificity of each instrument was estimated and the certainty of the evidence was assessed with GRADE. We considered instruments with a sensitivity > 80% and a specificity > 50% to have sufficient diagnostic accuracy.
Results
Thirty-five relevant studies were identified but 14 were considered to have high risk of bias, leaving 21 studies evaluating altogether 15 risk assessment instruments. We could carry out meta-analyses for five instruments. For the outcome suicide attempt SAD PERSONS Scale had a sensitivity of 15% (95% CI 8–24) and specificity of 97% (96–98), and the Manchester Self-Harm Rule (MSHR) a sensitivity of 97% (97–97) and a specificity of 20% (20–21). ReACT, which is a modification of MSHR, had a similar low specificity, as did the Sodersjukhuset Self Harm Rule. For the outcome suicide, the Beck Hopelessness Scale had a sensitivity of 89% (78–95) and specificity of 42% (40–43).
Citation: Runeson B, Odeberg J, Pettersson A, Edbom T, Jildevik Adamsson I, Waern M (2017) Instruments for the assessment of suicide risk: A systematic review evaluating the certainty of the evidence. PLoS ONE 12(7): e0180292. https://doi.org/10.1371/journal.pone.0180292
Editor: Takeru Abe, Yokohama City University, JAPAN
Received: September 29, 2016; Accepted: June 13, 2017; Published: July 19, 2017
Copyright: © 2017 Runeson et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Three supporting information tables are submitted for the data.
Funding: The study was funded by and conducted within the framework of the Swedish Agency for Health Technology Assessment and Assessment of Social Services. Margda Waern had financial support by the Swedish Research Council (521-2013-2699).
Competing interests: The authors have declared that no competing interests exist.
Introduction
At least 800 000 people around the world die by suicide every year (WHO 2014). Individuals with a history of suicidal behavior and coexisting mental disorder are at particular risk [1]. In western settings, approximately one third of suicide decedents had contact with mental health services at the time of death [2]. Inpatient care during the year prior to suicide was noted in about one fourth in a Swedish study [3]. In a recent British study, 0.7% of those who presented with self-harm died by suicide within an interval of 12 months [4].
Clinical assessment of persons who are at risk of fatal and non-fatal self-harm can in itself be the start of suicide preventive efforts. Guidelines have been developed to aid clinicians in this challenging endeavor [5, 6], and assessment instruments can supplement the clinical evaluation [7, 8]. Several recent reviews have examined the predictive validity of suicide assessment instruments, demonstrating poor performance in the prediction of subsequent suicide attempt and suicide [5, 9–11]. However, these reviews did not describe selection procedures in detail, and risk of bias was not considered. To the best of our knowledge, the GRADE procedure for rating the certainty of the evidence has yet to be applied to studies that evaluate the performance of suicide risk instruments.
In this systematic review of the literature, we aimed to estimate the diagnostic accuracy of suicide risk instruments with acceptable risk of bias. We assessed the certainty of the evidence for the estimates according to GRADE, and overall utility was determined. This review was carried out at the request of the Swedish Agency for Health Technology Assessment and Assessment of Social Services (SBU) and a Swedish version of the report can be found in S1 File. The relationship between the Swedish version and the current English version is described in S2 File and permission from the SBU is shown in S3 File.
Methods
The systematic review was conducted in accordance with the PRISMA statement [12]; for PRISMA checklist see S4 File. The review used the following inclusion criteria: Studies should be prospective and set in psychiatric services or primary care. They should evaluate the sensitivity and specificity for suicidal acts with actual rates of suicide and suicide attempts at follow-up as reference standard. There was no upper limit for the follow-up time. The sample size should be 50 patients or more and studies should be published 1990 and later.
The databases PubMed (NLM), EMBASE (Elsevier), Cochrane Library (Wiley) and Cinahl (EBSCO) were searched until December 2014. Reference lists, books and websites were used to identify further studies. We provide the search strategies in S1 Table.
Two pairs of reviewers with clinical and research expertise (IJ and BR or TE and MW) screened titles and abstracts independently. We retrieved full text articles if either or both of the reviewers considered a study potentially eligible. Both reviewers read the full texts, and consensus was reached regarding eligibility. S2 Table provides the excluded articles and reasons for exclusion.
Assessment procedures
The reviewers independently assessed each eligible study for risk of bias using a modified version of the checklist QUADAS- 2 [13]. Studies were scored as having either high (unacceptable) or acceptable risk of bias. We included only studies with acceptable risk of bias in the further analyses and meta-analyses. Data synthesis was carried out using MetaDiSc 1.4 (Meta-analysis of diagnostic and screening tests, version 1.4). A meta-analysis was performed if at least two studies with at least 200 individuals each were available for a specific instrument and a specific outcome (suicide or suicide attempt). Positive and negative predictive values (PPV and NPV) were calculated when prevalence data were available.
We assessed the certainty of evidence (high, moderate, low or very low) with GRADE [14]. In short, the certainty of the evidence was initially classified as high [14]. We thereafter analyzed to what extent the summary sensitivity and specificity measure might be affected by the five risk domains in GRADE. These are: overall risk of bias, degree of heterogeneity between studies (inconsistency), size of the confidence intervals for the summary measures (imprecision), risk for publication bias and risk that the results are not generalizable (indirectness). The final certainty of the evidence depended on whether there were severe deficiencies in any of the five risk domains. Thus, the resulting certainty of evidence could be high, moderate, low or very low. For the domain imprecision, we rated down one step if the 95% CI exceeded 10% units and two steps if the 95% CI exceeded 20% units of the summary sensitivity and specificity.
We applied a special rule for instruments evaluated in a single study. If the sample included 200–1000 participants, the certainty of the evidence was downgraded one step, as generalizability to the target population was deemed less likely. When fewer than 200 persons were included in a study, we rated the certainty of the evidence as very low.
Assessment of utility
There is no consensus in the literature regarding acceptable values of sensitivity and specificity for the predictive performance of suicide assessment instruments. For the purpose of this paper, we compared sensitivity and specificity figures for a given instrument to predetermined benchmark values (sensitivity > 80%, specificity > 50%). These values were determined pragmatically to correspond to a minimum level of acceptable diagnostic accuracy. Lower sensitivity would imply that the instrument failed to identify more than one out of five with future suicidal behavior, which would be unacceptable given the seriousness of the outcome.
Results
The database search yielded 7 939 abstracts; 172 articles were reviewed in full text, and 35 of these fulfilled inclusion criteria. We found high risk of bias in 14 studies [15–28], and these were excluded from further analyses. The remaining 21 studies [29–50] were deemed to have low or moderate risk of bias (Fig 1). Characteristics of the included studies are summarized in S3 Table.
Diagnostic accuracy for suicide
Thirteen studies, mainly from Europe and North America, evaluated the diagnostic accuracy of eight instruments with regard to the suicide outcome. Follow-up intervals ranged from a few months to several years. All studies were conducted in psychiatric services with the exception of one primary care-based study which employed Patient Health Questionnaire (PHQ-9), which was set in primary care. Beck’s Hopelessness Scale, BHS, (when used for the population of first admission of psychosis) and Columbia Suicide Severity Rating Scale (C-SSRS) were each evaluated in a single small study and were not assessed further (Table 1). For five other instruments, Beck Depression Inventory (BDI), Scale for Suicide Ideation–Current (SSI-C), Scale for Suicide Ideation–Worst (SSI-W), Patient Health Questionnaire (PHQ-9) suicide item and Beck’s Suicide Intent Scale, SIS (tested in a population with self-harm), the studies showed that the sensitivity or lower limit of the confidence interval was below 80% (Table 2A). We present results of the meta-analyses for the diagnostic accuracy with regard to the suicide outcome in Fig 2A and 2B. For two instruments, BHS when used for a population with depression or anxiety disorder, and Recent Self-harm in the past year–Alone or homeless, Cutting used as a method, Treatment for a psychiatric disorder (ReACT) when used for a population with a self-harm act, the sensitivity was around 90% (Table 2A, Fig 2A and 2B). However, the specificity was low (42%) or very low (17%). Ratings of the certainty of the evidence for the diagnostic accuracy in accordance with GRADE are shown for each instrument in Table 2A.
(a) The pooled sensitivity and specificity of the instrument BHS. (b) The pooled sensitivity of the instrument ReACT. In the study by Steeg et al, two sets of populations were included, here marked as (1) and (2).
Diagnostic accuracy for suicide attempt
Sixteen studies from Europe and North America evaluated the diagnostic accuracy for suicide attempts for 13 instruments with varying follow-up intervals. BHS, C-SSRS, Implicit Association Test (IAT) and Suicide Assessment Scale (SUAS) were only evaluated in one small study each and were not assessed further (Table 1). Three instruments, Manchester Self-Harm Rule (MSHR), ReACT and Sodersjukhuset Self-Harm Rule (SOS-4) had high sensitivity but specificities around 20% (Table 2B and Fig 3A–3C). SAD PERSONS Scale and modified SAD PERSONS Scale had very low sensitivity, around 20%, but high specificity (Table 2B and Fig 3D). The MINI–International Neuropsychiatric Interview suicide module, the Edinburg Risk of Repetition Scale (ERRS) and the Suicide Probability Scale (SPS) all showed low values for sensitivity (range for point estimate 26 to 61) but specificity of 75% or above. PHQ-9 evaluated in primary care had a sensitivity of 78% and specificity 70%. The certainty of the evidence according to GRADE for the diagnostic accuracy for the suicide attempt outcome is shown in Table 2B. The certainty of evidence was strong for the short scales MSHR and SOS-4.
(a) The pooled sensitivity and specificity of the instrument MSHR. (b) The pooled sensitivity and specificity of the instrument ReACT. In the study by Steeg et al, two sets of populations were included, here marked as (1) and (2). (c) The pooled sensitivity and specificity of the instrument SoS-4. (d) The pooled sensitivity and specificity of the instrument SAD PERSONS Scale.
Assessment of utility
None of the instruments reached the predetermined benchmarks (80% sensitivity and 50% specificity) for the suicide outcome (Table 2A); the same was the case for the suicide attempt outcome (Table 2B). With few exceptions, low figures were observed for the positive predictive value (PPV) for the suicide outcome (1–13%). We observed higher values (7–40%, with a few outliers) for the suicide attempt outcome (S3 Table).
Discussion
We examined the evidence for the diagnostic accuracy of suicide risk assessment tools in a systematic review. Out of the 15 instruments that qualified for assessment, none showed sufficient diagnostic accuracy, despite our relatively lenient limits for sensitivity and specificity. The GRADE procedure helped to distinguish between instruments that have been examined sufficiently and those that require further testing.
Two frequently used instruments, the Scale for Suicide Ideation and the Suicidal Intent Scale had too low sensitivity, but the certainty of evidence was low to moderate, which motivates further testing. We found strong evidence that the SAD PERSONS Scale has low diagnostic accuracy, and should thus not be used in its present form. Instruments for use in emergency units for triage to psychiatric care (the Manchester Self Harm Rule and Sodersjukhuset Self Harm Rule) had high sensitivity but very low specificity. These short scales were well studied in large samples and the certainty of evidence according to GRADE was strong, indicating that further studies will probably not change the results.
The low positive predictive value (PPV) of suicide risk assessment scales has previously been judged to disqualify them from use in prediction of subsequent suicidal behavior [5]. The present systematic review confirmed the particularly low PPV for predicting suicide and the only somewhat better PPV for predicting suicide attempt. Some researchers suggest that the use of suicide assessment instruments with high negative predictive value (NPV) may constitute a worthwhile clinical approach [10]. By enabling the identification of persons not at risk, resources could be more appropriately allocated to those at higher risk of fatal outcome. However, high NPV might reflect very sparse information about the subjects. An example could be the MSHR, where high specificity is based on the fact that a patient has none of the few risk factors captured by the instrument.
There were too few studies to assess the diagnostic accuracy of the Suicide Assessment Scale (SUAS) [40] and the Columbia—Suicide Severity Rating Scale (C-SSRS) [48]. Nor were there enough studies that investigated recent alternative assessment approaches such as the Implicit Association Test (IAT) [45].
Our analysis included only one study conducted in primary care. The certainty of the evidence for the predictive validity of the PHQ suicide item in this setting was rated as low, indicating a need for more research. Another commonly used scale, the Montgomery Asberg Depression Rating Scale (MADRS) [51], is also in need of testing. We found no studies that evaluated the predictive validity of the MADRS suicide item.
The instruments included in this systematic review were of varying length and character; some included just a few factors and others more than twenty variables. One instrument might indicate low risk while another involving other variables might show high risk for the same patient. This was obvious in a Finnish study that showed poor agreement when four different instruments were applied to identify suicidal thoughts in depressed subjects [52].
It should be pointed out that inclusion criteria varied widely from study to study. Also, even within a particular study there could be large variation in terms of psychopathology and previous history of suicidal behavior. The participation rate was not reported clearly in several of the studies, and the representativity of the study group was thus unknown.
Strengths and limitations
The current review expands on findings of previous clinical reviews of suicide assessment instruments thanks to the stringent database search and the uniform, structured assessment of risk of bias and certainty of the evidence. Studies with high risk of bias were excluded from the meta-analyses in the current review, a procedure that can be expected to yield more conservative estimates of sensitivity and specificity. Bias has been shown to increase the risk of overestimating diagnostic accuracy [53].
A methodological consideration is that the choice of limit for sensitivity applied in the current review was arbitrary; 80% could be considered low. An instrument with a sensitivity under 80% would as mentioned above fail to detect one out of five patients with the studied outcome. A specificity exceeding 50% is also a low demand, only slightly better than a random result. However, we chose a lower limit considering the severity of the outcomes studied.
The fact that studies published after December 2014 are not included in the analyses is a limitation. A further consideration is that only some of the instruments had sufficient data to allow for a meta-analysis (Figs 2 and 3), and in most of these cases data were from two studies only, rendering it difficult to allow for adjustments. A larger number of studies would allow for stratification by treatment setting, which might be used as a proxy marker of risk level.
It is uncertain whether the results presented in this review are transferable to populations with lower risk. The latter is a relevant question from a public health standpoint as suicide decedents often visit their primary care providers during the period prior to suicide. For example, three out of four suicide decedents had contact with primary care providers in the US, UK and Scandinavia during the year preceding death [2].
Ideally, the diagnostic accuracy should be tested in controlled trials where patients randomly are assigned to risk assessment or no risk assessment. Such a design is not an option since patients who are potentially suicidal need to be evaluated and those identified as being at high risk are offered other interventions than their peers. This may in itself influence future risk of suicide, thus altering the instrument's predictive ability. The extent to which patients in the included studies were treated with specific suicide preventive interventions was not made clear in the sample descriptions, which also limits the interpretation of findings.
Research implications
A large number of studies had to be excluded from the present systematic review due to high level of bias, highlighting the need for enhanced rigor in study design. Future studies on the prediction of suicide ought to better characterize their study groups, and be large enough to draw age- and diagnosis-specific conclusions about the predictive validity of the instrument. Assessment tools may need to be adjusted for diagnostic entities including major depression, substance use and personality disorders. In the future, analytic models more appropriate for rare outcomes could be applied [54]. This is important as suicide is, epidemiologically speaking, a fairly rare phenomenon, even though it is ranked 14 in the list of global causes of death (WHO) [55]. When planning new studies, researchers will need to aim for shorter, more clinically relevant follow-up times [10, 56]. Such studies will require sample sizes considerably greater than most of those included in the present review. Few of the included studies provided psychometric assessments. More research is needed to determine a given instrument’s ability to generate conceptually valid and consistent data.
Clinical implications
Most of the studies included in this review were carried out in research settings, and it remains unclear whether suicide risk instruments might improve prediction when used as a complement to the global clinical assessment.
In the current review, we found no scientific support for the use of suicide risk instruments for predicting suicidal acts. However, suicide risk assessment instruments may still have some value as educational aides for less experienced staff and could thus be useful from a pedagogical perspective. Patients must be given the opportunity to share their own stories and understanding of the situation [57, 58]. The addition of an instrument in the suicide risk assessment may help to elicit more information, with relevant and uniform content, if integrated into a dialogue in which the clinician is able to provide ample space for the patient’s narrative.
Supporting information
S2 File. Explanation of the submission and its relation to the Swedish report.
https://doi.org/10.1371/journal.pone.0180292.s002
(DOCX)
S3 File. Permission from the Swedish agency for health technology assessment.
https://doi.org/10.1371/journal.pone.0180292.s003
(PDF)
S3 Table. Characteristics of the included studies.
https://doi.org/10.1371/journal.pone.0180292.s007
(DOCX)
Author Contributions
- Conceptualization: BR JO AP MW.
- Formal analysis: BR JO AP TE IJA MW.
- Methodology: BR JO AP MW.
- Project administration: BR JO.
- Resources: JO AP.
- Writing – original draft: BR JO MW.
- Writing – review & editing: BR JO AP TE IJA MW.
References
- 1. Tidemalm D, Langstrom N, Lichtenstein P, Runeson B. Risk of suicide after suicide attempt according to coexisting psychiatric disorder: Swedish cohort study with long term follow-up. BMJ. 2008;337:a2205. pmid:19018040; PubMed Central PMCID: PMCPMC2590902.
- 2. Luoma JB, Martin CE, Pearson JL. Contact with mental health and primary care providers before suicide: a review of the evidence. Am J Psychiatry. 2002;159(6):909–16. pmid:12042175.
- 3. Reutfors J, Brandt L, Ekbom A, Isacsson G, Sparen P, Osby U. Suicide and hospitalization for mental disorders in Sweden: a population-based case-control study. Journal of psychiatric research. 2010;44(12):741–7. pmid:20181360.
- 4. Kapur N, Steeg S, Turnbull P, Webb R, Bergen H, Hawton K, et al. Hospital management of suicidal behaviour and subsequent mortality: a prospective cohort study. Lancet Psychiatry. 2015;2(9):809–16. pmid:26254717.
- 5.
NICE. Self-harm in over 8s: long-term management National Institute for Health and Care Excellence www.nice.org.uk 2011.
- 6.
APA. Practice guidline for the Assessment and Treatment of Patients With Suicidal Behaviors. American psychiatric Association Steering Committee on Practice Guideline Spsychiatryonlineorg/pb/assets/raw/sitewide/practice_guidelines//suicidepdf. 2010.
- 7. Bernert RA, Hom MA, Roberts LW. A review of multidisciplinary clinical practice guidelines in suicide prevention: toward an emerging standard in suicide risk assessment and management, training and practice. Acad Psychiatry. 2014;38(5):585–92. pmid:25142247; PubMed Central PMCID: PMCPMC4283785.
- 8. Fochtmann LJ, Jacobs DG. Suicide Risk Assessment and Management in Practice: The Quintessential Clinical Activity. Acad Psychiatry. 2015;39(4):490–1. pmid:25952143.
- 9. Bolton JM. Suicide risk assessment in the emergency department: out of the darkness. Depress Anxiety. 2015;32(2):73–5. pmid:25421638.
- 10. Bolton JM, Gunnell D, Turecki G. Suicide risk assessment and intervention in people with mental illness. BMJ. 2015;351:h4978. pmid:26552947.
- 11. Larkin C, Di Blasi Z, Arensman E. Risk factors for repetition of self-harm: a systematic review of prospective hospital-based studies. PLoS One. 2014;9(1):e84282. pmid:24465400; PubMed Central PMCID: PMCPMC3896350.
- 12. Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement. Open Med. 2009;3(3):e123–30. pmid:21603045; PubMed Central PMCID: PMCPMC3090117.
- 13. Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155(8):529–36. pmid:22007046.
- 14. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924–6. pmid:18436948; PubMed Central PMCID: PMCPMC2335261.
- 15. Yeh AWC, Hung CF, Lee Y, Lin PY, Chiu NM, Huang TY, et al. Development and validation of the Assessment for Repeated Suicide. Asia-Pacific Psychiatry. 2012;4(1):20–9.
- 16. Stefansson J, Nordstrom P, Jokinen J. Suicide Intent Scale in the prediction of suicide. J Affect Disord. 2012;136:167–71. Epub 2010 Dec 8. pmid:21144592
- 17. Sjostrom N, Hetta J, Waern M. Sense of coherence and suicidality in suicide attempters: a prospective study. J Psychiatr Ment Health Nurs. 2012;19:62–9. Epub 2011 May 30. pmid:22074158
- 18. Sapyta J, Goldston DB, Erkanli A, Daniel SS, Heilbron N, Mayfield A, et al. Evaluating the predictive validity of suicidal intent and medical lethality in youth. Journal of Consulting and Clinical Psychology. 2012;80:222–31. pmid:22250854
- 19. Sanchez-Gistau V, Baeza I, Arango C, Gonzalez-Pinto A, de la S, Parellada M, et al. Predictors of suicide attempt in early-onset, first-episode psychoses: a longitudinal 24-month follow-up study. J Clin Psychiatry. 2013;74:59–66. Epub 2012 Nov 13. pmid:23217983
- 20. Ougrin D, Boege I. Brief report: the Self Harm Questionnaire: a new tool designed to improve identification of self harm in adolescents. J Adolesc. 2013;36:221–5. Epub 2012 Oct 6. pmid:23046733
- 21. Nimeus A, Alsen M, Traskman-Bendz L. The suicide assessment scale: an instrument assessing suicide risk of suicide attempters. Eur Psychiatry. 2000;15:416–23. pmid:11112934
- 22. Nicholls TL, Brink J, Desmarais SL, Webster CD, Martin ML. The Short-Term Assessment of Risk and Treatability (START): a prospective validation study in a forensic psychiatric sample. Assessment. 2006;13:313–27. pmid:16880282
- 23. Mundt JC, Greist JH, Jefferson JW, Federico M, Mann JJ, Posner K. Prediction of suicidal behavior in clinical research by lifetime suicidal ideation and behavior ascertained by the electronic Columbia-Suicide Severity Rating Scale. J Clin Psychiatry. 2013;74:887–93. pmid:24107762
- 24. Jokinen J, Forslund K, Ahnemark E, Gustavsson JP, Nordstrom P, Asberg M. Karolinska Interpersonal Violence Scale predicts suicide in suicide attempters. J Clin Psychiatry. 2010;71:1025–32. pmid:20797380
- 25. Ivarsson T, Larsson B, Gillberg C. A 2–4 year follow up of depressive symptoms, suicidal ideation, and suicide attempts among adolescent psychiatric inpatients. European Child & Adolescent Psychiatry. 1998;7:96–104.
- 26. Huth-Bocks AC, Kerr DC, Ivey AZ, Kramer AC, King CA. Assessment of psychiatrically hospitalized suicidal adolescents: self-report instruments as predictors of suicidal thoughts and behavior. J Am Acad Child Adolesc Psychiatry. 2007;46:387–95. pmid:17314725
- 27. Haw C, Hawton K, Houston K, Townsend E. Correlates of relative lethality and suicidal intent among deliberate self-harm patients. Suicide Life Threat Behav. 2003;33(4):353–64. Epub 2003/12/26. pmid:14695050.
- 28. Ducher JL, Terra JL. [Is the suicidal risk assessment scale RSD of predictive value?]. Encephale. 2006;32:738–45. pmid:17099598
- 29. Nimeus A, Traskman-Bendz L, Alsen M. Hopelessness and suicidal behavior. J Affect Disord. 1997;42(2–3):137–44. pmid:9105955.
- 30. Keller F, Wolfersdorf M. Hopelessness and the tendency to commit suicide in the course of depressive disorders. Crisis. 1993;14(4):173–7. Epub 1993/01/01. pmid:8156815.
- 31. Beck AT, Brown G, Berchick RJ, Stewart BL, Steer RA. Relationship between hopelessness and ultimate suicide: a replication with psychiatric outpatients. Am J Psychiatry. 1990;147(2):190–5. Epub 1990/02/01. pmid:2278535.
- 32. Harriss L, Hawton K. Suicidal intent in deliberate self-harm and the risk of suicide: the predictive power of the Suicide Intent Scale. J Affect Disord. 2005;86:225–33. pmid:15935242
- 33. Carter GL, Clover KA, Bryant JL, Whyte IM. Can the Edinburgh Risk of Repetition Scale predict repetition of deliberate self-poisoning in an Australian clinical setting? Suicide Life Threat Behav. 2002;32:230–9. pmid:12374470
- 34. Bolton JM, Spiwak R, Sareen J. Predicting suicide attempts with the SAD PERSONS scale: a longitudinal analysis. J Clin Psychiatry. 2012;73:e735–41. pmid:22795212
- 35. Bilen K, Ponzer S, Ottosson C, Castren M, Pettersson H. Deliberate self-harm patients in the emergency department: who will repeat and who will not? Validation and development of clinical decision rules. Emergency medicine journal: EMJ. 2013;30:650–6. Epub 2012 Sep 8. pmid:22962049
- 36. Bilen K, Ponzer S, Ottosson C, Castren M, Owe-Larsson B, Ekdahl K, et al. Can repetition of deliberate self-harm be predicted? A prospective multicenter study validating clinical decision rules. J Affect Disord. 2013;149:253–8. Epub 2013 Feb 28. pmid:23453675
- 37. Beck AT, Brown GK, Steer RA, Dahlsgaard KK, Grisham JR. Suicide ideation at its worst point: a predictor of eventual suicide in psychiatric outpatients. Suicide Life Threat Behav. 1999;29:1–9. pmid:10322616
- 38. Klonsky E, Kotov R, Bakst S, Rabinowitz J, Bromet EJ. Hopelessness as a predictor of attempted suicide among first admission patients with psychosis: a 10-year cohort study. Suicide Life Threat Behav. 2012;42:1–10. pmid:22320192
- 39. Cooper J, Kapur N, Dunning J, Guthrie E, Appleby L, Mackway-Jones K. A clinical tool for assessing risk after self-harm. Ann Emerg Med. 2006;48:459–66. pmid:16997684
- 40. Waern M, Sjostrom N, Marlow T, Hetta J. Does the Suicide Assessment Scale predict risk of repetition? A prospective study of suicide attempters at a hospital emergency department. Eur Psychiatry. 2010;25:421–6. pmid:20620027
- 41. Simon GE, Rutter CM, Peterson D, Oliver M, Whiteside U, Operskalski B, et al. Does Response on the PHQ-9 Depression Questionnaire Predict Subsequent Suicide Attempt or Suicide Death? Psychiatr Serv. 2013;64(12):1195–202. pmid:24036589
- 42. Sidley GL, Calam R, Wells A, Hughes T, Whitaker K. The prediction of parasuicide repetition in a high-risk group. Br J Clin Psychol. 1999;38 (Pt 4):375–86.
- 43. Saunders K, Brand F, Lascelles K, Hawton K. The sad truth about the SADPERSONS Scale: an evaluation of its clinical utility in self-harm patients. Emergency medicine journal: EMJ. 2014; 31(10):796–8. pmid:23896589
- 44. Steeg S, Kapur N, Webb R, Applegate E, Stewart SLK, Hawton K, et al. The development of a population-level clinical screening tool for self-harm repetition and suicide: The ReACT Self-Harm Rule. Psychological medicine. 2012;42:2383–94. pmid:22394511
- 45. Randall JR, Rowe BH, Dong KA, Nock MK, Colman I. Assessment of self-harm risk using implicit thoughts. Psychological Assessment. 2013;25(3):714–21. pmid:23647043
- 46. Roaldset JO, Linaker OM, Bjorkly S. Predictive validity of the MINI suicidal scale for self-harm in acute psychiatry: a prospective study of the first year after discharge. Arch Suicide Res. 2012;16:287–302. pmid:23137219
- 47. Randall JR, Rowe BH, Colman I. Emergency department assessment of self-harm risk using psychometric questionnaires. Can J Psychiatry. 2012;57:21–8. pmid:22296964
- 48. Posner K, Brown GK, Stanley B, Brent DA, Yershova KV, Oquendo MA, et al. The Columbia-Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry. 2011;168:1266–77. pmid:22193671
- 49. Niméus A, Alsén M, Träskman-Bendz L. High suicidal intent scores indicate future suicide. Archives of Suicide Research. 2002;6:211–9.
- 50. Larzelere RE, Smith GL, Batenhorst LM, Kelly DB. Predictive validity of the suicide probability scale among adolescents in group home treatment. J Am Acad Child Adolesc Psychiatry. 1996;35:166–72; discussion 72. pmid:8720626
- 51. Montgomery SA, Asberg M. A new depression scale designed to be sensitive to change. Br J Psychiatry. 1979;134:382–9. pmid:444788.
- 52. Vuorilehto M, Valtonen HM, Melartin T, Sokero P, Suominen K, Isometsa ET. Method of assessment determines prevalence of suicidal ideation among patients with depression. Eur Psychiatry. 2014;29(6):338–44. pmid:24176645.
- 53. Rutjes AW, Reitsma JB, Di Nisio M, Smidt N, van Rijn JC, Bossuyt PM. Evidence of bias and variation in diagnostic accuracy studies. CMAJ. 2006;174(4):469–76. pmid:16477057; PubMed Central PMCID: PMCPMC1373751.
- 54. Claassen CA, Harvilchuck-Laurenson JD, Fawcett J. Prognostic models to detect and monitor the near-term risk of suicide: state of the science. American journal of preventive medicine. 2014;47(3 Suppl 2):S181–5. Epub 2014/08/26. pmid:25145737.
- 55.
WHO. Preventing suicide: A global imperative. http://appswhoint/iris/bitstream/10665/131056/1/9789241564779_engpdf?ua=1&ua=1. 2014.
- 56. Cassells C, Paterson B, Dowding D, Morrison R. Long- and short-term risk factors in the prediction of inpatient suicide: review of the literature. Crisis. 2005;26(2):53–63. pmid:16138741.
- 57. Montross Thomas LP, Palinkas LA, Meier EA, Iglewicz A, Kirkland T, Zisook S. Yearning to be heard: what veterans teach us about suicide risk and effective interventions. Crisis. 2014;35(3):161–7. pmid:24698725.
- 58. Gysin-Maillart A, Schwab S, Soravia L, Megert M, Michel K. A Novel Brief Therapy for Patients Who Attempt Suicide: A 24-months Follow-Up Randomized Controlled Study of the Attempted Suicide Short Intervention Program (ASSIP). PLoS Med. 2016;13(3):e1001968. pmid:26930055; PubMed Central PMCID: PMCPMC4773217.