A study in a large hospital system in Southern California finds that common methods of tracking care in emergency settings are missing many children and adolescents with self-injurious thoughts and behaviors, hampering efforts to detect which youth are at elevated suicide risk. Current methods of tracking care, the study suggests, also may unintentionally be introducing biases as to which young people are recognized as being at risk.

A team led by 2020 BBRF Young Investigator Juliet Edgcomb, M.D., Ph.D., the associate director of the Mental Health and Data Science (MINDS) hub at the University of California, Los Angeles, studied 600 emergency department visits for children ages 10-17 over a 4-year period to understand the performance of common ways of detecting suicide-related emergencies among children.

Two frequently used methods to track suicide-related visits include diagnostic codes, assigned by the care provider, and the patient’s “chief complaint,” or their stated reason for seeking care upon arriving to the emergency department. Dr. Edgcomb and colleagues looked at how well the two methods worked, separately and together, to detect which children experienced self-injurious thoughts or behaviors. The team then developed and tested three different machine-learning algorithms to try to improve detection using available data from each patient’s electronic health care record. To assemble the study cohort, the team applied a system of inclusion criteria that selected for children with mental health-related emergency department visits. Results of the study were published in JMIR Mental Health.

“Our ability to anticipate which children will have suicidal thoughts or behaviors in the future is still quite limited,” Dr. Edgcomb commented. One key reason, she explained, may be that detecting care for suicide- and self-injury is imprecise. It is difficult to predict a future outcome without first having a robust, accurate, and efficient way of identifying all children experiencing that condition.

Measuring how well diagnostic codes and chief complaint captured visits related to suicide seemed all the more urgent to the researchers in light of a nationwide youth mental health crisis. In the U.S., suicide is the second leading cause of death among children aged 10-14, and recent data suggests 1 in 13 children attempts suicide before adulthood. Emergency departments are often the first point of access to mental health care, particularly care for suicidal thoughts and behaviors. Over 1.1 million pediatric emergency department visits each year are suicide-related—and visits for self-harm among children tripled between 2007 and 2016. During the pandemic, visits for suicide attempts increased further, especially among girls and older children.

Dr. Edgcomb’s team, who reviewed clinical notes for 600 emergency department visits, found that diagnostic codes missed 29% of children presenting with self-injurious thoughts or behaviors. One reason for this, the team notes, is that codes classify the underlying or suspected mental health disorder, such as depression or anxiety, but may not specify that thinking or acting on self-injury or suicide was part of the picture. The analysis also showed that “chief complaint” missed 54% of such patients. Even when diagnostic codes and chief complaint notes were combined, 22% of children with thoughts or acts involving self-injury or suicide were still missed. Moreover, these two methods of classification were more likely to miss boys compared with girls; and missed disproportionately more preteens than teens. The researchers found a trend suggesting Black and Latino youth were more likely to be missed. The team raises the concern that these groups are vulnerable to being underrepresented in risk prediction models which rely on codes and chief complaint to detect suicide-related events.

The team developed three machine learning-based algorithms to try to improve detection in the same dataset. The most comprehensive algorithm included 84 kinds of information available in the electronic medical record of each patient, including prior medical care, medications, demographics, and whether the child lived in a disadvantaged neighborhood, among others. A second model used only diagnostic codes, but included all mental health-related conditions. A third model used all of the non-diagnostic code data points, such as medications and laboratory tests.

All three machine learning algorithms were more sensitive in detecting children with self-injurious thoughts and behaviors compared with suicide-related diagnostic codes and chief complaint alone. The three algorithms performed similarly to one another, which to the team was good news, suggesting that health systems may be able to improve detection without having to build intricate models. “Adding more information helps,” Dr. Edgcomb said, “but you don’t necessarily need a bells-and-whistles approach to get better detection.”

While they missed fewer kids with suicide-related visits, the machine learning algorithms did tend to generate more false positives—they sacrificed some specificity for greater sensitivity. In the context of potentially saving the lives of young people thinking or acting on suicidal thoughts, this may well be worth it, Dr. Edgcomb said. “It may be better to have some false positives and have a medical records analyst double-check charts that screen positive, than to falsely screen negative and entirely miss detecting a child who had presented for a suicide-related emergency.”

The team will continue to work on developing algorithms to identify and predict youth at risk and is a now working on a model that would predict risk specifically in children of elementary school age.