TALQ turns a year of Reddit posts into biweekly scores for 52 symptoms, and 13% of users entered in a crisis that remitted over six to nine months
Synopsis
TALQ has a large language model rate 52 depression-scale symptoms from the posts of 491 members of two Reddit mental health communities every two weeks for a year. The ratings held up across repeated sampling, across samples and against human annotations, and resolved into 12 replicable symptom factors. At the two-week scale only two lagged relations survived correction, self-harm content preceding emotional pain and sleep disturbance preceding overwhelm; over months, 13% of users entered in a crisis that remitted within six to nine months.
Interpretation
Depression and anxiety are usually measured every few weeks or months with brief questionnaires, so change between assessments goes unseen; TALQ instead scores what people write every two weeks across 52 symptoms. Clinical scales such as MADRS, PHQ-9 and BDI-II cover few symptoms and are completed at visits, while experience sampling is dense but usually tracks a few items over days to weeks. Across the first year of 491 sustained members of r/depression and r/Anxiety, 9,886 two-week windows contained posts, and rating all 52 symptoms five times per window yielded 2.54 million valid ratings.
Five repeated ratings of the same text agree only moderately on their own, but their mean is markedly more reliable, so each window and symptom is scored by the average of five ratings. The median intraclass correlation of single ratings was 0.64, ranging from 0.15 to 0.85 across symptoms, while the median for the mean of five reached 0.90. Agreement rose with the model's own 0-to-100 confidence, from a median intraclass correlation of 0.19 below confidence 40 to 0.90 above 80; the main analysis kept 7,094 windows from 481 users whose mean confidence exceeded 40 for all 52 symptoms.
The ratings predict labels assigned by human annotators, separating interpersonal risk factors, low self-esteem and depression severity in three Reddit datasets of single posts with a regularized regression on the scores. Nine lexicon and length features that need no language model did worse on the first three labels, with AUROC 0.64 to 0.68 against 0.75 to 0.89 for TALQ ratings, but matched or beat them on depression severity, AUROC 0.82 against 0.75. AUROC was 0.89 (95% CI 0.85 to 0.93) for perceived burdensomeness, 0.81 (0.76 to 0.86) for thwarted belongingness, 0.75 (0.70 to 0.81) for low self-esteem and 0.75 (0.67 to 0.83) for severe against minimum depression.
The 52 symptom ratings decompose into 12 factors that replicate across samples, including fearful preoccupation and social strain; at the two-week scale only self-harm content preceding emotional pain and sleep disturbance preceding overwhelm survived false-discovery-rate correction. Of 132 directed factor pairs, 16 had uncorrected P below 0.05 against about 6.6 expected by chance, and only two remained after correction, each running in one direction. Self-harm content predicted more emotional pain two weeks later while the reverse did not hold (P = 0.54), and sleep disturbance predicted more overwhelm while the reverse did not (P = 0.26); a 50-point impulse raised emotional pain by 4.5 points (95% CI 2.7 to 6.6) and overwhelm by 3.9 points (2.1 to 5.9), fading within six weeks.
Perspective
The demonstrated scope is the first year of posting by 491 highly active members of two English-language Reddit communities between 2009 and 2018, scored in two-week windows by Llama-3.1-8B-Instruct; the ratings are explicitly inferences about expressed mental states, not diagnoses.
Two-week windows average over the faster fluctuations that experience sampling captures, and autoregressive estimates from short panels are biased toward zero, a bias the half-panel jackknife reduces but may not remove, so the results are not evidence against symptom networks at shorter time scales; course-type membership agreed only weakly between halves of the year and is better reported as a probability than a label.
