A Mechanism Randomised Controlled Trial of a Three-Agent LLM-Augmented mHealth Intervention for Late-Life Loneliness in Older Adults

Loneliness in later life is associated with adverse physical and mental health outcomes. Conversational applications based on large language models have been proposed as a scalable form of support, but most published evaluations remain at the pilot feasibility stage, and few have used a comparator that holds interface richness, attention, and novelty constant. Without such a comparator, any observed benefit cannot be attributed to the language model rather than to the surrounding experience.

This trial addresses that gap using an architecturally matched comparator. Both arms deliver the identical mobile application: the same visual interface, navigation, three named conversational companions, tool layer, and in-app assessment prompts. The arms differ in the backend that generates companion responses and in the features that depend on the language model, which are available only in the Hybrid arm: (1) cross-session memory; (2) replies adjusted to the tone that the participant preferred at intake; (3) follow-up questions during reminiscence; (4) weekly life-story chapters drafted by the model and sent by notification when ready; (5) the option to ask Tung Tung about reading material; (6) activity suggestions and action plans personalised with intake information and memory; and (7) a weekly narrative summary on the progress page written by the model. In the Rule-based arm, the weekly life-story chapter is assembled at once from the participant's own answers, and the progress page shows charts and counts only. In the Hybrid arm, responses are generated by a large language model using each companion's system prompt, conversation history, and user input, and the arm exhibits language-model-distinctive conversational behaviours including anchoring on specific content, cross-session memory, explicit admission of unfamiliarity, mixed-content routing, and generative summarisation. In the Rule-based arm, responses are produced by a template system driven by keyword matching and conversational state, without cross-session memory and without those behaviours. Conversational content in this arm is not transmitted to any external service. A local safety classifier, hosted by the research team, is used in both arms.

Seventy-two community-dwelling Cantonese-speaking adults aged 60 or above who score at least 2 on the De Jong Gierveld Emotional and Social Loneliness Scale are allocated 1:1, stratified by baseline emotional loneliness using permuted blocks within stratum, with allocation concealment. Trained outcome assessors, who are unaware of allocation, explain the study, obtain written informed consent, and conduct the paper-based assessments at Week 0 and Week 4. The same assessor conducts both assessments for a participant. After the Week 0 assessment, the assessor leaves the room. The Principal Investigator then registers the participant, which triggers the allocation on the study server, and provides app onboarding. The allocation sequence is generated by a person who has no other role in the trial, and the Principal Investigator cannot read it. After the Week 4 assessment, the Principal Investigator conducts an exit interview. At Week 8, participants complete the follow-up questionnaires in the app.

The intervention period is four weeks, followed by four weeks of follow-up with continued application access and continuous usage logging. Participants are encouraged to use the application daily and at least five times per week; use is not mandatory. The primary analysis is a 2-1-2 multilevel structural equation model. Arm is the participant-level predictor. Perceived responsiveness is rated after conversations, and the model estimates each participant's latent mean. Emotional loneliness at Week 4, adjusted for baseline, is the outcome. The indirect effect is tested using a Monte Carlo confidence interval. The sample size is fixed by feasibility. In a sensitivity power analysis by simulation, sixty completers gave power of about 0.80 only when the effect of arm on perceived responsiveness was d = 0.8 and the standardised effect of perceived responsiveness on emotional loneliness was 0.4. The between-arm difference in emotional loneliness is reported as an effect estimate with a 95% confidence interval and is not tested confirmatorily. A falsifiable specification of the proposed mechanism is that an arm advantage appears on emotional loneliness but not on social loneliness.

A three-layer safety architecture operates in both arms: a Cantonese-calibrated distress detector that surfaces Hong Kong crisis resources in real time and alerts the research team; a daily 100% audit of Thought Exercise events against pre-specified scope criteria; and weekly Principal Investigator review of all flags and audits. Pre-specified pause criteria are an intervention-related serious adverse event, boundary-crossed events exceeding 10%, or distress detector recall below 0.90 on a real Cantonese corpus.

A pilot phase with six older adults preceded the randomised trial. Two pilot participants started on 15 October 2026 under a separate pilot consent form. Four further participants signed the main consent form and were registered on 19 October 2026 with an earlier allocation sequence. On that day, the Rule-based version of the app did not work correctly. The randomised analysis set was therefore restarted on 21 October 2026 with a newly generated allocation sequence, and these four participants were moved to the pilot phase. They continue to use the application and complete the scheduled assessments. Pilot-phase data are reported for feasibility and acceptability only and are excluded from all trial analyses. The randomised analysis set comprises participants enrolled from 21 October 2026 onwards; the target of 72 randomised participants, with about 60 expected to complete the trial, is unchanged.

Companion responses are generated by a large language model through the official DeepSeek API, accessed via Firebase Cloud Functions. The model version is not fixed, so the model identifier returned with each response is logged. Responses are conditioned on the companion's system prompt, conversation history, cross-session memory, and the current input. In reminiscence, the model asks one follow-up question after each answer, and at the end of each week a DeepSeek model drafts a short life-story chapter that a second model checks against the conversation. Four in-app tools (Action Loop, Thought Exercise, Education, Progress). Five LLM-distinctive behaviours are tagged in logs as a cumulative within-arm exposure variable. Conversational text is sent to the model provider; participants are advised not to include identifying information. Participants are encouraged to use the app daily and at least five times per week for four weeks; use is not mandatory and is logged.

Behavioral: Hybrid conversational companion application

Identical interface, navigation, companion names and personae, tool layer, notification schedule, and in-app assessment prompts. Replies are assembled from pre-written templates by keyword matching and conversational state; no language model is used. In free conversation, Siu Yan's replies are drawn in rotation from a pool of pre-written sentences. The daily check-in is a short conversation with fixed questions, followed by a pre-written response chosen by the participant's answer. In reminiscence, both arms use the same open questions; in this arm each answer is followed by a fixed acknowledgement, and the participant's own answers are arranged, unchanged, into a weekly life-story chapter. No cross-session memory. Conversational content is not sent to any external service; a local safety classifier, hosted by the research team, is used in both arms. Attention- and interface-matched active comparator. Recommended use, expected daily duration, and logging are identical to the experimen

Behavioral: Rule-based conversational companion application

Companion responses are generated by a large language model through the official DeepSeek API, accessed via Firebase Cloud Functions. The model version is not fixed, so the model identifier returned with each response is logged. Responses are conditioned on the companion's system prompt, conversation history, cross-session memory, and the current input. In reminiscence, the model asks one follow-up question after each answer, and at the end of each week a DeepSeek model drafts a short life-story chapter that a second model checks against the conversation. Four in-app tools (Action Loop, Thought Exercise, Education, Progress). Five LLM-distinctive behaviours are tagged in logs as a cumulative within-arm exposure variable. Conversational text is sent to the model provider; participants are advised not to include identifying information. Participants are encouraged to use the app daily and at least five times per week for four weeks; use is not mandatory and is logged.

Identical interface, navigation, companion names and personae, tool layer, notification schedule, and in-app assessment prompts. Replies are assembled from pre-written templates by keyword matching and conversational state; no language model is used. In free conversation, Siu Yan's replies are drawn in rotation from a pool of pre-written sentences. The daily check-in is a short conversation with fixed questions, followed by a pre-written response chosen by the participant's answer. In reminiscence, both arms use the same open questions; in this arm each answer is followed by a fixed acknowledgement, and the participant's own answers are arranged, unchanged, into a weekly life-story chapter. No cross-session memory. Conversational content is not sent to any external service; a local safety classifier, hosted by the research team, is used in both arms. Attention- and interface-matched active comparator. Recommended use, expected daily duration, and logging are identical to the experiment

Plazo: After valid conversation sessions, at most once per companion per day, Weeks 1 through 4

Brief perceived-responsiveness measure administered in the app after valid conversation sessions. Four single items, each rated from 1 to 7, cover Understanding, Validation, Caring, and Insensitivity (reverse-scored). A free-conversation session is valid when the participant sends at least three messages and receives replies; a check-in or reminiscence session is valid when the participant completes the module. In the pre-specified primary mediation model, each participant's latent mean of these ratings is the mediator; this is not an efficacy endpoint.

Plazo: Baseline (Week 0) and Week 4

3 items, 0-3. Change from baseline to Week 4. Outcome in the primary mediation model. Assessed on paper at Weeks 0 and 4 by a trained outcome assessor who is unaware of allocation; in principle, the same assessor conducts both assessments for a participant. Power for the indirect effect was examined in a sensitivity simulation; the trial is not powered for confirmatory testing of the between-arm difference, which is reported as an effect estimate with a 95% CI.

Plazo: Baseline (Week 0) and Week 8

De Jong Gierveld emotional subscale, 3 items, 0-3, self-administered in the app at Week 8 (the questionnaire opens on Day 56 and closes on Day 63).

Plazo: Baseline (Week 0) and Week 4

De Jong Gierveld social subscale, 3 items, 0-3. Tests the prediction that any difference between arms appears in emotional but not in social loneliness.

Plazo: After valid conversation sessions, Weeks 1 through 4

Understanding, Validation, Caring, and Insensitivity (reverse-scored), each rated from 1 to 7. Each dimension is examined as a mediator in a secondary analysis without correction for multiple comparisons.

Plazo: Baseline (Week 0), Week 4, and Week 8

Lubben Social Network Scale, 6-item version, completed on paper at Weeks 0 and 4 and in the app at Week 8. It is used to examine whether participants' social networks become smaller while they use the app. Within each arm, the Week 4 and Week 8 scores are compared with Week 0; no direction is predicted for the difference between arms.

Plazo: Weeks 0 through 4

De Jong Gierveld emotional subscale, 3 items, 0-3, completed in the app at Weeks 1 to 3. Together with Weeks 0 and 4, it is used to describe the trajectory of emotional loneliness with a mixed model that includes the interaction between arm and time, and to examine its within-person lagged association with weekly perceived responsiveness (exploratory).

Plazo: Weeks 1 through 4

Weekly perceived responsiveness questionnaire, 12 items, completed in the app.

Plazo: Weeks 1 through 8

One item asking how loneliness has changed compared with before the participant started using the app, with 7 options from much better to much worse, completed in the app.

Plazo: Weeks 1 through 4

Two items completed in the app, on the number of days with in-person contact and the number of days with remote contact in the past 7 days. Used to examine whether participants use the app in place of contact with other people (exploratory).

Plazo: Weeks 0, 4, and 8

Three-item UCLA Loneliness Scale.

Plazo: Weeks 0 and 4

No informado en este registro.

Plazo: Weeks 0 and 4

No informado en este registro.

Plazo: Weeks 1 through 4

Number of valid conversation sessions logged by the app, using the same definition of a valid session as Primary Outcome 1.

Plazo: Weeks 1 through 4

Ratings of the companions' responses in three randomly sampled valid sessions per participant, on understanding, validation, caring, and insensitivity. The Principal Investigator codes all sampled sessions; a second coder who is not told the arm codes about 20% of them.

Plazo: Weeks 1 through 8

Safety events reported separately by source (participant input, scans of the AI reply, questionnaires, and assessor entries).

Plazo: Weeks 0, 4, and 8 (optional at Week 8)

No informado en este registro.

Plazo: Week 4

No informado en este registro.

Plazo: Week 4

4 study-specific items.

Plazo: Weeks 4 and 8

3 items.

Plazo: Week 4

AIMHS answered at Week 4 about how the participant felt before using the app (then-test).

Plazo: Weeks 1 through 4

Three study-specific items, each rated from 1 to 7 after each weekly chapter is shown, on whether the chapter matches what the participant said, satisfaction with the chapter, and willingness to show it to family or friends.

Plazo: Weeks 1 through 8

One optional item with five faces, which can be answered more than once a day; each answer is time-stamped.

Plazo: Week 4

Generic acceptability questionnaire based on the Theoretical Framework of Acceptability (Sekhon, Cartwright and Francis, 2022), 9 items each scored from 1 to 5, covering affective attitude, burden, ethicality (2 items), intervention coherence, perceived effectiveness, self-efficacy, opportunity costs, and general acceptability. Items are reported separately by arm; no total score is formed. Completed on paper at Week 4; Cantonese translation by the research team.

A Mechanism Randomised Controlled Trial of a Three-Agent LLM-Augmented mHealth Intervention for Late-Life Loneliness in Older Adults