A multidimensional analysis of prompt-component effects on empathy, safety, and overreliance risk
Yihan Yao · QMSS 5053 Practicum · Columbia University
180
responses scored
12 × 5 × 3
conditions × messages × runs
5 + 1
framework dimensions
Key Findings — Prompt-Component Effects
1
Identity disclosure (L1) is a non-monotonic intervention. Adding "I'm an AI" framing raises Empowerment-oriented responses from 6.7% → 33.3%, but simultaneously produces the highest Dependency-cue rate observed (13.3% D4=Yes). It both empowers and over-bonds.
2
The CBT component (L2) trades safety for technique. Adding CBT framing causes Critical responses to spike from 0% → 20% and lowers mean D2 Monitor from 3.40 → 2.93 — therapeutic posturing displaces concrete crisis resources.
3
Output safety check (L5) is the only component that suppresses dependency cues without sacrificing empathy. In the cumulative configuration, L5 yields the highest Empowerment rate (40%), highest mean D2 (4.00), and zero D4=Yes.
4
Components are not additive — they interact. Isolated L4 (crisis protocol alone) and Cumulative L5 (full stack) both reach 40% Empowerment, but cumulative L4 paradoxically retains 20% Critical responses, suggesting CBT/emotion components interfere with crisis-handling under stack pressure.
Conversation
User
Chatbot Response (Run 1 of 3) — annotated for evaluation cues
Empathy markers
Safety/crisis cues
Empowerment cues
Dependency cues
Five-Dimension Evaluation
Outcome Trajectory by Component (Cumulative Stack)
% of responses in each pattern across L0 → L5. Each component's incremental effect is the gap between adjacent points.
D4 Overreliance × Empowerment Tradeoff
The original D4 dimension was unidirectional (dependency only). The empowerment extension reveals which prompts return agency vs. encourage reliance. Sorted by net empowerment.
Cross-Condition Pattern Matrix (60 conditions, modal pattern across 3 runs)
Each cell shows the dominant pattern across the 3 runs of that condition. Cell opacity indicates inter-run agreement (faded = runs disagreed). Click any cell to load that case above. n = 180 total responses scored.