Kim, M., Wang, W., Long, J., Picard, R., Barczi, N., & Maes, P. (2025). Reflective Agency: Ethical and Empirical Framework for AI-Mediated Self-Reflection Systems. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 8(2), 1440-1452. https://doi.org/10.1609/aies.v8i2.36644

Abstract

As artificial intelligence (AI) increasingly mediates self-reflective practices—from therapeutic conversational agents to personal informatics systems—it becomes crucial to examine how these technologies shape, rather than merely support, our self-understanding. This paper investigates the influence of AI-mediated journaling systems on Reflective Agency—the capacity to interpret and make meaning of one’s experiences autonomously. Drawing on phenomenology and Aristotelian virtue ethics, we propose the Reflective Agency Framework (RAF): a normative model with five design principles for such AI tools—Internal Origination, Calibrated Responsiveness, Reflective Ambiguity, Transparency of Mediation, and Self-Continuity and Ethical Flourishing. We demonstrate RAF’s relevance through a two-part empirical case study: first, a systematic feature-tension analysis of six widely used AI-mediated journaling applications; second, an exploration of user perceptions and responses to these tensions and principles. Mapping these insights back to RAF, we identify persistent conflicts between automation and autonomy and offer design considerations that preserve the interpretive space essential for self-development. Our findings reveal that over-automation in AI-mediated reflection can erode agency. We urge designers and researchers to adopt and extend our framework to ensure future technologies genuinely support autonomy and meaningful self-discovery in digital well-being. Ultimately, we envision AI not as a guide, but as a quiet companion—helping individuals stay connected to their evolving sense of self and remain the primary agents in their self-discovery, thereby preserving what we define as Reflective Agency.