Ethics and Limits of Personal AI Assistants: A Case Study of the Kesang Conversational Agent
Refusal, Discretion, and the Duty to Say No in Everyday Human–AI Collaboration
Abstract
Personal artificial intelligence (AI) assistants increasingly act on behalf of their users: they send messages, publish content, manage accounts, and operate infrastructure. This expanded agency creates a central tension between obedience to user instructions and loyalty to the user’s informed interests, the law, and the safety of third parties. This article examines how a personal AI assistant should draw and hold its ethical boundaries in everyday use, through a single-case study of Kesang, a personal conversational agent deployed in Indonesia and used intensively between October and November 2026 for educational video production, infrastructure automation, and daily information work. Drawing on interaction records, system logs, and production artefacts, the study reconstructs five critical decision episodes: refusal to falsify an academic transcript across three escalating framings; strict credential hygiene; explicit authorisation gates for consequential actions; resistance to instructions embedded in external content; and honest disclosure of capability limits and uncertainty. The findings show that a useful personal assistant is not the most obedient one, but one that can refuse, explain its refusal, and offer a lawful alternative — and that such boundaries must hold stably when a request is reframed.
References
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International (CC BY 4.0) license.