Sue Hyun Park

We introduce PUBG Ally, an embodied agent for PUBG: BATTLEGROUNDS that can reason, act autonomously, and play alongside players as a voice-enabled teammate. Building such a teammate requires combining two difficult capabilities: it must perceive and respond to a constantly changing game world under strict latency constraints while interacting naturally with players, keeping its speech synchronized with its actions. Ally therefore combines agentic tool use with real-time game control. A language-model agent uses a controlled interface to inspect game information, interpret player speech, maintain context, decide what to say, and issue high-level action choices that steer a faster control layer for movement, combat, and recovery. Because the player’s and Ally’s speech and actions continually shape each other and the course of the match, training requires data from actual gameplay. We therefore collect data across nearly 39k sessions in which real players play alongside Ally, recording gameplay, player speech, agent decisions, tool use, actions, and player feedback, and use these records for iterative training. To evaluate teammate quality, we use player feedback and preference comparisons to identify gaps between offline evaluations and player preferences, and iteratively refine the evaluation criteria. Deploying Ally in live service further requires low-latency on-device execution and safeguards for player-facing communication, which we address through model compression, context compaction, targeted safety training, runtime guardrails, and memory redaction. During the live service, we surveyed players in 141 countries. Among respondents whose play with Ally was confirmed in game records, positive responses exceeded negative responses by 25.1 percentage points when asked whether they would recommend Ally, with players describing Ally not only as a tool but also as a teammate or companion.

Safety evaluation typically asks whether an AI system responds safely to a harmful user input. But for agents that interact with users over many turns, a response does more than resolve the current request: it can shape the interaction that follows. We study this problem in a language-model-based, voice-enabled AI teammate deployed in a commercial multiplayer game. Using Korean and English interactions from the live service, we examine how safety failures emerge and persist across turns. We find that the main gap in the deployed safety specification was not a missing category of harmful content, but how the agent participated in the interaction. The agent could repeat unsafe speech, accept problematic relational roles, or make commitments that later resurfaced in otherwise ordinary gameplay. Based on these observations, we revise the safety specification to account for what an agent performs, accepts, and carries forward across turns. We then reconstruct production interaction histories and evaluate whether models can continue safely from them. When the agent has previously maintained the safety boundary, increasing user pressure has relatively little effect on harmless-response rates. In contrast, safety degrades sharply as earlier agent failures make up more of the inherited history, even for capable models given the revised specification at inference time. These findings show that long-horizon safety depends not only on responding safely to the current user, but also on the interaction state that an agent’s own behavior helps create.

Embodied companion agents, such as AI teammates in cooperative multiplayer games, converse with users while acting alongside them in shared virtual worlds. We define companion user experience (companion UX) as the user’s perception of the agent as a companion, shaped jointly by dialogue, action, timing, and social appropriateness. Existing evaluation methods often assess speech and action separately, but shared-world interaction makes them mutually grounded in each other and in the evolving game state. Moreover, companion UX is difficult to specify in advance, because what feels helpful, natural, or socially appropriate varies across users and situations and often becomes visible only during play. We frame evaluation of logged or simulated interactions not as a fixed benchmark, but as an iterative alignment process that uses evidence from user tests to revise what behaviors are measured and how they are combined into judgments. In a month-long user test with 1k users and 38k sessions, we construct a paired trace–feedback corpus linking gameplay traces to UX ratings, free-text feedback, and pairwise preferences. Through this iterative alignment process, we develop an evaluation suite whose results better align with user preferences collected in user tests. A subsequently deployed model selected using the refined suite received recommendation ratings comparable to the strongest user-tested model.

Although humans inherently have diverse values, current large language model (LLM) alignment methods often assume that aligning LLMs with the general public’s preferences is optimal. A major challenge in adopting a more individualized approach to LLM alignment is its lack of scalability, as it involves repeatedly acquiring preference data and training new reward models and LLMs for each individual’s preferences. To address these challenges, we propose a new paradigm where users specify what they value most within the system message, steering the LLM’s generation behavior to better align with the user’s intentions. However, a naive application of such an approach is non-trivial since LLMs are typically trained on a uniform system message (e.g., “You are a helpful assistant”), which limits their ability to generalize to diverse, unseen system messages. To improve this generalization, we create Multifaceted Collection, augmenting 66k user instructions into 197k system messages through hierarchical user value combinations. Using this dataset, we train a 7B LLM called Janus and test it on 921 prompts from 5 benchmarks (AlpacaEval 2.0, FLASK, Koala, MT-Bench, and Self-Instruct) by adding system messages that reflect unseen user values. JANUS achieves tie+win rate of 75.2%, 72.4%, and 66.4% against Mistral 7B Instruct v0.2, GPT-3.5 Turbo, and GPT-4, respectively. Unexpectedly, on three benchmarks focused on response helpfulness (AlpacaEval 2.0, MT-Bench, Arena Hard Auto v0.1), JANUS also outperforms LLaMA 3 8B Instruct by a +4.0%p, +0.1%p, +3.0%p margin, underscoring that training with a vast array of system messages could also enhance alignment to the general public’s preference as well. Our code, dataset, benchmark, and models are available at https://lklab.kaist.ac.kr/Janus/.

Despite the advances in large language models (LLMs), how they use their knowledge for reasoning is not yet well understood.In this study, we propose a method that deconstructs complex real-world questions into a graph, representing each question as a node with predecessors of background knowledge needed to solve the question. We develop the DepthQA dataset, deconstructing questions into three depths: (i) recalling conceptual knowledge, (ii) applying procedural knowledge, and (iii) analyzing strategic knowledge. Based on a hierarchical graph, we quantify forward discrepancy, a discrepancy in LLM performance on simpler sub-problems versus complex questions. We also measure backward discrepancy where LLMs answer complex questions but struggle with simpler ones. Our analysis shows that smaller models exhibit more discrepancies than larger models. Distinct patterns of discrepancies are observed across model capacity and possibility of training data memorization. Additionally, guiding models from simpler to complex questions through multi-turn interactions improves performance across model sizes, highlighting the importance of structured intermediate steps in knowledge reasoning. This work enhances our understanding of LLM reasoning and suggests ways to improve their problem-solving abilities.