publications
* Equal contribution † Equal advising
2026
- PUBG Ally Team: 16 more authors, Hyunseung Kim, Sungwoo Kim, Kangwook Lee, Minkyoung Park, Sue Hyun Park, Hyoseok Seol, Yujeong Son, Kiyoon YooTechnical Report
We introduce PUBG Ally, an embodied agent for PUBG: BATTLEGROUNDS that can reason, act autonomously, and play alongside players as a voice-enabled teammate. Building such a teammate requires combining two difficult capabilities: it must perceive and respond to a constantly changing game world under strict latency constraints while interacting naturally with players, keeping its speech synchronized with its actions. Ally therefore combines agentic tool use with real-time game control. A language-model agent uses a controlled interface to inspect game information, interpret player speech, maintain context, decide what to say, and issue high-level action choices that steer a faster control layer for movement, combat, and recovery. Because the player’s and Ally’s speech and actions continually shape each other and the course of the match, training requires data from actual gameplay. We therefore collect data across nearly 39k sessions in which real players play alongside Ally, recording gameplay, player speech, agent decisions, tool use, actions, and player feedback, and use these records for iterative training. To evaluate teammate quality, we use player feedback and preference comparisons to identify gaps between offline evaluations and player preferences, and iteratively refine the evaluation criteria. Deploying Ally in live service further requires low-latency on-device execution and safeguards for player-facing communication, which we address through model compression, context compaction, targeted safety training, runtime guardrails, and memory redaction. During the live service, we surveyed players in 141 countries. Among respondents whose play with Ally was confirmed in game records, positive responses exceeded negative responses by 25.1 percentage points when asked whether they would recommend Ally, with players describing Ally not only as a tool but also as a teammate or companion.
- When Agent Responses Become Interaction State: Safety in a Deployed AI TeammateSue Hyun Park, Minkyoung Park, Byeongju Kim, Seohyeon Jung, Youngin Cho, Jimin Hong, Hyunseung KimNeurIPS 2026 BiAlign Workshop Oral
Safety evaluation typically asks whether an AI system responds safely to a harmful user input. But for agents that interact with users over many turns, a response does more than resolve the current request: it can shape the interaction that follows. We study this problem in a language-model-based, voice-enabled AI teammate deployed in a commercial multiplayer game. Using Korean and English interactions from the live service, we examine how safety failures emerge and persist across turns. We find that the main gap in the deployed safety specification was not a missing category of harmful content, but how the agent participated in the interaction. The agent could repeat unsafe speech, accept problematic relational roles, or make commitments that later resurfaced in otherwise ordinary gameplay. Based on these observations, we revise the safety specification to account for what an agent performs, accepts, and carries forward across turns. We then reconstruct production interaction histories and evaluate whether models can continue safely from them. When the agent has previously maintained the safety boundary, increasing user pressure has relatively little effect on harmless-response rates. In contrast, safety degrades sharply as earlier agent failures make up more of the inherited history, even for capable models given the revised specification at inference time. These findings show that long-horizon safety depends not only on responding safely to the current user, but also on the interaction state that an agent’s own behavior helps create.
- Aligning Embodied Companion Agent Evaluation with User ExperienceSeohyeon Jung*, Sue Hyun Park*, Byeongju Kim*, Jimin Hong, Youngin Cho, Jaewoong Cho, Hyunseung KimNeurIPS 2026 BiAlign Workshop
Embodied companion agents, such as AI teammates in cooperative multiplayer games, converse with users while acting alongside them in shared virtual worlds. We define companion user experience (companion UX) as the user’s perception of the agent as a companion, shaped jointly by dialogue, action, timing, and social appropriateness. Existing evaluation methods often assess speech and action separately, but shared-world interaction makes them mutually grounded in each other and in the evolving game state. Moreover, companion UX is difficult to specify in advance, because what feels helpful, natural, or socially appropriate varies across users and situations and often becomes visible only during play. We frame evaluation of logged or simulated interactions not as a fixed benchmark, but as an iterative alignment process that uses evidence from user tests to revise what behaviors are measured and how they are combined into judgments. In a month-long user test with 1k users and 38k sessions, we construct a paired trace–feedback corpus linking gameplay traces to UX ratings, free-text feedback, and pairwise preferences. Through this iterative alignment process, we develop an evaluation suite whose results better align with user preferences collected in user tests. A subsequently deployed model selected using the refined suite received recommendation ratings comparable to the strongest user-tested model.
- Jiyeon Kim*, Hyunji Lee*, Dylan Zhou*, Sue Hyun Park, Seunghyun Yoon, Trung Bui, Franck Dernoncourt, Sungmin Cha, Minjoon SeoACL 2026Also presented at ICML 2026 CATS Workshop Oral
LLMs operating in dynamic real-world contexts often encounter knowledge that evolves continuously or emerges incrementally. To remain accurate and effective, models must adapt to newly arriving information on the fly. We introduce Online Adaptation to Continual Knowledge Streams(OAKS) to evaluate this capability, establishing a benchmark for online adaptation over streaming, continually updating knowledge. Specifically, the benchmark is structured as a sequence of fine-grained context chunks where facts change dynamically across time intervals. OAKS comprises two datasets: OAKS-BABI and OAKS-Novel, where individual facts evolve multiple times across context chunks. These datasets include dense annotations to measure whether models track changes accurately. Evaluating 14 models with varied inference approaches, we observe significant limitations in current methodologies. Both state-of-the-art models and agentic memory systems fail to adapt robustly on OAKS, demonstrating delays in state-tracking and susceptibility to distraction within streaming environments.
2025
- Seungone Kim, Juyoung Suk, Ji Yong Cho, Shayne Longpre, Chaeeun Kim, Dongkeun Yoon, Guijin Son, Yejin Cho, Sheikh Shafayat, Jinheon Baek, Sue Hyun Park, 21 more authorsNAACL 2025 Best Paper Award, Oral
As language models (LMs) become capable of handling a wide range of tasks, their evaluation is becoming as challenging as their development. Most generation benchmarks currently assess LMs using abstract evaluation criteria-like helpfulness and harmlessness-which often lack the flexibility and granularity of human assessment. Additionally, these benchmarks tend to focus disproportionately on specific capabilities such as instruction following, leading to coverage bias. To overcome these limitations, we introduce the BiGGen Bench, a principled generation benchmark designed to thoroughly evaluate nine distinct capabilities of LMs across 77 diverse tasks. A key feature of the BiGGen Bench is its use of instance-specific evaluation criteria, closely mirroring the nuanced discernment of human evaluation. We apply this benchmark to assess 100 frontier LMs using five evaluator LMs. Our code, data, and evaluation results are all publicly available at https://github.com/prometheus-eval/prometheus-eval.
- Seongyun Lee*, Geewook Kim*, Jiyeon Kim*, Hyunji Lee, Hoyeon Chang, Sue Hyun Park, Minjoon SeoICLR 2025
Vision-Language adaptation (VL adaptation) transforms Large Language Models (LLMs) into Large Vision-Language Models (LVLMs) for multimodal tasks, but this process often compromises the inherent safety capabilities embedded in the original LLMs. Despite potential harmfulness due to weakened safety measures, in-depth analysis on the effects of VL adaptation on safety remains under-explored. This study examines how VL adaptation influences safety and evaluates the impact of safety fine-tuning methods. Our analysis reveals that safety degradation occurs during VL adaptation, even when the training data is safe. While safety tuning techniques like supervised fine-tuning with safety datasets or reinforcement learning from human feedback mitigate some risks, they still lead to safety degradation and a reduction in helpfulness due to over-rejection issues. Further analysis of internal model weights suggests that VL adaptation may impact certain safety-related layers, potentially lowering overall safety levels. Additionally, our findings demonstrate that the objectives of VL adaptation and safety tuning are divergent, which often results in their simultaneous application being suboptimal. To address this, we suggest the weight merging approach as an optimal solution effectively reducing safety degradation while maintaining helpfulness. These insights help guide the development of more reliable and secure LVLMs for real-world applications.
2024
- Seongyun Lee*, Sue Hyun Park*, Seungone Kim, Minjoon SeoNeurIPS 2024Also presented at NeurIPS 2024 Pluralistic Alignment Workshop, ACL 2024 ConvAI Workshop Oral
Although humans inherently have diverse values, current large language model (LLM) alignment methods often assume that aligning LLMs with the general public’s preferences is optimal. A major challenge in adopting a more individualized approach to LLM alignment is its lack of scalability, as it involves repeatedly acquiring preference data and training new reward models and LLMs for each individual’s preferences. To address these challenges, we propose a new paradigm where users specify what they value most within the system message, steering the LLM’s generation behavior to better align with the user’s intentions. However, a naive application of such an approach is non-trivial since LLMs are typically trained on a uniform system message (e.g., “You are a helpful assistant”), which limits their ability to generalize to diverse, unseen system messages. To improve this generalization, we create Multifaceted Collection, augmenting 66k user instructions into 197k system messages through hierarchical user value combinations. Using this dataset, we train a 7B LLM called Janus and test it on 921 prompts from 5 benchmarks (AlpacaEval 2.0, FLASK, Koala, MT-Bench, and Self-Instruct) by adding system messages that reflect unseen user values. JANUS achieves tie+win rate of 75.2%, 72.4%, and 66.4% against Mistral 7B Instruct v0.2, GPT-3.5 Turbo, and GPT-4, respectively. Unexpectedly, on three benchmarks focused on response helpfulness (AlpacaEval 2.0, MT-Bench, Arena Hard Auto v0.1), JANUS also outperforms LLaMA 3 8B Instruct by a +4.0%p, +0.1%p, +3.0%p margin, underscoring that training with a vast array of system messages could also enhance alignment to the general public’s preference as well. Our code, dataset, benchmark, and models are available at https://lklab.kaist.ac.kr/Janus/.
- Miyoung Ko*, Sue Hyun Park*, Joonsuk Park†, Minjoon Seo†EMNLP 2024
Despite the advances in large language models (LLMs), how they use their knowledge for reasoning is not yet well understood.In this study, we propose a method that deconstructs complex real-world questions into a graph, representing each question as a node with predecessors of background knowledge needed to solve the question. We develop the DepthQA dataset, deconstructing questions into three depths: (i) recalling conceptual knowledge, (ii) applying procedural knowledge, and (iii) analyzing strategic knowledge. Based on a hierarchical graph, we quantify forward discrepancy, a discrepancy in LLM performance on simpler sub-problems versus complex questions. We also measure backward discrepancy where LLMs answer complex questions but struggle with simpler ones. Our analysis shows that smaller models exhibit more discrepancies than larger models. Distinct patterns of discrepancies are observed across model capacity and possibility of training data memorization. Additionally, guiding models from simpler to complex questions through multi-turn interactions improves performance across model sizes, highlighting the importance of structured intermediate steps in knowledge reasoning. This work enhances our understanding of LLM reasoning and suggests ways to improve their problem-solving abilities.
- Seongyun Lee*, Seungone Kim*, Sue Hyun Park, Geewook Kim, Minjoon SeoACL 2024 FindingsAlso presented at ACL 2024 ALVR Workshop, ICLR 2024 ME-FoMo Workshop
Assessing long-form responses generated by Vision-Language Models (VLMs) is challenging. It not only requires checking whether the VLM follows the given instruction but also verifying whether the text output is properly grounded on the given image. Inspired by the recent approach of evaluating LMs with LMs, in this work, we propose to evaluate VLMs with VLMs. For this purpose, we present a new feedback dataset called the Perception Collection, encompassing 15K customized score rubrics that users might care about during assessment. Using the Perception Collection, we train Prometheus-Vision, the first open-source VLM evaluator model that can understand the user-defined score criteria during evaluation. Prometheus-Vision shows the highest Pearson correlation with human evaluators and GPT-4V among open-source models, showing its effectiveness for transparent and accessible evaluation of VLMs. We open-source our code, dataset, and model.
- Seongyun Lee, Sue Hyun Park, Yongrae Jo, Minjoon SeoNAACL 2024
Large multimodal models suffer from multimodal hallucination, where they provide incorrect responses misaligned with the given visual information. Recent works have conjectured that one of the reasons behind multimodal hallucination is due to the vision encoder failing to ground on the image properly. To mitigate this issue, we propose a novel approach that leverages self-feedback as visual cues. Building on this approach, we introduce Volcano, a multimodal self-feedback guided revision model. Volcano generates natural language feedback to its initial response based on the provided visual information and utilizes this feedback to self-revise its initial response. Volcano effectively reduces multimodal hallucination and achieves state-of-the-art on MMHal-Bench, POPE, and GAVIE. It also improves on general multimodal abilities and outperforms previous models on MM-Vet and MMBench. Through qualitative analysis, we show that Volcano’s feedback is properly grounded on the image than the initial response. This indicates that Volcano can provide itself with richer visual information through feedback generation, leading to self-correct hallucinations. We publicly release our model, data, and code at https://github.com/kaistAI/Volcanogithub.com/kaistAI/Volcano