Alumni
Kyowoon Lee
Postdoctoral Researcher, 2024.10 -
- leekwoon [at] unist.ac.kr
7 notes
Research
2025
Counterfactual Activation Editing
A model-agnostic approach for inference-time control of prosody and pronunciation in pretrained text-to-speech systems.
2025
Counterfactual Activation Editing for Post-hoc Prosody and Mispronunciation Correction in TTS Models
Recent advances in Text-to-Speech (TTS) have significantly improved speech naturalness, increasing the demand for precise prosody control and mispronunciation correction. Existing approaches for prosody manipulation often depend on specialized modules or additional training, limiting their capacity for post-hoc adjustments. Similarly, traditional mispronunciation correction relies on grapheme-to-phoneme dictionaries, making it less practical in low-resource settings. We introduce Counterfactual Activation Editing, a model-agnostic method that manipulates internal representations in a pre-trained TTS model to achieve post-hoc control of prosody and pronunciation. Experimental results show that our method effectively adjusts prosodic features and corrects mispronunciations while preserving synthesis quality. This opens the door to inference-time refinement of TTS outputs without retraining, bridging the gap between pre-trained TTS models and editable speech synthesis.
2025
Local Manifold Approximation and Projection
LoMAP is a training-free method that keeps guided diffusion planning samples near the feasible data manifold in long-horizon tasks.
2025
Local Manifold Approximation and Projection for Manifold-Aware Diffusion Planning
Recent advances in diffusion-based generative modeling have demonstrated significant promise in tackling long-horizon, sparse-reward tasks by leveraging offline datasets. While these approaches have achieved promising results, their reliability remains inconsistent due to the inherent stochastic risk of producing infeasible trajectories, limiting their applicability in safety-critical applications. We identify that the primary cause of these failures is inaccurate guidance during the sampling procedure, and demonstrate the existence of manifold deviation by deriving a lower bound on the guidance gap. To address this challenge, we propose Local Manifold Approximation and Projection (LoMAP), a training-free method that projects the guided sample onto a low-rank subspace approximated from offline datasets, preventing infeasible trajectory generation. We validate our approach on standard offline reinforcement learning benchmarks that involve challenging long-horizon planning. Furthermore, we show that, as a standalone module, LoMAP can be incorporated into the hierarchical diffusion planner, providing further performance enhancements.
2023
Refining Diffusion Planner for Reliable Behavior Synthesis by Automatic Detection of Infeasible Plans
We propose a method to refine unreliable plans generated by diffusion-based planners for long-horizon tasks. We introduce a "restoration gap" metric to guide and improve these plans and uses an attribution map regularizer to prevent adversarial guidance. The approach is validated on benchmarks and enhances explainability by highlighting error-prone transitions in the generated plans.
2023
Variational Curriculum Reinforcement Learning for Unsupervised Discovery of Skills
We introduce Variational Curriculum RL (VCRL), a method for unsupervised skill discovery in reinforcement learning by using intrinsic rewards and curriculum learning. The proposed Value Uncertainty Variational Curriculum (VUVC) accelerates skill acquisition and improves state coverage compared to uniform approaches. The approach is validated on complex tasks, showing improved sample efficiency and successful real-world robotic navigation in a zero-shot setup.
2023
Adaptive and Explainable Deployment of Navigation Skills via Hierarchical Deep Reinforcement Learning
We propose a hierarchical framework for robotic navigation that learns a family of diverse low-level policies and a high-level policy to deploy them adaptively. Instead of relying on a fixed reward function, the method learns multiple navigation skills with varied reward functions and selects the most suitable one in real time. The approach is validated in simulation and real-world settings, demonstrating both adaptability and explainability of the agent's behavior.
12 papers