Yoonwoo Jeong
Ph.D. student in the Computer Vision Lab at POSTECH.
I am a Ph.D. student in Graduate School of Artificial Intelligence at POSTECH, working as a member of the Computer Vision Lab, advised by Prof. Minsu Cho.
My research centers on integrating 3D geometric understanding into vision foundation models β spanning neural radiance fields, multi-view reasoning, and visual perception in large multimodal models. I am broadly interested in building spatially-aware foundation models that can perceive, reason about, and interact with the 3D world.
Selected Publications
Under embargo
Paper #1 with the DINO team at Meta
Title, authors and details will appear here once the work is public.
Under embargo
Paper #2 with the DINO team at Meta
Title, authors and details will appear here once the work is public.
-
Multimodal LLMs & Generation
Lets image-generating multimodal models think in continuous latents, so more thinking gives better images.
LARC: Latent Reasoning in Continuous Space for Unified Multimodal Models
β Equal advising
- Multimodal LLM
- Generation
Problem
Unified multimodal models reason in discrete text tokens but draw images in a continuous space, so spending more on reasoning does not buy better images.
Key idea
LARC moves the reasoning into continuous space: the model interleaves short text spans with latent steps built from its own hidden states. Curriculum fine-tuning teaches latent reasoning, then self-evolving RL rewards each trace by how much it helps predict the target image.
Result
Beats its base model BAGEL and the token-reasoning Uni-CoT on text-to-image generation, image editing and visual reasoning, and shows clear test-time scaling: more latent steps, better images.
My role
Second author, on continuous latent reasoning that improves image generation in unified models.
BibTeX
@inproceedings{jeon2026larc, title = {LARC: Latent Reasoning in Continuous Space for Unified Multimodal Models}, author = {Jeon, Byungwoo and Jeong, Yoonwoo and Lee, Hyunseok and Cho, Minsu and Shin, Jinwoo}, booktitle = {Conference on Neural Information Processing Systems (NeurIPS)}, year = {2026}, } -
Multimodal LLMs & Generation
Keeps multimodal LLMs looking at the image while they think, so long reasoning stops washing out what they saw.
Vision-aligned Latent Reasoning for Multi-modal Large Language Model
β Equal advising
- Multimodal LLM
Problem
During long chains of thought, visual information gets progressively diluted, so multimodal LLMs struggle with multi-step reasoning and gain little from thinking longer.
Key idea
Before each reasoning step, VaLR generates latent tokens aligned with features from vision encoders such as DINOv3, keeping the reasoning grounded in perceptual cues.
Result
+19.9%p on VSI-Bench over Qwen2.5-VL (33.0% to 52.9%)
Raises VSI-Bench accuracy from 33.0% to 52.9% over Qwen2.5-VL, and keeps improving as reasoning gets longer.
My role
Second author, on injecting knowledge from vision foundation models into multi-view VLMs.
BibTeX
@inproceedings{jeon2026valr, title = {Vision-aligned Latent Reasoning for Multi-modal Large Language Model}, author = {Jeon, Byungwoo and Jeong, Yoonwoo and Lee, Hyunseok and Cho, Minsu and Shin, Jinwoo}, year = {2026}, eprint = {2602.04476}, booktitle = {International Conference on Machine Learning (ICML)}, archiveprefix = {arXiv}, } -
Multimodal LLMs & Generation
Gives multimodal LLMs an explicit 3D map of the scene, so they reason about space instead of guessing it.
Cog3DMap: Multi-View Vision-Language Reasoning with 3D Cognitive Maps
* Equal contribution
- Multimodal LLM
- 3D Understanding
Problem
Multimodal LLMs read multi-view images as mostly semantic tokens, so they must infer the 3D layout of a scene implicitly, and spatial questions suffer.
Key idea
Cog3DMap recurrently builds an explicit 3D memory from the images: one token per spatial location, carrying both semantic and geometric features. The LLM then reasons directly over this compact map.
Result
+8.7%p on VSTI-Bench over the previous best
New state of the art on VSTI-Bench (+8.7%p over the previous best) and VSI-Bench (+3.9%p), while using up to 90.2% fewer visual tokens on long sequences.
My role
Co-first author (equal contribution) of this work on 3D cognitive maps that strengthen the 3D reasoning of VLMs.
BibTeX
@misc{gwak2026cog3dmap, title = {Cog3DMap: Multi-View Vision-Language Reasoning with 3D Cognitive Maps}, author = {Gwak, Chanyoung and Jeong, Yoonwoo and Jeon, Byungwoo and Lee, Hyunseok and Shin, Jinwoo and Cho, Minsu}, year = {2026}, booktitle = {arXiv preprint}, eprint = {2603.23023}, archiveprefix = {arXiv}, } -
3D Understanding & Segmentation
Click one object in one photo β get it segmented consistently in every view.
MV-SAM: Multi-view Promptable Segmentation using Pointmap Guidance
- Segmentation
- 3D Understanding
Problem
SAM-style segmenters lack 3D awareness, so masks drift between views of the same scene unless you run costly per-scene optimization.
Key idea
MV-SAM lifts SAM image embeddings and your prompts into 3D with pointmaps reconstructed from unposed images, so a transformer decodes masks that agree across views. No 3D network or 3D labels: it trains on SA-1B alone.
Result
Outperforms SAM2-Video and matches per-scene optimization baselines on NVOS, SPIn-NeRF, ScanNet++, uCo3D and DL3DV.
My role
First author: an extension of SAM to multi-view inputs, trained without 3D or video datasets.
BibTeX
@inproceedings{jeong2026mvsam, title = {MV-SAM: Multi-view Promptable Segmentation using Pointmap Guidance}, author = {Jeong, Yoonwoo and Sun, Cheng and Wang, Yu-Chiang Frank and Cho, Minsu and Choe, Jaesung}, booktitle = {NVIDIA tech report}, year = {2026}, eprint = {2601.17866}, archiveprefix = {arXiv}, } -
3D Gaussian Splatting
Render 512-D feature maps from 3D Gaussians ~44Γ faster by sampling only the Gaussians that matter.
Quantile Rendering: Efficiently Embedding High-dimensional Features on 3D Gaussian Splatting
- Gaussian Splatting
- Segmentation
- 3D Understanding
Problem
Attaching high-dimensional features such as CLIP embeddings to 3D Gaussians makes rendering slow, because every Gaussian along each ray is blended, so prior work compresses the features.
Key idea
Q-Render samples only the Gaussians with dominant influence along each ray, found by analyzing transmittance changes. A Gaussian Splatting Network (GS-Net) predicts the features, so it generalizes across scenes.
Result
43.7Γ faster 512-D feature rendering
State-of-the-art open-vocabulary 3D segmentation on ScanNet and LeRF-OVS, with real-time rendering of full 512-D features.
My role
First author: a generalizable 3D-GS network for open-vocabulary segmentation.
BibTeX
@inproceedings{jeong2025quantile, title = {Quantile Rendering: Efficiently Embedding High-dimensional Features on 3D Gaussian Splatting}, author = {Jeong, Yoonwoo and Sun, Cheng and Wang, Yu-Chiang Frank and Cho, Minsu and Choe, Jaesung}, booktitle = {NVIDIA tech report}, year = {2025}, eprint = {2512.20927}, archiveprefix = {arXiv}, } -
3D Gaussian Splatting
Rebuild moving scenes in 3D from a casual phone video β no camera poses needed.
RoDyGS: Robust Dynamic Gaussian Splatting for Casual Videos
* Equal contribution
- Gaussian Splatting
- Neural Fields
Problem
Casual videos come without camera poses or scene geometry, which makes optimizing dynamic 3D scenes from them fragile.
Key idea
RoDyGS separates static and dynamic Gaussians and adds motion and geometry regularizers so the learned motion stays physically plausible. It also introduces Kubric-MRig, a benchmark with rich camera and object motion plus multi-view captures.
Result
Significantly outperforms previous pose-free dynamic neural fields and is competitive with pose-free static methods in rendering quality.
My role
Co-first author on this pose-free dynamic neural field pipeline that leverages geometric foundation models.
BibTeX
@inproceedings{jeong2024rodygs, title = {RoDyGS: Robust Dynamic Gaussian Splatting for Casual Videos}, author = {Lee, Junmyeong and Choi, Hoseung and Jeong, Yoonwoo and Cho, Minsu}, booktitle = {arXiv preprint}, year = {2025}, eprint = {2412.03077}, archiveprefix = {arXiv}, } -
Multimodal LLMs & Generation
Turns a text-to-image model into a novel-view generator with a small plug-in, without full fine-tuning.
NVS-Adapter: Plug-and-Play Novel View Synthesis from a Single Image
* Equal contribution
- Generation
- Neural Fields
Problem
Novel view synthesis from one image usually fine-tunes a whole text-to-image model, which is expensive and erodes its ability to generate diverse images.
Key idea
A plug-and-play adapter: view-consistency cross-attention aligns local details across views, and global semantic conditioning keeps generated views faithful to the reference image.
Result
Synthesizes geometrically consistent multi-views with strong benchmark results without full fine-tuning, and works with ControlNet and LoRA out of the box.
My role
First author (equal contribution): a parameter-efficient adapter for text-to-3D generative models.
BibTeX
@inproceedings{jeong2024nvsadapter, title = {NVS-Adapter: Plug-and-Play Novel View Synthesis from a Single Image}, author = {Jeong, Yoonwoo and Lee, Jinwoo and Kim, Chiheon and Cho, Minsu and Lee, Doyup}, booktitle = {European Conference on Computer Vision (ECCV)}, year = {2024}, eprint = {2312.07315}, archiveprefix = {arXiv}, } -
Neural Fields & 3D Reconstruction
Learn a NeRF and the camera itself β lens distortion included β without COLMAP.
Self-calibrating Neural Radiance Fields
- Neural Fields
Problem
NeRF needs accurate camera poses and intrinsics, usually from COLMAP, and real lenses add non-linear distortions that simple camera models miss.
Key idea
Jointly learn the scene and a generic camera model (pinhole, fourth-order radial distortion and a learnable noise model) using photometric consistency plus a new projected ray distance loss.
Result
Learns intrinsics and poses from scratch without COLMAP initialization and improves PSNR over baselines; it plugs into other NeRF variants.
My role
First author of the first pose-free NeRF optimization pipeline; I also led the open-source project.
BibTeX
@article{jeong2022scnerf, title = {Self-calibrating Neural Radiance Fields}, author = {Jeong, Yoonwoo and Ahn, Seokjun and Choy, Christopher and Anandkumar, Animashree and Cho, Minsu and Park, Jaesik}, journal = {{IEEE/CVF} Conference on International Conference on Computer Vision (ICCV)}, year = {2021}, eprint = {2108.13826}, archiveprefix = {arXiv}, }
Experience
-
Paris, France
Research Intern
Meta (DINO Team)
Managers: Maxime Oquab, Piotr Bojanowski
Collaborator: Eyal Benaroche
Developed a video generative model built on DINO representations.
-
Pohang, Korea
Undergraduate Research Intern
Mentor: Jongmin Lee
Developed robust visual correspondence models.
-
Pohang, Korea
Undergraduate Research Intern
Data Intelligence Lab, POSTECH
Worked on neural image style transfer.
Education
-
expected Pohang, Korea
Integrated M.S./Ph.D. in Artificial Intelligence
Pohang University of Science and Technology (POSTECH)
Advisor: Prof. Minsu Cho
Integrating 3D geometric understanding into vision foundation models.
-
Pohang, Korea
B.S. in Computer Science and Engineering
Pohang University of Science and Technology (POSTECH)
Started research as an undergraduate intern in the Data Intelligence Lab and the Computer Vision Lab.
-
Busan, Korea
High school
Korea Science Academy of KAIST
A science academy for gifted students, affiliated with KAIST.
Honors and Awards
Postechian Fellowship (2025)
|
Academic Services
| 2026 β Reviewer: ICLR, CVPR, ECCV |
| 2025 β Reviewer: CVPR, NeurIPS, TPAMI |
| 2024 β Reviewer: ECCV, NeurIPS Datasets and Benchmarks Track |
| 2022 β Reviewer: CVPR, ECCV, NeurIPS Datasets and Benchmarks Track |
| 2021 β Reviewer: WACV |
| 2020 β Reviewer: NeurIPS |