profile photo

Chen Yu    

[ Ph.D. Candidate | NUS ]     [ Research Interests: NeRF, SfM, CG, 3D Generation, SLAM ]

I'm a Ph.D. candidate in the National University of Singapore advised by Prof. Gim Hee Lee. I worked at Segway Robotics during 2020.07-2021.12 on enhancing the robustness of mapping algorithms for canteen robots and low-speed autonomous driving cars. Before that, I obtained my M.S. degree in the Department of Computer Science, EECS, Peking University, where I was advised by Prof. Yisong Chen and co-supervised by Prof. Shuhan Shen. I also spent four wonderful years at Beihang University, where I majored in software engineering.

NEWS  

  • [11.01.2024] I am awarded the Dean's Graduate Achievement Award!
  • [03.11.2023] I am awarded the Google PhD Fellowship 2023!
  • [15.08.2023] I received the Research Achievement Award by SoC, NUS!
  • [14.07.2023] Our paper on registering neural radiance fields accepted at ICCV 2023!
  • [28.01.2023] Our paper on neural radiance fields accepted by CVPR 2023!
  • [17.01.2023] Our paper on large scale Structure-from-Motion accepted by ICRA 2023!
  • [10.01.2022] I joined the CVRP lab of SoC, NUS as a phd student!
  • [23.05.2021] Our paper on rotation averaging got accepted at CVPR 2021!
  • [08.07.2020] I joined Segway Robotics to work as a SLAM engineer!
  • [02.07.2020] Our paper on large-scale SfM got accepted at Pattern Recognition 2020!
  • [20.06.2020] I successfully defended my M.S. thesis on distributed large-scale Structure-from-Motion!
  • [15.11.2019] We won the 2nd place in the 3D Reconstruction Track of the 2nd China Virtual Reality and Application Innovation Challenge
  • [20.04.2019] I joined the rising self-driving startup TuSimple as a research intern in the group of High-Definition Localization and Mapping!
  • [05.09.2018] I joined Megvii Beijing as an algorithm intern on the group of SLAM!
  • [11.09.2018]: Our paper on semantic SfM got accepted at PRCV 2018!

PUBLICATIONS

Hover over the to get the overall idea
SCALAR-NeRF

SCALAR-NeRF: SCAlable LARge-scale Neural Radiance Fields for Scene Reconstruction

SCALAR-NeRF aims at reconstructing large-scale scenes. We adopt a coarse-to-fine strategy, which a coarse global model is trained first, and then local models are trained with finer details and further conditioned by a global shared decoder. The final fusion can be guided by the global model.
Yu Chen, Gim Hee Lee

Under Review

Paper Web Page   Code   Cite

                
SCALAR-NeRF

Multi-Scale 3D Gaussian Splatting for Anti-Aliased Rendering

Zhiwen Yan, Weng Fei Low, Yu Chen, Gim Hee Lee

Under Review

Paper Web Page   Code   Cite

                
DReg-NeRF

DReg-NeRF: Deep Registration for Neural Radiance Fields

Given NeRF blocks that are in different coordinate systems, DReg-NeRF registers them together end-to-end. Firstly, we download 1.8K+ 3D meshes from Objaverse and render meshes into images to train our NeRF model. Subsequently, DReg-NeRF extracts the occupancy grid from each NeRF block and feeds them into a 3D FPN and transformer to regress correspondences.
Yu Chen, Gim Hee Lee

The International Conference on Computer Vision (ICCV) 2023

Paper Web Page   Code   Poster   Data   Cite
@article{DBLP:journals/corr/abs-2308-09386,
                  author       = {Yu Chen and
                                  Gim Hee Lee},
                  title        = {DReg-NeRF: Deep Registration for Neural Radiance Fields},
                  journal      = {CoRR},
                  volume       = {abs/2308.09386},
                  year         = {2023},
                  url          = {https://doi.org/10.48550/arXiv.2308.09386},
                  doi          = {10.48550/arXiv.2308.09386},
                  eprinttype    = {arXiv},
                  eprint       = {2308.09386},
                }
DBARF

DBARF: Deep Bundle-Adjusting Generalizable Neural Radiance Fields

DBARF jointly optimizes generalizable neural radiance fields and the relative camera poses without pose initialization. We extract feature maps from nearby views and construct a residual map based on the extracted features. The residual map and rendering loss are minimized during training.
Yu Chen, Gim Hee Lee

The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023

Paper Web Page   Code   Poster   Cite
@InProceedings{Chen_2023_CVPR,
                    author    = {Chen, Yu and Lee, Gim Hee},
                    title     = {DBARF: Deep Bundle-Adjusting Generalizable Neural Radiance Fields},
                    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
                    month     = {June},
                    year      = {2023},
                    pages     = {24-34}
              }
AdaSfM

AdaSfM: From Coarse Global to Fine Incremental Adaptive Structure from Motion

AdaSfM aims to adaptively recover camera poses and structures in large-scale outdoor scenes. We adopt a coarse-to-fine strategy for SfM. In AdaSfM, a coarse global SfM is reconstructed and then used to provide priors to the local incremental SfM reconstruction. The local incremental SfM can be run in a distributed way.
Yu Chen, Zihao Yu, Shu Song, Tianning Yu, Jianming Li, Gim Hee Lee

The IEEE International Conference on Robotics and Automation (ICRA) 2023

Paper Web Page   Code   Poster   Cite
@inproceedings{DBLP:conf/icra/ChenYSYLL23,
                  author       = {Yu Chen and
                                  Zihao Yu and
                                  Shu Song and
                                  Tianning Yu and
                                  Jianming Li and
                                  Gim Hee Lee},
                  title        = {AdaSfM: From Coarse Global to Fine Incremental Adaptive Structure
                                  from Motion},
                  booktitle    = {{IEEE} International Conference on Robotics and Automation},
                  pages        = {2054--2061},
                  publisher    = {{IEEE}},
                  year         = {2023},
                }
global RA

Hybrid Rotation Averaging: A Fast and Robust Rotation Averaging Approach

Hybrid rotation averaging hybrids a global rotation averaging method and a local rotation averaging method. The robustness is enhanced by pre-filtering wrong edges in the triplets. The resulted global rotations are used as priors to regularize the camera poses in incremental SfM.
Yu Chen, Ji Zhao, Laurent Kneip

The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2021

Paper Web Page   Code   Cite
@inproceedings{DBLP:conf/cvpr/Chen0K21,
                    author    = {Yu Chen and
                      Ji Zhao and
                      Laurent Kneip},
                      title     = {Hybrid Rotation Averaging: {A} Fast and Robust Rotation Averaging
                      Approach},
                      booktitle = {{IEEE} Conference on Computer Vision and Pattern Recognition},
                      pages     = {10358--10367},
                      year      = {2021},
                }
global RA

Graph-Based Parallel Large Scale Structure from Motion

GraphSfM splits and stitches scenes based on the graph connectivity. GraphSfM can reconstruct large-scale scenes efficiently in a distributed manner. We made the project publicly available and provided the first open-source distributed SfM implementation.
Yu Chen, Shuhan Shen, Yisong Chen, Guoping Wang

Pattern Recognition (PR) 2020

Paper Web Page   Code   Cite
@article{DBLP:journals/pr/ChenSCW20,
                author    = {Yu Chen and
                            Shuhan Shen and
                            Yisong Chen and
                            Guoping Wang},
                title     = {Graph-based parallel large scale structure from motion},
                journal   = {Pattern Recognition},
                volume    = {107},
                pages     = {107537},
                year      = {2020},
              }
global RA

Large-Scale Structure from Motion with Semantic Constraints of Aerial Images

Yu Chen*, Yao Wang*, Peng Lu, Yisong Chen, Guoping Wang (* denotes equal contribution)

Pattern Recognition and Computer Vision PRCV 2018

Paper Web Page   Code   Dataset   Cite
@inproceedings{DBLP:conf/prcv/ChenWLCW18,
              author    = {Yu Chen and
                          Yao Wang and
                          Peng Lu and
                          Yisong Chen and
                          Guoping Wang},
              title     = {Large-Scale Structure from Motion with Semantic Constraints of Aerial
                          Images},
              booktitle = {Pattern Recognition and Computer Vision - First Chinese Conference},
              series    = {Lecture Notes in Computer Science},
              volume    = {11256},
              pages     = {347--359},
              publisher = {Springer},
              year      = {2018},
            }

AWARDS  

EXPERIENCES

Segway Robotics

, July. 2020 - Dec. 2022
SLAM Algorithm Engineer (with an excellent team)
Visual-Inertial-Odometry SLAM, Marker SLAM, Outdoor Multi-sensor Fusion SfM

Segway Robotics

, Apr. 2020 - Jun. 2020
Algorithm Intern (with Jianming Li)
Visual-Inertial SLAM, Self-Calibration on scooters

TuSimple

, Apr. 2019 - Jul. 2019
Research Intern (with Dr. Ji Zhao)
High-Definition mapping, Rotation Averaging

Megvii Research

, Sep. 2018 - Dec. 2018
Algorithm Intern
Real-time 3D reconstruction, texture mapping