Motivation 💡
Neural fields have been widely adopted for learning novel view synthesis and 3D reconstruction from RGB images by modelling transport of light in the visible spectrum. This workshop focuses on neural fields beyond conventional cameras, including (1) learning neural fields from data from different sensors across the electromagnetic spectrum and beyond, such as lidar, cryo-electron microscopy (cryoEM), thermal, event cameras, acoustic, and more, and (2) modelling associated physics-based differentiable forward models and/or the physics of more complex light transport (reflections, shadows, polarization, diffraction limits, optics, scattering in fog or water, etc.). Our goal is to bring together a diverse group of researchers using neural fields across sensor domains to foster learning and discussion in this growing area.
Schedule ⏰
| 9:00 - 9:05 | Welcome & Introduction | |
| 9:05 - 9:30 | Keynote: Daniel Cremers | |
| 9:30 - 9:55 | Keynote: David Lindell | |
| 9:55 - 10:05 | Paper Spotlight: CryoFormers (Xinhang Liu) | |
| 10:05 - 10:15 | Paper Spotlight: Radar Fields (David Borts) | |
| 10:15 - 10:25 | Paper Spotlight: Beyond Ultra-NeRF (Magdalena Wysocki) | |
| 10:25 - 11:10 | Poster Session & Coffee Break | |
| 11:10 - 11:35 | Keynote: Jon Barron | |
| 11:35 - 12:00 | Keynote: Carl Vondrick | |
| 12:00 - 12:25 | Keynote: Tali Treibitz | |
| 12:25 - 1:00 | Panel Discussion Moderator: Or Litany Panelists: David Lindell, Jon Barron, Tali Treibitz |
Keynote Speakers 🧑🏫
David Lindell
University of Toronto
David Lindell is an Assistant Professor in the Department of Computer Science at the University of Toronto and founding member of the Toronto Computational Imaging Group. His work is at the intersection of machine learning, computational imaging, and computer vision. Along these lines he has worked on next-generation computational imaging systems for imaging around corners and through scattering media, and new machine learning algorithms for representing and processing signals. His work is relevant to a broad range of applications in computer graphics, vision, and remote sensing.
Daniel Cremers
TU Munich
Daniel Cremers is a Professor at Technical University of Munich where he holds the Chair of Computer Vision and Artificial Intelligence. His publications received several awards, including the 'Best Paper of the Year 2003' (Int. Pattern Recognition Society), the 'Olympus Award 2004' (now called 'German Pattern Recognition Award') and the '2005 UCLA Chancellor's Award for Postdoctoral Research'. For pioneering research he received a Starting Grant (2009), two Proof of Concept Grants (2014 \& 2018), a Consolidator Grant (2015) and an Advanced Grant (2020) by the European Research Council. In December 2010 he was listed among "Germany's top 40 researchers below 40" (Capital). On March 1st 2016, Prof. Cremers received the Gottfried Wilhelm Leibniz Award, the biggest award in German academia. In 2022 and 2023, he was listed among the top 10 most influential scholars in robotics of the last decade. He serves as co-founder, advisor and business angel to several startups.
Jon Barron
Jon Barron is a senior staff research scientist at Google Research, where he works on computer vision and machine learning. He received a PhD in Computer Science from the University of California, Berkeley in 2013, where he was advised by Jitendra Malik, and he received a Honours BSc in Computer Science from the University of Toronto in 2007. He received a National Science Foundation Graduate Research Fellowship in 2009, the C.V. Ramamoorthy Distinguished Research Award in 2013, and the PAMI Young Researcher Award in 2020. His works have received awards at ECCV 2016, TPAMI 2016, ECCV 2020, ICCV 2021, CVPR 2022, the 2022 Communications of the ACM, and ICLR 2023.
Tali Treibitz
University of Haifa
Tali Treibitz is heading the Viseaon marine imaging lab in the School of Marine Sciences in the University of Haifa since 2014. She received her PhD degree in electrical engineering from the Technion-Israel Institute of Technology in 2010. Between 2010-2013 she was a post-doctoral researcher in the department of computer science and engineering, in the University of California, San Diego and in the Marine Physical Lab in Scripps Institution of Oceanography. Her lab focuses on cutting edge research in underwater computer vision, scene, color and 3D reconstruction, automatic analysis of scenes, and autonomous decision making based on visual input.
Carl Vondrick
Columbia University
Carl Vondrick is an Associate professor of computer science at Columbia University. His research focuses on computer vision and machine learning. By training machines to observe and interact with their surroundings, his group aims to create robust and versatile models for perception. They often develop visual models that capitalize on large amounts of unlabeled data and transfer across tasks and modalities. Other interests include sound and language, interpretable models and high-level reasoning. Recent work includes 3D reconstruction from shadows and thermal reflections.
Accepted papers 🧑🤝
Xinhang Liu, Yan Zeng, Yifan Qin, Hao Li, Jiakai Zhang, Lan Xu, Jingyi Yu
David Borts, Erich Liang, Tim Brödermann, Andrea Ramazzina, Stefanie Walz, Edoardo Palladin, Jipeng Sun, David Brüggemann, Christos Sakaridis, Luc Van Gool, Mario Bijelic, Felix Heide
Magdalena Wysocki, Mohammad Farid Azampour, Felix Tristram, Benjamin Busam, Nassir Navab
Andrea Ramazzina, Stefanie Walz, Pragyan Dahal, Mario Bijelic, Felix Heide
Anagh Malik, Noah Juravsky, Ryan Po, Gordon Wetzstein, Kyros Kutulakos, David B. Lindell
Mingyang Xie, Haoming Cai, Sachin Shah, Yiran Xu, Brandon Y. Feng, Jia-Bin Huang, Christopher Metzler
Mikhail Okunev, Marc Mapeke, Benjamin Attal, Christian Richardt, Matthew O'Toole, James Tompkin
Ruoshi Liu, Sachit Menon, Chengzhi Mao, Dennis Park, Simon Stent, Carl Vondrick
Ruoshi Liu, Carl Vondrick
Mohamad Qadri, Kevin Zhang, Akshay Hinduja, Michael Kaess, Adithya Pediredla, Christopher Metzler
Benjamin Attal, Dor Verbin, Ben Mildenhall, Peter Hedman, Jonathan T. Barron, Matthew O'Toole, Pratul P. Srinivasan
Sriram Narayanan, Mani Ramanagopal, Mark Sheinin, Aswin C. Sankaranarayanan, Srinivasa Narasimhan
YuchenWeng, Zhengwen Shen, Ruofan Chen, Qi Wang, Shaoze You, Jun Wang
Florian Philipp Stilz, Mert Asim Karaoglu, Felix Tristram, Nassir Navab, Benjamin Busam, Alexander Ladikos
Takuma Nishimura, Andreea Dogaru, Martin Oeggerli, Bernhard Egger
Mert Özer, Maximilian Weiherer, Martin Hundhausen, Bernhard Egger
Mariam Hassan, Florent Forest, Olga Fink, Malcolm Mielle
Zimu Liao, Siyan Chen, furong, Yi Wang, Suzhongling, Hao Luo, Linning Xu, Bo Dai, Hengjie Li, PeiZhilin, Xingcheng ZHANG
Daniel Wedge
Parsa Mirdehghan, Maxx Wu, Wenzheng Chen, David B. Lindell, Kyros Kutulakos
NITHIN Sugavanam, Emre Ertin
Ziyuan Luo, Boxin Shi, Haoliang Li, Renjie Wan
Poster Format: Portrait (vertical) format, with a maximum size of 90x180 cm.
Related Works 🧑🤝
Below is a collection of related works in the field of neural fields beyond conventional cameras. Please feel free to get in touch to add other works as well.- PlatoNeRF: 3D Reconstruction in Plato's Cave via Single-View Two-Bounce Lidar CVPR 2024
- Dynamic LiDAR Re-simulation using Compositional Neural Fields CVPR 2024
- Eclipse: Disambiguating Illumination and Materials using Unintended Shadows CVPR 2024
- E-NeRF: Neural Radiance Fields from a Moving Event Camera RA-L 2023
- Ultra-NeRF: Neural Radiance Fields for Ultrasound Imaging MIDL 2023
- Transient Neural Radiance Fields for Lidar View Synthesis and 3D Reconstruction NeurIPS 2023
- Neural LiDAR Fields for Novel View Synthesis ICCV 2023
- Neural Fields for Structured Lighting ICCV 2023
- E2NeRF: Event Enhanced Neural Radiance Fields from Blurry Images ICCV 2023
- ORCa: Glossy Objects as Radiance Field Cameras CVPR 2023
- Humans as Light Bulbs: 3D Human Reconstruction from Thermal Reflection CVPR 2023
- SeaThru-NeRF: Neural Radiance Fields in Scattering Media CVPR 2023
- EventNeRF: Neural Radiance Fields from a Single Colour Event Camera CVPR 2023
- Neural Interferometry: Image Reconstruction from Astronomical Interferometers Using Transformer-Conditioned Neural Fields AAAI 2022
- Learning Neural Acoustic Fields NeurIPS 2022
- PANDORA: Polarization-Aided Neural Decomposition Of Radiance ECCV 2022
- Medical Neural Radiance Fields for Reconstructing 3D-aware CT-Projections from a Single X-ray EMBC 2022
- TöRF: Time-of-Flight Radiance Fields for Dynamic Scene View Synthesis NeurIPS 2021
- CryoDRGN2: Ab initio neural reconstruction of 3D protein structures from real cryo-EM images ICCV 2021
