Problem
Classical AR struggles with two things: making a virtual object hide correctly behind real geometry, and making it light up correctly under the scene's actual lighting. Traditional pipelines approximate each of these with handcrafted passes. NeRFs — implicit volumetric scene representations — give you a principled way to reason about both at once.
Approach
I trained per-object NeRFs and treated them as composable blocks. A Generalizing module at inference time merged blocks into an intermediate rendering, resolving both occlusion and illumination inside the NeRF's native rendering equation rather than as a post-process.
Key insight: if you render the virtual object inside the same NeRF volume as the scene capture, occlusion falls out for free from the volumetric integration, and illumination can be learned from the scene's own radiance field rather than estimated separately.
Contribution
- A composable NeRF block structure for AR scenes.
- A Generalizing module for merged intermediate renderings at inference time.
- Empirical study of the trade-offs vs. classical occlusion/illumination passes.
Status
Poster presented at the CSE Research Symposium, IIT Bombay (2023). The research informs the evaluation and rendering intuitions I apply to generative-model work today.
This was my first real exposure to the discipline of evaluating generative models. Everything I do today on LLM-judge rigs and held-out evaluation traces back to learning how to measure a NeRF honestly.
Stack
PyTorch · NeRF · OpenCV · CUDA · Blender (for synthetic scene generation)