Lightness perception in AR: effects of context and scission

Tasfia Ahsan¹, Minjung Kim², Richard F. Murray¹; ¹Department of Psychology and Centre for Vision Research, York University, ²Independent research scientist

Lightness constancy, the ability to perceive surface reflectance despite changes in illumination and context, is fundamental for interpreting naturalistic scenes in real and virtual environments. Optical see-through augmented reality (OST-AR) presents a particular challenge for lightness constancy, as it additively blends real and virtual content. This increases the luminance of virtual stimuli, and can make low-reflectance virtual surfaces (such as avatars with dark skin tones) appear transparent or washed out. How can we design scenes in AR that support a wide range of perceived reflectance? To address this question, we examined how several fundamental scene properties influence perceived reflectance in AR. In two experiments, participants viewed Voronoi textures in foreground and background planes and reported the lightness of a foreground test patch by comparing it to a printed reference palette with a range of achromatic Munsell values. We manipulated (1) foreground context, including surrounding texture; (2) background luminance and texture; (3) background motion; and (4) depth separation between foreground and background planes. Surprisingly, the strongest effects on perceived lightness came from changes to the foreground context of the test patch, and not from scission cues that distinguished between foreground and background. Reducing foreground context sharply compressed the range of perceived reflectance, and shifted it towards higher values. However, even substantial background manipulations — such as removing background texture, introducing background motion, or reducing depth disparity — produced only modest shifts in perceived reflectance. These findings show that using foregrounds with rich textures and luminance variation is critical for rendering realistic surfaces in OST-AR. These findings are promising developments, since designers have more control over the foreground, virtual stimulus than over the real-world background. We will discuss how these findings are consistent with current theories of lightness perception, and argue that such theories provide a principled foundation for improving AR design.

Acknowledgements: This work was funded by an NSERC Discovery Grant to RFM.