Current Issue

The Korean Journal of Cognitve & Biological Psychology - Vol. 33, No. 3

[ Original Article ]
The Korean Journal of Cognitve & Biological Psychology - Vol. 33, No. 3, pp.121-131
Abbreviation: KCBPA
ISSN: 1226-9654 (Print)
Print publication date 30 Jul 2021
Received 17 Feb 2021 Revised 13 Apr 2021 Accepted 03 May 2021
DOI: https://doi.org/10.22172/cogbio.2021.33.3.001

증강현실 환경에서의 3차원 깊이 지각
주성준 ; 이주형
부산대학교 심리학과

Three-dimensional depth perception in augmented reality
Sung Jun Joo ; Ju Hyeong Lee
Department of Psychology, Pusan National University
Correspondence to : 주성준, 부산대학교 심리학과, (46241) 부산광역시 금정구 부산대학로 63번길 2 사회관 119호 E-mail: sjjoo@pusan.ac.kr


ⓒ The Korean Society for Cognitive and Biological Psychology
Funding Information ▼

초록

증강현실은 4차 산업혁명의 근간이 되는 핵심 기술 중의 하나이다. 증강현실 환경에서 가상 물체는 실제 물리적인 환경에 제시되고, 사용자는 물리적인 물체뿐만 아니라 가상 물체와 상호작용을 할 수 있다. 사실적이고 몰입감 있는 증강현실 환경을 구축하기 위해서 가상 물체에 대한 오류 없는 3차원 깊이 지각은 매우 중요하다. 하지만 가상현실 기술과는 달리 증강현실 환경에 제시된 가상 물체의 3차원 깊이 지각에 대한 연구는 아직 미미한 단계이다. 또한, 증강현실 환경에 제시된 가상 물체를 활용한 사전 연구는 가상 물체의 3차원 깊이지각에 대해 과소추정 또는 과대 추정을 보이는 혼재된 결과를 보였다. 본 논문에서는 증강현실 환경에 제시된 가상 물체에 대한 3차원 깊이 지각에 대한 결과를 요약하였다. 시각 시스템이 3차원 시지각을 위해 활용하는 단서들이 가상 환경에서 어떻게 다른가에 대해 논의하였다. 특히, 양안 부등과 수렴이 증강현실 환경에서 3차원 깊이 지각 오차를 야기하는 주된 단서임을 살펴 보았다. 끝으로 본 논문에서는 3차원 깊이 지각 오차를 수정하는 방법과 수렴조절 불일치와 같은 몰입감 있는 증강현실 환경 구축의 문제점을 해결할 수 있는 새로운 디스플레이 기술에 대해 소개하였다.

Abstract

Augmented reality (AR) is one of the essential technologies for the fourth industrial revolution. In AR, virtual objects are embedded in the physical reality and observers can freely interact with virtual objects as well as physical objects. In order to create a realistic and immersive AR environment, three-dimensional (3D) depth perception of virtual objects must be errorless or at least with very little error. However, unlike virtual reality, 3D depth perception of virtual objects in AR has been less studied and the literature shows mixed results as to whether 3D depth perception of virtual objects is overestimated or underestimated. We reviewed 3D depth perception of virtual objects in AR. We discuss in depth how the 3D cues that the visual system uses to calculate or extract 3D egocentric distance (the distance between an observer and an object) may differ in AR. Specifically, binocular disparity and vergence might be the most important cues that cause 3D depth perception errors in AR. We introduce a new display technology, which might provide more immersive AR by resolving some problems in AR such as vergence-accommodation mismatch.


Keywords: augmented reality, 3D perception, depth perception, depth matching
키워드: 증강현실, 3차원 시지각, 깊이 지각, 깊이 매칭

Acknowledgments

이 논문은 부산대학교 기본연구지원사업에 의하여 연구되었음.


References
1. Azuma, R. T. (1997). A survey of augmented reality. Presence: Teleoperators & Virtual Environments, 6(4), 355-385.
2. Banks, M. S., Hoffman, D. M., Kim, J., Wetzstein, G. (2016). 3D Displays. Annual Review of Vision Science, 2, 397-435.
3. Bimber, O., & Raskar, R. (2005). Spatial augmented reality: Merging real and virtual worlds. A K Peters/CRC Press.
4. Bruder, G., Steinicke, F., Wieland, P., & Lappe, M. (2011). Tuning self-motion perception in virtual reality with visual illusions. Institute of Electrical and Electronics Engineers (IEEE) Transactions on Visualization and Computer Graphics, 18(7), 1068-1078.
5. Chicchi Giglioli, I. A., Pallavicini, F., Pedroli, E., Serino, S., & Riva, G. (2015). Augmented reality: A brand new challenge for the assessment and treatment of psychological disorders. Computational and Mathematical Methods in Medicine, 2015, 1-12.
6. Creem-Regehr, S. H., Stefanucci, J. K., & Thompson, W. B. (2015). Perceiving absolute scale in virtual environments: How theory and application have mutually informed the role of body-based perception. In Psychology of Learning and Motivation, 62, 195-224.
7. Creem-Regehr, S. H., Willemsen, P., Gooch, A. A., & Thompson, W. B. (2005). The influence of restricted viewing conditions on egocentric distance perception: Implications for real and virtual indoor environments. Perception, 34(2), 191-204.
8. Cumming, B. G., & DeAngelis, G. C. (2001). The physiology of stereopsis. Annual Review of Neuroscience, 24(1), 203-238.
9. Diaz, C., Walker, M., Szafir, D. A., & Szafir, D. (2017). Designing for depth perceptions in augmented reality. 2017 Institute of Electrical and Electronics Engineers (IEEE) International Symposium on Mixed and Augmented Reality (ISMAR), 111-122.
10. Drascic, D., & Milgram, P. (1996). Perceptual issues in augmented reality. Stereoscopic Displays and Virtual Reality Systems III, 2653, 123-134.
11. Durgin, F. H., Proffitt, D. R., Olson, T. J., & Reinke, K. S. (1995). Comparing depth from motion with depth from binocular disparity. Journal of Experimental Psychology: Human Perception and Performance, 21(3), 679-699.
12. El Jamiy, F., & Marsh, R. (2019). Survey on depth perception in head mounted displays: distance estimation in virtual reality, augmented reality, and mixed reality. The Institution of Engineering and Technology (IET) Image Processing, 13(5), 707-712.
13. El-Seoud, S. A., Mady, A., & Rashed, E. (2019). An interactive mixed reality ray tracing rendering mobile application of medical data in minimally invasive surgeries. International Journal of Interactive Mobile Technologies, 13(3), 29-39.
14. Ellis, S. R., & Bucher, U. J. (1994). Distance perception of stereoscopically presented virtual objects optically superimposed on physical objects by a head-mounted see-through display. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 38(19), 1300-1304.
15. Erkelens, C. J., & Collewijn, H. (1985). Eye movements and stereopsis during dichoptic viewing of moving random-dot stereograms. Vision Research, 25(11), 1689-1700.
16. Ferris, S. H. (1972). Motion parallax and absolute distance. Journal of Experimental Psychology, 95(2), 258-263.
17. Foley, J. M. (1980). Binocular distance perception. Psychological Review, 87(5), 411-434.
18. Fulvio, J. M., & Rokers, B. (2017). Use of cues in virtual reality depends on visual feedback. Scientific Reports, 7(1), 1-13.
19. Fulvio, J. M., Miao, H., & Rokers, B. (2021). Head jitter enhances three-dimensional motion perception. Journal of Vision, 21(3), 12-12.
20. Gao, Y., Peillard, E., Normand, J. M., Moreau, G., Liu, Y., & Wang, Y. (2019). Influence of virtual objects' shadows and lighting coherence on distance perception in optical seethrough augmented reality. Journal of the Society for Information Display, 28(2), 117-135.
21. Gogel, W. C., & Da Silva, J. A. (1987). Familiar size and the theory of off-sized perceptions. Perception & Psychophysics, 41(4), 318-328.
22. Hartley, R., & Zisserman, A. (2003). Estimation-2D projective transformations. Multiple View Geometry, 2nd ed.; Cambridge University Press: Cambridge, UK, 87-90.
23. Huang, F. C., Chen, K., Wetzstein, G. (2015). The light field stereoscope: Immersive computer graphics via factored near-eye light field displays with focus cues. The Association for Computing Machinery (ACM) Transactions on Graphics, 34(4), 1-12.
24. Huang, K. T., Ball, C., Francis, J., Ratan, R., Boumis, J., & Fordham, J. (2019). Augmented versus virtual reality in education: An exploratory study examining science knowledge retention when using augmented reality/virtual reality mobile applications. Cyberpsychology, Behavior, and Social Networking, 22(2), 105-110.
25. Jones, J. A., Swan II, J. E., Singh, G., Franck, J., & Ellis, S. R. (2009). The effects of continued exposure to medium field augmented and virtual reality on the perception of egocentric depth. Proceedings of the 6th Symposium on Applied Perception in Graphics and Visualization, 138.
26. Jones, J. A., Swan II, J. E., Singh, G., Kolstad, E., & Ellis, S. R. (2008). The effects of virtual reality, augmented reality, and motion parallax on egocentric depth perception. Proceedings of the 5th Symposium on Applied Perception in Graphics and Visualization, 9-14.
27. Kaufmann, H., & Schmalstieg, D. (2002). Mathematics and geometry education with collaborative augmented reality. The Association for Computing Machinery (ACM) Special Interest Group on GRAPHics and Interactive Techniques (SIGGRAPH) 2002 Conference Abstracts and Applications, 37-41.
28. Kham, K., & Lee, J.-H. (2003). The effect of inter-pupillary distance on stereopsis. Korean Journal of Cognitive Science, 14(3), 37-49.
29. Kytö, M., Mäkinen, A., Häkkinen, J., & Oittinen, P. (2013). Improving relative depth judgments in augmented reality with auxiliary augmentations. The Association for Computing Machinery (ACM) Transactions on Applied Perception (TAP), 10(1), 1-21.
30. Ling, H. (2017). Augmented reality in reality. Institute of Electrical and Electronics Engineers (IEEE) MultiMedia, 24(3), 10-15.
31. Linton, P. (2020). Does vision extract absolute distance from vergence?. Attention, Perception, & Psychophysics, 82, 3176-3195.
32. Livingston, M. A., Ai, Z., & Decker, J. W. (2009). A user study towards understanding stereo perception in head-worn augmented reality displays. In 2009 8th Institute of Electrical and Electronics Engineers (IEEE) International Symposium on Mixed and Augmented Reality, 53-56.
33. Livingston, M. A., Swan, J. E., Gabbard, J. L., Hollerer, T. H., Hix, D., Julier, S. J., Baillot, Y., & Brown, D. (2003). Resolving multiple occluded layers in augmented reality. In The Second Institute of Electrical and Electronics Engineers (IEEE) and The Association for Computing Machinery (ACM) International Symposium on Mixed and Augmented Reality, 2003. Proceedings, 56-65.
34. McIntire, J. P., Havig, P. R., Harrington, L. K., Wright, S. T., Watamaniuk, S. N., & Heft, E. L. (2014). Clinically normal stereopsis does not ensure a performance benefit from stereoscopic 3D depth cues. 3D Research, 5(3), 1-23.
35. Mon-Williams, M., & Tresilian, J. R. (1999). Some recent studies on the extraretinal contribution to distance perception. Perception, 28(2), 167-181.
36. Mon-Williams, M., & Tresilian, J. R. (2000). Ordinal depth information from accommodation?. Ergonomics, 43(3), 391-404.
37. Mon-Williams, M., Tresilian, J. R., & Roberts, A. (2000). Vergence provides veridical depth perception from horizontal retinal image disparities. Experimental Brain Research, 133(3), 407-413.
38. Norman, J. F., Dawson, T. E., & Butler, A. K. (2000). The effects of age upon the perception of depth and 3-D shape from differential motion and binocular disparity. Perception, 29(11), 1335-1359.
39. Pallavicini, F., Serino, S., Cipresso, P., Pedroli, E., Chicchi Giglioli, I. A., Chirico, A., ... & Riva, G. (2016). Testing augmented reality for cue exposure in obese patients: an exploratory study. Cyberpsychology, Behavior, and Social Networking, 19(2), 107-114.
40. Palmisano, S., Gillam, B., Govan, D. G., Allison, R. S., & Harris, J. M. (2010). Stereoscopic perception of real depths at large distances. Journal of Vision, 10(6), 1-16.
41. Parker, A. J. (2007). Binocular depth perception and the cerebral cortex. Nature Reviews Neuroscience, 8(5), 379-391.
42. Peillard, E., Argelaguet, F., Normand, J. M., Lécuyer, A., & Moreau, G. (2019). Studying Exocentric Distance Perception in Optical See-Through Augmented Reality. 2019 Institute of Electrical and Electronics Engineers (IEEE) International Symposium on Mixed and Augmented Reality (ISMAR), 115-122.
43. Philbeck, J. W., & Loomis, J. M. (1997). Comparison of two indicators of perceived egocentric distance under full-cue and reduced-cue conditions. Journal of Experimental Psychology: Human Perception and Performance, 23(1), 72-85.
44. Poggio, G. F. (1995). Mechanisms of stereopsis in monkey visual cortex. Cerebral Cortex, 5(3), 193-204.
45. Pouget, A., & Sejnowski, T. J. (1994). A neural model of the cortical representation of egocentric distance. Cerebral Cortex, 4, 314-314.
46. Qian, N. (1997). Binocular disparity and the perception of depth. Neuron, 18(3), 359-368.
47. Robinett, W., & Rolland, J. P. (1991). A computational model for the stereographic optics of a head-mounted display. In SPIE 1457 Stereographic Displays and Applications, 11, 140-160.
48. Rolland, J. P., Gibson, W., & Ariely, D. (1995). Towards quantifying depth and size perception in virtual environments. Presence: Teleoperators & Virtual Environments, 4(1), 24-49.
49. Rolland, J. P., Meyer, C., Arthur, K., & Rinalducci, E. (2002). Method of adjustments versus method of constant stimuli in the quantification of accuracy and precision of rendered depth in head-mounted displays. Presence: Teleoperators & Virtual Environments, 11(6), 610-625.
50. Rolland, J. P., Meyer, C., Arthur, K., & Rinalducci, E. (2002). Method of adjustments versus method of constant stimuli in the quantification of accuracy and precision of rendered depth in head-mounted displays. Presence: Teleoperators & Virtual Environments, 11(6), 610-625.
51. Rosales, C. S., Pointon, G., Adams, H., Stefanucci, J., Creem-Regehr, S., Thompson, W. B., & Bodenheimer, B. (2019). Distance Judgments to On-and Off-Ground Objects in Augmented Reality. 2019 Institute of Electrical and Electronics Engineers (IEEE) Conference on Virtual Reality and 3D User Interfaces (VR), 237-243.
52. Sahm, C. S., Creem-Regehr, S. H., Thompson, W. B., & Willemsen, P. (2005). Throwing versus walking as indicators of distance perception in similar real and virtual environments. The Association for Computing Machinery (ACM) Transactions on Applied Perception (TAP), 2(1), 35-45.
53. Soranzo, A., Lugrin, J. L., & Wilson, C. J. (2013). The effects of belongingness on the simultaneous lightness contrast: a virtual reality study. Vision Research, 86, 97-106.
54. Statistics Korea. (2018). Annual survey of industrial activity in 2017.
55. Steinman, R. M., Cushman, W. B., & Martins, A. J. (1982). The precision of gaze. Human Neurobiology, 1, 97-109.
56. Swan, J. E., Jones, A., Kolstad, E., Livingston, M. A., & Smallman, H. S. (2007). Egocentric depth judgments in optical, see-through augmented reality. Institute of Electrical and Electronics Engineers (IEEE) Transactions on Visualization and Computer Graphics, 13(3), 429-442.
57. Thompson, W., Fleming, R., Creem-Regehr, S., & Stefanucci, J. K. (2011). Visual perception from a computer graphics perspective. CRC Press.
58. Tittle, J. S., Norman, J. F., Perotti, V. J., & Phillips, F. (1998). The perception of scale-dependent and scale-independent surface structure from binocular disparity, texture, and shading. Perception, 27(2), 147-166.
59. Tittle, J. S., Todd, J. T., Perotti, V. J., & Norman, J. F. (1995). Systematic distortion of perceived three-dimensional structure from motion and binocular stereopsis. Journal of Experimental Psychology: Human Perception and Performance, 21(3), 663-678.
60. Ventura, S., Baños, R. M., Botella, C., & Mohamudally, N. (2018). Virtual and augmented reality: New frontiers for clinical psychology. State of the Art Virtual Reality and Augmented Reality Knowhow, 99-118.
61. Vishwanath, D. (2014). Toward a new theory of stereopsis. Psychological Review, 121(2), 151-178.
62. Von Hofsten, C. (1976). The role of convergence in visual space perception. Vision Research, 16(2), 193-198.
63. Wanger, L. C., Ferwerda, J. A., & Greenberg, D. P. (1992). Perceiving spatial relationships in computer-generated images. Institute of Electrical and Electronics Engineers (IEEE) Computer Graphics and Applications, 12(3), 44-58.
64. Westheimer, G. (1979). Cooperative neural processes involved in stereoscopic acuity. Experimental Brain Research, 36(3), 585-597.
65. Willemsen, P., Gooch, A. A., Thompson, W. B., & Creem-Regehr, S. H. (2008). Effects of stereo viewing conditions on distance perception in virtual environments. Presence: Teleoperators and Virtual Environments, 17(1), 91-101.
66. Wu, W., Tošić, I., Berkner, K., & Balram, N. (2015). Depth-disparity calibration for augmented reality on binocular optical see-through displays. Proceedings of the 6th The Association for Computing Machinery (ACM) Multimedia Systems Conference, 120-129.