Hou, J. F., Viswanath, S., Dilibal, C., Shende, T., Wu, B., Dagdeviren, C.†, “Real-time 3D ultrasound in augmented reality accelerates training and narrows novice–expert performance gaps”, Nature Communications Engineering, 5:1-17, 2026.

Abstract

Ultrasound imaging requires users to infer three-dimensional anatomy from two-dimensional slices, imposing steep training demands that limit broader adoption. Here we present AR-VIU, a mixed-reality platform that streams real-time volumetric ultrasound as point-cloud renderings into an augmented-reality headset with true-scale spatial registration. To isolate the contributions of volumetric imaging and immersive display, we tested four conditions—two-dimensional imaging on a screen, two-dimensional imaging in augmented reality, three-dimensional imaging on a screen, and three-dimensional imaging in augmented reality—in a controlled study with 18 participants (9 novices, 9 experts). Participants performed object recognition and localization tasks. The augmented-reality volumetric system was associated with the highest accuracy, lowest variability, and near-elimination of the novice-expert performance gap. These results demonstrate technical feasibility for real-time three-dimensional ultrasound in mixed reality and establish an evaluation framework for perceptual and cognitive performance in clinically relevant scenarios, with near-term applications in training and education.