AR(Augmented Reality) is an interactive experience of a real-world environment where the objects that reside in the real world are enhanced by computer-generated perceptual information, sometimes across multiple sensory modalities, including visual, auditory, haptic, somatosensory and olfactory.
Users play the guitar with Hololens that displays some support information; (1)proper 3D hand models, (2)positions that must be held with hingers, (3)a mirror of the guitar neck tracked based on structural features and (4)evaluations and advices considering frequency features.
00:00-00:10 C code proper model and positions are displayed, play it correctly.
00:10-00:20 Advices are shown 'cause of played incorrectly.
00:20-00:33 Correctly again, then switched to E code thanks to red indicator become full.
Note my environment details.
- Unity v2017.3.1f1 (main frame)
- Holographic Remoting (for remoting to HMD)
- Visual Studio 2017 v15.6.7 (sub frame; for building to UWP)
- opencv3.4.4 (C++ native)
- Computer: HP Probook 470 G5(Intel Core i7-8550U, 8GB RAM, NVIDEO GeForce 930MX)
- HMD: Microsoft HoloLens 1st
- Mic: Sony ECM-PC60
- Shun Miura, Toshihiko Ando: "A Suggestion of an AR-supported System for Learning Guitar Playing Using HMD", The Institute of Electronics, Information and Communication Engineers Society Conference 2020, D-15-10, p.124(2020).
- Shun Miura, Toshihiko Ando: "A Suggestion of an AR-supported System for Learning Guitar Playing", Information Processing Society of Japan Interaction 2020, 1P-77, pp.461-463(2020).
- Shun Miura, Toshihiko Ando: "Improvement of an AR-supported System for Learning Guitar Playing", The 82nd National Convention of Information Processing Society of Japan, 2ZF-06, pp.4-529-4-530(2020).
- Shun Miura, Toshihiko Ando: "Improvement of an AR-supported System for Learning Guitar Playing", The 82nd National Convention of Information Processing Society of Japan, 2ZF-06, pp.4-529-4-530(2020).
- Shun Miura, Takuo Suganuma, Toshihiko Ando: "Design of an AR-supported System for Learning Guitar Playing", The 81st National Convention of Information Processing Society of Japan, 2ZJ-08, pp.4-803-4-804(2019).