Immersive virtual environments (VEs) provide participants with computer-generated environments filled with virtual objects to assist in learning, training, and practicing dangerous and/or expensive tasks. But for certain tasks, does having every object being virtual inhibit the interactivity? Further, does the virtual object?s visual fidelity affect performance? Overall VE effectiveness may be reduced if users spend most of their time and cognitive capacity learning how to interact and adapting to interacting with a purely virtual environment. We investigated how handling real objects and how self-avatar visual fidelity affects performance on a spatial cognitive task in an immersive VE. We compared participants? performance on a block arrangement task in both a real-space environment and several virtual and hybrid environments. The results showed that manipulating real objects in a VE brings task performance closer to that of real space, compared to manipulating virtual objects.