In this paper, the observation technique on the micro environments with a real-time virtual reality camera system, which is constructed with a dynamic focusing lens and a smart vision sensor using the “depth from focus” criteria, has been discussed In the operation with a microscope, such as the micro-surgery, DNA operation and etc., the small depth of a focus makes bad controllability to actuate 3D micro objects with actuators. For example, if the focus is on the object, the actuator could not be seen in the microscope. On the other hand, if the focus is on the actuator, the object could not be observed. In this sense, the image, which includes all image information is at teach focus values, i.e. “all-in-focus image” is useful to actuate objects in microenvironments. However, one drawback on the all-in-focus image is that there is no information about the depth of objects. Then, it is also important to reconstruct a micro 3D environment in real-time to actuate objects in the micro virtual environments. Important factors to realize real-time system with the “depth from focus” criteria, which can obtain the all-in-focus image and micro 3Dreconstruction, simultaneously, had been discussed. Finally, the real-time system is constructed by a dynamic focusing lens, which can change the focus in high frequency, and a smart vision system, which is capable in capturing and processing the image data in high speed with SIMD architecture.