This paper describes a method of estimating position and orientation of a camera for constructing a mixed reality (MR) system. In an MR system, three-dimensional (3-D) virtual objects should be merged into a 3-D real environment at right position in real time. To acquire the user's viewing position and orientation is the main technical problem of constructing an MR system. The user's viewpoint can be determined by estimating position and orientation of a camera using images taken at the viewpoint. Our method estimates the camera pose using screen coordinates of captured color fiducial markers whose 3-D positions are known. The method consists of three algorithms for Perspective n-Points (PnP) problems and uses each algorithm selectively. The method also estimates the screen coordinates of untracked markers that are occluded or are out of the view. It has been found that an experimental MR system that is based on the proposed method can seamlessly merge 3-D virtual objects into a 3-D real environment at right position in real-time and allows users to look around an area in which markers are placed.