In recent years, GPU (Graphic Processor Unit) has become an import accelerator for conventional applications. User has to program in GPU-based environments, such as CUDA, and it usually requires detailed tuning for good performances. Also since GPU has high Single Precision (SP) performance while its Double Precision (DP) performance falls short, it has limited application in scientific computing. In this paper, our algorithm aims at accelerating the solving of dense linear equation on hybrid CPU-GPU system. We adopt iterative refinement to utilize the high SP capability of GPUs while achieving DP precision requirements. Specifically, we implement algorithm with utilize both GPU and CPU for computation-intensive parts by overlapping computations. Its performance reaches up to 236GFLOP/s, which is by far better than the result achieved by DP-only algorithms.