Grid computing provides new solutions for numerous complex problems. It is an inevitable trend to implement the distributed parallel computing of large-scale problems with the grid. This paper presents two implementations for distributed parallel computing on Globus Toolkit, a wide-used grid environment. The first implementation, Loosely Coupled Parallel Services is used to achieve the large-scale parallel computing that can be broken down into independent sub-jobs by using the corresponding implementation framework, and the second implementation, Grid MPI Parallel Program is able to deal with specialized applications, which can't easily be split up into numerous independent chunks, by using the proposed implementation framework. Finally, Two examples of large-scale parallel computing based on proposed implementations are achieved and the experimental results are shown. We make a beneficial attempt to implement distributed parallel computing on grid computing environments.