In this paper, the parallelization aspects of the accelerated waveform relaxation algorithms for the transient simulation of semiconductor devices on parallel distributed memory computers are mainly studied. These methods are competitive with standard pointwise methods on serial architectures, but are significantly faster on parallel computers. Here we are making use of an improved parallel version of the Conjugate Gradient Squared method (ICGS) combining elements of numerical stability and parallel algorithm design, for solving the resulting sequence of time-varying sparse linear differential-algebraic initial-value problems (IVP) arising at each linearization step with waveform Newton. We reorganize the algorithm such that all inner products, matrix-vector multiplications and vector updates of a single iteration step are independent and communication time required for inner product can be over-lapped efficiently with computation time of vector updates. Therefore, the bottleneck of the performance, namely the cost of global communication on parallel distributed memory computers can be significantly reduced. The resulting ICGS algorithm maintains the favorable properties of the original algorithm while not increasing computational costs. Compared with other accelerated approaches such as convolution SOR and waveform GMRES techniques on waveform relaxation algorithm and pointwise methods, some very preliminary experimental results carried out on a massively parallel systems have demonstrated the advantages of the proposed approach.