In this paper, voltage scaling strategies for scheduling aperiodic tasks under average delay constraints are studied. Dynamic voltage scaling in single processor systems is formulated as a constrained stochastic optimization problem for which the optimal solution can be obtained using a combination of Lagrange relaxation and the value iteration method. For multiprocessor systems, we present a two-phase approach. In the first phase, the speed settings and static workload distribution of the processors are optimized to minimize the total power dissipation. Dynamic voltage scaling techniques are then applied to each individual processor in the second phase. Both homogeneous and heterogeneous systems have been investigated. Based on queueing theory, the proposed algorithms guarantee conformity to the average delay constraint. Moreover, our simulation experiments have shown they are effective for minimizing power consumption.