Voice over Internet protocol (VoIP) applications must typically choose a tradeoff between the bits allocated for forward error correcting (FEC) and that for the source coding to achieve the best speech quality at a given packet loss rate. In this paper, we present a new scheme to optimize the speech quality subject to the bandwidth constraints and the packet loss rate. The scheme adopts adaptive multi-rate (AMR) speech codec along with a FEC scheme based on exclusive OR (XOR) operations. Retransmission is also taken into account if the round trip time (RTT) is within a certain limit. We use a simplified E-model as objective metric. Subjective listening tests show that our scheme improves the perceptual speech quality significantly compared to the non-adaptive baseline speech transmission system.