Streaming video over Internet suffers varied quality degradation from packet delay, loss and spatiotemporal error propagation. This paper first proposes a perceptually optimized INTRA refreshing (POIR) H.264 video coding scheme, which stops error propagation quickly. It was inspired by the high-level recognition feature of the human visual system (HVS), and takes into consideration the error concealment (EC) function of decoder in an effort to optimize the end-toend distortion. Further, a spatio-temporal EC method at decoder based on refined boundary match criterion is introduced. Simulation illustrates that the POIR scheme:(1) maintains a best rate-distortion tradeoff point by selecting the number and placement of INTRA updated macroblock; (2) outperforms random intra refresh (RIR) by several dBs in terms of PSNR depending on sequences, especially at low coding bit rate and at high packet loss rate; (3) delivers a good tradeoff between performance and computational load, which is critical for real-time situations and portable devices.