Service Oriented-Architecture (SOA) is becoming more and more important as a paradigm for the development of distributed software systems. In this approach user applications are built of a set of collaborating services running in a distributed environment of Internet. Although the idea of SOA appeals strongly to the software development practitioners, there are also numerous pitfalls that cause risks of different severity for SOA software reliability. The paper provides a survey of these critical risks and proposes a framework for Failure Modes, Effects and Criticality Analysis (FMECA) of SOA systems based on Web Services.