We present a simple, fully decentralized approach to improving robustness of existing unstructured overlay networks against a selective deletion of nodes (e.g., attacks on network hubs). The approach is based on modifying the network by adding or rewiring links at random. We quantify the effectiveness of adding randomness to the network by computing the curve governing the tradeoff between the number of modifications and the increase in robustness. For certain networks, a relatively modest amount of randomization can significantly improve the average path length (quantifying performance degradation) and the size of the largest connected component (quantifying network availability) after an attack.