We design and implement Virtual Private Grid (VPG), a shell that can easily and securely utilize a large number of machines distributed over multiple administrative domains. Today, many people have an access to a large number of machines across multiple subnets or geographically distributed places. These machines are managed by different administrators, and for the sake of security and administration cost, they impose various restrictions on their use. Methods to work around these restrictions are found on a case-by-case basis and require human intervention. There-fore, it increases the user's cost to utilize remote machines significantly, and consequently decreases the utilization of computational resources. VPG works around these restrictions automatically and can easily utilize a large number of machines in multiple administrative domains. We run VPG on approximately 100 nodes (270 CPUs). Experimental results show that VPG utilizes remote machines more efficiently than other job submission tools.