Computing Commodities Market in the Cloud
by Linda Dailey Paulson
Once an IT buzzword, cloud computing has evolved over the past couple of years to represent a widespread practice for using remote computing resources. And now it's catching on among enterprises as a financial commodity.
When Enomaly, a Toronto-based, cloud service software provider, launched SpotCloud.com earlier this year, the company called it the world's first cloud computing spot marketplace, allowing customers to buy and sell computing capacity in a global open market.
Reuven Cohen, Enomaly's founder and chief technologist, distinguishes SpotCloud from traditional cloud-hosting services, such as Amazon's Elastic Compute Cloud (EC2), which sell computing resources to customers. SpotCloud is focused on providing a marketplace for the exchange of computing capacity anywhere in the world at the best price, says Cohen.
Some experts think it's too still early for this kind of market to flourish. Enterprises are focused on building private clouds at present, according to George Hamilton, the Yankee Group's principal analyst in enterprise research." Not that spot markets don't exist and won't evolve," he says. "They do and will."
Evolving Computing Service Markets
Sun Microsystems and Archipelago Exchange proposed an exchange for excess CPU cycles in 2005. However, when Archipelago merged with the New York Stock Exchange before the launch, the idea was shelved. That proposal was for a distributed grid or batch-type applications, Cohen says, whereas SpotCloud uses virtual machines (VMs).
Amazon Web Services (AWS) was the first company to offer cloud capacity on a spot market/spot pricing basis on a useful scale, says Jonathan Eunice, president and principal analyst at Illuminata, a technology analysis firm. Amazon's Spot Instances let buyers bid on unused Amazon EC2 Instances. This provides cloud computing capacity, often at a market-driven price lower than standard Amazon pricing.
Cohen says SpotCloud evolved from Enomaly's infrastructure-as-a-service software platform for web-hosting firms and data center service providers. They saw buyers that couldn’t find regional cloud providers and, at the same time, providers that faced challenges finding new customers. SpotCloud was formed as an intermediary between buyers and sellers. It works much like other commodity spot markets, such as those for agricultural products and energy, says Cohen, except it trades in computing capacity. Firms with excess capacity, such as data centers, sell that surplus so that firms with short-term needs can bid for it.
The model is similar to digital distribution platforms for mobile devices, such as Apple's App Store or the Google Android Market, says Cohen. Buyers meet sellers in a central location, and sellers define prices — either statically or dynamically — via the SpotCloud API. Prices fluctuate on the basis of factors such as usage or demand. The seller defines the price duration and can offer a renewal at a price that depends on market conditions.
Participating in SpotCloud's Market
SpotCloud takes a percentage of each transaction. Cohen didn't say how the percentage is determined, but published reports suggest the transaction fee is between 10 and 30 percent of the selling price.
Participating firms don't have to reveal their identity, although they can if they choose, says Cohen. This permits some sellers to offer their excess capacity at reduced costs without exposing their regular pricing. Others might choose instead to reveal their identity and charge more based on brand recognition.
The SpotCloud platform is built on the Google App Engine, which Cohen says provides global coverage and affordable bandwidth. The application is delivered and provisioned based on raw disk images. It monitors usage and automatically bills buyers and pays providers. An algorithm records and reports a rating for each SpotCloud provider according to metrics such as bandwidth, network performance, and availability.
Buyers can specify the country or city in which to run their VMs. This capability to keeping computing resources closer to the end user can improve performance, says Cohen, especially for applications such as load testing, system optimization, and content delivery.
To use SpotCloud, sellers need a cloud infrastructure platform — either Enomaly's Elastic Computing Platform (ECP) or the OpenStack platform that Rackspace developed with NASA. With the SpotCloud API, users need have only an IP address and the capabilities to start and stop a VM and to report utilization data.
SpotCloud vets market participants, says Cohen, but also recommends that buyers fully secure their appliances and use a stateless approach to operations, sending no data with the VM but instead requesting it after deployment.
Timing and the Market
Cohen says that prior attempts to establish computing resource exchanges have failed because of timing. Cloud computing's popularity has helped people understand the advantages of using remote, regional computing resources. In its early trial and current beta releases, SpotCloud has already attracted more than 1,200 service providers. If 100 percent of their resources were available, they would provide access to more than 100,000 servers — that's enough to support 400 terabytes of RAM or 100 petabytes of storage.
Even though Cohen thinks the time is right, he also says that not every company or industry is a fit for the SpotCloud marketplace. The service would be ideal for location-specific testing or for testing application scaling across a thousand servers, he says. It's not appropriate for ongoing computing needs that require a specific level of service. For that requirement, enterprises would still need a hosting service.
But the Yankee Group's Hamilton doesn’t see large-scale spot markets as feasible until numerous concerns are satisfactorily resolved. These include security, compliance concerns, data protection, and the multi-tenancy issues that arise when several different firms use the same physical computing infrastructure.
Illuminata's Eunice says it’s also inherently difficult to establish a commodity market without a true equivalence in the product being sold. "If someone's using a broker or a market to buy, say, 100 units of cloud computing power," he explains, "they need to know what those 100 units mean and that they'll be satisfied regardless of whether their order is fulfilled by provider A, B, or C."
Eunice thinks it's too early for asset-like pricing in cloud computing. "Commodity and brokered markets require a high level of sameness among the different vendors," he says. "It will never be exact, but it has to be close." Right now, the services provided by Amazon differ from those provided by GoGrid, Rackspace, and Verizon in terms of expected level of service, management tools, available programming interfaces, and other details. Even though the definitions have been conceptually similar, he doesn't yet see the interchangeability necessary to create "a real spot market that goes beyond just a single cloud service provider."
Linda Dailey Paulson is a freelance technology writer based in Portland, Oregon. Contact her at ldpaulson@yahoo.com.
