Big Data Jobs will be Plentiful

Lack of skilled workers could limit firms' grasp of opportunities

By Peggy Albright

Everybody, it seems, recognizes the opportunity to harness and exploit big data for the benefit of business and society. And everybody, it seems, is concerned about the lack of enough computing professionals in the market who have the particular skills considered essential to this new field.

The market conditions are great for those who want to pursue a career in big data. While many will seek out extra training or academic coursework to gain new expertise needed for these new job opportunities, they might want to make sure they specialize in a particular vertical market, too.

"These problems are going to be very vertically oriented," said Brian Hopkins, a principal analyst at Forrester Research. "I don't think there is going to be a huge demand for generalist data scientists or developers who don't know a particular business skill."

To give some examples, Hopkins noted that a retail chain might want to perform big data analyses for detailed market studies or supply chain optimization. A telecommunications service provider might want to analyze data to find ways to understand customer retention and reduce churn. An energy firm might want to evaluate data gathered from sensors placed on oil rigs or along pipelines to discover opportunities to make operations more efficient.

"These are very different problems with very different mathematics and different analyses and different outcomes," Hopkins said. "Developers who can learn about a particular business vertical and the complexities of the analytics needed to get the value of the data will be well-positioned within their companies or across a number of companies."

Big data needs

Big data has already reached every sector in the global economy, according to the McKinsey Global Institute. The institute has noted that the computer, electronics products, and IT sectors have already begun to see productivity and economic gains from big data and that the insurance and government services industries will also benefit substantially.

McKinsey expects certain industries and business to drive particular, substantial business value from big data. It determined, for example, that a typical retailer in the United States can improve operating margins by 60 percent by embracing big data. It said the US healthcare industry can use big data to create annual financial benefits of about $300 billion, primarily by reducing the costs to deliver healthcare. The firm foresees about $100 billion in new revenues worldwide for companies that provide services based on personal location data, and up to $700 billion in value to the customers who use those services.

The firm noted that the availability of a skilled workforce will affect companies' abilities to go after these lucrative opportunities. Within the next six years, it said, businesses in the United States will need 140,000 to 190,000 people for "deep analytical" positions and they will also need 1.5 million managers and analysts to analyze big data and make decisions based on their findings.

To help spur the big data industry while addressing government needs, the Obama Administration just announced that it is putting up US $200 million in new R&D funding to pursue US big data projects at the National Science Foundation, the National Institutes of Health, the Department of Defense, the Department of Energy, and the US Geological Survey.

Expanding the workforce needed to develop and use these technologies is a key objective of the initiative, which will also partner with universities to develop new graduate programs to train data scientists and other important experts. The program includes funding for training and contests.

Big data was also a leading topic at the recent World Economic Forum in Davos, Switzerland. The WEF issued a plea to governments, development organizations and businesses to find ways to use big data from mobile and other connected devices that citizens use in their everyday activities.

The WEF wants to find ways to capture and use the data to provide better services in health, education, financial services, and agriculture for people who live in poverty. The WEF noted that organizations will need skilled technicians and data scientists to perform this work, and urged organizations to create incentives that will encourage individuals to work in these fields.

Data stewards and data scientists

As more and more businesses in these various sectors begin looking for experts to perform their big data functions, they will be using job titles that are new to many organizations.

Jeff Hammerbacher, cofounder and chief scientist at Cloudera, said one up-and-coming role is that of the "data steward"—someone who can define the structure of the new data used in a company, ensure that it's delivered and made available to the enterprise for analysis, communicate that to people in the enterprise, and make sure people employ best practices when accessing data.

"That sort of role, which is more data-management-focused than data-analysis-focused, is very promising," he said.

He noted that the data steward role could be added to an existing systems administrator or database administrator position, or it could be a separate position. 

"Data scientists" are also expected to be very valuable in organizations.  David Bayless, a data scientist at LexisNexis, said people who work in this role are often distinguished from data analysts or processing experts by their focus on "trying to find new ways to combine data to discover new things." The data scientist, for example, will perform cutting-edge work to develop algorithms that people in the data department will then use to understand business data.

Bayless said that data scientists need to employ the scientific method to make sure their work delivers reliable results. People who hold data scientist jobs usually have very strong backgrounds in mathematics and computing or physics and they often have a PhD or at least a master's degree in one of these fields.

Reflecting his emphasis that people combine technical skills with industry- or business-specific expertise, Hopkins, at Forrester Research, recommends obtaining advanced education in fields like statistics or parallel computing. He also recommends looking into other high-demand fields such as data mining, predictive analytics, or human computer interaction. 

"If I were a programmer or an IT professional and I had an inclination for the business, I would do two things," he said. "I would get a PhD in computer science, focusing on statistics and parallel computing, and I would go get an MBA." CW (19 June, 2012)