ReadyNotes

Sample Selection

 

Lessons Learned and Challenges

Working in a global context obviously has advantages but also drawbacks. While the positive side accounts for time-zone effectiveness or reduced cost in various countries, we should not close our eyes in front of the severe disadvantages. In fact the business case is surely not a simple trade-off of different cost of engineering in different regions. Working in a globally distributed project means overheads for planning and managing people. It means language and cultural barriers. It creates jealousy between the more expensive engineers being afraid of loosing their jobs, while forced to train their much cheaper counterparts. We will try in this Ready Note study to summarize experiences and to share the best practices from projects of different type and size that involve several locations in different continents and cultures.

Big savings in GSE have been reported only from (business) processes which are well defined and performed already before offshoring started, and which need not much control [Corbett05, Meta04b]. This includes maintenance projects (under the condition that the legacy software has some type of description) where some or all parts could be distributed, technical documentation (i.e., creation, knowledge management, packaging, translation, distribution, maintenance) or validation activities. Development projects have showed good results in all cases where tasks have been well separated so that distributed teams would have direction and ownership.

Global development projects failed if tasks were broken down too much, such as asking a remote engineer to do the verification for software developed concurrently in another site [Carmel99, Carmel01, Grinter99, Herbsleb00]. Here distance effects and lack of direct communication slow down development rather than help it. The single biggest source of difficulties in GSE is related to communication across sites, bad communication hindering both coordination and insufficient management processes [Cramton05, O’Hara94, Krishna04]. For instance, continuous integration of insufficiently verified and encapsulated software components fails if done remote to the parallel ongoing software development. Distributed teams working on exactly the same topic (e.g., the famous follow-the-sun pattern of developing a piece of software in different time zones) posed highest challenges for coordination and often resulted in severe overheads that would be measurable or tangible only later (e.g., features misinterpreted, insufficient quality, lack of ownership and responsibility, etc.).

The challenges in GSE can be summarized as follows :

  • Lack of strategy and shared values in parent organization resulting in insufficient collaboration, unclear work split and ownership. Roadmaps might be fragmented or insufficient visibility provided on the strategy that both contribute to insecurity of teams and cause sub-optimal results. A clear sign for lack of strategy is the senior manager announcing, “We will work in India because it is cheaper” or the engineering lead explaining, “Any work can be done by virtual teams”. A major underlying reason for dysfunctional global work is different values and underlying society factors [House04, O’Hara94, Krishna04]. We often label this superficially as “culture issues” or even worse as “soft factors”, claiming that we can not handle it with our limited management and software education. For instance time perception in a society has profound impact on many behaviors such as insufficient planning and monitoring which can not be cured only as symptoms. A culture deeply rooted in the present will always be portrayed as lazy and unfocussed by a society rooted into the future and therefore demanding accurate planning. The same applies to societies that value entrepreneurship and spontaneous (re)actions to events as opposed to those that prefer clearly outlined roles and responsibilities. Such differences must be recognized, considered and dealt with. A shared value system and continuous team building activities will help as well as rotating employees across these different societies.
     
  • Insufficient communication due to distance, time zones and cultural barriers. Note that distance impacts start at around 10-15 meters, i.e., far earlier than what one would usually assume. People talk and share only if they are close to each other and frequently run into each other without this being planned. Lucent and others did extensive studies on communication in global teams and found that 15 % of software development is informal communication [Herbsleb00, Herbsleb03, DeMarco99]. Distributed teams are less effective than a co-located team working on the same task.
     
  • Dispersed work organization. The global nature of project and product work obscures a holistic view of project success factors. More sites add cost due to overhead management, separated and dysfunctional processes, tools and teams. Tools help but will not be enough to build a distributed team. Process immaturity is a key roadblock and cause of inefficiencies and rework. Gartner and Standish report 10 % management overhead, i.e. 1 person to synchronize for 10 persons allocated an offshored task. Our own experiences show that having two sites working on the same development project immediately adds 10-20 % cost and moves away visibility. Overall effort overheads are ca. 35 % if done in 2 places due to interface control, management, replication, frictions, etc. [Jones01, Herbsleb00, Ebert04, Grinter99, Mockus01].
     
  • Inadequate global management resulting in micromanaged tasks or lack of visibility. Often project managers would fear the lack of control and establish very small fragmented tasks to stay in control. This is unhealthy micromanagement that to all experience creates a lack of buy-in from the teams as they expect that the manager would interfere anyway and create trouble. On the other end is insufficient visibility starting with estimates and continuing with change management and progress tracking. Global team management often suffers from biased attitudes. Functional and regional rivalries exacerbate the tendency to claim credit for success and shift blame for failures. We experienced in several such global product lines that roadmaps and features are overly volatile because of local optimization on per regional customer basis. Our experiences show that change rate of requirements will in consequence be much higher than industry average (1-3 % per month). Lucent reports 30-100 % delays for multi-site change requests and overall project delays if a project is distributed across sites [Herbsleb00].
     
  • Isolated learning. Improvements derived from past experiences are rarely applied beyond the originating organizational silo. We found that in GSE individual sites—even if working in the same product line—have their own tools and processes. Different countries or regions would launch independent infrastructure optimization in order to differentiate. This is often amplified by dysfunctional regional competition as many companies have established to challenge “high-cost” countries with “low-cost” countries. For that reason the parent organization would hesitate to provide all necessary support due to fears that work would be taken away. Additional obstacles in sharing experiences arise from insufficient risk mitigation related to intellectual property or third party access to tools and knowledge repositories. SAP reports, "Distributed development is slower and less forgiving in case of mistakes. We need to communicate more but we have less capacity to communicate. Effects of mistakes are not easily apparent and tend to be hidden by regional owners longer than possible in a centralized development."
     
  • Less agility compared to co-located teams. This is almost certain as soon as an integrated task is done in different sites. Workflows, monitoring and engineering processes must be strengthened to assure that different stakeholders collaborate well. This is perceived as overhead by the teams and if not well-trained they try to escape causing major trouble during development [Grinter99, Herbsleb03, Olsson00].
     
  • Insufficient contract management. Contracts are absolutely crucial for managing external suppliers. They must include defined and measurable SLAs to assure appropriate quality levels. For internally hosted GSE it might be wise (depending on organization structure) to govern by means of internal contracts and SLAs. They have the big advantage that targets and metrics are agreed upfront and would not need continuous debates with senior management if some delivery is late. Certainly such internal contracts and SLAs together with a culture of accountability and clearly assigned responsibility also avoid the political game of finger pointing to “the others” that did not do their job well.
     
  • Unknown legal environment. This is a major trap for any global activity; independent of whether it is sales or engineering [NeoIT03, CEB04]. It means to get very familiar with local laws, such as contracts, liability, intellectual property or human resource management. If you don’t have enough experience and exposure, we strongly recommend using consulting to set it up at fastest speed BEFORE starting any global development. Never, ever rely on the legal support from a supplier in the host country to which you want to expand your engineering teams.
     
  • Higher employee turnover rate. We see different local patterns of employee turnover rates across the world. Some can be explained by the legacy clichés; such as Europeans or Chinese being married with their company while Indians or US-Americans are on a continuous job hunt. Both are not true and especially challenging when managing such diversity in different cultures. For instance it is often said that India has high turnover rates. However the truth is that India’s IT industry is in so fast growth that many engineers are continuously approached from other companies with even more interesting offers. On the other hand we have experienced ourselves that it is feasible to manage an Indian engineering team with turnover rates similar to those in Europe over many years. It all depends on management, culture, responsibilities and ultimately motivation.

With these challenges, reported cost reduction from GSE is much less than the above-mentioned potential of 50 % savings if the same process is split across the world with changing responsibilities . Several such highly distributed software engineering projects reported a 10-15 % cost reduction after a 2-3 year learning curve. Initially outsourcing demands up to 20 % additional efforts. Over 20 % of outsourcing contracts are cancelled in the first year .

To externalize insufficient engineering processes creates extra cost and learning curve driven delays—on both sides. These additional costs sum up to 20-40 % of regular costs of engineering. The learning curve for transferring an entire software package to a new team (e.g., location) takes 12 months [Karolak98, CEB04, Herbsleb00]. Our own experiences [Ebert01a, Ebert01b, Ebert04] and research show that the effectiveness for software design and coding grows in a learning curve with 50 % effectiveness reached after 1-3 months and 80 % after 3-5 months. This obviously depends on process maturity and technology complexity. Each of the following bullets accounts for a 5-10 % increase to project cost:

  • Supplier and contract management.
  • Coordination and interface management, especially with fragmented / distributed processes.
  • Project management and progress control.
  • Training, knowledge management, communication.
  • IT infrastructure, global tools licenses.
  • Liability coverage, legal support.

We found several best practices reported from different global software teams, such as :

  • GE: After years of no results with reducing SW cost, J. Welch concluded the need for a clear and simple policy to split work—or nothing would happen. The GE policy demands 30 % of software being developed locally in North America while 70 % is developed in one of seven offshore design centers.
  • Ford: Never split projects on too many different areas (i.e., departments, regions). Favor projects with less than three areas involved in design and test.
  • Lucent: Best results are achieved if coherent tasks are co-located. If resources are scarce, co-locate functions rather than products or projects. Create a sufficiently large pool of similar resources to ensure flexibility, continuous mentoring and learning, and mobility of resources. Certain independent process steps can be separated from each other and distributed across sites (at the known overhead cost): 1) requirements mgmt / product mgmt; 2) development; 3) system test and interworking test. If work is to be distributed, it is better to do it for well-defined contents (i.e., such as a mobile communication protocol standard), but not for flexible and innovative projects.
  • Thales: Effective offshoring needs strong and aligned processes and tools.
  • SAP: Very strong focus on global team management with shared values and excellent collaboration environment.
  • Bosch: Common language across projects and regions is achieved by using standard processes based on CMM and CMMI.

Product Descripotions, Author Bios, Table of Conents, Purchase