In the context of industry Productivity is defined as the effectiveness of effort as measured in terms of output per unit of input. Clearly, productivity is a key metric that every business venture aims to improve upon and hence it is essential to choose an appropriate production methodology that helps a company optimize productivity. For the purpose of this article, in the context of the software development industry, we shall define the per unit of input to be one person hour of effort by a member of a development team. Thus we shall exclude all the other overheads such as executive management, marketing, sales, support, and the contributions of the administrative staff.
All commercially successful products in the market today rely on geographically distributed design and manufacturing teams. So it is no surprise that the software product industry has also been quick to adopt the distributed model for pretty much the same reasons as the rest of the product manufacturing industry. The challenge, however, is that due to its ephemeral nature, a software product does not lend itself to a typical lifecycle of market analysis, design, development, quality assurance, distribution, sales, and support. This has led to the success of methodologies such as Agile that aim to compress the development lifecycle by requiring the members of a development team to collaborate very closely on all aspects of the product. This high degree of collaboration required across a distributed team not only complicates the measurement of the productivity metric but also requires adoption of unique strategies to improve upon the same.
A brief history of Software Development Methodologies
In the early days of modern computing software development was very closely coupled to the associated hardware on which it was intended to run. This allowed the development team to make a fair number of assumptions about the operating environment such as the operating system and available resources. This rigidity allowed for very formalized product development methodologies to be implemented rather successfully since the expected use and behavior of a product could be predicted with a high degree of accuracy. However, with the advent of the Internet followed by the World Wide Web, software has increasingly been decoupled from the hardware platform and this has led to a drastic change in how a software product is defined let alone how it is designed and implemented. The growing role of mobile devices in our daily lives has further changed the software development process.
The Agile software development methodology evolved out of this need to address the unpredictability of how the requirements for a software product change during its lifecycle from inception to design to implementation. It aims to avoid analysis paralysis and minimize waste of development effort by empowering the development team to start building working versions of the product using the shortest possible iteration cycles typically no more than two calendar weeks in duration. The methodology as it is known today is an amalgamation of a number of methodologies that preceded it unified by the Agile Manifesto published in 2001:
“We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
Individuals and interactions over Processes and tools
Working software over Comprehensive documentation
Customer collaborationover Contract negotiation
Responding to change over Following a plan
That is, while there is value in the items on the right, we value the items on the left more. – Beck, Kent; et al. (2001). “Manifesto for Agile Software Development”. Agile Alliance.
Of the four manifesto items we shall focus on the first two items in the context of evaluating and improving productivity.
Pair Programming – Pros and Cons
By choosing to emphasize on individuals and interactions over processes and tools, the adoption of Agile requires that the team members be highly motivated allowing the team itself to be self-organizing. More importantly collaboration between team members should be as efficient as possible thereby promoting the practices of co-location and pair programming. Let us look at how these concepts can be applied to improve the productivity of distributed teams.
Co-location is the obvious challenge with a distributed team. However, it can be addressed by adopting a number of techniques and tools to ensure that the team members are collaborating as efficiently as they would be had they been co-located. Furthermore, as long as a suitable overlap is maintained between the working hours of a distributed team, productivity can be improved due to a longer “working day” for interdependent tasks.
The pair programming model helps address one of the key challenges with the agile development methodology: lack of detailed elaboration of the problem statement for a task so as to avoid “Analysis Paralysis”. Consequently, a pair of programmers is able to work through a complex problem statement by leveraging their collective understanding and considering multiple solutions in an objective manner. Thus, pair programming also improves learning due to the constant knowledge sharing happening at a level of detail that otherwise is simply not possible. By rotating team members in to different pairs not only further improves the collective knowledge of the team but also helps in team building and consequently job satisfaction thereby resulting in a highly motivated team. So then is productivity a mute point? Interestingly the jury still seems to be out on that!
A large number of studies have been undertaken on evaluating the impact of pair programming on the overall productivity of a team (1 – 4). While the general consensus seems to be that the total effort expended on a task by the pair either increases or remains the same, the benefits realized are harder to quantify in terms of code quality metrics such as defects and code readability. Furthermore, since team members will have varying degrees of experience and expertise the various pairing combinations such as “Expert-Expert”, “Expert-Novice”, “Novice-Novice”, may result in non-optimal situations such as high costs, watching the master, or disengagement. These challenges are further magnified when practicing pair programming across distributed teams.
Alternatives to Pair Programming
It is interesting to note that while the Agile manifesto emphasizes on producing working software it does not necessarily limit documentation to only user stories. However, documentation typically is restricted to user stories since it allows for a clear hand-off point for a task from the non-technical members of the team such as Product Owners and/or Story Authors to the technical members of a team such as the Developers and the QA Engineers. Unfortunately this also limits the design activities to information architecture and user experience. Software architecture is often considered redundant since it typically only focuses on creating models rather than working code. Furthermore, scaffolding features provided in practically all the technology stacks automates the generation of code for CRUD operations for entities for most of the scenarios. Subsequently, techniques such as Model Driven Architecture and Round Trip Engineering are rarely adopted in Agile projects especially if all members of the team are co-located.
However, for distributed teams adoption of appropriate tools can enable implementation of Model Driven Architecture and Round Trip Engineering techniques so as to improve productivity by optimally leveraging the varying levels of expertise available at different locations. Product owners collocated with subject matter experts can collaborate on architecture models to address complex aspects of the product at an abstract level by adopting techniques such as Domain Driven Design and using modelling languages such as UML to generate models that are automatically converted to “compilable” code and handed over to the remote team which would then continue with techniques such as Test Driven Development. Round trip engineering would then be employed to allow updates to the models, if any, to be jointly reviewed at the end of each sprint/iteration. Similarly, Rapid Application Development (RAD) platforms such as WaveMaker and OutSystems and Application Platform as a Service (aPaaS) providers such as Mendix can be used to generate skeletons that can then be elaborated upon by remote teams.
This approach can be further extended once suitable maturity has been reached so as to implement the Software Factory model wherein there is higher emphasis on assembling new product features using reusable code widgets. Software product development can thus benefit from the high productivity usually associated with product factories.
To pair or not to pair: Conclusion
It is clear that there is no silver bullet that will guarantee optimal productivity across all scenarios. Hence we shall conclude by laying out some guidelines for choosing an approach especially when working with distributed teams.
To start with it is important to first baseline the current productivity of your team(s) based on an appropriate metric such as velocity, control charts based on cycle time, etc. and a suitable time horizon. Once a suitable baseline has been arrived at, set a realistic and tangible goal for the improvement that is desired over the chosen time period. More often than not a blanket statement is made about improving productivity and that is a recipe for disaster since it is bound to make the team(s) feel under appreciated. The challenge here is that budgetary constraints are rarely exposed to the entire team and hence it is difficult to justify the need for productivity improvements in the first place.
Assuming that the current model is neither based on pair programming nor on software factories the choice for a new methodology must take in to account the existing culture of and skill distribution within the team. Pair programming puts certain specific demands on “the workstation” from perspective of space and hardware requirements such as a large and preferably dual monitor setup. Occupant density also plays an important role in preventing high noise levels that would otherwise be far too distracting for developers who can no longer don their favorite headsets to block out ambient noise. Finally, conscious effort must be undertaken to ensure that pairing is truly a two way street by encouraging different pairing techniques such as ping-pong pairing and ensuring that pairs are frequently changed. It might also be possible to pair a developer with a story author or a QA engineer while employing techniques such as Behavior Driven Development (BDD) or Acceptance Test Driven Development (ATDD) using tools such as Cucumber. In such a scenario the non-developer member of the pair would focus on developing tests using the Gherkin language while the developer focuses on the coding tasks.
Adoption of Model Driven Architecture and RAD platforms requires procurement of specialized tools to enable effective round trip engineering resulting in additional cost burden on the project. Some of the latest frameworks such as Spring 4, Lombok, etc significantly reduce the boiler plate code required to add support for typical architectural concerns such as logging, data access, security, exposing web service end points, etc. thereby allowing developers to focus on business logic development. Hence, investments can instead be made in to “up-skilling” the team to use these new frameworks and new language features such as lambda functions, reactive streams, etc. and allowing for capacity to refactor legacy code bases. Finally, a strong governance model based on feature based branching and promiscuous merging should be implemented to ensure that Continuous Integration (CI) practices are continually being improved upon to achieve lower defect leakage.
In conclusion classical techniques of pair programming are less likely to provide any significant benefits in terms of productivity over adoption of automation tools and modern frameworks especially across distributed teams.
References:
1. Cockburn, Alistair; Williams, Laurie (2000), The Costs and Benefits of Pair Programming, Proceedings of the First International Conference on Extreme Programming and Flexible Processes in Software Engineering (XP2000).
2. Williams, Laurie; Kessler, Robert (2003), Pair Programming Illuminated, Addison-Wesley, pp. 27–28, ISBN 0-201-74576-3.
3. Lui, Kim Man (2006), Pair programming productivity: Novice–novice vs. expert–expert, International Journal of Human–Computer Studies, 64 (9): 915–925.
4. Arisholm, Erik; Hans Gallis; Tore Dybå; Dag I.K. Sjøberg (2007), Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise, IEEE Transactions on Software Engineering, 33 (2): 65–86.
Adventures in Cognitive Computing 