Software Delivery Process with Agility Built-In

INTRO

The IT industry began the Agile transformation about 25 years ago. The revolution was caused by a great deal of frustration originating from the fact that most “large projects” were deemed as failures (either not delivered on time, or not delivering the right scope).

By examining the agile manifesto we can see that it is divided into two parts:

• Professional relationships – individual interactions together with the customer collaboration are valued over processes and tools together with contract negotiations.
• Delivering value – working software together with responding to change are valued over comprehensive documentation together with following the plan.

These are the two qualities of an agile delivery process among many others that Quality Assurance Engineers strive to assure every day. Focusing on creating an environment that values people is important to preserve a team’s intrinsic motivation. Delivering value on the other hand is where the money comes from.

The goal of this article is to describe how to build a process that allows software developers to deliver business value while embracing the fact that the only constant in the world is change.

PROFESSIONAL RELATIONSHIPS

According to intrinsic motivation research, it is important for people to feel engaged and motivated. We can look at this aspect from an even broader perspective. It teaches us how to create a working environment that keeps people motivated. Let’s analyze every motivator and check how it relates to various agile/scrum practices:

• Order – Scrum in terms of structure definition, finds a sweet spot between a lack of rules which leads to chaos, and too many rules, which lead to a rigid process. It defines just enough rules to have order:
  • Three roles (developer, scrum master, product owner) – mind that developers are people who develop a solution (it can be either a Business Analyst, UI/UX expert, QA, Programmer… anyone who is involved in the software development process). They work as an integrated team as opposed to in silos.
  • Three artifacts (product backlog, iteration backlog, and increment).
  • Cadence is defined by cyclic meetings (daily scrum defines a small cycle, and planning/retro/demo defines the iteration cycle).
• Relatedness – one of the visible results of the Agile Movement is a change in how boundaries are drawn within organizations. It is recognized that to deliver a solution, people with certain skills (analysis, programming, testing, etc.) need to communicate often. Hence instead of keeping them in separate teams/departments, they are put on the same team.

Building relationships between these people make those, who are motivated by relationships, work more efficiently. Of course, team members had relationships working in separate departments, but they were fueling the delivery of a subproduct (be it a document, an untested code, etc.) instead of the product itself. Change of focus discourages local optimizations of the process. Nowadays instead of document-based communication, we rely on face-to-face communication. Documents are still used, yet they are intended to record and store progress, decisions, and status.

• Acceptance/Inclusion – some people need to feel that they are accepted by the group and are part of the team. At least two practices support this motivator:
  • Every day the entire team has a stand-up meeting to agree on the next steps which need to be taken to reach the goal of the iteration.
  • Each team has a Scrum master that is dedicated to building an effective team from a group of people.
• Goals – for some it is important to deliver a valuable product that is aligned with their life goals. The alignment is very subjective and everyone contributes to the decision. Yet the product is what scrum is all about. The sole goal of a Scrum team is to deliver a valuable, high-quality product as a result of every iteration.
• Curiosity – the first statement of the Agile manifesto is about uncovering better ways of developing software. Additionally one of the twelve rules is about “continuous attention to technical excellence”. Application of this rule supports those who look for a better solution and regularly evaluate if the industry or organization itself came up with an idea of how to solve a particular problem more effectively.
• Freedom – as opposed to Power is about me being able to self-organize myself. Scrum redefines it slightly to a team that self-organizes itself. In Scrum, it is the team who decides (and commits to) what will be delivered in the upcoming iteration and who can change the process it follows to make it better.
• Power – as opposed to Freedom is about me influencing others. In Scrum, each team member is perceived as an equal professional whose knowledge and skills are required to deliver the increment. In practice though, within any group of people, sooner or later a leader manifests. While everyone’s opinion is equally important, naturally the leader’s opinion influences others more. The important thing is that nobody becomes a leader because of the number of working experience years. My experience shows that the best teams have more than one leader and the leadership effectively floats between all the members.
• Social Status – let’s be honest, this motivator is not part of a typical Scrum environment. Boasting about titles, “I’m a Director” or “I’m a Leader” or “I’m a Manager” in the Scrum world does not exist…
• Idealism (need for social justice / Honor) – these are highly subjective as a code of honor and the perception of social justice can differ a lot between people, and therefore, I won’t comment on this topic.

In the real world, organizations can inadvertently undermine these motivators. Here are a few examples of common practices:

• Cut the Scrum Team in pieces (e.g. business analyst/programmer/tester) – sub-teams are no longer delivering value, they deliver subproducts. Additionally, since there are fences within the team, pushing the subproduct over these fences back and forth takes too much time. The easiest way to check the environment against Scrum Team cuts is to listen to people at different meetings. Statements like “We’re done with development, now there is only testing left”, “We’ve finished our part now it’s your turn” or “We didn’t finish the story as a business analyst, didn’t analyze it properly” clearly show the barriers within the team.
• Transforming a scrum meeting to a reporting meeting loses the value of acceptance and agreement on the next steps. Instead, it assesses the progress of a particular person. Such a meeting is not for the team but for the management to track progress.
• Do what you are told to do – especially when there is no explanation why it is needed to do what we’re told to do. People who are fueled by curiosity, freedom, or power will definitely lose all motivation.
• Pushing a team to deliver a defined scope within an iteration – freezes all three corners of a project management triangle… Agility is gone together with freedom and self-organization.
• Nominating a leader – I see a few issues with nominating a leader. First of all, it might lead to a mistake in nominating someone who is not a natural leader of the team. There is a risk of increasing tension within the team, hence lowering productivity. Team members might start thinking that the nominated person is responsible for organizing the way the team works. As a result, it may inhibit the development of other team members since they don’t feel the responsibility. Consequently, it increases the risk by lowering a team’s bus factor. On the other hand, nominating a leader feeds the ego of the nominated person… Then a question arises: do we want to grow leaders whose ego needs to be fed?

To learn more about intrinsic motivators important on your team I encourage you to try a moving motivators game. The game has been designed to ease the change process that the team is expected to go through. It can be divided into two parts:

1. Learn what motivates each team member.
2. Learn how a particular person perceives the change through motivators’ lenses.

While there may be no change that the team needs to go through, the first part of a game can still be used to get to know each other better. There is even a nice deck of cards to use during one of the retrospection sessions.

DELIVERING VALUE

Delivering value quickly is the second cornerstone of the Agile development process. Many Agile practices focus on shortening the feedback loop (test-driven development, behavior-driven development, three amigos, cross-functional team, etc.). Many of these practices often culminate in a Continuous Integration practice (in this post I don’t want to go into nuances of continuous integration/delivery/deployment hence I decided to use “continuous integration” as a generic term), which strives to automate the entire software delivery pipeline.

CONTINUOUS INTEGRATION

When it comes to the continuous integration practice, an ideal situation is when nothing needs to be done outside of a delivery pipeline to deliver the increment. Conceptually delivering software with a CI Pipeline involves the following steps:

1. Make a code change and raise a Pull Request (PR) to merge the change into the main branch (it is recommended to integrate changes at least daily).
2. The change is reviewed by another teammate and the Continuous Integration server is triggered to run the PR pipeline.
3. Once approved by a team member and the CI server, merge the change into the main branch.
4. The CI server runs the Main Pipeline and deploys the package to the appropriate environment.

The main difference between the PR Pipeline and the Main Pipeline is that at the end of a Main Pipeline deployment of the package to some environment can take place.

PR is checked by the CI server

Change to a main branch results in creating a package that is then deployed on one of the environments.

It is vividly important that there are as few differences between these pipelines as possible. The main reason is to get feedback as soon as possible. It is counterproductive to be in a situation where the PR pipeline passes but then the Main Pipeline fails. The following is what is assumed to be included in the pipeline:

1. Build the package.
2. Unit/Integration tests.
3. Configure/build the test environment.
4. Deploy the package to the (ideally) isolated environment.
5. System tests.
6. Performance tests (usually focused on micro-benchmarking – getting quick feedback is essential).
7. Contract tests/rollback.
8. List of various types of tests that should be included depends on the business context.
9. Publish a deployable package or deploy the package to a particular environment.

The last point differentiates the Main Pipeline from the PR pipeline. It is required to pay special attention to the time needed to run the entire pipeline. The general rule is that the quicker the pipeline is, the shorter the feedback loop is, hence waiting time is reduced. Nevertheless, in my opinion, the entire pipeline shouldn’t be longer than 25+/-5 mins. If the 20 mins threshold is crossed, we should start looking for ways to optimize the pipeline. The first step is always increasing parallelization. It can be applied on at least two levels:

• Pipeline level – e.g. running different types of test steps together.
• Pipeline step level – e.g. running tests in many threads – in this case, it is crucial that tests are designed with parallelization in mind (this is a subject for another blog post).

What about other aspects of the testing process? Can everything be done within the Continuous Integration pipeline?

AGILE TESTING QUADRANTS

Agile testing quadrants can be used as a nice thinking framework allowing us to assess if there is any part of the testing process that remains uncovered:

Each quadrant deserves a separate blog post. It is important that the CI Server covers Q1/Q2 entirely and Q3/Q4 partially. There is still room for manual testing, but it is pushed to a higher level as all the tasks that can be automated are automated and performed happily by the CI server.

CONCLUSION

Understanding the importance of people, professional relationships, and delivering value in the Agile software delivery process is vital for every person working in an agile environment. Quality engineers play an outstanding role in this process. By fostering a motivated, collaborative environment and streamlining the delivery process, Agile teams can improve their efficiency and ensure high-quality software solutions.

I work at Klika Tech, which comprises an agile team of experienced, solution-focused, and business-driven technology professionals who thrive on solving problems together with our clients. We are a full-service agile-driven IT company committed to crafting ideas into lovable products.