Thursday, December 19, 2013

Distributed Agile - Ten Principles



One of my goals this year was to define a set of principles for geographically distributed Agile teams. To accomplish this goal, I consolidated the lessons learned from several projects, interacted with project teams and collaborated with industry experts. That approached helped!  I have described these principles in my article titled ’10 Principles for Success in Distributed Agile Delivery’ published in Cutter IT Journal (Vol.26, No.11, Nov 2013).

The term ‘Distributed Agile’ continues to trigger critical questions, thought provoking discussions, and exciting debates. Such discussions and debates may appear to cause a restraining force against adoption;  but that is not true!  I am saying this because an increasing number of enterprises have started adopting Agile methods in geographically distributed teams. To know more, read this article.

Free PDF Article (limited time offer)  -  Download this article by visiting http://www.cutter.com/offers/rbavani.html and completing the registration form.

Happy Reading!

Friday, December 13, 2013

How Do You Manage Dependencies in Agile Projects?


I am writing this post to share my thoughts on dependency management and write about how it impacts Agile teams. The motivation to do this is my previous post 'Yet another Dreadful Dependency'.  We hear stories similar to this in our work life. Don't we?

A task we perform can be independent or dependent (on something else – say another task). It is independent when we are free to start and end the task without having to worry about anything else. It is dependent when we have to think and do it in a logical sequence by considering all dependent tasks.

Sometimes you need to wait for another task to finish so that you can start a new task. Sometimes you wait for another task to finish so that you can finish your current task. There can be a situation when you wait for a task to start so that you can finish your current task. Also there can be a situation when you wait for another tasks to start so that you can start a new task. That’s how we have 4 types of task dependencies FS, FF, SF, and SS. This is not a traditional project management stuff. This applies to all projects including Agile projects. When you think in terms of these 4 types of dependencies, you can streamline every iteration or Sprint by minimizing dependency issues.

I think, we need to think deeper than this. Can we categorize task dependencies in some other way? Do we have techniques to deal with such dependencies? Let me explain.

How do we categorize dependencies? We can group dependencies under several categories such as

Technology Dependency – This category is about dependencies arising from architecture, design, coding, components etc. Here are some examples.
  1. Program A cannot function until Program B is developed
  2. We have to change the program as component X has changed
  3. We cannot develop layer-Y in our architecture until layer-Z is ready
Data Dependency ¬ This is about dependencies related to data readiness to complete project tasks. These dependencies are often ignored in projects as project teams find ways to create data in some way or the other. I am sure we learned ‘logical dependency’ and ‘physical dependency’ in our DBMS course. I am not talking about those here. Here are some examples.
  1. Acceptance Tests cannot start until we get adequate data for testing
  2. Data sources A, B and C need to be in sync to start integration testing
  3. Defect X cannot be reproduced until the logger is turned on to capture the trace of events
Application/Service Dependency – In this category we come across dependencies involved in developing, maintaining or testing your current application with reference to other related or dependent modules, applications or services.

Build Dependency - I have not included build dependency under technology dependency because dependencies related to build management deserve a separate mention. When we create builds we identify dependencies and compile programs in the right order. Also, when it comes to several interrelated applications or a suite of applications, build dependency becomes a pivotal point. There are several techniques and tools to identify and manage build dependencies.

Operational Dependency – These category includes other operational dependencies such as release dependencies, data migration dependencies, connectivity dependencies and so on.

These sound more conceptual and theoretical. Can we look at dependencies in a different way, analyze and understand them?

Is there a simpler way? Good question. Fair enough! With all these categories of dependencies, our discussion can become overwhelming. I know. We need to have a simpler way of identifying and categorizing dependencies. A logical way is to identify dependencies at three levels – product roadmap planning, release planning and iteration or Sprint planning. However, awareness on different categories of dependencies is necessary. With that awareness we won’t miss out any.

For each dependency, all we need to do is ask these questions.
  1. Is this an internal dependency (local to the project) or external dependency?
  2. When will it hit us - short, medium or long term? How much time do we have to plan and manage it?
  3. Is this dependency going to last for a week or couple of weeks or months or more? Or is it perpetual or recurring?
These questions help us understand the circle of influence (internal or external), estimated time of start, and expected life or duration. Without these we cannot understand the gravity or impact of dependencies. Without these, we will not be able to reduce anxiety and spend our efforts on the most important tasks. With out these we will not be able to identify the right techniques to handle dependencies.

What are the techniques?  There are many techniques. Here are some pointers to those techniques.
  1. Learn and apply design by contract and practice DIP (Dependency Inversion Principle). A downloadable PDF on design principles is available at http://www.objectmentor.com/resources/articles/Principles_and_Patterns.pdf.  It includes information on DIP. If you are a hardcore programmer, read this post for more information http://www.pelshoff.com/2012/02/dependency-management
  2. Use stubs, mocks etcetera to manage the situation until you get the real objects to depend on.
  3. Don’t ignore data dependencies. If something is necessary – for example, production data, you cannot live without it. Make it a criteria. Don’t accept junk data for testing.
  4. Application/Service dependencies can be haunting. What if a service breaks in real time? A way to handle this is to build robust exception handling mechanism. In addition to this, check dependency issues through automated scripts or snoop error logs. Here is a PDF file on managing dynamic service dependencies - http://proceedings.utwente.nl/13/1/S5-1.pdf.
  5. When you work with multiple teams (for example, Scrum of Scrums or with multiple dependent projects) dependency management has to be handed through proactive thinking, asking questions and triggering discussions in meetings. In addition to asking ‘Are we dependent on other teams?’ it is necessary to think ‘Is there a team that depends on us?’ 
If you are a researcher or student here is a PDF white paper from OOPSLA 2005 - http://www.lattix.com/files/dl/wp/oopsla05.pdf.

I think I have shared a good number of concepts and practices here.  How do you manage dependencies? Do you use any tools or techniques? Let us discuss.

Wednesday, December 4, 2013

Yet another Dreadful Dependency!


I introduced Joe to you in Oct 2013 through our conversation on his question ‘Velocity Dip or Double Dip – What to do?’ Subsequently I wrote a post on how he affirmed ‘Readiness at the Starting Point Matters!’ to his product owner. Last month we met again to discuss about ‘Building on Legacy Code’.  I trust you read all these posts. I am writing this post to share another story. This story is about dependencies in software projects.

Last week I was in a 1-day workshop on emerging technologies. To my surprise, I met Jim and his technical architect, Alan in that workshop. We had some interesting conversation over lunch.

During the first five minutes of our conversation Alan was little skeptical on how iterative and incremental development works in large projects. That is how I perceived him because of his crisp answers and body language. He was not ready to open up and talk about issues on the ground. I wanted to understand his views. I had no other option but asking him pointed questions. And I started.

“Alan, tell me. How is it going in your projects? I know you have multiple projects – big and small. What is one single big issue that bothers you?”

Alan smiled and responded. “Well there are multiple issues. There are issues because of dependencies. How do we handle them? I am working with a large team – there are multiple teams. These teams are spread across three locations. Last week, we were about to complete a feature. That is when one of our team members alerted about a dependency. Our code depends on what other teams are developing. This is not the first time. In spite of multiple reminders, the other team did not act on time and complete their work. We are just a month away from our release! Believe me, it is yet another dreadful dependency! ”

“Let me add some more information.” Jim chimed in. “This is a large project. Initially we thought that our teams spread across these three locations could function smoothly. Because we put them on three separate modules. I can say, fairly independent modules. Now, we are deep inside the project. We understand that on implementation point-of-view – I mean, in terms of design and coding, there are several dependencies. Those are hurting us now!”

Looking at Alan, I probed, “That is, you are not able to finish something because something else is not finished by another team?”

He nodded and said, “Yes. Sort of. You can say that.”

I wanted to know more about what happened in their project. I continued. “Joe, I think we are good at managing ‘Finish to Start or FS’ dependencies. For example let us assume that we plan a trip from point A to point C with one hop in point B. Let us say want to go by air. The entire trip is covered in two flights - one from point A to B and the other from B to C, we know that there is a dependency there. I mean unless you reach point B on or before X hours, you cannot start from point B at Y hours. That’s is obvious. We plan for sufficient hours between the arrival of the first flight and departure of the second. We do this in our projects too. Simple. Isn’t it?”

Joe smiled with his short response. “Yes. We do. But I am not sure if we identify all dependencies and explore all possibilities. Could it be the drawback of iterative and incremental development? I don’t know.”

Alan, with his curious expression “I partly agree with Joe. I have a question. Finish to Start is interesting. What are the other types? Is there a Finish to Finish? How about a Start to Finish and a Start to Start? Do we have all these types? FS, FF, SF, SS?”

“Well, Yes. That is correct. I think the dependency issue you told me about in your project is a Finish to Finish dependency. That means that unless that other team finishes some work, you cannot finish your work.”

“Right! Absolutely!”

“So, what do we do about these?”

“Plan, execute, and monitor progress. These three are important for all types of dependencies. I mean plan, execute and monitor. Obviously! We know that. In spite of these if you come across issues, you will have to manage those issues! What do we do when our first flight from A to B is delayed? If we carry a refundable ticket and if there is ticket available in a flight on the same day, we reach after few hours of delay. If we carry a non-refundable ticket, and the flight operators are different then the solution is costlier - we get some refund but we need to find out way home by buying another ticket or doing something else to travel. That will be an expensive ticket! So, what is the take away here? We need to plan. We need to build some safety net or a backup plan in some cases. That is because dependency is a risk. If a dependency is perceived as a big risk, you plan to mitigate it. In our example, the mitigation plan is your refundable ticket with enough overlay. In spite of this mitigation plan if that dependency becomes an issue – for example, due to bad weather or flight delays you miss your second flight, you have to manage that issue. That will impact your journey. You may not reach on time!”

“Interesting example! So, do you say that we must plan for all dependencies?”

“Well, I think we must be aware of all dependencies and their impact. The next step is to plan. Without awareness we cannot plan. Without planning we cannot put in measures to reduce the impact of dependencies.”

“Of course! I agree. Is this all part of Scrum or XP or something else?”

“I don’t think so. And you know this better than me! These is a project management practice. You must be able to pick and choose practices like these and include them in your methodology. Then it becomes your hybrid methodology. Methodology is project specific. If that is what matters, that is going to work for you!”

“Well, I think it is clear now! Thanks a lot!” said Alan.

“Sure. We need to go now. Our next session will start in five minutes. I will share some more thoughts on project dependencies next time.”

With a smile, Alan acknowledged, “Thank you. And yes, we talked about these theoretical concepts. It is worth exploring some hands-on techniques as well. May be next time.”

“Of course. I will share those in my next blog post as I am on a long leave and may not be able to meet you until Jan 2014. Or we will discuss about those techniques and all that when we meet.”

With that parting note we started walking towards the conference room looking forward to an interesting post-lunch session.

Dear Readers, I am sure this post is a long but a fast read. How do you manage dependencies? Did this post trigger any thoughts in your mind? Feel free to share!

Monday, December 2, 2013

Distributed Agile: Questions and Answers



Tom Cagley (http://tcagley.wordpress.com/) interviewed me on distributed agile. This interview happened sometime in July 2013. We spoke on phone - Tom was in US (East Coast) and I was in India (Pune).

Tom asked me several interesting questions and engaged me in a great conversation. Here are some samples.

1) What do we mean when we say "distributed agile"? How do words like off-shoring, outsourcing affect your definition?

2) Why do organizations seem to have a difficult time with agile with multiple organizations and locations?

3) What are the principles that you think should guide the adoption of distributed Agile?

4) Recently you have been blogging about Agile in stage-gated environments. How is Agile possible in that environment?

5) Recently you spearheaded the creation of the Global Distributed Agile Consortium. Please tell the SPaMCAST listeners about the project and where you see it heading?

My answers to all his questions are available in Tom’s podcast at http://spamcast.libsyn.com/s-pa-mcast-266-raja-bavani-distributed-agile-revisited.

Tom runs 'Software Process and Measurement' (SPaM) casts regularly. The Software Process and Measurement Cast provides a forum to explore the varied world of software process improvement and measurement. The SPaMCast covers topics that deal with the challenges on how work is done in information technology organizations as they grow and evolve. In a nutshell, the cast provides advice for and from practitioners, methodologists, pundits and consultants!

Go visit his site and listen his podcasts and read his blog post at regular intervals! It is worth an experience! Many impressive interviews by industry leaders in Software Engineering are waiting there for you!