Exams Are Never Fun

I mentioned a little while ago that I was required by QUT to do an Agile PM Practitioner Certification before they would let me tutor the postgraduate Agile Project Management course (IFN700). Well, technically I think it was probably Yellow House that required the certification, as they are the organization who a lot of the material for the course (specifically the Agile PM stuff) is sourced from. I assume they want to make sure that the people sharing the material actually know it well enough, which I can totally understand.

I took the exam for the Practitioner certification a few weeks ago.

It was surprisingly hard, and leaving the exam I was pretty sure that I had failed.

I’m going to use this blog post to talk about my experience completing the exam, in the hopes that it might prove useful for anything else who hopes to take it. The exam itself was 60 questions worth of multiple choice, and was open book, allowing you to bring a printed copy of the Agile PM Handbook. Being an open book exam, I found that the difficulty came from two places.

1. Whether or not you understand the subject matter enough to apply it to a problem (good, that’s what should be tested).
2. Whether or not you can understand the questions (bad, artificial difficulty as a result of badly formed questions).

Applying Knowledge

The exam was actually pretty good about not giving questions that just required you to regurgitate words from the handbook, which was nice.

The entire thing was centered around a case study involving a situation where Agile project management was necessary, in this case the refurbishment of an old building as a hotel. The difficulty was in making sure that you understood the case study, the problems defined within and how those problems could be solved using the processes and practices outlined in Agile PM.

This is the kind of thing that I like to see in a certification, proving that you can combine everything that you have learnt with some context to solve what seem to be realistic issues. I wasn’t fond of the fact that the problem space wasn’t IT related though, because honestly, that’s mostly where you are going to see Agile projects. It felt like the exam used a non-IT case study as a way of really emphasising “Agile can be used for non-IT things, we swear!” which made it feel like it was trying a bit hard.

Apart from the case study itself, the only thing I didn’t like about the exam was that there was no opportunity to ask questions to give more insight into the case study and the problems therein, a key tactic when dealing with problems in reality. In fact, if there is one thing you should always do, it’s ask questions. Questions can only lead to more information, which leads to making better decisions.

The only context that was available for the exam was the information that some person (or likely group of people) put together, that in their opinion fully defined everything that you needed to know.

A document that tells you everything you need to know? Sounds kind of like a requirements specification to me, and we all know how well those work.

Cognitive Load

The second component of the exam that made it difficult was the high cognitive load attached to the questions and the way in which they needed to be answered.

Essentially the questions required you to keep a significant amount of information in your head all at once in order to answer them effectively, information not necessarily related to the problem. They were multiple choice questions, but were structured in a way that made them difficult to understand. This is disappointing because one of the most important tenets of an Agile process (for me at least) is clarity of communication.

To give a concrete example, some of the questions consisted of two statements, like the following:

The fruit salad for the release party should contain sliced banana.

The Project Manager is extremely fond of banana.

Answering questions like these involved what is almost a truth table, like the following:

The answer is whatever row matches with the way in which you believe the two statements should be interpreted. So for my example, assuming the Project Manager is indeed fond of banana, the answer would probably be a.).

I can see what the writers of the exam were trying to accomplish. They had to have a multiple choice exam so that they could mark it electronically, and so that there would be a definite right answer to every question, but they didn’t want to make the mistake of making the questions too easy, so they wrote questions in ways that were more complicated than normal.

Being an exam for an Agile Certification (which is kind of weird in itself) I would have preferred essay questions. There are not always nice clean answers when doing Agile (regardless of what the people who write the guidelines and processes would like to believe), and I think being able to write what you would do in the situation, and give an explanation for why, would have made for a far better exam, and thus a more meaningful certification. Harder to mark of course, but not everything can be easy.

Conclusion

Well, I’ve managed to make it through the entire blog post without saying whether or not I actually passed the exam, so without further ado…

I passed.

I actually did a lot better than I thought I was going to do, which surprised me, because leaving the exam I had already accepted the fact that I had failed based on how I felt I did.

The main outcome of this is that I can continue to tutor the Agile Project Management courses at QUT, which is great fun. Really, that’s the only reason I got this certification. As a general rule of thumb I don’t really put a lot of stock in certifications, regardless of their source. I suppose this is because I’ve seen a lot of people with certifications who don’t really know anything. They learned to regurgitate just enough to pass the certification exam without really trying to understand the subject matter, or applying it to a meaningful problem.

The cynic in me says that the main reason for this is bad certifications, focused on pushing through as many people as possible in order to maximise training revenue. The optimist in me is strangely quiet, and has been for a long time.

Agile certifications are even weirder to me than technical ones, as Agile is all about people, and certifications tend to verify processes and practices (things that are easily marked) over softer skills (much harder to mark). If you have an Agile certification of some sort, what does that even tell me? That you understand the concepts I suppose, but its not like they are hard to understand in the first place? I don’t see this practitioner certification as being particularly useful to my career, especially considering no-one uses Agile PM in Australia anyway.

Maybe if I travel to Europe it might be useful. I hear they use it there.