Major projects come and go. It’s being able to reflect on the successes and pitfalls – and implementing lessons learned going forward – that allows future engineers to improve.
Many teams refer to this period as a project post-mortem, but human resources tech company Justworks sees it differently. Built In NYC caught up with engineering Manager Justin McKee about his process for continuous improvement, whether it’s working around the new COVID-19 business normal or identifying avoidable bugs in upcoming projects.
For Colin Allen, program manager at Bluecore, the aha moment that allows his team to move forward often comes from understanding the reasoning behind initial execution – like team structure or product prioritization – and adjusting as necessary.
“I find the most interesting revelations from a post-mortem come from understanding how culture manifests itself in code,” Allen said.
McKee doesn’t believe that a project ever actually dies. Rather, it continues carrying important lessons on how to better plan for the future. And he and his team act accordingly.
What does a typical post-mortem look like for your team, and how do you structure those meetings to ensure you’re making the most of that time?
At Justworks, we refer to these meetings as Post-Project Kaizens (PPK). The term “kaizen” refers to a system of continuous improvement and aligns with our goals of these after-action meetings: to identify key learnings and opportunities for improvement. We chose to avoid the term “post-mortem” because it connotes a sense of finality. In our experience, a project is rarely ever “dead” – it lives on in lessons learned and future maintenance.
Our Post-Project Kaizen template includes: description, timeline/key dates, accomplishments, problem areas, lessons learned, opportunities for improvement, and post-project action items.
A great way to start the PPK is to celebrate the team’s accomplishments through the project. Then, we spend most of our time discussing lessons learned and opportunities for improvement that were surfaced in the previous two sections. Any action items are assigned a responsible party and a due date.
What’s one of the most valuable revelations or lessons that has come from a project post-mortem, and how has that helped your team grow?
When the U.S. government passed COVID-19 legislation providing assistance to businesses, Justworks immediately began supporting customers in navigating the complex legal requirements and transferring emergency funds.
The organization mobilized to work together to provide an accurate and swift turnaround. This entailed working across functions and fundamentally rethinking boundaries of responsibility. Our PPK highlighted teams’ tremendous accomplishments, but it also identified challenges raised by this new way of working.
We faced moments of uncertainty around ownership and expectations, inefficient resourcing and avoidable bugs. Rather than relying on familiar team-specific ways of working, we could have coordinated better with centralized knowledge repositories and designated point people.
These learnings were invaluable when the organization needed to quickly implement subsequent pieces of legislation. Due to our PPK, we can still work together quickly without uncertainty or frustration. We now create a separate mission-specific project board rather than distributing tickets across team boards.
In our experience, a project is rarely ever dead – it lives on in lessons learned and future maintenance.”
What’s one thing you’ve done to improve your post-mortems over time?
One of the best-received changes we made was to publish our description and timelines in advance of the meeting, rather than attempting to reconstruct relevant events in real time. As a result, we all start the PPK with a shared understanding of the project details, and we have more time to discuss the juicy takeaways rather than sweat details.
At Bluecore, Allen prepares his team to ask the “five whys” at each post-mortem. The method derives from a technique used in retrospectives to discover the root cause of an issue. Asking “why” five times per problem informs the basis of subsequent inquiries.
What does a typical post-mortem look like for your team, and how do you structure those meetings to ensure you’re making the most of that time?
To get the most of a post-mortem, good preparation is key. Before the meeting, document the incident timeline to capture the known facts (who, what, when and where). Distribute this to the participants before the meeting, so that the team steps into the meeting ready to ask the “five whys.”
In the meeting, it is key to have a blameless environment to ensure a constructive conversation. Those with the most context should feel safe to contribute to the conversation without fear of repercussion. Dig into systemic issues and focus on actionable changes as outputs.
Time management is also key. Ensure that there are 10 minutes at the end of the meeting to review the action items generated so that they can be assigned and scheduled for completion. Without follow up, little value is gained from any retrospective.
The most interesting revelations from a post-mortem come from understanding how culture manifests itself in code.’’
What’s one of the most valuable revelations or lessons that has come from a project post-mortem, and how has that helped your team grow?
I find the most interesting revelations from a post-mortem come from understanding how culture manifests itself in code. Past decisions, team structure and product prioritization can help explain a feature’s complex systems working in unexpected or unplanned ways.
The challenge with this type of fragility is that it is very hard to predict future trends. Through post-mortems, the team has learned that the only sustainable approach to maintaining a complex system is by shifting their efforts from reactive to proactive.
The team is taking two different tacks to holistically reduce the flakiness in the system. One is to commit to fixing the system after the immediate issue is remediated. Secondly, the team is investing in proactive programmatic alerts, which are preferable to bugs, a lagging indicator.
What’s one thing you’ve done to improve your post-mortems over time? What were the results?
Traceability has been key. Over time, I have gotten better at ensuring that action items are written up into Jira tickets and that they are completed in a reasonable timeframe. Every team is busy, so accountability is key to ensure that technical debt is addressed.
