Beyond Guesswork: The Art and Science of Project Estimation
Elevating your project estimation game
👋 Hi, this is Sagar with this week’s newsletter of Mindful Matrix. I write about software engineering, growth mindset, and key learnings, along with candid reflections on my 9+ years of experience. Each piece resonates with my heart and mind, and is conveyed with genuine passion.
Today, I’m going to talk about the one of the most frequent tasks engineers face - Project estimations, and how to apply a systematic approach to predict the unpredictable. If you find the post helpful, please share it with your friends and coworkers. Enjoy!
Estimations —sounds familiar, doesn't it? I'm sure many of you, at some point in your career, have been handed the task of project sizing/estimation. And if by some chance you haven't yet, don't worry, your turn is likely just around the corner. As you climb the career ladder, mastering the art of project estimation becomes not just an advantage but a necessity.
It's a critical skill that significantly influences the success of a project and it’s not just guessing some numbers; it's fundamental in setting realistic expectations for stakeholders, aligning team efforts, and managing resources effectively. A well-done estimation lays the groundwork for project success.
After doing countless project estimations, here’s what I’ve learned from my numerous mistakes and how you can master this vital responsibility.
🔑Key Considerations in Estimation
Understanding Project Scope
To give yourself the best chance of accurately estimating the project, this initial(and likely the most crucial) step includes -
📝Thoroughly understand and document the project requirements
🤝Engaging early with all stakeholders, gathering their inputs and expectations, and translating these into technical requirements.
🎯❌Define what is within the scope and what falls outside
In a project I was involved in, my initial over-optimism led me to underestimate the scope, resulting in a significant delay. The lesson? Spend ample time in this phase.
Risk Analysis
🔍Risk assessment : Identify potential risks and dependencies early in the project and factor them into your estimates. This might include technological risks, staffing uncertainties, or dependencies on external teams/vendors.
🛡️Risk mitigation : Developing strategies to mitigate these risks is crucial, and would contribute to the upper bound of the estimated range. This proactive approach can save you from future headaches.
E.g., if a project depends on a third-party API, the risk of the API becoming unavailable should be accounted for in your project plan, and consider the effort required to mitigate this risk in your estimates.
In one of the early projects in my career, I overlooked the time required for the application security review conducted by our security certifier. This oversight resulted in a significant portion of the project timeline being consumed by addressing and fixing issues identified during the security review. Lesson learned: it's crucial to account for all dependencies, and potential strategies for risk mitigation in your project planning.
Estimation strategies
Estimating the time required to complete the project is your next step. This would mostly help in determining the number of developers needed. For effective time estimation consider following strategies -
🔨 Break it Down : For accuracy in estimation, use 'Divide and Conquer' strategy.
Divide the project into smaller, manageable, and possibly independent modules. Estimate the timeline of each module.
Divide the project, or even modules, into different phases with specific, measurable, and achievable technical goals for each phase. Estimate the timeline of each phase.
📊 Historical Analysis : Analyzing similar past projects can provide valuable insights.
💡Expert opinion - Leveraging expert opinions brings invaluable insights as they adept at identifying potential pitfalls and complexities that might not be apparent at the outset.
⏳Ramp up period : Consider your team's expertise with the project's systems and technologies. Estimates can change based on their skills. Adding a ramp up period to your estimates is often helpful.
⏰Add Buffer time : Don’t ignore the 'Unknown Unknowns'. Set aside a buffer for unforeseen challenges. Consider adding buffer for unexpected delays, testing, and iterations based on feedback during different phases of the project.
I often fell into the trap of underestimating the time needed for code reviews, overly optimistic and not fully considering how complex feedback, reviewer availability, and time zone differences could extend the process. Lesson learned : Adding a buffer time for code reviews ensures more accurate project estimates.
Transparency and Communication
💬 Estimation Transparency - Make it clear to stakeholders that early estimates, especially before HLD, are subject to change. Clearly explain any assumptions or constraints affecting these estimates. Clarify what your estimates cover, such as if they only include the initial code deployment after development testing, but exclude QA. This sets realistic expectations, ensures timely involvement of needed resources, and builds trust
📣Estimation Communication : Effectively handling early estimate challenges is about clear communication. Use three-point estimation (best, worst, most likely cases) for initial forecasts, offering a range of possible outcomes. It's best to use ranges to cover risks and unknowns early on. Over time, you can refine these ranges as you become more certain of your estimates.
E.g., Saying “Based on our preliminary analysis and considering the current variables and unknowns, we estimate the project to take between 6 to 9 months to complete. This range accounts for potential risks and uncertainties we've identified at this early stage (e.g. if API to fetch order history is provided by Ordering service or not, and if not then we would have to build that or if it’s available then we need to see if it’s scalable to support our usecase) . We'll refine this timeline as we delve deeper into the design phase and have a clearer view of the complexities involved.”
This would help stakeholders in understanding rationale behind the chosen range and preparing for potential variations.
🕰️Timing your estimation
Pre High-Level Design(HLD) : You may need to provide estimates before completing the HLD, often when details aren't clear yet, but stakeholders still want early figures for planning resources and schedules. It's a bit like guessing the amount of paint needed for a house when you've only seen the land. Focus on drawing from some key considerations mentioned in the above section.
Post HLD : Project estimation should be an iterative process, but the most critical estimation often occurs after the HLD is completed. HLD provides a clear overview of the project's scope, architecture and major components, enabling more accurate estimations.
Post Low-Level Design(LLD) : While post-HLD estimation gives a macro-level view, estimating post-Low-Level Design (LLD) can fine-tune the details, accounting for specific coding tasks and unit testing efforts.
Estimating too early can lead to vague and unreliable projections, while doing it too late might not leave enough room for resource allocation and scheduling. Be open to changes. Rigidity in scoping can lead to unrealistic estimates.
✨Conclusion
Effective project estimation requires a blend of technical understanding, management skills, and foresight. By considering the scope, timing, and potential pitfalls, and by actively engaging with the project team and stakeholders, you can ensure more accuracy and efficiency in project estimations.
Please note that this article, or any guide for that matter, isn't designed to unveil a secret formula for pinpointing the precise time or effort needed for your project. Given the nature of project work, especially on a larger scale, encountering unknowns is more often the rule than the exception. Instead, the goal in this article was to equip you with strategies and insights with systematic approach to predict the unpredictable, that will significantly narrow the gap between your initial estimations and the actual outcomes of your project delivery.
"The difference between a successful project and an unsuccessful one often lies in the quality of its initial estimation"
If you found this useful, please share it with your network and consider subscribing for more such insights.
If you haven’t subscribed, or followed me on LinkedIn, I’d love to connect with you. Please share your thoughts, feedback, and ideas, or even just to say hello!
In case you missed my previous articles…
Attitude: The Secret Ingredient in Software Engineering and Beyond
We all have dealt with astrology of estimations in our life. However, it is a practice and you get better with time.
However I have seen junior engineers giving "less" estimates in enthusiasm whereas seniors give seemingly "more" estimates for the same tasks. Part experience and part risk management I would say.