Shivam ChhunejaFreeCAD, the open-source 3D CAD modeler, has garnered quite a community with over 12,000 commits, 1,100 contributors, and more than 5,000 pull requests (PRs) to date. Known for its versatility in mechanical engineering and product design, FreeCAD is a go-to for makers and professionals alike.
But are they actually nailing their PR process, or just racing through merges? With an average PR merge time of 7.5 days, FreeCAD's workflow efficiency is up for debate. Let's break down the numbers and see what's really happening behind the scenes.
We are using Middleware Open Source to get the data and insights for these open source repos, so if you want to recreate or get data of your favorite God level repo you know where to go!
If you're curious to start a conversation with fellow engineering leaders, join The Middle Out Community! Don't forget to subscribe to our newsletter below for exclusive case studies!
Channeling Our Inner Sam Spade: Cracking the Case of FreeCAD's PR Efficiency
As we at Middleware dug into FreeCAD's Dora metrics, we couldn't resist going full "Sam Spade," curious about what made this repository tick. So, we dove in to explore the patterns, looking for what makes them tick!. What we found sheds light on some best practices that other repos can take notes from.
Background on Dora Metrics
Dora Metrics are a set of key performance indicators used to measure software development velocity and efficiency. They include:
Deployment Frequency
Lead Time for Changes
Mean Time to Restore (MTTR)
Change Failure Rate
These metrics help organizations understand their development processes and identify areas for improvement.
Key Findings
Exemplary Metric: Merge Time
One standout metric for FreeCAD is its Merge Time for PRs. Efficient merge times are a testament to streamlined code reviews and strong team coordination.
June 2024 -- Average Merge Time: 15.66 Hours
In June, FreeCAD's PR management was running relatively smoothly, with an average merge time of 15.66 hours.
July 2024 -- Average Merge Time: 20.4 Hours
July showed a bit of a slowdown, with an average merge time stretching to 20.4 hours. This change could be attributed to more complex PRs
August 2024 -- Average Merge Time: 8.04 Hours
In August, things really picked up pace, with the average merge time dropping significantly to just 8.04 hours. This lightning-fast rate suggests that the team and community were hyper-efficient in handling pull requests.
Average Merge Time: Approximately 11.02 hours.
Rapid Merges: PRs like #14693, #14691, and #14688 were merged within less than a day.
Factors Contributing to Efficiency
Ownership: Consistent authors like PaddleStroke, Roy-043, and wwmayer contribute to swift merges.
Structured PR Review: Automated checks via GH Actions, including workflows like CI_master.yml and sub_lint.yml, catch issues early.
Also read: Is freeCodeCamp Sacrificing Quality for Speed with Their Rapid Deployments?
Metric that needs Improvement: Cycle Time
While freeCAD has made commendable strides in reducing Cycle Time, with figures decreasing from 171.12 hours to 130.8 hours and then to 121.68 hours, there's still room for improvement.
Continuing this trend could further enhance their efficiency. Addressing bottlenecks and optimizing workflows could help freeCAD push these numbers even lower, achieving even faster turnaround times and reinforcing their reputation for swift, effective development.
Average Cycle Time: Approximately 21.42 hours.
Fluctuation: Inconsistencies arise, as seen in longer cycle times for complex PRs like #14674.
Factors Contributing to Delays
Complex PRs: More intricate PRs understandably take longer.
Reviewer Availability: Delays due to the availability and assignment of reviewers.
Other Key Metrics
Lead Time
The reduction in Lead Time from 2.51 days in June to 1.81 days in August is a clear indicator of enhanced efficiency in moving code through the development pipeline. This is a commendable achievement, suggesting that freeCAD has improved its processes for deploying code, leading to faster delivery of new features and fixes. This efficiency boost likely contributes positively to the overall project velocity and responsiveness.
First Commit to Open
The increase in the time taken from First Commit to Opening a Pull Request, from 3.77 hours in June to 12.01 hours in August, is a bit concerning. This rise might indicate bottlenecks or delays in the initial stages of the contribution process. Addressing this could involve streamlining the process for handling code submissions, possibly by improving workflows or increasing the availability of reviewers to handle initial requests more promptly.
First Response Time
Although there's a slight improvement from 11.42 hours in June to 10.15 hours in August, there's still potential for faster response times. This metric is crucial for maintaining an active and engaged contributor base, as prompt responses can enhance collaboration and prevent contributors from losing interest or moving on to other projects. Focusing on reducing this time further could improve contributor satisfaction and keep the development process moving smoothly.
Nature of Work: Feature-Focused Development
FreeCAD is rich in diverse contributions, spanning feature development, bug fixes, and optimization work.
Feature Development
Major focus, e.g., Assembly: TNP by PaddleStroke.
Bug Fixes
Substantial effort in maintaining and fixing issues, e.g., Fem: Fix height of reference list widget in spring constraint task panel by marioalexis84.
Optimization and Refactor
Significant, seen in PRs like Guard all uses of basic_random_generator for thread safety by bgbsww.
Each of these contributions highlights the diverse nature of work being done on FreeCAD---from feature development that solves long-standing issues to bug fixes that refine the user experience, to optimizations that ensure the software can handle future demands.
Impact on Project and Community
Efficient PR management leads to better code quality, more frequent releases, and higher contributor satisfaction. FreeCAD's emphasis on swift PR reviews encourages further contributions.
Takeaways
Structured Workflows: Implement CI/CD pipelines to automate checks, as FreeCAD's CI_master.yml ensures consistency.
Active PR Review Culture: Prioritize swift reviews to avoid PR pile-ups.
Clear Ownership Assignments: Assign code owners to specific modules for better PR management.
Balanced Work Distribution: Encourage a mix of new features, bug fixes, and refactoring for a well-maintained codebase.
DORA Score: 8/10
After taking a closer look at FreeCAD with our Dora Metrics toolkit, here's the rundown: FreeCAD scores a respectable 8/10. Its PR merge times are impressively quick, thanks to a committed community that's always in motion. But---and there's always a "but"---Cycle Time is where FreeCAD hits a few speed bumps, with complex PRs dragging down the average.
The takeaway? FreeCAD's team is doing a solid job, but a little change here and there could take them from great to exceptional. Our analysis, stacked against Google's Dora benchmarks, highlights FreeCAD's wins and its opportunities for improvement. Want to see how your project compares or optimize for efficiency? Analyze it using Middleware OSS.
Conclusion - FreeCAD: A PR Powerhouse or Just Polished?
FreeCAD's got some slick moves when it comes to handling PRs---those rapid merge times are nothing short of impressive. But like a dance partner with two left feet, its Cycle Time could use a little more rhythm. With just a bit of fine-tuning, FreeCAD could be on its way to even greater efficiency.
If you're curious to pick the brains of fellow engineering leaders, join The Middle Out Community!
Trivia
FreeCAD was started in 2002 by Jürgen Riegel, Werner Mayer, and Yorik van Havre, making it one of the oldest free and open-source 3D CAD modeling tools
