We at MVP Rockets have been working on helping visionary entrepreneurs launch kickass products and as we turn 1 year old, we take some time to reflect on our journey. We would be writing a series of blogs that explain all the learnings from our journey. If you have any queries, feel free to mail me at giri@mvprockets.com
What’s the dilemma?
Our entrepreneurs often demand high-quality and appealing products, coded effectively within a short period.
This poses some unique challenges for us in terms of speed and quality. Providing both together becomes the major issue we face.
- If speed is the primary focus, lean startup and Evolutionary design is the way to go. We start building a product on the insights and requirements that we clearly know and iteratively build as we uncover more of them. Developer tests in the local machine + pushes to QA —> PM tests on QA machine —>Pushes to prod. If anything breaks we fix them & repeat the process.
This is typically centered on Pareto principles. Having a website is better than not having one. We would ideally not take a call on not going live with a website because of small alignment issues. So, we compromise on perfection, for the sake of speed.
- If quality is all they want, a detailed engineering process, with an elaborate testing process could be a solution. A good rule of thumb for a team in this structure is 1:4:2 – 1 Product manager, 4 engineers, 2 QA.
Every time a developer completes a feature, he/she tests it in the local machine. Then, it gets pushed to the QA server, where a QA does a test pass. Then, it goes to the UAT, where the product manager does a QA pass. Finally, it gets pushed to the production system. All these processes make it time-consuming and prone to human errors.
The issue here is when you optimise for speed, quality suffers and if you optimise for quality, it takes more time. What we realised is that this problem can not be solved by merely altering/introducing processes. It required us to adopt a certain core philosophies – that have come to become a core part of our product/engineering philosophy.
Over the last 1 year, Praveen, Satish & our tech leadership have come up with some innovative solutions to the problems that we would love to share with you and gather your views to improve ourselves.
What’s our Philosophy?
Who doesn’t make mistakes? Everyone does that, even the most highly paid engineers from the most elite of the colleges make mistakes. I haven’t found one engineer who has not made any blunder.
So it’s foolish to think that we can avoid mistakes while building a product. Escaping mistakes is never the solution. What’s easier and more effective, is to build a process that allows one to identify and fix the problem in the shortest possible time.
There are a bunch of changes that we did with the design process that helped us to improve. We adopted atomic design, used consistent frameworks and storybooks and much more. These helped us be more predictable in terms of quality of designs & avoid any surprises later. But that’s for another blog on another day. Let’s stick to the technical process here.
We used a golden trio of continuous integration and continuous development (CI/CD) pipelines, Test-driven development (TDD) and Code metrics to achieve our objectives.
CI/CD pipelines
Earlier, when I was working on a product as a product manager, making a production release would take 7-8 hours. The process involved pushing the entire code to the QA server where the QA team tested it and then pushed it to the UAT.
Thereafter, our product team examined it, and then it was pushed to the product, after which a sanity plus smoke test was done on the product server, for all the key happy flows. All these processes took 7-8 hours on average to get done. If the product server identified any problem, the entire code was rolled back and the process was repeated after debugging.
This led to several unintended consequences such as:
- Our releases were often limited to once or twice a month. We were often scared of a prod push.
- We would be super stressed on the day of the release.
- The releases were often large with several fixes and features, creating more issues.
This was a vicious cycle that very few teams can come out of. The first thing we do to ensure that we never live in an everlasting state of fear about releases is that we always start a project by setting up the continuous integration and delivery (CI/CD) pipelines by using tools like Bitbucket.
At MVP Rockets, our production releases take a maximum of 15 minutes. We often do multiple product releases every day and our teams spend less than 5 percent of their time on issues and bugs. But this is not attributed to CI/CD pipelines alone. They wouldn’t be achieved without our next practice, which is Test-driven development.
Here’s a quick look at our process for CI, CD pipeline.
Test-driven development
In the same company, where I was a product manager – Over 60% of our developers and product managers time was spent on bugs and issues. If you think about it, this is insane.
The reason is simple. Over time, most products become complex & the features are super intertwined. People move on to other products or to other companies. The new guys make changes in the system without really understanding the intricacies of the entire system, making a larger mess. In some time, the system is such a hot mess, no one knows what will break when we make any change. This is the worst nightmare for any product manager or a developer.
Once you get to this, there is no way other than scrapping it all and building a new one from scratch. Now, don’t even get me started on the product revamp.
If you were to live through a production push (as explained above) in this hot mess, the biggest problem is, you would have made a change in one part of the product that broke another part, which was not tested the day the release was made. 2 days later there is an escalation from the support team that some part of the product is not working. It takes 2 people 4 hours to figure out that the change that went live 2 days back was the reason this feature is broken. You fix it, only to come back the next day to find there is another feature that is broken.
The root cause for this is that the product is being tested manually. When a product is complex, testing the entire product end to end can take days & human error is always possible. This is the antichrist of being agile. “Test Driven Development” is our solution to this problem.
In simple words, Test-driven development means that the developers write test cases before they write the actual code. It is the most critical aspect of our development.
Product managers and Developers would agree that the most complex issues or bugs often arise in the backend or Application Programming Interface (APIs), rather than in the frontend.
Given that we work primarily in MVPs and V1s of a product, we strive to have 100 percent code coverage on the backend code, rather than on the frontend and integrated use case testing.
This works well as we capture above 80 percent of the issues, stopping them from becoming a massive one with an effort of just 30 to 35 percent. The increase in utility of writing frontend and integration test cases does not justify the increase in cost and time for writing them. We use Chai and mocha for unit testing.
Code Metrics
We are currently working on over 10 products that we are developing and it’s impossible for our CTO to look into each product to ensure they are coming up well.
The only way to scale quality is to set metrics that will help us understand the overall health of the product. We track code coverage and code smells for identifying the quality of a product & use Sonarqube for the same.
Some of the metrics we look at are:
- Duplication Percentage
- Code Coverage
- Code Smells
- Security Hotspots
The first time I saw this, I was blown away. Measuring the quality of a code and taking decisions based on these metrics was all new to me. But once I saw it, there is no going back. Some screenshots from one of our product below –
This has enabled us to:
- Deliver high-quality products within a short period.
- Create incredible learning and engineering culture where developers would love to work.
One year of constant efforts and dedication has given us fruitful results. We have been able to deliver high-quality products within a short period and carry out multiple product releases over the month with each release taking a maximum of 15 minutes.
Moreover, we are constantly trying to make an incredible learning and engineering culture where developers would love to work.
We haven’t mastered the entire process, but we are way better than we were a year ago. Getting such impactful results in such a short time gives us immense happiness and we are constantly improving ourselves to deliver our best and achieve more success in the coming years.
The next issue that we are facing is at the frontend, which is about getting a pixel-perfect, consistent user interface (UI) in a quick time. We are addressing this with a combination of Tailwind/Bootstrap, Atomic Design and Storybook. We will address this in our next post.
Over and Out!