Introducing Continuous Delivery and amaysim

[authors note: this is a re-publication of a blog post originally published in January 2018]

Two years ago amaysim’s delivery teams would release to production once a month. A year ago we would release to production up to ten times a month. In the last four weeks we’ve made over a 100 production changes.

In 2018 we’ll soon be making dozens of production changes a day…

Hello and welcome to Engineering @ amaysim and our first post :). I’m Richard, the CTO at amaysim and I’m really proud to be able to give you a quick introduction to us and everything we do as both an Engineering group and a Technology-driven company.

To talk about amaysim is to talk about change. As a company we have an appetite for change, innovation and disruption like no other organisation I’ve worked with.

amaysim has recently transformed from being an award-winning, listed telco to a multi-vertical provider. We’ve always offered our customers Mobile services, but in the last 6 months we’ve also launched Broadband and Energy (Gas, Electricity and Solar) services as well as the chance to purchase phones and tablets from our shiny new E-commerce store. We have over one million happy mobile customers who consistently rank us as their favourite telco.

The Journey

As an Engineering group we’ve spent the last couple of years on a journey of transformation in the way we build and ship software. We’ve moved from monoliths to Micro-services (..to Nano-services). We’ve adopted containerisation as a build and deployment approach of choice and now run almost all of our stacks using Docker and Rancher. We’ve invested heavily in Serverless (the philosophy and the framework) and also proudly run many services in production using this approach. Naturally we’re 100% deployed on AWS (and have been for quite some time).

The story of amaysim Engineering over the last 2.5 years is one of growth, change, constant innovation and continuous improvement. The results have been fantastic and have truly helped transform our business.

We’ve been set challenging goals by the business, which means as an Engineering group we’ve had to step up, in order to be able to rapidly deliver on said targets. We’ve supported the business’s appetite for diversification, to rapidly onboard new verticals and tie them together with a seamless customer experience designed by our amazing Digital Products team. We’ve helped integrate and launch several new businesses in less than a year — I often remind the team:

“how often do you launch one new business, let alone three in a single year??”

There are many things that have made this transformation a reality, but one of the core drivers for how we deliver software and a foundation principle has been Continuous Delivery.

Continuous Delivery

The guiding principle behind our culture and practices here at amaysim Engineering has always been Continuous Delivery. At a high level the questions we constantly asking ourselves are:

Are we consistently delivering value back to our customers?

Can we find ways to do this more quickly?

And how can we do this while maintaining the high quality levels that are expected of us?

At purely an Engineering-level, Continuous Delivery has a much more specific meaning for us:

Is all of our software always in a state that it can be deployed, by choice, at any time?

Or even better (and more aspirational) is the ultimate goal of Continuous Deployment:

Is every change we make being continuously deployed into production?

Both interpretations of Continuous Delivery are lofty ambitions that are challenging and difficult to obtain, as documented in a wealth of online literature. The goal to achieve Continuous Delivery forced us to look inwards at our processes, our tools and our ways of working. It demanded that we look close and hard at our delivery practices and answer and address difficult questions. When we distilled it down, those questions were largely related to confidence:

Are we confident that what we’re building is the right thing for our customers? If not, how do we ensure that we minimise waste and can learn and pivot quickly as needed?

Are we confident we can release many, many times a day, at any time of the day or week?

Are we confident that if something does break, we’ll know about it first and can resolve it before most of our customers even realise?

At amaysim we’ve found that the answers to these questions requires the considered coming-together of a blend of areas: building a strong and durable Engineering Culture, making bold but pragmatic Technology Choices and establishing a core set of shared Delivery Practices that help guide us. And, of course, having a team of amazing people who actually make it all happen.

Culture

The aspiration of Continuous Delivery has driven us to define and adopt a culture that celebrates continuous improvement, innovation, integration and automation. We work hard to try and find that delicate, but crucial balance between evangelism and pragmatism, that allows us to keep moving forwards and continuously improving, whilst also still delivering value back to our customers on a regular basis.

The culture we’ve grown and fostered is one that encourages learning and improvement by being both reflective and retrospective, but also forward-thinking and consciously proactive. We don’t just do things because that’s the way they’ve always been done. There are no sacred cows. And we always, always try and leave things better than when we found them, even down to the smallest code change.

We aim for small, incremental, iterative changes as much as possible and use them as opportunities to test and learn. We apply this philosophy to the features and services we deliver to our customers, but also equally facing inwards we apply it to our own ways of working and our technology choices.

We value fast feedback loops and aggressively drive these to be as small as possible. We trust, guide and support our Engineering teams to deliver solutions under this umbrella and actively provide space for them to experiment, learn and innovate.

We’ve enthusiastically embraced the ‘DevOps’ mindset and it’s core tenets of close collaboration and communication. amaysim Software Engineers, supported and enabled by an awesome DevOps enablers group, own their code into production and the stacks upon which it’s deployed. We focus on high levels of testing, monitoring, measuring, logging and pre-emptive alarming that all help provide confidence that the software we release is stable, secure and running as designed.

We promote consistency in our ways of working and focus on uplifting the skillsets of our Engineering team. Key to all this has been fostering and amplifying the appetite for change that is so embedded in amaysim’s wider company culture and using it to help us as we strive to improve the way we deliver software.

Two years of this has taken us to a point now where all of our core applications can be deployed at any time, with high levels of confidence. We’ve actually reached our first milestone of Continuous Delivery across our core suite of applications. This is a huge achievement, one we’re really proud of and a validation of the culture we’ve built together.

Technology

From a technology-perspective we’ve also come a long way in a short space of time. Like many other organisations, we’ve seen the value that a Micro-Services architecture can bring and have adopted this approach wholeheartedly. However, one does not simply just switch to building Micro-services. It’s taken a lot of thought, planning, consistency and considerable technical innovation to make this a scalable, reliable and practical approach for delivering software.

To support this we’ve made the move to containerisation; building immutable artifacts with Docker, Docker-Compose and Make. We’ve defined our ‘3 Musketeers’ pattern and toolset to standardise and enable this (more on this in our next post..). We deploy our containers onto AWS and manage them using Rancher. This approach to building and deploying our software gives us consistency, allows us to deploy and release more quickly, removes boilerplate and enables us to build 12 Factor Apps by default.

Moving on from containers, we’ve also jumped with both feet into the Serverless world and have growing numbers of Serverless applications running in production. We were really proud to launch our first fully serverless business a couple of months ago. We’ve spoken at meetups and conferences about this and we’re collectively pumped for the potential that this approach to delivering software has.

Chief Architect Steve Ringo & Yun Zhi Lin speaking at Serverlessconf in NYC, Sept 2017

For the interested, we also have a goody-bag of Docker, Serverless and other utilities available as open source on our github page. Check it out!

Not content with just Micro-Services, we’ve also begun the journey to Micro-Frontends using our own flavour of JAMStack. React, Components and lots of other fun pieces make up this new way of building discreet front-end applications and we’re really excited to talk more about this approach in future posts.

And finally, we’ve also spent time taking difficult, so-called ‘legacy’ systems (you know, the ones that actually got us to where we are as a company) and applying best Engineering practices to work with them more effectively. We’re proud to say that we’ve also brought them along on the journey to Continuous Delivery. No one has been left behind.

Delivery Practices

We’ve also put a lot of focus on our core delivery practices and have spent considerable time streamlining and iterating on our processes to allow us to reach the goal of Continuous Delivery. This process is always ongoing, but we’ve invested much effort and experimentation in answering questions that enable this, such as:

What is our definition of done? How does this allow us to iterate and release more quickly? How does this add to and increase confidence in the solutions we’re delivering?

How is our definition of done consistently applied across all of our delivery teams?

How do we plan, manage and report on the delivery process and ensure that a continuous pipeline of the most valuable features are shipped to our customers?

Like many organisations, we’ve shaped our own blend of ‘agile’, which continues to evolve, grow and expand as we do. Again, we’re looking forward to sharing more about this crucial part of our process.

The Future

How to sum up... ?

This is just a small snippet, a teaser, of what we’ve been up to for the last couple of years. As well as our journey to Continuous Delivery, there’s heaps more we’ll talk to you about very soon: how we grew from two delivery teams to nine in just six months, how we’ve built a companion Engineering practice in Manila, how we run geographically dispersed teams across multiple cities and countries and how we went from midnight releases to daytime, zero downtime changes across the board.

It’s been an amazing voyage so far and I’m excited that my team and I have the chance to start sharing our story of change and innovation, and also our journey onwards as we continue to grow. Stay tuned..

Has any of the above piqued your interest? Does amaysim sound like the sort of place where you think you could make an impact? Do you thrive in organisations where you are empowered to bring change and constant improvement? Why not take a few minutes to learn more about our open roles and opportunities and if you like what you see then say hi, we’d love to hear from you..

Shout-out to all the lawyers..

The views expressed on this blog post are mine alone and do not necessarily reflect the views of my employer, amaysim Australia Ltd.