Software at Scale 12 - John Micco: Cloud Transformation Architect, VMWare

Software at Scale

John Micco is a Cloud Transformation Architect at VMWare, where he works on CI/CD systems. He’s worked in the CI/CD space for nearly twenty years at various companies like Netflix and Google, he’s authored several research papers, and was a keynote speaker at the International Conference of Software Testing and Verification (ICST). Our collaboration helped shape my work on Athena at Dropbox.

This episode is extremely technical. We talk about the management of CI servers and systems in large companies, common problems faced, and themes in emerging solutions. Large-scale CI/CD management is far from being commoditized due to the custom configuration of every company’s build tooling.

Apple Podcasts | Spotify | Google Podcasts

Highlights

0:00 - What is a “Cloud Transformation Architect”?

6:00 - Sharing knowledge in the CI/CD space

7:00 - A comparison between starting a car company and starting a software company, and why standardization on tools is so much trickier in software

11:30 - The scale of testing systems. Managing a system that handled 10k RPS of test result data. The quadratic growth in compute resources required to manage a testing system

18:44 - “Only 7% of code changes had problems discovered by tests”

23:36 - Is unit testing overrated?

30:00 - At what point can companies no longer afford to run all tests for every code change

35:00 - What is Culprit Finding / Regression ID? What is auto-rollback / test quarantine?

41:00 - Culprit Finding at Dropbox

45:00 - Dealing with flakiness at the test layer vs. the platform layer. An ongoing challenge for VMWare

60:00 - Increase in demand for CI compute when developers don’t have any meetings

65:00 - Bazel migration at VMWare

73:00 - Developing an interest in CI/CD systems. And why it’s an exciting space where there’s a lot of innovation, and why more innovation needs to happen

80:00 - How can someone tell whether they’re investing enough in developer productivity / their CI/CD experience for engineers? The trade-offs that companies should consider when deciding to run tests.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.softwareatscale.dev