Android Backstage, a podcast by and for Android developers. Hosted by developers from the Android engineering team, this show covers topics of interest to Android programmers, with in-depth discussions and interviews with engineers on the Android team at Google. Subscribe to Android Developers YouTube → https://goo.gle/AndroidDevs
…
continue reading
Player FM - Internet Radio Done Right
Checked 6M ago
Added four years ago
Content provided by Lee Atchison. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Lee Atchison or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://player.fm/legal.
Similar to Modern Digital Applications with Lee Atchison
The power of Data is undeniable. And unharnessed - it’s nothing but chaos. Making data your ally. Using it to lead with confidence and clarity. Host Jess Carter is solving problems in real-time to reveal what’s possible. Helping communities and people thrive. This is Data Driven Leadership, a show brought to you by Resultant.
…
continue reading
The American healthcare system is one of the most innovative in the world. But it’s also riddled with complex challenges, such as access to affordable medications, inefficiency and administrative burdens, and communication barriers between providers. There’s clearly a better way—and at Surescripts, we have a unique sightline into what that may be. In this series, host Melanie Marcus, Chief Marketing Officer of Surescripts, sits down with today’s most inspiring and innovative leaders in healt ...
…
continue reading
Show notes are at https://stevelitchfield.com/sshow/chat.html
…
continue reading
Investor Shayle Kann is asking big questions about how to decarbonize the planet: How cheap can clean energy get? Will artificial intelligence speed up climate solutions? Where is the smart money going into climate technologies? Every week on Catalyst, Shayle explains the world of climate tech with prominent experts, investors, researchers, and executives. Produced by Latitude Media.
…
continue reading
1
Defensive Security Podcast - Malware, Hacking, Cyber Security & Infosec
Jerry Bell and Andrew Kalat
5k
200
Defensive Security is a weekly information security podcast which reviews recent high profile cyber security breaches, data breaches, malware infections and intrusions to identify lessons that we can learn and apply to the organizations we protect.
…
continue reading
Every Friday and Sunday, Slate’s popular daily news podcast What Next brings you TBD, a clear-eyed look into the future. From fake news to fake meat, algorithms to augmented reality, Lizzie O’Leary is your guide to the tech industry and the world it’s creating for us to live in.
…
continue reading
Material is a weekly discussion about the Google and Android universe. Your intrepid hosts try to answer the question, “What holds up the digital world?” The answer, so far, is that it’s Google all the way down. Hosted by Andy Ihnatko and Florence Ion.
…
continue reading
TechStuff is getting a system update. Everything you love about Tech Stuff now twice the bandwidth with new hosts, Oz Woloshyn (Sleepwalkers) and Karah Preiss (Sleepwalkers). Oz and Karah bring humour and wit to the table as they break down what's happening in tech...and what it says about us. TechStuff is the podcast where technology meets culture. We speak to the folks building the future to understand what tomorrow will look like and how our technology is changing us: how we live, how we ...
…
continue reading
A tech podcast for the gadget lovers and tech heads among us from the mind of Marques Brownlee, better known as MKBHD. MKBHD has made a name for himself on YouTube reviewing everything from the newest smartphones to cameras to electric cars. Pulling from over 10 years of experience covering the tech industry, MKBHD and co-hosts Andrew Manganelli and David Imel will keep you informed and entertained as they take a deep dive into the latest and greatest in tech and what deserves your hard earn ...
…
continue reading
Player FM - Podcast App
Go offline with the Player FM app!
Go offline with the Player FM app!
))
Daily Deals
Podcasts Worth a Listen
SPONSORED
T
The Big Pitch with Jimmy Carr
1 Phil Wang Pitches Psychological Thriller Starring WHO?! 24:35
24:35 
Play Later 
Play Later 
Lists 
Like 
Liked24:35
Play Later
Play Later
Lists
Like
LikedIt’s the very first episode of The Big Pitch with Jimmy Carr and our first guest is Phil Wang! And Phil’s subgenre is…This Place is Evil. We’re talking psychological torture, we’re talking gory death scenes, we’re talking Lorraine Kelly?! The Big Pitch with Jimmy Carr is a brand new comedy podcast where each week a different celebrity guest pitches an idea for a film based on one of the SUPER niche sub-genres on Netflix. From ‘Steamy Crime Movies from the 1970s’ to ‘Australian Dysfunctional Family Comedies Starring A Strong Female Lead’, our celebrity guests will pitch their wacky plot, their dream cast, the marketing stunts, and everything in between. By the end of every episode, Jimmy Carr, Comedian by night / “Netflix Executive” by day, will decide whether the pitch is greenlit or condemned to development hell! Listen on all podcast platforms and watch on the Netflix Is A Joke YouTube Channel . The Big Pitch is a co-production by Netflix and BBC Studios Audio. Jimmy Carr is an award-winning stand-up comedian and writer, touring his brand-new show JIMMY CARR: LAUGHS FUNNY throughout the USA from May to November this year, as well as across the UK and Europe, before hitting Australia and New Zealand in early 2026. All info and tickets for the tour are available at JIMMYCARR.COM Production Coordinator: Becky Carewe-Jeffries Production Manager: Mabel Finnegan-Wright Editor: Stuart Reid Producer: Pete Strauss Executive Producer: Richard Morris Executive Producers for Netflix: Kathryn Huyghue, Erica Brady, and David Markowitz Set Design: Helen Coyston Studios: Tower Bridge Studios Make Up: Samantha Coughlan Cameras: Daniel Spencer Sound: Charlie Emery Branding: Tim Lane Photography: James Hole…
News Report: Google Cloud Next
Manage episode 294678596 series 2937999
Content provided by Lee Atchison. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Lee Atchison or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://player.fm/legal.
Google Cloud Next is coming to the Moscone Center in San Francisco from April 6 until April 8. This is Google’s big cloud event of the year.
Links and More Information
The following are links mentioned in this episode, and links to related information:
- Announcement — Google Cloud Next (https://cloud.google.com/blog/topics/google-cloud-next/join-us-at-google-cloud-next-2020)
- Registration — Google Cloud Next (https://cloud.withgoogle.com/next/sf)
- Use code GRPABLOG2020 for $500 USD off a full price ticket.
- Modern Digital Applications Website (https://mdacast.com)
- Lee Atchison Articles and Presentations (https://leeatchison.com)
- Architecting for Scale, published by O’Reilly Media (https://architectingforscale.com)
Special Report
Hello everyone and welcome to Modern Digital Applications. This is just a very quick news break...
Those of you using Google Cloud will want to hear this. Google Cloud Next is coming to the Moscone Center in San Francisco from April 6 until April 8. This is Google’s big cloud event of the year.
According to the information provided by Google: Google Cloud Next brings together a global cloud community of leaders, developers, and influencers to help you get inspired and solve your most pressing business challenges.
If you would like to attend, check out the show notes for a link to the registration page. Also in the show notes is a code that is good for $500 off a full price ticket.
You can expect that Google will have a host of new product announcements around this time, so even if you don’t attend, listen up in April for more news from Google Cloud.
31 episodes
ShareManage episode 294678596 series 2937999
Content provided by Lee Atchison. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Lee Atchison or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://player.fm/legal.
Google Cloud Next is coming to the Moscone Center in San Francisco from April 6 until April 8. This is Google’s big cloud event of the year.
Links and More Information
The following are links mentioned in this episode, and links to related information:
- Announcement — Google Cloud Next (https://cloud.google.com/blog/topics/google-cloud-next/join-us-at-google-cloud-next-2020)
- Registration — Google Cloud Next (https://cloud.withgoogle.com/next/sf)
- Use code GRPABLOG2020 for $500 USD off a full price ticket.
- Modern Digital Applications Website (https://mdacast.com)
- Lee Atchison Articles and Presentations (https://leeatchison.com)
- Architecting for Scale, published by O’Reilly Media (https://architectingforscale.com)
Special Report
Hello everyone and welcome to Modern Digital Applications. This is just a very quick news break...
Those of you using Google Cloud will want to hear this. Google Cloud Next is coming to the Moscone Center in San Francisco from April 6 until April 8. This is Google’s big cloud event of the year.
According to the information provided by Google: Google Cloud Next brings together a global cloud community of leaders, developers, and influencers to help you get inspired and solve your most pressing business challenges.
If you would like to attend, check out the show notes for a link to the registration page. Also in the show notes is a code that is good for $500 off a full price ticket.
You can expect that Google will have a host of new product announcements around this time, so even if you don’t attend, listen up in April for more news from Google Cloud.
31 episodes
All episodes
×
1 Notice: New podcast starting later this summer 2:12
2:12 Play Later Play Later Lists Like Liked2:12
Play Later Play Later Lists Like LikedModern Digital Applications is changing and coming back after its year long hiatus. Join us for the launch of Modern Digital Business! Modern Digital Business will be coming later this summer. If you'd like to be informed when it's ready to launch, please go to mdb.fm/launch . We hope to see you there!…
1 The Great Repo Debate with Kevin Goslar, p2 34:56
34:56 Play Later Play Later Lists Like Liked34:56
Play Later Play Later Lists Like LikedMy guest today is Kevin Goslar. Kevin is the Senior Vice President for Technology Strategy at Originate, a digital agency that helps organizations with digital transformation best practices. He has a PhD in business informatics, and is an avid software developer. He currently is the maintainer for Git Town, an open-source project that provides a high-level CLI for Git. Previously, Kevin worked as a software developer at Google, which is where he was exposed to Mono-repos. Kevin is a Git expert and process advocate, and he’s here to discuss with me the pros and cons of monorepos vs polyrepos. This is part 2 of my interview with Kevin.…
1 The Great Repo Debate with Kevin Goslar, p1 33:15
33:15 Play Later Play Later Lists Like Liked33:15
Play Later Play Later Lists Like LikedMy guest today is Kevin Goslar. Kevin is the Senior Vice President for Technology Strategy at Originate, a digital agency that helps organizations with digital transformation best practices. He has a PhD in business informatics, and is an avid software developer. He currently is the maintainer for Git Town, an open-source project that provides a high-level CLI for Git. Previously, Kevin worked as a software developer at Google, which is where he was exposed to Mono-repos. Kevin is a Git expert and process advocate, and he’s here to discuss with me the pros and cons of monorepos vs polyrepos.…
My guest today is Beth Long. Beth worked at New Relic where she held roles in both engineering and marketing, including two years leading the Reliability Engineering team, which owned the tooling and process for incident response and analysis. She also led New Relic's collaboration with the SNAFU Catchers, a group of researchers in- vestigating how tech companies learn from incidents. Beth recently left New Relic to join the startup Jeli.io, where she leads the engi- neering team working on the industry's first incident analysis platform. Links Beth Long, Engineering Manager at Jeli.io LinkedIn: https://www.linkedin.com/in/beth-adele-long/ Twitter: https://twitter.com/BethAdeleLong Featured in this episode: Jeli.io (https://jeli.io) Learning From Incidents with Jeli (https://leeatchison.com/atscale/2020/12/07/learning-from-incidents-with-jeli/) S3 Outage Mentioned in this Episode (https://thenewstack.io/dont-write-off-aws-s3-outage-fat-finger-folly/)…
My guest today is Beth Long. Beth worked at New Relic where she held roles in both engineering and marketing, including two years leading the Reliability Engineering team, which owned the tooling and process for incident response and analysis. She also led New Relic's collaboration with the SNAFU Catchers, a group of researchers in- vestigating how tech companies learn from incidents. Beth recently left New Relic to join the startup Jeli.io, where she leads the engi- neering team working on the industry's first incident analysis platform. Links Beth Long, Engineering Manager at Jeli.io LinkedIn: https://www.linkedin.com/in/beth-adele-long/ Twitter: https://twitter.com/BethAdeleLong Featured in this episode: Jeli.io (https://jeli.io) Learning From Incidents with Jeli (https://leeatchison.com/atscale/2020/12/07/learning-from-incidents-with-jeli/) S3 Outage Mentioned in this Episode (https://thenewstack.io/dont-write-off-aws-s3-outage-fat-finger-folly/)…
1 ICYMI: Reducing the Risk of Your Cloud Migration 10:03
10:03 Play Later Play Later Lists Like Liked10:03
Play Later Play Later Lists Like LikedThe scheduling of a cloud migration is a complex undertaking that should be thought and planned in advance. But in order for a migration to be successful, it’s important that you limit your risk as much as possible during the migration itself, so that unforeseen problems don’t show up and cause your migration to go sideways, fail outright, or result in unexpected outages that negatively impact your business.…
1 ICYMI: Avoid Downtime When Migrating Data to the Cloud 11:38
11:38 Play Later Play Later Lists Like Liked11:38
Play Later Play Later Lists Like LikedMoving your data is one of the trickiest parts of a cloud migration. During the migration, the location of your data can have a significant impact on the performance of your application. During the data transfer, keeping the data intact, in sync, and self-consistent requires either tight correlation or—worse—application downtime. Moving your data and the applications that utilize the data at the same time is necessary to keep your application performance acceptable. Deciding how and when to migrate your data relative to your services, though, is a complex question. Often companies will rely on the expertise of a migration architect, which is a role that can greatly contribute to the success of any cloud migration. Whether you have an on-staff cloud architect or not, there are three primary strategies for migrating application data to the cloud.…
1 Future of Cloud Identity with Thomas Curran, p2 17:35
17:35 Play Later Play Later Lists Like Liked17:35
Play Later Play Later Lists Like LikedMy guest today is Thomas Curran. Thomas is a cloud executive with many years of experience, including VP of Technology and Innovation at Deutsche Telekom and Technology Advisor at Deutsche Börse. He is the co-founder of the Ory Software Foundation, which is the owner of a very popular open source, go-based, identity management library named Kratos, along with other open source identity management tools. Now, Thomas is co-founders of Ory Corp, an Open Source Identity Infrastructure and Services company. Thomas is with me today from his office in Munich, Germany, to talk about application identity management. As means of full disclosure, I’ve worked with Thomas personally for many years, first meeting him back when he was at Deutsche Börse. I’m now currently working directly with Thomas at Ory, architecting their new cloud infrastructure. Links and More Information * Thomas LinkedIn (https://www.linkedin.com/in/thomasaidancurran/) * Ory (https://ory.sh) Tech Tapas — History of the Term SaaS When did software as a service start? Well, that depends on what you mean by the term… depending on how you define SaaS, the answer is either the early 1960s, or somewhere around 2005. Back in the early days of computing, all applications ran on a centralized computer. Users accessed the computers remotely. Initially via punch cards and later via remote terminals. The centralized nature of the application is, by a true definition, Software as a Service. But the modern definition of SaaS is tied much more closely with cloud computing. SaaS now-a-days refers to software running centrally, typically in a public or private cloud environment, and is shared among multiple users. A thin client of some sort — either a web browser or a thin mobile application — is used to front the centralized application. From a business model standpoint, users don’t buy SaaS software, instead they rent or lease access to it with monthly or annual fees. Alternatively, the service could be free and supported by advertising or other monetization processes. This is the heart of the business model for social media, for example. So, SaaS is an old term that has been given new meaning in recent years. But it’s the recent definition that has really changed the way people think and build software today. Tech Tapas — Amazon S3 Amazon S3. A highly durable, highly available file and object storage mechanism in the cloud. This service is the go to service for most companies that want to store huge quantities of data in the cloud, or for long term persistent object storage. S3 was designed with the goals of being highly available, highly durable, and highly scalable. The design goal for availability is 99.99%, with a durability of objects of 99.999999999 (that’s 11 9’s). How available? The 4 9’s availability translates to a total of 52 minutes of downtime per year. How durable? The 11 9’s durability means that if every man, woman, and child in the world had an object in S3, then Amazon would lose at most one of those objects, approximately once every 15 years. These are amazing goals, and is one of the reasons S3 has such a great reputation as a high quality object storage system. S3 was one of three initial AWS services and was a big part of AWS’s early success.…
1 Future of Cloud Identity with Thomas Curran, p1 18:04
18:04 Play Later Play Later Lists Like Liked18:04
Play Later Play Later Lists Like LikedMy guest today is Thomas Curran. Thomas is a cloud executive with many years of experience, including VP of Technology and Innovation at Deutsche Telekom and Technology Advisor at Deutsche Börse. He is the co-founder of the Ory Software Foundation, which is the owner of a very popular open source, go-based, identity management library named Kratos, along with other open source identity management tools. Now, Thomas is co-founders of Ory Corp, an Open Source Identity Infrastructure and Services company. Thomas is with me today from his office in Munich, Germany, to talk about application identity management. As means of full disclosure, I’ve worked with Thomas personally for many years, first meeting him back when he was at Deutsche Börse. I’m now currently working directly with Thomas at Ory, architecting their new cloud infrastructure. Links and More Information * Thomas LinkedIn (https://www.linkedin.com/in/thomasaidancurran/) * Ory (https://ory.sh)…
The scheduling of a cloud migration is a complex undertaking that should be thought and planned in advance. Typically, a migration architect is involved and makes the difficult technical decisions of what to migrate when, in concert with the organization management to take into account the business needs. But it’s important for a migration to be successful that you limit your risk as much as possible during the migration, so that unforeseen problems don’t show up and cause your migration to go sideways, fail, or result in unexpected outages that negatively impact your business. When scheduling the migration, there are a number of things you should keep in mind to increase the likelihood of a successful migration and reduce the risk of the migration itself. Here are five key methods to reducing the risk of your cloud migration, and hence increase your overall chance for success. Links and More Information The following are links mentioned in this episode, and links to related information: • Modern Digital Applications Website (https://mdacast.com) • Lee Atchison Articles and Presentations (https://leeatchison.com) • Architecting for Scale, published by O’Reilly Media (https://architectingforscale.com) • Advising and Consulting Services by Lee Atchison (https://atchisontechnology.com) • Course: Building a Cloud Roadmap, 2018-2019 (https://leeatchison.com/classes/building-a-cloud-roadmap/) Key #1. Limit the complexity of migrating your data The process of migrating your data from your on-premise datastores to the cloud is, itself, the hardest, most dangerous, and most time-consuming part of your migration. There are many ways to migrate your data…some of the methods are quite complex and some of them are very basic. Some of them result in no need for downtime, others require significant downtime in order to implement. There is a tradeoff you need to make between the complexity of the migration process and the impact that complexity has on the migration, including the potential need for site downtime. While in some scenarios you must implement a complex data migration scheme to reduce or eliminate downtime and reduce risk along the way, in general I recommend choosing as simple of a data migration scheme as possible given your system constraints and business constraints. The more complex your data migration strategy, the riskier your migration. By keeping the data migration process as simple as practical given your business constraints, you reduce the overall risk of failure in your migration. Be aware, though, that you may require a certain level of migration complexity in order to maintain data redundancy and data availability during the migration itself. So the ultimate simplest migration process may not be available to you. Still, it’s important that you select the simplest migration process that achieves your business and technical migration goals. Key #2. Reduce the duration of the in-progress migration as much as possible. Put another way, do as much preparation work before you migrate as you can, and then once you start the migration, move as quickly as possible to completing the migration, postponing as much work as possible until after the migration is complete and validated. By doing as much preparation work before the migration as possible and pushing as much cleanup work to after the migration as possible, you reduce the amount of time and complexity of the migration itself. Given that your application is most at risk of a migration related failure during the migration process itself, reducing this in-migration time is critical to reducing your overall risk. For example, it may be possible to accept a bit lower overall application performance in the short term—during the migration, in order to get to the end of your migration quicker. Then, after the migration is complete, you can do some performance refactorings to improve your overall performance situation. While postponing an important performance improvement is not normally ideal as it increases your technical debt. In this case, the delay may be more beneficial, because it allows the migration itself to complete quicker which reduces your overall migration related risk. Remember, your application is most at risk for something going wrong after the migration starts until the time when the entire application is moved to the cloud and the migration is complete. By postponing this performance improvement work until post-migration, you are able to keep the time of migration to as short a period as possible, hence reducing your risk. Key #3. Leave yourself as many options to back out a migration step as you can. The more options you have to revert a step the less risky your migration will be as a whole. As you take each step of the migration, think before you execute the step. Ask yourself how you could back out of the step if something happened that requires you to backoff. Every step you take that can be reversed if a problem occurs, means your risk of executing the step is reduced. If you take a step that cannot be reversed, you are operating under a higher risk. Key #4. Be conscious of interim performance issues and the impact on system availability. If you are doing your cloud migration a service at a time…module at a time…or application at a time, there will be long periods when some services, modules, and applications are in the cloud…and some are still on prem. Be very careful during these interim times. Latency of communications between components that are in the cloud and components that are still on-prem, will be significantly higher than if both components are either in the cloud, or both are on-prem. This means that, during the migration when some components have migrated and some have not, your overall system latency will be different…potentially significantly worse…than it was before the migration or after the migration. This latency may impact the usability of your system, and it may impact the availability of your system, since a significant change in latency between some components could cause unseen defects to become exposed and cause problems. During the migration, as some components have been moved, your application will act differently…and slower…than it will any other time. Plan for these differences and use these differences in your determination of which modules to move when. Key #5. Do as much refactoring before you migrate as you possibly can. While this is a corollary of many of the previous keys, it is important to stress here as well. The more work you can do to prepare your application for the cloud before you move it, the more likely you will succeed in making the migration successful and the quicker the migration will take overall. … So, in summary, the five keys to reducing your cloud migration risk are: • Limit the complexity of data migration. • Reduce the duration of all in-progress migrations. • Leave yourself backout options. • Be conscious of interim performance issues. • Do as much refactoring before you begin the migration. Keeping track of these five keys to risk reduction while you create your migration plan will help reduce your overall migration time and…most importantly…reduce the risk of a migration related failure. Tech Tapas – Services vs Microservices There is some controversy in the industry about use of the terms service and microservice. I personally do not like the term “microservice”. Why is that? It’s because the term “microservice” implies a specific sizing of a service that is not necessarily a healthy assumption. Yes, many services are small, some are truly “micro”, but many are much larger too. The appropriate size for a service is based on context and is subject to many concerns and criteria. In my mind the use of the term “microservices” biases this discussion. However, I recognize that the term microservice has gained strong popularity in the industry. There are also people that pigeon hole the use of the term “service” as part of the broader and older category of “SOA” or “Service Oriented Architectures”. They further pigeon hole the term to refer to a particular type of architecture offering that was popular a decade or more ago. I find these comparisons inaccurate and confusing, and I do not believe the reference is reasonable. This makes “service” sound old an ancient, which is not at all true. As far as which term to use… My personal preference is to use the term “service”, but I recognize many people use the term “microservice”. So, I tend to use both terms in my discussions with other companies, depending on context. In my mind, both the term service and microservice mean the exact same thing. And as such, I use the two terms interchangeably. When you hear me talk about services, microservices, service architectures, etc. I am referring to the exact same thing.…
Amazon Web Services provides a cloud certification program to encourage and enable growing your AWS cloud technical skills to help you grow your career and your business. Have you wondered what it takes to become AWS certified? In this episode, I conclude my interview with Kevin Downs, a trial by fire expert on the AWS certification program, as we discuss the AWS cloud certification program, and how to best utilize it. And then, what was the first AWS service? This is AWS Certifications, on Modern Digital Applications. Links and More Information The following are links mentioned in this episode, and links to related information: Modern Digital Applications Website ( https://mdacast.com ) Lee Atchison Articles and Presentations ( https://leeatchison.com ) Architecting for Scale, published by O’Reilly Media ( https://architectingforscale.com ) Advising and Consulting Services by Lee Atchison ( https://atchisontechnology.com ) AWS Certifications (https://aws.amazon.com/certification/) A Cloud Guru (https://acloudguru.com) Kevin Downs Twitter (https://twitter.com/kupsand) Kevin Downs LinkedIn ( https://www.linkedin.com/in/kevin-downs/ ) This episode is part 2 and final part of my interview with Kevin Downs.…
Amazon Web Services provides a cloud certification program to encourage and enable growing your AWS cloud technical skills to help you grow your career and your business. Have you wondered what it takes to become AWS certified? In this episode, join me with Kevin Downs, a trial by fire expert on the AWS certification program, while we discuss the AWS cloud certification program, and how to best utilize it. And then, what was EC2 like in the old days? Back before it was actually useful? This is AWS Certifications, on Modern Digital Applications. Links and More Information The following are links mentioned in this episode, and links to related information: Modern Digital Applications Website ( https://mdacast.com ) Lee Atchison Articles and Presentations ( https://leeatchison.com ) Architecting for Scale, published by O’Reilly Media ( https://architectingforscale.com ) Advising and Consulting Services by Lee Atchison ( https://atchisontechnology.com ) AWS Certifications (https://aws.amazon.com/certification/) A Cloud Guru (https://acloudguru.com) Kevin Downs Twitter (https://twitter.com/kupsand) Kevin Downs LinkedIn ( https://www.linkedin.com/in/kevin-downs/ ) This episode is part 1 of 2 of my interview with Kevin Downs.…
Likelihood and Severity. Two different measures for two different aspects of measuring risk in a modern digital application. They are both measures of risk, but they measure different things. What is the difference between likelihood and severity? And why does it matter? In this episode, I’ll discuss Likelihood and Severity, how they are different, and how they are both useful measures of risk in a modern digital application. Links and More Information The following are links mentioned in this episode, and links to related information: • Modern Digital Applications Website ( https://mdacast.com ) • Lee Atchison Articles and Presentations ( https://leeatchison.com ) • Architecting for Scale, published by O’Reilly Media ( https://architectingforscale.com ) • Advising and Consulting Services by Lee Atchison (https://atchisontechnology.com) • Learning Path - Risk Management ( http://leeatchison.com/classes/learning-path-risk-management/ ) • O’Reilly Learning Path Course ( https://learning.oreilly.com/learning-paths/learning-path-microservices/9781492061106/ ) Microservice architectures offer IT organizations many benefits and advantages over traditional monolithic applications. This is especially true in cloud environments where resource optimization works hand-in-hand with microservice architectures. So it’s no mystery that so many organizations are transitioning their application development strategies to a microservices mindset. But even in the realm of microservices, building and operating an application at scale can be daunting. Problems can include something as fundamental as having too few resources and time to continue developing and operating your application, to underestimating the needs of your rapidly growing customer base. At its best, failure to build for scale can be frustrating. At its worst, it can cause entire projects—even whole companies—to fail. Realistically, we know that it’s impossible to remove all risk from an application. There is no magic eight ball — no crystal ball — that allows you to see in the future and understand how decisions you make today impact your application tomorrow. Risk will always be a burden to you and your application. But, we can learn to mitigate risk. We can learn to minimize and lessen the impact of risk before problems associated with the risk negatively impact you and your applications. I’ve worked in many organizations, and have observed many more. Planning for problems is very hard and something most organizations fail to do properly. Technical debt is often a nebulous concept. Quantifying risk is the first step to understanding vulnerability. It also helps set priorities and goals. Is fixing one potential risk more important than another? How can you decide if the risks aren’t understood and quantified. In this episode, we’re going to talk about how to measure risk, so that you can build, maintain, and operate large, complex, modern applications at scale. There is a great quote by Donald Rumsfeld, twice former secretary of defense for the United States. It starts “Reports that say that something hasn’t happened are always interesting to me”. He goes on to say: “because, as we know, there are known knowns, there’re things we know we know. We also know there are known unknowns, that is to say we know there are some things we do not know.” “But there are also unknown unknowns. The ones we don’t know we don’t know. And if one looks throughout the history of our country and other free countries, it is the latter category that tend to be the difficult ones.” This is true in running a country, and a country’s military, and it is true in running a modern digital application at scale. This quote encompasses the entire meaning of risk management in one single quote. Risk management is about dealing with the unknown unknowns. You will often hear me talk about my big game example. This is the example where you invite 20 of your closest friends over to your house to watch the big game on your brand new big screen TV. Only, once the party — and the game — start, the power goes off in your home. Your big day is over, and your friends go home disappointed. Now, what would you do if you called the power company to report this outage, and their response was “What are you complaining about, you have power most of the time. In fact, we see you have power 95% of the time. Who cares if power goes out the other 5% of the time?” Who cares indeed. The reality is the power company can’t operate in this way. They cannot be satisfied with “good enough” service. They have to strive to provide power to you 100% of the time. 24 hours a day. 7 days a week. They have to strive for perfection. This difference from perfection. This extra 5%, it’s driven by the expected actions and problems that we see. It’s driven by the unknowns. It’s driven by the things we don’t even know that we don’t know. It’s driven by the unknown unknowns. Preparing for these unknown unknowns, is what risk management is all about. Risk, like anything else, can be quantified. There are two fundamental metrics that matter most when quantifying risk. Likelihood…and severity. Likelihood is the measure of the chance of a particular risk triggering . Or, put another way, it’s the measure of the chance of a particular risk occurring . We say “what’s the likelihood of our pipes freezing tonight?” Or “what’s the likelihood of us getting rain tomorrow" Or “what’s the likelihood of a tornado hitting our house?” Likelihood measures the possibility of an event happening. The likelihood of you getting rain tomorrow, for instance, is most definitely significantly higher than the likelihood of your house getting hit by a tornado tomorrow. That’s likelihood. Severity is the measure of the cost of a risk that triggers. If a risk occurs, what really happens and how severe are the ramifications? Using the examples above, the severity of rain hitting you on the head is pretty low. The severity of your pipes freezing is greater. But the severity of a tornado hitting your house — well, that is severe — the impact of each of those three things happening is different, and that difference is measured by severity. It’s important to keep these two things distinct and understand the difference between them. Likelihood is IF an event will occur. Severity is WHAT is the cost of the event occurring. The chance of rain tomorrow might be high (likelihood) but it doesn’t hurt you too much if it does (severity). The chance of your house getting hit by a tornado is very low (likelihood) but the impact of that event would be catastrophic for you (severity). These two measures work together to quantify the risk of a particular event. These two measures together are what we use to track and measure risk…whether that risk is a weather related risk, or a risk of an application failure in your business systems. In my book, architecting for scale, I give an example of risk measurement in a modern application by utilizing a T-Shirt e-commerce store example. We can measure the risk of a failure of components of this application using likelihood and severity. For example… The e-commerce store probably has a top ten list component — a service that generates a top ten list of products sold through the site. What’s the risk of the top ten list not appearing? The likelihood of the list not appearing is probably relatively low — it’s a simple component without a lot of complexity to it. Likewise, the severity of the problem of the top ten list not appearing is also low. If customer’s can’t see the top ten list, it doesn’t significantly impact their buying experience. This would be a low likelihood, low severity problem. In shorthand, it’d be a low/low risk. But what about the order database? What’s the risk if the order database stops accepting new orders? Well, once again, the likelihood of that happening is probably relatively low. It’s an important subsystem that we’ll assume is probably well maintained. But if it does happen, the severity of that problem is quite high. If you can’t accept orders, your entire business suffers. This would be a low likelihood, high severity problem. Shorthand, a low/high risk. Moving on, let’s say your store uses a custom font to make the display more visually pleasing. What’s the risk of the font not loading in a user’s browser? Well, the likelihood of this happening might in fact be quite high. You can imagine scenarios where a user’s browser has a poor internet connection and the font file doesn’t load correctly. Or maybe you are using a 3rd party font service to provide the fonts dynamically. The likelihood of this problem occurring could actually be quite high. But what about the severity? Here the severity is probably quite low. If the custom font doesn’t load, the browser will just substitute a different font for the page. The page will still work, it just might not look quite as visually appealing as you desire it to be. This would be a high likelihood, low severity problem. Shorthand, a high/low risk. Finally, let’s take a look at the t-shirt photos that appear in the store. These are the pictures of products that customer’s might buy. What’s the risk of the photos not appearing on a page? Well, the likelihood of this risk could be high, because showing photos on a page means loading them from a cache server or maybe a 3rd party CDN, and this system might not be working quite right. The photos may not be available, or the user’s internet connection could flake out and not show them. The likelihood of this problem occurring is, in this example, high. What about severity? Well, it’s hard to imagine that a customer would buy a t-shirt that they could not see a photograph of, so if the photos aren’t appearing, that could have a big impact on your business since people would buy fewer t-shirts. The severity of this problem is also high. This would be a high likelihood, high severity problem. In shorthand, this would be a high/high risk. These are four examples of problems that might occur in an e-commerce store, and the risk measurement associated with them happening. Now that we can measure the risk, we can use that measurement to prioritize work to mitigate or remove those risks. We can imagine that mitigating or removing a high/high risk, would be more critical than a high/low or low/high risk, and all of them would be more important than working on a low/low risk. We can properly determine which risks are most helpful for us to work on, and we can measure the impact of our work to mitigate those risks. In future episodes, I will continue the topic of risk management and discuss tools and techniques for monitoring, reporting, and mitigating risk in our applications with the ultimate goal of reducing the impact that risk has on our availability of our applications.…
Building a scalable application that has high availability is not easy. Problems can crop up in unexpected ways that can cause your application to stop working and stop serving your customer’s needs. No one can anticipate where problems will come from and no amount of testing will identify and correct all issues. Some issues end up being systemic problems that require the correlation of multiple systems in order for the problems to occur. Some are more basic, but are simply missed or not anticipated. Links and More Information The following are links mentioned in this episode, and links to related information: Modern Digital Applications Website ( https://mdacast.com ) Lee Atchison Articles and Presentations ( https://leeatchison.com ) Architecting for Scale, published by O’Reilly Media ( https://architectingforscale.com ) Application availability is critical to all modern digital applications. But how do you avoid availability problems? You can do so by avoiding those traps that cause poor availability. There are five main causes of poor availability that impact modern digital applications. Poor Availability Cause Number 1 Often, the main driver of application failure is success. The more successful your company is, the more traffic your application will receive. The more traffic it receives, the more likely you will run out of some vital resource that your application requires. Typically, resource exhaustion doesn’t happen all at once. Running low on a critical resource can cause your application to begin to slow down, backlogging requests. Backlogged requests generate more traffic, and ultimately a domino effect drives your application to fail. But even if it doesn’t fail completely, it can slow down enough that your customers leave. Shopping carts are abandoned, purchases are left uncompleted. Potential customers go elsewhere to find what they are looking for. Increasing the number of users using your system or increase the amount of data these consumers are using in your system, and your application may fall victim to resource exhaustion. Resource exhaustion can result in a slower and unresponsive application. Poor Availability Cause Number 2 When traffic increases, sometimes assumptions you’ve made in your code on how your application can scale are proven to be incorrect. You need to make adjustments and optimizations on the fly in order to resolve or work around your assumptions in order to keep your system performant. You need to change your assumptions on what is critical and what is not. The realization that you need to make these changes usually comes at an inopportune time. They come when your application is experiencing high traffic and the shortcomings start becoming exposed. This means you need a quick fix to keep things operating. Quick fixes can be dangerous. You don’t have time to architect, design, prioritize, and schedule the work. You can’t think through to make sure this change is the right long term change You need to make changes now to keep your application afloat. These changes, implemented quickly and at the last minute with little or no forethought or planning, are a common cause of problems. Untested and limited tested fixes, quickly thought through fixes, bad deployments caused my skipping important steps. All of these things can introduce defects into your production environment. The fact that you need to make changes to maintain availability, will itself threaten your availability. Poor Availability Cause Number 3 When an application becomes popular, your business needs usually demand that your application expand and add additional features and capabilities. Success drives larger and more complex needs. These increased needs make your application more complicated and requires more developers to manage all of the moving parts. Whether these additional developers are working on new features, updated features, bug fixes or other general maintenance, the more individuals that are working on the application, the more moving parts that exist, the greater the chance of a problem occurring that brings your application down. The more your application is enhanced, the more likely there is for an availability problem to occur. Poor Availability Cause Number 4 Highly successful applications usually aren’t islands unto themselves. Highly successful applications often interact with other applications, either applications that are part of your application suite, or third party applications. Third party applications can be provided by vendors or partners. They can be external SaaS services. Or, they can be integrations with customer systems. The more dependencies you have, the more exposed you are to problems introduced by those other external systems. Your availability will ultimately become tied to the availability and quality of those external applications. The more dependencies you have, the more fragile your application becomes. Poor Availability Cause Number 5 As your application grows in complexity, the amount of technical debt your application has naturally increases. Technical debt is the accumulation of desired software changes and pending bug fixes that typically build up over time as an application grows and matures. Technical debt, as it builds up, increases the likelihood of a problem occurring. The more technical debt you have, the greater the likelihood of an availability problem. Conclusion All fast-growing applications have one or more of these problems. These problems are the sort of problems that increase the risk of having a problem with availability. Potential availability problems can begin occurring in applications that previously performed flawlessly. The problems can quietly creep up on you, or the problems may start suddenly without warning. But most applications, growing or not, will eventually have availability problems. Availability problems cost you money, they cost your customer’s money, and they cost you your customer’s trust and loyalty. Your company cannot survive for long if you constantly have availability problems. Focusing on these five causes will go a long ways to improving the availability of your applications and systems. Tech Tapas — Database backup test failure I want to tell you a story. You tell me if this is ok or not. This was from a conversation I had heard in a company I was working with. The conversation was a message from one engineer to their peers, They were trying to update them on the situation of a production database. The message went like this: “We were wondering how changing a setting on our MySQL database might impact our performance…” “…but we were worried that the change might cause our production database to fail.” “Since we didn’t want to bring down production, we decided to make the change to the replica database instead…the backup database…” “After all, it wasn’t being used for anything at the moment.” Of course, you can imagine what happened next, and you would be right. The production database had a hardware failure, and the system automatically tried to switch over to use the replica database. But the replica database was in an inconsistent state due to the experimentation that was going on with it. As such, the replica database was not able to take on the job as the new master…it quickly became overwhelmed…and then it failed as well. Both the original master, and the replica failed. The replica, who’s sole purpose for existence was to take over in case the master failed, wasn’t able to do so because it was being tinkered on by other engineers. Those other engineers didn’t understand that, just because the replica wasn’t actively servicing production traffic, that doesn’t mean it wasn’t being used. It’s entire job was to sit in wait to take over if necessary. By experimenting on that replica database, they were inadvertently impacting production. They were introducing risk into the production system — risk that wasn’t appropriate. Risk that could — and in this case did — cause serious problems. This, by the way, was a true story. But it also is not an uncommon story. I hear similar sorts of problems occur in many engineering conversations, and many operations management conversations. Not having a clear understanding or appreciation for how certain actions impact the risk management plans for a production system can be disastrous. This is why active and continuous risk management planning is critical for production networks to stay operational.…
We often hear that being able to scale your application is important. But why is it important? Why do we need to be able to suddenly, and without notice, scale our application to handle double, triple, or even ten times the load it is currently experiencing? Why is scaling important? In this episode, I am going to talk about four basic reasons. Four reasons why scaling is important to the success of your business. And then, what is the dynamic cloud? This is Application Scaling, on Modern Digital Applications. Links and More Information The following are links mentioned in this episode, and links to related information: Modern Digital Applications Website ( https://mdacast.com ) Lee Atchison Articles and Presentations ( https://leeatchison.com ) Architecting for Scale, published by O’Reilly Media ( https://architectingforscale.com ) Why you must scale We often hear that being able to scale your application is important. But why is it important? Why do we need to be able to suddenly, and without notice, scale our application to handle double, triple, or even ten times the load it is currently experiencing? Why is scaling important? There are many reasons why our applications must scale. A growing business need is certainly one important reason. But there are other reasons why architecting your application so it can scale is important for your business. I am going to talk about four basic reasons. Four reasons why scaling is important to the success of your business. Reason #1. Support your growing business This is the first, and the most basic reason why your application has to scale. As your business grows, your application needs grow. But there is more to it than that. There are three aspects of a growing business that impact your application and require it to scale. First, is the most obvious. As you get more customers, your customer’s make more use of your applications and they need more access to your website. This requires more capacity and more growth for the IT infrastructure for your sites. But that’s not the only aspect. As your application itself grows and matures, typically you will add more and more features and capabilities to the application. Each new feature and each new capability means customers will make more use of your application. As each customer uses more of your application, the application itself has to scale. Simply by your business maturing over time, even if the size of your customer base doesn’t grow, the computation needs for your application grow and your application must scale. And finally, as your business grows and matures, and your application grows and matures, your more complex application will require more engineers to work on the application simultaneously, and they will work on more complex components. Your application might be rearchitected to be service based. It might add additional external dependencies and provisions. You will have to support more deployments and more updates. Your application and your application infrastructure will need to scale to support larger development teams and larger projects. This means you need more mature processes and procedures to scale the speed at which your larger team can improve your application. Reason #2. Handle surprise situations The second reason you need to be able to scale your application is to handle surprise situations and conditions. All businesses have their biggest days. These are the days where traffic is at the heaviest. These are days like Black Friday in retail, or the day of the Super Bowl for companies that advertise during that event, or open enrollment periods, or start of travel season. But your business may have unexpected business bumps. These are the traffic increases that occur not because of a known big event, but because of an unknown or unexpected event. When an event occurs that is favorable to your business, you need to be ready to handle the increased load that occurs to support the event. If you cannot handle the increased load, you risk losing the new business, and you risk disappointing your existing customers. Sudden business success can kill you if you can’t scale to meet the need. Just ask Robinhood Financial. Robinhood Financial is an investment company that provides investment management services. On Monday, March 2nd, 2020, Robinhood faced a business crisis. They faced a sudden increase in business. On that day, the United States stock market had a record-breaking day. This record-breaking day resulted in a record number of account signups and customer market transactions. This is good news for a company such as Robinhood. The problem was that their traffic load was not only high, it was too high. They needed to be able to respond to a huge spike in traffic to their application. Unfortunately, they were unable to keep up with the sudden demand. The result was a failure of their systems…and their application. The Robinhood Financial site was down…for a day and a half. This was during a peak stock market time, a time when their customer’s needed them the most. As a result, they lost out on a huge amount of easy, new business; and they created hardship and disappointment for many of their existing customers. Potential new customers and existing customers alike, were disappointed. A potential opportunity for huge growth and huge upside for the company…instead turned into a major negative event for the company. An event their founders had to publicly apologize for. All because they couldn’t scale to handle the surprise traffic load. To be successful, companies must be able to scale to meet sudden and unexpected traffic demands. Reason #3. Handle a partial outage The third reason is a sneaky one. You need to be able scale in order to handle partial application outages. Partial outages can be a big problem for businesses. You have a large application, distributed across the globe in multiple data centers.or availability zones, if you are operating in the cloud. You spread your application out like this for improved redundancy, availability, accessibility, and resiliency. But now, one of your data centers goes down. Of course, since you are operating in more data centers, a single data center outage is far more likely. This means more chances for something to go wrong in any one of them. But when a data center goes down, the traffic that would normally be sent to that data center has to be re-routed to other data centers. This results in a big uptick in traffic to those other data centers. Can those data centers handle the increased traffic? If not, those data centers could go down as well. The result is your application can fail and become unavailable, due to excessive traffic. This seems counter intuitive, but your plan to increase availability, just made your application less available. Your plan for improved redundancy by increasing the number of data centers, actually made your application more fragile. By increasing the number of data centers you were using, you increased the risk of a data center failure. And your application isn’t able to scale to handle the increased traffic needs of a data center failure. The result is an application melt down. A step to improve availability makes availability worse. Can your other data centers accept the sudden challenge of handling the additional traffic that is sent to them from a failed data center? Can you respond to this sudden need for scale? You must, or your application is at risk. Reason #4. Maintain availability The fourth reason is to maintain availability. As your application gets more complex, it requires more interactions between many different components to work correctly. If one of those components begins to act sluggishly, it can cause performance issues in downstream services. These downstream performance issues can become worse, and more critical problems can occur, such as transaction timeouts, corruption, data loss, and ultimately, upset customers. A single service, slowing down for some simple reason, can cascade into a larger problem. And if your application can’t scale, the likelihood of individual components saturating and slowing down becomes a matter of when it will happen, not if if will happen. Lack of scalability turns into lack of availability. Lack of availability turns into failed customer expectations. Failed customer expectations turns into a negative impact on your business. Scaling is Critical Scaling is critical to your business success. Whether your business is growing or not, you need to be able to handle the growing and spiky traffic needs of your customers…at anytime…or risk application failure, upset customers, and a business failure. Scaling isn’t just important, it is a business necessity. Tech Tapas — Dynamic Cloud There are two ways that people utilize the cloud...the first is by taking an application that is designed to run anywhere, and run it in an infrastructure that was created in the cloud. This is typically called the static cloud, because you create resources, such as servers, that are long lived and use them to operate the application. The resource usage typically does not change much — or at all — over the long term as the application runs. The other way is to only allocate the resources you absolutely need, when you need them. Given that it is very easy to allocate and free resources in a cloud — especially a public cloud — it’s relatively easy to build an application that allocates the resources it requires when it requires them, and frees the resources when they are no longer required. This is called the dynamic cloud. The dynamic cloud is where the true power of cloud computing exists, and where the true benefits of using the cloud can be unlocked. The ability to only consume the resources you absolutely require at the moment, coupled with the ability to quickly allocate the additional resources you require as your application needs increase — gives you incredible capabilities in building highly scalable applications that can meet your needs no matter the amount of traffic sent to them, yet conserve money when traffic is low. When you perform a lift-n-shift migration of an application to the cloud, you typically move the application from a static data center to operating in a static cloud. You typically do not take advantage of the dynamic capabilities of cloud computing. Too often such application migrations end up being disappointments, because the application does not run any better in the cloud than it did in your own data center, yet the cloud resources may end up actually costing more money when used statically than equivalent resources in a static data center. The only way to truly see the advantage of using cloud computing is to utilize the dynamic cloud to build dynamic applications. Then you only consume — and pay for — the resources you require at the time you require them, yet you can increase the resources available to your application very quickly to handle sudden increases in traffic. Whether you are doing dynamic auto scaling, or using dynamic services such as Amazon DynamoDB, Google Big Table, AWS Lambda, or Azure Functions, using the cloud in a dynamic fashion — using the dynamic cloud — is the key to effectively utilizing the cloud to improve, and hence modernize, your application.…
M
Modern Digital Applications with Lee Atchison
Modern Digital Applications is a podcast focused on helping corporate decision makers, executives, and architects create or extend their digital business with the help of modern applications, processes, and software strategy. The podcast is hosted by Lee Atchison , a recognized industry thought leader and pundit in cloud computing. Lee has over 30 years industry experience, and has been working on cloud computing since the early days of the cloud. Lee has committed his career to architecting and building high scale, cloud-based, service oriented, SaaS applications. He has a specific expertise in building highly available systems. Lee has consulted with leading organizations on how to modernize their application architectures and transform their organizations at scale; including optimize for cloud platforms, utilize service-based architectures, implement DevOps practices, and design for high availability. This experience led him to write his book Architecting for Scale, published by O’Reilly Media . Lee is an industry expert and pundit and is widely quoted in publications such as InfoWorld, Diginomica, IT Brief, Programmable Web, and CIO Review. He has been a featured and keynote speaker at events across the globe from London to Sydney, Tokyo to Paris, and all over North America. In each episode, Lee will give you insights into some aspect of building, modernizing, or scaling your digital applications, and the teams and culture that drive them. The podcast episodes will contain feature stories, new and noteworthy industry information, and some Tech Tapas, little bits of technology knowledge and data. We’ll also feature interviews with technology experts and other industry pundits. Each episode is designed to be short and concise, easily digestible on a single morning commute trip. This is ... Modern Digital Applications. Links and More Information Lee Atchison Articles and Presentations Architecting for Scale — O’Reilly Media Modern Digital Applications Podcast…
M
Modern Digital Applications with Lee Atchison
Ken Gavranovic was the Executive Vice President and GM for product at New Relic. In early 2019, Ken and I were in Boston together for an event, and we recorded an interview discussion about Risk Management in modern digital applications. Both Ken and I have experience dealing with Risk Management issues in current and past assignments. I discuss Risk Management in my book, Architecting for Scale. Ken used a very similar risk management technique in his past corporate management gigs. In this interview, we compare notes and make recommendations on best practices for Risk Management that everyone can use. Links and More Information The following are links mentioned in this episode, and links to related information: Modern Digital Applications Website ( https://mdacast.com ) Lee Atchison Articles and Presentations ( https://leeatchison.com ) Architecting for Scale, published by O’Reilly Media ( https://architectingforscale.com ) Risk Management with Ken Gavranovic Video ( https://leeatchison.com/2019/02/06/managing-risk-in-modern-enterprise-applications/ ) Ken Gavranovic Twitter ( https://twitter.com/kgavranovic ) Ken Gavranovic LinkedIn ( https://www.linkedin.com/in/gavranovic/ ) Risk Management Interview Ken: I know we both talk to a lot of customers. One of the questions is, where do I get started? What are some of the patterns we see in enterprises and our own experiences? We have an awesome opportunity to talk to a lot of companies doing digital transformation, but what is something that I can just go do tomorrow to get started? Lee: One of the things I find it’s very easy to wrap your mind around is risk management. How do you build a risk matrix to track the issues and the risks you have within your system? I like to talk to companies about that because it gets people starting to think about what their system is doing, what problems they have, and how they deal with them. It gets them thinking beyond just the problem/resolution cycle, and more into a pro/con and risk assessment process. What is the benefit of fixing something versus the benefit of mitigating it versus the benefit of simply ignoring it? I like to talk about that because it gets conversations going within the company about the sorts of things that are important to them. Creating a risk matrix is an important first step for anyone who is thinking about trying to improve their availability, trying to improve their scalability, or trying to modernize their application in many different ways. It helps get a grip on the issues that already exist in your system and what you are currently doing to manage those risks. Ken: I 100% agree. I remember in a previous role, I had a couple hundred-million-dollar project, I had some challenges. We created a risk matrix which helped us solve those challenges. So I thought it might be helpful for people watching this video. Let’s double click and see what this might look like. From my perspective, I think the key questions that need to be asked, those questions need to be asked in a bottoms-up way, not top down. Agreed? Lee: Yes, definitely. Ken: It’s not people at the top of the organization that are giving you the answers. It’s the team level that gives you the answers you need. Let me give you my shot and tell me where I miss. First of all, the things that can go into the risk are the things that can go bump in the night. Lee: Most people already have an idea of the things that keep them up at night. Things they think about, worry about. The things they think about on a regular basis, and that is a good place to start. Ken: That makes sense. So, bottom up, by team, just create a list. Just list all the things that we think are some sort of risk to the project. These are things you know you should be resolving, but instead you have a habit of prioritizing feature development work over it instead. Next, is to think about the likelihood that this risk will actually happen. Lee: I tell people they need to think about two values for every risk item they come up with. Create a spreadsheet, and list all the risks as rows in the spreadsheet. Each individual risk, line by line. Then, for each risk, add two values in separate columns: likelihood and severity. That is, how likely is this risk to happen and if it does happen, how much negative impact will I have with it. They should do this for every risk in the matrix, before they even begin to think about fixing or mitigation. Ken: I think it’s important to share that this is what we’ve seen, not just from personal experience, but from a lot of companies that we work with. Lee: Right. Ken: What types of values should I use for likelihood and severity? Some people say I should score it from 1 to 10. I think that’s too granular. I like to keep it simple. Just use: Low, Medium, and High. Lee: I agree with you. You do run into people that want to be highly analytic. They want to use numbers, say, from 1 to 100 and they end up arguing about whether a particular risk is a 35 or a 36. This is way too granular. Keep it simple. Ken: Sometimes teams like to use their SPRINT approaches of throwing numbers, such as using cards. Lee: Yeah, if you really want a more rigorous process, you can do something similar to the SPRINT throwing numbers approach, but just use three playing cards, say Ace, Five, and Ten. Then everyone can vote with a card and use that to determine High/Medium/Low. But that sort of process is only for people that really want a truly analytic solution. It can be done much simpler than that. Often, items are clear to everyone that they are a high or a low or somewhere in between. Ken: So, whether you use cards, or just use Low/Medium/High, or whatever. At the end of the day, the most important thing is to keep it simple. It’s not about a big debate. Lee: Exactly. Ken: At this stage, we are not trying to get into a great level of detail. Just a high-level description, likelihood, severity. Next thing for the matrix is, is this risk currently instrumented? Does it have observability? If this risk were to occur, would you know that it is occurring from a notification from an automated system, or would you find out from your customers telling you? Lee: That’s a fantastic way to think about it. It’s one thing to know that if something goes wrong, what’s going to happen. It’s another thing to know that you’ll know when it happens. Ken:** Agreed. Lee: And, certainly when we talk later about mitigation, you absolutely need to know that knowing when a risk is occurring is a critical aspect of risk management. This is especially true for your high severity risks, whether or not they are high likelihood or low likelihood. Ken: Kicking starting a program like this in an enterprise is obviously hard. You need top-down leadership to support this process that we are going to do. Lee: Yes. Ken: Risk matrix, containing lines with items, likelihood, severity, monitored or not monitored. Ok, what else, or is it just that simple? Lee: Well, coming up with that list is going to get you 80% of the way to what you need. That’s because it gets you and your organization thinking about what’s going on. That’s the most important benefit of this process. You start thinking about risk and the impact risk has on your system. What’s going to happen during this risk discovery process is the engineers in the room, their minds are quickly going to go to the next thing, which is mitigation. They are going to start to think about how to handle the risk. But, you are right. If you get nothing done but create that list of risks and put them in the matrix during the first meeting or two, that’s all you need and your world will be a whole lot better, just by simply having that matrix. Ken: Right. Another point I want to throw out there and see if you agree, is around RCA and incident response processes. I think when you have an incident, during the RCA you should always check if this issue was already on the risk matrix. If it was not there, then it should be added, and some time should be spent on why it wasn’t added in the first place. Maybe a team wasn’t as aggressive, and they didn’t want to put everything in the matrix. Because, going back to no surprises, you want to understand why this incident was a surprise. One of my favorite phrases is, “surprises, not a fan of giving or receiving”. If you have a risk matrix and it’s done right, anything that goes bump in the night should have been known and on the risk matrix ahead of time. Lee: Exactly, you know, every time you have an outage or an incident of any sort, you end up with some sort of post mortem whether it’s formalized or not. One of the key questions has to be, “did you know about this ahead of time?”, and that comes back to the risk matrix. Because, if you didn’t know about it, that’s a problem. It needs to be added to the risk matrix, so you understand that risk fully. But if you did know about it, you should also verify that the actual severity of the incident matches the severity you thought it should be on the risk matrix. Were you right or wrong in your estimates? You can gain a lot of knowledge when an incident occurs by answering questions like this with the help of a risk matrix. Ken: So, let’s assume that as a leader, I’ve told my organization to build a risk matrix. They’ve done the process, I now have this risk matrix. From an execution point of view, I think there are two things that need to happen next. First, you look at the high/highs – high likelihood, high severity. In some cases, removing these risks might involve rewriting. But the high/highs that you can fix, you should prioritize the work and get them fixed. Second, you always have business partners. I’m a big believer that you should take that risk matrix and present it, at the executive level, to your business partners. You show the high/highs, the medium/mediums, or whatever they are. Now, as a company, think about one of two things. Should we focus on fixing these high/highs or should we all take a breath and say we are willing, for whatever reason, to take this risk on as a company. You go into that with open eyes, blameless culture, and state your willingness to take that risk together. Lee: Yes, and that’s really critical too. Because no matter what, you are not going to remove all the risk from the system. You aren’t going to fix all the problems, nor is trying to do that necessarily the right investment for you. The right level of risk is whatever level your organization – your extended organization – is comfortable with. The business cost of the risk, the development cost of fixing it, all of these things have to fit together. But once you know what your risk is, you can evaluate whether you and the culture of your company, and your customers, and the business you provide, are comfortable with that level of risk. Now, for the things that you are not comfortable with, you have to address these right away. You have to either mitigate these risks or remove them. But the other risks, the ones where you are comfortable with the level of risk, it’s not necessarily a good investment to work on resolving those things. Because there are going to be higher priority issues you want to work on. Ken: Another important aspect is from the funding perspective. I look at the risk matrix as a living document. My thoughts are, you should run this exercise at least twice a year. Then, when you have incidents, you should update the risk matrix to match those incidents. The risk matrix should be accurate and maintained. Lee: Absolutely. Ken: Now, most companies fund on an annual basis. My perspective is a lot of times people forget about risk when it comes to funding. In some companies, what is funded are the “bright shiny objects”. That’s where the money is invested. So, for companies that are technology leaders, you should bring the risk matrix to the budgeting discussions. That way you can make sure everybody is clear and all discussions are open on what we are investing in and why we are investing in it. The risk matrix is part of the budgeting process. Lee: Yes, it’s definitely a feedback into your budgeting process. But it’s also at a much lower level a feedback back into your SPRINT planning process. Ken: Totally agree. Lee: You use it to determine what you can accomplish this SPRINT, and how much you want to spend on doing risk management activity during this specific SPRINT versus doing new features or dealing with other problems. Ken: I know many enterprises, if they are really focused on the customer experience say High/Highs must be done first, unless it involves a full rewrite. If you go into an organization that has a lot of technical debt, that may not be the case, you do as many as you can each SPRINT. Lee: Yes, absolutely. But the one important thing to consider is that fixing does not have to mean removing it. It might be creating a mitigation for it that reduces its severity or likelihood to an acceptable level. Ken: It might move from a High likelihood to a medium or might take it from a High impact to a medium or low. Lee: And just by doing that you’ve brought it to down to within the comfort level of your organization. And once it’s in the comfort level of the organization, that’s a very successful place to be. Ken: And you and I have seen this at hundreds of global companies. So, from a best practices’ standpoint, it really makes a lot of sense. Have a risk matrix, update it semi-annually and when incidents happen. Review it during the RCA process. Rinse and repeat. Then, take what you have and use that in the budgeting process. Anything else we should add? Lee: The only additional thing is that individual teams need to own their own risk matrices – remember, they are built bottom up. Individual teams need to have responsibility for their own risk matrices and be held accountable for the content. Then, they all need to bubble up to a high-level list that is known at the highest levels of the organization. Ken: I agree. And the initiative and guidance to do it needs to come top-down, because it’s important to the entire organization. Lee: Yes. Ken: The actual work itself happens bottoms-up. Totally agree. I’d like to thank Ken for his involvement in this interview and for the insights he provided to the very important topic of Risk Management. Additionally, thank you to Ken for providing the recording equipment he and I used for the interview. Tech Tapas — History of the term Cloud What is the history of the term “cloud” as it is used in Cloud Computing? Well, that’s an interesting question and unfortunately there are probably as many answers as there are people who work in the cloud. So, it’s very hard to answer. But most people who have looked into this subject believe the term originally came in the 1980’s from the telephone companies of the day. Network engineers who drew diagrams of portions of their networks would often draw a big blob to indicate a portion of their network that they weren’t dealing with at the time. Rather than drawing these blobs as simple circles, squares, or rectangles, since those icons represented real entities in their diagrams, instead they drew the blob using rounded segments interconnected that made it look “nebulous” and nondescript in shape…it was, after all, suppose to represent a nebulous and nondescript part of the network. In fact, this nebulous and nondescript shape looked like a cloud. So much so that these engineers started talking about the part of the network that they weren’t focusing on at the time — the part external to their area of concern — as being out in the cloud. This usage expanded into software architects as they build their software diagrams and flow charts as well. They used the symbol for a similar purpose. The term cloud computing, though, is much more recent. Some would argue that early server farms were really cloud computing, but the term really wasn’t popularly used back then. Some would say that Google Compute Platform (GCP) provided some of the earliest cloud computing technology to the industry. Others would say that Salesforce.com was a major early creator of cloud technology. But I think the real mainstream usage of the term cloud computing among technical professionals was popularized with the start of Amazon Web Services. AWS mainstreamed cloud computing — and hence mainstreamed the term cloud computing. This happened in the mid 2000’s. But software running in the cloud was still something that was reserved for technical people to talk about. Mainstream non-techy people didn’t yet know what the term cloud was all about. This was certainly my personal experience. I was working at AWS in the early days, and I had a hard time telling my non-techy friends what I did for a living. They just didn’t know what I meant by “cloud computing”. They didn’t know what I meant when I said I worked “in the cloud”. They didn’t understand what the cloud was. Like many of the changes in modern popular tech culture, that changed once Apple computer came into the picture. On October 12, 2011, Apple introduced iCloud to the Apple universe, and overnight, the word “cloud” was a part of mainstream culture. Those friends of mine who couldn’t understand what I was doing when I said I “worked in the cloud”, now understood, at least at some level, what the cloud was all about. Apple brought the term cloud to the mainstream. I know I will likely get many people who disagree with my analysis on who invented the term cloud and who popularized its use. That’s because there really isn’t a single right answer. But I do believe that the biggest events in the history of the term “cloud computing”, were — first the network engineers, then AWS, then Apple computer. Each of those three groups played a role in bringing the world “cloud” into our everyday lives.…
M
Modern Digital Applications with Lee Atchison
1 How to Improve Application Availability, p2 14:43
14:43 Play Later Play Later Lists Like Liked14:43
Play Later Play Later Lists Like LikedModern applications require high availability. Our customers expect it, our customers demand it. But building a modern scalable application that has high availability is not easy and does not happen automatically. Problems happen. And when problems happen, availability suffers. Sometimes availability problems come from the simplest of places, but sometimes they can be highly complex. In this episode, we will continue our discussion from last week with the remainder of the five strategies for keeping your modern application, highly available as well. This is How to Improve Application Availability, on Modern Digital Applications. Links and More Information The following are links mentioned in this episode, and links to related information: Modern Digital Applications Website ( https://mdacast.com ) Lee Atchison Articles and Presentations ( https://leeatchison.com ) Architecting for Scale, published by O’Reilly Media ( https://architectingforscale.com ) Robinhood Announcement ( https://blog.robinhood.com/news/2020/3/3/an-update-from-robinhoods-founders ) How to Improve Availability, Part 2 Building a scalable application that has high availability is not easy and does not come automatically. Problems can crop up in unexpected ways that can cause your application to stop working for some or all of your customers. No one can anticipate where problems will come from, and no amount of testing will find all issues. Many of these are systemic problems, not merely code problems. To find these availability problems, we need to step back and take a systemic look at our applications and how they works. What follows are five things you can and should focus on when building a system to make sure that, as its use scales upwards, availability remains high. In part 1 of this series, we discussed two of these focuses. The first was building with failure in mind. The second was always think about scaling. In part 2 of this series, we conclude with the remaining three focuses. Number 3 - Mitigate risk Keeping a system highly available requires removing risk from the system. When a system fails, often the cause of the failure could have been identified as a risk before the failure actually occurred. Identifying risk is a key method of increasing availability. All systems have risk in them. There is risk that: A server will crash A database will become corrupted A returned answer will be incorrect A network connection will fail A newly deployed piece of software will fail Keeping a system available requires removing risk. But as systems become more and more complicated, this becomes less and less possible. Keeping a large system available is more about managing what your risk is, how much risk is acceptable, and what you can do to mitigate that risk. This is Risk management, and it is at the heart of building highly available systems. Part of risk management is risk mitigation. Risk mitigation is knowing what to do when a problem occurs in order to reduce the impact of the problem as much as possible. Mitigation is about making sure your application works as best and as completely as possible, even when services and resources fail. Risk mitigation requires thinking about the things that can go wrong, and putting a plan together now, to be able to handle the situation when it does happen. For example, consider a typical online e-commerce store. Being able to search for product on the e-commerce store is critical to almost any online store. But what happens if search breaks? To prepare for this, you need to have “Failed Search Engine” listed as a risk in your application risk plan. And in that risk, you need to specify a mitigation plan to execute if that risk ever triggers. For example, we might know from history that 60 percent of people who search our site end up looking at and buying our famous red striped shirt. So, if our search service stops functioning, rather than simply failing, we could display an appropriate “I’m Sorry” page, followed by a list of our most popular T-Shirts, including our red striped shirts. For some number of customers, this would be a success. For the rest, it might create alternatives for them other than simply leaving in frustration. Combine this I’m sorry page with showing a coupon for 10% off their next visit, and you’ve turned a bad customer experience into an experience that might just create some return customers. This is an example of a risk mitigation plan. It’s a plan that you build in advance of a potential but serious problem, and be able to implement it if that problem occurs. This is a great example of risk mitigation. Other risk mitigation plans might be entirely technical. They might involve failover servers, or rapid response plans to resolve an issue. Whatever the plan is, risk mitigation is the process of creating and putting these plans into place. Number 4 - Monitor availability You can’t know if there is a problem in your application unless you can see the problem. Make sure your application is properly instrumented so that you can see how the application is performing. Proper monitoring depends on the specifics of your application and needs, but usually entails some of the following capabilities some or all of the following: Server monitoring. Monitoring the health of the server infrastructure that your application is running on. This might be physical resources, or cloud-based virtual resources. Configuration change monitoring. Understanding when and how your system infrastructure changes and how those changes impact the operation of your application. Application performance monitoring. Look inside your application and services to make sure they are operating the way you expect them to operate. Synthetic testing. Monitor how your application works from an external perspective in order to catch problems as customers may see them before they actually see them. Monitoring involves improving two aspects of modern application operation. MTTD and MTTR. That’s mean time to detection and mean time to resolution. Looking at key performance indicators for changes in patterns and alerting you of those changes will improve your mean time to detection. Giving you a wealth of data that can be used to diagnose the source of a problem will improve your mean time to resolution. Both of these are important measures for improving application availability. Number 5 - Respond to issues in a predictable and well defined manner Monitoring systems are useless unless you are prepared to act on the issues that arise. This means being alerted when problems occur so that you can take action. Additionally, you should establish processes and procedures that your team can follow to help diagnose issues and easily fix common failure scenarios. For example, if a service becomes unresponsive, you might have a set of remedies to try to make the service responsive. This might include tasks such as running a test to help diagnose where the problem is, restarting a daemon that is known to cause the service to become unresponsive, or rebooting a server if all else fails. Having standard processes in place for handling common failure scenarios will decrease the amount of time your system is unavailable. It will help with improving Mean Time to Resolution. Additionally, they can provide useful followup diagnosis information to your engineering teams to help them deduce the root cause of common ailments, in order to reduce the likelihood of a reoccurrence of a problem, or the occurrence of a similar problem. When an alert is triggered indicating that a service is or might be failing, the owners of that service must of course be alerted so they can deal with the issue in a timely manner. However, other teams that are closely connected to the problem service may also want to be alerted. If you own a service that depends on the failing service, you might want to be informed of the problem even before it impacts your service, so that you could take preventative measures or institute actions from your risk management plan before they become critical. Additionally, if the failing service is a consumer of your service, you may want to be aware of the fact that traffic patterns from the failing service may change as the problem occurs and is being resolved. You may want to keep a close eye on your service to make sure any changes do not negatively impact you. Documented processes and operations are an essential part of this process. Support artifacts should be well documented and available to all parties that require them. They should also be frequently reviewed and updated, and updating support artifacts should be a regular part of your process for adding new features and capabilities. These processes and procedures are especially useful, after all, because outages often occur during inconvenient times — times such as the middle of the night or on weekends—times when your on-call team might not perform at peak mental efficiency. These recommendations will assist your team in making smarter and safer moves toward restoring your system to operational status. Summary No one can anticipate where and when availability issues will occur. But you can assume that they will occur, especially as your system scales to larger customer demands and more complex applications. Preparation and planning are critical to improving availability and maintaining availability as your application scales. That’s it for the five focuses to help improve your modern application availability. More information on these five focuses can be found in my book, Architecting for Scale, published by O’Reilly Media. A link can be found in the shownotes. Tech Tapas — Can’t Scale? Time to go out of business Why is it essential that your application scale? Well, why not ask Robinhood Financial. Robinhood is an investment company that provides investment management services for its tech-savvy clientele. Robinhood learned the hard way the cost of success. On Monday, March 2nd, 2020, the United States stock market had a record-breaking day. After previous significant drops due to virus pandemic scares, good news caused the stock market to rally. The result was record-breaking traffic in the stock market. For a young investment company, like Robinhood Financial, this would typically be considered great news! The company thrives on new account signups and on customer market transactions. Both of these were available in record numbers to Robinhood on this day. The problem? Their traffic was *too* high. You see, companies like Robinhood need to be able to respond to variable loads, as spikes in traffic occur all the time. Still, this record-breaking traffic spike and the volatile market conditions that went with it were more than Robinhood’s infrastructure could handle. The result? There systems started to fail. This failure created a “thundering herd” effect, as Robinhood founders described it, leading to a failure of their DNS system. The result was that Robinhood systems were down for approximately one and a half days. One and a half days. This one and a half day outage occurred during a peak stock market time, a time when their customer’s needed them most. During their most critical time, their systems were unavailable. That’s often the problem, scaling related outages more often than not occur during good times, not the bad times. This can very quickly turn a significant business opportunity for success...into an utter failure. Scaling and availability problems can take the moment of greatness you’ve been working towards all your life, and turn it into an event that can shutter your business. This isn’t true just for Robinhood — all modern companies face problems like this. One of the most challenging things for an online business to handle...is success. Success can be the killer of a business. Success can shut you down. Avoiding problems like this is why it is essential that you consider scaling and availability needs of your application well before your needs arise. The day when success is staring you in the face is too late to be planning for scaling in your system architecture. In the shownotes, I have a link to the announcement of the outage by Robinhood in their blog, along with links to other useful information. There are many resources you can use to help you build in scalability and availability into your business processes and your applications. My book, Architecting for Scale, published by O’Reilly Media, has lots of useful high level information on the systems and processes involved in building highly scaled, highly available applications. You can also listen to this podcast. I talk often about scaling and availability topics as it relates to building and operating your modern digital applications.…
M
Modern Digital Applications with Lee Atchison
1 How to Improve Application Availability, p1 13:26
13:26 Play Later Play Later Lists Like Liked13:26
Play Later Play Later Lists Like LikedModern applications require high availability. Our customers expect it, our customers demand it. But building a modern scalable application that has high availability is not easy and does not happen automatically. Problems happen. And when problems happen, availability suffers. Sometimes availability problems come from the simplest of places, but sometimes they can be highly complex. In this episode, we will discuss five strategies for keeping your modern application, highly available as well. This is How to Improve Application Availability, on Modern Digital Applications. Links and More Information The following are links mentioned in this episode, and links to related information: Modern Digital Applications Website ( https://mdacast.com ) Lee Atchison Articles and Presentations ( https://leeatchison.com ) Architecting for Scale, published by O’Reilly Media ( https://architectingforscale.com ) How to Improve Availability, Part 1 Building a scalable application that has high availability is not easy and does not come automatically. Problems can crop up in unexpected ways that can cause your application to stop working for some or all of your customers. These availability problems often arise from the areas you least expect, and some of the most serious availability problems can originate from extremely simple sources. Let’s take a simple example from a real world application that I’ve worked on in the past. This problem really happened. The software was a SaaS application. Customer’s could login to the application and they received a customized experience for their personal use. One of the ways that the customer could tell they were logged in is that an avatar of themselves appeared in the top right hand corner. It wasn’t a big deal, but it was a handy indicator that you were receiving a personalized environment. We’ve all seen this sort of thing, it’s pretty common in online software applications now-a-days. Anyway, by default, when we showed the page, we read the avatar from a 3rd party avatar service that told us what avatar to display for the current user. One day, that third party system failed. Our application, which made the poor assumption that the avatar service would always be working, also failed. Simply because we were unable to display a picture of the user in the upper right hand corner, our entire application crashed and nobody could use it. It was, of course, a major problem for us. It was harder too because the avatar service was out of our control. Our business was directly tied to a 3rd party service we had no control over, and we weren’t even aware of the dependency. A very minor feature crashed our entire business…Our business crashed because of an icon. Obviously, that was unacceptable. How could we have avoided this problem? There were a thousand solutions to the problem. By far the easiest would have been to notice and catch any failure of the 3rd party service in realtime, and if it did fail, show some default generic avatar instead. There was no need to bring down our entire application over this simple problem. A simple check, some error recovery logic, some fallback options, that’s all it would have taken to avoid crashing our entire business. No one can anticipate where problems will come from, and no amount of testing will find all issues. Many of these are systemic problems, not merely code problems. To find these availability problems, we need to step back and take a systemic look at our applications and how they works. What follows are five things you can and should focus on when building a system to make sure that, as its use scales upwards, availability remains high. Number 1 - Build with Failure in Mind As Werner Vogels, CTO of Amazon, says: “Everything fails all the time.” You should plan on your applications and services failing. It will happen. Now, deal with it. Assuming your application will fail, how will it fail? As you build your system, you need to consider availability during all aspects of your architecture, design, construction, and testing. What design constructs and patterns have you considered or are you using that will help improve the availability of your software? Are you using simple but effective methods for detecting problems with other services? This can be simply catching errors. It might mean appropriate retry logic. But it could also involve things like circuit breaker patterns in order to validate external systems are behaving as they should. Circuit breaker patterns are particularly useful for handling dependency failures because they can reduce the impact a dependency failure has on your system as a whole, without having to continuously rediscover the problem and impact performance. Once you detect a failure, what can you do? Is this a hard dependency that you cannot recover from if it fails? Or can you replace the call with a temporary result that may degrade the experience but allow other functions to continue? This is the case with the avatar service, you can show a default avatar as a degraded experience, yet still the main part of your service can continue operating. Beside taking care of failures of systems that you depend on, you need to consider failures of systems that depend on you. What happens if a service that is calling your system behaves poorly? Can you handle excessive load from your consumers? Can you handle bad incoming requests and provide graceful responses? Sometimes, denial-of-service attacks can come from “friendly” sources. A service calling you could have a bug that causes it to go into a tight infinite loop, calling you at an unacceptably high rate. Can you detect this and throttle traffic appropriately in a way that doesn’t impact valid requests that might also be coming at a high rate? Number 2 - Always think about scaling Speaking about scaling… Always be thinking about scaling during all aspects of your system architecture, design, and construction. Thinking about scaling should be ingrained into your culture and part of every decision you make. Just because your application works now does not mean it will work tomorrow. Most web applications have increasing traffic patterns. A website that generates a certain amount of traffic today might generate significantly more traffic sooner than you anticipate. As you build your system, don’t build it for today’s traffic; build it for tomorrow’s traffic. Build it for traffic spikes and bulges. Build it for your biggest days. This might involve things as simple as increasing the size and capacity of your databases. But it might involve rethinking a design to reduce dependency on certain types of resources, or utilize caching and other scale-based-optimization techniques. Think about what logical limits exist to your scaling. Where are your sensitive and vulnerable spots? What can you do today to reduce your impact on these vulnerable spots? Can content you are currently generating dynamically, can it be generated statically and cached instead? Be creative. You might be surprised what scaling can do to your thought process. One simple example from my early days with Amazon. We were looking at how to scale an early version of the menu bar on the top of the amazon.com home page. That page was dynamically generated on every page load. It “had to be”, because the banner contained a “Hi Dave” personalized message for each customer. It had to be generated dynamically. Until we discovered an interesting fact. A significantly large portion of our traffic came using only one of a hundred or so possible names. By creating static versions of the banner for each of these high traffic names and caching these, we would only have to dynamically create the banner for pages that were personalized using a different, less common name. The result was that a huge amount of our dynamic traffic could be cached. Now, I don’t know if they ever implemented that approach or if there were other techniques that ultimately were used for improving scale, that’s all irrelevant now. But the point is that this was a big lesson for me and I’ve always remembered it. It can be surprising what scale can do to help you make things more optimized. *** High scale is the friend of optimization, not the enemy. That is worth repeating… *** High scale is the friend of optimization, not the enemy. We often think of optimization as a tool to fight the negative impact of increased scaling…negative impacts like resource depletion. But if you only take one lesson out of this discussion, take this. The higher scale an application operates at, the more options available to you to improve system optimization and efficiency. The more you scale, the more you can optimize for scale. That’s it for part one of this story in this episode. We will continue with the remaining three focuses for improved availability in the next episode. More information on these five focuses to improve availability can be found in my book, Architecting for Scale, published by O’Reilly Media. A link can be found in the shownotes. Tech Tapas — What happened to HP cloud? What happened to HP Cloud? HP Cloud never was at the scale of Amazon Web Services, nor Microsoft Azure, nor Google Cloud. Yet it was a player for awhile in the early cloud days. Like much of HP at the time, HP cloud was focused from the beginning on meeting the needs of enterprise customers. HP had a great idea — focus on the enterprise — at the time before enterprise was a huge focus for any of the major cloud providers. While AWS was growing focusing on the small-to-medium sized businesses and individuals, HP was focusing on the enterprise. But by the time HP was able to execute on their enterprise strategy, the big three public cloud providers — Amazon, Microsoft, Google — all had switched their focus to the enterprise customer as well. This left HP out in the cold. HP is now virtually entirely out of the public cloud space. In fact, in 2015, HP announced it was shutting down its Helton public cloud offering and instead started focusing only on private and hybrid clouds. HP ceded the public cloud market to Amazon, Microsoft, and Google... HP has been able to make some traction in this space...the private and hybrid cloud space... The reason? Their focus on using the OpenStack technology, a technology that has not shown much traction in the public cloud markets, but has gained significant traction in the private and hybrid cloud markets. Maybe HP found a groove that will work for them. But in this space, they’ll have to compete with the likes of IBM, which is also focusing on the hybrid cloud space as well... How long will HP remain a player in the private and hybrid cloud niche? We’ll have to wait to see, but as far as the public cloud is concerned, HP is nowhere to be found. That’s a shame, I would like to see HP — and IBM — both be larger players in the public cloud. We need more and better competition for AWS in the public cloud. But it looks like that competition isn’t going to come from HP.…
M
Modern Digital Applications with Lee Atchison
1 How to maintain availability with multiple AWS accounts 12:50
12:50 Play Later Play Later Lists Like Liked12:50
Play Later Play Later Lists Like LikedIn this episode, we know that using multiple availability zones helps increase your application availability and resiliency by distributing our application across multiple disperse data centers. But did you know that availability zones don’t necessarily give you the separation you expect? In fact, it is entirely possible to have two instances of a service running in two distinct availability zones, but actually have them running in the same data center, in the same physical rack, and possibly even on the same physical server! How can this be? And even more importantly, how can we avoid it? The answer involves understanding how availability zones work and how they are structured. And then, one of the oddest cloud services created is also one of the first cloud services. Before AI and before machine learning, humans actually powered a part of the cloud. This is, Life with Multiple AWS Accounts. Links and More Information The following are links mentioned in this episode, and links to related information: How to maintain availability when using multiple AWS accounts ( https://www.infoworld.com/article/3444860/5-pain-points-of-modern-software-development-and-how-to-overcome-them.html ) Modern Digital Applications Website ( https://mdacast.com ) Lee Atchison Articles and Presentations ( https://leeatchison.com ) Architecting for Scale, published by O’Reilly Media ( https://architectingforscale.com ) Distributing Your Application When building a modern, high-performant application at scale, it’s important to make sure the individual application instances are distributed across a variety of data centers in such a way that if any given data center goes offline, the application can continue to function relatively normally. This is an industry-wide best practice, and an important characteristic to architect into your applications in order to make them sufficiently resilient to data center problems. The same philosophy occurs when you build your application in the cloud. Except, when you build a cloud-based application, you typically do not have visibility into which data center a particular server or cloud resource is located. This is part of the abstraction that gives the cloud its value. Not having visibility into which data centers your application is operating in makes it difficult to build multi data center resiliency into your applications. To solve this problem, AWS created a cloud abstraction of the data center that allows you to build on this level of resiliency without being exposed to the details of data center location. The abstraction is the availability zone. AWS availability zones An AWS availability zone is an isolated set of cloud resources that allows specifying a certain level of isolation into your applications. Resources within a single availability zone may be physically or virtually near each other, to the extent that they can be dependent on each other and share subcomponents with each other. For example, two EC2 servers that are in the same availability zone may be in the same data center, in the same rack, or even on the same physical server. However, cloud resources that are in different availability zones are guaranteed to be separated into distinct data centers. They cannot be in the same data center, they cannot be in the same rack, and they cannot be using the same physical servers. They are distinct and independent from each other. Hence, the solution to the resiliency problem, you can build your application to live in multiple availability zones. If you construct your application so instances of your application are distributed across multiple availability zones, you can isolate yourself from hardware failures such as server failures, rack failures, and even entire data center failures. Using multiple availability zones allows you to build in application resiliency. Availability zones as data centers Loosely, availability zones can be thought of as data centers. At a first level of approximation, this is a roughly reasonable assumption to make. However, there is not a one-to-one mapping of availability zones to data centers. When you create your AWS account, your availability zone names are mapped to individual data centers in a dynamic fashion. This means one AWS account may have an availability zone named us-east-1a mapped to data center #4, and another account may have the same availability zone mapped to data center #2. Even worse, a given data center may map to different availability zones in different accounts. For example, data center #4 in account #1 may be used for availability zone us-east-1a, but the same data center in account #2 may be used for availability zone us-east-1b. You can find out how your availability zones are mapped to specific data centers in a given account by looking in the AWS console in the Resource Access Manager (RAM). This shows a mapping of availability zone names to an AZ ID. An AZ ID is a unique identifier that can be effectively used as a data center identifier. It shows the mapping, for your current account, of each availability zone to its associated data center’s AZ ID. This mapping is shown for your currently selected region, but you can simply switch regions to show the mapping for any region in your account. So, when you are using multiple accounts, don’t use availability zone to determine data center redundancy. Instead, you should use the AZ ID. If two availability zones in different accounts have different AZ IDs, you can be sure that those two availability zones are in distinct data centers. Using the AZ ID, rather than the availability zone name, is a safe way to ensure your applications live in distinct data centers across multiple accounts. Tech Tapas — Mechanical Turk One of the oddest cloud services created was also one of the first cloud services. In 2005, Amazon’s Jeff Bezos had an idea and created the Mechanical Turk service. Mechanical Turk, or MTurk, was a service that allowed one to programmatically request execution of micro tasks that could only be performed by a human being. The service allowed software applications to request the execution of a task, and on the back end a human was used to execute the task. In the days before high quality Artificial Intelligence, it was a way of automating many tasks that only a human could perform. One early use for mechanical Turk was to use it to compare product pictures to determine if pictures were of the same product or different products. A single micro task might be to compare one picture to another and respond back yes or no if they represented the same product. This technique was used to improve the quality of the Amazon catalog. Another common task was to examine a picture of a sales receipt and record what information was found on the receipt. Workers on the back end were paid for each task they completed, typically making on average 10 cents for each micro task they completed. I remember when I was working at Amazon the day that Jeff presented to the employees the Mechanical Turk service. It was highly innovative. Of course, most of what it could do is now, or soon will be, replaced by machine learning algorithms and Artificial Intelligence. But still, it was a fantastic innovation at the time.…
M
Modern Digital Applications with Lee Atchison
1 Special Edition: The Great Serverless Debate, Redux — Special Guest Clay Smith, part 2 28:57
28:57 Play Later Play Later Lists Like Liked28:57
Play Later Play Later Lists Like LikedThis is a special edition of Modern Digital Applications. July 9th, 2018 was the launch of a podcast episode. It was an episode of the “ Modern Software Podcast ”, a podcast sponsored by New Relic , and hosted by New Relic’s Fredric Paul and Tori Wieldt . This particular episode was titled “ The Great Serverless Debate ”. It was a debate between myself, and a good friend of mine, Clay Smith . Clay and I were guests on the show. That episode was a huge success, and I still get asked questions about it today. It seemed to me that it was time for an update of that debate…a redux if you will. Since New Relic’s Modern Software Podcast isn’t active right now, I thought I would take on the challenge myself and host a redo of the great debate — based on what we know about serverless in 2020. So, on February 21, 2020, Clay and I got together for an update to our views on the world of serverless. This is The Great Serverless Debate, Redux. This is the second part of that interview. Links and More Information The following are links mentioned in this episode, and links to related information: Modern Digital Applications Website ( https://mdacast.com ) Lee Atchison Articles and Presentations ( https://leeatchison.com ) Architecting for Scale, published by O’Reilly Media ( https://architectingforscale.com ) The Great Serverless Debate ( https://leeatchison.com/2018/07/11/the-great-serverless-debate/ ) Clay Smith - Twitter ( https://twitter.com/smithclay ) Clay Smith - LinkedIn ( https://www.linkedin.com/in/smithclay/ ) Clay Smith’s Monitoring Monitoring Newsletter ( https://monitoring2.substack.com/ ) Modern Software Podcast - Fredric Paul @TheFreditor ( https://twitter.com/TheFreditor ) Modern Software Podcast - Tori Wieldt @ToriWieldt ( https://twitter.com/ToriWieldt ) Lee’s Guest — Clay Smith Clay Smith is a good friend of mine. He was a senior software engineer at several early-stage startup companies and has been building serverless solutions for many years now, from mobile backends to real-time APIs with Amazon Web Services. Clay was a senior Developer Evangelist at New Relic, which is where Clay and I met. Clay’s newsletter is “Monitoring Monitoring”. You can subscribe to the newsletter at https://monitoring2.substack.com/ . Questions/Issues Discussed Is Lambda living up to the hype? Is there an end to the hype anytime soon? Has Fargate lived up to the hype? What is the role of containers vs FaaS? What is the role of Kubernetes? What types of problems are suited for FaaS and what kind of problems are not? How good was our guesses in 2018 for the state of serverless in 2020? What will be the state of serverless in 2022? We need a better term for FaaS than the generic term “serverless” Use of FaaS as a glue between external services Quotes Lee: “One of the problems with Lambda is…you are not making your service boundaries based on functionality, you are making service boundaries based on the limitation of the technology.” Clay: “It’s been a great two years for event driven architectures in general.” Clay: “There’s a new class of startups giving a Heroku type experience that look very interesting…” Lee: “Lambda is very good…for the hook and glue use case” [system integration] Clay: “API Glue” Clay: “We both definitely agree that the glue aspect is still the killer use case…” Clay: “Greatest disappointment around FaaS the past two years is the DevOps…workflows. These sorts of solutions are still a total mess.” Lee: “When Lambda first came out, that was my number one concern…how do you deal with them operationally. We are still operating the same way we were [then] with the tooling available now. It isn’t yet to the level it needs to be to be a true partner in the operations or DevOps organizations. … No-one has figured out how to do it yet.” Lee: “When you move from a monolith to services, you aren’t decreasing the complexity, you are moving it out of the code and into the system of service interactions. With Lambda, there aren’t the tools yet to manage the complexity between interacting Lambda functions reasonable.” Administrative Clay appeared as a personal guest, and not a representative of his employer. All views expressed by Clay are his own and not of his employer. This podcast, “Modern Digital Applications”, is not affiliated with the “Modern Software” podcast. All views expressed in “The Great Serverless Debate, Redux” are those of Clay and Lee personally and not related to the “Modern Software” podcast. The original “Great Serverless Debate” was created by Fredric Paul on the “Modern Software" podcast.…
M
Modern Digital Applications with Lee Atchison
1 News Report: Architecting for Scale, 2nd Edition 1:28
1:28 Play Later Play Later Lists Like Liked1:28
Play Later Play Later Lists Like LikedThe second edition of my book, Architecting for Scale, is now available! Links and More Information Book Website ( https://architectingforscale.com ) Amazon.com ( https://www.amazon.com/Architecting-Scale-Maintain-Availability-Manage/dp/1492057177/ ) O’Reilly ( http://shop.oreilly.com/product/0636920274308.do ) Special Report Hello everyone and welcome to Modern Digital Applications. This is just a very quick news break... The second edition of my book, Architecting for Scale, is now available! This edition is dramatically updated and improved. It includes new content on cloud computing, microservices, and serverless computing. All the content has been updated and it has been significantly reorganized. The book is now available for purchase on amazon.com , other technical bookstores, or directly from O’Reilly Media. It’s also available included as part of your O’Reilly Safari subscription. Check it out! If you have any questions or comments, feel free to reach out to me. Links to the websites where you can find out more about the book or purchase it are contained in the shownotes for this episode. Thank you and enjoy!…
M
Modern Digital Applications with Lee Atchison
1 Special Edition: The Great Serverless Debate, Redux — Special Guest Clay Smith 22:56
22:56 Play Later Play Later Lists Like Liked22:56
Play Later Play Later Lists Like LikedThis is a special edition of Modern Digital Applications. July 9th, 2018 was the launch of a podcast episode. It was an episode of the “ Modern Software Podcast ”, a podcast sponsored by New Relic , and hosted by New Relic’s Fredric Paul and Tori Wieldt . This particular episode was titled “ The Great Serverless Debate ”. It was a debate between myself, and a good friend of mine, Clay Smith . Clay and I were guests on the show. That episode was a huge success, and I still get asked questions about it today. It seemed to me that it was time for an update of that debate…a redux if you will. Since New Relic’s Modern Software Podcast isn’t active right now, I thought I would take on the challenge myself and host a redo of the great debate — based on what we know about serverless in 2020. So, on February 21, 2020, Clay and I got together for an update to our views on the world of serverless. This is The Great Serverless Debate, Redux. Links and More Information The following are links mentioned in this episode, and links to related information: Modern Digital Applications Website ( https://mdacast.com ) Lee Atchison Articles and Presentations ( https://leeatchison.com ) Architecting for Scale, published by O’Reilly Media ( https://architectingforscale.com ) The Great Serverless Debate ( https://leeatchison.com/2018/07/11/the-great-serverless-debate/ ) Clay Smith - Twitter ( https://twitter.com/smithclay ) Clay Smith - LinkedIn ( https://www.linkedin.com/in/smithclay/ ) Clay Smith’s Monitoring Monitoring Newsletter ( https://monitoring2.substack.com/ ) Modern Software Podcast - Fredric Paul @TheFreditor ( https://twitter.com/TheFreditor ) Modern Software Podcast - Tori Wieldt @ToriWieldt ( https://twitter.com/ToriWieldt ) Lee’s Guest — Clay Smith Clay Smith is a good friend of mine. He was a senior software engineer at several early-stage startup companies and has been building serverless solutions for many years now, from mobile backends to real-time APIs with Amazon Web Services. Clay was a senior Developer Evangelist at New Relic, which is where Clay and I met. Clay’s newsletter is “Monitoring Monitoring”. You can subscribe to the newsletter at https://monitoring2.substack.com/ . Questions/Issues Discussed Is Lambda living up to the hype? Is there an end to the hype anytime soon? Has Fargate lived up to the hype? What is the role of containers vs FaaS? What is the role of Kubernetes? What types of problems are suited for FaaS and what kind of problems are not? How good was our guesses in 2018 for the state of serverless in 2020? What will be the state of serverless in 2022? We need a better term for FaaS than the generic term “serverless” Use of FaaS as a glue between external services Quotes Lee: “One of the problems with Lambda is…you are not making your service boundaries based on functionality, you are making service boundaries based on the limitation of the technology.” Clay: “It’s been a great two years for event driven architectures in general.” Clay: “There’s a new class of startups giving a Heroku type experience that look very interesting…” Lee: “Lambda is very good…for the hook and glue use case” [system integration] Clay: “API Glue” Clay: “We both definitely agree that the glue aspect is still the killer use case…” Clay: “Greatest disappointment around FaaS the past two years is the DevOps…workflows. These sorts of solutions are still a total mess.” Lee: “When Lambda first came out, that was my number one concern…how do you deal with them operationally. We are still operating the same way we were [then] with the tooling available now. It isn’t yet to the level it needs to be to be a true partner in the operations or DevOps organizations. … No-one has figured out how to do it yet.” Lee: “When you move from a monolith to services, you aren’t decreasing the complexity, you are moving it out of the code and into the system of service interactions. With Lambda, there aren’t the tools yet to manage the complexity between interacting Lambda functions reasonable.” Administrative Clay appeared as a personal guest, and not a representative of his employer. All views expressed by Clay are his own and not of his employer. This podcast, “Modern Digital Applications”, is not affiliated with the “Modern Software” podcast. All views expressed in “The Great Serverless Debate, Redux” are those of Clay and Lee personally and not related to the “Modern Software” podcast. The original “Great Serverless Debate” was created by Fredric Paul on the “Modern Software" podcast.…
M
Modern Digital Applications with Lee Atchison
In this episode, Our customers depend on them, and our businesses depend on them. Without modern web apps, most businesses would not survive. This is the second in a two part series on the principles for modernizing your enterprise web application. And then, what is STOSA and how does it help with modern digital applications? Links and More Information The following are links mentioned in this episode, and links to related information: Modern Digital Applications Website ( https://mdacast.com ) Lee Atchison Articles and Presentations ( https://leeatchison.com ) Architecting for Scale, published by O’Reilly Media ( https://architectingforscale.com ) STOSA — Single Team Oriented Service Architecture ( https://stosa.org ) Main Story - Five Principles for Modernizing Your Enterprise Web Applications, p2 Modern web applications have a lot riding on them. Our customers depend on them, and our business depends on them. Without modern web applications, many businesses would not survive. Modern web applications must scale to meet our biggest needs without suffering outages or other availability issues. In order to meet and exceed this high bar, we must build, manage, and monitor our applications using modern principles, processes, and procedures. There are five guiding principles that a modern application requires in order to meet these expectations. In previous episodes, we discussed principle 1, using service-based architectures; principle 2, organizing teams around services; and principle 3, using DevOps processes and procedures. In this episode, we will finish the remaining two principles. Starting with… Principle #4. Use dynamic infrastructures. Customer traffic loads on our applications vary considerably. The upper bound or maximum amount of traffic your application needs to support can never be known accurately. In the past, as application traffic grew, we simply threw additional hardware at the problem. But as traffic variations increase, and customer usage patterns become more complicated. This simple solution is no longer reasonable. Simply adding hardware may be fine to handle expected peaks, but what do you do when an unexpected spike in usage arrives? When I was at Amazon, there was a term that was used. It was called the “Johnny Carson Effect”. It’s origin comes from the day that Johnny Carson died. On that day, there was a huge uptick in demand for any videotape of old episode’s of Johnny Carson’s The Tonight Show. Demand was huge, and given the suddenness of the event, it was all unexpected demand. Now-a-days, the term “going viral” is both a positive and a negative for most businesses. Going viral means huge attention given to you and your business. But it also means huge and unexpected traffic volume. The last thing you want is for your application to fail right at the point when everyone is watching. How can you handle the unexpected surge from such an unexpected event? It is no longer possible to throw large quantities of hardware at an application. You never know how much hardware is enough hardware to handle your possible maximum need. Additionally, when your application is not operating at a high volume, what do you do with the extra hardware? Typically, it’s sitting idle...which is a huge a waste of resources...and even more importantly...a waste of money. Especially if your traffic needs are highly variable or highly spiky, simply adding hardware to handle your peaks is not an effective use of resources. Instead, you must add resources to your applications dynamically, as they are needed and when they are needed. These resources can be applied when your traffic needs are high, and they can be released when they are no longer needed. This is what dynamic infrastructures are all about. The only way to implement a true dynamic infrastructure for an application with a highly variable load is to use the dynamic capabilities of the cloud. In the cloud, resources can be added on demand, and released when they are no longer needed, becoming available for other uses. To be successful, modern applications must make use of dynamic infrastructures to not only optimize their operational costs, but maintain availability during all types of traffic patterns. I have many articles I’ve written, and I’ve given many talks on dynamic infrastructures and the dynamic cloud. For more information, go to leeatchison.com and click on “Articles”, or click on “Presentations”. Principle #5. Maintain high visibility into all aspects of your application. It is impossible to keep a modern application working if you do not have visibility into what your application is doing and how it is performing. This is a basic truism of modern applications. Without proper visibility into the operation of your application, you have no idea if the application is meeting the needs of your customers, or of your business. Without proper visibility, you have no way of knowing if a ticking time bomb is ready to go off and make your application suddenly unavailable. **With** proper visibility, you can understand what your application is doing, you can understand how it’s doing it, and you can understand if there are any lurking issues that might impact the application’s ability to perform those operations in the foreseeable future. Further, while using tools such as synthetic testing and infrastructure performance monitoring is important, this level of monitoring is not sufficient. You cannot completely understand how your application is performing without an internal view of its operations. This point is worth restating. You cannot understand how your application is performing without an ***internal*** view of its operation. This is only possible using application performance monitoring, log monitoring, and trace analysis. These three tools are critical to the ability to have proper visibility into the operation of your modern application. With proper visibility, comes the ability to build and maintain your ever growing, ever more complex, modern applications. That’s it, that’s the last of the five principles required to modernize your enterprise applications. In summary, the five principles are: Use service-based architectures, organize your teams around your service architecture, use DevOps principles methodologies, use dynamic infrastructures, and maintain high visibility. Our applications *are* getting more complex, and becoming more intertwined with the fundamental operation of our business. As such, the expectations of our customers...expectations on reliability, functionality, and availability...continue to grow. As we modernize our applications to meet these growing needs, these five principles will you meet the needs of your customers and your business. More information on these five principles is available in my article published in InfoWorld, “How to modernize enterprise web applications”. A link to this article is available in the show notes. Tech Tapas Single Team Owned Service Architecture ... STOSA. STOSA is an operational model for how to structure your organization to effectively manage a multi service or micro service architected application. For your teams and organization to be STOSA compliant, they must: Have an application that is constructed using a service-based architecture or a microservice-based architecture. Have multiple development teams that are responsible for building and maintaining the application. Each and every service in your application must be assigned to a development team, who owns that service. The team that owns each service should be well documented and readily available to everyone in the organization. No service should be assigned to more than one development team, however individual development teams may own more than one service. Teams are responsible for all aspects of managing the service, from service architecture and design, development, testing, deployment, monitoring, and incident resolution. Services have strong boundaries between them, including well-documented APIs. The service owns its own data. Data is part of the service. If a service needs to access data stored in a different service, it must use one of the well-documented APIs to access that data. Services maintain internal service-level agreements (SLAs) between them that are monitored and violations reported to the owning team. A STOSA-based application is an application for which all services follow these rules. A STOSA-based organization is one in which all service teams follow the preceding rules and all applications are STOSA applications. I developed this model based on my experiences building and operating large scaled applications. The model first appears in my book Architecting for Scale, by O’Reilly Media. You can learn more about the STOSA model by going to stosa.org . That’s s-t-o-s-a-dot-o-r-g. Links are available in the show notes.…
M
Modern Digital Applications with Lee Atchison
In this episode, Our customers depend on them, and our businesses depend on them. Without modern web apps, most businesses would not survive. I’ll present the first in a two part series on the principles for modernizing your enterprise web application. And then, what does any of this have to do with my son and his wage reporting application? Links and More Information The following are links mentioned in this episode, and links to related information: Modern Digital Applications Website ( https://mdacast.com ) Lee Atchison Articles and Presentations ( https://leeatchison.com ) Architecting for Scale, published by O’Reilly Media ( https://architectingforscale.com ) Main Story - Five Principles for Modernizing Your Enterprise Web Applications Modern web applications have a lot riding on them. Our customers depend on them, and our business depends on them. Without modern web applications, many businesses would not survive. Modern web applications must scale to meet our biggest needs without suffering outages or other availability issues. In order to meet and exceed this high bar, we must build, manage, and monitor our applications using modern principles, processes, and procedures. There are five guiding principles that a modern application requires in order to meet these expectations. Principle #1 Using service-based architectures Modern applications are large and complex, too complex to be handled as a single entity by a single team. Instead, multiple teams are required to develop, test, support, and operate these applications. This complex task is impossible when the application is a single monolith. To handle a large complex application, split the application into multiple independent services. Then, you can assign different pieces of the application to different teams, allowing parallel development and operation. By keeping modules isolated, they can be built, tested, and operated in isolation. Application problems can be more easily correlated and isolated to individual services, making it easier to decide which team should be called on to work on an issue when a problem arises. Scaling is not just about the amount of traffic an application receives, but about the size and complexity of the application itself. Using service-based architectures makes scaling the application easier and allows larger and more complex applications to be dealt with reasonably. In future segments, we will talk about the remaining four guiding principles that are required for enterprise applications to modernize. In the next segment, principle #2 is about team organization. Our applications *are* getting more complex, and becoming more intertwined with the fundamental operation of our business. As such, the expectations of our customers are growing, and the demands for reliability, scalability, and functionality from management are keeping pace. Only by modernizing our applications can we make our applications meet the needs of our customers and our business. Principle #2. Organize teams around services. Architecting your application as a service-based application is only part of the answer to modernizing your enterprise systems. Once you have your application split up into services, it is essential that you structure your development teams around those services. It is critical that a single team owns each service. And when I say “own”, I mean complete ownership...front to back...top to bottom. This includes development, testing, operation, and support. All aspects of the development and operation of each service should be handled by one and only one team. There is a model for application management that stresses these ownership values. STOSA, which stands for Single Team Owned Service Architecture, provides guiding principles on team-level ownership, providing clear boundaries between services and promoting clear understanding and expectations between teams. We will talk about STOSA and the operating principles behind the STOSA model in much greater detail in future segments. Until then, you can read more about STOSA at stosa.org . Principle #3. Use DevOps processes and procedures. Now that we have team organization and ownership taken care of, the next principle is to focus on the policies and practices that those teams use to build and operate their services. For modern applications, teams must make use of modern devops processes and procedures to build and maintain their services. Many organizations claim that they “use devops,” but ultimately they do not operate their organizations in a true devops manner. True devops techniques require a complete organizational flush of all processes and systems in order to incorporate the principles of devops within the entire organization. You cannot accomplish this transformation by simply “creating a DevOps team,” though many organizations try. DevOps is a cultural transformation. And only by undergoing this cultural transformation can an organization truly build and operate modern applications at scale. Without DevOps, your application will tend to stale in some spots and the work you have done to separate and “service-ize” your application will be for naught. You will not be able to build and maintain your applications unless you adopt...and truly become...a true DevOps culture within your organization. There are many good sources of information on DevOps. But, certainly, no DevOps bookshelf is complete without The Phoenix Project. You can find a link to The Phoenix Project in the shownotes. In upcoming segments, we will talk about the remaining guiding principles that are required for enterprise applications to modernize. More information on these five principles is available in my article published in InfoWorld, “How to modernize enterprise web applications”. A link to this article is available in the show notes. Tech Tapas My son has an app on his iPhone that he uses to record his income for some of the government benefits he gets. Once a month, he has to run this app to simply enter the amount of money he’s earned that month. He simply enters the number and his social security number into the app, presses submit, and he’s done. That’s it. The problem is, the application on his iPhone only works between the hours of 8am and 5pm, Eastern Standard Time, Monday through Friday. If you try and use the application any other time of the day or night, the application pops up an error message and says the application is offline right now, and you can’t use it. How many of you can operate applications like that? How many of you can make it so your customers only use your mobile applications during business hours, and prohibit them from using them at any other time? I know the answer…none of you. It is simply not possible in today’s modern world. People expect our applications to function, and they expect them to function all the time. Anytime they attempt to use your application, they expect it to work, without exception. 24 hours a day, 7 days a week…always. That is what’s required of modern applications. There is not acceptable downtime. I sometimes here people say they have “nearly 100% uptime!”. But then when you dig in deeper, you find out that they bring their application down for a “maintenance window” every Sunday evening. How is that 100% uptime? The answer is, it’s not. In the modern world, we have to perform all of our maintenance live, and we have to have our applications up and operational all the time. In our customer’s minds, there is no excuse for anything other than that.…
M
Modern Digital Applications with Lee Atchison
1 Avoiding Downtime while Migrating Data to the Cloud 14:26
14:26 Play Later Play Later Lists Like Liked14:26
Play Later Play Later Lists Like LikedIn this episode, the hardest part of your cloud migration is moving your data to the cloud. Moving your data to the cloud, without suffering planned or unplanned downtime, can be a challenge. I’m going to give you three strategies that will help you avoid downtime during the migration of your critical application data to the cloud. And in Tech Tapas, we are going to take a look at what it means to fly two mistakes high, and how that relates to application availability. Links and More Information The following are links mentioned in this episode, and links to related information: 3 Strategies to Avoid Downtime When Migrating Data to the Cloud ( https://blog.newrelic.com/engineering/migrating-data-to-cloud-avoid-downtime-strategies/ ) Modern Digital Applications Website ( https://mdacast.com ) Lee Atchison Articles and Presentations ( https://leeatchison.com ) Architecting for Scale, published by O’Reilly Media ( https://architectingforscale.com ) Main Story - 3 Strategies to Avoid Downtime when Migrating Data to the Cloud Moving your data is one of the trickiest parts of a cloud migration. During the migration, the location of your data can have a significant impact on the performance of your application. During the data transfer, keeping the data intact, in sync, and self-consistent requires either tight correlation or—worse—application downtime. Moving your data and the applications that utilize the data at the same time is necessary to keep your application performance acceptable. Deciding how and when to migrate your data relative to your services, though, is a complex question. Often companies will rely on the expertise of a migration architect, which is a role that can greatly contribute to the success of any cloud migration. Whether you have an on-staff cloud architect or not, there are three primary strategies for migrating application data to the cloud: Offline copy migration Master/read replica switch migration Master/master migration It doesn’t matter if you’re migrating an SQL database, a noSQL database, or simply raw data files—each migration method requires a different amount of effort, has a different impact on your application’s availability, and presents a different risk profile for your business. Strategy 1: Copy Data While Application is Offline An offline copy migration is the most straightforward method. Bring down your on-premise application, copy the data from your on-premise database to the new cloud database, then bring your application back online in the cloud. An offline copy migration is simple, easy, and safe, but you’ll have to take your application offline to execute it. If your dataset is extremely large, your application may be offline for a significant period of time, which will undoubtedly impact your customers and business. For most applications, the amount of downtime required for an offline copy migration is generally unacceptable. But if your business can tolerate some downtime, and your dataset is small enough, you should consider this method. It’s the easiest, least expensive, and least risky method of migrating your data to the cloud. Strategy 2: Read Replica Switch The goal of a read replica switch migration is to reduce application downtime without significantly complicating the data migration itself. For this type of migration, you start with your master version of your database running in your on-premise data center. You then set up a read replica copy of your database in the cloud with one way synchronization of data from your on-premise master to your read replica. At this point, you still make all data updates and changes to the on-premise master, and the master synchronizes those changes with the cloud-based read replica. The master-replica model is common in most database systems. You’ll continue to perform data writes to the on-premise master, even after you’ve gotten your application migrated and operational in the cloud. At some predetermined point in time, you’ll “switchover” and swap the master/read replica roles. The cloud database becomes the master and the on-premise database becomes the read replica. You simultaneously move all write access from your on-premise database to your cloud database. You’ll need a short period of downtime during the switchover, but the downtime is significantly less than what’s required using the offline copy method. Strategy 3: Master/Master Migration This is the most complicated of the three data migration strategies and has the greatest potential for risk. However, if you implement it correctly, you can accomplish a data migration without any application downtime whatsoever. To begin, create a duplicate of your on-premise database master in the cloud and set up bi-directional synchronization between the two masters, synchronizing all data from on-premise to the cloud, and from the cloud to on-prem. Basically, you’re left with a typical multi-master database configuration. After you set up both databases, you can read and write data from either the on-premise database or the cloud database, and both will remain in sync. This will allow you to move your applications and services independently, on your own schedule, without needing to worry about your data. At the completion of your migration, simply turn off your on-premise master and use your cloud master as your database. It’s important to note, however, that this method is not without complexity. Setting up a multi-master database is quite complicated and comes with the risk of skewed data and other untimely results. For example, what happens if you try and update the same data simultaneously in both masters? Or what if you try to read data from one master before an update to the other master has synchronized the data? As such, this model only works if your application’s data access patterns and data management strategies can support it. You’ll also need application specific synchronization and sync resolution routines to handle sync-related issues as they arise. If your application, data, and business can handle this migration method, consider yourself fortunate and use it. It’s the cleanest and easiest of the three strategies. Mitigate migration risks Any data migration comes with some risk, especially the risk of data corruption. Your data is most at risk while the migration is in progress; swift and determined execution of the migration is critical. Don’t stop a migration until you have completed the process or you have rolled it back completely. And never stop a migration halfway through—half-migrated data isn’t useful to anyone. Risk of data corruption is especially high when migrating extremely large datasets. Offline data copy and transfer tools such as AWS Snowball can help manage the migration of large quantities of data, but they do nothing to help with your application’s data usage patterns during a migration. Even if you use a transfer device such as Snowball, you’ll still need to use one of the migration strategies described above. Tech Tapas — Two Mistakes High I fly model R/C airplanes. There is an expression we use in that hobby while you are learning to fly. It is “keep your plane two mistakes high”. What does it mean to “keep two mistakes high”? Well, when you are flying, and you are doing some maneuver, such as a stunt of some sort. If you make a mistake, the result is your plane will typically lose some altitude. Now, while you are recovering from that mistake — and you are now lower in altitude — what happens if you make another mistake? If your plane isn’t high enough to recover from the second mistake...well...it’s bad news for you and your plane... You always want to be high enough so you can recover from a mistake, even while you are recovering from a mistake... You always want to stay high enough so you don’t crash, no matter what goes wrong. This is where the expression “keep two mistakes high” comes from. You always want to perform stunts high enough so that if you make a mistake, you have room to recover even if you make a mistake during the recovery. This is a good analogy for maintaining availability in our most critical modern web applications. Keeping two mistakes high in our critical modern applications means that even when something is going wrong with our application, you want to be able to keep your application running reliably enough so that you can afford for something else to go wrong while you are still recovering from the main problem. This applies to many different aspects of modern application availability planning, from dealing with hardware failures, to data redundancy, to capacity planning, to performing retries in your inter-service API calls, to risk management and disaster planning. Keeping “two mistakes high” is a lesson about redundancy. It’s a lesson about availability, and it’s a lesson about resiliency. It applies to model aviation, and it applies just as effectively to modern application development and operation. In future episodes of Modern Digital Applications, I will be using this principle in giving you guidance and suggestions to keeping your critical modern applications operating.…
M
Modern Digital Applications with Lee Atchison
Google Cloud Next is coming to the Moscone Center in San Francisco from April 6 until April 8. This is Google’s big cloud event of the year. Links and More Information The following are links mentioned in this episode, and links to related information: Announcement — Google Cloud Next ( https://cloud.google.com/blog/topics/google-cloud-next/join-us-at-google-cloud-next-2020 ) Registration — Google Cloud Next ( https://cloud.withgoogle.com/next/sf ) Use code GRPABLOG2020 for $500 USD off a full price ticket. Modern Digital Applications Website ( https://mdacast.com ) Lee Atchison Articles and Presentations ( https://leeatchison.com ) Architecting for Scale, published by O’Reilly Media ( https://architectingforscale.com ) Special Report Hello everyone and welcome to Modern Digital Applications. This is just a very quick news break... Those of you using Google Cloud will want to hear this. Google Cloud Next is coming to the Moscone Center in San Francisco from April 6 until April 8. This is Google’s big cloud event of the year. According to the information provided by Google: Google Cloud Next brings together a global cloud community of leaders, developers, and influencers to help you get inspired and solve your most pressing business challenges. If you would like to attend, check out the show notes for a link to the registration page. Also in the show notes is a code that is good for $500 off a full price ticket. You can expect that Google will have a host of new product announcements around this time, so even if you don’t attend, listen up in April for more news from Google Cloud.…
M
Modern Digital Applications with Lee Atchison
1 How to Use Service Tiers and Redundancy in Outer Space 15:32
15:32 Play Later Play Later Lists Like Liked15:32
Play Later Play Later Lists Like LikedIn this episode, we conclude our three part series on Service Tiers, and how they can be used to prevent disasters in applications using service based architectures. We also take a look at the very first AWS service. Any guess what that service is? Finally, what does redundancy look like in outer space? And just how effective was the space shuttle application systems at keeping the shuttle safe? Links and More Information The following are links mentioned in this episode, and links to related information: Modern Digital Applications Website ( https://mdacast.com ) Lee Atchison Articles and Presentations ( https://leeatchison.com ) How Service Tiers Can Help to Avoid Microservices Disasters ( https://thenewstack.io/how-service-tiers-can-help-to-avoid-microservices-disasters/ ) Architecting for Scale, published by O’Reilly Media ( https://architectingforscale.com ) Main Story How to Use Service Tiers So, now that we’ve defined service tiers, how do you use service tiers? Service tiers have two distinct uses. Helping determine required responsiveness to problems, and requirements for dependencies between individual services. Responsiveness Let’s first talk about responsiveness. The service tier level of a service can be used to determine how fast or slow a problem with a service should be addressed. Of course, the higher the significance of a problem, the faster it should be addressed. But, in general, ***the lower the service tier number, the higher importance the problem likely is…and therefore the faster it should be addressed***. A low-to-medium severity problem in a Tier-1 service is likely more important and impactful than a high severity problem with a Tier-4 service. Given this, you can use the service tier, in conjunction with the severity of the problem, together to determine how fast of a response your team should have to a problem. Should we be alerted immediately, 24 hours a day, 7 days a week and fix the problem no matter what time of the day or night? Or is this a problem that can wait until the next morning to fix? Or is it a problem we can add to a queue and fix it when we get to it in our overall list of priorities? Or should we simply add it to our backlog for future consideration? Service tiers, in conjunction with problem severity, can give you the right procedural guidelines for how to handle a problem. You can use them to set SLAs on service responsiveness. You can even use them to set availability SLAs for your services. For example, you could create a policy that says that all Tier 1 services need to have an availability of 99.95%. This might dictate that all high severity problems must be resolved within 2 hours of identification, meaning that you must have an on call support team available 24 hours a day, 7 days a week and that support team must have enough knowledge and experience to fix any serious problems that arise. This would likely mean the owning development team would need to comprise the support rotation for this service. Meanwhile a Tier 3 service might be able to have an availability SLA of only 99.8%. A Tier 4 service might not even have an availability SLA. This would mean that all but the most serious problems could probably wait until the next business day to be fixed, meaning an on call support role may not be needed, or may not need to be as formal or have tight mean time to repair goals. Service Tiers help set policy on responsiveness requirements for your services, which can then dictate many requirements for your other policies and procedures. Interservice Dependencies Now, let’s talk about how service tiers can help with inter service dependencies. Given that services at different service tier levels have different responsiveness requirements. This impacts your dependency map between services and assumptions you can make about your service dependencies. For example, if a Tier-4 service, a low priority service, makes a call to a Tier-1 service, a high priority)service, then it probably is safe for the Tier-4 service to assume that the Tier-1 service will always respond and will always be available. If for some reason the Tier-1 service does not respond, it would typically be acceptable for the Tier-4 service to simply fail itself. After all, if a Tier-1 service for your application is down, significant efforts will be immediately put into place to try and resolve that service problem. The fact that a Tier-4 service is also down will not be of any significant consequence. Think of the case where your web application is down because users cannot log in…this would be a Tier 1 service problem. In such a situation, how concerning will it be that the marketing email service is down, delaying the delivery of your day’s marketing emails? The marketing email service is a Tier-4 service problem, so it is dwarfed by the Tier 1 login problem. But the reverse is not true. If a Tier-1 service depends on a Tier 4 service, , then that Tier-1 service ***must*** ***have*** developed contingency plans and failover recovery plans for when that Tier-4 service might be down. After all, you don’t want a Tier-1 service to fail simply because a much lower priority Tier-4 service is not functioning. As an example, let’s say you have a Tier-1 web application running. The web application displays the current customer’s avatar in the upper right hand corner whenever they are logged in. It does this by calling the customer avatar service, which is likely a Tier-3 service. After all, if you cannot display the avatar, it really has no significant impact on the customer’s experience. So, what happens to the Tier-1 web application if it tries to get the customer’s avatar from the Tier-3 avatar service, and the Tier 3 service is down? In this case, it would *not*, under any situation, be acceptable for the Tier-1 application to fail simply because the avatar service was failing. Instead, the Tier-1 service, when it calls the avatar service, should have safe guards in place in order to handle service failures. It should have contingency plans in place for what to do if the tier-3 service call fails. In this case, the web application can simply not show the avatar. The web application can simply not show the avatar if it is not available. But it would not be acceptable for the entire application to fail simply because it couldn’t display the avatar! This may sound like an unrealistic example, but I actually saw this happen once, with a company that was not paying attention to service levels. They had a SaaS application that went down and was completely unavailable to their customer’s, simply because they could not load the customer’s avatar due to a problem in the external avatar service. It was an embarrassing problem for them to admit. Therefore, it is not acceptable for a Tier 1 service to fail simply because a dependent Tier 3 service is unavailable. Instead, the Tier 1 service should be built in such a way that it can work around and deal with failures of the Tier 3 service. So, we can see that Service Tiers can be used to determine the criticalness of inter service dependencies. If a lower priority service, such as Tier 4 service, depends on a higher priority service, such as a Tier 1 service, then the Tier 4 service can generally ignore failures of the Tier 1 service. But the reverse is not true. If a higher priority service, such as a Tier 1 or 2 service, depends on a lower priority service, such as a Tier 3 or 4 service, then the Tier 1 or 2 service must be written to handle and managed failures of the lower priority dependent service. This is the power of service tiers. Tech Tapas - First AWS Service What was the first cloud service that Amazon put out as part of Amazon Web Services? The first cloud service was the Simple Queue Service. The Simple Queue Service, or SQS, is still available today. It provides a message queuing system for delivery of messages between loosely connected application components. SQS was launched publicly in November of 2004. AWS was relaunched in March of 2006 as a combination of three initial services: SQS, the S3 object storage service, and the EC2 virtual server service. These were all very early implementations. The EC2 instances had no persistent storage attached, so they could not be used to build persistent storage systems, such as relational databases. The services were also all exposed to the public internet, so there was no security available for any of the services. The EC2 instances were openly vulnerable on the public internet. EC2 didn’t get persistent attached storage until EBS was launched in 2008. It didn’t get private network security until VPC launched in September of 2009. It was after those two capabilities were added that EC2 became a viable option for application computation, and cloud computing took off with a vengeance. Tech Tapas - Space Shuttle Redundancy One requirement of modern digital applications is that they must maintain a high degree of resiliency...resiliency to failure. The ability to stay operational even in adverse conditions, and the ability to successfully recovery easily and quickly when problems do occur in a manner that reduces or eliminates the impact on the end user is critical to modern digital applications. There is an older, yet great example of building high level of resiliency into a complex software application. That is the United States Space Shuttle program. The United States Space Shuttle is a program that ran from 1981 to 2011. The space shuttle program had its problems, disastrous problems. But there was one aspect of the program that worked incredibly well...their computers and software systems. The computer systems aboard the space shuttle were state of the art when they were built. They had to be. A failure of the software on board the space shuttle would be disastrous. It was critical that the software not fail. To that end, a sophisticated system to maintain availability and reliability was employed. Let’s take the main computer system. There were four identical, redundant copies of the main computer aboard the shuttle. They consisted of identical hardware, identical inputs and outputs, and ran an identical copy of the software. Every computation the computer had to make, was made simultaneously but independently by all four computers. The results of the four computations were then compared, and the results were expected to be the same from all four computers. If they were, all was well. If, however, the results differed among the computers, then the four computers voted on which result was correct. The winning result was used, and the computers that generated the losing results were disconnected and turned off for the duration of the voyage. The losers were disabled. This is what I call the ultimate in democratic voting! The space shuttle could operate with as few as two computers operating, but the mission might be changed or shortened. But as long as two of the four computers were operating, the shuttle could be operated safely. What happened if the computers voted and the result was a tie? Well, then a fifth computer system was consulted. This system was a very different computer...a much simpler one. It had software for only the most critical calculations, and that software was written and the hardware was built by completely independent teams than the ones that built the main computer. That way, a problem that caused the main computers to fail, would not likely impact this fifth computer. If the main computers voted, disagreed, and tied, then the fifth computer would become the tie breaker. During the 30 year operation of the space shuttle, there was never a case where a serious life threatening problem occurred that was a result of a software problem, even though the software was the most complex software ever built for a space program at the time. It was a sophisticated system for high availability, especially for the 1970s when this system was first built. But sometimes its useful to look at older techniques, such as this, when deciding how to build and operate modern digital applications. We will be talking about the space shuttle software system in more detail during a main story in a later episode.…
M
Modern Digital Applications with Lee Atchison
Amazon recently announced a reduction in the cost of their Elastic Kubernetes Service. They lowered the cost of the Amazon EKS service by 50%. The new pricing is greatly appreciated, but is still a barrier to entry in using the EKS service for small clusters. Links and More Information The following are links mentioned in this episode, and links to related information: AWS EKS Pricing ( https://aws.amazon.com/blogs/aws/eks-price-reduction/ ) AWS EKS (https://aws.amazon.com/eks/) AWS ECS (https://aws.amazon.com/ecs/) Modern Digital Applications Website ( https://mdacast.com ) Lee Atchison Articles and Presentations ( https://leeatchison.com ) Architecting for Scale, published by O’Reilly Media ( https://architectingforscale.com ) Special Report Amazon recently announced a reduction in the cost of their Elastic Kubernetes Service. As of January 21, 2020, the price of Amazon EKS will be lowered by 50%. The previous price for using the EKS service was 20 cents per hour for each Kubernetes cluster managed. This works out to approximately $144 per month. The new price for running an EKS cluster will now be 10 cents per hour, which works out to approximately $72 per month. It should be noted that this is the price necessary to operate the cluster, regardless of the number of workers contained in the cluster. You must still pay for the computation costs — either EC2 instance hours or Fargate compute resources — in order for the cluster to be useful. The cost reduction is for the “overhead” cost of maintaining the cluster, which covers the Amazon cost of managing the cluster. This price has always been an issue with the EKS service. The non-Kubernetes Amazon container management service, ECS, does not have this overhead cost — all you pay for is the cost of running the workers. Only EKS has this overhead. So, the reduction of this overhead cost should be seen as significant reduction in the barrier to entry in using the EKS service. While I applaud AWS for taking this step, I still believe this overhead cost is too high. It still makes EKS less attractive than ECS for small clusters. The cost is not an issue for large clusters, since you can amortize the cost out over the size of the cluster. But if you run a large number of small clusters, this overhead cost can be significant. For example, if you are running 100 clusters, the overhead cost is around $87,000 per year. Overhead. Now that’s a lot less than the $175,000 it was before the price change, but it’s still quite a bit of overhead. That said, I applaud AWS for making this price change. It’s a consistent strategy they use — pass on cost savings to their customers as their costs go down due to volume. It’s a strategy very much appreciated by their customers. A link to the announcement is given in the show notes, along with a link to the EKS and ECS services.…
M
Modern Digital Applications with Lee Atchison
Hello and welcome to Modern Digital Applications. In this episode, we continue our three part series on Service Tiers, and how they can be used to prevent disasters in applications using service based architectures. We also take another look at Amazon S3, and ask the question, how large is S3? The answer might surprise you. All of this, in this episode of Modern Digital Applications. Links and More Information The following are links mentioned in this episode, and links to related information: Modern Digital Applications Website ( https://mdacast.com ) Lee Atchison Articles and Presentations ( https://leeatchison.com ) How Service Tiers Can Help to Avoid Microservices Disasters ( https://thenewstack.io/how-service-tiers-can-help-to-avoid-microservices-disasters/ ) Architecting for Scale, published by O’Reilly Media ( https://architectingforscale.com ) News Links to stories mentioned in the news section: * DevOps World ( https://www.cloudbees.com/devops-world ) Main Story In the last episode, we introduced the concept of service tiers, and we talked specifically about Tier 1 service tier, which is used to label the most critical, highest priority services in your application. In this episode, we’re going to talk about the definition of the rest of the four service tier levels that are defined. In review, a Tier 1 service is a service that is mission critical to your application. If a Tier 1 service is down, your application is down. A tier 1 service is a service where if it fails, it has a significant impact on your customer’s and/or your business’s bottom line After Tier 1, the next level of service is Tier 2. Tier 2 services can have an impact to your business, but are typically less critical than a Tier 1 service. A Tier 2 services is a service where a failure of the service can cause a degraded customer experience in a noticeable and meaningful way but does not completely prevent your customer from interacting with your system. A great example of a tier 2 service is a search service. This would be a service that handles requests to search by your customers for products, product information or other content from your application. If the search box on your website stopped working, it would have a major impact on customers…especially if a customer is depending on search to find something on your site…but it would not make the application completely unusable to your customers. Compare this to a tier 1 service, such as the login service. If customer’s can’t login to their application, they can’t do anything. So a tier 1 service means a failure keeps the customer from doing anything. A tier 2 service failure means they can’t perform some important functionality, but other parts of the application are still available to them. Tier 2 services can also be services that impact your backend business processes in significant ways, but may not be directly noticeable to your customers. For example, if a service that managed order processing in a fulfillment center fails, this would have a significant impact on your ability to fulfill orders in a timely manner. This would have an impact on your business, and a potential impact on your customers if a package arrived late as a result, but it doesn’t completely bring down your website or application, and customers can still do things, such as create new orders or check on the status of existing orders. A customer may not even notice this failure, yet it could still impact them. That’s for tier 2 services. Tier 3 The next level is Tier 3. A Tier 3 service is one that can have minor, unnoticeable or difficult-to-notice customer impact. Or it could have limited effects on your business and systems. The key words here are: minor, difficult-to-notice, and limited. The best example of a tier 3 service is a recommendations service. Think back to the last time you were on the website for a retail store. You click on a product to look at it. On the detail page for that product, you often will also see a list of recommended or related products that you might be interested in. You can click on those, and see and potentially purchase these alternate products. These recommendations are created by a recommendations service. Now what happens if the recommendation service is down or failing? In this case, you can still view products on the website. You can still purchase product, and you can still have that product shipped to you. It’s just that the list of alternate recommended products…the upsell products if you will…won’t be visible on the site. Most customers would not even notice the missing feature. Or if they did notice, it would be only a limited inconvenience. Yes, it’d be nice if the feature was there, and it probably does have a financial impact to not have those recommendations showing, but it isn’t a significant impact. That is a tier 3 service. Another example of a tier 3 service might be a service that pops up a message on your website when a customer first arrives, and offers to help them find something on your site. Those sorts of popups are useful, provide a customer service, and generate more business for you. But if the service that generates that popup is down, the impact to customers would be minor, and perhaps unnoticeable…since it was a nice to have not a required capability for using your site. Tier 4 The lowest level of service is a Tier 4 service. A Tier 4 service is a service that, when it fails, causes no significant effect on the customer experience and does not significantly affect the customer’s business or finances. A good example of a Tier 4 service is a the service that generates the weekly sales report for your company. Many managers in your company probably find that report important, but if they don’t get it Monday morning but rather get it Tuesday morning, it won’t have a significant impact on your business…an undoubtedly it will have no impact on your customers. The same might be true with a service that sends out marketing emails. If you send out a monthly newsletter advertising products to past customers, if that service fails, then you may have to delay sending out the monthly newsletter. Yes, that would have an impact to you, but probably not a significant impact on the business and customer’s probably wouldn’t even notice the delay at all. Tech Tapas Amazon does not disclose much information on the size of any of the AWS services. However, in June of 2012, Amazon disclosed that S3 just reached a huge milestone by storing it’s one trillionth object. That’s one trillion objects. One million million objects. Let’s try and put that into perspective with some comparisons. This is equivalent to 142 objects for every man, woman, and child on the entire planet. Put another way, that’s 10 objects for every man, woman, and child who has ***ever*** lived, ever. Or, a bit over three objects for every star in the Milky Way Galaxy. That’s a lot of objects. But how many objects does it store today? Amazon has not released any updated numbers, but given the growth of AWS as a whole since 2012, it’s not unrealistic to think this number may be multiple orders of magnitude higher than that. AWS advertises a durability for S3 of 99.999999999%. That’s 11-9’s. Assuming that is accurate, that means of the one trillion objects stored in S3 in 2012, AWS would have lost at most ten objects a year. That’s amazingly resilient. I think it’s safe to say, without too much concern of exaggeration, that Amazon S3 is perhaps the most provably scalable, reliable, durable service ever built. Which, of course, is why it continues to grow...and scale...…
M
Modern Digital Applications with Lee Atchison
Show Notes Welcome to the inaugural episode of Modern Digital Applications! I’m very glad you are listening, and I hope you’ll find this podcast informative and helpful. My goal is to try and keep the episodes short, so that they can be consumed during a single morning average commute trip to work. Please, let me know how I am doing and what I can to improve. But, let’s get started. In this episode, we begin a three part series on Service Tiers, and how they can be used to prevent disasters in applications using service based architectures. We also take a look at Amazon S3, and the history of SaaS. Links and More Information The following are links mentioned in this episode, and links to related information: Modern Digital Applications Website ( https://mdacast.com ) Lee Atchison Articles and Presentations ( https://leeatchison.com ) Architecting for Scale — O’Reilly Media ( https://architectingforscale.com ) How Service Tiers Can Help to Avoid Microservices Disasters ( https://thenewstack.io/how-service-tiers-can-help-to-avoid-microservices-disasters/ ) Architecting for Scale, published by O’Reilly Media ( https://architectingforscale.com ) Main Story Bringing down an entire application is easy. All it takes is the failure of a single service and the entire set of services that make up the application can come crashing down like a house of cards. Just one minor error from a non-critical service can be disastrous to the entire application. There are, of course, many ways to prevent dependent services from failing. However, adding extra resiliency in non-critical services also adds complexity and cost, and sometimes that extra cost is not needed. What if a service, let’s call it Service A, is consuming another service, let’s call it Service B. If the called Service, Service B, is not critical to the operation of the calling Service, Service A, then why should Service A fail if Service B has a problem? Surely, we should be able to build Service A so that it can survive the failure of Service B. And if Service B is not critical to the functioning of Service A, does Service B need to have the same level of resiliency has Service A? No, of course not. As we build our dependency map between each of our services and the services they depend on, we will find that some of the dependencies are critical dependencies, and some of them are non-critical dependencies. How do we determine which dependencies are critical and which are not critical? One important tool for making this determination is to use something called Service Tiers. What Are Service Tiers? A service tier is simply a label associated with a service that indicates how critical a service is to the operation of your business. Service tiers let you distinguish between services that are mission critical, and those that are useful and helpful but not essential. Service tiers can be used to determine whether the interaction between dependent services is a critical dependency, or a non-critical dependency. Service Tiers are a great way to help you prioritize where and how you invest in making your services, and their dependencies, more resilient. This allows you to build higher scaled applications that are much more highly available for the same amount of effort. To see how this works, let’s take a look at the various service tier labels and how you can determine which label to apply to which services. Assigning Service Tiers All services in your system, no matter how big or how small, should be assigned a service tier. In the model of service tiers that I use and recommend, there are four distinct tiers, four distinct levels if you will, that allow you to specify the criticalness of a service. Let’s talk about each of these four levels. Tier 1 The highest level is Tier 1. Tier 1 services are the most critical services in your system. A service is considered Tier 1 if a failure of that service will result in a significant impact to customers or to the company’s bottom line. Let me repeat that: A service is Tier 1 if a failure of that service will result in a significant impact to customers or to the company’s bottom line. Significant impact to customer’s. Or significant impact to the company’s bottom line business. If a service can impact either one, then it’s a Tier 1 service. What are some example Tier 1 services? A Login service: This is a service that provides a method for people to login to your application. This is tier 1 because if a customer can’t login to your applications, it creates a significant negative impact to your customers. Credit card processing service. This is the service that handles customer payments. If customer’s are unable to have credit card payments processed, then this would have an impact on your company’s bottom line. Permission service: This would be a service that tells you what features a given user may have access to. If this service is unavailable, then customer’s won’t be able to get access to the features of your application that they have access to. Order-accepting service: This would be a service that lets customers purchase a product on your website. Obviously, if customers can’t purchase product, that impacts your bottom line, as well as the customer experience! A failure of a Tier-1 service is a serious concern to your company. In all, there are four service tier levels that we have specified. Tier 1, as we discussed above, is used for the most critical of your services. The remaining three tiers, Tier 2, Tier 3, and Tier 4, are for progressively lower priority services. In our next episode, we will be talking about each of these other service tier levels and what they mean. After we have defined the four tiers, we will discuss how we can utilize service tiers to make our application better, more reliable, more available, and for less money. If you want to learn more about service tiers, you can read about them in my book, Architecting for Scale, or in my article published in The New Stack titled “How Service Tiers Can Help Avoid Microservice Disasters”. Links to both of these sources is available in the show notes. Tech Tapas - Amazon S3 Amazon S3. A highly durable, highly available file and object storage mechanism in the cloud. This service is the go to service for most companies that want to store huge quantities of data in the cloud, or for long term persistent object storage. S3 was designed with the goals of being highly available, highly durable, and highly scalable. The design goal for availability is 99.99%, with a durability of objects of 99.999999999 (that’s 11 9’s). How available? The 4 9’s availability translates to a total of 52 minutes of downtime per year. How durable? The 11 9’s durability means that if every man, woman, and child in the world had an object in S3, then Amazon would lose at most one of those objects, approximately once every 15 years. These are amazing goals, and is one of the reasons S3 has such a great reputation as a high quality object storage system. S3 was one of three initial AWS services and was a big part of AWS’s early success. Tech Tapas - History of the Term SaaS When did software as a service start? Well, that depends on what you mean by the term… depending on how you define SaaS, the answer is either the early 1960s, or somewhere around 2005. Back in the early days of computing, all applications ran on a centralized computer. Users accessed the computers remotely. Initially via punch cards and later via remote terminals. The centralized nature of the application is, by a true definition, Software as a Service. But the modern definition of SaaS is tied much more closely with cloud computing. SaaS now-a-days refers to software running centrally, typically in a public or private cloud environment, and is shared among multiple users. A thin client of some sort — either a web browser or a thin mobile application — is used to front the centralized application. From a business model standpoint, users don’t buy SaaS software, instead they rent or lease access to it with monthly or annual fees. Alternatively, the service could be free and supported by advertising or other monetization processes. This is the heart of the business model for social media, for example. So, SaaS is an old term that has been given new meaning in recent years. But it’s the recent definition that has really changed the way people think and build software today.…
Welcome to Player FM!
Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.
Daily Deals
Similar to Modern Digital Applications with Lee Atchison
Android Backstage, a podcast by and for Android developers. Hosted by developers from the Android engineering team, this show covers topics of interest to Android programmers, with in-depth discussions and interviews with engineers on the Android team at Google. Subscribe to Android Developers YouTube → https://goo.gle/AndroidDevs
…
continue reading
The power of Data is undeniable. And unharnessed - it’s nothing but chaos. Making data your ally. Using it to lead with confidence and clarity. Host Jess Carter is solving problems in real-time to reveal what’s possible. Helping communities and people thrive. This is Data Driven Leadership, a show brought to you by Resultant.
…
continue reading
The American healthcare system is one of the most innovative in the world. But it’s also riddled with complex challenges, such as access to affordable medications, inefficiency and administrative burdens, and communication barriers between providers. There’s clearly a better way—and at Surescripts, we have a unique sightline into what that may be. In this series, host Melanie Marcus, Chief Marketing Officer of Surescripts, sits down with today’s most inspiring and innovative leaders in healt ...
…
continue reading
Show notes are at https://stevelitchfield.com/sshow/chat.html
…
continue reading
Investor Shayle Kann is asking big questions about how to decarbonize the planet: How cheap can clean energy get? Will artificial intelligence speed up climate solutions? Where is the smart money going into climate technologies? Every week on Catalyst, Shayle explains the world of climate tech with prominent experts, investors, researchers, and executives. Produced by Latitude Media.
…
continue reading
1
Defensive Security Podcast - Malware, Hacking, Cyber Security & Infosec
Jerry Bell and Andrew Kalat
5k200
Defensive Security is a weekly information security podcast which reviews recent high profile cyber security breaches, data breaches, malware infections and intrusions to identify lessons that we can learn and apply to the organizations we protect.
…
continue reading
Every Friday and Sunday, Slate’s popular daily news podcast What Next brings you TBD, a clear-eyed look into the future. From fake news to fake meat, algorithms to augmented reality, Lizzie O’Leary is your guide to the tech industry and the world it’s creating for us to live in.
…
continue reading
Material is a weekly discussion about the Google and Android universe. Your intrepid hosts try to answer the question, “What holds up the digital world?” The answer, so far, is that it’s Google all the way down. Hosted by Andy Ihnatko and Florence Ion.
…
continue reading
TechStuff is getting a system update. Everything you love about Tech Stuff now twice the bandwidth with new hosts, Oz Woloshyn (Sleepwalkers) and Karah Preiss (Sleepwalkers). Oz and Karah bring humour and wit to the table as they break down what's happening in tech...and what it says about us. TechStuff is the podcast where technology meets culture. We speak to the folks building the future to understand what tomorrow will look like and how our technology is changing us: how we live, how we ...
…
continue reading
A tech podcast for the gadget lovers and tech heads among us from the mind of Marques Brownlee, better known as MKBHD. MKBHD has made a name for himself on YouTube reviewing everything from the newest smartphones to cameras to electric cars. Pulling from over 10 years of experience covering the tech industry, MKBHD and co-hosts Andrew Manganelli and David Imel will keep you informed and entertained as they take a deep dive into the latest and greatest in tech and what deserves your hard earn ...
…
continue reading
Player FM - Podcast App
Go offline with the Player FM app!
Go offline with the Player FM app!
