Rajat Monga is a director of engineering at Google where he works on TensorFlow. TensorFlow is a framework for numerical computation developed at Google. The majority of TensorFlow users are building machine learning applications such as image recognition, recommendation systems, and natural language processing–but TensorFlow is actually applic ...…

There are hundreds of different databases. There are tens of continuous delivery products. There is an ocean of cloud providers and CRM systems and monitoring platforms and sales prospecting tools. The range of available software products is so diverse that it can be overwhelming to figure out which products to buy. Siftery is a company that wa ...…

A message broker is an architectural component that sends messages between different nodes in a distributed system. Message brokers are useful because the sender of a message does not always know who might want to receive that message. Message brokers can be used to implement the “publish/subscribe” pattern, and by centralizing the message work ...…

Stripe processes payments for thousands of businesses. A single payment could involve 10 different networked services. If a payment fails, engineers need to be able to diagnose what happened. The root cause could lie in any of those services. Distributed tracing is used to find the causes of failures and latency within networked services. In a ...…

Bitcoin and Bitcoin Cash are two cryptocurrencies with similar properties. But the supporters of each of these Bitcoin versions have strongly divergent opinions on the direction of the Bitcoin project. At the center of this debate is the subject of block size. Bitcoin’s block size determines how many transactions fit into each block that is min ...…

Most new frontend webapps today use ReactJS. An increasing number of mobile apps are created using the cross-platform components of React Native. GraphQL, Facebook’s open source data-fetching middleware tool is being used by more and more companies, who are finding that it simplifies their development. Facebook’s open source suite of technologi ...…

Machine learning tools are rapidly maturing. TensorFlow gave developers an open source version of Google’s internal machine learning framework. Cloud computing provides a cost effective, accessible way of training models. Edge computing allows for low latency deployments of models. But even if you are a kid with a laptop who has learned all the ...…

Let’s Talk Bitcoin is one of the most popular podcasts about cryptocurrencies. Adam B. Levine started it after three other podcasts he started did not get the traction he had hoped for. Adam parlayed the success of Let’s Talk Bitcoin into a network of podcasts–the Let’s Talk Bitcoin Network–which also includes one of my favorite shows, Epicente ...…

Monitoring a Kubernetes cluster allows operators to track the resource utilization of the containers within that cluster. In today’s episode, Ilan Rabinovitch joins the show to explore the different options for setting up monitoring, and some common design patterns around Kubernetes logging and metrics gathering. Ilan is the VP of product and c ...…

Laura Shin is the host of Unchained, a podcast about cryptocurrencies and decentralized technology. For every episode, Laura does significant research and preparation, so the content turns out polished and high quality. Her enthusiasm for the subject of cryptocurrencies comes through in her reporting. Podcasting about cryptocurrencies requires ...…

Social networks can make you feel connected to a global society. But those social networks are controlled by a corporate entity. The profit motivations of the corporation are not directly aligned with the experience of the users. Mastodon is an open source, decentralized social network. Eugen Rochko started building Mastodon in response to his ...…

Go is a language designed to improve systems programming. Go includes abstractions that simplify aspects of low level engineering that are historically difficult—concurrency, resource allocation, and dependency management. In that light, it makes sense that the Kubernetes container orchestration system was written in Go. Erik St. Martin is a cl ...…

Tammy Butow has worked at Digital Ocean and Dropbox, where she built out infrastructure and managed engineering teams. At both of these companies, the customer base was at a massive scale. At Dropbox, Tammy worked on the database that holds metadata used by Dropbox users to access their files. To call this metadata system simply a “database” is ...…

Software engineers have interacted with operations teams since software was being written. In the 1990s, most operations teams worked with physical infrastructure. They made sure that servers were provisioned correctly and installed with the proper software. When software engineers shipped bad code that took down a software company, the operati ...…

The Interplanetary File System (IPFS) is a decentralized global, peer-to-peer file system. IPFS combines ideas from BitTorrent, Git, and Bitcoin, creating a new way to store and access objects across the Internet. When you access an object on almost any website, you are accessing the object via a location address—a URL. The URL tells you where ...…

The Ethereum community started as a small group of dedicated engineers. It has ballooned to thousands of engineers, entrepreneurs and investors, all of whom have a stake in the direction of Ethereum. Ethereum is an open source project, and the direction of a popular open source project can get complex. Ethereum is figuring out how to govern its ...…

The pubsub pattern allows a developer to create channels, which messages can be written to and read from. Pubsub messaging is useful for multicast messaging–when you want to publish messages from a producer, and have multiple consumers who are subscribed to the publisher receive those messages. Almost any application that reaches a high level o ...…

Most technology companies rely on open source software projects. But open source software projects are often maintained by a group of people that is not affiliated with any particular company. When an open source project develops too much technical debt, it can become a tragedy of the commons. Who is responsible for maintaining these open sourc ...…

Zcash is a payment and consensus system that allows users to transfer money to each other with strong guarantees of privacy. Zcash implements the same core features of Bitcoin, with the added functionality of shielded payments. Shielded payments are private, and they are enabled by a novel cryptographic technique called zk-SNARKS: zero knowledg ...…

“The Federal Reserve System is fraudulent. Whatever its stated purpose, its effective purpose is to create a mechanism of deficit spending by politicians, through the insidious invisible taxation of monetary debasement (aka inflation).” These are the words of Erik Voorhees, the CEO of crypto financial exchange ShapeShift. Long before he started ...…

We sign many different types of contracts throughout our lives. We sign a mortgage to get a loan for a house. When we go to the hospital, we sign a piece of paper that defines how our medical data can be shared between organizations. These pieces of paper represent our opting into an agreement that will be mediated and enforced by computer inte ...…

Ethereum is a system for running decentralized smart contracts. In the current implementation of Ethereum, every smart contract gets deployed to every full node. Whenever a user wants to call a smart contract, that smart contract gets executed on each full node–across the entire network. The current model for smart contract execution needs to b ...…

Soumya Basu is a PhD student at Cornell, where he studies distributed systems problems associated with cryptocurrencies. Soumya is advised by Emin Gun Sirer, a Cornell professor who previously appeared on the show to discuss smart contract security. Soumya joins the show today to talk about a variety of issues in the cryptocurrency space. We fi ...…

Consensus protocols are used to allow computers to work together. A consensus protocol lets different servers agree on the state of a system. For decades, these protocols have been used to establish consensus among database nodes, application servers, and other infrastructure that runs within an enterprise. More recently, new consensus protocol ...…

The DAO was a system of smart contracts on the Ethereum blockchain that investors put millions of dollars into. Back in May 2016, it was the largest crowdfunding event in history, and we discussed it in detail in a previous episode with Matt Leising. The DAO was hacked due to a security vulnerability, and this event led to a hard fork of Ethere ...…

Data streams about the weather can be used to predict how soybean futures are going to change in price. Satellite data streams can take pictures of the number of cars on the road, and judge how traffic patterns are changing. Search engines can aggregate data from different queries and determine what people are most interested in. Data streams d ...…

To use a web application, you probably open a web browser or a mobile app. To access an Ethereum application, many people use an Ethereum browser. In previous episodes, we explored Metamask and Mist, which are Ethereum browsers for the desktop. In today’s episode, we explore Status, a mobile Ethereum browser. Status founders Jarrad Hope and Osk ...…

Anthony Diiorio was involved with Ethereum since the earliest days. He was one of the first people to see the Ethereum ideas presented by Vitalik Buterin, and he invested deeply in Ethereum–both financially and by helping to establish the early Ethereum community. Anthony started Decentral in 2014, which is a hub for his projects in the cryptoc ...…

A financial exchange is an operationally intensive business. You have customers making a high volume of transactions, your service has to be low latency and highly available, and you are dealing with a lot of money. A cryptocurrency exchange has all of the complexity of a typical financial exchange–and then some additional complexity. ShapeShif ...…

Cryptocurrency speculation has pulled in a large population of people who do not know what they are investing in. If you hear about an investment of $1000 turning into $1M, it’s tempting to get sucked in yourself. For most of these everyday people, the game is completely rigged. A large percentage of market activity is driven by “pump and dumps ...…

In the finance industry, many people have a computer on their desk called a Bloomberg terminal. A Bloomberg terminal contains news, stock prices, communication tools, and other features that make it worth a high subscription price. And people in finance can afford to pay that high subscription because their decisions can cause a gain or loss of ...…

Most applications today run on a cloud provider like AWS. They are built with a framework like Ruby on Rails. They use a set of APIs like Stripe and Twilio for middleware services. This is the era of “web 2.0.” With decentralized systems, we are starting to get a feel for what “web 3.0” might feel like. The futuristic idea of “web 3.0” works of ...…

Decentralized applications can be built on the Ethereum blockchain. Just as the Bitcoin blockchain is a distributed, append-only ledger of financial transaction history, Ethereum is a distributed, append-only ledger of computational transaction history. New kinds of applications can be built on the Ethereum blockchain—and just like every new te ...…

Decentralized applications might someday offer alternatives to modern monopolies. Uber, Airbnb, Facebook, Amazon—all of these services could be recreated on a decentralized stack of technologies like Ethereum, IPFS, and Golem. Fully decentralized services could be more transparent, cheaper, and more efficient. But let’s be realistic. Today, eve ...…

For a decade, Bitcoin’s proof-of-work system has run without disruption. In a proof-of-work scheme, Bitcoin miners compete to solve a cryptographic puzzle associated with a block of transactions. Every ten minutes, all the Bitcoin miner nodes race to be the first to solve a block of transactions. Only one miner wins each block, meaning the othe ...…

Humans organize into groups. There are lots of group types: religions, corporations, national governments, state governments, citizenries, clubs, musical bands. Every group has governance. Governance defines the rules, and the ways that rules change. The United States requires citizens to pay taxes. A corporation requires you to show up to work ...…

Smart contracts are programs that run on the Ethereum blockchain. A smart contract developer pays Ether to deploy the contract. When a contract is deployed, every full node on the Ethereum blockchain has a copy of the contract code in that node’s address space. Every full node needs to hold a copy of every smart contract. This allows every full ...…

Joseph Bonneau is co-author of Bitcoin and Cryptocurrency Technologies, a popular textbook. At NYU, he works as an assistant professor exploring cryptography and security. His YouTube lessons teaching Bitcoin have hundreds of thousands of views. His material offers clear explanations of how Bitcoin works. Since Joseph has a clear understanding ...…

Farms have lots of data. A corn farmer needs to monitor the chemical composition of soil. A soybean farmer needs to track crop yield. A chicken farmer needs to count the number of eggs produced. If this data is captured, it can be acted upon—for example, a dry farm can automatically turn up its irrigation system. Or the data can simply be studi ...…

Dogecoin was started in 2013 as a joke. Jackson Palmer forked Bitcoin and created his cryptocurrency as a play off the “doge” meme. The currency became popular as a means of reddit users “tipping” each other. If I made a comment on reddit that you liked, you might send me some Dogecoin. This use case allowed people to share the idea of Dogecoin ...…

There are two factors that limit the rate at which transactions are accepted into the Bitcoin blockchain: block time and block size. Block time defines how often a new block is appended onto the blockchain. Block size defines how many transactions fit into a new block. As of March 2018, the current block time and block size allow for about 7 tr ...…

Bitcoin is an immutable, append-only blockchain ledger that reaches consensus through proof-of-work. The contents of the ledger are financial transactions–people sending and receiving Bitcoin currency to each other. Since Bitcoin, there have been other cryptocurrencies that have similar properties–like Ethereum and the IPFS/Filecoin system. Sim ...…

The easiest way to train a computer to recognize a picture of cat is to show the computer a million labeled images of cats. The easiest way to train a computer to recognize a stop sign is to show the computer a million labeled stop signs. Supervised machine learning systems require labeled data. Today, most of that labeling needs to be done by ...…

Apache Spark is a system for processing large data sets in parallel. The core abstraction of Spark is the resilient distributed dataset (RDD), a working set of data that sits in memory for fast, iterative processing. Matei Zaharia created Spark with two goals: to provide a composable, high-level set of APIs for performing distributed processing ...…

Steve Herrod led engineering at VMWare as the company scaled from 30 engineers to 3,000 engineers. After 11 years, he left to become a managing director for General Catalyst, a venture capital firm. Since he has both operating experience and a wide view of the technology landscape as an investor, he is well-equipped to discuss a topic that we h ...…

On Software Engineering Daily, we have been covering the “serverless” movement in detail. For people who don’t use serverless functions, it seems like a niche. Serverless functions are stateless, auto-scaling, event driven blobs of code. You might say “serverless sounds kind of cool, but why don’t I just use a server? It’s a paradigm I’m used t ...…

Earlier this year we did several shows about Cloud Foundry, followed by several shows about Kubernetes. Both of these projects allow you to build scalable, multi-node applications–but they serve different types of users. Cloud Foundry encompasses a larger scope of the application experience than Kubernetes. Kubernetes is lower level, and is act ...…

Kafka is at the center of modern streaming systems. Kafka serves as a database, a pubsub system, a buffer, and a data recovery tool. It’s an extremely flexible tool, and that flexibility has led to its use as a platform for a wide variety of data intensive applications. Today’s guest is Gwen Shapira, a product manager at Confluent. Confluent is ...…

Streaming architecture defines how large volumes of data make their way through an organization. Data is created at a user’s smartphone, or on a sensor inside of a conveyor belt at a factory. That data is sent to a set of backend services that aggregate the data, organizing it and making it available to business analysts, application developers ...…

When you go to a website where a video is playing, and your video lags, how does the website know that you are having a bad experience? Problems with video are often not complete failures–maybe part of the video loads, and plays just fine, and then the rest of the video is buffering. You have probably experienced sitting in front of a video, wa ...…