3factor app - Reliable Eventing

Coding Blocks

We discuss the second factor of Hasura’s 3factor app, Reliable Eventing, as Allen says he still _surfs_ the Internet (but really, does he?), it’s never too late for pizza according to Joe, and Michael wants to un-hear things.

This episode’s full show notes can be found at https://www.codingblocks.net/episode116, just in case you’re using your podcast player to read this.

Sponsors

• Datadog.com/codingblocks – Sign up today for a free 14 day trial and get a free Datadog t-shirt after creating your first dashboard.
• O’Reilly Velocity Conference – Get expert insight on building and maintaining cloud native systems. Use the code BLOCKS during registration to get 20% off of most passes.
• Educative.io – Level up your coding skills, quickly and efficiently. Visit educative.io/codingblocks to get 20% off any course.

Survey Says …

What's the first thing you do when picking up a new technology or stack?

Take the survey here: https://www.codingblocks.net/episode116. News

• Thank you to everyone that left us a review:
  • iTunes: !theBestCoder, guacamoly, Fishslider
  • Stitcher: SpottieDog
• We have photographic evidence that we were in the same room with Beej from Complete Developer at Atlanta Code Camp.

The Second Factor – Reliable Eventing

• Don’t allow for mutable state. Get rid of in memory state manipulation in APIs.
• Persist everything in atomic events.
• The event system should have the following characteristics:
  • Atomic – the entire operation must succeed and be isolated from other operations.
  • Reliable – events should be delivered to subscribers at least once.

Comparing the 3factor app Eventing to Traditional Transactions Traditional application 3factor application Request is issued, data loaded from various storage areas, apply business logic, and finally commit the data to storage. Request is issued and all events are stored individually. Avoid using async features because it’s difficult to rollback when there are problems. Due to the use of the event system, async operations are much easier to implement. Have to implement custom recovery logic to rewind the business logic. Recovery logic isn’t required since the events are atomic. Benefits of an Immutable Event Log

• Primary benefit is simplicity when dealing with recovery. There’s no custom business logic because all the event data is available for replayability.
• Due to the nature of persisting the individual event data, you have a built in audit trail, without the need for additional logging.
• Replicating the application is as simple as taking the events and replaying the business logic on top of them.

Downsides of the Immutable Event Log

• Information isn’t (instantly) queryable, not taking into account snapshotting.
  • CQRS (command query responsibility segregation) helps to answer this particular problem.
• Forcing event sourcing on every part of the system introduces significant complexity where it may not be needed.
• For evolving applications, changing business needs require changes to the event schema and this can become very complex and difficult to maintain.
  • Upcasting: converting an event record on the fly to reflect a newer schema. Problem with this is you’ve now defeated the purpose of immutable events.
  • Lazy upcasting is evolving the event records over time, but that means you’re now maintaining code that knows how to understand each version of the event records in the system, making it very difficult to maintain.
  • Converting the entire set of data any time a schema needs to change. Keeps things in sync but at a potentially large cost of taking the hit to update everything.
• Considerations of event granularity, i.e. how many isolated events are too much and how few are not enough?
  • Too many and there won’t be enough information attached to the event to be meaningful and useful.
  • Too few and you take a major hit on serialization/deserialization and you run the risk of not having any domain value.
  • So what’s the best granularity? Keep the events closely tied to the DDD intent.
• Fixing bugs in the system may be quite a bit more difficult than a simple update query in a database because events are supposed to be immutable.

Resources We Like

• 3factor app (3factor.app)
• 3Factor Canonical App (GitHub)
• Atomic commit (Wikipedia)
• DDD & co., part 2: Semantics over CRUD (thenativeweb.io)
• Event Sourcing (martinFowler.com)
• CQRS (martinFowler.com)
• What they don’t tell you about event sourcing (medium.com)
• A Practical Guide to GraphQL: From the Client Perspective (educative.io)

Tip of the Week

• Use docker system to manage your Docker environment.
  • Use docker system df to see the the disk usage.
  • Use docker system prune to clean up your environment.
• How To View Clipboard History On Windows 10 (AddictiveTips.com)
• Use docker-compose down -v to also remove the volumes when stopping your containers.
• Intel 660p M.2 2TB NVMe PCIe SSD (Amazon)
• Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems (Amazon)