Scalawags #17 - Shapeless

thescalawags's podcast

our guest today is Miles Sabin, author of Shapeless

• https://twitter.com/milessabin
• https://github.com/milessabin/shapeless

Intro (0:00)

• Shapeless has a theme tune, provided by Ed Kmett and featuring Bruce Lee
  • https://www.youtube.com/watch?v=pGFf3SRP1bE
• Sabin is pronounced “Sabin”
• Josh catches and eats raccoons, as does nearly everyone in western Pennsylvania

What is Shapeless? (5:00)

• Shapeless is just HLists, right?
• Shapeless is just a “showoff project”, right?
• Miles’ dog is heard from and briefly seen
  • Or is it a guest dog?
• Miles is shapeless, Dick is shameless

What is an HList? (9:50)

• HList crosses tuples with lists:
  • H is for heterogeneous
  • Variable length like list
  • Each element has its own compile-time type, like a tuple
• The simplest possible HList implementation is just 6 lines of Scala
• They're useful for abstracting over arity
  • e.g. of tuples, of case classes, of functions

Generic programming (15:00)

• Shapeless began with Miles trying to do generic programming, a la Haskell’s Scrap Your Boilerplate, in Scala
  • http://www.haskell.org/haskellwiki/Generics
  • http://www.haskell.org/haskellwiki/Scrap_your_boilerplate
• Generic programming is “the kind of polymorphism your language doesn’t support yet”?
• Generic programming is “second-class polymorphism”?
• Generic programming abstracts over the “shape” of diverse data types
• Arity is an example; so is depth of nesting
• Generic tree transformation, generic folds on trees, serialization and deserialization. XML, JSON.
• The best of both worlds: avoiding boilerplate while staying type-safe
• Michael Pilquist's scodec library uses Shapeless to do binary serialization and deserialization
  • https://github.com/scodec/scodec

HLists again (26:05)

• there are other HList implementations around:
  • Jesper Nordenburg's Metascala
  • https://www.assembla.com/spaces/metascala/wiki
  • Mark Harrah and Runar Bjarnason's blog series
  • http://apocalisp.wordpress.com/2010/06/08/type-level-programming-in-scala/
  • Mark's HLists inside sbt, now overseen by Josh
  • there's HList code in Josh's book Scala in Depth, too
  • http://www.manning.com/suereth/
• sbt has AList and KList too
• Josh: are the HLists in Shapeless optimized for some particular uses?

Generic programming again (30:00)

• again, HList lets you abstract over arity
• example: composing validations
  • Scalaz internally has boilerplate code for this for a fixed set of arities only
• Miles disclaims any desire to reinvent Scalaz

KList and AList (34:15)

• KList requires the element types to share an outer type constructor
• AList allows multiple type constructors
• Daniel says KList and AList are unsound in Scala because of variance
  • there’s an “unchecked-variance loophole”
• Shapeless doesn't have AList and doesn't need KList
  • in Shapeless a KList is just an HList with a constraint
  • the constraint is maintained by requiring an implicit witness
• Shapeless uses Poly for mapping functions over HLists
• Daniel says Lars Hupel says there are problems combining Poly with Scalaz’s Unapply?
  • Miles is skeptical, asks for gist
• Josh wonders if sbt ought to unify HList, KList, and AList too
• what's the trick for unifying them? Poly!

Polymorphic functions (43:10)

• Poly is a polymorphic function (a function whose behavior varies based on the input type)
• In Haskell, the varying behavior is achieved using typeclasses. Thanks SPJ!
• Poly is like a “locally defined typeclass”, defined for a finite set of (quantified) types
• A natural transformation maps F[A] to G[A]. It's a particular kind of polymorphic function.
• Miles has yet to write part 3 of his blog post series on polymorphic functions
  • part 1: http://www.chuusai.com/2012/04/27/shapeless-polymorphic-function-values-1/
  • part 2: http://www.chuusai.com/2012/05/10/shapeless-polymorphic-function-values-2/
  • part 3 is the Duke Nukem Forever of type-level programming
• A Poly can be mapped over an HList

Algebraic data types (47:00)

• But we don't just care about HLists
  • HList is a particular data “shape”
  • Shapeless aims to be generic over all shapes
• “Algebraic” data types
  • Sum types (e.g. Either, sealed traits) and product types (e.g. tuples, case classes)
  • Complex data types can be understood (and processed generically) as recursively nested sums and products
• Coproducts
• We're all writing boilerplate code for folds and transforms all the time
• If we could do it generically, our code would be smaller

Performance, compile-time and runtime (56:00)

• Would our smaller code also be slower?
  • Expressiveness vs. performance?
• Daniel: Java 8 JVM helps
• Dick: can you have a million-element HList?
• Dale somebody asks: compile-time performance penalty, runtime performance penalty, or both?
• Shapeless tests often test whether something compiles, rather than what the answer is
• discussion on what factors affect Scala compile times
  • Daniel: don't scapegoat cake pattern
  • Josh: if your compile times are too long, don't obsess over which language features or patterns compile slowly; concentrate on architecture and controlling dependencies

Labels and lenses (1:05:05)

• Dick wants to hear about lenses
• Miles needs to explain Generic (formerly Iso) first
• Generic is a small macro, one of only a few in Shapeless
• Generic maps back and forth between case class types and HList types
• “Labels”, brand new in recent months
  • Adding labels to Generic gives LabeledGeneric
  • In a sum type (family of case classes), the labels are the class names
  • In a product type (particular case class), the labels are the field names
  • You need the names to do serialiation and deserialization
• New Shapeless lenses built on top of LabeledGeneric
  • Try them in Shapeless 2.1-SNAPSHOT
• Scala's SelectDynamic is used also
• Miles also recommends looking at Monocle
  • by Julien Truffaut
  • https://github.com/julien-truffaut/Monocle
  • Attempts to translate Ed Kmett's Haskell lens library to Scala
• Miles would like to take Shapeless lenses in the same direction
  • But doing so would involve depending on Scalaz
  • Scalaz good, more dependencies bad. Dilemma
  • Shapeless lenses may become a separate project, with the Scalaz dependency
• (we apologize for all the beeping in this section and the next)

Listener Q&A (1:17:05)

• (we apologize for losing the askers’ names)
• “What happened to unboxed tagged types?”
  • Miles: See migration guide. They've moved.
  • Tagged types are like newtype in Haskell
  • Labels are implemented using tagged types, with the singleton types of strings
• “What’s the Haskell equivalent to Shapeless?”
  • GHC.Generics for generic programming
  • http://www.haskell.org/haskellwiki/GHC.Generics
  • Andres Löh, http://www.andres-loeh.de
• “Is anything in Shapeless flaky and I should wait before using it?”
  • Miles: the way you define Polys inline is likely to change.
  • But relax because if anything goes wrong it will go wrong at compile time.
• “What is the Either-like structure you mentioned that we get from a sealed trait?”
  • Miles: That’s the coproduct. It only works if what you’re doing is just like an ADT in Haskell.
  • Obligatory animal example
  • HLists are nested pairs, coproducts are nested Eithers
  • Daniel: It's Lisp at the type level.
• SHAPELESS IS ABOUT GENERIC PROGRAMMING NOT HLISTS
• Miles enjoys doing long rambly Shapeless workshops
• Miles will speak at Scala Days in Berlin, with Jon Pretty, about 10 years of doing Scala
• Josh fights Bruce Lee and guess what: Bruce wins