Jos Dirksen

Full-stack developer.

Scalaz features for everyday usage part 2: Monad Transformers and the Reader Monad

13 minute read

For the second article of the "Scalaz features for everyday usage" we'll look at the subject of Monad transformers and the Reader monad.Let's start with Monad Transformers. Monad transformers come in handy when you have to deal with nested Monads, which happens suprisingly often. For instance when you have to work with nested Future[Option] or Future[Either], your for-comprehensions can quickly become unreadable, since you have to handle the None and Some cases for an Option and the Success and Failure cases explicitly. In this article I'll show some examples where Monad transformers come in handy, and how you can work with these.

The following articles are currently available in this series:

Working without Monad transformers

Like we mentioned in the introduction, Monad transformers are really useful when working with nested Monads. However, when would you encounter these? Well, a lot of Scala database libraries tend to be async (using Futures) and in some cases return Options. For instance you might query for a specific record which returns an Future[Option[T]]:

# from Phantom cassandra driver (without implicits), which returns Some(Record) if found
# or None if nothing can be found
def one(): Future[Option[Record]]

# or you might want to get the first element from a Slick query
val res : Future[Option[Row]] = db.run(filterQuery(id).result.headOption)

Or you might just have your own trait or service defining functions that will eventually return an Option or an Either as a result:

# get an account, or none if no account is found
def getAccount() : Future[Option[Account]]

# withdraw an amount from an account, returning either the new amount in the account
# or a message explaining what went wrong
def withdraw(account: Account, amount: Amount) : Future[\/[String, Amount]]

Lets look at an example piece of ugly code you would get when not using Monad transformers:

def withdrawWithoutMonadTransformers(accountNumber: String, amount: Amount) : Future[Option[Statement]] = {
  for {
    // returns a Future[Option[Account]]
    account <- Accounts.getAccount(accountNumber)
    // we can do a fold, using scalaz for the typed None, since a None isn't typed
    balance <- account.fold(Future(none[Amount]))(Accounts.getBalance(_))
    // or sometimes we might need to do a patten match, since we've got two options
    _ <- (account, balance) match {
      case (Some(acc), Some(bal)) => Future(Accounts.withdraw(acc,bal))
      case _ => Future(None)
    }
    // or we can do a nested map
    statement <- Future(account.map(Accounts.getStatement(_)))
  } yield statement
}

As you can see when we've got to work with nested Monads, we need to handle the nested one in the right side of each step in the for-comprehensions. Scala is rich enough in its language that we've got plenty of different way to do this, but the code doesn't get more readable. We have to resort to nesting maps or flatmaps, using folds (in the case of an Option) or sometimes have to resort to pattern matching, if we're interested in multiple Options. There are probably other ways to do this, but all in all, the code doesn't get more readable. Since we have to deal with the nested Option explicitly.

Now with Monad transformers

With Monad transformers we can remove all this boilerplate, and get a very convenient way of working with these kind of nested constructs. Scalaz provides Monad transformers for the following types:

BijectionT
EitherT
IdT
IndexedContsT
LazyEitherT
LazyOptionT
ListT
MaybeT
OptionT
ReaderWriterStateT
ReaderT
StateT
StoreT
StreamT
UnWriterT
WriterT

While some of these might seem a bit exotic, ListT, OptionT, EitherT, ReaderT and WriterT can be applied in whole lot of use cases. In this first example lets focus on the OptionT Monad transformer. First lets look at how we can create an OptionT monad. For our example we're going to create an OptionT[Future, A], which wraps an Option[A] inside a Future. We can create these like this from an A:

scala> :require /Users/jos/.ivy2/cache/org.scalaz/scalaz-core_2.11/bundles/scalaz-core_2.11-7.2.1.jar
Added '/Users/jos/.ivy2/cache/org.scalaz/scalaz-core_2.11/bundles/scalaz-core_2.11-7.2.1.jar' to classpath.

scala> import scalaz._
import scalaz._

scala> import Scalaz._
import Scalaz._                                ^

scala> import scala.concurrent.Future
import scala.concurrent.Future

scala> import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.ExecutionContext.Implicits.global

scala> type Result[A] = OptionT[Future, A]
defined type alias Result

scala> 1234.point[Result]
res1: Result[Int] = OptionT(scala.concurrent.impl.Promise$DefaultPromise@1c6ab85)

scala> "hello".point[Result]
res2: Result[String] = OptionT(scala.concurrent.impl.Promise$DefaultPromise@3e17219)

scala> res1.run
res4: scala.concurrent.Future[Option[Int]] = scala.concurrent.impl.Promise$DefaultPromise@1c6ab85

Note that we define an explicit type Result to be able to make point work. If you don't do this you'll get helpful error messages regarding type construction:

scala> "why".point[OptionT[Future, String]]
<console>:16: error: scalaz.OptionT[scala.concurrent.Future,String] takes no type parameters, expected: one
              "why".point[OptionT[Future, String]]

You can only use point when you're dealing with the innervalue of the monad transformer. If you've already got a Future or an Option, we need to use the OptionT constructor instead.

scala> val p: Result[Int] = OptionT(Future.successful(some(10)))
p: Result[Int] = OptionT(scala.concurrent.impl.Promise$KeptPromise@40dde94)

With a Monad transformer we can automatically unwrap a nested Monad. Now that we now how to convert our values to an OptionT, lets look back at the example we previously saw, and rewrite it like this:

def withdrawWithMonadTransformers(accountNumber: String, amount: Amount) : Future[Option[Statement]] = {

  type Result[A] = OptionT[Future, A]

  val result = for {
    account <- OptionT(Accounts.getAccount(accountNumber))
    balance <- OptionT(Accounts.getBalance(account))
    _ <- OptionT(Accounts.withdraw(account,balance).map(some(_)))
    statement <- Accounts.getStatement(account).point[Result]
  } yield statement

  result.run
}

Nice right? Instead of all the noise massaging the types into the correct type, we just create OptionT instances and return those. To get the stored value out of the OptionT we just call run.

Even though it is already much more readable. We now have the noise for create the OptionT. Even though it isn't much noise, it still is kind of distracting.

And some more syntax cleanup

We can even clean it up a bit more:

// with Monad transformers
type Result[A] = OptionT[Future, A]

/**
  * Unfortunately we can't use overloading, since we then run into
  * type erase stuff, and the thrush operator not being able to find
  * the correct apply function
  */
object ResultLike {
  def applyFO[A](a: Future[Option[A]]) : Result[A] = OptionT(a)
  def applyF[A](a: Future[A]) : Result[A] = OptionT(a.map(some(_)))
  def applyP[A](a: A) : Result[A] = a.point[Result]
}

def withdrawClean(accountNumber: String, amount: Amount) : Future[Option[Statement]] = {

  val result: Result[Statement] = for {
    account <- Accounts.getAccount(accountNumber)         |> ResultLike.applyFO
    balance <- Accounts.getBalance(account)               |> ResultLike.applyFO
    _ <- Accounts.withdraw(account,balance)               |> ResultLike.applyF
    statement <- Accounts.getStatement(account)           |> ResultLike.applyP
  } yield statement

  result.run
}

In this approach we just create specific converters to get the results into an OptionT monad. The result is that the actual for-comprehensions is very readable, without any cruft. And at the right hand side, out of immediate eyesight, we do the conversion to OptionT. Note that this isn't the most clean solution, since we need to specify different apply functions. Overloading here doesn't work, since after type erasure the applyFO and applyF will have the same signature.

Reader monad

The Reader Monad is one of the standard monads provided by Scalaz. The Reader monad can be used to easily pass configuration (or other values) around, and can be used for stuff like dependency injection.

The Reader monad solution

The Reader monad allows you to do dependency injection in scala. Whether that dependency is a configuration object, a reference to some other service, doesn't really that much. We start with an example, since that best explains how to use a reader monad.

For this example we'll assume we have a service which requires a Session to do stuff. This could be a database session, some web service session, or something else. So lets replace the previous sample to this, and for now simplify it a bit by removing the futures:

trait AccountService {
  def getAccount(accountNumber: String, session: Session) : Option[Account]
  def getBalance(account: Account, session: Session) : Option[Amount]
  def withdraw(account: Account, amount: Amount, session: Session) : Amount
  def getStatement(account: Account, session: Session): Statement
}

object Accounts extends AccountService {
  override def getAccount(accountNumber: String, session: Session): Option[Account] = ???
  override def getBalance(account: Account, session: Session): Option[Amount] = ???
  override def withdraw(account: Account, amount: Amount, session: Session): Amount = ???
  override def getStatement(account: Account, session: Session): Statement = ???
}

This seems a bit annoying, since everytime we want to call one of the services, we need to provide an implementation of Session. We could of course make Session implicit, but then we still need to make sure it is in scope, when we call the functions of this service. It would be nice if we could find a way to inject this session somehow. We could of course do this in the constructor of this service, but we could also use a Reader monad for this, which changes the code to this:

// introduce a Action type. This represents an action our service can execute. As you can see in
// the declaration, this Action, requires a Session.
type Action[A] = Reader[Session, A]

trait AccountService {
  // return an account, or return none when account can't be found
  def getAccount(accountNumber: String) : Action[Option[Account]]
  // return the balance when account is opened, or none when it isn't opened yet
  def getBalance(account: Account) :Action[Option[Amount]]
  // withdraw an amount from the account, and return the new amount
  def withdraw(account: Account, amount: Amount) : Action[Amount]
  // we can also get an account overview statement, which somehow isn't async
  def getStatement(account: Account): Action[Statement]
}

object Accounts extends AccountService {
  override def getAccount(accountNumber: String): Action[Option[Account]] = Reader((session: Session) => {
    // do something with session here, and return result
    session.doSomething
    some(Account())
  })

  override def getBalance(account: Account): Action[Option[Amount]] = Reader((session: Session) => {
    // do something with session here, and return result
    session.doSomething
    some(Amount(10,"Dollar"))
  })

  override def withdraw(account: Account, amount: Amount): Action[Amount] = Reader((session: Session) => {
    // do something with session here, and return result
    session.doSomething
    Amount(5, "Dollar")
  })

  override def getStatement(account: Account): Action[Statement] = Reader((session: Session) => {
    // do something with session here, and return result
    session.doSomething
    Statement(account)
  })
}

As you can see, we're not returning the result, but wrap the result in a Reader. The cool thing is that we now can start composing stuff, since the Reader is just a monad.

def withdrawWithReader(accountNumber: String) = {

  for {
    account <- Accounts.getAccount(accountNumber)
    balance <- account.fold(Reader((session: Session) => none[Amount]))(ac => Accounts.getBalance(ac))
    _ <- (account, balance) match {
      case (Some(acc), Some(bal)) => Accounts.withdraw(acc,bal)
      case _ => Reader((session: Session) => none[Amount])
    }
  statement <- account match { case Some(acc) => Accounts.getStatement(acc)}
  } yield statement
}

This won't return the actual final value, but will return a Reader. We can now run the code by passing in a Session:

// function returns 'steps' to execute, run execute these steps in the context of 'new Session'
withdrawWithReader("1234").run(new Session())

When you look back at the withdrawWithReader function you can see that we once again have to manage the Option monad explicitly, and make sure that we always create a Reader as a result. Luckily, though, Scalaz also provides a ReaderT, which we could use to automatically handle a specific type of Monad. In the following code we've shown how to do this for this example:

// introduce a Action type. This represents an action our service can execute. As you can see in
// the declaration, this Action, requires a Session.
type Action[A] = ReaderT[Option, Session, A]

trait AccountService {
  // return an account, or return none when account can't be found
  def getAccount(accountNumber: String) : Action[Account]
  // return the balance when account is opened, or none when it isn't opened yet
  def getBalance(account: Account) :Action[Amount]
  // withdraw an amount from the account, and return the new amount
  def withdraw(account: Account, amount: Amount) : Action[Amount]
  // we can also get an account overview statement, which somehow isn't async
  def getStatement(account: Account): Action[Statement]
}

object Accounts extends AccountService {
  override def getAccount(accountNumber: String): Action[Account] = ReaderT((session: Session) => {
    // do something with session here, and return result
    session.doSomething
    some(Account())
  })

  override def getBalance(account: Account): Action[Amount] = ReaderT((session: Session) => {
    // do something with session here, and return result
    session.doSomething
    some(Amount(10,"Dollar"))
  })

  override def withdraw(account: Account, amount: Amount): Action[Amount] = ReaderT((session: Session) => {
    // do something with session here, and return result
    session.doSomething
    Some(Amount(5, "Dollar"))
  })

  override def getStatement(account: Account): Action[Statement] = ReaderT((session: Session) => {
    // do something with session here, and return result
    session.doSomething
    Some(Statement(account))
  })
}

def withdrawWithReaderT(accountNumber: String) = {
  for {
    account <- Accounts.getAccount(accountNumber)
    balance <- Accounts.getBalance(account)
    _ <- Accounts.withdraw(account, balance)
    statement <- Accounts.getStatement(account)
  } yield statement
}

withdrawWithReaderT("1234").run(new Session)

As you can see, not that much has changed. The main thing we changed was changing the declaration of Action to use a ReaderT instead of a Reader, and we change the trait and the implementation to work with that. Now when you look at the withdrawWithReaderT function you can see that we don't need to handle the Option anymore, but it is handled by our ReaderT (which actually is a Kleisli but that's something for another artile). Cool right?

While this works great for just an Option, what would happen if we go back to the original example and want to deal with an Option nested inside a Future, and these once again inside an Reader? Well at that point, we might pass out of the scope of "Scalaz features for everyday usage", but the basic set up is the same:

// introduce a Action type. This represents an action our service can execute. As you can see in
// the declaration, this Action, requires a Session.
type OptionTF[A] = OptionT[Future, A]
type Action[A] = ReaderT[OptionTF, Session, A]

trait AccountService {
  // return an account, or return none when account can't be found
  def getAccount(accountNumber: String) : Action[Account]
  // return the balance when account is opened, or none when it isn't opened yet
  def getBalance(account: Account) :Action[Amount]
  // withdraw an amount from the account, and return the new amount
  def withdraw(account: Account, amount: Amount) : Action[Amount]
  // we can also get an account overview statement, which somehow isn't async
  def getStatement(account: Account): Action[Statement]
}

/**
  * Normally you would wrap an existing service, with a readerT specific one, which would handle
  * all the conversion stuff.
  */
object Accounts extends AccountService {
  override def getAccount(accountNumber: String): Action[Account] = ReaderT((session: Session) => {
    // do something with session here, and return result
    session.doSomething
    // Assume we get a Future[Option[Account]]
    val result = Future(Option(Account()))

    // and we need to lift it in the OptionTF and return it.
    val asOptionTF: OptionTF[Account] = OptionT(result)
    asOptionTF
  })

  override def getBalance(account: Account): Action[Amount] = ReaderT((session: Session) => {
    // do something with session here, and return result
    session.doSomething
    // assume we get a Future[Option[Amount]]
    val result = Future(some(Amount(10,"Dollar")))
    // convert it to the Action type, with explicit type to make compiler happy
    val asOptionTF: OptionTF[Amount] = OptionT(result)
    asOptionTF
  })

  override def withdraw(account: Account, amount: Amount): Action[Amount] = ReaderT((session: Session) => {
    // do something with session here, and return result
    session.doSomething
    // assume we get a Future[Amount]
    val result = Future(Amount(5, "Dollar"))
    // convert it to the correct type
    val asOptionTF: OptionTF[Amount] = OptionT(result.map(some(_)))
    asOptionTF
  })

  override def getStatement(account: Account): Action[Statement] = ReaderT((session: Session) => {
    // do something with session here, and return result
    session.doSomething
    // assume we get a Statement
    val result = Statement(account)
    // convert it to the correct type
    result.point[OptionTF]
  })
}

def withdrawWithReaderT(accountNumber: String) = {
  for {
    account <- Accounts.getAccount(accountNumber)
    balance <- Accounts.getBalance(account)
    _ <- Accounts.withdraw(account, balance)
    statement <- Accounts.getStatement(account)
  } yield statement
}

// this is the result wrapped in the option
val finalResult = withdrawWithReaderT("1234").run(new Session)
// get the Future[Option] and wait for the result
println(Await.result(finalResult.run, 5 seconds))

We define a different ReaderT type, where we pass in an OptionT instead of just an Option. This OptionT will handle the Option/Future transformation. When we've got a new ReaderT we of course need to lift the results from our service calls to this monad, which needs some type coersion for the compiler to understand everything (Intellij, also doesn't understand this anymore). The result though is very nice. The actual for-comprehension stays exactly the same, but this time can handle Option inside Future inside a Reader!

Conclusions

In this article we've looked at two parts of Scalaz which really come in handy when dealing with nested monads, or when you want to better manage dependencies between components. The cool thing is, that it is fairly easy to use Monad Transformers together with the Reader monad. The overall result is, is that with a couple of small steps, we can completely hide the details of working (in this case) with the Future and Option moand, and have nice and clean for-comprehensions and other monadic goodies.

Updated:

You may also enjoy

Working with nix-shell

11 minute read

Managing runtime dependencies using Nix We’re setting up a new project at work, and wanted to apply some lessons learned from previous projects. One of the ...

Exploring Cats Effect

13 minute read

In two previous articles we explored how you can use ZIO to easily create programs in a type-safe manner: Exploring ZIO - Part I Exploring ZIO - Part I...

Exploring ZIO - Part I

10 minute read

I’ve done a lot of Kotlin for the last two years, and have mainly followed Scala and done some pet projects. Since a lot of interesting stuff has been going ...