Exploring ZIO - Part I

I’ve done a lot of Kotlin for the last two years, and have mainly followed Scala and done some pet projects. Since a lot of interesting stuff has been going on in the Scala space for the last couple of months, I want to start writing some more about it, and my experiences with it. So expect some articles on Scala 3 and ZIO. For the first of these set of articles, I’d like to focus a bit on ZIO. I’m exploring the documentation, watched some presentations, but nothing works as best, as just writing a simple application with it.

So in this first part on ZIO, I’m going to look at a simple layered ZIO application that (for now) consists out of the following:

• Loading Configuration: How to load configuration (PureConfig), and provide that as a ZLayer to other services.
• REST endpoint with HTTP4s: I’ll add a very simple REST layer (only the basics for now), which uses information from the configuration to start a server on a specific host and port.

In the end we should have a simple application that starts an HTTP Server, loads configuration in a safe way, and provides us with a simple application that we can build upon for future articles, when we explore more features of ZIO.

ZIO Project setup

If you want to play along, everything should work out of the box with Metals:

Before we can do anything, we need to define the dependencies (build.sbt). When I started playing around with this, there were some dependencies issues between the version of cats, the version of HTTP4S and the zio-cats-interop stuff, so after some checking, the following is a good starting point (I’ll have a look and update this to a newer version in the future).

import Dependencies._

ThisBuild / scalaVersion := "2.13.4"
ThisBuild / version := "0.1.0-SNAPSHOT"
ThisBuild / organization := "com.example"
ThisBuild / organizationName := "example"

// need to use an older version, since the newest version of http4s
// supports the latest cats version, while the zio-interop-cats one
// doesn't support this yet.
val Http4sVersion = "1.0.0-M4"

lazy val root = (project in file("."))
  .settings(
    name := "zio-playground",
    libraryDependencies += scalaTest % Test,
    libraryDependencies ++= Seq(
      "org.http4s" %% "http4s-blaze-server" % Http4sVersion,
      "org.http4s" %% "http4s-dsl" % Http4sVersion,
      "dev.zio" %% "zio" % "1.0.6",
      "dev.zio" %% "zio-streams" % "1.0.6",
      "dev.zio" %% "zio-interop-cats" % "2.4.0.0",
      "com.github.pureconfig" %% "pureconfig" % "0.15.0",
      "org.slf4j" % "slf4j-api" % "1.7.5",
      "org.slf4j" % "slf4j-simple" % "1.7.5"
      "dev.zio" %% "zio-test" % "1.0.6" % "test",
      "dev.zio" %% "zio-test-sbt" % "1.0.6" % "test"
    )
  )

Looking at the dependencies above, nothing out of the ordinary. Note the last two dependencies which we’ll use for testing part of the application.

Providing a configuration

For the configuration part we’re going to use (PureConfig), which is a typesafe, secure way of loading configurations. How this library works is a bit out of scope, but basically what you do is, you define a set of case classes defining the configuration model, and fill them by loading a configuration file. For this part we have a very basic configuration file:

api-config {
  endpoint = "localhost"
  port = 8081
}

To load this configuration we call ConfigSource.default.load[Config], this returns us with an Either where we have a list of configuration exceptions, or the loaded configuration. To wrap this in ZIO we need to lift it into a ZIO type. We’ll use the Task type from ZIO for this. A Task is one of the default type aliases:

  type IO[+E, +A]   = ZIO[Any, E, A]         // Succeed with an `A`, may fail with `E`        , no requirements.
  type Task[+A]     = ZIO[Any, Throwable, A] // Succeed with an `A`, may fail with `Throwable`, no requirements.
  type RIO[-R, +A]  = ZIO[R, Throwable, A]   // Succeed with an `A`, may fail with `Throwable`, requires an `R`.
  type UIO[+A]      = ZIO[Any, Nothing, A]   // Succeed with an `A`, cannot fail              , no requirements.
  type URIO[-R, +A] = ZIO[R, Nothing, A]     // Succeed with an `A`, cannot fail              , requires an `R`.

So we’ve got an operation that can either fail with a Throwable or succeed with an A. To lift our Either we just call fromEither:

Task.fromEither(
  ConfigSource.default
    .load[Config]
    .left
    .map(pureconfig.error.ConfigReaderException.apply)
  )

Since the load returns a list of ConfigReaderFailures, we just map the left part into an exception, before we can convert it into a ZIO Task. Now that we’ve got this configuration, we can use it in for-comprehensions, pass it to other components that need it and anything else you want to do with it. For this example we’re going one step further, and we’ll define the loading of the configuration as a service, which we can define as a dependency for other services. For ZIO the standard way of doing this is by using ZLayer. I found the documentation a bit hard to follow, but once you start playing around with it, it becomes a lot easier to understand. Below I’ve put the complete source for the configuration service, with inline comments explaining the various steps. Note that this structure is the proposed convention from ZIO, so it’s something you’ll see coming back in multiple sources.

import zio.Task
import zio.Has
import zio.ZIO

import pureconfig.ConfigSource
import pureconfig.generic.auto._
import zio.ZLayer
import zio.Layer

// define the domain model used for configuration
case class Config(apiConfig: ApiConfig)
case class ApiConfig(endpoint: String, port: Int)

// define the configuration dependency to inject into the environment
// and the related functions
object configuration {

  // Custom type to inject
  type Configuration = zio.Has[Configuration.Service]

  // contains the service definition, and multiple implementations
  // based on the environment we want to run in.
  object Configuration {

    // we just have the load function inside the service to load
    // some configuration
    trait Service {
      val load: Task[Config]
    }

    // we can have multiple implementations of this service. This is the
    // live implementation, which loads the config through pureconfig
    val live: Layer[Nothing, Configuration] = ZLayer.succeed {
      new Service {
        override val load: Task[Config] = Task.fromEither(
          ConfigSource.default
            .load[Config]
            .left
            .map(pureconfig.error.ConfigReaderException.apply)
        )
      }
    }

    // helper function which access the correct resource from our environment.
    val load: ZIO[Configuration, Throwable, Config] = ZIO.accessM(_.get.load)
  }
}

What we do here is first define the case classes that map to the configuration file, that is just something specific to pure-config. Next we define the configuration object, where we define the ZIO / ZLayer specific stuff. We define a Configuration type, which is the type we can have other services depend on. This is the type which uniquely identifies that functionality offered by this service. To define this we use zio.Has[A], which following the scaladoc, does the following:

The trait Has[A] is used with ZIO environment to express an effect’s dependency on a service of type A. For example, RIO[Has[Console.Service], Unit] is an effect that requires a Console.Service service. Inside the ZIO library, type aliases are provided as shorthands for common services, e.g.:

After the type definition, we define a trait with the specific functions provided by this service. In our case this is just the load function (we’d actually could do better by loading during service creation, and assign it to a lazy val or use memomize). The next step we need to do is define the different implementation we have of our service. We could have specific ones for testing, for specific environments and so on. In this case we have one, for which it is the convention to call that one live:

val live: Layer[Nothing, Configuration] = ZLayer.succeed {
  new Service {
    override val load: Task[Config] = Task.fromEither(
      ConfigSource.default
        .load[Config]
        .left
        .map(pureconfig.error.ConfigReaderException.apply)
    )
  }
}

Here we load the configuration, and lift it into the Task. We create an anonymous instance, and return that as the result of the ZLayer.succeed function. ZLayer.succeed means that service creation of this instance will always succeed. In our case that is true, since we only load the config at the point it is needed. There are other ZLayer functions which allow you to create managed resources, or where the creation of the service returns an effect (e.g a Task). For now though, since nothing can go wrong here, we use ZLayer.succeed.

The final part of this service, is that we provide a convenience method called load, which mirrors the load function from our service:

 val load: ZIO[Configuration, Throwable, Config] = ZIO.accessM(_.get.load)

This means that if there is a Configuration in the provided environment, we can use this helper function to directly access the load function, and load the configuration. We’ll show you how this is used further down in this article, when we start tying services together.

Testing a configuration

So far we’ve seen quite some code, but how do we now that this will do what it needs to do. For this ZIO provides some testing libraries, where we can test whether the service we just created does what it needs to do. So let’s write a test:

import org.scalatest.flatspec.AnyFlatSpec
import org.scalatest.matchers.should.Matchers
import zio.test.DefaultRunnableSpec
import zio.test._
import Assertion._

object LiveConfigLoaderSpec extends DefaultRunnableSpec {
  override def spec: ZSpec[Environment, Failure] =
    suite("ConfigLoaderSpec")(
      testM("Load the configuration") {
        for {
          config <- configuration.Configuration.load
        } yield {
          assert(config.apiConfig.port)(equalTo(8081))
        }
      }
    ).provideCustomLayer(configuration.Configuration.live)

}

What we do here is that we call the load function we just defined in the companion object, and based on the passed in layer, we can check whether it works. In this example we pass in the configuration.Configuration.live layer. So now when we call configuration.Configuration.load, we should call the load of the pureconfig based configuration.

Running the above test will show something like this:

+ ConfigLoaderSpec
  + Load the configuration
Ran 1 test in 751 ms: 1 succeeded, 0 ignored, 0 failed

At this point we’ve got a simple configuration service, but we haven’t got an application yet. So before we move on the http4s stuff, we’re going to set up a minimal application.

Minimal application

Once again, I’ll show the annotated code, and then go into some of the details:

import zio.ExitCode
import zio.URIO
import zio.ZEnv
import configuration.Configuration
import example.services.http4sServer
import zio.ZIO

/** Base entry point for a ZIO app, which runs the logic within
  * a specified environment
  */
object Application extends zio.App {

  // now run the application, by providing it with the application layer
  override def run(args: List[String]): URIO[ZEnv, ExitCode] =
    myAppLogic.provideLayer(Configuration.live).exitCode

  // we can do more stuff here. For now just load the config, cause why not.
  val myAppLogic =
    for {
      config <- Configuration.load
    } yield (
      println(config)
    )

}

Here we extend from zio.App which gives us the override def run(args: List[String]): URIO[ZEnv, ExitCode] function to implement as entrypoint in the application. What we do in this entry point is call our program, and when we do that we provide a set of layers containing our services. In this example we just provide the layer we just defined, where we can load the configuration. In out program myAppLogic we load the configuration (which is present in the environment of our ZIO type.). The result is very simply that we see this:

$ sbt run
[info] welcome to sbt 1.5.0 (AdoptOpenJDK Java 11.0.7)
[info] loading settings for project zio-playground-build-build from metals.sbt ...
[info] loading project definition from /home/jos/dev/git/smartjava/zio-playground/project/project
[info] loading settings for project zio-playground-build from metals.sbt ...
[info] loading project definition from /home/jos/dev/git/smartjava/zio-playground/project
[success] Generated .bloop/zio-playground-build.json
[success] Total time: 1 s, completed May 17, 2021, 4:56:17 PM
[info] loading settings for project root from build.sbt ...
[info] set current project to zio-playground (in build file:/home/jos/dev/git/smartjava/zio-playground/)
[info] running example.Application
Config(ApiConfig(localhost,8081))

What we did above is pretty much the same as we did in the test. We provide a program to execute, and when we execute that program we pass in a set of services that can be accessed through the environment.

Adding the HTTP4S server dependency

For the last part of this article, we’ll also add a very simple HTTP4S server that will just return ok (we’ll look at the mapping to case classes later). For the HTTP4S server we want it to use the configuration layer we defined earlier. To get the HTTP4S server running we need to take a couple of extra steps to make it work correctly with ZIO.

To create the HTTPServer we use the following function on the BlazeServerBuilder

def apply[F[_]](executionContext: ExecutionContext)(implicit
        F: ConcurrentEffect[F],
        timer: Timer[F]): BlazeServerBuilder[F]

If you look at this signature you can see that, besides an explicit executionContext we also need an implicit Timer[F] and implicit ConcurrentEffect[F], where F is the type we want to work with. Since we’re already working with a Task in the config service, lets do that as well for this part. So we want to use the builder like this: BlazeServerBuilder[Task](someExecutionContext).

This won’t work however, since we don’t have the correct implicits in scope, and don’t use the types from Cats. Luckily zio provides an interop library that provides us with the correct implicits. For instance the ConcurrentEffect[Task] and Timer[Task are provided by this implicit function from the CatsEffectInstances class:

  implicit final def taskEffectInstance[R](implicit runtime: Runtime[R]): effect.ConcurrentEffect[RIO[R, *]] =
    new CatsConcurrentEffect[R](runtime)

  implicit final def zioTimer[R <: Clock, E]: effect.Timer[ZIO[R, E, *]] =
    zioTimer0.asInstanceOf[effect.Timer[ZIO[R, E, *]]]

As you can see the taskEffectInstance needs a Runtime as implicit parameter, so we need to make sure that our service that we’re defining has access to the ZIO Runtime. And looking back at the initial apply function for the BlazeServerBuilder we can also see that it needs an executionContext. Luckily an executionContext is also provided by the ZIO Runtime.

Now that we now which dependencies we need we can create the service. First we’ll look at the basic structure, and then a bit closer at how to create the HTTP4s server:

import org.http4s.server.Server
import zio._
import example.configuration.Configuration

object http4sServer {

  type Http4sServer = zio.Has[Http4sServer.Service]

  object Http4sServer {
    type Service = Has[Server]
    val live: ZLayer[ZEnv with Configuration, Throwable, Service] = ...
  }
}

The code above defines our server, and the signature for the live instance. As we mentioned we want to use information from the Configuration dependency to configure the service, and we need to have access to the zio runtime to correctly set up HTTP4s. ZIO has helper functions for creating the runtime, so lets look at the implementation of the live instance.

val live: ZLayer[ZEnv with Configuration, Throwable, Service] =
  ZLayer.fromManaged {
    for {
      // we should probably cache this, and make this a function, where we
      // pass in the relevant configuration
      config <- Configuration.load.toManaged_

      // The implicit runtime provided the implicits needed to create the ConcurrentEffect.
      // This is done by the zio / cats interop imports.
      server <- ZManaged.runtime[ZEnv].flatMap {
        implicit runtime: Runtime[ZEnv] =>
          BlazeServerBuilder[Task](runtime.platform.executor.asEC)
            .bindHttp(config.apiConfig.port, config.apiConfig.endpoint)
            .withHttpApp(Routes.routes)
            .resource
            .toManagedZIO
      }
    } yield (server)
  }

Before we look at the fromManaged, let’s first look at the way we create the server. As we mentioned we need access the runtime, ZIO provides a useful helper for this called ZManaged.runtime, where we create a runtime based on the passed in type. Using this runtime, we can create the BlazeServerBuilder. When using cats we would use the resource to safely start and stop the server. In ZIO world this is called a ZManaged:

A ZManaged[R, E, A] is a managed resource of type A, which may be used by invoking the use method of the resource. The resource will be automatically acquired before the resource is used, and automatically released after the resource is used.

The resource from the call to .resource can be converted to a ZManaged by calling toManagedZIO. Since this returns a ZManaged type, we also need to make sure that the other types in our for-comprehension are also ZManageds. So we use the toManaged_ function to convert the result from the Configuration.load to a ZManaged. At the end of the for-comprehension we can then convert the resulting ZManaged into a ZLayer by calling the fromManaged.

This seems like a lot of work, but it nicely shows how well ZIO can play with existing Cats types.

Defining a minimal API and running the application

Without going too much into detail on the http4s specifics, the following are the routes we support for now (don’t look at the implementation, since it’s just some placeholders for future implementation)

object Routes {

  private val dsl = Http4sDsl[Task]
  import dsl._

  def routes = HttpRoutes
    .of[Task] {
      case GET -> Root / "users" / IntVar(id) => {
        Created("A value here")
      }
      case POST -> Root / "users" => {
        Created("And another one here")
      }
    }
    .orNotFound
}

And with some small updates our main program looks like this:

package example

import zio.ExitCode
import zio.URIO
import zio.ZEnv
import configuration.Configuration
import example.services.http4sServer
import zio.ZIO

/** Base entry point for a ZIO app, which runs the logic within
  * a specified environment
  */
object Application extends zio.App {

  // combine the configuration layer and the standard environment into
  // a new layer.
  val defaultLayer = Configuration.live ++ ZEnv.live
  // the complete application layer also consists out of the http4server, so
  // add that to the default layers list
  val applicationLayer = defaultLayer >>> http4sServer.Http4sServer.live

  // now run the application, by providing it with the application layer
  override def run(args: List[String]): URIO[ZEnv, ExitCode] =
    myAppLogic.provideLayer(applicationLayer).exitCode

  // we want a service, but return never
  val myAppLogic: URIO[http4sServer.Http4sServer.Service, Unit] = ZIO.never
}

We first combine the ZEnv.live and Configuration.live layers into a first layer using the ++ operator. Next we pass these dependencies to the http4sServer layer. The >>> operator uses the provided left hand operand as dependencies and returns the layer on the right hand side. This final layer is passed into the application using provideLayer. Our application is just a ZIO.never, which means that our application will keep running until interrupted.

[info] running example.Application
[zio-default-async-4] INFO org.http4s.blaze.channel.nio1.NIO1SocketServerGroup - Service bound to address /127.0.0.1:8081
[zio-default-async-4] INFO org.http4s.server.blaze.BlazeServerBuilder -
  _   _   _        _ _
 | |_| |_| |_ _ __| | | ___
 | ' \  _|  _| '_ \_  _(_-<
 |_||_\__|\__| .__/ |_|/__/
             |_|
[zio-default-async-4] INFO org.http4s.server.blaze.BlazeServerBuilder - http4s v1.0.0-M4 on blaze v0.14.13 started at http://127.0.0.1:8081/

Summary

So that’s it for now. We’ve only scratched the surface of ZIO, but this should at least give you a nice idea on how the layering and dependency management works. If I don’t get distracted by something else, I’ll try and post a couple of other articles about ZIO.