Getting started with FP in Kotlin and Arrow: Typeclasses

I’ve recently made the effort to try and pick up Kotlin (again). The last couple of years I’ve been doing mostly scala work (with some other languages in between), and was wondering how the functional concepts of scala programming transfer to Kotlin. Main reason is that I don’t see myself returning to Java any time soon but, Kotlin seems to fix many of the verbose parts of Java. With the Arrow Kotlin also gets some FP concepts, so it looked like the right time to really dive into Kotlin, and see how FP in Kotlin holds up.

We’ll just start this series with looking at how we can do typeclasses using Arrow in kotlin. Before we start a quick note. In these examples I’ve used Gson for JSON marshalling, a better approach would probably have been using a more functional / immutable JSON library. So I might change that if I ever get the time for it.

Setting up the environment

Since this is the first article, we’ll start with setting up the environment. I’ve just followed the instructions on Arrow, and ended up with the following gradle files:

build.gradle

plugins {
    id 'org.jetbrains.kotlin.jvm' version '1.3.0'
}

group 'org.smartjava'
version '1.0-SNAPSHOT'

repositories {
    mavenCentral()
    jcenter()
    maven { url 'https://jitpack.io' }
    maven { url 'https://dl.bintray.com/spekframework/spek-dev' }
}

dependencies {

    testImplementation ("org.spekframework.spek2:spek-dsl-jvm:$spek_version")  {
        exclude group: 'org.jetbrains.kotlin'
    }
    testImplementation 'org.amshove.kluent:kluent:1.45'
    testRuntimeOnly ("org.spekframework.spek2:spek-runner-junit5:$spek_version") {
        exclude group: 'org.junit.platform'
        exclude group: 'org.jetbrains.kotlin'
    }

    compile "org.jetbrains.kotlin:kotlin-stdlib-jdk8"
    compile "ch.qos.logback:logback-classic:1.2.3"
    compile "io.ktor:ktor-gson:$ktor_version"

    compile "io.ktor:ktor-server-core:$ktor_version"
    compile "io.ktor:ktor-server-netty:$ktor_version"

    compile "io.arrow-kt:arrow-core:$arrow_version"
    compile "io.arrow-kt:arrow-syntax:$arrow_version"
    compile "io.arrow-kt:arrow-typeclasses:$arrow_version"
    compile "io.arrow-kt:arrow-data:$arrow_version"
    compile "io.arrow-kt:arrow-instances-core:$arrow_version"
    compile "io.arrow-kt:arrow-instances-data:$arrow_version"
    kapt    "io.arrow-kt:arrow-annotations-processor:$arrow_version"

    compile "io.arrow-kt:arrow-free:$arrow_version" //optional
    compile "io.arrow-kt:arrow-instances-free:$arrow_version" //optional
    compile "io.arrow-kt:arrow-mtl:$arrow_version" //optional
    compile "io.arrow-kt:arrow-effects:$arrow_version" //optional
    compile "io.arrow-kt:arrow-effects-instances:$arrow_version" //optional
    compile "io.arrow-kt:arrow-effects-rx2:$arrow_version" //optional
    compile "io.arrow-kt:arrow-effects-rx2-instances:$arrow_version" //optional
    compile "io.arrow-kt:arrow-effects-reactor:$arrow_version" //optional
    compile "io.arrow-kt:arrow-effects-reactor-instances:$arrow_version" //optional
    compile "io.arrow-kt:arrow-effects-kotlinx-coroutines:$arrow_version" //optional
    compile "io.arrow-kt:arrow-effects-kotlinx-coroutines-instances:$arrow_version" //optional
    compile "io.arrow-kt:arrow-optics:$arrow_version" //optional
    compile "io.arrow-kt:arrow-generic:$arrow_version" //optional
    compile "io.arrow-kt:arrow-recursion:$arrow_version" //optional
    compile "io.arrow-kt:arrow-instances-recursion:$arrow_version" //optional
    compile "io.arrow-kt:arrow-integration-retrofit-adapter:$arrow_version" //optional
    compile "org.jetbrains.kotlin:kotlin-script-runtime:1.3.10"
}

compileKotlin {
    kotlinOptions.jvmTarget = "1.8"
}
compileTestKotlin {
    kotlinOptions.jvmTarget = "1.8"
}

apply plugin: 'application'
apply plugin: 'kotlin-kapt'

mainClassName = 'io.ktor.server.netty.EngineMain'

generated-kotlin-sources.gradle

apply plugin: 'idea'

idea {
    module {
        sourceDirs += files(
                'build/generated/source/kapt/main',
                'build/generated/source/kapt/debug',
                'build/generated/source/kapt/release',
                'build/generated/source/kaptKotlin/main',
                'build/generated/source/kaptKotlin/debug',
                'build/generated/source/kaptKotlin/release',
                'build/tmp/kapt/main/kotlinGenerated')
        generatedSourceDirs += files(
                'build/generated/source/kapt/main',
                'build/generated/source/kapt/debug',
                'build/generated/source/kapt/release',
                'build/generated/source/kaptKotlin/main',
                'build/generated/source/kaptKotlin/debug',
                'build/generated/source/kaptKotlin/release',
                'build/tmp/kapt/main/kotlinGenerated')
    }
}

gradle.properties

kotlin.code.style=official
ktor_version=1.0.1
arrow_version=0.8.1
spek_version=2.0.0-alpha.2

Note that if you want to use the annotation processing and you use Intellij Idea for development, you might run into the issue that no sources are being generated for the Arrow annotations. The reason is that Intellij uses it’s own system to build the sources, and not necessary your complete gradle file. This can easily be fixed though by changing enabling the following setting:

At this point we should be able to use the annotations provided by Arrow and all the other provided FP functionality.

Creating typeclasses

I’m just going to assume you already know what typeclasses are. If not there are many resources out there that explain what they do and why they are really useful:

• Scala typeclass explained: Implement a String.read function
• Wikipedia typeclass explanation
• Cats: typeclasses
• Haskell: typeclases

Typeclasses are also often used to marshal types from JSON to a domain object and back again. So in this example we’re going to use that as the example. We’ll start with the interface definition of our marshallers:

typealias Result<T> = Either<Exception, T>

object Marshallers {

    fun <T : Any>convertToJson(m: T, ev: JsonMarshaller<T>): JsonElement = ev.run { m.toJson() }

    interface JsonMarshaller<T : Any> {
        fun T.toJson(): JsonElement

        companion object {}
    }
}

object Unmarshallers {

    fun <T : Any>convertFromJson(j: String, ev: JsonMarshaller<T>): Result<T> = ev.run { fromJson(j) }

    interface JsonMarshaller<T : Any> {
        fun fromJson(j: String): Result<T>

        companion object {}
    }

}

Here you can see that we’ve defined an interface that can handle marshalling a T to a JsonElement and one that does the opposite. Unmarshal an incoming String to a Either<T>. We’ve also added a helper method that takes an instance of a typeclass and calls the relevant function. Before we look at implementations of these typeclasses let’s take a quick look at a very simple model.

object Model {
    data class Product(val productId: String, val description: String) {
        companion object {}
    }

    data class OrderLine(val lineId: Int, val quantity: Int, val product: Product) {
        companion object {}
    }

    data class Order(val orderId: String, val orderLines: List<OrderLine>) {
        companion object {}
    }
}

Create typeclass instances for the marshallers

The first thing we’re going to do is define the marhsallers. The marshaller will take a T and convert it to a JSONElement. With Kotlin, creating this is pretty simple. Let’s start with the most simple approach. A quick note on this code and the code later on in the article. This is purely to demonstrate how typeclasses work in Kotlin. In real code we would do this differently (especially use a different JSON library), so don’t look to much at the something convoluted code in the marshallers and the unmarshallers.

    val orderLineMarshaller = object : JsonMarshaller<Model.OrderLine> {
        override fun Model.OrderLine.toJson(): JsonElement {
            val orderLine = gson.toJsonTree(this).asJsonObject
            orderLine.add("product", Marshallers.convertToJson(this.product, MarshallerInstances.productMarshaller))
            return orderLine
        }
    }

    val productMarshaller = object : JsonMarshaller<Model.Product> {
        override fun Model.Product.toJson(): JsonElement = gson.toJsonTree(this)
    }
    
    val orderMarshaller = object : JsonMarshaller<Model.Order> {
        override fun Model.Order.toJson(): JsonElement {
            val orderLines = this.orderLines.map { Marshallers.convertToJson(it, orderLineMarshaller) }
            val withoutOrderLines = this.copy(orderLines = listOf())
            val json = gson.toJsonTree(withoutOrderLines)

            val f = JsonArray()
            val acc = orderLines.foldLeft(f) { res, el -> res.add(el); res}
            val result = json.asJsonObject
            result.remove("orderLines")
            result.add("customorderlines", acc)

            return result
        }
    }

In this code fragment you can see that we create a numbr of instances of the JsonMarshaller interface. And use the Marshallers.converToJson function to convert the nested objects to JSON. We could of course also have used the gson.toJsonTree(this) call everywhere, but that wouldn’t have allowed us to easily customize the way the various objects are marshalled.

Now lets set up a simple test to see what is happening when we run this code:

   describe("The product marshaller instance") {

        val product = Model.Product("product", "description")

        it("should marshall an product to a JSON value") {
            val result = MarshallerInstances.productMarshaller.run {
                product.toJson().toString()
            }

            result `should be equal to` """{"productId":"product","description":"description"}"""
        }
    }


    describe("The orderline marshaller instance") {

        val orderLine = OrderLine(1, 2, Model.Product("", ""))

        it("should marshall an orderline to a JSON value") {
            val result = MarshallerInstances.orderLineMarshaller.run {
                orderLine.toJson().toString()
            }

            result `should be equal to` """{"lineId":1,"quantity":2,"product":{"productId":"","description":""}}"""
        }
    }

    describe("The order marshaller instance") {

        val orderLines = listOf(OrderLine(1,2, Model.Product("", "")), OrderLine(2,3, Model.Product("", "")))
        val order = Order("orderId", orderLines)

        it("should marshall an order to a JSON value") {
            val result = MarshallerInstances.orderMarshaller.run {
                order.toJson().toString()
            }

            result `should be equal to` """{"orderId":"orderId","customorderlines":[{"lineId":1,"quantity":2,"product":{"productId":"","description":""}},{"lineId":2,"quantity":3,"product":{"productId":"","description":""}}]}"""
        }
    }

All these tests pass, since we’re just calling them directly. We can also use the function defined in the Marshallers object:

fun <T : Any>convertToJson(m: T, ev: JsonMarshaller<T>): JsonElement = ev.run { m.toJson() }
...
val orderLine = OrderLine(1, 2, Model.Product("", ""))
Marshallers.convertToJson(orderline, MarshallerInstances.orderLineMarshaller)

The big advantage of calling it like this, is that the compiler checks for us that we’ve used the correct typeclass instance. If we try to call this with a typeclass instance of the incorrect type, the compiler starts complaining:

val orderLines = listOf(OrderLine(1,2, Model.Product("", "")), OrderLine(2,3, Model.Product("", "")))
val order = Order("orderId", orderLines)
Marshallers.convertToJson(order, MarshallerInstances.orderLineMarshaller)

This will fail:

e: fskotlin/src/test/kotlin/org/smartjava/typeclasses/MarshallersSpecTest.kt: (51, 21): Type inference failed: Cannot infer type parameter T in fun <T : Any> convertToJson(m: T, ev: Marshallers.JsonMarshaller<T>): JsonElement
None of the following substitutions
(Model.OrderLine,Marshallers.JsonMarshaller<Model.OrderLine>)
(Model.Order,Marshallers.JsonMarshaller<Model.Order>)
(Any,Marshallers.JsonMarshaller<Any>)
can be applied to
(Model.Order,Marshallers.JsonMarshaller<Model.OrderLine>)

As you might have noticed, we have to pass in the specific instance of the typeclass that we want to use. I come from a scala background and am used to passing in typeclasses as implicit parameters (or use a context bounded type parameter). With that approach it is enough for the typeclass to be (implicitly) in scope, so we don’t have to explicitly pass it into the function.
For Kotlin there is a proposal (KEEP-87), which proposes something similar. There is also a reference implementation available, that already allows you to play around with it. So in a future article I’ll do the same as in this article, but then with that implementation.

Now let’s also quickly implement the unmarshallers to get back from JSON to Kotlin.

Create typeclass instances for the marshallers

For a very naive implementation of the unmarshallers we can create something like this:

object UnmarshallerInstances {

    val gson = Gson()
    val parser = JsonParser()
    
    val productUnmarshaller = object : JsonUnmarshaller<Model.Product> {
        override fun fromJson(j: String): Result<Model.Product> = try {
            Either.right(gson.fromJson(j, Model.Product::class.java))
        } catch (e: Throwable){
            Either.left(e)
        }
    }
    
    val orderLineUnmarshaller = object : JsonUnmarshaller<Model.OrderLine> {
        override fun fromJson(j: String): Result<Model.OrderLine> = try {
            // first use the productUnmarshaller to get a Result<Product>
            val jsonObject = parser.parse(j).asJsonObject
            val jsonProduct = jsonObject.get("product")
            val product = productUnmarshaller.fromJson(jsonProduct.toString())

            // if product is right, convert it to a product, else we get the error.
            product.map{ p -> Model.OrderLine(jsonObject.get("lineId").asInt, jsonObject.get("quantity").asInt, p)}
        } catch (e: Throwable){
            Either.left(e)
        }
    }

    val orderUnmarshaller = object : JsonUnmarshaller<Model.Order> {
        override fun fromJson(j: String): Result<Model.Order> = try {

            val jsonObject = parser.parse(j).asJsonObject
            val jsonOrderLines = jsonObject.get("customorderlines").asJsonArray

            // convert using the correct unmarsahller
            val orderLines = jsonOrderLines.map { ol -> orderLineUnmarshaller.fromJson(ol)}

            // now we've got a List<Result<OrderLine>>. We'll reduce it to a Result<List<OrderLine>
            // so that we only continue of all succeed. Missing pattern matching and scala collections here.
            val rs = Either.Right(listOf<Model.OrderLine>())
            val orderLinesK = orderLines.foldLeft<Result<Model.OrderLine>, Result<List<Model.OrderLine>>>(rs) { res, ol ->
               when (res) {
                    is Either.Left -> res
                    is Either.Right -> when(ol) {
                        is Either.Left -> ol                               // set the error
                        is Either.Right -> Either.right(res.b.plus(ol.b))
                    }
                }
            }

            // and finally return the unmarshalled object
            orderLinesK.map{ ols -> Model.Order(jsonObject.get("orderId").asString, ols)}
        } catch (e: Throwable){
            Either.left(e)
        }
    }
}

Without going into too much detail here. You can see that we do some custom JSON unmarshalling here to convert our custom generated JSON back to our data classes. The productUnmarshaller is really straightforward, and we just use the standard gson unmarshall functionality. For the orderLineUnmarshaller we reuse the productUnmarshaller and use the map function to creae a OrderLine is the Product was parsed successfully. And for the complete order, we reuse the orderLineUnmarshaller to convert the incoming data to a List<Result<OrderLines>>. We fold this into a Result<List<OrderLines> failing if one of the orderLines is a Left. Finally we use this result to create the Ordere. At this point we also see some limitations in the type interference of Kotlin. We need to make explicitly pass in the types for the foldLeft function, so that the Kotlin compiler understands what is happening.

For completeness sake lets also create some tests for this, so that we know that the marshalles actually do what they’re supposed to do.

object UnmarshallersSpecTest: Spek({

    describe("The product unmarshaller instance") {

        val productJson = """{"productId":"product","description":"description"}"""
        val product = Model.Product("product", "description")

        it("should marshall an product to a JSON value") {

            val result = UnmarshallerInstances.productUnmarshaller.run {
                fromJson(productJson)
            }

            result.isRight() `should be` true
            result.map {
                it `should equal` product
            }
        }

        it("should return a Left when JSON is invalid") {
            val result = UnmarshallerInstances.productUnmarshaller.run {
                fromJson("invalid")
            }

            result.isLeft() `should be` true
        }
    }


    describe("The orderline unmarshaller instance") {

        val orderLine = OrderLine(1, 2, Model.Product("", ""))
        val orderLineJson = """{"lineId":1,"quantity":2,"product":{"productId":"","description":""}}"""

        it("should marshall an orderline to a JSON value") {
            val result = UnmarshallerInstances.orderLineUnmarshaller.run {
                fromJson(orderLineJson)
            }

            result.isRight() `should be` true
            result.map {
                it `should equal` orderLine
            }
        }

        it("should return a Left when JSON is invalid") {
            val result = UnmarshallerInstances.orderLineUnmarshaller.run {
                fromJson("invalid")
            }

            result.isLeft() `should be` true
        }
    }

    describe("The order unmarshaller instance") {

        val orderLines = listOf(OrderLine(1,2, Model.Product("", "")), OrderLine(2,3, Model.Product("", "")))
        val order = Order("orderId", orderLines)
        val orderJson = """{"orderId":"orderId","customorderlines":[{"lineId":1,"quantity":2,"product":{"productId":"","description":""}},{"lineId":2,"quantity":3,"product":{"productId":"","description":""}}]}"""

        it("should marshall an order to a JSON value") {
            val result = UnmarshallerInstances.orderUnmarshaller.run {
                fromJson(orderJson)
            }

            result.isRight() `should be` true
            result.map {
                it `should equal` order
            }
        }

        it("should return a Left when JSON is invalid") {
            val result = UnmarshallerInstances.orderUnmarshaller.run {
                fromJson("invalid")
            }

            result.isLeft() `should be` true
        }
    }
})

Add this point we’ve seen a little bit of the Arrow functionality already. We’ve used the Either typeclass, and also used the foldLeft and map extensions Arrow provides on top of the base classes. In the next section we’ll look a bit closer at using the typeclasses directly and use a custom annotation from Kotlin, that helps in making typeclass instances easier.

Use Arrow functionality: extension annotation

In the previous code fragments we’ve seen that we can create type classes by implementing an interface and assigning it to a value which we then can use directly. If you’ve only got a small number of typeclasses to create this works nice, but you do have to create the wrappers and assign values yourself. With the @extension annotation, Arrow generates code which makes it easy for you to get the instance for a specific type.

The following example shows how to use the @extension annotation for our Unmarshallers

    @extension
    interface OrderLineUnmarshallerInstance : JsonUnmarshaller<Model.OrderLine> {
        override fun fromJson(j: String): Result<Model.OrderLine> = try {
            // first use the productUnmarshaller to get a Result<Product>
            val jsonObject = parser.parse(j).asJsonObject
            val jsonProduct = jsonObject.get("product")
            val product = productUnmarshaller.fromJson(jsonProduct.toString())

            // if product is right, convert it to a product, else we get the error.
            product.map{ p -> Model.OrderLine(jsonObject.get("lineId").asInt, jsonObject.get("quantity").asInt, p)}
        } catch (e: Throwable){
            Either.left(e)
        }
    }

As you can see not that much has changed. The main thing that has changed is that we now define an interface, and not an val or fun. What Arrow does, it will generate code that looks like this:

package org.smartjava.fskotlin.orderline.jsonUnmarshaller

import arrow.core.Either
import kotlin.String
import kotlin.Suppress
import kotlin.Throwable
import kotlin.jvm.JvmName
import org.smartjava.fskotlin.Model.OrderLine.Companion
import org.smartjava.fskotlin.OrderLine
import org.smartjava.fskotlin.OrderLineUnmarshallerInstance

@JvmName("fromJson")
@Suppress(
        "UNCHECKED_CAST",
        "USELESS_CAST",
        "EXTENSION_SHADOWED_BY_MEMBER",
        "UNUSED_PARAMETER"
)
fun fromJson(j: String): Either<Throwable, OrderLine> = org.smartjava.fskotlin.Model.OrderLine
   .jsonUnmarshaller()
   .fromJson(j) as arrow.core.Either<kotlin.Throwable, org.smartjava.fskotlin.OrderLine>

fun Companion.jsonUnmarshaller(): OrderLineUnmarshallerInstance = object : org.smartjava.fskotlin.OrderLineUnmarshallerInstance {  }

As you can see this will create an OrderLineUnmarshallerInstance on the companion object of the OrderLine class, so we can easily access it, and create a helper function for easy access to the fromJson function. Throughout the Arrow code base this is used extensively, which is a good thing, to make sure that all the typeclasses follow the same pattern. For your own projects, I don’t think you should need this, and it’s probably easier and more straightforward to just define the typeclasses directly and assign them to a fun or a val.

Use Arrow functionality: Foldable typeclass

Before ending this article, I want to clean up a couple of code fragments by using some other arrow typeclasses. The first one is the orderMarshaller:

    val orderMarshaller = object : JsonMarshaller<Model.Order> {
        override fun Model.Order.toJson(): JsonElement {
            val orderLines = this.orderLines.map { Marshallers.convertToJson(it, orderLineMarshaller) }
            val withoutOrderLines = this.copy(orderLines = listOf())
            val json = gson.toJsonTree(withoutOrderLines)

            val f = JsonArray()
            val acc = orderLines.foldLeft(f) { res, el -> res.add(el); res}
            val result = json.asJsonObject
            result.remove("orderLines")
            result.add("customorderlines", acc)

            return result
        }
    }

While this works, it isn’t the best way to do this. We first have to explicitly map the orderLines to a List<JsonElement>, we can’t use this list directly, but have to convert it again to a JsonArray, before we can use it. What would be nice is if we could have a JsonMarshaller which would be able to automatically marshall a list or set of T to a JsonArray. If we look at the previous code what we need to do is some mapping from T to a JsonElement and some folding. To go from T to JsonElement we’ve already got out JsonMarshaller, and Arrow provides a typeclass called Foldable that allow us to fold over a specific container. With these two typeclasses we can create a JsonMarshaller instance that is able to create the JsonArray for arbitrary foldable containers like this:

    /**
     * For the type of Kind<F, T> e.g ListK<T> we can automatically fold them using the provided typeclass. What
     * we need to know is evidence that it's foldable, and evidence on how to apply the marshaller to the embedded T
     * elements.
     */
    fun <F, T: Any>foldableToJsonArrayMarshaller(fev: Foldable<F>, ev: JsonMarshaller<T>) = object: JsonMarshaller<Kind<F, T>> {
        override fun Kind<F, T>.toJson(): JsonElement = fev.run {
            foldLeft(JsonArray()) { b, a ->
                b.add(ev.run { a.toJson() })
                b
            }
        }
    }

Here we use the provided foldable, to invoke foldLeft on the container, and the provided marshaller to convert the T elements contained in the foldable to a JsonElement. All the converted Ts are aggregated into a JsonArray, which is returned. Ignore the Kind<F, T> stuff for now, since that is something for a different article. It is enough to know for now, that this is the way Arrow allows us to use higher kinded types for extensions. With this generic marshaller, we can now also change the orderMarshaller to something like this:

    val orderMarshaller = object : JsonMarshaller<Model.Order> {
        override fun Model.Order.toJson(): JsonElement {
            val json = gson.toJsonTree(this.copy(orderLines = listOf())).asJsonObject
            val orderLinesJson = MarshallerInstances.foldableToJsonArrayMarshaller(foldable(), orderLineMarshaller).run { ListK(orderLines).toJson() }

            json.remove("orderLines")
            json.add("customorderlines", orderLinesJson)

            return json
        }
    }

And you can see that we use the foldableToJsonArrayMarshaller and pass in a foldable() which can be retrieved from the List companion object, and our marshaller. The rest is done in the same way as for the other typeclasses. This makes our code much more clean, and if we have other containers with T elements, we can reuse this foldableToJsonArrayMarshaller marshaller.

Conclusion

All in all it isn’t that hard to use and create typeclasses in Kotlin. With Arrow we get a lot of functionality, that makes functional programming, working with monads and typeclasses much easier. What you do see is that in certain parts the standard library of Kotlin is limited. Pattern matching is limited, and the lack of implicit arguments in functions makes using typeclasses kind of intrusive. Besides that, the lack of a good API on top of immutable collections in Kotlin makes me miss standard functionality in other functional languages.

But, Arrow in itself looks great. While it takes some effort to get my head in the right place, the provided type and data classes work as expected (so far) and I’ll continue experimenting with the various parts to see how Arrow and Kotlin can work together.