Assert with JUnit for easier debugging in Android Studio

[GAumala]

I've been using Kluent in Android projects for years because the syntax is great and I really like writing tests with it. However, there has always been a little problem when comparing "big" structures like data classes or with several fields or long lists. The error message printed by assertion methods of kotlin.test get really hard to read when the toString() methods of the compared objects are very large. For reference here is how it is implemented:

fun assertEquals(message: String?, expected: Any?, actual: Any?): Unit {
        assertTrue({ messagePrefix(message) + "Expected <$expected>, actual <$actual>." }, actual == expected)
    }

As you can see the message gets printed on a single line. When it gets wrapped in a terminal it's really hard to figure out what is going on. Consider this example that compares lists of timestamps:

    @Test
    fun test_equal_lists() {
        val listA = listOf(1590026561, 1590026579, 1590026590, 1590026597,
            1590026608, 1590026619, 1590026639, 1590026648)
        val listB = listOf(1590026561, 1590026579, 1590026590, 1590026597,
            1590026608, 1590026619, 1590028639, 1590026648)
        listA `should be equal to` listB
    }

One of them is off by a single digit. This is the console output :

This list isn't really that long, but it's really hard to spot that wrong digit. It would be nice if the objects could be printed on different lines so that they can be compared side by side. Unfortunately no matter how we print it, it's going to be hard figure out errors on a console because toString() implementations don't follow any standards whatsoever.

I think the best bet here is to leave this job to existing tooling. Android Studio has great integration with JUnit 4. When you create a new project, you get example tests generated with JUnit 4. This tests use org.junit.Assert.* methods for assertion. When assertEquals fails it prints each object on a different line as I suggested earlier. But the really cool thing here is that when the assertion fails you get a nice "Click to see difference" link that opens a new window that highlights the difference. You can see it here:

To get this in my tests I can easily implement should be equal to like this:

import org.junit.Assert.assertEquals

infix fun <T> T.`should be equal to`(expected: T?): T {
    assertEquals(expected, this)
    return this
}

This would be pretty easy to implement, but it's a significant change nevertheless. A good thing is that the API remains the same. The only problem that I see is that if your setup doesn't include JUnit 4, then there aren't much benefits. I'd assume that if this works on Android Studio, it works on all of JetBrains IDEs as well, but I really don't know. Let me know what you guys think.

[MarkusAmshove]

Hi 👋 and thanks for raising this issue!

I remember that we were using junit 4 a while back, but I can’t remember or find the reason we switched to kotlin.test. Or maybe kotlin.test switched out junit Inder their hood.

The reason for us using junit was the same you’re talking about, the great experience in IDEs, including diff viewers for comparison failures.

I think it might be a good idea to raise an issue at JetBrains and see what they think about implementing the same behavior for kotlin.test failures, because then they can even support that in Multiplattform Kotlin projects.

Depending on their answer, I think we could go and either implement formatters ourselves or fall back to junit again for JVM and Android 🙂

[GAumala]

Ok, I've been looking up things in Kotlin's source code for hours and this isn't a bug on kotlin.test, we're just using it wrong.

The folks at JetBrains seem to want to provide good testing tools without getting coupled to a single testing framework. Naturally, Kluent should do the same.

Under the hood kotlin.test uses an Asserter interface to delegate the actual assertions. When the assertions API loads, it uses ServiceLoader to look for an AsserterContributor implementation in the classpath that can contribute a different Asserter implementation. JetBrains already provides contributors for both JUnit 4 and 5 in different artifacts.

So, to get diff viewers with Kluent, all I have to do is add this line to my build.gradle:

    testImplementation "org.jetbrains.kotlin:kotlin-test-junit:$kotlin_version"

No additional configuration is required. Maybe we could add this to the README.

[MarkusAmshove]

Anither thing worth looking at is if we can reuse java.lang.AssertionError since the screenshot with the diff prompt suggests that IntelliJ might look for that Exception, maybe expecting a specific shape.

But in the meantime I'll add your solution to the README 🎉

[GAumala]

I think AssertionError is already used everywhere. IntelliJ doesn't seem to care about the exception type. Any type works as long as the message uses this template somewhere:

"expected:<$expectedObj> but was:<$actualObj>"

[MarkusAmshove]

I've added a note to the README :-)