Kotlin for a Python and C++ Programmer

A while ago I got interested in Kotlin as a possible type-safe alternative to Python for our system. A lot of the non-performance-critical things are done in Python. As a lot of people have discovered, Python works well for small programs. But small programs have a tendency to get bigger, and to take on a life of their own. Maintaining large Python programs is hard, and refactoring them, for example to change the way objects relate to one another, is just about impossible. You're sure to miss some corner case which will show up as a runtime error much later.

Our CLI was the obvious candidate for experimenting with Kotlin. This is several thousand lines of Python and predictably, it has become hard to maintain. My first efforts with Kotlin were not very successful. It is based on Java and inherits some mis-features and design baggage from there, which stopped me from doing what I wanted. Also, the build environment is a nightmare.

Then recently I took another look at this project, and saw a way around my previous problem. As a result, I have since written a complete Kotlin implementation of the CLI. It's a nice piece of code and much easier to maintain and work with than its Python equivalent.

Here's a summary of the good and bad points of Kotlin, based on my experience.

• Really, really good: the Kotlin language. It's a delight to use with lots of features that lead to compact, uncluttered code, yet totally type-safe. More on that later.
• Very good: the IDE (called Idea). Intuitive and easy to get used to, and makes writing code just so easy. Only problem is that it occasionally crashes, taking the system with it.
• OK but not great: libraries. Like Python, Java supposedly has libraries for everything. But finding them is hard, and figuring out how to use them from Kotlin is even harder.
• Awful beyond belief: the build system, a dog's breakfast of several different tools (Gradle, Maven, Ant, who knows what else). As long as you stay within the IDE, life is mostly good. But at some point you generally need to build a stand-alone app. There is no documentation for how to do this, and what you can find online is confusing, contradictory and rarely works. Probably if you come from a Java background all this seems normal.

Idea: the IDE

Kotlin is joined at the hip to its IDE, which comes from the company that invented the language (Jetbrains). The same company also produces PyCharm for Python, and the two are very similar. It's everything you could ask from an IDE. The instant typing-time error checking has spoiled me completely, and now I expect Emacs to do the same when writing C++ - except of course that it doesn't.

The biggest problem, running it on Ubuntu, is that every so often it freezes and takes the entire GUI with it. If you can log in from another system, "killall -9 java" kills it and allows the system to keep going. Otherwise, you just have to reboot.

It does a good job of hiding the complexities of the build system, as long as you want to stay entirely in the IDE. But the problem with anything "automagic" is what happens when it goes wrong, or doesn't do what you need. The build system is a nightmare (see later) and the IDE offers no help at all in dealing with it if you want to create a stand-alone app.

It sometimes gets confused about which library symbols come from, and flags errors that aren't there. It still lets you run the compiler, so it's only a nuisance. It also isn't very helpful when you add a library that comes from a new place. You have to hand-edit an obscure Gradle file, and then restart Idea before it understands what you have done.

The Language

Once I got my head around it, Kotlin is the nicest language I have ever used. It particularly lends itself to functional-style programming, but there's nothing to stop you using it like C or Fortran. The few irritating things are a result of its Java legacy - fundamentally, Kotlin is just syntactic sugar over the top of Java. Some of the really nice things:

• inobtrusive strong typing: everything's type is known at compile time, yet you rarely have to be explicit about types. The compiler does an excellent job of figuring out types from context, like auto in C++ but much better.
• ?. and ?: operators: between them these make dealing with nullable values very clean and simple. ?. lets you write in one line what would take a string of nested if statements in C++ or Python. 
• lambda functions: all languages now support lambda (anonymous) functions, but in both C++ and Python they're an afterthought, and it shows. In Kotlin they are an integral part of the way the language is meant to be used, making them a very clean and natural way to express things.
• the "scope functions": a collection of highly generic functions that make it easy to do functional programming. For example, the 'let()' function allows you to execute some procedural style code using the result of a functional call chain. 'also()' makes it very easy to write chainable functions.
• generic sequence handling functions: 'map()' does the obvious job of applying a function to every element of a sequence or collection. There are plenty more that simplify all kinds of common requirements, for example to trim null elements from a list after some processing.
• string interpolation: the string "foo = $foo" replaces the last part with the value of foo, converted as appropriate to a string. That started with Perl and is now available in Python 3. Kotlin takes it further though, allowing complex expressions and figuring out the string syntax, e.g. "foo = ${x.getFoo("bah")+1}".
• extension functions: it's easy to define new functions as if they were member functions of a class. They can only access public members of the class, but to the user they work exactly as if they were part of the base class definition. For example I wrote a function String.makePlural() which figures out the plural of a noun. This has always struck me as an obvious improvement but it has never even been considered for C++ (nor Python as far as I know).

There's nothing really bad about the language. The "generic" support is fairly feeble compared to C++ templates. In C++ a template parameter type behaves exactly as any other type, for example you can instantiate it. And no validity checking is applied until you instantiate the template, which is very flexible.

Kotlin's generics are built on the Java equivalent. You have to specify exactly what the template can and can't do in the function definition, meaning you can't for example write a numeric function using the normal arithmetic operators. (It doesn't help that there is no common supertype for integers and floats that supports arithmetic). Once you accept this, it's not too hard to work around it. But it's a shame.

Libraries

The great thing about Python is that you can find libraries to do just about anything. The same is supposedly true of Java, too. Since Kotlin can easily call Java, that should mean you can find libraries to do just about anything in Kotlin too. But you can't.

I first ran into this when trying to use a Rest API from Kotlin. Python has an excellent library for this, called Requests. After a bit of googling, I found that someone had ported it to Kotlin, calling it khttp. Then I spent a couple of hours trying to figure out how to get Kotlin to actually use it. That ought not to be hard, but you have to tell the build system where to find a library, i.e. a URL. And none of the documentation, for any library, tells you this. Or sometimes it does, but it's wrong.

I did finally get khttp working, and it was good. But when I returned to my project a few months later it had simply disappeared. It was a single-person project, and the maintainer had got bored with it and moved on to something else. There were bits and pieces about it on the web, and maybe you could get it from here and patch it from there, but it didn't look like a good path.

So I googled around some more, and found another library. It's called Fuel, and it does allow you to make Rest requests. But it is obscure and barely documented. For example, when Rest returns an error, there is no straightforward way to access the details. You can do it in a very clumsy way. But even then, it uses the type system in such an obscure way that there is no way to write a common function that will work across multiple request types (Get, Put, Post and so on). You have to repeat the same ten lines of ugly impenetrable code.

One of the much-vaunted features of Kotlin is coroutine support, allowing you to run lightweight threads that maintain their own stack and state. This looked useful to handle parallel Rest requests, which I needed. Even though it is documented as part of the language, it isn't really. It's part of a library, and has to be explicitly imported. But from where? Everything you can find on the web says you need to "import kotlinx.coroutines". But that doesn't work. Eventually I did figure it out, but I never did convince the IDE. That showed an error right up until I decided I didn't need coroutines anyway.

Another example: a CLI needs the equivalent of GNU readline, so commands can be recalled and edited. The good news is that someone has ported the functionality to Java, in a library called JLine. In fact, they've done it three times - there is JLine, JLine2, and JLine3. They're all different in undocumented ways. But anyway there's hardly any documentation. To find out how to show history (equivalent of the shell 'history' command) I ended up reading the code.

The experience with other libraries has been the same:

• the documentation is between non-existent and very poor
• figuring out where to get the library from is near impossible
• even though there is probably a library for what you want, finding it is a challenge

The Build System

When you start writing Kotlin, you work entirely in the IDE and you don't even have to think about the build system. When you want to run or debug your program, you click on the menu, it takes a second or two to build, and it runs. Life is good.

But if you're writing a program it's probably because you want it to do something. And for that you most likely need to be able to run it without the IDE - from the command line, file explorer or similar. In the Java world, that means creating a .jar file. And that is where the fun begins.

You might reasonably suppose that the IDE would have a button somewhere, "turn this into a Jar file". But it doesn't, nothing like it. So you google it, thinking the menu item you need must just be buried somewhere. But no. What you find is incredibly complicated suggestions about editing files that don't even exist in your environment.

When you do finally manage to persuade something to create a Jar file, and you try to run it, you get a message about not having a manifest for something or other. If you're an experienced Java hand, this may mean something to you. All I know is that the IDE has agreed to build something, but missed out something vital.

Eventually, somehow, I managed to create a menu item that successfully built a runnable Jar file. Problem is, I have no idea how. When I created a trivial "hello world" program just for this exercise, I could never get it to work again. And then somewhere along the way I did something wrong, and the menu item disappeared, never to return.

Ironically, I did once find an article by someone from Jetbrains saying "of course when you write a program, you want to be able to run it without the IDE. Here's what you need to do." The instructions were simple, and they worked. The trouble is, no matter what search I do, I have never managed to find the article again.

Java programs were originally built using something called Ant. That was too complicated, so it was overlaid with another tool called Maven. Then that was too complicated too, so it was overlaid with something called Gradle. That came with its own language, but Kotlin invented a variant where the build requirements are described in a Kotlin mini-program.

So far so good, but all of these tools are mind-numbingly complicated and poorly documented for the casual user. Such documentation as you can find, assumes complete familiarity with the world of Java. Just because Gradle sits on top of Maven, doesn't mean you can ignore Maven. You still sometimes have to go and edit Maven files, which use XML. I've always viewed XML as an object language that only computers should deal with, just like Postscript or PDF. But the Java world is in love with it.

This all really starts to matter if you want to build your Kotlin program as part of some larger project - for example, our entire application. That is built with Make, and heaven only knows Make is a nightmare. But it's a familiar nightmare.

The IDE creates a "magic" file called gradlew. It isn't mentioned in any instructions, nor what to do with it. But a friend told me that './gradlew build' will build a stand-alone jar file from the command line - and sometimes it does. Luckily that worked for me "real" program, though when I tried it with a toy hello-world program it didn't.

Summary

Kotlin is great language, and a pleasure to use. Sadly, the nightmare build system, and the lack of any help from the IDE in dealing with it, means it is not really ready for prime time as part of a serious application or project.