retro-gtk: Postmortem

This article is the first of a small series about retro-gtk, a library I develop in tandem with Games and which allows it to use Libretro cores. This first article focuses on the initial goals of the library, its design and the problems that arose during its development, while the next ones will focus on what I am working on to fix these problems.

Libretro? retro-gtk? Are These Edible?

The Libretro project defines an API to be implemented by so-called Libretro cores — typically video game console emulators — to expose them as shared libraries with a common ABI. These cores can then be used by so-called Libretro frontends via this API. Here is the main C header if you want to know what it actually looks like.

You can see Libretro as a videogame console emulator plugin definition without a plugin system to make it usable.

Initially, retro-gtk was designed and implemented as a library easing the use of Libretro cores from higher level languages like Vala. It allowed to dynamically load cores and to use them via a GObject API mimicking the names and the behavior of the Libretro one, while overcoming some of its limitations. The main limitation of Libretro is that you can't pass user defined data when calling a core's function and get it back when the core calls back, which makes it impossible as-is to get the identity of the calling-back core and hence make it impossible to have multiple cores running at the same time. This is something we want to avoid as having a parametrable singleton would artificially hinder retro-gtk's API and by extension the code of its users.

You can see retro-gtk as a GObject-based plugin system based on the Libretro plugin definition.

To ease it's development, the library was written in Vala, which at the time seemed like a good candidate to simplify the implementation of a GObject introspectable library — also, when I started writing it I was more proficient in Vala than in GObject C. To allow multiple cores to coexist, two solutions were explored. The first solution consists in storing the calling core in a thread-specific static variable and running each core in their own threads. This forces cores to be run from a different thread and doesn't allow reentrant calls from callbacks. The second solution consists in pushing the calling core on a static stack before each call to one of it's module's functions, and in poping it out of the stack just after. This allows reentrant calls from callbacks but forces cores to be run from the same thread.

A third solution could consist in a mixture of the first two, using thread-specific static stacks, forcing usage of multiple threads and allowing reentrance, but thread-specific static variables being a non-standard compiler feature, the second solution was retained.

Emergent Problems

While developing and using retro-gtk I noticed several problems. Here are the main ones.

Staying close to Libretro's API means staying close to a large and tedious to use API. Each user of retro-gtk would have to solve the same problems in about the same way, hence even though retro-gtk simplifies discovering the available cores, simplifies managing its resources and makes Libretro available from other languages it — by design — doesn't fix the complexity of the original API, making it not as useful as it could be.

You may be wondering how annoying is the API to use. Well, here is a pseudo-code (hence, simplified) example of what Libretro and by extension retro-gtk requires you to do for something as trivial as loading a game into a core.

load_game(core, medias):
    assert medias.length > 0
    core.init()
    if core.has_disk_interface():
        core.disk_interface.open_tray()
        foreach media in medias:
            core.disk_interface.add_index()
            gameinfo = prepare_gameinfo (core, media)
            core.disk_interface.set_index(gameinfo)
        core.disk_interface.close_tray()
    else:
        assert medias.length == 1
        gameinfo = prepare_gameinfo (core, medias[0])
        core.load_game(gameinfo)

prepare_gameinfo(core, media):
    if core.needs_media_path:
        return new gameinfo_from_path(media.path)
    else:
        file = new file(media.path)
        content = file.read()
        return new gameinfo_from_content(content)

The tricks used to allow multiple cores to run side by side were working, but not as well as expected. Storing the calling core on a static stack or in a thread-specific static variable means that if the core calls back from a different thread than the calling one, the identify of the caller can't be retrieved: either the caller has already been removed from the stack or the the thread-specific variable hasn't been set in the calling-back thread. In both cases you get a wrong value, making cores behaving this way not usable at all. A solution would be to ensure having only one core loaded at the same time but as explained earlier, that's a no-go.

The goals of staying close to the original low-level API while exposing it to higher levels is tedious, and hinder the inclusion of some of the more complex features of the original API. And again, doing this offers no real added value to retro-gtk's users.

And finally, it's a bit out of retro-gtk's scope but some Libretro cores aren't very stable and lead users of retro-gtk to crash. It would be great for retro-gtk to fix that in some way.

So… What Now?

All of this really doesn't sound good, but don't worry as the next article will focus on what I am working on to improve the library and make it actually usable and useful!