CRT Filter in Games — SUSE Hack Week

SUSE Hack Week 0x10 finished on previous Friday, during it I wanted to support hardware rendering in retro-gtk.

This project had two sides, the first one was to allow the Libretro cores to draw their video with OpenGL, the second one was to render their video using OpenGL to allow filtering it via shaders. It was unrealistic to do both sides of this project in one week and, being at the time a complete beginner when it comes to OpenGL, I didn't plan to succeed any of these projects but just to learn a lot.

The week before the Hack Week, I took some time to study GtkGLArea and to implement a skeleton for the RetroGLDisplay widget, which was similar to RetroCairoDisplay but was drawing nothing. The GTK+ part of the project being ready, I was able to focus on the OpenGL part during the Hack Week.

Friday

The first day I looked for OpenGL tutorials and I managed to implement an "Hello, triangle" tutorial. I played a bit more with it and got the triangle blinking and then I turned it into a rectangle. Nothing fancy, but you have to start somewhere! 😀

Then I started playing with textures and I managed to load the video output of the core into a texture. I rendered it by blitting it onto the widget's framebuffer with the correct proportions, but the color and orientation were wrong. It was time for the opening session of lightning talks in Nuremberg where I presented the project and showed what I managed to accomplish so far.

By the end of the day, RetroGLDisplay was on par with RetroCairoDisplay: it was simply blitting the video from the Libretro core onto the widget, but it did it correctly.

Saturday, Sunday

I looked for more resources, tutorials, etc. about OpenGL shaders, especially in the context of video filtering.

Monday

I studied and started exploring how to use shaders. It turned out that to draw the video with shaders the widget has to draw a textured rectangle, but by this time I removed the code to draw shapes… oops. 😣 No problem, I just had to reimplement drawing a triangle, load some shaders from files, compile them at runtime and… it wasn't easy. 🙃

Tuesday

After more work, I managed to render the video correctly with simple shaders doing nothing special. I initially wanted to use shaders from RetroArch but they were too complex to use for the beginner I was (and still am). I started looking for other shader sources and decided to try the one ones from Higan, which are GLSL only and simple to understand, I managed to use their scanline shader!

I played with some more complex shaders and managed to get the crt-simple filter to work.

Wednesday

To simplify selecting the shaders, I needed to add some manifest describing how to use them so retro-gtk could load them easily. Higan describes its filters using the *.shader/manifest.bml format — a *.shader directory containing the shaders and a manifest.bml file describing them —, rather than implementing a custom parser I decided to rewrite the manifests as *.filter key files I can easily handle with GKeyFile. The manifests are so short and so few that it's way simpler that way. I also added to the manifests fields containing the authors, the licenses and an URL.

I checked how well RetroGLDisplay ran in Games, then I added the CRT value to RetroVideoFilter and used it in Games to select the crt-simple filter. The bicubic and sharp filters were also included to implement the SMOOTH and SHARP filter values respectively.

Thursday

On the last day of the Hack Week, I cleaned up the code and fixed the last bugs I encountered. All of this seemed rock solid so I merged them on master for both retro-gtk and Games!

During the closing session of lightning talks I was happy to present a project which turned out way better than I expected! 😁

It marked the end of the Hack Week, but not of the project!

Friday, Saturday

exalm complained on IRC of slowdowns with the hardware accelerated renderer, and actually so did I on some of my personal machines. 😐 Bastien Nocera suggested to synchronize the rendering requests sent to the widget with the ticks of the widget's drawing clock, avoiding some useless draws and hence saving time and power. It did improve the performances in fullscreen on my tests, but the perfs were abyssal when there is an overlay widget on top of RetroGLDisplay. None of our tests managed to find a slow point on the retro-gtk side, but GTK+ developer Benjamin Otte suggested to check gdk_cairo_draw_from_gl() as it may be what causes the slowdowns.

Other small performance improvements could be implemented and so I they were. I dropped the bicubic and sharp GLSL filters to instead directly blit the texture as was done on Monday, shader-based rendering is still used for the CRT filter and will be used for any other filter in the future. The video from the Libretro cores was first converted into a GdkPixbuf by the CPU before being loaded into a texture, the texture is now loaded directly from the video output of the core, avoiding to do the color conversion on the CPU.

What's Next

Despite the performance problems I'm overall happy with how this project turned out. We need to explore the function pointed at by Bejamin, but in the mean time rather than reverting the OpenGL support we could introduce an environment variable to use RetroCairoDisplay instead, similarly to WEBKIT_DISABLE_COMPOSITING_MODE in WebKitGTK+.

Currently the way video filtering is implemented doesn't exactly match the one from Higan, which means not all its filters can be used. I would like retro-gtk to support any Higan filter so we can add some HD/cartoony one like the supernxsai family, some LCD one and filters like Game Boy mimicking filters.

That was exhausting, but fun! 😄