reference_frontend

Description

This is a barebones minimal reference frontend for the libretro API, designed for developers (not end users), to show the most basic functionality necessary to run a libretro core in a playable state in a cross-platform manner (at least Windows and Linux, probably OSX as well). By playable I mean that input, audio and video all work to a minimum degree. SRAM and save states are also supported. And most importantly, both software and OpenGL-accelerated cores are supported.

This project requires Qt5 version 5.9 or later (Qt6 is also supported but as of 6.0 the multimedia module has not been ported yet, so you will not have sound), and a C++11 toolchain.

Building

The simplest way to build this project from the command-line is with qmake && make -j$(nproc). This should work on Linux as well as Windows via MSYS2 at the least. If you prefer an IDE-based approach, you can also build with either Qt Creator, or Visual Studio with the Qt add-in, just open the "reference_frontend.pro" file and click Build in either case.

Program/API flow

The basic idea is that we will be loading in a dynamic library from disk at runtime; this is the core that you want to run. It is assumed that you already have access to a compiled core library whether built yourself or acquired from elsewhere. After the library is loaded, we will acquire some function pointers from the library and then later call those functions to load content and/or render frames inside the core. Finally, we will display those frames (video and/or audio) on the screen, and relay inputs from the user back to the core.

The general flow of logic is as follows:

• frontend/main.cpp

  • Initial entry point of the program itself, where main() is located.
  • Process command-line arguments to set the appropriate paths to the core library and content file we want to run.
  • Construct our "MainWindow" class that handles the high-level GUI functions like drawing the window and receiving input.

• core/core.cpp

  • A static instance of this class is created automatically once the main window is shown. It handles all of the communication with the core library. The other files in the core/ folder are all functions in this same class, just broken out by their common functions like audio or video.
  • The core library is loaded in library.cpp (Core::load()) and all necessary function pointers are resolved. Most of these functions are ones that we will call ourselves which directly do something inside the core (like load a content file, save state or render a frame), but some of them (most of the ones with "set" in the name) only take pointers to functions of our own and are then later called as necessary by the core (for example to signal when a video frame has finished rendering).

• core/library.cpp

  • As soon as the "retro_set_environment" callback is registered, the core will start calling our Core::environment() callback any time it wants, including even before we call retro_init(). This interface lets the core issue a set "command" to the frontend with some optional data attached to it. See the section on core/env.cpp below for the necessary commands you should implement.
  • Outside of the environment stuff, the first function we must call after loading the core is retro_init(). This is done in the Core::load() function.

• core/content.cpp

  • Next, we call retro_get_system_info() (in Core::run()) which tells us generic info about the core, like its name, version, and what file extensions it supports for content.
  • The next step is to ask the core to load some content. First we must fill in a "struct retro_game_info" with the path to the content as well as its size and a pointer to the content data itself (optional in some cases, see "need_fullpath" in libretro.h)
  • Now we call retro_load_game() and give it our retro_game_info struct we just filled in. Some cores might check the content for validity at this point and return an error if so.
  • retro_get_system_av_info() is called to fill in a "struct retro_system_av_info" for us, this holds audio/video details about the "system" (which may be content-specific) like its video size/framerate or its audio sample rate.
  • If the core is hardware-accelerated, we now have enough info to create an offscreen framebuffer object (FBO) as the target for the core to render frames to. The API calls this "get_current_framebuffer" approach "rather obsolete" but other frontends and even RetroArch seems to continue to use it.
    • Now call context_reset().
  • At this point our frontend prepares the audio output (in Core::setupAudio()) by specifying the desired output format (number of channels, sample rate etc). If your audio renderer cannot natively handle the sample rate output by the core, then you may have to handle the resampling yourself.
  • Now we load any existing SRAM into the core that the user may have saved previously (Core::loadSRAM()).
  • Finally a repaint() function is called on a timer that, at a glance, would appear to run as fast as possible (it uses 0 as the timeout parameter). More on that later.

• frontend/video.cpp

  • The repaint timer will cause the main window to fire its paintGL() and paintEvent() functions in this file. After the main window is done compositing each frame, a frameSwapped() signal is fired by the underlying framework.
  • If the core is hardware-accelerated, paintGL() will "blit" the offscreen framebuffer to the "default" onscreen framebuffer and the contents will become visible.
  • After frameSwapped() is fired, retro_run() is called to render a single frame (Core::onGotFrameSwap() in core/video.cpp).
    • When retro_run() is called, the core will fire our "retro_set_video_refresh" and "retro_set_audio_sample_batch" callbacks (Core::videoRefresh() and Core::audioSampleBatch()). (there is also a "retro_set_audio_sample" but it is not used much)
      • The parameters given to the video refresh callback are only useful for software-rendered cores. It always provides the exact geometry and a pointer to the image data it's providing on every frame. Pay extra close attention to any changes. NOTE: Video dimensions can change dramatically from one frame to the next! It's up to you to notice (and handle) this. It may change anywhere between the base and max geometry of the AV info. The "pitch" may also contain extra padding you need to account for when reading the image data.
  • Once the core's frame is rendered internally, we then draw the resulting image onto our main window, if this is a software-rendered core.
  • Normally this whole process would cause the core's video/audio output to play back at warp speed, but because our main window is backed by an OpenGL surface, we get vsync for free, and that "zero timer" really only fires at the refresh rate of the monitor because it is stalled by the vsync.
    • To prove the vsync point, if you were to call setSwapInterval(0) in main.cpp (it's commented out), the core should play back as fast as possible.
    • This also means that cores which do not render at (at least very close to) the refresh rate of the monitor will play back at the wrong speed. Dealing with this is left up to the individual developer and out of the scope of this project. You may need to duplicate/drop frames or employ other techniques to achieve proper time sync.

This should cover the main sequential steps necessary to get a core running. Other asynchronous/event-based handling is covered below.

Other considerations

• frontend/audio.cpp

  • Our MainWindow::onAudioSampleBatch() callback handles the audio frames output by the core. The format is always signed 16-bit interleaved stereo.
    • For example the data in memory looks like: frame 1 - left, frame 1 - right, frame 2 - left, frame 2 - right, etc.

• frontend/input.cpp

  • This project only uses the keyboard and mouse for input as an easy example. Supporting gamepads and other peripherals is out of the scope of this project.
  • Once a key is pressed or released on the keyboard, the main window callbacks keyPressEvent/keyReleaseEvent are fired. In there we set the state of any button we care about in a "inputState" array for later.
  • When the running core needs to know what buttons are being pressed, it checks them during retro_run() using our "retro_set_input_state" callback (InputState::getInputState() in common/input.cpp).
  • The core may also fire a "retro_set_input_poll" callback which is just a suggestion to the frontend on when to check the state of its inputs/buttons (the core still won't know the actual state until it calls its retro_set_input_state callback).
    • In our case we don't need this input_poll callback because the keyboard key states are already being tracked as soon as they are pressed, since the underlying framework fires its own callback for it.
    • Your own frontend may alternatively choose to poll its own input state in other ways either before, during or after retro_run(). This is what RetroArch's "polling behavior" setting does. The "normal" setting relies on the core to call input_poll for you, and early/late means the frontend calls it in that way relative to retro_run() (before or after).

• core/env.cpp

  • The following commands I found to be necessary for minimum core functionality in most cases:
    • RETRO_ENVIRONMENT_SET_VARIABLES (tells the frontend what core options are available)
    • RETRO_ENVIRONMENT_GET_VARIABLE (asks the frontend for the current value of a core option)
    • RETRO_ENVIRONMENT_SET_SUPPORT_NO_GAME (tells the frontend we support running without any content loaded)
    • RETRO_ENVIRONMENT_SET_INPUT_DESCRIPTORS (tells the frontend what buttons it supports on what kind of devices, for however many users)
    • RETRO_ENVIRONMENT_GET_VARIABLE_UPDATE (asks the frontend if it changed any core variables)
    • RETRO_ENVIRONMENT_GET_SAVE_DIRECTORY (if the core needs to handle saving on its own, this asks the frontend for a path to do so in)
    • RETRO_ENVIRONMENT_GET_SYSTEM_DIRECTORY (if the core needs to load a BIOS or other support files, this asks the frontend for a path to find them in)
    • RETRO_ENVIRONMENT_GET_LOG_INTERFACE (asks the frontend for a callback to send core log messages to)
    • RETRO_ENVIRONMENT_SET_HW_RENDER (asks the frontend if hardware-accelerated cores are supported, and if so, instructs the frontend to set some extra callbacks)
    • RETRO_ENVIRONMENT_SET_GEOMETRY (tells the frontend that the core has changed its base video output size (may be temporary, can change at any time))
    • RETRO_ENVIRONMENT_SET_SYSTEM_AV_INFO (tells the frontend there was a major change in the A/V setup of the core, in RetroArch's case this often means an entire teardown/reinit of the whole application, but we don't really need to for the scope of this project)

• frontend/mainwindow.cpp

  • When the window is closed, we make sure to save our SRAM and core options to disk before attempting to stop the content/core in any way, just in case it crashes. We don't want the user to lose their work.

Usage

Usage: ./reference_frontend [options] core rom
reference_frontend

Options:
  -h, --help        Displays help on commandline options.
  --help-all        Displays help including Qt specific options.
  -v, --version     Displays version information.
  -f, --fullscreen  Start in fullscreen mode
  -m, --mute        Start with audio muted

Arguments:
  core              Core library to use
  rom               ROM file to use

Any platform that supports Qt 5.9+ and a GPU with OpenGL 2.0+ (and proper drivers) should be able to run this. If you're on Linux, both X11 and Wayland should work, including directly on the console via DRM/KMS using the -platform eglfs option.

Key bindings

The key bindings are hardcoded as follows. Re-binding of the keys is outside the scope of this project. Basic mouse support for cores that utilize it should also work.