In this thread, gdkchan asked about A/V synchronization in his emulator. Here, I'll discuss the method I developed for nemulator.
Most emulators sync to the audio playback rate and adjust the video rate to accommodate. This results in tearing (no vsync) or stuttering when scrolling (vsync). nemulator syncs to the video frame rate, and continuously adjusts the audio rate to match. The result is smooth video and glitch-free audio. Here's a block diagram of how it's done.
Input is audio at its native rate (1.79MHz in the case of NES). The input is low-pass filtered and then passed to the arbitrary rate resampler. The output from the resampler is high pass filtered, then stored in an output buffer. When the soundcard needs data, the callback function is invoked, pulling data from the buffer.
Each frame, a frame's worth of audio data is processed. When it is placed in the circular output buffer, the distance between the write and play cursors is measured. The goal is to keep this distance consistent between frames so that the audio buffer doesn't underflow, resulting in audio glitches, and so that no blocking occurs while waiting for the buffer to drain, resulting in delayed frames/video glitches.
The feedback block:
- Low pass filters the distance measurement
- Adds that measurement to a circular buffer containing the last 60 measurements (1 second)
- Computes the slope of those measurements
- Adjusts the resampler output rate based on how far the distance is from the target and whether it's converging on or diverging from that target
I've created a sample SDL app that implements this system. Source code is available on github, or you can download a Windows version here (requires sdl2.dll). I've only tested this on Windows, but it should work on other platforms. A few notes:
- nemulator uses DirectSound without callbacks, which permits for more accurate measurement of the audio buffer's cursors and, thus, better synchronization
- Accuracy in this app is limited by the callback frequency (every 512 samples), so some tweaking may be required
- Pre- and post- resampler filtering is not implemented
- Input to the resampler should be at least 8x oversampled
- The green line represents the distance measurement, the red line represents the resampler output frequency
- If your display isn't running at 60Hz, this won't work properly
James
Most emulators sync to the audio playback rate and adjust the video rate to accommodate. This results in tearing (no vsync) or stuttering when scrolling (vsync). nemulator syncs to the video frame rate, and continuously adjusts the audio rate to match. The result is smooth video and glitch-free audio. Here's a block diagram of how it's done.
Code:
audio callback -> sound card
^
|
|
input -> low pass -> resampler -> high pass -> output buffer
^ |
| |
+---------feedback----------+
^
|
|
input -> low pass -> resampler -> high pass -> output buffer
^ |
| |
+---------feedback----------+
Input is audio at its native rate (1.79MHz in the case of NES). The input is low-pass filtered and then passed to the arbitrary rate resampler. The output from the resampler is high pass filtered, then stored in an output buffer. When the soundcard needs data, the callback function is invoked, pulling data from the buffer.
Each frame, a frame's worth of audio data is processed. When it is placed in the circular output buffer, the distance between the write and play cursors is measured. The goal is to keep this distance consistent between frames so that the audio buffer doesn't underflow, resulting in audio glitches, and so that no blocking occurs while waiting for the buffer to drain, resulting in delayed frames/video glitches.
The feedback block:
- Low pass filters the distance measurement
- Adds that measurement to a circular buffer containing the last 60 measurements (1 second)
- Computes the slope of those measurements
- Adjusts the resampler output rate based on how far the distance is from the target and whether it's converging on or diverging from that target
I've created a sample SDL app that implements this system. Source code is available on github, or you can download a Windows version here (requires sdl2.dll). I've only tested this on Windows, but it should work on other platforms. A few notes:
- nemulator uses DirectSound without callbacks, which permits for more accurate measurement of the audio buffer's cursors and, thus, better synchronization
- Accuracy in this app is limited by the callback frequency (every 512 samples), so some tweaking may be required
- Pre- and post- resampler filtering is not implemented
- Input to the resampler should be at least 8x oversampled
- The green line represents the distance measurement, the red line represents the resampler output frequency
- If your display isn't running at 60Hz, this won't work properly
James