NEStronic Game Music Synthesizer (a work in progress)

So there's this little project I've been working on for the past few weeks, which I think is at a good stage to start showing off. Its still very much a work-in-progress, but the most critical components are now working.

What I'm building is a contraption that can play NES game music using an authentic RP2A03 CPU/APU to do the audio synthesis. Beyond the RP2A03 itself, however, I'm attempting to do this with entirely modern components.

My system is basically composed of two halves:

The NES half contains the RP2A03, an SRAM chip, and EEPROM chip, an audio amplifier, and an I2C interface.

The "modern microcontroller" half currently contains an Espressif ESP32-WROOM and a microSD card interface. Eventually, I intend for it to have a display and some buttons for control. (I'm also seriously considering wrapping up a version of this project into an alarm clock, which uses the ESP32's Wi-Fi for network time sync.)

The system currently works by reading the music from VGM files, and generating a command stream to tell the 2A03 what to poke at its APU registers.

Here's a video of my first end-to-end test:
https://www.youtube.com/watch?v=97jic_WRrwY

And a follow-up playing a different track:
https://www.youtube.com/watch?v=eafaFr9Q_rU

And here's the work-in-progress schematic of the NES half of the system:


At the outset of this project, the VGM format seemed like an ideal thing to start with. Lots of rips already available, and the format was literally "poke this value into that register, sleep for X samples, poke this value into that register, etc." In very little time, I was actually able to write a quick-and-dirty C program that could convert VGM into a dump of 6502 assembly (or data easy to iterate over in 6502 assembly). Of course it filled my memory to play only a few seconds of Super Mario Bros this way, but that was enough to give me a huge grin.

As I got deeper into it, I also learned about NSF. It seems like NSF has an even larger catalogue of rips and is more populate in the chiptune community. Unfortunately, NSF seems to only be viable on emulators (or if you have just the right combination of "architecturally correct NES /w whichever bank switching chip a particular game felt like using"). So when/if I decide to consider dealing with NSF files, I think I'm going to have to basically convert it to VGM first. (Either by hacking an emulator on my PC, or by running a stripped down emulator on the ESP32 that captures APU writes and sends them across.)

Anyway, enough rambling for now, and I'll save any involved back-stories and inspirations for a future blog post. I just felt like I had to get some sort of forum post about this project out there.

NSF always uses the same bankswitching hardware, or none at all. The problem for your current design is that the no-bankswitching-at-all model expects something resembling a 32K ROM in the upper half of the address space.

As far as VGM consuming too much memory, I'd think you should be able to build a message-passing loop between the 2A03 and the ESP32 ...

For whatever it's worth, a bare 2A03 doesn't have to worry about where it maps things: the bus conflict that would happen on reads from $4016 and $4017 won't happen (because that's external, between the 74'368 on the mainboard and whatever other hardware is present) OR on any conflict that would happen on reads from $4015 (because the NES hardware special cases that address and ignores the data bus on reads from that address).

(When the 2A03's TST pin 30 is high, reads from the entire range from $4000 through $401F ignore the external data bus entirely)

I know its probably lost in the middle of my attempt at writing a "brief" post, but this is actually exactly what I did. I only tried cramming data into the EEPROM in my early "let's see if I can make sound" tests. My later tests (which are in those video links) use the ESP32 to read in the VGM file and generate a sequence of I2C commands that tell the 2A03 what to do. This works quite well, doesn't require much 6502 code, and is how I plan to make this all work.

Glad to see that was the obvious thing to have done

I'd think you'd also be able to emulate the 6502 and NSF bankswitching hardware on the ESP32, and relay the audio writes to the physical 2A03... basically dynamically generating the VGM at play time. That's probably also what you meant ...

Something I'd think would be fun from a musical point of view, although not particularly useful from the point of view of replaying existing soundtracks, is using some kind of reconfigurable clock generator to choose what frequency the 2A03 runs at. Could be nice to get an extra octave off the bottom, or more precise tuning with higher notes, or different DPCM tuning. (Then again, most of the VGMs of Master System games I've listened to are dramatically improved by transposing everything down an octave. The 2A03 has a much less limited musical range, but I can't say that won't be true here too.)

Is it because their developers weren't aware of the alternate mode for pulse 3 that's 4 octaves down? That's two octaves below even the 2A03 triangle.

This is an idea that has never occurred to me, but actually something worth considering as part of a follow-on project (in addition to exploring add-on audio chips only used on the Famicom). Programmable oscillators are now a thing (a Sega Genesis VGM player project used them in lieu of weird crystal frequencies) so its definitely do-able.

Thankfully I actually was able to still get a new crystal at the weird frequency used by the 2A03, otherwise I'd have already built in this capability.

So 6 * NTSC is "weird" now?

Is 4 * NTSC still gettable, and is PLL multiplication by 3/2 practical?

6×NTSC crystals have been verging on "special order" territory for years now.

Not that special order is that much of a problem.

No PLL is needed; extracting the 3rd harmonic of a square wave using an adjustable LC tank is an easy thing.

As I continue to work on this project, I decided to go ahead and start writing blog posts to describe it in more detail.

Here's the first post:
Introducing the Nestronic! (Part 1)

This blog post mostly covers what I already said in my first post on this forum thread, but goes into a lot more detail. (In the next post, I plan to cover the system architecture and schematics.)

As far as the actual project, the schematics are mostly finalized. I'm now neck-deep in PCB layout, and hope to get to the point of building some real (non-breadboard) hardware in the next month or so (time and resource permitting).

It feels pretty crazy that I do actually have this all assembled across three breadboards, but the general flakiness and noisiness of breadboard circuit connections is getting old.

I didn't actually have problems with assembling my (8051-based) college final microcontroller project class on two of those breadboards ... but it was a brand-new one purchased just for the project.

Most of these were brand-new too.
Of course its often hard to determine which breadboards are "good" vs "crap" before you buy them. They all have certain issues beyond that, which may or may not manifest in your project.

The basics of this system actually worked just fine. I don't think I ever had a problem where the parts of the 2A03 "computer" failed to communicate with each other, or the ESP32 couldn't talk to some peripheral because of these breadboard issues.

The most annoying breadboard, mechanically-speaking, was the triple-wide one I moved the 2A03 system onto mid-way through the project. It took a lot more force and fiddling to get component pins into its holes, versus all the others. I think I have a lot of resistors with squiggly pins as a result of that.

From a flaky-connection perspective, I had the most issues with the connections between the audio output pins, amplifier, and speaker. It seemed like poking (or breathing) on various resistors and wires in that chain would determine whether or not I got good sound out of it.

Most of the other electrical issues came down to the effects of noise. I could detect the frequency of the crystal oscillator from all over the breadboard. However, I don't think this caused any serious issues. But what did cause sporadic issues, is the noise that appeared on the OUT ($4016) pins of the 2A03. The "low" value for these pins tended to float between ~600mV and ~800mV depending on when I measured it. When probed with an oscilloscope, I found ~300mV p/p noise on it. (Since I'm using those pins to control whether the amplifier is on or off, and the amplifier chip wants it <800mV for "off", this obviously caused issues.) Fingers crossed that this vanishes once I go to real PCB, otherwise I'll need to add a buffer between the two chips.

The only really big difference I see between my board and yours is that I more aggressively made ground/power supply rails into a "plane", by connecting the +5V and ground rails at both top and bottom. (Yes that causes ground/power supply loops, no it seemed to have worked out) My design mixed baseband composite video on the same board as a bunch of discrete logic (implementing a genlocked framebuffer)

Looking at your board, power consumption in any of the left column will cause power supply sag as seen by the 2A03, and there's also no bypass capacitor for it.

But don't let me stop you from actually changing to a PCB.

Yeah, I have had issues with voltage drops across this breadboard setup. They were actually far more noticeable on the 3.3V rail the ESP32 is hanging off of, as they didn't actually cause problems on the 2A03's 5V rail. As its currently configured, the regulators have separate wires to each vertical track, which helps with it. I'll obviously do better on the PCB layout.

The bypass capacitors are missing on the breadboard because they were actually causing problems. When I had them installed, the crystal oscillator circuit sometimes didn't work correctly on startup. However, the actual schematic has bypass capacitors on all the ICs, so they'll definitely be on the real PCB.

The next blog post is ready:

Nestronic System Architecture

This post details the architecture of the system, and shows all the circuit schematics.

As far as actual development goes, PCB layout is mostly complete and I've begin to work on the design of the enclosure. Once I have a bit more certainty on my reflow soldering assembly solution (long story), I'll actually order the PCB and components to start putting this thing together for real.

Just finished getting the first prototype PCB assembled and mounted in a half-enclosure. Here's a video of it in action:
https://www.youtube.com/watch?v=izMFPKmD5ZU

And now with the rest of the enclosure put together:
https://www.youtube.com/watch?v=pXp97wzkQEE

Finally finished writing the next blog post, which covers the assembly and testing of the main board for the first real prototype:

Building the Nestronic Prototype (Part 3)
http://hecgeek.blogspot.com/2018/05/nestronic-3.html

(Maybe someday I'll blog without a multi-week lag before doing something and having a writeup ready... Someday...)

Oh. Yeah. Oops.

I wonder what the simplest way is to remap the RAM to available to hold DPCM for the DMA unit to handle.

I'd be tempted to split the 8K into two 4K windows, divided by A14, and enable RAM any time A13 was low. Then there'd be 4K for internal RAM at $0000, mirrored to $8000, and 4K for DPCM at $C000, mirrored under the registers at $4000. (You don't have to worry about a bus conflict on reads from $4015, in case anything does: the 2A03 internally prevents it). But something more complicated is probably nicer anyway.

Splitting the RAM into two 4K windows is one idea I've considered. Another idea is to simply mirror it at $8000, and make sure I don't allow any addresses into the DMC registers that overlap RAM being used by my program code.

The former idea gives me a simpler memory model, while the latter frees up a little bit extra RAM for this purpose. (Yet another idea would be to add a second SRAM chip, but that would require a lot more signal routing.)

Problem is that DPCM DMA can only fetch from $C000-$FFF1¹, so you have to prevent the writes to $40xx from colliding with zero-page usage. But I guess you're only running your own software, so maybe that's acceptable.

¹you can start a DMA at some high address and when it overflows it wraps around—after fetching from $FFFF it will fetch from $8000—but that's just goofy and overlaps with the vectors.

Or replace the 8K RAM with a 32K RAM. Would need only two extra traces over the 8K RAM.

I originally thought the DMC start address could be set so I could wrap around and begin at $8000. But upon double-checking, it seems that's not the case. Thanks for making me double-check.
That being said, I can still mirror my SRAM to $C000-$DFFF and hopefully be okay. My VGM player code is going to have to remap DMC memory blocks anyway, since I can't place them on top of my program ROM, so I'm prepared to deal with that.

I hadn't considered using a 32KB SRAM (or some other size), and splitting it across areas of the memory map, but its also not a bad idea. If done right, it might even make the address decoding glue simpler than mirroring the 8KB SRAM.

I'm definitely going to sketch out all these ideas before I go ahead and make the "Rev B" design changes to my board. Until then, I guess I just can't play the SMB3 soundtrack.

If anyone is interested in following along with the actual development, the entire project (code + schematics) has recently been moved up to GitHub:
https://github.com/dkonigsberg/nestronic

Since I just finished building revision B of the project design, I threw together an assembly video. It covers the whole process, start-to-finish. Hopefully its decent:
https://youtu.be/UlO0NQTAaJU