About a year ago I wrote a tool that would automatically take a (monaural) audio file and convert it to a series of bytes you could write to the NES noise channel control registers. This is specifically useful for making percussive sounds, and works best on snare drum and high hat samples.
I kept on meaning to go and make it less fragile but never got around to it. I figure it'd be better to post it here now, so that either 1- someone else benefits from it 2- someone else inspires me to fix it.
There are a few samples inside. You probably are most interested in noise.asm and noise.obj. (I'm using the xa65 assembler, because it's the only one in debian.). The actual converter itself is analyze.pl -- it's verbosely documented, so if you want to rewrite it in a different language you should be ok.
How it works:
The tool takes a reference set of sounds generatable by the noise hardware, takes their FFTs, and then compares this to the FFTs of a file to be converted, 1/60th of a second of audio at a time, finding the best match. (This is a kind of
granular synthesis)
Caveats:
* This tool needs perl, PDL, and PDL::Audio.
* I wrote it on Linux. It might work on windows, but I've not even looked at it.
* I generated the reference audio using FCEU which I'm certain isn't very accurate.
* The closest-FFT algorithm is the wrong one to choose for the short-loop mode of the noise hardware, so tonal things convert things badly
* Conversion is actually simply wrong for converting files not sampled at 48kHz. (it's a tempo-preserving pitch shift of the file by 48kHz/input rate)
Anyway: -h-t-t-p-:-/-/-e-a-m-p-.-o-r-g-/-l-i-/-n-o-i-s-e-g-r-a-i-n-.-7-z- [937kB] It's only so big because I'm including the reference noisewaveforms.wav -- generating it involved stupid hackery.edit x2: Please use the C version:
Attachment:
preanalyze.c [6.28 KiB]
Downloaded 119 times
, it doesn't have any of the build obnoxiousness of the PDL version
Attachment:
preanalyze2.c [8.58 KiB]
Downloaded 116 times
is the same but also produces a residual (FFT resynthesis of error signal)
A demonstration NES image for the C version is at
Attachment:
newnoise.nes [16.02 KiB]
Downloaded 130 times
, or read on from
http://nesdev.com/bbs/viewtopic.php?p=69538#p69538
Wow this sounds absolutely awesome ! Very usefull for percussion and sound effects too.
Cool idea but somewhat limited if only using the noise channel. There are pitch/tonal qualities to many percussion sounds that can't be simulated with noise alone.
I always found the best way was to do it by ear - load a drum sample into a sample editor and play it back at 1/2 or 1/4 speed. You can then clearly here the changes in pitch over time and mimic this in your wavetable tables.
Having said that, I didn't actually get to hear the generated sounds (for some reason many of the files came out garbage when I unpacked the archive) so they could be awesome
The reference sounds good enough. The noise channel isn't that hard to emulate correctly once you understand how the shift register works.
Can't test it now though, since Cygwin's perl doesn't come with PDL.
neilbaldwin wrote:
Cool idea but somewhat limited if only using the noise channel. There are pitch/tonal qualities to many percussion sounds that can't be simulated with noise alone.
Yeah -- I've seen the simulators that use one sine and one white noise source behind a VCF, that work remarkably well. My rationale for not digging into that was 1- it's a lot harder 2- the NES only has 5 channels so I'd rather see what I can do with one. Since the noise channel only has 512 (really 481) different unique sounds it can make, searching the space by brute-force is straight-forward.
Quote:
I always found the best way was to do it by ear - load a drum sample into a sample editor and play it back at 1/2 or 1/4 speed. You can then clearly hear the changes in pitch over time and mimic this in your wavetable tables.
I never thought of trying to do it by ear -- but that makes sense.
Quote:
Having said that, I didn't actually get to hear the generated sounds (for some reason many of the files came out garbage when I unpacked the archive) so they could be awesome :)
Some of them are compellingly close, many are so-so. However, I should fix this problem -- does 7z complain about the files, or something else? (MacOS, Windows, and linux disagree about what marks a newline in a text file -- if it's just the text files that are giving you trouble)
It might be easier just to fit a 2-pole linear predictive model to the sound using the Levinson-Durbin algorithm, determine its pole frequency, and find the noise pitch that has the closest pole frequency.
lidnariq wrote:
neilbaldwin wrote:
Cool idea but somewhat limited if only using the noise channel. There are pitch/tonal qualities to many percussion sounds that can't be simulated with noise alone.
Yeah -- I've seen the simulators that use one sine and one white noise source behind a VCF, that work remarkably well. My rationale for not digging into that was 1- it's a lot harder 2- the NES only has 5 channels so I'd rather see what I can do with one. Since the noise channel only has 512 (really 481) different unique sounds it can make, searching the space by brute-force is straight-forward.
Quote:
I always found the best way was to do it by ear - load a drum sample into a sample editor and play it back at 1/2 or 1/4 speed. You can then clearly hear the changes in pitch over time and mimic this in your wavetable tables.
I never thought of trying to do it by ear -- but that makes sense.
Quote:
Having said that, I didn't actually get to hear the generated sounds (for some reason many of the files came out garbage when I unpacked the archive) so they could be awesome
Some of them are compellingly close, many are so-so. However, I should fix this problem -- does 7z complain about the files, or something else? (MacOS, Windows, and linux disagree about what marks a newline in a text file -- if it's just the text files that are giving you trouble)
Don't get me wrong, I think it's a noble effort and if you could make it work with pitch too I'd definitely use it.
I'm currently writing a NES audio engine (known around these parts as 'Nijuu') and my drum synthesis wavetables use any or all of the NES hardware voices. I recently uploaded an editor I made on the NES itself to enable you to live-preview your sounds;
http://dutycyclegenerator.com
(apologies for the lack of navigation, scroll down to bottom and work up)
Re. 7z. That was my fault. Years ago I'd installed some stand-alone (flaky) unzipper for 7z files. I've just thrown it in the bin and unarchived it properly and it's not corrupted any more.
I've still not heard the sounds yet though - the .NES file won't open in Nestopia (I'm on OSX).
I got the code compiling with ASM6. Pretty interesting experiment though I was right about the sounds
I would really like to see this capable of creating FTI (FamiTracker Instrument) files for the noise channel.
Thought I'd help those who are too lazy to compile xa65 for themselves.
http://average.truechiptilldeath.com/ne ... .5-w32.zip
There's the "Windows version" compiled with Cygwin.
Also... I can almost get it working in Windows, but it is really picky on the version of Types.pm used. Could you supply that please?
B00daW wrote:
Also... I can almost get it working in Windows, but it is really picky on the version of Types.pm used. Could you supply that please?
That's so weird. anyway,
http://eamp.org/Types.pmtepples wrote:
It might be easier just to fit a 2-pole linear predictive model to the sound using the Levinson-Durbin algorithm, determine its pole frequency, and find the noise pitch that has the closest pole frequency.
That would make sense... My background is more in DIP than audio, so I've not really done any work on pole estimation. It didn't help that my preceding class on DSP used a book where the author was very down on same.
The other problem is that the different values written to $400E have different loudnesses, too. Which is why I'm searching the entire 512-entry space of frequency and volume rather than just the 32-entry space of frequency.
neilbaldwin wrote:
Don't get me wrong, I think it's a noble effort and if you could make it work with pitch too I'd definitely use it.
Do you mean fix the sample-rate thing? Or using two voices?
lidnariq wrote:
neilbaldwin wrote:
Don't get me wrong, I think it's a noble effort and if you could make it work with pitch too I'd definitely use it.
Do you mean fix the sample-rate thing? Or using two voices?
Yep, using multiple voices. I appreciate that that may make it massively more complicated.
Seems like I'm the only one concerned about this though so don't go out of your way to accommodate me - I'm quite happy using my little editor.
Though if you like a challenge......
Absolutely awesome sounds like a mild compliment in this case. I would never have imagined that you could make the noise channel sound so much like real samples while still only updating its registers once per frame. It's always fun to see how people's accomplishments in NES music still surprises us.
Dwedit wrote:
The reference sounds good enough. The noise channel isn't that hard to emulate correctly once you understand how the shift register works.
Can't test it now though, since Cygwin's perl doesn't come with PDL.
Check out ActivePerl and use the xa65 compile above. I just ran the MakeFile.PL in the root of PDL and dumped whatever libs from PDL it needed in Perl's /lib directory.
lidnariq wrote:
tepples wrote:
It might be easier just to fit a 2-pole linear predictive model to the sound using the Levinson-Durbin algorithm, determine its pole frequency, and find the noise pitch that has the closest pole frequency.
That would make sense... My background is more in DIP than audio
If the "I" in DIP refers to two-dimensional images, I might still be able to help you. Ever heard of deconvolution to reverse a blurring operation? If you have, I could explain what I mean using related terms.
Quote:
The other problem is that the different values written to $400E have different loudnesses, too. Which is why I'm searching the entire 512-entry space of frequency and volume rather than just the 32-entry space of frequency.
I took differential equations, linear algebra and group theory in college. My professors always used to recommend solving problems by changing the domain: "go somewhere, do something else, and come back". The "somewhere" in question might be the Laplace domain, the z-transform domain, the Fourier domain, etc. In this case, you could 1. change the volume of the input sample such that the sum of squares matches that of your reference samples, 2. match against reference samples' power spectra, and 3. change the volume back.
I can't get the necessary dependencies working on Cygwin at all. Would someone post some WAV or NES files demonstrating the program?
Dwedit wrote:
I can't get the necessary dependencies working on Cygwin at all. Would someone post some WAV or NES files demonstrating the program?
I thought I'd included them in the .7z -- the .wav files were converted and are in the noise.obj (rename it to noise.nes) -- they map as:
cymrsmp.wav - A
hat.wav - B
bd.wav - select
pwrsnare.wav - start
Note that all of these are not sampled at 48kHz, so all of them are erroneously pitch-shifted by my tool.
tepples wrote:
If the "I" in DIP refers to two-dimensional images, I might still be able to help you. Ever heard of deconvolution to reverse a blurring operation? If you have, I could explain what I mean using related terms.
Sure! That'd be useful.
tepples wrote:
In this case, you could 1. change the volume of the input sample such that the sum of squares matches that of your reference samples, 2. match against reference samples' power spectra, and 3. change the volume back.
Right -- the problem is that the highest frequencies of noise (0-3 = can be thought of us 4, 8, 16, and 32 point FIR ~lowpass all-ones filters) all have their first zeroes at frequencies so much higher than we can hear (447kHz, 224kHz, 112kHz, 56kHz respectively) that there's basically no useful attenuation in the audible spectrum to match those by anything except volume -- they're basically just white. (And in the 24kHz bandwidth my sound card can produce, they functionally are white)
I'd like to hear what the program outputs for drumloops.
Dwedit wrote:
I'd like to hear what the program outputs for drumloops.
No good. It keeps the snare and hat sounds; roughly as expected, the bd and tom don't convert. Cymbals convert so-so.
http://eamp.org/noise.nes
now, in addition-
Down - sample # 9350 from freesound (
http://www.freesound.org/samplesViewSingle.php?id=9350)
Left - sample #3455 from freesound (
http://www.freesound.org/samplesViewSingle.php?id=3455)
Still cool how it sounds like it plays the cymbals and hi-hats simultaneously on the DOWN sample.
Dwedit wrote:
Still cool how it sounds like it plays the cymbals and hi-hats simultaneously on the DOWN sample.
In a traditional NES music engine, that would correspond to treating the cymbals as "instruments" and the hi-hats as "sound effects".
Anyway, here's the deal:
Linear prediction tries to find a filter that "predicts" the next sample given the last few samples. Analysis (used in compression) is like convolution, giving the signal a more "white" spectrum; synthesis (used in decompression) is like deconvolution, reconstituting the original spectrum. Once you have the filter, you can take its
Z-transform to find the "poles" and "zeroes", which are related to which frequency bands a filter increases or decreases.
tepples wrote:
Dwedit wrote:
Still cool how it sounds like it plays the cymbals and hi-hats simultaneously on the DOWN sample.
In a traditional NES music engine, that would correspond to treating the cymbals as "instruments" and the hi-hats as "sound effects".
I have my drums on a virtual track to achieve this effect. It's applied to all 4 voices though, not just noise. Everything gets rendered to virtual APU registers until the very end of the code where it's written to the hardware. That way I can write/overwrite/modify anything that gets written to the (virtual) registers so I end up with a snapshot of activity during each refresh.
The only problem I have is that there's no real priority system other than the numerical order of the tracks, which the virtual drum track is highest and therefore always takes precedence.
This discussion has given me some ideas of how to change my code to make it more flexible.
To dig up this post again from the dead, bregalad was pestering me about it on the wiki, and I said "eh... needs to be rewritten in C"
So I did. It now deals correctly with sample rates other than 48kHz. It now assumes an "ideal" white spectrum by directly calculating the spectrum of the lowpass caused by the noise hardware rather than comparing to a sampled reference. The gain calculation seems incorrect, but it's the same one I was using before.
source code:
http://eamp.org/li/preanalyze.c
demonstration rom:
http://eamp.org/li/noise.nes
noise.nes- samples play on A,B,select,start,down,left. Down and Left are a comparison of the old analyzer to the new one respectively.
To build this you'll need fftw3 and libsndfile. I don't have a windows machine to build this on but it should be much easier than fighting with PDL.
I release it to the public domain, or if you're in a place where that's not allowed, under the WTFPL.
The demo isn't really convincing of this approach being better than just making drums manually, though I may be missing the point of the tool. Thinking about the big picture, the following comes to mind:
With traditional percussion, composer uses those that best elicit the desired "feeling" when listening. Presumably the development of drums themselves was shaped by this as well, so we have drums that form a good sound palette.
With this tool, sound palette is drums put through tool, forming a sound palette that might not be optimal anymore. Traditional drum sounds are somewhat limited by the physics of drums, so the best palette using them is shaped by these limitations. If different limitations were present, a different palette might be better. Converting a palette based on the physical limitations of drums to the limitations of the NES is unlikely to give the best palette for the NES.
Making new drum sounds on the NES and tuning them until they sound good is likely to yield the best drum palette for a NES. Here, it's not their similarity to physical drums that's important (though that may play a role), it's just their effect on the listener that's refined.
Anyway, just a thought that came to mind when scanning this thread and listening to the demo.
Well I didn't ask for it to be converted to C, but thank you for doing it.
I tried to compile it under windows but failed miserably. I downloaded both libraries you are mentionning, one comes in the form of an installer and I'm not sure where it gets installed, and the other I was able to get. Anyways, gcc still says both .h files are missing and can't compile the thing.
(you know, no offense but not everyone here is a PHD in computer science).
@blargg : you have a perfect point. I'm more interested in this tool for curiosity than anything else. It could be used as an inspiration tool to make drums manually too (instead of "blindly" copy it's output).
You can make noise splash which doesn't sound like real drums at all but goes well along a song. Castlevania is a good example of this, it comes with 2 kind of noise splashes for all percussion, and this works well. Wizard & Warriors II would be another example of this.
Also, I think the drum kit in Ferrari Grand Prix Challenge (from Neil
) and Blaster Master sounds absolutely amazingly close to a real drum kit with kick, snare and hats. It's hard to make a crash cymbal without having the other drums stop playing unfortunately.
blargg wrote:
The demo isn't really convincing of this approach being better than just making drums manually, though I may be missing the point of the tool.
Why do we need yet another DPCM converter? It's just another tool, this one unique. Does it produce perfect results? I'd be surprised if it ever did, but it can be informative and a good starting place.
Bregalad wrote:
(you know, no offense but not everyone here is a PHD in computer science).
Sorry about the tone; you reminded me that I really shouldn't have left it as the pile of crap it honestly was.
lidnariq wrote:
blargg wrote:
The demo isn't really convincing of this approach being better than just making drums manually, though I may be missing the point of the tool.
Why do we need yet another DPCM converter? It's just another tool, this one unique. Does it produce perfect results? I'd be surprised if it ever did, but it can be informative and a good starting place.
Sorry, I wasn't meaning to bash your tool, I was just sharing a thought that came to mind as I read about it/listened to the demo. When I'm making things, I also consider them from a practical standpoint, without reference to their research/future value, because it often leads to new ideas that can make it more practical, rather than entirely research-oriented. Maybe I misunderstood it, but I took it to be converting drums to noise only, no DPCM. Thus I was talking about manually coming up with noise writes to make a drum, not DPCM at all.
I think your tool is interesting, and could easily be used as the basis for something. For example, converting a drum, then manually tweaking the output. Or using the code as a basis for one that converts things for other channels. I was just considering it in the context of converting drum samples without any further tweaking.
Demo program fails on FCEUX.
Dwedit, it works for me with FCEUX and all the other emulators I can get my hands on. Did it get downloaded erroneously as text?
Never mind, my mistake, forgot I had various sound channels disabled at the time.
I was thinking and it would be cool to run this tool in real-time, with the NES playing along, so you could hear how well it matches the source material.
BTW, when making problem reports, it's good to describe the symptoms, rather than just that it doesn't work.
All the links are down, anybody have these (or could lidnariq repost them please?). I would like to try how well the Streemerz sound effects convert, many of them are noise based.
Fixed the link in the first post
Thanks, here's a Windows build if anybody's interested:
http://thefox.aspekt.fi/preanalyze-1.zip
By the way, it would also be nice to have the demo ROM (link on 2nd page is broken) if you still have that.