Some of you may recall my previous thread where I wasn't happy with any of the currently-offered NES palettes on various emulators as well as the NESRGB board. I had decided to take on the challenge to generate the most accurate palette based on the NTSC-USA front-loader NES's composite video output. While I had finalized my 'eyeballed' palette, I had intended to do a more accurate direct-measure/capture of the NES composite video output, but I didn't have the equipment at the time to do it... Well I do now, and I just finished the conversion!
So here's how I went about it:
1. NES front-loader RCA-style video output jack hooked into video capture device calibrated to NTSC color bars.
2. Test ROM loaded into the NES that fills the screen with a single palette entry (advanced to the next entry via the controller).
3. Uncompressed video recorded for several seconds of each palette entry.
4. Screen caps taken of each palette entry shown in the video.
5. Screen caps cropped to the visible color shown, and then Gaussian-blurred to average the color.*
* Part 5 had to be done because the NES doesn't display solid colors with its composite output. Instead, they appear with 'fish-netted' type interference. Here's a pic showing the original captured NES color with my Guassian-blurred single color entry next to it:
As you can see, the Gaussian blurring does a good job of approximating the intended color.
So when I got done generating the captured palette, I was quite surprised at how close my eyeball job was comparison:
I'm calling the matter finished with the direct-captured version. If you're interested in trying it out, here's the download to the .pal file:
http://www.firebrandx.com/downloads/fbxnespalettecapture.zip
Just a small warning, most image filters on the PC
fail to account for gamma.
This is great! Gonna integrate this into my graphics pipeline right away.
Comparison GIF showing this new palette vs Nintendulator's palette. (The Darker one is the new one)
Attachment:
nintendulator_vs_new_palette.gif [ 10.14 KiB | Viewed 9300 times ]
This capture card palette has a much greener dark olive ($08) than what I've seen on TVs.
Speaking of "dark olive", I might need to start a new topic about coming up with names of colors that programmers can use when communicating with artists.
Wow, gray $10 looks so light. Are you sure you don't have the screen brightness too high?
For me, the problem with the emulator palettes like Nestopia's YUV version is the blue-green shades on the right are far too green. I noticed it when playing the first dungeon of Zelda. They have it as pretty much half blue and half green, but the real hardware favors the blue far more than the green. I noticed this in my eyeball tests, and the capture card also detected the same thing.
tepples wrote:
This capture card palette has a much greener dark olive ($08) than what I've seen on TVs.
Speaking of "dark olive", I might need to start a new topic about coming up with names of colors that programmers can use when communicating with artists.
$08 looks kind of green like that on my LCD TV I think.
I like the idea of agreeing on names of the colours.
I've made colour labels in my program to make it easier to remember what colour is what. I just made up names based on how they looked like in the Family BASIC CGSET chart and the Nintendulator palette. If better names was agreed on, I'd love to use those instead.
Here it is (also includes some comments of what games are using what colours):
Code:
;Colour constants:
;Only the 52 unique/safe colours out of the 64 possible colours are included.
;Excluded colours are: The "forbidden greyscale" ($0D, $1D, $2D and $3D), the
;white $20 (since it's identical to $30) and all of the blacks that are
;identical to $0F (the $0E-scale, $1F, $2F and $3F).
;Colour scales goes from 0-3 (darker to lighter).
;Colour names should only be seen as a general guidance.
;For example waterblue and skyblue may both be used for water or sky,
;yellow may also contain orange and purple may contain pink and so on.
;-------------------------------------------------------------------------------
;Black and White
col_black = $0F ;classic black backdrop colour
col_white = $30 ;most commonly used white
;-------------------------------------------------------------------------------
;Grey
col_grey0 = $00 ;dark grey, DQ mountains
col_grey1 = $10 ;light grey, DQ mountains
;-------------------------------------------------------------------------------
;Blue
col_waterblue0 = $01 ;FF menu blue
col_waterblue1 = $11 ;DQ water/sky, Rockman armor (dark part)
col_waterblue2 = $21 ;FF sea, thief hair
col_waterblue3 = $31 ;FF sea
col_skyblue0 = $02
col_skyblue1 = $12 ;DQ hero blue armor
col_skyblue2 = $22 ;Super Mario Bros sky
col_skyblue3 = $32
col_plumblue0 = $03
col_plumblue1 = $13
col_plumblue2 = $23 ;Akumajou Dracula whip
col_plumblue3 = $33
col_purple0 = $04
col_purple1 = $14
col_purple2 = $24 ;Lady pink (Donkey Kong)
col_purple3 = $34 ;Akumajou Dracula whip
;-------------------------------------------------------------------------------
;Red
col_pink0 = $05
col_pink1 = $15
col_pink2 = $25 ;Lady pink (Family BASIC), cursor (Donkey Kong)
col_pink3 = $35 ;Lady skin (Family BASIC), DQ common skin
col_red0 = $06 ;Donkey Kong brown
col_red1 = $16 ;Mario red
col_red2 = $26
col_red3 = $36 ;Mario skin, FF common skin
col_yellow0 = $07
col_yellow1 = $17 ;Lady hair (Family BASIC), barrel brown (Donkey Kong)
col_yellow2 = $27 ;Samus yellow, Lady hair (Donkey Kong), DQ tree stem
col_yellow3 = $37 ;Zelda I ground, DQ desert/hill
col_brown0 = $08
col_brown1 = $18 ;FF monk hair, MetaFight bush
col_brown2 = $28 ;FF thief/monk skin, MetaFight bush
col_brown3 = $38 ;Rockman face and rockbuster beam
;-------------------------------------------------------------------------------
;Green
col_grassgreen0 = $09
col_grassgreen1 = $19 ;Luigi green, FF forest
col_grassgreen2 = $29 ;Link green (Zelda I), DQ grass, FF forest
col_grassgreen3 = $39
col_leafgreen0 = $0A ;MetaFight bush
col_leafgreen1 = $1A ;DQ tree/grass, FF grass/tree/forest
col_leafgreen2 = $2A ;Link green (Zelda II), R.O.B screen flash
col_leafgreen3 = $3A
col_bluegreen0 = $0B
col_bluegreen1 = $1B
col_bluegreen2 = $2B
col_bluegreen3 = $3B
col_turquoise0 = $0C ;DQ title screen backdrop, DQ slime
col_turquoise1 = $1C
col_turquoise2 = $2C ;FF stream, Rockman armor (light part)
col_turquoise3 = $3C
;-------------------------------------------------------------------------------
;64 Famicom colour chart:
;
; ^ Hex:
; |------------------
; | 00 10 20 30 GREY ～ WHITE
; |------------------
; | B 01 11 21 31
; | L 02 12 22 32
; | U 03 13 23 33
; | E 04 14 24 34
; |------------------
; | R 05 15 25 35
; | E 06 16 26 36 EXISTING COLORS
; H D 07 17 27 37
; U 08 18 28 38
; E------------------
; | G 09 19 29 39
; | R 0A 1A 2A 3A
; | E 0B 1B 2B 3B
; | E 0C 1C 2C 3C
; |-N----------------
; | 0D 1D 2D 3D <----- Forbidden greyscale (0D, 1D, 2D, 3D)
; | 0E 1E 2E 3E BLACK
; | 0F 1F 2F 3F
; |------------------
; v
; <-DARK-----LIGHT->
Funny, I always thought of the x7 column as oranges and the x8 column as yellows.
And why are the xD grays 'forbidden' ?
$0D has a voltage lower than black and can confuse the sync parsing in some TVs.
$1D is equal to $0F.
$2D and $3D are unavailable on 2C03 RGB PPUs, but they're so close to $00 and $10 that they're not too useful anyway.
And I too was about to assign the name "orange" to column 7 and "yellow" to column 8. I guess I need to finish my proposal and post it.
dougeff wrote:
Funny, I always thought of the x7 column as oranges and the x8 column as yellows.
And why are the xD grays 'forbidden' ?
Ninjad by Tepples about the "forbidden colours". Also any of these forbidden colours may fade into the bad $0D (if you fade by subtracting with $10) unless your fading routine is designed to work around it. So there's so many reasons not to use them I thought.
I named the x7 column from col_yellow2 since it's the colour of Samus armor and it's known to be yellow. Colours are generally named after one colour in their column not their "actual colour". And for names like bluegreen and turquoise, I'm not sure which is closer to blue and which is closer to green or if they are words with identical meaning even.
What about the 7 emphasis states, Firebrandx?
Emphasis isn't used all that much by games, pretty much a few games that run in deemphasize all colors mode for some reason, Final Fantasy 1 during battle transitions, and Noah's Ark.
We have
an imcomplete list of emphasis-using games, but I would presume that a "quest for the most accurate NES palette" shouldn't ignore 87.5% of the colours it can produce just because not many games use them.
Quote:
What about the 7 emphasis states, Firebrandx?
I second that. I would like to see how the palette looks under the various color emphasis states. Especially the 'all bits set' state (darker colors).
I think it would be a good idea to document exactly what capture device was used (and preferably other steps taken, like what software was used for the gaussian blur, and the settings). We may never find the one true NES palette, but it could still be interesting to have a collection of palettes from a variety of different devices.
As for naming the colors, I have also thought that it could be a nice thing to have. E.g. in quick test ROMs it would be nice to be able to say that you want some sort of a blue instead of specifying the exact color number. Even better if we can standardize on the naming.
Yeah, I'd expect there are a lot of things that could happen to the signal at each stage. Amplifier nonlinearity, clipping, intentional or unintentional filter effects, quantization, etc. Despite this, I think it's very much worthwhile to have an empirically measured "NES palette". It's always good to measure the real world case as well as you can.
I'm thinking about trying again with a new capture device. While most of the colors loo spot-on when I try them in an emulator, some others don't look right. For example the intro in Faxanadu when he's walking toward the world tree. The colors there don't strike me as being how it should be.
At any rate, which device would you guys suggest? I know for HD recordings, the debate seems to be between Avermedia and Elgato.
Also can you guys explain to me about the color emphasis and how I'd go about capturing that? I thought there was just the basic color palette and that's it. But if you're saying there's more to it, then I'd need a new test rom that gives me access to testing those colors. Thanks!
What ROM are you using for your tests? If there's source code it can probably be easily modified to do emphasis too.
I actually was writing a palette test ROM today, which lets you play with and observe how emphasis works:
viewtopic.php?f=3&t=13264Basically emphasis lets the NES "emphasize" red, green, blue, or any combination of the three. This means there are 8 different possible variations of the palette, i.e. the 64 entries becomes 512.
Family BASIC V3 allows emphasis bits to be set with the new command FILTER (
page 107-108 in the V3 manual) and Nintendo also gave all emphasis combinations a name there:
no color = 000
red = 001
green = 010
yellow = 011
blue = 100
magenta = 101
sky-blue = 110
white = 111
There's no command for greyscale mode but that can easily be achieved by poking $2001.
rainwarrior wrote:
What ROM are you using for your tests? If there's source code it can probably be easily modified to do emphasis too.
It was a ROM somebody on these forums sent me a link to that they had made when I originally requested a palette test ROM.
However, if you've already got one you've been working on, maybe I could just use that one? It would need to work in the EverDrive N8 on real hardware (NTSC).
At any rate, I looked into possibly the Avermedia brand of capture devices, and it looks like I'd need to also get their composite converter cable as their devices don't readily accept composite input.
I made a suitable test ROM for you:
hereThis test ROM lets you create a full screen of any color, and you can toggle all the emphasis bits. Should run on NTSC/PAL/whatever, it's very simple, no special hardware tricks.
Don't worry about the greyscale toggle, all that does is force all colors to use column 0 of the palette, it doesn't produce any new colors.
Firebrandx wrote:
it looks like I'd need to also get their composite converter cable as their devices don't readily accept composite input.
Doesn't adding more conversion steps to the process increase the chances of the colors being modified along the way?
I've got a question:
I own two TVs of exactly the same brand. I have access to the hidden system menu where I can adjust all possible values.
And guess what: Setting the same values in both TVs does not produce the same output.
So, what makes you think that creating an "accurate" NES palette is a thing that would even be possible in theory when not even two identical TVs with exactly identical values produce even slightly the same output?
Good point, but we still need a standard to work with, even if reality may vary. Its useful as a development tool to have a set of agreed upon hues, IMHO.
dougeff wrote:
Good point, but we still need a standard to work with, even if reality may vary. Its useful as a development tool to have a set of agreed upon hues, IMHO.
Why? Every TV is different. So, why should the colors that one person captured with the random TV that he happened to own be somehow declared more standard than any other TV? If the NES doesn't provide a standard in the first place, why should developers create one if it's proven that there are no objective values since even two "identical" TVs create a different output?
If you really need a reference point, don't try to come up with a definite palette. Instead, just use other games as reference.
You want to use the color red? Take the color from Mario's pants in "Super Mario Bros."
Skin color? Mario in SMB3.
And so on.
Besides, I always found the colors in FCE Ultra quite decent.
There is an NTSC standard for translating composite waveforms into RGB intensities, but I don't think TVs actually follow it. Instead they add quirks that in the manufacturer's opinion enhance the subjective appearance of live-action scenes that the manufacturer thinks viewers are most likely to watch, such as "yellow boost" in Japanese TVs.
DRW wrote:
So, what makes you think that creating an "accurate" NES palette is a thing that would even be possible in theory when not even two identical TVs with exactly identical values produce even slightly the same output?
Every one you can measure and survey gets you a better average. With enough samples, you can get a really good idea of where the "ideal" version should lie, and just how much variation there is between sets. This is all good information; just because there isnt "one true palette" doesn't mean we can't measure and make use of the characteristics of real world palettes.
DRW wrote: Why?
Because we're all nerds, apparently, and appreciate the experiment Firebrandx is doing.
rainwarrior wrote:
This is all good information; just because there isnt "one true palette" doesn't mean we can't measure and make use of the characteristics of real world palettes.
Sure, but in this case, you would need to calculate the average of a whole bunch of TVs.
So, if your experiment consisted of getting 100 TVs from different times, brands and models, setting them all to standardized color values, measuring the NES palette and then calculating the average value for each color, then I would say this makes sense.
But if it's just hooking up a screen capture device to a TV and then somehow trying to get the most accurate value in RGB form, this is really a vain attempt. Because you would get totally different results if you used another TV, even if everything else is the same. You cannot get an "accurate" palette if you just use
one TV. Because not even this TV is identical to the one of the same brand that was produced next to it in the factory.
Unlike the NES itself, TVs vary greatly among each other. So, the only way to get an accurate palette is to do the described procedure on a whole bunch of TVs and calculate the average value.
I'm a little skeptical about the existence of an ideal NES RGB palette myself, but I appreciate the effort in the attempt to find it. Having a few real world examples is nice, even if they don't match everyone's personal experiences.
It makes me wonder whether there's even an ideal palette for RGB systems such as the Genesis and Super NES, given the differences between the RGB values and the result after they've been encoded to composite by the console and decoded back to RGB by the TV.
Yeah, I wonder about that too. But since those systems have RGB palettes to begin with, people don't usually care much, and just assume the colors reach the TV intact. Some emulators don't even map the range to 0..255 properly, and just shift the smaller ranges left, leaving the list bits clear.
DRW wrote:
Because you would get totally different results if you used another TV, even if everything else is the same.
It's not totally different from TV to TV, it's subtly different. This is a very important distinction! Most will be relatively close to the average, and even one point of data on it is immensely better than no points of data.
tokumaru wrote:
But since those systems have RGB palettes to begin with, people don't usually care much, and just assume the colors reach the TV intact.
If we can assume as an axiom that Super NES "colors reach the TV intact", then the proper approach to derive an NES palette is to find, for each NES color, the Super NES color that produces a composite waveform with the same amplitude and phase.
tepples wrote:
a composite waveform with the same amplitude and phase.
If my understanding of the NTSC signal is correct, I think you'd want to match the
amplitude,
phase,
and "center height" as in this
diagram. (I'm not sure what the best term for "center height" is.)
"center height" = "luma"
Ok, fine, "mean" or "average".
tokumaru wrote:
Firebrandx wrote:
it looks like I'd need to also get their composite converter cable as their devices don't readily accept composite input.
Doesn't adding more conversion steps to the process increase the chances of the colors being modified along the way?
I worry about this too, which is why I looked at many different devices on the market. I tried the Hauppauge PVR 2 with the composite video cable you can order for it, but the picture came out scrambled like it couldn't handle 240p or something. That was very disappointing, because it would have been a direct link from the console to the device without the need of a converter.
I think I should at least give the Avermedia coverter + capture method a try though.
The most important thing about NES palettes is the fact that several colors are out of gamut, meaning they feature R, G, or B values that are below 0 or above 255. Below 0 is usually clipped to 0, but above 255 has no "right" way of fixing. It's mostly the blues that are out of gamut, but some purples and at least one red is too. An average, run-of-the-mill NES palette will just clip those colors, which alters either the hue, the saturation, the luminance, or any combination of those three things.
The only "accurate" way to portray those colors is to darken the whole palette until those colors enter the gamut. That, above all else, will give you the most meaningfully "accurate" palette, but I'd only use one of those if I were designing graphics for an actual NES console. For just playing games, it's usually fine to have a palette with a couple of clipped colors, just as long as the hues of most other colors looks right to you, and because which hues look "right" depends on the person, having a palette generator with a hue knob helps.
It would appear that there are as many "accurate" NES palettes as there are people with capture devices. Meanwhile, both developers and players back in the day would just tweak their TV sets to make them look "nice". At least most of these "accurate" NES palettes still look halfway decent, unlike that desaturated slop from those "accurate" Commodore 64 palettes.
Why not hook it up to a scope and analyze it that way? As in, remove as many variables as possible. Capture cards are nice, but I wouldn't rely on them to be extremely accurate; there purpose isn't accuracy relative to game console video output.
NewRisingSun wrote:
It would appear that there are as many "accurate" NES palettes as there are people with capture devices. Meanwhile, both developers and players back in the day would just tweak their TV sets to make them look "nice". At least most of these "accurate" NES palettes still look halfway decent, unlike that desaturated slop from those "accurate" Commodore 64 palettes.
^This.
tomaitheous wrote:
Why not hook it up to a scope and analyze it that way? As in, remove as many variables as possible. Capture cards are nice, but I wouldn't rely on them to be extremely accurate; there purpose isn't accuracy relative to game console video output.
HardWareMan got it:
viewtopic.php?p=112191#p112191Fact of the matter is, we already know exactly how the digital portion works, and the outside-the-IC part is so simple there's nothing variable there. And all of this is accurately simulated by Bisqwit's and Drag's palette generators.
The
only possible remaining variation in colors would be if the output impedance from the NES varies significantly across the 10 different voltages the PPU can emit. And even then, that should really just come down to a little variation in net hue rotation, net brightness, or net saturation.
NewRisingSun wrote:
At least most of these "accurate" NES palettes still look halfway decent, unlike that desaturated slop from those "accurate" Commodore 64 palettes.
That's because the video generator circuitry inside the C64's VIC-II chip has been accurately modelled. If the resulting colours are unsatisfactory then one should adjust the brightness/contrast/hue settings in the emulator's CRT emulation settings... just like an actual CRT.
Oh, and lets not forget the whole PAL/NTSC thing.
Custom RGB palettes for machines whose chipsets natively generate composite/Chroma+Luma signals should be consigned to the past.
lidnariq wrote:
Fact of the matter is, we already know exactly how the digital portion works, and the outside-the-IC part is so simple there's nothing variable there. And all of this is accurately simulated by Bisqwit's and Drag's palette generators.
What more beyond this is needed? (Beyond brightness/contrast/hue controls)
Anything else is at risk of being contaminated by differences in colour decoding hardware and rose-tinted filters.
Hojo_Norem wrote:
That's because the video generator circuitry inside the C64's VIC-II chip has been accurately modelled.
Oh, I'm not denying that. What I am saying however is that the desaturated slop is the result of improperly converting the VIC-II's accurately-modeled NTSC waveform to RGB. Most C64 color generators (and some capture devices) ignore (1) the color burst as a chroma amplitude reference (2) the 7.5% black-level setup (3) the NTSC to sRGB primaries conversion. All three of these contribute to desaturating the output, and have nothing to do with the user adjusting an imagined CRT. This is only superficially off-topic, as it applies at least in principle to the NES as well, though the difference these things make there is admittedly rather subtle, unlike on the C64.
Hojo_Norem wrote:
Custom RGB palettes for machines whose chipsets natively generate composite/Chroma+Luma signals should be consigned to the past
I find them useful for circumventing imperfect built-in color generators.
NewRisingSun wrote:
the color burst as a chroma amplitude reference
Could you provide an instance of hardware that uses the colorburst amplitude to scale composite or OTA video? Every single one I've ever looked exclusively uses sync tip depth.
I stated that the color burst amplitude that should be used as a reference for the
chroma amplitude (i.e. saturation), not the entire video composite signal. The entire video signal should only be scaled by the sync tip level, as you mentioned. But to answer your question, my Sony KV-21X4D CRT unfortunately indeed uses the color burst amplitude to scale the entire composite signal (both in NTSC and PAL modes), which wreaks havoc with the output of my Tandy 1000 TX.
If I could only find that service manual I once had, I could even tell you what picture processing IC inside that TV does the dastardly deed. Edit: A Motorola MC44002P.
lidnariq wrote:
Fact of the matter is, we already know exactly how the digital portion works, and the outside-the-IC part is so simple there's nothing variable there.
Is this known for both the NES and Super NES?
Quote:
The only possible remaining variation in colors would be if the output impedance from the NES varies significantly across the 10 different voltages the PPU can emit. And even then, that should really just come down to a little variation in net hue rotation, net brightness, or net saturation.
Slew rate effects may produce a saturation-dependent hue rotation, where the more saturated colors ($1x and $2x) are rotated more than the ones with weaker chroma ($0x and $3x).
NewRisingSun wrote:
It would appear that there are as many "accurate" NES palettes as there are people with capture devices. Meanwhile, both developers and players back in the day would just tweak their TV sets to make them look "nice". At least most of these "accurate" NES palettes still look halfway decent, unlike that desaturated slop from those "accurate" Commodore 64 palettes.
Speaking of which, has anybody noticed how muddy the colors look on the AVGN videos?
psycopathicteen wrote:
NewRisingSun wrote:
It would appear that there are as many "accurate" NES palettes as there are people with capture devices. Meanwhile, both developers and players back in the day would just tweak their TV sets to make them look "nice". At least most of these "accurate" NES palettes still look halfway decent, unlike that desaturated slop from those "accurate" Commodore 64 palettes.
Speaking of which, has anybody noticed how muddy the colors look on the AVGN videos?
You have an extra layer there, since James records his sessions with a DVD recorder for a playthrough.
At what quality on the DVD recorder? DVD recorders have 704-pixel* modes and 352-pixel modes. The pixel counts refer to luma; chroma is half as fine. So long as there are at least 280 chroma pixels across, which the 704-pixel mode has, it should be able to cover the entire bandwidth of the NES's composite output, or even the Super NES's S-Video signal if your DVD recorder happens to be compatible with your Super NES (mine isn't except through RF) and the game isn't using 512px mode. Cheap real-time MPEG-2 encoders, however, might mess with this.
Commodore 64 is a different story. Its pixel clock is 16/7, and 426.67 pixels fit across the 4:3 screen width (including 53.33 pixels of border at each side). This is enough for luma in all modes or for chroma in multicolor modes, but it's just barely not enough for chroma in high resolution mode. I know the C64's native output is S-Video, but that pixel clock (8 high resolution pixels per 3.5 color subcarrier cycles) is 14% too wide even for S-Video without artifacts. I wonder how its chroma output is actually defined.
In either case, at least flat areas of color shouldn't be affected.
* The 704-pixel modes actually store 720 pixels, including 16 undisplayed "nominal analog blanking" pixels at the sides that are out of the 4:3 or 16:9 frame.
So I wrapped up my finalized NES 'accuracy' palette. It's an amalgam of a few personal tweaks, the hue alignment from my direct-capture results, and uses the top-end values from the 'natural' palette. The end result looked absolutely kosher on every game I tried (using an Everdrive N8 to have access to 99.9% of the library). I tested all the Mega Mans, Castlevanias, Zeldas, Punch-Out!!, Ninja Gaiden, Double Dragons, Bionic Commando, and many other games. I tested for so long that I got burnt out. This is finally the palette I can rest on.
Here's the zipped palette file:
http://www.firebrandx.com/downloads/fbxnesaccuracypalette.zipOne thing I will point out is I discovered some displays cannot handle the blueish-purple color entries coming from the real NES hardware. As a result, some graphics that were meant to be purple came out as straight blue. This is where the direct-capture hue values came in real handy in that it shows the purple tinge on those entries correctly. Examples of the bluish-purple would be in the intro to Mega Man 5 (where it shows Proto Man in blueish-purple in front of the blue robots behind him), and the first stage of Double Dragon II's floor is also bluish-purple. ON some displays, these will appear entirely blue.
This is what the palette looks like, in case you want to use it in a paint program:
Attachment:
fbx-palette.png [ 2.93 KiB | Viewed 4894 times ]
Its colors are less muted than
the palette that I had been using.
The Python program that produced this image:
Code:
#!/usr/bin/env python
"""
indexed image palette replacer
made by Damian Yerrick, licensed under CC Zero
"""
from PIL import Image
with open("FBX-Accuracy.pal", "rb") as infp:
palette = infp.read()
palette = palette[:192]*4
# this png file is written by savtool available at
# https://pineight.com/nes/#editor
im = Image.open("savtool-swatches.png")
im.putpalette(palette)
im.save("fbx-palette.png")
On request, I slightly reduced the purple tinge that shows up on Protoman in the intro of Mega Man 5. While the intended look was meant to have a purple tinge, many TVs have trouble interpreting it, and so most people remember seeing those colors in a more blue hue. The same entries also show up in Metroid quite a bit, so it was the best compromise between 'intended' and 'what people remember'.
The file has been uploaded to the same link.
Regarding the purples. On Double Dragon II's street, my CRT shows a color that aligns with Nestopia's default palette. If I shift the tint control (from center) two notches to the left to the red, I will see Firebrandx's color.
Although I did not download Firebrandx's original final palette, if I keep my CRT's tint control at the level to see Firebrandx's purple street, I can make out a purplish outline for Protoman in Mega Man V. With his newer palette, I see the blue color I would see if my tint control was in the center.
I believe Super Mario Bros. and Super Mario Bros. 2 use the same palette entry for the sky in the overworld stages. However, if you look at the Japanese commercial for each game (easily available on youtube), the TV shows a more purplish sky for SMB2 than for SMB. Issues like these must cause nothing but headaches for people who want a consistent color palette.
So I finally got to the bottom of why my direct-capture palette conversions looked like crap compared to my eyeballed method. Turns out there was some horribly incorrect color conversion going on with blown out colors. I was able to make a new direct-capture palette where FINALLY everything is beautiful and accurate to the real console!
Here's the finalized Direct-Capture palette:
http://www.firebrandx.com/downloads/FBX-Composite-DC-Final.zipEvery game I reviewed looks 100% kosher with it, and it's all purely a math-driven palette (i.e. no eyeballing).
Great Hierophant wrote:
I believe Super Mario Bros. and Super Mario Bros. 2 use the same palette entry for the sky in the overworld stages. However, if you look at the Japanese commercial for each game (easily available on youtube), the TV shows a more purplish sky for SMB2 than for SMB. Issues like these must cause nothing but headaches for people who want a consistent color palette.
SMB1 and SMB2J have the same sky color, $22. SMB2US does not--its sky color is $21. $x2 is a weird choice for sky; most games use $x1 or $xC. I think the popularity of emulator palettes with highly inaccurate $x2 hues is related to people misremembering SMB1's sky color (confusing it with other games). If you play SMB1 on real hardware after playing a bunch of other NES games I guarantee your reaction will be "huh, why is the sky so purple?"
Sorry to post yet another palette update, but really the only complaint I got about the new direct capture palette on other forums was that it was too low on saturation. I made a new one with heavier saturation here:
http://www.firebrandx.com/downloads/FBX ... urated.zipIt could be just considered an alternative if you like the more vibrant colors.
Cheers!
I wrote up a web page showcasing my latest (and last) palette. It also goes into full detail on how each color was derived:
http://www.firebrandx.com/nespalette.htmlI really cannot stand to work on this project any further, so no matter what, I'm retired from the project.
Besides, those pics look superb if I say so myself.
Saturating the entire palette doesn't preserve apparent brightness, you throw off the brightness when you do that. Notice how the first blue column is lighter than the second column, but only in the saturated palette.
A mirror for good measure
To clarify: Though $0D is #110011 (dark purple) in the PNG, that's not the case on the actual hardware. Small changes to $0D are something I do to keep certain tools from using $0D when looking for the "first black color in the palette".
I added some FAQs and also a preview image to the web page (
http://www.firebrandx.com/nespalette.html).
Last night as a safety check, I recorded a new session of the color output and exclusively used Photoshop to import, crop, and average all the colors since I was getting complaints of Paint Shop Pro being 'inaccurate'. The results were 99.99% identical, with differences only being 1 tick in 1 or 2 color channels (imperceptible to the human eye). This degree of error is so slight that I don't have to worry about apologizing and yet again releasing a new palette.
Anyway, no complaints on other forums I've posted it in, and in fact some fans are having nerdgasms over how unbiased the palette is. For example in SMB, the bricks are brown as opposed to the "YUV" palette's bias towards red. The direct-capture palette also reveals just how wrong the YUV palette is on the last dark color entry (Mega Man 2's Wily Stage sky background uses it).
Dwedit wrote:
Saturating the entire palette doesn't preserve apparent brightness, you throw off the brightness when you do that. Notice how the first blue column is lighter than the second column, but only in the saturated palette.
FYI the saturated palette uses the default saturation balance of the XRGB-mini. The previous unsaturated palette was actually with saturation set to 25 instead of the default 32. So while you may think it's less accurate, it technically is more accurate. I realized I needed to make the saturated version when I tested SMB. The bricks were considerably darker at 25 than they were at the default 32.
Tim was kind enough to make new firmware versions for the NESRGB boards that incorporate the direct-capture palettes I made. They come in two flavors:
#1 = Playchoice 10 -- FBX Saturated -- FBX Unsaturated
#2 = Playchoice 10 -- FCUX Palette -- FBX Saturated
This should cover those that prefer the tonal consistency of the unsaturated palette as well as those that prefer the vibrancy of the saturated version.
I've made my own zip link of the firmware files here:
http://www.firebrandx.com/downloads/nesrgbfbxpalettes.zipPlease be aware that you need to be familiar with using an Altera USB blaster to update the NESRGB boards. Some instructions can be found on Tim's page here:
http://etim.net.au/nesrgb/background_fault/Some users have already made the update and reported they are working.
Quote:
no complaints
I'm using the FirebrandX palette. It looks pretty close to my actual NES. Better than the default palettes in every emulator I use, at least.
Just got done using the Altera USB blaster on my own NESRGB board. I'd never done anything like this before, so I was thrilled that there were zero problems and it worked first try.
I can tell you the saturated palette running on the NESRGB board looks identical to my umodded NES hooked into the same 55" LCD display! I couldn't be happier with the outcome (finally screens like the title to Castlevania II have the right colors)!
OCD 100% satisfied, and a HUGE THANK YOU to Tim for taking the time to indulge me on this obsessive goal.
BTW, I'm betting the unsaturated palette will be better for those with CRT setups, as saturation can be adjusted directly from the CRT as needed.
Alas, just when I thought I'd be done with the project, I began noticing the browns and olive greens were distinctly different compared to the YUV palette. Games like Contra, Castlevania III, and Faxanadu clearly intended olive colors to be more brown than green. So what I've done is make a combination palette alternative, where it uses the core purple-red-brown-olive colors combined with the green-cyan-blue-purple colors of my direct capture work and also a few tweaks to a few colors.
http://www.firebrandx.com/downloads/fbxyuvcustomized.zip
After spending the past couple days arranging all my palette efforts and comparing them to the YUV palette, I had to face the fact that there were really only 4 entries blatantly 'off', while everything else was within reasonable range. As such, I've settled on only correcting those 4 entries in the dark green-cyan areas of the palette. Web site is updated with screenshots showing how the improvement looks in-game compared to the YUV palette:
http://www.firebrandx.com/nespalette.htmlHere's the palette with the corrected entries highlighted in red:
It's weird coming nearly full-circle, but ultimately those colors were my biggest complaint anyway.
Mind you to post a picture of SMB title screen for comparison? This palette is much much like nesdev palette, but a bit brighter.
On a request, I was able to revisit my unsaturated direct-capture palette and manually correct the green bias the Framemeister was causing on about 6 entries in the table. End result is a palette table that mimics the contrasts of the console tones, but looks identical in hue to the original YUV palette (save for the green-cyan entries the YUV palette is known to be off on). Together with my corrected YUV palette, the two look very similar, but one is vivid while the other mimics physical contrast.
Web site has been updated with tables and pic examples, and I swear this is absolutely finished. Every entry now looks like it should:
http://www.firebrandx.com/nespalette.html
Tim came through on the final palette firmware update I wanted:
http://www.firebrandx.com/downloads/nesrgbfbxfinal.zipI've already used it to update my NESRGB, and everything looks beautiful!
I've uploaded full instructions on my project page. Hit "refresh" until you see the update date of January 19th, that's the new version with the installation instructions:
http://www.firebrandx.com/nespalette.htmlDone and done!
Apologizes if this has already been covered, but were you using a calibrated monitor during this process?
zeroone wrote:
Firebrandx has a portion of a scan from "the original programmer" indicating a preference for the purplish-tint for SMB's blue sky. Does anyone know the original source for that quote?
Great Hierophant wrote:
Firebrandx has a portion of a scan from "the original programmer" indicating a preference for the purplish-tint for SMB's blue sky. Does anyone know the original source for that quote?
It's from an interview with Shigeru MIYAMOTO as part of a published book in Japan titled
ファミリーコンピュータ 1983-1994 (Family Computer 1983-1994).
ccovell has a reference to it here:
http://www.chrismcovell.com/levelxbook.htmlFor what it's worth, I still maintain that the whole "quest for an accurate NES palette" is an eye-of-the-beholder trait, simply because the display devices these games were being developed using at the time were subject to luminance/chrominance/hue variance. In other words: the way it looks to you, the developer, on your CRT on your desk, may not be how it ends up looking to little Tommy on his CRT TV set at home. The displays the character designers (and systems testers!) were using at the time are clearly depicted (
page 1,
page 2) -- they're NTSC CRTs. (Hint: now you know why the designer there has 2 CRT displays on his desk: one for
the development system which used VGA, and one hooked up to the Famicom directly. (There's several reasons for this actually, but "best colour representation" is certainly one of them))
I find it utterly amazing that the "quest for the most accurate palette" (a.k.a. The Autistic Sperglord's Adventure) has been going on -- quite literally -- for nearly 20 years (the earliest *I* heard of this subject coming up was late 1996 when emulation was starting). :P
Miyamoto-san's words, as the designer (not programmer) of Super Mario Bros., do carry a certain weight in the Super Mario Bros sky debate. The translated text on Firebrandx's page implies that he was looking at palette entries 21, 22 and 11 or 2c. ("At that time, you could only have about three colors for the blue sky on the Family Computer.") Of course, he does not quantify how much purple he saw in the palette entry he used. But if he wanted a purer blue sky, $21 would be the preferable palette entry.
His preference does not mean that Nintendo had implemented any standard to calibrated all its NTSC monitors to show identical colors. The designers at Capcom, Konami, Rare or Sachen may have seen different colors when testing their games.
That is a very good point, especially when you consider Rare and Sachen (as examples) being based in PAL territories (UK and Hong Kong respectively.)
Would the RGB palettes for the games on the Nintendo VS System be any use in trying to guess intention? They seem to have different palettes for each game? Or at least a choice of PPU with that had different palettes?
From looking at MAME, unless it is wrong, it would seem that the sky in VS Super Mario Brothers has a purplish tint where as the bricks in VS Castlevania are pinkish / purplish in colour.
Now i don't know whether each game got to choose its own colour palette specifically for the game, but it looks like each one had a separate PPU and there were multiple PPUs to choose from with different palettes? So I would have thought that the designers would have chosen the colours for these palettes to match the home Famicom releases. Now this isn't necessarily the case - they may have wanted to change some colours they couldn't achieve on the home system, or they may have just done a quick and dirty palette conversion or they may not have done a good job of matching anyway. But if they did try and match colours then perhaps it gives some insight.
In terms of the Player's Choice 10 System, I imagine that this just has 1 RGB palette for all of the games (and therefore less helpful for showing intent), considering that the games are the same as the home versions just with a timer, unlike the VS System games.
Mind you even on the VS System, it seems that colour calibration of the monitor makes a huge difference, compare:
VS Super Mario Brothers:
Purplish blue
https://youtu.be/-lPW7Zx8uRQ?t=999Sky blue
https://www.youtube.com/watch?v=mKk2TkKRAC0VS Castlevania:
Even more purplish than MAME (probably partially due to recording conditions)
https://www.youtube.com/watch?v=nxhhThfvMBAMuch more subdued purple like how i imagine the Famicom may have been:
https://www.youtube.com/watch?v=t67qmcG7mnQMAME's is somewhat of a much lighter pinkish purple.
To be honest, I would imagine a lot of the arcade monitors aren't calibrated that well, as it also seems people tend to have an affinity for oversaturating and blowing out their colours for some reason because "more colourful = apparently better" even when it ruins the shading and subtlety.
But assuming MAME is correct, it would imply that the dark purple colour should indeed be slighlty purplish with a bit of red, rather than mostly red (so maybe the old version was more accurate in this regard).
Of course, this is relying on the assumptions I talk about above.
Again, in the end it's all personal preference unless you question the people who worked on the games.
What do you think?
The Famicom Titler and Famicom TV have the same 2C03 PPU as the PlayChoice.
The 2C03 palette is known, and it differs in several ways from what you get when you run the NES output through any reasonable NTSC decoder.
I'm still interested in what would happen if you assume that a Super NES should be able to produce the same color as an NES through the same monitor. Anybody with a scope, an NES flash cart, and Super NES flash cart should be able to find the RGB values out of the S-PPU that produce the same center level, RMS amplitude, and phase as each NES PPU color.
notindeed wrote:
To be honest, I would imagine a lot of the arcade monitors aren't calibrated that well, as it also seems people tend to have an affinity for oversaturating and blowing out their colours for some reason because "more colourful = apparently better" even when it ruins the shading and subtlety.
The 50-unique-color 2C03 palette entirely consists shades that are 0% or 100% saturation when viewed in HSL space.
The 60-unique-color 2C04 palette adds a few more colors, but precisely one has a middling saturation value (a sandy/peachy color)
Hmm from reading
this and
this, it would seem that although there were a lot of PPUs, they were mainly different for security purposes - scrambling the colour locations in the palettes of games, rather than actually changing the colours themselves.
Or was it a case of both?
The 2C03 and 2C05 are garish versions of the 2C02 palette ... just missing two extra greys.
It appears that the 2C04 took advantage of the cheapo DRM scrambling the 2C03's palette to add another 10 colors to address a few shortcomings.
Yes, after reading more, the VS Arcade palettes are all basically the same but mixed around and some with a few extra colours, like you say. So i guess they don't give as much insight as i had first thought - mixed up palette locations don't really count as different palettes per game.
I did remember seeing scans of the Akumajou Dracula manual though, and they give some idea:
slightly washed out and yellowish scans but if you colour correct it, it still looks desat pinkish rather than outright red. Sort of a dark pinkish burgundy.
(Also it looks like they originally used the stairs colour for the main blocks judging from those pics).
With GIMP auto-white balance:
Obviously, scans of photos of a tv screen in a printed manual won't give you a definitive swatch, but they will give some idea / insight into what was intended.
Judging from the pictures, I adjusted 05 to be 95, 20, 47 as an approx of something closer to the "in-between dark red and pinkish, slightly lighter but desat" that it appears to be from the pictures.
Default Nostalgia (FBX) palette left, approx edit right:
Now I'm not saying the edit is correct, I just wanted to get people's thoughts.
Also, I am preferring the olive green changes of Nostalgia FBX - it certainly adds more depth to the shading - That background in the underground sections of Wood Man's stage in Rockman 2 for example.
From what I understand, the NES-Classic (FBX) palette is just your rip off the palette that Nintendo have put out on their new emulation system?
It's just a bit confusing as you then talk about adjusting the olive colours to be brown (like in previous versions) but don't actually attach a palette that represents that. However considering old versions seem to have the adjusted brown instead of olive green, i would say the new more accurate ones are a lot better - they are close enough to brown to work as that and also add more depth to the shading when combined with brown.
Anyway, it is looking better, good job on all the effort. A lot of this is very subjective so don't worry too much - it all depends on how the different developers had their tvs / monitors calibrated when working on them. A compromise that looks decent in all games is all we can wish for
Thoughts?
notindeed wrote:
... I did remember seeing scans of the Akumajou Dracula manual though, and they give some idea: ... slightly washed out and yellowish scans but if you colour correct it ... Obviously, scans of photos of a tv screen in a printed manual won't give you a definitive swatch, but ... Now I'm not saying the edit is correct, I just wanted to get people's thoughts. ... Thoughts?
You have already stated my thoughts. Summarised: using a printed book as a "reference", requiring further "tweaking" (because of the nature of the medium), is just as susceptible to the problem as the one I explained
here -- but now it's in paper form (subject to whatever inks and paper pulp were used during that specific printing run, blah blah blah). I too can "colour correct" anything I want to look pretty much however I want -- kinda like those hue and colour dials on your old TV, re: chroma amplitude, and those implemented even in high-end CRTs.
In short: eye of the beholder.
Why is it so hard for people to accept there really is no "perfect palette" given how all of this works? Two decades and counting.
I have a different theory.
That's basically what i said, in the end it is all personal preference.
Or rather, unless you can know how the developers monitors were calibrated and what they saw, you can't really know.
And even if you get it "right" for one game, other developers may have had their monitors set differently.
And then you consider that the NTSC standard changed in 1987, differing for both Japan and America (see
viewtopic.php?f=21&t=13687#p162402 ).
This means if you capture things today your captures are probably from the wrong colour space.
I am guessing a lot of games released after they change would have been made on tvs with the old colour space anyway, but at some point they would have updated, knowingly or otherwise.
So yes, you will never get a palette that is perfect, but if people here want to try then you don't really need to insult them.
I do agree it's a bit of a wild goose chase, but at the same time palettes from this thread look a lot more natural than i have seen in a lot of emulators with super saturated crazy colours, so it's not all wasted effort.
I think what we have in today's emulators (ones that allow for a palette of your choice, *and* adjustment of brightness, saturation, contrast, and hue, ex. Nestopia) is sufficient -- especially in comparison to what we had in the late 90s emulator-wise. The NES's NTSC artefacting is unique/interesting to say the least.
I have no problem with people continually "adjusting" their palettes to match, well, whatever they happen to think "is the most accurate", it's just that... well, you actually paraphrased it quite well: "... it's a bit of a wild goose chase". What we've got today (and have had for a while now) is pretty wonderful.
notindeed wrote:
Or rather, unless you can know how the developers monitors were calibrated
Even the developers' monitors literally mean shit.
For example, regarding the topic above: So, Miyamoto says that he wanted a purple-like sky for "Super Mario Bros.", that's why the sky color is supposed to be purple.
WRONG!
Miyamoto was only the designer for "Super Mario Bros.",
not for the NES itself. So, when he says that he took this specific color because it matched what he wanted for the sky, this says
nothing about what the designers of the NES had in mind when defining this color. It only means that Miyamoto's TV happened to show the sky color in a puple-like way. But the design choices of this 1985 game still say nothing about the intended specifications of the 1984 console.
So, even knowing how the developers calibrated their monitor means nothing when it comes to a general-purpose all-games-including NES palette.
And I want to repeat what I said earlier: I own two CRT TVs of exactly the same model. I have access to the hidden system menu. When calibrating both TVs with the exact same values, both TVs produce a different output.
So, the "most accurate NES palette" is not only an OCD-like attempt, it is
objectively impossible.
That's not like finding the most accurate color for NES cartridge shells where you have thousands of shells that all look alike. For the NES, not even two identical TVs show the same colors. How can there, even in theory, be any kind of "most accurate" palette? The best you can do is: "I'm trying to create a palette that comes closest to my personal TV when it's set to the settings that I chose for it."
Some of you people still think that Nintendo had a bunch of 100 completely identical monitors that they used in-house and gave to their developers and the marketing department, right? That's why you think you can listen to game designers or have a look into old Nintendo magazines to find the "definite" palette, right?
Guess what: Miyamoto's TVs and the TVs those pictures were taken with were no better or worse or any more or less "definite" than your or my CRT TV.
It is an analog signal, consisting of voltages. There is no "most accurate" palette.
It's not like with the "Mona Lisa", where the colors are actually physically there and you can check the original painting for it. For the NES, there was
never an "original" or "master" color palette to begin with.
If you think that the word of a game developer has any weight on the topic at hand, simply ask me. I programmed a game for the NES. My color choices were based on FCE Ultra/fceux. There you have it: Me, an NES developer, took FCE Ultra's palette as the basis. Now you have the most accurate palette. End of story.
DRW wrote:
It is an analog signal, consisting of voltages. There is no "most accurate" palette.
What could be more accurate than the palette that, when fed to the Super NES S-PPU, produces the same voltages?
tepples wrote:
What could be more accurate than the palette that, when fed to the Super NES S-PPU, produces the same voltages?
While this may get you the most *technically* (hopefully my use of the word is understood given the context) accurate palette, it changes absolutely nothing with regards to what the original artist/graphics designer saw and chose (because of how it looked on their setup). In other words: while what Miyamoto saw as "a slightly purple hue" to the sky may be true for him/others around him, but a different developer with different equipment (different calibration -- or more likely no post-factory calibration at all!) might have used the same colour in their game because it was blue. Both accurate, i.e. both were what were intended by the designer.
I remember back in the late 90s having a conversation along these lines with folks doing emulators. I even went as far as to use my ATI All-in-Wonder Pro card to capture NES output in attempt to make an "accurate palette". Guess what my comparison base was? The TV itself. The "adjustment" of the palette was pretty obvious: either I adjusted the colours in the palette that was captured, or I adjusted the hue/colour on the TV itself to match what the PC/capture had. Which is "accurate?" (The answer is: both are) It's not like there's a Pantone colour chart/swatch for the NES. ;-)
I feel like my above two paragraphs are starting to get philosophical, almost a kind of Schrodinger's box-type scenario.
I honestly think designers/developers simply looked at how their characters showed up on the graphics design workstations and on the TV hooked up to the Famicom and said "yeah, that's about right" and went with it. I can't imagine long-winded meetings about palettes or "well that colour looks a little purple-ish, so let's go with the other index" -- I imagine it was more like "hey Curt, can you try making the protagonists hat more green? No? Okay, that's fine".
tepples wrote:
What could be more accurate than the palette that, when fed to the Super NES S-PPU, produces the same voltages?
Would this work?
Does the SNES work with definite RGB values respectively values that can at least be converted to RGB? If yes, would another console that also works with RGB values produce the same colors when the same voltages are used?
And if you can answer all this with yes, then why didn't you say this earlier, let someone do it and be done with this whole topic instead of letting the people here hopelessly philosophize about stuff like what Miyamoto saw when he chose the colors for his games or what some magazine shows?
koitsu wrote:
While this may get you the most *technically* (hopefully my use of the word is understood given the context) accurate palette, it changes absolutely nothing with regards to what the original artist/graphics designer saw and chose (because of how it looked on their setup).
Well, this one cannot be solved, but this shouldn't be the problem anyway. Unless you try to create one specific color for each game, it doesn't matter what Miyamoto saw, just like it doesn't matter what Miyamoto saw on his TV when choosing the colors for SNES games: The accuracy of the palette doesn't change one little bit with the knowledge about game developers' TVs.
Proof: Bitmaps on the PC have definite RGB values. But if a designer has a shitty screen and therefore converts his drawn images into bitmap files and uses the wrong colors, then this doesn't mean that the RGB value in the bitmap is not a 100 % correct value of what the sprite looks like
in the game. It might not match the drawn image, but there's a 100 % correlation between the three bytes for the color in the bitmap file and the way the image is shown in the game. Therefore, the game has a definite palette that is identical among all PCs, Macs and all types of x86 emulators.
So, I do believe that
if this can be done, we really have "the" definite palette.
Of course, this only works if there's really a definite 1:1 assignment between the analog signal and the SNES. And the output needs to be identical when I use another console with an RGB-based palette.
And of course you need to be able to decide which of the 16000 colors actually corresponds to the specific NES colors. I don't know if people will be able to decide which of the 50 or so red shades that look like NES color $16 will be
the identical value to $16.
Or do you have the equipment to measure the actual analog signal (voltage etc.) and then you simply cycle hrough all the colors individually until their analog signals matches with the one on the NES?
And will there be a perfect match? What if one of the NES colors has, let's say, a color brightness of 83.6253 %, but on the NES, you only find colors with 83.6250 % and 83.6256 %?
I don't think you'll find SNES colors that generate the same square waves as NES colors. You might find colors that appear the same, but the waveforms will be different.
DRW wrote:
tepples wrote:
What could be more accurate than the palette that, when fed to the Super NES S-PPU, produces the same voltages?
Would this work?
Does the SNES work with definite RGB values respectively values that can at least be converted to RGB?
Yes. Each palette entry controls the level of each of red, green, and blue channels on a scale from 0 to 31.
Quote:
If yes, would another console that also works with RGB values produce the same colors when the same voltages are used?
Did you mean "another console" as in another Super NES Control Deck? Each unit with the same motherboard and PPU2 revision should produce the same signal given the same RGB triple. There are three revisions that matter as far as I can tell: 1/1/1 (launch), 2/1/3 (most), and 1CHIP (late and all mini). And even between those three, the variation shouldn't be noticeable.
Or did you mean "another console" as in a different platform entirely, such as a Super NES Control Deck vs. a GameCube with Game Boy Player, whose palette also uses 5-bit-per-channel RGB?
Quote:
And if you can answer all this with yes, then why didn't you say this earlier, let someone do it and be done with this whole topic
The reason I haven't done so is that I don't own an oscilloscope nor know how to use one nor where to rent one. The reason others haven't done so is that they appear not to be interested.
Quote:
Or do you have the equipment to measure the actual analog signal (voltage etc.) and then you simply cycle hrough all the colors individually until their analog signals matches with the one on the NES?
Oscilloscopes exist, though I don't happen to own one.
Quote:
And will there be a perfect match? What if one of the NES colors has, let's say, a color brightness of 83.6253 %, but on the NES, you only find colors with 83.6250 % and 83.6256 %?
Having a brightness between, say, 25/31 and 26/31? That's a possibility. But someone with an oscilloscope should be able to mark that it's between two values. And at that point, bickering about it would become even more sperglord-ish than I'd condone.
Dwedit wrote:
I don't think you'll find SNES colors that generate the same square waves as NES colors.
This 3.58 MHz square wave consists of some energy at DC that sets the luma, a 3.58 MHz sine wave that sets the chroma, plus some energy at 10.74 MHz (the third harmonic) and higher. The "color" as a TV receives it is defined by the luma and chroma. So if you make a low-pass filter with a corner frequency anywhere between 4.2 MHz (the standard corner for RF) and 10 MHz, you'll end up finding Super NES colors that generate the same
sine waves as NES colors after filtering.
tepples wrote:
Or did you mean "another console" as in a different platform entirely, such as a Super NES Control Deck vs. a GameCube with Game Boy Player, whose palette also uses 5-bit-per-channel RGB?
This.
Would the SNES RGB values be of any actual help, though? I mean, there's no guarantee that the same RGB values that an SNES happens to transform into NES-like colors will look anything like NES colors on a modern computer or television. Unless you're making an NES emulator for the SNES, I don't see how knowing these RGB values would help with anything.
DRW wrote:
Miyamoto was only the designer for "Super Mario Bros.", not for the NES itself. So, when he says that he took this specific color because it matched what he wanted for the sky, this says nothing about what the designers of the NES had in mind when defining this color. It only means that Miyamoto's TV happened to show the sky color in a puple-like way. But the design choices of this 1985 game still say nothing about the intended specifications of the 1984 console.
I actually think I remember reading an interview where Miyamoto claimed that he took part in choosing the colors for the Famicom. Take his whatever way you like. (I'm not sure how much influence he could've actually had -- the color generation method is such that the hues are pretty much evenly spaced, but maybe he could've had a say in the brightness/saturation of the colors.)
Quote:
It is an analog signal, consisting of voltages. There is no "most accurate" palette.
Well, I agree, but it has nothing to do with it being an analog signal. It has to do with how people (and TV manufacturers) are interpreting/implementing the standards.
TL;DR: Get rid of fixed palettes all together and replace with proper PAL/NTSC emulation with suitable user adjustable controls.
-----
If it hasn't been done already, then the PPU needs to be decapped and the video output circuitry traced and converted to a software algorithm. Next step is to feed the output of this algorithm into a software PAL/NTSC decoder... one with user adjustable realtime brightness/contrast/colour/tint controls.
Colours don't look right? Do what you did with your old TV and just play with the settings until it does.
The thing is, even I know that this is a rather steep task to perform. Hell, take the C64 emulation scene, the correct PAL colour values have been known for years (the pepto palette, derived from the workings of the VIC-II) but people still complain. VICE64 and other emulators implement proper PAL emulation so colour mixing tricks and other GFX techniques can be displayed properly, but people complain about the image not looking 'sharp and pixely'. They say that the colours look too washed out, while ignorant to the fact that the saturation can be turned up in the emulator just like you would turn up the colour knob on your telly.
Which brings me to the argument about the designer's opinions on the colours. Back in the day, most TV sets were adjusted with easy to reach, quick to turn controls. It would be easy for one person to adjust a TV to his or her liking, only for the next person to come along to set it to their preferences. For all we know, Miyamoto could have had the colour dial on his TV all the way to the max and could have thought that everybody else's TVs were dull looking.
Hojo_Norem wrote:
TL;DR: Get rid of fixed palettes all together and replace with proper PAL/NTSC emulation with suitable user adjustable controls.
Decoding the composite signal to a texture doesn't work so well if you're trying to run an emulator on a device whose CPU clock speed is well below 1 GHz, or if you're (say) trying to port an NES game to another retro platform while preserving colors as closely as possible.
Quote:
If it hasn't been done already, then the PPU needs to be decapped and the video output circuitry traced
Visual 2C02 has done this.
Quote:
and converted to a software algorithm. Next step is to feed the output of this algorithm into a software PAL/NTSC decoder... one with user adjustable realtime brightness/contrast/colour/tint controls.
That already exists. The problem is that not all TVs are adjusted the way you remember, and not all decoders in all makes and models of TV operate exactly the same. They may differ in their strategy to reduce the appearance of dot crawl or in their nonlinear hue warping to enhance skin tones. How many TV models do we want to emulate?
The point of comparing filtered composite output to that of the Super NES S-PPU is that the S-PPU is also capable of producing RGB at the same pixel rate and nonstandard scanline length as the NES. So if your video processing chain distorts NES video in a certain manner, it ought to be distorting Super NES video in the same way.
tepples wrote:
Decoding the composite signal to a texture doesn't work so well if you're trying to run an emulator on a device whose CPU clock speed is well below 1 GHz, or if you're (say) trying to port an NES game to another retro platform while preserving colors as closely as possible.
If you can't generate/decode the actual composite signal in real-time, can't you at least pre-compute an RGB palette by encoding a dummy video signal containing flat colors and decoding it according to the user's settings? Sure you wouldn't have the other peculiarities of the composite signal, like the color bleed out the dot crawl, but the basic colors would, in theory, be more accurate.
Quote:
The point of comparing filtered composite output to that of the Super NES S-PPU is that the S-PPU is also capable of producing RGB at the same pixel rate and nonstandard scanline length as the NES. So if your video processing chain distorts NES video in a certain manner, it ought to be distorting Super NES video in the same way.
But once you get those RGB values and use them in another device, the colors will not be coming from an SNES anymore, so they'll go through completely different transformations before they reach the display, won't they?
tepples wrote:
DRW wrote:
It is an analog signal, consisting of voltages. There is no "most accurate" palette.
What could be more accurate than the palette that, when fed to the Super NES S-PPU, produces the same voltages?
The SNES's video output path is completely unrelated to the NES's, so it's really no help. The SNES PPU outputs analog RGB which is converted to an NTSC or PAL signal by a separate encoder chip. The encoder is an off-the-shelf part, not a Nintendo custom, and different production runs of SNESes used different chips. And on top of that there are various resistors and capacitors between the PPU and the encoder and between the encoder and the multi-out jack, which introduces even more variability due to tolerances, aging, etc. on each of those components.
And even if you decide that the NTSC encoder in your SNES contains special Nintendo pixie dust, you still can't make it output "the same voltages" as a NES. The NES PPU outputs square waves, whereas proper NTSC encoders output sine waves. And many of the NES colors are more saturated than anything an RGB-to-NTSC encoder can output (not coincidentally, those are the colors that vary the most between TVs) You can program the SNES palette to RGB(31, 0, 0) and it's
still not as red as the NES color $06.
lidnariq wrote:
The only possible remaining variation in colors would be if the output impedance from the NES varies significantly across the 10 different voltages the PPU can emit. And even then, that should really just come down to a little variation in net hue rotation, net brightness, or net saturation.
Here is a morsel of evidence for this phenomenon.
Attached please find find a raw capture of the output of a Sharp Twin Famicom (model AN-505BK), sampled at eight times the color burst frequency (28.6363.. MHz). They were made with an older Hauppage ImpactVCB card, for which custom drivers are available (credit to reengine) that allow getting the raw, undecoded signal, before the card or its drivers mess with it.
This particular game scene has color 0xC ("cyan") in all four levels in sufficiently large horizontal areas to allow measuring the phase at all four levels. If the color burst/color 0x8 is at 180 degrees, then color 0xC is nominally at 300 degrees.
The archive also includes the output of my custom NTSC decoder as a .BMP file. I configured it to use the sync amplitude as a reference for overall signal gain, which makes the entire image brighter while clipping the 3x colors a bit in the RGB colorspace.
The phases of color 0xC that I actually measure (using the unclipped YIQ signal) are:
0x0C: 298.565552
0x1C: 292.832520
0x2C: 288.335663
0x3C: 285.327179
That's quite a considerable a difference of almost 15 degrees between the reference and the brightest version of the color. All four levels were measured within the same horizontal line (147 in the .BMP file).
Since the shift increases with brightness and not with saturation, then assuming I understand what has been written in this thread correctly, it is not slew rate effects (saturation) that matter, but output impedance (level).
At least some game programmers must have had this in mind. For example, the Combat screen in Fire Emblem Gaiden normally uses colors $16 and $26 for the enemy stats panel. If the enemy is almost defeated, the panel goes dark, but instead of going to colors $06 and $16, it goes to colors $05 and $16, exactly what one would do to account for a phase shift between the brightness levels.
This phenomenon should not be observed with a PAL console, as the PAL system is specifically designed to difference out such differential phase shifts. Instead, the 3x colors become more desaturated compared to a NTSC system adjusted to similar phase and baseline saturation. This matches my observation of the PAL NES' picture.
NewRisingSun wrote:
The phases of color 0xC that I actually measure (using the unclipped YIQ signal) are:
0x0C: 298.565552
0x1C: 292.832520
0x2C: 288.335663
0x3C: 285.327179
There's definitely not that much precision. But ... even assuming the fractional part is fictitious...
First I counted the total number of samples between the first end of hsync, and the last end of hsync. From that, I discovered that there was a 57ppm difference between the reference colorburst frequency of the NES and that of the capture card. (Hence 3.999772 below rather than 4)
Using this really simple matlab/octave script:
Code:
% -*- octave -*-
KunioSkyFH = fopen("Kunio-Sky.raw");
KunioSkyData = fread(KunioSkyFH,Inf,"uint8");
Axis1 = KunioSkyData .* cos([1:rows(KunioSkyData)]'*pi/3.999772);
Axis2 = KunioSkyData .* sin([1:rows(KunioSkyData)]'*pi/3.999772);
% 5th order filter at 1.4MHz when sampled at 28.636MHz
% we don't care too much about having a "correct" filter.
% just enough to preserve small-ish horizontal details
[LPNum,LPDen] = butter(5,1.4/28.636);
Axis1LP = filter(LPNum,LPDen,Axis1);
Axis2LP = filter(LPNum,LPDen,Axis2);
plot(atan2(Axis1LP,Axis2LP)*180/pi);
hold on;
plot(180+KunioSkyData);
hold off;
I extracted this one scanline:
Attachment:
Kunio-Sky_scanline147_highlighted.jpg [ 25.4 KiB | Viewed 4012 times ]
And got this plot of phase (top: samples; bottom: instantaneous chroma angle, note that it is nonsense during black or white pixels)
Attachment:
scanline147.png [ 28.8 KiB | Viewed 4012 times ]
If I zoom in (not included), I see ≈12 degrees of difference between the instantaneous angle of color $0C and color $2C. Maybe I made a mistake? You only saw ≈10 degrees.
Quote:
Since the shift increases with brightness and not with saturation, then assuming I understand what has been written in this thread correctly, it is not slew rate effects (saturation) that matter, but output impedance (level).
It would be lovely if you could test using other hardware. Other revisions of the 2C02, and other analog paths out of the PPU could easily change this exact behavior.
Quote:
At least some game programmers must have had this in mind. For example, the Combat screen in Fire Emblem Gaiden normally uses colors $16 and $26 for the enemy stats panel. If the enemy is almost defeated, the panel goes dark, but instead of going to colors $06 and $16, it goes to colors $05 and $16, exactly what one would do to account for a phase shift between the brightness levels.
I'm not convinced that's related. I have seen pixel artists talk about intentional hue shifts as things get darker.
lidnariq wrote:
There's definitely not that much precision
No, I was just giving you the raw numbers.
lidnariq wrote:
It would be lovely if you could test using other hardware. Other revisions of the 2C02, and other analog paths out of the PPU could easily change this exact behavior.
The AN505-BK has a G revision PPU. I have another Twin Famicom with an E revision, and then of course I could capture directly at the output pins of both PPUs.
According to my results, the phase shift seems to be not constant for all the phases within one of the four levels either. I measure color $31 at 319.9 degrees (nominal: 330), or a shift of 10.1 degrees, and color $36 at 113.1 degrees (nominal: 120), or a shift of just 6.9 degrees. I'm hoping that I'm just doing something wrong, because it would really be annoying and unfortunate if that were true. (This is from the same setup as the other captured image. I'll do the alternative methods once I have a flash cartridge that allows me to upload and run a program that displays, and allows me to capture, all colors at once.)
If the shift is one amount for even phases but a different amount for odd phases, then the duty cycle of the master clock may have something to do with it.
Do you have a flashcart? If so, it would make things much easier if you just use either
Blargg's Full Palette demo or anything else where I can identify specific color by definition/position rather than having to cross-reference it to a picture?
I should probably do this same matlab graph on the previous oscilloscope log of HardWareMan's Dendy.
Quote:
Do you have a flashcart?
Nope. I don't really have an overview of which one to get for a 60-pin system.
For a 60-pin system, the EverDrive N8 should be good enough.
NewRisingSun wrote:
color $31 at 319.9 degrees (nominal: 330)
color $36 at 113.1 degrees (nominal: 120)
Since I bothered to play around with the Pac-Land image and write down some numbers:
Blue $31 at 316-323° (nom: 330)
Blue $11 at 323-335° (nom: 330)
Magenta $15 at 84-90° (nom: 90)
Peach $36 at 109-117° (nom: 120)
Brown $07 at 150-158° (nom: 150)
Orange $27 at 140-146° (nom: 150)
Green $1B at 268-274° (nom: 270)
I clearly need to put more effort into filter design to reduce those ranges.
I don't use Matlab, but I can post the C source to my NTSC decoder that takes the .raw captured file and outputs a .rawraw RGB file, if that helps you.
NewRisingSun wrote:
Quote:
Do you have a flashcart?
Nope. I don't really have an overview of which one to get for a 60-pin system.
The Everdrive N8 has a Famicom 60-pin option, though I can also confirm that the PowerPak and Everdrive N8 72-pin both work on a Famicom using simple 72-60 pin converters.
You need to perform two simple soldering mods to get the most out of your PowerPak with just about all 72-to-60 pin converters and it the stacked PowerPak and converter can get a bit wobbly, but otherwise you will be good.
NewRisingSun wrote:
I don't use Matlab, but I can post the C source to my NTSC decoder that takes the .raw captured file and outputs a .rawraw RGB file, if that helps you.
Eh, the matlab/octave fragment was more for proving to myself that your methodology is correct. It clearly is, and now I think the only remaining parameter to be explored is just how much variation we should anticipate across all the hardware variants.
Is there an easy way to calculate how "non-ideal" either not a square wave or not a sine wave this is? Is the "skewness" test the right one?
Skewness of 10000 points of a sampled ideal sine wave appears to be approximately 0, but skewness of the huge patch of color $31 is 0.126 and the mean is very slightly higher than the median (Δ=0.336).
Skewness is a measure of asymmetry. A square wave with non-equal duty cycles is indeed asymmetrical, though I am not sure how this translates to the phase of the fundamental frequency. Tepples suggested hue-specific phase shifts resulting from hue-specific duty cycles. If non-equal duty cycles of a square wave translate into phase shifts at the subcarrier (fundamental) frequency, then the phase shift amount should be correlated with skewness in sign and magnitude.
For the record, I have used the same setup to measure the composite video output of a Tandy 1000 computer. It can output six hues at two luminance levels. The two levels are different for each of the hues. I measure no level-specific phase shift beyond the noise level between the two luminance levels of each hue:
Code:
Color 1: 0.008638 (blue, nominal: 0)
Color 2: 223.742996 (green, nominal: 225)
Color 3: 269.025116 (cyan, nominal: 270)
Color 4: 92.503235 (red, nominal: 90)
Color 5: 46.509937 (magenta, nominal: 45)
Color 6: 180.940750 (yellow, nominal: 180)
Color 9: 359.858380 (blue, nominal: 0)
Color A: 223.127350 (green, nominal: 225)
Color B: 268.470940 (cyan, nominal: 270)
Color C: 91.619957 (red, nominal: 90)
Color D: 46.090595 (magenta, nominal: 45)
Color E: 179.678253 (yellow, nominal: 180)
This tells me that whatever I am measuring from the Famicom is not an artifact introduced by my capturing hardware, otherwise I would be measuring level-specific phase shifts in the the Tandy 1000's composite video output as well. I have attached that capture as well, in case you want to try your skewness test on that file.
I have received my Everdrive N8. I have used "Blargg's Full Palette" to make raw capture files that show all 52 colors. The attached archive contains three files:
- Palette-RP2C02E-RCA.raw: Signal from the yellow RCA socket at the back of the Sharp Twin Famicom AN-500B (PPU: RP2C02E-0)
- Palette-RP2C02G-PPU.raw: Signal from the PPU's output pin of the Sharp Twin Famicom AN-505BK (PPU: RP2C02G-0)
- Palette-RP2C02G-RCA.raw: Signal from the yellow RCA socket at the back of the Sharp Twin Famicom AN-505BK (PPU: RP2C02G-0)
I was unable to get any signal from the output pin of the RP2C02E-0.
I am posting these files without comment for now, because I am curious about what conclusions other people draw from them, before I share mine.
My biggest observation is what I see in the first scanline of the 2C02E output of colors $1x :
At this point, I have no idea what to think any more. Colors $12, $16, and $1A are a different phase offset than the other ones, not restricted to just even or odd phases.
Do the affected colors remain $12, $16, and $1A even across resets? Perhaps the phase might get distorted more when signal level transitions coincide with the pixel clock.
As far as I can tell, they remain the same. Attached find four different capture files from the RP2C02E, each taken after another RESET. Note that the Everdrive UI appears between the RESET and the palette test program.
I have captured the output (see attachment) of a PAL NES with an RP2C07 PPU, from the yellow RCA socket. I have not modified the capture program itself, which is why it only captures five sixths of the 1000 ms/50Hz field. Note how the RP2C07, unlike the RP2C02, never illuminates the border area with background color 0.
The first picture shows the result I get when I use my "Simple PAL" decoder --- properly accounting for the phase shift in the V channel from scanline to scanline, but not averaging each scanline's demodulated chroma signal with the delayed previous scanline's demodulated chroma signal. If the signal is affected by phase distortion, then we should see
Hanover bars - opposite hue errors from scanline to scanline:
The second picture shows the result I get when I use my "Full PAL" decoder --- averaging each scanline's demodulated chroma signal with the delayed previous scanline's demodulated chroma signal:
These results indicate to me that the PAL NES is just as susceptible to considerable phase shifts as the NTSC NES, but because the PAL system is designed to eliminate the effects of phase shifts, PAL televisions will display colors that are almost exactly the nominal hues minus 15.0 degrees.
Capturing regular television pictures and decoding them with my "Simple PAL" decoder exhibits only very slight Hanover bars, indicating little phase distortion:
RAW... OK... but how to parse the file, as RGB or any other format?
In case it isn't obvious to others reading this, the reason you don't get Hanover bars from a broadcast signal is the 90 degree swing of broadcast TV's color burst compared to the nonstandard 120 degree swing from the 2C07.
Zepper wrote:
RAW... OK... but how to parse the file, as RGB or any other format?
Implement a PAL or NTSC decoder. Raw values are just direct immediate voltages, higher = brighter.
tepples wrote:
In case it isn't obvious to others reading this, the reason you don't get Hanover bars from a broadcast signal is the 90 degree swing of broadcast TV's color burst compared to the nonstandard 120 degree swing from the 2C07.
You're confusing the phase of the color subcarrier
relative to the horizontal sync signal with the phase of the color subcarrier
relative to itself. The 2C07 does not switch the phase of the color subcarrier
relative to itself by 120 degrees from scanline to scanline, and so the 2C07's non-standard scanline duration has nothing to do with the presence or absence of Hanover bars.
... You know, I actually saw evidence of this phase error when I made my
bitbanged RS170 test. The colorburst was always a little greener than the corresponding field of hue 8, and it changed depending on the nominal brightness of that field of hue 8.
And I didn't know what I was looking at and told myself it was experimental error at the time. Oops!
It looks like FireBrandX
updated his palettes yet again. It sounds like "PVM Style D93 (FBX)" would be probably be the most applicable one.
