Written in Python. Uses a different transformation to get the RGB values than Kevin's. I've tried the palette it generates in FCEU and it looks much better than Kevin's does with some games (like FF3j) The docstrings and the sites referenced in them pretty much explain everything.
Code:
"""AWJ's NTSC NES palette generator.
Based on Kevin Horton's palette generator
and on information from these web pages:
http://unusedino.de/ec64/technical/misc/vic656x/colors/
http://en.wikipedia.org/wiki/YIQ
http://en.wikipedia.org/wiki/YUV
"""
import array
import math
hue = (0, 285, 315, 345, 15, 45, 75, 105, 135, 165, 195, 225, 255, 0, 0, 0)
"""Table of hue values (theta coordinate on Q-I plane)
Assumption: That the 12 non-grey hues are equally spaced
30 degrees apart, and offset 15 degrees from the axes.
This assumption allows hue 8 (colors 08, 18, 28, 38) to be
very close to pure yellow (R = G > B), and for hue 6 (06, etc.)
to be fairly close to pure red (R > G = B).
The alternate assumption, that every third hue is aligned with
an axis, results in no hue being close to pure yellow or pure red.
TODO: (by someone else)
verify hues using a video signal separator and a real NES
"""
sat = 0.146
"""Saturation value (radius coordinate on Q-I plane)
Assumption: That all 12 non-grey hues have the same saturation.
According to Kevin Horton (and parsimony) this is the case.
Currently this is an arbitrary value, chosen such that
color 08 ends up with a blue level of exactly zero.
"""
luma = ((0.50, 0.75, 1.00, 1.00),
(0.29, 0.45, 0.73, 0.90),
(0.00, 0.24, 0.47, 0.77))
"""Table of luma (Y) values
Assumption: That these values (stolen from Kevin Horton's
QBASIC palette generator) are accurate :)
Are color 00 and 2D, and 10 and 3D, really this close?
All hand-hacked NES palettes I've seen make them
distinctly different shades of grey from each other
(3D brighter than 10, and 2D much darker than 00)
TODO: (by someone else)
verify values, ideally on several hardware versions
(frontloader, toploader, original Famicom, AV Famicom)
"""
def yiq2rgb(y, i, q):
"""Convert a YIQ color to RGB.
Transformation matrix stolen from here:
http://www.mathworks.com/access/helpdesk/help/toolbox/images/ntsc2rgb.html
"""
r = y + 0.956 * i + 0.621 * q
g = y - 0.272 * i - 0.647 * q
b = y - 1.106 * i + 1.703 * q
return (r, g, b)
# Main program begins here
nespalette = array.array('B')
for i in range(4):
for j in range(16):
h = math.radians(hue[j])
if j > 13:
s = 0
y = 0
elif j == 0:
s = 0
y = luma[0][i]
elif j == 13:
s = 0
y = luma[2][i]
else:
s = sat
y = luma[1][i]
r, g, b = yiq2rgb(y, s * math.sin(h), s * math.cos(h))
# clip RGB values
if r < 0.0: r = 0.0
if g < 0.0: g = 0.0
if b < 0.0: b = 0.0
if r > 1.0: r = 1.0
if g > 1.0: g = 1.0
if b > 1.0: b = 1.0
# TODO: add gamma correction
# convert from [0..1] to [0..255]
r = round(r * 255.0)
g = round(g * 255.0)
b = round(b * 255.0)
# add this color to the palette
nespalette.fromlist([int(r), int(g), int(b)])
outputfile = file("awj.pal", "wb")
nespalette.tofile(outputfile)
outputfile.close()
Hmm... I see plenty of views but no replies. Let me try to explain what's going on a bit more, particularly what the deal is with this YIQ stuff.
As you'll know if you've hooked up a DVD player, a television video signal is composed of three components: a brightness or "Luma" (Y) component plus two components that determine color, a blue component (called U, Pb or Cb depending on scaling) and a red component (V, Pr or Cr).
Colorspaces based on brightness, blue and red components are called YCC colorspaces and there are many different ones, the difference between them being the scaling of the three components with respect to each other. The colorspace color television uses, whether NTSC or PAL, is YUV. The colorspace digital video on a DVD uses, on the other hand, is YCbCr. This gets converted to YUV by multiplying the blue component by one constant and the red component by another.
The reason Kevin Horton's old palette generator produces inaccurate colors (oversaturated reds and weak blues) is that he mistakenly used an RGB conversion formula for one of the other YCC colorspaces out there, not YUV.
Inside the innards of your television set, the YUV components undergo a linear transformation to get RGB values. The Wikipedia entry for YUV includes the matrix to transform from RGB to YUV. What a television performs, obviously, is the mathematical inverse of that transformation.
Now, the color information in a color NTSC signal (the stuff that comes in on the "C" pin on an S-Video jack, or the "Chroma" RCA jack on an oldschool Commodore monitor) consists of two components: I and Q. These are not the red and blue color components, like you might expect; rather, they're a pair of coordinates on a plane which need to be rotated around the origin in order to get V and U. The hue (aka tint) control on a television controls the angle of rotation.
The reason NTSC uses this seemingly Rube Goldberg-esque setup, which is responsible for the unflattering alternate expansion "Never The Same Color", has to do with human color perception and optimal usage of bandwidth. There's a brief explanation in the Wikipedia entry for YIQ. Basically the I axis, which distinguishes the hues to which humans are most sensitive, gets the lion's share of bandwidth--in a standard broadcast signal, I uses nearly three times as much bandwidth as the less important Q.
Note that a PAL color signal is completely different from NTSC, and actually directly encodes U and V, the blue and red components. No rotation is required, and therefore PAL television sets don't need a hue control (and don't have one, unless they're dual-standard TVs that can take NTSC input)
The "canonical" angle to rotate the I and Q coordinates by in order to get V and U is 33 degrees. In other words, a television with hue correctly calibrated (by using a standard color bar pattern) will rotate the coordinates by exactly this angle. The matrix found in the yiq2rgb() function in my program combines in a single transformation the 33° rotation of I and Q, with the conversion from YUV to RGB.
So what has all of this got to do with the NES? Well, the NTSC NES PPU's video output is a composite NTSC signal consisting of Y, I and Q components. If we know what values of Y, I and Q the PPU produces for a given palette color, we can use the YIQ-to-RGB transformation to get the exact RGB colors that will be generated on a canonically calibrated television.
Several years ago, Kevin Horton determined the Y value for each NES color and incorporated his findings into his palette generator (though as I mention in the program documentation I'm just a little bit suspicious of some of those values, and it would be nice if someone could reproduce and confirm them). Determining experimentally the I and Q values for each hue is not as easy to do; you can buy a Y/C splitter for ten bucks, but splitting the C signal into I and Q requires more specialized hardware. Therefore, for now, we have to settle for determining them inductively.
The most parsimonious theory is that the I and Q values for the twelve nongrey hues are evenly spaced points on a circle. Furthermore, it is most likely that the twelve points are either aligned on the I and Q axes or that the axes pass midway between two adjacent points. The reason is that if either of these is the case, then the PPU only needs to be able to generate 6 different nonzero levels of I and 6 different nonzero levels of Q. If the twelve points are not aligned with the axes, on the other hand, the PPU has to be able to generate 12 different nonzero levels of I and Q.
Given that hue 8 is known to be more or less yellow, there are three arrangements of the 12 hues that seem reasonable:
1: Hue 6 lies on the positive I axis. This makes hue 8 slightly green.
2: Hue 7 lies on the positive I axis. This makes hue 8 slightly orange.
3: The positive I axis passes midway between hues 6 and 7. This makes hue 8 almost pure yellow.
My program goes with arrangement 3, for two reasons. The first is simply that it's the median hypothesis out of the three candidates. The second is that I tested all three with an assortment of commercial games, including games whose colors on the real console I remember well (my NES is in a closet a few hundred miles away, but the colors of Ultima: Quest of the Avatar's overworld are permanently etched into my brain) The games were evenly split between those that looked good with 1 and 3, and those which looked good with 2 and 3. The clincher game was Final Fantasy 3 (despite the fact that I've never played it on a real NES) Both 1 and 2 had obvious problems with this game: 1 resulted in very purple water, while 2 resulted in brownish grass and trees in towns.
The other half of the question is the saturation--the radius of the circle plotted by the 12 hues. If the saturation is too high, the RGB values for some colors go out of gamut and have to be clipped, resulting in distorted hues. What I did was choose the largest saturation such that no color in the lower two brightnesses suffered clipping. This was probably too conservative of me--the hues are certainly accurate, but the colors are quite dull. What I'm going to try next is to use the largest saturation such that no color's red component goes out-of-gamut (Blue is the first component to go out-of-gamut with excessive saturation, followed by red and finally green)
i have implemented the ntsc -> rgb conversion using the yiq transformation matrix, and i am quite happy with the results.
i am using a simplified version of kevin's converter algorithm, with luma values at regular intervals: 0.0, 0.25, 0.5, 0.75, 1.0.
there's also no need for the color angle table. color angles simply start at 255 degrees (or whatever your "tint" preference is) and increase 30 degrees for each hue.
good work on this.
This is good stuff here.
Here's the rgb output if anyone wants it (as I would) (in order):
[r, g, b]
(row 1)
[128, 128, 128]
[46, 77, 130]
[65, 64, 148]
[87, 53, 146]
[105, 48, 125]
[115, 50, 90]
[114, 58, 51]
[102, 70, 18]
[83, 84, 0]
[61, 95, 2]
[42, 100, 23]
[32, 98, 58]
[34, 90, 97]
[0, 0, 0]
[0, 0, 0]
[0, 0, 0]
(row 2)
[191, 191, 191]
[86, 118, 171]
[106, 105, 189]
[128, 94, 187]
[146, 89, 165]
[156, 91, 130]
[155, 99, 91]
[143, 111, 59]
[124, 125, 41]
[102, 135, 43]
[83, 141, 64]
[73, 139, 99]
[74, 131, 138]
[61, 61, 61]
[0, 0, 0]
[0, 0, 0]
(row 3)
[255, 255, 255]
[158, 190, 242]
[177, 176, 255]
[199, 166, 255]
[218, 160, 237]
[228, 162, 202]
[227, 170, 163]
[215, 183, 130]
[195, 196, 112]
[173, 207, 114]
[155, 212, 136]
[145, 210, 170]
[146, 202, 210]
[120, 120, 120]
[0, 0, 0]
[0, 0, 0]
(row 4)
[255, 255, 255]
[201, 233, 255]
[221, 220, 255]
[243, 209, 255]
[255, 204, 255]
[255, 205, 245]
[255, 213, 206]
[255, 226, 173]
[238, 239, 156]
[216, 250, 158]
[198, 255, 179]
[188, 254, 214]
[189, 246, 253]
[196, 196, 196]
[0, 0, 0]
[0, 0, 0]
And here it is in binary format:
NTSC.pal
Do you realize there is not a primary color anywhere near that palette? That is awfully faded or pastel-like, surpassing FCE Ultra's NTSC emulation.
Great Hierophant wrote:
That is awfully faded or pastel-like, surpassing FCE Ultra's NTSC emulation.
That's due to the default saturation in the first version being too low, as I said. I've subsequently changed it to 0.221. Saturation is very much an "adjust-to-taste" setting: the only way to get a canonical value would be to compare the PPU's chroma output to the standard NTSC color bar pattern. If you incorporate this formula into a NES emulator then saturation (and gamma, which I've also subsequently added) should be user-adjustable.
To incorporate gamma correction, add the following lines to the program after clipping the RGB values:
Code:
# correct gamma
r = pow(r, 2.2 / gamma)
g = pow(g, 2.2 / gamma)
b = pow(b, 2.2 / gamma)
You'll also have to add a value for gamma to the program, of course. The default value should probably be 2.2, which results in no gamma correction.
The next frontier in palette emulation is to determine what the color emphasis bits do at the level of NTSC output. I have a theory, but no way to confirm it or to get the exact numbers, since I don't have a devcart.
Nice. A couple of points:
Quote:
we can use the YIQ-to-RGB transformation to get the exact RGB colors that will be generated on a canonically calibrated television.
Note that most NTSC consumer TV sets are not and can't ever be "canonically calibrated", older devices even less than newer and cheaper devices less than expensive ones. Using the by-the-book decoder matrix will result in a palette that is "accurate" in that it looks the same as on a studio-quality monitor, but not "accurate" in that it looks the way the games were designed to look.
NES graphics artists most likely used and optimized for consumer-grade equipment, which systematically and intentionally deviates from the NTSC specification (i.e. wide-angle color demodulation, flesh-tone boost); with Japanese TV sets using different algorithms than US TV sets (for example, Sony TVs have an "AXIS" setting in service mode that can be set to "US" or "JAPAN").
Quote:
Hue 6 lies on the positive I axis. This makes hue 8 slightly green.
This one is correct. Hue 8 is a yellowish green on a NTSC-faithful color decoder at the "canonical" hue setting.
Quote:
Both 1 and 2 had obvious problems with this game: 1 resulted in very purple water, while 2 resulted in brownish grass and trees in towns.
That's exactly the problem you're always going to have with a 100% faithful NTSC decoder, no matter what your tint setting is. All Japanese games are designed for Japanese consumer TV sets, which place the I and Q axis about 105 to 120 degrees apart (standard NTSC uses 90 degrees, as we know) and boost yellows (I>Q colors) to make Asian fleshtones more "natural" on the "colder" 9300K phosphors that they use in Japan.
I made a hack to the palette generator that generates one I like a little better.
It doesn't matter how saturated I make the colors, blue always ended up too strong, and red always ended up too weak, so I just flip flopped them (red got outputted to b, and blue got outputted to r), and then I adjusted the hues to make the colors right, and it made a nice palette... the yellow got slightly more brown as it got darker, and red now can get strong without blue getting over saturated.
But that was just my personal preference, and I know it's probably not accurate, but it looks good to me. I read that red on a TV was the strongest color... like if you turned the color saturation WAY up, the red would get really strong and bleed into the other colors first.
Isn't it impossible to emulate the NES color accurately without handling where pixels fall relative to the pixel clock or something like that? This seems the direction to go for accurate color rendering. A palette can only go so far, and if the emulator isn't taking the effect of the pixel's position into account, each palette is a tradeoff and will make some games look better than others.
That should only apply to "artifact colors", like on the Apple ][ or the CGA's 160x200 composite color mode, where the color is determined by which monochrome pixel columns are illuminated.
NewRisingSun wrote:
Nice. A couple of points:
(snip)
NES graphics artists most likely used and optimized for consumer-grade equipment, which systematically and intentionally deviates from the NTSC specification (i.e. wide-angle color demodulation, flesh-tone boost); with Japanese TV sets using different algorithms than US TV sets (for example, Sony TVs have an "AXIS" setting in service mode that can be set to "US" or "JAPAN").
This is very interesting information. Can you provide more details as to how US and Japanese sets differ from each other and from the NTSC specification, or does it vary from manufacturer to manufacturer?
If you do not emulate "artifact colors", then the bricks in underground levels of Super Mario Bros. won't look like those on real NES hardware. Instead, they might look like the bricks on a VS Unisystem or a PlayChoice 10.
Anyway, the output of the PPU in palette values $x1-$xC is a square wave that toggles between the values used for the $x0 and $xD gray series. This should help you determine the correct saturation.
As for hue spacing, at least $x1, $x3, $x5, $x7, $x9, $xB are spaced evenly, and so are $x2 through $xC even. The spacing between $x1 and $x2, between $x3 and $x4, and so on seems like it would depend on the duty cycle of the master clock (21.48 MHz, 6*colorburst), as the PPU's hue generator is based on a 12-step counter clocked on positive and negative edges.
tepples wrote:
Anyway, the output of the PPU in palette values $x1-$xC is a square wave that toggles between the values used for the $x0 and $xD gray series. This should help you determine the correct saturation.
So colors 01-0C oscillate between the values of 00 and 0D, 11-1C oscillate between 10 and 1D, et cetera? That would mean that the lightest row of non-gray colors is approximately half as saturated as the other three rows. Great information. How accurate are kevtris's values for the luminances of the gray colors?
I daren't suppose you know the details of how the color emphasis bits affect the output...
"artifact colors" are unfortunately outside the scope of a simple palette-generating algorithm, though it would be interesting (but probably beyond my skill) to add them to an emulator.
NewRisingSun wrote:
Quote:
Hue 6 lies on the positive I axis. This makes hue 8 slightly green.
This one is correct. Hue 8 is a yellowish green on a NTSC-faithful color decoder at the "canonical" hue setting.
Aha, it must be about the same hue as the uniforms in the original Star Trek series. The actual uniforms were pale green, but they appeared yellow on consumer television sets.
Sorry, that last post was me.
If you do not emulate "artifact colors", then the bricks in underground levels of Super Mario Bros. won't look like those on real NES hardware. Instead, they might look like the bricks on a VS Unisystem or a PlayChoice 10.
And they won't shimmer when you scroll the screen (bothersome or not, it was part of the NES experience). I take it Nintendo's arcade boxes used RGB (why use lossy modulation schemes when you can just send the color components directly to the CRT?).
blargg wrote:
I take it Nintendo's arcade boxes used RGB (why use lossy modulation schemes when you can just send the color components directly to the CRT?).
Yes, NES-based arcade hardware generated RGB from a palette stored in a PROM, like typical arcade video hardware of the time (more modern arcade hardware has fully software-defined palettes, like a SNES or a PC's VGA card). The only unusual thing about Nintendo's arcade hardware was that the color PROM was embedded right in the PPU rather than being on a separate chip; games with different palettes used different PPU models. This may have been intended as a form of copy protection.
tepples wrote:
If you do not emulate "artifact colors", then the bricks in underground levels of Super Mario Bros. won't look like those on real NES hardware. Instead, they might look like the bricks on a VS Unisystem or a PlayChoice 10.
The NES does not use "artifact colors", period. Forget about them, they only apply to the Apple ][ and the CGA 160x200 composite color mode. It doesn't apply the NES simply because the NES generates colors differently. The word "artifact colors" has a specific meaning which does not apply to the NES PPU.
What you're describing is the result of the fact that the PPU generates the luminance and the chrominance portion of the video signal in the same process and that it generates square waves instead of sine waves --- it has nothing to do with "artifact colors". The "shimmering" by the way only occurs in the NTSC PPU, the PAL PPU has constant "jagginess".
I don't even know whether that's worth emulating --- after all, it doesn't affect the COLOR of a pixel, only the placement.
Quote:
NES-based arcade hardware generated RGB from a palette stored in a PROM,
Right, but keep in mind that the palette found in RGB PPUs is not accurate in that it would correctly represent what NES games should look like --- several PlayChoice 10 games (Volleyball, Punch-Out) actually had their color values changed in the game code from their regular cartridge versions to look better on the PC10 PPU.
Quote:
That would mean that the lightest row of non-gray colors is approximately half as saturated as the other three rows.
It is true that x1-xc colors oscillate between the x0 and xd values, HOWEVER that information is only relevant when you're dealing in voltage levels --- it's unusable if you're trying to deduce saturations from a "TV-captured" palette, because there is not a set of x0/xd values that is not clipped --- for 0x and 1x, the d color is clipped at zero; for 2x and 3x, the 0 color is clipped at 255 (or 1.0, whatever your notation is).
Chances are that the 3x colors are just as saturated as the rest (level-wise), but appear less saturated because most TVs "hard clip" the input signal at 120 IRE, thus reducing modulation and therefore, saturation.
Quote:
Can you provide more details as to how US and Japanese sets differ from each other and from the NTSC specification, or does it vary from manufacturer to manufacturer?
The coefficients indeed differ from manufacturer to manufacturer and from model to model, but the algorithms are more-or-less the same. For example, a given US TV set might use a color decoder with the following characteristics (taken from a Sony service manual):
R-Y: 112° ´ 0.83, G-Y: 252° ´ 0.3 (B-Y: 0° ´ 1)
This would translate into the following decoder matrix:
R = Y + 1.539*V - 0.622*U
G = Y - 0.571*V - 0.185*U
B = Y + 0.000*V + 2.000*U
This matrix represents a typical "red boost", supposedly making flesh tones more "natural" on 9300K phosphors (manufacturers use 9300K phosphors because they look "brighter" than the 6500K that the standard calls for). Looks a little like Chris Covell's old captured palette.
Again, other sets might use different coefficients, but the algorithm is basically the same, at least for US sets.
Japanese TV sets are a different story, because Asian fleshtones are different; instead of a "red boost", you have a "yellow boost". Basically, "Yellowness" is (I-Q), so, to boost yellows, you'd do something like this:
Yellowness = (I-Q);
if (Yellowness > 0) {
I = I + Yellowness*factor;
Q = Q - Yellowness*factor;
}
"factor" would of course be < 1. The decoding matrix would be more-or-less like the above with a little less gain in the red channel, I suppose.
Only as a general idea though, I don't know the exact coefficients for any given set because newer Japanese TVs (to whose service manuals I have access) don't use "yellow boost" anymore because everyone over there uses Digital TV now and thus these "picture improvement" features have become obsolete. Consequently, the matrices listed in today's Japanese TV's service manuals are basically like the same as the NTSC standard's.
Anyway, maybe you'd like to see my calculated "Japanese" palettes, which I like very much and match those old Japanese TVs I've seen:
http://www.lau-net.de/~nl2305/6500K.pal
http://www.lau-net.de/~nl2305/9300K.pal
Use the first one if your PC monitor is set to 6500K (it usually is if you've calibrated your monitor for photoshop) --- it will simulate the blueish picture of a 9300K monitor; if your PC monitor is set to 9300K (which most monitors are factory-setting-wise), use the second one, it's like the "raw output". Make sure you're getting the right one for accuracy --- if you use the 6500K.pal on a 9300K monitor, it'll look way too blue; if you use the 9300K.pal on a 6500K monitor, it'll look too yellowish.
Notice how the yellow boost that is applied in these palettes makes SMB3's color combinations of 0x27 and 0x36 actually look sensible, or likewise 0x29 and 0x38 in Ice Climber and others.
NewRisingSun wrote:
It is true that x1-xc colors oscillate between the x0 and xd values, HOWEVER that information is only relevant when you're dealing in voltage levels --- it's unusable if you're trying to deduce saturations from a "TV-captured" palette, because there is not a set of x0/xd values that is not clipped --- for 0x and 1x, the d color is clipped at zero; for 2x and 3x, the 0 color is clipped at 255 (or 1.0, whatever your notation is).
Chances are that the 3x colors are just as saturated as the rest (level-wise), but appear less saturated because most TVs "hard clip" the input signal at 120 IRE, thus reducing modulation and therefore, saturation.
So, can someone use a scope and get the actual voltage levels for all the x0 and xD colors?
NewRisingSun wrote:
The coefficients indeed differ from manufacturer to manufacturer and from model to model, but the algorithms are more-or-less the same. For example, a given US TV set might use a color decoder with the following characteristics (taken from a Sony service manual):
R-Y: 112° x 0.83, G-Y: 252° x 0.3 (B-Y: 0° x 1)
This would translate into the following decoder matrix:
R = Y + 1.539*V - 0.622*U
G = Y - 0.571*V - 0.185*U
B = Y + 0.000*V + 2.000*U
Hmm, it looks like I misunderstood what those angles meant. I assumed that 112° and 252° were the angles of the hues that decoded to pure red (R > G = B) and pure green (G > R = B) relative to the hue that decoded to pure blue (B > R = G) For a standard NTSC decoder, the angles of the pure primary colors from the Q axis are:
R = tan-1[(-0.647 - 1.703)/(-1.106 + 0.272)] = 70.5°
G = tan-1[( 0.621 - 1.703)/(-1.106 - 0.956)] = 207.7°
B = tan-1[( 0.621 + 0.647)/(-0.272 - 0.956)] = -45.9°
The angle between red and blue is 116.4° and the angle between green and blue is 253.6°, which are very close to the angles given in the Sony datasheet, so you can see why I thought along these lines.
Previously you referred to I and Q being 105°-120° apart on Japanese decoders, but now you're talking in terms of U and V. Could you clarify what you meant earlier in reference to Japanese equipment?
NewRisingSun wrote:
Anyway, maybe you'd like to see my calculated "Japanese" palettes, which I like very much and match those old Japanese TVs I've seen:
http://www.lau-net.de/~nl2305/6500K.palhttp://www.lau-net.de/~nl2305/9300K.pal
I think I'd rather see the algorithm you used to create those palettes. What I'm really interested in is not creating a single "perfect" NES palette, but an algorithm with parameters which can be tweaked to simulate the colors produced on a variety of displays. This algorithm could be used in emulators of other
Before I forget, I had one question about the PAL version of the NES. Since PAL uses YUV instead of YIQ, are the hues on a PAL PPU "rotated" one step compared to a NTSC PPU? I've never seen a PAL NES in operation, but the "PAL palettes" for emulators I've seen (such as the one hardcoded into FCEUltra) all make hue x8 orange-yellow, not green-yellow.
This time not only did I forget my name, I left a sentence unfinished. What I meant to say in the second-last paragraph was "This algorithm could be used for emulation of other composite video hardware similar to the NES PPU in the way they produced colors, such as the Atari TIA family and the Commodore VIC, VIC II and TED (used in the VIC20, C64/C128 and Plus4/C16 respectively)
Quote:
For a standard NTSC decoder, the angles of the pure primary colors from the Q axis are:
Nonono, that's completely wrong. If you want to convert RGB values to NTSC and back, you need to use the proper matrices. Trigonometry alone won't get you anywhere until you're using the proper scale factors.
See
http://wwwzenger.informatik.tu-muenchen ... COL_29.htmQuote:
Previously you referred to I and Q being 105°-120° apart on Japanese decoders, but now you're talking in terms of U and V. Could you clarify what you meant earlier in reference to Japanese equipment?
It's just different ways of talking about the same thing:
V = 0.877*(R-Y)
U = 0.493*(B-Y)
I = V plus 33 degrees
Q = U plus 33 degrees.
R-Y is at 90 degrees, as is V.
B-Y is at 0 degrees, as is U.
I is at 123 degrees.
Q is at 33 degrees.
You can decode the same signal either using I and Q, or U and V; the result is the same except that the hue offset is different, which is okay since all NTSC decoders have a hue control anyway.
I and Q, as well as U and V, are 90° apart in standard; they are further apart (up to 120°) in common implementations.
Quote:
Since PAL uses YUV instead of YIQ, are the hues on a PAL PPU "rotated" one step compared to a NTSC PPU?
The PAL NES creates hues that are rotated half a step (15°) vs. the NTSC PPU, making color x8 look like SMPTE yellow (R=G). Forget about "hard-coded" palettes, they're based on what emulator authors think "looks good". Yes, the PAL NES actually creates different colors than the NTSC NES.
Quote:
I think I'd rather see the algorithm you used to create those palettes.
Well, what did I just post before, about "Yellowness" and such...
Quote:
This algorithm could be used for emulation of other composite video hardware similar to the NES PPU in the way they produced colors, such as the Atari TIA family and the Commodore VIC, VIC II and TED (used in the VIC20, C64/C128 and Plus4/C16 respectively
Well, these are American machines, so try the US decoder matrix I gave you. You can also convert SNES RGB values to YUV using the standard encoder matrix and back to RGB using the "Sony US TV" decoder matrix if you're in the mood.
NewRisingSun wrote:
Quote:
For a standard NTSC decoder, the angles of the pure primary colors from the Q axis are:
Nonono, that's completely wrong. If you want to convert RGB values to NTSC and back, you need to use the proper matrices. Trigonometry alone won't get you anywhere until you're using the proper scale factors.
See
http://wwwzenger.informatik.tu-muenchen ... COL_29.htmI think you misunderstood what I was doing. I was using the "proper" matrix:
Code:
[ R ] [ 1 0.956 0.621 ][ Y ]
[ G ] = [ 1 -0.272 -0.647 ][ I ]
[ B ] [ 1 -1.106 1.703 ][ Q ]
What I was doing was
inverting this matrix to find the NTSC hues of SMPTE red, SMPTE green and SMPTE blue. An equivalent (and simpler) way of doing it would have been to start with the inverse matrix:
Code:
[ Y ] [ 0.299 0.587 0.114 ][ R ]
[ I ] = [ 0.596 -0.274 -0.322 ][ G ]
[ Q ] [ 0.211 -0.523 0.312 ][ B ]
and just plug in RGB triplets of (1, 0, 0), (0, 1, 0) and (0, 0, 1) (Yes, I know these are too saturated to be legal NTSC colors, but the math is simplest this way...)
Doing this I get:
RGB(1, 0, 0) = YIQ(0.299, 0.596, 0.211)
hue = tan-1(0.596 / 0.211) = 70.5°
RGB(0, 1, 0) = YIQ(0.587, -0.274, 0.523)
hue = tan-1(-0.274 / -0.523) = 207.7°
RGB(0, 0, 1) = YIQ(0.114, -0.322, 0.312)
hue = tan-1(-0.322 / 0.312) = -45.9°
...which are exactly the same hues as I got by inverting the YIQ-to-RGB matrix (as expected).
Since you've made it clear that this isn't what those numbers mean in the first place, this discussion is probably meaningless.
By the way, the RGB-to-YIQ matrix on that page you linked to has a couple of the signs backwards.
NewRisingSun wrote:
Quote:
Previously you referred to I and Q being 105°-120° apart on Japanese decoders, but now you're talking in terms of U and V. Could you clarify what you meant earlier in reference to Japanese equipment?
It's just different ways of talking about the same thing:
Yes, I already know what the relationship between I/Q and U/V is. Sorry, I understand what you meant now.
NewRisingSun wrote:
Quote:
Since PAL uses YUV instead of YIQ, are the hues on a PAL PPU "rotated" one step compared to a NTSC PPU?
The PAL NES creates hues that are rotated half a step (15°) vs. the NTSC PPU, making color x8 look like SMPTE yellow (R=G). Forget about "hard-coded" palettes, they're based on what emulator authors think "looks good". Yes, the PAL NES actually creates different colors than the NTSC NES.
I suspected such, but every NES dever I asked insisted that NTSC and PAL units produced the exact same colors. Thanks for the confirmation.
Quote:
(Yes, I know these are too saturated to be legal NTSC colors, but the math is simplest this way...)
Both 100% red and 100% blue are completly legal NTSC colors, in fact, the 0.877 and 0.493 coeffecients were deliberately chosen so that 100% red and 100% blue, which do occur in nature, WOULD be legal. 100% yellow is "illegal" in NTSC because the upper peak of the modulation exceeds 120 IRE (reaches about 133 IRE).
By the way, there's one more systematic difference between Japanese and "real" (American) NTSC: the NTSC standard calls for a 7.5% setup, Japanese devices however use 0%. I think you can even select between these two on some camcorders...
Quote:
Since you've made it clear that this isn't what those numbers mean in the first place, this discussion is probably meaningless.
Your equations would be right if it was about the R axis, not the R-Y axis...
Quote:
By the way, the RGB-to-YIQ matrix on that page you linked to has a couple of the signs backwards.
Right, I've been meaning to email the author of those pages a couple of times, but always forgot...
Quote:
I suspected such, but every NES dever I asked insisted that NTSC and PAL units produced the exact same colors. Thanks for the confirmation.
I must say though that I have never seen an American NES unit in operation, I have only seen a PAL NES as well as Japanese Famicoms, Twin Famicoms and AV Famicoms; so MAYBE the American NES produces hues like the PAL NES, though I think that's extremely unlikely, as the PPU is the same.
By the way, you wrote that you'd like to emulate the output for a variety of displays; that would mean you'd also have to adjust to different phosphor types, effectively necessitating complete color management, Photoshop-like. Basically what you'd want is a general conversion algorithm for RGB values between sets of given phosphor chromaticities. I've written a couple of C code lines for this, tell me if you need anything.
NewRisingSun wrote:
Both 100% red and 100% blue are completly legal NTSC colors, in fact, the 0.877 and 0.493 coeffecients were deliberately chosen so that 100% red and 100% blue, which do occur in nature, WOULD be legal. 100% yellow is "illegal" in NTSC because the upper peak of the modulation exceeds 120 IRE (reaches about 133 IRE).
By the way, there's one more systematic difference between Japanese and "real" (American) NTSC: the NTSC standard calls for a 7.5% setup, Japanese devices however use 0%. I think you can even select between these two on some camcorders...
Yeah, I remember reading somewhere about the difference in black levels between standard NTSC and "NTSC-J".
I was wondering a while ago exactly where the 0.877 and 0.493 coefficients in the YUV matrix came from. I was able to work out that their ratio determined the angle of G-Y, and presumably was chosen so that G-Y would be more or less equidistant from R-Y and B-Y (it's actually just a bit closer to B-Y)
NewRisingSun wrote:
By the way, you wrote that you'd like to emulate the output for a variety of displays; that would mean you'd also have to adjust to different phosphor types, effectively necessitating complete color management, Photoshop-like. Basically what you'd want is a general conversion algorithm for RGB values between sets of given phosphor chromaticities. I've written a couple of C code lines for this, tell me if you need anything.
It would be great if you could point me to a general formula for converting between color temperatures. The page you linked to has the following matrices for converting between NTSC and EBU (=PAL?) colors:
Code:
[Rntsc] [0.6984 0.2388 0.0319][Rebu]
[Gntsc] = [0.0193 1.0727 -0.0596][Gebu]
[Bntsc] [0.0169 0.0525 0.8450][Bebu]
[Rebu] [ 1.4425 -0.3173 -0.0769][Rntsc]
[Gebu] = [-0.0275 0.9350 0.0670][Gntsc]
[Bebu] [-0.0272 -0.0518 1.1081][Bntsc]
Are these color temperature conversions, or something else? (And are they even correct, or are some of the signs reversed again?) The page mentions that NTSC and EBU have different "white points", which I understand is related to color temperature (but am likely completely wrong)
As you've probably guessed by now, my knowledge of this stuff is pretty fragmentary and mostly gleaned from a lot of Googling and doing math by hand. My background is in biology, not color science...
AWJ wrote:
I was wondering a while ago exactly where the 0.877 and 0.493 coefficients in the YUV matrix came from.
Well, the low peak of the video signal must not go below a level of -0.33:
Luma - Chroma > -0.33
The nature of quadrature modulation means we can write it like this:
Y - sqrt (V² + U²) > -0.33
For R=1.0 or B=1.0, this would be violated if we just used V=R-Y and U=B-Y. Therefore, we need to scale them in such a way that R=1.0 and B=1.0 are just within range.
Let coR, coG, coB be given (they'd be 0.299, 0.587 and 0.114 in current implementations), and scR, scB the scale factors for R-Y,B-Y that we're looking for. The following must be fulfilled simultaneously (sorry, don't know how to do this in HTML):
The first line is for R=1.0, which means Y=coR, R-Y=1-coR, B-Y=0-coR. The second line is for B=1.0, you get the idea.
Have a lot of fun working out the scale factors from that.
Quote:
It would be great if you could point me to a general formula for converting between color temperatures. The page you linked to has the following matrices for converting between NTSC and EBU (=PAL?) colors:
The general algorithm would be to convert from (linear!) RGB values for the source phosphor to CIE XYZ values, and then from those to linear RGB values for the target phosphor. Setting up RGB<->XYZ matrices from a given set of chromaticities is somewhat involved and best shown using C code, not a formula.
Quote:
Are these color temperature conversions, or something else? (And are they even correct, or are some of the signs reversed again?) The page mentions that NTSC and EBU have different "white points", which I understand is related to color temperature (but am likely completely wrong)
Color temperature means white point. These equations are basically right, however meaningless since those "NTSC primaries" this page and other people keep talking about are completely obsolete (no real non-ancient NTSC TV uses them); unless you're trying to emulate the look of a
1954 RCA CT-100 TV set, you'd use "real world" primaries from actual monitor profiles, not these theoretical NTSC/SMPTE-C/EBU values.
I've been trying to come up with a NES palette of my own for the longest time. I tried using the vid capture method to grab colors (or more than one card) but the results of that are way off. I'd tried Kevin Horton's generator but could never get satisfactory results. I just found your (AWJ) method a few days ago, and it's better than Kevin's but still doesn't do it for me
What I ended up doing a while back is color calibrating my TV and monitor as best I could and just winging my own pallete by eyeballing it from the NES playing on the TV. I'm still not %100 pleased with it, but I think it's pretty good. Like NewRisingSun said, I don't think there is a perfect hard coded palette.
My results from my palette are interesting though, the hue shifts are quite uneven and so is the saturation, but to a lesser extent. I also noticed there is quite a difference between the color from RF output and composite output. The composite colors seem to be distributed better than RF (no surprise) but I actually found the RF colors to be more fitting and pleasing to the eye in most cases. BTW, I'm using an NTSC TV/NES.
I am very interested in what you guys come up with here
NewRisingSun, this is a somewhat strange question, but it seems you may know or be able to find this out. The TV I'm using for my NES and other older consoles is a 27" RCA model# 27F630T. I'm wondering if you know how to enter the service mode on this? I've looked elsewhere on the net but have never found a combo that works. My screen geometry is messed up, it's rotated I would guess about 5 - 10 degrees. Very apparent with computer generated images.
I don't know anything about current RCA sets --- they're not even sold here in Europe.
NewRisingSun wrote:
I don't know anything about current RCA sets --- they're not even sold here in Europe.
Are there any sets sold under the name THOMSON? Those are made by the same company as current RCA or current GE sets.
NewRisingSun wrote:
R-Y: 112° ´ 0.83, G-Y: 252° ´ 0.3 (B-Y: 0° ´ 1)
This would translate into the following decoder matrix:
R = Y + 1.539*V - 0.622*U
G = Y - 0.571*V - 0.185*U
B = Y + 0.000*V + 2.000*U
How did you get those values from those angles ? When I don't understand something, I just do trial-and-error, and the only one I could find was tan(252)/2=1.539
R = Y + sin(112°)*0.83*2 + cos(112°)*0.83*2
G = Y + sin(252°)*0.3*2 + cos(252°)*0.3*2
B = Y + sin(0°)*1*2 + cos(0°)*1*2