Very interesting stuff! I'll be interested to see what you find, and the difference game mode would make on a LCD on HDMI.

It would be interesting to make a game that just displays some bars when you hold down the A button -- an NES version of this: http://www.displaylag.com/the-lag...

That was my original idea but the problem is a software emulator like the Retron 5 that won't let you run any game you want. Using a simple/common/licensed game like Balloon Fight guarantees I can run the same test on any system.

I might have to do the white bars when I switch to an optical sensor instead of using the camera anyways.

This is interesting. Wish I could do this.

I've been wanting to write some sort of NES "Hello, World"; maybe it will be "Super Press A For White Bars".

Very cool project.

I am actually surprised some of the LCD display latencies are so low.

It would probably be worth mentioning the brand, since it is reasonable to believe that no two tv manufacturers are using exactly the same signal conversion hardware, and extrapolating these results to all brands may paint a rosier picture than reality.

Then again, if your point us to show the best case frame difference between HDMI and composite for marketing the hdmi NES I think you have succeeded.

I agree with Arch, input lag can vary wildly among manufacturers, models of HDTV sold by a manufacturer, and even within the same model of TV. The input lag comparision is interesting, but isn't that useful outside the exact type of HDTV you're testing on Bunny.

I'll third that. I didn't even realize it a few years ago when I thought Nintendo coded up a POS lazy lagged and unplayable SMAS for Wii, ended up being the stupid TV hating on sub-HD system stuff. Taking it to a Panasonic I have it ran as well as my legit cart so I kept a second copy I got. There's a website out there I don't have bookmarked where people are trying to stir up a community effort of as many makes, models, and variants of LCD, LED, OLED and Plamsa tvs and assessing the input lag so people will know what to buy or not if they intend to use older sub-HD devices so they don't run as useless lagged up crap.

I fear the day my Panasonic dies as it's solid, even displays old games with a sharp clean image that basically fills the screen.

Yup I will test other TVs and make a cool chart, but the main focus is the console/emulator so using the same display is important. At least one of my screens is on displaylag.com so I can see how it compares.

Fascinating stuff! I'm always impressed with bunnyboy's ability to engineer solutions, and the fact that he's considered stuff like using a common licensed game is brilliant in its own right.

I wonder if a CRT HDMI would make a difference. My PS3 using HDMI to my CRT has noticably less input delay vs. HDMI to LCD using that rocksmith game.

What CRT do you have with HDMI input?

My TV is a panasonic CT34WX15N..  It doesn't do full 1080p though only 1080i and 720p, which apparently isn't the right format. but I assume there are others that do 1080p?  The difference in input delay was significant enough that when I moved my PS3 from the CRT to the LCD, I couldn't play rocksmith at all anymore and I had to move it back.

Like all HDMI CRTs I have seen, it doesn't do 720p.  Either 1080i or 480p only.  Should be much less lag because it is only doing digital->analog conversion instead of any kind of processing or upscaling.  At most it would have a couple scanline buffer to do the pixel doubling on 480p.

Again composite is significantly slower than HDMI in all cases.  Turning on game mode saves 10mS which is significant but still slower.  In HDMI game mode only saves ~2mS.  Labeling the input as PC ends up the same as game mode (within the error margin of the tests).

Next is Nestopia connected to the display above through HDMI.  The USB hardware can add 0-12mS just for polling the controller, so results will vary wildly.  I got 52mS and 68mS, so the usb/emulator/buffers/output adds around 2+ frames making it as slow as composite.  Will do more tests later with options like vsync.  I doubt filters or scaling will add any measurable delay.

What I need is a pro player to come over and see which of the delays is noticable, and which is instant death  

I've got a Sony KV-34XBR800 that has DVI in which I use an adapter from HDMI, which does support 720p I believe. It would be interesting to see if there is lower lag in this versus an LCD.

Interesting, the spec sheet says that one converts 720p to 1080i. Want to mail it here?

Looks like there's a CRT on ebay that does 720p, but it is 5 hours away....

Display lag doesn't have my tv on their site, looked at the back it's a Panasonic Viera model TC-26LX70 (dated Nov 2007 so I must have got it in 08.)

I'm not sure how people are rating this but newegg and crutchfield claims it has a 9ms response time. It's a 720p(or 1080i) tv and runs at 60hz. I do know it's not as bright as the LED tv but the way I tweaked it it's fairly close and the image is super sharp on it.

I've been considering doing something like this myself, but with a slightly different approach to keep the results more controllable. In short, a special ROM that changes the color of the whole screen from black to white when a button is pressed, and then a controller as well as a light sensor connected to an oscilloscope (or other device capable of measuring time between trigger pulses.)

There are four interesting data points, imo.
1) The time a button is pressed.
2) The time when the button press is registered. (This would depend on the poll rate of the controller device.)
3) The time of the earliest display change.
4) The time of the last display change.

The point of separating 1 and 2 is isolating one aspect of the measurement as "poll to output time" instead of "input to output time" to make the technical part of the measurement more predictable. You now can now isolate the press to poll time as a random term between 0 and x where x is the time between polls.

The point of 3 and 4 is again more predictability. 3 and 4 would in practice correspond to the top and bottom of the screen a classical CRT will draw the screen progressively, meaning that you might hit the jackpot and get a <1 ms input to output latency if you press the button at the right time, and look at the top of the screen. The bottom of the screen on the other hand is guaranteed a minimum latency of 16.7 ms. (Assuming 60 Hz and checking the controller once in each VBlank only.)

An LCD on the other hand might have a different strategy where the data is drawn to the screen faster than 16.7 ms, and only a smaller portion of the time of a frame is used for drawing. This would make the difference for the best and worse case scenario for the same frame (ie the top and the bottom of the screen) smaller. I don't know if any TVs/screens actually do this, but it's worth keeping in mind.

The result that composite was slower than HDMI is not surprising, considering it will likely double or triple buffer the screen contents before drawing it. The screen drawing might not even be be synched to the video signal for composite on a digital TV, meaning that your latency might vary slightly depending on the phase relationship between the signal and the composite decoder at the time.

Yet another issue is the jitter introduced by the timing of the controller polling. Even more ideally you would add yet another data point, an "end of frame" in the emulator, which could be signalled in some good way such as perhaps outputting it on one of the pins on an LPT port, or recording the a high precision timestamp, assuming that this could be correlated well with the rest of the rig, again to isolate another individual step in the process.

I hope I've given some good ideas. I want to do something like this myself, but I haven't managed had the time and motivation to set it up yet.

Response time is different from input lag. Response is the time for a pixel to change from black to white (or a similar measurement). 9mS would be considered slow now, 6mS is common and 2mS is fast. The input lag testers need the pixel to change so that response time is included in the overall number on sites like displaylag.com.

That was my original plan, but I couldn't find a light sensor that would work.  Did you ever get to that point?  I wonder how much the brightness matters for sensors like the bodnar.  I am ending the test when I see the first sign of the new graphics which still might be too dark for it to register.  That could mean a 5mS difference between my camera results and an optical sensor.

Mine is already starting at the poll, but another LED on the button itself was the next step.  I think that is only significant for emulators where the hardware poll does not align with the NES game poll.  It would be awesome to measure a tetris player to see how much time they have between button press and button poll!

I am already picking a consistent point on screen, so that shouldn't make a difference other than a constant add or subtract from the final number.  The lcd/plasma panel won't be able to draw any sooner simply because the NES/emulator won't have output the video data yet.

I think displaylag.com had more info about how they average top/bottom/middle to account for different drawing routines but I'm not sure that is significant enough to care about. 

I don't think LPT port is low latency enough, there are just too many layers of software and hardware in the way.   Not sure of anything else that would be faster without some extreme hardware tho.  Are there any USB 3.0 keyboards to use the num lock light?  

You didn't list what your using with those inputs, but I'm assuming you're using a stock NES for composite and your HDMI NES for the HDMI input. If that's the case, the higher composite lag would be due to the TV applying de-interlacing to the 240p signal to convert it to 480p.

240p isn't interlaced, or it would be 240i. The screen in the first display is 1080p, so both the HDMI (720p) and composite (240p) need to be upscaled. I would have assumed those would take similar time and the analog->digital conversion would be very fast, but apparently not!

I'm assuming you've created a custom NES software to test the inputs easier, is there any chance that your software may also have some lag? I know some games have input lag, although I'm hard pressed for examples right now.

Excellent stuff though man, CRT Composite is the way to go for competitive NES gameplay, thanks for that tidbit.