Broken FDS emulation: Kiki Kaikai - Dotou Hen

Broken FDS emulation: Kiki Kaikai - Dotou Hen
by on (#63043)
Hi everyone. I began to write a guide on how to play this FDS port of the Taito arcade game Kiki Kaikai (the series that came to be known as Pocky & Rocky on the SNES), but I have come across a strange problem. There are two boss fights that crash just about every modern NES emulator that I have tried, including FCEUX, Nestopia, Nintendulator, Mednafen, and VirtuaNES. The _only_ emulator that the game works properly on is the "old" emulator NNNesterJ. It's very strange, where those five other emulators hang, NNNesterJ goes swimmingly right along. (Interesting to note, for those of you who can read Japanese, a warning to this effect is present on this page: http://www25.atwiki.jp/famicomall/pages/244.html which you can run through Google Translator if need be.)

It's not too hard to get to the crash point, although it takes a few minutes of walking (if you avoid all of the enemies.) The ROM that I'm using (and the only one that seems to be available) is from the TOSEC set with a CRC of 4C791779. From the start point, walk straight up. Keep following the path north until you get the first chance to travel east (you'll cross a few bridges just before that point.) Head east as soon as you are able, and then take the very next path south. This will lead you down and around to the entrance of a cave. You need 50 Ofuda cards to enter the cave, which is what you start with (you earn Ofuda cards by defeating enemies with your wand). Once inside, the path to the boss never branches. You travel north, then east, and the north. At the top of the second north path is the boss room. The expected result (and the one which NNNesterJ produces) is that the room scrolls into view, turns blue, and a large boss starts to attack you. The actual result in the rest of the emulators I've tried is that they hang before the room turns blue, stuck playing one note.

I know this is the sort of thing that is incredibly low priority in the grand scheme of things, but I thought it might be interesting for some of the more active emulator developers to look at and try to figure out what NNNesterJ is doing differently (correctly or perhaps incorrectly) that allows the game to play in this one particular emulator. I would love to know myself. Thanks for taking a look!

Procyon

by on (#81527)
no to raise the dead but thats interesting. just tried this game out and i wonder what the problem could be....

by on (#81544)
Does it actually crash the emulator into closing itself, or does the game just freeze?

by on (#81545)
Drag wrote:
Does it actually crash the emulator into closing itself, or does the game just freeze?


the emulator neither freezes nor crashes, the game hangs when you enter the boss's room. Weird how nesterj could play this right but no others...seems like something blocks the boss from appearing or something...

Give it a try in nestopia, when you reach one of the boss's rooms it just kinda hangs, and nothing happens, the music freezes on one note, i used rewind to leave the room, but i know the emu does not crash.

by on (#81546)
Illegal opcodes maybe? Maybe one not documented very well? Weird thing. And you can't track it down?

by on (#81547)
3gengames wrote:
Illegal opcodes maybe? Maybe one not documented very well? Weird thing. And you can't track it down?


i have no clue whats causing it, i always thought fds was pretty much 100 percent working and great. i came across this thread yesterday and since then it has peaked my interest. comparing code from NNNnesterj to nestopia is hard to do lol, :shock:


***update. My mistake the game does in fact crash.

by on (#81552)
The linked page indeed said that VirtuaNES reported an illegal opcode error.
Maybe someone who have time would try to trace the code there with a debugger enabled emulator (Nintendulator, FCE*whatever* or even No$NES).

by on (#81562)
I walked to the 1st boss with Nestopia debug mode and it throws illegal opcodes BRK and JAM causing CPU JAM! At titlescreen throws Expression: modulator.writing with Fds::Sound::WriteReg7.

by on (#81563)
plasturion wrote:
I walked to the 1st boss with Nestopia debug mode and it throws illegal opcodes BRK and JAM causing CPU JAM! At titlescreen throws Expression: modulator.writing with Fds::Sound::WriteReg7.


so we need to erase/modify the illegal opcodes then... so it threw an error with sound, writereg7...hmm.

by on (#81564)
I think maybe something cause that this opcodes appears. So best way is try to find out if WriteReg7 don't make any mess.

Let's see what is doing - Nestopia NstFds.cpp

Code:
      NES_POKE_D(Fds,4088)
      {
         sound.WriteReg7( data );
      }
      
      void Fds::Sound::WriteReg7(uint data)
      {
         NST_VERIFY( modulator.writing );

         if (modulator.writing)
         {
            Update();
            std::memmove( modulator.table, modulator.table + 1, Modulator::SIZE-1 );
            modulator.table[Modulator::SIZE-1] = Modulator::steps[data & REG8_MOD_DATA];
         }
      }


and NNNesterJ 0.23 in fdssnd.c
Code:
static void FDSSoundWrite(uint32 address, uint8 value)
{
   if (0x4040 <= address && address <= 0x407F)
   {
      apu->fdssound.op[1].wave[address - 0x4040] = LinearToLog(((int32)value & 0x3f) - 0x20);
   }
   else if (0x4080 <= address && address <= 0x408F)
   {
      int ch = (address < 0x4084);
      FDS_FMOP *pop = &apu->fdssound.op[ch];
      apu->fdssound.reg[address - 0x4080] = value;
      switch (address & 15)
      {
...
         case 8:
            {
               static int8 lfotbl[8] = { 0,1,2,3,-4,-3,-2,-1 };
               uint32 v = LinearToLog(lfotbl[value & 7]);
               apu->fdssound.op[0].wave[apu->fdssound.waveaddr++] = v;
               apu->fdssound.op[0].wave[apu->fdssound.waveaddr++] = v;
               if (apu->fdssound.waveaddr == 0x40)
               {
                  apu->fdssound.waveaddr = 0;
               }
            }
            break;

that's only suggest, I'm not sure it can be somehow connected with this issue, however the soundtrack is changing while the boss appears. It takes only last hex digit of address, so we're looking case 8?

Damn... I need to register if I want using VC++ express. The time trial has expired. Maybe this time someone else wants to have fun ;).

by on (#81565)
Is it running code from RAM? Maybe your CPU is causing something in RAM to be changed wrong and crash?

by on (#81566)
3gengames wrote:
Is it running code from RAM? Maybe your CPU is causing something in RAM to be changed wrong and crash?


whatever it is, 5 diff emus are all proving to do the same thing including nestopia, virtuanes and others....

by on (#81576)
Perhaps the image really seriously is broken (bad "dump", I guess), and the fact that nesterj allowed the game to run was due to a bug?

It could also be the result of copy protection of some kind; I know disk-based c64 software had some hardware-implemented copy protections, such as purposefully unreadable sectors.

by on (#81578)
I've checked where fce ultra debug tool stuck when the boss seems to appear and...
Code:
00:803B:03        UNDEFINED
00:803C:01 03     ORA ($03,X) @ $02F3 = #$72
00:803E:61 62     ADC ($62,X) @ $0000 = #$D3
00:8040:F0 00     BEQ $8042
00:8042:00        BRK
00:8043:00        BRK
00:8044:92        UNDEFINED // Here debug trace stuck
00:8045:24 49     BIT $0049 = #$00
00:8047:92        UNDEFINED
00:8048:24 49     BIT $0049 = #$00
00:804A:DB        UNDEFINED
00:804B:B6 6D     LDX $006D,Y @ $0080 = #$FE
00:804D:DB        UNDEFINED
00:804E:B6 6D     LDX $006D,Y @ $0080 = #$FE
00:8050:04        UNDEFINED

looks like this is not so good place to code execute.

by on (#81581)
So looks like something needs to be added or modified...maybe an exception to address while boss appears and music changing. Isn't fce issue the same as nestopia?

by on (#81585)
I have no Idea where is the problem, i think somewhere else, that part asm code are located in few places, so we should trace the code little bit earlier. I don't think is this bad dump, no way that NNNesterJ can emulate bad dumped fds image, but how - it skips this op code? So if I put NOP (EA) to UNDEFINED and BRK it may help?

...and yes it looks the same to Nestopia, FCE Ultra or even NesDS.

by on (#81586)
Yea I would start by giving that a try. Who knows what oddities this may fix if any. Could be the issue. :). Ill be back in town on monday and ill be messing with this too. Let us know soon plasturion. :)

by on (#81598)
You should try making a CPU log of the instructions *leading* to that place. Maybe make a movie until the point it crashes, then seek to a couple of frames before the crash and start logging. My money is on a bad dump.

by on (#81601)
Code:
$8F18:A2 01     LDX #$01                   A:04 X:02 Y:00 S:FE P:nvUBdizC
$8F1A:E4 5D     CPX $005D = #$02           A:04 X:01 Y:00 S:FE P:nvUBdizC
$8F1C:B0 29     BCS $8F47                  A:04 X:01 Y:00 S:FE P:NvUBdizc
$8F1E:BD D8 04  LDA $04D8,X @ $04D9 = #$13 A:04 X:01 Y:00 S:FE P:NvUBdizc
$8F21:30 21     BMI $8F44                  A:13 X:01 Y:00 S:FE P:nvUBdizc
$8F23:A8        TAY                        A:13 X:01 Y:00 S:FE P:nvUBdizc
$8F24:BD 0E 05  LDA $050E,X @ $050F = #$FF A:13 X:01 Y:13 S:FE P:nvUBdizc
$8F27:C9 FE     CMP #$FE                   A:FF X:01 Y:13 S:FE P:NvUBdizc
$8F29:F0 39     BEQ $8F64                  A:FF X:01 Y:13 S:FE P:nvUBdizC
$8F2B:C9 FD     CMP #$FD                   A:FF X:01 Y:13 S:FE P:nvUBdizC
$8F2D:F0 1C     BEQ $8F4B                  A:FF X:01 Y:13 S:FE P:nvUBdizC
$8F2F:C9 F7     CMP #$F7                   A:FF X:01 Y:13 S:FE P:nvUBdizC
$8F31:F0 1E     BEQ $8F51                  A:FF X:01 Y:13 S:FE P:nvUBdizC
$8F33:B9 40 DB  LDA $DB40,Y @ $DB53 = #$64 A:FF X:01 Y:13 S:FE P:nvUBdizC
$8F36:85 B0     STA $00B0 = #$56           A:64 X:01 Y:13 S:FE P:nvUBdizC
$8F38:B9 78 DB  LDA $DB78,Y @ $DB8B = #$73 A:64 X:01 Y:13 S:FE P:nvUBdizC
$8F3B:85 B1     STA $00B1 = #$67           A:73 X:01 Y:13 S:FE P:nvUBdizC
$8F3D:86 AB     STX $00AB = #$04           A:73 X:01 Y:13 S:FE P:nvUBdizC
$8F3F:20 48 8F  JSR $8F48                  A:73 X:01 Y:13 S:FE P:nvUBdizC
$8F48:6C B0 00  JMP ($00B0) = $7364        A:73 X:01 Y:13 S:FC P:nvUBdizC
$7364:7F        UNDEFINED                  A:73 X:01 Y:13 S:FC P:nvUBdizC
$7367:FF        UNDEFINED                  A:47 X:01 Y:13 S:FC P:nvUBdizC
$736A:18        CLC                        A:46 X:01 Y:13 S:FC P:nvUBdizC
$736B:20 40 80  JSR $8040                  A:46 X:01 Y:13 S:FC P:nvUBdizc
$8040:F0 00     BEQ $8042                  A:46 X:01 Y:13 S:FA P:nvUBdizc
$8042:00        BRK

This is log and the clue why the code is going outside is instruction $8F48:6C B0 00 JMP ($00B0) = $7364, and this is good. I noticed is needed while 2 sprites or more are displayed. Accumulator and $00B0 register value are deciding, so looks like this is some calculating issue.

[Here] is little more of them.

Maybe loading from disk to ram in most common emus are not perfect, so at $7364 should be some code loaded from the disk, and maybe only NNNesterJ can do it. In that case it would be nice to compare logs from NNNesterJ, but it hasn't debugger/logger.

Someone said before that can be something with CPU so in that case this is bad dump and illegal changing something in RAM. Bad working CPU in NNNesterJ don't change something important in RAM, so "bad" instruction is omitted and then works fine. Sounds fun.

Any other ideas?

by on (#81615)
Is it possible to omit the instruction? Or maybe add the exception?

by on (#81616)
I should said "Maybe", I just suggest loud what it can be and I think is rather impossible what i wrote, I still don't know where and what is wrong (emu, dump, CPU, FDS), maybe others can find it.

Hey, I didn't check it wisely, NNNesterJ has some tools after all like memory viewer, it should be enough to compare with logs. lets see....

Heres the memory page of FCEUX where it try to code execute but there's nothing (too much of 00 and FF, maybe data from disk wasn't transfered to this area?)
Image

------------
and the 2nd one - correct one of NNNesterJ - nice and fluffy...
Image

by on (#81625)
edit*

by on (#81627)
What's the byte before it jumps to the crap code?

by on (#81634)
3gengames wrote:
What's the byte before it jumps to the crap code?


U on to something 3gen? :)

by on (#81635)
Just wondering if the byte before it could could make it seem like code, looks trashed from what you gave us though.

by on (#81638)
Well, it does indeed seem like NNNesterJ has some valid code where other emulators don't. Here's short snippet of disassembly from NNNesterJ's $7364 (ignore the label):
Code:
L0027           := $0027
        lda     $050E,x
        cmp     #$FF
        bne     L0027
        lda     #$20
        sta     $072A,x


I don't know anything about FDS though to speculate why this could happen.

by on (#81639)
Since it's RAM, log where it changes that piece of RAM I guess?

by on (#81640)
3gengames wrote:
What's the byte before it jumps to the crap code?

You can check it in logs that i've posted before
Code:
 $8F3F:20 48 8F  JSR $8F48                  A:73 X:01 Y:13 S:FE P:nvUBdizC
 $8F48:6C B0 00  JMP ($00B0) = $7364        A:73 X:01 Y:13 S:FC P:nvUBdizC

So the answer is 8F, is it helpful?
I think this part is correct (about instruction) is using many times and if it don't game should crashed at the start.
I bet that's because of FDS wrong emulation (maybe).

... I'll try to find logs when RAM is changing...

at first I checked how it looks loading 2nd stage of Kiki Kaikai and give some screens of NNNesterJ.
Image
0 - its old data
1 - after 1st procedure - loaded some data that filled all $7300 - $7400
2 - after 2nd loading - it ends exactly on $7363
3 - after 3rd loading - starts from $7364, and this stage not happen in FCEUX, so there are old data from 1st procedure.

I can send you some logs from FCEUX when it ending 2nd procedure, and some more until the game starts rendering (loading is over)... but I think it's useless.
[Here] is the big one 320M log compressed to 1.5M
It takes some time (few seconds) when 2nd loading procedure ends and starts 3rd.

by on (#81710)
Nice work plasturion! Seems like everytime ya think ya got the nes emus figured out ya find another snag. Lol. Looks like the person who coded that was in the know! :)

by on (#81733)
I found maybe another step to make sure is this really FDS issue.
Let's talk about 2 other previous Nester releases that runs FDS:

1. unofficial release of original Nester emu (public beta4) made by Toshiya Takeda. There was the first additional FDS feature that runs Kiki KaiKai correctly. FDS support is inside Mapper/020.cpp

2. NesterJ emu by Mikami Kana, which is slightly different to T.Takeda FDS version and this change exist utill NesterJ 0.503 as Nes_Mapper_FDS.cpp. With this version FDS works bad (Kiki KaiKai hangs when you meet the boss, exactly the same how all other popular emus: Nestopia, FCE, Virtuanes, etc), then he for some reason put the files - Mapper/020.cpp of T.Takeda (or Takeda takes Mikami files to enhance previous release, so it could be Mikami work (didn't checked)) with NesterJ 0.51 and Kiki KaiKai works here good again.


Looks like is it FDS, isn't it? And because of that both of files are very similar, comparing this files maybe you can simply find something interesting.

by on (#81737)
i totally agree thats its most likely an fds issue nothing more. im about to start comparing nestopia and the correct NNNnesterj. hard to compare but its certainly doable and fixable., and there is ALWAYS a solution to every problem haha. 8)

by on (#81758)
I think you can be sure more or less, about difference 0.503 and 0.51 of NesterJ is somehow possible to check is it only FDS issue very simple, without any knowledge of FDS. Just rename 020.cpp to Nes_mapper_FDS.cpp and replace it with compiling NesterJ v0.503. If Kiki Kaikai works you can said you are 95% certain that's only FDS. However It needs VC++ 6.0, and is not friendly to compile.

Comparing only 020.cpp and NES_mapper_FDS.cpp, maybe you can also reduce possibilities of what's wrong with another thing (much of them are unchanged), and then you can compare with your emu source code.

by on (#81759)
plasturion wrote:
I found maybe another step to make sure is this really FDS issue.
Let's talk about 2 other previous Nester releases that runs FDS:

Can you post these two files (020.cpp that works correctly, and Nes_Mapper_FDS.cpp that doesn't work correctly) somewhere?

by on (#81761)
[Here] - (good) and (bad) NesterJ FDS files
I have a bad news. I compiled NesterJ 0.503 with (correct) 020.cpp and it wont help. So the problem can be somewhere else afterall.

[Here] is site about Kiki Kaikai games with some screenshots and graphics.
Pretty nice shooter! I like those snes version.

by on (#81778)
So im assuming once the issue is figured out. Most likely it's either fds.cpp or mapper 20.cpp file in nestopia I think. : )

Perhaps it fix other various oddities in nestopia fds games.

by on (#81780)
It's kinda fun, you allways discover something new, i noticed that FDS support is also inside snss.cpp. So at first I'll try to make it similar like 0.51.
...it's done and nothing happen, ok i give up now, let's hear other oddities.

by on (#81790)
plasturion wrote:
It's kinda fun, you allways discover something new, i noticed that FDS support is also inside snss.cpp. So at first I'll try to make it similar like 0.51.
...it's done and nothing happen, ok i give up now, let's hear other oddities.


you mean you added the 2 together to compile for nester? or you added to nestopia?

oddities...hmmm ill have some soon :) LOL

by on (#81792)
I working only with both version of NesterJ 0.51 and 0.503 to find some clue of correct FDS emulate.

For now I can make only three things to do:
- make NesterJ 0.51 the same problem like FCEUX putting (bad) 0.503 files
- brick NesterJ 0.51 somehow to works bad like FCEUX.
- fix NesterJ 0.503 giving, modify (good) 0.51 files (snss.cpp, 020.cpp, maybe other)

If you made only one of that, it would be nice.
There's no sense trying to put this to Nestopia if you don't know what, but good luck if you find something, let me and others know.

by on (#81794)
plasturion wrote:
I working only with both version of NesterJ 0.51 and 0.503 to find some clue of correct FDS emulate.

For now I can make only three things to do:
- make NesterJ 0.51 the same problem like FCEUX putting (bad) 0.503 files
- brick NesterJ 0.51 somehow to works bad like FCEUX.
- fix NesterJ 0.503 giving, modify (good) 0.51 files (snss.cpp, 020.cpp, maybe other)

If you made only one of that, it would be nice.
There's no sense trying to put this to Nestopia if you don't know what, but good luck if you find something, let me and others know.


gotcha. :) im looking at all the sources too. ill update as things progress as well my friend. :) ive also been taking classes since im not the best at this stuff. im using other code as ways to help learn how things link together and reading up on nes documents and what fds sources i can find.

by on (#81872)
I think i've got it...

I've removed some code of NesterJ 0.51 and now works incorrect like FCEUX (and all other emus), and fixed 0.503 adding that new little part of code removed from 0.51.
In that case now we can be certain, that only this part of code is responsible for correct work of Kiki Kaikai. (look at Nes_rom.cpp)

What do you think about this?
Code:
   if(image_type == 1)
   {
      screen_mode = 1;
      mapper = 20;

      fds = (ROM_banks[0x1f] << 24) | (ROM_banks[0x20] << 16) |
         (ROM_banks[0x21] <<  8) | (ROM_banks[0x22] <<  0);

       for(i = 0; i < ROM_banks[4]; i++)
       {
         uint8 file_num = 0;
         uint32 pt = 16+65500*i+0x3a;
         while(ROM_banks[pt] == 0x03)
         {
            pt += 0x0d;
            pt += ROM_banks[pt] + ROM_banks[pt+1] * 256 + 4;
            file_num++;
         }
         ROM_banks[16+65500*i+0x39] = file_num;
      }
   }

by on (#81887)
Great work plasturion! ;). This might be of huge help to other emus that want to make the kiki update. Lol. U deserve a round of brew lol. :lol:

Seriously though that's awesome that something new is always to be learned.

Since you have become the nestopia saviour would it be easier to put this in or is that a diff thing from the others? So u got fceultra to run it right? I just wanted to make sure that's what u were saying. I myself have been studying the Nes ppu and the nestopia ppu code. This emu is pretty advanced coding work.

by on (#81889)
Wait, are you guys just trying to get the game to run or actually looking for the emulation inaccuracy that prevents it from running?

by on (#81893)
*Spitfire_NES* wrote:
So u got fceultra to run it right? I just wanted to make sure that's what u were saying. I myself have been studying the Nes ppu and the nestopia ppu code. This emu is pretty advanced coding work.

like fceu means like all other emus... I compiled only NesterJ 0.51 to make works like 0.503, and 0.503 to works like 0.51, only to make ensure where's the problem.

Quote:
Wait, are you guys just trying to get the game to run or actually looking for the emulation inaccuracy that prevents it from running?

And that's a good question. But what can be inaccurate? Emulation of NesterJ or all the other emus, so maybe there's something in known FDS docs, that can be incorrect, or something missing? Or this code might be helpful only to emulate Kiki, but is wrong and it can broke other FDS games? We need an expert opinion.

...about trying to put this to Nestopia, maybe now it's a good idea... It has high compatibility and we need some test.

by on (#81901)
plasturion wrote:
What do you think about this?
Code:
   if(image_type == 1)
   {
      screen_mode = 1;
      mapper = 20;

      fds = (ROM_banks[0x1f] << 24) | (ROM_banks[0x20] << 16) |
         (ROM_banks[0x21] <<  8) | (ROM_banks[0x22] <<  0);

       for(i = 0; i < ROM_banks[4]; i++)
       {
         uint8 file_num = 0;
         uint32 pt = 16+65500*i+0x3a;
         while(ROM_banks[pt] == 0x03)
         {
            pt += 0x0d;
            pt += ROM_banks[pt] + ROM_banks[pt+1] * 256 + 4;
            file_num++;
         }
         ROM_banks[16+65500*i+0x39] = file_num;
      }
   }

As far as I can tell, this code scans through all disk sides, calculates number of files on that side, and then patches the file amount block with the correct number. What would this accomplish, I don't know. Seems like a hack, since FDS can't really be doing the same.

by on (#81902)
This "throw chunks of mystery code around and see if they stick" mentality to fixing emulation problems is kinda disappointing.

*Spitfire_NES* wrote:
I wonder what that all means....lol :p

Your enthusiasm is commendable, but...if you don't actually know what the code is doing, perhaps you shouldn't be trying to "fix" the problem, lest you introduce even more somewhere down the road.

by on (#81904)
I tried manually patching the correct amount of files in the file amount block (wrote a small Python tool to calculate the actual number of files), and guess what, now it works:
Code:
Comparing files Kiki Kaikai - Dotou Hen (1987)(Taito Corp.).fds and KIKI KAIKAI
- DOTOU HEN (1987)(TAITO CORP.) - PATCHED.FDS
00000049: 08 09
00010025: 0C 0D

The original question of course still stands... is this just a bad disk or is there something that can be done to improve FDS emulation. Patching the number of files on load time doesn't seem like the Right Thing(TM) to do. :)

by on (#81906)
Weird. Was that some form of copy protection back in the day (specifying the wrong number of files so disk copying tools would fail to copy everything)? Does an actual FDS even look at that value?

by on (#81907)
BMF54123 wrote:
This "throw chunks of mystery code around and see if they stick" mentality to fixing emulation problems is kinda disappointing.

*Spitfire_NES* wrote:
I wonder what that all means....lol :p

Your enthusiasm is commendable, but...if you don't actually know what the code is doing, perhaps you shouldn't be trying to "fix" the problem, lest you introduce even more somewhere down the road.


I thought they were trying to pinpoint the problematic code.

Hmmm weird thing...

by on (#81919)
thefox wrote:
I tried manually patching the correct amount of files in the file amount block (wrote a small Python tool to calculate the actual number of files), and guess what, now it works:
Code:
Comparing files Kiki Kaikai - Dotou Hen (1987)(Taito Corp.).fds and KIKI KAIKAI
- DOTOU HEN (1987)(TAITO CORP.) - PATCHED.FDS
00000049: 08 09
00010025: 0C 0D

The original question of course still stands... is this just a bad disk or is there something that can be done to improve FDS emulation. Patching the number of files on load time doesn't seem like the Right Thing(TM) to do. :)


Good Job!!!
[Here] is two of Nestopia dir log files - original and patched kiki image, the additional files appeared with patched form. So it's strange that number of existing files is written inside the real image FDS, or the image format? Why recalculating this value by emulation is incorrect (like any other dir command in file system), because the hardware can't do it, ...maybe it can?

by on (#81926)
Great job fox!

by on (#81927)
...i wish i could have a [x] button somewhere. =]

by on (#81928)
........

by on (#81934)
BMF54123 wrote:
Weird. Was that some form of copy protection back in the day (specifying the wrong number of files so disk copying tools would fail to copy everything)? Does an actual FDS even look at that value?


Yes it looks like it has something with copy protection...
Quote:
Apparently, some FDS disks implement a very simple copy protection scheme, which the game relies on in order for the game to refuse to work on the copied disk. Normally, the number of files that exist on an FDS disk is stored in the second block recorded on it. However, some games maintain "invisible" files, which are basically files that exist beyond what the file count number in the file count block indicates. This poses somewhat of a problem for copy software like FDSLOADR, since these tools rely on the file count block, and don't assume that there is any valid data past the last file found on the disk. This means that when these types of disks are copied, the invisible files will be lost, and when the game loads the files that do exist, the game's going to give the user heat about there being a file missing or somthing, gumming up the works. However in practice, when an FDS disk is programmed, the unused end of the disk is usually completely zeroed out, and this makes detecting the end of the disk simple: just wait to find a GAP period of extreme length. Except in rare cases, this model for detecting the true end of an FDS disk should generally provide the best results for copying the complete contents for all types of FDS disks.


If the file count block stores info about number of common files, how the real FDS is able to know about of hidden files? Is this realize somehow by BIOS? Maybe it counts while scanning disc surface.

by on (#81961)
plasturion wrote:
If the file count block stores info about number of common files, how the real FDS is able to know about of hidden files? Is this realize somehow by BIOS? Maybe it counts while scanning disc surface.

The "problem" is emulators require that very same BIOS! If it does any counting, it should count on emulators as well.

I'm wondering if there can be any condition that when dumping this game the file count would have accidentally be bumped down by 1, but the dumper didn't notice because the game seems to work fine in the beginning.

I think the only way to figure this one out is to see if the game does anything funky in its loading routines that would allow it to work correctly on the actual hardware, unless somebody actually can copy the game back to a FDS disk and verify that it works on hardware. Personally I've lost interest in this issue, too many open questions.

by on (#82039)
I think we can treat this issue rather like a bad dump, and we no need to know if this image works on the real FDS hardware.

Let's see some changelog part of T.Takeda when he add this autopatching feature to Nester.
Quote:
2001/03/22
code fixed 20
tested 20

# support 'VS Super Skater (Bootleg)'
# support 'Yuushi no Monshou (Deep Dungeon)'.

# change disk save file format (same as fds format / can load old files).
# change disk state save file format (can not load old files).
# calcurate number of files in fds image when load.


Yuushi no Monshou (Deep Dungeon) is Deep Dungeon II and can be found in earlier TOSEC romset pack as Yuushi no Monshou (1987)(Humming Bird Soft)[b].zip, and as another file know as Deep Dungeon II - Yuushi no Monshou (Japan) (v1.1) [b].fds and both are the same.
I confirmed that this image is the very same issue like Kiki kaikai. I used nestopia log to see all the file list with both sides, the last files are:
Quote:

Fds: Disk 1 Side A: 44k in 12 files, 17b trailing data..
Fds: file: "KYODAKU-", id: 0, size: 224, index: 0, address: 0x2800, type: NMT
...
Fds: file: "OPEN6/25", id: 0, size: 8736, index: 11, address: 0xA000, type: PRG
-------------------------------------------------------------------
Fds: Disk 1 Side B: 47k in 13 files, 18b trailing data..
Fds: file: "FLOOR12F", id: 1, size: 5960, index: 0, address: 0xC800, type: PRG
...
Fds: file: "MAP0PRO", id: 0, size: 2752, index: 12, address: 0xC800, type: PRG



and this is fixable in the same way like Kiki (by incrementation number of files by one), so patched rom show as additional (probably hidden) files that the common emulation has now access:
Quote:
Fds: Disk 1 Side A: 44k in 13 files..
Fds: file: "KYODAKU-", id: 0, size: 224, index: 0, address: 0x2800, type: NMT
...
Fds: file: "OPEN6/25", id: 0, size: 8736, index: 11, address: 0xA000, type: PRG
Fds: file: "CHARA", id: 96, size: 144, index: 12, address: 0xB300, type: PRG
-------------------------------------------------------------------
Fds: Disk 1 Side B: 47k in 14 files..
Fds: file: "FLOOR12F", id: 1, size: 5960, index: 0, address: 0xC800, type: PRG
...
Fds: file: "MAP0PRO", id: 0, size: 2752, index: 12, address: 0xC800, type: PRG
Fds: file: "DUMMY", id: 170, size: 1, index: 13, address: 0xDFF0, type: PRG



there's also "correct" dump of previous version of this game know as Deep Dungeon II - Yuushi no Monshou (Japan) (v1.0)
Quote:
Fds: Disk 1 Side A: 44k in 13 files..
Fds: file: "KYODAKU-", id: 0, size: 224, index: 0, address: 0x2800, type: NMT
...
Fds: file: "OPEN5/12", id: 0, size: 8736, index: 11, address: 0xA000, type: PRG
Fds: file: "CHARACTE", id: 96, size: 144, index: 12, address: 0xB300, type: PRG
-------------------------------------------------------------------
Fds: Disk 1 Side B: 47k in 13 files, 18b trailing data..
Fds: file: "FLOOR12F", id: 1, size: 5960, index: 0, address: 0xC800, type: PRG
...
Fds: file: "MAP0COM", id: 0, size: 1568, index: 11, address: 0xD800, type: PRG
Fds: file: "MAP0PRO", id: 0, size: 2752, index: 12, address: 0xC800, type: PRG

...and the same way while changing amount files of second disk side to 14 like in previous example we receive the DUMMY file, 1st side seems ok. I only wonder is this file has impact to code execute and is it used in english translation. (Maybe CHARA / CHARACTE is only save file, cause deleting this file relay only the same value we are talking about. In that case v1.1 has deleted save file)

In the same way (incrementation amount files by 1) I've tried to found dumps of other games that could has similar issue but I didn't found, even correct Doki Doki that someone confirmed on board that has copy protection, all the list was complete with visible DUMMY file. So even the hardware can work with hidden files, ripping tool probably check is there hidden files and recalculate this value makes all of them visible to avoid emulation problem, (and maybe there's some condition that this "bad" dumps occur (noticed that when I incremented amount files, sometimes SAVE files was appeared, and usually is placed as last additional file, maybe ripping tool decrease files number to hide unneeded or previous deleted SAVE file, but is not perfect)). If is true I think there's no need to improve emulation, even "bad dump" means "real dump"

In that case T.Takeda could know about weak point of dumping tool and wanted to enhance support with this kind of bad dumped FDS images. Lower value amount files of Kiki Kaikai, could be also result of bug code while playing, before was dumped. Last question is, does other emulator need this autopatching solution of T.Takeda to auto-fix bad dumps and enhance support of current romset receiving all deleted needed or not files every single game? (It could generate problems with unneeded, last deleted files).

[Here] is requested patch - Kiki Kaikai fixed.

by on (#82524)
I extensively tested this game with the new fix on this game today and was able to fight 2 of the 4 bosses before it goes back to freezing on a boss room. Would incrementing more invisible files perhaps prevent this. Just wanted to report my findings.

by on (#82526)
OK... so you confirm that this patch is working now. I think that's obvious that this value should be correct (all the necessary files must be visible (except some unnecessary save or extra files deleted before was dump)). If you increase more invisible files you make more visible files deleted or in other words "decrease amount of visible files with necessary data", and game cannot be able to access to this data, because this value during loading files tells that they not exist (that's the reason why kiki cannot load boss files into $7364 while loading second stage).

... i'm afraid you didn't found anything, this was told before thousand times, check that post before you said "i wonder what all of this mean", and please try to understand.

by on (#82537)
I did check. I started a new game and it was patched correctly. I fought the first 2 bosses successfully and when I got to the third it froze.

I may not completely understand how the fds works but im not so ignorant that I don't know if the game froze. Ill try again and see what happens. I was just reporting my findings here. :)

by on (#82539)
Did you patched game with the same name like ips patch, and with header that contains "FDS" at the beginning? After applying patch into image - do you have the same file list (look at last files in nestopia log)?
Quote:
Fds: Disk 1 Side A: 51k in 9 files..
Fds: file: "KYODAKU-", id: 0, size: 224, index: 0, address: 0x2800, type: NMT
...
Fds: file: "end", id: 30, size: 8124, index: 7, address: 0x6000, type: PRG
Fds: file: "WORKSS", id: 80, size: 33, index: 8, address: 0xDC10, type: PRG
-----------------------------------------------------
Fds: Disk 1 Side B: 48k in 13 files..
Fds: file: "mchar", id: 16, size: 6348, index: 0, address: 0x6000, type: PRG
...
Fds: file: "kitune", id: 27, size: 1180, index: 11, address: 0x7364, type: PRG
Fds: file: "jyzsoh", id: 28, size: 1114, index: 12, address: 0x7364, type: PRG

Did you played the same game before you applied this patch? You must to know that nestopia create some overriding patch image to avoid changing real file (write to disk is common, so it's a safety solution).
After all of this try to copy patched game into different folder or rename it.

IMPORTANT!!! You can't use previous created nestopia save state (created before you patched game) - they for sure contains wrong (old) amount files value.

...or do you suggest that during playing something happen and occur that game freeze in boss room like before? Please report more details.

[Try this patch]
It has deleted (hidden) save file ("WORKSS"), and visible boss file ("jyzsoh"). I think it's more correct, we don't need this previous (dumper) save file, maybe it's corrupted or loading this makes amount files of side B to wrong value. Anyway when you die, and choose first option this user save file will be created (overwritten).
I've played little more and beaten 5 different bosses or sub-bosses (don't know which one are you talking about, which one are third) I fought some fox, fatman, longman, even some tanuki that dressed up my cloths, and boss room never freezed. Maybe try to make some screen or describe where did you found this freeze rooms.

by on (#82608)
thanks plast. i will try it again and see if produces the same results. :)
Re: Broken FDS emulation: Kiki Kaikai - Dotou Hen
by on (#158079)
Hi, there.
I tried the game with the updated patch and I had the same problem where the fourth boss freezes. I think it's in a specific room (the one labled 四). The game freezes there. If you could take a look at it and possibly fix it, I'd really appreciate it!

If/when you look at it, use these cheats to enter the boss room:
004A 09 and 004B 09
These will give you enough tarot(?) cards to enter the boss room. They probably didn't want the player getting to the boss without enough ammunition to kick it's ass. I wouldn't mind playing through the game on NesterJ, but I can't figure out how to access the memory viewer, if you can even edit the memory addresses on it. I'm using FCEUX 2.2.2 to play through the game.

EDIT: I tried the patched ROM on VirtuaNES and I was able to defeat the seven bosses.

Thanks again.