« the importance of labeling your tapes *properly* | Main | vm heaven »

August 23, 2006

katamari hell

over the past week or so i have been trying unsuccessfully to capture and encode the latest runs on the katamari damacy series of games.

the fun started when i received a large number of tapes, perhaps between twenty and thirty of them. each tape had between one and about four or five unique runs on it. in most cases these runs were no more than two or three minutes in length. the runs were also most often scattered throughout the tape.

i decided to immediately (on capture) follow the instructions sent by the runners and locate each run on its tape, recording with the dvd recorder only those important parts of each tape. thus, i would use only a handful of dvds to capture the entire project, rather than the matching twenty or thirty discs to twenty or thirty tapes. this would be my first major mistake.

radix came calling a few days later repeating a verifier report of extreme audio desync in the verification copies of the katamari material. i was shocked and appalled, but not nearly as much as i would have been had i not already had a hell of a week trying desperately to keep the requested old material flowing into the flash encoder (but that's another story). one more thing going wrong seemed almost like the natural flow of things.

i quickly determined that the audio started out in sync, but actually became progressively earlier as each disc wore on. i had seen this before when i had tried to index at the same time multiple vobs created at different times, but this couldn't be the case now - the vobs were all part of the one large whole my dvd recorder seems to like to write to the disc (for all discs regardless of length or content, it writes five equal sized vobs together representing all of the material on the disc).

the only other viable explanation meant that i would have to somehow correct audio desync caused by so-called "title" breaks, or breaks inserted when i would press stop and then later record again on the dvd recorder to capture another run.

i began to experiment with various pieces of software (vob2mpg, etc) trying to change the format of the vobs so that dgindex could successfully compensate for the title breaks. none of my experiments were successful, and so i created this thread at doom9. as a side note, my very favorite thing about using open source software is that, in cases of technical questions about the software, the author him- or herself is on hand not only to answer said questions faithfully, but also to recommend different software if the present software cannot do the job.

i followed neuron2's advice and used projectx (an impressive piece of work, by the way) on the katamari vobs. this produced vobs which, when indexed with dgindex, had in-sync audio!

but there was a problem - to make a long story short, it turned out projectx had actually deleted sections of the video to keep the audio in sync. besides being totally unacceptable from a speed running standpoint, this also caused me to have to completely remark the coordinates of all fifty something segments once again in order to get proper verification copies out to the verifiers.

unfortunately, i was not aware of the frame dropping issue introduced by projectx until a verifier reported it to me.

at this point i wondered whether i shouldn't have simply bitten the bullet and used all thirty something dvds to recapture the entire project without using more than one "title" per dvd. i immediately started recapturing the first tape in this way.

then i had an idea: why not get my old dvd player out (bought to test sda dvds before i sent them out way back when) and use it to make an analog copy of the dvds with the corrupted timestamp information? the resulting analog copies would be slightly inferior in terms of video quality (being mpeg-2 compressed twice), but they would each consist of only one title, eliminating all of my problems processing the material. i got out my old dvd player and began the copying process, switching over to a new spindle of dvd-rs.

then i found out that the new dvd-rs are incompatible with my dvd recorder.

at first i didn't want to believe it, because every type of high speed verbatim media i had bought before had been compatible. but all the signs were there: the long finalization process and the single, tiny vob and accompanying table of contents file that resulted.

not wanting to slap down $40 for new media that day, i went to take out my old dvd-rws with poor compatibility i had used with limited success in the early days of the dvd recorder.

but before i could attempt to copy the original katamari dvds again, i again had a thought: why not simply raw read the entire "incompatible" disc and scan the data thereon for vob headers? assuming the dvd recorder wrote the vobs in place (and being write-once media, i could imagine no differing scenario), i knew i would be able to recover the captured video regardless of the mangled table of contents file (created so because the dvd recorder couldn't read properly the media it had just written to). i had already done this once a few months before for kridly using simple unix software written by djgrenola for that sole purpose.

this time, though, i decided to check out ballofsnow's "recovering data from unfinalized dvds" links, and in them i found the wonderful windoze program, isobuster. this program was so impressive to me (it duplicated under doze the functionality and flexibility of grenola's work without any of the manual labor that had been required with the command-line unix version of the recovery process) that i bought it ($30 us) on the spot when it asked for a registration code to complete recovery of the vobs. i figured that it was a good deal, as another spindle of dvds would cost me at least $30, and isobuster would remove in a user-friendly fashion on the encoding machine the need to have compatible media for my dvd recorder.

with a viable capture process notwithstanding the incompatibility of my media i finished making analog copies of the original katamari dvds with corrupt timestamp information.

i next tried for several hours to create a script to "realign" the segment coordinate information from either the original capture attempt or the vobs as "repaired" by projectx. (i was trying to avoid having to mark all of the segment breaks for a third time.) this was my second major mistake, as after several hours of completely fruitless effort, i discovered that the corruption of the timestamps produced desync at every title break that not only was not linear in its growth - it actually went backwards (became more in sync) some of the time, completely destroying the viability of an automated realignment.

so i set out to mark all of the segment breaks a third time. it turned out that the timestamp corruption had also somehow caused entire runs to "slip through the cracks" in the previous encoding attempts - on the second disc alone, i have now marked over twice as many segments as the disc originally appeared to contain. how this is possible i am not sure, as i remember checking the output of the vobs with corrupted timestamps several times, and every time it totaled something reasonable, something just under two hours in length ...

lessons learned from this fiasco are many (i no longer have to have media compatible with my dvd recorder to capture, for one), but i still get the feeling that i in fact traded one lousy situation (capturing with the all-in-wonder) for a much more lousy situation (capturing with the dvd recorder). for example, it will be necessary in the future in the case of a thirty-tape project to use thirty dvds to capture the material.

the quality is better now, and perhaps that should be all that matters in the end - but i find myself time and time again in situations where i have to spend several days "learning my way out of a wet paper bag" as it were. this isn't rocket science, but without the necessary experience, people are just going to be totally screwed when they try to do this stuff themselves - especially if they get a dvd recorder as retarded as mine in this area (and mine is still recommended in the dvd recorder thread in the sda forum!).

one day this community will be delivered from this sorry state of technological affairs.

that day is not today.

Posted by njahnke at August 23, 2006 8:51 PM

Comments

Methinks you need a vacation, then again you'd probably wind up with another dozen tapes or so by then.

Posted by: MatrixTN Author Profile Page at August 23, 2006 10:06 PM

Oh my, a fair number of those katamari vhs tapes were mine. Eek, I can't believe how much trouble the katamari project caused.

I hope the videos turn out to be worth the trouble.

Posted by: Tom_Batchelor Author Profile Page at September 30, 2006 5:44 PM

Post a comment

Thanks for signing in, . Now you can comment. (sign out)

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)


Remember me?