movable type ate my blog, so this is just temporary until i can roll my own solution that won't corrupt itself to the point where a fresh install won't fix templating issues. sorry for the inconvenience and don't miss the new entry immediately below.



ds ii

2009-04-05 19:58:31

it's been almost eighteen months since i last processed a ds run, and i've learned a lot in that time. i knew that if dsgamer or anyone else submitted a new run, i'd do things differently from last time.

well, dsgamer submitted not only an improvement on his any percent run of prime hunters, but a new 100% run, as well. on top of that, greenalink did a run of mario 64 ds. when all three runs were accepted by verifiers, i knew it was "time to get serious" (as mega man would say) about processing ds footage.

perl has become my new best friend, and i knew it had integration with imagemagick, gd, etc. a few google searches later, i had decided on using imagemagick ("perlmagick"), since it came with a well-tested autocrop function.

originally i was going to try to determine the z-distortion (i don't know the proper term) of the ds screen(s) and correct for that (basically if the top of the screen is further away from the camera than the bottom), but the only implementation i found of that functionality i couldn't get working with any of my test footage (it never changed anything), so i decided to put that off until next time. i felt like i had enough on my plate with making sure the autocrop threshold was correct all the time.

basically, i set a default threshold and try to autocrop the current frame. if the autocrop succeeds, then i note how much was cropped and move on to the next frame. if it doesn't succeed, then i gradually turn up the threshold until it does succeed. if it never succeeds, i don't save the crop data. if it succeeds but it's cropped off way too much as determined by a sanity check, then i don't save the crop data.

i do this for a certain number of frames (i ended up using two seconds' worth for dsgamer's runs), then go back and crop all of those frames by the means of the values i saved for each side. this is much improved from how i was doing it before using avisynth, which could only look at a single frame to determine values. i also don't think i could get the values that the avisynth plugin was using very easily due to how simple the avisynth language is, but i could be wrong on that.

in other words, the old method was very high maintence - i had to go five seconds at a time and manually adjust the threshold. a lot of times bloom from the screen (if the screen was very bright and you could see the ds's body glowing in the dark, for example) would throw off the threshold. i got around that this time by starting with a rather high threshold and then throwing out crop data that deviated too much from an arbitrary norm. (by "deviated too much" i am talking about how if the screen was very dark the high starting threshold would sometimes cause it to crop off too much, for example the area outside the hud in human form.)

a new feature for next time might be to determine the norm on the fly. i guess that would be means for the whole segment. actually i thought i was going to have to do that this time, but dsgamer's runs came out perfect on the first try. it was a good thing, too, because it took the better part of a week for one of the computers in the lab where i work to process them (i think most of it was i/o though since i had to read twice and write once a .png file for every single frame in the runs). i'd also like to learn how to read and write yv12 and do everything through mplayer/mencoder pipes next time for a big speedup. normally i wouldn't care so much about the speed, but i think this may make it into anri someday (even though only two people have ever submitted ds runs to the site), and it's totally unworkable for most people's computers the way it is now.

i did the audio on dsgamer's runs the same way i did it last time - by putting the over-the-air audio on the left channel and the audacity-recorded audio on the right channel, wearing headphones, and adjusting the audacity audio's offset until i couldn't distinguish the right and left channels. (i then cut the over-the-air audio.)

greenalink's run i did totally differently because he recorded it totally differently. he was able to keep his ds much more still than dsgamer was probably because of how the game is controlled differently, so autocrop was not really needed. i just figured out a set of rotation and cropping values for each individual segment and applied those. he also recorded both screens all the time. he and i talked about this quite a bit and we decided that since mplayerc lets you zoom in on one screen in the video, leaving in both screens would be the best option since some people might want to see what's going on on the bottom screen even when he's controlling a character. it also goes along with the site's general ethos, not throwing out potentially valuable data that could be used by someone to improve the run.

greenalink's run didn't need anything done to the audio (except maybe normalization) since he recorded both the video and the audio using a dvd recorder, so it was already synced.

all three runs will be going up on the site soon, and i hope you enjoy them.

here are the tools i made or used to process them this time:
mplayer/mencoder (to dump to pngs/encode video from pngs)
ds.pl (to process the pngs)
avisynth rotate (to rotate greenalink's segments before cropping)

by the way, getting perlmagick to compile under os x was a total pain in the ass. it took me several hours of googling and experimentation. i ended up manually linking the perlmagick sources against the imagemagick libraries delivered by macports.

no more musical chairs

2009-02-01 18:25:51

i forgot to put the segments of the rosenkreuzstilette run in proper order before making the verification-stage avisynth scripts: segment 8 was actually segment 5, and segments 5-7 all had to move up one. it sounds simple enough, but there is no way to rename everything in situations like this without having at least one file have a temporary name. and if it's me doing the renaming i will often second-guess what i am doing halfway through the operation because i am ocd like that.



so i cooked up a quick perl script to just give every file in the directory a temporary name and then rename everything back based on a transformation rule. the old segment numbers are the keys of %rehash and the values are the new numbers. i use %files to remember what the names used to be after the files are given their temporary names. i made the temporary names random (with a check to make sure they're unused) since i may have more than one copy of the same file in the directory (so renaming them to their md5 hash or something is out).



enjoy.





#!/usr/bin/perl -w

#nathan jahnke 


my $natename = 'rks';

my $thedir = "/p/${natename}";

my %rehash = ( #old, new

	8=>5,

	5=>6,

	6=>7,

	7=>8,

);

my %files;


opendir(P,$thedir);

my @files = readdir(P);

closedir(P);


for (@files) {

	if (-f "${thedir}/${_}") {

		my $newname = &newname;

		system('mv', '-nv', "${thedir}/${_}", "${thedir}/${newname}");

		$files{$newname} = $_;

	}

}


for (keys %files) {

	my $newname;

	if ($files{$_} =~ m/^${natename}-v-(\d+)\.(.+)$/) {

		my $segnum;

		$segnum = $1;

		$segnum = $rehash{$segnum} if $rehash{$segnum};

		$newname = "${natename}-v-${segnum}.${2}";

	} else {

		$newname = "${files{$_}}";

	}

	system('mv', '-nv', "${thedir}/${_}", "${thedir}/${newname}");

}


exit 0;




sub newname {

	my $retvar = '';

	while (-e "${thedir}/".$retvar) { #try until we get a unused name

		$retvar = &random;

	}

	return $retvar;

}


sub random {

	my $retvar = rand(1);

	$retvar =~ s/^0\.//;

	return $retvar;

}


audio has frames too ii

2008-11-27 20:19:41

so it turns out the current release version of perian causes audio desync when playing an ffvhuff/pcm avi. moral of the story: if you're trying to resync something, for the love of god, make sure the instrument you are using to check sync is not itself causing desync. it turns out that the "skipping frame!" messages from mencoder were actually correct in that they were also skipping video frames to compensate for the mutilated ac3 input - which means that all along all i had to do was do a simple sync correction (albeit one i have to discover myself) to fix output from tiki's borked dvd recorder - assuming i first make nmf using mencoder e.g.:

mencoder -dvd-device /p/mm6 dvd://1 -fps 59.94 -vf-add tfields=0 -vf-add scale=320:240 -ofps 59.94 -ovc lavc -lavcopts vcodec=ffvhuff -oac pcm -o nmf.avi

dgindex, because it doesn't drop video frames to compensate for the bad ac3 ones, will cause randomly progressive desync. though i need to remember to use it for timing in cases when the in-game timer isn't used, such as this mm6 run (the release versions of such runs will actually be several seconds shorter than their official times indicate).

audio has frames too

2008-11-23 21:21:15

tiki sent me his new mm6 run today in dvd format and i was dismayed to notice that it apparently had random audio desync after indexing using dgindex (with the audio leading the video). the vobs played fine in e.g. mplayer, so i wasn't sure whether his dvd recorder was making a new chapter and chopping up the audio every few minutes or what.

after several hours of playing around, my initial guess turned out nearly correct: the "Skipping frame!" messages i was seeing every few hundred frames from mencoder trying to transcode the file were because the audio was bad, not the video. actually, i should have known this from the start - if it had been the video, then the video would have been leading the audio, not vice versa as i initially observed at points. i had also forgotten that the term "frame" also applies to many audio formats. using mencoder's -noskip option together with the -mc 0 option seemed to restore a semblance of audio sync in an nmf - enough to carefully stretch the audio to correct the progressive desync to an acceptable level.

tiki has been advised of the situation. right now we are guessing it was because he used rf (from his nes). pal nes is known to generate a nonstandard signal (enough to cause massive frame droppage with more sensitive capture devices), so maybe the ntsc nes in combination with rf confused his dvd recorder? i'll post again as i learn more.

more decentralisation

2008-07-27 13:42:16

so i spent about $600 and got two sata-2 samsung 750 gig 7200 rpm disks with 32 megs of cache each as well as a synology ds 207+. i looked at the reviews on newegg and the specs and it looked like a good enough unit, but what really won me over was the description of the bundled software (yes, really) on the synology website. it seemed to me like someone inside the company really understood what it means to be a network software (e.g. gmail) developer today, and thankfully ($600 later) it turned out i was right - the software is amazing:

nas_dstation_settings.png nas_dstation_tasks.png nas_network.png nas_status.png nas_terminal.png nas_volume.png nas_winmac.png

as you can see, i set up the disks in a raid1. this is the new sda project drive ("p"), replacing an old sata 500 gig in v5 (my hp pc). unfortunately the old 500 started locking up a few months ago and after it did it three times last week i decided i had better not wait any longer. i do daily backups of all the in-progress sda stuff but i didn't want to work off of the backup if i could avoid it, which i did. actually even now i'm not sure whether the lockups were caused by the drive, the cable or the controller (on the hp motherboard) but to be honest i don't have the time to be troubleshooting things like that, especially when the lockups are more or less random. also no matter the outcome i would have had to have spent money anyway so i felt this purchase, subsidized 50% with sda funds (even though i have no reason to buy it except sda), was inevitable.

softwarewise:

BusyBox v1.1.0 (2008.06.20-16:34+0000) Built-in shell (ash)

Enter 'help' for a list of built-in commands.


p> uname -a

Linux p 2.6.15 #639 Sat Jun 21 00:30:16 CST 2008 armv5tejl unknown

p>


i have to say that the built in torrent client (rtorrent with the web gui) isn't bad. the only thing that seems to be missing is down/up rate caps in the web gui (it may be available in the windoze-only standalone app - i haven't checked) but it's not the end of the world. having the torrent client run on the same machine as the project drive is important and this morning i was able to shut down v5 for the first time in years, which was pretty cool (literally). it's by far the biggest power user out of all the v6 machines (by perhaps an order of magnitude - the other three are two laptops and a mac mini) so having it off when i'm not encoding is a boon.

the nas looks like it has a ~2 gig solid state drive for its software, which comes as an image file which is then flashed to it over the network (it doesn't come with any software preloaded, just a disc with an image on it to get you started). upgrades happen this way as well - of course i updated it to the latest after i got it and it worked great from my mac.

p> df -h

Filesystem                Size      Used Available Use% Mounted on

/dev/md0                  2.3G    235.0M      2.1G  10% /

/tmp                     62.0M    108.0k     61.9M   0% /tmp

/dev/md2                684.7G     38.8G    645.9G   6% /volume1

p>


overall i'm quite pleased with it. it works as advertised.

the wine whine

2008-06-28 11:43:31

so wine 1.0 came out last week (firefox 3 sort of stole its thunder) but i caught it on slashdot and i thought, what the hell.

i'd stayed away from wine previously because i was almost certain nothing as complex as my use of avisynth would run 100% satisfactorily. needless to say, i was quite surprised when it compiled on one of my macs and played an unmodified sda avisynth script in vdub correctly.

that was over a week ago, and i'm still not entirely over the weirdness factor of running the whole anrichan family of tools without running any microsoft code. so i guess i'm not as elated as you might imagine - more like dumbfounded. it's been one of those rare moments when technology catches me off guard.

i've read that wine works even better under linux due to apple's lame x implementation, so i'm now considering what to do with the one remaining machine i own that runs windoze as its primary os ("v5").

it's not perfect by any means - as far as i can tell, pipes aren't supported in batch files, which meant i had to reimplement everything in unix shell language. it hasn't been a big deal except in the case of anri.bat, which will need major reworking, and that's a damn shame, because i thought at first that i had "discovered" a universal anrichan.

i'm also having problems with processes, e.g. avs2wav and bepipe, not being able to close files that are on network volumes. the process will complete successfully, but the close call hangs, meaning i have to hit ctrl-c to continue the encode script - unacceptable. i've actually seen this before under windoze (though not working on sda stuff), so i doubt it's wine's fault per se - just something i'll have to dump more time into.

i also can't seem to figure out how to install codecs that include right-click -> install .inf files. i grabbed the dll call out of a real copy of doze and tried that, and it does run, but doesn't seem to produce any result under wine. i'm lucky the anrichan installer installs lagarith, because i probably wouldn't have access to that codec under wine for doing nmf otherwise.

speaking of nmf, here's my unix nmf command, my v command and my mq commmand as references for how i reimplemented snow's batches in unix shell language. sumichan is essentially unchanged because i'm still using avisynth - the only difference is the batch file writing for calling nmf.bat on each of the slices, which now obviously just writes calls to nmf instead. syntax is like 
nmf "P:\\projname\\runname" 1
(the 1 for setting "delete temporary files" to true, from snow's batches).

look for the first runs produced with the help of wine on sda soon.

sumi-chan

2007-10-07 02:56:19

tonight saw the successful launch of my virtual multithreading solution, sumi-chan.

first of all, some background: right now, i own three modern computers, all of which are powered by dual core processors. the brilliant among us will have already concluded that this means i have a total of six (6) processor cores available. all the computers share storage over a gigabit lan.

unfortunately, the high quality deinterlacing filter mvbob() is not multithreaded. this means it can only use one core at a time. in the past i have set up multiple instances of vdub with e.g. one fourth of the total segments in a run to each, then started them all at the same time, but of course not all the segments are identical in length, so there would always be inefficiency as some vdubs would finish their deinterlacing before others.

avisynth is a very useful tool, and it was only a matter of time before i used it to automate a process by which any given input could be split up across a given number of threads, then automatically recombined later. i call this process sumi-chan for lack of a more creative descriptor (if the name were to describe the software properly, it would be something like "avisynth filter virtual multithreading system" - so long! so boring!).

usage is simple - i just run sumichan.sh with the avisynth scripts' basename and the desired number of cpu cores across which to split up the work, then run the resulting batch files on each real machine (one batch file for each cpu core). thus, entering 6 for sumi-chan's second argument results in six sets of avisynth scripts and six batch files.

due to rounding errors, if the number of frames in the source material is not divisible by the number of cpu cores specified, a few frames will be "squeezed" to the final work unit. this is unlikely to result in any significant inefficiency, as mvbob() usually processes between 1 and 2 frames per second even in extremely complex scenes. changes in said complexity throughout each work unit will result in much more significant inefficiency (compared to a true multithreading solution whereby all cpu cores would be simultaneously busy or not busy).

sumi-chan will appear as an optional feature in anri-chan 3 (we're getting close to the release of 2 right now). it ought to make encoding d1 material much more realistic for most people.

incidentally, i'm using quantizer 1 vfw xvid (via vdub) for the nmf. i've been extremely pleased with both the cpu and disk costs associated with this codec (they are much smaller than those associated with lagarith - and it crashes less to boot!).

of course, i have to include the obligatory crap quality picture of my new virtual computer, dubbed "v6". cores 1 and 2 are in the macbook on the left, while cores 3, 4, 5 and 6 are monitored on the mac mini's screen, but are in reality split equally between v5 (my pc) and the mini. (the terminal windows lower down on the mini's screen are left over from working on the prime hunters run as discussed in the previous entry.)

diy ds

2007-10-05 16:12:27

it's been almost eighteen months since i posted the ds recording guidelines, and now the first ds run has been submitted to sda.

DSGamer3002 followed those guidelines to the letter for his prime hunters run, and this was reflected in the quality of the recording.

however, the original recordings needed to be cropped in order to get rid of their useless black borders. the catch was that the cropping parameters needed to be dynamic - that is, where and how much i need to crop changed every few seconds.

as i'm sure you've figured out by now, i came up with a way to accomplish this in a semi-automated fashion, and the rest of this blog entry will be devoted to this methodology.

the first step was to locate a simple autocropping plug-in for avisynth. this came in the form of the aptly-named autocrop, found at warpenterprises. thankfully, autocrop() offers a great deal of customization of the autocropping algorithm. the user can set not only the crop/don't crop threshold, but the number of sample frames the plug-in takes into consideration within the video, as well.

the crop threshold is probably 8-bit luma, while the sample frames are, according to the plug-in's author, taken equidistantly from within the source video. however, no matter how many samples the plug-in uses, there will always be only one set of cropping settings per video. thus, whenever dsgamer's tired hands moved ever so slightly, the screen would no longer be contained entirely within the initially-calculated cropping values.

the obvious solution was to make many more, smaller videos out of dsgamer's run - enter acrop.

this bash script takes two arguments: the basename of the video and audio originals and the number of frames in the original video. the script then creates a number of avisynth scripts, each of them including a number of source declarations and autocrop() invocations for bits of the original video.

i call each section of video on which autocrop() operates a "window". the window size is not arbitrary, but is determined by how long a sequence of absolute black will be on screen in the source video. if any window consists entirely of black, then autocrop() will naïvely produce 0x0 video, which is a contradiction in terms - avisynth has no ability, from what i have seen, to resize a 0x0 frame. thus, even though it's all black anyway, windows consisting entirely of black must be avoided. the window size, then, should be perhaps one or two seconds longer than the longest stretch of absolute black in the source video.

i found taking the correct number of samples, too, to be crucial: the process will fail (producing 0x0 video) if either too many or too few are taken.

for convenience and efficiency of processing, i included several windows in each new script file. for this speed run i created scripts 1000 frames long, 5 200-frame windows in each. because the source video was exactly 30 fps, a 10-minute segment would correspond to around 20 individual script files.

it was not possible to include every script file in one master script for two reasons. one was the extremely long startup time associated with stacking multiple instances of autocrop() on top of one another. on my core 2 duo 2 ghz, it took between 10 and 20 seconds just to open one script with 5 windows in it. the other was that each window represents one directshowsource() declaration for the source video - in other words, there was one copy of the entire source video in memory for each window in the currently open script(s). avisynth is extremely ram-inefficient in this way, but you can't blame it, actually, because we're talking about windoze here.

of course, my default autocrop() threshold of 100 was not always optimal. in fact, i had to preview every window in vdub to make sure it was not too high or too low. many times i simply replaced every 100 with e.g. 70 because the game environment in that particular segment was darker and autocrop() was getting fooled and cropping off parts of the game screen. it was also occasionally necessary to increase the threshold when e.g. the screen went totally white, resulting in light reflecting off surrounding objects and back onto the area around the ds lite's screens, in turn resulting in autocrop() leaving in the plastic border around said screens.

after i determined the correct autocrop() parameters for each window of a segment, i aligned the audio (recorded separately in audacity by dsgamer via an 1/8" miniplug cable) by placing the over-the-air audio from the original source video on the left channel and the correct, line-in recorded audio on the right channel. i knew that the "new" audio was lined up when i could not distinguish that sound was coming first in either the left or the right channel. i adjusted the correct audio's delay by using delayaudio() in a master avisynth script file which loaded all of the individual pieces of the segment in question (since saved from the original scripts as lagarith-compressed new master files).

i would like to emphasize my success with this relatively novel method of audio realignment. in sharp contrast to the usual method when it comes to audio realignment, i did not even have to look at the video to know the audio was aligned - i literally did the work with my eyes closed.

and so the world's first high quality downloadable nintendo ds speed run was prepared for distribution on sda. i hope that everyone enjoys dsgamer's hard work and my automation-laced lack thereof when his run is posted on sda soon.

i almost forgot - here's the obligatory screenshot of my initial autocrop() parameters discovery process.

framerate decimation hell (or: deflickering revisited)

2007-09-26 00:28:13

i'm encoding the ocarina of time 100% run for the third time right now (second time was due to a bad statid).

F1: ||||||



F2: | | |



F3: | |





in the above picture each | represents one frame. this drawing depicts one complete cycle and only one complete cycle, so the first frame not shown is the first frame of the next cycle. it would be solid blue like the first frame of this cycle.

oot is d4 f3. from the f1 original i was going to f2 first, then to f3 to avoid odd (deflickered) frames (shown above with a bar through them). otherwise i have to use retard bob which does a marginal job of undoing the deflickering and takes exponentially more cpu. more on deflickering in these two blog entries (incidentally this is the same reason i put up this topic).

the frame marked with blue is fine. the problem as i understand it is that sometimes instead of the motion happening in the green frame it happens in the purple or red frame (and gets lost as you can see on the f3 line). in these cases the pattern of motion in f1 is:

K D D K D D or
K D D D K D respectively, instead of
K D K D D D

where K is a motion frame and D is a duplicate (no motion). when i was testing this i saw only sequences of the last type which are ok. but there are often sequences of the first or second type. and once this starts it apparently continues for some time, thus the long, painful stretches of f6 people are seeing.

so basically the game isn't dropping frames at regular intervals. sometimes it will be ok to go -> f2 -> f3 but usually not.

the good news is that i think i've worked out a solution with the help of donald graft's decimate():

changefps(f1/2)
decimate(cycle=3)

this seems to be picking the correct frame (either green or red) from the f2 constellation, probably because it's checking to see that each frame is unique. this means i probably won't have to go back to the horrid retard bob.

mac mini

2007-09-16 16:08:00

so i bought a "top of the line" mac mini.

it's a core 2 duo 2.0 ghz with 2 gigs of ram (it isn't "apple" ram, of course).

total cost was $1,017.28. cost to sda was $339.09. 1/3 of the machine was paid for with sda funds for the following reasons:

1) the recent increase in the number of d1 f1 runs i need to encode for the site meant less and less idle time for my macbook. i was concerned that i would run out of cpu power to meet demand, especially after the warm reception of metroid prime 3 coupled with its length: the current estimate for an optimal any percent run is around three and half hours - not counting cutscenes. for reference, it takes about a week to encode 5 hours of d1 f1 video (deinterlacing to nmf and encoding all seven qualities) using four processor cores (assuming full speed from them all). i had been counting on most runs being d4, but this appears to be coming to an end.

2) related to 1), i had no quick way to take 10-20 gig of content in to campus to upload while the macbook is locked down encoding.

only 1/3 of the cost was covered by sda because i could have gotten a better machine for the money had it not been a mac (i wanted a desktop mac because using the macbook as one was suboptimal). granted, this will help me do non-encoding work for sda more efficiently, but i feel that is a rather weak reason to take more money from our reserves in this crazy time. also, regarding 2), the other half of the reason i have to have the macbook on campus is school, which has yet to demonstrate decisively that it is beneficial to m2k2sda (though it has been making gains of late: some work on anri-chan was actually done as part of my on-campus job!).

i backed down from saving up for the next beast machine (codenamed V6) because it no longer made sense economically. there are many factors involved, but to summarize:

3) i need more power right now (see 1) and 2)). an eight-core machine costs about $4,000. i wouldn't be able to afford that until may at the earliest.
4) there has been no solution found for multithreaded deinterlacing. therefore i will have to write my own (already discussed as a feature for a future version of anri-chan). once i do this, it will no longer matter how many physical computers are performing the deinterlacing - the load will be divided evenly between the available number of cores.
5) following from 4), continuing to pretend otherwise would mean that the macbook and V5 would be considered useless after the purchase of V6. together they have just under 50% of the theoretical power of V6, so this would be a great waste. by buying a weaker machine now and writing software to tie it together with my existing two, i can have 75% of the power of V6 right now for 1/4 of the price.

therefore, i now think of V5, the macbook and the new mac mini together as V6.

obligatory birth sequence documentation:

anri-chan and total independence

2007-06-30 19:12:29

it's been over two years since m2k2sda became monetarily self-sufficient.

money changes hands at least once every month, as i have to pay the $99 monthly m2k2sda server bill. when the google ads on the site earn us more than $99 a month, then sda requires no outside investment to continue to operate so long as it is continually updated (it's thought that otherwise, site traffic, and therefore ad revenue, would go down, while the server bill would remain the same, thereby reversing the aforementioned positive trend).

unfortunately, the amount of labor grenola, mike, snow, radix and i pour into sda, if paid, would equate to a sum much, much larger than $99 a month. therefore, when i declared sda monetarily self-sufficient two years ago, i was referring only to actual money earned and spent.

now it's time to take care of the rest.

for years, automating my job was a hopeless fantasy. the number of correct decisions required to encode material for the site was simply too high, and i knew of no way to have the computer make them for me. anyone who has seen the knowledge base and the tech support forum knows that asking inexperienced people to do my job as i do it is a losing proposition.

so i tried instead to make sda's revenue grow - to make the site pay us back for our work on it. i learned many things from my adventures trying to sell sda content on dvd, but probably the most significant was that it doesn't work: sda may never generate enough revenue to pay its workers, and sometimes i think it's better that way.

but the fundamental problem remained: sda needs people to add new content, and if i died today, i thought to myself, would someone be there to replace me? i decided that even though i can't control my mortality, i can control how i will be seen after the fact.

little by little i chipped away at the problem, making generic.avs into a highly concentrated, library-driven block of boolean action. generating the scripts for new projects became a matter of enabling or disabling variables. now one script could do it all.

but there still had to be someone there to make decisions based on the content of the source video. i had automated the ntsc/pal distinction, but the rest of them seemed hopeless: how should i tell the computer to check whether every second or every third frame is the same as the previous? and even if this could somehow be done - what about the games with a varying framerate?

therefore, i created the concepts of d (dimensions), f (framerate) and fdp (framerate decimation paradigm) to quantify the differences between games. if a database of every game's d, f and fdp could be created, i reasoned, then surely some sort of script could be written that would take those values and configure generic.avs, reducing my job to reading numbers out of list and typing them in to a window. even if the game in question weren't yet included in the list, it could be added by any tech support staff member after seeing a brief, automatically-generated sample of the game's video. it would be the next best thing to full automation.

as many of you are already aware, i decided to call this script anri-chan after the anime character who (like mickey and goethe before her) used magic to create numerous inferior copies of herself in a failed attempt to automate her job. thus, i am anri-chan, and anri.bat is an inferior copy of me. (when i began the project, i wasn't even aware that dgindex accepted arguments on the command line, and so i thought that automating the steps involving the user interface might be impossible, thereby dooming it all to failure.)

but these open source tools came through for me in the end, and so i was able to produce a proof-of-concept much more quickly than i had anticipated - and even better - it worked: other people were actually encoding acceptable video using anri-chan!

even vista couldn't stop me.

and as if that weren't enough, the free, open source model impressed me yet again today, as b-man in the sda forum went and wrote the anri-chan encoding settings manager for me. i have always said that it is not only me who can do this (just that i was the first to be chosen), and now i have been proven right. it really is true: if you build it, they will come.

i will finally make sda eternal by taking away the cost of its maintenance (at least as far as my job is concerned).

so now it's just a matter of writing comforting documentation ("when the program says do x, that's when you do x!") and officially releasing the software. with this i will simultaneously increase submissions, decrease my workload and therefore ensure the future of the site.

two years wasn't so long to wait, after all.

time for lossy nmf

2007-06-23 22:11:58

[20:53:11] <nate> i've decided i'm never using lossless nmf again

[20:58:18] <Radix> why>
[20:58:32] <DJGrenola> you had to ask didn't you
[20:58:33] <DJGrenola> heheh
[21:00:27] <nate> disk read locks
[21:00:31] <nate> i have too much cpu with 2 cores
[21:00:40] <nate> so what happens when i put a 4 core cpu in there
[21:00:46] <nate> i buy a raid? i don't think so
[21:01:00] <nate> more cpu just means less relative cost to decode x264 original
[21:01:10] <nate> can't use it with interlaced stuff but that's not an issue
[21:01:16] <nate> this is post-mvbob stuff
[21:01:39] <nate> basically what happened is the zoid ss flv was cut off
[21:01:49] <nate> because it was one of those times p: filled up while i was in kansas
[21:01:54] <nate> and i only just caught it today
[21:02:01] <nate> so now it's taking ~12 hours to reencode the flv
[21:02:07] <nate> when it's less than half that cpu wise
[21:02:33] <DJGrenola> I'd definitely consider looking into ffvhuff in ffdshow
[21:02:39] <DJGrenola> I use that for everything
[21:02:47] <nate> if it's lossless then it's too big
[21:02:51] <nate> well
[21:02:59] <nate> if it's anything like huffyuv anyway
[21:03:04] <DJGrenola> it's a little smaller than huffyuv is, ~20% or so
[21:03:09] <nate> 3:1 compression for all #000000 is pathetic
[21:03:38] <Radix> have you tried a compressed partition?
[21:04:04] <DJGrenola> :o
[21:04:10] <nate> nope ... but do you really think blind compression i.e. not knowing it's compressing video will do a whole lot
[21:04:19] <DJGrenola> that's a clever idea
[21:04:27] <nate> certainly can't be more than the 3:1 the shit lossless ones get
[21:04:32] <Radix> i dunno, zip compress a lossless avi and compare to huffyuv
[21:04:51] <Radix> huffyuv was built for speed, not compression ratio
[21:06:33] <nate> ok making a test file
[21:06:39] <Radix> also try zip compressiong a huffyuv avi
[21:06:45] <Radix> preferrably the same one i guess :-p
[21:26:26] <nate> ok this is pal d1 f1
[21:26:44] <nate> samus wearing the varia fusion suit kills the sheegoth and gets the wave beam, then leaves c.i.temple
[21:33:30] <nate> clip is about 34 seconds long

1.9G test_rgb24.avi
924M test_rgb24.avi.zip
680M test_huffyuv.avi
611M test_huffyuv.avi.zip
467M test_lagarith.avi
467M test_lagarith.avi.zip
328M test_x264.avi
328M test_x264.avi.zip

[22:02:54] <nate> quantizer 0 x264 avi it is
[22:03:12] <nate> though it seems lagarith would come close
[22:04:32] <nate> xp (10 megabaud) mpeg-2 original from dvd would be just over 42.5 meg
[22:05:10] <DJGrenola> wonder if snow has a lossless mode
[22:19:57] <nate> trying quantizer 1 etc
[22:37:46] <nate> looks like quantizer 1 and quality 1 are similar in size to the quantizer 0
[22:38:01] <nate> so what i'm going to do is no quantizer two pass 10 megabaud and look at the thing closely to see if i can see any artifacts
[...]
[23:13:42] <nate> the difference between the original and 10 megabaud is slight
[23:13:50] <nate> visible ... but not necessarily bad
[23:13:57] <nate> what it did was smooth out some areas
[23:14:03] <nate> typical of h.264 i guess
[23:14:18] <nate> the thing is that a lot of what it smoothed out was actually not supposed to be there in the first place
[23:14:22] <nate> because it was artifacts from mvbob
[23:14:34] <nate> so sometimes the lossy looks better than the lossless (1, 3) [23:16:54] <nate> it's certainly better than mpeg-2!
[23:17:50] <nate> near key frames they're almost identical (2)

1: original | 10 megabaud 2-pass x264 2: original | 10 megabaud 2-pass x264 3: original | 10 megabaud 2-pass x264

[23:26:31] <nate> so 10 megabaud 2-pass x264 avi is the new standard nmf
[23:26:35] <nate> hurrah

generic.avs grows stronger

2007-03-23 00:31:36

tonight i finally got automatic (well, almost) d working in generic.avs (the script from which all other sda scripts are created).

just set "false" to "true" on line 7 to enable d1 (d4 is default). pal autodetection remains in from the last version, which i can't remember if i posted here or not. change "false" to "true" on line 23 to enable the statid (after you've finished filling out the trim(), for example).

additionally, you'll need to change all occurrences of "dx" to your project name, which should be the same as your index and ac3 basenames. you also may need to change your drive letter (i use p: for my project drive). finally, watch out for the path to the sda logo pngs in the statid subroutines (and, obviously, the LoadPlugin()s at the very top!). all the other options from lines 10-19 depend on the input video source and desired output quality.

enjoy!

more memory is always better

2007-02-10 13:25:43

big thanks to radix for maxing out v5 with 3 gb (it only had 1 gb before) ram this xmas.

i had long suspected that the embarrassing "access violation" errors spewed by avisynth were the result of running out of memory, but this was very difficult to test, as they tended to occur randomly, even under low memory conditions. one of the reasons i suspected it was overshooting while allocating was that on my macbook, now with 2 gb of ram, i didn't see the problem on the same scale as i did on v5. the only alternative was bad ram in both of the machines (since it had happened at least once on both).

i finally uncovered the connection last month during a series of tests i was running for OPERATION ZWEI, an sda-related secret project some members of the sda team are working on right now (expect to see its release sometime this summer). these tests involved extremely memory-intensive avisynth scripts, and i later decided not to accomplish the effect in this way, because the avisynth system is simply broken when it comes to memory allocation (thanks, doom9 guys!). however, this situation afforded me repeated chances to see the exact same "access violation" error pop up -- again, pseudo-randomly -- but this time while watching the task manager intently.

i was able to distinguish two different types of access violation errors -- one thrown when the avisynth mother process reached doze's 2 gb allocation limit and another when the process would attempt to allocate memory beyond the total (3 gb) available to doze. it was at this point that i realized that radix had basically saved m2k2sda from these pesky access violations making their way into final encodes, because under normal conditions i don't use more than 1 gb of ram on v5. surely 2 gb free should be enough for any normal encode to allocate freely without the danger of running over the edge.

and indeed, i haven't seen or heard of a new encode afflicted with the problem since installing the new ram.

more automation

2007-01-17 22:59:26

in the spirit of ballofsnow's megui-like batch files for sda, i've created a similar xvid.bat. this batch uses the vdub.exe command line driver to encode xvid videos loading m1.vcf, m2.vcf and l.vcf -- mq first pass, mq second pass and lq third pass, respectively.

the input syntax for this batch varies slightly from the others due to the need to insert LQ in varying places in the filename depending on whether the run is segmented or not. to help me remember it (and to speed up master batch creation), i've also created a generic.bat.

flickerbat ii

2007-01-17 21:27:05

more work on the deflickering problem followed the previous entry.

working together through most of the night, grenola and i shot ideas back and forth that culminated in a new avisynth function, nate_retard_bob_2(). this function corrects the "blur polarity" problem discussed at length in the previous entry by maintaining a consistent aspect ratio in clip processing. changing the aspect ratio was resulting in the top half of the output having more or less blur than the bottom half.

additionally, the new function includes a sharpen() invocation to (at least superficially) recover some of the original quality in the output. this trick works best on very clean (i.e. dvd) source material. when used on material that was recorded using vhs, it tends to only draw out the many problems with such a picture.

inspired by this productive night, i streamlined my generic-v.avs (from which all other avisynth scripts for sda are made), collecting some other blocks of interdependent code into functions i can more easily enable and disable as required.

nate_retard_bob_2() is available as a part of generic-v.avs.

flickerbat

2007-01-12 23:28:48

this entry is long overdue. i will now describe in detail what i know about the so-called deflickering problem and present my attempts at solving it to date.

first, a brief introduction. there is a tendency of interlaced displays to "flicker" when showing rough edges or straight lines. some of you may have noticed this in ties or other articles of clothing with complex patterns worn by news anchors--the patterns will seem to move or jump about rapidly. this is a side effect of the psychological trick interlaced displays use to approximate full motion--what is not moving seems to be moving.

video games suffer from flickering more so than your average tv program because they contain more of the types of material that make flickering obvious, such as straight lines (usually in menus) and rough edges (usually in text). some early attempts at deflickering involved switching the hardware from d4 (half resolution) to d1 (full resolution) when navigating menus. one notable game to do this was metal gear solid. changing to full resolution allows the game console to display different information in each field. flickering is reduced as the jagged edges are reduced. for this reason, some people refer to deflickering as "antialiasing", quite similar in lineage to the methods used to hide "jaggies" from discerning gamers' eyes. (by the way, deflickering can be "experienced" firsthand by entering the super smash bros. melee options and enabling and disabling the option. what looks better on an interlaced scanning display looks worse when it comes to progressive scan.)

there is nothing wrong with games running in full resolution. the missing motion information can be accurately reconstructed with an advanced deinterlacing algorithm such as mvbob(). further, as previously mentioned, "hybrid" games like metal gear solid usually run at f2 (half framerate) outside of menus, so i can simply drop one of the fields to simultaneously deinterlace and de-deflicker in these cases (no mvbob() necessary!).

however, another type of deflickering has recently become much more common with the rise in console emulation (that is, running emulators on consoles). sonic mega collection and the mega man anniversary collection both deflicker their output. they accomplish this by blurring one field a quarter pixel up and the other a quarter pixel down. i have taken pictures of this in action:



the right image occurs one half-frame after the left. that is, each image represents one field. if you turn your attention to the top non-black line in the left picture, you will notice that it is different from the line immediately below it. it is exactly 50% white (or, if you prefer, 50% black!) because it has been blurred with the preceding (black) line. a similar situation can be verified in the second picture by looking at its last non-black line--the only difference is that this time, the picture was blurred down, not up.

material that was once d4 is now effectively d1. as you can imagine, playing back this video without any form of de-deflickering will present an unacceptable level of "bobbing", if you wish to call it that. after pondering this sad situation for many hours, i concluded (with the help of grenola) that the only solution is to blur the down-blurred field up and the up-blurred field down, canceling out the deflickering (thanks go to grenola for this particular method):

converttorgb32

field1=SelectEven.PointResize(640,480).addborders(0,0,0,1).bilinearresize(320,240)

field2=SelectOdd.PointResize(640,480).addborders(0,1,0,0).bilinearresize(320,240)

Interleave(field1,field2)


unfortunately, things are not so simple. upon performing this so-called "retard bob" (so dubbed after "smart bob"), one field will seem clearer when it comes to certain objects, while the other will seem clearer when it comes to other objects. the exact cause of this behavior is not yet known, but one theory involves the quite imperfect horizontal analog representations of luma and chroma changes. that is, different brightness levels and colors at the source will produce different levels of blur after deflickering. the problem now moves beyond my ability to correct.

having said that, "retard bob" has improved things significantly:



mega man's energy bar was used as a reference for attempting blur level equalization. however, rush and the cloud maker towers at the bottom of the frame are clearly more blurry in the left (blur compensated) picture than in the right one. this would seem to dictate less blur compensation. however, applying any less blur compensation results in mega man's energy bar coming out of "blur sync" and "flickering" in the final product. this "neo-flickering" (or, if you prefer, "bobbing") is considerably more noticeable at f3 (third framerate) than at f1 (full framerate) for obvious reasons. unfortunately, the divx medium and low quality as well as the h.264 low quality of all 2d runs are encoded at f3 in order to ensure playability on old hardware (can't be f1) as well as to avoid problems with 60 hz blinking in most games (can't be f2). the ability to encode at f2 would neatly solve this entire situation, as i mentioned earlier when discussing metal gear solid, and this is precisely what i did for the recent sonic 3 & knuckles and sonic 1 runs submitted by mike89 (old school sonic is very special in that it contains no salient 60 hz blinking effects).

using gaussresize() i have settled on a "not-so-happy medium" blur (p) of 40:

converttorgb32

notblurred=SelectEven.PointResize(640,480).addborders(0,0,0,1).gaussresize(320,240,p=40)

blurred=SelectOdd.PointResize(640,480).addborders(0,1,0,0).bilinearresize(320,240)

Interleave(notblurred,blurred)


("notblurred" refers to the field needing blur compensation based on mega man's energy bar.)

however, as shown and explained above, some "neo-flickering" or "bobbing" will be visible no matter the blur compensation. for this reason, i am recommending that emulation be completely avoided when recording speed runs. as previously discussed, there are exceptions, mario 64 and similar 3d games constituting one such class (because the games run at d4 f2, i can simply drop one of the fields to neatly solve the problem).

concluding, i wish to mention that all game boy advance games played using the game boy player suffer from deflickering. i have employed many solutions over the years, including old school "retard bob" in virtualdub (even field quarter scanline up, odd field quarter scanline down in the "field bob" filter) and even mvbob. however, because virtually all gba games run at f1, none of these solutions is optimal for the above explicated reasons. the jury is still out on deflickering.

codecs++;

2006-11-11 13:20:18

joining me this month are the windows build of xvid from koepi.org and a patched ffmpeg build that transcodes to flv.

completely replacing divx with this build of xvid (as recommended by djgrenola) has finally liberated me from the long-standing vdub randomly closing problem. i don't know why it didn't occur to me to try xvid earlier, actually--i guess i was convinced i would lose something in the process of moving over, but that has not happened as far as i can tell. indeed, sda users have been watching xvid-encoded mq and lq avi for a while now without reporting any problems, so i have to consider this codec replacement a resounding success.

the new flv creation process recommended by mike hearn is still under evaluation, but i am encouraged by its fruits thus far. the only major sticking point for me has been that it demands that the input video be upside-down (flipped vertically), and so i must create a distinct .avs file for flv encoding now (as opposed to just using the mq mp4's), but this is really quite minor compared to never again getting shot down by one of the macromedia flash video encoder's myriad avs input bugs or having to set the video and audio bitrate every time i reboot the machine or wondering how much better quality/smaller of a file i could make had i multipass encoding--and i could go on.

the divx codec and the macromedia flash video encoder were dragging me down without a doubt, and i am grateful to the community for their continued support of my work.

destiny

2006-11-04 00:00:36

i like how i don't have to question anymore whether or not i'm really dedicated to sda--obviously, if i weren't dedicated, i wouldn't still be here.

of course, that's a positivist view. if i decided tomorrow that sda was not for me anymore, then suddenly i wouldn't look so dedicated, after all. but i like to force myself into the probably more realistic viewpoint that every second i spend working on sda means i'm that much more "into it".

things have been exciting around here lately. i'm sure y'all have noticed grenola and mike doing updates, and indeed, they are together doing the entire update process now, leaving radix to watch over his creation like a proud father and to occasionally yell, "no, no, you're doing it all wrong!" mike responds to runners, no matter how annoying, with super speed, and grenola has made short work of automating much of the remaining parts of the sda updating process. it's safe to say sda is operating better than never, ever before.

on the home front, i am very pleased with my new dvd recorder, an lg dr1f9h (say that five times fast!). media compatibility is far better than with my old panny, and as an added bonus, the lg writes dvd+rw without the need for a finalization step. that means that i went from using isobuster to raw read the vobs off of every dvd i recorded (because all the media i bought was incompatible with the panny) to not even needing to finalize the disc to take the vobs directly off of it in v5. and on top of all of that, the title corruption bug that precipitated "katamari hell" is nonexistent. good deal for only $200.

the lg has tons of other crazy features i'll probably never use (like dv in and hdmi out), but i think it's a good sign that i'm getting all these earmarks of a high-end dvd recorder at only $200 while over at walmart cheap (but still perfectly good) dvd recorders are down below $100. it's never been clearer to me that my prediction of dvd recorders meeting and then beating vcr prices will happen by this coming summer. that's significant, of course, because i plan to phase out my free vhs capture service at that time. the reason is because i must help the quality of the videos on the site in any way i can, and if that means coercing people into buying dvd recorders, then so be it. at $40 a pop they'll be less expensive than a single next generation game.

i'd like to take this opportunity to thank tom batchelor for his continued monetary support of sda. clearly he's not the only generous one around here, but he is obviously the one with the most money, and i appreciate his confidence in me and my operation. when other people believe in you, you often find a way to make things happen regardless of whether you share their confidence.

i will keep it coming.

vm heaven

2006-09-05 11:37:55

with the help of parallels desktop for mac, i've successfully doubled my studio's cpu capacity. running windoze xp as a guest os inside the virtual machine on my macbook, i appear to be encoding using x264 with little to no speed penalty. (windoze is essential to my encoding process because, among other reasons, i am dependent on avisynth and its filters, specifically mvbob().)

the software is $80 us, but i'm using their free 14-day trial right now (a wise business decision on their part, to offer said trial -- who in their right mind would purchase such expensive software without definitive proof beforehand that it does what they want?). running windoze on my macbook before was never very productive, and mostly consisted of me playing civilization iv. i had virtually no space left on the cramped internal drive for it, and my 250 gig external drive is formatted hfs+, so windoze couldn't even read it. of course, with a virtual machine, the "drive" is just a file, making the host machine's filesystem irrelevant.

this situation (sda annexing the macbook) became necessary when i realized how vastly superior -- and therefore essential -- the mvbob() deinterlacing filter was to its alternatives. with v5 completely tied up making huffyuv new master files of the large amount of d1 f1 runs that have come in recently, i simply needed more cpu power to work on actually encoding those files v5 was creating, lest i find myself still working on the same runs two weeks later, while others continue to flood in.

the setup isn't perfect. for one thing, the parallels "network adapter" seen by windoze xp is only 10 megabaud. this means that the huffyuv compression is just not sufficient to get the d1 f1 file over the network from v5 fast enough to saturate the macbook's cpu. however, i'm seeing about 80% average cpu use on the macbook with d1 encodes, so that's really no big problem. also, because the machine can encode mq and lq (d4) stuff more quickly than the d1 hq and iq, the cpu use goes down for those jobs, the fans spin down, and the machine gets a network-bottleneck-supplied respite from the heat. i believe that this is important, because the model of macbook i own has been shown to have a heat problem coming from a thermal paste application oversight on apple's part. i would fix the paste, but i would void my applecare warranty.

here's a screenshot for your viewing pleasure. note that one of the two "c" drives mounted on the os x desktop is in fact the virtual machine's c: "drive" (as broadcast via smb). transferring files back and forth is made quite easy in this way.

katamari hell

2006-08-23 20:51:58

over the past week or so i have been trying unsuccessfully to capture and encode the latest runs on the katamari damacy series of games.

the fun started when i received a large number of tapes, perhaps between twenty and thirty of them. each tape had between one and about four or five unique runs on it. in most cases these runs were no more than two or three minutes in length. the runs were also most often scattered throughout the tape.

i decided to immediately (on capture) follow the instructions sent by the runners and locate each run on its tape, recording with the dvd recorder only those important parts of each tape. thus, i would use only a handful of dvds to capture the entire project, rather than the matching twenty or thirty discs to twenty or thirty tapes. this would be my first major mistake.

radix came calling a few days later repeating a verifier report of extreme audio desync in the verification copies of the katamari material. i was shocked and appalled, but not nearly as much as i would have been had i not already had a hell of a week trying desperately to keep the requested old material flowing into the flash encoder (but that's another story). one more thing going wrong seemed almost like the natural flow of things.

i quickly determined that the audio started out in sync, but actually became progressively earlier as each disc wore on. i had seen this before when i had tried to index at the same time multiple vobs created at different times, but this couldn't be the case now - the vobs were all part of the one large whole my dvd recorder seems to like to write to the disc (for all discs regardless of length or content, it writes five equal sized vobs together representing all of the material on the disc).

the only other viable explanation meant that i would have to somehow correct audio desync caused by so-called "title" breaks, or breaks inserted when i would press stop and then later record again on the dvd recorder to capture another run.

i began to experiment with various pieces of software (vob2mpg, etc) trying to change the format of the vobs so that dgindex could successfully compensate for the title breaks. none of my experiments were successful, and so i created this thread at doom9. as a side note, my very favorite thing about using open source software is that, in cases of technical questions about the software, the author him- or herself is on hand not only to answer said questions faithfully, but also to recommend different software if the present software cannot do the job.

i followed neuron2's advice and used projectx (an impressive piece of work, by the way) on the katamari vobs. this produced vobs which, when indexed with dgindex, had in-sync audio!

but there was a problem - to make a long story short, it turned out projectx had actually deleted sections of the video to keep the audio in sync. besides being totally unacceptable from a speed running standpoint, this also caused me to have to completely remark the coordinates of all fifty something segments once again in order to get proper verification copies out to the verifiers.

unfortunately, i was not aware of the frame dropping issue introduced by projectx until a verifier reported it to me.

at this point i wondered whether i shouldn't have simply bitten the bullet and used all thirty something dvds to recapture the entire project without using more than one "title" per dvd. i immediately started recapturing the first tape in this way.

then i had an idea: why not get my old dvd player out (bought to test sda dvds before i sent them out way back when) and use it to make an analog copy of the dvds with the corrupted timestamp information? the resulting analog copies would be slightly inferior in terms of video quality (being mpeg-2 compressed twice), but they would each consist of only one title, eliminating all of my problems processing the material. i got out my old dvd player and began the copying process, switching over to a new spindle of dvd-rs.

then i found out that the new dvd-rs are incompatible with my dvd recorder.

at first i didn't want to believe it, because every type of high speed verbatim media i had bought before had been compatible. but all the signs were there: the long finalization process and the single, tiny vob and accompanying table of contents file that resulted.

not wanting to slap down $40 for new media that day, i went to take out my old dvd-rws with poor compatibility i had used with limited success in the early days of the dvd recorder.

but before i could attempt to copy the original katamari dvds again, i again had a thought: why not simply raw read the entire "incompatible" disc and scan the data thereon for vob headers? assuming the dvd recorder wrote the vobs in place (and being write-once media, i could imagine no differing scenario), i knew i would be able to recover the captured video regardless of the mangled table of contents file (created so because the dvd recorder couldn't read properly the media it had just written to). i had already done this once a few months before for kridly using simple unix software written by djgrenola for that sole purpose.

this time, though, i decided to check out ballofsnow's "recovering data from unfinalized dvds" links, and in them i found the wonderful windoze program, isobuster. this program was so impressive to me (it duplicated under doze the functionality and flexibility of grenola's work without any of the manual labor that had been required with the command-line unix version of the recovery process) that i bought it ($30 us) on the spot when it asked for a registration code to complete recovery of the vobs. i figured that it was a good deal, as another spindle of dvds would cost me at least $30, and isobuster would remove in a user-friendly fashion on the encoding machine the need to have compatible media for my dvd recorder.

with a viable capture process notwithstanding the incompatibility of my media i finished making analog copies of the original katamari dvds with corrupt timestamp information.

i next tried for several hours to create a script to "realign" the segment coordinate information from either the original capture attempt or the vobs as "repaired" by projectx. (i was trying to avoid having to mark all of the segment breaks for a third time.) this was my second major mistake, as after several hours of completely fruitless effort, i discovered that the corruption of the timestamps produced desync at every title break that not only was not linear in its growth - it actually went backwards (became more in sync) some of the time, completely destroying the viability of an automated realignment.

so i set out to mark all of the segment breaks a third time. it turned out that the timestamp corruption had also somehow caused entire runs to "slip through the cracks" in the previous encoding attempts - on the second disc alone, i have now marked over twice as many segments as the disc originally appeared to contain. how this is possible i am not sure, as i remember checking the output of the vobs with corrupted timestamps several times, and every time it totaled something reasonable, something just under two hours in length ...

lessons learned from this fiasco are many (i no longer have to have media compatible with my dvd recorder to capture, for one), but i still get the feeling that i in fact traded one lousy situation (capturing with the all-in-wonder) for a much more lousy situation (capturing with the dvd recorder). for example, it will be necessary in the future in the case of a thirty-tape project to use thirty dvds to capture the material.

the quality is better now, and perhaps that should be all that matters in the end - but i find myself time and time again in situations where i have to spend several days "learning my way out of a wet paper bag" as it were. this isn't rocket science, but without the necessary experience, people are just going to be totally screwed when they try to do this stuff themselves - especially if they get a dvd recorder as retarded as mine in this area (and mine is still recommended in the dvd recorder thread in the sda forum!).

one day this community will be delivered from this sorry state of technological affairs.

that day is not today.

the importance of labeling your tapes *properly*

2006-07-10 02:51:32

with the rush of submissions after the long hiatus finally winding down (or winding up, as the case may well be), i thought i would take some time to hunt through the wall of vhs in my bedroom closet for two runs apparently located on unlabeled tapes: ajbolt89's 45:42 hard 10% in metroid zero mission and pirate109's pal 22% in metroid prime in order to recapture them correctly:

the high quality of aj's run was improperly cropped and not fixed when i captured it a second time for that exact reason, while the high quality version of one of p109's segments (the one containing geothermal core) was not run through smart bob, resulting in a bobbing d2 picture, not to mention the fact that the current capture has only mono audio and is missing a few thousand frames due to a bad signal written by a bad vcr. i hope to correct the latter two more minor errors with my new stereo pal vcr (compliments of banjonator) and my full frame time base corrector, respectively.

both of these primary errors (the cropping and the failure to run smart bob) are critical, and i feel as though i do not have a choice about recapturing these runs, especially when radix's luigi's mansion run and 19duke84's 0:50:25 run in silent hill 2 were just recaptured with the new dvd recorder method for no other reason than 'being rather dark'.

back to the monster in my closet: 'unlabeled' in this context means only that the tape lacks a side label. any other kind of label is more or less useless to me, especially if the tape in question is in a box - having to actually take a tape out of its box, much less extract it from the wall in the first place, is something i should not have to do in the interest of not spending hours in my closet looking for a particular run.

all this only adds to feelings i've had recently that no one really has any clue what it's like to be me. it's clear to me that i have problems no one else on earth has ever dreamed of: if a tower of vhs tapes falls on me and no one hears me yell out, does anyone feel sorry for me?

it's important to distinguish, though, between intentional and unintentional harm. i'm sure no one who sent vhs tapes to me without side labels had any idea what a catastrophe they were contributing to. i will not embarrass them here, but some of the most popular and successful speed runners on the site are in fact guilty of labeling their tapes either insufficiently or not at all. in the case of aj, this run (his most recent submission) seems to be the only exception to an otherwise stellar labeling record ...

i started by the door and worked my way back through the closet. finding aj's run halfway through the final column was a mixed blessing - i was disappointed to find it near the end of all the tapes, but i was elated to find it with a descriptive top label, which saved me from going through each and every one of these:

(aj's tape is on the ground between the two groups.)

many in the unlabeled group also lack top labels, meaning i would have to put each tape into the uber vcr one at a time to discover what it might hold. actually, i have been considering doing this anyway, side labeling them as i go, in order to forever banish any further hour-long treks through vhsia as occurred tonight.

i remembered that p109's 22% run had been sent in on tapes enclosed in huge plastic cases, so for his run i only had to extract every tape in a large plastic enclosure. there were far fewer of these than unlabeleds, and i separated them out while searching for aj's.

i am comforted by the progress dvd recorders have made in the community over the past few months, cutting out not only the meddling, inferior nature of vhs, but in many cases, the role of snail mail in delivering the runs to me, as well. if a run is submitted only in its purest digital form - .vob files - then the role of archiving the originals falls to me, and i of course am solely responsible for the labeling methodology in that case.

hey, maybe it'll all make a crazy story to tell the grandchildren.

sda megui profiles

2006-07-08 05:14:01

as promised, here they are.

final settings for h.264 encoding

2006-07-06 14:46:55

i use x264 and megui. maybe can put up profiles later. for now:

lq: reduction to d4 where applicable
reduction to f2 (3d games) or f3 (sprite based games) where applicable
automated two pass 128 kbps
min quantizer 17
audio: le-aac cbr 64 kbps

mq ("normal quality"): reduction to d4 where applicable
reduction to f2 (3d games) or f3 (sprite based games) where applicable
automated two pass 512 kbps
min quantizer 17
audio: le-aac cbr 64 kbps

hq: automated two pass 2048 kbps
min quantizer 17 (d4) 19 (d1)
audio: le-aac cbr 128 kbps

iq: automated two pass 5000 kbps
min quantizer 17 (d4) 19 (d1)
audio: le-aac cbr 320 kbps

in case the average bitrates for hq and iq are the same, iq will be overwritten by hq, keeping the hq name.

new apt

2006-07-05 21:59:47

i'm not quite done moving into my new apartment yet, but the sda-related parts are pretty much complete.

here's the "wall of vhs" i've created in my closet (anyone care to estimate how many speed run tapes are up there right now?):



and here's the capture assembly (minus the new pal vcr) mostly set up in its new stand:

comments

2006-06-29 22:26:14

sorry, everyone, but i had to disable unregistered commenting on entries due to receiving 20+ spams a day. please spend five minutes registering with typekey if you would like to comment on an entry. thanks.

snow ftw

2006-06-16 02:19:21

using the following avisynth script (last line's the only relevant one):

AC3source(MPEG2source("C:\re4\1\re4_1.d2v"),"C:\re4\1\re4_1.ac3")

BilinearResize(640,480).Crop(0,0,0,470).AddBorders(0,0,0,10)

assumeframebased.complementparity


... i've managed for the first time to see the difference between the latest method of deinterlacing in use (the leakkernelbob avisynth plug-in) and actually ordering the fields so they appear in progressive order in an f2 (half interlaced framerate, full progressive framerate) clip.

i was expecting this new (reordering) method to come out on top, but i was mistaken:

cswap.PNG (new) deint.PNG (old)

for reference, leakkernelbob was called as follows:

LeakKernelBob(order=1,threshold=0,sharp=true,twoway=true,map=false).changefps(29.97)


while the two methods offer quite similar results, there seem to be more interlacing artifacts in the field swapping/reordering method, especially if you look at the ladder and leon's hair. while i'm not entirely sure what to attribute these artifacts to, i suspect they might have to do with how this video was captured (lossy mpeg-2 compressed file from dvd recorder). leakkernelbob gets rid of these artifacts somehow during the course of its operation, apparently.

i'm encouraged by this result, not only because it reaffirms my current methodology, but also because a constant f2 is not something i can necessarily count on in sources.

macbook

2006-05-24 22:49:29

so i got my new macbook today, and yes, this is sda-related, because i intend to process content as well as upload it using the new machine.

first of all, the pictures:



my first impressions:

May 24 09:45:20 jbg ok, here we go
May 24 09:45:23 jbg macbook coming out of box
May 24 09:45:28 SABERinPSYCHE go go go
May 24 09:45:36 jbg the box is extremely small
May 24 09:45:46 jbg like you think of a backpack's inside
May 24 09:48:16 jbg pics definitely going in blog
May 24 09:48:27 jbg this puppy is going to be producing runs, not just distributing them so
May 24 09:48:29 jbg it's appropriate :P
May 24 09:49:05 jbg and there it is
May 24 09:49:29 jbg feels smaller than my old one
May 24 09:49:39 jbg maybe 60-70% of the weight
May 24 09:50:45 jbg the battery has a full charge
May 24 09:50:47 SABERinPSYCHE thought you said "60-70% of my weight"
May 24 09:50:55 SABERinPSYCHE that's class
May 24 09:50:57 jbg the polycarbonate of the case is like a mirror
May 24 09:51:02 jbg like you think of the back of an ipod
May 24 09:51:16 SABERinPSYCHE hah
May 24 09:51:23 jbg it feels indestructible
May 24 09:51:23 SABERinPSYCHE mine was used
May 24 09:51:33 jbg but i guess we'll see about that won't we
May 24 09:51:44 SABERinPSYCHE looks like the Tails side of Two Face's coin
May 24 09:51:51 jbg ;-[
May 24 09:52:33 jbg keyboard feels almost identical to the old one (good)
May 24 09:53:02 jbg magnetic latch is excellent
May 24 09:53:24 jbg camera ran out of batteries ... oh well
May 24 09:53:42 jbg the feet are indeed part of the case
May 24 09:53:46 jbg time to boot up
May 24 09:53:50 SABERinPSYCHE yeah now how does that work
May 24 09:53:52 SABERinPSYCHE the latch
May 24 09:53:55 SABERinPSYCHE is it strong?
May 24 09:54:07 jbg the apple logo has appeared before the startup chime completed
May 24 09:54:23 jbg the machine is silent from a normal working distance
May 24 09:54:29 jbg i can't hear that the hard drive is there
May 24 09:54:38 jbg yeah it's quite strong
May 24 09:54:42 jbg won't come open accidentally
May 24 09:54:48 jbg have to jam your fingers in there and pull
May 24 09:55:06 jbg the machine feels small to me
May 24 09:55:15 jbg the screen aspect ratio is obvious
May 24 09:55:38 jbg language selection screen
May 24 09:55:49 jbg the computer just said "english" out loud
May 24 09:56:38 jbg english, japanese, french, german, spanish, italian, portuguese, dutch, swedish, danish, norwegian, finnish, simplified and traditional chinese, korean
May 24 09:57:16 jbg i just realized i didn't realize the screen is glossy ... good
May 24 09:59:16 Jimmy what's your blog?
May 24 09:59:43 Jimmy oh, incidentally, check out the 5th and final design for my website: http://www.exmembersof.com/
May 24 09:59:57 jbg i like how the trackpad is wide like the screen
May 24 10:00:03 jbg http://nate.metroid2002.com/blog
May 24 10:00:27 jbg looks sweet man
May 24 10:00:29 jbg good choice of color
May 24 10:01:44 jbg ah the command key on the right side of the spacebar is back
May 24 10:01:51 SABERinPSYCHE ugh
May 24 10:02:04 Jimmy do you use wordpress for your blog?
May 24 10:02:09 SABERinPSYCHE I mean I guess for versatility it's a good design call, but I'd never use that
May 24 10:02:17 SABERinPSYCHE meh on wordpress
May 24 10:02:28 SABERinPSYCHE it's good but cutenews just owns it
May 24 10:02:43 Jimmy well, I programmed my own database/php system
May 24 10:02:49 Jimmy I was just asking :P
May 24 10:02:56 Jimmy and what's wrong with my design?
May 24 10:02:56 jbg mt
May 24 10:03:34 SABERinPSYCHE no no
May 24 10:03:47 SABERinPSYCHE the cmd key being on the right side of the spacebar on nate's macbook
May 24 10:03:51 Jimmy oh
May 24 10:03:53 Jimmy :)
May 24 10:04:01 SABERinPSYCHE yeah yours looks fine
May 24 10:04:16 Jimmy I don't think I've _ever_ used the right command key
May 24 10:05:15 jbg it's for mac stuff like command period
May 24 10:05:30 jbg command comma etc
May 24 10:06:28 jbg i haven't noticed any dead pixels
May 24 10:06:48 jbg hahaha the snapshot it took of me is hilarious
May 24 10:06:50 jbg fakest of fake smiles
May 24 10:07:26 jbg go .mac ad
May 24 10:08:10 jbg haha
May 24 10:08:36 jbg ah and there's software update ;-[
May 24 10:08:53 jbg the smell is overwhelming
May 24 10:09:41 jbg easy to remember mac id
May 24 10:09:43 jbg on the wireless
May 24 10:10:53 jbg spotlight is much, much faster than on here
May 24 10:11:13 jbg hah you know i never thought of a name for it
May 24 10:11:58 jbg trackpad scrolling works great
May 24 10:12:02 jbg something i might actually use looks like
May 24 10:14:25 jbg 16.77 gig used, 54.44 gig available
May 24 10:14:33 jbg er oops that's the capacity sorry
May 24 10:14:39 jbg 38.68 gig available
May 24 10:14:45 jbg i got the smallest drive because it's so easy to upgrade ;-[
May 24 10:15:48 jbg the speakers are invisible
May 24 10:16:38 * inate (camera@reality.org) has joined
May 24 10:16:38 * Alizee sets modes [+oa inate inate]
May 24 10:16:56 inate the screen is very bright and clear ... i can read this even with my glasses on sitting from a normal distance
May 24 10:17:08 Jimmy all that video stuff is gibberish to me :\
May 24 10:17:18 inate heh yeah i get really technical
May 24 10:17:22 inate i'll post pics tonight
May 24 10:17:28 inate and then there can be something for you :P
May 24 10:17:40 Jimmy heh
May 24 10:17:50 inate i don't know why i'm running software update if i have to repartition to put doze on here
May 24 10:18:00 * inate goes to download bootcamp
May 24 10:18:23 inate applications open instantly
May 24 10:18:27 inate this is really bizarre
May 24 10:18:33 inate laptops are supposed to suck
May 24 10:18:43 Jimmy heh
May 24 10:19:07 inate safari reminds me of firefox on v5
May 24 10:19:16 inate which is good
May 24 10:20:29 inate Thank you for your interest in features of the next version of Mac OS X, Leopard. Donb
May 24 10:20:36 inate urg ircii
May 24 10:25:05 inate gmail works much better
May 24 10:28:33 inate insane
May 24 10:28:40 inate it says the existing mac os x partition remains intact
May 24 10:30:46 inate brb
May 24 10:34:55 * Jimmy awtches some muppet show before bed
May 24 10:34:59 Jimmy night guys
May 24 10:38:16 * Jimmy has quit (Connection reset by peer)
May 24 11:02:48 * inate has quit (Quit: Leaving)
May 24 11:14:46 jbg the magsafe power connector is genius
May 24 11:14:51 jbg i didn't realize that it's actually held in by the magnets
May 24 11:15:02 jbg so you can just kind of tug it and it comes "off"
May 24 11:15:03 jbg not "out"
May 24 11:15:26 jbg i guess what i mean is it's held on by the magnets
May 24 11:15:33 jbg like if you stick two magnets together
May 24 11:15:38 jbg that's what the power connector is like
May 24 11:16:16 jbg so this is how they solved the power adapter problems
May 24 11:23:17 * SABERinPSYCHE does the Protoman whistle
May 24 11:23:49 jbg :P

and finally, specs: 13" display, intel 2.0 ghz core duo cpu, 2x256 megs of ram (machine's achilles heel right now), dvd±rw (4x ±r write) optical drive, 60 gb 5400 rpm sata hard drive, white color.

jade empire

2006-04-30 18:17:52

here's the avisynth script i'm using to process the jade empire run (using the new leak kernel deinterlacer provided by ballofsnow):

clip=avisource("C:\je1.avi")


clip.BilinearResize(640,480).Crop(0,0,0,470) ¬

.AddBorders(0,0,0,10).LeakKernelBob(order=1 ¬

,threshold=16,sharp=true,twoway=true,map=false)


jade empire is interesting in that the framerate varies so wildly, all the way from f1 to f3 or even less. i continue to be satisfied with the results provided by the leak kernel deinterlacer, even given the difficult and unpredictable frame duplication present in this game.

video moving up and down one pixel every frame?

2006-04-28 18:44:04

no problem - just use this avisynth script:

clipp=avisource("C:\nmf\vob1.avi").assumefieldbased

clip1=SelectEven(clipp).Crop(0,0,0,240).AddBorders(0,0,0,0)

clip2=SelectOdd(clipp).Crop(0,0,0,239).AddBorders(0,1,0,0)

Interleave(clip1,clip2)


incidentally, this has been seen in runs submitted on dvd.

the transition begins in earnest

2006-04-28 12:31:59

having received as many runs on dvd as on vhs in the past two weeks, i am now confident that dvd is the best thing to ever happen to sda.

the difference in quality is just night and day, or whatever other clichéd analogy you'd like to use. processing runs (the actual work i do, anyway) is reduced by maybe 90%, not having to babysit everything while the analog capture takes place, and the benefits don't end there - think about how much easier it is for the encoder to handle video without the excessive amounts of noise apparently introduced only by vhs. the audio is cleaner (no denoising required in most cases), the brightness and contrast are always correct in the case of ntsc (black is black and white is white) and vhs hell who?

on account of dvd being proven so clearly superior to vhs, i have decided to phase out vhs capture starting on 1 june, when i'm finished moving into my new apartment. vhs capture services for sda through metroid 2002 will continue for one year from that date, but as of 1 june 2007, you will need to buy a dvd recorder to submit runs to sda. by that time, though, recorders should cost well under 100 dollars or euros, so it will not be an undue burden on you to come up with that between now and then. on top of that, i doubt that most retailers (except perhaps walmart) will even be selling vhs anything by that time, so it ought to be pretty easy for you to convince whomever you live with to invest in a dvd recorder.

r.i.p. akai

2006-04-10 21:39:52

the akai vs-g420 that faithfully served metroid 2002 speed demos archive for over two years died tonight. i used it to capture every pal run submitted to m2k2sda on vhs up to and including frezy_man's recent mm3 run.

despite the condition of frezy_man's tape, i do not believe that his tape killed the vcr. these symptoms:

tmnt.mp4 (1.6 MB; quicktime mp4)

... resemble the alignment problem the first uber (jvc hr-s9911u) that was defective had, and so i think that it might have simply been time for the akai to go. on top of that, because it uses european power, i have always had to use this enormous TEMPORARY USE ONLY power adapter for it, and i have no idea what kind of power it was giving the akai.

the good news for frezy_man is that i should be able to capture his tmnt run when i get a new pal vcr. the good news for everyone is that i will now be buying a 60 hz stereo pal vcr of a good reputation from one of my friends in europe. the bad news for me is that i will soon have a much lighter wallet.

but that's how it goes.

vhs hell iii: somebody shoot me

2006-04-09 17:29:37

this time it was the mega man 3 etc tape from freddy andersson (frezy_man).

attempting to fast forward and rewind the tape failed with tape spewing into the machine (the same thing that always happens no matter what goes wrong). i carefully extracted the tape and disassembled it. this was the scene (click the thumbnails for larger versions):



a third of the left tape reel had broken off and become caught in the rest of it, locking the tape in place. i carefully extracted the piece of the reel. then i noticed that another third of the reel was broken at the center in both places and was about ready to come off as well (it was only being held on by the outside of the wheel). i decided to try to use mailing tape to hold the wheel together better while i attempted to capture the run. i cut tiny pieces of mailing tape and applied them to the two broken center joints of the source reel.

i was able to capture the mm3 run, but i was careful not to fast forward the tape except while playing it (what i tell you to never do!), lest the precarious situation detonate. i've never replaced a tape reel before, and i wouldn't know the right way to go about attaching the tape to the new reel, not to mention how boring it would get winding an olympic-length european tape off by hand, so i was glad to avoid having to do that.

and the fun doesn't stop here! frezy_man forgot to tell me about this tmnt run he wanted to submit that is located near the beginning of the same tape, so the damaged reel will now have to take most of the tape back up ...

runner forgot to mark a segment?

2006-04-07 10:36:53

no problem ...


#!/bin/bash

i=1
j=1
while `test ${i} -le 41`
do
if `test ${j} -le 9`
then
ii=0${j}
else
ii=${j}
fi
if `test ${j} -eq 15`
then
mv -vi slyf-i-41.avi SlyCooper_255_part15_IQ.avi
mv -vi slyf-h-41.avi SlyCooper_255_part15_HQ.avi
mv -vi slyf-m-41.avi SlyCooper_255_part15.avi
mv -vi slyf-l-41.avi SlyCooper_255_part15_LQ.avi
else
mv -vi slyf-i-${i}.avi SlyCooper_255_part${ii}_IQ.avi
mv -vi slyf-h-${i}.avi SlyCooper_255_part${ii}_HQ.avi
mv -vi slyf-m-${i}.avi SlyCooper_255_part${ii}.avi
mv -vi slyf-l-${i}.avi SlyCooper_255_part${ii}_LQ.avi
i=`expr ${i} + 1`
fi
j=`expr ${j} + 1`
done

exit 0

revolution to the origin

2006-02-18 09:32:37

after applying the statid to the sonic 2 single-segment run caused a screech in the audio of the mq/lq and total sound desync in the hq/iq, i decided to reevaluate how i process runs.

the problem, according to radix, is that encoding a waveform to mp3 changes its length. this means that any videos appended to a video with mp3 audio will have problems with sound synchronization. unfortunately, because the run has to be sent to a verifier before the statid is applied in the overwhelming majority of cases, the prospect of encoding everything twice rears its ugly head.

radix suggested that i save each segment's audio to a file before encoding the video the first time. the implications of doing this, however, are not pretty. an insane level of automation would be required to prevent me from having to treat each segment of a multisegment run individually multiple times (first on the initial audio save and compress, then again when applying the statid).

what i'm trying now is actually running all of the filters on the segments, but then, instead of compressing them to divx, recompressing them to huffyuv. the processed (denoised/normalized) audio is also copied to the "new master files" at this time. for d4 runs, only one file is required; for d1, both a d1 and a d4 master file is required (the d1 for iq/hq and the d4 for mq/lq).

the most obvious drawback to this method is the large amount of disk space required. further, much of that disk space will be in use until the run is ready to be encoded with statid, and the verification process sometimes takes months.

advantages, however, are numerous: no more missed false starts at the beginning of segments with no keyframe in sight forcing a complete recapture of the run (with huffyuv, every frame is a keyframe), no more compressing everything only to find the run is rejected by the verifier (only 1-pass high quality would be created for verification purposes), no more desynced audio, and, most of all, no more six sets of operations i have to go through on the keyboard for each segment i want to encode (i/i/h/m/m/l). now, i just have to write a shell script to either write a jobs file or write a batch file for vdub's command line driver (i am undecided yet as to which would be the better option) that takes all of the "new master files" and performs the six operations on them. because the final version of the audio is included in each file, i don't have to worry about the location of the processed .wav file that came from the original captured video (indeed, it can be deleted as soon as the "new master files" are created). on top of that, i can safely apply the statid using the same shell script with little to no actual work on my part. plus, i only need one statid file for d4 video, which i only have to compress to huffyuv (right now i have to do two-pass encoding twice, once for the f1 statid and once for the f2 or f3 statid), and, to top it all off, the statid will now be encoded at the same bitrate as the rest of the file, eliminating potential problems with buggy players before they even start.

but do i have the disk space? a test with the recent legend of zelda new game plus run by trebor shows that the d4 "new master file" will take up almost as room as the original 720x480 capture. i can only assume that a d1 "new master file" would take up over twice the room of the original capture. this means over 100 gigs of space would be gone for a d1 run only two hours in length until the run could be verified. of course, a change in the verification process is all that would be necessary to largely negate this problem - i simply wouldn't capture the run until a verifier for it were lined up. i suppose i'll talk to radix when he wakes up.

avisynth to the rescue

2006-02-13 11:12:12

unable to find the time to get gbpvr working on v5, i've been forced to go back to the old dv bridge (the pre-summer 2004 capture device) to capture pal.

now, this isn't like the days of yore when i would capture pal nes and get several frames dropped every second - the standalone tbc ensures that few frames are dropped, and following from pal's superior image reproduction, the image quality is quite acceptable. a few minor problems still cropped up, but nothing warranting any further delays in releasing the now-ancient prince of persia run or the amazing new mega man runs from freddy andersson. i will simply replace the videos with better ones when i get gbpvr working.

incidentally, the delay there is not so much gbpvr's or my fault, but rather that gbpvr is exactly what it sounds like - a pvr, and not some random capture program like vdub. what i am trying to do - get it to simply display and capture s-video in - seems to be very difficult for it, and people are encouraged to look to other programs (such as graphedit or vdub) if they need to do what i need to do. the problem is, i have read that you can force the region using gbpvr with a tv wonder elite, and that is exactly what i need to do.

and now for the main focus of this entry ... upon opening the .avs file in virtualdub for the mega man 4 run, i noticed that it was jumping up and down exactly one pixel every other half-frame. thinking it was probably a symptom of either freddy's vcr, the horrid pal nes video signal, my old dv bridge, or maybe even all three, i googled for a solution and found this page.

using a .avs file inspired by the information at the bottom of that page, i was able to successfully correct the jumping without applying any quality-mangling filters in vdub. as always, i've provided before and after videos for your enjoyment.

that .avs file can be used to correct any f2 (half framerate) frequency jumping in any video. unfortunately, it can't help with insipidmuckywater's recent genesis runs, as that motion was far too irregular to even attempt an automated correction with a fixed measurement.

statid reborn

2006-02-01 12:51:11

they're much less exciting this time, but for whatever they're worth, statids (short for "station ids") now come with every new run released on sda through metroid 2002 (me). hopefully, they will cut down on other sites stealing the content and releasing it without crediting either me or (more importantly) the one who created the run.

here's a sample for your viewing pleasure:



it appears for five seconds before every segment of the run.

r.i.p. d.v.d. ii

2006-01-27 13:32:51

this time i'm stopping everything, so the data dvd service as well as the old video dvds will no longer be available.

this comes after the rousing success of our new host, which should ensure download speeds near or at the maximum of a given connection for new and popular runs. the greater use of bittorrent will also help people on poor connections get things more reliably.

i simply cannot justify making the data dvd service available any longer after all the time and money it has cost me, and especially how it has proven to be more of a hindrance than a help to the site. maybe someday, when we find someone as dedicated as we (radix and i) are to help with the site, data dvd duties could be assigned to them, but for now, i'm through with it.

and as the site grows more popular, the video dvds, which i was able to use as a source of income for the site only thanks to the obliviousness or the kindness of the game companies, make less and less sense. i can no longer ignore their capability to harm the site if some company hostile toward their existence finally gets wind of them, and orders have once again slowed to a trickle on top of that.

i apologize for any inconvenience this may cause. if you had your heart set on a specific video dvd, you may contact me via email, and i'll see what i can do.

don't let the door hit your huge butt on the way out, vhs

2006-01-21 20:53:38

so everyone is buying dvd recorders, and this is a good thing, but i'm sorry to report that they aren't buying them fast enough.

insipidmuckywater's brilliantly executed runs of genesis games i fondly remember almost met a cruel fate among the hundreds of interlocking parts that make up the guts of his vcr: witness the horror (DivX, 5.0 MB)!

this was after setting up everything so it was like the last time i encountered something like this. my guess is that neither of the tbcs could do much about it, because the image was rolling with a lower frequency than 60 hz - probably something more like 10 or 20 hz, and probably not constant.

luckily, an amazing virtualdub filter was all that was required to restore the video to an acceptable level of watchability (DivX, 6.2 MB). i used rather heavy-duty settings with the filter, turning everything up to the most cpu-intensive levels, meaning that these runs took an order of magnitude longer to process than they should have. however, given how short they are, i barely felt it.

anyone care to place a bet on whether vhs will nearly destroy someone else's runs before it dies a death that's more than fifteen years overdue?

just a quick note

2006-01-04 19:47:55

to show this pic of v5 in its environment:

SV300019.JPG

just brought the raid online, so that's ~480 gig of space at 2x normal speed. still no word on pal capture; been having enough hell as it is trying to keep capture working with all the crap hp installs on their machines. i'll probably wipe it tomorrow and install the 64-bit windoze xp, and i'll be done with those problems ... and probably create entirely new ones for myself.

woot

2005-12-23 09:05:56

tentatively going to say that this is the end of my problems with the tv wonder elite.

(i'm at work right now and have to wait until i get home to actually try this.)

v5 est arriv&eacute;

2005-12-21 12:14:21

the first thing i noticed about the new machine (well, other than the fact that it weighed thirty pounds) was that it was silent. and i don't mean quiet, i mean i couldn't even tell it was on when i was sitting three feet away from it. this is a radical change from my other computers, which all produce various levels of white, sometimes more greyish noise depending on their temperament at that particular moment. i thought that this must be why hp seems to be in love with amd - with intel you're unlikely to make a machine anyone in their right mind would ever call "quiet." (unless you're apple, anyway.)

the next thing that happened was that v4 (my old 1.1 ghz athlon) died. it was the damnedest thing - i was taking a disc out of the optical drive, and as the drive tray was retracting, it was as if the power was cut to the machine. the tray stood about two thirds of the way retracted, so i had to close it the rest of the way manually. i tried everything: plugging the machine in somewhere else, disconnecting the monitor and all the usb stuff, opening up the case and watching for dead fans, everything. no dice. it would turn on, but after two, maybe three seconds, it would go into what looked like a sleep or low power mode, with the power led still lit, yet none of the case buttons were responsive. i had to turn the power supply off to get it to turn off.

well, i thought ... v5 has no choice but to live up to the hype now!

so i dismantled both machines and transferred all my stuff over from v4 to v5. this included a floppy drive, my dvd burner, two maxtor 250 gig ata/133 disks, my old soundblaster live 5.1 and the new tv wonder elite. i was very impressed by hp's proprietary mounting system: you screw in the mounting screws on one side of the drive before you put the drive in, and then the drive slides down rails, and one of the screws is caught by a piece of green plastic that holds the drive in place. to remove the drive, you just lift up on the plastic catch and slide the drive out. because the case screws are also human-operable, this means that you can realistically do all kinds of work inside the computer without ever needing to touch a tool (except for some indoor plumbing or something to ground yourself first, of course).

i had to change the boot order in the bios (windoze kept trying to boot from v4's old c: drive and failing), but with that done, everything proceeded well. i tested out capture and dvd duplication and things looked good there, or at least as good as i expected them to ... v4's death is actually significant for at least one reason, that being that i as yet have not been able to capture pal properly using the new tv wonder elite. the problem is not that the card does not understand a pal signal; the problem is that i can't seem to change the output resolution of the card to anything other than 720x480 or (in ntsc mode) 720x240. this is unacceptable for pal, as the bottom (576 - 480) lines are cut off. i've tried asking in several places to get answers on this to no avail, but i'm still holding out for a solution lest i have to deal with returning the card and paying $200 more for a pci-x aiw (as explored in previous entries here).

luckily, i managed to capture the latest prince of persia run (pal) with v4 before it died. i'll have to rerecord and sync the audio, as the level mysteriously went up to several hundred percent higher than it was supposed to be about one third through the capture, but i'll take that over not being able to capture the run any day.

i've already packaged up the v4 motherboard and its agp aiw and addressed the box to radix, so he should get his "new" computer and capture assembly right after xmas. here's hoping it was the power supply that was bad and that i didn't just send him a dead motherboard ...

no change

2005-12-17 14:18:26

just got the tv wonder elite. thought i would do some quick tests comparing it to the old aiw in v4 (my old 1.1 ghz athlon) before i get v5 (the new sda beast), which, incidentally, just shipped today, almost a week early. because the free printer i got with the computer arrived in two days (with free shipping) and now this, i am under the impression that hp is nimble, especially unbelievably so now during the height of the holiday season. then again, maybe i'm playing captain kirk to hp's scotty - they promise me a build date for the computer of 23 december, and then they ship it on the 17th. go miracle worker. anyway, here's hoping v5 comes before i leave for my parents' on the 24th; otherwise, i'll have to wait until i get back on the 27th to take it out of the box.

as for the tests, here's the old all-in-wonder and here's the new tv wonder elite. radix was able to distinguish that the gradient from black to blue on the left side of the image is smoother with the tv wonder elite, but with those filter settings on the tv wonder, it looks like i will be neither gaining nor losing any quality going to the new card. very good news, considering the plan b was $200 more expensive.

upgrade + tax

2005-12-14 23:32:14

decided on the ati tv wonder elite instead of the 3x more expensive all-in-wonder. interesting about this card are two things: it supposedly has ati's latest tuner, which makes analog cable signals look like they came off of a dvd, and there are apparently mac drivers for it. perhaps i will be able to capture in os x on the new sda beast sooner rather than later.

i bought the card today from buy.com for $106.66 and the machine from hp for $1099.00 ($949.00 after rebates). full specs on the machine below:

HP Pavilion d4100e customizable Desktop PC
* - Microsoft(R) Windows(R) XP Home Edition with SP2
* - AMD Athlon(TM) 64 X2 4200+ dual-core - 2.2GHz
* - FREE UPGRADE from 512MB to 1024MB 400MHz (2x512)
* - 80GB 7200 rpm SATA Hard Drive
* - FREE UPGRADE from CD-RW to 16X DVD+/-R/RW
* - Front USB, FireWire IEEE 1394 and audio ports
* - NVIDIA GeForce 6200SE w/TurboCache 256MB support
* - Integrated 5.1 Capable Sound w/ front audio ports
* - HP Multimedia Keyboard, HP Optical Mouse

will be adding to it either two or three of my maxtor 250 gig ata/133 drives, my sony dru-710a dual layer burner, my soundblaster and the tv wonder elite. two of the maxtor 250s will form a raid 0, because i highly doubt that one of them by itself can serve the huffyuv fast enough to avoid becoming a bottleneck. (with the newly-released divx 6.1 it'll use both cores for a single job, increasing my encoding speed from amd 3400 to amd 8400.) operating systems will be windoze xp 64-bit and mac os x, as soon as they crack the release version of x86 tiger so that it runs on non-apple-blessed hardware.

the journey continues

2005-12-11 13:02:48

ladies and gentlemen, it's upgrade time here at m2k2sda.

my roommate, rachel, has decided to take back her pc, which i have used for the past sixteen months or so to process all of the incoming material for the sites. i should probably give some background about how this situation arose.

when we were moving into the new apartment in august of 2004, it became clear to us that i was the one who needed the insane speed of her new pc, which she had just built with her parents' money. therefore, i traded with her, loaning her my old athlon 1.1 ghz for her athlon 64 3400+. i ended up upgrading the machine several times, giving it a second dvd writer and several 250 gig hard drives. i never really liked the thing, though, mostly because it wasn't mine, and because it had a few kooky hardware problems that i was never able to track down (it doesn't help when running the program inside a debugger stops it from crashing).

a few days ago, she announced that she had bought quake 4 and wanted her machine back to play it. i knew that my old pc is actually something like eight times slower than hers, meaning that it would take eight days to export a run that takes just over 24 hours to export now. it was time to kick my upgrade plans into high gear.

i had been saving since classes started this semester for a new "sda beast," but i had been planning on waiting until january or even later to actually buy it. i spent about an hour on the phone with my parents last night, and i was able to get them to push my graduation gift back from may to the present time, ensuring that i can buy the new machine now, and averting potential disaster as runs continue to come in at a rate that is certainly brisker than one per week.

but what should it be? dad was quick to point out that since 2004, prices on custom built brand-name units have continued to fall, but prices on individual parts (such as cpus) have not. we seem to have hit a wall where it's cheaper to buy a bare bones pc from hp or dell than to buy the exact same parts and assemble it yourself. he said that he thought it was because the "microsoft tax" the pc manufacturers pay has gone down so much, but to me the reason is irrelevant; i just wanted to know how i could get the most power for the least money.

and so i went to the hp site and customized a d4100e series pavilion, the highest powered one i saw with amd inside. (as anyone following the cpu wars knows, intel is not a good buy right now.) i selected the fastest cpu available, an amd 64 dual core 4200+. it came to about $1,100 after a buttload of rebates. i agreed to pay $400 of it, and my parents would shoulder the rest. but then i noticed something: there was no agp slot on this motherboard! my all-in-wonder capture card is agp ... so that meant either back to the drawing board or buying a new capture card.

luckily, it looks like i can get the all-in-wonder x800 xl (a pci express 16x card) for less than $150, and my parents also agreed to pay for it. because i'll be at my parents' around christmas, i decided to wait a few days to ensure that the new sda beast is not delivered during that time and has to wait in the cold for me to come home. thus, i'll have my first reports and captures from the new machine right at the end of the year. at this point i can only imagine what improvements the folks at ati have made to their all-in-wonders over the past two years.

incidentally, radix is probably the biggest winner in all of this: he gets my old 1.1 ghz athlon with motherboard and all-in-wonder. what's so special about that, you ask? well, if you weren't aware, he's currently using a 450 mhz computer ... a cd-rom drive that can't even read 700 mb discs ... and a capture device ... well, i wouldn't even call that thing a capture device. suffice to say that he's going to have a blast capturing his own awesome runs with my old hardware.

r.i.p. d.v.d.

2005-11-02 12:18:11

i started this blog fully intending to limit its focus to m2k2sda-related entries. however, today i feel that i need to divulge some background information in order to adequately explain a decision i have made: newly submitted speed runs will no longer go on video dvd for sale at dvd.metroid2002.com.

as some of you may be aware, radix's life has taken a turn for the worse over the last couple months. mine, by contrast, has rarely been so great. thanks to a well-timed encounter with a professor, i finally know now what i am going to do with my life. as a result, school has actually become something i look forward to and enjoy now. ironically, this blissful obsession with things outside of the sites serves the same function as radix's misery: "the site will suffer."

i was thinking the other day about how many runs i had sitting on my desk waiting to be captured, and i wondered how many of their dvd versions would actually be sold in my lifetime. i knew that the answer, at least for the foreseeable future, would be "not many." in fact, i haven't gotten a single order in several weeks. that said to me that i would soon spend the equivalent of several days working on dvd masters that would sit in storage until the end of time.

i knew when i raised the prices on the products that it might kill the whole operation, and i knew that if it did, i would have to swallow the fact that our worthless capitalist society is not capable of sustaining me doing this video dvd thing. of course, it's much harder for me to accept it now than it was to think about it then. a part of me wants to say "no" to giving up and press ahead toward some unseen salvation. but i know that such delusional acts can be dangerous. i can do my best for you only by carefully evaluating my options and then picking the most logical one, regardless of how i feel about it.

please do not misunderstand and think that m2k2sda is in financial straights or something. that is far from the truth. the sites are quite self-sufficient monetarily thanks to our advertising partner, google. the dvds, taken by themselves, however, are not. the market is simply too small to warrant production of these things at this time. i will continue to make the existing video dvds available for purchase, and all the runs on the site will continue to be available on data dvd upon request.

this effectively cuts down the amount of work i have to do for each submitted run by about two thirds, so hopefully you will notice a decrease in the time from submission to posting after the runs currently in the queue are taken care of.

i apologize profusely to anyone who thought that their run, currently in the queue, was going to appear on video dvd.

thank you for reading.

kaeru

2005-10-28 18:57:35

new power supply is installed and functioning normally.

$58.99

2005-10-26 10:04:36

ordered a replacement 400w power supply last night with two-day shipping. it hasn't shipped yet. (the original was 480w, but that's like twice as expensive, so i figure i'll have to make do without.)

just got a new email while writing this saying it had shipped. so i'll expect it on friday.

out of the frying pan ...

2005-10-24 10:40:19

now the power supply died.

is this why the drives died?

the solid-state era can not come soon enough

2005-10-16 11:12:20

had two drives die in as many weeks.

first one was a worst-case scenario, the c: drive in the capture/encode/burn machine. luckily i had a notion it was going when i opened the case trying to troubleshoot this crap (thought maybe if i pulled one of the sticks of 512 meg and it stopped, the troubleshooting would be over). with the case open i could hear that a drive was "thunking" (louder than clicking) and that this coincided with a complete freeze of the software on the machine. definitely a drive dying, but i had to disconnect each one one at a time to determine that it was indeed the c: drive.

it took several tries to boot successfully after this, and i knew the end was near, so i stopped everything and tried to get everything off of the c: drive. unfortunately, this couldn't have happened at a worse time - i was right in the middle of importing dvd master files to the mac to prepare them for burning, so the mac had little room to spare. luckily i was able to get everything off in time, even given the extra burden of holding on to all those dvd master files at the same time. it was pretty scary, though, as i was forced to use the raid to hold important data, and the last time i played with disconnecting drives, the raid controller forgot about the raid and i lost all the data on it (that was what happened to my ecco speed run, if anyone was wondering about that).

i was pretty peeved when i found out that this drive was in fact over a year old (they sold 250 gig drives two years ago!?) and as such i would not be getting a free replacement from maxtor. thus, i was forced to slim down even more and take everything off of the old g: drive (also a maxtor 250, but much newer than c:) and format it to make it the new c:. this process took several days, and in the end i was forced to simply format over the dvd master for the adventures of lolo 3 run (i decided to recapture it from the vhs later for dvd).

my parents took pity on me and notified me of a deal at compusa last weekend where i could get another maxtor 250 for $80 after instant rebate, and said that they would pay for it. that got me back up to the 750 gig in the machine i need to comfortably store all of the versions of more than one run at the same time (e.g. huffyuv original, rgb16 field dominance corrected dvd master and divx hq/mq/lq).

then, yesterday, i was absolutely thrilled to hear the same thunking sound coming from the machine again. this time it was one of the two sata drives that together make up the ~250 gig raid0 i use to capture to. indeed, the seventh segment of the pal legend of mana run was shown to have several seconds of nearly 100% dropped frames - a sign that the raid had been going out earlier than yesterday. useless as a capture drive and even more useless as a place to put important data (e.g. hq/mq/lq divx files), i disconnected the raid this morning so that the machine could successfully boot.

so, i'm down to 500 gig after all, and probably for some time. submissions are coming in much less quickly now, and so i think if i'm really diligent about getting dvd versions burned immediately after i capture something (i.e. the same day), it shouldn't impede my ability to quickly make runs ready for the verification stage. but submissions can't reach the level they did last summer. radix is already quite favorable to the idea of making the submission guidelines more stringent, and i think this might be the straw that breaks the camel's back in that area.

framerate

2005-09-29 18:19:49

cpu constraints force me to most of the time only release full framerate (usually 59.94 fps in ntsc and 50 fps in pal) video in the hq. and for 2d games that likely employ 60 hz blinking effects (basically every 2d game ever made up until a few years ago) i have to cut the framerate down even further to 19.98 fps (ntsc) or ~16.67 (pal) in the mq/lq to avoid invisible sprites/no blinking. that means that even though the mario 2 run has issues, the hq still looks way better. imo, once you get hooked on full framerate, you can't go back.

when it's possible (i.e. i have enough cpu time), i try to release a fourth quality retaining the full framerate, but usually having half the bitrate of hq. this won't help more people play it, as it's the framerate that is usually the problem, but it will cut the download time in half. this is never done with d1 runs as i feel they need all 4 megabaud to look decent. in situations where i release all four qualities, the new quality steals the name 'high quality' (so ~2 megabaud), while what used to be high quality becomes 'insane quality' (~4 megabaud). can't remember if such a run has actually been released on the site as yet. it should become more common, as submissions have starkly dropped in the past few weeks.

also, if you've had trouble playing hq in the past, you might try downloading hq of a d4 run (320x240 in ntsc; usually anything done on a system released before 1999 except saturn) to see if that plays, as even my ancient power mac g4 dual 1 ghz (using only one of the cpus) can play that fine, while it chokes on d1.

it begins

2005-09-15 19:49:51

recapturing tape 1 of 2 of the 0:55 now.

it's a surprisingly, abnormally bright tape ... during calibration i had to turn the brightness all the way down to 135 - that's 50% of the normal correction (usually 138, with 128 as the default), my "one-size-fits-all" solution so that i don't have to recalibrate the brightness for every single run i get. i guess i was unwittingly quite lucky to have her back in the days of the dv bridge with its permanently weak brightness pickup.

the picture looks spectacular ... way better than i'd hoped. also, i don't have to go through the receiver to lower the audio level, which is great news, as well. i wish there were something i could do for her that would make her feel as good as i do right now because of how perfect her stuff is. i mean, this isn't like, "oh, this run would be perfect IF ..."

this run is perfect. period.

vhs-hell ii: the sequel

2005-09-12 21:18:21

i encountered what was easily the worst video signal i've ever seen today - and my expensive equipment rose to the task.

this was the mario 2 all levels run. at first when i saw it i thought there was no way i was going to be able to do it, because the vsync was virtually gone, so the image was rotating vertically every few seconds. it later turned out that it was only doing that in certain spots, but then there was still that the odd field was several lines higher than the even field, for a truly seizure-inducing 60 hz vibration of the picture. i tried dropping every other half-frame (making 29.97 fps, or 'f2' as i call it), and that solved it, but there would be no chance of a high quality version if i went with that solution.

then i tried capturing without the standalone tbc just to see what would happen. the shuddering problem (where the odd field was several lines higher than the even field) was gone, but i couldn't capture anything - without the standalone tbc's help, up to half the frames were being dropped by the capture card. the tbc would be necessary to capture this run.

so i played with the settings on the uber vcr a bit and tried turning on the "stabilizer" (which disengages the uber's tbc in exchange for a processor that tries to hold the vsync on the picture). bingo - the image was stabile through the standalone tbc, which allowed it to be captured. it's still really nothing to look at - distortion everywhere - but the fact that i didn't drop any frames due to the bad signal means there can be a full framerate (hq) version and a dvd version of this run.

here's to my over $600 in technology truly being worth every penny once and for all.

another productive day

2005-09-11 21:46:48

finished capturing all of the runs in the queue today, as well as recapturing the final run in the 'tertiary group' (runs already on dvd but needing recapturing) - d.darch's single-segment 1% in fusion (dvd #001).

everything should be imported into imovie and ready for encoding by this time tomorrow: approximately five to ten dvds will be appearing soon with most of the recent content from vhs, as well as a redone #001.

i am eagerly anticipating finally recapturing scarlet's 0:55 100% tomorrow in the first step toward the #003 special edition dvd set. this blog will probably turn into 'scarlet central' in the coming weeks as my focus shifts from capturing and encoding new runs to working on her dvds. the number of ideas i have for it is already quite insane, and tons of stuff always pops up while i'm working on projects like it, so it ought to be a blast. this should be the most awesome thing i have created in my entire life: success is a foregone conclusion when i am this enthusiastic.

also today radix discovered the lost 'divx data files location' option in the 'advanced' settings of divx 6 - it had been 'lost' since divx 5 - but, unfortunately, even setting it to a different location did not allow me to successfully run more than one multipass job at a time, as every instance of virtualdub always crashed with a divx out of bounds memory access violation at the end of the job. radix recommended running the vdubs as different users to try to mitigate this, but i'll have to play with it some other time.

vhs-hell (oder: ich bin ein kluger held)

2005-09-10 13:11:00

well, today marked the largest vhs catastrophe to date, and the closest i've come to losing a run.


when i began to fast forward the second tape of the single seg legend of mana run (i always fast forward and rewind every tape before capturing it to get rid of any sticky spots in the tape), it made a horribly loud noise, like putting your ear next to a motorcycle engine or something. i am not exaggerating at all when i say this noise was far louder than i ever would have expected could be produced by a vhs unit. the noise grew in pitch and volume as the tape sped up, and i feared that something bad was about to happen, so i decided to stop the tape and take it out.

as i tried to pull the tape out of the vcr, my worst fears were realized - the tape was outside of its enclosure and caught somewhere inside the unit. i tried reinserting the tape to see if it would come uncaught the next i ejected it. no luck, but there was more tape outside of the enclosure now, enough to pull the enclosure completely out of the vcr and reach in with my hands to try to dislodge the tape from the inner workings of the unit. luckily, the tape was apparently only caught in one place, and i was able to quickly disentangle it and extract the whole mess from the unit.

once removed, i used my trusty dvd labeling pen to turn the tape's uptake wheel. unfortunately, the tape stopped coming when all the slack was on the uptake wheel. i tried turning the uptake wheel harder, but the source wheel wouldn't budge. i got out my screwdriver set and opened the carriage.

the inside of the enclosure looked normal enough. however, as i turned it over (the screws are on the bottom of vhs tapes for some reason), pieces of the one-way locking mechanism fell out of the carriage. so i had to hunt around for a tiny, thin spring and the white plastic piece it held in place to reassemble the carriage. (the one-way locking mechanism consists of two pieces of plastic and two springs that fit over them. these plastic pieces act like gears touching the two reels, making sure that the tape does not move backwards unless it is being rewound.) after this reassembly i was able to get the source wheel turning again.

when i put the reassembled tape back in the vcr, it made it to the end of the tape fine, but when it tried to rewind, it ate the tape again. i wondered if i hadn't incorrectly reassembled the one-way lock. but some tape was on the source wheel ... so it had successfully rewound maybe a minute or two. that said to me there was something wrong with the carriage outside of the one-way locking mechanism (either that or the uber vcr had failed). i left for work not knowing if the tape would be capturable.

when i got back from work, i opened up a brand new tape i had and began to transplant the legend of mana tape into its carriage. but when i tried to take the legend of mana tape out of its original carriage, i noticed that the source wheel was caught on something. it all made sense when i saw that the metal seesaw-like strip used to hold the reels flush with the carriage had partially broken off of the carriage and was leaning down into the reel. the seesaw strip had become caught in the source reel! no wonder it couldn't turn ... and that also explained where the motorcycle engine noise had been coming from (the seesaw thrashing against the source wheel as the tape wound).

i carefully extracted the seesaw strip from the source reel and set the tape down into the new carriage. the second half of the run is capturing normally now. i expect that there will be a considerable decline in the picture quality in the area of tape that had come outside of the enclosure the first time the vcr ate the tape, though i don't have to worry about the second area, as it was after the run finished on the tape.

dvds, part 3

2005-09-03 18:39:56

just formally launched the new data dvd service and raised the prices on the existing products to $15 each with no bulk discount. hopefully it will go over well.

too loud

2005-09-03 16:04:04

radix pointed out some audio overpeaking in the capture of sparky's new metroid prime frigate escape record. turns out that my jvc hr-s9911u (the "uber" vcr) has all kinds of controls, but not a playback volume control. therefore, it was necessary to connect an old sony receiver between the full frame tbc (datavideo tbc-1000) and the all in wonder. i'm not happy with this solution, because the audio has to go through two adapters to get back to rca and to the card (i had to use the receiver's headphone port to be able to control the output volume), but it will work for now. the problem is not actually with the aiw, but with my capture machine's crappy built in sound hardware (the lowest recording level for the line in is too high).

looking forward to trying out the tbc with a pal tape soon (as i don't have a built-in tbc on my pal vcr).

static audio adjustment in vdub

2005-08-29 11:06:19

radix taught me today how to correct a static (unchanging) audio offset error in an avi using vdub.

you just go to audio -> interleaving and change the "delay audio track by" number to either a positive or a negative number. then hit ok and change both video -> and audio -> to direct stream copy, and save as a new avi. you'll need to check the new avi to see if you used too much or too little correction. i used numbers from about -20 ms up to -200 ms to correct the progressively worsening (yet static within each file) audio offset errors in smilingjack13's hard 22% run's later segments. (see the previous entry for more on how these errors were introduced.)

all in all i'm quite pleased with this, as it saved me not only from having to set up new jobs for all of the affected segments, but also from having to encode them to divx again as well (cpu cycles are one of my most precious resources here).

d1 vs d2 audio update

2005-08-28 13:55:02

so it turns out the audio got progressively further behind in the dvd master as well as the divx master, just not as severely so. this means that i'll need to process each file's audio individually (which won't be quite as bad as it could have been were normalization necessary in this run) as well as correct the offset in each segment in imovie before i sent the project to idvd. will think twice about capturing in multiple files in the future, even though the disk space wasted in the process may be extreme.

d1 versus d2 audio

2005-08-27 13:14:15

just learned the avisynth syntax for joining multiple files today: 
clip = avisource("file1.avi", "file2.avi")
... and so i used it to join all of the smilingjack13 hard 22% files (most with one segment per file) (the reason i do this is so i only have to process the audio once, rather than once per source file).

unfortunately, as i was marking the segment boundaries for the five divx passes in the appended mess in vdub, it sounded as though the audio was getting progressively further and further behind. will have to check out one of the final files to verify this, but if true, it would mean that i would have to process the audio twice for such projects - once for the dvd master (d1) and once for the divx master (d2 (field split via avisynth)). most times that wouldn't be much of an issue, but for complex audio with lots of normalization false positives (i.e. loud cable tv programming wedged between segments, loud pops from pesky vhs hardware), it will be a pain.

my theory is that the d1 audio didn't match the d2 frames because the d2 appended mess's framerate was calculated all from the first file's framerate by avisynth, and i guess the later files had a slightly slower framerate than the first one did. perhaps the new tbc will help with this.

extend luma black point is a no go

2005-08-27 00:54:47

so i was reading phaeron's blog and iirc he said he had the 'extend luma black/white point' feature in 1.6.x in order to correct issues with the old ati all in wonder dropping detail in dark areas. sounds like me, right?

well, i decided that this looked better than this, so i'll be sticking to brightness +10 (138) with the old drivers for any adjustments i make on capture. as far as i can tell, that luma black point adjustment is just a fancy term for a preset contrast adjustment, and there are big problems with doing it that way, especially for the divx encoder (wasted bits keeping track of noise in the margins that's not all #000000, etc). the detail regained in such an adjustment was just not worth how ugly the adjustment made the rest of the picture.

also, i received my new datavideo tbc-1000 yesterday, and this sonic adventure dx (ntsc) run is the first thing captured with it. i'm dropping about 1 frame every 2 minutes with a dvd master importing into imovie over the network. however, i'm pretty sure that the dropped frames are still the drivers' problem. too bad the newer drivers don't work for crap with vdub on this machine. i'll try to remember to post an update on whether the dropped frames are 'extra' (vdub reports 'no next dropped frame found' when hitting }) or actual lost information.

edit: looks like the dropped frames were "real" after all, about 70 out of 204776 captured.