sda capture blog

Uhh...

This is for the output of avisynth from DVD deinterlaced and the input of the final encode, right?

1) The screenshots show contrast loss. For example in the 3rd one the ball reached true yellow (FFFF00) whereas in the encoded one is stays a little lower. It's easily visible.

2) 2-pass means doing MVBob twice. The time lost will be higher than the one saved...

3) It provides inconsistent quality, as with any bitrate-based encoding. For example, in the newest Soldier of fortune 2 speedrun, some jungle segments still look terrible at insane quality, as 5mbps fall very short.

If the settings are the same you will be using,

4) Exhaustive motion search? That's crazy and useless. Doesn't it slow down the encoding several times?

5) B-frames and B-pyramid in AVI? These settings will cause a 4-frame lag (2 for the encoder to do 2-bframes, 1 for the AVIfile decoder for b-frames and the last one for b-pyramid), which will cause AV-desync (and is ugly anyway). If using directshow, it'll still be a 2-frame delay.

Why do you need such small filesize, when you were doing fine except for the bandwidth? Fixed quant at 5 or 10 or so should be indistinguishable and avoid the disk bottleneck.

If you are going to use AVI, by all means, disable B-frames, otherwise it's just more trouble. Open the file you just posted in VDub: The first 2 frames are dropped frames. If you advance frame by frame, the first keyframe will appear, but then it'll freeze for another 2 frames, for a total delay of 4, as I explaned above.

For more speed, you could try XviD, 1 pass quality mode, disable B-frames, disable trellis, and set the quantizer to 2. This will be much faster for encoding and decoding, more robust for AVI, and still provide a reasonable filesize. You don't need (or want) the hyper-complexity of H264 for this sort of stuff.

thanks very much for your comments. this is still a work in progress as i only found out after i wrote the entry how much of a hack x264 in vfw is. unfortunately i haven't yet found a superior replacement, believe it or not. i can't use commandline because mp4box won't mux lossless audio and ffmpeg overflows and hangs encoding x264, and i can't use avidemux because it won't open avs.

1) yes, but is this compounded with each encode?
2) it's a very complex issue made even more complex by the fact that mvbob is limited to one core. therefore for single segment stuff i'm not sure there would really be any more time lost versus going through a temporary lagarith (remember that my cpus are so fast that even lagarith will read lock the disk). mvbob twice is nothing compared to mvbob 14 times or the disk read locking and choking the cpus 14 times (standard 7 qualities, 2 passes each).
3) inconsistent versus ... constant quality? i have to admit i'm not quite sure how cq even works, so i was afraid to try it. also keep in mind that sda iq is 5 megabaud while this would be 10.
4) again, cpu use is irrelevant coming from lagarith because the disk read locks. less insane settings would be used if this disk bandwidth thing were not already in my way.
5) yeah, i found out about the unfortunate state of b-frames into avi after i committed this entry. again, i'm not sure what to do at this point. b-frames are my best friend for game footage.

as i said before, rate can't go much above 10 megabaud without losing efficiency due to the storage bottleneck, and so the goal is to make good lossy nmf at that rate.

xvid? seriously? i'm not sure i can handle the thought of fixed macroblock size on white text on a black background. just seems like shooting myself in the foot especially for all digital captures.

thanks again for the reply.

I don't understand how can you be so IO-bottlenecked. Perhaps there's something I don't know about your setup, but:

10mbps = 1.25 megabytes / second

Now let's assume the hard disk can provide 40 mbytes/s (a rather conservative estimate). For a 30fps source, you'd have to be encoding at 960fps for this to be the bottleneck. For a 60fps source, at 1920fps.

I can believe a lossless source can be a bottleneck (the lagarith encode you mention in the post is 115mbps), but for a lossy one, I think you should be able to go much higher (than 10mbps). Then there's the issue the encoding is VBR, which means for a given bitrate some parts will "bottleneck" more than others as bitrate varies...

1) I have no idea, you'll have to compare the final encode with the source for contrast loss...

2&3) Constant quantizer at 10-15 or so, or whatever gives you something in the neighborhood of 10-20mbps for "normal" sources... My point is that since you don't need a given filesize, 2 pass is overkill (and suboptimal since it gives you a size at the expense of varying quality, but you ideally want good quality always at the expense of varying size). Of course, with constant quantizer, bitrate will vary a lot depending on content, but at this stage bitrate isn't important as long as it is much less than lossless.

I see this "lossy nmf" thing as a way to speed up things overall, but I don't think obsessing over not being bottlenecked is good, even at 50mbps you'd still be 2x as fast as lagarith, is it really worth sacrificing quality for the sake of not being bottlenecked in some cases?

4) But this will not come from lagarith, it'll come from DVD->MVBob, right? esa is not really better than umh (in fact, it can very well be worse), and IIRC it was several times slower...

5) Same thing I've said... I don't think low bitrate is so important, you don't need compression efficiency. For example 15mbps w/o B-frames should look better than 10 with... do you really need so much compression for an intermediate step?

About XviD, I'd give it a try just for checking it out. It's not as terrible as you think it is, I believe it's a better quality/speed tradeoff for something like this, but if you say you are so bottlenecked, then I don't know anymore...

It does have a different distinct look to it, sharper (but perhaps noisier) than H264, which might be good or bad for what you want. At quantizer 2 it might very well be considered "almost lossless" with a very competitive bitrate (the benefit from sophisticated codecs like H264 is less as the bitrate goes up...)

A pedantic remark: "macroblocks" are always of fixed size (16x16) on both codecs. H264 does have a smaller transform (4x4 vs 8x8) that gives it advantage on sharp edges and artificial sources, so your point stands :)

yes, the disk bottleneck does seem very strange. it looks like the d1 f1 lagarith is supposed to be ~14 megabytes per second, and you would think a decent 7200 rpm sata disk could read twice that fast at the very least. yet 120 fps x264 encoding is a fantasy for me, and the cpus are only at about 50% usage each reading that kind of file. much smaller sources e.g. vobs will solve the problem, and looking at the disk access led seemed to confirm that.

about cq, do you think only 1 pass is really ok? i was seeing some pretty nasty ghosting artifacts with cq like 10, talking about stuff like trails being left behind from a large object moving quickly across a black background leaving the background not so black (this was x264 as well).

interesting info about xvid. i'm wondering how good the vfw one is. i guess i'll run some tests and see what quant ~2 looks like visually and sizewise. if it's acceptable then that will also reduce the cpu load decoding the nmf once it's made. the thing that kind of gets me is that all dvd source stuff is run through a realtime hardware mpeg-2 encoder most often at only 5 megabaud and even i can't tell usually (that is without doing things the viewer won't do like blow up tiny sections to huge sizes). it's mainly only if there's a lot of interference (like nes over rf) or like i said, really high contrast jaggy text that i can tell, and so if i can get around that with the very high bitrate xvid then i'm sold. no more screwing with ghetto vfw x264.

I've never seen artifacts like that in x264 with such low quants... something must have gone wrong :(

Keep in mind that the difference between 1-pass and 2-pass is just ratecontrol, that is, selecting the qp for every frame. If a 2-pass encode selects qps>10 for every frame, then it should always look worse than a qp=10 encode.

I've used fixed qp in the past and it worked as I expected, and provided good quality even with high values (>20), that is, with no surprises in that respect. Not much more that I can say.

Xvid for vfw is very robust (and has the same performance as the command line one). It can even do B-frames without delay (when encoding in VDub and using packed bitstream), although I'd advise against B-frames for this purpose (they are useful for increasing compression, but at a given qp they'll lower quality).

Also, if everything goes right, it's less prone to banding and ghosting than x264 (though at qps

It surprises me too how good the realtime mpeg2 encoders are, but there's more magic to it that what meets the eye: you don't have the original to compare. They have gotten very good at avoiding artifacts, but they might be losing detail. I thought most of them aimed for 8mbps at top quality though; 5mbps sounds like a tough challenge.

Thanks for signing in, . Now you can comment. (sign out)

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)

URL:
Remember me? YesNo

Comments: