emcodem wrote: ↑Tue Dec 10, 2024 12:58 pm
I'm suer Frankie has something to say to that?
Other than "I hate frame properties"?
No but seriously and jokes aside, unfortunately when Avisynth introduced frame properties on top of clip properties, it was a bit of a storm. In theory frame properties are great: they're populated by the indexer when you index a file and passed through the filterchain so that each filter is aware of what it's dealing with and can perform the appropriate action. Also, in theory, those are then passed all the way down to the encoder, which means that it also knows what it's dealing with. In the case of FFAStrans, before you had two options:
A) you dealt with everything yourself and then only really used the encoder to encode
B) you leveraged on the filter builder to do stuff
There was no in between. Let me explain.
FFAStrans deals with every file that is thrown at it as if it were to solve one problem at a time and this is done by something called "the filter builder" (i.e filter_builder.au3). Once a file is submitted to FFAStrans, it gets analysed by a combination of ffprobe, exiftools and Mediainfo and then, once the filter builder knows what's is dealing with in input and what the target output is, it "builds" the "filterchain", hence the name. Such a component goes through the problems one at a time to check whether it's interlaced or not, what the frame rate it, what the resolution is, what the chroma location is, what the chroma sampling is, what the colormatrix, transfer and primaries are, what the bit depth is etc and what the target needs to be to then build a bridge from the input to the target. It's based on a huge series of rules, it's several lines long and it's a marvelous piece of code that Grandmaster Steinar wrote and improved over the years. Anyway, that's basically what happens when you put an encoding node there in a workflow. In the former FFAStrans version this would be option B, i.e let the filter_builder take care of it. Now, given that there weren't frame properties in Avisynth and given that clip properties only really carried a limited subset of information, we had to work around that. In particular, Avisynth wasn't passing anything down the line to FFMpeg in terms of properties, but that's because historically Avisynth was supposed to be the frame server doing all the various transformations and then have its uncompressed audio and video stream go to an encoder. For instance, I often use it to encode with x264 and create the raw_video.h264, with BePipe + NeroAAC to create the raw_audio.aac and MP4Box to create the final_output.mp4 using a BAT. The idea being that no other transformation should be needed on the Avisynth output. Anyway, this isn't exactly the case with FFMpeg. As ironic as it may sound, FFMpeg is a collection of tools, so not only it can call encoders like x264 via libx264, FDK AAC etc and muxers like the MP4 muxer as part of lavf, but it's also a frameserver itself as it can perform different kinds of filtering via the filtercomplex as it includes things for scaling in sws, it includes z.lib, etc. Suddenly we have a situation which the original Avisynth developers didn't envision: having the output from Avisynth go into another frameserver before being converted again and going into the actual encoder. This is what happens in FFAStrans. So... if the filter_builder is meant to create a filterchain in FFMpeg and it does it based on the input but the input is an Avisynth script which doesn't have any properties at all, how does it know what it's receiving and where it needs to go? Well... simply put: it doesn't. What we did was a huge implementation of passing properties around, so basically when you had an input, FFAStrans was gonna analyse it anyway before going through the indexing step (i.e before going through the A/V Decoder) and then once you inserted a filter (I'm talking about one of the actual filters built into FFAStrans) we were gonna update the properties internally 'cause I mean we knew what was gonna happen and what the settings were gonna do on the input. Like, if you were to insert a deinterlacing node with BWDIF 2x and the input was interlaced, we knew that it was gonna bob deinterlace it, so we updated those info under the hood so that once you got to the encoding node, the filter builder knew that it was dealing with a progressive source. This obviously was only possible for the built-in filters, but not with the custom Avisynth scripts. Like, suppose you add a LinearTransformation filter to go from BT2020 HLG to BT709 SDR, then we would know that we were dealing with BT709 SDR, but if you did it yourself in a custom Avisynth script cause you wanted to use Reinhard Tonemapping rather than a LUT, then we wouldn't have had any way to know that the output was now BT709 already and that the filter_builder didn't have to convert it before calling the encoder. This led us to create a tickbox in the custom Avisynth script called "adapt" (I don't remember the exact name but it started with "adapt") in FFAStrans 1.2.0 come 2021 that basically told the filter builder "don't do anything, I've got everything under control" and only really do the straightforward things that could be inferred by the clip properties like the chroma sampling conversion, the resolution conversion etc, but definitely nothing matrix/transfer/primaries related etc. This obviously wasn't ideal, so either you had to go with option A, do everything in custom Avisynth script to give the filter builder something perfect and then encode or just not use those custom Avisynth scripts and let the filter builder do everything. Obviously one could have also just created a workflow like:
watchfolder -> A/V Decoder -> Deinterlacing node -> Custom Avisynth Script (denoise) -> Encoder -> Delivery
and left the "adapt" option unticked to let the filter builder do its thing as you were not gonna change other properties of the video, just denoising it, hence why the adapt was a tickbox that could be ticked and unticked. Once again, this whole thing wasn't ideal because let's assume for a moment that your source is 4:2:0 with chroma_location "top_left" but your denoise only works in 4:2:2 interleaved (i.e YUY2). In your Avisynth Script you could have just used a simple ConverttoYUY2() then the denoise like SpatialSoften(4, 4, 8) and then back to 4:2:0 with ConverttoYV12(). You didn't tick the "adapt" so you would still be able to take advantage of the filter_builder converting everything else to your destination like resolution changes, matrix/transfer/primaries etc and everything will be hunkydory, right? Well, not quite. You see, the filter_builder will do that with the information it has from the input and adjusted as per the last known internal node (the deinterlacing one), but once you inserted the deinterlacing node you just changed the chroma location from top_left to left in the YV12-YUY2-YV12 roundtrip without telling the filter_builder, so it will still assume a top_left chroma location and the output will be wrong. Frame Properties were introduced to solve EXACTLY this problem. Suddenly, the filter_builder was no longer blind, it wasn't kept in the dark about what it was receiving as Avisynth finally had a way of telling exactly what was passing and this was a game changer. We removed all the old logic and allowed Avisynth to talk directly to the filter builder which not only simplified the handling of internal filters but even the custom Avisynth script now allowed people to truly have an hybrid approach that was neither A nor B but something in between, I'll call it option C:
C) do something I really care about myself and let the filter builder do the rest
This is where we are now with FFAStrans 1.4.0.xx, so this brings the question: what's with the "clear frame properties" tickbox? If frame properties are great, then why would you wanna remove them and not use them in the first place?
Well... the answer is complicated and it would it would make this post from "long" to "downright unreadable", so in a nutshell, carrying frame properties isn't always a good thing. As we're transitioning from clip properties to frame properties, we're in a pretty dim situation right now. Some filters embraced frame properties and use them to adjust their settings which is something you not always want especially if you're telling a function what to do explicitly, some other don't update frame properties after they've done their own things but just carry them over, which is bad as it then falls to the user to change them and some others don't even pass them over as they were made with very old versions of Avisynth in mind like those for Avisynth 2.5 (we're currently at 3.7.3) so again it falls to the user to populate them manually with PropSet(). Unfortunately, there's no simple way to look at it and there isn't a simple solution that fits everything, but essentially we tried offering people the possibility to do everything regardless of frame properties and, given that they didn't exist up until recently, turning on the "clear frame properties" by default was in theory a mechanism to protect people from suddenly having their functions doing other things rather than what they thought they were doing, but obviously this backfired in some situations. The migration to frame properties is there to stay and I'm sure it will bring more benefits in the long run, but at this moment in time there sure are a lot of headaches, which is why I often joke about it by saying "I hate frame properties". Truth to be told, I don't hate them, I'm sure they're gonna bring so many benefits and they've already been a game changer for lots of my workflows that went from doing literally everything myself to leveraging on the filter builder more and only address manually the things I truly care about. The only thing is that, unlike VapourSynth which was born with them and therefore everything is aware of them, in Avisynth they have just been introduced and it will take a long time for everything to be aware of them and also, realistically, some filters will never be updated so you're gonna need some workaround anyway, but this just shows how long of a journey it's been for this amazing 24 years old frameserver.
