For some reason my folder monitor/watch folder stopped working - and I can't quite figure it out. Is there a way to see what is happening? I removed the application totally and stared with a fresh install just to make sure I didn't mess it up, but it still won't recognize the watchfolder.
Here is a screen grab of the basic setup I cannot get to begin!
https://imgur.com/a/aJ0ONa3
Thanks for any help you might be able to offer!
Watchfolder stopped working?
Re: Watchfolder stopped working?
Hey oneskinnydave,
the logs should be written to Processors\db\cache\def_log, but in my installations this folder is empty and i have no clue why, @admin might be able to say why but he is currently not available.
So we try to debug without any log.
First: i see you checked most of the "deny attributes" checkboxes, does it work when you check none of them?
Second: not sure if you are aware about what "check growing files continuously" does, it is a very unusual option for watchfolder operations. "Once" in combination with "forget missing" is the most common setting.
Third:
What i do for debugging is in the folder monitor processor, after a configuration change, hit history -> forget and rebuild (make sure some files that should be included by the watchfolder are there). Then i go to \Processors\db\cache\wfs\%wf_id%\mons\%proc_id% and check if the json files for all the concurrent files in the folder are there. The json files in this cache\wfs folder is the database for remembering what was processed, whats incoming and such.
Let me know your experience with above procedure..
the logs should be written to Processors\db\cache\def_log, but in my installations this folder is empty and i have no clue why, @admin might be able to say why but he is currently not available.
So we try to debug without any log.
First: i see you checked most of the "deny attributes" checkboxes, does it work when you check none of them?
Second: not sure if you are aware about what "check growing files continuously" does, it is a very unusual option for watchfolder operations. "Once" in combination with "forget missing" is the most common setting.
Third:
What i do for debugging is in the folder monitor processor, after a configuration change, hit history -> forget and rebuild (make sure some files that should be included by the watchfolder are there). Then i go to \Processors\db\cache\wfs\%wf_id%\mons\%proc_id% and check if the json files for all the concurrent files in the folder are there. The json files in this cache\wfs folder is the database for remembering what was processed, whats incoming and such.
Let me know your experience with above procedure..
emcodem, wrapping since 2009 you got the rhyme?
-
oneskinnydave
- Posts: 6
- Joined: Fri Sep 23, 2022 11:23 pm
Re: Watchfolder stopped working?
Awesome - thanks so much, I'll try that out today, I appreciate all your help!
-
oneskinnydave
- Posts: 6
- Joined: Fri Sep 23, 2022 11:23 pm
Re: Watchfolder stopped working?
It must have been some combination of those checked boxes, but it's working again - thank you SO much for your help!!!
Re: Watchfolder stopped working?
As a humble suggestion for the future, this is how, in general, watchfolders should be set for most of the scenarios:
A few notes:
Accept Files: *.*
will collect literally ANYTHING you put inside
The flag on "Recurse" will find files in nested folders, so even if you put inside a folder called "ABC\AAA\File.mxf" it will find it and process it.
The flag on "Forget Missing File" will allow you to reprocess the same file if you need it for whatever reason; like if you put it back in the watchfolder without changing the name.
Now, a few things about the common misconceptions: you do not need to use "Check growing files" 'cause all files will be locked anyway. This means that even if a person is copying via SMB Share a file in a watchfolder, such a file won't be picked up 'till it has been fully copied anyway. This is because there's a check on whether the file is locked in the storage or not and a storage with a good file system should behave (especially if all machines writing on it are on Windows).
What the "Check for growing files" will do is... well... yes, check for growing files and wait another sleep cycle, which can potentially slow everything down, especially if you have lots of watchfolders, 'cause the files discovery in FFAStrans is single thread (due to an AutoIT limitation) inside a workflow, so it will check each Watchfolder one by one in an ordered manner.
Of course there are also use cases for that option, for instance we had cases in which some linux boxes were writing files on a particularly hosted SMB share by them and didn't behave correctly, but again, those are edge cases.
Most of the time you want the config to be like the way I showed you and rest assured that if you set it that way, it will work
A few notes:
Accept Files: *.*
will collect literally ANYTHING you put inside
The flag on "Recurse" will find files in nested folders, so even if you put inside a folder called "ABC\AAA\File.mxf" it will find it and process it.
The flag on "Forget Missing File" will allow you to reprocess the same file if you need it for whatever reason; like if you put it back in the watchfolder without changing the name.
Now, a few things about the common misconceptions: you do not need to use "Check growing files" 'cause all files will be locked anyway. This means that even if a person is copying via SMB Share a file in a watchfolder, such a file won't be picked up 'till it has been fully copied anyway. This is because there's a check on whether the file is locked in the storage or not and a storage with a good file system should behave (especially if all machines writing on it are on Windows).
What the "Check for growing files" will do is... well... yes, check for growing files and wait another sleep cycle, which can potentially slow everything down, especially if you have lots of watchfolders, 'cause the files discovery in FFAStrans is single thread (due to an AutoIT limitation) inside a workflow, so it will check each Watchfolder one by one in an ordered manner.
Of course there are also use cases for that option, for instance we had cases in which some linux boxes were writing files on a particularly hosted SMB share by them and didn't behave correctly, but again, those are edge cases.
Most of the time you want the config to be like the way I showed you and rest assured that if you set it that way, it will work
Re: Watchfolder stopped working?
To mention: the scenario FranceBB describes is for optimizing the performance in a special scenario also it is ONLY compatible when files arrive in watchfolders through "windows explorer copy" or move operations which is usually only the case when the watchfolders are being filled manually by humans that work on windows clients. It would not work for example for file export from NLE to the watchfolder and most other scenarios where it is not a human manually copying the file into the watchfolder.
Check growing "once" is compatible to nearly any thinkable scenario AND covers above usecase, so it is the most compatible setting that we offer.
Check growing "continuously" is compatible to the same amount of scenarios as "once" but can lead to huge amounts of RAM/CPU usage for the watchfolders. It should never be used unless you have exactly this usecase:
Check growing "once" is compatible to nearly any thinkable scenario AND covers above usecase, so it is the most compatible setting that we offer.
Check growing "continuously" is compatible to the same amount of scenarios as "once" but can lead to huge amounts of RAM/CPU usage for the watchfolders. It should never be used unless you have exactly this usecase:
- File arrives in watchfolder first time -> will be processed BUT workflow does not move/delete file from watchfolder
- Later in time, the same file is updated e.g. overwritten and it shall be reprocessed
emcodem, wrapping since 2009 you got the rhyme?