Page 1 of 2

ULTIMATE BACKUP for FFAStrans System

Posted: Sun Feb 14, 2021 12:06 am
by authorleon
Hello all,

Are you looking for one stop solution to backup your FFAStrans. Look no further.

Why did I make this
I am rather obsessed with backups and very paranoid.

Concept
To backup the FFAstrans folder with date and time stamp.

Requirements
7zip to be installed and 7z.exe to be placed under "C:\Windows\System32".
You need to create a .bat file.
Just copy the code below into a text file and change the extension from .txt to .bat
Every time you run this.BAT file. It will automatically create a backup and you can see the picture right at the bottom is an example.
You can even put this in the Windows Start-up folder so it will execute when your computer starts.

Performance
I have tweaked the code so the compression software uses a multi threaded CPU

Restore process
Just unzip the folder to the C drive and it will replace the existing ffastrans folder. For those who are a bit more paranoid. Delete the ffastrans first and then decompress the folder to the C drive.

Code: Select all

@echo on
:Version 1.0 (AuthorLeon)
Setlocal enabledelayedexpansion
CONSOLESTATE /Show
color a

cd c:\
ping 127.0.0.1 -n 1 > nul
mkdir c:\.temp

:COPY FILES FROM C:\FFAStrans to c:\.temp
robocopy /is /S /E "C:\FFAStrans" "c:\.temp"

:MAKE FFAStrans BACKUP FOLDER
CD /D "C:\FFAStransBackups"
For /f "tokens=1-3 delims=/ " %%a in ('date /t') do (set mydate=%%a-%%b-%%c)
For /f "tokens=1-2 delims=/:" %%a in ('time /t') do (set mytime=%%a%%b)
set mydir="FFASTRANS_%mydate%_%mytime%"
:mkdir %mydir%


CD /D "C:\.temp"
rem 7za a %mydir%.7z
7z a -t7z -m0=LZMA2:d64k:fb32 -ms=8m -mmt=30 -mx=1 --  "C:\FFAStrans\Backups\%mydir%.7z" "C:\.temp\*"


REM move "%mydir%.7z" "C:\FFAStrans\Backups"
CD /D "C:\"
REM PAUSE
rmdir /Q /S c:\.temp

CD /D "C:\FFAStrans\"
start "" "FFAStrans.exe"
REM ping 127.0.0.1 -n 1 > nul
exit
Image

Thank you all

Re: ULTIMATE BACKUP for FFAStrans System

Posted: Sun Feb 14, 2021 9:31 am
by momocampo
Hi authorleon,
It's a good idea but I don't think backup All FFAStrans folders is necessary. "db" folder (tickets, cache, workflows, etc) should be enough. You can use the wenbinterface to schedule a backup as well.
Create a command executor node, type your backup command inside and schedule it each week for example.
Cheers.
B.

Re: ULTIMATE BACKUP for FFAStrans System

Posted: Sun Feb 14, 2021 11:27 am
by authorleon
Hello Momocampo.

Thank you. I do understand what you are saying. Of course you can back up the DB folder. But I have quite a few portable application under C:\FFAStrans for example Process Lasso to manage CPU using as I have 64 cores (Thread ripper) and so on. The above backup solution backs-up everything. As it is a startup script, you can define how things should load. On every startup (turn on pc), everything will back backed up.

Thank you :)

Author

Re: ULTIMATE BACKUP for FFAStrans System

Posted: Sun Feb 14, 2021 6:41 pm
by momocampo
Hi,
Ok got it. I still think using webinterface to start your script might make more sense. FFAStrans is often installed on servers, so it doesn't restart often..
Anyway thanks for this clever procedure.
;)
B.

Re: ULTIMATE BACKUP for FFAStrans System

Posted: Sun Feb 14, 2021 10:33 pm
by authorleon
momocampo wrote: Sun Feb 14, 2021 6:41 pm Hi,
Ok got it. I still think using webinterface to start your script might make more sense. FFAStrans is often installed on servers, so it doesn't restart often..
Anyway thanks for this clever procedure.
;)
B.
That could be grate using the web interface.

Doubt: If the farm is running, can you make a backup? Hence the reason for executing it in the beginning and then starting up FFAStrans.

Re: ULTIMATE BACKUP for FFAStrans System

Posted: Mon Feb 15, 2021 8:24 am
by emcodem
Aye,

I love the idea of this, i am thinking that we should provide a plugin processor that allows to specify a directory name to backup and a destination for the zip, include and exclude patterns as well as a "count of backup files to keep" on target...

To the topic itself: well just shutting down ffastrans.exe, at least in farm environment isn't really doing anything but closing the worfklow builder GUI for the user. All the operations are done by exe_manager and processors.exe and their child processes. It shouldnt be too much of an issue to zip the files of a running instance anyways.

It might also not make too much sense to backup the tickets folder as you would end up with old jobs being reprocessed as soon as you restore the stuff.

If we'd like to make such a solution working in a generalized way, here are some more points of interest:

Code: Select all

mkdir c:\.temp
This will not work if the user don't have UAC disabled, better to use %temp% instead

Code: Select all

CD /D "C:\.temp"
It would be preferrable to avoid any CD command as users migth prefer to have everything on UNC locations, especially in Farms when installed as a service

Code: Select all

ping 127.0.0.1 -n 1 > nul
Don't think thats really neccessary

Code: Select all

robocopy, 7z 
Instead of assuming the programs are available in c:\windows\system32, i'd place a copy of them relative to the batch in "bin" directory and call like "%~dp0bin\robocopy.exe" "%~dp0bin\7z.exe"

Re: ULTIMATE BACKUP for FFAStrans System

Posted: Thu Feb 18, 2021 10:26 am
by authorleon
emcodem wrote: Mon Feb 15, 2021 8:24 am Aye,

I love the idea of this, i am thinking that we should provide a plugin processor that allows to specify a directory name to backup and a destination for the zip, include and exclude patterns as well as a "count of backup files to keep" on target...
...
Hey emcodem,

How are you? I hope you are well. I will take your suggestions on-board. Thank you!!!!

Re: ULTIMATE BACKUP for FFAStrans System

Posted: Thu Feb 18, 2021 1:43 pm
by emcodem
Hey there, nice that you ask :D ...besides the social distancing starts to hurt i am very fine, thanks for asking :D
You think you can exclude some not so interesting folders like the tickets?

Re: ULTIMATE BACKUP for FFAStrans System

Posted: Thu Feb 18, 2021 2:02 pm
by authorleon
Hey,

I am glad you're well.

Of course I can exclude folders. Consider it done. What would be great is a list of folders I should exclude.

That would be fantastic.

I'm also integrating a retention plan. In other words delete back ups after 21 days for example.

Thanks.

Re: ULTIMATE BACKUP for FFAStrans System

Posted: Thu Feb 18, 2021 2:24 pm
by emcodem
We would need to exclude any currently running job info, thus:
  • \Processors\db\cache\tickets can be completely excluded
  • \Processors\db\cache\status can be completely excluded
  • Processors\db\cache\monitor we could also exclude this but to go the full way and keep some history info, we should "include" only the log.txt file from here but exclude all other
  • Processors\db\cache\jobs this is a matter of discussion, it can get huge if you have many jobs and it is not really needed but if we decide to keep the log.txt file from above, we should also keep those guys here.
  • api_log and exe_log ...well again a matter of discussion... they are not needed to restore functionality
I'd include all other directories because, as you say, people might have enriched the tools folders with their own stuff.
Also, i think there is no need for robocopy, just zip the stuff, thats it...