Pre Run Job? CPU Throttle?

[sexy-trousers]

I'm evaluating your product (1.10 build 213) on a Win2k3 server, and i was wondering if there is a way to (natively) run an executable/script/batch file before a scheduled backup job (i.e. to dump ms sql databases and other data that won't be able to be backed up normally).

Obviously I can just schedule a batch file to run these things, then run the backup via a command line, but as you had a "post run" option, i would have thought you would also have a "pre run" option - does it exist somewhere?

Also, is there any way to throttle the amount of cpu time the backup consumes? I can alter the priority, but if nothing else pressing is executing, AIS will still consume a high amount of cpu cycles (i.e. generally 60-100%). The reason i ask is because I am evaluating AIS to backup my web server, which obviously has a bunch of other server based apps running concurrently (web, ftp and mail server etc.). So if i could restrict AIS to only consuming, say 40% of the cpu max. during a backup, that would be beneficial...

Thanks in advance for your feedback!

(PS AIS seems to be a great product, you should be proud... )

TJ

[Barry]

We only found a way of altering the priority of jobs, this may be set separately for auto-run and normal run jobs via Tools / Program Options / Advanced. AISBackup does yield time to other applications many times during the backup process. There is a task monitoring software called TaskInfo 2003 which should indicate if AISBackup behaves properly when a web related task is fired:

http://www.iarsn.com/

We recently decided to update AISBackup (from build 215) to enable it to stop and then restart services during a backup, this can be found on our download page, as of today as version 1.10.1 pre-release. Use the Manage Backup / Task Manager option and enter the service within square brackets into the Program field, e.g. [MSSQLSERVER]. AISBackup automatically closes MS Exchange if any of the Information Store database has been selected for backup.

The post run job was added in response to a request from a customer who wanted to do some extra processing on a completed backup. If the pre-run job is still required then we will add it.

Barry

[sexy-trousers]

Adding a pre-run option would be good as stopping and starting the MS SQL service isn't really an option, running an osql command to dump selected databases to a file is less invasive. Good idea to add the service control into the product though - i'm sure i'll find a use for it ... The pre-run can also run purge jobs etc (like clearing out BADMAIL folders, truncate logs etc.) to minimize the size of data to backup (even though i know i could just exclude them, but not "truncate" a file).

Also, using an ftp site as a destination, is there a threshold I can set the maximum size I want the backup folder to get to before it begins uploading files? Or perhaps add an option to upload as you go?

I don't know if your product can run multiple threads etc. (to backup/upload in parallel) but if i backup my entire server (which could be say 70gb of a 80gb drive), unless i have another 80gb to hold the temp. backup files until the backup is complete, and ready to upload, i assume the backup would fail - is that right?

So i guess i am making a feature request to do the following:

1. Create a pre-run execution option.

2a. Create an option to chunk the ftp uploads once a certain job file size reaches X.

or

2b. Be able to optionally upload backed up files in tandem to the backup itself in a second thread (my first choice - if i had one <grin>)

Thanks, TJ

[Barry]

The pre and post run options were the easy ones, so this has been done. As a bonus we now interpret the environment variables correctly so that, for example, %windir%/notepad.exe constructs may be used in any of the three parameters of the pre and post jobs. AISBackup waits for the pre-run job to terminate before execution is resumed, AISBackup does not wait for the post run job to finish.

The easiest way for us to implement the FTP option is to optionally upload and delete each zip file as it is created.

I will e-mail you the location of AISBackup with the pre/post update.

Barry

[sexy-trousers]

The Pre-Job processing works a treat - thanks for the quick response.

A couple of quick questions though...

1. Can the pre & post jobs be defined as part of the regular job defination? This way you are able to run defined jobs from a shortcut and still have them complete correctly (i.e. to dump databases to files prior to a backup etc.) - perhaps this functionality would be better included in the task manager section instead of the Auto-Run section?

2. In your last post in this thread you mentioned how you would implete the ftp-as-you-go feature - was this included with the beta .exe you provided to me? I didn't see the option anywhere and the program didn't appear to operate as an ftp-as-you-go job.

P.S. I have ordered a copy off your site - thank you, it's a very nice product

TJ

[Barry]

The backup job specific pre and post run jobs are ready, the ‘Global’ pre and post run jobs are still available to give the greatest flexibility. The on-the-fly-upload of FTP backups is been worked on. We expect these changes to be uploaded in a day or two.

The pre and post job options are accessed by using the menu option Manage Backup / Edit Backup Job Parameters / Pre and Post Jobs. The On-the-fly upload of FTP backups is accessed by the menu option Manage Backup / Edit Backup Job Parameters / FTP Settings / On the fly upload. I’ll e-mail you and anybody else who is interested when this update is available for testing.

Barry

[sexy-trousers]

Barry, you continue to amaze... Thanks..

[Barry]

Build 216 of AISBackup has been uploaded as version 1.10.1 pre-release. This contains the enhanced pre and post job options and the optional on-the-fly upload, download and deletion of FTP backup files. Note: If the test option is specified the backup files will have to be downloaded for the test phase this will increase network traffic.

http://www.aiscl.co.uk/downloadaisbackup.htm

Barry

[sexy-trousers]

Would the test phase lend itself to have a new "quick test" option in addition to the regular test, so that a local backup file can be tested before an on-the-fly upload takes place?

Sorry to keep asking for things, it seems all i do is take, take, take, but I think these are all good suggestions - feel free to ask me to be quiet when ever you've had enough

Thanks again, TJ

[Barry]

Good idea! Although some Users may prefer to test the backup from the actual destination - which wasn't an option with AISBackup prior to this update. I guess that you also realise that the test is very likely to be from cache rather than actual disk – but that also makes the test phase look blisteringly fast.

This option has been uploaded as build 217. Set the Test Before Upload option using the menu option: Manage Backup / Edit Backup Job Parameters / FTP Settings / Test Before Upload

See my previous post for the download link.

Barry

[sexy-trousers]

Blinding! Thanks for the update, at this rate, I'll be part of your development team before the end of the year!

[Barry]

The manual needs updating

[sexy-trousers]

I was thinking more of an executive project manager role - I think I have enough on my plate at the moment though, thanks for the offer

TJ

[sexy-trousers]

As there is no way to throttle how much CPU AISBackup consumes, and the priority doesn't really help on relatively inactive machines or servers. e.g. If a backup kicks off on a machine with a web server (even set to run at low priority), the backup will peg the processor so that the web server can’t receive/respond to web requests.

Do you think there could be an option added to build in a delay or sleep between processing X number of files or perhaps when a timing threshold is reached, i.e. a timer triggers a X (say 1 second) millisecond delay/sleep every Y (say 10 seconds) milliseconds? It would be a easy (although inefficient) way of building in some sort throttling.

The reason for all of this is I have a couple of monitoring programs poll my machine to make sure everything is running, and that the web server is ok and CPU & memory usage is within bounds. During a backup the web server doesn’t respond and the CPU is max’ed out – thereby sending me a barrage of email and SMS alerts nightly.

If an optional periodic sleep or pause were coded into the job properties, I think the extended high CPU usage could be avoided. Obviously most people want their backup to run as fast as possible, but I would image there are people out there like me who would like the option of controlling AISbackup’s resource usage somewhat.

… just a suggestion – the product is fantastic, and I would highly recommend it to anyone who’s reading

TJ

[Barry]

Have you tried the Low Priority setting? This sets AISBackup to the Idle_Priority_Class, see here: http://msdn.microsoft.com/library/defau ... rities.asp

There are techniques that can be used to throttle AISBackup, these essentially can be used to make AISBackup sleep for x cycles every y cycles, but this would still be true on quiescent systems’ unless dependencies are applied (and should AISBackup periodically check system status – e.g. decelerate if IIS starts and accelerate if IIS stops). We’ll try and knock up a prototype for you to evaluate.

The current version of AISBackup often yields time to other processes and threads. It is in this area of the program that the throttle would be applied, i.e during the CPU intensive compression phase.

I ran a graphic intensive application during during a backup to disk using both the normal and low priorities. There was an obvious difference in performance between both settings.

Barry