Long backup misbehaving

Use this forum for help with AISBackup
Post Reply
Hughg
Posts: 94
Joined: Sat Feb 01, 2003 11:25 pm

Long backup misbehaving

Post by Hughg »

I'm backing up some large files to a remote sftp server. The backup starts fine, but when it gets to the large files (4GB) it seems to get stuck. It shows the SFTP link as active, but the Upload Progress bar is at 100% and it seems to stay like this for days. The file appears to have been transferred (it appears in the "Transferred" folder), and the "Transferring" folder is empty. Any idea what's going on?
Barry
Site Admin
Posts: 1529
Joined: Tue Aug 20, 2002 3:16 pm

Re: Long backup misbehaving

Post by Barry »

Hi Hugh

I just backed up a 4.3GB file to A local Synology NAS and that worked without error, the files are split at 4GB. I restored the file to another location on my PC.

This may be a timeout issue: There is an option to split files at 2GB instead of the default 4GB, this may be set by using Manage Backup / Change Backup Job Settings then select ‘Split files at 2GB’. If that does not work the you can set the size at which files are split by entering a value in megabytes using the option Manage Backup / Change Backup Job Settings / FTP Settings (button) the enter a value in ‘Spit Files at (MB)', I used 512 in this test, and one of my 6.76 GB YouTube music videos.

Barry
Hughg
Posts: 94
Joined: Sat Feb 01, 2003 11:25 pm

Re: Long backup misbehaving

Post by Hughg »

Hi Barry

I tried reducing the split to 512MB and ran it again. I attach the log below. There are a lot of ftp errors, but after it reset the ftp system it started uploading a file "Democritus to Tollkeepers_00014_000.AIL" The File Progress went to 100%, then the Upload Progress did the same, and at that point it seems to have hung again.
Hugh

AISBackup started: April 21, 2023 09:14:29
Backup Session: 656 Started on April 21, 2023 at 09:14:31 Version: 6.0 (505)
Platform: Windows 10 Pro (22H2), 64-bit
Path: C:\Program Files (x86)\Aquarius IS Consultancy\AISBackup\AISBACKUP.exe
User Account: hughg
Running as Administrator
Checking Connection to FTP Server: tollkeepers.gunzclan.org
Calculating backup
Files Selected For Backup: 702 (8,077.54 MB)
Already Backed up: 292,966
Excluded From Backup: No files or folders have been excluded.
Backup destination: L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers.aib, Volume ID: [DATA 2], Signature: [000000000000000100000000]
70 New Folders.
Backing up NTFS Security Settings and Alternate Data Streams.
Test Method: File Integrity Method.
Backing up: 702 Files of 8,077.00 Megabytes
Excluding: 0 Files (See Modify Files and Folders/Exclusions)
Already on Backup: 292,966 Files
Compression Level: High
Estimated Size of Backup: 3.95 GB
Folder Security Phase
Backing up NTFS folder security settings
Finished Backing up NTFS folder security settings
Backing up 1 Exclusively Open File
Initialised Volume Shadow Copy Service for Drive: D:\
Testing Backup
Test Completed
Checking for redundant backup data (Advanced Sessions)
============ Pruning 228 files from Session: 644 (Expired Weekly Backup: 05/2023)
L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_7681.aib: Removing 1 File
L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_7688.aib: Removing 1 File
L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_7693.aib: Removing 3 Files
L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_7697.aib: Removing 1 File
L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_7707.aib: Removing 1 File
L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_7717.aib: Removing 3 Files
L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_7729.aib: Removing 1 File
L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_7732.aib: Removing 1 File
L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_7751.aib: Removing 1 File
L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_7756.aib: Removing 1 File
L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_7771.aib: Removing 1 File - Backup file no longer used: Deleted
L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_7773.aib: Removing 1 File
L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_7781.aib: Removing 1 File
L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_7786.aib: Removing 21 Files
L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_7788.aib: Removing 18 Files - Backup file no longer used: Deleted
L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_7793.aib: Removing 29 Files
L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_7800.aib: Removing 9 Files
L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_7804.aib: Removing 1 File
L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_7806.aib: Removing 127 Files
L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_7807.aib: Removing 2 Files
L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_7808.aib: Removing 1 File - Backup file no longer used: Deleted
L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_7809.aib: Removing 1 File - Backup file no longer used: Deleted
L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_7810.aib: Removing 1 File - Backup file no longer used: Deleted
L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_7811.aib: Removing 1 File - Backup file no longer used: Deleted
Finished checking redundant backup data
702 Files were backed up and tested successfully
Completed successfully
Backing up Backup Contents File Database Files
Time Calculating Backup: 5 Minutes 1 Second
Time Backing up: 8 Minutes 26 Seconds
Time Testing Backup: 24 Minutes 25 Seconds
Total Time: 51 Minutes 3 Seconds
Time Deleting Old Versions: 10 Seconds
Backed up at 955.94 MB per minute
Compression Ratio: 45%
Not Compressible 7164.36 MB 89% (Not included in compression ratio)
Tested backup at 330.73 MB per minute
Longest wait for SFTP to respond: 4 seconds
SFTP Replace Failed: L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00014_000.AIL - L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00014_000.AIL Exception During Upload: tgsftp_putfile: Unknown Error. - sftp_recvdata: timeout, no data received for 60 secondsFATAL ERROR: did not receive SFTP response packet from serverFATAL ERROR: unable to understand SFTP response packet from server: did not receive a valid SFTP packet/mnt/DroboFS/AISBackup/Democritus to Tollkeepers/Democritus to Tollkeepers_00014_000.AIL: open for write: fxp_open_recv: no pktin, possibly not connected
SFTP Replace Failed: L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00007_000.AIL - L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00007_000.AIL Exception During Upload: TTGPuttySFTP exception NULL == find234(sftp_requests, &i, sftp_reqfind) at line 138 in ..\sftp.c
SFTP Replace Failed: L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00013_000.AIL - L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00013_000.AIL Exception During Upload: TTGPuttySFTP exception NULL == find234(sftp_requests, &i, sftp_reqfind) at line 138 in ..\sftp.c
SFTP Replace Failed: L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00012_000.AIL - L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00012_000.AIL Exception During Upload: TTGPuttySFTP exception NULL == find234(sftp_requests, &i, sftp_reqfind) at line 138 in ..\sftp.c
SFTP Replace Failed: L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00006_000.AIL - L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00006_000.AIL Exception During Upload: TTGPuttySFTP exception NULL == find234(sftp_requests, &i, sftp_reqfind) at line 138 in ..\sftp.c
SFTP Replace Failed: L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00004_000.AIL - L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00004_000.AIL Exception During Upload: TTGPuttySFTP exception NULL == find234(sftp_requests, &i, sftp_reqfind) at line 138 in ..\sftp.c
SFTP Replace Failed: L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00003_000.AIL - L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00003_000.AIL Exception During Upload: TTGPuttySFTP exception NULL == find234(sftp_requests, &i, sftp_reqfind) at line 138 in ..\sftp.c
SFTP Replace Failed: L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00005_000.AIL - L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00005_000.AIL Exception During Upload: TTGPuttySFTP exception NULL == find234(sftp_requests, &i, sftp_reqfind) at line 138 in ..\sftp.c
SFTP Upload Failed: L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00014_001.AIL - L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00014_001.AIL Exception During Upload: TTGPuttySFTP exception NULL == find234(sftp_requests, &i, sftp_reqfind) at line 138 in ..\sftp.c
SFTP Replace Failed: L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00013_001.AIL - L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00013_001.AIL Exception During Upload: TTGPuttySFTP exception NULL == find234(sftp_requests, &i, sftp_reqfind) at line 138 in ..\sftp.c
SFTP Upload Failed: L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00004_001.AIL - L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00004_001.AIL Exception During Upload: TTGPuttySFTP exception NULL == find234(sftp_requests, &i, sftp_reqfind) at line 138 in ..\sftp.c
SFTP Upload Failed: L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00007_001.AIL - L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00007_001.AIL Exception During Upload: TTGPuttySFTP exception NULL == find234(sftp_requests, &i, sftp_reqfind) at line 138 in ..\sftp.c
SFTP Replace Failed: L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00011_001.AIL - L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00011_001.AIL Exception During Upload: TTGPuttySFTP exception NULL == find234(sftp_requests, &i, sftp_reqfind) at line 138 in ..\sftp.c
SFTP Upload Failed: L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00004.aii - L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00004.aii Exception During Upload: TTGPuttySFTP exception NULL == find234(sftp_requests, &i, sftp_reqfind) at line 138 in ..\sftp.c
SFTP Upload Failed: L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00014.aii - L:\AISBackup\Democritus to Tollkeepers\Democritus to Tollkeepers_00014.aii Exception During Upload: TTGPuttySFTP exception NULL == find234(sftp_requests, &i, sftp_reqfind) at line 138 in ..\sftp.c
Processing Failed Uploads
SFTP: Resetting FTP Connection.
SFTP: Re-creating FTP Interface.
Barry
Site Admin
Posts: 1529
Joined: Tue Aug 20, 2002 3:16 pm

Re: Long backup misbehaving

Post by Barry »

Hi Hugh

I looks like AISBackup isn't able to replace some files on the server, this may be because of security settings or the server is holding the files open for write. Is it possible for you to make another backup job that only backs up this failing file for testing?

Barry
Hughg
Posts: 94
Joined: Sat Feb 01, 2003 11:25 pm

Re: Long backup misbehaving

Post by Hughg »

Hi Barry

I could, if I knew which file it was that's causing the trouble. All I have is the .ail file. Is there a way of finding out which file that comes from?

Hugh
Barry
Site Admin
Posts: 1529
Joined: Tue Aug 20, 2002 3:16 pm

Re: Long backup misbehaving

Post by Barry »

Hi Hugh

If you can e-mail me the backup job files by using the Help / Bug Report option I should be able to find out which file this is.

Barry
Post Reply