Can I back up a Linux network share?

Use this forum for help with AISBackup
Post Reply
cyclone
Posts: 14
Joined: Tue Feb 17, 2004 11:11 am
Location: Silicon Valley

Can I back up a Linux network share?

Post by cyclone »

My work laptop is running Linux with Samba, which means I can mount its hard disk via Windows. I'm attempting a backup, and it appears to be proceeding as expected. Can you tell me whether there might be any problems I should be aware of?

Bill
Barry
Site Admin
Posts: 1529
Joined: Tue Aug 20, 2002 3:16 pm

Backing up Linux via Samba

Post by Barry »

Hi Bill

It is possible to backup and restore via a samba server. This has been tested by backing up an Apache web-site, deleting some of it and then doing a restore.

The FTP option can also be used if a Linux FTP has been installed.

Some AISBackup Users backup Windows systems’ to Samba shares too.

AISBackup cannot, however, backup and restore the whole Linux Operating system as there is no mechanism for backing up and replacing opened exclusive files over a network.

Barry
cyclone
Posts: 14
Joined: Tue Feb 17, 2004 11:11 am
Location: Silicon Valley

Post by cyclone »

After trying I think there may be a problem. AIS Backup spent over thirty minutes calculating the backup before I gave up (the progress bar would go from about 50% back to 30% repeatedly). My assumption is that perhaps it's not handling linked directories properly and it's getting into some sort of recursive loop between links. It could be that I just didn't give it enough time, but it's over a 100MB LAN and the Linux system is just a basic laptop, so shouldn't have an unusually large number of files.

Any thoughts?
Barry
Site Admin
Posts: 1529
Joined: Tue Aug 20, 2002 3:16 pm

Linux Backup

Post by Barry »

The initial stage of calculating a backup cannot be timed correctly, therefore we move the progress bar back periodically. This is to show that AISBackup has not hung. We will allow the progress bar more time in the next release of 1.9.

Let us know how much data and the approximate depth of directories on the Samba share and we will run a test based on your set-up here.

Barry
cyclone
Posts: 14
Joined: Tue Feb 17, 2004 11:11 am
Location: Silicon Valley

Post by cyclone »

Hi, Barry,

It's a standard SuSE 9.0 Professional installation on a laptop. When I made a backup using tar it was about 4GB using gzip. There are about 260K files, the deepest directory I could find was about ten levels deep (in the Linux source tree). As I mentioned, I'm on a 100MB LAN. The desktop running AIS Backup is a 512MB P4-2.26GHz, the laptop is a 384MB 1.4GHz Centrino. I'd be interested in hearing how long is reasonable for calculating the backup; I assumed it had gotten into some sort of recursive directory loop.

Can you confirm that linked files will be handled properly? That is, on a restore, the link would be restored as pointing to the original directory rather than restored independently?

Bill
Barry
Site Admin
Posts: 1529
Joined: Tue Aug 20, 2002 3:16 pm

Linux directory recursion loop found

Post by Barry »

You are quite right. There is a folder called Proc which contains a folder called 4574 which contains a folder called Root, which is a pointer to ‘ the real Root (\)’, which of course holds Proc, 4574 etc… AISBackup would continue to navigate this virtual tree until it runs out of system resources. Normally AISBackup gets an access not allowed from root, but in this case it was able to navigate the root branch again. The virtual directory root has an attribute of 16 which means it looks like a regular folder, therefore we cannot exclude it programmatically (unless somebody can tell us different).

The work around is to exclude folder \root (click Exclusions, and add \root to the exclusions list). Also exclude the folder called Dev (right click / exclude files and folders), this contains a file called ftape whose reported size was 0 but after a rather long period of ‘hanging’ AISBackup was aborted after backing up >1GB of this zero length file :roll:.

As I mentioned in an earlier post AISBackup is not suitable for disaster recovery of a Linux system so I guess there are many more system files that could be excluded. I am not a Linux expert so cannot comment too much on this subject.

I selected root minus the above exceptions which consisted of 22,897 files of 4,647.15 MB, which backed up in 1 hour 5 minutes (using the test version of AISBackup which backs up in half the time as version 1.8 and 1.9 pre-release).

As for the linked files scenario, tell me how to set one up and I’ll test it for you. I guess that at the moment the file will be seen as two or more distinct files and will be backed up as many times. I also imagine that if there is a concept of a linked directory and this points to an owner directory of itself the recursion will happen again. The work around for this is to exclude all files and folders from linked folders. There may be an attribute can be checked on the linked folders to prevent the ‘root’ problem and any other linked directories, although this did not work with Root. Can you help here?

Barry
cyclone
Posts: 14
Joined: Tue Feb 17, 2004 11:11 am
Location: Silicon Valley

Re: Linux directory recursion loop found

Post by cyclone »

Barry wrote:You are quite right. There is a folder called Proc which contains a folder called 4574 which contains a folder called Root, which is a pointer to ‘ the real Root (\)’, which of course holds Proc, 4574 etc… AISBackup would continue to navigate this virtual tree until it runs out of system resources.
Does this happen during the "calculating backup" phase? That's as far as I got, aborting after about thirty minutes. Is there any mechanism in V1.9 for me to see what files it's working on so I can see where there might be recursive linkage?
Barry wrote:The work around is to exclude folder \root (click Exclusions, and add \root to the exclusions list). Also exclude the folder called Dev (right click / exclude files and folders), this contains a file called ftape whose reported size was 0 but after a rather long period of ‘hanging’ AISBackup was aborted after backing up >1GB of this zero length file :roll:.
I had excluded /proc, but it's quite likely there are other links which recurse back to a parent directory.
Barry wrote:As I mentioned in an earlier post AISBackup is not suitable for disaster recovery of a Linux system so I guess there are many more system files that could be excluded.
Yes, I recognize that this wouldn't be suitable for restoring a damaged system. The beauty of Linux is that a new vanilla install can be done quite easily from original media and then all of the modifications, which generally aren't system files, can be restored. I'd say there's much less complexity than with Windows' registry and other protected files.
Barry wrote:As for the linked files scenario, tell me how to set one up and I’ll test it for you. I guess that at the moment the file will be seen as two or more distinct files and will be backed up as many times. I also imagine that if there is a concept of a linked directory and this points to an owner directory of itself the recursion will happen again. The work around for this is to exclude all files and folders from linked folders. There may be an attribute can be checked on the linked folders to prevent the ‘root’ problem and any other linked directories, although this did not work with Root. Can you help here?
Sure. There are two kinds of linked files in Linux. One is a hard link which is simply a second (or more) filename pointing to an existing file. According to "info ln":

A "hard link" is another name for an existing file; the link and the original are indistinguishable. Technically speaking, they share the same inode, and the inode contains all the information about a file--indeed, it is not incorrect to say that the inode _is_ the file.

I think the main issue a hard link presents during a backup is whether it's possible to not store the file once for each link to it. That is, if you have file.txt, and filelink.txt which is a hard link to file.txt (created by "ln file.txt filelink.txt"), each 5K, you'd hope the backup is only storing file.txt and a link to it, rather than storing it twice. Tar on Linux does this properly - if you did "tar cf test.tar file.txt filelink.txt" and then listed it via "tar vf test.tar" you'll get "file.txt, filelink.txt link to file.txt" as the output.

The other type of link, symbolic (or soft) link, is more dicey from a backup standpoint, because it does typically point to a directory, so could lead to infinite recursion. It's purely a pointer to another directory, and I'd say is fairly analagous to a shortcut in Windows. Here's the "info ln" on it:

"Symbolic links" ("symlinks" for short), on the other hand, are a special file type in which the link file actually refers to a different file, by name. When most operations (opening, reading, writing, and so on) are passed the symbolic link file, the kernel automatically "dereferences" the link and operates on the target of the link. But some operations (e.g., removing) work on the link file itself, rather than on its target.

You create a symbolic link by typing from the command line "ln -s [actual directory] [target name]". So if you're in your home directory, say /home/barry, and you want to create a link in this directory called Root which points to /root, you'd type "ln -s /root Root". Then if you were in /home/barry and typed "cd Root" you'd end up in /root. You can identify a symbolic link by "ls -l [filename]. So if you're in /home/barry and type "ls -l root" you'll see something like:

lrwxrwxrwx 1 barry users 6 2004-02-19 14:58 Xfer -> /mnt/x

where the initial "l" indicates it's a link. I'm not sure how this gets represented uniquely when viewed across Samba from a Windows machine, however, although there's surely a way to distinguish a link from an actual directory. Again, tar handles this fine.

Hope this mini-tutorial helps - I think being able to back up mounted Samba filesystems would be a great asset for AIS Backup.

Bill
Barry
Site Admin
Posts: 1529
Joined: Tue Aug 20, 2002 3:16 pm

Post by Barry »

Hi

I have got an update on Samba:

To stop Samba identifying to Windows directories that are symbolic links add the parameter to the appropriate share section of smb.conf:

follow symlinks = no

For good measure I also tried adding:

Hide unreadable = yes

But this caused severe performance issues.

AISBackup did not end up in a recursive loop but if all folders are selected there are many files that fail to backup, mainly owing to permission issues.

I do not think I can help with linked files (or rather I do not know if I can help) because they look to Windows like regular files.

Barry
cyclone
Posts: 14
Joined: Tue Feb 17, 2004 11:11 am
Location: Silicon Valley

Post by cyclone »

Great, I'll try that and report back how it works.

Bill
Barry
Site Admin
Posts: 1529
Joined: Tue Aug 20, 2002 3:16 pm

Duplicate filenames

Post by Barry »

Hi Bill

I have noticed a problem with duplicate filenames, obviously they are not duplicate on Linux systems because filenames are case sensitive. I tried dragging and dropping two files using Windows Explorer to a Windows folder, the second asked "Replace 4KB file with 1KB file" (as expected) but Windows actually transferred the 4KB file again. This was under Windows XP.

This problem also causes an internal problem with AISBackup, which we will fix before the final release of version 1.9 (even if for some reason we cannot backup all files).

Barry
Barry
Site Admin
Posts: 1529
Joined: Tue Aug 20, 2002 3:16 pm

Post by Barry »

The following samba parameters fix the backup problem with case sensitive filenames:

preserve case = yes
case sensitive = yes


Restore is a problem because the file version information held within AISBackup has to be case in-sensitive for Windows file systems. Right click / Restore As does work correctly.

Please wait for build >= 204 for the fixes to the backup of case sensitive filenames.

Barry
Post Reply