Radmind and "Applefiles"
When using radmind on OS X radmind has to deal with some filesystem quirks that are unique to OS X. A file is not just a file. Besides the contents of the file, each file has metadata, information about the file. The most basic metadata is the file name. Other metadata includes permissions, creation date, last modification date, size, etc.,. OS X also has metadata called extended file attributes. Files can have an arbitrary number of extended attributes and the attributes have a name and data.
Radmind stores typical metadata like owner, permissions, and modification dates in transcripts. Extended attributes need to be saved somehow or else some files could actually be destroyed. Folders on OS X also have metadata beyond the typical and that data is stored in 2 places, as an extended attribute named com.apple.FinderInfo and in .DS_Store files that are in the folder.
Although some extended attributes (like the com.apple.ResourceFork) are absolutely required or else the file is destroyed, most is completely irrelevant and we do not ever want to upload it to the radmind server. The radmind tool fsdiff is the tool that senses if a file has extended attributes or not and it does not have any way of deciding if extended attributes should be saved or not, so it saves it all. So if you want to avoid uploading files with extended attributes to the server, you need to remove the extended attributes before you run fsdiff.
The different attributes are stored in different forks. Forks are just different ways of linking data with the same file. In the pre-OS X days of Mac OS, the "resource" forks stored images and other data like GUI information, and the "data" fork stored the normal data stored in files.
Extended attributes and radmind
This is what fsdiff prints when it scans a file and folder with extended attributes.
d ../Help/Contents 0755 0 0 AAAAAAAAAAABAAABAAAAAAAAAAAAAAAAAIAAAAAAAAA= a ../Help/Contents/Info.plist 0755 0 0 1217351645 836 4sKSZzoaSYAHSoI/wY+lXImdGQY=
The capital letters after the permissions on the directory is base64 encoded com.apple.FinderInfo extended attributes. The "a" that starts the file line is the flag that the extended attributes for the file has been combined with with the file data, which is what radmind calls an "applefile."
Note, the term "applefile" is used exclusively by the radmind community, so be aware that non-radmind users wont know what an applefile is. However, the applefile is really an AppleSingle, which is a format defined by Apple.
If you open an applefile in BBEdit you will see something like this:
> ^ : %o ,TEXT 2 2 ??30 2MWBB O^^<?xml version="1.0" encoding="UTF-8"?> ...
The first part is binary data that is the meta information (it looks weird because the binary data is being show as if it were ascii characters, many of which are unprintable so there are spaces for those spots). The
"<?xml" begins the actual file.
If you ever plan on editing files on the radmind server, you do NOT want to edit the file if it is an applefile. If you really must, you need to remove all applefile data and then change the transcript from "a" to "f".
The applefile manpage
Here is the "applefile" manpage (this man page comes with radmind). Yes, I'm just going to quote the whole thing. Skip it if you want, but it is kind of required reading.
To provide support for Mac OS X, radmind is aware of the Mac OS meta- data used on HFS+-formatted drives. When lcreate uploads a file denoted in the transcript as an applefile (type 'a'), it encodes the file and the file's metadata to the server as an AppleSingle file. The AppleSin- gle file is an archive containing a file's relevant metadata and data in a single file stream. The radmind AppleSingle file has the following format: 1) The AppleSingle header, consisting of a 4-byte magic number, 4-byte version number, 16-byte filler, 2-byte number denoting the number of entries in the archive, and 3 header entries, each describing the items in the archive and containing a 4-byte entry id, 4-byte offset, and 4-byte length; 2) The file's Finder information (32 bytes), which stores the file's Type and Creator codes, as well its position in the Finder window; 3) The file's Resource Fork, which contains things like images, sounds, and other data not easily or appropriately stored in the Data Fork; 4) The file's Data Fork, the portion visible from the Finder, and the only part which is considered valid by flat file systems. The radmind tool lapply similarly decodes the applefile from the server and restores it to the client Mac OS X machine running on an HFS+-for- matted drive.
Desirable extended attributes
So do you know what extended attributes are needed and which are not needed? Right now I only know of a few reasons for distributing extended attributes with radmind.
Hidden folders (notice the 1 B? That tells the Finder the folder is hidden):
d ./Users/template 0770 502 80 AAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
Icon files for folders (File name is "Icon\r":
a ./Applications/Microsoft\bOffice\bX/Icon\r 0755 0 0 1003474800 58422 PS1n4dm97H+In25geiIvxuScc04=
Icon files for files. There is no way to see this within a transcript. It will just look like any "a" line.
Alias files. There is no way to see this within a transcript. It will just look like any "a" line.
File that end with ".webloc" or ".textClipping" files (they are really the same thing, just different extension):
a ./Users/template/Desktop/Report\ba\bproblem.webloc 0644 606 20 1322003670 271022 jmTWIrG7PDgPnsqck1xXsSPVyUI=
If you are dealing with a file or folder that doesn't have either of those, that means it is probably useless information. Although it isn't mandatory that you remove it, it makes things cleaner.
As a side note, .DSStore files also store metadata for folders, and besides icon positions on the Desktop, we haven't found a single reason to distribute a .DSStore file.
Useless extended attributes
What types of extended attributes can you get rid of? Probably everything except the few mentioned above. But honestly, you never really know what is required without looking. But one thing you can do is get rid of the common attributes that you will see when you've installed something for the first time. Do that with these 2 commands.
xattr -dr com.apple.FinderInfo / xattr -dr com.apple.quarantine /
After you get rid of these attributes, you will want to see if there are any others remaining. You can do that with this command.
ls -lR@ | grep " "
That grep is looking for the tab character (between the quotes). You can't type that command in the shell because the tab key is used for tab completion. There is probably a way to do it but I don't know how. So to run that command, I saved it to a file and ran the file. I saved it to ~/print_xattrs and I ran it with
sh ~/print_xattrs. There is probably a better way to do this, but I don't exactly know what to look for to find it.
What other type of extended attributes are out there? I scanned my hard drive and this is my list. I don't really want to figure out how much of this would really be needed. Most of it is com.apple.FinderInfo and not needed. Most of this is also created by the OS, not by installers.
app_description AppCrashCount AppDuration bug_type com.apple.AddressBook.ImageTransform.ABClipRect_1 com.apple.AddressBook.ImageTransform.ABClipRect_1_hash com.apple.backup.SnapshotNumber com.apple.backup.SnapshotVersion com.apple.backupd.BackupMachineAddress com.apple.backupd.HostUUID com.apple.backupd.ModelID com.apple.backupd.SnapshotCompletionDate com.apple.backupd.SnapshotContainer com.apple.backupd.SnapshotStartDate com.apple.backupd.SnapshotState com.apple.backupd.SnapshotType com.apple.backupd.SnapshotVolumeFSEventStoreUUID com.apple.backupd.SnapshotVolumeLastFSEventID com.apple.backupd.SnapshotVolumeUUID com.apple.backupd.VolumeBytesUsed com.apple.backupd.VolumeIsCaseSensitive com.apple.diskimages.fsck com.apple.diskimages.recentcksum com.apple.DTDeviceKit.screenshot.device_id com.apple.FinderInfo com.apple.metadata:_kTimeMachineNewestSnapshot com.apple.metadata:_kTimeMachineOldestSnapshot com.apple.metadata:com_apple_backup_excludeItem com.apple.metadata:FileSyncAgentExcludeItem com.apple.metadata:kMDItemAttributeChangeDate com.apple.metadata:kMDItemDownloadedDate com.apple.metadata:kMDItemFinderComment com.apple.metadata:kMDItemIsScreenCapture com.apple.metadata:kMDItemLastUsedDate com.apple.metadata:kMDItemScreenCaptureType com.apple.metadata:kMDItemSupportFileType com.apple.metadata:kMDItemWhereFroms com.apple.metadata:kMDLabel_zya2exypzrhulknkk5enqbj33y com.apple.Preview.UIstate.v1 com.apple.quarantine com.apple.ResourceFork com.apple.TextEncoding com.apple.xcode.PlistType com.apple.XcodeGenerated com.dropbox.attributes com.macromates.caret com.vmware.backupReenabled displayName LAST_UPLOADED name NSImageMetadata os_version ReopenPath SampleWeight SubmissionSkipped SystemCrashCount UserDuration UserMissingLibrary version
Digging into extended attributes
If you want to learn how to figure it out what to keep and what not to, here are a few commands to show you how to identify extended attributes from Terminal.
Whenever you use
ls -l, it will list an @ when the file has extended attributes.
ls -al /Users/template/Desktop/ ... -rw-rw-r--@ 1 template staff 134 Nov 22 16:14 Report a problem.webloc
To view the names of the extended attribute, just add "@" (for attribute) to ls.
ls -al@ /Users/template/Desktop/ ... -rw-rw-r--@ 1 template staff 134 Nov 22 16:14 Report a problem.webloc com.apple.FinderInfo 32 com.apple.ResourceFork 270794
To actually view the extended attribute, use the xattr command with -p (for print).
xattr -p com.apple.FinderInfo /Users/template/Desktop/Report\ a\ problem.webloc 00 00 00 00 00 00 00 00 04 50 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
To delete extended attributes, use xattr again with -d (for delete). If you forget the flags, just type "man xattr".
xattr -d com.apple.FinderInfo Report\ a\ problem.webloc ls -al@ /Users/template/Desktop/ ... -rw-rw-r--@ 1 template staff 134 Nov 22 16:14 Report a problem.webloc com.apple.ResourceFork 270794
If you want to see what com.apple.FinderInfo is then use a tool that comes with the developer tools, /Developer/Tools/GetFileInfo. This is for a hidden directory.
/Developer/Tools/GetFileInfo /Volumes directory: "/Volumes" attributes: aVbstclinmedz created: 02/03/2012 17:32:10 modified: 03/08/2012 17:36:27
This is for a file.
/Developer/Tools/GetFileInfo /Users/template/Desktop/Report\ a\ problem.webloc file: "/Users/template/Desktop/Report a problem.webloc" type: "\0\0\0\0" creator: "\0\0\0\0" attributes: avbstclinmedz created: 11/22/2011 16:14:30 modified: 11/22/2011 16:14:30
This is the meaning of the letters (this comes from the man page on GetFileInfo). Lowercase is off, uppercase is on.
a Alias file b Has bundle c Custom icon (allowed on folders) d Located on the desktop (allowed on folders) e Extension is hidden (allowed on folders) i Inited - Finder is aware of this file and has given it a location in a window. (allowed on folders) l Locked m Shared (can run multiple times) n File has no INIT resource s System file (name locked) t "Stationery Pad" file v Invisible (allowed on folders) z Busy (allowed on folders)
Here are some links to discussions extended attributes.
Copyright 2016 James Reynolds