This is an unofficial mirror of pondini.org by James Pond, who is unfortunately no longer with us.

All rights belong to James' beneficiaries.

No guarantee is made about the accuracy or completeness of this mirror.

 

Time Machine - Troubleshooting

D10a  Clues to the "Skip Folders" bug

Previous        Frequently Asked Questions        Home     Troubleshooting      Contact      Next

 

  Previous           Frequently Asked Questions          Home        Troubleshooting         Contact        Next

The actual cause is unknown so far.  It appears to happen only when certain 3rd-party apps are installed, but exactly which ones, under what circumstances, and how it happens, remains a mystery.

What is clear is the mechanism:  some process is adding instructions to the Time Machine Parameter List (/Library/Preferences/com.apple.TimeMachine.plist) telling Time Machine not to back up 7 top-level folders, including /Applications, /Library, /System, and the hidden /private folder (user home folders, other top-level folders, and other volumes aren't affected, so are backed-up normally).


The absence of those folders from backups means:

  1. •The backups aren't complete, so don't appear for selection when doing a full system restore via the Recovery HD.

  2. •Since user accounts are stored in /private, Setup/Migration Assistant can't recreate them, much less associate them with their home folders, so the backups aren't even recognized by the Assistants.   The backup drive or sparsebundle doesn't appear as a selection option, as if it's not connected.

  3. •These exclusions are "Fixed Path" exclusions, in a separate portion of the plist, that users cannot set via the Time Machine Preferences window, and are not shown there.  Thus users have no idea there's a problem, as they rarely look for anything besides data in their home folders, which appear normally via the Time Machine browser.


"FIXED PATH" EXCLUSIONS

Extra Apple and 3rd-party apps may set these exclusions, for such things as cache files that exist and are used only by that app, by passing them to the CSBackupSetItemExcluded Apple API.  The file/folder path is then placed in the ExcludeByPath section of the plist.  This is separate from the SkipPaths section set by and displayed via the Time Machine Preferences window. 

(That API can also set "Sticky" exclusions that follow a file that's moved or renamed, but those don't go in the plist and aren't a problem here.  Both kinds can also be set via the tmutil command, but it seems unlikely to be the culprit, as it probably calls the same API.) 

In some circumstances, another section is added: IncludeByPath.  That contains the 7 top-level libraries, all of which should be backed-up unless specifically excluded by the user.


WHAT GOES WRONG

In a relatively few cases, instead of adding the 7 libraries to the IncludeByPath section, they're added (or moved) to the ExcludeByPath section instead.   Thereafter, of course, the backups are incomplete.

In most, but not all, of the cases I've seen, Microsoft/PlayReady appears, as in the samples below.  Drive Genius is also in a few of them, and some other 3rd-party apps are named, too.  The significance is, well, unknown.

THEORIES: 

  1. •Something in the API puts the 7 libraries in the wrong section under some odd circumstances.

  2. •The backupd process, when updating that section, does it wrong in some cases.  It seems that cfprefsd is updating the plist when a backup runs. 

  3. •Further investigation shows frequent changes to the plist while a backup is running;  in many, but not all, cases the libraries are excluded momentarily (10 seconds at most), then included.  That sounds like whatever is supposed to include them again fails on occasion, triggering the bug.

  4. •An Install failed after excluding the libraries temporarily, but before re-including them.  From an Install Log:

  5. Sep  4 16:03:11 JPsiMac10v8.local installd[1588]: PackageKit: prevent user idle system sleep

  6. Sep  4 16:03:11 JPsiMac10v8.local installd[1588]: PackageKit: suspending backupd

  7. Sep  4 16:03:11 JPsiMac10v8.local installd[1588]: PackageKit: Executing script "preinstall" in /private/tmp/PKInstallSandbox.yPluFZ/Scripts/com.apple.pkg.<deleted>.hklmYy

  8. Sep  4 16:03:12 JPsiMac10v8.local install_monitor[1598]: Temporarily excluding: /Applications, /Library, /System, /bin, /private, /sbin, /usr

  9.  

  10. followed later by:

  11.  

  12. Sep  4 16:03:39 JPsiMac10v8.local install_monitor[1598]: Re-included: /Applications, /Library, /System, /bin, /private, /sbin, /usr

  13. Sep  4 16:03:40 JPsiMac10v8.local installd[1588]: PackageKit: releasing backupd

  14. Sep  4 16:03:40 JPsiMac10v8.local installd[1588]: PackageKit: allow user idle system sleep

  15. Sep  4 16:03:40 JPsiMac10v8.local installd[1588]: PackageKit: ----- End install -----

  16. In addition, similar behavior seems to occur when installing Sophos (and perhaps other) anti-virus updates, which apparently can happen hourly.

A DAMAGED PLIST    (from the same Mac as the one in the blue box):





The ExcludeByPath section now contains the 7 volumes in addition to the "real" one.

The IncludeByPath section is gone.




SkipPaths remains unchanged.

 
A NORMAL PLIST  (with a "Fixed Path" exclusion, from a
different Mac:




The ExcludeByPath section has been added.

IncludeByPath is added sometimes, sometimes not.




SkipPaths items excluded by the user are unchanged.

 

A NORMAL PLIST  (without "Fixed Path" exclusions) will look something like this:




There are no ExcludeByPath or IncludeByPath sections.



Note the SkipPaths section containing items selected via the Preferences Window.

 

SOME RELATED THREADS in the Apple Support Community forums

As usual, there's lots of duplicate and/or irrelevant info in them, but any number of users have seen this problem.  Many had already fixed it, so no longer have the evidence, thus haven't reported it.

In addition, before we realized there was a problem, we assumed several users who reported not being able to restore and/or use Setup/Migration Assistant had excluded System Files and took their denials with a grain of salt, or had damaged backups.  I didn't bookmark any of those threads.

Plus, several users contacted me directly, and never posted in the forums. 

  1. Serious Time Machine bug on Mountain Lion

  2. UPDATED: Serious Time Machine bug on Mountain Lion

  3. Trouble recovering latest TM backup to new SSD

  4. Problems With Migration Assistant & Time Capsule

  5. Mountain Lion: Time Machine Backups Get Very Big!

  6. Time Machine restore point failure - a WARNING!

  7. Time machine running out of space quickly-Why?

  8. Time Machine intermittently backs up a large amount unnecessarily

  9. Cannot Restore from Time Machine on Time Capsule 10.8.4

  10. Time Machine doesn't back everything up

  11. TimeMachine: Restore incremental backup

  12. Time Machine only backing up Users folder

  13. time machine not backing up system files


Not to belabor the point, but some users may have posted elsewhere, or not at all.

But the MAIN CONCERN is, since it's rare for a user to notice this before it's too late,

HOW MANY OTHERS HAVE IT, BUT WON'T KNOW UNTIL IT'S TOO LATE ????

SAMPLES

The Time Machine plist is a binary file, so can't be deciphered easily with most text editors;  a Plist Editor app such as the paid PrefEdit, will.

Or, if you're comfortable with Terminal:

  1. •Copy the file to your Desktop

  2. •Convert it to XML via Terminal:    plutil -convert xml1 ~/Desktop/com.apple.TimeMachine.plist

  3. •Then you can open it with a text editor that recognizes XML.  It may display in XML format, or as in the samples below.  TextEdit displays as shown.