MacInTouch Amazon link...

system synchronization

Channels
Other, Questions

Ric Ford

MacInTouch
Econ Technologies (ChronoSync) notes the problem ... The ChronoSync user manual has more details, including the suggestion of creating multiple smaller tasks that you then combine into a larger sync operation.
The ~/Library folder is a real can of worms, mixing critical files that need to be synched (e.g. email) with others that should not be (e.g. hardware-specific licenses/parameters).

Meanwhile, here are some notes I found in Bombich Software's Carbon Copy Cloner documentation about all these kinds of issues:
 


The ~/Library folder is a real can of worms, mixing critical files that need to be synched (e.g. email) with others that should not be (e.g. hardware-specific licenses/parameters). Meanwhile, here are some notes I found in Bombich Software's Carbon Copy Cloner documentation about all these kinds of issues:
I agree with Ric and many others that trying to sync Home folders between computers has many issues, even if they use the same OSX. If different versions, consider it to be impossible - too many differences in preferences, email databases, etc. From a tech standpoint, it’s a nightmare in working with customers who want to sync info between computers and not worry about which computer the work is done on. Then, I would recommend a backup program that has two-way sync abilities. One such that I have used is Tri-Backup. I would only sync specific folders such as Documents, Music and Pictures (only if same OSX, due to versions of Photos file format) and any specific work folders that are needed. This way the data files, which are the most important, are kept in sync, no matter which computer is used to edit those files. It’s like using Dropbox, but only local file syncing, no online copy.
 


Ric Ford

MacInTouch
Here's a question I haven't answered yet: Would using disk images (e.g. a "sparse bundle disk image" created in Disk Utility) offer any benefits for organization/sync vs. folders? (I can imagine creating a disk image, for example, to hold all one's photos.)
 


I'm looking to drop Dropbox, and Sync.com is a promising alternative.
Thanks for the tip. I haven't used Dropbox (or anything similar) in a while, and I'd rather not. I like the look of Sync (Sync is like Dropbox, but 100% private), but noted that while they offer apps for Mac, Windows, iOS and Android, there's nothing for Linux. So I looked around, and found:
Sync.com said:
Can I install Sync on Linux?
A Linux app is on our long-term roadmap, however, not available at this time. In the mean time you can access the Sync.com web panel on Linux using a supported web browser."
Looked around a little more, and found:
Sync.com said:
What is the Sync.com web panel?
The Sync.com web panel provides secure web-based access to your cloud storage and account settings.
Looks good enough for me, while they get up to speed with Tux.
 


Ric Ford

MacInTouch
I like the look of Sync but noted that while they offer apps for Mac, Windows, iOS and Android, there's nothing for Linux....
I appreciate the notes on Linux status, but I had already set up a free Sync account and downloaded the Mac app to two macOS 10.12 Sierra computers - seemed to work well - then I simply used a web browser to access Sync from Linux Mint, which also seemed to work fine. I also got the iOS app. I haven't really stress-tested anything, but so far, so good. Next will be copying files from Dropbox, setting up access for people who need it, and finally dropping Dropbox....
 


Ric Ford

MacInTouch
... as many have noted, a file-sharing option is a major requirement/benefit — I'm looking to drop Dropbox, and Sync.com is a promising alternative.
Thanks for the tip. I haven't used Dropbox (or anything similar) in a while, and I'd rather not. I like the look of Sync (Sync is like Dropbox, but 100% private), but noted that while they offer apps for Mac, Windows, iOS and Android, there's nothing for Linux. So I looked around
Just a few more quick notes:

There can be some delay in Sync'ing... I just shut down one Mac, not realizing it hadn't finished a sync of 5.1 MB of files (at roughly 20 Mbps) and was surprised to find the folder incomplete on another Mac, so I had to go back and restart the first one to finish up.

I also didn't realize it immediately, but Sync is a menubar app, so that's the place to go to see and manage operations in more detail (vs. the Sync folder in the Finder).

Lastly, I'm using Sync successfully within Path Finder 7 (instead of the Finder).
 


re: Sync vs Dropbox...

I have two questions:
1) What's wrong with Dropbox?​
2) There are a lot of apps with built-in support for Dropbox. I assume that switching means you give those up as well?​
 



Dropbox fails to provide end-to-end encryption and, in fact, accesses and processes your files and metadata
To which I will add that Dropbox, in an effort to save storage, was found to be running de-dupe on a massive scale. Imagine how much space that might save by storing only one instead of millions of copies of a top-selling song MP3? I've not followed Dropbox since I no longer use it, but that de-dupe process had a couple of potential gotchas. The MP3 you download probably won't be "yours." Some providers were embedding ID info into their downloads, to facilitate tracking of pirated music. The other, and much more hypothetical possibility, is the metadata of a data file you upload might exactly match someone else's previous upload....

Ben Sandofsky posted on Twitter that the "new" Dropbox app uses half a gig of memory on a Mac. His observation has been repeated around the internet and on podcasts, but I've not found other sources.

I read the information Sync provides about encryption, and it seems very credible. My own concerns about any service like that is it could be used as a honeypot, a way to attract the kind of users three-letter agencies and law enforcement really want to find. We saw that, when a "secure" US email provider simply shut down, and given our national security orders including gags, offered no comment. Not just being paranoid, if you really, truly, want your "stuff" private, don't put it on the cloud, and, if you do, use a pre-transmission encryption program of your choice, not one provided by the cloud service.
There can be some delay in Sync'ing... I just shut down one Mac, not realizing it hadn't finished a sync of 5.1 MB of files (at roughly 20 Mbps) and was surprised to find the folder incomplete on another Mac
From an early experience with a shared "business" Dropbox account, there's a variety of reasons sync might break — time-based ones, in particular. If your share crosses ISPs, there may be a slight variance in their "internet time" — the same for even two local computers on your own LAN. Shutting down one system with files open... [not good].
 


To which I will add that Dropbox, in an effort to save storage, was found to be running de-dupe on a massive scale. Imagine how much space that might save by storing only one instead of millions of copies of a top-selling song MP3? I've not followed Dropbox since I no longer use it, but that de-dupe process had a couple of potential gotchas. The MP3 you download probably won't be "yours." Some providers were embedding ID info into their downloads, to facilitate tracking of pirated music. The other, and much more hypothetical possibility, is the metadata of a data file you upload might exactly match someone else's previous upload....
Dropbox could be de-duping in one of two ways:
  • Check if the entire file is the same
  • Check if blocks of the file are the same
In your MP3 example:
  • If looking at the entire file, your file would be different due to the embedded ID info. So, your file would be stored independently
  • If looking at file blocks, the duplicate blocks would be de-duped, but the blocks that contain the embedded ID info would be stored independently
So in either case, the risk of Dropbox doing cross-user de-duplication is not that you'd download someone else's file. Rather, there is a different privacy implication. Doing cross-user de-duplication opens the possibility of side-channel attacks, similar to the various CPU cache attacks that have been in the news.

Let's say a government has a super-secret document. They want to know if anyone else has a copy of it. So the agent puts the document in a Dropbox. If the document uploads instantly, then that means that Dropbox already had the document uploaded by some other Dropbox user. Time to submit a subpoena to Dropbox to find out who those users are.
 


the risk of Dropbox doing cross-user de-duplication is not that you'd download someone else's file.
My understanding, I think I picked up from Steve Gibson's Security Now Podcast a very long time ago, was that Dropbox designed a de-dupe system so all of Dropbox would have only one version of popular songs. Again, it's been a long time, but I believe users began to notice, similar to what I recall about iTunes Match, not getting back the same album art, or getting back a "clean" version of a track NSFW. The question there would be what's the canonical song? One Dropbox determines is exactly what would rip from a CD, or the first version a user uploads? If, in fact, the goal of Dropbox was to save storage by storing only one (or a limited variety) of popular songs, Dropbox would have learned to handle purchaser data embedded, for example, in music purchased from iTunes.

StackExchange:
Let's say a government has a super-secret document. They want to know if anyone else has a copy of it. So the agent puts the document in a Dropbox. If the document uploads instantly, then that means that Dropbox already had the document
Ingenious! But...
If the document truly is super-secret, uploading it to Dropbox ends that. Instantly? I'm pretty sure the document would have to upload in its entirety, and possibly sit for some time in a user's account, before Dropbox servers could calculate its ID code to compare for de-duplication.
 


My understanding, I think I picked up from Steve Gibson's Security Now Podcast a very long time ago, was that Dropbox designed a de-dupe system so all of Dropbox would have only one version of popular songs. Again, it's been a long time, but I believe users began to notice, similar to what I recall about iTunes Match, not getting back the same album art, or getting back a "clean" version of a track NSFW.
Since they're running a file archiving service, not a music library, getting back anything less than a bitwise copy of what you uploaded would be unacceptable.

In the case of iTunes, Apple very explicitly stated that your songs would be replaced with copies from the iTunes library wherever possible, so there was never any deception (although there have definitely been bugs).

In the case of Dropbox, they have (as far as I know) made no such claims. So, some kind of algorithm to look for music files and perform semantic matching against other similar files would probably violate all kinds of contractual obligations.
If the document truly is super-secret, uploading it to Dropbox ends that. Instantly? I'm pretty sure the document would have to upload in its entirety, and possibly sit for some time in a user's account, before Dropbox servers could calculate its ID code to compare for de-duplication.
Depends on how the service works. If the client-side software performs the checksum/hash calculation before the upload, then you might be able to notice something.

Of course, it would only prove that somebody else has a file matching a hash, probably meaning that someone else has a file. They'd still need a court order to discover who else uploaded the file, and even with such an order, it might not be all that easy to determine it, depending on how their storage system is organized.
 


Dropbox fails to provide end-to-end encryption and, in fact, accesses and processes your files and metadata on their servers (e.g. to provide previews, searching, etc.), as well as having had security issues. In addition, the company recently raised its prices unilaterally.
Thanks for that. I had no idea that Dropbox was reading my files. Is that documented? That sucks, and may be enough of a reason to leave. I'll note that if you want end-to-end encyrption, you can have it with Dropbox by using Boxcryptor, free if you only use it on one site. (For $40 per year, you can use it "on over 30 sites", including iCloud drive, Google, OneDrive etc.)

I've used it for a couple of years now, and it's about as simple as it can be and still work. Instead of dropping confidential files on Dropbox, you drop them on the Boxcryptor folder. As long as you have BoxCryptor running, you access them the same way, and they are instantly decrypted for you.

The rub to Dropbox was my issue #2 - a zillion different apps use it as an option. Nothing else seems to come close. So, giving up Dropbox, you're giving up a lot of convenience, too.

Personally, I run my own server close to the backbone for my clients, and I have a Synology here in the house. Between the two, I have a lot of options, and I'm happy with the no-cost level at Dropbox. I like Amazon S3 (or DreamObjects) for some things, but for me, until I can access the internet at terabyte speeds, it's all a compromise.

The question then becomes what do you want to compromise?

If you don't want someone else to have it, don't put it on the public web, and if I'm not using BoxCryptor, I basically don't trust anyone, so most everything simply stays right here....
 


I appreciate the notes on Linux status, but I had already set up a free Sync account and downloaded the Mac app to two macOS 10.12 Sierra computers - seemed to work well - then I simply used a web browser to access Sync from Linux Mint, which also seemed to work fine. I also got the iOS app. I haven't really stress-tested anything, but so far, so good. Next will be copying files from Dropbox, setting up access for people who need it, and finally dropping Dropbox....
It's important to consider how the synchronization program handles errors. Does it put up an error dialog? Send you an email? Fail silently?

For example, Sync (and other similar programs) won't synchronize files containing certain "forbidden" characters.

Sync handles such files by ignoring them during the synchronization process. The Sync program won't proactively inform you about files that it ignored. You're expected to go check for yourself.

The forbidden names are enforced on all platforms. Even if you're syncing one Mac to another, Sync won't copy any files with names that aren't kosher on Windows.

What names are forbidden? Sync documentation is rather alarming (and ambiguous), but it's a good place to start your investigation:

Thus I recommend that you do some testing before committing your workflow to a certain synchronization tool.

For example, try synchronizing a directory containing files with interesting names. Here are a few examples to get you started:

"Test file with forbidden character \.pdf"​
"Test file with forbidden character sequence ~$.pdf"​
"Test file with some forbidden characters ;'"<>,.?/!@#$%^^&%&*()_+=-`{}[]"'|\"​
Sync fails to synchronize files with these names (among many others, of course).
 


Ric Ford

MacInTouch
Thanks for that. I had no idea that Dropbox was reading my files. Is that documented?
This may give some sense of the kinds of things Dropbox is doing now:
Fast Company said:
How Dropbox is finally breaking free of the folder
... The existing menu that pops out from Windows’ tray and MacOS’s menu bar doesn’t look much different, but it’s been retooled to show the files that your colleagues are sharing, editing, and commenting upon: “It’s not just about your sync activity or files that you’ve edited, but what’s going on with everyone in your group,” explains Adam Nash, Dropbox’s VP of product. The menu also offers newly sophisticated search, similar to that in the web version, that plumbs the content of files rather than just scanning their names.
You might also find this interesting...
The Phoblographer said:
Dropbox Partners With Unsplash in Deal That Rips off Photographers
... Dropbox paper is simple. Sign up, create documents and project boards in the cloud, and then collaborate with your team. Oh, and now you can use over 850,000 images that Unsplash has swiped the image rights to for free. That’s right. Millions upon millions of people could very well use your image as a stock image in their online documents with Dropbox Paper and you won’t receive a single penny. Marvelous!
 



Ric Ford

MacInTouch
Maybe iCloud turns out to be the most private after all ?
Nope, Apple also fails to provide end-to-end encryption.
EFF said:
Meanwhile, some additional, interesting Dropbox reading material:
Wikimedia said:
Criticism of Dropbox
Criticism of Dropbox centers around various forms of security and privacy controversies surrounding Dropbox, an American company specializing in cloud storage and file synchronization. Issues include a June 2011 authentication problem that let accounts be accessed for several hours without passwords, a July 2011 Privacy Policy update with language suggesting Dropbox had ownership of users' data, concerns about Dropbox employee access to users' information, July 2012 email spam with reoccurrence in February 2013, leaked government documents in June 2013 with information that Dropbox was being considered for inclusion in the National Security Agency's PRISM surveillance program, a July 2014 comment from NSA whistleblower Edward Snowden criticizing Dropbox's encryption, the leak of 68 million account passwords on the Internet in August 2016, and a January 2017 accidental data restoration incident where years-old supposedly deleted files reappeared in users' accounts.
 



Ric Ford

MacInTouch
OK... although again, you could use Boxcryptor, if that is your only complaint.
See above. Other options include Cryptomator and encrypted Mac disk images, but neither seems as convenient as Sync.com. (I haven't yet reviewed the filename issues highlighted in Jeff Stearns's post.)

I'm not very interested in having large companies analyzing all my documents with their A.I. systems, though that's becoming more and more prevalent, so that's mainly what I'm looking for in alternatives to iCloud, Dropbox, Google Drive, etc.
 




I'd consider Sync.com as an alternative to Dropbox. What do we know about the company, other than it has a pretty website and seems to be in Canada?

And how do my friends download my files - I send them a URL (like Dropbox) and a password?
 


I'm looking to drop Dropbox, and Sync.com is a promising alternative.
Have you considered running Nextcloud? It appears to have all the security you need as well as Linux clients. It can run on your own Linux machine so your data need never leave your control.

I have it running as part of my NethServer install using VirtualBox on an old Mac. For me it does everything I used to use Dropbox for but without data limits or cost.
 


Ric Ford

MacInTouch
Have you considered running Nextcloud? It appears to have all the security you need as well as Linux clients.
That's been on my research list but I hadn't gotten to it yet — it seemed a little complicated at a glance, but it's good to hear it's working for you. Definitely something to investigate further - thanks!
 


The Synology "Drive" application works well as a user-managed and controlled replacement for Dropbox and other cloud services. It's encrypted at rest, encrypted in transit, and accessible only by authorized users with appropriate permissions. I chose and set the passwords and activated required Two Factor Authentiction which uses Google Authenticator.

Of course, there's that basic matter of trust. Best I can tell, Synology, the company, really can't access the passwords and keys I set up, but just like a cloud service such as Dropbox, there's a risk with anything connected to the internet and an appliance that's both connected to the Internet and which receives proprietary system updates over the 'net.

I've heard a lot about Nextcloud and ownCloud. As server-based solutions they've seemed a bit beyond my reach. The sorta' turn-key Synology has been challenge enough! In looking for context about this thread, I found today that it's possible to run Nextcloud on a local Raspberry PI-based applicance. Not that I'm moving away from my new Synology, but an intriguing idea...
Own Your Bits said:
NextCloudPI
NextCloudPi is a Nextcloud instance that is preinstalled and preconfigured, and includes a management interface with all the tools you need to self host your private data in a single package. This is an official open source community project that aims at making it easier for everyone to have control over their own data. NextCloudPi is free, thriving only on your help, feedback and support.
 


Dropbox just announced that symlinks will no longer be followed:


Dropbox syncs any files and folders that reside in your Dropbox folder, of which there can be only one. Up until now, you could also sync files and folders outside of the Dropbox folder by creating a symlink from Dropbox to that file/folder.

For example (and this is just an example): I could create a symlink ~/Dropbox/iMac_Desktop that pointed to ~/Desktop, and all of the Desktop contents would be synced. On dropbox.com it would appear as though I had an iMac_Desktop folder in my dropbox, containing all of those items.

If you did nothing else, then Dropbox would sync the iMac_Desktop to other devices as being inside the Dropbox folder. But you could do the same symlink trick on other devices, too. If I create a symlink from ~\Dropbox\iMac_Desktop to my MacBook's Desktop, then Dropbox would push all the file changes out to that machine -- true two-way syncing to folders outside of the Dropbox.

This feature was useful when you wanted to sync application data across machines, where it wasn't possible or convenient to have the data actually in the Dropbox. For example, perhaps the application is insistent that the data be located at a particular location.

Dropbox's abandonment of this long-standing and perfectly functioning feature doesn't eliminate the ability to sync application data. There always were two ways to approach it. The first, as described above, was to symlink from Dropbox to the external location. The other is to work in the opposite direction: put the application data in the Dropbox, but symlink (or hardlink) from where the application wanted the data to be to the actual Dropbox location. The second method will still work.
 


Ric Ford

MacInTouch
OK. I wasn't aware that Apple was also violating our privacy. ("...analyzing all my documents with their A.I. systems...")
This isn't exactly what I meant but may be illustrative:
Apple said:
Privacy Policy
... We also use personal information to help us create, develop, operate, deliver, and improve our products, services, content and advertising, and ... pre-screening or scanning uploaded content...
This isn't directly related to iCloud but illustrates some dangers of silent Apple processing of our documents:
Eclectic Light Co. said:
What I was getting at was more along this line:
Apple said:
https://support.apple.com/en-us/HT207023
Photos scans your library for significant people, places, holidays, and more...
And, of course, Apple is pushing very hard in "analytics", "machine learning" and AI:
Apple Jobs said:
 



Ric Ford

MacInTouch
... how do my friends download my files - I send them a URL (like Dropbox) and a password?
Yes, with a free Sync.com account, you can share a file as a URL link with an optional password, a label, and an "enhanced privacy" option (end-to-end encyption of the file transfer).

Paid accounts get additional options, such as expiry dates, file requests (for an upload), download limits, preview-only (not download), email notifications, etc.

See the Sync Help Desk for lots more details.
 



I'm a long time user of Box and that includes both the Box Sync app and Box Drive. Box Sync will synchronize files on your desktop with your Box account. Box Drive works a little differently in that you have access to your cloud based Box files via a desktop interface. Both work well unless you are trying to manage multiple Box accounts for different entities. Then you have to resort to browser based logins since Box Sync and Box Drive only support access to a primary account.
 





Since using Mojave 10.14.6 with DropBox 82.4.155, all custom folder icons (folders in Dropbox) disappeared and reverted to default icons. I'd like Mac users to be informed to minimise their inconvenience.you can read more here:
Dropbox Community said:
Dropbox is interfering with macOS functionality, without user agreement, with disastrous consequence for Mac users relying on custom folder icons.

Dropbox's position:
To keep a consistent structure throughout your Dropbox account, customized permissions and icons may revert back to their default state, for example into simple folder icons, to ensure efficient syncing.

Please note though that the team is aware of your comments and we thank you for your feedback.
 



Since using Mojave 10.14.6 with DropBox 82.4.155, all custom folder icons (folders in Dropbox) disappeared and reverted to default icons. I'd like Mac users to be informed to minimise their inconvenience. ... Dropbox is interfering with macOS functionality, without user agreement, with disastrous consequence for Mac users relying on custom folder icons.
Thanks for the warning, Jean-Luc. That is very good information.

I think the issue is larger than Dropbox, though. Every syncing service has quirks that may not be obvious to end users, as services attempt to replicate functions which may be specific to particular operating systems while using technologies that are quite independent of the original operating system.

In some cases, it may be as dramatic as not syncing files with certain characters in their names (every platform I've looked at has its own list of "illegal" characters, and behaviors sometimes vary depending on whether you are using Windows or macOS) or as subtle as not faithfully capturing metadata that is normally hidden from end users. As Howard Oakley has detailed in multiple posts, even Apple does not preserve all expected file metadata when syncing across iCloud.

The bottom line is that it is safest to assume that any cloud syncing operation is imperfect and to avoid anything but the most basic, explicitly supported cloud functions. Even if you switch to a cloud platform that works for you today, there is no guarantee that it will work in the same way tomorrow.
 


Ric Ford

MacInTouch
... The bottom line is that it is safest to assume that any cloud syncing operation is imperfect and to avoid anything but the most basic, explicitly supported cloud functions. Even if you switch to a cloud platform that works for you today, there is no guarantee that it will work in the same way tomorrow.
All excellent and critical points. Just one additional thought: Could some of these problems be avoided through use of disk images - e.g. keeping all one's files in a disk image that is stored in the cloud with an ultra-generic filename (to avoid filename compatibility problems)?
 


Could some of these problems be avoided through use of disk images - e.g. keeping all one's files in a disk image that is stored in the cloud with an ultra-generic filename (to avoid filename compatibility problems)?
Disk images are solid solutions for many use cases, though Zip archives may be better solutions in some circumstances, particularly in multi-platform environments. I like that it is simple to create Zip archives through a single Finder "Compress..." menu command or a right-click/ctrl-click.

Also, if the intention is to mount the disk image file and work with its contents interactively, especially if it is a writeable image, the strong recommendation would be to keep the disk image file as small as possible to maintain interactive performance, especially over a network. It's also probably a matter of data integrity, too, as corruption of a large disk image file (writeable or not) can ruin your day, while smaller files may provide less risk.

There are other complications that can arise with disk images and multi-user sharing, so there really isn't a perfect solution, as far as I can tell. To be fair, we've been searching for a perfect syncing/sharing solution for as long as we've been connecting computers into networks with shared drives.
 


Disk images are solid solutions for many use cases, though Zip archives may be better solutions in some circumstances, particularly in multi-platform environments.
Zip archives are particularly convenient on Windows 10, because you can open them just like a folder and browse the contents. You can even (with some limits) open documents in Zip files without having to unpack the archive.
 


Amazon disclaimer:
As an Amazon Associate I earn from qualifying purchases.

Latest posts