{"id":40797,"date":"2021-12-08T15:00:03","date_gmt":"2021-12-08T14:00:03","guid":{"rendered":"https:\/\/staging.kinolios.com\/?p=40797"},"modified":"2024-07-04T21:23:23","modified_gmt":"2024-07-04T20:23:23","slug":"backups-et-integrite-des-donnees-partie-i-freefilesync","status":"publish","type":"post","link":"https:\/\/staging.kinolios.com\/en\/blog\/cinematography\/post-production\/backups-et-integrite-des-donnees-partie-i-freefilesync\/","title":{"rendered":"[En\/Fr] Tutorial : Backups and data integrity part I : FreeFileSync"},"content":{"rendered":"
\n
\n
\n
\n

Backup management and local network sync with FreeFileSync<\/h3>\n

Presentation of Freefilesync and how I use the software to synchronize files between multiple computers connected to the same local network and to help in maintaining my offline backups.<\/p>\n

<\/p>\n

As always, a full list of the articles in this blog can be found HERE<\/a>.<\/em><\/p>\n\n

<\/div>\n

Introduction<\/strong><\/h4>\n
<\/div>\n

This post is the first part of my workflow taking care of preserving the integrity of my data and media files over time.<\/p>\n

My current method of backup up my data is to have a local work copy on a SSD drive on my desktop and to create two copies on two separates hard drives which are kept disconnected and stored in a safe place. I personally use HGST 4To drives after having read the reliability results from Backblaze\u2019s website, which is a cloud storage company using consumer grade hard drives in their servers (link to the results HERE<\/a>). They upload each quarter the reliability results from each of their drive pools and HGST always places among the bests in regards to low failure rates. In a low intensity scenario as is mine (the disks are only spun up a couple times a month at best), it\u2019s perhaps not necessary to have the best reliability but it\u2019s certainly a peace of mind and a way to limit risks. For the same reason, I don\u2019t use external drives as I recall having read that the drives used are the ones failing the standards for desktop use and are therefore lower grade and more likely to fail.<\/p>\n

\"\"
The two disks are protected and stored in plastic cases padded with foam.<\/em><\/figcaption><\/figure>\n
<\/div>\n

The disks are connected to the desktop via a USB 3.0 dock (accepting both 3.5\u2033 and 2.5\u2033 drives) with the capability to access both drives simultaneously (which is a big help to perform backups).<\/p>\n

\"\"<\/p>\n

<\/div>\n

In theory, I should have a third copy of the data on a third drive stored in a separate location from my work, to prevent complete loss of data if a disaster struck my workplace.<\/p>\n

I don\u2019t use and trust Cloud storage services, as I don\u2019t want to depend on internet access or on service providers who could interrupt access at any time and without any means of data recovery.<\/p>\n

<\/div>\n

I first started by copy-pasting manually my media files on the two drives, which quicky led to disparities when I deleted old or superfluous files on one drive and not on the other. Have I deleted this folder on the second drive ? Have I replaced this old file with the new on both disks ? This quickly led to headaches.<\/p>\n

Rather than clone one drive on the other each time I made changes, which could take several hours or even days to rewrite the entire disk, I tried to find a software solution :<\/p>\n

<\/div>\n

Software presentation
\n<\/strong><\/h4>\n
<\/div>\n

FreefileSync is a free and open source folder comparison and synchronization software (even for commercial use). It\u2019s available on Windows, MacOs and Linux at this LINK<\/a>. If I only use basic functionalities, the software can also sync to cloud services (like Google Drive) or via FTP client.<\/p>\n

<\/div>\n

Backup to external storage<\/h4>\n
<\/div>\n

Freefilesync is a perfect fit for my use : create a mirror copy of one disk to the other by comparing the data on them and only copying the changes, not rewriting the whole disk everytime.<\/p>\n

This is how I set it up :<\/p>\n

First I created a new config file on FreeFileSync.<\/p>\n

I added my source and target disks in the two corresponding boxes then opened the comparaison parameters (F6 or the Gear icon).<\/p>\n

\"\"<\/p>\n

<\/div>\n

In my case, I chose the \u00ab date and file size \u00bb comparison method as it\u2019s sufficient.<\/p>\n

\"\"<\/p>\n

<\/div>\n

In the second tab \u00ab Filters \u00bb, I didn\u2019t touch the default settings.<\/p>\n

\"\"<\/p>\n

<\/div>\n

In the third tab \u00ab Synchronization \u00bb, I selected \u00ab Mirror \u00bb to only copy the changes detected between Disk 1 to Disk 2. I also checked \u00ab Detect moved files \u00bb. This is particularly useful to prevent massive drive rewrites if you just rearranged some folders, especially with some media files exceeding hundreds of Gbs in size. I left the delete option on \u00ab Recycle bin \u00bb, but you can enable versioning to keep older versions of deleted files if you want.<\/p>\n

\"\"<\/p>\n

<\/div>\n

Once the parameters set, you can save it to reuse it later (\u00ab Save as \u00bb) or start the comparison to see what changes will be copied over (\u00ab Compare \u00bb, F5) before the software start modifying the drives.<\/p>\n

All seems correct, so I start the synchronization (\u00ab Synchronise \u00bb, F9)<\/p>\n

\"\"<\/p>\n

<\/div>\n

I now have two drives with identical data on them. I add the new rushes or media on the first disk, delete what is not needed anymore, then start the comparison and the synchronization on FreeFileSync to echo the changes on the second drive.<\/p>\n

<\/div>\n

Backing up, synchronizing data between two computers on the local network.<\/h4>\n
<\/div>\n

Being on the move a lot, I have a laptop so I can work away from my main workstation. I soon had trouble keeping my work files up to date between the two computers. Still against the idea of using a Cloud service (even moreso for sensitive work files), I configured FreeFileSync to automatically and effortlessly sync my data between the two when they are connected to the same local network.<\/p>\n

<\/div>\n
Setting up the Network Share on computer 1 and connecting to it from computer 2<\/h5>\n

I first had to create a network share on my work computer, which will serve as an exchange folder between the two computers.

\n\nThe following photos will unfortunalty be in French but the written tutorial should contain the correct translations.<\/p>\n

To do this, select the folder you want to share, right-click then \u00ab Give access to \u00bb then \u00ab Specific users \u00bb.<\/p>\n

\"\"<\/p>\n

<\/div>\n

In the window that opens, select specific users or \u00ab Everyone \u00bb depending on your use then \u00ab Share \u00bb.<\/p>\n

\"\"<\/p>\n

<\/div>\n

On the other computer, open an Explorer window and go to \u00ab My PC \u00bb, on the \u00ab Computer \u00bb tab, select \u00ab Connect a network share \u00bb. In the window that opens, select the drive letter that you want then find the network share by clicking \u00ab Browse \u00bb (the network discovery can take some time). Write down the name of the folder somewhere before exiting, which should look something like \u00ab \\\\PCNAME\\NetworkShareFolderName \u00bb.<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

<\/div>\n
Setup on computer 1<\/h5>\n

It is now time to synchronize the work folder with the network share folder (this way I have two copies of my data on two different drives in case one goes bad). The setup is a little more complicated to make the whole process automated : it uses the RealTimeSync function of FreeFileSync, which can detect changes in real time and synchronize in the background without user interaction.<\/p>\n

On FreeFileSync, I redo all the setup described above, but instead of choosing \u00ab Mirror \u00bb in the comparison settings, I use \u00ab Two Way \u00bb. The software will reflect the changes whether they come from the local folder or from the folder in the network share. It also creates a database file to keep track of what has been modified, deleted or moved and keep everything consistent.<\/p>\n

\"\"<\/p>\n

<\/div>\n

Au lieu d\u2019enregistrer simplement le fichier de configuration, il faut cette fois choisir l\u2019option \u00ab Enregistrer en tant que fichier de commande… \u00bb<\/p>\n

\"\"<\/p>\n

<\/div>\n

In the window that opens, select \u00ab Run minimized \u00bb and \u00ab Auto-close \u00bb so that the software doesn\u2019t create a new window each time a file is modified in the watched folders and everything runs in the background.<\/p>\n

\"\"<\/p>\n

<\/div>\n

The saved file will be in .ffs_batch format. If launched as is, it will open Freefilesync but we want to use the RealTimeSync function. To do this, open the RealTimeSync.exe utility present in the FreeFileSync installation folder. In the window that opens, import the .ffs_batch file created previously using File > Open. The utility loads the configuration and all you have to do is save it with Files > Save as with the .ffs_real extension.<\/p>\n

\"\"<\/p>\n

By clicking \u201cStart\u201d, the service will be visible in the taskbar and will actively monitor any changes to the folders saved in the script.<\/p>\n

\"\"<\/p>\n

<\/div>\n
Creating a task with Windows Task Scheduler<\/h5>\n

The problem is that each time your computer will reset\/shut down or your session is closed, the script will have to be launched again. To do this automatically, we will have to use the Windows task scheduler : type in the Windows Search bar \u00ab task scheduler \u00bb and open it.<\/p>\n

In the right window, select \u00ab Create task\u2026 \u00bb. In the window that opens, choose a name for your task and select \u00ab Run with highest privileges \u00bb to prevent conflicts. Verify that the settings for \u00ab Configure for : \u00bb is on Windows 10.<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

<\/div>\n

Go to the \u00ab Triggers \u00bb tab. In \u00ab Begin the task \u00bb, select \u00ab At startup \u00bb. The Freefilesync documentation uses \u00ab On workstation unlock \u00bb but the task wasn\u2019t always launched on my machine, with \u00ab at startup \u00bb I haven\u2019t had any issues.<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

<\/div>\n

In the \u00ab Actions \u00bb tab, click \u00ab New \u00bb and in the window that opens, choose \u00ab Start a program \u00bb. In the \u00ab Program\/script \u00bb field, click Browse and go to your Freefilesync install folder and select RealTimeSync.exe. In the \u00ab Add arguments \u00bb field, copy the path to your .ffs_real config file.<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

<\/div>\n

Test if the service launches by restarting the computer.<\/p>\n

<\/div>\n
Setup of computer 2<\/h5>\n

On the other computer, redo the process to create a script in Freefilesync between the local folder and the network share folder. As it is connected through the network, it\u2019s best to use the network name of the folder and not the drive letter you\u2019ve mapped it to. For example, use \u00ab \\\\PCNAME\\NetworkShareFolderName \u00bb (the folder name you\u2019ve wrote down above) instead of \u00ab V : \u00bb.<\/p>\n

Just as above, save the config as a .ffs_batch, then open the RealTimeSync utility, import the .ffs_batch file then save it as a .ffs_real file then create a task in Windows Task Scheduler. A warning, if this computer is a laptop, during the task creation, go to the \u00ab Conditions \u00bb tab and uncheck \u00ab Start the task only if the computer is on AC power \u00bb and \u00ab Stop if the computer switches to battery power \u00bb, otherwise the sync will not work on battery power.<\/p>\n

\"\"<\/p>\n

<\/div>\n

I had to go an extra step so that RealTimeSync could access the network share drive. I don\u2019t know why, but windows wouldn\u2019t let the service access the folder unless I had manually navigated to it via an elevated explorer window, even after changing the permissions\u2026 To make it easy, I created on my desktop a .bat file with \u00ab run as administrator \u00bb checked that opens the networkshare drive with explorer then exits the window :<\/p>\n

explorer \\\\PCNAME\\NetworkShareFolderName<\/p>\n

taskkill \/im explorer.exe
\nexit<\/p><\/blockquote>\n

<\/div>\n

Conclusion<\/h4>\n
<\/div>\n

Freefilesync allows me to automatically manage the various additions \/ deletions of files and data and to reflect these changes on all my backups. It also aids in synchronizing my work files between my different computers. However, nothing guarantees the integrity of this data and it is not possible to know if it has been deteriorated by time or during a transfer, with a disk sector becoming corrupted for example. The next part will therefore focus on ensuring the sustainability of these data by generating and verifying checksums.<\/p>\n

Blog index<\/em> <\/a><\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>","protected":false},"excerpt":{"rendered":"

gestion de backups et synchronisation r\u00e9seau avec freefilesync Pr\u00e9sentation du logiciel Freefilesync et de mon utilisation dans le cas de synchronisation de fichiers de travail entre plusieurs postes connect\u00e9s en r\u00e9seau local et d’aide \u00e0 la gestion de backups de m\u00e9dias et donn\u00e9es.<\/p>","protected":false},"author":1,"featured_media":40895,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[62],"tags":[82,64,68,58],"_links":{"self":[{"href":"https:\/\/staging.kinolios.com\/en\/wp-json\/wp\/v2\/posts\/40797"}],"collection":[{"href":"https:\/\/staging.kinolios.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/staging.kinolios.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/staging.kinolios.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/staging.kinolios.com\/en\/wp-json\/wp\/v2\/comments?post=40797"}],"version-history":[{"count":9,"href":"https:\/\/staging.kinolios.com\/en\/wp-json\/wp\/v2\/posts\/40797\/revisions"}],"predecessor-version":[{"id":52604,"href":"https:\/\/staging.kinolios.com\/en\/wp-json\/wp\/v2\/posts\/40797\/revisions\/52604"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/staging.kinolios.com\/en\/wp-json\/wp\/v2\/media\/40895"}],"wp:attachment":[{"href":"https:\/\/staging.kinolios.com\/en\/wp-json\/wp\/v2\/media?parent=40797"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/staging.kinolios.com\/en\/wp-json\/wp\/v2\/categories?post=40797"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/staging.kinolios.com\/en\/wp-json\/wp\/v2\/tags?post=40797"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}