Differences between revisions 17 and 44 (spanning 27 versions)
⇤ ← Revision 17 as of 2021-02-20 11:56:16 →
Size: 2463
Editor: Kristian Kallenberg
Comment:
← Revision 44 as of 2021-03-22 16:49:20 → ⇥
Size: 5008
Editor: Kristian Kallenberg
Comment:
Deletions are marked like this. Additions are marked like this.
Line 4: Line 4:
 * Easy easy and simple configuration  * Easy and simple configuration
Line 6: Line 6:
 * File ignore patterns
Line 43: Line 44:
<gui enabled="true" tls="true" debugging="false"> <gui enabled="true" tls="false" debugging="false">
Line 45: Line 46:
    <user>kale</user>
    <password>password hash</password>
Line 57: Line 56:
== GUI == === GUI ===
Line 63: Line 62:
Start by setting up a username, a password and force https. This is done in the settings dialog in the GUI tab. Your installation should now be secured. Start by setting up a username, a password and force https. This is done in the settings dialog in the GUI tab. Your installation should now be secure.
Line 66: Line 65:

== Transfer speed ==

When transferring files, the default behavior of syncthing is to flush for every file transferred. This makes transfer of many small files very slow. The version of syncthing provided in Debian Buster does not have an option to disable syncthings use of the fsync call. In newer versions of syncthing, this can be found in `~/.config/syncthing/config.xml` as `<disableFsync>false</disableFsync>`. Changing that option to `true` will make syncthing transfer files much faster, on the expense of risking data loss. Currently the version of syncthing in Debian Bullseye does have this functionality implemented.

=== Using syncthing from bullseye ===

Set your default release in `/etc/apt/apt.conf.d/99defaultrelease` by adding.
{{{
APT::Default-Release "stable";
}}}

Add the bullseye repository to `/etc/apt/sources.list.d/bullseye.list`
{{{
deb http://deb.debian.org/debian/ bullseye main contrib non-free
}}}

Then setup pinning in `/etc/apt/preferences.d/buster`
{{{
Package: *
Pin: release a=buster
Pin-Priority: 1000
}}}

And in `/etc/apt/preferences.d/bullseye`
{{{
Package: syncthing
Pin: release a=bullseye
Pin-Priority: 500
}}}

Install syncthing from bullseye
{{{
apt-get update
apt-get -u install syncthing/bullseye
}}}
=== Using eat my data ===
A library called libeatmydata has been created to stop applications like syncthing from using fsync, by replacing the fsync library call.

{{{
apt-get install eatmydata
}}}

Link syncthing to eatmydata
{{{
ln -s /usr/bin/eatmydata /usr/local/bin/syncthing
}}}

Now change the init script for the syncthing service in `/lib/systemd/system/syncthing@.service` to use the version of syncthing in `/usr/local/bin`, and eatmydata will take care of the fsync call.
{{{#!highlight bash
[Unit]
Description=Syncthing - Open Source Continuous File Synchronization for %I
Documentation=man:syncthing(1)
After=network.target

[Service]
User=%i
ExecStart=/usr/local/bin/syncthing -no-browser -no-restart -logflags=0
Restart=on-failure
SuccessExitStatus=3 4
RestartForceExitStatus=3 4

# Hardening
ProtectSystem=full
PrivateTmp=true
SystemCallArchitectures=native
MemoryDenyWriteExecute=true
NoNewPrivileges=true

[Install]
WantedBy=multi-user.target
}}}

Let systemd know that you changed a service
{{{
systemctl daemon-reload
}}}

Finally restart the syncthing service
{{{
systemctl restart syncthing@kale.service
}}}
Line 77: Line 158:
* https://theselfhostingblog.com/posts/how-to-set-up-a-headless-syncthing-network/  * https://theselfhostingblog.com/posts/how-to-set-up-a-headless-syncthing-network/
 * https://docs.syncthing.net/advanced/folder-disable-fsync.html
 * https://wiki.debian.org/SimpleBackportCreation

Syncthing

Syncthing is a modern file synchronization tool. It synchronizes files between various devices and operating systems. It is a great tool for automatic backup of personal files and smartphones. It has several usefull features:

  • Easy and simple configuration
  • File versioning
  • File ignore patterns

For this setup we will use GlusterFS as the file store backend.

Software

Start by installing syncthing 

apt-get install syncthing

Filesystem

Configure your system as a GlusterFS client, so you have the following in your fstab. 

/etc/glusterfs/home.vol /srv/home glusterfs defaults,_netdev,rw 0 0

Configuration

For each user that is going to synchronize against this system, create a directory for them in /srv/home and give it the correct ownership. 

mkdir /srv/home/kale
chown -R kale:kale /srv/home/kale

For each user add the syncthing service to default runlevel and start it. 

systemctl enable syncthing@kale.service
systemctl start syncthing@kale.service

When the syncthing service starts, it creates a set of config files in the users home directory. Edit /home/kale/.config/syncthing/config.xml and look for the section below. Change the address, so the service listens on all interfaces <address>0.0.0.0:8384</address>, instead of just localhost. Notice that when you make this change, your syncthing service is public. Everyone can change it. 

   1 <gui enabled="true" tls="false" debugging="false">
   2     <address>127.0.0.1:8384</address>
   3     <apikey>apikey</apikey>
   4     <theme>default</theme>
   5 </gui>

Restart the service for the specific user. 

systemctl restart syncthing@kale.service

GUI

It should now be possible to connect to the syncthing GUI on https://syncthing:8384 and configure it.

Credentials

Start by setting up a username, a password and force https. This is done in the settings dialog in the GUI tab. Your installation should now be secure.

syncthing-settings-gui.png

Transfer speed

When transferring files, the default behavior of syncthing is to flush for every file transferred. This makes transfer of many small files very slow. The version of syncthing provided in Debian Buster does not have an option to disable syncthings use of the fsync call. In newer versions of syncthing, this can be found in ~/.config/syncthing/config.xml as <disableFsync>false</disableFsync>. Changing that option to true will make syncthing transfer files much faster, on the expense of risking data loss. Currently the version of syncthing in Debian Bullseye does have this functionality implemented.

Using syncthing from bullseye

Set your default release in /etc/apt/apt.conf.d/99defaultrelease by adding. 

APT::Default-Release "stable";

Add the bullseye repository to /etc/apt/sources.list.d/bullseye.list 

deb http://deb.debian.org/debian/ bullseye main contrib non-free

Then setup pinning in /etc/apt/preferences.d/buster 

Package: *
Pin: release a=buster
Pin-Priority: 1000

And in /etc/apt/preferences.d/bullseye 

Package: syncthing
Pin: release a=bullseye
Pin-Priority: 500

Install syncthing from bullseye 

apt-get update
apt-get -u install syncthing/bullseye

Using eat my data

A library called libeatmydata has been created to stop applications like syncthing from using fsync, by replacing the fsync library call. 

apt-get install eatmydata

Link syncthing to eatmydata 

ln -s /usr/bin/eatmydata /usr/local/bin/syncthing

Now change the init script for the syncthing service in /lib/systemd/system/syncthing@.service to use the version of syncthing in /usr/local/bin, and eatmydata will take care of the fsync call. 

   1 [Unit]
   2 Description=Syncthing - Open Source Continuous File Synchronization for %I
   3 Documentation=man:syncthing(1)
   4 After=network.target
   5 
   6 [Service]
   7 User=%i
   8 ExecStart=/usr/local/bin/syncthing -no-browser -no-restart -logflags=0
   9 Restart=on-failure
  10 SuccessExitStatus=3 4
  11 RestartForceExitStatus=3 4
  12 
  13 # Hardening
  14 ProtectSystem=full
  15 PrivateTmp=true
  16 SystemCallArchitectures=native
  17 MemoryDenyWriteExecute=true
  18 NoNewPrivileges=true
  19 
  20 [Install]
  21 WantedBy=multi-user.target

Let systemd know that you changed a service 

systemctl daemon-reload

Finally restart the syncthing service 

systemctl restart syncthing@kale.service

Instances

References

None: Syncthing (last edited 2022-02-12 09:39:51 by Kristian Kallenberg)