Category: Linux

Install Nzbget Usenet Downloader

This article describes how to download and install nzbget on a debian server.

Usenet is a distributed system where users read and post messages – articles or posts, and collectively termed news) to one or more categories, known as newsgroups. They have been around since before the internet as we know it existed – think of it as a large, distributed forum. It was originally designed to transmit text messages but has since proved exceedingly popular for binaries i.e. programs and other files.

Nzbget is a downloader client for Usenet newsgroups binaries – it’s fast, efficient, has a small system footprint and of course it’s free! While transmission-daemon will take care of the torrent downloads, nzbget in turn takes care of newsgroup binary downloads.

A binary file in a newsgroup is split into a number of “part” files to allow posting – once all files have been downloaded, the original file can be reassembled – should one part file be missing or corrupted however then reassembly is not possible. Nzbget downloads all the part files and reassembles the file, notofying the user should anything be amiss.

The last thing to say is that while many internet providers have a limited Usenet service for their customers, to take real advantage of newsgroups you will need to take out a subscription – a quick google for providers will soon have you up and running and many have a free trial period.

Htpcguides have already written an excellent guide on installing nzbget on debian and ubuntu systems so this tutorial has been taken from their site and modified only to run as the user and group – nobody:nogroup – which we are configuring.

Install unrar

Unrar is the program that nzbget uses to unpack and reassemble the part files – you can install this with apt-get install unrar OR build it from source as described below to take advantage of multicore PCs which all of them are nowadays.

Why bother? Well, newsgroup binaries tend to be in a rar format so the faster we can decompress them the better – let’s use as many cores as possible!

We are going to build the latest unrar from source adapted from these instructions.

1. Make sure you have building tools:
   • sudo apt-get install build-essential -y
2. Enter your temporary directory
   • cd /tmp
3. Download the latest unrar source tarball
   • wget rarlab.com/rar/unrarsrc-5.2.7.tar.gz
4. Unpack the unrar source
   • tar -xvf unrarsrc-5.2.7.tar.gz
5. Enter the unrar source
   • cd unrar
6. Get the number of processors for compiling faster
   • nproc
7. I have 2 so when building I add 2 after -j
   • make -j2 -f makefile
8. Now install unrar
   • sudo install -v -m755 unrar /usr/bin
9. Cleanup your unrar source and unpacked directory
   • cd ..
   • rm -rf unrar
   • rm unrarsrc.5.2.7.tar.gz

Install NZBGet

2. Download the latest stable NZBGet installer – check the page and insert the correct link:
   • http://nzbget.net/download/
   • wget https://github.com/nzbget/nzbget/releases/download/v16.4/nzbget-16.4-bin-linux.run

3. Install NZBGet on Debian 8.x:
   • sudo sh nzbget-16.4-bin-linux.run --destdir /opt/nzbget
4. Remove the installer
   • rm nzbget-16.4-bin-linux.run

5. Change ownership of the NZBGet installation to our nobody user
   • sudo chown -R nobody:nogroup /opt/nzbget
6. Update your NZBGet configuration file to run as  nobody
   • sudo sed -i "/DaemonUsername=/c\DaemonUsername=nobody" /opt/nzbget/nzbget.conf
7. Create the NZBGet systemd service
   • sudo nano /etc/systemd/system/nzbget.service
8. Paste the working NZBGet systemd script for Debian
   • [Unit]

     Description=NZBGet Daemon
     Documentation=http://nzbget.net/Documentation
     After=network.target
     
     [Service]
     User=nobody
     Group=nogroup
     Type=forking
     ExecStart=/opt/nzbget/nzbget -c /opt/nzbget/nzbget.conf -D
     ExecStop=/opt/nzbget/nzbget -Q
     ExecReload=/opt/nzbget/nzbget -O
     KillMode=process
     Restart=on-failure
     
     [Install]
     WantedBy=multi-user.target

9. Enable the NZBGet systemd service
   • sudo systemctl enable nzbget.service
10. Start the NZBGet systemd service for Debian
    • sudo service nzbget start
    • Confirm you have access to NZBGet on http:/your_ip_here:6789, the default username is nzbget and password is tegbzn6789.
11. Change the default username and password under “SETTINGS: ControlUsername and Control Password” – we will need this later when we integrate nzbget into Sonarr:
    
    • 
12. Lastly and definitely not least, enter your news server(s) address under SETTINGS -> NEWS-SERVERS. There are 4 parameters per server which must be configured:
    • Host – the address of your news server
    • Port – can be anything but generally 119 for non-SSL (unencrypted ) connections and 563 for SSL connections
    • Username and password for your Usenet provider

Install transmission-daemon

You could be forgiven for thinking that a simple “apt-get install” would suffice to install transmission-daemon and thoretically it does.

However! We want this set up securely and neatly as there are other components which need to interact with transmission and its files and directories and ideally we will have all of these running as user “nobody:nogroup”.

Install the Software

merlin@merlin:~$ sudo apt-get install transmission-daemon

Configure transmission-daemon to run as user nobody

If we now check our running processes, transmission is configured to run as user “debian-transmsission” – totally fine and secure but not quite what we want:

merlin@merlin:~$ ps aux | grep transmission
debian-+  1356  0.1  0.5  30708  6064 ?        Ssl  20:45   0:00 /usr/bin/transmission-daemon -f --log-error

We need to make the following edits and permission changes:

1. Stop the daemon:

   • /etc/init.d/transmission-daemon stop

2. Edit /etc/init.d/transmission-daemon and change the user:
   • change USER=debian-transmission to USER=nobody:nogroup and save the file
3. As debian now uses systemd to manage services we also need to change the service “Unit” to reflect the different user. Edit the /lib/systemd/system/transmission-daemon.service file and change the following:
   •  User=debian-transmission to User=nobody
   • Add the absolute path of the config file to the end of the ExecStart line to workaround a slight bug with $HOME variables, the line should read:

     ExecStart=/usr/bin/transmission-daemon -f --log-error --config-dir=/var/lib/transmission-daemon/info

   • Save the file and reload the system configuration:

     sudo systemctl daemon-reload

4. We now need to make sure that user nobody has permissions to the relevant files and directories:

   • merlin@merlin:~$ sudo chown -R nobody:nogroup /var/lib/transmission-daemon/

   • merlin@merlin:~$ sudo chown -R nobody:nogroup /etc/transmission-daemon

Set the UDP Send and Receive Buffers

The daemon will now start and run as user nobody:

merlin@merlin:~$ ps aux | grep trans
nobody     418  0.5  1.7  60628 18512 ?        Ssl  09:38   0:14 /usr/bin/transmission-daemon -f --log-error --config-dir=/var/lib/transmission-daemon/info

Tailing /var/log/syslog while restarting the service shows the following errors:

UDP Failed to set receive buffer: requested 4194304, got 327680 (tr-udp.c:78)
Failed to set send buffer: requested 1048576, got 327680 (tr-udp.c:89)

Transmission is requesting a 4MB receive buffer and a 1MB send buffer but not getting them. This can be swiftly corrected by adding the values to the sysctl.conf file and then refreshing with “sysctl -p”:

echo 'net.core.rmem_max = 4194304' >> /etc/sysctl.conf
echo 'net.core.wmem_max = 1048576' >> /etc/sysctl.conf
sysctl -p

You can also try using larger buffers to tweak performance e.g. 16MB receive buffer and 4MB send as below:

echo 'net.core.rmem_max = 16777216' >> /etc/sysctl.conf
echo 'net.core.wmem_max = 4194304' >> /etc/sysctl.conf

Once the values have been applied, a restart should be smooth and error free.

Change Default User Credentials & Allow Network GUI Access

As this is a headless server we are unlikely to be accessing it from localhost very often. Default settings allow only connections from 127.0.0.1 so edit the /etc/transmission-daemon/settings.json file and either add allowed IPs to the whitelist or disable the whitelist altogether (change rpc-enabled to false) – the latter should only be done if the server is not directly connected to the internet where anyone could access it!

1. Stop the daemon before editing, otherwise your changes will be overwritten when the service is stopped / started later:

   merlin@merlin:~# sudo /etc/init.d/transmission-daemon stop

2. Edit /etc/transmission-daemon/settings.json, look for the settings and change them as applicable:
   • Delete the hashed password between the quotes in this line and replace it with your new plain text password – transmission will automatically hash it as soon as it restarts:

     "rpc-password": "{b21ebef271608102725706c76924ca596ca0d3e6CA5En8YL",

   • Find the following line and change the username:

     "rpc-username": "transmission",

   • Either add your allowed IP(s) to the whitelist or set the enabled parameter to false to allow access from any IP:

     "rpc-whitelist": "127.0.0.1", "rpc-whitelist-enabled": true,

3. Restart the service and you should now be able to connect via either the web gui or a front-end app e.g. Transmission Remote GUI

Implement a Blocklist (Optional)

In the Transmission Remote GUI go to Tools -> Transmission Options -> Network (WAN). Select “Enable blocklist” and enter the URL of your favourite blocklist provider. Generally the following is ample:

http://list.iblocklist.com/?list=bt_level1&fileformat=p2p&archiveformat=gz

Click OK and then Tools -> Update Blocklist

References

• http://0pointer.de/blog/projects/systemd-for-admins-3.html
• https://falkhusemann.de/blog/2012/07/transmission-utp-and-udp-buffer-optimizations/

Debian Server Install

This article details how to create a standard, headless debian server install. For our linux media server project we will be using the latest version distribution available at the time of writing: Debian 8.3.

You can also use the process below to build a standard debian server. It will be headless, i.e. no desktop environment e.g. Gnome as we don’t want to waste resources on running a desktop environment we won’t be using after setup is complete.

Instructions

1. Download either the netinstall image iso or the first complete image iso in the series
   • The net installer is about 315MB, the complete CD is around 650MB. If you are going to ever install more than one machine then the complete CD is far better value!
2. Create a new virtual machine (or prepare your physical server if that’s what you will use)
3. Go through a default install as shown below:
   1. Choose “Install” from the boot up screen
   2. Configure your language, keyboard and country fro the options given – here we are using UK English
   3. Give your server a hostname, in this case we are using “merlin”
   4. When asked to input a root password, we advise leaving this blank – this disables root login which is more secure
   5. Enter a username – we are using “merlin” again here. This user will be able to carry out root permission commands using “sudo”, i.e. it will be a member of the “wheel” group
   6. Accept all the default disk partitioning prompts (guided, use entire disk) then click to finish and write the partitions to disk
   7. When prompted to scan another disk, answer “no” – CD1 is enough to create what we need for our debian server
   8. Choose “yes” to use a network mirror in the next step. Don’t worry! We will not be downloading large amounts in the initial configuration but we will need it for the installation of individual packages later on. Pick your country and the relevant local network mirror site for quicker access to packages.
   9. For the “Select and install software” use the arrow and spacebar  keys to deselect “Debian Desktop Environment” and “print server”, select “SSH server” and leave “standard system utilities” selected as shown below:
   10. Choose the default to install grub to the master boot record, in our case /dev/sda and then “continue to finish the installation.
4. Set a static IP address
   • This is important so that you can always find your server! The article > here < details how to do this via the /etc/network/interfaces file.
5. Make sure you can connect via SSH
   • Use a terminal client like putty to make sure you can connect – this install will all be done via console or SSH as we do not have a desktop environment.
6. Run

   sudo apt-get update

   and then

   sudo apt-get upgrade

   to make sure you have all the latest updates, including security patches.

Give “nobody” a Home!

If you are following this guide as part of the Linux Media Server project then you must create a home directory for the nobody user – this is so that our media applications have somewhere central to store their configuration data.

For this, we need to edit the /etc/passwd file, create some directories and assign permissions:

1. In /etc/passwd change the following line:

   • nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin

     to

     nobody:x:65534:65534:nobody:/home/nobody:/usr/sbin/nologin

2. Create the necessary directory, including one for NzbDrone to save configuration into:

   • mkdir -p /home/nobody/.config/NzbDrone/

3. Assign permissions:

   • chown -R nobody:nogroup /home/nobody/

You should now have completed your debian server install – why not carry on to the next srticle in the Complete Linux Media Server!