Using a Raspberry Pi to remotely operate and administer a
UFOCapture camera system
Introduction
This document contains intructions and tips for using a Rapsberry Pi to remotely control and
monitor a UFOCapture camera system. The main advantage of using a Raspberry Pi like this
is that it can be used to turn on the camera system automatically at sunset, something that
can be difficult or expensive to perform by other methods. With the Raspberry Pi, this is done
by electronically toggling the power button of the main computer by means of a relay switch
connected to the wires of the power button and operated via the general purpose input-output
(GPIO) pins on the Raspberry Pi. In addition to switching on the main computer, the Pi can
be used to control the power to the camera unit itself using a second relay. The instructions for
this basic setup are given in Section 1.
The flexibility of the Raspberry Pi allows a range of more advanced automation and monitoring
features to be added with relative ease. These include automatically checking the weather forecast
for the night before switching on, hosting and post-processing the videos, checking disk usage
and using a web interface to allow remote monitoring and control of the system and observing
schedule via a website. The more advanced steps involved in this are outlined in Section 2.
This guide is not intended to be an exhaustive set of instructions, rather a list of the hardware
and software required and the basic steps necessary to get the system up and running. If you
get stuck, feel free to contact me with any technical questions or suggestions.
Nick Rowell ([email protected])
August 14, 2016
Contents
1 Basic setup: turning on the
1.1 Hardware . . . . . . . . .
1.2 Software . . . . . . . . . .
1.3 Instructions . . . . . . . .
camera
. . . . .
. . . . .
. . . . .
2 Advanced features
2.1 Checking weather conditions . . .
2.2 Remote monitoring and retrieval of
2.3 Video hosting on the Pi . . . . . .
2.4 Automated disk administration . .
2.5 Remote monitoring using a website
2.6 Additional stuff that may be useful
system
. . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
3
3
5
6
.
.
.
.
.
.
9
9
10
13
18
20
22
3 Things that didn’t work
3.1 Network share drive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
22
22
. . . . . . . . .
the video clips
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1
Basic setup: turning on the camera system
This section outlines the steps required to turn on the camera and host computer automatically
at a preset time. While this is possible with BIOS features such as Wake-on-LAN and Wake-onAC-power, both of these have drawbacks that will prevent most NEMETODE nodes from using
them: many older network cards don’t support Wake-on-LAN (which would require a second
computer to send the signal anyway), and Wake-on-AC-power does not adjust for the changing
daylight hours without an expensive astronomical timeswitch. Instead, we’ll connect wires to
the terminals of the power button and use a relay switch, controlled by the Pi, to electronically
‘press’ the button.
In the basic setup described below, the Raspberry Pi runs a short program once per day that
turns on the main computer and the camera at 30 minutes after sunset, and turns the power to
the camera off at the end of observing the next morning. Note that automatically starting the
UFOCapture software once the main computer has been turned on can be done via Windows as
described in other NEMETODE documents. Also, shutting down UFOCapture and turning off
the main computer at the end of the night can also be done automatically using settings within
UFOCapture, as described in the user manual.
1.1
Hardware
The bits of hardware required for this project are listed in this section. They can all be bought
relatively cheaply from the following outlets:
• The Pi Hut (thepihut.com/)
• Ebay (www.ebay.co.uk/)
Raspberry Pi
The Raspberry Pi is a small, low cost and low power single-board computer, which can be
bought for around £30 from the Pi Hut. This guide assumes a fresh installation of the Raspbian
operating system as explained at www.raspberrypi.org/help/noobs-setup/. Several variants
of the Raspberry Pi have been produced such as the Model B shown in the figure below. For this
project basically any of them will do, although the Model B is probably the easiest to obtain.
Figure 1: Raspberry Pi model B
Real time clock
The Raspberry Pi doesn’t have a built in clock, and the internal time is reset to midnight
whenever it is switched off. In order for the observing schedule to be set the correct time needs
to be known, and this is most easily done by using a real time clock module. They cost about £5
from the Pi Hut. This guide assumes the real time clock has been installed and set up already
by following the instructions at thepihut.com/blogs/raspberry-pi-tutorials.
Figure 2: Real time clock
WiFi dongle
An internet connection is necessary in order to download the software required for the basic
setup described in this section. In addition, some of the advanced features described in the
next section use an internet connection while the system is operational. In both cases a direct
ethernet connection is fine, although a WiFi dongle can be more convenient depending on where
the Pi is to be located. Many USB WiFi dongles will work out of the box (check the list at
elinux.org/RPi_VerifiedPeripherals if in doubt), and the one pictured below can be bought
from the Pi Hut for £6.
Powered USB hub
If the WiFi dongle is plugged into one of the USB ports on the Pi itself, there will very likely
be problems with the WiFi connection being intermittenly dropped due to the dongle drawing
more power than the Pi can provide through the USB ports. This is especially likely if a longer
range WiFi dongle is used. In this case, the use of a powered USB hub is required. These can
be bought from the Pi Hut for around £10.
Figure 3: WiFi dongle (left) and powered USB hub (right)
Two Channel Relay board
A two channel relay board is required in order to turn on the main computer and the power to
the camera separately. The device pictured below can be bought on Ebay for £2.79. Although
these are often sold as 5V devices, the switching voltage of the GPIO pins on the Pi is 3.3V so
the 3.3V source on the Pi is used.
Figure 4: 5V Two Channel Relay Board Module
Push switch
An optional push-to-make switch can be connected across the terminals of the relay to provide
an easy manual override to turn on the main computer immediately. Note that a similar override
for the power to the camera is not advisable due to the larger current - an easy solution is to
add extra connector sockets to the camera power lead so that the relay can be bypassed by
unplugging the cable.
Jumper wires etc
A few jumper wires are required (usually the female-female type) to connect the GPIO pins of
the Pi to the input pins of the relay. These can be bought in bulk on Ebay and cost a few pounds
for forty or so.
1.2
Software
The Raspberry Pi should be set up with a fresh installation of the Raspbian operating system,
and with a real time clock installed. An internet connection is required in order to download
and install the software described in this section.
WiringPi
The WiringPi library allows simple control of the GPIO pins from the command line. It is used
within the camera startup script to switch the relays on and off. It has to be downloaded from the
git repository and built manually. This is done using the following commands, which download
the software to /home/pi/wiringPi/.
pi@raspberrypi ~ $ cd ~/
pi@raspberrypi ~ $ git clone git://git.drogon.net/wiringPi
pi@raspberrypi ~ $ cd wiringPi/
pi@raspberrypi ~ $ ./build
Sunwait
Sunwait is a small C program for calculating sunrise and sunset, as well as civil, nautical, and
astronomical twilight times. It has features that make it useful for home automation tasks. In
this project, it is used to schedule the startup sequence to run in the evening at a specific time
relative to the local sunset. It is available online via the website www.risacher.org/sunwait/,
and can be downloaded and installed using the following commands.
# Make a directory to contain sunwait
pi@raspberrypi ~ $ mkdir ~/sunwait
pi@raspberrypi ~ $ cd ~/sunwait/
# Download and extract sunwait source
pi@raspberrypi ~ $ wget www.risacher.org/sunwait/sunwait-20041208.tar.gz
pi@raspberrypi ~ $ tar -xzvf sunwait-20041208.tar.gz
pi@raspberrypi ~ $ cd sunwait-20041208/
# Build sunwait
pi@raspberrypi ~ $ make
# Install links
pi@raspberrypi ~ $ sudo ln -s /home/pi/sunwait/sunwait-20041208/sunwait
/usr/local/bin/sunwait
# Verify that sunwait has been installed...
pi@raspberrypi ~ $ /usr/local/bin/sunwait -p 0N 0W
1.3
Instructions
Figure 5 shows a block diagram of the main components of this project and how they should
be connected together, including the GPIO pin assignments. If different pins are used then the
scripts below should be updated accordingly.
Figure 5: Block diagram of basic setup. An optional push switch can be added across the
terminals of relay 1 in order to provide an override to turn on the main computer manually.
In order to connect the relay terminals to the power button of the main computer, it’s necessary
to open up the computer and track down where the power button connects to the motherboard
(make sure the computer is turned off and unplugged!). If you’re lucky, there’ll simply be two
wires from the button to a dedicated socket on the motherboard, and the button can simply be
removed and the jumper wires connected directly between the relay terminals and the socket. It’s
also possible that the button goes through an intermediate interface and the connection to the
motherboard is harder to track down. In my computer this was the case - the button was built
into a small PCB along with some other electronics and attached to the motherboard through
a large ribbon cable. Some testing with a multimeter was necessary to track down which of the
wires were connected to the button. Figure 6 shows the Pi and relay board (in the small black
box) plugged into the on button port through the ribbon cable.
Figure 6: Raspberry Pi and relay board (in the small blue/black boxes) connected to the main
computer power button port through the ribbon cable.
The system is based on two shell scripts that are run at specified times. The first script, called
pin_setup.sh, is run once whenever the Pi reboots and sets up the chosen GPIO pins to operate
as outputs. The second script, called startup.sh, runs once per day in the afternoon and has
the job of scheduling the system startup sequence for the end of nautical twilight on the current
date. The two scripts are shown below. These should be saved on the Pi at /home/pi/NEMETODE/.
Script pin_setup.sh
#!/bin/bash
# Shell variables to store pins connected to relay
pin1=10
pin2=24
/usr/local/bin/gpio -g mode ${pin1} out
/usr/local/bin/gpio -g write ${pin1} 1
/usr/local/bin/gpio -g mode ${pin2} out
/usr/local/bin/gpio -g write ${pin2} 1
Script startup.sh
#!/bin/bash
# Shell variables to store pins connected to relay
pin1=10
pin2=24
# Latitude and longitude of observing site, in floating-point degrees
latitude=56.126957N
longitude=4.085583W
# Scheduling system startup for 30 mins after sunset
/usr/local/bin/sunwait sun down +00:30:00 ${latitude} ${longitude}
# Optional weather check (see later)
# Turning on the main computer
/usr/local/bin/gpio -g write ${pin1} 0
sleep 1
/usr/local/bin/gpio -g write ${pin1} 1
# Turning on the camera
/usr/local/bin/gpio -g write ${pin2} 0
# Scheduling system shutdown for 30 mins before sunrise
/usr/local/bin/sunwait sun up -00:30:00 ${latitude} ${longitude}
# Turning off the camera (the computer is turned off by UFOCapture)
/usr/local/bin/gpio -g write ${pin2} 1
These two scripts can be scheduled to run automatically at certain times of day using the cron
daemon, which is a task scheduling service that operates on the Pi and which executes commands
at preset times as specified by the user. This is done by adding an entry for each script in the
cron table, which is a list of all the tasks that the cron daemon runs. The cron table can be
opened for editing using the command crontab -e. The following entries should be added at
the end of the file:
# Schedule the pin_setup.sh script to run whenever the Pi reboots
@reboot /home/pi/NEMETODE/pin_setup.sh
# Schedule the startup.sh script to run every day at 15:00
00 15 * * * /home/pi/NEMETODE/startup.sh
Once these entries are added to the cron table they will be picked up by the cron daemon
whenever the Pi is next rebooted, and the system will then be up and running.
2
Advanced features
This section describes a few optional extra features that can be added to make the system more
flexible and intelligent. Most of the features are implemented by adding extra commands to the
shell scripts described in the previous section.
2.1
Checking weather conditions
The basic setup described earlier turns on the camera system at a predetermined time regardless
of the weather conditions. A very useful extension to the basic setup is to add an automated
weather check to allow the Pi to decide whether the conditions are suitable for observing and only
turn on the camera system if so. This is achieved by looking up an aviation weather report for a
nearby airport, which is presented in a standardized format that makes it simple to automatically
check the cloud cover.
Weather-Util
The piece of software that is used to do this is simply called Weather (see fungi.yuggoth.org/
weather/), and is available as a Debian package called weather-util that can be easily installed
on the Raspberry Pi and operated using the following commands:
# Install Weather-Util
pi@raspberrypi ~ $ sudo apt-get install weather-util
# Check the latest weather report from e.g. Dundee (EGPN)
pi@raspberrypi ~ $ weather EGPN
Searching via station...
[caching result Dundee / Riverside, United Kingdom]
Current conditions at Dundee / Riverside, United Kingdom (EGPN) 56-27N 003-01W
Last updated Apr 18, 2016 - 12:20 PM EDT / 2016.04.18 1620 UTC
Temperature: 55 F (13 C)
Relative Humidity: 43%
Wind: from the W (270 degrees) at 17 MPH (15 KT) (direction variable)
Sky conditions: mostly clear
In order to get a weather forecast for a nearby airport you need to know it’s International Civil
Aviation Organization (ICAO) code. This is a four letter code that starts EG for airports in the
United Kingdom (in the example above, EGPN represents Dundee): a list of all the airports in
the UK can be found online at en.wikipedia.org/wiki/List_of_airports_by_ICAO_code:_E.
The basic idea is to insert a few extra commands in the startup.sh script that look up the
weather report, strip out the ‘sky conditions’ part and compare it against a set of known values
to decide whether the conditions are suitable for observing. This is performed by the following
lines, which should be added to the startup.sh script just below the # Optional weather check
comment (replace the EGPF code with that of your chosen airport):
# Get full weather forecast for Glasgow Airport (EGPF)
forecast=$(/usr/bin/weather EGPF)
# Extract sky conditions string (note we need to put $forecast in quotes otherwise
# the new lines disappear and we can’t grep out the sky conditions line)
sky=$(echo "${forecast}" | grep ’Sky conditions’ | awk -F ’:’ ’{print $2}’)
# Trim white space from start & end of string so it’s easier to compare
skyTrimmed="$(echo -e " ${sky} " | sed -e ’s/^[[:space:]]*//’ -e ’s/[[:space:]]*$//’)"
sky=$skyTrimmed
if [ "$sky" == "overcast" ] || [ "$sky" == "mostly cloudy" ]; then
# Conditions not good: exit script early to cancel system startup
exit
elif [ "$sky" == "partly cloudy" ] || [ "$sky" == "mostly clear" ] || [ "$sky" == "clear"
]; then
# Conditions are good: continue with the startup script
else
# Can’t interpret sky conditions: cancel observing as a precaution
exit
fi
Troubleshooting
If a weather report isn’t available In developing this system I noticed that weather reports
are intermittently not available for the chosen airport. The solution to this is to use a set of
nearby airports, and when the weather report isn’t available for the first choice then move on to
the next until a report is returned. This can be done programmatically by checking whether the
shell variable sky contains anything after the weather call has returned.
2.2
Remote monitoring and retrieval of the video clips
This section describes how to remotely login to the camera host computer in order to retrieve
the video clips and perform other maintenance. This is achieved by running an ‘SSH server’ on
the main computer which handles incoming connections. A few extra steps are necessary to get
this working on a home broadband network. Note that the main computer needs to be switched
on and not running UFOCapture in order to allow the file transfer to proceed. This section
is mainly useful as a stepping stone to the later sections, which build on these ideas to allow
remote connection to the Pi itself for retrieval of the video clips during the daytime when the
main computer is turned off.
Installing an SSH server As long as the camera host computer has an active internet connection, then it can be connected to remotely by installing and running an SSH (Secure-SHell)
Server on it. This is a piece of software that listens for incoming connections using the SSH
protocol, which allows users to login remotely to inspect files and e.g. transfer them to another
machine. There are many options for SSH Server applications, and the recommended one in
this guide is Bitvise www.bitvise.com/ssh-server, which is free for personal non-commercial
use. This should be installed and set up to run automatically whenever Windows starts up.
Specifically, the SSH ‘Server’ application should be installed and not the SSH ‘Client’, which is
used for outward connections and is of no use here.
Port forwarding on home router It’s likely that the main computer will not connect to the
internet directly, but will be part of a home network and access the internet through a router.
In this case, the router needs to be configured to forward incoming SSH connections to the main
computer. This is referred to as port forwarding, and is a reasonably common thing to set up.
The basic steps are as follows:
1. From any computer that is connected to the home network, open a web browser and
navigate to the router configuration page. This will have a standard web address (e.g.
192.168.1.254 for a BT Home Hub 4) - if in doubt, check Google for the address for your
router. Another method is to use the command ipconfig (on Windows) or route -n (on
Linux) and look for the IP address listed under ‘Gateway’.
2. Make sure that the camera host computer is turned on and connected to the network. Look
up it’s IP address and hostname on the local network by looking for a list of connected
devices on the router config page.
3. Following the instructions for your router, forward port 22 (or SSH services) to the IP address or hostname associated with the main computer. There should be a ‘port forwarding’
section of the router config page that sets this up.
You should now be able to connect remotely to the main computer using an SSH connection as
shown below.
~$ ssh [email protected]
[email protected]’s password:
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.
C:\Documents and Settings\CJ1>
In this example, CJ1 is the Windows account username and 86.136.2.98 is the external IP
address of the home router. The next section explains how to find out the external IP address
and to set up a more memorable hostname to use instead. Also, if you have trouble logging in
then check the tips in the troubleshooting section below.
Looking up the external IP address Home internet connections are nearly always assigned
a dynamic IP address, that is, the external IP address of the connection will change from time
to time. Depending on the service provider, this can happen anywhere from once per month
to several times per day. You will need to know the current IP address in order to connect
to the Raspberry Pi remotely, which can be a pain if it changes frequently. One way to look
up the current external IP address is to go to a website such as www.whatismyip.com from a
device connected to the internet through the same network as the Raspberry Pi. However this
means that someone needs to be present to do this. Another solution is to run a script on the
Raspberry Pi several times a day to check the current IP address using a simple command (such
as $ wget -qO- http://ipecho.net/plain), then to automatically email it to you. However,
a much cleaner and more flexible solution is to assign a fixed hostname (web address) to your
home router so that you can login using that rather than the IP address. This is described in
the next section.
Assigning a hostname using Dynamic DNS A Dynamic DNS service is an application
that maps the IP address of a computer to a fixed hostname that is much easier to remember.
As the name suggests, it constantly monitors for changes in the IP address so that the hostname
is always up to date. There are a variety of free dynamic DNS providers that do this, and I
would recommend No-IP (www.noip.com). The steps that are involved in setting this up are
quite simple:
1. Set up a (free) account with No-IP and create a hostname (e.g. mymeteorcamera.ddns.net).
2. If your computer connects directly to the internet (and not through a home router) then
you will need to install a small piece of software provided by No-IP, as explained on their
website.
3. Otherwise, if your computer connects to the internet through a home router, then it’s very
likely that the router will already support the use of dynamic DNS and you just need to
enter your account details in the right place. Look up your router on Google, or navigate
to the router configuration page (e.g. 192.168.1.254) and have a search for Dynamic DNS.
4. You might need to restart your computer and/or router for the settings to take effect.
That’s it - now you should be able to connect to the computer over SSH using the modified
command below.
~$ ssh [email protected]
[email protected]’s password:
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.
C:\Documents and Settings\CJ1>
Compressing and transferring the video files The raw video files tend to be very large
and would take a long time to transfer over the internet. This can be sped up a great deal by
first compressing the files using a program called 7zip, which provides lossless compression at
a high ratio. This should be downloaded from www.7-zip.org and installed on the main computer. A folder containing the entire night’s video clips and other files can then be compressed
by navigating to it and running the following command. In this example, the folder is located at
E:\2016\201604\20160415, and the 7zip executable is installed at C:\Program Files\7-Zip\7z.exe.
C:\> E:
E:\> cd 2016\201604
E:\2016\201604>"C:\Program Files\7-Zip\7z.exe" a 20160415.7z 20160415
The compressed directory can then be transferred to a remote machine using scp as in the
following command:
# Example of transferring compressed folder of video files using scp under Linux
# Command format: <username>@<IP>:<remote path> <local path for copy>
# Note the double-backslashes for Windows file path
~$ scp [email protected]:E:\\2016\\201604\\20160415.7z 20160415.7z
Troubleshooting
Port forwarding Depending on your router, you might run into problems when the main
computer is restarted as it may get assigned a new IP address on the local network that is
different from the one that SSH connections are forwarded to. In this case, either forward the
port to the hostname of the main computer (which doesn’t change) rather than the IP address,
or configure the router to always assign the same IP address to the main computer.
Login problems for Windows user accounts Note that the Windows account name as
displayed on the login screen can differ from the account username required for login via SSH. If
your SSH connection is being declined due to errors along the lines of ‘The supplied user name
could not be looked up’, then this may be the case. To find out the account name for SSH,
go to Control Panel→Admin Tools→Computer Management →Local Users and Groups→Users.
The ‘Name’ is what you must use to connect via SSH, the ’Full Name’ is what is displayed on
the login screen. You may also need to change a firewall setting in Bitvise: open Easy Server
Settings, and for ‘Open Windows Firewall’ select ‘Open Port(s) to any Computer’
2.3
Video hosting on the Pi
Because the main computer needs to be switched on in order to remotely access the video clips,
the method outlined in the previous section is of limited use with the current project, where the
main computer is only turned on at night to perform the meteor observing. As the Raspberry Pi
is switched on permanently, it is a much better host for the videos as they can then be accessed
at any time. This section describes how to automatically transfer all the video clips to the Pi at
the end of each night and make them accessible from there.
Connecting to the main computer from the Pi The first step is to set up the Pi to make
SSH connections to the main computer. An SSH client on the Pi is required for this. Note that
the corresponding SSH server on the main computer should be installed following the instructions
in the previous section. It’s very likely that the Pi will already have a SSH client installed - you
can check by typing ssh -V at the command line:
pi@raspberrypi ~ $ ssh -V
OpenSSH_6.0p1 Debian-4+deb7u2, OpenSSL 1.0.1e 11 Feb 2013
If the SSH client is not installed then check on Google for installation instructions (probably just
sudo apt-get install ssh).
The next step is to configure the SSH client and server to allow passwordless connections from
the Pi to the main computer. This is necessary in order to automate the SSH connection for
use in scripts – if this wasn’t done then you would have to type in the password every time the
Pi tried to connect to the main computer, and automation wouldn’t be possible. For this, we
need to generate public and private SSH keys on the Pi, which are used to uniquely identify the
Pi when connecting to the SSH server on the main computer. These are files called id_rsa.pub
and id_rsa, and are located in the /.ssh/ folder. First, check if they already exist:
pi@raspberrypi ~ $ ls ~/.ssh
known_hosts
pi@raspberrypi ~ $
If not (as in this example), then you can create new ones using the command ssh-keygen -t
rsa -C "NEMETODE RPi":
pi@raspberrypi ~ $ ssh-keygen -t rsa -C "NEMETODE RPi"
Generating public/private rsa key pair.
Enter file in which to save the key (/home/pi/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/pi/.ssh/id_rsa.
Your public key has been saved in /home/pi/.ssh/id_rsa.pub.
The key fingerprint is:
a3:31:42:1e:1b:76:f7:a0:a2:55:9b:fd:03:33:29:e5 NEMETODE RPi
The key’s randomart image is:
+--[ RSA 2048]----+
|
+ .
|
| o X .
|
|. o O E
|
|.o + + =
|
|. + o S
|
|
. + o
|
|
.
|
|
|
|
|
+-----------------+
Verify that the public and private keys are now present:
pi@raspberrypi ~ $ ls ~/.ssh
id_rsa id_rsa.pub known_hosts
pi@raspberrypi ~ $
id_rsa is the private key, which remains on the Pi. id_rsa.pub is the public key, and is required
by computers that you want to connect to without a password. Transfer id_rsa.pub to the
camera host computer. Then, in Bitvise SSH server config, install the public key following the
instructions at https://www.bitvise.com/wug-publickey. You should then be able to connect
from the Pi to the main computer using SSH without a password:
pi@raspberrypi ~ $ ssh [email protected]
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.
C:\Documents and Settings\CJ1>
Note that the IP address to use is the address of the main computer on the local network (and
not the external IP address or hostname).
Transferring the videos and shutting down the main computer The next step is to
set up the Pi to copy all the previous night’s video clips after the observing schedule has ended.
First, the UFOCapture observing schedule should be updated to stop UFOCapture shutting
down the main computer once the observing is complete. Next, the following commands can be
added to the startup.sh shell script in order to take a copy of all the previous night’s clips, and
then shut down the main computer when complete:
# Local videos location
videopath=/home/pi/NEMETODE/videos
# Get yesterday’s date in order to determine directory names
yyyy=$(date --date="yesterday" +"%Y")
mm=$(date --date="yesterday" +"%m")
dd=$(date --date="yesterday" +"%d")
# IP address of main computer on local network
remote_ip="192.168.1.97"
# Make local directory on the Pi to store the latest videos
local_dir=${videopath}"/"${yyyy}"/"${yyyy}${mm}"/"${yyyy}${mm}${dd}
mkdir -p ${local_dir}
# Set permission on the new directory so that we can delete contents
chmod 0777 ${local_dir}
# Get name of folder on main computer containing last night’s videos
remote_dir="E:\\"${yyyy}"\\"${yyyy}${mm}"\\"${yyyy}${mm}${dd}"\\"
# Copy all of last night’s videos and other files to the local directory
scp CJ1@${remote_ip}:${remote_dir}* ${local_dir}
# Issue command to shut down the main computer
ssh CJ1@${remote_ip} ’shutdown -s’
A couple of things to note: UFOCapture should be configured to save all the night’s video to the
same directory with the date set according to the start of the night (i.e. to not split the night’s
footage into pre- and post-midnight folders). This assumes that UFOCapture saves files into
a directory structure like e.g. \2016\201608\20160810 for clips obtained on the night starting
10th August 2016. The videopath should be updated to wherever you want to store the videos
on the Pi; the remote_ip should be set to the IP address of the main computer on your local
network, and CJ1 should be updated to whatever the username of your Windows account is.
This part of the script should be invoked after UFOCapture has stopped detecting: this can
be achieved by using a sunwait command to pause the startup.sh script until e.g. 30 minutes
after UFOCapture has stopped (in order to allow for small differences between the sunrise times
estimated by UFOCapture and Sunwait).
Connecting to the Pi to access the videos In order to connect to the Pi during the
day to download the videos, a SSH server should be installed on the Pi to handle incoming
connections. The Pi usually comes with a SSH server already installed; you can check this by
using the command which sshd, which will return something like /usr/sbin/sshd if a SSH
server is present:
pi@raspberrypi ~ $ which sshd
/usr/sbin/sshd
Note that you may need to enable the SSH server via the raspi-config, as explained in
www.raspberrypi.org/documentation/remote-access/ssh/. The next step is to update the
configuration of the network router to forward incoming SSH connections (port 22) to the Pi
rather than the main computer, as explained in section 2.2. Make sure to change the password
on the pi to something more secure, as the SSH connection will be public facing:
pi@raspberrypi ~ $ passwd
Changing password for pi.
(current) UNIX password:
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
pi@raspberrypi ~ $
Note that it’s still possible to connect to the main computer remotely over SSH by first connecting
to the Pi then to the main computer. Passwordless SSH to the Pi can be enabled by copying the
public SSH key of the remote machine to the ~/.ssh/authorized_keys file on the pi:
nrowell@hydra:~$ cat ~/.ssh/id_rsa.pub | ssh [email protected] ’cat >>
.ssh/authorized_keys’
[email protected]’s password:
You can then SSH to the Pi without having to type in the password.
Video postprocessing It’s handy to perform some basic processing on the videos and images
in order to make them more presentable, especially if they’re eventually presented on a website
running on the Pi. There are a few good pieces of software available for the Pi which can help
with this. These can be installed as shown below:
# Installing avconv (video processing/compression)
pi@raspberrypi ~ $ sudo apt-get install libav-tools
# Install zip (compression & archiving)
pi@raspberrypi ~ $ sudo apt-get install zip
# Install imagemagick (image processing)
pi@raspberrypi ~ $ sudo apt-get install imagemagick
As an example, in my system once the video clips are transferred to the Pi they are processed to
produce compressed MP4 and WMV versions of the raw video footage, rotated by ninety degress
as my camera operates in portrait mode. The raw data files are zipped up for easy transfer, and
the peak-hold image is rotated for use as a thumbnail image on the website. The commands for
doing all this are as follows:
# Loop over all the previous night’s clips
for AVI in *.avi; do
# Get file name with ’_NR.avi’ extension stripped off
NAME=$(echo ${AVI} | sed ’s/_NR.avi//’)
# Make compressed versions of the raw footage
avconv -i ${NAME}_NR.avi -vcodec libx264 -crf 0 -an -vf transpose=1 ${NAME}.mp4
avconv -i ${NAME}_NR.avi -vcodec wmv2 -crf 0 -an -vf transpose=1 ${NAME}.wmv
# Make thumbnail image
convert ${NAME}_NRP.jpg -transpose -flop ${NAME}.jpg
# Zip up the raw data files
mkdir ${NAME}
mv ${NAME}_* ${NAME}
zip -r ${NAME}.zip ${NAME}
rm -r ${NAME}
done
This can be added to the end of the startup.sh script to run automatically at the end of the
night, once the raw data has been transferred off the main computer to the Pi.
Troubleshooting
Loss of network connection on the Pi Experience with my system showed that from time
to time the Pi can lose the network connection and require a restart of the networking service.
Since switching to using a powered USB hub (rather than plugging the WiFi dongle directly
into the Pi) these events decreased in frequency dramatically, so in most cases they were due to
the dongle drawing too much power from the Pi USB ports. However, even with the powered
USB hub this happens occasionally, and I haven’t tracked down the root cause. A simple fix to
this is to use a short shell script to check the status of the network connection and restart it if
necessary. The following script does this:
#!/bin/bash
# Script network_check.sh: monitor for loss of network and restart if necessary.
# Ping four packets to router (replace 192.168.1.254 with the
# IP address of your router on the local network).
ping -c4 192.168.1.254 > /dev/null
# Check if this was successful, and restart the network if not
if [ $? != 0 ]; then
echo $(date) "> Lost network! Restarting..."
sudo /etc/init.d/networking stop
sleep 10
sudo /etc/init.d/networking start
sleep 60
fi
This script is then scheduled to run once every 15 minutes by adding the following entry to the
cron table:
# Monitor for loss of network every 15 mins
0,15,30,45 * * * * /home/pi/NEMETODE/scripts/network_check.sh
2.4
Automated disk administration
A couple of useful tasks to automate are monitoring the disk usage on both computers, and
defragmenting the hard drive on the camera host computer. These can both be achieved by
adding some extra commands to the same shell script as before, to be executed at the end of
the night after the videos have been copied and processed, and before the main computer is shut
down.
For monitoring disk usage, the technique I use is to write the free and used disk space on each
computer to a file. This can be achieved using the following commands in a shell script:
# Get disk usage for main computer (replace username and IP address with your own)
mainCompDriveE=$(ssh [email protected] "echo ’’ | wmic logicaldisk get
size,freespace,caption" | grep E:)
echo $mainCompDriveE | awk ’{print $2}’ > /home/pi/NEMETODE/logs/free_main.txt
echo $mainCompDriveE | awk ’{print $3}’ > /home/pi/NEMETODE/logs/size_main.txt
# Get disk usage for Raspberry Pi
df -hBk . | grep /dev/root | awk ’{print $3}’ | sed ’s/K//’ >
/home/pi/NEMETODE/logs/used_pi.txt
df -hBk . | grep /dev/root | awk ’{print $4}’ | sed ’s/K//’ >
/home/pi/NEMETODE/logs/free_pi.txt
After this script has finished, the files free_main.txt and size_main.txt in the /home/pi/NEMETODE/logs/
directory will contain single lines reporting the free space and total size of the main computer
disk; the files used_pi.txt and free_pi.txt report the used and free disk space on the Pi. These
files can either be checked manually, or they can be used to create a nice visualisation of the
disk usage for inclusion on a website, which is what I do. Figure 7 shows the effect that can be
achieved using HTML meter tags.
Figure 7: Disk usage dashboard
The snippet of website code which does this, using a combination of HTML and PHP, is shown
below:
<h2>Disk usage</h2>
<div id="dashboard">
<h3>Raspberry Pi</h3>
<?php
// Get the disk usage stats from file
$used = intval(file_get_contents("/home/pi/NEMETODE/logs/used_pi.txt"));
$free = intval(file_get_contents("/home/pi/NEMETODE/logs/free_pi.txt"));
$total = $used + $free;
echo "<meter value=’".$used."’ min=’0’ max=’".$total."’></meter><br>";
echo $used."K / ".$total."K";
?>
</div>
<div id="dashboard">
<h3>Main computer</h3>
<?php
// Disk free space and size for main computer
$free_main =
round(floatval(file_get_contents("/home/pi/NEMETODE/logs/free_main.txt"))/1000.0);
$size_main =
round(floatval(file_get_contents("/home/pi/NEMETODE/logs/size_main.txt"))/1000.0);
$used_main = $size_main - $free_main;
echo "<meter value=’".$used_main."’ min=’0’ max=’".$size_main."’></meter><br>";
echo $used_main."K / ".$size_main."K<br>";
?>
</div>
The defragmenting of the main computer hard drive can also be automated by adding another
few lines to the startup.sh shell script, to be executed after the videos have been transferred
and before the computer is shut down at the end of the night. These lines are shown below:
defrag=$(ssh [email protected] ’defrag e: /a’)
echo $(date) "> Defrag status report: " $defrag >> ${logfile}
# Check if ’You should defragment this volume’ appears anywhere in the defrag status:
if [[ $defrag == *"You should defragment this volume"* ]]
then
echo $(date) "> Defragmenting the main computer E: drive" >> ${logfile}
defragPerf=$(ssh [email protected] ’defrag e:’)
echo $(date) "> Defrag performance report: " $defragPerf >> ${logfile}
fi
Again, update the Windows account username and IP address accordingly.
Writing a log file Also included in the example above is the use of the echo command to
write some log messages to a file on the pi. Basically, the shell variable logfile contains the
path to a text file to which log messages will be written. The benefit of this is that the progress
and current state of the system can be monitored: in my system, I use lots of logging and keep
an eye on the log file to check that everything is working OK. For clarity, a lot of logging has
been removed from the shell scripts included in this document.
2.5
Remote monitoring using a website
One of the most useful and time saving elements of my Raspberry Pi system has been the
use of a webserver on the Pi to allow easy access to the videos and other monitoring data
through a simple website frontend (http://gargunnockmeteorcam.ddns.net/). The website can
be used to download or delete the videos, inspect the log file, check the disk usage and perform
other monitoring. Also, using interactive elements like buttons in conjunction with a server-side
language like PHP can allow direct operation of the GPIO pins, for example to immediately turn
on the main computer with a click.
Going through all the code for my website would be too much information - so instead this
section lists some tips and the main bits of software you would require to get a simple website
up and running.
Software The main piece of software that is required is Apache webserver (http://httpd.
apache.org). A webserver is an application that serves out webpages in response to HTTP
connections from remote machines. It runs in the background while the Pi is switched on. With
this alone, you can set up a simple website using plain HTML code. However, there are more
powerful languages for programming websites, perhaps the most useful of which is PHP. PHP
is a scripting language that runs on the webserver itself (server-side, i.e. on the Pi) and is used
mainly to create dynamic web content, e.g. a table containing all the video clips that happen
to be on the hard drive at the time the page is loaded. It is also useful for other tasks such as
reading and writing files on the server, and for executing commands e.g. to turn on and off the
GPIO pins on the Pi. Javascript (not to be confused with Java) is a scripting language that
runs on the client (client-side, i.e. in the website visitor’s browser) and is useful for creating
interactive elements on webpages, such as buttons. JQuery is a powerful library of Javascript
functions, which you can access by downloading a copy of the library to the Pi. All three of
these pieces of software are installed using the commands below:
# Install Apache and PHP
pi@raspberrypi ~ $ sudo apt-get install apache2 apache2-utils
pi@raspberrypi ~ $ sudo apt-get install php5 libapache2-mod-php5
# Download the jQuery library
pi@raspberrypi ~ $ cd /var/www/
pi@raspberrypi ~ $ sudo wget http://bit.ly/jqsource -O jquery.js
Port forwarding on home router As with the SSH server described previously, the home
router needs to be configured to forward incoming HTTP connections (port 80) to the Pi. Repeat
the steps decribed in section 2.2, this time forwarding port 80 to the Pi. Once this is set up, and
the webserver is running on the Pi, if you open a web browser and put the external IP address of
the home internet connection into the address bar (or the hostname, if you’ve set one up) then
you should be directed to the default website of the Apache webserver running on the Pi. This is
usually located somewhere like /var/www/index.html. This file can now be updated to contain
the code for your website.
Password protection It’s possible to make some or all of the website pages and content
password-protected, so that users that navigate to the page are prompted for a password in
order to access certain restricted areas. This could be used, for example, to allow the latest
video clips to be viewed by all users but to hide the log file and disk usage stats behind a
password. There are two steps to setting this up; the first is to create a file, readable by the
webserver, which contains the usernames and passwords cleared to access the restricted content.
The second step is to specify to the webserver which directories should be accessible only via a
password.
The password file is usually called .htpasswd, and is created using a dedicated program called
htpasswd as follows:
# Create the password file at /etc/apache2 and create a user ’nemetode’
pi@raspberrypi ~ $ sudo htpasswd -c /etc/apache2/.htpasswd nemetode
New password:
Re-type new password:
Adding password for user nemetode
The folders that are to be password protected are specified by editing the ‘virtual hosts’ file, which
is located at /etc/apache2/sites-enabled/000-default.conf. One entry should be added for
each directory that is to be protected, between <Directory> ... </Directory> tags as shown
below:
<VirtualHost *:80>
...
# Enter path of restricted content
<Directory "/var/www/html/protected">
AuthType Basic
AuthName "Restricted Content"
# Enter path to htpasswd file
AuthUserFile /etc/apache2/.htpasswd
Require valid-user
</Directory>
</VirtualHost>
Password protection will be enabled the next time the webserver is restarted, and users will be
prompted for a password whenever they try to navigate to pages or files within the restricted
directory.
2.6
Additional stuff that may be useful
Automated email alerts Having the Pi send you emails when certain things happen can be
very useful. In order to set this up, the packages ssmtp and mailutils need to be installed on
the Pi:
pi@raspberrypi ~ $ sudo apt-get install ssmtp
pi@raspberrypi ~ $ sudo apt-get install mailutils
These packages allow emails to be sent using simple one-line commands. However first you need
to specify the email address from which you want the emails to be sent. The easiest thing to do
is to create a new gmail account solely for this purpose, especially as the password will be saved
in a plain-text file. Then, open up the file /etc/ssmtp/ssmtp.conf and add the following lines:
root=postmaster
mailhub=smtp.gmail.com:587
hostname=raspberrypi
AuthUser=<your gmail address>
AuthPass=<your gmail password>
FromLineOverride=YES
UseSTARTTLS=YES
Now, emails can be sent from the command line using echo "Message body" | mail -s "Subject"
<recipient email address>. For example, the following script will check the current external
IP address of the Pi and send it by email to a given address:
#!/bin/bash
# Get current IP address
CURRENT_IP=$(/usr/bin/wget -qO- http://ipecho.net/plain)
# Send the current IP by email
echo ${CURRENT_IP} | mail -s "[RaspberryPi] New IP address" <recipient email address>
3
Things that didn’t work
This section records things that I tried that didn’t turn out to be successful.
3.1
Network share drive
I had various attempts to get the main computer to write the video files directly to the Pi rather
than to it’s own hard drive. This would be very useful as it would avoid the hassle of transferring
the video files and having multiple copies of them on the different disks. The most promising
technique was to connect the Pi and the main computer directly using an Ethernet cable, then to
set up a folder on the Pi hard drive (SD card) as a network share drive that the main computer
can directly write to. However, while this worked fine for the various images that UFOCapture
writes, the video data was corrupted presumably because the network and/or Pi couldn’t handle
the large data rate.