Introduction to the Maths Computer
System
December 6, 2016
Introduction
This booklet is divided into three parts.
• The first part introduces the local services. Even if you know
Unix you should read this section.
• The second part introduces Unix. We hope there is enough
there to keep you afloat until you can attend the University’s
Introduction to Unix course.
• The third part is a glossary of Unix terms. It will include some
extra commands not covered in part 2.
We have aimed to make all examples in this booklet work accurately
for new users. Old users may have slightly different setups but
should still find much of this booklet useful.
Your login-name on the Maths Linux system is your “CRSid”. Your
“CRSid” is generated from your initials and used by various systems
within the University e.g. Email (Hermes), Web authentication
(Raven).
This booklet uses “crsid” in place of your login-name in many of the
examples.
CRSid = Common Registration Service identifier.
1
Assistance: The Helpdesk
Our computing documentation is at
http://www.maths.cam.ac.uk/computing/
To contact us by email
[email protected]
and telephone at Xt 66100
If the problem is urgent, then email the really-urgent address and/or
phone the extension 66100.
[email protected]
When emailing us please include your crsid (university login name),
computer name, office number and phone.
Email (Hermes) and Raven services are run by the University’s
central IT services with their own pages.
http://www.ucs.cam.ac.uk/linkpages/newcomers
2
Part I: Local IT Services
The majority of people using the departmental computer system have
a Linux desktop. People with different desktops should still read this
booklet as you are likely to run your code on Linux.
Any member of the department who happens to have no desktop
provided should contact us via [email protected]. If you
cannot log into your desktop please check whether you have access
by visiting the online list of computers and entering your crsid.
http://www.maths.cam.ac.uk/computing/comptable/
Search for your computer’s name. If it is not there, please email us
with your research group, office number and the name of the desktop
computer. If it is there, please double-check that you have typed your
password correctly and that the computer is plugged into a network
socket before emailing us.
We run a version of Linux called Ubuntu 14.04.
https://help.ubuntu.com/
3
Laptops, Tablets and Smartphones
We provide both wireless and wired connections for your own
mobile devices and computers.
If you wish to use a wired connection please only connect to a
free network port which is marked with a sticker and/or an orange
network cable. The sticker will say something similar to
This data outlet is connected to the CMS laptop
network. Laptops may be connected here.
Connecting your laptop to any other port will not work. This
includes your desktop’s port since that is on a different network.
If you need to use this network and there are no laptop network
ports available in your office, please contact us by emailing
[email protected] with your office number and if possible the
number of the free network port you need to have added to the laptop
network.
For assistance with network connections see
https://www.maths.cam.ac.uk/computing/laptops/
Should you have a problem connecting or printing, please if possible
include the output from accessing the following page from your
laptop when emailing us.
https://www.wireless.cam.ac.uk/person/self/usage/
(wireless)
http://lapserv.maths.cam.ac.uk/authstatus/ (wired)
4
Virus Checker
Please make sure any computers you use other than the department
computers have up-to-date virus checkers on them. The University
has a license for some products which you can use free of charge.
http://www.ucs.cam.ac.uk/support/anti-virus/
If your computer has a virus and is attempting to spread it to
other computers, we will have to disconnect it until the problem is
resolved.
5
Linux Computers
In addition to your desktop computer you also have remote access to
many other computers on our network via ssh (see later).
To see the names, locations and spec of different computers check
http://www.maths.cam.ac.uk/computing/comptable/
Enter your CRSid to see just the computers to which you have login
access.
If the computer you want to use is switched off, you can switch it on
over the network with wake computer name from the command line
or via this web page.
https://things.maths.cam.ac.uk/it-admin/wakeup.
cgi
Be considerate towards other computer users. When running your
code on a computer that is in another office, make sure you leave the
office (console) user with enough resources to move their mouse and
type.
Do this by using the nice command. We have more information on
this topic at
http://www.maths.cam.ac.uk/computing/policy/longjobs.html
6
Windows Computers
We have a variety of ways for people to access Microsoft Windows
applications.
There are a few public computers running Microsoft Windows 7.
The Desktop Services / Managed Cluster Service (DS/MCS)
Computers in room GL.04 and the Part III room in the basement
of Pav B are accessible to department members. These computers
are dual boot Windows 7 / Ubuntu Linux and are managed by the
University Computing Service.
http://www.maths.cam.ac.uk/computing/mcs/
The windows remote desktop server which is called RDS is
accessible from the linux desktops.
Applications → Office → Windows Applications
Or if using the command line in a terminal or xterm:
$ windows_applications &
On this you will find all the standard Microsoft Windows programs
as well as some mathematical software.
http://www.maths.cam.ac.uk/computing/win7/
7
Printing and Scanning
Each pavilion has at least one public scanner and a number of public
printers. In general you have access to all of these. The webpage
below provides the locations and instructions for use of these.
http://www.maths.cam.ac.uk/computing/printing/
When a printer is flashing its orange maintenance light, the small
LCD display should tell you why. It may be out of paper, it may
have a paper jam (try to remove the jammed paper yourself, being
careful not to tear it) or it may be out of toner. Please change the
toner yourself if you feel confident to. Some pavilion secretaries
keep a supply of toners close by, and DAMTP toners are kept in
room BL.05 (basement of pavilion B). Old toners should be placed
in any of the black toner recycling bins and unwanted output in the
blue paper recycling bins.
If you are unable to resolve a printer issue yourself, please email
help@maths giving the name and location of the printer. If a printer
is out of toner and you cannot find a replacement cartridge please
speak to a secretary.
There are two poster printers. Please carefully check any conference
posters for errors before printing the final version on the poster
printer.
http://www.maths.cam.ac.uk/computing/printing/
poster.html
8
Printing Tips and Rules
If a printer isn’t working the best thing to do is remove your job from
the printer queue (use the lprm command) and send it to a different
printer.
If a PDF file won’t print, try using a different PDF viewer.
To save paper, think before you print, print double-sided where
possible, and consider printing more than one page per side of A4.
Please use black and white printers where possible because the
colour printers cost three times more to print the same black and
white page.
Be careful to use the correct media for overheads/transparencies as
the wrong ones can melt inside the unit, make it unusable for several
days, and cost both time and money to remove.
We do not charge for printing, instead we expect responsible usage.
http://www.maths.cam.ac.uk/computing/printing/
cant_print.html
http://www.maths.cam.ac.uk/computing/printing/
rules.html
9
Remote Access
Whichever departmental Linux computer you log into you will see
the same files (home directory) and the same environment. If your
office is moved, there is no need to move your computer.
To connect to a departmental Linux computer from any Windows
machine, use Winex which will give you the same graphical
environment as when sitting at the Linux computer.
http://www.maths.cam.ac.uk/computing/win7/winex.
html
The Linux computers shut themselves down when idle and unused
to save energy, but ssh.maths.cam.ac.uk is always switched on.
We have further information on this topic at
http://www.maths.cam.ac.uk/computing/remoteaccess/general.
html
If you are using a laptop and want to see your department home
directory from it see:
https://www.maths.cam.ac.uk/computing/laptops/
homedir.html
10
Quota and Backups
Your home directory (where you store files by default) has a quota
and is backed up regularly. If you delete or overwrite a file you
wanted please refer to
http://www.maths.cam.ac.uk/computing/files/
backups/deletedfile.html
Up in the top right hand side of your desktop is a quota icon. If you
hover your mouse over the square with a percent in it, you will see
how much quota you have and how much you are using. The quota
command will give you the same information from the command
line.
Keep an eye on the files you download and create. By default
these are placed in the Downloads or Desktop subdirectory of your
home directory, and their disk usage soon mounts up. Items in your
Wastebasket or Recycle Bin also count towards your quota. You
need to empty the Wastebasket or Recycle Bin to completely remove
them.
http://www.maths.cam.ac.uk/computing/files/quota.
html
11
Additional File Storage
You have access to additional file space in the store spaces. These
have no quotas and only optional limited backup (two weeks of).
Like your home directory, the store spaces are accessible from any
department Linux computer.
We also have scratch spaces which are the hard disks of different
computers and are accessible only from the computer itself. These
are wiped when a computer is reinstalled. They are useful if you
want to write a file directly instead of across the network.
For how to use store and scratch space see:
http://www.maths.cam.ac.uk/computing/files/storage.html
For a list of which storage spaces are available type store-space.
Please speak to your supervisor if you think your research group
should have its own storage space. The cost is £250 per terabyte.
Please remember that store space will not be backed up unless you
enable this feature with store-space snapshots GROUPNAME and
scratch space will not be backed up at all.
Please keep all shared areas tidy, preferably by creating a directory
named after your crsid and placing your files within.
12
Email and calendars
The University email service is called Hermes.
The most
straightforward way to access it from anywhere in the world is via
webmail.
https://webmail.hermes.cam.ac.uk/
The notes on how to set up other email programs are at
http://www.ucs.cam.ac.uk/email
http://www.ucs.cam.ac.uk/email/muasettings
The department supported email programs are pine, thunderbird and
webmail. If you need assistance setting up your email program, we
can help you provided you are running one of these. Otherwise
beyond pointing you at which settings to use we may not be able
to actually configure your program.
Information for mobile devices is at
http://www.ucs.cam.ac.uk/mobiledevices
The university’s calendar service is:
http://www.ucs.cam.ac.uk/googleapps/calendar.html
13
Your @cam address and the lookup service
Your official Cambridge email address is [email protected]. It is
important to make sure that email to this address is forwarded
somewhere that you read regularly. By default, it will be forwarded
to your Hermes account. You can check and edit your “@cam
delivery address” at
http://www.lookup.cam.ac.uk/
Please also check and fill out the other information on your
University lookup page. You can do this as soon as you have a Raven
account. The lookup service is used to search for people and for the
university email address book service.
14
Email Lists
Each member of the department should be on the
[email protected] email list.
A list of Maths email lists with links to department-specific lists is at
http://www.maths.cam.ac.uk/internal/email_lists/
You can see which @lists.cam.ac.uk lists your @cam address is
subscribed to and subscribe to other lists from your lists page
https://lists.cam.ac.uk/mailman/
http://www.ucs.cam.ac.uk/email/lists/listmanaging.
html
For more information on email lists see
http://www.ucs.cam.ac.uk/email/lists
15
Computing News
We maintain a message of the day on the internal webpage telling of
important changes to the system.
http://www.maths.cam.ac.uk/computing/news/
The early hours of Wednesday morning are reserved for computer
reboots. We try to announce these at least a few days in advance on
the message of the day.
Note: If you are writing code that will run over a number of days,
we recommend you checkpoint it. While the Wednesday morning
reboots are planned, sometimes we have a power cut or someone
accidentally switches off a computer.
http://www.maths.cam.ac.uk/computing/policy/longjobs.html#
Preparing
Some people prefer to read the computer announcements by email.
We keep two lists. One which has all the announcements sent
to it (maths-motd-all), the other which only has really important
ones sent to it (maths-motd-important). Messages about upcoming
reboots are sent to both lists.
To sign up for these and other @lists.cam.ac.uk mailing lists, go to
https://lists.cam.ac.uk/mailman/
16
Software
The list of numerical and mathematical software, including Matlab
and Mathematica, is at
http://www.maths.cam.ac.uk/computing/software/
If we don’t have software you need and it is free, you can try to install
it yourself. Some members of the department make their installed
code available for others to use. This saves multiple people from
installing the same code.
http://www.maths.cam.ac.uk/computing/software/
own.html
http://www.maths.cam.ac.uk/computing/software/
user-maint.html
Consider attending the free training course on how to install Unix
software.
http://www.training.cam.ac.uk/ucs/theme/unix?providerId=
36407
The UCS also sells software, but generally not for personal use so it
is best to ask us to order any software you need.
17
Scientific Computing
To learn about scientific computing please see the documentation.
• http://www.maths.cam.ac.uk/computing/hpc_sci/
• http://www.maths.cam.ac.uk/computing/mpi.html
You can search for “Scientific Computing”training courses at
http://www.training.cam.ac.uk
18
Guests and Visitors
Never give out your passwords.
If someone comes to visit you and they’d like WiFi access, generate
a wireless ticket for them here (select Colleague Tickets tab).
https://www.wireless.cam.ac.uk/person/self/
Or bring them to CMS Reception who can create longer-term tickets.
If you have a formal academic or official visitor staying more than
a few days see your group or pavilion secretary, preferably a few
days before the person arrives. They will issue your visitor with the
appropriate access.
If a new person is joining your group, your group/pavilion secretary
is the person to contact to arrange computer accounts. Please contact
the secretary at least a week before the person is due to arrive.
19
Privacy
All files you create on the department system are by default private
(readable only by you) and your email is also private.
If you want new files you create to be readable by other people, edit
the line at the end of your .bashrc file
umask 0077
to read
umask 0022
It is expected that people will not look at files not belonging to them
which they have no good reason to look at.
In Part 2 of this booklet we explain more about how to change who
can read your files.
20
Purchases
If you wish to buy a computer from your research grant funds, please
first check
http://www.maths.cam.ac.uk/computing/buying/
Then contact help@maths.
The University is a member of SUPC, which in summary means it
can purchase some hardware at discounted prices.
http://www.ucs.cam.ac.uk/support/
hardware-support/hwoverview
Broken Computers
The department fixes its own broken computers. Please email us if
your computer is broken, as we have no way to distinguish a broken
machine from one which is simply turned off.
For personal computers, a department computer officer may take a
quick look at your broken hardware (time dependent).
21
Sharing Resources
Please remember to consider others when working. For example
if you need to run code on lots of computers, leave some of the
better ones for others. Make sure your code doesn’t slow down the
computer for interactive use.
http://www.maths.cam.ac.uk/computing/policy/longjobs.html
In store and scratch spaces, look at the total space available (df -h .).
Archive and compress or remove your files as you finish with them.
When a space fills a lot of people are inconvenienced.
Please remove personal data from the system before you leave.
It is impossible for the computing staff to know which files left
behind are left intentionally to share with others, and which are
left by accident. Clearing files when you leave frees up space for
newcomers.
If you are using the shared software area, make sure you keep your
index file up to date so that others can see what software you are
sharing with them.
http://www.maths.cam.ac.uk/computing/software/
user-maint.html
Leave public computers in a good state and report any problems with
them to the helpdesk.
22
Saving Energy
Always use the commands safeshutdown and safereboot to shut
down or reboot your desktop computer. These check that no-one
else is using it before shutting it down.
http://www.maths.cam.ac.uk/computing/powersaving.
html
If your computer is unresponsive and safereboot does not work,
email help@maths and one of the computer officers will
investigate.
If you want your computer or phone moved to a different network
socket, or have an unwanted computer in your office, email help@
maths.cam.ac.uk. Do not move it yourself as most network
sockets are not connected and will not work.
Never unplug computers or turn them off at the wall. This
prevents other users from waking them up remotely to run a job.
Disconnecting the keyboard, monitor or mouse for use with a laptop
is OK.
Please leave your monitor on when you go home. It will
automatically go into a power-saving mode using virtually no
energy, and the computer will be able to keep track of the screen
resolution.
Office printers, but not public printers, can safely be turned off.
23
Health and Safety
If you get pain in your arms, wrist or hands from typing, STOP and
take a break. Don’t make it worse by continuing to type. The internet
has enough stories of people who could not work for months because
they ignored initial work related upper limb disorder pain.
http://www.uis.cam.ac.uk/health-and-safety/
upper-limb-disorders
If you need equipment to help you work more comfortably, for
example a different mouse or keyboard, contact the helpdesk.
24
Part 2 - Introduction to Unix
Our computer network consists mainly of Ubuntu 14.04 Linux
computers. The type of computer on your desktop depends largely
on what your group has purchased.
Many day to day tasks like reading email (webmail or thunderbird),
browsing the Web (firefox), running mathematical software like
Mathematica or Matlab, changing the appearance of your desktop
and moving files between folders can be managed with little
knowledge of the underlying system.
However, once you start to program, install software or analyse data
you will need to learn more.
A note on terminology: technically Linux is a Unix-like operating
system, however the terms “Linux” and “Unix” are often used
interchangeably, as in this booklet.
25
Education and Learning (basic and advanced)
If you have not used Unix before (or only used it for web-browsing
and email) please attend the UCS “Introduction to Unix” Course as
soon as possible. We strongly recommend this course.
The course runs over two half day sessions and is repeated several
times throughout the year.
http://www.training.cam.ac.uk/ucs/event/721643
For more advanced use of the shell and command line, please attend
the “Simple Shell Scripting For Scientists” course.
http://www.training.cam.ac.uk/ucs/event/722116
There are also courses offered in scientific computing.
To sign up for a course, start from
http://www.training.cam.ac.uk/ucs/theme
and choose the topic from the left-hand menu e.g. “Unix (including
Linux) Systems & Use” or “Scientific Computing”.
You will need your Raven password to sign up.
We have a Quick Reference List of Unix Commands online:
http://www.maths.cam.ac.uk/computing/linux/unixref.html
For detailed information about a command, use “man”. For example
‘man rm’ and ‘man rmdir’ to see the manuals on removing files
and directories.
26
Starting
The default department desktop is Xubuntu. We have others installed
which you can use, but Xubuntu is the desktop we know most about.
Other desktops may not be customised for the Maths system as much
as Xubuntu is. In particular they do not all have the quota icon or a
customised menu to the Maths applications (Applications → Maths
Programs).
For writing scientific programs and analysing data you need
some knowledge of the Command Line Interface (CLI) to the
UNIX operating system. This booklet contains many examples
of commands to type into the CLI. To obtain a Terminal window
(equivalent to a Command Prompt under Windows) click on the
black monitor icon at the top of your screen, or find it in the menus
(Applications → Accessories → Terminal Emulator).
When you have finished working on your computer, please log out
by selecting Log Out from the System menu. This will allow it to
shut itself down to save energy.
To reset your desktop to the default state, type xreset.sh, log out
and select the Xubuntu desktop (from an icon at the top right of the
login screen) before logging back in.
27
Shortcuts
The cursor keys (left and right, up and down) allow editing of the
current command and recalling of previous commands in an intuitive
manner.
Note also that pressing {TAB} when part-way through typing a file
name will cause the rest of the file name to be filled in automatically
if it is unique. This trick also works for command completion (type
his then press {TAB}).
The wild cards ‘?’ and ‘*’ can be used to stand for any one character,
and any string of characters respectively:
# ls lists files, rm removes (deletes) files
$ ls
a.dat b.dat results.dat write-up.txt
$ ls ?.dat
a.dat b.dat
$ ls *.dat
a.dat b.dat results.dat
$ ls *
a.dat b.dat results.dat write-up.txt
$ rm *
$ ls
$
In this way the CLI is much more powerful than using the mouse
and the file-manager.
‘rm *’ should be used with considerable caution.
‘rm -r *’ should be used with EVEN more caution as the -r means
a recursive removal of all files and directories.
28
Filenames
As you may surmise from the previous page, having a “?” or “*” in
a filename can cause confusion. So can spaces in filenames. While
using these characters in filenames is not absolutely forbidden,∗ until
you are much more confident with Unix it is best to stick to the
following characters.
• Letters and numbers.
• Underscore ( ).
• Hyphen (-) and full stop (.), but not at the beginning of a
filename.
Unix filenames are case-sensitive i.e. it is possible to have three
different files in the same directory called MYFILE, MyFile and
myfile.
∗ The only character which is absolutely forbidden in a Unix filename is “/”, because this
separates the components of a path like /home/eva/test files.
29
Programs and Applications
This booklet uses the terms ‘application’ and ‘program’ interchangeably.
People tend to use the term ‘program’ when they use the Command
Line Interface (CLI) to type in the name of the program, and
‘application’ when they click a menu item or double click an
application icon.
We use the Terminal application to access the CLI. Click on the
black Terminal icon, or select from the menus (Applications →
Accessories → Terminal Emulator).
Terminal allows a shell or command interpreter to run. This
obeys our commands and produces a prompt consisting of the
machine name followed by the current directory. (A directory is
what Windows, MacOSX and some mobile phones call a “folder”.)
Assuming you are in your home directory your prompt will look like
this∗
crsid@machine:˜ $
Owing to space restrictions, in this booklet the prompt will be shown
as a simple $.
∗
˜ means your home directory, ˜user1 means user1’s home directory.
30
Starting Programs and Applications
From the CLI programs which are on your PATH are started by
typing their name:
$ firefox &
(the & sign causes a background process. This means you get
your prompt back immediately so that you can start other programs
without waiting for the firefox program to exit.)
The PATH and where a program is found on the PATH can be seen
with
$ echo $PATH
$ type firefox
Not all the locations you will put programs are on your PATH. To
start a program you have downloaded (e.g. with firefox):
$ cd Downloads
$ ./prog-01
(Without the & this is a foreground process. This will run the
program from the CLI and only return you to the shell prompt when
the program finishes/exits.)
You will learn more about this on the recommended CLI course.
31
Home Directory
Under the Places menu (which is beside Applications) is your Home
Folder marked with a house icon and your CRSid. This is your initial
working space. By default all your files are stored here.
The Home Folder is known as your Home Directory in UNIX
terminology.∗ In a similar manner the Folders in your Home
Directory are known as Directories in UNIX terminology.
An alternative way of viewing the files and directories in your Home
Directory is to use the CLI. You can see what files are present by
typing ‘cd’ to move to your home directory and then ‘ls’ to list the
contents.
$ cd
$ ls
private/ public_html/ shared/
public/ results.dat
thesis.tex
Regardless of which Unix computer you log into, you will always
start in your home directory. This means that you can use any of the
computers in your office or on the benches.
Your home directory is called /home/crsid or ˜ for short. ls ˜
will list the contents of your home directory without changing your
current directory, and cd ˜ is an alternative way to return to your
home directory.
∗ This can be likened to ‘My Documents’ under Windows or Home on a Mac.
32
Quota
As explained in Part One your Home Directory has a quota. The icon
at the top right hand side of your screen shows your quota usage.
Alternatively you can check your quota with the CLI:
$ quota
Disk quotas for user evatest3 (uid 1016):
Filesystem blocks
quota
limit
grace
rpc-serv1.maths.cam.ac.uk:/local/home/stats
83972 2097152 2097152
files
1337
This person is using about 82MB (83972 / 1024) of a 2GB quota.
We do not have a limit on how many files people can have.
Be careful not to run out of quota because then odd things
happen. For example your web browser may end up with corrupted
bookmarks, you may be unable to save a file in your home directory,
or the system may not allow you to log in.
To get more quota email help@maths and specify how much more
quota you need.
33
quo
Hidden Files
Your home directory contains some “hidden” files and directories
whose names start with a full stop. To list these files use View →
Show Hidden Files in the file manager. Or on the command line
$ ls -a
.bashrc .emacs .gconfd/ .gnome2/ .login .vimrc
Only edit these “dot files” if you know what you’re doing. An error
in these can prevent the associated application from working or lock
you out of your account. If you do run into difficulties from trying to
modify these files email [email protected]. It may also help to
reset your desktop with xreset.sh or your “shell configuration”
with /alt/bin/shellreset.
You can learn about the commands in your .bashrc file with
man bash
Press “/” to search for the command you are interested in.
34
Who can read your files—page 1
There are three sets of people—you, your “Unix group” and
everyone else on the system. Use the Properties function in the File
Manager (Places → CRSid, or double-click the Home icon on your
desktop) to view or amend who can read or execute a file. Select the
file or directory with the right mouse button, then select properties
and then the permissions tab.
Our Unix groups are loosely based on research/department groups,
but sometimes two or more smaller research groups are combined. If
you want to have some files which are readable (or writable) by your
research group but not by everyone on the system, it’s best to email
help@maths and ask them to create a small group for you. Tell them
exactly who needs access to these files.
You can also change who has access to your files with the CLI. This,
while initially harder to learn, is useful when you have many files
you would like to update at once.
For this you use the ‘chmod’ command.
For more information see:
http://www.maths.cam.ac.uk/
computing/linux/unixinfo/perms.html
Technically even if someone can read a file it doesn’t mean they
should read the file unless they have the owner’s explicit permission
to do so.
35
Who can read your files—page 2
By default no-one else can read any of your files. To make all
your new files readable by everyone else on the system, replace the
“umask” line at the end of your .bashrc with
umask 0022
To make an existing directory and everything in it readable by
everyone else, type
chmod -R go+rX my_directory
CAUTION: Don’t do this for your home directory. Some computergenerated “hidden” files may contain passwords or other confidential
information.
36
Who can read your files—page 3
$ ls -l thesis.tex
-rw-r--r-- 1 crsid
group
−rw−r−−r−− 1 mjr19
Access permissions
File owner
3410 Oct
12 15:13 thesis.tex
2 15:13 thesis.tex
Length (bytes)
File’s group
File type
− normal file
tcm
3410 Oct
File name
Time last modified
Link count
d directory
(1 for files, 2 or more for directories)
l link
Access permissions:
File
r Readable
w Writable
x eXecutable
Directory
ls works
File deletion and creation work
Can access files within directory.
The first three characters refer to the file’s owner.
The next three to people in the file’s group.
The final three to everyone else.
‘ls -ld *’ will list actual directories in long format, whereas
‘ls -l *’ will list the contents of each directory in long format.
37
Creating Documents
A text editor is not a word processor, and vice versa.
Word processors like Microsoft Word break up lines spontaneously
and concern themselves with the minutiæ of typography. This can be
useful when trying to format a document nicely for publication (see
next page on LATEX for an alternative which is better at formatting
maths) but not when programming.
LibreOffice is the Unix system’s equivalent of Microsoft Office. It
is under Applications → Office. LibreOffice can read and write
Microsoft Office documents.
Text editors are used to edit plain text files. This includes source code
e.g. shell, C or Fortran programs, LATEX, and HTML—basically,
anything you can type in. Windows has a very basic text editor
called Notepad. Linux has more advanced text editors which can
automatically highlight parts of your source code in different colours
to make it easier to read.
Text editors are usually started from the command line, or can be
found in the menus under Applications → Accessories. Of the many
text editors available, we know most about emacs, gedit and vim.
38
LATEX
LATEX is a text processing system which is particularly suited for
producing technical documents. It encourages users to focus on
the structure and content of their documents rather than the precise
details of the layout.
LATEX is used in a slightly different manner from WYSIWYG (What
You See Is What You Get) word processors. First you create a .tex
file using a text editor, then you convert it into a form which you can
view on screen.
Windows users can create LATEX documents with the TeXnicCenter
program. Linux users can type the following commands:∗
latex file.tex
xdvi file.dvi &
dvips -Pps2 file.dvi
# Create file.dvi
# View file.dvi
# Print file.dvi to printer ps2
pdflatex file.tex
evince file.pdf
# Create file.pdf
# View and optionally print file.pdf
The UCS run a LATEX training course.
http://www.training.cam.ac.uk/ucs/event/721788
∗ The # sign indicates that the rest of the line is a “comment” which the computer will ignore.
You do not need to type these, but if you are cutting and pasting from an electronic form of this
booklet it is harmless to leave them in.
39
Viewing files
For text files there is no need to open an editor just to view their
content. To view a file using the mouse, double click on that file. If
this fails, right-click and choose a program to view the file with.
To view a text file with the CLI use the command ‘cat’ or ‘less’.
‘less’ will show you the file page by page while ‘cat’ will display
the whole file at once and you may need to use the terminal scroll
bar to see the beginning of the file.
For non-text files like PDF files, use the appropriate installed
software readers, for example evince for PDF files:
$ evince thing.pdf &
If you do not know what to use to open the file try using:
$ xdg-open thing.jpg
cat is also useful for merging the content of two or more text files.
$ ls
paper1.txt paper2.txt
$ cat paper1.txt paper2.txt > papers-combined.txt
$ ls
paper1.txt paper2.txt papers-combined.txt
Use > to send output to a file, and >> to append output to a file.
Use | to send the output to the input of another command.
For example to display all processes, one page at a time:
$ ps -A | less
40
Simple File Operations
Files can be copied, moved (renamed) and removed (deleted).
The CLI commands are ‘cp’, ‘mv’ and ‘rm’.
Be Careful!: deletion is irreversible!
$ ls
result.dat thesis.dvi thesis.tex
$ cp thesis.tex new_book.tex
$ rm thesis.dvi
$ mv result.dat results.dat
$ ls
new_book.tex results.dat thesis.tex
The best way to learn all file operations using the graphical file
manager is to experiment and read through the help pages (from the
File Manager window, Help → Contents → Read Online).
Files in the “Rubbish Bin” are not actually deleted and still count
towards your quota though they are not backed up. To empty
your Rubbish Bin, double click on the “Rubbish Bin” icon on your
desktop, then File → Empty Wastebasket.
Or type rm -rf .Trash from the command line.
41
Tidiness
It is best to place files in tidy groups in sub-directories, rather than
having everything in one directory. The CLI offers ‘mkdir’ which
creates a directory, and ‘rmdir’ will remove one provided it is
empty. The ‘cd’ command changes the current directory.
$ mkdir test
$ ls
thesis.dvi
private/
test/
$ rmdir test
$ ls
thesis.dvi
private/
$
results.dat
thesis.tex
results.dat
thesis.tex
For the adventurous you can remove directories with contents (and
all sub-directories) using the ‘rm’ command with the -rf options:
$ rm -rf stuff
Be careful!
To create a new sub-directory using the graphical interface open your
Home Folder then File → Create Folder.
42
Where Is It?
If you created a file but you can’t remember where you put it, there
are various search mechanisms.
find is a standard Unix command, hlocate is a local command which
searches your home directory. You can use either of the following
two commands to search for a file with “thesis” in its name:
find ˜ -name "*thesis*" -print
hlocate thesis
You can also search by content, although this is slower than
searching by name as the shell needs to look inside every file. To
search for a file within the current directory which contains the word
“Internet”:
find . -type f -print | xargs grep Internet
More on searching for files:
http://www.maths.cam.ac.uk/computing/files/finding.html
If it turns out that the file was deleted rather than simply mislaid, see
the earlier page on backups.
http://www.maths.cam.ac.uk/computing/files/
backups/deletedfile.html
43
Trees
Directories form a tree: each directory has one parent directory, and
may have multiple sub directories. A file name is assumed to refer
to the current directory. Other locations can be specified by forming
a path using ‘/’ to separate the components of the path, ‘..’ to refer
to a directory’s parent, ‘.’ to start a relative path and ‘/’ to start an
absolute path.
$ ls -F
a/ results.dat
$ cp results.dat a/results.dat
$ ls -RF
.:
a/ results.dat
./a:
results.dat
$ ls ./a
results.dat
$ mkdir b
$ cd b
$ cp ../a/results.dat .
$ cd ..
‘ls -F’ distinguishes directories by placing a ‘/’ after their names.
‘ls -R’ lists all sub directories in a recursive fashion.
‘cd’ typed on its own returns one to ones home directory.
A similar ‘tree’ effect can be seen with the File Manager by using
View → View as List and expanding and collapsing the directories.
Recursion
It is a very useful thing to have Unix commands recurse through a
directory tree. ‘man ls’, ‘man grep’ etc. will help. Remember
that using a recursive command on a BIG directory tree can take a
LONG time, so think before hitting the Enter key.
44
Stopping Runaway Programs
It is usual to run several processes (programs) simultaneously.
A web-browser runs as an individual process, as does an email
program, an editor and a compiler.
Occasionally it is necessary to exercise some direct influence over
processes.
For example you may want to stop a program running, maybe you
opened xv∗ to see what it did and now can’t get out of it. From
another terminal or xterm use the ‘killall’ command.
$ killall xv
The command ‘ps -A’ will show all the processes running on a
machine. This may be overwhelming, so
$ ps -u crsid
or
$ ps -A | grep crsid
may be more helpful. Also pgrep and pkill on Linux.
Each process has a unique identifier, its Process-ID (PID). To request
that a process with a PID of 1234 exits, type
$ kill 1234
If that fails, try to force the process to shut down, although this gives
the application no opportunity to shut itself down neatly.
$ kill -KILL 1234
If that fails, email [email protected].
Note that you cannot stop processes you do not own!
∗ xv is a useful application for cropping and changing the size of images. When you start it
you need to know to click with the right mouse button to get the menu
45
Sharing Resources
When you run programs on different machines it is important to
share the available resources with other people. Running code on
every computer will leave nothing for anyone else.
The Maths online documentation has a list of rules for running code
and advice on how to make your program ‘nice’ to both the system
and other users. Following these rules allows others to use the
computer while your code is running. This is essential on our shared
network.
http://www.maths.cam.ac.uk/computing/policy/longjobs.html
The command ‘top’ will produce a continuously-updated list of
running processes. Don’t leave it running for ever: press ‘q’ after
a while.
To see how much memory a computer has
$ cat /etc/motd
or see the computer list:
http://www.maths.cam.ac.uk/computing/comptable/
46
Large Files
Data files tend to be large. Often too large for storing in your home
directory. Programs that write data to your home directory may quit
unexpectedly when your quota fills up. ‘store’ areas, described in
Part 1, are dedicated storage typically used for long term storage of
data.
To see which store areas you have access to, type store-space.
Everyone has access to their department’s store area and some
research groups have paid for store areas of their own.
To create your store space in one of the areas listed, type e.g.
store-space create DPMMS
This will create a directory /store/DPMMS/crsid which can then be
accessed with the ‘cd’ command like any other directory.
Owing to the automounter only having the store areas actually in use
mounted at any given time, you cannot access /store/DPMMS via
the GUI without first going via the command line. To get around
this you can create what is called a symbolic link
$
$
$
$
cd
mkdir mydata
cd mydata
ln -s /store/DPMMS/crsid/ mystorespace
Now when you double click on your folder mydata, you will see
mystorespace. Double clicking on it will bring you direct to the
store space you created.
47
Where to Store Files—Scratch Spaces
In addition to your home directory and the store areas, each
computer has its own ‘scratch space’ for storing files. This
was described briefly in part 1 of this booklet. Note that you can
only access a computer’s scratch space on that computer.
$ cd /local/scratch/public/
$ mkdir /local/scratch/public/crsid
Or use the GUI interface. Start by double-clicking on File System
then go through local, scratch and public.
Use a computer’s ‘scratch space’ to store data from running
code on that computer. It is much quicker to write data locally
than write data over the network (e.g. to a ‘store’ area or homedirectory).
When your code has finished move the datafile to a more reliable
storage place, for example /store space, if you want to keep it.
The “scratch areas” can also be used to store Internet downloads or
images as these are recoverable if the disk dies and not so valuable
that you should use your home directory quota on them.
An unsafe area for saving files is /tmp.
48
Accessing Maths Computers Remotely
A list of Maths Linux computers:
http://www.maths.cam.ac.uk/computing/comptable/
You can wake up computers from here if they are powered down.
To log into any Maths Unix machine from another:
$ ssh computer1
From a Unix computer outside Maths use an ssh client to connect
to the full name e.g. ssh.maths.cam.ac.uk (which is always
switched on).
$ ssh -C -Y [email protected]
Sometimes options need to be passed to ssh. For example ‘-Y’ when
running remote graphical applications like matlab or firefox and ‘-C’
to compress data if your home broadband is slow.
For Windows computers “putty” is the recommended ssh client.
Note: this will only give you a text-based session. If you want a
graphical session use Winex instead.
http://www.chiark.greenend.org.uk/˜sgtatham/putty/
http://www.maths.cam.ac.uk/computing/win7/winex.html
http://www.maths.cam.ac.uk/computing/remoteaccess/general.
html
49
File Transfer
Use ‘rsync’ to synchronise files between different areas. For
example to copy your Maths home directory to your laptop:
laptop$ mkdir my_maths_files
laptop$ rsync -avz \
[email protected]:/home/CRSid/ my_maths_files
Copy files from your laptop to a Maths data area (this copies the
directory laptop data into /store/DAMTP/crsid):
laptop$ rsync -avz \
laptop_data ssh.maths:/store/DAMTP/crsid/
Note that the EOL \ means the command continues on the next line.
See also
http://www.maths.cam.ac.uk/computing/files/
linux_file_transfer.html
50
Conclusion
The UNIX Command Line Interface (CLI) is fantastically useful and
learning it sooner rather than later is advised.
http://www.training.cam.ac.uk/ucs/event/721643
Should you really, really not have time to attend this half day course,
you can download, print and read the course notes instead.
http://www.ucs.cam.ac.uk/docs/course-notes/unix-courses/
UnixCLI
Leaving learning these skills until the last minute when you have
a thousand data files to manipulate and no time to learn, means
repeating the same action over and over and over and over. . . . The
shell and CLI provide tools to greatly simplify this sort of activity.
51
Part 3 - Glossary of Terms
This section lists some useful command and summarises their use.
To find out more about any of them run man command-name or use
google.
There are lots of additional programs under the Applications menu
on your desktop.
awk
Manipulates files.
bash
Default shell. Look for “Bash unix shell” on wikipedia for a brief
history and list of keyboard shortcuts. For more information man
bash.
bunzip2
Uncompress a file.
bzip2
Compress a file to save space.
cal
Shows this month’s calendar. For the whole year run cal 2016.
cd
cd when run on its own changes to your home directory.
52
cd when given an argument changes to that directory
cd /store/DAMTP/crsid
cat
Display a file.
Or concatenate (merge) two or more files together.
cat file1 file2 > file3
chmod
Change the permissions on a file, usually to grant or deny others read
access or to make a file executable.
cp file1 file2
Copy file one to file two. Be careful. This may overwrite file two if
it exists.
cp file1 file2 file3. . . dir will copy multiple files to a directory.
date
Show date and time.
du dirs. . .
Show disk usage of the directories. Useful when you are running
down on quota and trying to figure out where your space is used up.
To list your files and directories by size, typically from your homedirectory run du -sk * .??* | sort -n
diff filename1 filename2
53
Show differences between two files.
echo text
Repeats its arguments. Useful for displaying the values of system
variables e.g. echo $PATH and in scripts for keeping the user
informed.
emacs
Traditional unix editor. Graphical. Has old fashioned computer
games under its Tools menu.
env
Display environment variable settings.
file filename
Guess the type of file. Useful when a file has no extension (.pdf, .txt
etc). While it is not always accurate, if it says that a file is any sort
of “text”, you can safely read it without messing up your screen.
find
Find a file. Typically used when you know a file is somewhere
in your home-directory or a certain data space and you cannot
remember which sub-directory it is in.
find . -name “test∗” Find all files starting with the string “test”
under the current directory.
firefox
Web browser.
54
firefox -ProfileManager start firefox by creating or selecting a new
profile. It can be useful to try a new profile when something
goes wrong and you can’t tell if it’s your firefox or happening for
everyone.
gedit
Graphical unix editor.
grep string filename
Find some text in a file.
gvim
Graphical edition of the traditional unix editor vim.
head filename
Displays the top ten lines of a text file.
history
Displays the last commands you ran.
LATEX
LATEX is used for writing mathematical papers; its strength is in
displaying mathematical equations. The programs ‘kile’ and
‘texstudio’ are used for writing LATEX documents. These
programs give you the ability to use all the functionality of LATEX in
a graphical environment, for writing, compiling and viewing LATEX
documents.
These programs contain LATEX reference material and the Internet
contains numerous guides to LATEX.
55
kile
GUI LATEXprogram.
kill
kill pid Ask a process to exit.
kill -KILL pid Tell a process to exit.
killall program-name
Causes all instances of program to be killed.
less filename
View a file, page at a time. Keystrokes include:
• q - exit, ↑, ↓ up and down one line
• b - back one page, {space} - next page
• G - end of file, 1G - beginning of file
lpr file
Print a file (text or Postscript) to default printer.
To print to a named printer append the printer name. lpr -Pprintername filename
lpq
List printer queue (can add -P).
ls
56
List contents of directory.
To list contents with sizes etc., and sorted by modification time ls -ltr
man command
On-line manual.
man -k keyword Search on-line manual for a keyword.
mkdir directory
Make directory.
mv file1 file2
Move (rename) file1 to file2, deleting file2 if it exists.
Move multiple files to a directory
mv files. . . directory
pico
Unix editor.
pine
Email program.
ps
Shows running processes.
ps aux list all processes.
ps -elf list all processes.
quota
57
Show filespace quota.
rm filename
Deletes Files.
rmdir directory
Remove directory (if empty).
sed
text replacement.
ssh computer-name
Connect to another computer.
sort
For sorting files.
tail filename
Displays the last ten lines of a text file.
texstudio
Graphical LATEX program.
thunderbird
Email program.
tidy
Validate a webpage. It’s important to keep webpages accessible to
all.
58
$ tidy -asxhtml file.html > newfile.html
time command
Time a simple command.
top
View process activity. Press q to quit.
tr
Changing characters.
vim
Traditional unix editor.
w
w shows who is using a computer, what command they are running
and the load on the computer.
wc filename
Count lines and words in text file.
whereis program-name
Tells you where a program is installed.
who
Who is logged onto a computer.
xdg-open
59
Open a file. This program will take a guess at what file type a file is
and open it with the appropriate piece of software.
xterm
Opens a new command shell in a window.
60
*, 27
/, 43
?, 27
>, 39
>>, 39
xdg-open, 39
xterm, 29
xv, 44
backups, 11
cat, 39
cd, 31, 41
chmod, 34
cp, 40
evince, 39
groups, 34
home directory, 31
kill, 44
killall, 44
less, 39
ls, 31, 36, 43
mkdir, 41
mv, 40
path, 30
permissions, 36
ps, 44
rm, 27, 40, 41
rmdir, 41
terminal, 29
text editors, 37
wild cards, 27
61