Convey System Administration Guide
November 2010
Version 1.6
901-000002-000
 Convey ComputerTM Corporation 2009-2010.
All Rights Reserved.
1302 East Collins
Richardson, TX 75081
i
The Information in this document is provided for use with Convey Computer Corporation (“Convey”)
products. No license, express or implied, to any intellectual property associated with this document or
such products is granted by this document.
All products described in this document whose name is prefaced by “Convey” or “Convey enhanced“
(“Convey products”) are owned by Convey Computer Corporation (or those companies that have licensed
technology to Convey) and are protected by patents, trade secrets, copyrights or other industrial property
rights.
The Convey products described in this document may still be in development. The final form of each
product and release date thereof is at the sole and absolute discretion of Convey. Your purchase,
license and/or use of Convey products shall be subject to Convey‟s then current sales terms and
conditions.
Trademarks
The following are trademarks of Convey Computer Corporation in the United States and other countries:
Convey Computer
The Convey Logo
Convey HC-1
Trademarks of other companies
Red Hat and Red Hat Enterprise Linux are registered trademarks of Red Hat, Inc.
Fedora and RPM are trademarks of Red Hat, Inc.
SUSE and openSUSE are registered trademarks of Novell, Inc.
Linux is a registered trademark of Linus Torvald
Python is a registered trademark of the Python Software Foundation
GNU is a trademark of The Free Software Foundation
PERL is a trademark of the Yet Another Society DBA The Perl Foundation
Ubuntu and Canonical are registered trademarks of Canonical Ltd.
Debian is a registered trademark of Software in the Public Interest, Inc.
Convey System Administration Guide v1.6
ii
Revisions
Version
Description
1.0
May 2008. Original printing.
1.1
August 2008. New system setup procedures.
1.2
May 2009. Reworked for Beta Release of HW, OS, and SW.
1.3
June 2009. Revised reboot instructions when coprocessor is hung.
New scripts.
1.4
September 2009. Recovery DVD usage instructions. New SW repo
conventions.
1.5
April 2010. New licensing mechanism. New DVDs for updating
systems. New cross-development system install instructions.
1.6
November 2010. DoWebUpdate command for updating Convey
servers. Web interface for adding/changing web access
users/passwords.
Convey System Administration Guide v1.6
iii
Table of Contents
1
Overview ..................................................................................................................... 6
1.1 Intended Audience .................................................................................................. 6
1.2 Topics Covered ....................................................................................................... 6
1.3 Other Suggested Documents ................................................................................. 6
2
Installing Convey Software ......................................................................................... 7
2.1 Internet connection ................................................................................................. 8
2.2 Convey Software Repositories for Cross-Development Systems ......................... 8
2.2.1 Where to install the Convey .repo/.list files ..................................................... 8
2.2.2 Downloading and installing the Convey repository description files ............... 9
2.3 Convey Server Software ......................................................................................... 9
2.3.1 Convey O.S. Installation ................................................................................ 10
2.3.2 Installing Convey O.S. from a DVD ............................................................... 10
2.3.3 Convey Firmware Installation ........................................................................ 10
2.3.4 Convey Coprocessor Recipe Packages........................................................ 10
2.3.5 Updating Convey OS, firmware, and Recipe Packages ............................... 10
2.3.6 Installing other Convey Software products on a Convey Server .................. 11
2.4 Installing Convey Software on Cross-Development Systems.............................. 11
2.4.1 Compatible Linux® Distributions ................................................................... 11
2.4.2 SELinux Considerations ................................................................................ 12
2.4.3 Available Convey Software Development Packages .................................... 12
2.4.4 Identifying the Appropriate Installation Tool .................................................. 13
2.4.5 Installing Convey Software Packages ........................................................... 14
3
Updating Convey Software Packages ..................................................................... 15
3.1 Updating Convey Software Packages using a GUI ............................................. 16
3.2 Installing older versions of Convey Software Packages ...................................... 16
3.3 Version Numbers and Package Filenames .......................................................... 16
3.4 Manual Installation ................................................................................................ 16
3.5 Additional Installation Notes for Ubuntu/Debian and openSUSE ........................ 18
3.5.1 How to downgrade gcc and g++.................................................................... 18
3.6 Installing Convey Software without an Internet Connection ................................ 18
4
Booting and System Management ........................................................................... 20
4.1 Overview ............................................................................................................... 20
4.2 Powering Up the Convey Server .......................................................................... 20
4.3 First Power-Up and Boot ...................................................................................... 20
4.3.1 Intel Remote Management Module ............................................................... 20
4.3.2 First Boot ........................................................................................................ 20
4.3.3 Manual Configuration .................................................................................... 21
4.4 Boot Options ......................................................................................................... 21
4.4.1 enable/disable ................................................................................................ 22
4.4.2 caemode ........................................................................................................ 22
4.4.3 memory .......................................................................................................... 22
4.4.4 window ........................................................................................................... 22
4.4.5 pagesize ......................................................................................................... 22
4.4.6 interleave ....................................................................................................... 23
4.5 Coprocessor Modules ........................................................................................... 23
4.5.1 CNYMP Module ............................................................................................. 23
Convey System Administration Guide v1.6
iv
4.5.2 CNYSYS Module ........................................................................................... 23
4.6 Kernel Configuration ............................................................................................. 24
4.7 Convey System Services ...................................................................................... 24
4.7.1 Convey Configuration Files ........................................................................... 25
4.8 Coprocessor Sanity Test Program ....................................................................... 25
4.9 When to Shutdown the Convey Server ................................................................ 26
4.10 Activating a Coprocessor Recipe ......................................................................... 26
5
Convey System Memory Management .................................................................... 27
5.1 Overview ............................................................................................................... 27
5.1.1 Intended Audience ......................................................................................... 27
5.1.2 Terminology ................................................................................................... 27
5.2 NUMA Memory Configuration ............................................................................... 27
5.2.1 Physical Memory Configuration..................................................................... 28
5.3 Address Extension Feature .................................................................................. 28
5.3.1 Fully-mapped mode ....................................................................................... 28
5.3.2 Dynamic-mapped mode ................................................................................ 29
5.4 Memory Interleaving ............................................................................................. 29
5.4.1 Binary Interleave ............................................................................................ 30
5.4.2 31-31 Interleave ............................................................................................. 30
5.4.3 31-31 Interleave Page Allocation .................................................................. 30
5.4.4 31-31 Interleave Faults .................................................................................. 31
5.5 Variable Page Size Support ................................................................................. 31
5.5.1 Maximum Coprocessor Page Size ................................................................ 32
5.5.2 Linux HugeTLB Feature ................................................................................ 32
5.6 Memory Migration ................................................................................................. 32
6
Personality Management.......................................................................................... 34
6.1 Personality Signatures .......................................................................................... 34
6.2 Default Base Signature ......................................................................................... 34
6.3 System Default Signatures ................................................................................... 35
6.4 Convey Personality Management Script .............................................................. 35
7
Recovery DVDs ........................................................................................................ 37
7.1 How to use the Convey System Recovery DVD .................................................. 37
7.1.1 Preparation .................................................................................................... 37
7.1.2 Boot from DVD ............................................................................................... 37
7.1.3 Configure the Installation and Installing the OS ............................................ 38
7.1.4 System Configuration........................................................................................... 38
7.2 Fixing the Convey SW Repository Files ............................................................... 39
7.3 Adding Convey Provided License Keys ............................................................... 40
7.4 Installing the Remaining Convey Software Products ........................................... 40
7.5 Checking the Management Processor ................................................................. 40
7.6 Updating the Server to the latest SW Versions .................................................... 41
7.7 Restoring a Convey Server without a DVD drive ................................................. 41
8
Customer Support Procedures................................................................................. 42
9
Licensing ................................................................................................................... 43
9.1 Convey Licensed Software ................................................................................... 43
9.2 Old Licensing Scheme .......................................................................................... 43
Appendix A - Installing YUM on older SUSE™ Distributions ........................................... 44
Index
. ................................................................................................................... 45
Convey System Administration Guide v1.6
v
1 Overview
This guide describes how to perform various tasks that are typically performed by a system
administrator. These tasks require system administrator privileges.
Please note that this document contains clickable hyperlinks. If you are browsing this document
on Convey‟s web site, or downloaded all of Convey‟s documents to the same directory, those
clickable hyperlinks should work. If you only download a single document, clicking on the
hyperlinks will fail.
1.1
1.2
Intended Audience

Administrators of a Convey hybrid-core server

Users of desktop Linux systems (who have root access) used for cross-development of
applications for a Convey hybrid-core server

Administrators of customer provided Linux® systems used for cross-development of
applications for a Convey hybrid-core server
®
Topics Covered
How to update Convey server specific software, including:

Convey OS and patches

Convey server firmware
How to install and update other Convey software on a cross-development system or a Convey
server, including the

Convey compilers, assembler, linker, and system libraries

Convey provided coprocessor personalities

Convey documentation
How to reboot a Convey server
How to check for updates for Convey‟s software packages
Personality Management
Recovering a Convey Server using the recovery DVDs
Customer Support Procedures
Licensing
1.3
Other Suggested Documents

Convey HC1 Service Guide – Describes how to install, upgrade, and troubleshoot the
HC1 hardware
Convey System Administration Guide v1.6
6
2 Convey’s Support Website
Convey provides a technical support website for customers: http://www.conveysupport.com.
This website contains a variety of useful information for developers, system administrators,
installers, etc.
A Convey provided web access user name and password are required to access the site,
although anyone with such a username/password can create new web access usernames, by
visiting http://www.conveysupport.com/protected/webaccess.html.
The most recent versions of all Convey documentation is provided, as well as older versions of
most documents.
Support bulletins, FAQs, a bug reporting/tracking system, software repositories, and other
resources are available.
All users of Convey servers or development software tools should have access to the Convey
support website.
Convey System Administration Guide v1.6
7
3 Installing Convey Software
In general, Convey recommends using the Convey provided software repositories for most
Convey software installations. These repositories are accessed using the standard Linux
software installation tools (i.e. yum, zypper, apt-get), and require the system to be connected to
the internet. Users with a large number of Convey servers or cross-development systems may
wish to create a local copy of the Convey software repositories, or download the individual
packages and install them with the low level package installer provided by their Linux distribution.
See Installing Convey Software without an Internet Connection for more details.
Some Convey software products require a license to be installed in order to use that product. As
of the April 2010 release, all Convey software products may be installed on a compatible system
before the license is installed. Convey servers are delivered with appropriate licenses already
installed, based on the sales order for the server. Licenses for cross-development systems must
be installed by a system administrator. Typically, Convey will email the contact person identified
on a sales order for a cross-development system with instructions for obtaining the appropriate
license(s).
3.1
Internet connection
Most of the directions below assume that a working internet connection exists. If a proxy server is
required, the appropriate proxy configuration should be updated for the software installation tool
of choice1. In addition, the wget command requires that the system‟s network settings correctly
specify any required proxy for http access to conveysupport.com.
If the system does not have a working internet connection, or you wish to install the Convey
software from a Convey Software DVD, see the section titled Installing Convey Software
without an Internet Connection.
3.2
Convey Software Repositories for Cross-Development Systems
On user provided cross development systems, the Convey software repository description files
must be installed before other Convey software can be installed from those repositories. On
Convey servers, the software repository description files are already installed.
3.2.1 Where to install the Convey .repo/.list files
On rpm based Linux distributions (Redhat, SuSE, Fedora, CentOS, Convey‟s OS),

Repository description filenames have a .repo suffix

All repository description files are located in either
/etc/yum.repos.d/ (for Redhat, CentOS, Convey‟s OS, Fedora, and older Suse/OpenSUSE
systems), or
/etc/zypp/repos.d/ (for newer OpenSUSE systems).

The Convey .repo files should be installed in whichever directory exists and already
contains other .repo files.
On Debian based Linux distributions (including Debian and Ubuntu),
1
For Convey Servers, see /etc/yum.conf
Convey System Administration Guide v1.6
8

Repository description filenames have a .list suffix

The repository description files are located in /etc/apt/sources.list.d/, and the Convey
.list files should be installed there. This directory may be empty before installing the
Convey repository description files.
3.2.2 Downloading and installing the Convey repository description files
Once you know which directory to install Convey‟s repository description files in, you may
download them from the Convey Support website by:

Logging into the system you want to install Convey‟s software products on, as the root
user

cd … (where … is the directory that contains all of your system‟s repository description
files, as described above)

wget http://conveysupport.com/repofiles/hc1-1/filelist

wget -i filelist

On an RPM based system, delete the unneeded .list files just downloaded, and on a
Debian based system, delete the unneeded .repo files just downloaded

See the wget man page for options required when an http proxy server must be used

If the wget utility is not already installed on your system, most Linux distributions provide it
in the vendor‟s software repository.
Alternatively, you can click on each repository description file in your browser, and copy/paste the
contents of the file to the same name file on your Linux system.
After installing the Convey repository description files, each of those files must be edited, and the
strings “username” and “password” must be replaced with the username and password provided for
your use when any Convey software product was purchased. E-mail
[email protected] for the name of the contact at your site that can provide the
appropriate username and password.
If your site is using a local copy of the Convey repositories, it may not be password protected. In
this case, you should delete the string “username:password@” from each repository description file.
3.3
Convey Server Software
If you are installing Convey‟s cross-development tools on a non-Convey system, please skip
over this Convey Server Software information.
Convey servers come with the following items pre-installed (package names are in this font):

Many CentOS packages

Convey OS packages (Convey enhancements to CentOS)
convey-kernel (or convey-kernel_beta), convey-platform_utils, conveyplatform_base

Convey Firmware package
convey-mp_fw

Convey Coprocessor recipe package(s)
convey-recipe-release

Convey coprocessor base image package
convey-aebase-image
Convey System Administration Guide v1.6
9

Convey Personality packages(s) with an Application Engine FPGA bit image
(these are optional licensed products)
convey-aesp (where “sp” can be any Convey provided personality name)
as well as many of the Convey Software packages described later, that can be installed on both
Convey servers and non-Convey systems running a compatible Linux distribution.
Additional licensed packages may be installed with the standard yum package manager, from the
repositories on Convey‟s support website, or from a Convey Software DVD.
3.3.1 Convey O.S. Installation
A Convey server must have the Convey provided Linux® distribution installed. No other Linux®
distributions may be installed on a Convey server. The Convey provided Linux distribution is preinstalled on all Convey servers. This distribution (and various updates) is also available on
Convey‟s support website, http://conveysupport.com.
Convey does not recommend installing Convey‟s Linux distribution on non-Convey systems.
Since Convey‟s Linux distribution is based on CentOS 5.3, CentOS may be installed on crossdevelopment systems if you want the most similar Linux distribution on those systems.
3.3.2 Installing Convey O.S. from a DVD
If the Convey server includes the optional DVD drive, Convey‟s O.S. may be installed using a
Convey supplied recovery DVD. You should never install Convey‟s O.S. from a DVD unless
instructed to do so by the Convey Support group (typically required only when replacing the main
disk in a Convey server). Please contact Convey if you believe you need to reinstall the O.S. on
a Convey server.
3.3.3 Convey Firmware Installation
A Convey server, in addition to the Convey provided Linux® distribution, requires the Convey
firmware package (convey-mp_fw) to be installed. This package is pre-installed on all Convey
servers. The Convey firmware package should not be installed on non-Convey systems.
3.3.4 Convey Coprocessor Recipe Packages
Recipe packages contain (or indirectly reference) “bit image files” that are loaded into some of the
FPGAs that are part of the Convey coprocessor. A recipe is a Convey tested set of these images
that are known to work well together. Typically, only the “released” recipe should be activated on
a Convey server unless a Convey support representative recommends installing another recipe.

The “released” recipe is always in a package named convey-recipe-release.

Installing or updating a receipe does not activate that recipe. Any installed recipe may be
activated with the pickRecipe or activaterecipe commands in /opt/convey/sbin.
Recipe packages should not be installed on non-Convey systems.
Note that when a recipe package is installed, it copies the relevant files to
/opt/convey/fpgaimages, and then removes itself. The relevant files will still be present on the
Convey server. This automatic removal allows older recipes to be installed after a newer recipe
has been installed.
3.3.5 Updating Convey OS, firmware, and Recipe Packages
Updates to all Convey server software packages may be performed with the yum tool, if either
internet access or a locally mounted Convey DVD repository is available.
IMPORTANT NOTE: After updating or installing any Convey OS, firmware, or recipe packages,
the host system needs to be shutdown for at least two minutes, and then powered up again. Any
package that requires a shutdown will indicate so during installation.
Convey System Administration Guide v1.6
10
For example, to update every Convey package installed on a system, logon the system as root,
and enter the command:
yum
update
convey-\*
and answer “yes” when prompted.
Occasionally the yum database becomes out-of-date or corrupted, and yum won‟t be able to find or
update packages. If you see unexpected error messages while using yum, you can purge the yum
database with the command:
yum
clean
all
New packages, such as new recipe packages, or newly purchased personality packages, may be
installed with the command:
yum
install
convey-xxx
where convey-xxx is the package name to be installed.
3.3.6 Installing other Convey Software products on a Convey Server
See the very next section, Installing Convey Software on Cross-Development Systems.
3.4
Installing Convey Software on Cross-Development Systems
Other Convey Software products, including Convey‟s software development products, may be
installed on a Convey server or on any compatible user provided Linux® system. When the
Convey compilers, assembler, linker, personality tools, and, optionally, the personality
development kit are installed on a non-Convey system, they provide a cross-development and
debug environment that permits developing and debugging code for the Convey server and
coprocessor, using a software simulator. See the Convey Programmers Guide for more
information.
Any other Convey products/packages that were purchased, such as the Convey Math Libraries,
the Personality Development Kit, or a Convey Application Engine fpga image (for a particular
personality), can be individually installed.
If you are installing or updatingConvey software products on a Convey server, you may skip the
®
section titled Compatible Linux Distributions below.
The directions below assume that an appropriate Linux® distribution is already installed on an
x86-64 system. If a proxy server is required, the appropriate proxy configuration should be
updated for the software installation tool of choice2, or you may use the manual installation
instructions below. See the section titled Installing Convey Software without an Internet
Connection for systems that do not have a working internet connection.
3.4.1 Compatible Linux® Distributions
The Convey software development products were developed and have been extensively tested
on the 5.1, 5.2 and 5.3 releases of the CentOS and Red Hat 64-bit distributions. If you have no
preference for any particular Linux distribution for your cross-development systems, Convey
recommends you install either the CentOS or Red Hat distribution.
The Convey PDK and other Convey software development products have been installed and
verified to work on an x86-64 system running the following 64-bit Linux® distributions:
Red Hat®
2
SUSE™
Fedora™ Core
OpenSUSE™
For Convey Servers, see /etc/yum.conf
Convey System Administration Guide v1.6
11
Desktop 5.1
Desktop 10
8 and 9
10.3 and 11.0
CentOS 5.2/5.3
Ubuntu® 8.0.4
Red Hat
Enterprise
Server 5.1-5.3
Debian
Different minor releases of these distributions are also likely to work. Newer major releases of
these distributions are likely to work if no substantial incompatibilities with respect to libc, python®,
and perl™ are introduced by the new release.
In addition to the base distribution, the “Software Development” super-package was typically
installed (x86-64 versions of gcc and g++ are required). Most live distributions (Linux distributions
that boot off of a DVD or CD but are not installed on a hard disk) are likely to require additional
packages that most DVD based distributions include automatically.
Other packages that are required, but typically included by default, are python®, and perl™. yum,
zypper, or apt-get is required if you want to install and update software using the Convey provided
repositories.
Other 64 bit Linux® distributions that have gcc 3.2 (or newer) and a corresponding g++ compiler,
GNU™ libc 2.5, a recent version of zypper, rpm or apt-get, and python 2.4 installed are likely to
work well, but have not been tested by Convey Computers. Most up-to-date Linux® distributions
with a 2.6.xx kernel will meet the above criteria, after any necessary optional packages are
installed.
Convey software development products are available over the internet, as RPM™ and Debian®
packages. These packages may be downloaded individually, or installed using the provided yum
and Debian repositories. These packages are also available on a CD by request (email your
request to [email protected]).
3.4.2 SELinux Considerations
If SELinux is enabled on a cross-development system, please see the SELinux FAQ at
www.conveysupport.com/protected/faqs for more information about configuring SELinux
(required in order for the Convey compilers to operate correctly).
3.4.3 Available Convey Software Development Packages
Users of Debian based distributions should note that any underscores in the package names
below will be replaced with dashes in the Debian package name (i.e. convey-sp_per -> conveysp-per).
Note that the full filename for a given package includes version and architecture information. The
rpm packages have an architecture specification of x86_64, while the debian packages have an
architecture specification of amd64. These two architecture specifications are equivalent.
Convey software packages should only be installed on systems with x86-64 compatible
processors.

The convey-1st package provides a few administrative scripts and adds /opt/convey/bin to
everyone‟s PATH

The convey-per-tools package provides the Convey personality database and some
personality management tools

Packages containing Convey compilers, support libraries, debugger, … (install all of these
packages together)
Convey System Administration Guide v1.6
12
o


convey-utils64-opensource
o
convey-platform_libs
o
convey-dp_per
o
convey-fap_per
o
convey-none_per
o
convey-pdk_per
(only for use with the convey-pdk package)
Convey Personality Development Kit for developing a custom personality
convey-pdk
Convey math libraries
convey-cml
OpenMPI
convey-openmpi
InfiniBand Drivers
o

o
convey-sp_per
o

convey-utils64
o
o

o
Convey personality packages for software development
o

convey-open64 (also installs the following)
convey-ofed-basic, convey-ofed-hpc, convey-ofed-all (only install one of these)
Documentation not included in another package
o
convey-manuals
3.4.4 Identifying the Appropriate Installation Tool
One of the following installation tools (or a compatible equivalent) is required to utilize the Convey
repositories. If none of these tools are installed, either install the appropriate tool or skip to the
section titled Manual Installation.

yum

zypper

apt-get
On all Redhat, Fedora, SUSE, and older openSUSE distributions, yum is the tool of choice. It
may be necessary to install yum on SUSE and openSUSE distributions (available on the
openSUSE installation media and from the openSUSE repositorty).
On openSUSE 10.3 and 11.x distributions, zypper is the tool of choice and should be installed by
default.
On Debian distributions (including Ubuntu), apt-get is the tool of choice.
On all other distributions, check to see if one of these tools is available, or can be installed from
the repositories provided by that distribution. If none of these tools can be used, skip to the
section titled Manual Installation.
Once you know which installation tool is available, follow the repository setup directions for that
tool below.
Convey System Administration Guide v1.6
13
3.4.5 Installing Convey Software Packages

Log in as root

The example commands below, for installing a package, use the yum command.
Substitute zypper or apt-get if appropriate for your Linux distribution. Remember to
change “_”s to “-“s in package names for Debian based distributions.

Install the convey-1st package first
yum install convey-1st

Install any Convey provided licenses as instructed in the accompanying email. Email
[email protected] for your site‟s contact person who can provide the
appropriate licenses and instructions.

Install the Convey personality tools package
yum install convey-per-tools

Install the Convey compilers, debugger, and other software development tools (install all 4
packages together)
yum install convey-open64

Install any of the personality packages you need
yum install convey-sp_per
yum install convey-none_per
…

Install any licensed packages you have a valid license for, such as
yum install convey-cml
yum install convey-pdk
yum install convey-sp_ae

If you would prefer /opt/convey/bin to appear before /usr/bin in $PATH (and $path) on crossdevelopment systems, execute (as root) the script /opt/convey/sbin/preferoptconveybin, log
off your terminal, ssh, or telnet session and start a new terminal session. This script will
automatically be executed when certain Convey server specific software packages are
installed on a Convey server. The effect of this script can be undone by editing the files
/etc/profiles.d/convey.sh and /etc/profiles.d/convey.csh, and moving /opt/convey/bin to
the end of the path.
Convey System Administration Guide v1.6
14
4 Updating Convey Software Packages
4.1
Updating Convey Sofware on a Convey Server
On a Convey server, Convey provides a script to update most Convey SW packages (except
those that are customer specific/proprietary), and ensure that compatible versions of FPGA
recipes, AE personality firmware, management processor firmware, and other Convey software
are installed, and the proper recipe is activated. This script may be used by:

Log in as root

yum update convey-support-tools
if this command complains that there is "No Match for argument", install the package with:
yum install convey-support-tools

/opt/convey/support/DoWebUpdate
This script updates all installed Convey provided software on your Convey Server,
including various firmware packages, the firmware recipe, coprocessor personalities,
development tools, the Convey OS and associated CentOS packages. This script ensures
that the various firmware, recipe, operating system, and other packages are compatibile
and up-to-date.
4.2
Updating Convey Software on non-Convey systems
All previously installed Convey software development packages and Convey server specific
packages may be upgraded to the newest released version with the usual Linux update
mechanism for your distribution. For Convey OS, this is:
yum
update
convey-\*
Note that yum‟s metadata cached on your system is typically reused for at least an hour, and that
metadata may need to be reloaded from the Convey repositories if you are attempting to use a
repository that has just been updated. For Convey servers, the command
yum
clean
metadata
will flush the existing metadata, and the next yum command will reload metadata from all the
repositories.
Also note that the database of installed software on most Linux distributions does not care where
a package has been installed. Therefore, if you install a newer version of a package in a location
other than the default, you may need to remove that version of the package before you can install
it in the default location.
On Debian based distributions, the apt-get command is used to update Convey packages.
apt-get
update
apt-get
upgrade
(flushs old metadata)
convey-\*
On systems where zypper is available, packages may be updated with:
zypper
update
--type
package
convey-\*
Zypper may report multiple available versions for some packages. Please choose the newest
(bigger version number) version of each package.
Convey System Administration Guide v1.6
15
4.3
Updating Convey Software Packages using a GUI
After the Convey repositories have been setup, the standard installation tools provided with a
Linux distribution, including graphical user interfaces to those tools (i.e. YaST, aptitude, Synaptics
Package Manager), may be used to install, upgrade, and remove Convey supplied packages.
Almost all Convey supplied packages have names that start with „convey-„, and are provided in
both .rpm and .deb versions in corresponding yum and debian repositories. Customer proprietary
packages developed by Convey typically do not have “convey-“ as part of the package name.
Note that the debian package names have dashes („-„) in their names wherever the rpm package
names have underscores. This difference in package names is an artifact of the tool used to
automatically convert rpm packages to debian packages.
4.4
Installing older versions of Convey Software Packages
Some of the native Linux installation tools support installing older versions of a package.
First, remove any installed versions of the package (except for kernel and AE personality image
packages). Then install the package specifying the desired version.
Installation
Tool
Sample Syntax
zypper
zypper
yum
yum install convey-1st-0.9.0-0
apt-get
apt-get install convey-1st=0.9.0
install
convey-1st=0.9.0
Note that the convey-kernel package and the personality packages with application engine images
(i.e. convey-sp_ae) may (and should) have multiple versions installed concurrently. Do not remove
any older (or newer) versions of these packages when installing a different version.
Note that some personality packages (convey-…_per) generally include all of the older versions of
that particular personality.
4.5
Version Numbers and Package Filenames
Throughout the discussion of package installation, package version numbers are depicted as
w.x.y-z, always as part of a package filename, i.e. convey-1st-w.x.y-z.x86_64.rpm. The actual
packages on the website and installed on a system will typically have numeric version numbers,
such as 1.1.1-1 or 1.2.1-2. w is the major version number, x is the minor version number, and y
is the very minor version number. z is the so-called release number, and in Convey packages is
typically an internal version number, and may not be strictly numeric.
The major version number (w) is changed whenever a major new release of a product occurs.
The minor version number (x) is changed whenever minor functionality changes or some
significant bug fixes are being released. The 3 rd version number (y) is changed when minor
changes are being released.
4.6
Manual Installation
These instructions assume either the rpm or dpkg command is available. On some distributions,
another installation program may be appropriate.
You will need the Convey assigned username and password for your site in order to access the
packages on the Convey support website. No password is needed to explore the Convey
Software DVD. In the URLs below, “release” may be replaced with one of the other release
Convey System Administration Guide v1.6
16
stream names currently on the Convey support website, if you have an appropriate password for
that release stream (i.e. beta).
All Convey created packages have a filename that starts with “convey-“.
Review the list of available packages in the following directories:
http://conveysupport.com/protected/install/hc1-1/convey_sdk/x86_64/RPMS/
http:// conveysupport.com/protected/install/ hc1-1/convey_sdk/updates/x86_64/RPMS/
http:// conveysupport.com/protected/install/ hc1-1/convey_platform_sdk/x86_64/RPMS/
http:// conveysupport.com/protected/install/ hc1-1/convey_platform_sdk/updates/x86_64/RPMS/
and for Convey servers only, the following directories also:
http:// conveysupport.com/protected/install/ hc1-1/convey_platform/x86_64/RPMS/
http:// conveysupport.com/protected/install/ hc1-1/convey_platform/updates/x86_64/RPMS/
http:// conveysupport.com/protected/install/ hc1-1/convey_centos/...
(in the convey_centos directory, please select the CentOS release # that matches your
system)
Note that there may be multiple versions of the each package. Typically, you want to
download/install the newest version of each package (the one with the largest version number).
The Convey Software DVD provides a similar structure.
To download and install one or more packages from Convey‟s website:
Login as root (typically with the su command)

cd dir
where dir is the directory you want to store the packages

Check which Convey packages are already installed, and remove any you want to
upgrade or replace (except for kernel and application engine fpga image packages):


o
On rpm based systems, use
rpm –qa | grep convey
rpm –-erase package-name(s)
o
On Debian based systems, use
aptitude search convey-\*
apt-get remove package-name(s)
If aptitude is not available, use the graphical interface to the software installer
provided by your distribution to seach for packages with „convey‟ in their name
retrieve the packages you want to install
o
wget http://username:[email protected]/protected/...
replacing „username‟ and „password‟ with the Convey provided username and
password for web site access, or
o
copy the packages from the Convey Software DVD
Install each package
o
rpm
–ihv
o
dpkg
–i
convey-…-w.x.y-z.x86_64.rpm, or
convey-…-w.x.y-z_amd64.deb (Debian based distributions)
The above steps can be repeated at any time to update the installed Convey software with a
newer version of any package.
Convey System Administration Guide v1.6
17
4.7
Additional Installation Notes for Ubuntu/Debian and openSUSE
Some distributions, including Ubuntu 8.0.4, openSUSE 10.3, and openSUSE 11.0, have a
problem linking 32 bit C/C++ applications with the newer version of gcc and g++. In particular,
when compiling the examples in the pdkSample directory that is part of the PDK personality, the
linker will complain that it can‟t find the library stdc++. If you receive this error message, you
need to downgrade to an older version of gcc and g++.
4.7.1 How to downgrade gcc and g++
First, see what versions of gcc and g++ are already installed:
ls /usr/bin/gcc* /usr/bin/g++*
If version 3.4 of gcc and g++ is not listed, please install the 3.4 version of those compilers from
your Linux distributions CD or vendor repository.
After the 3.4 versions of gcc and g++ are available, execute the following commands logged in as
root:
rm
ln
ln
gcc g++
-s gcc-3.4
-s g++-3.4
gcc
g++
Any scripts or Makefiles that use „gcc‟ or „g++‟, without a version number, will now use the older
3.4 version of those products. The specific versions of gcc and g++
(i.e. gcc-4.1) can still be used directly, and the system can be restored to its original state by
linking gcc and g++ to the newest install version of those products.
4.8
Installing Convey Software without an Internet Connection
If a reliable internet connection is not available on your system, you may install the Convey
Software Products several different ways:

Install Convey Software products using a Convey provided Update DVD
o

Email or call Convey support, and request a Convey Update DVD, and follow the
directions that come with the update DVD, or
Make your own DVD or USB installation media:
o
On another computer that has internet access, browse to
http://conveysupport.com/protected/install/hc1-1, and for each directory of interest
therein:
o
Descend into directorties of interest, until you find the .rpm or .deb packages,
o
right-click on each package whose name ends with „.rpm‟ for Red Hat /
SUSE based distributions, or „.deb‟ for Debian/Ubuntu based distributions,
and pick Save Target As (in Internet Explorer) or Save Link As (in
Firefox/Netscape) to save each file to a local disk.
o
You may also want to save the repository database files (Packages.gz for the
debian repository, and the repodata directory for the yum repository) (you
must preserve the entire existing directory structure to create a usable
repository).
o
You may also use the wget command (with the –r option) to download an
entire repository directory structure, and then copy it to a USB memory stick,
a writable CD/DVD, or some other portable media.
Convey System Administration Guide v1.6
18
o
It may be easier to use a copy of the repositories if you create a separate CD
or USB memory stick for each repository of interest, where the repodata or
Packages.gz file is in the root directory of the CD or USB memory stick.
o
Do not copy the convey-platform repository for use on a non-Convey system.
The packages in this repository should only be installed on a Convey HybridCore server.
o
If the CD or USB memory stick includes the repository database files
(Packages.gz and/or the repodata directory) then the software installation
tools provided with your Linux distribution can probably treat the CD or USB
memory stick as a repository (possibly after enabling the CD/USB media as a
repository), and you may install the Convey packages like any other
package, using yum, zypper, or apt-get. You might need to create the
repository description files by hand, to point to the correct subdirectory on the
CD or USB memory stick.
o
On Ubuntu, a USB memory stick can be enabled as a Software Source by
adding a line such as
deb file:/media/UsbName /
to /etc/apt/sources.list, where „UsbName‟ is the name the USB memory stick
was mounted as in /media.
The Software Sources tools in Ubuntu distributions can also add a CD as a
Software source.
o
If you cannot use the CD or USB memory stick as a Linux repository:
o
Copy the packages (.repo or .deb files) from the Convey Software CD/DVD
(or other portable media containing the Convey packages) onto your Linux®
system (/tmp/rpms/ in the example below). Delete any Convey server
specific packages if you are installing on a non-Convey server, especially
any Linux kernel packages.
o
Login as root (typically with the su command), and enter the following
commands for Red Hat or SUSE based distributions:
cd /tmp/rpms
rpm –ihv convey*
or these commands for Debian based distributions, such as Ubuntu:
cd /tmp/rpms
dkpg --install *.deb
o
The above steps should be repeated whenever a new release of any Convey
Software Product is required, except the command should be
rpm –uhv package-file-name
or
dpkg –list | grep convey # lists installed convey packages by
name
dpkg --remove package-name
# not the package-file-name
dkpg --install package-file-name
o
Each package-file-name to be installed must be spelled out completely,
including the .rpm or .deb suffix, but a package-name does not include the
version number, the architecture, or the .rpm/.deb suffix
Convey System Administration Guide v1.6
19
5
Booting and System Management
5.1
Overview
This chapter describes the process of booting the system, initialization of the Convey
coprocessor, and management of the system.
5.2
Powering Up the Convey Server
When the server is first plugged in, the blue Chassis Identity light will begin blinking. It will blink
for about one minute. Once it has stopped blinking, you can push the front panel button to power
up the Intel server. Do not power up the Intel server until the blue Chassis Identity light has
stopped blinking.
5.3
First Power-Up and Boot
The Convey System comes pre-installed with Convey Linux, Convey software products, and
basic system configuration.
5.3.1 Intel Remote Management Module
If your system is configured with the Intel Remote Management Module (RMM), then this is the
recommended interface (console, keyboard, and mouse) for initial system power-on and
configuration. The MAC address for the RMM will be on a label on the outside of the server.
The Intel website, intel.com, provides manuals and information about the RMM and configuration
utilities. The psetup(1) utility that can be downloaded from the Intel website is useful in attaching
to the RMM and assisting in the initial configuration. The psetup(1) utility identifies all RMMs on
the local subnet and presents them based on their MAC address.
A link to the psetup utility is available on the Convey Support website, conveysupport.com. Click
on the “Convey Hardware Help” tab on the top menu bar.
The RMM is initially configured with DHCP and the default user/password is „admin/password‟.
5.3.2 First Boot
The system is configured such that on the first boot at your site, the system enters a graphical
configuration menu. This allows initial configuration of several aspects of the system. Some of
the configuration tasks available are:

Change the password for the root user.

Add an initial user to the system (not root) and/or setup NIS.

Setup firewall rules.

Configure the network interface.

Configure date/time, time zone, and Network Time Protocol.

Install additional software packages via CDs.
If at some point later you would like to invoke this graphical configuration again, login as root and
execute the following commands:
chkconfig firstboot on
touch /etc/reconfigSys
Convey System Administration Guide v1.6
20
reboot
5.3.3 Manual Configuration
The following configuration updates are required at a minimum to place the system in general
operation and connectivity to your network if you are not using the FirstBoot reconfiguration
interface just described in the previous section. Login as root and perform the following actions.

Select a password for super-user and update using the passwd(1) utility.

Select a host-name for the machine and update the HOSTNAME variable in the file
/etc/sysconfig/network.

Select an IP-ADDRESS for the system and update the IPADDR variable in the file
/etc/sysconfig/network-scripts/ifcfg-eth0.

Add a host-name and IP address entry to the file /etc/hosts.

If you are enabling a serial connection to the machine, add the following boot options to
the kernel command line in the file /boot/grub/grub.conf.
o

5.4
earlyprintk=serial,ttyS1,115200 console=ttyS1,115200
Reboot the system.
Boot Options
The Linux operating system is rich with boot options to control many aspects of system
configuration. The goal for the Convey system is to provide as seamless as possible system
initialization. Toward that end, a properly installed Convey system is designed with a set of
defaults that allow the system to be fully configured and functional without any additional boot
options. The system defaults are as follows:

The Convey coprocessor is fully enabled.

To the extent possible based on the installed physical memory on the Host server, all
memory on the coprocessor is configured and accessible.

The memory interleave for the coprocessor defaults to 31-31 interleaving.

The page size for the coprocessor is set to the maximum available based on board
configuration.

A 2 node NUMA configuration is established for memory domains by the kernel. Node 0
is the Host server and Node 1 is the Convey coprocessor.

Address extension mode is configured as fully-mapped.
The Convey boot options follow similar syntax to most standard Linux options. All Convey boot
options are preceded by the keyword convey followed by a comma separated list (no spaces) of
keyword-value pairs. Note, keywords do not have to have an associated value. All keywords and
values are lowercase. For example:
convey=keyword1=value1,keyword2,keyword3=value3
Multiple ‘convey‟ boot option tags are also permitted:
convey=keyword1=value1
convey=keyword2
convey=keyword3=value3
The following Convey boot options are provided to allow system administrators detailed control
over the coprocessor configuration.
Convey System Administration Guide v1.6
21
5.4.1 enable/disable
The enable/disable keywords control the discovery and initialization of the Convey
coprocessor. By default, the coprocessor is enabled. The disable option results in the
coprocessor not being initialized or available by the system. The memory on the coprocessor
board is not configured. All other Convey boot options are ignored.
5.4.2 caemode
The caemode keyword controls the behavior of the Convey Address Extension mode. There are
currently 2 options available: fully-mapped (default) and dynamic-mapped. The caeinfo(1)
command provides detailed information on the address extension configuration. Examples are:
convey=caemode=fully-mapped
convey=caemode=dynamic-mapped
5.4.3 memory
The memory keyword allows the administrator to restrict the amount of memory configured on the
coprocessor. By default, the system will configure all coprocessor memory up to limits that may
be imposed by the physical memory configuration of the Host server. The option takes a single
value that is the amount of memory to configure for the coprocessor. This can be expressed as a
decimal value, hex value, or shorthand memory-suffix notation (k (kilobytes) = 1024; m
(megabytes) = 1048576; g (gigabytes) = 1073741824). Examples are:
convey=memory=0x200000000
convey=memory=2g
convey=memory=1024m
convey=memory=0
The last example is a valid technique to configure the coprocessor with no local memory.
5.4.4 window
The window keyword allows the administrator to set the amount of Host memory allocated to the
memory window. The window size is constrained to a minimum of 4G and a maximum equal to
the size of coprocessor memory. The window size must allow a minimum of 4G of free memory
to be available on the Host when the configuration is complete. Currently, the window size must
be a multiple of 1G bytes. The window size can be expressed as a decimal value, hex value, or
memory-suffix notation. Examples are:
convey=window=0x200000000
convey=window=8g
convey=window=8192m
5.4.5 pagesize
The pagesize option allows the administrator to override the architectural maximum page size of
the coprocessor. This value will be ignored if it exceeds the architectural maximum page size for
the installed board. Once the system is booted, the configured page size can be queried using
the cnyinfo(1) command. The page size value can be expressed as a decimal value, hex
value, or memory-suffix notation. Examples are:
convey=pagesize=256k
convey=pagesize=1m
convey=pagesize=4096
Convey System Administration Guide v1.6
22
5.4.6 interleave
The interleave option allows the administrator to configure specific interleave schemes for the
coprocessor. The coprocessor interleave can only be set at boot time and is configured during
discovery and initialization. There are currently 2 types of interleave supported: binary and
3131. Examples are:
convey=interleave=binary
convey=interleave=3131
The coprocessor interleave can be queried using cnyinfo(1).
5.5
Coprocessor Modules
The initialization and configuration of the Convey coprocessor as a seamless part of the system is
accomplished by two loadable Linux system modules. The CNYSYS module and the CNYMP module
are dynamically loaded at boot time and perform all discovery and initialization functions. The
command lsmod(1) displays all currently loaded modules in the system including the Convey
modules. The following sections describe in detail the function of these modules.
5.5.1 CNYMP Module
The first Convey module to load during boot is CNYMP. This module provides out-of-band
communication with the management processor (MP) on the Convey coprocessor board. This
communication is provided over a dedicated PCIE link with the MP. If the CNYMP module fails to
load and initialize, the system will boot but the coprocessor will be inaccessible. The CNYMP
module provides the following general features:

Discovery and initialization of the MP PCIE link

General I/O communication path with the MP

Supports remote login for field personnel

Transport path for dynamic personality loads and coprocessor firmware

Error handling and logging between the Host server and the MP
5.5.2 CNYSYS Module
The CNYSYS module loads immediately following the load of the CNYMP module. The CNYSYS
module depends on the CNYMP module for communication with the coprocessor management
processor (MP). If the CNYMP module fails to initialize then the CNYSYS module will also fail. Failure
of the CNYSYS module to initialize correctly results in the system booting without coprocessor
access. The main functions of this module are:

Discovery and initialization of the Convey coprocessor

Setting the coprocessor memory interleave

Configuration of coprocessor memory into the system

Configuration of the coprocessor interrupt path

Management of the personality cache on the MP

Handling coprocessor faults and traps

Coprocessor allocation policies (attach/detach)

Debug control (GDB save/restore context)
Convey System Administration Guide v1.6
23
As just mentioned, one of the key functions of CNYSYS is configuration of coprocessor memory into
system. Coprocessor memory is configured into Linux and is accessible by all the standard
memory management policies. For example, x-86 applications requiring large amounts of
memory will spill over into coprocessor memory based on the NUMA system policies configured
and not anything unique to the Convey system. The memory on the coprocessor board will be
part of the memory statistics in /proc/meminfo.
From an application perspective, the CNYSYS module provides the allocation polices for acquiring
access to the coprocessor. The Convey Programmer’s Guide provides details of these policies
and programming models. But it is important to note that the Convey coprocessor is modeled as
an extension to the x-86 instruction set and not a separate stand-alone processor. For that
reason, the coprocessor will not show up in the file /proc/cpuinfo as an independent processor.
5.6
Kernel Configuration
This section describes the basic kernel configuration for the Convey Linux operating system. The
Convey Linux kernel was developed based on the Centos 5.4 kernel source base. Centos5.4 is a
2.6.18 released Linux kernel with extensions and fixes.
The overriding goal for the Convey Linux development was maintaining complete binary
compatibility. This extends to middle-ware, system call interface, tools, drivers, and loadable
modules. There are no changes to any system libraries or header files.
The kernel configuration has been tailored for the Intel Shoffner platform and its devices. There
are several configuration selections made specifically for the Convey platform and its extensions.
Some of the key kernel configuration options are:

Kernel crashdump support

64-bit Intel Processor (32-bit kernel is not supported)

No forced preemption (server platforms)

NUMA (multi-node Non-Uniform-Memory-Access; 8 node maximum)

Sparse Memory design

Maximum CPUs (8)

Timer Frequency (1000)

Kernel Profiling
The Convey Linux kernel has been fully qualified with the configuration described. Changes to
this configuration, addition of drivers and/or modules, or specification of complex boot options
may result in conflicts within the system and untested code paths.
5.7
Convey System Services
A service, as defined under Linux, is a shell script in the /etc/init.d directory. The Convey
platform supports a new service: convey. The convey service supports the full set of service
commands (start, stop, status, restart, reload). In general, the convey service is responsible for
the following events:

Setting of Convey specific sysctl(1) variables

Setting of the /proc/sys/kernel/printk level

Load and initialization of the CNYMP module

Load and initialization of the CNYSYS module
Convey System Administration Guide v1.6
24

Configuration of the management processor
The utility, chkconfig(1), provides an easy to use interface to query the status of a service and
the run levels it is enabled for. For example, the following command,
chkconfig –list convey
will display the following line indicating the run levels that will execute this service.
convey
0:off
1:off
2:on
3:on
4:on
5:on
6:off
This indicates that the convey service is enabled for run levels 2, 3, 4, and 5. See the man page
for chkconfig(1) for a description of managing these run levels.
The service(8) utility provides an interface to execute service commands (stop, start, etc.)
against a specific service. The most frequent use of this command is to query the loaded state of
the Convey modules.
service convey status
5.7.1 Convey Configuration Files
This section describes the Convey specific configuration files associated with the services just
described. The Convey configuration files are maintained in the directory /etc/convey on Convey
installed systems. Each configuration file has a Convey installed version and may have a local
site-specific version. The local site-specific configuration files can override or extend Convey
configuration options.
The current configuration file set is:
5.8
modules.conf
Defines the device names, permissions, owner, and group for the CNYSYS
and CNYMP modules.
modules_local.conf
Local site definitions that override the installed modules.conf.
mp.conf
Configuration parameters for the Convey management processor. This
includes hostname, Ethernet settings, console settings, and logging
configuration.
mp_local.conf
Local site definitions for management processor. This will typically include
the IP/Mask for the management processor Ethernet.
sysctl.conf
The format of this file is identical to /etc/sysctl.conf and contains Convey
specific sysctl variable settings.
sysctl_local.conf
Local site sysctl variable settings.
yum.conf
This file is referenced from /etc/yum.conf and contains Convey specific
yum(8) configurations. No local version of this file is defined.
Coprocessor Sanity Test Program
The program /opt/convey/sbin/cnydiag.aebase exercises basic functionality of the Convey
coprocessor, including memory access, coprocessor scalar instructions, internal datapaths in the
coprocessor, and an application engine vector instruction. The coprocessor must be unattached,
Convey System Administration Guide v1.6
25
and booted with “binary” memory interleave. If the server is booted with 31-31 memory interleave
enabled, the vector test will fail, but the earlier tests should all pass.
5.9
When to Shutdown the Convey Server
If the coprocessor is attached by a process but appears to be hung (the application won‟t
abort/exit), or when the Convey coprocessor is not attached (see the man page for cnyps) and
you cannot run the sanity test program successfully, please follow these steps:

Log onto the Convey server as root, and run
/sbin/shutdown –r 1 rebooting to reset the coprocessor

After the host finishes rebooting, try running the sanity test program again. If it still fails,
please contact Convey Customer Service.
5.10 Activating a Coprocessor Recipe
Installing a recipe package does not cause the coprocessor to use that recipe.
A Convey server can have many recipes installed, but only one is “active” at a time. To activate a
particular recipe, Convey provides two tools:

/opt/convey/sbin/pickRecipe
o

This script lists the available (installed) recipes, and lets the user select one to be
activated.
/opt/convey/sbin/activerecipe recipe-filename
where recipe-filename is one of the “recipe...” files in /opt/convey/fpgaimages.
Important Note: After activating a recipe, you must shutdown the host system, wait for for two or
more minutes, and then power up the host.
Convey System Administration Guide v1.6
26
6 Convey System Memory Management
6.1
Overview
This chapter provides high level details about the Linux operating system physical and virtual
memory management for the Convey Coprocessor. The goal behind these changes is to provide
high memory bandwidth and performance for the coprocessor. There are four concepts that form
the basis of Convey memory system that will be discussed.

NUMA Memory organization

Memory Interleaving

Variable Page Size Support

Inter-Node Memory Migration
The Convey Reference Manual will be referred to frequently for general description of the
system organization and the hardware architecture.
6.1.1 Intended Audience
This chapter is intended for system administrators and application designers. The goal is to
provide sufficient details about Convey memory management to enable development of high
performance applications. Administrators need to understand how the operating system
manages the Convey memory architecture and presents this to applications.
6.1.2 Terminology
The following terms are used to describe various components of the Convey system as it relates
to memory management.
6.2
Host memory
This refers to memory physically installed on the Intel x-86 Server.
Coprocessor
memory
This refers to memory physically installed on the Convey coprocessor.
Memory window
This refers to the physical memory on the Host server that is reserved/used
to enable coherent access to memory on the coprocessor.
NUMA Memory Configuration
Physical memory is populated on the commodity Intel motherboard and on the Convey
Coprocessor board thus forming a classic Non-Uniform-Memory-Architecture (NUMA). The
degradation in memory latency and bandwidth when accessing remote memory (non-local to the
processor) is significant and must be considered in any coprocessor application.
The Convey Linux kernel manages the physical memory of the system by using kernel nodes to
separately configure the physical memory associated with the Intel motherboard and the Convey
coprocessor. By convention, the coprocessor node will be the highest numbered node in the
system. For most system configurations, node 0 will be the Intel board and node 1 will be the
coprocessor board. This also implies the Linux kernel is configured with NUMA enabled and a
maximum node count of 8.
All available Linux techniques and tools are supported for managing a multi-node NUMA
configuration. The /sysfs file system describes each node and its attributes. For example, the
file /sys/devices/system/node/node1/meminfo provide all the memory statistics for the
Convey System Administration Guide v1.6
27
coprocessor node. The file numastat in the same directory provides statistics on page allocations
with respect to the NUMA configuration. The numastat(1) utility is available from the numactl
package. This outputs a nicely formatted display of the numastat information for all nodes in the
system. Finally, the file /proc/meminfo provides memory statistics for the entire system (both
nodes).
6.2.1 Physical Memory Configuration
The previous section discussed the memory configuration from a logical Linux perspective. The
physical implementation is different due to the architecture of the Convey coprocessor. Physical
memory installed on the coprocessor board is actually implemented as a cache of memory on the
Intel server board. This design is an artifact of the Intel Front Side Bus (FSB) and providing a
single cache-coherent address space for the system. The implications of this design are as
follows:
6.3

Because coprocessor memory is a cache, a portion of host memory (less than or equal to
the size of memory on the coprocessor board) is configured as coprocessor memory.

The Linux kernel will automatically reserve and configure host memory equal to the size
of coprocessor memory if available (default). No boot options are required.

The Linux kernel requires a minimum of 4 gigabytes of physical memory for the host
server. The remainder of physical memory may be configured as coprocessor memory
(memory window) based on coprocessor memory size.

If insufficient host memory is configured, the memory boot option is used, or the window
boot option is specified, the host memory configured for the coprocessor can be less than
the physical memory size installed on the coprocessor (even zero).
Address Extension Feature
The initial release of the Convey system only supported 16G of coprocessor memory and
required that 16G of Host memory be reserved for access to the coprocessor (one to one
mapping; i.e. fully-mapped). The Shoffner X-86 motherboard supports a maximum of 128G of
physical memory (16 x 8G DIMMS) that would have limited the maximum amount of memory
supported on the coprocessor given the previous architectural rules.
The Convey Address Extension feature was developed to enable support for maximum memory
configurations of the Host Intel server and the Convey coprocessor (128G Host memory; 128G
coprocessor memory) without requiring the one to one mapping of Host memory to coprocessor
memory. A region of Host memory (known as the memory window) is still reserved to enable
accesses to coprocessor memory but it is not required to be a one to one mapping. This new
configuration capability is known as dynamic-mapped.
The architecture now supports a hardware mapping table between the Host memory window and
coprocessor memory that enables mapping a region of the memory window onto any region of
coprocessor memory. Memory window regions can be remapped dynamically as the X-86
references different regions of coprocessor memory. The remapping process is managed by the
operating system in a manner similar to virtual memory page faults. The caeinfo(1) utility
provides detailed information on the hardware and software management of the memory window.
The kernel boot option, caemode, allows the system administrator to select between the 2
modes: fully-mapped, dynamic-mapped. The following sections describe each in detail.
6.3.1 Fully-mapped mode
Fully-mapped mode is the system default and provides a full mapping of the Host memory
window onto coprocessor memory. In this mode, each byte of coprocessor memory must be
mapped by memory on the Host. The hardware mapping table is initialized at boot time to a one
Convey System Administration Guide v1.6
28
to one mapping of the Host memory window onto coprocessor memory. The size of the memory
window is configured by the operating system at boot time based on the following constraints:

Boot option convey=memory=### sets an upper limit on the size of coprocessor
memory configured.

Boot option convey=window=### sets an upper limit on the size of the Host memory
window.

A minimum of 4G of Host memory is reserved for the system.

A successful configuration must result in 3-4G of Host memory being free and allocatable
for the kernel and applications.
The following example demonstrates the configuration rules for fully-mapped mode. A system
with 64G of Host memory and 128G of coprocessor memory without any boot options will
configure a memory window of approximately 58G and coprocessor memory of 58G. This results
in 70G of coprocessor memory not being configured. The 6G of Host physical memory remains
for the operating system, system data structures, and 3-4G of free memory (this assumes 2-3G
consumed by the operating system).
6.3.2 Dynamic-mapped mode
Dynamic-mapped mode must be specified via the boot option caemode. The size of the memory
window is configured at boot time based on the following constraints:

Boot option convey=window=### sets an upper limit on the size of the Host memory
window.

The Host memory window defaults to a size equal to 25% of the size of configured
coprocessor memory.

A minimum of 4G of Host memory is reserved for the system.

A successful configuration must result in 3-4G of Host memory being free and allocatable
for the kernel and applications.
Using the example from the prior section, a system with 64G of Host memory and 128G of
coprocessor memory would configure a memory window of size 32G. The full 128G of
coprocessor memory would be configured into the system and available. 32G of Host memory
remain for the operating system, system data structures, and applications.
Determining the size to configure the Host memory window is completely application specific. If
the application has limited X-86 access of its data in coprocessor memory, then smaller windows
should provide good performance. As the application increases its X-86 accesses or the
accesses are very random/distributed, memory window region swapping will occur. The size of
the memory window will affect the amount of swapping in this case.
6.4
Memory Interleaving
This section describes the supported memory interleaving schemes on the Convey coprocessor.
The coprocessor supports two types of interleaving; binary and 31-31 as described in XXX. The
coprocessor memory interleave is set at boot time and currently defaults to 31-31 interleave if no
boot option is specified. See the chapter 5.4.6 interleave for a detailed description of interleave
boot options.
The interleave schemes are independent of the type of DIMMs installed in the coprocessor
(standard DIMMs or scatter-gather DIMMs). The interleave scheme affects which memory
controllers, DIMMs, and banks are accessed based on memory address. The performance
obtained by the interleave scheme is a function of the memory access pattern of the application
and is not discussed in this section.
Convey System Administration Guide v1.6
29
The type of interleave configured is determined at boot time and cannot be reconfigured after the
coprocessor has been brought online. The kernel performs a coprocessor discovery and
configuration process during boot. The coprocessor attributes are logged into the file
/var/log/messages that includes the configured interleave type. The configured interleave can
also be determined using the cnyinfo(1) utility or displaying the file
/sys/devices/system/node/node1/interleave/summary (note, this assumes the coprocessor is
node 1).
6.4.1 Binary Interleave
The binary interleave scheme is designed to provide near peak performance for unity stride
applications. All available memory will be configured; there are no interleave holes in the physical
address space.
6.4.2 31-31 Interleave
The 31-31 interleave scheme was designed to meet the following system requirements:

Provide as high a bandwidth as possible for all memory access strides, with particular
focus on power of two strides.

Keep each memory line (64 bytes) on a single memory controller. This is required to
simplify the cache coherency protocol.

Maintain the interleave pattern across virtual memory page crossings. This helps large
strides where only a few accesses are to each page.

All virtual addresses must map to unique physical addresses.
The scheme uses a two level hierarchical interleave approach. The Convey memory system
using Scatter/Gather DIMMs has 1024 memory banks. The 1024 banks are divided into 32
groups of 32 banks each. The first interleave level selects one of 31 groups of banks. The
second interleave level selects one of 31 banks within a group. Note that one of the groups of the
32 groups of banks is not used. Similarly, one bank within each group of banks is not used. The
prime number (31) of banks and groups of banks is used to maximize the sustainable memory
bandwidth for as many different strides as possible. This skipping of banks results in a 6% loss of
available memory and a decrease in peak memory bandwidth of 6%. A coprocessor with 16
Gbytes of physical memory and 31-31 interleave configured, will have an available physical
memory of 15.04 Gbytes (a loss of 960 Mbytes).
6.4.3 31-31 Interleave Page Allocation
The 31-31 scheme is a 2 level interleave. The first level has a required mapping function and the
second level is a best effort. If a physical page of the required mapping cannot be found on the
coprocessor for a specific virtual address, then one of several actions may take place:

If the memory policy for the virtual memory region allows, the memory allocator will
attempt to allocate a page out of host memory (x86 Intel server board).

If host memory is also not available or the memory policy does not permit off node
allocation, then the process is pended and the paging daemon is woken to free additional
memory on the node.

Note, host memory does not have the mapping requirement for 31-31 interleave. The
mapping function only comes into play for allocation of coprocessor memory.
The cny3131(1) utility provides detailed information about allocation patterns, color utilization,
page order usage, etc. The „p-miss‟ column when displaying primary color statistics indicates
that a page of the correct mapping was not available. This would trigger one of the above
Convey System Administration Guide v1.6
30
actions. Allocation of a host memory page or starting of the paging daemon will result in
performance degradation for the coprocessor application.
6.4.4 31-31 Interleave Faults
The Linux system call interface and general programming model gives any application almost
complete control of its virtual address space. For example, the mmap(2) system call permits the
application to specify the exact virtual address at which to map a new object. This feature is used
heavily by the dynamic library loader but rarely by other applications. Use of this capability can
result in invalid physical-to-virtual mappings for a coprocessor application with 31-31 configured.
For this reason, it is strongly advised not to specify the mapping virtual address to mmap(2).
The Linux kernel makes an extreme best effort to avoid and prevent invalid 31-31 mappings. But
given the virtual address space control an application has, there is not a 100% guarantee. For
this reason, the coprocessor has a detection algorithm in the TLB walker that will detect an invalid
page mapping and generate a SIGBUS to the application. Detailed information is dumped to the
system console and system logs. This information should be captured and sent to Convey
Customer Support.
6.5
Variable Page Size Support
This section discusses the Linux operating system support for Convey variable page size support.
The goals for providing variable page size support are:

Reduce the number of TLB misses and page faults by the coprocessor

Increase the utilization of the coprocessor TLB cache

Allow the coprocessor TLB to cover all physical memory
The Convey Reference Manual (Chapter 4) describes the architectural details behind the
coprocessor variable page size support. The system supports page sizes 4K to 4M and every
power of 2 page size in between. The kernel aggressively allocates the largest page possible
during the following events:

Page faults (these can be copy-on-write)

File system reads

Reads from the swap area

Data migration
The allocation of contiguous memory (large pages) is applied only to processes that are currently
attached to the coprocessor. Host X-86 processes cannot benefit from the large page allocation
feature of the coprocessor.
The allocation of large pages is applied only to Convey attached processes and is independent of
the node the memory is allocated from. Because the coprocessor can access Host and
Coprocessor memory, large pages in either memory domain improves TLB performance. Note,
that due to the memory activity in the Host domain, the ability to allocate the maximum large page
is reduced.
The /proc pseudo-filesystem implements several files that provide detailed information about a
process. From a memory management point of view, the files maps, numa_maps, and smaps contain
data on the virtual address space of the process. The data is organized around each memory
mapped region with the address space. There are man pages for proc(5) and numa_maps(5) that
cover the format of this information. Because of the importance of page placement and page size
on application performance, a new /proc file has been added for Convey systems. The file pmaps
contains detailed data on physical memory placement and page size for the entire virtual address
Convey System Administration Guide v1.6
31
space. This is extremely useful when tuning coprocessor applications. See the man page
pmaps(5) for more detail.
6.5.1 Maximum Coprocessor Page Size
The current architecture of the coprocessor is designed to support a maximum page size of 4M.
The architecture data of the coprocessor is acquired by the kernel at discovery time during boot.
The maximum page size can also be controlled at boot time (boot option) or dynamically at runtime. The boot option, pagesize, defines the maximum page size to be used for the coprocessor
up to the architectural limit. Also, the maximum page size of the coprocessor can be queried and
set by the file /sys/devices/system/node/node1/pagesize (assuming node1 is the coprocessor).
The following command allows setting of the maximum page size to 16k bytes.
echo 16k > /sys/devices/system/node/node1/pagesize
This change takes effect immediately on the next memory allocation for an attached coprocessor
application. This can be useful when diagnosing system problems or performance anomalies.
6.5.2 Linux HugeTLB Feature
The HugeTLB feature in Linux enables use of X86 2Mbyte pages by applications and is fully
supported by the Convey Linux operating system. The coprocessor can access HugeTLB pages
mapped into an application. However, because this feature is more restrictive than the
coprocessor page size support and due to the complexity of maintaining 31-31 interleave
mappings, coprocessor memory cannot be reserved into the HugeTLB memory pool. In addition,
the HugeTLB feature does not currently have an API to specify the node from which to reserve
memory for the pool. HugeTLB memory will always be allocated out of Host memory.
Applications being ported to use the Convey coprocessor that depend on the HugeTLB feature
will execute correctly. The issue is one of performance. If the application port results in the
coprocessor heavily accessing virtual memory that maps HugeTLB pages, changes will need to
be made to allocate coprocessor memory or to be able to take advantage of page migration. This
is covered in detail in the Convey Programmers Guide.
6.6
Memory Migration
This section describes memory migration within the Convey system architecture. The Convey
system is configured as two NUMA nodes typically numbered 0 and 1. Memory migration allows
the moving of the physical location of pages between nodes while the process is running. The
virtual addresses the process sees do not change but the physical location of pages mapped to
those virtual addresses do. The goal of memory migration is to reduce the latency of memory
access by moving pages near to the processor accessing the memory. This is critical toward
achieving maximum performance of the application.
It is important to note that page migration is a best effort by the kernel. All references to a page
must be removable at the time of migrate or the page is dropped from the migration list.
Examples of this are pages that are locked for I/O, shared pages that still have references from
other processes, or dirty pages in the process of being cleaned. Other factors include lack of
physical memory on the target node of the migration.
The Convey Programmers‟ Reference Guide describes the library API available for data
management. The routine cny_migrate_data() provides a user interface for migrating virtual
regions within a process to/from the coprocessor. This is built on top of standard Linux system
calls and is provided as an easy to use interface that is cognizant of the Convey coprocessor
system architecture. The routine cny_memory_locale() assists in identifying the location of a
specified virtual region within the application. The information provided by cny_memory_locale()
may not be completely accurate if the memory has not been referenced following a migration.
When the routine cny_migrate_data() is applied to a memory region that is unpopulated (empty),
the pages are not faulted in by this call. Instead, the memory policy for the region is updated with
Convey System Administration Guide v1.6
32
the target node of the migration. Later, the pages will be faulted into memory of the target node
thus avoiding a memory to memory copy. However, the cny_memory_locale() will still return
empty until the pages are faulted into memory.
Convey System Administration Guide v1.6
33
7 Personality Management
7.1
Personality Signatures
The Convey Coprocessor supports multiple personalities, and multiple versions of each
personality. Although the user can exercise substantial control over which particular personality a
given program will use, several tools are available for a system/site manager to guide users
towards particular personalities and versions thereof. Note that there are multiple personality
upgrading steps, only two of which are described here. A full discussion of personality upgrading
is available in the Convey Programmers Guide.
A complete signature identifies a particular version of a coprocessor personality. Signatures are
composed of five numeric values, such as „2.1.1.1.0‟. The five fields (in order) are:

personality number

major version number

minor version number

hardware model number

canonical instruction set version number
The personality number indicates the type of instructions implemented in a particular personality,
such as single precision vector, double precision vector, etc.
The major and minor version numbers indicate various updates to the personality, including bug
fixes, performance improvements, and new instructions.
The hardware model number indicates which version of the Convey coprocessor this signature
supports. Signatures that are identical except for the hardware model numbers are compatible,
and an object file compiled for a particular signature can be executed using a compatible
signature.
The fifth numeric value, the canonical instruction set version number, is reserved for future use. If
specified as part of the signature, it must be zero.
7.2
Default Base Signature
After installing the convey-per-tools package for the first time (or anytime thereafter), the system
administrator may update the file
/opt/convey/personalities/DefaultBasePersonality
This file contains a single default base signature for the system. This default base signature is
used to fill in some missing parts of a user specified signature when compiling or assembling
source code. If the user specifies a zero (or does not specify a value) for the personality number
or the hardware model number, the corresponding value from the default base signature will be
used. The major and minor version numbers in the default base signature are not used for
anything.
This „upgrading‟ of the user specified signature allows the system administrator to:

select which personality number to use when the user does not specify one, such as
„single_precision‟,
Convey System Administration Guide v1.6
34

select which hardware model number to use when the user does not specify one,
typically the architecture version number of the Convey coprocessors installed on the
Convey server this program will be run on, and

If the personality number in the default base signature is zero or an invalid personality
number, the user must specify a valid personality number when compiling or assembling
source code.
Convey recommends that the default value for the hardware model number be set to the value for
the Convey coprocessor the program will be run on. Although signatures with different hardware
model numbers are compatibile, the compilers use the hardware model number to optimize code
for better performance.
Users of the Convey compilers and assembler will typically specify a personality number, and
possibly a major and minor version numbers, but will not usually specify a hardware model
number.
At the time this users guide was written, the default signature installed the first time the convey1st package is installed was 2.1.1.1.0, which specifies the single precision personality number,
and a hardware model number of 0 (the first version of the Convey coprocessor released. Future
installations of Convey packages will not overwrite this file, even if the convey-per-tools package
is uninstalled.
7.3
System Default Signatures
After installing the convey-per-tools package for the first time (or anytime thereafter), the system
administrator may update the file
/opt/convey/personalities/SystemDefaults
This file contains zero or more signatures. The system administrator can modify this file to
establish default signatures for each personality. These personality specific default signatures
are used to „upgrade‟ a user supplied partial signature. When the user specifies a partial
signature, that signature is upgraded (if possible) to match an actual installed personality.
Typically, the major and/or minor version numbers are upgraded. See the upgrading discussion
in the Convey Programming Guide for more information.
For example, if the SystemDefaults file contains 2.2.3.1.0, and the user specified “sp” as a
partial signature when compiling a routine (“sp” maps to “2” in the Convey Personality Database),
then the partial signature will be upgraded to 2.2.3.x, where the value (x) for the hardware model
number is obtained from the system‟s default base signature value (see above).
7.4
Convey Personality Management Script
After installing the convey-per-tools package for the first time (or anytime thereafter), the
/opt/convey/sbin/cpm script may be used to manage personality signatures in the personality
database. The cpm script allows the system administrator to disable and enable particular
personalities. A disabled personality, although still installed on the system, cannot be used to
compile, assemble, or execute coprocessor code. The cpm script can also be used to list installed
signatures, and get additional info about a particular signature.
Convey System Administration Guide v1.6
35
Usage:
/opt/convey/sbin/cpm action [personality list]
action
What it does
list
Lists the specified signatures, if installed.
$ cpm list 2.1.1.1.0
1.1.1.1.0 produces
Personality
Compiler License
Coproc Nickname(s)
Number
Support Status
Support
--------------- ------- ------------------ ------ -------------------------1.1.1.1.0
Yes
Permanent
No
double dp double_precision
double_vector dp_vector
2.1.1.1.0
Yes
Permanent
No
single sp single_precision
single_vector sp_vector
If no signatures are specified, all signatures in the personality database will be listed.
A “compiler support” value of “yes” means the compiler can compile code for that
signature. A “coproc support” value of “yes” means an executable requiring that
personality may be executed on this system. The “license status” field shows if that
personality has a valid license on this system. A valid license for a personality is
required in order to exeute a program using that personality.
info
More verbose than „list‟, the „readme‟ file for the specified signatures is also included.
enable
The specified signatures are enabled.
disable
The specified signatures are disabled. A disable signature cannot be used by the
compiler.
Convey System Administration Guide v1.6
36
8 Recovery DVDs
Convey provides a recovery DVD with each Convey Server shipped (1 set of DVDs per cluster or
head node). Additional DVDs can be ordered through Convey‟s Support group.
This recovery DVD allows the system to be restored after a catastrophic disk or system failure,
with a modest amount of user interaction.
WARNING: The System Recovery DVD will completely erase the systems main hard
drive, and all data on that drive will be lost.
Note: the recovery process for non-head nodes (Convey server nodes WITHOUT a DVD drive) is
described later, near the end of this chapter.
8.1
How to use the Convey System Recovery DVD
The Convey System Recovery DVD installs CentOS plus Convey platform extensions and
enhancements.
In general, recovery / installation from the DVD involves the following steps:

System preparation including:
o
Attaching a graphical (PC compatible) display to the VGA adapter, and attaching a
USB keyboard /mouse, or
o
Logging into the previously configured Intel RMM console for the host server

Booting off of the DVD.

Reviewing and possibly modifying the installation parameters.

Allowing the configured installation to install.

Removing the DVD and booting off of the installed disk.

Configuring the system via CentOS‟s post install configuration menus.

After the system is completely installed and running, the second DVD may be used to
install various additional Convey products, including compilers, personalities, …
8.1.1 Preparation
Convey recommends that Convey OS installs be performed using the CentOS GUI installation
program. The GUI installation program offers the full functionality. To use the graphical
installation program, either:

attach a PC compatible graphical display, USB keboard, and USB mouse to appropriate
ports located on the back panel of the Convey sever, or

Log into the previously configured Intel RMM host server console from a web browser
8.1.2 Boot from DVD
Install the Convey System Recovery DVD into the servers DVD drive. If necessary, configure
the system BIOS to boot from the DVD and allow the system to boot. A successful DVD boot will
result in a CentOS splash sceen being displayed on the console with the following text menu
options:
- To install or upgrade in graphical mode, press the <ENTER> key.
Convey System Administration Guide v1.6
37
- To install or upgrade in text mode, type: linux text <ENTER>
- Use the function keys listed below for more information.
[F1-Main] [F2-Options] [F3-General] [F4-Kernel] [F5-Rescue]
To abort the installation process at this time, reboot your machine and then eject the boot media
DVD.
To continue with the installation, press enter and allow the system installer to load.
8.1.3 Configure the Installation and Installing the OS
After loading the Anaconda CentOS system installer, the CentOS welcome menu will be
displayed; proceed through the following menus to configure the installation. Please note that
network, firewall, system name, and a few other items will be configured after the installation,
during the first hard disk boot:
1. Click Next to enter the disk partitioning menu.
2. Partition the disk as to your local requirements; a recommended partitioning is as follows:
a. 2G /boot partition
b. 64G Swap partition
c.
The remainder of the disk allocated to the “/” partition.
3. Configure the time zone
4. Create a root password
5. Add additional packages; all recommended and required Convey base platform RPMs
are already selected
6. Click next to arrive at the final installation menu; you may abort the installation at this
point by rebooting your machine and then ejecting the boot media.
7. Click next and wait for the installation to complete.
When the installation is complete the DVD will be ejected (remove the DVD from the drive) and
the system will reboot. Upon a successful disk reboot the system will enter the CentOS “first boot”
configuration GUI; proceed through these menus to configure the Convey Server according to
your local requirements.
8.1.4 System Configuration
Additional system configuration occurs during the first, post install, boot. A GUI steps through the
following menu options; complete details of the menu options can be found at
http://www.centos.org/docs/5/. This is a summary of the configuration menus:
1. Cent OS welcome menu:
Click Forward
2. Keyboard configuration menu:
Select the appropriate keyboard type.
Click Forward when complete.
3. Root Password menu:
Create a root password.
Click Forward when complete.
Convey System Administration Guide v1.6
38
4. Network configuration menu:
The default configuration is DHCP. Click Change Network Configuration to
configure static IP addresses; a pop-up menu will be displayed that provides IP, DNS,
/etc/hosts, and other configuration panels. Note if you click outside of the network
configuration popup then the window manage will push the network pop up to the
background; use the Alt+Tab keyboard short cut to bring the network configuration
menu to the foreground. Click on the popup X button at the upper right to exit the
network configuration.
Click Forward when complete.
5. Firewall configuration menu:
The firewall is disabled by default; configure as appropriate for your installation.
Click Forward when complete.
6. SELinux configuration menu:
SELinux is disabled by default; configure as appropriate for your installation.
Click Forward when complete.
NOTE: if you enable SELinux, please see the SELinux FAQ at
www.conveysupport.com/protected/faqs for more information about configuring SELinux
(required in order for the Convey compilers to operate correctly).
7. Kdump configuration menu:
kdump is enabled by default; Convey recommends that it remain configured with 128MB
of reserved memory.
Click Forward when complete.
8. Timezone configuration menu:
Select the appropriate timezone.
Click Forward when complete.
9. Date and Time configuration menu:
Verify the current time and date; adjust as necessary. Use the NTP configuratioin tab to
enable and configure NTP.
Click Forward when complete.
10. Create User configuration menu:
Define additional local users. Select the Use Network Login button to configure NIS.
Click Forward when complete.
11. Sound Card configuration menu:
Click Forward to skip.
12. Additional CDs menu:
Click Finish to complete the basic Convey / Centos server platform installation.
13. After several seconds, the CentOS graphical login menu is displayed.
8.2
Fixing the Convey SW Repository Files
You need to add the Convey provided username and password (for web access) to every Convey
SW Repository file in /etc/yum.repos.d.
Edit every file in /etc/yum.repos.d, whose name begins with “convey-“, and insert your company‟s
Convey web access username and password into the URL entry. For example, if your site‟s
username / password was abc / xyz, then a URL line that initially looked like
baseurl=http://conveysupport.com/protected/install/hc1-1/convey_sdk/x86_64/RPMS/
would look like this after editing:
Convey System Administration Guide v1.6
39
baseurl=http://abc:[email protected]/protected/install/hc1-1/convey_sdk/x86_64/RPMS/
8.3
Adding Convey Provided License Keys
After restoring the Convey OS, various optional products need to be installed, some of which
require a Convey provided license.
You may request a copy of the license(s) from the Convey support group (email
[email protected] from an email address associated with the organization that
purchased the Convey server please).
8.4
Installing the Remaining Convey Software Products
See section Installing Convey Software on Cross-Development Systems. This section
applies to Convey servers as well as cross-development systems.
The Convey host system should now be fully configured and operational.
8.5
Checking the Management Processor
After following all the above steps, run the command: /opt/convey/sbin/mpinfo
If the Management Processor (MP) is communicating with the host system, something similar to
the following will be appear:
# /opt/convey/sbin/mpinfo
Hostname
: your hostname
Active recipe
: recipe.beta-0.9.3-09_07_08
Next recipe
: none
Active firmware
: firmware-0.0-0.0.1124-1
Committed firmware
: firmware-0.0-0.0.1124-1
Trial firmware
: none
U-Boot
: 1.3.0-rc3 (May 13 2009 - 16:03:38)
Configured eth0
: static 192.168.22.182/24:
Current eth0
: 192.168.22.182/24:192.168.22.5 up
Console baud
: 115200
Xinetd
: on
Clock Frequency
: 150.000 MHz
If the management processor appears to not be communicating with the host system, the
following steps should correct the problem:

Do a yum update convey-\* (as root) to update all of Convey‟s packages to the latest
available (and known to be consistent) versions. Alternatively, explicitly install the specific
versions of the convey-kernel (or convey-kernel_beta), the convey-mp_fw, and the
convey-uboot packages that you were using before the system was restored.
If convey-mp_fw or convey-uboot was one of the packages updated by the above
command, you must also shutdown the host as shown below

Shutdown the host (/sbin/shutdown -h now bye), wait at least two minutes, and
power up the host via the RMM or the host systems power button.

Try the mpinfo command again. If the MP is still not communicating with the host
system, please contact Convey support ([email protected]).
Convey System Administration Guide v1.6
40
8.6
Updating the Server to the latest SW Versions
If you didn‟t do a yum update as described in the preceeding sub-section, you may wish to
upgrade all the installed Convey SW to the latest released versions. The command
yum update convey-\*
(logged in as root)
will upgrade all installed Convey SW packages, and may require a shutdown/reboot.
8.7
Restoring a Convey Server without a DVD drive
Contact the Convey support group ([email protected]) for detailed instructions.
Convey System Administration Guide v1.6
41
9 Customer Support Procedures
Please use email or the toll free phone number for simple questions. The issue tracker web
interface is ideal for reporting bugs and requesting enhancements to Convey products.
Email
[email protected] : Convenient for
quick simple questions.
Issue Tracker Web Interface
This is the preferred contact method when additional
information needs to be passed on to Convey, such as
source files, screen dumps, etc.
If you already have a Convey Issue Tracker username
and password, browse to rt.conveysupport.com.
If you need an issue tracker username and password,
browse to conveysupport.com/issuetracker.html and
follow the directions there.
Call Convey‟s toll-free number
1-866-338-1768
Select the “Customer Support” extension (“2” when this
document was written). If you are transferred to
voicemail, please leave a message with your name,
your company‟s name, your phone #, and a brief
description of the problem. If you prefer to receive an
e-mail response, please leave your e-mail address.
This number will usually be answered 9am – 5pm
Central Time (USA), if a customer service
representative is available.
A customer service representative will typically acknowledge all email, voicemail, and new issues
reported via the issue tracker within 24 hours, Monday-Friday (except holidays).
For extremely urgent issues, please send an email to [email protected] and call
the toll-free #.
Convey System Administration Guide v1.6
42
10 Licensing
10.1 Convey Licensed Software
Beginning with the April 2010 release, all of the Convey products may be installed on a Convey
Server or customer provided cross-development system, but a license is required to use some of
the features and capabilities provided by those packages.
For a Convey server, the required licenses are typically installed before the server is delivered.
For a cross-development system, and for an add-on order for new software for a Convey server,
the necessary license(s) are typically e-mailed to the designated contact, along with instructions
for installing those licenses.
The following are examples of what features and capabilities are licensed:

Using a Convey provided vector personality when executing on the coprocessor

Use of the Convey coprocessor simulator

Use of the Convey compiler suite
10.2 Old Licensing Scheme
Prior to April 2010, the packages containing the Convey personality tools, the FPGA personality
images, the personality development kit, some of software development utilities, and the Convey
Math Libraries were licensed. Those older packages can only be installed if a license key for that
package is already present in the file /opt/convey/licenses. Usually, Convey Customer Support
would have emailed the license keys to each customer‟s designated contact person. Send e-mail
to [email protected] for help with license keys.
The is no need to remove the /opt/convey/licenses file when installing products that use the
new licensing mechanism described above. You may wish to install older versions of some
Convey packages someday, and the /opt/convey/licenses file may be required to install those
packages.
Convey System Administration Guide v1.6
43
Appendix A - Installing YUM on older SUSE™
Distributions
Start up yast, then

in the Software part, pick Software Management, then

in the search bar, type yum (and click search), then

select the package named yum, then

click on accept (you may have to accept other packages due to dependencies)
On Suse distributions that support zypper (openSUSE 10.3 and later), please use zypper, not
yum.
Convey System Administration Guide v1.6
44
Index
.
31-31 interleave
apt-get
binary interleave
boot options
booting
configuration files
Convey software distribution CD
Convey System Services
customer support
default base signature
default signatures
distributions, Linux
First Boot
installation, manual
Installing Convey Software
installing convey software without an internet
connection
Intel Remote Management Module
kernel configuration
licensing
linux distributions
Convey System Administration Guide v1.6
29
12
29
20
19
24
17
23
41
33
34
10
19
16
7
17
19
23
42
10
memory interleaving
memory migration
NUMA Memory Configuration
package version numbers
personalities
personality upgrading
Powering Up the Convey Server
Recovery DVDs
repository description file
RMM
SDK/Optional Products DVD
signature
software repository description files
system memory management
System Recovery DVD
updating Convey software packages
variable page size
wget
yum
zypper
28
31
26
15
33
33
19
36
7
19
39
33
7
26
36
14
30
16
12
12
45