EDG Testbed installation and
configuration with LCFGng
Maite Barroso - WP4
[email protected]
Contents

Current EDG Testbed structure and configuration

EDG configuration files




Description
Where to get them
How to use them
Upgrade of a basic LCFG server and client to an EDG testbed server
and client
EDG Testbed installation and configuration with LCFG - n° 2
EDG Logical Machine Types
1. User Interface (UI)
4. Computing Element (CE)

Gatekeeper
(Front-end Node)

2. Resource Broker (RB)
3. Information Service (IS)
Worker Nodes (WN)
5. Storage Element (SE)
6. Replica Catalog (RC)
EDG Testbed installation and configuration with LCFG - n° 3
EDG Testbed Configuration
UI
User Interface
CE
Computing Element
SE
WN
WN
Worker WN
Node
Worker WN
Node
Worker WN
Node
Worker WN
Node
Worker Node
Worker Node
Storage Element
Minimal Testbed
Proxy
RB
Proxy renewal
Resource Broker
LCFG
Installation
Server
MDS
RC
Meta Data Server
Replica Catalog
EDG Testbed installation and configuration with LCFG - n° 4
EDG Configuration files

EDG configuration files stored in the DataGrid cvs repository:
http://marianne.in2p3.fr/datagrid/testbed1/repositories/index.html
 In

the main directory edg-release/
Two types of LCFGng files needed:


RPM configuration files: list of RPMS to be installed on a node
Component configuration files: files containing the configuration
information associated to each LCFG component in LCFG syntax
(key-value pairs)
EDG Testbed installation and configuration with LCFG - n° 5
RPM configuration files
EDG Testbed installation and configuration with LCFG - n° 6
RPM config files
 Stored
in the cvs directory edg-release/rpmlist/
 They
are built following a hierarchical structure of files containing
standard cpp directives like #define, #include, #ifdef, comments
with /* */, etc...
 These
files should be located in the LCFGng server, in the directory:
/opt/local/linux/6.2/rpmcfg
 They


include two types of files:
header files (.h)
template configurations
EDG Testbed installation and configuration with LCFG - n° 7
RPM config files: header files (I)

The RPM config file is a list of RPMs specifications, each of the
following form:
[prefix]rpmname-version-release[/arch][:flags]
 version-release
wildcards are supported :
* as the release field will imply the latest version/release of the specified
RPM found in the RPM repository
 [prefix]


+
used to replace a previously listed package with another version or
release of the same package
-
used to remove an already listed package
 [/arch]
overrides the default architecture
EDG Testbed installation and configuration with LCFG - n° 8
RPM config files: header files (II)

[:flags] consists of one or more single character flags as defined
below:







:f
install this package without making any checks at all.
:d
install this package without making any dependency checks.
:r reboot this machine when this RPM is installed, removed or upgraded.
(not yet fully implemented).
:i
ignore any already installed RPM matching the given RPM spec.
:c
preserve this RPM's config files across upgrades.
:t
install this package without invoking trigger scripts.
:s
install this package without running any pre or postinstall scripts.
NOTE: for more detailed information, use man updaterpms
EDG Testbed installation and configuration with LCFG - n° 9
RPM config files: header files (III)

Example: ComputingElement-rpm.h
/*
ComputingElement.h
========================================================
Customized list for a ComputingElement - Globus 2 BETA 21
*/
#ifdef OPENPBS
openpbs-*-*:s
#endif
globus_common-gcc32dbg_dev-2.0-21
globus_common-gcc32dbg_pgm-2.0-21
globus_common-gcc32dbg_rtl-2.0-21
globus_common-gcc32dbgpthr_dev-2.0-21
globus_common-gcc32dbgpthr_rtl-2.0-21
cyrus-sasl-1.5.11-2:s
EDG Testbed installation and configuration with LCFG - n° 10
RPM config files: template configuration (I)
 One
template per machine-type:

BASE-rpm: Base list for minimal system

CE-rpm: Computing Element list

NM-rpm: Network Monitoring list

RB-rpm: Resource Broker list

RC-rpm: Replica Catalogue list

SE-rpm: Storage Element list

UI-rpm: User Interface list

WN-rpm: Worker Node list
EDG Testbed installation and configuration with LCFG - n° 11
RPM config files: template configuration (II)

Each template is organized in a hierarchical way, with a list of #includes
of the needed header files. An example, CE-rpm:
/* CE-rpm
=====================================================
ComputingElement high-level RPM list
*/
Rh62 list of RPMs
/* RedHat 6.2 + updates */
#include "rh62-release-rpm.h"
#include "rh62-updates-rpm.h“
/* LCFG support */
#include "lcfg-common-rpm.h"
#include "lcfg-client-rpm.h"
/* OpenSSH */
#include "openssh-rpm.h“
LCFG list of RPMs
Computing Element
specific list of RPMs
/* ComputingElement specific packages */
#include "ComputingElement-rpm.h"
EDG Testbed installation and configuration with LCFG - n° 12
RPM config files: template configuration (III)

But they can also contain lists of RPMs, without including them from any
file. An example, CE-rpm:
/* Packages added to satisfy dependencies */
libxml-1.8.6-2


Or overwrite a previously defined RPM version:
+jade-1.2.1-9:s
+sgml-common-0.1-7:s
+docbook-3.1-3:s
+stylesheets-0.13rh-4:s
+openldap-servers-1.2.12-3:s
Or include #define directives:
/* Define architecture */
#define ISARCH_i386 1
#define ARCH i686
EDG Testbed installation and configuration with LCFG - n° 13
Component configuration files
EDG Testbed installation and configuration with LCFG - n° 14
Component config files: format (I)

Contain the needed configuration information expressed in LCFG syntax, keyvalue pairs known as resources:
hardware.keytable
us

The key specifies the LCFG component and the parameter, e.g. hardware.keytable

Types



Existing types: integer, boolean and string (default)
Type definition:
@size <%integer%>
@enable <%boolean%>
References to other component’s resources:
updaterpms.rpmcfg
<%update.initrpmcfg%>
NOTE: Remember that a machine DOES NOT have access to other machines resources.

The + symbol indicates that the new definition overrides any precedent one:
+hardware.keytable

en
If you don’t want to overrride but to add, use the EXTRA macro instead:
EXTRA(hardware.keytable ) en
EDG Testbed installation and configuration with LCFG - n° 15
Component config files: format (II)

More complex structures can also be defined:
@users
usercomment_$ userhome_$
users
usercomment_$
userhome_$

Example:
auth.users
edginfo mysql
auth.usercomment_edginfo
EDG Info user
auth.userhome_edginfo
/tmp
auth.usercomment_mysql
user for mysql
auth.userhome_mysql

/home/mysql
To add a new element use the macro EXTRA:
EXTRA(auth.users)
postgres
auth.usercomment_postgres
auth.userhome_postgres
user for postgres
/home/postgres
EDG Testbed installation and configuration with LCFG - n° 16
Component config files

Stored in the cvs directory edg-release/source/
 These
files should be located in the LCFGng server, in the
directory: /var/obj/conf/server/source
 Organized
inclussion:




in a hierarchical way, using simple inheritance by file
One file per functional entity to be configured, e.g.
FabricMonitoring-cfg.h, Users-cfg.h, pbs-cfg.h
One file per machine type, e.g. ComputingElement-cfg.h,
WorkerNode-cfg.h
One file per site, with the site dependent configuration information,
e.g. site-cfg.h
Last level on the hierarchy: one file per machine, file named after
the unqualified DNS name of the client, containing the host
dependent configuration information, e.g. testbed006
EDG Testbed installation and configuration with LCFG - n° 17
Component config files: example 1
/* testbed006
testbed006
==============================================
COMPUTING ELEMENT- NG VERSION
*/
/* Host specific definitions */
#define HOSTNAME testbed006
Specific host configuration
/* Some useful macros */
#include "macros-cfg.h"
/* Site specific definitions */
Site config files
#include "site-cfg.h"
/* Basic RH62 client resources */
Feature/component
configuration (Linux, users…)
Config files per
machine type
#include "redhat62-cfg.h"
/* Users */
#include "Users-cfg.h"
/* Computing Element specific resources */
#include "ComputingElement-cfg.h"
+auth.rootpwd
a1Z1WykXTe2PU
EDG Testbed installation and configuration with LCFG - n° 18
Component config files: example 2
ComputingElement-cfg.h
/* globus
Globus component configuration
needed in a Computing Element
------------------------------------------------------- */
EXTRA(profile.components)
globus
profile.version_globus
1
globus.ng_reconfig
configure
EXTRA(boot.services)
globus
/* globus.conf content */
globus.globusconfenabled
yes
globus.globusconf
/etc/globus.conf
inet component configuration
needed in a Computing Element
/* inet: add gatekeeper service to /etc/services,enable mount
------------------------------------------------------- */
EXTRA(inet.etcservices)
gatekeeper
inet.etcsrvconf_gatekeeper
globus-gatekeeper 2119/tcp
EXTRA(inet.allow)
mountd portmap gatekeeper
inet.allow_mountd
SITE_ALLOWED_NETWORKS
inet.allow_portmap
SITE_ALLOWED_NETWORKS
inet.allow_gatekeeper
ALL
EDG Testbed installation and configuration with LCFG - n° 19
Component config files: example 3
site-cfg.h
/*site-cfg.h
============= SITE SPECIFIC CONFIGURATION ============
*/
/* COMMON GRID DEFINITIONS -------------------------------- */
#define GRID_RB_II_HOST
lxshare0315.cern.ch
#define GRID_RB_II_PORT
2170
/* COMMON SITE DEFINITIONS -------------------------------- */
/* FQDN of your LCFG server */
#define SITE_LCFG_SERVER
lxshare0315.cern.ch
/* Your domain */
#define SITE_LOCALDOMAIN
cern.ch
/* Sysadmin e-mail */
#define SITE_MAILROOT
[email protected]
/* The gateway/router */
#define SITE_GATEWAYS
137.138.1.1
/* Allowed networks (useful for tcpwrappers) */
#define SITE_ALLOWED_NETWORKS
127.0.0.1, 192.168., 137.138.
/* DNS */
#define SITE_NAMESERVERS
137.138.16.5
EDG Testbed installation and configuration with LCFG - n° 20
SERVER UPGRADE
1) Assume LCFGng base server up & running
2) Populate the RPM repository at /opt/local/linux/6.2/RPMS
with all needed RPMs from EDG central repository (don’t forget to
generate the header files!)
3) Copy the needed RPM config files from cvs (edg-release/rpmlist)
to the LCFGng server, /opt/local/linux/6.2/rpmcfg. This
files normally don’t need to be modified.
4) Copy the needed component config files from cvs (edgrelease/source) to the LCFGng server,
/var/obj/conf/server/source.
EDG Testbed installation and configuration with LCFG - n° 21
SERVER UPGRADE
5) Adapt them to configure your clients:
5.1) Change the file site-cfg.h with your site settings.
5.2) Create/modify the configuration file for every client, naming it with
its unqualified DNS name, e.g. testbed006 (template configuration files
called testbed00x can be found in cvs).
 Decide what type of machine you want to have, e.g. Computing
Element
 Include the needed config files according to your machine type, e.g.
ComputingElement-cfg.h for a Computing Element
 Add the host-dependent configuration (host name, root password…)
5.3) Compile and generate the XML node profile
/usr/sbin/mkxprof –v –s testbed006
5.4) If it is a new client, edit /etc/dhcpd.conf and add entries for your
client machine. Don’t forget to restart the service afterwards!
EDG Testbed installation and configuration with LCFG - n° 22
CLIENT UPGRADE
5) Reboot the client
6) Check that all the components started in the correct way: have a
look to the log files located at:
/var/obj/log/component_name
7) You can always start/stop/restart the components manually:
/etc/obj/component_name start
EDG Testbed installation and configuration with LCFG - n° 23