Open Source on IBM i
Václav Matoušek
Consultant, IBM CZ
Agenda
•
•
•
•
•
•
•
•
•
Open Source Technologies
Samba
Node.js
Python
GCC
Git
Tools
Orion
Cloud.init
Open Source Technologies (2017)
5733OPS
5733OPS
5733OPS
5733OPS
5733OPS
5733OPS
5733OPS
5733OPS
5733OPS
5733OPS
5733OPS
5733OPS
5733OPS
5733OPS
5733OPS
5733OPS
*BASE
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
IBM i Open Source Solutions (Samba)
IBM i Open Source Solutions Option 1 (Node.js)
IBM i Open Source Solutions Option 2 (Python 3)
IBM i Open Source Solutions Option 3 (GCC scripts)
IBM i Open Source Solutions Option 4 (Python 2)
IBM i Open Source Solutions Option 5 (Node.js v4)
IBM i Open Source Solutions Option 6 (Git)
IBM i Open Source Solutions Option 7 (Tools)
IBM i Open Source Solutions Option 8 (Orion)
IBM i Open Source Solutions Option 9 (cloud init)
IBM i Open Source Solutions Option 10 (Node.js v6)
IBM i Open Source Solutions Option 11 (Nginx)
IBM i Open Source Solutions Option 12 (TBD)
IBM i Open Source Solutions Option 13 (TBD)
IBM i Open Source Solutions Option 14 (TBD)
IBM i Open Source Solutions Option 15 (TBD)
IBM i Release
Group PTF
Level
i 7.3
SF99225
3
i 7.2
SF99223
3
i 7.1
SF99123
3
How to obtain the new Open Source for IBM i - 5733OPS
Step 1: Install any requisite products
• PASE
– PASE (SS1 option 33) is required for all 5733-OPS options.
• 5733-SC1
– Various options within 5733-OPS require zlib or OpenSSL to function which are provided by
5733-SC1
– For IBM i 7.1, you will need 5733-SC1 7.1. For IBM i 7.2 and up, you will need 5733-SC1 7.2
– Make sure to install the latest SC1 PTFs as well.
• XMLSERVICE
– The Python, Node, ... language toolkits require XMLSERVICE to function.
– If you do not plan to use the toolkit, XMLSERVICE is not required, but it is recommended
– A version for XMLSERVICE is installed with latest HTTP PTF Group (see the
IBM i Preventive Service Planning page) and installed in the QXMLSERV library.
– The latest PTF versions of the toolkit will default to using the QXMLSERV installation of
XMLSERVICE. Alternatively, the toolkits can be configured to use an alternate XMLSERVICE
installation. See XMLSERVICE install documentation for more details on installing and the
toolkit documentation to configure the XMLSERVICE location.
• Others
– Please see individual option pages for any other dependencies that option may have that is
not listed here
How to obtain the new Open Source for IBM i - 5733OPS
Step 2: Obtain the 5733-OPS media
• The new License Program Option 5733-OPS "Open Source for IBM i" is being included as part if
the IBM i Bonus Package. This means that if you are current on your IBM i software maintenance
you are entitled to this new offering. Below are the designated ways that you can obtain this new
support.
• Starting at 5733-OPS GA, it will begin shipping with IBM i 7.1 and later Operating System Keyed
Media Set. All orders with ship date of GA and later would receive this new media.
• If entitled to the IBM i V7 OS at GA, 5733OPS will become part of the 7.1 and later Operating
System ESD download package under 5770-SS1 base FC's 5817, 5818, 5819, 5827, 5828, and
5829 the package is titled F_MULTI_NLV_110_IBM_i_Open_Source_Solutions on ESS
• At GA 5733OPS will become part of the 7.1 and later Operating System ESD download package
under the IBM i Evaluation, Try & Buy and NLV download section, under 5770-SS1 base FC's
5817, 5818, 5819, 5827, 5828, and 5829 the package is titled
F_MULTI_NLV_110_IBM_i_Open_Source_Solutions on ESS
How to obtain the new Open Source for IBM i - 5733OPS
Step 3: Install the 5733-OPS option placeholders
• 5733-OPS media is shipped with empty, undefined placeholders.
• These options will be defined and filled with new content via PTF
• IBM recommends you install all product options for the following reasons:
– some options have cross-option dependencies
– as new options are defined and delivered, you will receive that function via PTF without
having to go back and install the product option (since it's already installed)
How to obtain the new Open Source for IBM i - 5733OPS
Step 4: Install 5733-OPS enablement and superseding PTFs
• To install each option individually, see each option's wiki page for information on which PTFs to
apply to enable that option
• Alternatively, just install the latest Open Source Solutions Group PTF to install and update all 5733OPS options installed. See here for the group applicable to your IBM i release.
• Note: You cannot install PTFs for products or product options that are not installed. Installing PTFs
will not install those product options automatically. When new product options are defined, you will
have to ensure that product option is installed prior to ordering the enablement PTFs for that option.
• Note: 5733-OPS PTFs are designed to be installed on IBM i 7.1 and up, thus may depend on fixes
in multiple OS or LPP release levels. For example, a Python database PTF may depend on a fix in
CLI for IBM i 7.1 and 7.2. When ordering such a PTF, you can disregard warning messages about
PTFs for a different release than the IBM i release that you're on.
IBM i support for Samba
Samba is an open source SMB-based networking protocol for providing fast, stable, and secure file
access. File serving, something we have had on IBM i for a long time is provided with IBM i NetServer.
IBM i. NetServer has many great features when it comes to file serving, although in some instances
performance has been an issue. Samba is a server that uses TCP/IP on IBM i to interact with
Microsoft® Windows® clients or servers as if it is a Windows file and print server. Samba is not
intended to be a full replacement for file serving on IBM i, but rather give customers an additional
option. For example, Samba does not support Kerberos, automatic CCSID conversions, or integration
with IBM i auditing exit programs. For those features, IBM i NetServer is the best choice. But, if you
require just basic file serving with performance, then Samba might be the choice for you.
Consider the following key features:
– A fast, modern, lightweight Common Internet File System (CIFS) file server
– An FTP style CIFS client (smbclient)
– NTLM and NTLMv2 authentication with session security enhancements
– Enhanced error code reporting
– Better protocol compatibility with current Windows clients
– SMB 2.0 protocol support
Samba is provided as an addition to the IBM i Operating System and is shipped with a simple PTF.
IBM i support for Samba - continuing
• Please see the Services Guide for instructions on the latest PTF number, install
instruction, and how to start Samba: Samba on IBM i Service Guide
– http://www-01.ibm.com/support/docview.wss?uid=nas8N1020089
• Samba use the same port as NetServer (445). There is needed to stop NetServer
(ENDTCPSVR *NETSVR) before start Samba deamon.
• Once you have Samba running on IBM i, there is nothing special that you need to do.
Its just Samba. All the instructions, best practices and such are the same across
Samba running on any platform.
Node.js
• Node.js is an open source
project based on Google
Chrome V8 JavaScript Engine.
It provides a platform for
server-side JavaScript
applications running without
browsers. The event-driven,
non-blocking I/O model makes
it lightweight and efficient to
use. It also provides several
built-in modules to simplify
programming, especially for
networking applications. There
are many more third-party
modules that can be easily
installed with the built-in npm
(node packaged modules) tool
to extend Node.js.
DB2 for i Access APIs
• Methods
– setConnAttr
– getConnAttr
– conn
– disconn
– close
– debug
– validateStmt
– setStmtAttr
– getStmtAttr
– exec
– execSync
– prepare
– prepareSync
– bindParam
–
–
–
–
–
–
–
–
–
–
–
–
–
–
bindParamSync
execute
executeSync
nextResult
commit
rollback
closeCursor
close
fetch
fetchSync
fetchAll
fetchAllSync
numFields
numRows
–
–
–
–
–
–
–
–
–
fieldType
fieldWidth
fieldNullable
fieldName
fieldPrecise
fieldScale
validStmt
stmtError
debug
Toolkit for IBM i
• Global Object itoolkit
– xmlToJson
iCmd
iSh
• Class iConn
– iConn
debug
add
run
getConnection
setTimeout
• Class iPgm
– addParam
addReturn
toXML
• Class iSql
– iSql
addQuery
fetch
commit
prepare
execute
• Class iWork
– iWork
getSysValue
getSysStatus
getSysStatusExt
getJobStatus
getJobInfo
getDataArea
• Class iProd
– iProd
getPTFInfo
getProductInfo
getInstalledProducts
• Class iUserSpace
– iUserSpace
createUserSpace
setUserSpaceData
getUserSpaceData
deleteUserSpace
• Class iNetwork
– iNetwork
getTCPIPAttr
getNetInterfaceData
• Class iObj
– iObj
retrUsrAuth
retrCmdInfo
retrPgmInfo
retrSrvPgmInfo
retrUserInfo
retrUserAuthToObj
addToLibraryList
• Class iDataQueue
– iDataQueue
sendToDataQueue
receiveFromDataQueue
clearDataQueue
Node modules
• Node provides the concept of "modules"
– Libraries (like service programs?) that you can use in your program
– Sometimes included with the Node.js install
– Or, provided as downloadable packages (lots and lots of them available)
– Or, you can write your own.
– Bring them into your program with the require statement
Node.js
Module installation manager
• NPM is used from the PASE command like (via Putty or 5250)
– npm list will list the packages currently installed
– npm install <package> will install a package
– npm update <package> will update an existing package
– npm uninstall <package> will remove an existing package
– npm search <keyword> will search for a package
• DNS must be setup
• By default packages are listed/installed in the local project. Add --global to make it
install system-wide. (or -g for short!)
• By default, it tries to use the most current version of a package available. You can add
@VERSION to the package name to install a particular version, such as npm install
[email protected] –global
Node.js
Install module manually - example
•
•
•
•
Download module to your PC.
Use ftp to transfer it to IBM i.
Unzip file.
npm install –g ./python-shell-master
• cd /QOpenSys/QIBM/ProdData/OPS/Node6/lib/node_modules
• npm list
• Then you can use code like
• var PythonShell =
require('/QOpenSys/QIBM/ProdData/OPS/Node6/lib/node_modules/python-shell');
Node.js
What Is Long Term Support and Why Does It Matter to Me?
• First and foremost, if you haven’t read the Essential Steps: Long Term Support (LTS) for Node.js by
Rod Vagg, Technical Steering Committee Chairperson at the Node.js Foundation and the Chief
Node Officer at NodeSource, do so. It’s a very helpful source for understanding our release cycle
process. If you only have two minutes now, here is a quick summary:
• The point of establishing an LTS plan for Node.js is to build on top of an existing stable release
cycle by delivering new versions on a predictable schedule that have a clearly defined extended
support lifecycle. It is an essential requirement for enterprise application development and
operations teams. It also affects companies that provide professional support for Node.js.
• As stated above, the first LTS release line is v4 “Argon," beginning at v4.2.0 and currently standing
at v4.2.1. The next LTS release line will begin in 12 months around the first week of October 2016.
All LTS release lines will begin at the same time each year.
• All LTS release lines are assigned a “codename” drawn from the names of the elements on the
Periodic Table.
• The LTS release line will be actively maintained for a period of 18 months from the date the LTS
release line begins. After 18 months have passed, it will transition into Maintenance mode.
• There will be no more than two active LTS release lines at any given time. Overlap is intended to
help ease migration planning.
• Once a Current release line becomes LTS, no new features or breaking changes will be added to
that release. Changes are limited to bug fixes for stability, security updates, possible npm updates,
documentation updates and certain performance improvements that can be demonstrated to not
break existing applications.
Node.js Long Term Support (LTS) Release Schedule
Python
• Python is a widely used high-level programming language for general-purpose
programming, created by Guido van Rossum and first released in 1991. An interpreted
language, Python has a design philosophy which emphasizes code readability (notably
using whitespace indentation to delimit code blocks rather than curly braces or
keywords), and a syntax which allows programmers to express concepts in fewer lines
of code than possible in languages such as C++ or Java. The language provides
constructs intended to enable writing clear programs on both a small and large scale.
• Python features a dynamic type system and automatic memory management and
supports multiple programming paradigms, including object-oriented, imperative,
functional programming, and procedural styles. It has a large and comprehensive
standard library.
Python – package installation manager
Difference between 'python setup.py install' and 'pip install'
On the surface, both do the same thing: doing either python setup.py install or pip install < PACKAGE-NAME> will install
your python package for you, with a minimum amount of fuss.
However, using pip offers some additional advantages that make it much nicer to use.
• pip will automatically download all dependencies for a package for you. In contrast, if you use setup.py, you often
have to manually search out and download dependencies, which is tedious and can become frustrating.
• pip keeps track of various metadata that lets you easily uninstall and update packages with a single command:
pip uninstall <PACKAGE-NAME> and pip install --upgrade <PACKAGE-NAME>. In contrast, if you install a package
using setup.py, you have to manually delete and maintain a package by hand if you want to get rid of it, which
could be potentially error-prone.
• You no longer have to manually download your files. If you use setup.py, you have to visit the library's website,
figure out where to download it, extract the file, run setup.py... In contrast, pip will automatically search the
Python Package Index (PyPi) to see if the package exists there, and will automatically download, extract, and
install the package for you. With a few exceptions, almost every single genuinely useful Python library can be
found on PyPi.
• pip will let you easily install wheels, which is the new standard of Python distribution. More info about wheels.
• pip offers additional benefits that integrate well with using virtualenv, which is a program that lets you run multiple
projects that require conflicting libraries and Python versions on your computer. More info.
• pip is bundled by default with Python as of Python 2.7.9 on the Python 2.x series, and as of Python 3.4.0 on the
Python 3.x series, making it even easier to use.
Python
Install package manually - example
•
•
•
•
•
•
•
•
•
Download required package to PC.
Use FTP to trasfer to IBM i
Use gzip and tar from tools to unpack
Copy unpacked directory to target directory
(/QOpenSys/QIBM/ProdData/OPS/Pythonx.x/lib/pythonx.x/site-packages, where x.x is
2.7 or 3.4)
Go inside transfered directory (cd json2html-1.1.1)
Issue command: pythonx.x ./setup.py install
> pip list
json2html (1.1.1)
Issue command: pip2 list (x.x = 2.7) or pip3 list (x.x = 3.4)
pip (8.1.1)
setuptools (20.3.1)
Issue command export LC_ALL=C, if you have problem with locale)
six (1.10.0)
Then you can use code like:
wheel (0.29.0)
$
– import json2html
– from json2html import *
inside your python code without errors.
GCC
• GCC is a popular C language compiler used for compiling Open Source projects. One way to get it
is to use the 5733-OPS Option 3
• Option 3 of 5733-OPS ships a set of scripts that enable you to install RPMs from perzl.org. This
website contains open source software compiled for AIX, which can be made to install and run
inside PASE. These scripts were developed in an Open Source project by the Young i Professionals
and now live at the ibmichroot BitBucket project. IBM periodically replaces GCC option 3 via PTF,
however, you may download newer versions at bitbucket Downloads.
• PTF
– The latest PTF is SI58604. PTF creates /QOpenSys/QIBM/ProdData/OPS/GCC, which
contains all materials.
• FIX
– I required newer versions of m4 and gmake for perzl gcc 4.8.3 builds. I posted binaries to
Yips OPS Product section.
• Chroot Scripts provides two main scripts:
– chroot_setup.sh -- set up a chroot location enabled for PASE development safely away from
your PASE installation (*see about chroot below). A *SECOFR profile is assumed for
chroot_setup.sh operations.
– pkg_setup.sh - download and/or install selected perzl.org rpms, AIX binaries for PASE. For
IFS root file system installs, pkg_setup.sh may be run without using chroot_setup.sh.
Git
• Git is an open source distributed Source Control Management (SCM) system, also
called a Version Control System (VCS). Git is the most popular SCM for Open Source
Software (OSS) and allows easy sharing of code using sites like GitHub, Gitlab, and
Bitbucket - though a cloud provider is not required for code hosting.
• You can learn more about Git at https://git-scm.com/
• The latest PTF for Git support is: SI63502.
Major Git commands
• git config. Sets configuration values for your user name, email, gpg key, preferred diff
algorithm, file formats and more. ...
• git init. Initializes a git repository – creates the initial '.git' directory in a new or in an
existing project. ...
• git clone. ...
•
•
•
•
git add. ...
git rm. ...
git commit. ...
git status. ...
GitHub is home to open source projects
Tools
• The OPS tools option will contain many open source tools commonly found on Unix
and Linux. The tools live under /QOpenSys/QIBM/ProdData/OPS/tools/bin to prevent
clashing with software installed from AIX toolbox, Perzl or Bull Freeware RPMs or AIX
tools shipped with PASE.
– NOTE: Shared libraries shipped by 5733-OPS are for use by 5733-OPS only,
unless noted otherwise. They are not guaranteed to be binary compatible
and should not be built against.
• You can access them by fully qualifying the path to the program you want to
call, eg.
– > /QOpenSys/QIBM/ProdData/OPS/tools/bin/bash --version
GNU bash, version 4.3.42(1)-release (powerpc-ibm-os400)
Tools
Orion
• What is Orion?
• You can gain some initial insight by heading over to orionhub.org. As quoted from the website, the
concept is "A modern, open source software development environment that runs in the cloud." The
topic of using browser-based development tools has been of great interest to me the past few
years. Six years ago, I would have said, "There’s no way I can use a browser-based development
environment." However, my perception has changed a lot. To put it into perspective, how much time
did you spend updating SEU back in the day? Exactly! It was installed once for all developers.
Browser-based tools afford us similar efficiencies. The kicker: SEU is a server-side editor. So we've
come full circle and now have a new server-side (browser-based) source code editor. Its name is
Orion.
• Orion goes further than just an editor. It wasn't created for IBM i. Rather, it’s existed for a number of
years. Orion was developed by the Eclipse Foundation, among a few other browser-based
development environments.
• It’s important to understand that Orion does not replace RDi. RDi is very feature-rich and very "IBM
i aware." Orion not so much. In some ways, that's OK, although I do hope it grows in IBM i
capabilities. Check out the Orion docs to learn about the full set of features.
Orion
How to install Orion
1.Ensure that a valid Java is installed. Orion requires Java 7 or newer.
2.Install 5733OPS option 8.
3.Install PTF SI63852 or latest superceded
How to start/stop Orion
To start Orion, run the following from a terminal:
/QOpenSys/QIBM/ProdData/OPS/Orion/orion
To stop Orion, run the following from a terminal:
/QOpenSys/QIBM/ProdData/OPS/Orion/stopOrion
How to access Orion
Orion is a web application. By default it runs on port 2025.
Access orion by accessing http://<hostname>:2025 in your web browser
Orion user ID's
Orion does not use IBM i user profile authentication. Instead, users can create their own user ID's for
Orion. Orion can also be configured to use Github or Google logins via OAuth authentication
Orion itself always runs under the QTMHHTTP user ID.
Orion
• New! Orion 13 update - March 2017
– Our work to provide IBM i language enhancements to Orion continues with the
release of PTF SI63852. This PTF is built using Orion 13 as a base and adds
syntax highlighting support for CL, COBOL, DDS, and UIM. Additionally, the freeform RPG support that was introduced in Orion 11 now includes the ON-EXIT
opcode and the MAX and MIN built-in functions from the latest updates to the
RPG language.
Orion – initial page before sign in or register
Orion – initial page after register
Orion – first node.js program
Orion
Download GitHub project
Orion
Download GitHub project - continuing
Orion
How to administer Orion
Lots of information is available at the Orion/Server admin guide at eclipse.org.
The admin guide references orion.ini and orion.conf files, which are used for many of the Orion
admin tasks. These files are located in the following locations:
•/QOpenSys/QIBM/UserData/OPS/Orion/orion.ini
•/QOpenSys/QIBM/UserData/OPS/Orion/orion.conf
Preconfigured links directory
Orion supports the ability to directly access files in the filesystem rather than through the user's
workspace. This ability is called "Link to Server" and is available from the "File->New" actions.
The PTF's have created the /QOpenSys/etc/orion/links directory, and the orion.conf file has been
preconfigured to allow this directory to be used for the "Link to Server" feature via the
orion.file.allowedPaths property
By default, the /QOpenSys/etc/orion/links has permissions of 777, meaning it is
readable/writable/modifiable by any user on the system. Before using this directory, we
recommend that you restrict the permissions based on your needs. Since Orion always runs under
the QTMHHTTP user profile, so this directory must be accessible by QTMHHTTP!
Cloud-init
Cloud-init is a script that enables virtual machine activation and initialization, and is widely used for OpenStack.
Before you capture a virtual machine, install the cloud-init initialization package.
1. Download and install the required license programs at http://www-304.ibm.com/servers/eserver/ess/index.wss:
• 5733OPS with option base, 2, 3, 4 and 9.
 Cloud-init is packaged in option 9 of 5733OPS
 Option 2, 3 and 4 provide the environment to run cloud-init on IBM i
• 5733SC1 with *ALL option
• 5770DG1 with *ALL option
2. Install all required PTFs, as listed in the blog Install and enable cloud-init on IBM i.
3. Install the dependent cloud-init RPMs
• Use call qp2term to get into the PASE environment.
• Run cd /QOpenSys/QIBM/ProdData/OPS/cloudinit/install/.
• If the IBM® i virtual machine has an internet connection, and you would prefer to install the dependencies from
the internet, then use the following command to download and install the RPMs:
cloud-init-setup.sh -download_and_install.
• If you want to download the dependencies from the internet into the
/QOpenSys/QIBM/ProdData/OPS/cloudinit/install/ directory without installing them, use the following command:
cloud-init-setup.sh -download_only
• If you have the dependent cloud-init RPMs in the /QOpenSys/QIBM/ProdData/OPS/cloudinit/install/ directory, use
this command to install the RPMs from the local IBM i virtual machine:
cloud-init-setup.sh -install_only
• If the IBM i virtual machine does not have access to the internet, download the dependencies from other systems
in the /QOpenSys/QIBM/ProdData/OPS/cloudinit/install/pkg_cloud-init-0.7.5.lst. directory. Then, upload the RPMs
into the /QOpenSys/QIBM/ProdData/OPS/cloudinit/install/ directory of your IBM i virtual machine and run
cloud-init-setup.sh -install_only.
Cloud-init
4. Capture a virtual machine.
• In order to capture a virtual machine, you must enable the cloud-init that is installed on the IBM i virtual machine.
Run CALL PGM(QSYS/QAENGCHG) PARM(*ENABLECI) on the virtual machine. After you enable the cloud-init
on the server, power off the virtual machine, and it will be ready for capture.
Note:
This command must be run with a user profile that has the following access privileges:
*ALLOBJ
*AUDIT
*IOSYSCFG
*JOBCTL
*SAVSYS
*SECADM
*SERVICE
*SPLCTL
NGINX
• NGINX is a free, open-source, high-performance HTTP server and reverse proxy, as
well as an IMAP/POP3 proxy server. NGINX is known for its high performance,
stability, rich feature set, simple configuration, and low resource consumption.
Děkuji za pozornost