New Mexico State University
Information and Communication Technology (ICT)
Standard Operating Procedure
for implementation in:
UC4 Applications Manager
(Formerly APPWORX)
Version 4.1
1
1.0 What is UC4 Application Manager? ............................................................................. 3
1.1 UC4: A Process Improvement Tool ................................................................................... 3
2.0 Why Use UC4? ................................................................................................................... 4
2.1 UC4 is Object-Oriented ...................................................................................................... 4
3.0 Naming Guidelines ........................................................................................................... 5
3.1 Naming Convention............................................................................................................ 5
3.2 Naming Standard Examples ............................................................................................... 6
4.0 Notifications ....................................................................................................................... 7
4.1 Notification Details ............................................................................................................ 8
5.0 Job/Process flow Development .................................................................................... 8
5.1 Design Structure Implementation ....................................................................................... 9
5.2 Definition and Terms .......................................................................................................... 9
5.3 UC4 Object Implementation Document ........................................................................... 10
5.4 Jobs .................................................................................................................................. 14
5.5 Process flows (Job Steams) .............................................................................................. 14
5.6 Error Handling in Jobs...................................................................................................... 15
5.7 Notes within Jobs/Process flows ...................................................................................... 16
6.0 Job/Process flowTesting .............................................................................................. 16
6.1 Summary of Testing Deliverables .................................................................................... 17
7.0 UC4 Request Handling .................................................................................................. 17
8.0 UC4 Security..................................................................................................................... 18
8.1 Requests for UC4 Access ................................................................................................. 19
9.0 UC4 Queue’s..................................................................................................................... 19
2
1.0 What is UC4 Application Manager?
UC4 is a powerful application job scheduling tool that meets the needs of operators,
programmers, and system administrators throughout the life cycle of an application. UC4
allows operators to submit jobs on an ad hoc basis, view the output online, and print the output
to a system printer or a local Windows printer. UC4 provides programmers the tools to set up
sophisticated job scheduling without writing scripts. Instead, users can create logical
conditional statements with a few mouse clicks. System administrators will find UC4 roles and
security are powerful tools for managing access to UC4.
1.1 UC4: A Process Improvement Tool
UC4 software automates enterprise scheduling, accelerates business processes, and simplifies
application management. When fully implemented, UC4 can reduce IT operating expenses,
speed the development process, and improve processes.
Reduce Operating Expenses
UC4 includes features that enable ICT to approach full automation of operations. These
features include predecessors, conditions, and substitution variables. Underlying these features
is the ability of UC4 to make scheduling decisions based on queries of NMSU databases or
scans of output. UC4 is a “smart” scheduler.
Speed Development
In a traditional development environment, scripts drive operations. Scripts incorporate the
information required to run one or more programs on a set schedule, direct output, and handle
exceptions. The problem with scripts is the time required to write and maintain them. For
example, if a printer definition changes, it must be changed in every script.
UC4 takes the individual components of a script such as programs, schedules, printers, and
variables, and allows the definition of discrete objects. These objects can then be combined
into an unlimited number of combinations, to handle any operation. The advantage of this is it
allows a change to an object in one place, to roll over to every other use of the object.
3
2.0 Why Use UC4?
There are a number of benefits to automating complex business processes with UC4, including
the following:





Creating sophisticated job streams without writing complex scripts.
Pinpointing problems in a job stream, and restarting a job stream at any point of failure.
Balancing system load using UC4 schedules, queues, and agent groups.
Running UC4 on Unix and Windows platforms.
Controlling job stream execution based on the state of the database with dynamic
substitution variables.
2.1 UC4 is Object-Oriented
In a traditional operations environment, systems personnel created shell scripts to control batch
processing of background tasks. If system administrators were somewhat sophisticated,
structured scripts would be created that reuse some standard information. Structured scripts
however, can only be taken so far, and required significant time to maintain.
With UC4, a variety of objects are used to accomplish work UC4 takes the structured approach
to its logical conclusion: object-oriented operations. Each element in the system is defined as
an object. Once defined, UC4 objects can be combined in an infinite number of combinations
to accomplish operational tasks.
UC4 objects include:
Jobs
Process flows
Roles
Applications
Calendars
Role Authorities
Program Types
Queues
Users
Libraries
Thread schedules
Logins
Printers
Data types
Agents
Printer groups
Substitution variables
Agent groups
Spoolers
Objects are combined to define jobs. Jobs are combined with other objects to create process
flows that run batch processes. All of this is accomplished without the use of scripts.
4
3.0 Naming Guidelines
The following naming guidelines have been compiled in order to maintain control of work
procedures. These guidelines define the subtle details that make the difference between success
and failure of maintaining UC4 within ICT and NMSU.
3.1 Naming Convention
A comprehensive set of naming convention guidelines for UC4 objects, particularly jobs,
process flows, and aliases for process flow components are provided below.
The most efficient naming of jobs/process flows used in UC4 will be kept clean, (i.e. simple,
short and familiar) manageable, and comprehensive.
The following naming conventions will be utilized in UC4 for all objects (i.e. Application,
Project, Queue, Notifications, Output Scans, Job or Process flow…etc):
1. The first character group will serve as a qualifier for the primary data custodian of each
Application used in UC4, These will be tied to the UNIX directory they will reside in
for purposes of continuity.
The following will be utilized as the first level qualifier:
ADM – Admissions
ADV – Advancement
APWX - UC4
AR – Accounts Receivable
COG – Cognos
DEMO – Generic qualifier for demonstration/training purposes
FIN – Finance
FINAID – Financial Aid
FMS – Facilities Management System
HR – Human Resources/Payroll
LMS – Learning Management System
NMSU – Utility/Generic Programs
ODS – Operational Data Storage
STU – Student
5
2. The next five (5-9) characters will provide a more descriptive name for the project/task
they are associated with.
The following are examples of different type descriptions.
ADSTR – Ad-Astra
AP – Accounts Payable
COA – Chart of Accounts
DLY – Daily Job/Process flow
ERB – Educational Retirement Board
MTHLY – Monthly Job/Process flow
PAY – Payroll Specific Jobs/Process flows
SPA – Special Projects Administration
TK20 – College of Education TK20 Project
W2 – W2 specific jobs/process flows
WKLY – Weekly Job/Process flow
3. The following characters (maximum of 30 chars total) can describe the job/process
flow to a more detailed level if necessary.
3.2 Naming Standard Examples
UC4 Object
Application
Queue
Library
Jobs
Script called by job
Process flows
Schedules
Notifications
Substitution Variables
Calendars
Naming Standard
Application Acronym
Application Acronym
Application Acronym

Each application shall
have its own sub
directories under the
script directory
Application Acronym as a Prefix
Should be named the same as the
job calling it
Application Acronym as a Prefix
ending with PROCESS
Same name as Job or Process flow
the object are scheduling
Application Acronym followed by
NOTIFICATION

Application level
notification assigned to
the UC4 Application
Object
Same as Object they are associated
with
Application Global Calendar
6
Example
STU
STU
STU_TK20
STU_TK20_CODES_EXTRACT
STU_TK20_CODES_EXTRACT.sh or
STU_TK20_CODES_EXTRACT.sql
STU_TK20_DATA_EXTRACT_BY_ID_PROCESS
STU_TK20_NOTIFICATION
#stu_current_semester
STU_CAL_NAME
4.0 Notifications
The UC4 notification object can alert a client to several different conditions of a job run. It can
send a notification either the first time or every time the following status of a job/process flow
occurs:
Finished
The job goes into a FINISHED status.
Deleted
The job is deleted from the Backlog.
Aborted
Any failure status such as ABORTED or TIMED OUT.
Hold
The job is put into a HOLD status.
Running
The job goes into a RUNNING status.
Completed
The job completes without regard to status.
Queued
The job goes into QUEUED status.
Requested
Restart
Finished
The job is requested or run with AWRUN.
The job was restarted and has finished with a decimal value Job ID rather than
an integer.
Notification is a valuable tool in UC4 when used properly. It can help any “lights out”
organization avoid delays in resolving job issues especially when the job/process flow are run
by UC4, but are essential to someone outside ICT. For example, the completion status of a
particular job/process flow may be vital to the functionality of NMSU.
All types of notification are handled by the UC4 notification feature. Clients or On-Call
personnel can receive notifications by email, page, or any other type of device.
Notifications are defined as objects in UC4 then assigned to jobs, applications, and program
types.
By using the available variables in the messages, the notification object can apply to any job.
After a notifications object has been defined, it can be assigned to jobs, applications, and
program types. If the object is assigned to an application, the notification will apply to any job
assigned to that application. The same holds true if the object is assigned to a program type.
The ability to assign notification objects to applications and program types makes it easy to set
up notifications for an entire class of jobs.
7
4.1 Notification Details
Multiple details can be created for each notification. It is recommended that the use of
Departmental Listservs be utilized instead of individual email addresses. The use of Listservs
allows for more efficient maintenance. As staff changes, no changes will be needed in UC4 as
changes will be made at the Listserv level.
5.0 Job/Process flow Development
The UC4 application philosophy ensures that development, testing, and production occur
simultaneously while minimizing downtime to our mission-critical production instance. The
UC4 methodology, when used effectively, ensures that modifications and enhancements occur
quickly and safely in the NMSU application environment.
ICT realizes the importance of compartmentalizing development, testing, and production
environments. The key mission of all database applications is to get them up and running and
keep them running. Yet in attaining the goal, ICT must contend with constantly changing
requirements and requests to improve database performance, flexibility, and functionality. ICT
at NMSU must keep enterprise applications running 24/7, thus the need for a solution that
ensures modifications and enhancements occur quickly and safely in our production instance.
The UC4 methodology provides this solution.
Development
Testing
Production
Although this methodology may appear obvious, this is often not the case for NMSU. The
three-tiered approach encourages NMSU to maintain separate development, testing, and
production instances. The advantages of this approach are many:
 Separating production from development/ testing decreases the chance of an untested
program going awry and tying up mission-critical resources and personnel.
 Most decisions can be made ‘up front’ with already established requirements and goals.
 All applications are managed to a single design standard. Cross-platform, crossapplication, and cross-phase conflicts are avoided since any change made in one
instance can be passed onto another.
8
In other words, no matter where in the process a developer may be, he/she can anticipate and
develop a specific component independently and in advance of other components. Many
organizations take a system-level approach, where each database instance is administered
independently. Incompatibilities arise when developers design for one instance only to
discover that the production instance was modified to accommodate the request of yet another
developer. Since each phase is ‘blind’ to each other, decisions in one instance, can directly
conflict with requirements in another instance.
UC4 addresses these concerns through its application design philosophy. Export and Import
Managers (ICT_USA) are the functional managers to make this happen.
5.1 Design Structure Implementation
NMSU has two UC4 instances: UC48DEV and UC48PRD. Developers create/test scripts in
their own libraries then have them moved into UC48DEV libraries for QA Testing. Developers
will use job NMSU_UC4_LIBRARY_COPY to accomplish this. The document can be used to
request ict_usa do the copy in pre-approved situations.
If modifying existing objects, the developer can request a migration of these objects to
UC48DEV from UC48PRD to ensure a clean copy of the production version.
After testing, all effected objects will be migrated to UC48PRD from UC48DEV.
In any instance, all objects will adhere to these practices. Also, when requesting that an object
be moved into the development/production environment, it must be accompanied with the
proper documents. No request for a move to either environment will be accepted without them.
Also, migration document from UC48DEV to UC48PRD must contain the job ID and
timestamp of the successful test run. The test run needs to be within 7 days of the request as
most output expires after 7 days.
5.2 Definition and Terms
Developer
Refers to process developer. Responsible for creating new jobs and programs.
Refers to quality assurance personnel (i.e. ICT_EIS, ICT_RS). Responsible for
QA/Testing
testing new jobs or diagnosing existing jobs.
Refers to System or Database Administrator (i.e. ICT_USA, ICT_DBA).
Responsible for maintaining and implementing QA and production databases
Sys. Admin
with jobs and programs supplied by the developer.
The location of the UC4 repository. NOTE: It is possible for UC4 to reside in
UC4 Instance one location, but run jobs on one or more remote databases.
Refers to your existing instance of UC4 – the one with objects you want to
Source
move.
Target
Refers to the new instance you created – the one you want to receive objects.
9
5.3 UC4 Object Implementation Document
PRODUCT NAME = <job abbr>_<product name>
Author: Name of Developer
Department: Department Name
Version: <version number >
Last modified: 10/26/2005
Information and communication technologies
10
Project Overview
Client Name: Name of office/person that commissioned the project (Use ICT if internal)
Description: High level description of the product. Usually cut/paste from project request
Product files
Located at: /users/username/job/<product_name>
Filename
Description
<product_name>_<action>.shl Shell script to be run by UC4. This script will contain
description, version and parameter information within
itself as comments.
<product_name>_<action>.sql script to be run by UC4. This script will contain
or src
description , version and parameter information within
itself as comments.
Project output files
Located at: Oracle directory path name OR $AWHOME/out/<product_name>
Filename
Description
<product_name>_<output_name>. Extract files from the above programs
<dat | csv etc>
UC4 job information
Job information
Job X
Name
Description
<product_name>
<job name>
Job description for the use of the developers in the future
Application
Library
Program Type
Description
Program
Name
Program
Location
<UC4_JOB>
AW_NMSU leave unchanged if not sure
AWEXECS leave unchanged if not sure
Common shell programs leave unchanged if not sure
<product_name>_<action>.shl
Parameters
Name
Default Value
Type
<$var_name>
<default value>
UC4
<$var_name2>
<default value>
Scheduled
Provide explanation of any of the above if required
Login Tab
General Tab
User from dropdown list (this will be the login ID used on UC4)
Execution Options for Queue groups and Notification e-mail groups
$AWHOME/NMSU/exec/<product_name>
$AWHOME/sql/<product_name>
11
Process flow Information
Process flow
Name
Description
Jobs
<NAME OF PROCESS FLOW>
Description, detailed enough for person running it to understand
<List all jobs and the order in which they are to be run>
Scheduling (REQUIRED)
Name
Type
Users
Schedule
<NAME OF JOB/PROCESS FLOW>
Manual or schedules
Name of department/ people
Time:
Recurrence: daily/weekly/weekdays/monthly – specify if process is
to run or not run on holidays or weekends
Special cases:
Dependency
Dependant jobs/Process flows
Update to Development with version number (REQUIRED unless developer
moves their own scripts via job NMSU_UC4_LIBRARY_COPY)
Installation to
Development
UC4 development instance
Bulleted installation steps
Update Migration to Production with version number (REQUIRED)
Installation to
Production
UC4 Job ID &
Timestamp
UC4 Production instance
UC4 ID number and date when successfully tested (must be within the
last 7 days)
Bulleted installation steps including process flow
information
Clean-up (REQUIRED)
Instructions for flagging old jobs/process flows as inactive, which are being replaced by
this job/process flow.
12
Notes (REQUIRED)
Cut and paste the following into the notes section for the job/process flow
Instructions for error recovery.
Urgency/ contact times
Contact information
Email address:
Time to call:
13
5.4 Jobs
Jobs-are the basic building block in UC4. For each program that needs to run (for example:
FTP, database load), a job must be created. A job contains all the information required to
execute a program or script on the server and handle its output. When a job is created, it will
specify the program location, input and output parameters. Jobs are run both individually and
as components of UC4 process flows. Furthermore, a job can be a component of any number
of process flows. If a job definition is changed, the change is applied to every process flow that
includes it.
For purposes of continuity the following will apply to UC4 at NMSU:







Design jobs where possible for generic use (i.e. FTP/SFTP/SCP, Move/Copy,
Zip/Compress, Data Loading, and emailing)
Try to solve a larger need with the least amount of coding
Standard solutions for common needs save time and effort, both in development and
support.
No output/log files are to be placed in UC4 directories EXCEPT for $AW_HOME/out.
All files in this directory are deleted at the age of 15 days.
If files are stored outside of UC4, it will be the responsibility of the developer for
cleanup/rotation.
In jobs and scripts, NO hard coding of libraries, logins, or parameters will be allowed.
The script should be named the same as the job calling it. In cases where multiple
scripts are called from the main script, all related scripts have to have the same
Application Acronym prefix.
5.5 Process flows (Job Steams)
Jobs are combined to create process flows. Process flows are equivalent to job streams and run
any number of jobs. Process flows include scheduling and exception handling information.
When jobs and process flows are added to a process flow, these objects are referred to as
‘process flow components’. The job limit in UC4 for any process flow is 999.
Again, for the sake of continuity in the implementation of process flows in UC4, following
these simple rules will guarantee that NMSU utilizes all that this tool offers.
 Design process flows to be dynamic.
 Build Sub-Routines around common tasks that are linked together.
 Use of Numeric, Dynamic, and Static Substitution Variables.
 In process flows and scripts, NO hard coding of libraries, logins, or parameters will be
allowed.
14
5.6 Error Handling in Jobs
Behind every program type object is a program type script. The program type script performs
all the main work of running the program specified in a job definition. Specifically, the
program type script accomplishes one or more of the following six tasks:






Program execution
Parameter passing
Error determination
Output registration
Debug/administration
Termination
For UC4 to function as the tool it can be, it is imperative that all jobs incorporate error
handling. Developers must ensure custom code exits with an error code in an abend/abort
situation. Instances and each object will need evaluation for the proper function. If programs
do not exit with an error, UC4 assumes the process completed successfully and will continue to
process the process flow/successor.
Here is an example of SQL for error handling in a script:
WHENEVER SQLERROR EXIT sql.sqlcode
OR
err=$?
Exit $err
The code err=$? traps the exit status of the last command executed.
OR
if errorlevel 1 set err=%errorlevel%
The code err=%errorlevel% traps the exit status of the last command executed.
15
5.7 Notes within Jobs/Process flows
UC4 provides a Notes area for descriptive instructions for two types of customizable notes:
General and Abort.


General notes can contain information on goals and requirements, or existing security
and access issues.
Abort notes can contain information on what action to take if a job aborts or fails, who
to contact if a job aborts, or what considerations exist when running a job.
There are three locations within UC4 where notes can be entered. Suggested uses for notes are
described in the table below:
Note Type
Job
Process flow
Process flow
Component
Description
Useful when the job is going to be requested ad hoc, or when the notes for
the job would be useful-regardless of how the job was invoked.
Can be used to provide information which is relevant to the entire process
flow.
Can be used to provide specific information about one component in a
process flow.
Users can view job, process flow, and process flow component notes from the Backlog,
History, and Submit window. If a job aborts, an operator can view the specific abort notes and
make more effective operational decisions.
ICT would like to encourage the use of notes in both the initial creation and as a change control
tool in both jobs and process flows. This will benefit Operations and On-Call personnel in the
proper handling of aborts. As a change control tool for example, this can be used to detail
history of any changes to UC4 objects. It would also benefit clients/programmers in future
maintenance.
6.0 Job/Process Flow Testing




Testing in UC4Dev is mandatory. No objects will be moved from development to
production, without a full test in the UC4 development libraries.
Tests will not be considered valid using scripts from personal libraries.
The sign off mechanism that will be utilized at ICT is incorporated into the UC4 Object
Implementation Document, referred to previously. This includes, but is not limited to;
subvar, parameters, login changes….etc.
When requesting that an object be moved into the development/production
environment, it must be accompanied with the proper documents. No request for a
move to either environment will be accepted without them.
16



The migration document from UC4Dev to UC4Prd must contain the job ID of the
successful test run. Test run must be within the last 7 days (output only stays on the
server 7 days by default).
If the object fails user testing, it will be returned to the developer for changes.
If the object passes testing, it will be imported into the production instance.
6.1 Summary of Testing Deliverables






Developer request from ICT_USA export of existing objects from UC4Prd to UC4Dev
using the UC4 Object Implementation Document (UOID).
Developer tests scripts in own library.
Developer requests from ICT_USA movement of script into UC4 Devl library using
AOID or moves scripts themselves using the provided job
(NMSU_UC4_LIBRARY_COPY).
User tests objects in UC4Dev and signs off with developer.
Developer records jobs number of successful test in UOID and requests from ICT_USA
export of effected objects into UC4Prd.
ICT_USA forwards UOID to UCC for scheduling/parameter changes.
7.0 UC4 Request Handling





Any/All UC4 change requests will have a cut-off time of 3:30pm Monday thru Friday to
ensure inclusion in that evening’s run.
Anything received after the 3:30pm cut-off time will be guaranteed to be completed by
the end of next business day.
Exception for TRUE emergencies will be validated by an ICT Director. An emergency
change must be documented and a request to move the change from production to
development must be received by ICT within 3 workdays.
Requests for any update to UC4 must be accompanied with an UC4 Object
Implementation Document. Any requests submitted without this standard form will be
refused. This document has been incorporated into the standard practices and an
example is previously included in this document and can be provided either hard copy
or via email.
This includes request to both ICT_USA and ICT_UCC.
17
8.0 UC4 Security
Security is always a major concern with client/server programs. UC4 allows us to control
security on three levels.



Access to UC4 is controlled with user logins.
Access to UC4 functions is controlled with roles.
Access to databases and hosts is controlled with logins.
Using the three security levels, ICT can fine tune UC4 to give different groups access to only
the areas of UC4 a client may need.
When ICT adds a user to UC4, his/her user name, password, login group, maintenance role, and
default output directory is defined. ICT can assign UC4 specific options such as View All User
Outputs for the output window.
ICT can control access to UC4 windows and objects using roles. Think of roles as containers
to which ICT can add objects and users. Users have access to all the objects within that
container. The number of objects and users that can be assigned to a role is unlimited.
Objects and users can be assigned to more than one role. Users have access to these objects
based on the roles assigned. ICT can further restrict access by designating Edit authorization to
some roles and not to others.
Developer Access.
 Developers will have modify access in the development instance (UC48DEV) to all
objects except for system objects (agents, system utilities, security, etc).
 Developers will have full read access to the production instance to all objects except for
system objects.
 Developers will not have modify access to any production instance objects.
 Developers will not be able to submit jobs or processes in the production instance. This
is an end-client function for objects not scheduled.
 Developers will have a Maintenance User Group of ict_interface_edit so any object they
create will be a member of that user group (development instance only).
Sysadmin Access.

Full modify access to any UC4 instance will be restricted to staff of UCC and USA.
End Client Access.

End Clients should only have read access to those objects they need to request (dev and
prod). Ex: only the process flow, and not the individual jobs
18
8.1 Requests for UC4 Access
At NMSU, it is recommended that only those individuals who are allowed to request Banner
security shall also be designated as authorized to request UC4 security (i.e. SIM, ADM). Each
designated group (i.e. adm, uar, sar, finaid, fin) shall designate a contact person within that
group. All requests for an UC4 login and role authorities will be submitted to
[email protected] for evaluation and processing.
NMSU Policy 2.35.1.1.3 states “Employee access to institutional data is revoked
immediately upon separation”, which is compliant with data privacy regulations (FERPA,
HIPAA, GLBA, FISMA, The Red Flags Rule and PCI DSS) and best information security
practices.
Security Admin will adhere to the above mentioned policy. Using a report generated daily,
employees who have terminated or moved from active employee status to retiree will be
revoked on that day.
Users transferring to a new department are not considered modifications but new accounts.
When a user is leaving a department, regardless of the reason, Security Admin will identify
these through report run daily. The user’s account will be revoked after their last day in the
department. The user’s new department will need to submit a new request for UC4 system
access.
9.0 UC4 Queue’s
Controlling the load on the NMSU system is critical. In UC4, ICT can control the workload by
setting the number of concurrent jobs that can pass through a queue. Queues control the flow
of jobs. All jobs must pass through an UC4 queue to be executed. ICT_UCC can control queue
throughput by assigning a queue to different thread schedules and priorities. UCC can also
inactivate one or more queues. An unlimited number of queues can be defined.
Thread schedules control the number of concurrent jobs that can run through a queue. When
ICT defines a thread schedule, it includes the number of threads, the minimum threads, and the
start and stop times for the schedule. A thread schedule can also be divided into sub schedules,
allowing a change to the number of threads for different times of the day.
If a queue is inactive, UC4 will still submit scheduled jobs to the queue but those jobs will have
a status of QUEUE WAIT. When the queue is reactivated, UC4 will process those jobs based
on priority settings.
19
Queues are being developed and implemented by ICT_UCC for use in controlling
downtime/maintenance and controlling the number of jobs run simultaneously. This is useful
to restrict jobs from running during system intense times (i.e. grading), or when upgrades or
maintenance is being performed. Queues are also a powerful tool to limit jobs/process flows
that are resource intense to run only during certain times of the workday.
Again, for continuity, NMSU will utilize the naming convention standards as defined in section
3.0 of this document.
20