Design Guide How to use Web Services with NMSDK Amir Mukeri July 17, 2012 Table of Contents 1 INTRODUCTION ......................................................................................................................... 3 1.1 WEB SERVICES ...................................................................................................................................... 3 1.2 RELEASES ............................................................................................................................................... 3 1.3 SCOPE OF THIS DOCUMENT ................................................................................................................. 3 2 BUSINESS STATEMENT ............................................................................................................ 4 3 WEB SERVICES AND NMSDK .................................................................................................. 5 3.1 OVERVIEW ............................................................................................................................................. 5 3.2 USE CASES FOR THE WEB SERVICES APIS ....................................................................................... 5 3.2.1 Orchestration Tool ......................................................................................................................... 5 3.2.2 Custom programmatic bindings .............................................................................................. 6 3.2.3 Existing ZAPI client ........................................................................................................................ 6 3.2.4 Unsupported API ............................................................................................................................. 6 3.3 WEB SERVICES SUPPORT IN NMSDK ............................................................................................... 6 3.4 LANGUAGES SUPPORTED BY NMSDK ................................ ERROR! BOOKMARK NOT DEFINED. 3.5 WSDL IN NMSDK ............................................................................................................................... 7 3.5.1 Version Matrix .................................................................................................................................. 7 3.5.2 Example вЂ“ To generate Java stubs on Windows using Apache CXF ........................ 7 3.6 SOAP ...................................................................................................................................................... 8 3.7 AUTHENTICATION ................................................................................................................................ 8 3.8 ITERATIVE APIS .................................................................................................................................... 9 3.9 OVER-THE-WIRE REQUEST/RESPONSE............................................................................................ 9 3.9.1 Request вЂ“ SOAP ................................................................................................................................ 9 3.9.2 Response вЂ“ SOAP ............................................................................................................................. 9 3.10 SAMPLE CODES ................................................................................................................................ 10 3.10.1 HelloDfm.java .............................................................................................................................. 12 3.10.2 StorageService.java ...................................................................................................................20 2 1 Introduction 1.1 Web Services A Web Service, as defined by the W3C, is a software system designed to support inter-operable machine-to-machine interaction over a network. It is a standardized way of integrating web-based applications using XML messages following the SOAP, using WSDL to describe the service and UDDI to enlist available services. Web Services allow different applications from different sources to communicate with each other without time-consuming custom coding. Since all communication is in XML, Web Services are platform independent and thus are not tied to any one operating system or programming language. п‚· SOAP (Simple Object Access Protocol) is a lightweight XML-based messaging protocol used to encode the information in Web Service request and response messages before sending them over a network. SOAP messages are independent of any operating system and may be transported using a variety of Internet protocols including HTTP, SMTP and MIME. п‚· UDDI (Universal Description, Discovery and Integration) is a web-based distributed directory that enables businesses to list themselves on the Internet and discover each other, similar to a traditional phone book. п‚· WSDL (Web Services Description Language) is an XML-formatted language used to describe a Web Service's capabilities as collections of communication endpoints capable of exchanging messages. WSDL is the language that UDDI uses. 1.2 Releases DataFabric Manager 4.0 onwards provides support for the Web Services APIs. The NMSDK bundles the OnCommand Web Services APIs in the form of a WSDL file that you can use with any third-party management tool, orchestration tool, or customer-created tools. You can use the WSDL to develop applications. To obtain further help to develop Web Service applications, see the Web Services sample codes. 1.3 Scope of this document By the end of this document, you should be able to: п‚· Locate WSDL file in NMSDK and extract stubs from WSDL file п‚· Locate the Web Services sample codes, and understand the Web Services sample codes HelloDfm and StorageService. 3 2 Business Statement NetApp exposes APIs in the form of Zephyr APIs, which is NetApp proprietary standard and thus requires a basic level of learning for any developer to start his development efforts. Also many third party tools look to integrate with NetApp using industry standard open interfaces. With big TelcoвЂ™s and Service Providers embracing cloud/automation, the need for a standard open interface has been a big ask. To cater to the above needs NetApp provides industry standard SOAP based support through NMSDK. One can find an equivalent Web Service API for all DataFabric Manager server APIs for OnCommand Core package. NMSDK also exposes an WSDL which can be used with any third party toolkit like gSOAP, Apache Axis, Apache CXF, Perl SOAP::Lite etc to integrate with NetApp solutions. 4 3 Web Services and NMSDK 3.1 Overview The OnCommand Web Services APIs expose all DataFabric Manager server APIs through SOAP. This enables the features like monitoring, provisioning and cloning, backup and recovery, disaster recovery, and performance monitoring to be transportprotocol-agnostic. 3.2 Use Cases for the Web Services APIs 3.2.1 Orchestration Tool If you are a storage automation administrator who would like to integrate services provided by DataFabric Manager Server into an orchestration tool, then you will want to use the SOAP protocol, because the orchestration tool has built-in SOAP support. The administrator can enable this in the environment using DataFabric Manager server Web Services APIs. п‚· The administrator imports the WSDL into the orchestration tool, and configures various tasks in a workflow to call DataFabric Manager server APIs via SOAP. п‚· The administrator must also configure a username and password to be used for authentication. п‚· The administrator executes the new workflow, and the tasks call DataFabric Manager server APIs via SOAP. 5 3.2.2 Custom programmatic bindings If you are a storage automation administrator who wants to integrate the DataFabric Manager server services into various custom scripts, the DataFabric Manager Web services APIs lets you use third-party tools to generate SOAP bindings from the WSDL to interact with the custom scripting languages. A storage automation administrator is integrating services provided by OnCommand into various custom scripts. п‚· The administrator uses a third party tool to generate SOAP bindings from the WSDL for the scripting language. п‚· The administrator modifies the scripts to use the generated bindings to send SOAP messages. п‚· The administrator must also configure a username and password to be used for authentication. п‚· The administrator executes the scripts, and the scripts call DataFabric Manager server APIs via SOAP. 3.2.3 Existing ZAPI client If you are already a NetApp customer using DataFabric Manager Server with Data ONTAP API support, then you can upgrade to the DataFabric Manager server SDK version, providing a new SOAP interface without any interruption to the current Data ONTAP API support. п‚· The customer upgrades to the DFM version 4.0 or OnCommand providing a new SOAP interface. п‚· The customer's current integration is able to continue using ZAPI. 3.2.4 Unsupported API If you are already using Web Services APIs and need functionality not provided by a supported SOAP API. п‚· The customer coordinates with the account team. The account team informs the customer that the required functionality is provided via an unsupported API. The customer is willing to try it. The account team gives the customer the full WSDL (WSDL containing all APIs regardless of status), and the customer is able to call the required unsupported API via SOAP. Note: no "upgrade" to the DataFabric Manager server server is required to use the APIs provided by the full WSDL. 3.3 Web Services support in NMSDK п‚· Web Services APIs for DataFabric Manager server that can be used with various SOAP tool-kits, such as Axis, Apache CXF etc to develop custom Web Service applications. Note: Data ONTAP currently does not support Web Services 6 п‚· WSDL file for DataFabric Manager server APIs for OnCommand Core Package. п‚· Basic and Advanced sample codes for DataFabric Manager server Web Services for OnCommand Core Package support C# and Java Apache CXF toolkit. 3.4 WSDL in NMSDK The DataFabric Manager server APIs are supported through the industry standard SOAP protocol, with the help of Web services APIs. The supported features on the DataFabric Manager server are made available to all the clients through the WSDL file containing all the available APIs. 3.4.1 Version Matrix Version Matrix (below) lists the supported DataFabric Manager APIs for OnCommand Core Package, for each version of the SDK and link to WSDL file. You can use the following tables to verify the supported API version for each version of the SDK: NetApp Manageability SDK version 4.1 4.0 1.0 DataFabric Manager server APIs for OnCommand Core Package 5.0 4.0 3.8 3.7.1 3.6R1 WSDL for DataFabric Manager server APIs for OnCommand Core Package 5.0 4.0 N/A 3.4.2 Example вЂ“ To generate Java stubs on Windows using Apache CXF Step 1: Set the environment variables: п‚· JAVA_HOME - Enter the variable value as the installation path for the Java Development Kit. 7 п‚· CXF_HOME - Enter the value as the installation path for CXF. Step 2: Run вЂњwsdl2javaвЂќ executable and provide the location of wsdl file as shown below. Step 3: Below is the output of the above command. Java stubs are generated at the location mentioned below. 3.5 SOAP SOAP is available over HTTP and HTTPS using the same ports that ZAPI uses (but different url contexts). Currently the ports are configured with the following respective dfm options (with their default values listed): п‚· п‚· HTTP Port 8088 HTTPS Port 8488 The url context will be /apis/soap/v1. So, if the default port is being used, the full endpoint for HTTPS would be https://servername:8488/apis/soap/v1. For comparison, the equivalent ZAPI endpoint is https://servername:8488/apis/XMLrequest. 3.6 Authentication The SOAP implementation will use HTTP Basic Authentication (with username/password). This is the same authentication mechanism that ZAPI uses. 8 Starting with OnCommand/DFM version 5.0 Certificate Based Authentication is also supported. 3.7 Iterative APIs Iterative APIs allow you to operate on a large set of objects by dividing that set of objects into manageable sub-sets, repeatedly executing an API call. To repeatedly call a DFM API, you can use *-iter-start, *-iter-next, and *-iter-end. 3.8 Over-the-wire Request/Response The following snippets show an example of how the same API is exposed via SOAP. 3.8.1 Request вЂ“ SOAP 3.8.2 Response вЂ“ SOAP 9 3.9 Sample Codes You can use the Web Services APIs for OnCommand Package to retrieve information about the DataFabric Manager server version, vFiler units management, usage metrics, and so on. The Web services API sample codes can be used with Web Services frameworks likeвЂ”Apache CXF and Microsoft .NET. 10 The following list describes sample codes that demonstrate the use of a few Web Services APIs. Name Description Sample code in Java HelloDfm.java Hello DataFabric Manager MultiStore management Sample code in C# HelloDfm.cs Sample code to demonstrate how to get the version number of the DataFabric Manager server by using the HTTP or HTTPS protocol. Sample code to manage vFiler units. You can Vfiler.java Vfiler.cs create and delete vFiler units; and create, list, and delete vFiler templates. Usage Metrics Sample code to demonstrate how to compute and UsageMetrics.java UsageMetrics.cs retrieve space utilization and I/O measurement for a dataset node. Storage service Sample code to demonstrate how to manage StorageService.java management storage services and provision storage. You can create a new storage service with backup or mirror protection policy, list or delete storage service, provision and de-provision storage, and resize storage. 11 - 3.9.1 HelloDfm.java HelloDfm.java is a basic sample code to retrieve the version of DataFabric Manager server using the Apache CXF Web service Framework. DataFabric Manager 4.0 server Web Services Description Language (WSDL) provides the XML prototype of the operations supported by DataFabric Manager 4.0 server. 1. The section (below) of WSDL shows the definition of the DfmAbout API operation. The DfmAbout operation description identifies the valid input and output structure of the DfmAbout Application Programming Interface (API). So, the DfmAboutRequest input message needs to be passed along with the DfmAbout API, at the time of invocation. The DfmAboutResponse message will be returned if the API invocation is successful. The DfmAbout operation is the method exposed by the server to enable the DataFabric Manager clients to retrieve the version of the server. 2. If you navigate through the help document to the Web Services API section, you can see the API definition for the DfmAbout API. The input and output parameters of the DfmAbout API are DfmAbout and DfmAboutResult. 12 The DfmAbout element has an optional IncludeDirectorySizeInfo parameter which returns the size and file count information of the DataFabric Manager directories. Note: We are not going to use this parameter in the hello dfm sample code to retrieve the dfm server version. 3. The DfmAboutResult element consists of a list of parameters about the DataFabric Manager server. We will be using the version parameter in the hello dfm sample code. 13 4. The DfmAbout XML element in the WSDL file defines the input parameter value of the DfmAbout API operation. 5. The highlighted section(below) shows the definition of the DfmAboutResult output message in the WSDL file. This creates a prototype for the output structure 14 The parameters of the output structure CpuCount, DfmDirectories, Hostname, Install Dir, and so on, are listed in the DfmAboutResult XML element tag. 6. The version tag is the string value that will be extracted from the DfmAboutResult output message. 7. This section of the WSDL file defines the input and output message structure in the DataFabric Manager server WSDL. 15 8. This portion of the WSDL file defines the service tag used by the DataFabric Manager server to define the service location of the web server. 9. Import the netapp.management package, which contains all the client side stubs. 10. Create HelloDfm class and declare DfmInterface member to invoke the DFM Web Services APIs. Declare default HTTP port as 8088 and HTTPS port as 8488 for DataFabric Manager server communication. 11. This section of the code creates variables for the command-line arguments, DataFabric Manager server IP address, user name, and password. The option of using HTTP or HTTPS as the transport protocol is also initialized in this section. 12. The PrintUsageAndExit function is highlighted. The -s switch is used for selecting HTTPS as the transport protocol and the -i switch is used to ignore the server certificate validation process. 16 13. After the user input arguments are validated, create Interface method to create a client proxy to invoke the web services APIs. 14. Frame the address for SOAP request over HTTP with the given server and port number. 15. Create an instance of the DFMService and extract the dfmInterface through getDfmPort method 16. Set the server binding properties like user name, password, and end point address for the DataFabric Manager server interface. 17. The next function call in the main method is the DfmAbout method. 18. This section shows the creation of an instance of the DfmAbout class defined in the stub file. You can invoke the DfmAbout API through the dfmInterface instance and assign the output of the API invocation to the instance of the DfmAboutResult class. 19. You can extract the version from the DfmAboutResult instance. The extracted version is printed on the console. 17 20. In the HelloDfm sample code, when the DfmAbout API is invoked, the API call is redirected to this stub code. 21. The parameters of the DfmAboutResult API are also defined in the DfmService stub class. 22. From the command prompt, CD to the sample code folder: WebServices>CXF>basic set the CXF_HOME and JAVA_HOME environment variables to the folder where apache-CXF and JDK is installed. Run MAKE.BAT to generate and compile the client side stubs. MAKE.BAT also compiles HelloDfm sample code. Run "TEST.BAT HelloDfm" to list the usage of the sample code. 23. Run TEST.BAT HelloDfm with the DataFabric Manager server IP address, administrator username, and password. This displays the version of the DataFabric Manager server. 18 24. To use the HTTPS transport protocol, run the code and include the -s switch operator. Use -i switch to ignore the server certification. 25. Sample output: 19 3.9.2 StorageService.java This sample code demonstrates how to manage storage services and provision storage using storage service. A storage service contains one or more resource pools, one or more provisioning policies and a protection policy. The service abstracts the physical resources and policies from the subscriber. 1. Import the netapp.management package, which contains all the client side stubs. 2. Create StorageService class and declare DfmInterface member to invoke the OnCommand Web Services APIs. Declare default HTTP port as 8088 for DataFabric Manager server communication. 3. Extract the DataFabric Manager server, user name and password information from the command line arguments. 4. Call the create Interface method to create a client proxy to invoke the Web Services APIs. 20 5. Frame the address for SOAP request over HTTP with the given server and port number. 6. Create an instance of the DFMService and extract the dfmInterface through getDfmPort method 7. Set the server binding properties like user name, password, and end point address for the DataFabric Manager server interface. 8. After creating the dfmInterface, parse the command line arguments given by the user. 9. createService method creates a storage service. This method requires the name of the service, name of the protection policy, provisioning policy and resource pool name attributes for primary and secondary nodes. This method will internally invoke storageServiceCreate Web Service API to create a new storage service. 10. Create an instance of StorageServiceCreate and set the name of the new service and the protection policy. 21 11. Set the node name and resource pools node attributes for the primary node. 12. After setting the node name and resource pool attributes for the primary node, set the node name and resource pools attributes for the secondary node. The secondary node name can be either a Backup node or Mirror node. 13. Invoke the storageServiceCreate API to create a storage service. This API will return StorageService Create Result instance, which contains the ID of the new storage service. 14. The listService method will list information of all storage services on the DataFabric Manager server. 22 15. Create an instance of StorageServiceListInfoIterStart and set the name of the service. 16. Invoke the storageServiceListInfoIterStart Web Service API to start the iteration to list storage services. This API will return the ServiceListInfoIterStartResult structure, which will contain the tag and record values for the storageServiceListInfoIterNext API. 17. Create an instance of the StorageServiceListInfoIterNext API and set the maximum number of records to retrieve and the tag value from the storageServiceListInfoIterStart API. 18. Invoke the storageServiceListInfoIterNext API to return a list of storage service records. 19. Iterate through each storage service information record 20. Extract the name of the storage service and protection policy. 21. Extract the information about storage service nodes from the storage service info object. 22. Iterate through each node in the storage service and list the node name, provisioning policy name and resource pool node attributes. 23 23. Extract and list the datasets attached to the storage service. 24. The provision Storage method will provision the storage with specified size and exports the storage over CIFS protocol. Provision the storage using storageServiceDatasetProvision API. This will create a new dataset with the specified storage service. Set the provisioning request with the specified name and size of the storage container while creating the new dataset. 25. Create an instance of StorageServiceDatasetProvision and set the dataset name and service name. 26. Create an instance of ProvisionMemberRequestInfo API and set the storage 24 container name and the size attributes. 27. Set the provision member request info. 28. Create an instance of DatasetExportInfo API and set the export protocol to CIFS. Set the CIFS share permissions to "full control" for "everyone" 29. Invoke the storageServiceDatasetProvision API request to provision storage. This API will return the Provision Result structure, which contains the ID of the newly provisioned Dataset. 30. From the command prompt, CD to the sample code folder: WebServices>CXF>advanced set the CXF_HOME and JAVA_HOME environment variables to the folder where apache-CXFand JDK is installed. Run MAKE.BAT to generate and compile the client side stubs. Run "TEST.BAT StorageService" to list the usage of the sample code. 25 31. Create a Gold storage service with mirror protection policy. 32. List the storage services and the details of the protection policy and node attributes. 33. Provision 100 GB of storage for finance department under gold storage service. 26