Redmond Protocols Plugfest 2016
Enhanced User Experience For Protocol Test Suites
Vivian TIAN
Software Engineer
Agenda
• Protocol Test Manager
• Super Logging with MMA
• MMA parser update to Windows protocols
• Q&A
Protocol Test Manager
Protocol Test Manager
• Background
• Partner number increases year by year
• Partners show interests to run test suite during product development phase
• Introduction
• Provide a step-by-step wizard for protocol test suite run.
• Support environment auto detection, configuration, case run and case run result investigation.
Protocol Test Manager
• PTM Capability
• Guide: Embedded deployment guide illustration
Protocol Test Manager
• PTM Capability
• Guide: Embedded deployment guide illustration
• Detect: SUT capability auto detection
Protocol Test Manager
• PTM Capability
• Guide: Embedded deployment guide illustration
• Detect: SUT capability auto detection
• Filter: Cases filtered by features and SUT capability
Protocol Test Manager
• PTM Capability
• Guide: Embedded deployment guide illustration
• Detect: SUT capability auto detection
• Filter: Cases filtered by features and SUT capability
• Configure: GUI Test suite configuration
Protocol Test Manager
• PTM Capability
• Guide: Embedded deployment guide illustration
• Detect: SUT capability auto detection
• Filter: Cases filtered by features and SUT capability
• Configure: GUI Test suite configuration
• Run: GUI test case run w/o VS dependency
Protocol Test Manager
• PTM Capability
• Guide: Embedded deployment guide illustration
• Detect: SUT capability auto detection
• Filter: Cases filtered by features and SUT capability
• Configure: GUI Test suite configuration
• Run: GUI test case run w/o VS dependency
• Investigate: Integrated test run logs
Protocol Test Manager
• PTM Capability
• Guide: Embedded deployment guide illustration
• Detect: SUT capability auto detection
• Filter: Cases filtered by features and SUT capability
• Configure: Test suite configuration recommendation
• Run: GUI test case run w/o VS dependency
• Investigate: Integrated test run logs
• Rerun: Rerun and profile supported
Protocol Test Manager
• PTM Capability
ptmcli.exe <-p|-profile profileName> [-s|-selected] [-r|-report reportfile]
[-outcome pass,fail,inconclusive,notrun] [-sortby name|outcome]
[-separator comma|space]
• Guide: Embedded deployment guide illustration
• Detect: SUT capability auto detection
• Filter: Cases filtered by features and SUT capability
-p | -profile profileName Specifies the file name of profile of the test
suite to run.
-s | -selected
• Configure: Test suite configuration recommendation
Turn this switch on to run only the test cases
selected in the Run Page of PTM. Otherwise, run
all the test cases in the profile.
• Run: GUI test case run w/o VS dependency
-r | -report reportfile
• Investigate: Integrated test run logs
• Rerun: Rerun and profile supported
-outcome
Specifies the outcome of the test cases to be
included in the plain text report.
Saparate using comma without space.
Possible values: pass,fail,inconclusive,notrun
Default value: pass,fail.
-sortby
Specifies the way to sort the test cases in the
report.
Possible values: name outcome.
Default value: name
-separator
Specifies the separator in the plain text report.
Possible values: space comma
Default value: space
• Cmd: Command line supported
Specifies the file name of the plain text report.
Protocol Test Manager
• PTM Capability
Deploy
Guide
• Guide: Embedded deployment guide illustration
Detect
SUT
Cmd
Line
• Detect: SUT capability auto detection
• Filter: Cases filtered by features and SUT capability
• Configure: Test suite configuration recommendation
• Run: GUI test case run w/o VS dependency
• Investigate: Integrated test run logs
• Rerun: Rerun and profile supported
• Cmd: Command line supported
TS
Configure
Rerun with
Profile
Filter
Cases
Investigate
Run
Case
Protocol Test Manager
• PTM Infrastructure :
PTM Command Line
GUI Main Window
Detector
Kernel
-IValueDetector detector
-DetectLog DetectLogCallback
Load(string detectorAssembly)
delegate void DetectLog()
delegate void
DetectionCallback()
delegate DetectionOutcome
DetectionDelegate()
BeginDetection()
RunDetection()
FileServer Plugin
RDP Plugin
AdFamily Plugin
ADOD Plugin
Kerberos Plugin
…
Super Logging With MMA
Super Logging With MMA
Run-time information for test case failure investigation
•
Test case log - by log file
Network traffic - by Microsoft Message Analyzer/Netmon/Wireshark
Encrypted message – by Visual Studio Debug Tool
•
•
•
•
Super logging
•
Combine all valuable logging information in MMA
Test Log
Network Traffic
Encrypted Message
MMA
Super Logging With MMA
ETW Controller
Event Tracing
Session
Log
Files
Events
Events
Events Delivery in real-time
Logged Events
Protocol Test
Suites
ETW Provider
ETW
Consumer
MMA
MMA Parser
MMA Parser
• Freshly released MMA parser aligned with Window Server 2016 TP5
• Release with latest MMA 1.4: https://www.microsoft.com/en-us/download/details.aspx?id=44226
Parser
Updates
RSVD
Add new messages/operations
2.2.4.34 RSVD_BLOCK_DEVICE_TARGET_SPECIFIER Structure
2.2.4.35 RSVD_BLOCK_DEVICE_TARGET_SPECIFIER_SNAPSHOT Structure
2.2.4.36 SVHDX_APPLY_SNAPSHOT_PARAMS Structure
2.2.4.37 SVHDX_TUNNEL_QUERY_VIRTUAL_DISK_CHANGES_REQUEST Structure
2.2.4.38 SVHDX_TUNNEL_QUERY_VIRTUAL_DISK_CHANGES_REPLY Structure
Update decoding logic to parse existing messages correctly.
RSVD_TUNNEL_CHANGE_TRACKING_GET_PARAMETERS
RSVD_TUNNEL_CHANGE_TRACKING_START
RSVD_TUNNEL_CHANGE_TRACKING_STOP
SQOS
Add fields for new protocol version 1.1 in messages
2.2.2.2 STORAGE_QOS_CONTROL_REQUEST Structure
2.2.2.3 STORAGE_QOS_CONTROL_RESPONSE Structure
SMB2
Update field name ”ErrorData” to “ErrorContextData” in below structure:
2.2.2.1 SMB2 ERROR Context Response
Add validation check for padding field in structure 2.2.13.2.15 SMB2_CREATE_APP_INSTANCE_VERSION
Update fields definition in structure 2.2.26 SMB2 LOCK Request:
LockSequence:uint is split into 2 fields: LockSequenceNumber:4bit + LockSequenceIndex:28bit
Resources
• PTM on Github:
• https://github.com/Microsoft/WindowsProtocolTestSuites/tree/4bdee2811d6
e7e1d90b329d88ff2fc7646301cfd/ProtocolTestManager
• PTM Plugin on Github
• https://github.com/Microsoft/WindowsProtocolTestSuites/tree/4bdee2811d6
e7e1d90b329d88ff2fc7646301cfd/ProtocolTestManager/Plugins
Redmond Protocols Plugfest 2016
Questions or Comments?