Weekly OpenADE Meeting Notes
Tuesday, June 2, 2015
OpenADE Task Force Topics
•
Green Button Connect My Data Testing and Certification (target fall 2014)
–
–
–
•
Issues Raised and Implementation Questions
–
–
–
–
–
–
–
–
–
–
•
 Complete function block descriptions
 Complete test case requirements
Amend CMD test requirements if gaps are discovered in dry run or other process
 How to use BR=bulkID with application to account and account groupings, as well as, large ThirdParty
collections of Authorizations.
 Service Request 83 – including Function Block for optional customer info (service point address, etc.)
 Service Request 84 – having scope selection screen on Data Custodian Site vs 3rd Party site (need to write
up description)
 Service Request 85 – Duplicating TOU and CPP from ReadingType to IntervalReading as in SEP 2.0
Service Request 86 – Desire to add digital signature to Green Button data to protect against tamper.
 Service Request 90 – Error in GB TestPlan: FB_07,08 AccumulcationBehavior should be 4
 Service Request 91 – Error in Test Plan: FB_04
 Mega Third Party
 Service Request 92 – Certificate reference
Enhancement of UsageSummary.CostAdditionalDetail to indicate kind of billing determininant
New Resources for OpenADE Exchange requested
–
–
Tariff Model Resource
 Customer Information Resource
GET Subscription w/wo query
parameters – Default required
behaviors
Problem: The amount of data that is in a:
GET /Batch/Subscription/{subscriptionId}
can potentially be very large. What is the obligation of a DC when that
message is received without query parameters? What about the desire
of some DC’s to provide only a fixed size response of say “X” months?
What about when a subscription contains 1/10/1000 UsagePoints?
• Alternatives
– Requires date range in GET /Batch/Subscription/{subscriptionId}
• DC responds with 202 if data is too large to return right away
– DC decides maximum depth to return when no query parameters
• TP may get whole history (from scope parameter HistoryLength) or may get
less.
– Term in Scope for default GET Subscription depth
• The scope can indicate for the specific subscription what the default length is.
The DC decides during scope selection what to include in the offered scopes.
CIM Tariff Model
cl a ss Ta r i f f P r of i l e
Document
C ust omer s::
P r i ci ngSt r uct ur e
+PricingStructures
0..*
+Tariffs
0..*
Document
IdentifiedObject
C ust omer s::
Ta r i f f
Charges associated with
time threshholds
+Tariffs
0..*
+TariffProfiles
0..*
0..*
+ReadingType
Document
+TariffProfiles
Ta r i f f P r of i l e
+
M et er i ng::
Rea di ngTy pe
tariffCycle: String [0..1]
Charges associated with
value threshholds
+TariffProfiles
0..*
+ConsumptionTariffIntervals
+TimeTariffIntervals
+ConsumptionTariffIntervals
0..*
0..*
0..*
Ti meTa r i f f I nt er v a l
+
+
0..1
+TouTariffIntervals
sequenceNumber: Integer [0..1]
startTime: Time [0..1]
+ConsumptionTariffIntervals
C onsumpt i onTa r i f f I nt er v a l
0..* +
+
0..*
0..*
+TimeTariffIntervals
sequenceNumber: Integer [0..1]
startValue: Float [0..1]
0..*
+ConsumptionTariffIntervals
«Compound»
A ccount i ngU ni t
IdentifiedObject
+Charges
+
0..*
+
+
+ParentCharge
0..1
C ha r ge
+Charges
kind: ChargeKind [0..1]
fixedPortion: AccountingUnit [0..1]
variablePortion: PerCent [0..1]
0..*
+ChildCharges
0..*
+
+
+
+
value: Float [0..1]
energyUnit: RealEnergy [0..1]
monetaryUnit: Currency [0..1]
multiplier: UnitMultiplier [0..1]
Additional Information for
UsageSummary.CostAdditionalDetail
• Add an enumeration tag to lineitem to
indicate what kind of charge it is
• itemKind
– Tax
– TOU
– Demand
– ThirdParty energy provider fee
– ThirdParty energy provider usage
 Solutions to Add Customer Account
Numbers
Make nonobfuscatedId
Add link
Extend each Class with
IdentifiedObject.name
Add Atom
Extension

Account/Agreement Topology
Separation of PII containing Resource
RetailCustomer from Subscription*
PII Containing
information
RetailCustomer
Anononymous
EUI
Subscription
CustomerAccount
UsagePoint
Customer
Agreement
Normal ESPI
Resources
ServiceLocation
PostionPoint
Key
Authorization
ServiceSupplier
New Resource
Existing Resource
Non-Resource
Class
EndDeviceAsset
PricingStructure
*This data structure is to be developed on an aggressive schedule
based on HelpDesk issue #83 and PAP10 NAESB Std REQ.18. No
single API request can retrieve both PII and Anonymous data
cl a ss C ust omer P r of i l e
OrganisationRole
Model
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Need to determine
which are
IdentifiedObjects ->
resources
Customers::Customer
{root}
+ kind: CustomerKind [0..1]
+ specialNeed: String [0..1]
+ pucNumber: String [0..1]
+ status: Status [0..1]
+ billingCycle: String [0..1]
+ priority: Priority [0..1]
+Customer
Agreement
+ budgetBill: String [0..1]
+ locale: String [0..1]
+CustomerAccounts
Customers::CustomerAgreement
::Document
1 ::IdentifiedObject
+ createdDateTime: DateTime [0..1]
+ description: String [0..1]
+ loadMgmt: String [0..1]
+ lastModifiedDateTime: DateTime [0..1] 0..*
+ mRID: String [0..1]
::Agreement
+ revisionNumber: String [0..1]
+ name: String [0..1]
+ signDate: Date [0..1]
+ status: Status [0..1]
«deprecated»
+ validityInterval: DateTimeInterval [0..1]
::IdentifiedObject
+ vip: Boolean [0..1]
::Document
+ description: String [0..1]
+ createdDateTime: DateTime [0..1]
+CustomerAccount + mRID: String [0..1]
+ lastModifiedDateTime: DateTime [0..1]
+ name: String [0..1]
+ revisionNumber: String [0..1]
+CustomerAgreements
1
+ status: Status [0..1]
0..*
::IdentifiedObject
Document
Customers::CustomerAccount
UsagePoints of RetailCustomer
Location of premise
Account ID
Sub Account (SA) ID—Service Agreement / Account is
name depending on utility
+ description: String [0..1]
+ mRID: String [0..1]
Customer name, nickname (or short name)
+ name: String [0..1]
Address and info
+CustomerAgreements
+CustomerAgreements
+CustomerAgreements
0..*
+CustomerAgreements
0..*
0..*
0..*
SDG&E provides only email address and UPID
correspondence csv email and UsagePoint ID (Customer
WorkLocation
C ust omer s::Ser v i ceLoca t i on
Obfuscated Key)
+ accessMethod: String [0..1]
+ siteAccessProblem: String [0..1]
MeterID
+ needsInspection: Boolean [0..1]
ServicePointId
::Location
+ type: String [0..1]
+ mainAddress: StreetAddress [0..1]
Pnode
+ServiceLocations + secondaryAddress: StreetAddress [0..1]
LoadAggregationPoint, SubloadAggregationPoint
0..* + phone1: TelephoneNumber [0..1]
+ phone2: TelephoneNumber [0..1]
+ electronicAddress: ElectronicAddress [0..1]
Climate zone
+ geoInfoReference: String [0..1]
+ direction: String [0..1]
Account open date
+ status: Status [0..1]
::IdentifiedObject
Account close date
+ description: String [0..1]
+ mRID: String [0..1]
SA Open and Close date
+ name: String [0..1]
MDM Agent Id (who does meter read)
ServiceSupplierId
PaymentMetering::Serv iceSupplier
+ServiceSupplier
EnergyServiceProviderId (may be same as service
+ kind: SupplierKind [0..1]
1 + issuerIdentificationNumber: String [0..1]
supplier)
Demand Response Provider
M et er i ng::Dema ndResponseP r ogr a m
May need list of Ids for service providers rather than
+DemandResponsePrograms
+ type: String [0..1]
explicit?? (0..* relationship{role, href})
0..*
+ validityInterval: DateTimeInterval [0..1]
Related assets ???? For example pool pump and pool
pump participation in a program.
C ust omer s::
+PricingStructures
P r i ci ngSt r uct ur e
Related programs ????
0..*
+ code: String [0..1]
C ommon::
P osi t i onP oi nt
+PositionPoint
+
0..1 +
+
+ServiceLocation
xPosition: String [0..1]
yPosition: String [0..1]
zPosition: String [0..1]
0..1
+ServiceLocation
0..1
+UsagePoints
M et er i ng::
U sa geP oi nt
0..*
+ServiceLocation
0..1
AssetContainer
M et er i ng::EndDev i ce
+EndDevices
+
0..* +
+
+
isVirtual: Boolean [0..1]
isPan: Boolean [0..1]
installCode: String [0..1]
amrSystem: String [0..1]
::Asset
+
+
+
+
+
+
+
+
+
+
+
type: String [0..1]
utcNumber: String [0..1]
serialNumber: String [0..1]
lotNumber: String [0..1]
purchasePrice: Money [0..1]
critical: Boolean [0..1]
electronicAddress: ElectronicAddress [0..1]
lifecycle: LifecycleDate [0..1]
initialCondition: String [0..1]
initialLossOfLife: PerCent [0..1]
status: Status [0..1]
::IdentifiedObject
+
+
+
description: String [0..1]
mRID: String [0..1]
name: String [0..1]
RetailCustomer
•
 API
–
–
–
–
–
–
•
GET .../resource/Batch/RetailCustomer/{id}
GET .../resource/RetailCustomer/{id}
…
GET .../resource/RetailCustomer/{id}/CustomerAccount/{id}/CustomerAgreement/{id}
GET .../resource/CustomerAgreement/{id}
GET .../resource/Batch/BulkRetailCustomerInfo/{id}
 Authorization
–
Scope string addition?
•
•
–
–
Access tokens to access? Individual with access_token, bulk with client_access_token
Thoughts:
•
•
•
•
 Simpler to use the one authorization and the one bulkID for this data which would be used with …/Bulk/{bulkId} and with …/BulkRetailCustomer/{bulkId}
X Desire to select which APIs query parameters are valid for – but, currently we only test for the specific query paramters needed for IntervalBlock date ranges. Other
parameters and which resources they apply to are optional.
 Leave generic query parameters as is for all resources but no requirements to support any specific set by resource.
FB
–
–
–
–
•
X RCInfo=AC where – A=Address included , C=AccountInfo included
X RCBulkId=123 for access to account info in bulk
X Has RetailCustomer – Just one FB_4X with RCInfo
X Has Additional Detail indicated by content of RCInfo
 Alternative – just like EUI data have separate function blocks for groupings of data (along resource lines) and they are part of scope=FB_xxxxxxxxxx
Need to work out FBs that make sense for RetailCustomer
 Document as a separate document to maintain isolation of the PII and related discussions (CustomerResourceModelHelpdesk83.docx)
Older or other slides
Will build deck with new content
over time.
Mega Third Party
• Host Third Party for smaller third parties
– Small solar companies that don’t want to implement ESPI
technical requirements
– Have simple web site
– Want customer data
• Why does small third party interface to mega third
party have to be standardized?
– Have consulting company they are hosting GB on behalf of
– Retail Customer authorizes little third party and mega third
party to access the data
• How does mega third party track the authorization
which occurs from the little third party
– Wants ability to tag a OAuth sequence to associate with
the little third party – perhaps a state parameter???
Issues
• What if one customer authorizes for two different
provider using mega-third party
– PG&E only permits one authorization for actual
customer/third party pair.
– Same customer goes to Solar1 and Solar2 who are
using the services of MegaTP
– How can DC know one authorization from the next
–  Consensus: let this be private matter between
MegaThirdParty and customer. No additional protocol
required.
• What if wordpress directs to DC and not TP?
GBCMD Test Harness
• Using Stunnel Proxy to isolate https from http on test
harness
– Three message types:
• 1) DataCustodian to Mock Server
– https  http
– Use stunnel server config (tpserver.conf)
• 2) SOAPUI to DataCustodian using groovy script and httpbuilder
– http https
– Use stunnel client config (tpclient.conf) – host must be what is in
ApplicationInformation remapped to port 8080.
• 3) SOAPUI to DataCustodian via Selenium
– https web browser integration
– Use Selenium browser with what is in ApplicationInformation bypassing
stunnel for these messages
 Certified Link
<link rel="related" href="https://cert.greenbuttonalliance.org/92348981231"/>
•
•
•
•
1.
2.
3.
4.
Added to feed (or if only entry, added to entry)
Assigned by GBA at test request submission
Until certified, may return “in progress” or some useful status
Optional request of return type for GET (returned by GBA site):
GET https://cert.greenbuttonalliance.org/92348981231
GET https://cert.greenbuttonalliance.org/92348981231?format=JSON
GET https://cert.greenbuttonalliance.org/92348981231, Header parameter Accept:
application/json
Return (XML Version) – exact form tbd.
Chunking – When the DataCustodian
needs to provide data in “chunks”
• Have huge data set that DC wants to provide
in “chunks” over HTTPS
• Use common URI – Subscription or Bulk / id
– Chunked by index and date?
– Use ESPI query parameters (FB_37) to distinguish
chunks
<?xml version="1.0" encoding="UTF-8"?>
<BatchList xmlns="http://naesb.org/espi" xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance
xsi:schemaLocation="http://naesb.org/espi espiDerived.xsd">
<resources>../espi/1_1/resource/Batch/Bulk/1?start-index=1&max-results=1000&published-max=2012-04-02T04:00:00Z
&published-min=2012-04-02T04:00:00Z</resources>
<resources>../espi/1_1/resource/Batch/Bulk/1?start-index=1&max-results=1000&published-max=2012-04-02T04:00:00Z
 &published-min=2012-04-02T04:00:00Z</resources>
</BatchList>
Deferred Response when DC does not
have data ready
DC
Request:
HTTP GET
https://localhost/DataCustodian/espi/1_1/Subscription/1
Content-type: application/xml
TP
HTTP returns 202
Request:
HTTP POST
https://localhost/ThirdParty/espi/1_1/Notification
Content-type: application/xml
Response:
Request:
HTTP GET
https://localhost/DataCustodian/espi/1_1/Subscription/1
Content-type: application/xml
HTTP returns 200 and Data
• ThirdParty makes asynchronous request
• DataCustodian does not have the data ready – returns HTTP 202
• Some time later (when ready no guarantees or time limit) DataCustodian sends
Notification with original URL in BatchList
• DataCustodian provides the result
Green Button Data File Summary
• Can there be a “digest” of what is in a Green Button
data set (i.e. feed) so that the whole data set returned
by a GET or DMD can be judged for content without
reviewing the contents
– Different Usage Points in file may have different contents
– Content may be dependent on when data is retrieved
– Simple xpath statements can be used to understand the
contents
– Unsuitable data is likely to be asked for once not
repeatedly
• Consensus: Not needed
New XML Schema Tags
• certificationNumber
– Provides information about the certificate issued to the creator of the
file
• contentHash
– Provides a checksum value of the files contents the recipient of the file
may use to confirm the file has not been tampered with and all
elements of the file were received
– Response to OpenADE Help Desk item 86
• http://osgug.ucaiug.org/HelpDesk/Lists/servicerequests/DispForm.aspx?ID=86
&Source=http%3A%2F%2Fosgug%2Eucaiug%2Eorg%2FHelpDesk%2FLists%2Fs
ervicerequests%2FGreenButton%2Easpx
• Consider RFC 4287 Atom Syndication Format section 5.1 which
describes Digital Signatures for Atom:
https://tools.ietf.org/html/rfc4287#section-5.1
Certification Structure
link type=“text/html”
href=“https://cert.greenbuttonalliance.org/certificate/987654321987654321” />
Possible Options
• GET
https://cert.greenbuttonalliance.org/92348981231?ThisGuysScopeIs=FB_
01040507”
• PGE -92348981231 – scopes=1_4_5,1_4_10,1_4_5_10
Variable return types
• Query Parameter:
– GET
https://cert.greenbuttonalliance.org/9234898123
1?format=XML
• Accept Header Parameter:
– Accept: Application/XML
• Atom link types
– <link type=“XML” href=“https://foo.bar.com”/>
Certificate Solutions
• Create new required resource for GB
• *** Create a required feed related link
• Look at other atom feed attributes to
implement
[FB_14] OAD011 [NEG] Malformed Refresh Token Request
• Verify Data Custodian rejects a malformed Refresh Token Access
Token Request
– Refresh_token field-value pair that was issued to another Third Party
• The current CMD test harness can only simulate a single Third Party
application and thus is not able to present a refresh_token request using
another Third Party’s assigned refresh_token
• Two application information structures would need to be registered at the
Data Custodian under test to be able to support this requirement
– Refresh_token field-value pair has expired
• A short lived refresh_token would need to be available to perform this test
requiring the Data Custodian under test to be able to modify the
refresh_token expiration period
• The authorization server’s token expiration test should not be based on the
type of token issued, therefore the existing test in OAD016 [NEG] Invalid
Access Token Request (Access Token contained in the Authorization Header
has expired) makes this test redundant
Cell Phone API Model
• How to do authorization for cell phone APP
– Still need to involve App developer (DC may not
accept request from any app)
– API FBs
– Customer and his app authentication
• Get an access token
– Differs in certificate management and security
• Security
• Privacy
– Retail Customer vs Subscription
One Data Custodians Potential Design
Dimensions for UsageSummary
Nettable: total cost = sum of nettable components
NEM: Net Metering
Implementation Perspective to Billing data
Meter Relationships (Additive-Conjunctive, Subtractive)
Special Bill Items
oNEM Statement
oShadow Bill Info
oLevel Pay Plan customers
Rates (bill periods: pricing changes, seasons)

NEM:
o Nettable
 UDC
 Commodity
 Taxes (DWR, City and State)
 Discounts
 Care
 FERA
 Employee discounts
o Non-nettable
 Demands
 Minimum charge
 Customer charge
 Event days
 GHG

Non-NEM customers:
o Nettable
 UDC
 Taxes (DWR, City and State)
 T&D
 Discounts
 Care
 FERA
 Employee discounts.
o Non-nettable
 Event days
 GHG
Topics
• Multiple ReadingTypes in file not referenced
by contents
–  Proposal – permit (right now excluded)
• Definition of Net metering FB_07 vs. FW/REV
metering FB_08
FB_03
• espiderived.xsd
– thirdPartyUserPortalScreen vs. client_uri
• Appears to duplicate same function
• client_uri is optional by dynamic registration OAuth protocol
• Solution
– Make client_uri and thirdPartyUserPortalScreen optional in schema
– Authorization.authorizedPeriod and publishedPeriod should be optional since
not needed in authorizations for client admin and registration
• Solution
–
–
–
–
–
Make both fields optional in the schema
Ensure they are present for access_token based authorizations
If present validate authorizedPeriod and publishedPeriod are valid date format
If either authorizedPeriod or publishedPeriod is present, both are required
Allow duration to be present with “0” values implying non-expiring authorizations
• grant_types for ApplicationInformation
• Should it be set of grant_types that DC supports?
• Spring requires separate client_id for client_credentials flow
• Solution:
– Nothing needs to change
FB_03
• grant_types test assertion in FND002 re: redirect_uri
– Solution:
• Remove the test from FND002 and OADxxx
• response_types should be “code”
– Solution:
• Add test to validate content of response_types is “code”
• Lifetime of client_access_token
– If shortlived, you need to do client_credentials each day to
get a new one
• This forces you to use the “secret” often which is a greater risk
than client access token
• What does this do the lifetime of the “Authorization”
CMD T&C
T&C Plan
Now
(10/14/2014)
Test
implemented
Tests defined
12/23
February
Event
Any needed
adjustments
to testees
code
GreenButton Connect My Data Conformance
Testing Requirements Review
•
•
For Review
•
FB_03 Core GB CMD
•
FB_39 PUSH model-REST Notifications/bulk transfer
•
FB_34 SFTP for bulk
•
FB_35 REST for Bulk
•
FB_13 Security
•
FB_14 Authorization and Authentication
Not yet ready for review
•
FB_19 Partial data update
•
FB_40 Offline RetailCustomer authorization
•
FB_37 Query parameters
CMD Test Development Plan
• Phase I - 11/17/2014..11/21/2014
– Ron/Don Complete Spreadsheet with Test Requirements and
Test Steps.
– In parallel John/Marty get scheduled with consenting Data
Custodians for an initial test – GET ServiceStatus which requires
a target URL and a client_access_token for a preconfigured test
Third Party.
• Phase II - 11/19/2014..11/26/2014
– OpenADE/OpenESPI Participants review test requirements and
procedures and report by exception
– Don/Ron are building tests
• Phase III – 12/1/2014..12/31/2014
– Don/Ron are building tests
– John/Marty are running tests with consenting Data Custodians
Set UUT Into Test Harness
• Harness acts as a TP
– Need test third party account
• At least three Test Accounts
– Two authorized for this third party
– One known and authorized for any other third party
• Create / Exchange ApplicationInformation for Test TP
• Create / Exchange
– TP client_access_token
– TP registration_access_token
– Two Subscription access_token (may included OAuth
authorization process)
– Third subscription access_token (not owned by TP and used in
negative tests)
Data Custodian Test Capabilities
• Any given data custodian might fall into three
possible capability categories:
– The DC has the ability to clear created
authorizations
– The DC has multiple accounts to authorize so that
when a new authorization is needed it can be
created
– The DC has to accept that one test failure may
preclude going on to perform other tests
Issues
• How to expire an access token so it can be tested along
with refresh_token
–  Testee can manually or otherwise “expire” an access token
• Removal of client_credential flow testing until dynamic
registration
–  Support the API
– Don’t support in minimum required
• How the transition from “scope selection” which is not
OAuth, to the OAuth sequence which must originate at the
Third Party occurs.
– Need to review Authorization document (needs corrections) and
implementers need to check what they are implementing
DataCustodian Registration Values to Communicate
with the Certification Test Harness
• thirdPartyNotifyUri
– https://services.greenbuttondata.org/CertificationThirdParty/es
pi/1_1/Notification
• thirdPartyScopeSelectionScreenURI
– https://services.greenbuttondata.org/CertificationThirdParty/es
pi/1_1/RetailCustomer/ScopeSelection
• thirdPartyUserPortalScreenURI
– https://services.greenbuttondata.org/CertificationThirdParty/es
pi/1_1/RetailCustomer/home
• client_name
– Certification Test Harness
• client_uri
– https://services.greenbuttondata.org/CertificationThirdParty
GBCMD Testing and Certification
Status
•
Test Project and Harness
– Need to add target UUT configuration and refactor tests
•
FB_3 Core Green Button Connect My Data
– Status: Tests almost complete
•
FB_13 Security and Privacy
– Status: Initial set of test complete; need to adjust to test harness and needs some small
enhancements
•
FB_14 Authorization and Authentication
– Status: Repertoire of test cases initially identified by Don Coffin and they need to be reviewed
and implemented … implementation begun
•
FB_19 Partial Update Data
– Status: not started
•
FB_37 Query Parameters
– Status: not started
•
FB_39 PUSH Model
– Status: substantially complete
•
FB_34 Bulk SFTP
– Status: substantially complete (not on github)
•
FB_35 Bulk REST
– Status: substantially complete (not on github)
Authorization Resource
• Currently, client_access token can retrieve the collection of
authorizations for the specific third party. A concern was raised that
theft of that one token would provide access to all tokens (in the
Authorization resource) a serious vulnerability. Proposed solutions:
1.
Keep API and schema constant but require the omission of
access_token and refresh_token tags.
2. Make access to Authorization only based on the contained
access_token. That is, the client_access_token can only retrieve the
corresponding Authorization resource; registration_access_token
can only retrieve the corresponding Authorization resource; the
individual access_tokens can only retrieve the corresponding
Authorization resource.
Consensus solution:
3.  Remove access tokens from the schema.
Access Tokens
• Reference:
http://www.greenbuttondata.org/espi/access_tokens/
• ACUDR
• access_token
• client_access_token
• upload_access_token (used only in FB 45)
• datacustodian_access_token (used only in FB_33)
• registration_access_token
• refresh_token ?
CMD Test Development Plan
References
• All the test development is being done on the
https://github.com/energyos/OpenESPIGreenButtonCMDTest project, and,
• The test requirements and test procedures maintained in
the spreadsheet at https://github.com/energyos/OpenESPIGreenbuttonDataSDK/tree/master/GreenButtonTestingReq
uirements.
• You can enter issues for discussion on either project as you
see appropriate.
– For Test Code Issues: https://github.com/energyos/OpenESPIGreenButtonCMDTest/issues
– For Test Requirements Issues:
https://github.com/energyos/OpenESPIGreenbuttonDataSDK/issues
RETAIL CUSTOMER
Object Identification
• Objects are instances of classes
• Objects need to be identified uniquely because
– Data in a repository needs to be identified as to where
it came from
– Updates to data (for example from raw to validated)
need to identify that it’s the same data that has been
updated
– Devices from which data originates often needs to be
associated with the data
– Devices need to be labeled multiple ways for various
purposes – e.g. in a building topology (2nd floor
floodlight), in an electrical hierarchy (branch 2 load 3)
IEC 61970 IdentifiedObject
Master resource identifier issued by a model authority.
The mRID is globally unique within an exchange context.
Global uniqeness is easily achived by using a UUID for the
mRID. It is strongly recommended to do this.
A simple string to identify the object.
cl a ss Na mes
I dent i f i edO bject
+
+
+
aliasName :String [0..1]
mRID :String [0..1]
name :String [0..1]
+IdentifiedObject
1
+Names 0..*
Na me
+
name :String [0..1]
+Names
0..*
1
+NameType +
+
Na meTy pe
+NameTypes
name :String [0..1]
0..*
description :String [0..1]
The Name class provides the means to define any
number of human readable names for an object. The
name can be further attributed by a NameType and a
NameTypeAuthority.
Na meTy peA ut hor i t y
0..1
+NameTypeAuthority
+
+
name :String [0..1]
description :String [0..1]
IdentifiedObject
• mRID – usually a UUID that represents the object
instance
• name – simple string to identify the object
• Name – a class that allows additional names to be
used for the same object in different hierarchies.
– different naming authorities may have the right to
name devices for their own purposes
– it is important to identify the naming authority and
naming convention (type)
– These must also be properties of the object to which
they represent since it is the same unique object
ESPI Mapping of IdentifiedObject to
Atom
• ESPI endpoints expose resources as described by Atom,
IETF RFC 4287.
• Representations are identified as media type
“application/atom+xml”
• ESPI namespace and types (“http://naesb.org/espi”) are
used for objects in <content> element
• espi:mRID is implemented by atom:id
– UUIDs are used, as specified in IETF RFC 4122
• espi:description is implemented by atom:title
• atom:published and atom:updated are used
• Associated objects use atom:link (rel=“related”)
• espi:name is implemented a resource.name
 Solutions to Add Customer Account
Numbers
Make nonobfuscatedId
Add link
Extend each Class with
IdentifiedObject.name
Add Atom
Extension

Account/Agreement Topology
Separation of PII containing Resource
RetailCustomer from Subscription*
PII Containing
information
RetailCustomer
Anononymous
EUI
Subscription
CustomerAccount
UsagePoint
Customer
Agreement
Normal ESPI
Resources
ServiceLocation
PostionPoint
Key
Authorization
ServiceSupplier
New Resource
Existing Resource
Non-Resource
Class
EndDeviceAsset
PricingStructure
*This data structure is to be developed on an aggressive schedule
based on HelpDesk issue #83 and PAP10 NAESB Std REQ.18. No
single API request can retrieve both PII and Anonymous data
cl a ss C ust omer P r of i l e
OrganisationRole
Model
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Need to determine
which are
IdentifiedObjects ->
resources
Customers::Customer
{root}
+ kind: CustomerKind [0..1]
+ specialNeed: String [0..1]
+ pucNumber: String [0..1]
+ status: Status [0..1]
+ billingCycle: String [0..1]
+ priority: Priority [0..1]
+Customer
Agreement
+ budgetBill: String [0..1]
+ locale: String [0..1]
+CustomerAccounts
Customers::CustomerAgreement
::Document
1 ::IdentifiedObject
+ createdDateTime: DateTime [0..1]
+ description: String [0..1]
+ loadMgmt: String [0..1]
+ lastModifiedDateTime: DateTime [0..1] 0..*
+ mRID: String [0..1]
::Agreement
+ revisionNumber: String [0..1]
+ name: String [0..1]
+ signDate: Date [0..1]
+ status: Status [0..1]
«deprecated»
+ validityInterval: DateTimeInterval [0..1]
::IdentifiedObject
+ vip: Boolean [0..1]
::Document
+ description: String [0..1]
+ createdDateTime: DateTime [0..1]
+CustomerAccount + mRID: String [0..1]
+ lastModifiedDateTime: DateTime [0..1]
+ name: String [0..1]
+ revisionNumber: String [0..1]
+CustomerAgreements
1
+ status: Status [0..1]
0..*
::IdentifiedObject
Document
Customers::CustomerAccount
UsagePoints of RetailCustomer
Location of premise
Account ID
Sub Account (SA) ID—Service Agreement / Account is
name depending on utility
+ description: String [0..1]
+ mRID: String [0..1]
Customer name, nickname (or short name)
+ name: String [0..1]
Address and info
+CustomerAgreements
+CustomerAgreements
+CustomerAgreements
0..*
+CustomerAgreements
0..*
0..*
0..*
SDG&E provides only email address and UPID
correspondence csv email and UsagePoint ID (Customer
WorkLocation
C ust omer s::Ser v i ceLoca t i on
Obfuscated Key)
+ accessMethod: String [0..1]
+ siteAccessProblem: String [0..1]
MeterID
+ needsInspection: Boolean [0..1]
ServicePointId
::Location
+ type: String [0..1]
+ mainAddress: StreetAddress [0..1]
Pnode
+ServiceLocations + secondaryAddress: StreetAddress [0..1]
LoadAggregationPoint, SubloadAggregationPoint
0..* + phone1: TelephoneNumber [0..1]
+ phone2: TelephoneNumber [0..1]
+ electronicAddress: ElectronicAddress [0..1]
Climate zone
+ geoInfoReference: String [0..1]
+ direction: String [0..1]
Account open date
+ status: Status [0..1]
::IdentifiedObject
Account close date
+ description: String [0..1]
+ mRID: String [0..1]
SA Open and Close date
+ name: String [0..1]
MDM Agent Id (who does meter read)
ServiceSupplierId
PaymentMetering::Serv iceSupplier
+ServiceSupplier
EnergyServiceProviderId (may be same as service
+ kind: SupplierKind [0..1]
1 + issuerIdentificationNumber: String [0..1]
supplier)
Demand Response Provider
M et er i ng::Dema ndResponseP r ogr a m
May need list of Ids for service providers rather than
+DemandResponsePrograms
+ type: String [0..1]
explicit?? (0..* relationship{role, href})
0..*
+ validityInterval: DateTimeInterval [0..1]
Related assets ???? For example pool pump and pool
pump participation in a program.
C ust omer s::
+PricingStructures
P r i ci ngSt r uct ur e
Related programs ????
0..*
+ code: String [0..1]
C ommon::
P osi t i onP oi nt
+PositionPoint
+
0..1 +
+
+ServiceLocation
xPosition: String [0..1]
yPosition: String [0..1]
zPosition: String [0..1]
0..1
+ServiceLocation
0..1
+UsagePoints
M et er i ng::
U sa geP oi nt
0..*
+ServiceLocation
0..1
AssetContainer
M et er i ng::EndDev i ce
+EndDevices
+
0..* +
+
+
isVirtual: Boolean [0..1]
isPan: Boolean [0..1]
installCode: String [0..1]
amrSystem: String [0..1]
::Asset
+
+
+
+
+
+
+
+
+
+
+
type: String [0..1]
utcNumber: String [0..1]
serialNumber: String [0..1]
lotNumber: String [0..1]
purchasePrice: Money [0..1]
critical: Boolean [0..1]
electronicAddress: ElectronicAddress [0..1]
lifecycle: LifecycleDate [0..1]
initialCondition: String [0..1]
initialLossOfLife: PerCent [0..1]
status: Status [0..1]
::IdentifiedObject
+
+
+
description: String [0..1]
mRID: String [0..1]
name: String [0..1]
RetailCustomer
•
API
–
–
–
–
–
–
•
GET .../resource/Batch/RetailCustomer/{id}
GET .../resource/RetailCustomer/{id}
…
GET .../resource/RetailCustomer/{id}/CustomerAccount/{id}/CustomerAgreement/{id}
GET .../resource/CustomerAgreement/{id}
GET .../resource/Batch/BulkRetailCustomerInfo/{id}
Authorization
– Scope string addition?
•
•
RCInfo=AC where – A=Address included , C=AccountInfo included
RCBulkId=123 for access to account info in bulk
– Access tokens to access? Individual with access_token, bulk with client_access_token
•
FB
– Has RetailCustomer – Just one FB_4X with RCInfo
– Has Additional Detail indicated by content of RCInfo
February Event – Preliminary Planning
• Celebrate:
“Birth of the Green Button Ecosystem”
– Testing and Certification Complete for DMD/CMD
– UCAIug ITCA fully established
– Initial Data Custodians successfully certified for
DMD and CMD
– Shower Successful T&C Adopters with Fame and
Congratulations
• Venue and participation TBD in coming weeks
Best Practice Reading Quality Flags
•
•
•
•
•
•
•
ReadingType.defaultQuality contains the default quality that applies to all
corresponding IntervalReading data.
IntervalReading.ReadingQuality.quality allows specific Intervals to override the
default in ReadingType
UsageSummary.qualityOfReading if present overrides default in ReadingType for
those IntervalBlocks within the scope of the UsageSummary.billingPeriod
If IntervalReading data are modified, DataCustodian should notify of this change so
ThirdParty can retrieve the changes.
If UsageSummary.qualityOfReading overrides the ReadingType or IntervalReadings,
the IntervalReading qualities would change and a subsequent retrieval (not
required) of the IntervalBlocks would come with the corresponding quality flag.
The qualityOfReading flag for Usage Summary will indicate latest overriding quality
of previously provided interval values corresponding to BillingPeriod dates
The Default Quality (from ReadingTypeRef) for OverallConsumptionLastPeriod will
indicate quality of total billed usage
Requests that are not inside
authorization period Date of
request April 5
1/1/2014
12/31/20
(i) 3P Requested Period (completely outside)
9/1/2013
12/31/2013
(ii) 3P Requested Period (partially outside)
11/1/2013
3/31/2014
 Consensus:
i – agree on 403
ii – agree on 403
Expected Result?
(i)
(ii)
Options:
1. Return 403 unauthorized request for both complete and partial scenarios
2. Return 400 bad request for both cases (since the request params are bad but the tokens are
valid)
3. Return 200 with
a. Feed for the partially authorized period (per example: 1/1/14 – 3/31/14)
b. Empty feed for the period which is completely outside the authorized period (per
example: 11/1/2013 – 12/31/2013)
FB3 - Core REST Services
• [R] GET resource/ApplicationInformation/{ApplicationInformationID}
–
–
–
–
•
•
•
•
•
[POS] Accept of valid request
[NEG] Reject by invalid ID
[NEG] Reject by invalid access-token
[POS] Results valid to schema and include required fields for OAuth and TP/DC
interaction
[C] GET resource/Authorization
[C] GET resource/Authorization/{AuthorizationID}
[A] GET resource/Batch/Subscription/{SubscriptionID}
[C] GET resource/ReadServiceStatus
POST: How to test for TP Notification
– Trigger
– Uses notification URI from ApplicationInformation
– Expected content – at least one URI that can be GET’d?
FB_03 Core GB CMD
• Covers “core services”, resources and access control
–
–
–
–
–
–
–
–
–
–
atom:entry, atom:feed
GET, PUT, POST, DELETE (negative only)
ApplicationInformation
Authorization (feed)
Authorization (entry)
Subscription (entry) only available through
batch/subscription
ReadServiceStatus (entry)
Access token expiration testing?
Authorization expiration?
Notification move to FB_39?
FB_03 Core GB CMD
• atom:entry / ApplicationInformation
– Using required access token of
R(registration_access_token)
• Verify successful GET and response contents (which subset of app
info is required in the response?)
– If it is required by OAuth2 dynamic registration it must be present
– Other fields not derived from OAuth2?
– i.e. ContactInfo used in the case of a failed notification to TP
• Verify negative response to PUT, POST, DELETE: 403 – forbidden
– Using other access tokens(A,C) or none
• Verify negative response to GET, PUT, POST, DELETE: 403 –
forbidden
• No token: 401 – unauthorized
FB_03 Core GB CMD
• atom:feed / Authorization
– Using required access token of C(client_access_token)
• Verify successful GET and response contents (which subset
of Authorization info is required in the response?)
• Verify negative response to PUT, POST, DELETE: 403 –
forbidden
– Using other access tokens (AR) or none
• Verify negative response to GET, PUT, POST, DELETE: 403 –
forbidden
• No token: 401 – unauthorized
FB_03 Core GB CMD
• Authorization(C)- all fields(except error fields),
some have of which have req. values
• Subscriptions(A) - feed with at least 1 UsagePoint
• ReadServiceStatus(C) – all fields with content of
0/1
– Using required access tokens
• Verify successful GET and response contents
• Verify negative response to PUT, POST, DELETE
– TBD?: Using other access tokens or none
• Verify negative response
FB_39 Push Model – REST notifications
•
Send Notification to TP
–
–
–
–
–
•
•
•
pre-test set up – DC needs to know TP stand-in URI (FB_33 could automate this?)
DC UT has manual “trigger method” to generate notification
ApplicationInformation
Authorization
Subscription
Verify well-formed contents of the Notification
Get the data (w/ correct access token) and validate the data
[NEG] Test GET out of bounds and deferred response (should be moved to or is
already present in other FB tests)
– Authorization no longer valid (how?) – MOVE to FB_14
– Data no longer available (i.e. TP took too long for requesting the data) – SFTP error 2 ref to file
that does not exist / REST GET error
– Issue REST GET with wrong token (what token should the test use?) applies specifically to
FB_39 to ensure notification data is secure
•
[NEG] If TP Notify fails, verify by demonstration that failure was detected
– TP is off-line
FB_34 SFTP for Bulk
– Prerequisites
•
•
Pre configured Authorizations
Authorizations need bulk scope
– Notification of Bulk sent to test harness
•
i.e. sftp://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/Batch/Bulk/{BulkID}
– Test harness retrieves data via SFTP
– Validate the data
•
•
•
•
Pass schema
Must contain 1 feed w/ 1 or more entry(s)
Verify there is a valid authorization for each entry and the scope of each authorization for each
entry has BR=BulkId
Use https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/Authorization for
list of authorizations to check against
– Is URI a pointer to a folder or a pointer to a file?
– SFTP GETALL – could retrieve a set of files from a folder or
– SFTP GET – single file
FB_35 REST for Bulk
– Notification of Bulk
• Authorizations must be present
• Authorizations need bulk scope
• i.e.
sftp://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/B
atch/Bulk/{BulkID}
– Test harness retrieves data via REST GET
– Validate the data
• Pass schema
• 1 feed w/ 1 or more entry(s)
• Verify there is a valid authorization for each entry and the scope of each
authorization for each entry has BR=BulkId
• Use
https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/
Authorization for list of authorizations to check against
FB 13: Security Testing
• Cyber Security and Privacy Test Requirements
– Based on Authorization.docx section 2.7
• From SGIP SGCC Committee review of REQ.21
• Reviewed with NIST Cyber Security staff
• NAESB REQ.21 section
• Initial set of test requirements on next slide
FB 13: Security Testing
Initial Set of Test Requirements
•
•
•
•
•
•
•
•
[TR_TC001] Test software shall issue a service request over an SSL session and shall
verify that the response HTTP header contains the following fields and information –
fields TBD
[TR_TC002] Verify that REST request headers include – fields TBD
[TR_TC003] Verify that the Data Custodian implements TLS 1.2.
[TR_TC004] Verify that when communicating with a Retail Customer the Data Custodian
negotiates the highest level of TLS mutually supported.
[TR_TC005] Verify that when communicating with a Retail Customer the Data Custodian
rejects TLS_RSA_WITH_NULL_SHA cipher suites.
[TR_TC006] Verify that when communicating with a Retail Customer at a minimum the
Data Custodian accepts the TLS_RSA_WITH_AES_128_CBC_SHA cipher suite.
[TR_TC007] Verify that when communicating with a Third Party the Data Custodian
negotiates the highest level of TLS mutually supported.
[TR_TC008] Verify that the Data Custodian maintains an unexpired unrevoked RSA
certificate with a public key length of at least 2048 bits.
FB 13: Security Testing
Initial Set of Test Requirements
•
•
•
•
•
•
[TR_TC009] Test software or manual inspection shall verify that the Data Custodian RSA
certificate was issued by a Certificate Authority (CA) that has been successfully audited
according to the criteria of ETSI or WebTrust.
[TR_TC010] Test software or manual inspection shall verify that Tokens and IDs
communicated by the Data Custodian are opaque and if based on actual Customer
information that they are randomized using a secure method to protect privacy.
[TR_TC011] Test software or manual inspection shall verify that Tokens and IDs
communicated by the Data Custodian consist of at least 48 bits and can be the random
number part of an RFC2422 UUID.
[TR_TC012] Manual inspection of supporting documentation shall confirm that the Data
Custodian implementation utilizes software libraries which are FIPS 140-2 level 1 or
higher and listed on the CMVP website.
[TR_TC013] Verify that the Third Party implements TLS 1.1 or higher.
[TR_TC014] Verify that when communicating with a Retail Customer the Third Party
negotiates the highest level of TLS mutually supported.
FB 13: Security Testing
Initial Set of Test Requirements
•
•
•
•
•
•
[TR_TC015] Verify that when communicating with a Data Custodian the Third Party
negotiates the highest level of TLS mutually supported.
[TR_TC016] Verify that the Third Party maintains an unexpired unrevoked RSA
certificate with a public key length of at least 2048 bits.
[TR_TC017] Test software or manual inspection shall verify that the Third Party RSA
certificate was issued by a Certificate Authority (CA) that has been successfully audited
according to the criteria of ETSI or WebTrust.
[TR_TC018] Test software or manual inspection shall verify that Tokens and IDs
communicated by the Third Party are opaque and if based on actual Customer
information that they are randomized using a secure method to protect privacy.
[TR_TC019] Test software or manual inspection shall verify that Tokens and IDs
communicated by the Third Party consist of at least 48 bits and can be the random
number part of an RFC2422 UUID.
[TR_TC020] Manual inspection of supporting documentation shall confirm that the
Third Party implementation utilizes software libraries which are FIPS 140-2 level 1 or
higher and listed on the CMVP website.
FB_14: Authorization and Authentication (OAuth)
Initial Set of Test Requirements
•
•
•
•
•
•
•
•
•
•
•
•
[TR_OA001] Verify Data Custodian provides an Authorization Endpoint per OAuth 2.0 specification
[TR_OA002] Verify Data Custodian provides a Token Endpoint per OAuth 2.0 specification
[TR_OA004] Verify Data Custodian provides client with Third Party ID value per OAuth 2.0 specification
[TR_OA005] Verify Data Custodian provides client with Third Party Secret value per OAuth 2.0 specification
[TR_OA007] Verify Data Custodian rejects an Authorization Code Request with NO "Response Code" parameter
[TR_OA008] Verify Data Custodian rejects an Authorization Code Request with NO "client_id" parameter
[TR_OA009] Verify Data Custodian rejects an Authorization Code Request with NO "scope" parameter
Check on “redirect_uri” parameter
Check on “state” parameter
[TR_OA010] Verify Data Custodian rejects an Authorization Code Request containing multiple "response_type"
parameters
[TR_OA011] Verify Data Custodian rejects an Authorization Code Request containing multiple "client_id" parameters
[TR_OA012] Verify Data Custodian rejects an Authorization Code Request containing multiple "scope" parameters
FB_14: Authorization and Authentication (OAuth)
Initial Set of Test Requirements
• [TR_OA013] Verify Data Custodian rejects an Authorization Code Request containing multiple "redirect_uri"
parameters
• [TR_OA014] Verify Data Custodian rejects an Authorization Code Request containing multiple "state" parameters
• [TR_OA015] Verify Data Custodian rejects an Authorization Code Request containing an INVALID parameter
• [TR_OA016] Verify Data Custodian rejects an Authorization Code Request with an INVALID "Response Code" parameter
• [TR_OA017] Verify Data Custodian rejects an Authorization Code Request with an INVALID "client_id" parameter
• [TR_OA018] Verify Data Custodian rejects an Authorization Code Request with an INVALID "redirect_uri" parameter
• [TR_OA019] Verify Data Custodian rejects an Authorization Code Request with an INVALID "scope" parameter <Note:
May need various forms of Invalid SCOPE Parameter Values ????>
• [TR_OA020] Verify Data Custodian properly validates Retail Customer while processing a valid Authorization Code
Request
• [TR_OA021] Verify Data Custodian proper handles a Retail Customer who fails to pass authentication testing while
processing a valid Authorization Code Request
• [TR_OA022] Verify Data Custodian properly obtains the Retail Customer's authorization for the Third Party to access
their data while processing a valid Authorization Code Request
• [TR_OA023] Verify Data Custodian properly processes a Retail Customer's denial to allow a Third Party to access their
data while processing a valid Authorization Code Request
FB_14: Authorization and Authentication (OAuth)
Initial Set of Test Requirements
• [TR_OA024] Verify Data Custodian properly processes a Retail Customer's authorization to allow a Third Party to access
their data while processing a valid Authorization Code Request
• [TR_OA027] Verify Data Custodian properly authenticates and accepts an Access Token Request with a valid HTTP
BASIC Authorization header
• [TR_OA028] Verify Data Custodian rejects an Access Token Request with an INVALID HTTP BASIC Authorization header
• [TR_OA029] Verify Data Custodian properly authenticates the authorization_code contained in the "code=" parameter
of an Access Token Request was issued to the "client_id" in the Access Token Request
• [TR_OA030] Verify Data Custodian accepts an Access Token Request containing an authorization_code ("code="
parameter) issued to the "client_id" in the Access Token Request
• [TR_OA031] Verify Data Custodian rejects an Access Token Request with NO "grant_type" parameter
• [TR_OA032] Verify Data Custodian rejects an Access Token Request with NO "code" parameter
• [TR_OA033] Verify Data Custodian rejects an Access Token Request with NO "redirect_uri" parameter
• [TR_OA034] Verify Data Custodian rejects an Access Token Request with NO "client_id" parameter
FB_14: Authorization and Authentication (OAuth)
Initial Set of Test Requirements
•
•
•
•
•
•
•
•
•
[TR_OA035] Verify Data Custodian rejects an Access Token Request with multiple "grant_type" parameters
[TR_OA036] Verify Data Custodian rejects an Access Token Request with multiple "code" parameters
[TR_OA037] Verify Data Custodian rejects an Access Token Request with multiple "redirect_uri" parameters
[TR_OA038] Verify Data Custodian rejects an Access Token Request with multiple "client_id" parameters
[TR_OA039] Verify Data Custodian rejects an Access Token Request with a "client_id" parameter and a HTTP BASIC
authorization field
[TR_OA040] Verify Data Custodian rejects an Access Token Request containing an authorization_code ("code="
parameter) NOT issued to the "client_id" in the Access Token Request
[TR_OA041] Verify Data Custodian rejects an Access Token Request containing an INVALID authorization_code
("code=" parameter)
[TR_OA042] Verify Data Custodian rejects an Access Token Request not containing a "redirect_uri" parameter if the
original Authorization Request contained a "redirect_uri" parameter
[TR_OA043] Verify Data Custodian rejects an Access Token Request containing a "redirect_uri" parameter if the
"redirect_uri" does NOT match the "redirect_uri" parameter contained in the original Authorization Request
parameter
FB_14: Authorization and Authentication (OAuth)
Initial Set of Test Requirements
•
•
•
•
•
•
•
•
•
•
•
[TR_OA044] Verify Data Custodian rejects an Access Token Request containing a "redirect_uri" parameter if the original
Authorization Request did NOT contain a "redirect_uri" parameter
[TR_OA045] Verify Data Custodian rejects an Access Token Request containing a previously used authorization_code
[TR_OA046] Verify Data Custodian rejects an Access Token Request containing an expired authorization_code
[TR_OA047] Verify Data Custodian issues a properly formatted Access Token Response (grant_type=authorization_code)
[TR_OA050] Verify Data Custodian issues a properly formatted Access Token Response (grant_type=client _credentials)
[TR_OA051] Verify Data Custodian rejects an Access Token Request with multiple "grant_type" parameters
[TR_OA052] Verify Data Custodian rejects an Access Token Request with multiple "scope" elements
[TR_OA053] Verify Data Custodian rejects an Access Token Request with an INVALID "scope" parameter <Note: May need various
forms of Invalid SCOPE Parameter Values ????>
[TR_OA056] Verify Data Custodian rejects a Refresh Token Request with NO "grant_type" parameter
[TR_OA057] Verify Data Custodian rejects a Refresh Token Request with NO "refresh_token" parameter
[TR_OA058] Verify Data Custodian rejects a Refresh Token Request with multiple "grant_type" parameters
FB_14: Authorization and Authentication (OAuth)
Initial Set of Test Requirements
•
•
•
•
•
•
•
•
[TR_OA059] Verify Data Custodian rejects a Refresh Token Request with multiple "refresh_token" parameters
[TR_OA060] Verify Data Custodian rejects a Refresh Token Request with multiple "scope" parameters
[TR_OA061] Verify Data Custodian properly authenticates and accepts a Refresh Token Request with a valid HTTP
BASIC Authorization header
[TR_OA062] Verify Data Custodian rejects a Refresh Token Request with a INVALID HTTP BASIC Authorization header
[TR_OA063] Verify Data Custodian rejects a Refresh Token Request containing a "refresh_token" element that was
NOT issued to the requesting Third Party Application
[TR_OA064] Verify Data Custodian rejects a Refresh Token Request containing a "refresh_token" element that is
expired
[TR_OA065] Verify Data Custodian rejects a Refresh Token Request containing a "scope" element that does NOT
match the "scope" element value used to obtain the original Access Token.
[TR_OAxxx] Verify Data Custodian handling of expired Access Token and Refresh Token
FB_19: Partial update of data
• Is this requirement for upload role and not
core data custodian?
[FB_37] Query Parameters
• Support published_min, published_max
[FB_40] Offline RetailCustomer
Authorization
• Manual Creation of ApplicationInformation,
Authorization(s)
ReadingType Attributes
• PG&E notices that their MDM uses various
ReadingType attribute values that are not the
same as those in the DMD test suite.
• Question – should meter data reflect the
diversity of the meter system or the meaning
of data conveyed to the ThirdParty
• Folks will look at their data and thinking
further.
Interpretation of Quality Flags for UsageSummary,
ReadingType, and IntervalReading
• ReadingType.defaultQuality contains the default quality that applies
to all corresponding IntervalReading data.
• IntervalReading.ReadingQuality.quality allows specific Intervals to
override the default in ReadingType
• If IntervalReading data or quality tags are modified, DataCustodian
should notify of this change so ThirdParty can retrieve the changes.
• UsageSummary.qualityOfReading if present indicates that those
IntervalBlocks within the scope of the UsageSummary.billingPeriod
may have changed quality as well. Third Party may want to retrieve
data again to see the revisions if any.
• The DC may indicate to the TP that IntervalBlock data has changed
by sending a notification for the IntervalBlocks that changed.
Testing and Certification Issues
• Test Third Party role for testing DataCustodian
• Test accounts (how many ; real-or-not ; how much
history?)
• Test Set up – Application Information
• How to put DC in reproducible state – reset?
• Minimum FBs to test –
– FB_3, 13, 14, 19,37,39
• Alternative – Test Environment that is same as the Live
environment
– Same Certs etc…
– Does it need to be real data?
ElectricPowerUsageSummary
• Make general UsageSummary and deprecate
ElectricPower one…
• Let query determine period not current/last
• How to rename fields to remove current/last
ambiguity for past requests
• Determine what required fields might be and
some possible new FB to support ecosystem
interoperability
• Single Demand fields too limiting for modern
tariffs.
UsageSummary Recommendataions
• Create new UsageSummary (which is NAESB REQ.18
name)
• Add new tags recommended by PG&E
• Retain all existing tags and make UsageSummary and
ElectricPowerUsageSummary identical – but mark old
one deprecated for backwards compatibility – new
implementations will have to accept either Resource
on input
• Revise descriptions of existing tags to make clear what
goes with billing period etc.
• Provide documentation on how to interpret query
parameters for GET UsageSummary
UsageSummary Use Cases
•
•
•
•
I ask for summary today (with day later publishing)
I ask for 3 months last year (query parameters?)
Billing period is non-calendar
John: I ask for an arbitrary period “roll-up” of
consumption
• GET UsageSummary
min=1/15/2014&max=2/28/2014&rollup=True
– In PGE concept – you get 2 UsageSummarys – one for
billing period January and one for billing period February
– In John concept – you get 1 UsageSummary with totals for
1/15..2/28
Documentation Issues
• Do in GreenButtonAuthorization.docx section
2.4
– Use Cases for Authorization Termination
(revocation) -– DC-oriented control of termination process due
regulatory requirements in Ca.
• Do in section 2.8
– Behavior of GET UsageSummary with query
parameters
Service Request 83 – including Function Block for optional customer info
(service point address, etc.)
•
Requirements
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
•
UsagePoints of RetailCustomer
Location of premise
Account ID
Sub Account (SA) ID—Service Agreement / Account is name depending on utility
Customer name, nickname (or short name)
Address and info  from Lynn will provide more information
SDG&E provides only email address and UPID correspondence csv email and UsagePoint ID (Customer Obfuscated Key)
Current ESPI resources will never return PII
GET Subscription does not contain PII
Single Authorization covers entire Subscription and Authorization Scope
MeterID
ServicePointId
Pnode
LoadAggregationPoint, SubloadAggregationPoint
Climate zone
Account open date
Account close date
SA Open and Close date
MDM Agent Id (who does meter read)
ServiceSupplierId
EnergyServiceProviderId (may be same as service supplier)
Demand Response Provider
May need list of Ids for service providers rather than explicit?? (0..* relationship{role, href})
Related assets ???? For example pool pump and pool pump participation in a program.
Related programs ????
Implementation
–
Resource Definition
–
REST service to exchange resource(s)
–
Function Block(s)
–
–
Optionality vs Required
Possible Scope spec
•
•
•
Probably multiple resources are good idea
GET only
Wholesale vs Retail
For May 20 Topics
• Use Case for “verified for billing”
– Added
<xs:enumeration value="19">
<xs:annotation>
<xs:appinfo>revenue-quality</xs:appinfo>
<xs:documentation>data that is valid for billing purposes</xs:documentation>
</xs:annotation>
</xs:enumeration>
• ServiceStatus – return data
– Simple status or, outstanding batchlists
– Consensus: Don’t really need this extension because the
DC can determine if it wants to send a notification of what
hasn’t been retrieved at its discretion.
• Revised Authorization document
• Use Case for Small ThirdParty / Mega ThirdParty …
maybe another day in future
Service Status
• Consensus: Don’t really need this extension because the DC
can determine if it wants to send a notification of what hasn’t
been retrieved at its discretion.
• As is in standard
<?xml version="1.0" encoding="UTF-8"?>
<ServiceStatus xmlns="http://naesb.org/espi" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://naesb.org/espi espiDerived.xsd">
<currentStatus>1</currentStatus>
</ServiceStatus>
• Enhanced to add current outstanding batchlist
<?xml version="1.0" encoding="UTF-8"?>
<ServiceStatus xmlns="http://naesb.org/espi" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://naesb.org/espi espiDerived.xsd">
<extension>text</extension>
<currentStatus>65535</currentStatus>
<batchList>
<resources>../espi/1_1/resource/Batch/Bulk/1?start-index=1&max-results=1000&amp;published-max=2012-04-02T04:00:00Z&amp;
 published-min=2012-04-02T04:00:00Z</resources>
<resources>../espi/1_1/resource/Batch/Bulk/1?start-index=1&max-results=1000&amp;published-max=2012-04-02T04:00:00Z&amp;
 published-min=2012-04-02T04:00:00Z</resources>
</batchList>
</ServiceStatus>
Authorization
• What happens when authorization changes –
UsagePoints or period
– When Authorization changes, place
authorizationUri in notification to ThirdParty
which can then re-establish its state
What can you negotiate with Scope?
• FBTerms – data content, CMD services
• ValueTerms – default durations and blocking,
history length, subscription frequency (i.e.
daily data cycle)
• ResourceTerms – specific resources available
by api, bulkID assignments, bulkaccount
• Other?
Scope Negotiation
Oversimplified sequence diagram of Use Case #2
showing essence of scope negotiation
TP
DC
RC
Logon
Logon
HTTP Redirect with
Scope={scope1} {scope2} …
Authorization request
Scope={scope2}
Authorization response
Scope={scope2}
access-token
resourceUri
authorizationUri
referenceId
…
Scope issues
•
•
Limit Scope to access-token and minimal exchange requirements
Add list of UPs in a subsequent GET request
– Could include UPs, optional location, additional data
– We would define new resource that has this data
•
Are there options?
–
–
•
FB_XX Minimum data –
» UsagePoint
FB_XY Optional data
» location
Should it be a different namespace and XSD?
– We need to make sure they are mutually exclusive – the usage and the PII containing data
– Namespace and separate schema minimize the opportunity for comingling of data
•
Single authorization with multiple UPs with different scopes
– Don suggested that the scope is a union of capabilities. You need to get the data to see details
– Jerry suggests scope be provided with UP?
CSV from GB Data
GBData.XML
GreenButtonDa
taStyleSheetCS
V.xslt
XSLT
Transform
CSV File that
opens in Excel
========================================
Usage Information
For: Front Electric Meter
========================================
Meter Reading Information
Type of readings: Electricity
Fifteen Minute Electricity Consumption Real energy in kilowatt-hours Two-Phase Residential Service
========================================
Detailed Usage
========================================
Start date: 2012-03-01 00:00 for14 days
========================================
Interval Blockdata for period starting:
2012-03-01 00:00 for 1 day
========================================
Energy consumption time period
Usage(Real energy in kilowatt-hours)
Cost(US Dollar)
Events occurred
2012-03-01 00:00 to 2012-03-01 00:15
0.282
0.01
8
2012-03-01 00:15 to 2012-03-01 00:30
0.323
0.01
7
2012-03-01 00:30 to 2012-03-01 00:45
0.294
0.01
2012-03-01 00:45 to 2012-03-01 01:00
0.331
0.01
2012-03-01 01:00 to 2012-03-01 01:15
0.321
0.01
2012-03-01 01:15 to 2012-03-01 01:30
0.316
0.01
2012-03-01 01:30 to 2012-03-01 01:45
0.299
0.01
Notification
TP
DC
HTTP POST
Content-type: application/atom+xml
<?xml version="1.0" encoding="UTF-8"?>
<BatchList xmlns="http://naesb.org/espi" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://naesb.org/espi espiDerived.xsd">
<resources>https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/Subscription/
1</resources>
</BatchList>
A Couple of Use Cases
• Use Case 1: How to do Gas and Electric in one Authorization
–
–
–
–
An Authorization
Two UsagePoints
One Gas One Electric
Different Scope
• Use Case 2: CISR based Authorization
– Customer logs in has id for utility website
– Each login has multiple electricity accounts
– Each account can be multiple usage points
– Customer login id becomes obfuscated {referenceId} which can be used in
REST Uris of the form: /espi/1_1/resource/RetailCustomer/{referenceId}/**
– Authorization enables a subcriptionID and authorizationID which is (internally)
correlated to the customer and the subselection of usagepoints
Discussion on Authorization Structure
•
•
•
•
•
•
•
•
•
•
Authorization enables the following URLs:
<resourceURI>http://localhost:8080/DataCustodian/espi/1_1/resource/Batch/Subscription/1</resourceURI>
<authorizationURI>http://localhost:8080/DataCustodian/espi/1_1/resource/Authorization/1</authorizationURI>
/espi/1_1/resource/RetailCustomer/<referenceID>/UsagePoint/...
(SA == UsagePoint, CISR == subscription == authorization)
with Access-token
GET /espi/1_1/resource/RetailCustomer/<referenceID>/UsagePoint
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
<feed>
<entry>
<id>urn:uuid:40BE6242-F7E6-4B51-828E-59B5FC0C35F0</id>
<link rel="self" href="https://localhost:8080/DataCustodian/espi/1_1/resource/RetailCustomer/9B6C7065/UsagePoint/5446AF3F"/>
<link rel="up" href="https://localhost:8080/DataCustodian/espi/1_1/resource/RetailCustomer/9B6C7065/UsagePoint"/>
<link rel="related" href="https://localhost:8080/DataCustodian/espi/1_1/resource/RetailCustomer/9B6C7065/UsagePoint/5446AF3F/MeterReading"/>
<link rel="related"
href="https://localhost:8080/DataCustodian/espi/1_1/resource/RetailCustomer/9B6C7065/UsagePoint/5446AF3F/ElectricPowerUsageSummary"/>
<link rel="related" href="https://localhost:8080/DataCustodian/espi/1_1/resource/LocalTimeParameters/01"/>
<title>a galaxy far, far away</title>
<content>
<UsagePoint xmlns="http://naesb.org/espi">
<ServiceCategory>
<kind>0</kind>
</ServiceCategory>
</UsagePoint>
</content>
<published>2012-05-03T04:00:00Z</published>
<updated>2012-05-03T04:00:00Z</updated>
</entry>
...
</feed>
Customer Information Resource
•
Requirements
–
–
–
–
–
–
UsagePoints of RetailCustomer
Location of premise
Account ID
Sub Account (SA) ID -- Service Agreement / Account is name depending on utility
Customer name
SDG&E provides only email address and UPID correspondence csv email and UsagePoint ID
(Customer Obfuscated Key)
– Current ESPI resources will never return PII
– GET Subscription does not contain PII
– Single Authorization covers entire Subscription and Authorization Scope
•
Implementation
–
–
–
–
Resource Definition
REST service to exchange resource(s)
Function Block
Possible Scope spec
NAESB REQ.18 Extended Customer Information
cl a ss NA ESB P A P 10 EU I Di a gr a m
Ser v i ceSuppl i er
+
+
IdentifiedObject +CustomerAuthorisations
CustomerAuthorisation
This data is already
part of the PAP10
parent model to
ESPI – REQ.18
+
+
+
+
+
+
+
+
+CustomerAgreements
authorizationServer :anyURI [0..1]
authorizedPeriod :DateTimeInterval [0..1] 0..*
name :String [0..1]
validityInterval :DateTimeInterval [0..1]
accessToken :String32 [0..1]
publishedPeriod :DateTimeInterval [0..1]
resource :anyURI [0..1]
status :UInt8 [0..1]
0..*
+ServiceSuppliers
+
name :String [0..1]
0..1
+CustomerAgreement
+ServiceDeliveryPoints
0..*
+ServiceDeliveryPoints
0..*
IdentifiedObject
EndDev i ceA sset
C ust omer
+
name :String [0..1]
name :String [0..1]
0..*
+EndDeviceAssets
0..1
+Customer
Ser v i ceDel i v er y P oi nt
+
name :String [0..1]
0..*
+ServiceDeliveryPoints
Ta r i f f P r of i l e
name :String [0..1]
0..1
+TariffProfile
Ser v i ceC a t egor y
+
cl a ss C ust omer sO v er v i ew
+ kind :ServiceKind [0..1]
0..1
+ServiceCategory
+UsagePoints
P osi t i onP oi nt
+UsagePoint
+
+UsagePoint
+
0..*
0..1
+UsagePoints
0..*
+PositionPoint +
IdentifiedObject
0..1
0..*
UsagePoint
+
+
+
+
1..*
C ust omer A gr eement
+Customers 0..*
+
kind :SupplierKind [0..1]
name :String [0..1]
xPosition :String [0..1]
yPosition :String [0..1]
zPosition :String [0..1]
0..1
name :String [0..1]
description :String [0..1] +UsagePoint
status :Integer
roleFlags :RoleFlags
This data is part of CIM and associated
with CustomerAgreement
ServiceLocation may be equal to
ServiceDeliveryPoint which is no longer
in CIM
WorkLocation
Ser v i ceLoca t i on
+
+
+
accessMethod :String [0..1]
siteAccessProblem :String [0..1]
needsInspection :Boolean [0..1]
::Location
+
+
+
+
+
+
+
+
+
type :String [0..1]
mainAddress :StreetAddress [0..1]
secondaryAddress :StreetAddress [0..1]
phone1 :TelephoneNumber [0..1]
phone2 :TelephoneNumber [0..1]
electronicAddress :ElectronicAddress [0..1]
geoInfoReference :String [0..1]
direction :String [0..1]
status :Status [0..1]
::IdentifiedObject
+
+
+
+
aliasName :String [0..1]
description :String [0..1]
mRID :String [0..1]
name :String [0..1]
Common Information Model (CIM) Customer
Overview
IEC 61968 and IEC 61970
cl a ss C ust omer sO v er v i ew
Document
+CustomerAccounts
0..* +
+
+CustomerAccount
billingCycle :String [0..1]
budgetBill :String [0..1]
1
+Customer 1
C ust omer
kind :CustomerKind [0..1]
specialNeed :String [0..1] +Customer
pucNumber :String [0..1]
1
status :Status [0..1]
priority :Priority [0..1]
locale :String [0..1]
«deprecated»
+ vip :Boolean [0..1]
+
+
Agreement
loadMgmt :String [0..1]
+CustomerAgreements
+PricingStructures
+CustomerAgreements
0..*
+ServiceCategory
0..*
startDate :Date [0..1] 0..*
endDate :Date [0..1]
+PricingStructures +
+
0..* +
+
+
+
0..1
IdentifiedObject
Ser v i ceC a t egor y
0..*
Document
+Tariffs
accessMethod :String [0..1]
siteAccessProblem :String [0..1]
needsInspection :Boolean [0..1]
+CustomerAgreements
0..*
C ust omer A gr eement
0..* +
+
+
+
+
P r i ci ngSt r uct ur e
Document
Ta r i f f
0..*
+CustomerAgreements
Ser v i ceLoca t i on
+ServiceLocations
+CustomerAgreements
0..*
OrganisationRole
+
+
+
+
+
+
WorkLocation
C ust omer A ccount
code :String [0..1]
revenueKind :RevenueKind [0..1]
taxExemption :Boolean [0..1]
dailyEstimatedUsage :Integer [0..1]
dailyCeilingUsage :Integer [0..1]
dailyFloorUsage :Integer [0..1]
kind :ServiceKind [0..1]
+ServiceCategory
+PricingStructures
0..*
1
UsagePoint (from espiderived.xsd)
Obfuscated tariff ID
Obfuscated
customerAgmtID
Possible Arrangement of Data
“pulling the string”
RetailCustomer
UsagePoint
ServiceLocation
Customer
Agreement
EndDeviceAsset
Normal ESPI
Resources
ServiceSupplier
PostionPoint
Key
TariffProfile
Account Resource
Authorization
Existing Resource
ERP Resource
Possible Arrangement of Data
“pulling the string”
RetailCustomer
UsagePoint
EndDeviceAsset
ServiceLocation
Customer
Agreement
Key
New Resource
PostionPoint
Authorization
Existing Resource
Non-resource
included
TariffProfile
ServiceSupplier
FB3 - Core REST Services
– [TR_CR003] Verify ReadServiceStatus returns
“active” status
FB31 - Core REST Services
– [TR_CR001] Verify the Authorization can be retrieved using the
authorizationUri (from the authorization process in FB-14 or FB-40)
– [TR_CR002] Verify the Authorization resource does not contain PII by
inspection
– [TR_CR003] Verify ReadServiceStatus returns “active” status
– [TR_CR004] Verify Batch/Subscription/{subscriptionId} returns a valid Atom
feed with all UsagePoints and related data including all interval data
– [TR_CR005] Verify structured URIs are of the form
{DataCustodianResourceEndpoint}[/{keyterm}/{id}]* based on the structure of
Green Button APIs
– [TR_CR006] Verify /RetailCustomer/{retailCustomerID}/UsagePoint Returns list
of UsagePoints only under the Authorization
– [TR_CR007] Verify
Batch/RetailCustomer/{RetailCustomerId}/UsagePoint/{UsagePointId} Returns
all data under and including a single UsagePoint
– [TR_CR008] Verify that resources returned by the resourceUri are valid to the
schema, proper linking, and verify that the data meets the test requirements
based on PICS for content and consistency
FB 13: Security Testing
• Cyber Security and Privacy Test Requirements
– Based on Authorization.docx section 2.7
• From SGIP SGCC Committee review of REQ.21
• Reviewed with NIST Cyber Security staff
• NAESB REQ.21 section
• Initial set of test requirements on next slide
Initial Set of Test Requirements
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
[TR_TC001] Test software shall issue a service request over an SSL session and shall verify that the response HTTP header contains the following fields
and information – fields TBD
[TR_TC002] Verify that REST request headers include – fields TBD
[TR_TC003] Verify that the Data Custodian implements TLS 1.2.
[TR_TC004] Verify that when communicating with a Retail Customer the Data Custodian negotiates the highest level of TLS mutually supported.
[TR_TC005] Verify that when communicating with a Retail Customer the Data Custodian rejects TLS_RSA_WITH_NULL_SHA cipher suites.
[TR_TC006] Verify that when communicating with a Retail Customer at a minimum the Data Custodian accepts the
TLS_RSA_WITH_AES_128_CBC_SHA cipher suite.
[TR_TC007] Verify that when communicating with a Third Party the Data Custodian negotiates the highest level of TLS mutually supported.
[TR_TC008] Verify that the Data Custodian maintains an unexpired unrevoked RSA certificate with a public key length of at least 2048 bits.
[TR_TC009] Test software or manual inspection shall verify that the Data Custodian RSA certificate was issued by a Certificate Authority (CA) that has
been successfully audited according to the criteria of ETSI or WebTrust.
[TR_TC010] Test software or manual inspection shall verify that Tokens and IDs communicated by the Data Custodian are opaque and if based on
actual Customer information that they are randomized using a secure method to protect privacy.
[TR_TC011] Test software or manual inspection shall verify that Tokens and IDs communicated by the Data Custodian consist of at least 48 bits and
can be the random number part of an RFC2422 UUID.
[TR_TC012] Manual inspection of supporting documentation shall confirm that the Data Custodian implementation utilizes software libraries which
are FIPS 140-2 level 1 or higher and listed on the CMVP website.
[TR_TC013] Verify that the Third Party implements TLS 1.1 or higher.
[TR_TC014] Verify that when communicating with a Retail Customer the Third Party negotiates the highest level of TLS mutually supported.
[TR_TC015] Verify that when communicating with a Data Custodian the Third Party negotiates the highest level of TLS mutually supported.
[TR_TC016] Verify that the Third Party maintains an unexpired unrevoked RSA certificate with a public key length of at least 2048 bits.
[TR_TC017] Test software or manual inspection shall verify that the Third Party RSA certificate was issued by a Certificate Authority (CA) that has
been successfully audited according to the criteria of ETSI or WebTrust.
[TR_TC018] Test software or manual inspection shall verify that Tokens and IDs communicated by the Third Party are opaque and if based on actual
Customer information that they are randomized using a secure method to protect privacy.
[TR_TC019] Test software or manual inspection shall verify that Tokens and IDs communicated by the Third Party consist of at least 48 bits and can
be the random number part of an RFC2422 UUID.
[TR_TC020] Manual inspection of supporting documentation shall confirm that the Third Party implementation utilizes software libraries which are
FIPS 140-2 level 1 or higher and listed on the CMVP website.
[FB_14] Authorization and
Authentication (Oauth 2.0)
– Verifying response to invalid authorization request
(invalid access-token for resource)
– Verify rejection of request missing access-token
– Missing header parameters
– Invalidation of access-token at end of
authorization period
Function Blocks for CMD
FunctionBlocks for Green Button Connect My Data
[FB_3] Core Green Button Connect My Data
[FB_13] Security and Privacy classes
[FB_14] Authorization and Authentication (Oauth 2.0)
[FB_19] Partial update data
[FB_31] Core Rest Services
[FB_32] Resource Level REST
[FB_33] Management REST Interfaces
[FB_34] SFTP for Bulk
[FB_35] REST for Bulk
[FB_36] Third Party (Client) Dynamic Registration
[FB_37] Query Parameters
[FB_38] On Demand Requests
[FB_39] PUSH model
[FB_40] Offline RetailCustomer Authorization to Complement
OAuth
[FB_42] Third Party Core REST Services
[FB_43] Third Party Management REST Services
[FB_xx] Not a Function Block (Implementation Specific)
[FB_44] Security and Privacy for Simple Third Party
[FB_45] Security and Privacy for Certificate-based Third Party
Description
Core Services
HTTPS support
Oauth
IntervalBlocks without full data sets – e.g. just entrys containing
IntervalBlocks
Third Party Access to Subscription/Authorization
Third Party Access to UsagePoints, MeterReading, … and collections
GET PUT POST DELETE individual resources …
Optionally support the SFTP delivery of Bulk for Bulk request
Support the REST request for Bulk
Use Case 1
Without Notification
Notification followed by GET
This is a out of band authorization process without the
automated OAuth protocol exchange but producing the same
artifacts.
Implementation Specific RESTful API
Opaque vs Structured URIs
• No structure, support Opaque URIs using either HTTPS or FTPS
protocols in conjunction with the espiDerived.xsd schema. Make
Opaque URIs part of the CORE CMD function block.
• Optional support Structured URIs using either HTTPS or FTPS
protocols: make Opaque URIs part of the CORE CMD function
block, and structured URIs an optional Function Block in CMD
Testing & Certification in conjunction with
the espiDerived.xsd schema
• Required Structure, make structured URIs a requirement but allow
some variability – e.g. User versus RetailCustomer; Thus structured
URIs would be part of the CORE CMD function block in CMD Testing
& Certification in conjunction with the espiDerived.xsd schema.
•   Specific Required Structure based on espiDerived.xsd
Resource Names as described in two
documents: GreenButtonAtomLinks and Authorization document
Changes in espiderived.xsd from
espi.xsd
• *Enumerations: The largest volume of changes is in the explicit
documentation of the many enumerations in the standard. In the
standard, only a few examples from the IEC standard were provided
in a comment. Values that distinguish measurements of Wh, W, VAr,
VA, gas, water, etc… are tested for in DMD if corresponding FBs are
indicated.
• *Errors of data type corrected – value, cost, and currency all had
deficient data types that were recognized early on
• *Representation of conversion factors from UTC to Local Time:
LocalTimeParameters resource was added
• *Missing overallConsumptionLastPeriod was added to make
ElectricPowerUsageSummary rational as a record of billing period
consumption
• Support for OAuth 2.0: the second largest volume of changes to the
schemas is in support of CMD (no impact to DMD)
* Differences tested for in T&C
Test Requirements for CMD
Brainstorm
FB_3 Core Green Button Connect My Data
FB_13 Security and Privacy classes
FB_14 Authorization and Authentication (Oauth 2.0)
FB_19 Partial update data
FB_31 Core Rest Services
FB_32 Resource Level REST
FB_33 Management REST Interfaces
FB_34 SFTP for Bulk
FB_35 REST for Bulk
FB_36 Third Party (Client) Dynamic Registration
FB_37 uery Parameters
FB_38 On Demand Requests
FB_39 PUSH model
FB_40 Offline Authorization to Complement OAuth
FB_42 Third Party Core REST Services
FB_43 Third Party Management REST Services
FB_xx ot a Function Block (Implementation Specific)
• FB31 - Core REST Services
– Verify the authorization can be retrieved
– Lack of PII
– Ditto Batch/Subscription, Batch/RetailCustomer, and
UsagePoint
– Verify that resource returned is valid to schema and
links are correct
– Verify structured URIs
– Verify all required content is present (based on PICs)
– Could be FB_14
• Verifying response to invalid authorization request (invalid
access-token for resource)
• Verify rejection of request missing access-token
• Missing header parameters
• Invalidation of access-token at end of authorization period
For February 25
• John Teeter raises issue of path vs opaque
URIs for REST services for individual and
subscription resources
– Does the uri give any indication of what will be
retrieved or not?
Some URIs Found In GBDMD Files
•
URI ::= protocol://hostname:port/datacustodian/espi/1_1/resource/  resource endpoint of the server
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
<link rel="self" href="User/9b6c7063/UsagePoint/01"/>
<NS:link rel="self" href="/User/9b6c7063/UsagePoint/01"/>
<atom:link href="User/9b6c7063/UsagePoint/01" rel="self"/>
<link rel="self" href="User/25cd2af5c5f6f693f8f6d62852033843/UsagePoint/01" />
<link rel="self" href="RetailCustomer/10/UsagePoint/01"/>
<link href="RetailCustomer/115973279529374200002937445377/UsagePoint/0" rel="self"/>
<link rel="self" href="RetailCustomer/765786587/UsagePoint/01" />
<link rel="self" href="/User/9b6c7063/UsagePoint/01"/>
<atom:link href="User/9b6c7063/UsagePoint/01" rel="self"/>
<link rel="self" href="/User/9b6c7064/UsagePoint/01"/>
<atom:link href="/User/9b6c7063/UsagePoint/01" rel="self"/>
<link rel="self" href="/RetailCustomer/1/UsagePoint/J2753386"/>
<link href="/v1/User/455/UsagePoint/580" rel="self"></link>
<link rel="self" href="User/9b6c7063/UsagePoint/01"/>
<link rel="self" href="User/9b6c7063/UsagePoint/01"/>
<link rel="self" href="User/9e610ca8441264b3d21cad5b2a13d028/UsagePoint/01" />
<link href="/v1/User/12704625/UsagePoint/4218907" rel="self"></link>
<link href="/User/4685/UsagePoint/67" rel="self"/>
<link rel="self" href="User/00e308198d020442995dea12c013f77a/UsagePoint/01" />
<link rel="self" href="/User/1564408+15644/UsagePoint/0"/>
Opaque URIs
• Opaque URIs
– No need to test structure
– No need to recognize structure in sw
• Structured URIs
– Easier to recognize the links
– Easier to validate what you are doing by looking at them
– If I have interval block, I know all the possible URIs for that UsagePoint
• Possible Outcomes of OpenADE Discussion?
– No structure, support opaqueness
– Optional Structure, make structured URIs an optional Function Block
– Required Structure, make structured URIs a requirement but allow
some variability – e.g. User versus RetailCustomer
–  Single Required Structure – defined structure based roughly on
GreenButtonAtomLinks and Authorization documents
SFTP for Bulk Transfer
•
Pertinent to the SFTP discussion are the concepts that each Third Party has a defined relationship with the Data
Custodian.
–
–
–
•
For the purposes of Bulk transfer, this URI will be:
–
–
•
How to initiate the SSH connection?
What is the role if any of the client_credentials authorization to control access to SFTP enabled resources?
Discussion –
–
–
•
sftp://hostname:port/DataCustodian/espi/1_1/resource/Batch/Bulk/{bulkId}
where {bulkId} is a unique identifier assigned by the Data Custodian and the balance of the URI is presented in the
ApplicationInformation resource that both parties share (contains all relevant URIs and data for interchange via OAuth etc…).
The Third Party would then retrieve the bulk data by using an SFTP client with that URI. This is a straw man concept for
discussion on the call. Its advantage is that it in harmony with overall architecture of the Green Button Connect My Data RESTful
architecture and simply adds SFTP as a means of transfer when a large data set is to be returned.
Used to Retrieve the data using SFTP protocols
–
–
•
For automated exchange of information about his relationship there is a special Authorization obtained in Use Case #1 (see the
Authorization.docx -http://osgug.ucaiug.org/sgsystems/OpenADE/Shared%20Documents/Testing%20and%20Certification/GreenButtonTestPlan/ref
erenceMaterial/GreenButtonAuthorization.docx).
We anticipate that when the Data Custodian has data available, it sends an asynchronous Notification to the Third Party.
This Notification provides URIs of note that it is assumed the Third Party will want to retrieve.
After authorization of TP, they use Pene test, so what is benefit of access-token?
sftp user:pw, user=<tpname>, password=<tp client-credentials access-token>
Summary
–
–
sftp://hostname:port/DataCustodian/espi/1_1/resource/Batch/Bulk/{bulkId}
sftp user:pw, user=<tpname>, password=<tp client-credentials access-token>
Function Blocks for CMD
FunctionBlocks for Green Button
Connect My Data
[FB_3] Core Green Button Connect My Data
[FB_13] Security and Privacy classes
[FB_14] Authorization and Authentication (OAuth)
[FB_19] Partial update data
[FB_31] Core Rest Services
[FB_32] Resource Level REST
[FB_33] Management REST Interfaces
Description
Core Services
HTTPS support
Oauth
IntervalBlocks without full data sets (Ups,MR, …)
Third Party Access to Subscription/Authorization
Third Party Access to UsagePoints, MeterReading, … and
collections
GET PUT POST DELETE individual resources …
Optionally support the SFTP delivery of Bulk for Bulk
request
Support the REST request for Bulk
Use Case 1
[FB_34] SFTP for Bulk
[FB_35] REST for Bulk
[FB_36] Third Party (Client) Dynamic Registration
[FB_37] Query Parameters
[FB_38] On Demand Requests
Without Notification
[FB_39] PUSH model
Notification followed by GET
[FB_40] Offline Authorization to Complement OAuth
[FB_42] Third Party Core REST Services
[FB_43] Third Party Management REST Services
[FB_xx] Not a Function Block (Implementation Specific) Implementation Specific RESTful API
• Authorization Sequence
– Scope
– access-token
– Refresh-token
– resourceUri (the subscription)
– authorizationUri
– expiration of the access-token and refresh-token
– token-type
Proposed CMD Function Blocks
FunctionBlocks for Green Button Connect My Data
Description
[FB_3] Core Green Button Connect My Data
[FB_13] Security and Privacy classes
[FB_14] Authorization and Authentication (OAuth)
[FB_19] Partial update data
[FB_31] Core Rest Services
[FB_32] Resource Level REST
[FB_33] Management REST Interfaces
[FB_34] SFTP for Bulk
[FB_35] REST for Bulk
[FB_36] Third Party (Client) Dynamic Registration
[FB_37] Query Parameters
[FB_38] On Demand Requests
[FB_39] PUSH model
[FB_40] Offline Authorization to Complement OAuth
NEED to Discuss
[FB_42] Third Party Core REST Services
[FB_43] Third Party Management REST Services
[FB_xx] Not a Function Block (Implementation Specific)
Core Services
HTTPS support
Oauth
IntervalBlocks without full data sets (Ups,MR, …)
Third Party Access to Subscription/Authorization
Third Party Access to UsagePoints, MeterReading, … and collections
GET PUT POST DELETE individual resources …
Optionally support the SFTP delivery of Bulk for Bulk request
Support the REST request for Bulk
Use Case 1
Without Notification
Notification followed by GET
Implementation Specific RESTful API
Draft of API Allocations to FBs
Function Blocks
CRUD
API URL
[FB_3] Core Green Button
Connect My Data
GET
resource/ReadServiceStatus
[FB_31] Core Rest Services
GET
resource/ApplicationInformation/{ApplicationInformationID}
[FB_31] Core Rest Services
PUT
resource/ApplicationInformation/{ApplicationInformationID}
[FB_31] Core Rest Services
[FB_31] Core Rest Services
[FB_31] Core Rest Services
[FB_31] Core Rest Services
[FB_31] Core Rest Services
[FB_31] Core Rest Services
[FB_31] Core Rest Services
DELETE
GET
PUT
DELETE
GET
GET
GET
[FB_31] Core Rest Services
GET
[FB_31] Core Rest Services
GET
[FB_31] Core Rest Services
GET
[FB_31] Core Rest Services
[FB_31] Core Rest Services
[FB_31] Core Rest Services
GET
GET
GET
https://services.greenbuttondata.org/DataCustodian/espi/1_1/RetailCustomer/{RetailCustomerID}/UsagePoint/{UsagePointID}/ElectricPowerUsageSumary
https://services.greenbuttondata.org/DataCustodian/espi/1_1/RetailCustomer/{RetailCustomerID}/UsagePoint/{UsagePointID}/ElectricPowerUsageSumary/{ElectricP
owerUsageSummaryID}
resource/RetailCustomer/{RetailCustomerID}/UsagePoint/{UsagePointID}/MeterReading/{MeterReadingID}/IntervalBlock
resource/RetailCustomer/{RetailCustomerID}/UsagePoint/{UsagePointID}/MeterReading/{MeterReadingID}/IntervalBlock/{IntervalBlockID}
[FB_31] Core Rest Services
GET
resource/LocalTimeParameter
[FB_31] Core Rest Services
[FB_31] Core Rest Services
[FB_31] Core Rest Services
[FB_31] Core Rest Services
[FB_31] Core Rest Services
[FB_31] Core Rest Services
[FB_31] Core Rest Services
[FB_31] Core Rest Services
[FB_31] Core Rest Services
[FB_31] Core Rest Services
GET
GET
GET
GET
GET
GET
GET
GET
GET
GET
resource/LocalTimeParameter/{LocalTimeParameterID}
resource/MeterReading
resource/MeterReading/{MeterReadingID}
resource/RetailCustomer/{RetailCustomerID}/UsagePoint/{UsagePointID}/MeterReading
resource/RetailCustomer/{RetailCustomerID}/UsagePoint/{UsagePointID}/MeterReading/{MeterReadingID}
resource/ReadingType
resource/ReadingType/{ReadingTypeID}
resource/Subscription/{SubscriptionID}
resource/RetailCustomer/{RetailCustomerID}/UsagePoint
resource/RetailCustomer/{RetailCustomerID}/UsagePoint/{UsagePointID}
resource/ApplicationInformation/{ApplicationInformationID}
resource/Authorization/{AuthorizationID}
resource/Authorization/{AuthorizationID}
resource/Authorization/{AuthorizationID}
resource/Batch/Subscription/{SubscriptionID}
resource/Batch/RetailCustomer/{retailCustomerID}/UsagePoint
resource/Batch/RetailCustomer/{RetailCustomerId}/UsagePoint/{UsagePointId}
https://services.greenbuttondata.org/DataCustodian/espi/1_1/RetailCustomer/{RetailCustomerID}/UsagePoint/{UsagePointID}/ElectricPowerQualitySummary
https://services.greenbuttondata.org/DataCustodian/espi/1_1/RetailCustomer/{RetailCustomerID}/UsagePoint/{UsagePointID}/ElectricPowerQualitySummary/{Electri
cPowerQualitySummaryID}
Term
Scope
FBTerms
FBTerm
ValueTerms
Expansion
Scope
[ FBTerms ], [ ValueTerms ], [ ResourceTerms ];
“FB=“, { [FBTerm , ”_”} , FBTerm, ScopeDelimiter ;
“4” | “5” | “6” | “7” | “8” | “9” | “10” | “11” | “12” | “15” | “16” | “17” | “18” | “19” | “27” | “28” | “29”
{ ( "IntervalDuration=", nonNegativeNumber | namedFrequency),
| ( "BlockDuration=", nonNegativeNumber | namedFrequency),
| ( "HistoryLength=", nonNegativeNumber),
| ( "SubscriptionFrequency=", nonNegativeNumber | namedFrequency), ScopeDelimiter };
{ (“ApplicationInformation,” | “Authorization,” | “UsagePoint,” | “IntervalBlock,” | “MeterReading,” |
“ElectricPowerQualitySummary,” | “ElectricPowerUsageSummary,” | “ReadingType,” | “Subscription,” |
ResourceTerms
“LocalTimeParameters,” | (“BulkAccountCollection=”, nonNegativeNumber) | “BR=”, brID), ScopeDelimiter}
ScopeDelimiter
“;”
namedFrequency
“billingPeriod” | “daily” | “monthly” | “seasonal” | “weekly” |
nonNegativeNumber digit, { digit };
digit
0 | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9" ;
Where:
The ESPI resource – default is “Subscription”. If a Bulk resource is specified via the “BR” term, the value of the {bulkID}
is provided after the equals sign (“=”). There could be one or more terms in this list that express the granularity of
ResourceTerms
notifications about resource changes.
FBTerms
The function blocks supported (only data content FBs are listed)
ValueTerms
These are parameterized terms
IntervalDuration
This is the minimum default length of an interval in seconds (e.g. 900 for 15 minutes, 3600 for one hour, …)
This is the length of a block that contains the intervals (based on enumeration of MacroPeriodKind in ESPI above as
BlockDuration
namedFrequency)
This is the length of history buffer of records in number of Interval Blocks (e.g. 12 for a year if BlockDuration is
“monthly”). ote: this is what the DataCustodian offers; however, the buffer may not be full for transitional metering
HistoryLength
systems; in these cases less data will be returned until the buffer is full.
Used where the DC wants to provide for the reporting of multiple UsagePoints in a single Subscription. The number of
UsagePoints is represented by the value in the assignment statement – e.g. 4 UsagePoints would be
BulkAccountCollection BulkAccountCollection=4.
Green Button Connect My Data Testing
and Certification
•
Complete function block descriptions
–
Current:
•
•
•
•
–
[FB_3] Green Button Connect My Data
[FB_13] Security and Privacy classes
[FB_14] Authorization and Authentication (OAuth)
[FB_19] Partial update data
New?:
•
Core Rest Services
–
–
GET Batch/Subscription
…
•
Resource Level REST
•
•
•
•
•
•
•
SFTP for Bulk
REST for Bulk
Use Case 1: Client Registration
Query Parameters
On Demand Requests (as opposed to Notification followed by GET)
PUSH model
Offline Authorization to Complement OAuth – should this be outside the scope of standard and testing or standardized?
–
–
–
–
–
–
–
GET PUT POST DELETE individual resources …
No standard isolated way to get the token to a third party without OAuth
On exceptional basis some customers can’t be required to use a web account
Sometime commercial accounts don’t need privacy and want a service provider just to register the data.
Could use Notification service to tell TP about new authorizations made by DC. Out of band how RetailCustomer is identified to the TP
“transitive” model TP gets bulk data from DC and then becomes DC – can this architecture be of help here?
Possible provision by DC of access token for conveyence to thirdparty devoid of customer information. Maybe even encrypted for TP as in software
activations:
»
“Please provide this to your TP (the text between the ====)
»
=============================================
»
ashoiqwherfhdjnvcjq2dhijvkqnvoiikdfv
»
=============================================“
Questions
• retailCustomerID=authorization=subscription
– Corresponds to a single authorization
– Results in one or more usagePoints being associated with subscription
– Scope=
“FB=4,5,15;IntervalDuration=3600;BlockDuration=monthly;HistoryLen
gth=13;BulkAccountCollection=10”
• Says that the BulkAccountCollection has 10 usage points
Authorization provides two URIs that can be used:
resourceUri  GET this to retrieve usage data (all UPs)
authorizationUri  GET/PUT details of Authorization
Notification is a list of URIs
All nested resources under the UPs are accessible under the single
authorization
Service Request 83 – including Function Block for optional customer info
(service point address, etc.)
Service Request 84 – having scope selection screen on Data Custodian Site vs
3rd Party site
[85] Time of Use tier indicator alignment with
SEP 2.0
Here is a list of topics raised by you all that
we will touch on
• Issues Raised and Implementation Questions
– How to use BR=bulkID – relates to HD #61
– Service Request 83 – including Function Block for
optional customer info (service point address, etc.)
– Service Request 84 – having scope selection screen on
Data Custodian Site vs 3rd Party site
– Tariff Model Resource
• Green Button Connect My Data Testing and
Certification
– Complete function block descriptions
– Complete test case requirements
How to use BR=bulkID – relates to HD #61
• Application Profiles
– BulkID was proposed for large sets of authorizations
– One account level authorization on top of service level
accounts – how to do this
• Degrees of freedom we have now – can we cover
– Subscription – 1 or more Usage Points
• Granularity of a customer authorization
– BulkID
• “macro” for a large set of existing authorizations
– Is there another degree needed?
Contributed by Jerry Yip

Clarification/confirmation about ESPI standard: Does ‘shared resource key’ referenced in the
NAESB Ratified word doc correspond to Access Token for oAuth?
 Yes: This is the access token in the new Oauth 2.0 paradigm.

Formal Submission of Application Profile for bulk (vs. batch?) use case as part of GB/GBC
Conformance Testing Plan

Question: (options to address 1 Acct to many SA issue)
- Does UUID correspond to usage point (1-to-1 relationship)? Is there passing of UUIDs (as resource
terms in Scope section of GBAuthorization) during authorization sequence? (how would 3rd Party
know multiple usage points have been authorized via single oAuth sequence/login?)
- Can multiple access tokens be issued (1 token per SA) per oAuth session?
 Write up coming to test concept of BulkIDs
 An Authorization is one access_token
 How does Third Party get to know the depth of data (how many Ups) are in the authorization



 Perhaps an extension of scope string to have numUPs?
Request to consider scope selection screens at Data Custodian Portal instead of 3rd party portal
(Need customer to select SAs to share – only Data Custodian has that info) – also minimizes number
of redirects (?)
Customer info as optional functional block (atom feed) for authorization (sharing with 3Ps)
John suggests – prep a large multi account data set and test against a reference sw implementation
and measure. SFTP and Streaming, compressed and non-compressed method and compare.
=
How to use BR=bulkID with application to account and
account groupings, as well as, large ThirdParty
collections of Authorizations
• Establish Use Case Story for Commercial
Accounts
• Design Scope String(s) that convey it
• Repaint the storyboard with appropriate
content
Application Profile
•
Per footnote 1, pg 20 of GBAuthorization.doc:
–
•
•
•
•
•
Suggest: new FB or Application Profile to properly capture this scenario
[FB_31] Web Customer Manages Multiple Customer Accounts
(OR: 3.9 Application Profile)
For GBCMD, this FB/AP contains tests associated with a Web Customer accessing a Data Custodian’s Web Portal to manage multiple customer
accounts. Upon log in to the Data Custodian’s Web Portal, the web customer can manage multiple customer accounts, for which each customer
account can represent multiple usage points (for electricity and/or gas). This mostly impacts large agricultural and commercial customer accounts for
which a single web customer can represent hundreds to thousands of individual usage points – imagine a franchise manager with multiple branch
locations across a data custodian’s service territory.
In this scenario, the Web Customer should have the ability to authorize, deauthorize and change scope on an individual “usage point” basis and
optionally at the larger aggregated web customer or customer account basis. This includes the ability to perform one-time authorization of multiple
customer accounts by a single web customer to third party, and any subsequent scope changes (whether on an aggregated or individual basis) – third
party acknowledgement/communication of which customer accounts have been authorized, deauthorized or whose scope has changed needs to be
determined.
Notes:
–
–
–
–
–
–
•
A “Web Customer” may actually manage more than one “Retail Customer” where “Retail Customer” is an actual “Customer Account”. Thus identifying the
specific Retail Customer may be part of the scope selection on both sides. The scenarios in this section refer to the “Retail Customer” for simplicity.
Whether scope selection in this scenario should live on the 3rd party portal vs. the Data Custodian’s portal needs to be determined as well.
Collection has one description or multiple?
What is the scope string for this use case?
Is there a need for a bulkId in this case (maybe not).
ew Scope Resource Term= “BulkAccountCollection”
Scope= “FB=4,5,15;IntervalDuration=3600;BlockDuration=monthly;HistoryLength=13;BulkAccountCollection”
1/14/2014
–
To allow the TP to know how many Ups are being provided, suggest Add to BulkAccountCollection a number of UsagePoints BulkAccountCollection=nnn
UsagePoint Grouping in Commercial
Account Management
BulkId
Person
/web account
Via gui
1:n
Premise
Account
SubscriptionId
1:n
1:n
Service
Agreement
1:n
Service Point
1:1
UsagePointId
Meter
1:1
HAN Device
Scope= “FB=4,5,15;IntervalDuration=3600;BlockDuration=monthly;HistoryLength=13;BulkAccountCollection”