UNIFYING DEVOPS MODEL
Who generates the Generators?
ABOUTME.GROOVY
AboutMe.create {
name 'Stephan Pauxberger'
twitter '@black_build'
email 'stephan@blackbuild'
work {
title 'Buildmanager'
modeOfOperation FREELANCER
clients (
'Big companies',
'specific projects',
'usually green or brown field assignments'
)
}
}
WALKING THE LINE
THE PROBLEM
It‘s groundhog day
YOU ARE DOING GREAT!
• You set up your environments via Puppet
• You embrace CI/CD
• Your Jenkins jobs are automatically generated
• You have useful and up to date documentation in Confluence
• Your environment is fully monitored via Icinga
BUT WAIT!
LET‘S ADD A NEW ENVIRONMENT
• Create Puppet manifest(s)
• Generate Jenkins Deploy Jobs / Pipelines
• Request and setup a database
• Request firewall changes via support ticket
• Update system documentation
• Setup Icinga
• …
• Rinse and repeat
WE SHOULD HAVE KNOWN BETTER…
Don‘t
Repeat
Yourself
Single
Source
of Truth
THE APPROACH
Find yourself a model that can do both
WHEN IN DEVELOPMENT, DO AS
DEVELOPERS DO
Consume
Commit
Build/
Test
Store
Consume
Consume
WHAT WE NEED
• A Schema
• A Static Model
• A Build Process
• Tests
• Consumers
SCHEMA AND MODEL
• XML
• JSON
• YAML
• Ruby
• Groovy DSL
WHY GROOVY?
• Modular
• Type Safe (when activated)
• IDE support (code completion, tests)
• Loops and Conditions
• Methods
• Great Markup Support
• Lots of Libraries
• Great DSL, less Boilerplate
LET‘S GET SOME HELP
Supermodels don‘t just grow, they are made
LET‘S BUILD A SCHEMA
@DSL class Application {
@Key String name
@Default(field = "name") String title
String description
@Validate(message = "Need at least one environment")
Map<String, Environment> environments
}
@DSL class Environment {
@Key String name
Stage stage
Map<String, ServerGroup> serverGroups
}
… AND A MODEL
Application.create("MyApp") {
Wildfly.withTemplate(version: '8.0', configuration: MINIMAL) {
environments {
environment("Dev") {
stage DEVELOPMENT
serverGroups {
iis("frontend") {
hosts 'a.b.c.d', 'd.e.f.g'
clusterIp '1.2.3.4'
deployUser 'demo'
}
wildfly("application") {
host "abc"
}
}
}
environment("Test") {
…
}
}
}
}
… PUT SUGAR ON TOP
environments {
[1..5].each { count ->
environment("DEV$count") {
stage DEVELOPMENT
serverGroups {
iis("frontend") {
hosts "a.b.c.$count"
deployUser 'demo'
}
wildfly("application") {
host "abc$count"
}
}
}
}
}
def envFolder = new File("environments")
environments(envFolder.listFiles().collect { Environment.createFrom it })
A UNIT TEST
def 'Higher wildfly version must enter lower stages first'() {
given:
Application myApp = Application.createFrom(Configuration)
when:
List<Version> wildflyVersionsSortedByStage = myApp.environments.sort { it.value.stage }.collect { it.value.wildflyVersion }
then: 'versions already sorted'
assertAlreadySorted(wildflyVersionsSortedByStage)
}
AND USE IT
An on to the catwalk, we go
A CONSUMER
Docalot.run {
gather {
Application model = Config.createFromScript(Configuration)
main(PojoCollector) {
sourceObject model
splitter { model.environments }
}
domain(PropertiesCollector, 'status') {
sourceFolder new File("domains/data")
}
domain(FlagCollector, 'disabled') {
sourceFolder new File("domains/data")
}
}
convert {
baseDir new File("docs")
directory(Single) {
source "index"
target "."
}
directory(Splitted) {
source "projects"
}
}
publish {
output(Confluence) {
url "https://confluence.comp.com"
credentialsId "abc-xyz"
}
}
}
JENKINS PIPELINES
Config config = loadModel("Provider", Configuration)
String envName = currentBuild.parameters.targetEnv
Environment env = config.applications.myApp.environments[envName]
stage 'Download' {
withCredentials(...) {
sh "curl $nexusUrl -u $credentials -o deploy.zip"
}
}
stage "Unzip" {
unzip(file: 'deploy.zip')
}
stage "deploy" {
withClassPath(".") {
loadScript("Deploy.groovy").run(env)
}
}
stage "post deploy" {
...
}
WHEN IN BUILDMANAGEMENT, DO AS
DEVELOPERS (SHOULD) DO
Create Jobs
Commit Model
Build/
Test
Store
Configure Pipeline
Configure Deployment
ALL THE GOOD THINGS
• Unit-Tests
• Mocking
• Repeatability
• Tagging / Branching
• Continuous Integration / Inspection / Delivery
CONSUMERS
• Direct (native Java / Groovy)
• Jenkins JobDSL plugin (directly)
• Jenkins Pipelines (via plugin)
• Post Deploy Integration Tests (directly)
• With Templates / Converters (via Docalot or Markup Templates)
• Puppet Manifests
• Support Tickets (for firewall rules, via Rest API of ticket system)
• Documentation (via Docalot / Templates, pushed via CMS API)
FURTHER DOWN THE RABBIT
HOLE
Let‘s put some clothes on
ADVANCED TECHNIQUES:
COMPOSITE MODELS
Live Data (Filesystem,
Application)
ApplicationVersion
Static
getDeployedVersion()
getPreviousVersion()
<<Decorator>>
Environment
LiveEnvironment
EnvState
isBlocked()
getMessage()
getBlockedUntil()
From Icinga
COMPOSITE MODELS IN USE
config.applications.each { appName, app ->
div(class: 'app') {
div(class: 'appHeader', appName) {
app.environments.each { envName, env ->
div(class: 'env') {
div(class: 'title', env.title)
div(class: 'description', env.description)
div(class: 'version', env.version.installedVersion)
if (env.status) div(class: env.status.statusColor, env.status.message)
}
}
}
}
}
ADVANCED TECHNIQUES:
DECORATORS
CoreModel
name
environments
data
getEnvironmentsOfType()
<<Decorator>>
<<interface>>
HtmlProviderModel
HtmlProvider
toHtmlMarkup()
ADVANCED TECHNIQUES:
CRYPTO PROVIDERS
Environment
getPassword()
crypted
<<Decorator>>
<<interface>>
CryptoEnvironment
getPassword()
decrypted
CryptoProvider
LET‘S WRAP IT UP
And create our own collection
MY FUTURE…
RIGHT NOW
FUTURE
• Configure multiple Jenkins instances
• Support tickets
• Create necessary Jenkins Jobs (projects and
deploy jobs) -> 2500 jobs
• More documentation
• (Maven, Gradle, C#, C++, JavaScript with
Grunt / Webkit)
• Configure Jenkins Pipelines -> All Pipelines
have access to their model
• Documentation (System Overviews)
• Deployment
• Icinga
• Puppet
…AND YOUR FUTURE
• Take a look at the examples
• Go out and do some modelling
• Contribute your own examples and techniques
• Live the life and tell the tale
• Enjoy your new freedom as a buildmanagement rockstar
USED TECHNOLOGIES
• Jenkins (Pipeline, JobDSL)
• Klum Suite
• Gradle, to build the model
• KlumAST
• Nexus, to store models
• KlumWrap (to be released as OSS)
• Groovy
• KlumLock (to be released as OSS)
• Docalot
• Custom Jenkins Plugins
BUT, WHAT IS A ‚KLUM‘?
• (K)onvenient Library for Unifying Models
• Suite of libraries for Groovy
• Type safe and IDE friendly models
• Open Source
• A way to create Buildmanagement‘s Next Top
Model
Q&A
[email protected]
THANK YOU!