yardstick Documentation
Release 0.1.0
Kenny Freeman
December 30, 2015
Contents
1
yardstick
1.1 What is yardstick?
1.2 Features . . . . . .
1.3 Future . . . . . . .
1.4 Misc . . . . . . .
1.5 Credits . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
3
3
3
4
4
5
2
Installation
7
3
Usage
9
4
Development Guide
4.1 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11
11
12
5
Contributing
5.1 Types of Contributions .
5.2 Get Started! . . . . . . .
5.3 Pull Request Guidelines
5.4 Tips . . . . . . . . . . .
.
.
.
.
15
15
16
16
17
6
Credits
6.1 Development Lead . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.2 Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19
19
19
7
History
7.1 0.1.0 (2016-01-11) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21
21
8
Indices and tables
23
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
i
ii
yardstick Documentation, Release 0.1.0
Contents:
Contents
1
yardstick Documentation, Release 0.1.0
2
Contents
CHAPTER 1
yardstick
1.1 What is yardstick?
Yardstick is a python daemon for measuring system/service metrics and writing them to one or more destinations (eg.
influxdb, graphite etc). It is designed to allow you to easily create reader and writer plugins by creating a python class
and a corresponding yaml configuration file.
Warning: Yardstick is in development and not ready for use in production. That said, the system is usable in its
current state with the exception of loading plugins from modules. Functionality and code are likely to change.
1.2 Features
Supports metric reader and writer plugins to measure then transport metrics. Currently there are only testing plugins
created (a random metric generator and a metric logger).
• installation via pip/easy_install, run via entry point script yardstick –config /path/to/config
• logs are written to stdout and the system is designed to run under eg. supervisord , runit
• yaml configuration file format for both the main process and plugins
• metric reader and writer plugin system, plugins are loaded on startup based on configuration
• supports writing to multiple destinations simultaneously
• ‘0MQ <http://zeromq.org/>‘+ PUB/SUB connects reader(s) with writer(s) and, if configured, can make metrics
available outside the process (or host) via 0MQ PUB socket
• user defined configuration for 0MQ addressing (eg. ipc://, tcp://)
• internally it uses gevent for concurrency - all plugins and libraries used in them must support gevent and be
reasonably light on the one cpu available process wide
Here’s an example reader plugin:
# -*- coding: utf-8 -*# random metric generator
import random
import time
import sys
from ystick.plugins import reader
3
yardstick Documentation, Release 0.1.0
class randreader(reader):
def poll(self):
"""Return random metrics."""
## random => identifies eg. the service/class of metrics being measured
## name => identifies the metric being measured
m = "random name %d %r"%(random.randrange(sys.maxint), time.time())
self.log.debug("returning", metrics=m)
return m
to use it, create a yaml configuration file that points to either the module or file you created:
## test reader yaml configuration
name: testreader
## give it a uniq a name
file: /full/path/to/file.py
## provide the path to the python file
##module:
## or use the module name
class: randreader
## tell yardstick the class to load
##interval: 10
## optional: a non-default interval for poll()
then tell yardstick where to find your reader configuration:
readers: /path/to/your/config/
##writers: /path/to/writers/
##pub: tcp://127.0.0.1:10102
##sub: tcp://127.0.0.1:10103
##log: debug
##interval: 60
##
##
##
##
##
##
yardstick will load all *.yaml files here
ditto, for writer pluggins
use a non-default ZMQ address for reader pub
use a non-default ZMQ address for writer sub
increase the log level
change the default polling interval
yardstick will also call the onstart() and onstop() methods in your class on startup and shutdown.
1.3 Future
There are some broken/missing and planned features:
• todo: upload to pypi
• broken: loading plugins from modules
• missing: reader and writer plugins for various system/service metrics (eg. cpu, load, mem, disk, net, etc) - most
likely will use psutil
• most likely somewhat broken: better exception handling around various calls (eg. 0MQ sockets)
• nice to have: a processpoller plugin that spawns a configured short running process and reads metrics from
stdout
• nice to have: a processreader plugin that spawns a configured long running process and reads metrics from
stdout (likely just use processpoller)
• nice to have: a server plugin that exposes a 0MQ REQ socket for pull capability (eg. possible to plugin to
monitoring systems in addition to pushing metrics to timeseries)
1.4 Misc
• Free software: MIT license
• Documentation: https://yardstick.readthedocs.org.
4
Chapter 1. yardstick
yardstick Documentation, Release 0.1.0
1.5 Credits
This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.
1.5. Credits
5
yardstick Documentation, Release 0.1.0
6
Chapter 1. yardstick
CHAPTER 2
Installation
At the command line:
$ easy_install ystick
Or, if you have virtualenvwrapper installed:
$ mkvirtualenv yardstick
$ pip install ystick
7
yardstick Documentation, Release 0.1.0
8
Chapter 2. Installation
CHAPTER 3
Usage
To use yardstick you will need a:
• main configuration file eg. /etc/yardstick.yaml
• configuration directory for the readers eg. /etc/yardstick.conf.d/readers/
• configuration directory for the writers eg. /etc/yardstick.conf.d/writers/
• configuration file for each reader and writer that you want to use eg. /etc/yardstick.conf.d/readers/readera.yaml
/etc/yardstick.conf.d/writers/writera.yaml
• means of running it, eg. supervisord, runit or similar system
The main configuration file can be empty and if so yardstick will use the defaults in yardstick.common.default_config:
log:
conf:
readers:
writers:
pub:
sub:
inverval:
onstartto:
onstopto:
warn
/etc/yardstick.conf
/etc/yardstick.conf.d/readers/
/etc/yardstick.conf.d/writers/
127.0.0.1:10100
127.0.0.1:10101
30
5
5
##
##
##
##
##
##
##
##
##
log level: crit, error, warn, info, debug
path to configuration file
path to readers configs
path to writers configs
readers pub here
writers sub here
reader default polling interval
timeout onstart hooks
timeout onstop hooks
Each plugin that you want loaded must have a configuration file in the corresponding directory:
name: testreader
file: /path/to/plugin.py
##module: name.of.your.plugin.module
class: randreader
##interval: <seconds>
##any: other
##config: needed
##
##
##
##
##
##
##
the plugin needs a uniq identifier
either use the full path to the plugin
or the name of the module that can be found in sys.p
the name of the class to instantiate
define a non-default value for the poll() inteval
configuration for your plugin in yaml format
loaded on instantiation and available via self.confi
The setup.py script installs the yardstick command and it supports one argument - the configuration file to load:
yardstick --config /path/to/config.yaml
It is possible to run multiple yardstick processes with different configurations however you must use different end
points for the pub/sub addresses. When yardstick starts up it first launches the pub/sub bridge process and waits until
it is up - it will fail with ‘bind: socket in use’ errors if you run multiple instances with the same configuration and tcp://
endpoint(s).
Note: if you have configured ipc:// addressing you must ensure the path to the sockets exist as yardstick will not create
them.
9
yardstick Documentation, Release 0.1.0
10
Chapter 3. Usage
CHAPTER 4
Development Guide
Yardstick plugins were designed with ease of development in mind and currently supports reader and writer plugins.
The only difference between readers and writers is the name of the method to define. For readers define poll() and for
writers define push().
To create a plugin:
• subclass the ystick.plugins.reader or ystick.plugins.writer class (both are abstract base classes)
• define either the poll() or push() methods (note poll() takes no arguments aside from self, push accepts self and
a metrics string)
• optionally define the onstart() and onstop() methods to hook startup and shutdown (neither accept arguments
aside from self)
The reader and writer plugin objects inherit from ystick.plugins.plugin as a result you can access these instance variables:
• log - an instance returned from a call to structlog.get_logger().bind(src=’name’)
• config - an instance of ystick.config.conf with the plugins configuration loaded
Any logging needed can be done via structlog eg. self.log.info(‘message’, tag=value, tag2=value2) and configuration can be read from self.config.gte(‘key’, default=value). Additionally any default configuration can be set via
self.config.defaults(<dict>).
If you must override the constructor, its signature is __init__(self, log, config).
4.1 Notes
• the system uses gevent and plugin code plus any modules imported must support it
• plugins should not be cpu intensive otherwise they may starve the rest of the system of cpu time potentially
causing poll intervals to be missed and metrics to be dropped
• the system is designed to run io bound workloads and a reasonable number of plugins (eg. tens, perhaps hundreds)
• it is possible to connect to the PUB/SUB sockets (if configured such) and metrics are simply strings - the system
can be extended outside of the yardstick process and code
11
yardstick Documentation, Release 0.1.0
4.2 Examples
Writer plugin:
# -*- coding: utf-8 -*import
import
import
import
random
time
sys
os.path
from ystick.plugins import writer
class nullwriter(writer):
def onstart(self):
"""connect to eg. influxdb, perhaps start a greenlet, etc."""
pass
def onstop(self):
"""close any filehandles, sockets, etc."""
pass
def push(self, metrics):
self.log.debug("received", metrics=metrics)
Reader plugin:
# -*- coding: utf-8 -*import random
import time
import sys
from ystick.plugins import reader
class randreader(reader):
def onstart(self):
"""connect to a service, perhaps start a greenlet, etc."""
pass
def onstop(self):
"""close any filehandles, sockets, etc."""
pass
def poll(self):
"""Return random metrics."""
m = "random n %d %r"%(random.randrange(sys.maxint), time.time())
self.log.debug("returning", metrics=m)
return m
Before yardstick will load it (on next startup) you must define a yaml configuration and place it in either the reader or
writer configuration directory you have set:
## example plugin configuration
name: someplugin
## give it a uniq a name
file: /full/path/to/file.py
## provide the path to the python file
12
Chapter 4. Development Guide
yardstick Documentation, Release 0.1.0
##module:
class: someclass
##interval: 10
4.2. Examples
## or use the module name
## tell yardstick the class to load
## optional: a non-default interval for poll()
13
yardstick Documentation, Release 0.1.0
14
Chapter 4. Development Guide
CHAPTER 5
Contributing
Contributions are welcome, and they are greatly appreciated! Every little bit helps, and credit will always be given.
You can contribute in many ways:
5.1 Types of Contributions
5.1.1 Report Bugs
Report bugs at https://github.com/s4z/yardstick/issues.
If you are reporting a bug, please include:
• Your operating system name and version.
• Any details about your local setup that might be helpful in troubleshooting.
• Detailed steps to reproduce the bug.
5.1.2 Fix Bugs
Look through the GitHub issues for bugs. Anything tagged with “bug” is open to whoever wants to implement it.
5.1.3 Implement Features
Look through the GitHub issues for features. Anything tagged with “feature” is open to whoever wants to implement
it.
5.1.4 Write Documentation
yardstick could always use more documentation, whether as part of the official yardstick docs, in docstrings, or even
on the web in blog posts, articles, and such.
5.1.5 Submit Feedback
The best way to send feedback is to file an issue at https://github.com/s4z/yardstick/issues.
If you are proposing a feature:
15
yardstick Documentation, Release 0.1.0
• Explain in detail how it would work.
• Keep the scope as narrow as possible, to make it easier to implement.
• Remember that this is a volunteer-driven project, and that contributions are welcome :)
5.2 Get Started!
Ready to contribute? Here’s how to set up yardstick for local development.
1. Fork the yardstick repo on GitHub.
2. Clone your fork locally:
$ git clone [email protected]:your_name_here/yardstick.git
3. Install your local copy into a virtualenv. Assuming you have virtualenvwrapper installed, this is how you set up
your fork for local development:
$ mkvirtualenv yardstick
$ cd yardstick/
$ python setup.py develop
4. Create a branch for local development:
$ git checkout -b name-of-your-bugfix-or-feature
Now you can make your changes locally.
5. When you’re done making changes, check that your changes pass flake8 and the tests, including testing other
Python versions with tox:
$ flake8 yardstick tests
$ python setup.py test
$ tox
To get flake8 and tox, just pip install them into your virtualenv.
6. Commit your changes and push your branch to GitHub:
$ git add .
$ git commit -m "Your detailed description of your changes."
$ git push origin name-of-your-bugfix-or-feature
7. Submit a pull request through the GitHub website.
5.3 Pull Request Guidelines
Before you submit a pull request, check that it meets these guidelines:
1. The pull request should include tests.
2. If the pull request adds functionality, the docs should be updated. Put your new functionality into a function
with a docstring, and add the feature to the list in README.rst.
3. The pull request should work for Python 2.6, 2.7, 3.3, 3.4 and 3.5, and for PyPy. Check https://travisci.org/kenny-freeman/yardstick/pull_requests and make sure that the tests pass for all supported Python versions.
16
Chapter 5. Contributing
yardstick Documentation, Release 0.1.0
5.4 Tips
To run a subset of tests:
$ python -m unittest tests.test_yardstick
5.4. Tips
17
yardstick Documentation, Release 0.1.0
18
Chapter 5. Contributing
CHAPTER 6
Credits
6.1 Development Lead
• Kenny Freeman <[email protected]>
6.2 Contributors
None yet. Why not be the first?
19
yardstick Documentation, Release 0.1.0
20
Chapter 6. Credits
CHAPTER 7
History
7.1 0.1.0 (2016-01-11)
• First release on PyPI.
21
yardstick Documentation, Release 0.1.0
22
Chapter 7. History
CHAPTER 8
Indices and tables
• genindex
• modindex
• search
23