libzfs_core and ioctl
stabilization
Richard Yao (ClusterHQ)
About Me
•
•
•
Gentoo Developer
•
genkernel initramfs maintainer
•
Founded eudev project as original team lead.
•
Many miscellaneous improvements, most of which were upstreamed, including a patch in FreeBSD's
kernel (certain SCSI controllers are attached by the right driver now)
ZFSOnLinux Developer
•
Maintains Packaging for Gentoo, various Gentoo derivatives and SoftNAS.
•
Second by commit count. Many miscellaneous stability, compatibility and performance fixes, plus a
few minor features. Many more in various stages of development
Nascent Linux Kernel Contributor
•
Using 9p-virtio as a VM's rootfs works now
•
Another patch pending upstream review that makes atomic readonly bindmount creation actually
work.
Lets talk about ZFS system
integration and management
•
ZFS has great interactive management via the zfs and
zpool commands. It can even be scripted.
•
On Linux, they are used in more places than you think.
•
Backups and volume management
•
GRUB2 bootcode installation (some Linux
distributions link grub2-probe instead to libzfs)
•
System boot via the initramfs scripts
This way of doing things is
problematic
•
•
Developing management software around
interactive command tools requires string parsing,
which is easy to get wrong and hard to get right.
•
The strings printed are subject to change as
improvements are made.
•
Proper error handling is not always clear.
Those that do it right have problems whenever the
ioctl interface changes in an incompatible way.
Related issue of ioctl
interface changes
•
The introduction of libzfs_core in ZoL 0.6.3 severely broke the ioctl
interface.
•
Many initramfs archives that loaded ZFS had not been updated.
•
Plenty of developer time was lost assisting users
•
Many users thought that they had lost their pools.
•
Volunteers doing community support were dealing with it for months.
•
A webpage dedicated to fixing it reached first place on Google searches
for ZoL
•
The install process for / on ZFS was broken until Live media was
updated.
Aftermath of ZoL 0.6.3
/dev/zfs ioctl break
•
Many users have forgiven us with the understanding
that this is a one time incident, but they were *VERY*
upset.
•
Some long time system administrators think ZoL is unfit
for production due to this incident, but they appear to
be a small minority.
•
Allowing this to happen again could irreparably damage
ZFS' reputation on Linux and further delay deployment
of reliable storage. This must *NOT* happen.
libzfs_core (2013)
•
Provides a stable committed interface.
•
Developed to be a thin shim over the ioctl interface that provides a
subset of functions from the zfs command provides.
•
New ioctl design that uses nvlists introduced for this subset that
is encapsulated in the old zfs_cmd_t.
•
Omitted many key features such as replication streams, property
management, snapshot creation, renaming, listing, destroying.
•
Supports dataset creation with properties which are enumerated
types, but does not provide any way of getting what the index
properties are.
Why bother with libzfs_core enhancements
if we will have a stable ioctl interface?
•
Writing the code to interact with /dev/zfs in C writing
wrappers has benefits.
•
It reduces effort in implementing support in
consumers
•
It leaves less room for bugs
•
It makes those that do exist easier to catch.
•
It makes it easier for extensions to be supported.
Designing stable interfaces
•
Once you design a stable interface that is stable, it is around
forever.
•
•
You need to do your best to avoid mistakes.
How do you avoid mistakes?
•
Stop thinking you are not going to make mistakes and focus on
minimizing mistakes.
•
Look at history for things not to do (e.g. dup2 versus dup3)
•
Make trade-offs such as simplicity (security) versus generality
(performance).
The overall plan of action
•
Modify /dev/zfs to use string values for index properties (done, but not yet
upstreamed)
•
Extend libzfs_core with desired functions (WIP)
•
Rework ioctl interface (all changes are tentative until working code is reviewed)
•
Get cross platform review of working code
•
Implement bindings
•
•
Python initially. Bindings for other languages are welcome.
Use bindings in consumers
•
Flocker initially. Other consumers coming on-board not only welcome, but
encouraged.
ioctl stabilization plan
•
Initially only a subset of functionality will be stabilized, but it will be a large subset.
•
•
Things like zpool iostat will not be included.
The libzfs_core ioctl design will be extended
•
String command identifiers will be added to allow us to have 1 ioctl for the zfs
command and 1 ioctl for the zpool command.
•
We will add an opts nvlist to allow for future additions of flags.
•
We add sequence numbers that we hope we will never need to increment but will
allow us to make backward incompatible changes should it be necessary
•
Strings will be used for passing index property values moving forward.
•
Logic for handling replication streams will move into the kernel.
Logistics
•
Userland will migrate to the new ioctls whenever possible
while the kernel will fully support both for the time being.
•
This prevents another 0.6.3 style backward compatibility
issue.
•
Groundwork for using Linux ZFS tools in lx branded zones
on Illumos and Linux jails on FreeBSD will be in place.
•
Initial code is not ready for review yet, but should be
available within weeks.
Questions?