CPAN6
Mark Overmeer & Sam Vilain
YAPC::EU 2006, Aug 31 Birmingham UK
CPAN is..
“CPAN is the best thing
that ever happened
to Perl”
CPAN is..
“CPAN is the best thing
that ever happened
to Perl”
CPAN is..
“CPAN is the best thing
that ever happened
to Perl5”
CPAN is...

Just over 10 years young
younger than



Perl5
WWW
UNIX
CPAN is...

Just over 10 years young
younger than




Perl5
WWW
UNIX
Just over 10 years old
developed before break-through of




XML/SOAP/WSDL
CVS/SVN/SVK/GIT
spam, viruses
unicode
...concerns...

10k modules, a lot is bad quality or
poorly maintained

3500 authors, who is who?

who is making the rules of CPAN?
...concerns...

10k modules, a lot is bad quality or
poorly maintained

3500 authors, who is who?

who is making the rules of CPAN? GBD#2
What will happen when infected code is
detected on CPAN? Will Perl lose its
best thing?
...future...

Where will we store



Perl6 modules
pir, pbc, pil, pod,… modules
name-space battles!
parTcl? Brainf*ck?
...future...

Where will we store




Perl6 modules
pir, pbc, pil, pod,… modules
name-space battles!
parTcl? Brainf*ck?
Can we improve





security
= trust
professionalism = regulation/deregulation
deployment
= less content independent
platform integration = YaST, apt, ...
capacity
= name-space
management
Releases ≠ Revisions

Revisions





a logical sequence of development steps of a file
each step may solve some bugs
each step extends the code and documentation
steps are created by a single person
Releases



are created to be distributed
a defined `frozen' state of the software
(some combination of file versions)
published by someone who is responsible
CPAN ≠ YA VCS

CPAN is a collection of releases




distributing information
name-spaces and trust
searching in the collection
Version Control Systems manage (related)
(development) revisions of files



collecting pieces of information
traceable changes
tracing and solving bugs
Needs

improved security: alternatives to Pause-ID
Needs


improved security: alternatives to Pause-ID
professional approach: described rules,
especially how to resolve disputes/problems.
Needs



improved security: alternatives to Pause-ID
professional approach: described rules,
especially how to resolve disputes/problems.
extended deployment: support for other
programming languages (and other data)
Needs




improved security: alternatives to Pause-ID
professional approach: described rules,
especially how to resolve disputes/problems.
extended deployment: support for other
programming languages (and other data)
platform integration: environment friendly
Needs



improved security: alternatives to Pause-ID
professional approach: described rules,
especially how to resolve disputes/problems.
extended deployment: support for other
programming languages (and other data)

platform integration: environment friendly

capacity growth: more name-spaces
NO QUICK FIX!
but a good alternative
Split CPAN

CPAN6:

idea of having a collection of releases,
combined into archives


uploading releases
downloading releases



distributing the archives
combining archives
filtering archives
like mirrors
like ftp sites
sub-sets
CPAN6 is the DISTRIBUTION
Split CPAN


CPAN6 is the distribution
Pause6:

one archive
is one name-space
is set of releases (with some relation)




a group of releases relate to a project(name)
projects are allocated to authors
authors have an defined identity
an archive has well-defined rules
Pause6 is ADMINISTRATION
Split CPAN



CPAN6 is the distribution (network)
Pause6 is the administration (archives)
CPAN6.pm




installs Perl5, Perl6 etc distributions
blends into the platform
collects meta-data to publish releases
programming language specific component
CPAN6.pm for INSTALLATION
Split CPAN



CPAN6
Pause6
CPAN6.pm
is the distribution
is the administration
installs releases locally
Of course
perl -MCPAN6 'install Test::More'
will work!
CPAN
CPAN6 concept
CPAN6 structure



One system can host multiple archives, as
commissioner and/or deployer; combined in
one service (daemon)
Multiple repositories are combined into a
store
Stores (data) are not protected, and can
therefore be kept on other systems than the
related daemons.
Pause6 Implementation
Pause6 implementation

Allocates the name-space



project names are taken by a few authors
entrusting board members
limits to project names and version labels
Pause6 implementation

Allocates the name-space




project names are taken by a few authors
entrusting board members
limits to project names and version labels
User identities



PKI based signing
publisher, authors, board, processes proof right
to make changes with signatures.
keeps track on trust
Pause6 projects

Release has project-name, version and state
Each state change requires signatures

The release states are









uploading,
initiated by publisher
published,
distributable to deployers
embargo,
download blocked
released,
by the authors
deprecated, by the authors
expired,
by the authors
rejected,
by the board/authors
installed,
by the end-user
Pause6 projects

Projects are sets of releases

Each release is a set of files

The archive collects per release





checksums of each file (SHA-256 or better)
location of each file in the store
state of the release
user provided meta-data, like description for
searching
transport traces etc
Pause6 projects


Any set of files can be a release of a project,
not only Perl5 tar-ball distributions.
Other “project” types are:
Pause6 projects


Any set of files can be a release of a project,
not only Perl5 tar-ball distributions.
Other “project” types are:





constitution
user and daemon identities
archive references
license description
...
Pause6 projects


Any set of files can be a release of a project,
not only Perl5 tar-ball distributions.
Other “project” types are:





constitution
user and daemon identities
archive references
license description, etc
therefore


everything follows same release protocol
one transport implementation covers all aspects
Release
trails
Release
trails
CPAN6 archives
Hierarchy

Scribes can be configured to create archive
hierarchies, for example:






global cpan-perl5
business level sub-set
department sub-set
system local
personal installed
personal development
CPAN6 service daemon

Manage a set of archives





as commissioner
as deployer
trigger scribe processes
local or remote stores
Configuration


an archive with archive-references
archive board are the system administrators
CPAN6 Scribes


Copy published releases from commissioner
to deployers
Copy releases between archives, allowing





selection rules,
like “only last”, “license
GPL”
auditing,
content checks
trust calculation,
signature checks
transport protocol conversions
Used to build virtual hierarchies of archives
Example: project
perl -MCPAN6 'install DBD::Oracle'
is something like
LOC=$(pause6 project get http://cpan.org?project=DBD::Oracle)
cd $LOC
tar xzf *.tar.gz
perl Makefile.PL
make test install
Example: project
pause6 archive create ~/perl/devel as myperl
pause6 project create perl5/DBD::Oracle
mkdir ~/perl/dbd-oracle; cd $_
vi <anything>
make dist
pause6 release create DBD::Oracle 0.01 as ora
pause6 release add ora DBD-Oracle-0.01.tar.gz
pause6 release edit ora
pause6 release publish ora to perl5
# or myperl
DIR=$(pause6 release get perl5/DBD::Oracle)
Syntax subject to change
pause6 init
pause6 archive use http://cpan6.net/perl5
pause6 archive use http://pause-id.cpan.org as pauseid
pause6 identity default pauseid/MARKOV
Example: service CPANTS
pause6 archive create http://archives.cpan6.net/cpants
# Triggered daily by cron
pause6 archive query perl5 -c name,version release=last \
| while read NAME VERSION
do
LOC=$(pause6 release get “pause6/$NAME;$VERSION”)
Q=$(calculate_kwalitee $LOC)
pause6 project create cpants/$NAME 2>/dev/null
pause6 release create $NAME $VERSION as new
pause6 release add new $Q
pause6 release publish new to cpants
done
Status

CPAN6/Pause6 global design document

CPAN6 design and implementation doc

Pause6 design and implementation doc

platform integrated install tools
Needed...

... discussion and attention

extensions to the design, use-cases

contacts with other potential user groups

signing, licensing, trust algorithms

funding to create base implementation
Plans...



implement local archives (daemon-less)
implement network archives (daemon)
implement scribes (cron tasks)
Plans...





implement local archives (daemon-less)
implement network archives (daemon)
implement scribes (cron tasks)
create CPAN mirror in CPAN6/Pause6
create CPAN6.pm, based on CPAN.pm
Plans...








implement local archives (daemon-less)
implement network archives (daemon)
implement scribes (cron tasks)
create CPAN mirror in CPAN6/Pause6
create CPAN6.pm, based on CPAN.pm
start pir, pasm, partcl,... archives
start javascript, python, php, ... archives
create ftp-server wrappers (get public)