The ZTK builds on the automated test suites (unit and functional tests) from the individual projects it keeps track of. We use automated build systems, like buildbot, to run various combinations of differing Python versions, operating systems and packages and ensure everything works as expected.
The ZTK’s automated test suite builds on the individual packages’ unit and functional tests and creates a combined test runner that runs each packages’ test suite in isolation but ensures that the dependencies are satisfied using the ZTK versions under test.
The combined test runner is created using z3c.recipe.compattest – check the documentation for details.
If you take a ZTK checkout, you can run the tests yourself like this:
$ svn co svn://svn.zope.org/repos/main/zopetoolkit/trunk
$ python bootstrap.py
$ bin/buildout
$ bin/test-ztk && bin/test-zopeapp
If you work on a ZTK package and want to ensure that your changes are compatible with a specific version of the ZTK (but using the version of the package you’re working on instead of the version listed in the ZTK) then you can create a combined test runner in your buildout like this:
[buildout]
parts = compattest
extends = path-to-specific-ztk-version.cfg
develop = .
[versions]
<package-you-work-on> =
[compattest]
recipe = z3c.recipe.compattest
include = ${ztk:packages}
This is a list of servers that run regular builds of various parts of the code base.
The software currently being tested is not compatible with python 2.4. The tests run on OS X 10.6
| Project | OS X / Python2.4 | OS X / Python2.5 | OS X / Python2.6 | OS X / Python2.7 |
| Zope 2.12 | unsupported | unsupported | THA | unsupported |
| Zope 2.13 | unsupported | unsupported | THA | THA |
| Zope 2 trunk | unsupported | unsupported | THA | THA |
| ZTK 1.0 | THA | THA | THA | unsupported |
| ZTK 1.1 | unsupported | THA | THA | unsupported |
| ZTK trunk | unsupported | THA | THA | unsupported |
| Groktoolkit 1.2 | unsupported | THA | THA | unsupported |
| Groktoolkit trunk | unsupported | THA | THA | THA |
The winbot is configured to compile C-Extensions, while the ztk environment isn’t. A rumour is that pythons before 2.6 are unstable on Win64, therefore no testing support for those
| Project | Win32 / Py2.4 | Win32 / Py2.5 | Win32 / Py2.6 | Win32 / Py2.7 | Win64 / Py2.6 | Win64 / Py2.7 |
| Zope 2.13 | unsupported | unsupported | THA | THA | unsupported | unsupported |
| ZODB trunk | unsupported | winbot | winbot | winbot | winbot | winbot |
| ZTK 1.0 | unsupported | winbot | unsupported | |||
| ZTK 1.1 | unsupported | THA | THA | unsupported | unsupported | unsupported |
| ZTK trunk | unsupported | winbot | winbot | winbot | ||
| zc.buildout trunk | unsupported | winbot | winbot | winbot | winbot | winbot |
| Bluebream | winbot | winbot | winbot | winbot | winbot | winbot |
| Groktoolkit 1.2 | unsupported | THA | THA | unsupported | unsupported | unsupported |
| Groktoolkit trunk | unsupported | THA | THA | THA | unsupported | unsupported |
| Project | Linux32 / Py2.4 | Linux32 / Py2.5 | Linux32 / Py2.6 | Linux32 / Py2.7 | Linux64 / Py2.4 | Linux64 / Py2.5 | Linux64 / Py2.6 | Linux64 / Py2.7 |
| Zope 2.10 | unsupported | unsupported | unsupported | EPY | unsupported | unsupported | unsupported | |
| Zope 2.11 | unsupported | unsupported | unsupported | EPY | unsupported | unsupported | unsupported | |
| Zope 2.12 | unsupported | unsupported | THA | unsupported | unsupported | unsupported |
|
unsupported |
| Zope 2.13 | unsupported | unsupported | THA | THA | unsupported | unsupported | THA | THA |
| Zope 2.14 (trunk) | unsupported | unsupported | THA | THA | unsupported | unsupported | THA | THA |
| Zope 3.4 KGS | POV | POV | unsupported | unsupported | unsupported | unsupported | ||
| ZTK 1.0 | THA | THA | THA | unsupported | unsupported | |||
| ZTK 1.0 dev | unsupported | AFPY | AFPY | AFPY | unsupported | |||
| ZTK 1.1 | unsupported | THA | THA | unsupported | unsupported | THA | THA | unsupported |
| ZTK trunk | unsupported | THA | THA | unsupported | unsupported | THA | THA | unsupported |
| Bluebream | AFPY | AFPY | AFPY | AFPY | ||||
| Groktoolkit 1.2 | unsupported | THA | THA | unsupported | unsupported | THA | THA | unsupported |
| Groktoolkit trunk | unsupported | THA | THA | THA | unsupported | THA | THA | THA |
We would like to thank all providers of automated test facilities:
| Buildbot | Contact | Platforms | Python | Packages / Branches |
| The Health Agency (THA) | Jan-Jaap Driessen | OS X, Ubuntu 32/64, Windows | 2.4, 2.5, 2.6, 2.7 | ZTK, Zope 2, groktoolkit, many grok and ZTK packages |
| POV |
|
Linux 32/64 | 2.4, 2.5, 2.6, 2.7, 3.1 | KGS for Zope 3.4 |
| AFPY | Christophe Combelles | Linux 64 | 2.4, 2.5, 2.6, 2.7, 3.1 | ZTK 1.0 (trunks and releases), BlueBream template |
| Sebastien Douche | Linux 32/64 | 2.4, 2.5, 2.6 | KGS (3.4/3.5), ZTK, grok, BFG (trunk), zc.buildout | |
| EPY | Stefan Holek | Linux 64 | 2.6 | Zope 2, CMF, Plone |
| winbot (project sponsored by the Zope foundation) | Adam Groszer | Windows 32 and 64 bits | 2.4, 2.5, 2.6, 2.7 | ZTK (trunks and releases), BlueBream template, ZODB, |
To ensure a reasonable amount of communication from automated systems that reaches the Zope developers and keeps them aware of the overall build status we prefer not to send individual build information directly to the zope-dev@zope.org mailing list.
A separate list (zope-tests@zope.org) exists which you can send build notifications to. This list is usually not read by humans. However, a script aggregates the messages once per day and reports the overall build status back to zope-dev.
To ensure that your build output is correctly picked up, you need to
The subject line should be formatted like this:
<STATUS>: <SUBJECT OF TEST>
OK: Zope 2.12 on Linux 64-bit with Python 2.4
FAILED: Zope 2.6 on Windows XP 32-bit with Python 2.5
UNKNOWN: zope.interface on Linux 64-bit
Here is a sample message_formatter function for the buildbot MailNotifier
def message_formatter(mode, name, build, results, master_status):
"""Provide a customized message to BuildBot's MailNotifier."""
result = Results[results]
text = list()
# status required by zope-tests list
# http://docs.zope.org/zopetoolkit/process/buildbots.html
status = 'UNKNOWN'
if result == 'success':
status = 'OK'
if result == 'failure':
status = 'FAILED'
subject = '%s : %s / %s' % (status, master_status.getProjectName(), name)
text.append(subject)
text.append("Build: %s" % master_status.getURLForThing(build))
text.append('\n')
text.append("Build Reason: %s" % build.getReason())
text.append('\n')
source = ""
ss = build.getSourceStamp()
if ss.branch:
source += "[branch %s] " % ss.branch
if ss.revision:
source += ss.revision
else:
source += "HEAD"
if ss.patch:
source += " (plus patch)"
text.append("Build Source Stamp: %s" % source)
text.append('\n')
text.append("Blamelist: %s" % ", ".join(build.getResponsibleUsers()))
text.append('\n')
text.append("Buildbot: %s" % master_status.getBuildbotURL())
return {
'body': "\n".join(text),
'type': 'plain',
'subject': subject,
}
Some links to sample configs:
See also winbot details
Patrick Gerken (do3cc) is the voluntary coordinator for automated builds and nightly tests.
The responsibility of the coordinator is to help the community reach our goals regarding:
The coordinator’s tasks include: