Moving Talos from BuildBot to Mozharness

by jasonyeo

tl;dr Using BuildBot to run Talos is giving us too many headaches. We are moving it to Mozharness, a configuration driven script harness with full logging facilities that allows production infrastructure and individual developers to use the same scripts.

In case you don’t know, I’m currently interning at Mozilla as a release engineer in the RelEng team. Part of my job here is to port Talos from BuildBot to Mozharness. Buildbot is a continuous integration tool, kinda like Travis-CI and Jekins-CI. It is used at mozilla to automate our builds and tests. Talos, is a performance testing framework that loads a number of test pages and collect data from the system. It is usually scheduled by BuildBot to be ran after the builds have been completed.

Talos Py

Being called and ran by BuildBot has a number of downsides. The biggest problem is that other developers have to understand and setup buildbot on their own dev machines to test the production infrastructure environment that talos runs in.

But with mozharness developers just have to clone the mozharness code and execute it to test talos. Assuming that you wish to run the tp5o talos test suite and your dev machine is a 64 bit OS X machine, you can run these commands to start talos.

hg clone
cd mozharness
python scripts/ --suite tp5o --add-option --webServer,localhost --branch-name Ash --system-bits 64 --cfg talos/ --download-symbols ondemand --use-talos-json

The best part is that developers just have to copy and paste the commands that are logged in TBPL to test particular commands that may have failed in the production testing machines.

If you want to contribute to mozharness, the code is at our mercurial repository. If you want to find out more about mozharness, drop by our #releng IRC channel on the mozilla irc server. (