could we multithreading build tasks? (eg qunit/jsunit/junit/msunit)

Benjamin Anderson

just a random thought... if the build servers have quad-cores or the like and we multi-threaded some of the build processes (eg unit tests), wouldn't that be nice? anyone ever multithread unit tests before? theoretically it should be doable.

James Greene

Yes, I did this with all of the Cobalt *StaticContent builds for a while with the JsUnit and QUnit tests. It worked great locally on a beefy developer box but tended to timeout more frequently on the good-but-not-great CI build boxes. I've referenced some pertinent commits below but the build scripts have changed a lot since then, so its utility will be limited. The big difference was, of course, running multiple "PhantomJS.exe" processes simultaneously by utilizing Ant's `parallel` task. _Interesting Changesets:_ - [#222996](
http://tfsnpt.int.thomson.com:8080/tfs/web/UI/Pages/Scc/
ViewChangeset.aspx?changeset=222996&pguid=60798db5-fb71-4793-a6b6-79e322749a57): Initial introduction of parallelism - [#223672](
http://tfsnpt.int.thomson.com:8080/tfs/web/UI/Pages/Scc/
ViewChangeset.aspx?changeset=223672&pguid=60798db5-fb71-4793-a6b6-79e322749a57): Throttled back the max threads allowed for parallelism - [#224311](
http://tfsnpt.int.thomson.com:8080/tfs/web/UI/Pages/Scc/
ViewChangeset.aspx?changeset=224311&pguid=60798db5-fb71-4793-a6b6-79e322749a57): Removed parallelism In all changesets, seek out the "UnitTestAntScript.xml" and "ant-global.properties" files and compare with their previous versions. ~~~ The Document module team also tried to multithread their MSTest unit tests before but ran into race conditions in their mocking library (RhinoMocks) and so had to revert.

Benjamin Anderson

interesting... sure, if you can find the changesets quickly, I'd be interested in looking. Wonder why phantonjs would time out?

James Greene

In retrospect after much additional learning, we don't believe that PhantomJS timed out at all. Rather, it seems that the old version of Jetty (the dynamic HTTP server that we host the Static Content files with) we were using (a custom-wrapped version of Jetty 6) couldn't keep up with the load/demand without lagging, which led to the max timeouts that we specified being exceeded. We have since upgraded to Jetty 9 but have not yet attempted to recreate the multithreaded test run as we are working to correct some other related issues.

Benjamin Anderson

Thanks for that background, James. Do you think switching from good to great servers might help as well? Just wondering since so many people depend on these builds... if we all spoke up from the various communities and demonstrated the potential increases to velocity... if the servers might be candidates for upgrade? Just wondering if there's anything we could do to help out. Thanks for your efforts.

James Greene

Edited my original answer to add changesets for *StaticContent parallelism and recount what I heard from the Doc team.

James Greene

One would hope that a concerted urge from the developers to get better build boxes would be well received and acted upon... not positive of the reality. Personally, I expect my build boxes to have equal or _**BETTER**_ specs than my developer box but that's just me.

Benjamin Anderson

I'll second that and pass it along (fwiw)

Unknown

In Research Organizer (Foldering) we run our builds in parallel. More specifically, the unit tests, integration tests, and the endpoint tests are run in a parallel manner (basically, three threads of execution). Doing so reduces our build time from ~20 minutes to closer to 12 minutes (as measured on my machine). Like in James's case, we use Ant's parallel task to accomplish this.