Open tasks to improve Sakai testing
| WORK IN PROGRESS Following on from the Paris BOF, here's a starting list of tasks which any developers in the community can take on. Please update this page as you see fit. |
Add more sample tests
Revise existing tests to reflect "best practices"
Provide easier starts
It's a lot faster to fill in a form than to write an essay from scratch when faced with a blank sheet of paper. Similarly, time-crunched developers will be much more able to write tests if the basic setup is there already. For example, Aaron Zeckoski's App Builder encourages test development by generating placeholders for unit and project-bounded tests.
Possibilities include creating Maven archetypes and Eclipse templates for the various test styles, providing commented-out best-practices sample projects, adding EasyMock-based stubs to the App Builder templates, and adding optional generation of integration test areas to the App Builder.
Agree on Maven trigger to enable longer-running tests
Full front-to-back functional integration tests, Sakai service-level integration tests, and load tests are all too heavyweight to be run automatically on every build. Currently, there's no agreed-upon method for enabling all such tests. They have to be run manually for each project, and the method for running them varies. By agreeing on a Maven property or profile to be shared across projects, we can make sure that all tests pass as part of continuous integration and the release process.
As a first attempt, I'm using a "skipLongTests" property to enable or disable a few integration tests in Sakai 2.6.
Centralize support for user and request emulation
Much service-integrated tests and utilities need Sakai components to think they're being run by a particular user, and many also need an easy way to pretend to be in an HTTP request associated with a particular site and tool. Currently, we have many not-quite-copy-and-pasted variants of this functionality. It should be centralized in the test utilities package.
Write data loader for performance tests using ComponentContainerEmulator and compare its performance to the web services method
Script sample test-harness-powered integration tests using Groovy or Jython
Can Sakai service integration be used for all Spring tests without copy-and-paste?
To work with the emulated component manager, SakaiDependencyInjectionTests overrides two methods in AbstractDependencyInjectionSpringContextTests. Other Spring test-support classes could be used the same way by overriding the same two methods. Could AOP be used to overlay that logic rather than having to subclass each Spring class?
Continued improvements to test-harness
Jon Gorrono is coordinating this work at SAK-12334. Requirements were gathered at a Paris BOF.
Comments (2)
Jul 28
Linda M Place says:
Thanks for pulling this material together, Ray! I totally agree with the need t...Thanks for pulling this material together, Ray!
I totally agree with the need to centralize support for user and request emulation. Performance tests totally depend on the ability to emulate the same combination of user/site/tool over a large easily generated and predictable population. Chris has attempted to create such a population as part of our Community Performance Testing effort, so that might be a good place to start. We have already invested in designing an environment to test Sakai trunk code, so it would help us if that population could be agreed upon. Naturally, if another design would work better in the long run, we're willing to change.
We'd love to hear more about ComponentContainerEmulator as a possible data loader. Where can we find more info about it? Data loading is the most time consuming part of setting up a viable performance test environment.
Aug 12
Ray Davis says:
Sorry to be late responding, Linda I was off the job for a couple of weeks. Rig...Sorry to be late responding, Linda – I was off the job for a couple of weeks.
Right before leaving, I checked documentation on ComponentContainerEmulator into the Sakai trunk. It's been working well for me so far, and I'd be glad to help make a trial implementation of Chris's data load available for comparative testing. (Performance testing the data load routine that enables performance testing – ya gotta love it.) Ideal for me would be if you or Chris create a JIRA task for the collaboration and either assign to me or add me as a watcher.