May 30, 2014

End of Week 2 update for GSoC 2014

This blogpost is an end of week 2 update for Google Summer of Code 2014 IDLE Improvement's project. In this time, I have worked on building a non-buildbot human test framework for GUI components of IDLE.

0. Expectation

To have put down a standard for non-buildbot tests. Extended the framework for the same. To have substantially increased the number of modules under non-buildbot test coverage.

1. Accomplishments

A human test(or htest) is a way to test and check for any regression in those GUI components, which are either hard/impossible to test by typical unittests. The tester is presented with instructions(in a parent dialog) on what to test and what to expect. The tests are also modular in the sense that a single component will be isolated. This feature will be useful for current IDLE developers. Future developers may also use it to gain a finer understanding of modules.

A issue for this sub-project was created at the issue tracker. We managed to add human tests for nearly 30 modules in idlelib. To put things into perspective, it is upwards of 90% coverage*. Two utility functions run() and runall() were refactored into a single function run() - the benefit being that all tests re-use the same root dialog. The run() of  htest file can either be called directly or from modules. It has the capability to run individual tests or all tests. This was one of the key improvements.

Moving to UI/UX, the parent dialog(the dialog in which the tester is presented with instructions), now "stays in the same place" and does not "jump" around. This was done by making a tkinter Text widget behave like a Label widget with wrapping and a scrollbar.

All these changes have also been committed to the Python's mercurial repository.

2. Roadblocks

AutoCompleteWindow and Debugger have proved to modules, difficult to test individually. While they work when IDLE is run, there is difficulty in making these components function in an isolated environment. Though  stubbing of methods does make them work, the number of methods that have to be stubbed is so high that it is comparable to an actual IDLE instance. Deeper understanding of the IDLE architecture will be required to make any progress. Needless to say, I will continue trying to make it work even though it could take some time.

Currently, the user has to manually close a test before clicking [Next].We need to find a way to ensure when running multiple tests, when a user has finished a particular tests, and clicks [Next] to move onto a new test - all residuary widgets from previous tests have to be destroyed. While existing suggestions work for majority of tests, modules like ClassBrowser, PathBrowser are exceptions. Once a common thread is found, fixing this issue is going to be straightforward.

3. Critique

In this section, I will try to objectively critique progress from a 3rd party point of view, keeping in mind the proposal timeline.
I'd say, overall, the past two weeks have been a success, barring a few hiccups(read 2. Roadblocks). The 
if __name__ == '__main__': part of most modules have been cleaned up. The tests are more or less presentable, though the instructions presented to the tester could need more polishing.

4. Overview for the next two weeks.

In the next two weeks, I will try to increase the test coverage for IDLE. I will be giving priority to those which have open issues/bugs on the tracker. By doing so, those bugs will be fixed and we can reduce any such future regressions.

No comments:

Post a Comment