Sunday November 1, 2009Using Cucumber with SketchUpCucumber is a Ruby-based tool (technically, a domain-specific language) that helps programmers and their clients define and agree on tests of program behavior. These tests can be used to guide development, enforce acceptance criteria, and detect regressions.Although it is popular in the larger Ruby community, Cucumber has not been used (as far as I can tell) to develop Ruby-based extensions for Google SketchUp. This seems like an unfortunate situation; perhaps it's time to see what can be done about it...
BackgroundThe following information is largely intended for readers who are unfamiliar with Cucumber and/or SketchUp, but other readers should probably skim it, just in case...
Cucumber
Cucumber was created to support a software development technique called Behavior Driven Development (BDD). In this technique, development starts with a description of desired behavior (ie, a feature), then proceeds "inward" to create tests (ie, steps), make them pass, and then refactor the code into civility. A Cucumber feature describes some desired behavior, using language that the client should find comfortable. It begins with some context information, then specifies one or more scenarios composed of single-line steps. Here is a simple example:
Feature: Demonstration
In order to demonstrate Cucumber
As a blogger
I want to use some values from a Scenario
Scenario: Capture and use values
Given this example is about 'adding numbers'
And I specify values such as 1 and 2
Then the result should equal 3
And this step should be undefined
The typical client should have no trouble reading this description
and determining whether it expresses his or her understanding of the desired behavior.
Use of familiar concepts and language reduces the chance of ambiguity or confusion.
In short, the client can feel comfortable
about using Cucumber features as acceptance criteria.
However, the description is not really written in English. Rather, it uses a (minimalistic, but easily extensible) controlled natural language. So, the programmer can create a collection of step definitions that match specific steps and perform appropriate actions:
Given /^this example is about '(.*)'$/ do |topic|
puts " >> topic: '#{topic}'"
end
And /^I specify values such as (.*) and (.*)$/ do |v1, v2|
@v1, @v2 = v1, v2
end
Then /^the result should equal (.*)$/ do |result|
result == @v1 + @v2
end
In general, clients will never need to look at step definitions.
Ruby programmers, however, will recognize them as code blocks,
guarded by regular expressions (REs).
When a line in a scenario matches a step definition, Cucumber runs the corresponding step.
For flexibility, portions of the line (eg, 'adding numbers', 1) can be "captured"
and passed to the code inside the block.
When Cucumber is run, it generates output which reports on problems (eg, overlapping or missing steps, syntax errors) and summarizes the results:
Feature: Demonstration
In order to demonstrate Cucumber
As a blogger
I want to use some values from a Scenario
Scenario: Capture and use values # features/foo.feature:6
>> topic: 'adding numbers'
Given this example is about 'adding numbers' # features/step_definitions/foo.rb:1
And I specify values such as 1 and 2 # features/step_definitions/foo.rb:5
Then the result should equal 3 # features/step_definitions/foo.rb:9
And this step should be undefined # features/foo.feature:10
1 scenario (1 undefined)
4 steps (1 undefined, 3 passed)
0m0.003s
You can implement step definitions for undefined steps with these snippets:
Then /^this step should be undefined$/ do
pending
end
Note that Cucumber deals gracefully with undefined steps. It reports that they (and the scenarios that use them) are undefined, then generates sample code for the needed step definition(s). Sweet! When all of the tests pass and the code is in acceptable condition, the feature is ready for inspection by the client. If the program does not act as the client expects and/or desires, the feature and test descriptions can be examined to locate and resolve the discrepancy. Passing sets of descriptions are retained as a useful form of regression testing.
SketchUpSketchUp is a powerful and free (as in beer) tool for creating and rendering 3D models. These models can be exported to Google Earth, turned into animations, etc. Although SketchUp is most commonly used to model buildings, it can be used to model a wide variety of objects. However, it is limited to modeling surfaces, as opposed to 3D volumes. SketchUp Ruby, an embedded Ruby API, provides a convenient and flexible way to create and add extensions (aka macros, plugins, Rubies). These extensions can be used to import, examine, and modify data, place "observers" on objects, etc. So, the API provides a powerful mechanism for testing (eg, controlling, inspecting) extensions. If we could use this mechanism under Cucumber, the benefits of BDD would be available in a totally different environment! For more information on SketchUp (etc), see:
Analysis and DesignIt might be possible to make Cucumber run as a SketchUp extension, but this approach doesn't look either easy or robust. Cucumber and SketchUp are far too likely to get in each other's way. Also, if either program changed, we might have to rework our modifications. So, let's run them as separate processes, letting Cucumber initiate, interact with, and monitor SketchUp plugins. Cucumber can be used to test various kinds of programs, but it is most commonly used to test web servers. So, it supports the specification of many small steps (eg, HTTP requests), possibly accompanied by some setup and/or teardown code. To match this orientation to SketchUp, we need ways for Cucumber to:
High-level InteractionOur step definitions will need to interact with SketchUp at a high level: pushing buttons, selecting menu items, etc. Although SketchUp makes no provision for this, Apple's Open Scripting Architecture (OSA) and accessibility support give us a convenient "back door". OSA makes it possible to send arbitrary Apple Events to applications. Apple's Accessibility Inspector allows any sufficiently motivated and qualified coder to examine applications and develop event description code (eg, using AppleScript or RubyOSA). However, none of this is easy enough for lazy folks like me. Fortunately, PreFab Software's UI Browser offers a very convenient way to generate AppleScript code. Basically, the programmer points the UI Browser at an application, navigates to the widget of interest, and specifies the desired action. The resulting code can then be edited (eg, generalized), stored in a file, and used via the osascript command.
Low-level InteractionAlthough it is quite possible for a plugin to operate autonomously, it would be convenient for Cucumber to be able to direct (and monitor) the plugin at a lower level. For example, Cucumber could start up a plugin, then feed it a series of actions and report on the results. Distributed Ruby (DRb) should provide a convenient and powerful way for Ruby code in Cucumber to interact with objects in SketchUp (and vice versa). However, I haven't (yet) been able to require DRb within SketchUp. Anyone who knows how to do this (on Mac OS X) is implored to get in touch! In the meanwhile, I plan to use named pipes, temporary files, etc.
General ApproachBringing these details together, we arrive at the following general approach:
Thoughts on TestingSketchUp's API provides a wealth of methods that allow plugins to examine and modify the current model. Based on these, we should be able to build up some "helper" methods akin to the ones used to test Ruby on Rails applications. However, that's mostly a pipe dream at this point. Meanwhile, we need to develop testing approaches that work well with both Cucumber and SketchUp.
Case Study: StairsThe stairs.rb script, described in the Google SketchUp Ruby API Developer's Guide, is supposed to generate a simple staircase (ie, 12 treads and 12 risers). How can we verify that it is behaving correctly? For simplicity, we'll start with an empty model, then run the plugin. We can then examine the model to see what entities the plugin produced. Here are some assertions to consider testing:
These assertions are at a level that would work for a client, but they also look reasonably easy to test (famous last words :-). Once I have a bit more plumbing in place, I'll give them a try and find out where the monsters lurk...
Reducing ConfusionIt won't always be possible to start with an empty model. For example, the plugin being tested may need entities to operate on. So, there is lots of opportunity for the test code to get confused. Which entities (eg, edges, faces) "belong together"? Which entities did the plugin create, modify, etc? SketchUp can't answer these sorts of questions, but it provides a facility which can be used to do so. Any object can be given any number of attributes. If the plugin under test leaves a suitable trail of attributes, the test code should be able to follow it. Again, this is mostly speculation at this point, but I expect to use SketchUp attributes pretty heavily to reduce confusion in testing.
Project WikiFor brevity, I have glossed over quite a few implementation details. These may be found, in their current (evolving) state, in my SketchUp project wiki:
Using Cucumber with SketchUp
- posted at Sun, 01 Nov, 21:27 Pacific
| «e»
| TrackBack
|