Cucumber
is a Ruby-based tool
(technically, a domain-specific language)
that helps programmers and their clients define and agree on tests of program behavior.
These tests can be used to guide development, enforce acceptance criteria, and detect regressions.
Although it is popular in the larger Ruby community,
Cucumber has not been used (as far as I can tell) to develop Ruby-based extensions
for Google SketchUp.
This seems like an unfortunate situation;
perhaps it's time to see what can be done about it...]]>
Background
The following information is largely intended
for readers who are unfamiliar with Cucumber and/or SketchUp,
but other readers should probably skim it, just in case...
Cucumber
Cucumber was created to support a software development technique called
Behavior Driven Development (BDD).
In this technique, development starts with a description of desired behavior (ie, a feature),
then proceeds "inward" to create tests (ie, steps), make them pass,
and then refactor the code into civility.
A Cucumber feature describes some desired behavior,
using language that the client should find comfortable.
It begins with some context information,
then specifies one or more scenarios composed of single-line steps.
Here is a simple example:
Feature: Demonstration
In order to demonstrate Cucumber
As a blogger
I want to use some values from a Scenario
Scenario: Capture and use values
Given this example is about 'adding numbers'
And I specify values such as 1 and 2
Then the result should equal 3
And this step should be undefined
The typical client should have no trouble reading this description
and determining whether it expresses his or her understanding of the desired behavior.
Use of familiar concepts and language reduces the chance of ambiguity or confusion.
In short, the client can feel comfortable
about using Cucumber features as acceptance criteria.
However, the description is not really written in English.
Rather, it uses a (minimalistic, but easily extensible)
controlled natural language.
So, the programmer can create a collection of step definitions
that match specific steps and perform appropriate actions:
Given /^this example is about '(.*)'$/ do |topic|
puts " >> topic: '#{topic}'"
end
And /^I specify values such as (.*) and (.*)$/ do |v1, v2|
@v1, @v2 = v1, v2
end
Then /^the result should equal (.*)$/ do |result|
result == @v1 + @v2
end
In general, clients will never need to look at step definitions.
Ruby programmers, however, will recognize them as code blocks,
guarded by regular expressions (REs).
When a line in a scenario matches a step definition, Cucumber runs the corresponding step.
For flexibility, portions of the line (eg, 'adding numbers', 1) can be "captured"
and passed to the code inside the block.
When Cucumber is run, it generates output which reports on problems
(eg, overlapping or missing steps, syntax errors) and summarizes the results:
Feature: Demonstration
In order to demonstrate Cucumber
As a blogger
I want to use some values from a Scenario
Scenario: Capture and use values # features/foo.feature:6
>> topic: 'adding numbers'
Given this example is about 'adding numbers' # features/step_definitions/foo.rb:1
And I specify values such as 1 and 2 # features/step_definitions/foo.rb:5
Then the result should equal 3 # features/step_definitions/foo.rb:9
And this step should be undefined # features/foo.feature:10
1 scenario (1 undefined)
4 steps (1 undefined, 3 passed)
0m0.003s
You can implement step definitions for undefined steps with these snippets:
Then /^this step should be undefined$/ do
pending
end
Note that Cucumber deals gracefully with undefined steps.
It reports that they (and the scenarios that use them) are undefined,
then generates sample code for the needed step definition(s).
Sweet!
When all of the tests pass and the code is in acceptable condition,
the feature is ready for inspection by the client.
If the program does not act as the client expects and/or desires,
the feature and test descriptions can be examined to locate and resolve the discrepancy.
Passing sets of descriptions are retained as a useful form
of regression testing.
SketchUp
SketchUp is a powerful and free
(as in beer) tool
for creating and rendering 3D models.
These models can be exported
to Google Earth,
turned into animations, etc.
Although SketchUp is most commonly used to model buildings,
it can be used to model a wide variety of objects.
However, it is limited to modeling surfaces, as opposed to 3D volumes.
SketchUp Ruby,
an embedded Ruby API,
provides a convenient and flexible way to create and add extensions (aka macros, plugins, Rubies).
These extensions can be used to import, examine, and modify data, place "observers" on objects, etc.
So, the API provides a powerful mechanism for testing (eg, controlling, inspecting) extensions.
If we could use this mechanism under Cucumber,
the benefits of BDD would be available in a totally different environment!
For more information on SketchUp (etc), see:
Analysis and Design
It might be possible to make Cucumber run as a SketchUp extension,
but this approach doesn't look either easy or robust.
Cucumber and SketchUp are far too likely to get in each other's way.
Also, if either program changed, we might have to rework our modifications.
So, let's run them as separate processes,
letting Cucumber initiate, interact with, and monitor SketchUp plugins.
Cucumber can be used to test various kinds of programs,
but it is most commonly used to test web servers.
So, it supports the specification of many small steps (eg, HTTP requests),
possibly accompanied by some setup and/or teardown code.
To match this orientation to SketchUp, we need ways for Cucumber to:
- install plugins, then launch SketchUp
- run, interact with, and monitor plugins
High-level Interaction
Our step definitions will need to interact with SketchUp at a high level:
pushing buttons, selecting menu items, etc.
Although SketchUp makes no provision for this, Apple's
Open Scripting Architecture (OSA) and
accessibility support
give us a convenient "back door".
OSA makes it possible to send arbitrary
Apple Events to applications.
Apple's
Accessibility Inspector allows any sufficiently motivated and qualified coder
to examine applications and develop event description code
(eg, using AppleScript
or RubyOSA).
However, none of this is easy enough for lazy folks like me.
Fortunately, PreFab Software's
UI Browser
offers a very convenient way to generate AppleScript code.
Basically, the programmer points the UI Browser at an application,
navigates to the widget of interest, and specifies the desired action.
The resulting code can then be edited (eg, generalized), stored in a file,
and used via the osascript command.
Low-level Interaction
Although it is quite possible for a plugin to operate autonomously,
it would be convenient for Cucumber to be able
to direct (and monitor) the plugin at a lower level.
For example, Cucumber could start up a plugin,
then feed it a series of actions and report on the results.
Distributed Ruby (DRb)
should provide a convenient and powerful way for Ruby code in Cucumber
to interact with objects in SketchUp (and vice versa).
However, I haven't (yet) been able to require DRb within SketchUp.
Anyone who knows how to do this (on Mac OS X) is implored to get in touch!
In the meanwhile, I plan to use named pipes, temporary files, etc.
General Approach
Bringing these details together, we arrive at the following general approach:
- Using Cucumber, we create a working directory of configuration data,
AppleScript (interaction) and Ruby (plugin) code, etc.
We store the directory's path in an environment variable
and launch SketchUp, possibly specifying one or more documents.
- SketchUp loads assorted plugin files,
including one for Cucumber initialization.
- The Cucumber plugin saves the directory's path,
then processes each Ruby file in the directory
(eg, running require, creating an initialization flag file).
- The Ruby files perform various setup actions.
Some provide infrastructure (eg, "helper" methods).
Some create test methods and register them
as "Plugin" menu items (eg, Cuke_0001) in SketchUp.
- When Cucumber detects the last flag file, it waits a couple of seconds,
then uses AppleScript to select the menu item for the first test.
- Some test plugins may act as "servers",
allowing Cucumber to specify and monitor fine-grained activities.
- As each test plugin finishes,
it creates a "flag" file for the run.
Cucumber can use these (and a fallback timer) to pace its activities.
At this writing, I am able to use the command line
(supplemented by AppleScript and Ruby code)
to launch SketchUp, run plugins, display and dismiss message boxes, etc.
In short, the high-level plumbing is basically in place.
Hooking it up to Cucumber will take some work,
but I don't expect any show-stoppers to emerge.
Thoughts on Testing
SketchUp's API provides a wealth of methods
that allow plugins to examine and modify the current model.
Based on these, we should be able to build up some "helper" methods
akin to the ones used to test Ruby on Rails applications.
However, that's mostly a pipe dream at this point.
Meanwhile, we need to develop testing approaches
that work well with both Cucumber and SketchUp.
Case Study: Stairs
The stairs.rb script, described in the
Google SketchUp Ruby API Developer's Guide,
is supposed to generate a simple staircase (ie, 12 treads and 12 risers).
How can we verify that it is behaving correctly?
For simplicity, we'll start with an empty model, then run the plugin.
We can then examine the model to see what entities the plugin produced.
Here are some assertions to consider testing:
- There should be 24 faces and 73 edges (3 edges/face, plus one).
- All faces should be rectangular and have the same width.
- The faces should form a series, joined by "width" edges.
- The series should begin with a riser and end with a tread.
- Risers should be vertical and have the same height.
- Treads should be horizontal and have the same depth.
These assertions are at a level that would work for a client,
but they also look reasonably easy to test (famous last words :-).
Once I have a bit more plumbing in place,
I'll give them a try and find out where the monsters lurk...
Reducing Confusion
It won't always be possible to start with an empty model.
For example, the plugin being tested may need entities to operate on.
So, there is lots of opportunity for the test code to get confused.
Which entities (eg, edges, faces) "belong together"?
Which entities did the plugin create, modify, etc?
SketchUp can't answer these sorts of questions,
but it provides a facility which can be used to do so.
Any object can be given any number of attributes.
If the plugin under test leaves a suitable trail of attributes,
the test code should be able to follow it.
Again, this is mostly speculation at this point,
but I expect to use SketchUp attributes pretty heavily
to reduce confusion in testing.
Project Wiki
For brevity, I have glossed over quite a few implementation details.
These may be found, in their current (evolving) state, in my SketchUp project wiki:
]]>
