Story Runner in Plain English 21
Houston, we have Plain Text!
I just committed a first stab at a Plain Text Story Runner. It’s in RSpec’s trunk and will be (in some form) part of the next release.
Big thanks to Pat Maddox for the StoryPartFactory (which is now called StoryMediator) and to all on the rspec-users list who contributed their ideas and thoughts to the discussion about plain text stories.
Keep in mind that this is brand new and very experimental. I do not recommend that you start converting all your projects to using this.
That said …
A bit of background
[Update: modified to use And for multiple Givens, Whens or Thens]
The initial implementation of Story Runner supported syntax like this (slightly modified from Dan North’s article introducing rbehave):
Story "transfer to cash account",
%(As a savings account holder
I want to transfer money from my savings account
So that I can get cash easily from an ATM) do
Scenario "savings account is in credit" do
Given "my savings account balance is", 100 do |balance|
@savings_account = Account.new(balance)
end
And "my cash account balance is", 10 do |balance|
@cash_account = Account.new(balance)
end
When "I transfer", 20 do |amount|
@savings_account.transfer_to(@cash_account, amount)
end
Then "my savings account balance should be", 80 do |expected_amount|
@savings_account.balance.should == expected_amount
end
And "my cash account balance should be", 30 do |expected_amount|
@cash_account.balance.should == expected_amount
end
end
Scenario "savings account is overdrawn" do
Given "my savings account balance is", -20
And "my cash account balance is", 10
When "I transfer", 20
Then "my savings account balance should be", -20
And "my cash account balance should be", 10
end
end
While this is a really cool start, there are a couple of problems. One is that we’re constrained in the way we phrase things. Because the arguments become part of the phrase, we have to structure each phrase so that the argument comes at the end.
The other problem, for me, is that the differing levels of abstraction in the two scenarios make it difficult to read.
Enter Blockless Steps and Parameterized Steps
The first step in resolving this problem was to decouple the expression of the story from the steps, which is accomplished with the use of Parameterized Steps. Here’s how the story above might look:
[Update: using StepGroup/define instead of StepMatchers/add]
steps = StepGroup.new do |define|
define.given("my savings account balance is $balance") do |balance|
@savings_account = Account.new(balance.to_f)
end
define.given("my cash account balance is $balance" do |balance|
@cash_account = Account.new(balance.to_f)
end
define.when("I transfer $amount") do |amount|
@savings_account.transfer_to(@cash_account, amount.to_f)
end
define.then("my savings account balance should be $expected_amount" do |expected_amount|
@savings_account.balance.should == expected_amount.to_f
end
define.then("my cash account balance should be $expected_amount" do |expected_amount|
@cash_account.balance.should == expected_amount.to_f
end
end
Story "transfer to cash account",
%(As a savings account holder
I want to transfer money from my savings account
So that I can get cash easily from an ATM),
:steps => steps do
Scenario "savings account is in credit" do
Given "my savings account balance is 100"
And "my cash account balance is 10"
When "I transfer 20"
Then "my savings account balance should be 80"
And "my cash account balance should be 30"
end
Scenario "savings account is overdrawn" do
Given "my savings account balance is -20"
And "my cash account balance is 10"
When "I transfer 20"
Then "my savings account balance should be -20"
And "my cash account balance should be 10"
end
end
A bit nicer, yes? The steps coming first is a bit noisy, but that could be extracted to another file, or perhaps we can add a means of associating them with the Story after the Story has already been parsed so they can move below the Story.
That bit aside, look how much cleaner the Story reads now. And we can do a couple of additional things to make it even nicer. One thing you might notice is that the line about transfering (When “I transfer 20”) doesn’t specify which way the transfer goes. We can improve on that by enhancing that step:
steps = StepGroup.new do |define|
...
define.when("I transfer $amount from $source to $target") do |amount, source, target|
if source == 'cash' and target == 'savings'
@savings_account.transfer_to(@cash_account, amount.to_f)
elsif source == 'savings' and target == 'cash'
@cash_account.transfer_to(@savings_account, amount.to_f)
else
raise "I don't know how to transfer from #{source} to #{target}"
end
end
...
That lets us write the step as
When "I transfer 20 from savings to cash"
As you can see, this is a big step towards making stories more clear and flexible.
More on Steps
Another thing you may have noticed is that the Steps are grouped together somewhat arbitrarily. Thanks to a couple of handy convenience methods, you can easily build up libraries of these steps and make them as broad or as granular as you like. Perhaps we want the account steps available to many stories, but the transfer step only to this one. Here’s how you can handle that:
class AccountSteps < Spec::Story::StepGroup
steps do |define|
define.given("my savings account balance is $balance") do |balance|
@savings_account = Account.new(balance.to_f)
end
define.given("my cash account balance is $balance" do |balance|
@cash_account = Account.new(balance.to_f)
end
define.then("my savings account balance should be $expected_amount" do |expected_amount|
@savings_account.balance.should == expected_amount.to_f
end
define.then("my cash account balance should be $expected_amount" do |expected_amount|
@cash_account.balance.should == expected_amount.to_f
end
end
end
steps = AccountSteps.new do |define|
define.when("I transfer $amount") do |amount|
@savings_account.transfer_to(@cash_account, amount.to_f)
end
end
Here we’ve created a subclass of StepGroup, instantiated one and defined an additional ‘when’ that will only be available to this instance.
Goodbye quotes!
Once we were able to get rid of the blocks, the quotes made no sense. So we’ve added support for true plain text stories. So now our example can read like this:
Story: transfer to cash account
As a savings account holder
I want to transfer money from my savings account
So that I can get cash easily from an ATM
Scenario: savings account is in credit
Given my savings account balance is 100
And my cash account balance is 10
When I transfer 20
Then my savings account balance should be 80
And my cash account balance should be 30
Scenario: savings account is overdrawn
Given my savings account balance is -20
And my cash account balance is 10
When I transfer 20
Then my savings account balance should be -20
And my cash account balance should be 10
That gets stored in a plain text file and you can run it by running a ruby file that looks like this:
require 'spec'
require 'path/to/your/library/files'
require 'path/to/file/that/defines/account_steps.rb'
# assumes the other story file is named the same as this file minus ".rb"
runner = Spec::Story::Runner::PlainTextStoryRunner.new(File.expand_path(__FILE__).gsub(".rb",""))
runner.steps << AccountSteps.new
runner.run
And that’s it! It’s that simple. This is still in its very early phases and I’m certain there will be enhancements as people gain experience with it.
If you want to check it out yourself, grab the trunk and do the following:
cd trunk/rspec
ruby examples/stories/calculator.rb
ruby examples/stories/addition.rb
The first example uses Ruby with blockless steps. The second example uses a plain text story stored in examples/stories/addition.
Also, with a couple of small tweaks we’ll be able to consume plain text from any source (not just a local file) and feed it into the PlainTextStoryRunner. This means that we’ll be able to do things like email scenarios to an app that consumes email and runs the scenario against the system and emails you back a report! Crazy, huh?
Lastly, just a reminder, this is only in trunk right now (as of rev 2764), so if you want to explore it you’ll have to get the trunk.
Enjoy!!!!!
Neat. Still hate the reinforcement of capturing/duplicating user stories in programmer artifacts, though.
Wow, that is utterly cool, thanks!
Going to improve my stories anytime soon
Very cool David. I haven’t had a chance to play with the story aspect of rspec too much; this makes me even more interested.
@Scott: out of curiousity, why? I’d rather have our product team be somewhat constrained in writing user stories that we could parse in text files instead of bloated word docs that’re always changing…
This is REALLY great stuff David.
And having this in plain text will make it a lot easier to create a user interface for it to please the Bellware types ;-)
Excellent work, David. I think you did a remarkable job of integrating so much diverse feedback from different people on the mailing list.
I will be extremely interested to see how this plays out when put into practice with real customers (ie. ones who don’t know Ruby but can edit a plain text file). For them there may be an illusion of free-form simplicity there (ie. the appearance that they can write just about anything), when in reality if they’re going to add new scenarios they still need to make sure that their plain text fits within the constraints defined by the matchers (which for them will be unintelligible).
Only when put into practice will we know exactly how this interplay is going to work out. Evidently some of the time the customer’s scenarios will have to be tweaked, and some of the time the matchers themselves will need to be tweaked or augmented. At the moment exactly how this workflow is going to feel is up in the air, so I’m very curious to see if this turns out to be the Holy Grail of customer-authored acceptance tests.
Awesome, that looks great. I’m sure to start using it regularly now. Interested to see what (if any) improvements can be made to it in future.
Great stuff David! I’ve only been tinkering with Story Runner, but this looks like an excellent step forward.
Nice piece of work. Back a while ago when setting PHPSpec’s scope there was brief chatter about plain text parsing and such. It’s great to see this functionality in rspec to work with. All we need now are methods of storing and transporting such plain text between rspec and whichever project management system is in use ;).
Wincent, perhaps a next level is a story builder app which uses option lists and/or autocompletion for adding steps to a scenario from the existing ones, and lets the customer create new step clauses but validates syntax constraints.
VERY glad to see the expressions pulled out of the steps. A little worried about the plain text though. To me it seems to de-emphasize the fact that you’re essentially still writing “code” that needs to match a very specific syntax. Not sure if that’s good or bad…
re: concerns about customers living within the constraints of syntax:
I’m not sure the best way, but we should certainly consider tools to help them down the road. Perhaps an editor app that offers a selection of existing matchers (perhaps with different names). Of course this could get quite complex – how would they request new matchers? what the hell is a matcher (they might ask)? etc. But it’s certainly feasible.
At the very least, we could come up with a tool that reads in all the existing matchers and presents a list of them that can be presented on screen or printed. Who knows? I guess this is a whole new area to explore.
However, I don’t really believe that leaving in the quotes and do/end would make it any less of a problem to solve.
great work David (and everyone contributing + discussing rspec + rbehave) - Is the context of a Rails project, is the story runner format used for all levels of testing (models, controllers, integration) - or just @ the integration level?
Better phrasing on question: In a Rails project, CAN the story runner format be used
all levels of testing, or is it practical onlyintegration level. I just love the idea of plain-text test descriptions.Hi Ryan,
Story Runner is intended for User Stories – i.e. customer facing. This is definitely NOT intended for the more granular sort of specs we do with Spec Runner (i.e. describe it!).
With that, Story Runner will ship with a wrapper for Rails Integration Tests but definitely not anything more granular.
Great stuff! Many thanks
This is wonderful! It solves my biggest concern about the new rspec story runner.
Great Stuff Dave. Thanks. Agree with Justin about pulling out the plain text part though. Do we need to go to that extent, since it is a programmer’s tool anyway.
I would rather run a tool to get a specdoc like output to share the story with non-programmers.
OK I give, there is no way we are going to be able to match this awesomeness in C# on the NBehave project.
http://www.codeplex.com/NBehave
I am jealous!
Awesome job!
This is fantastic. I watched your Story Runner presentation / lightning talk at Rails Conf Europe and was instantly convinced that this is of real value for acceptance / integration test things.
I never really liked the Rails integration tests – maybe because of the missing structure.
I’m using the story runner tests for a project right now and works really well. The text based stories are even better and solve the “how do I explain all that quotes, do/end stuff to the customer”-problem.
That you and all contributers for that awesome testing / specing framework :)
Keep up the good work!
The plaintext format is interesting insofar as it makes tests more readable for the customer once written, but I still think that asking a lay user to authored acceptance tests is like asking a baby to change its own diaper. One way or another you’re going to have to lend a hand or get shit all over the place.
@benjamin_jackson: the goal is not to have customers write stories in a vacuum. The goal is to be able to collaborate on stories with customers and to then provide customers and testers the tools to use the stories as an exploratory tool.
By providing an input format that is plain text, we make that collaboration far simpler.
Also – this is just the first step. Check out Aslak’s prototype story editor. That should help you to see where this is going.