Myron Marston is taking over leadership of the RSpec project, and will be the lead maintainer of the rspec-core, rspec-expectations, and rspec-mocks gems.
Andy Lindeman is taking over as lead maintainer of the rspec-rails gem.
Myron Marston has been contributing to RSpec since the ramp up to the 2.0 release in 2010, and joined the core team in early 2011. In addition to solid contributions to the code base, Myron has taken responsibility for many bug reports, feature requests, pull requests, etc. He also makes a habit of answering RSpec-related questions on Stack Overflow and Twitter, and he does this all with thoughtfulness, patience, and wisdom.
I can’t think of a better choice to lead the RSpec project, so I invited him to do so. Thankfully, SEOmoz, Myron’s employer, is allowing him to work on RSpec during work hours, and so he accepted.
In addition to the overall project lead, Myron will be the lead maintainer of the rspec-core, rspec-expectations, and rspec-mocks gems. While Myron uses these core rspec libs every day for his work, he doesn’t do much with Rails, so we discussed and agreed that we’d ask somebody else to take care of the rspec-rails gem. Enter Andy Lindeman.
Andy Lindeman is the newest addition to the rspec core team. Like Myron, Andy takes great care in shepherding pull requests and answering questions in addition to making his own rock solid contributions. Andy also writes Rails apps and does Rails training at Big Nerd Ranch, which puts him in a great position to ensure that rspec-rails keeps up with changes in Rails remains a great choice for developing Rails apps.
Thanks in part to Big Nerd Ranch for their support of Andy’s work on RSpec, Andy has agreed to take the lead on rspec-rails.
When Dave Astels introduced Steven Baker’s new RSpec library to me back in 2005, I started submitting patches, Steven gave me commit rights and, a year later, he decided to move on to other work and offered me leadership of the project. I was overjoyed to accept.
In those days, I was teaching TDD/Refactoring courses for Object Mentor. I was encouraged to work on OSS in my bench time, and I was particularly interested in tools that helped to promote the documentation and design aspects of TDD. RSpec and some of the early definitions/discussions of BDD (“it’s all behavior!”) fit perfectly into my thinking and my world.
After I left Object Mentor, RSpec was part of my day to day work on Ruby applications, but most of my work on its maintenance moved to my spare time. This was fine at first, as I had the support of employers and family, but I found myself doing less and less of pretty much everything else that I enjoy and learn from.
Back in June, I joined a project team at DRW Trading that deals in a lot of Clojure and almost no Ruby. In the roughly 6 months since, I’ve been learning a new domain, a new language, new programming models, and even a new text editor. I consider myself extraordinarily fortunate to be able to learn all of these new things, but I’ve found myself less and less able to balance my work on RSpec with my job and with everything else I want to do.
And so, it is time.
Of course, I’ll continue to contribute to the project and surrounding conversation, and I look forward to seeing all my friends in the community at assorted conferences in the coming year(s). I’ll just be arriving without my RSpec Lead hat on.
Thank you to Steven Baker for handing me the wheel, and thank you to everybody who has participated in the project and all RSpec users for your support over these 6+ years. It’s been a great honor and a great pleasure.
]]>Thanks to all who contributed. Special thanks to Myron Marston and Andy Lindeman for their personal contributions to the code as well as a great job shepherding pull requests from several new contributors.
UPDATE: If you’re an rspec/rails/capybara user, be sure to read Andy Lindeman’s blog post on Capybara-2.0 and rspec-rails.
Enhancements
--default_path
can be specified as --default-path
. --line_number
can be
specified as --line-number
. Hyphens are more idiomatic command line argument
separators (Sam Phippen).format_docstrings { |str| }
config option. It can be used to
apply formatting rules to example group and example docstrings.
(Alex Tan).rspec-local
options file. This is intended to
allow individual developers to set options in a git-ignored file that
override the common project options in .rspec
. (Sam Phippen)Bug fixes
ExampleGroup#ancestors
. This is a core ruby method that
RSpec shouldn’t override. Instead, define ExampleGroup#parent_groups
. (Myron
Marston)shared_examples_for
, etc.) to just the objects that need it rather than
every object in the system (Myron Marston).fail_fast
so that it properly exits when an error occurs in a
before(:all) hook
(Bradley Schaefer).before(:all)
are always exposed
from after(:all)
, even if an error occurs in before(:all)
(Sam Phippen).rspec --init
no longer generates an incorrect warning about --configure
being deprecated (Sam Phippen).1 seconds
(Odin Dutton)Deprecations
share_as
is no longer needed. shared_context
and/or
RSpec::SharedContext
provide better mechanisms (Sam Phippen).RSpec.configuration
with a block (use RSpec.configure
).Enhancements
--color
option is configured. (Alex Coplan)raise_error
matcher (Myron Marston)=~
and match_array
matchers output a more useful error message when
the actual value is not an array (or an object that responds to #to_ary
)
(Sam Phippen)Bug fixes
include
matcher so that expect({}).to include(:a => nil)
fails as it should (Sam Phippen).be_an_instance_of
matcher so that Class#to_s
is used in the
description rather than Class#inspect
, since some classes (like
ActiveRecord::Base
) define a long, verbose #inspect
.
(Tom Stuart)Enhancements
and_raise
can accept an exception class and message, more closely
matching Kernel#raise
(e.g., foo.stub(:bar).and_raise(RuntimeError, "message")
)
(Bas Vodde)and_call_original
, which will delegate the message to the
original method (Myron Marston).Deprecations:
and_return
with should_not_receive
(Neha Kumari)Enhancements
#errors_on
(Woody Peterson)Bug fixes
should
and should_not
to CollectionProxy
(Rails 3.1+) and
AssociationProxy
(Rails 3.0). (Myron Marston)controller.controller_path
is set correctly for view specs in Rails 3.1+.
(Andy Lindeman)render
properly infers the view to be rendered in Rails 3.0 and 3.1
(John Firebaugh)be_new_record
are more useful (Andy Lindeman)One of the original ideas behind BDD was that testing should be about behavior at all levels. Effectively, all automated tests can be viewed as “functional tests” of entry points, be they user facing or internal, crossing procedural boundaries or not. In this talk we’ll explore approaches to focusing on the behavior of objects, using language that also serves to document the expected behavior in ways useful to both technical and non-technical stakeholders.
The Agile Testing & BDD eXchange is a one-day conference with talks, open-space discussions and focus on ‘quality time’ between presenters and attendees. There is a nice blend of technical talks like Uncle Bob Martin talking about how to apply SOLID principles to test design, and more process oriented talks like Ellen Gottesdiener’s talk on product definition through structured conversation.
You can read the full program and purchase tickets at http://bit.ly/S0Ypb8 (be sure to use the promo-code “bddxnyc-community-discount”).
What: Agile Testing & BDD eXchange NYC
When: October 1st 2012
Where: The Ace Hotel NYC
Twitter: #BDDXNYC
]]>Thanks also to Myron for all his work on two great new features: the new expectation syntax and support for stubbing constants.
Enhancements
--example
options. (Daniel Doubrovkine @dblock)subject(:article) { Article.new }
config.mock_with
and config.expect_with
yield custom config object to a
block if given
include_context
and include_examples
support a block, which gets eval’d
in the current context (vs the nested context generated by it_behaves_like
).config.order = 'random'
to the spec_helper.rb
generated by rspec
--init
.describe
onto just the objects that need it rather
than every object in the system (Myron Marston).Bug fixes
rspec spec\subdir
. (Jarmo Pertman @jarmo)--require
and -I
are merged among different configuration sources (Andy
Lindeman)Enhancements
expect
syntax so that it supports expections on bare values
in addition to blocks (Myron Marston).RSpec.configuration.expect_with(:rspec) { |c| c.syntax = :expect }
RSpec.configuration.expect_with(:rspec) { |c| c.syntax = :should }
RSpec.configuration.expect_with(:rspec) { |c| c.syntax = [:should, :expect] }
RSpec.configuration.add_should_and_should_not_to Delegator
Bug fixes
Numeric
values to be the “actual” in the be_within
matcher.
This prevents confusing error messages. (Su Zhang @zhangsu)should
and should_not
on BasicObject
rather than Kernel
on 1.9. This makes should
and should_not
work properly with
BasicObject
-subclassed proxy objects like Delegator
. (Myron
Marston)Enhancements
stub_const
API to stub constants for the duration of an
example (Myron Marston).Bug fixes
double.should_receive(:foo) { a }
was causing a NoMethodError when double.stub(:foo).and_return(a, b)
had been setup before (Myron Marston).any_instance
and dup
. (Sidu Ponnappa @kaiwren)double.should_receive(:foo).at_least(:once).and_return(a)
always returns a
even if :foo
is already stubbed."%i" % double.as_null_object
). (Myron Marston)should_receive
so that null object behavior (e.g. returning
self) is preserved if no implementation is given (Myron Marston).and_raise
so that it raises RuntimeError
rather than
Exception
by default, just like ruby does. (Andrew Marshall)Enhancements
spec/spec_helper.rb
sets config.order = "random"
so that
specs run in random order by default.render_template
to have_rendered
(and alias to render_template
for backward compatibility)Bug fixes
rspec-rails-uncommitted
that define Rspec::Rails
before rspec-rails
loads (Andy Lindeman)Explicit use of the “subject” abstraction is a code smell, and should be refactored to use a more intention revealing name whenever possible.
rspec-core supports a one-liner syntax to reduce the noise of common requirements like validations:
1 2 3 |
|
Without support for this syntax, the same example might look like this:
1 2 3 4 5 6 |
|
The benefit of this more verbose example is that it we can read it and understand all the parts right away: an object is initialized and assigned to a local variable, then that variable is used for an expectation.
The benefit of the one-liner is that it’s terse and expressive, but that comes at a cost: you can’t see what the expectation is being evaluated against, so you have to understand some underlying mechanics in order to isolate/understand a failure.
subject
abstractionBehind the scenes, the one-liner uses a “subject” abstraction supported by two
methods named subject
. One is a class method on ExampleGroup
, used to
declare the “subject” of all of the examples in the group:
1 2 3 4 |
|
The other is an instance method on ExampleGroup
. The first time it is called
within an example the block passed to the class’ subject
method is evaluated
and its result memoized, returning the same value from that and each subsequent
subject
call:
1 2 3 4 5 6 |
|
Here is what they look like together:
1 2 3 4 5 6 7 8 |
|
The problem with this example is that the word “subject” is not very intention revealing. That might not appear problematic in this small example because you can see the declaration on line 3 and the reference on line 6. But when this group grows to where you have to scroll up from the reference to find the declaration, the generic nature of the word “subject” becomes a hinderance to understanding and slows you down.
In this case, we’d be better served by using a method (or a let
declaration)
with an intention revealing name:
1 2 3 4 5 6 7 |
|
If we can do that, you might wonder why we have “subject” at all. Well, it was originally designed to never be seen:
Note in the example with subject { Article.new }
, that the subject
declaration is initializing an
Article
with no args. Since RSpec knows that the first argument to
describe
is the Article
class, it can store a similar block in the
background as a default, implicit subject declaration, leaving us with:
1 2 3 4 5 6 |
|
That’s a little better, but now subject
appears out of nowhere in the
example, leaving the reader to wonder where it came from. To remove the need
for explicitly referencing subject
, the example delegates should
and
should_not
to subject
when it is, itself, the receiver:
1 2 3 4 5 6 |
|
Starting that line with “should” seems a bit odd though, so the final step is to do it all in one line:
1 2 3 |
|
Now we have a completely implicit subject, and the result reads quite nicely in
both the code and the output when run with --format documentation
:
Article
should validate presence of :title
We still need to trust that something is doing some work for us but it’s all operating at the same level of abstraction, so we don’t have to try to interpret half of the functionality.
subject
Intention revealing names are crucial if you want to be able to quickly scan and understand code as you navigate around different parts of a system. This is as true for specs as it is for implementation, and the generic nature of the word “subject” makes it a poor choice when a more intention revealing name can be used.
subject
ever OK?Guidelines are guidelines; YMMV. In general I would recommend that if there is a reasonable way to use an intention revealing name instead of “subject”, you should. The only use case I can think of in RSpec in which another name can’t be used is the one liner syntax:
1 2 3 4 5 6 7 8 |
|
Here we have to use subject
because that’s the only way to tell RSpec where
to send should
and should_not
. In my opinion, any other explicit appearance
of subject
can and should be refactored to use an intention revealing name.
Based on feedback on this post, I added support for a “named subject,” which lets you reference the declared subject implicitly in one-liners and with an intention revealing name in standard examples:
1 2 3 4 5 6 7 |
|
This will be released with rspec-core-2.11. Keep your eyes out for it!
]]>Bug fixes
object.should_receive(:message).at_least(0).times.and_return value
object.should_not_receive(:message).and_return value
Bug fixes
]]>http://rubydoc.info/gems/rspec-core
http://rubydoc.info/gems/rspec-expectations
http://rubydoc.info/gems/rspec-mocks
http://rubydoc.info/gems/rspec-rails
http://relishapp.com/rspec/rspec-core
http://relishapp.com/rspec/rspec-expectations
http://relishapp.com/rspec/rspec-mocks
http://relishapp.com/rspec/rspec-rails
Enhancements
prepend_before
and append_after
hooks (preethiramdev)
Bug fixes
SPEC_OPTS
options.example.description
returns the location of the example if there is no
explicit description or matcher-generated description.Enhancements
start_with
and end_with
matchers (Jeremy Wadsack)expect {...}.to yield_control
expect {...}.to yield_with_args(1, 2, 3)
expect {...}.to yield_with_no_args
expect {...}.to yield_successive_args(1, 2, 3)
match_unless_raises
takes multiple exception argsBug fixes
be_within
matcher to be inclusive of delta.Bug fixes
exactly
or at_most
expectation is exceededBug fixes
render_views
called in a spec can now override the config setting. (martinsvalin)render_views
for anonymous controllers on 1.8.7. (hudge, mudge)process_view_paths
route_to
matcher with should_not
controller
is no longer nil in config.before
hooksrequest.path_parameters
keys to symbols to match real Rails
environment (Nathan Broadbent)Bug fixes
Enhancements
Bug fixes
--full_backtrace
optionconfig.filter_run
are respected when running
over DRb (using spork).rescue false
from calls to filters represented as Procsits
correctly memoizes nil or false values (Yamada Masaki)Enhancements
Bug fixes
respond_to?
and method_missing
in DSL-defined matchers.Enhancements
Bug fixes
as_null_object
to be passed to with
Enhancments
Bug fixes
mock_model(XXX).as_null_object.unknown_method
returns self againThis is my personal viewpoint, though it is not mine alone. YMMV.
ActiveRecord provides a declarative interface for describing the structure and behavior of a model:
1 2 3 4 |
|
While syntactically similar, these two declarations do fundamentally different things.
The validates_presence_of :title
declaration changes the behavior of
the save
method (and other methods that use save
), and should be specified
explicitly. Here’s an example using shoulda matchers:
1 2 3 |
|
Even though the matcher’s name looks just like the likely implementation, the
validate_presence_of
matcher specifies that you can not save an Article
without a non-nil value for title
, not that the
validates_presence_of(:title)
declaration exists.
The has_many
declaration exposes a comments
method to clients that appears
to be a collection of Comment
objects. Doing Test-Driven Development, you
would add this declaration when a specified behavior requires it e.g.
1 2 3 4 5 6 7 8 9 |
|
This example needs a comments
method that returns a collection in order to
pass. If it doesn’t exist already (because no other example drove you to add
it), this would be all the motivation you need to introduce it. You don’t need
an example that says it "should have_many(:comments)"
.
Some will argue that we don’t need to spec validations either, suggesting that
it "should validate_presence_of(:title)"
is testing the Rails framework,
which we trust is already tested. If you think of TDD as a combination of
specification, documentation, and regression testing, then this argument falls
short on the specification/documentation front because the validation is
behavior and, thus, the spec should specify the validation.
Even if you view testing as nothing more than a safety net against regressions,
the argument still falls down in the face of refactoring. If we add a Review
class that also has_many(:comments)
and validates_presence_of(:title)
, and
we want to extract that behavior to a Postable
module that gets included in
both Article
and Review
, we’d want a regression test to fail if we failed
to include either of those declarations in the Postable
module.
Another argument is that declarations supply sufficient documentation. e.g. we
can look at rental_contract.rb
and know that it validates the presence of
:rentable
:
1 2 3 4 5 6 7 8 9 10 |
|
This is an interesting argument that I think has some merit, but I think it would require an extraordinarily disciplined and consistent approach of using declarations 100% of the time in model files such that each one is the spec for that model, e.g.
1 2 3 4 5 |
|
100% may sound extreme, but as soon as we define a single method body in any one of the models, the declarative nature of the file begins to degrade, and so does its fitness for the purpose of specification. Plus, if we can only understand the expected behavior of a model by looking at both its spec and its implementation, we’ve lost some of the power of a test-driven approach.
Do you spec associations? If so, what value do you get from doing so? If not, have you run into situations where you wished you had?
Same questions for validations.
]]>The rails-3.2.0.rc2 release broke stub_model
in rspec-rails-2.0.0 > 2.8.0.
The rspec-rails-2.8.1 release fixes this issue, but it means that when you
upgrade to rails-3.2.0.rc2 or greater, you’ll have to upgrade to
rspec-rails-2.8.1 or greater.
Because rspec-rails-2.8.1 supports all versions of rails since 3.0, I recommend that you upgrade to rspec-rails-2.8.1 first, and then upgrade to rails-3.2.0.rc2 (or 3.2.0 once it’s out).
http://rubydoc.info/gems/rspec-rails/file/Changelog.md
While not 100% complete yet, we’ve made great strides on RSpec’s RDoc:
http://rspec.info is now just a one pager (desperate for some design love – volunteers please email rspec-users@rubyforge.org). All the old pages are redirects to the relevant RDoc at http://rubydoc.info. RSpec-1 info is still available at http://old.rspec.info.
We’ve still got Cucumber features up at http://relishapp.com/rspec, but we’re going to be phasing that out as the primary source of documentation. There are a lot of reasons for this, and I’ll try to follow up with a separate blog post on this topic.
You can now set default tags/filters in either RSpec.configure
or a .rspec
file and override these tags on the command line. For example, this configuration
tells rspec to run all the examples that are not tagged :slow
:
# in spec/spec_helper.rb
RSpec.configure do |c|
c.treat_symbols_as_metadata_keys_with_true_values = true
c.filter_run_excluding :slow
end
Now when you want run those, you can just do this:
rspec --tag slow
This will override the configuration and run onlly the examples tagged :slow
.
We added an --order
option with two supported values: rand
and default
.
rspec --order random
(or rand
) tells RSpec to run the groups in a random
order, and then run the examples within each group in random order. We
implemented it this way (rather than complete randomization of every example)
because we don’t want to re-run expensive before(:all) hooks. A fair tradeoff,
as the resulting randomization is just as effective at exposing
order-dependency bugs.
When you use --order random
, RSpec prints out the random number it used to
seed the randomizer. When you think you’ve found an order-dependency bug, you
can pass the seed along and the order will remain consistent:
--order rand:3455
--order default
tells RSpec to load groups and examples as they are declared
in each file.
We added an --init
switch to the rspec
command to generate a “spec”
directory, and “.rspec” and “spec/spec_helper.rb” files with some starter code
in them.
We discovered that the matcher DSL generates matchers that run considerably slower than classes which implement the matcher protocol. We made some minor improvements in the DSL, but to really improve things we re-implemented every single built-in matcher as a class.
]]>Nothing really changed in rspec-rails or rspec-mocks, but rspec-core and rspec-expectations have both gotten some nice improvements.
rspec-core offers a number of configuration options which can be declared on
the command line, in a config file (.rspec
, ~/.rspec
, or custom location),
as well as in an RSpec.configure
block (in spec/spec_helper.rb
by
convention). Before this release, some options, but not all, could be stored in
config files and then overridden on the command line. The problems were that it
was inconsistent (not all options worked this way), and we couldn’t override
options that were set in RSpec.configure
blocks.
With this release, almost all options declared in RSpec.configure
can be
overridden from the command line, and --tag
options can override their
inverses. For example, if you have this in .rspec
:
--tag ~slow:true
That means “exclude examples tagged :slow => true
”. So the following example
would be excluded:
it "does something", :slow => true do
# ...
end
You can also exclude that example from RSpec.configure
like this:
RSpec.configure do |c|
c.filter_run_excluding :slow => true
end
Note: the naming is different for historical reasons, and we will reconcile
that in a future release, but for now, just know that --tag
on the command
line and in .rspec
is synonymous with filter_run_[including|excluding]
in
RSpec.configure
.
Whether the default is stored in .rspec
or RSpec.configure
, it can be overridden
from the command line like this:
rspec --tag slow:true
The rspec
command has an --options
option that let’s store command line args in
arbitrary files and tell RSpec where to find them. For example, you could set things
up so your normal spec run excludes the groups and examples marked :slow
by putting
this in .rspec
:
--tag ~slow
Now add a .slow
file with:
--tag slow
Now run rspec
to run everything but the slow specs, and run rspec --options
.slow
or rspec -O.slow
to run the slow ones.
RSpec’s Rake task supports an rspec_opts
config option, which means you can
set up different groupings from rake tasks as well. The fast/slow example above
would look like this:
namespace :spec do
desc "runs the fast specs"
RSpec::Core::RakeTask.new(:fast) do |t|
t.rspec_opts = '--options .fast'
end
RSpec::Core::RakeTask.new(:slow) do |t|
t.rspec_opts = '--options .slow'
end
end
Or ..
namespace :spec do
desc "runs the fast specs"
RSpec::Core::RakeTask.new(:fast) do |t|
t.rspec_opts = '--tag ~slow'
end
RSpec::Core::RakeTask.new(:slow) do |t|
t.rspec_opts = '--tag slow'
end
end
true
value for tags/filtersThis is not new in rspec-2.8, but all the tags/filters in the example above can
be written without explicitly typing true
:
--tag slow
--tag ~slow
RSpec.configure {|c| c.filter_run_excluding :slow}
it "does something", :slow do
You have to set a config option to enable this in rspec-2.x:
RSpec.configure {|c| c.treat_symbols_as_metadata_keys_with_true_values = true}
In rspec-3.0, this will be the default, but without setting this value in 2.x you’ll get a deprecation warning when you try to configure things this way. It’s ugly, I know, but this enabled us to introduce the new behavior without breaking compatibility with some suites in a minor release.
With 2.8, you can now run the examples in random order, using the new --order
option:
--order rand
The order is randomized with some reasonable caveats:
This provides a very useful level of randomization while maintaining the
integrity of before/after hooks
, subject
, let
, etc.
If you want to run the examples in the default ordering (file-system load order for files and declaration order for groups/examples), you can override the order from the command line:
--order default
The randomization is managed by Ruby’s pseudo-randomization. This means that if you find an order dependency and want to debug/fix it, you can fix the order by providing the same seed for each run:
--order rand:1234
The seed is printed to the console with each run, so you can just copy it to the
command. You can also just specify the seed, which RSpec will assume means you want
to run with --order rand
:
--seed 1234
Every time you run the suite with the same seed, the examples will run in the same “random” order.
The Matcher DSL in rspec-expectations makes it dead simple to define custom matchers that suit your domain. The problem is that it is several times slower than defining a class to do so. While this doesn’t make much difference when you have a custom matcher that you use a few dozen times (where talking hundredths of seconds here), it does make a difference if every single matcher invocation in your entire suite suffers this problem.
The short term fix is that all of the built-in matchers have been re-implemented as classes rather than using the DSL to declare them. This has the added benefit of making it easier to navigate the code and RDoc
Longer term, we’ll try to refactor the internals of the matcher DSL so that it generates a class at declaration time. Eventually.
So that’s it. Nothing ground breaking. Nothing compatibility breaking. But some nice new features and improvements that will make your life just a little bit nicer when you upgrade. We’re doing a release candidate because enough changed internally that I want to give you time to try it out, so please, please do so! And please report any issues you’re having with this upgrade to:
Assuming there are no significant issues, I’ll release 2.8 final within a week or two.
Happy spec’ing!
David
]]>rspec
With the the 2.7.0 release, if you keep all of your specs in the conventional
spec
directory, you don’t need to follow the rspec
command with a path.
Just type rspec
.
If you keep your specs in a different directory, just set the --default_path
option to that directory on the command line, or in a .rspec
config file.
The RSpec::Core::RakeTask
invokes the rspec
command in a subshell. In
recent releases, it assumed that you wanted it prefixed with bundle exec
if
it saw a Gemfile
. We then added gemfile
and skip_bundler
options to the
task, so you could manage this in different ways.
It turns out that Bundler manages this quite well without any help from RSpec.
If you activate Bundler in the parent shell, via the command line or
Bundler.setup
, it sets environment variables that activate Bundler in the
subshell with the correct gemfile.
The gemfile
and skip_bundler
options are therefore deprecated and have no
effect.
NOTE: RSpec’s release policy dictates that there should not be any backward incompatible changes in minor releases, but we’re making an exception to release a change to how RSpec interacts with other command line tools.
As of 2.7.0, you must explicity require "rspec/autorun"
unless you use the
rspec
command (which already does this for you).
Enhancements
--default_path
command line option (Justin Ko)--line_number
options (David J. Hamilton)
path/to/file.rb:5:9
(runs examples on lines 5 and 9)xspecify
and xexample as temp-pending methods (David Chelimsky)--no-drb
option (Iain Hecker)--failure-exit-code
option (Chris Griego)Bug fixes
Rake::DSL
to remove deprecation warnings in Rake > 0.8.7 (Pivotal
Casebook)let
block once even if it returns nil
(Adam Meehan)--pattern
option (wasn’t being recognized) (David Chelimsky)require "rspec/autorun"
with the rspec
command (David
Chelimsky)at_exit
defines the exit code (Daniel Doubrovkine)Enhancements
to_i
(Alex Bepple & Pat Maddox)have_xxx
matcher (Myron Marston)count
(Matthew Bellantoni)Enumerable
before the action, supporting custom
Enumerable
types like CollectionProxy
in Rails (David Chelimsky)Bug fixes
have(n).xyz
documentation (Jean Boussier)safe_sort
for ruby 1.9.2 (Kernel
now defines <=>
for Object)
(Peter van Hardenberg)Enhancements
__send__
rather than send
(alextk)any_instance.stub_chain
(Sidu Ponnappa)any_instance
argument matching based on with
(Sidu
Ponnappa and Andy Lindeman)Changes
failure_message_for_should
or failure_message
instead of
description
to detect a matcher (Tibor Claassen)Bug fixes
any_instance.stub
. (Justin Ko)to_ary
to be called without raising NoMethodError
(Mikhail
Dieterle)any_instance
properly restores private methods (Sidu Ponnappa)Enhancments
ActiveRecord::Relation
can use the =~
matcher (Andy Lindeman)require 'rspec/autorun'
from generated spec_helper.rb
(David Chelimsky)bypass_rescue
(Lenny Marks)route_to
accepts query string (Marc Weil)Internal
Bug fixes
fixure_file_upload
reads from ActiveSupport::TestCase.fixture_path
and
misses RSpec’s configuration (David Chelimsky)primary_key
on class generated by mock_model("WithAString")
(David Chelimsky)This spec …
require 'spec_helper'
describe ListsController do
let(:list) { double("List") }
describe "GET 'index'" do
let(:expected) { [{id: "1", name: "test"}] }
before do
list.stub(:id){ "1" }
list.stub(:name){ "test" }
List.stub(:select){ [ list ] }
end
it "should return the list of lists" do
get :index, format: :json
response.body.should == expected.to_json
end
end
end
… plus this implementation …
class ListsController < ApplicationController
respond_to :json
expose(:lists) { List.select("id, name") }
def index
respond_with(lists)
end
end
… produces this failure:
Failure/Error: get :index, format: :json
ActiveSupport::JSON::Encoding::CircularReferenceError:
object references itself
Here’s why: there are three incorrect assumptions hiding behind the stubs!
select
takes an Array: List.select(["id","name"])
, but the example stubs it incorrectly.{"list":{"id":1,"name":"test"}}
, but the example doesn’t wrap it.Even if the stubs were properly aligned with reality, the reason for the error
is that respond_with(lists)
eventually calls as_json
on the list
object,
which, in this example, is an RSpec double that doesn’t implement as_json
.
We need to either use a stub_model
(which does implement as_json
), or
explicitly stub it in the example:
list.stub(:as_json) { { list: {id: 1, name: "test"} } }
But I’d avoid stubs altogether in this case. Stubs are great for well defined
(and understood) public APIs which are invoked by the code being specified.
In this case, we’re stubbing an API (as_json
) that is invoked by the Rails
framework, not the code being specified. If the Rails framework ever changes
how it renders json, this example would continue to pass, but it would be a
false positive.
Start with a request spec:
require 'spec_helper'
describe "Lists" do
describe "GET 'index.json'" do
it "returns the list of lists" do
list = List.create!(name: "test")
get "/lists.json"
response.body.should == [{list: {id: list.id, name: "test"}}].to_json
end
end
end
This shows exactly what to expect, so when working on clients we can refer directly to this without having to dig into internals.
Run this and it fails with uninitialized constant List
, so generate the list
resource:
rails generate resource list name:string
rake db:migrate
rake db:test:prepare
Run it again and it fails with ActionView::MissingTemplate
. Now we have a
couple of choices. The purist view says “write a controller spec”, but some
people say controller specs are unnecessary if there are already request specs
(or cukes) as they just add duplication.
For me, the answer depends upon the complexity of the requirement as it
compares to what we get for free from Rails. In this case, the only difference
between the requirement and what Rails gives us for free is that we constrain
the fields to id
and name
This is something we can implement in the model,
so I’d just implement this very simple controller code and move on:
class ListsController < ApplicationController
respond_to :json
def index
respond_with List.all
end
end
Now the request spec fails with:
expected: "[{\"list\":{\"id\":1,\"name\":\"test\"}}]"
got: "[{\"list\":{\"created_at\":\"2011-08-27T14:56:19Z\",\"id\":1,\"name\":\"test\",\"updated_at\":\"2011-08-27T14:56:19Z\"}}]"
We’re getting more key/value pairs than we want. I want the model responsible for constraining the keys in the json (Rails implements json transformations in the context of the model, so why shouldn’t we?), so I’d add a model spec:
require 'spec_helper'
describe List do
describe "#as_json" do
it "constrains keys to id and name" do
list = List.new(:name => "things")
list.as_json['list'].keys.should eq(%w[id name])
end
end
end
This fails with:
expected ["id", "name"]
got ["created_at", "name", "updated_at"]
I expect to see created_at
and updated_at
, but I’m surprised (initially) to
see that id
is missing. Thinking this through, it makes sense because the
example generates the list
using new
, so no id
is generated. To get id
to show up in the list of keys, we can use create
instead of new
, or we can
explicitly set it. I’m going to go with setting the id explicitly to avoid the
db hit, accepting the self-imposed leaky abstraction. It’s all trade-offs.
it "constrains fields to id and name" do
list = List.new(:name => "things")
list.id = 37
list.as_json['list'].keys.should eq(%w[id name])
end
Now it fails with:
expected ["id", "name"]
got ["created_at", "id", "name", "updated_at"]
Now we can implement the constraint:
class List < ActiveRecord::Base
def as_json
super({ only: %w[id name]})
end
end
Now the model spec passes, but the request spec fails with:
ArgumentError:
wrong number of arguments (1 for 0)
This is because the as_json
implementation fails to honor the Rails
API:
as_json(options = nil)
as_json
is called by the Rails framework with an options hash. Had we done
this without the request spec and weren’t aware of this information, we’d have
a bunch of passing specs but the app would blow up. Hooray for testing at
multiple levels!
So we add a new example to the model spec:
it "honors the submitted options hash" do
list = List.new(:name => "things")
list.id = 37
list.as_json(:only => :name)['list'].keys.should eq(%w[name])
end
This fails with wrong number of arguments (1 for 0)
as well, so now we adjust
the model implementation:
def as_json(opts={})
super({ only: %w[id name]}.merge(opts))
end
Now the model spec passes again, and so does the request spec! DONE!
The result is a very nice balance of clarity, speed (in spite of the one db hit in the request spec) and flexibility. Any new endpoints we add will get the same json representation because it is expressed in the model (heeding the principle of least surprise). The model spec not only specifies how the model should represent itself as json, but it helps to explain how the rails framework uses the model. All of this with no stubbing at all, and especially no stubbing of APIs our code isn’t invoking.
]]>Assuming you’re using Bundler to constrain your runtime environment (which you
are if you’re using Rails 3 defaults), then you are likely prefixing most shell
commands with bundle exec
.
Two important pieces of information in the comments:
./bin
to your path exposes a serious security risk. Proceed with caution.Here’s a little tip to help save you the prefix, without adding any aliases or functions to your environment.
bundle install --binstubs
export PATH=./bin:$PATH
bundle install --binstubs
creates a bin
directory at the root of your
project, and fills it with Bundler-enabled wrappers for all of the executables
installed by the gems listed in your Gemfile. This enables you to type
bin/rake
instead of bundle exec rake
, for example, ensuring that the
correct version of rake is loaded.
Now prepend ./bin
to your path and you can just type rake
.
First, there are three distinct issues related to the rake release:
Rake 0.9 includes backward-incompatible changes. Per the changelog:
## Version 0.9.0
* *Incompatible* *change*: Rake DSL commands ('task', 'file', etc.) are
no longer private methods in Object. If you need to call 'task :xzy' inside
your class, include Rake::DSL into the class. The DSL is still available at
the top level scope (via the top level object which extends Rake::DSL).
This conflicts with the way Rails, among others, uses Rake, among others. The workaround recommended by @dhh is to constrain the rake version in the Gemfiles in your Rails applications:
gem "rake", "0.8.7"
This is a perfectly fine short term solution to keep your applications running, but it won’t be long before a gem that your Rails application depends on, either directly or through the transitive property of dependencies, is going to specify any of:
gemspec.add_dependency 'rake', '0.9.0'
gemspec.add_dependency 'rake', '>= 0.9.0'
gemspec.add_dependency 'rake', '~> 0.9.0'
When that happens, you’ll need to loosen the constraint in your app if you want to upgrade any of the gems downstream from the gem that introduces this dependency. This is not a big deal because you can control the situation directly in your own Gemfile in your own application.
This advice should not, however, be applied to any gems that depend on Rake.
Let’s say you’re using two gems that both provide Rake tasks and therefore
depend on the rake gem. At some point the maintainer of gem aaa changes the
constraint to "= 0.8.7"
, and the maintainer of gem bbb keeps a looser
constraint: either ">= 0.8.7"
or "~> 0.8.7"
. You upgrade to the new version
of aaa and everything is fine because both constraints are satisfied by
rake-0.8.7.
A little while down the road, the constraint in bbb changes to "~> 0.9.0"
. At
this point you are unable to have the newest versions of aaa and bbb in the
same application. This may not seem like a big deal because you can choose to
not upgrade bbb at this point, but the further upstream the dependency (i.e.
aaa depends on bbb, which depends on ccc), the more likely you are to be
constrained in your upgrade choices.
In short, if you are maintaining a gem that applications or other gems depend on, you are doing end users a disservice by locking down any upstream dependency at one and only one version number.
Now here’s the catch: while some gem maintainers follow some sort of standard versioning and/or release policy, there are many that don’t. If you put in a looser version constraint on a gem whose maintainers introduce breaking changes in patch releases, you are also doing your users a disservice. More on this later.
Perhaps you’ve run into this interaction (or similar):
$ bundle install
$ rake db:migrate
You have already activated rake 0.9.0, but your Gemfile requires rake 0.8.7. Consider using bundle exec.
In this case, the application has an explicit dependency on rake-0.8.7, but
rake-0.9.0 is installed in the shell environment. When you type rake xxx
,
Rubygems activates the 0.9.0 version (the newest version installed), and then
tries to activate 0.8.7 when the app is running.
This is a catch 22 that we’ve been lulled into ignoring by the mere fact that
there have not been any rake releases for a couple of years (rake-0.9.0 was
released 2 years and 5 days after rake-0.8.7). We all expect to type rake xxx
and have it just work. Why not? It’s worked thus far, right?
During the two years of rake-0.8.7, Bundler was born. You may remember that the
Bundler team took a lot of heat during its early days. One of the complaints I
remember was that people didn’t want to have to type bundle exec
to run a
rake task. The result is that pretty much all apps that use Bundler and Rake
have this in their Rakefiles:
require 'rubygems'
require 'bundler'
Bundler.setup
This enables us to type rake xxx
and let Bundler manage loading every other
gem besides rake, which is already loaded by Rubygems. So now when we find both
rake-0.8.7 and 0.9.0 in our gem environment, and the app we’re working with depends
explicitly on 0.8.7, we have (at least) three options:
bundle install --binstubs
Now you can run this
bin/rake xxx
bundle exec rake
In either of the first two options, Bundler controls the activation of the rake
gem for you, which allows it to put the correct version on the $LOAD_PATH
.
gem uninstall rake
This only works if you’re using an isolated gemset for the current project (e.g. using rvm) or you simply don’t need rake-0.9.0 on your system. It also is not a very reliable way to deal with this if you have any sort of automated build or deployment system that is installing gems into a shared gem environment on the build or production servers.
The real problem here is not that we have to type a different command on the
command line. We humans can adapt and get used to doing that. The deeper
problem is that there are countless automation scripts out in the wild that
depend on rake xxx
. In order to support both versions of Rake, they will all
have to be changed to use one of the first two solutions noted above. The cost
of this is no small chunk of change, but it is nobody’s fault but our own for
failing to recognize the cyclical nature of using a versioned tool to run
applications that might require a different version.
On my team at DRW, we tried to constrain our rake dependencies to 0.8.7 as a temporary measure, but each time we installed into a new gem environment we found that rake-0.9.0 was being installed. It turned out that a gem we depended on was installing rake through a back door, and with no version constraint at all. The result was that neither Bundler nor Rubygems had any control over this installation relative to our application (Bundler told Rubygems to install this gem, and this gem silently installed rake). And, to make things more confusing, Bundler reported that it was installing rake-0.8.7 and said nothing about 0.9.0.
The maintainer of that gem released new versions right away, so that issue is now resolved, but it’s entirely possible that other gems you’re using are doing the same (or similar). Just something to keep your eye out for.
One issue this exposes is a lack of common understanding and agreement about how to manage releases and dependencies. The Rubygems Rational Versioning policy and Semantic Versioning are both very sound approaches that share a common scheme for version numbers:
A version has three parts: major, minor, and patch. For example, release 3.0.0 is a major release because the first number was incremented from 2 to 3, 3.2.0 is a minor release because the second number was incremented from 1 to 2, and 3.2.1 is a patch release because the third number was incremented from 0 to 1. Both specs state the following:
If everybody adhered to either policy, we’d all be able to declare our gem dependencies like this:
spec.add_dependency "foo", ">= 2.3", "< 3.0"
… or the following, oft misunderstood, shortcut for same:
spec.add_dependency "foo", "~> 2.3"
This tells Rubygems to install the newest version that is >= 2.3.0, trusting that no version 2.y.x will include breaking changes.
I’ll confess that I didn’t adhere to either approach with RSpec until the rspec-2.0.0 release, last October. I knowingly introduced breaking changes in the 1.x series and RSpec likely lost the confidence of a fair sum of users during that time.
The good news, vis a vis RSpec, is that we’ve been following Rubygems Rational Versioning since the rspec-2.0 release. While we’ve had a couple of regressions in the process (followed swiftly by patch releases that addressed them), there has been only one intentionally breaking change, and that was related to integration with another library. That change was announced, documented, and I don’t recall seeing any issues reported related to it.
We’re not doing SemVer yet because it is more strict than RRV, and RSpec does not currently meet all of its criteria. I do hope, however, to have RSpec on SemVer before the year is out.
… the reality is that getting every gem developer to commit to RRV or SemVer is very unlikely. What those of us who do can do, however, is try to provide a balance of flexibility and safety when we declare upstream dependencies. The rspec-expectations gem, for example, declares the following runtime dependency:
diff-lcs ~> 1.1.2
This expresses an opinion that it is safe for your application (that depends on rspec-expectations) to depend on any 1.1.x version of diff-lcs greater than or equal to 1.1.2, but it is not safe to depend on 1.2.0. While this provides a high degree of safety, it also provides low flexibility: if any other gem your app depends on depends on diff-lcs-1.2 in the future (not likely, since 1.1.2 was released in 2004, but that’s besides the point), you won’t be able to use it with the current release of rspec-expectations, even if the diff-lcs-2.2 release does not include any breaking changes.
If diff-lcs was still under regular maintenance, and it’s maintainers were committed to RRV or SemVer, then rspec-expectations would be able to use this dependency instead:
diff-lcs ~> 1.2
This would provide significantly more flexibility in rspec-expectations’s ability to play nice with other gems that also depend on diff-lcs in the same applciation over a longer period of time.
Note that every gem page on rubygems.org now includes a
recommendation to use the pessamistic constraint using a three-part version
number (e.g. rake ~> 0.9.0
). As just discussed, this provides safety, but
lacks long term flexibility.
So what should maintainers of gems that depend on rake do now? The likelihood
is that some end users will constrain their applications to rake 0.8.7
, and
others will constrain them to = 0.9.0
, ~> 0.9.0
, or >= 0.9.0
. Unless Jim
Weirich announces that rake will follow RRV or SemVer, we have to allow for the
possibility that rake 0.10.0
will introduce new breaking changes. In this
case, I think the responsible thing to do is make sure our gems work with both
rake-0.8 and 0.9, and specify the dependency like this:
spec.add_runtime_dependency 'rake', '>= 0.8.7', '< 0.10'
Trusting that no rake 0.9.x version will introduce breaking changes, this provides the greatest flexibility to end users without exposing them to the risk of breaking changes in rake-0.10.0.
I’m curious to hear what you think about all of this. Do you think this all makes sense? Do you think I’m over or understating the importance, complexity, or severity of these issues? Do you have a different approach to recommend in moving forward? I look forward to your feedback.
]]>