David Chelimsky: limiting scope of autotest

limiting scope of autotest

David — Wed, 05 Mar 2008 02:07:00 -0500

If you use autotest with rspec or test/unit, you’ve probably had this experience (or one like it):

You want to add some new behaviour to a model object, so you write a spec, watch it fail, make it pass, and then wait until the entire spec suite runs. Even if you’ve got a fast-running suite, this can be painful sometimes.

Wouldn’t it be great if you could limit the scope of what directories autotest observes? Well it turns out that you can! Recent releases of ZenTest include a find_directories attribute on the autotest object. Just add this to your .autotest file:

Autotest.add_hook :initialize do |at|
  unless ARGV.empty?
    at.find_directories = ARGV.dup
  end
end

and then you can say:

autotest app/models spec/models

and it will only observe those directories. This is nice and flexible, but I find that most of the time I’m wanting pairs like that: app/models and spec/models, or app/views/accounts and spec/views/accounts. In that case, I’d really like to just say:

autotest models

To accomplish that you can do this to the hook instead:

Autotest.add_hook :initialize do |at|
  unless ARGV.empty?
    at.find_directories = ["spec/#{ARGV.first}","app/#{ARGV.first}"]
  end
end

Want the best of both worlds? Try this:

Autotest.add_hook :initialize do |at|
  unless ARGV.empty?
    at.find_directories = ARGV.length == 1 ? ["spec/#{ARGV.first}","app/#{ARGV.first}"] : ARGV.dup
  end
end

The only limitation of this is that it’s based on directories, not files. Once in a while, when I’m bootstrapping a new object, I’ll keep the examples and the implementation in the same file until I’ve got things fleshed out a bit the object is ready to play nice with others. In that case, I might like to just point autotest to that one file. I started working on a patch for this for ZenTest, but I’m not sure it’s worth the extra effort. What do you think?

Regardless – happy auto-exemplifying!

"limiting scope of autotest" by Chris Anderson

Fri, 14 Mar 2008 13:21:13 -0400

David – you may be right.

I remember having some conversations and reading some threads where the upshot was “if you’re test suite is slow the problem is your tests” which is often the case. That and being frustrated by the changing plugin API are probably what gave me the impression that Autotest is opinionated software. I definitely don’t want to put words in Ryan’s mouth, so perhaps it’s my mistaken impression.

Anyway, autotest is nearly the most important piece of gear in my kit, and if I’m working, it’s running.

@steve it should be in your ~ directory

cat ~/.autotest

If it’s not, you can create it.

"limiting scope of autotest" by steve

Fri, 14 Mar 2008 12:37:26 -0400

help a noob—where is the .autotest file (on Mac OS X)

"limiting scope of autotest" by David Chelimsky

Fri, 14 Mar 2008 10:25:41 -0400

@chris – what gives you the idea that ZenSpider hates the idea of scoping autotest? He actually invited me to submit a patch for this before I realized it was just as easy to do it .autotest as described above :)

"limiting scope of autotest" by Anny

Thu, 13 Mar 2008 05:49:03 -0400

PS How can I get that little cloud before the comment link on the blog? Is that Haloscan and Blog template?;

"limiting scope of autotest" by Chris Anderson

Tue, 11 Mar 2008 19:39:04 -0400

David,

My setup uses a regex which allows autotest only to run the files that match it. this makes limited scoping to a set of models really easy.

usage:

AUTOTEST=’post|comment’ autotest

now autotest will only look at files that match ‘post’ or ‘comment’.

my implementation is crappy though – I just shoved some code into my copy of the autotest gem.

from ZenTest 3.9.1 autotest.rb line 360:

        # my horrible hack
        order = order.select do |k,v| 
           (ENV['AUTOTEST'] and ! ENV['AUTOTEST'].empty?) ?   
              Regexp.new(ENV['AUTOTEST']).match(k) : true
        end.uniq

ZenSpider seems to hate the idea of scoping autotest, so I doubt we’ll ever get this functionality built in. I don’ think there’s a hook you can use for this anymore, at least I didn’t see anything there in the code that looks promising.

"limiting scope of autotest" by Chris Anderson

Tue, 11 Mar 2008 19:38:46 -0400

David,

My setup uses a regex which allows autotest only to run the files that match it. this makes limited scoping to a set of models really easy.

usage:

AUTOTEST=’post|comment’ autotest

now autotest will only look at files that match ‘post’ or ‘comment’.

my implementation is crappy though – I just shoved some code into my copy of the autotest gem.

from ZenTest 3.9.1 autotest.rb line 360:

        # my horrible hack
        order = order.select do |k,v| 
           (ENV['AUTOTEST'] and ! ENV['AUTOTEST'].empty?) ?   
              Regexp.new(ENV['AUTOTEST']).match(k) : true
        end.uniq

"limiting scope of autotest" by David Chelimsky

Thu, 06 Mar 2008 22:48:34 -0500

@Jean-Michel – yes, I too come from the wonderful world of java tools and I do long for the day that my language of choice has the same level of TDD/Refactoring support. It’s moving along. NetBeans has some cool features now (including code completion for RSpec!) but it’s got a way to go.

"limiting scope of autotest" by http://www.21croissants.com

Thu, 06 Mar 2008 13:30:49 -0500

David said: @Jean-Michel – the “one file” thing is only cases in which I’m working on something very focused that has no incoming dependencies yet. So I’d fire up autotest, restricting it to the one file (or one spec and one implementation file), and get that working right, and then restart autotest with the full suite (or just the directories for the encompassing component).

Jean-Michel replies: Now I understand better your needs and your tip . I can’t live without Autotest either :-)

David said: As for execute_latest_failure – seems like that would have to be re-written all the time, as it names a specific example. Or am I missing something?

Jean-Michel replies: Yes. In fact, I was a bit ashamed of my hack. This is the full code:

def execute_latest_failure
    "" 
#    "-e 'CommunityController (when the session has been resetted) AJAX add_country should not throw exception'" 
  end

So every time, I need Autotest to execute the “ONE” spec I am working on, I commment the first line of execute_latest_failure and uncomment the second line … Pathetic I know

David said:

As for autotest not recognizing new files and new methods, this is new behaviour since the recent releases. From what I can tell this does not happen in with test/unit, so I need to figure out what’s going on there.

Jean-Michel replies: Sorry about the misunderstanding. I meant: as I write the spec before the code, it’s normal that the spec failed with “NoMethodError”. When I save my spec, it triggers Autotest and the spec fails. Then, I will add the new method in the class …

David said: I think autotest is the wrong place for adding code to your code. That should be in the editor, in my opinion. This is the sort of help java developers get from tools like eclipse and IntelliJ out of the box that could make a Ruby developer’s life a bit easier, but putting it in autotest makes autotest a bit too invasive IMO.

Jean-Michel replies: At the end of the day, you’re right, this is definitely a feature that IDEs should provide.

But Autotest code seems to be easier to plugin. I think I will play with it and try to implement this feature. I was addicted to it when I was a java developper working with Eclipse: the method does not exist? => CTRL+1 triggerred a quick fix and RETURN was suggesting to add the missing method. It was a very productive way of TDD :-) What is your background?

"limiting scope of autotest" by kat

Thu, 06 Mar 2008 12:41:17 -0500

I’ve been having the same experience as Niel – when going from red to green it doesn’t run the entire suite anymore.

"limiting scope of autotest" by David Chelimsky

Wed, 05 Mar 2008 15:41:08 -0500

@Neil – I don’t find that funny :)

Seriously – I’m having a different experience than you are.

"limiting scope of autotest" by Neil Wilson

Wed, 05 Mar 2008 14:27:10 -0500

@David,

Same with a red green cycle.

"limiting scope of autotest" by Neil Wilson

Wed, 05 Mar 2008 14:10:21 -0500

@David,

That’s funny, because autotest -f doesn’t do that here. If I touch a file after firing it up it just runs that spec and nothing else.

I’m using 3.9.1

"limiting scope of autotest" by JJ

Wed, 05 Mar 2008 10:37:37 -0500

Good codes. So far i have no bugs with them.

"limiting scope of autotest" by David Chelimsky

Wed, 05 Mar 2008 09:27:31 -0500

@Neil – the ‘-f’ option makes it so autotest does not run everything until you change something. As soon as you make a change it does its normal thing: as soon as you get red followed by green, it’s going to run the whole suite. I want to make it so it runs a subset of the suite to keep things moving faster.

@Jean-Michel – the “one file” thing is only cases in which I’m working on something very focused that has no incoming dependencies yet. So I’d fire up autotest, restricting it to the one file (or one spec and one implementation file), and get that working right, and then restart autotest with the full suite (or just the directories for the encompassing component).

As for execute_latest_failure – seems like that would have to be re-written all the time, as it names a specific example. Or am I missing something?

I think autotest is the wrong place for adding code to your code. That should be in the editor, in my opinion. This is the sort of help java developers get from tools like eclipse and IntelliJ out of the box that could make a Ruby developer’s life a bit easier, but putting it in autotest makes autotest a bit too invasive IMO.

"limiting scope of autotest" by Matt Darby

Wed, 05 Mar 2008 08:37:24 -0500

Great tip! I’ve wished for this functionality for months!

"limiting scope of autotest" by http://www.21croissants.com

Wed, 05 Mar 2008 07:57:52 -0500

Nice trick David!

I am not sure using autotest to point to only one file would have an added value: I really like when autotest detects a regression in a class which has nothing to do with the class I am specing because my changes had unexpected side effects …

However, some of my specs are running slowly (I am not using mocks everywhere yet!) so I monkey patched RSpec with the following hack so Autotest only runs the latest failed spec:

class Autotest::Rspec < Autotest

  def make_test_cmd(files_to_test)
    return "#{ruby} -S #{spec_command} #{add_options_if_present} #{execute_latest_failure} #{files_to_test.keys.flatten.join(' ')}" 
  end

  def execute_latest_failure
    "-e 'CommunityController (when the session has been resetted) AJAX add_country should not throw exception'" 
  end

Doing that, Autotest could run that only spec in 5s only instead of almost 45s for my CommunityControllerSpec!!!

Do you think I should propose a patch?

Another thought. As I am working in a TDD Style, I always write the spec first and when I add a new method to a class, Autotest first fails with:

NoMethodError in 'MyClassUnderBDD should  a_long_descriotion)'
undefined method 'the_method_just_added_coz_I_write_specs_first'

Which is fair enough ;-) This was done on purpose.

What do you think about Autotest asking you if you want to add the skeleton of the new method automatically to class under test? It would save me so many copy-pasted!!!

Jean-Michel Garnier

"limiting scope of autotest" by Neil Wilson

Wed, 05 Mar 2008 04:54:13 -0500

Don’t you just do

autotest -f

to skip the full run when autotest starts. Then it just runs the files that change.

Or did I not understand the question?