limiting scope of autotest

March 5th, 2008

If you use autotest with rspec or test/unit, you’ve probably had this experience (or one like it):

You want to add some new behaviour to a model object, so you write a spec, watch it fail, make it pass, and then wait until the entire spec suite runs. Even if you’ve got a fast-running suite, this can be painful sometimes.

Wouldn’t it be great if you could limit the scope of what directories autotest observes? Well it turns out that you can! Recent releases of ZenTest include a find_directories attribute on the autotest object. Just add this to your .autotest file:

Autotest.add_hook :initialize do |at|
  unless ARGV.empty?
    at.find_directories = ARGV.dup
  end
end

and then you can say:

autotest app/models spec/models

and it will only observe those directories. This is nice and flexible, but I find that most of the time I’m wanting pairs like that: app/models and spec/models, or app/views/accounts and spec/views/accounts. In that case, I’d really like to just say:

autotest models

To accomplish that you can do this to the hook instead:

Autotest.add_hook :initialize do |at|
  unless ARGV.empty?
    at.find_directories = ["spec/#{ARGV.first}","app/#{ARGV.first}"]
  end
end

Want the best of both worlds? Try this:

Autotest.add_hook :initialize do |at|
  unless ARGV.empty?
    at.find_directories = ARGV.length == 1 ? ["spec/#{ARGV.first}","app/#{ARGV.first}"] : ARGV.dup
  end
end

The only limitation of this is that it’s based on directories, not files. Once in a while, when I’m bootstrapping a new object, I’ll keep the examples and the implementation in the same file until I’ve got things fleshed out a bit the object is ready to play nice with others. In that case, I might like to just point autotest to that one file. I started working on a patch for this for ZenTest, but I’m not sure it’s worth the extra effort. What do you think?

Regardless – happy auto-exemplifying!

Posted by David Filed in Autotest

17 Responses to “limiting scope of autotest”

  1. Neil Wilson Says:
    March 5th, 2008 at 2:07 am

    Don’t you just do 

    autotest -f

    to skip the full run when autotest starts. Then it just runs the files that change.

    <p>Or did I not understand the question?</p>
  2. http://www.21croissants.com Says:
    March 5th, 2008 at 2:07 am

    Nice trick David!

    <p>I am not sure using autotest to point to only one file would have an added value: I really like when autotest detects a regression in a class which has nothing to do with the class I am specing because my changes had unexpected side effects &#8230;</p>


<p>However, some of my specs are running slowly (I am not using mocks everywhere yet!) so I monkey patched RSpec with the following hack so Autotest only runs the latest failed spec:</p>


    class Autotest::Rspec < Autotest

    def make_test_cmd(files_to_test) return "#{ruby} -S #{spec_command} #{add_options_if_present} #{execute_latest_failure} #{files_to_test.keys.flatten.join(' ')}" end

    def execute_latest_failure "-e 'CommunityController (when the session has been resetted) AJAX add_country should not throw exception'" end

    Doing that, Autotest could run that only spec in 5s only instead of almost 45s for my CommunityControllerSpec!!!

    <p>Do you think I should propose a patch?</p>

    Another thought. As I am working in a TDD Style, I always write the spec first and when I add a new method to a class, Autotest first fails with: 

    NoMethodError in 'MyClassUnderBDD should  a_long_descriotion)'
undefined method 'the_method_just_added_coz_I_write_specs_first'

    Which is fair enough ;-) This was done on purpose.

    <p>What do you think about Autotest asking you if you want to add the skeleton of the new method automatically to class under test? It would save me so many copy-pasted!!!</p>


<p>Jean-Michel Garnier</p>
  3. Matt Darby Says:
    March 5th, 2008 at 2:07 am

    Great tip! I’ve wished for this functionality for months!

  4. David Chelimsky Says:
    March 5th, 2008 at 2:07 am

    @Neil – the ‘-f’ option makes it so autotest does not run everything until you change something. As soon as you make a change it does its normal thing: as soon as you get red followed by green, it’s going to run the whole suite. I want to make it so it runs a subset of the suite to keep things moving faster.

    <p>@Jean-Michel &#8211; the &#8220;one file&#8221; thing is only cases in which I&#8217;m working on something very focused that has no incoming dependencies yet. So I&#8217;d fire up autotest, restricting it to the one file (or one spec and one implementation file), and get that working right, and then restart autotest with the full suite (or just the directories for the encompassing component).</p>


<p>As for execute_latest_failure &#8211; seems like that would have to be re-written all the time, as it names a specific example. Or am I missing something?</p>


<p>As for autotest not recognizing new files and new methods, this is new behaviour since the recent releases. From what I can tell this does not happen in with test/unit, so I need to figure out what&#8217;s going on there.</p>


<p>I think autotest is the wrong place for adding code to your code. That should be in the editor, in my opinion. This is the sort of help java developers get from tools like eclipse and IntelliJ out of the box that could make a Ruby developer&#8217;s life a bit easier, but putting it in autotest makes autotest a bit too invasive <span class="caps">IMO</span>.</p>
  5. JJ Says:
    March 5th, 2008 at 2:07 am

    Good codes. So far i have no bugs with them.

  6. Neil Wilson Says:
    March 5th, 2008 at 2:07 am

    @David,

    <p>That&#8217;s funny, because autotest -f doesn&#8217;t do that here. If I touch a file after firing it up it just runs that spec and nothing else.</p>


<p>I&#8217;m using 3.9.1</p>
  7. Neil Wilson Says:
    March 5th, 2008 at 2:07 am

    @David,

    <p>Same with a red green cycle.</p>
  8. David Chelimsky Says:
    March 5th, 2008 at 2:07 am

    @Neil – I don’t find that funny :) 

    <p>Seriously &#8211; I&#8217;m having a different experience than you are.</p>
  9. kat Says:
    March 5th, 2008 at 2:07 am

    I’ve been having the same experience as Niel – when going from red to green it doesn’t run the entire suite anymore.

  10. http://www.21croissants.com Says:
    March 5th, 2008 at 2:07 am

    David said: @Jean-Michel – the “one file” thing is only cases in which I’m working on something very focused that has no incoming dependencies yet. So I’d fire up autotest, restricting it to the one file (or one spec and one implementation file), and get that working right, and then restart autotest with the full suite (or just the directories for the encompassing component).

    <p><strong>Jean-Michel replies:</strong>

    Now I understand better your needs and your tip . I can’t live without Autotest either :-) 

    <p><strong>David said:</strong>

    As for execute_latest_failure – seems like that would have to be re-written all the time, as it names a specific example. Or am I missing something?

    Jean-Michel replies: Yes. In fact, I was a bit ashamed of my hack. This is the full code: 

    def execute_latest_failure
    ""

    "-e 'CommunityController (when the session has been resetted) AJAX add_country should not throw exception'"

    end

    So every time, I need Autotest to execute the “ONE” spec I am working on, I commment the first line of execute_latest_failure and uncomment the second line … Pathetic I know

    <p><strong>David said:</strong></p>


<p>As for autotest not recognizing new files and new methods, this is new behaviour since the recent releases. From what I can tell this does not happen in with test/unit, so I need to figure out what&#8217;s going on there.</p>


<p><strong>Jean-Michel replies:</strong>

    Sorry about the misunderstanding. I meant: as I write the spec before the code, it’s normal that the spec failed with “NoMethodError”. When I save my spec, it triggers Autotest and the spec fails. Then, I will add the new method in the class …

    <p><strong>David said:</strong>

    I think autotest is the wrong place for adding code to your code. That should be in the editor, in my opinion. This is the sort of help java developers get from tools like eclipse and IntelliJ out of the box that could make a Ruby developer’s life a bit easier, but putting it in autotest makes autotest a bit too invasive IMO.

    <p><strong>Jean-Michel replies:</strong>

    At the end of the day, you’re right, this is definitely a feature that IDEs should provide.

    <p>But Autotest code seems to be easier to plugin. I think I will play with it and try to implement this feature. I was addicted to it when I was a java developper working with Eclipse: the method does not exist? =&gt; <span class="caps">CTRL</span>+1 triggerred a quick fix and <span class="caps">RETURN</span> was suggesting to add the missing method. It was a very productive way of <span class="caps">TDD</span> :-)

    What is your background?

  11. David Chelimsky Says:
    March 5th, 2008 at 2:07 am

    @Jean-Michel – yes, I too come from the wonderful world of java tools and I do long for the day that my language of choice has the same level of TDD/Refactoring support. It’s moving along. NetBeans has some cool features now (including code completion for RSpec!) but it’s got a way to go.

  12. Chris Anderson Says:
    March 5th, 2008 at 2:07 am

    David,

    <p>My setup uses a regex which allows autotest only to run the files that match it. this makes limited scoping to a set of models really easy.</p>


<p>usage:</p>


<p><span class="caps">AUTOTEST</span>=&#8217;post|comment&#8217; autotest</p>


<p>now autotest will only look at files that match &#8216;post&#8217; or &#8216;comment&#8217;.</p>


<p>my implementation is crappy though &#8211; I just shoved some code into my copy of the autotest gem.</p>


<p>from ZenTest 3.9.1 autotest.rb line 360:</p>


            # my horrible hack
        order = order.select do |k,v|
           (ENV['AUTOTEST'] and ! ENV['AUTOTEST'].empty?) ?
              Regexp.new(ENV['AUTOTEST']).match(k) : true
        end.uniq

    ZenSpider seems to hate the idea of scoping autotest, so I doubt we’ll ever get this functionality built in. I don’ think there’s a hook you can use for this anymore, at least I didn’t see anything there in the code that looks promising.

  13. Chris Anderson Says:
    March 5th, 2008 at 2:07 am

    David,

    <p>My setup uses a regex which allows autotest only to run the files that match it. this makes limited scoping to a set of models really easy.</p>


<p>usage:</p>


<p><span class="caps">AUTOTEST</span>=&#8217;post|comment&#8217; autotest</p>


<p>now autotest will only look at files that match &#8216;post&#8217; or &#8216;comment&#8217;.</p>


<p>my implementation is crappy though &#8211; I just shoved some code into my copy of the autotest gem.</p>


<p>from ZenTest 3.9.1 autotest.rb line 360:</p>


            # my horrible hack
        order = order.select do |k,v|
           (ENV['AUTOTEST'] and ! ENV['AUTOTEST'].empty?) ?
              Regexp.new(ENV['AUTOTEST']).match(k) : true
        end.uniq

    ZenSpider seems to hate the idea of scoping autotest, so I doubt we’ll ever get this functionality built in. I don’ think there’s a hook you can use for this anymore, at least I didn’t see anything there in the code that looks promising.

  14. Anny Says:
    March 5th, 2008 at 2:07 am

    PS How can I get that little cloud before the comment link on the blog? Is that Haloscan and Blog template?;

  15. David Chelimsky Says:
    March 5th, 2008 at 2:07 am

    @chris – what gives you the idea that ZenSpider hates the idea of scoping autotest? He actually invited me to submit a patch for this before I realized it was just as easy to do it .autotest as described above :)

  16. steve Says:
    March 5th, 2008 at 2:07 am

    help a noob—where is the .autotest file (on Mac OS X)

  17. Chris Anderson Says:
    March 5th, 2008 at 2:07 am

    David – you may be right.

    <p>I remember having some conversations and reading some threads where the upshot was &#8220;if you&#8217;re test suite is slow the problem is your tests&#8221; which is often the case. That and being frustrated by the changing plugin <span class="caps">API</span> are probably what gave me the impression that Autotest is opinionated software. I definitely don&#8217;t want to put words in Ryan&#8217;s mouth, so perhaps it&#8217;s my mistaken impression.</p>


<p>Anyway, autotest is nearly the most important piece of gear in my kit, and if I&#8217;m working, it&#8217;s running.</p>


<p>@steve it should be in your ~ directory</p>


    cat ~/.autotest

    If it’s not, you can create it.