[Note: Due to length, this post covers only the beginning of the run method, up to getting the examples_to_run. It will be continued in future posts.]

Here we are at run. We aren't going to look through the runners themselves; you'll have to trust me that I looked at ExampleGroupRunner, and it calls the run method on the ExampleGroup. :) Let's get down and dirty with ExampleGroup now. Naturally, the actual run method is in module ExampleGroupMethods. Here's run:

     # File lib/spec/example/example_group_methods.rb, line 110
110:       def run
111:         examples = examples_to_run
112:         return true if examples.empty?
113:         reporter.add_example_group(self)
114:         return dry_run(examples) if dry_run?
115: 
116:         plugin_mock_framework
117:         define_methods_from_predicate_matchers
118: 
119:         success, before_all_instance_variables = run_before_all
120:         success, after_all_instance_variables  = execute_examples(success, before_all_instance_variables, examples)
121:         success                                = run_after_all(success, after_all_instance_variables)
122:       end

examples_to_run isn't in the rdoc, but it is in the sourcecode for ExampleGroupMethods. Here it is:

def examples_to_run
  all_examples = examples
  return all_examples unless specified_examples?
  all_examples.reject do |example|
    matcher = ExampleMatcher.new(description.to_s, example.description)
    !matcher.matches?(specified_examples)
  end
end

Wow, I don't know about you, but I like the rdoc format much better (I copied this out of aptana). I wonder if there is an option on rdoc to include private methods. I could regenerate the rdocs to include them. Let me take a look. Sure enough, it does have a -all option. I wonder what would be the best way to generate the docs. Well, I went into the lib\spec folder and executed the following line:

rdoc -S -N -a

It sure did a lot. Let's go see. Sweet. It included the private methods. So, I now have a rdoc that contains what I need.

Here's a better view of examples_to_run

     # File example/example_group_methods.rb, line 304
304:       def examples_to_run
305:         all_examples = examples
306:         return all_examples unless specified_examples?
307:         all_examples.reject do |example|
308:           matcher = ExampleMatcher.new(description.to_s, example.description)
309:           !matcher.matches?(specified_examples)
310:         end
311:       end

I do notice that rdoc still doesn't include have everything, as it doesn't have the examples method. Looking in Aptana, we see the following in example_group_methods:

def examples #:nodoc:
  examples = example_objects.dup
  add_method_examples(examples)
  rspec_options.reverse ? examples.reverse : examples
end

Ah, that #:nodoc: might be a good reason it doesn't show up in the rdoc. :) I can't seem to find an option in rdoc to override this, but I'm not going to look too hard.

The reason this takes a dup of example_objects is that the rest of method changes the list, adding method examples, and, if the reverse flag is set, the examples are run in reverse order. This is done via an intrusive update to the list, so we want to make a copy before we do something like that. Let's look at add_method_examples

     # File example/example_group_methods.rb, line 394
394:       def add_method_examples(examples)
395:         instance_methods.sort.each do |method_name|
396:           if example_method?(method_name)
397:             examples << new(method_name) do
398:               __send__(method_name)
399:             end
400:           end
401:         end
402:       end

Wow! This sorts the instance methods on the example group and loops them. If the method is an example method, it adds it to the list of examples. Wow, this is interesting code:

397:             examples << new(method_name) do
398:               __send__(method_name)
399:             end

So, this will create a new ExampleGroup with the method_name as the description and __send__(method_name) as the implementation. Wow! So, my understanding of this is that the method will be executed.

example_method? delegates to should_method?

     # File example/example_group_methods.rb, line 408
408:       def should_method?(method_name)
409:         !(method_name =~ /^should(_not)?$/) &&
410:         method_name =~ /^should/ && (
411:           instance_method(method_name).arity == 0 ||
412:           instance_method(method_name).arity == -1
413:         )
414:       end

At first, this looks like matcher methods to me, but it seems like these are actually being added to the list of examples to run, so they can't be that. I guess you can write methods that start with should(_not) and they will get run. It also makes sure that you don't take parameters. Let's dissect that regular expression to figure out the exact requirements:

!(a) && b && (c || d)

a = method should be either should or should_not with a ? on the end.

b = method name should start with should

c = 0 parameters

d = variable number of parameters (.arity == -1)

So, basically, it says that the method should either be a checker (should(_not)?) or a method that starts with should but has either no parameters or a variable number of parameters. Why the variable number of parameters? You can pass no parameters to those, as well. If we are going to run a method as an example, you don't want to have to provide a parameter. This could be a possible opening in the future to create parameterized tests.

Whew, we made it pretty far down a rabbit-hole, but let's get back to where we were when we started collecting up the examples:

     # File example/example_group_methods.rb, line 304
304:       def examples_to_run
305:         all_examples = examples
306:         return all_examples unless specified_examples?
307:         all_examples.reject do |example|
308:           matcher = ExampleMatcher.new(description.to_s, example.description)
309:           !matcher.matches?(specified_examples)
310:         end
311:       end

So, we've now got the all_examples list properly initialized, we return it unless specified_examples? Well, let's go take a look at this:

     # File example/example_group_methods.rb, line 313
313:       def specified_examples?
314:         specified_examples && !specified_examples.empty?
315:       end

     # File example/example_group_methods.rb, line 317
317:       def specified_examples
318:         rspec_options.examples
319:       end

So, if we have any examples off rspec_options and it isn't empty, then we have specified_examples. Where did this come from?

PS C:\ruby\lib\ruby\gems\1.8\gems\rspec-1.1.1> dir -Recurse | Select-String "rspec_options.examples"

lib\spec.rb:15:      @run || rspec_options.examples_run?
lib\spec\example\example_group_methods.rb:318:        rspec_options.examples

That seems very strange. Now, it is wrapped in specified_examples, so let's look for uses of that:

PS C:\ruby\lib\ruby\gems\1.8\gems\rspec-1.1.1\lib\spec> dir -Recurse -Include *.rb | select-string "specified_examples"

example\example_group_methods.rb:306:        return all_examples unless specified_examples?
example\example_group_methods.rb:309:          !matcher.matches?(specified_examples)
example\example_group_methods.rb:313:      def specified_examples?
example\example_group_methods.rb:314:        specified_examples && !specified_examples.empty?
example\example_group_methods.rb:317:      def specified_examples
example\example_matcher.rb:9:      def matches?(specified_examples)
example\example_matcher.rb:10:        specified_examples.each do |specified_example|

Ah, example matcher uses them. However, I don't see anyone actually adding to them.

Let's stop and think for a bit. The examples are off the rspec_options object. Let's search for uses of rspec_options with any sort of example stuff:

PS C:\ruby\lib\ruby\gems\1.8\gems\rspec-1.1.1\lib\spec> dir -Recurse -Include *.rb | select-string -Pattern "rspec_options\..*example.*"

example\example_group_methods.rb:160:        rspec_options.reverse ? examples.reverse : examples
example\example_group_methods.rb:240:        rspec_options.add_example_group self
example\example_group_methods.rb:244:        rspec_options.remove_example_group self
example\example_group_methods.rb:262:          rspec_options.reporter.example_started(example)
example\example_group_methods.rb:263:          rspec_options.reporter.example_finished(example)
example\example_group_methods.rb:318:        rspec_options.examples
runner\command_line.rb:19:            return $rspec_options.run_examples
runner\heckle_runner.rb:66:        success = @rspec_options.run_examples

Look at add_example_group and remove_example_group. This is in example_group_methods, which looks like it might be what we are interested in.

    # File runner/options.rb, line 71
71:       def add_example_group(example_group)
72:         @example_groups << example_group
73:       end

Okay, this starts to make sense. Let's go look in this for the examples property. Ah, it is a simple one created with attr_accessor.

For our purposes, example_group_methods.register is it. Now, who calls register. This seems like we could go back to previous parts of this serious to see it. I remember there being a register_behaviour in the old code, so it seems like we would have something similar here. Running a select for ".register" returns only a pointer to the comment about using register on the ExampleGroupFactory (example_group_factory.rb:13). Strange. So, my initial searching proves fruitless: it appears at this moment that nobody calls register on the ExampleGroup.

Looking in the specs for RSpec, I find these two specs:

describe '#register' do
        it "should add ExampleGroup to set of ExampleGroups to be run" do
          example_group.register
          options.example_groups.should include(example_group)
        end
      end

      describe '#unregister' do
        before do
          example_group.register
          options.example_groups.should include(example_group)
        end

        it "should remove ExampleGroup from set of ExampleGroups to be run" do
          example_group.unregister
          options.example_groups.should_not include(example_group)
        end
      end
end

Strange. Going to the root of rspec and doing a search for .register returns no uses of calling .register on an example_group except for the test. Could this be vestigial?

So, here's the point. If nothing ever gets added to example_groups, then the code that uses it is never called. So, anyone using specified_examples is going to always get false. The example_group_runner loops over them, but nobody adds. Very, very, very strange.

The important thing for this series (looping back around) is that we were inspecting the following method:

     # File example/example_group_methods.rb, line 304
304:       def examples_to_run
305:         all_examples = examples
306:         return all_examples unless specified_examples?
307:         all_examples.reject do |example|
308:           matcher = ExampleMatcher.new(description.to_s, example.description)
309:           !matcher.matches?(specified_examples)
310:         end
311:       end

So, if specified_examples? always returns false, then we will always just return all_examples. This excludes the loop that is rejecting examples. That looks strange anyway. We'll postpone the look at that.

Now, this post is getting really long, and we are only halfway through the primary method that we are investigating here:

     # File example/example_group_methods.rb, line 110
110:       def run
111:         examples = examples_to_run
112:         return true if examples.empty?
113:         reporter.add_example_group(self)
114:         return dry_run(examples) if dry_run?
115: 
116:         plugin_mock_framework
117:         define_methods_from_predicate_matchers
118: 
119:         success, before_all_instance_variables = run_before_all
120:         success, after_all_instance_variables  = execute_examples(success, before_all_instance_variables, examples)
121:         success                                = run_after_all(success, after_all_instance_variables)
122:       end

We still need to look at dry_run and then the rest of the setup before we even get to executing the samples. I'll continue in the next post.