complete coverage with SimpleCov

When using SimpleCov in a very ill-covered project, I got amazingly good results: SimpleCov just did not count not-covered files. So files that were never used in our test-suite, were just simply being ignored. While I understand that approach, it did not feel good. I want to measure absolute progress, and I want to know how badly it really is.

So, on a mission to count all uncovered files with simplecov, I encountered an issue in their github repo. In itself it contained/mentioned two solutions:

  • in a three year old comment, a solution to set a starting base-line, and merge it after the tests. Unfortunately, it did not work completely: all files were added, but with complete coverage. Doh.
  • a pull request, claiming to fix it: while it did add all files, files that had before 100% coverage, where now no longer.

Let's get technical. The first solution added a baseline, with for all lines the value nil. The second added the value 0 for each line. The value nil is what SimpleCov uses internally to count for a never line: a line that never matters. Which is either an empty line, a comment line, or begin and end of a class.

The zero is line that is not covered (a 1 is a line that was covered). When merging lines for files that had coverage, I assume the base-line 0 takes precedence over the coverage-calculated nil and so we end up with a non-covered line in the merged result. Bummer.

So I added my spec_helper.rb as follows:

if ENV["COVERAGE"]
  require 'simplecov'
  SimpleCov.start 'rails'
  SimpleCov.coverage_dir 'coverage/rspec'

  all_files = Dir['**/*.rb']
  base_result = {}
  all_files.each do |file|
    absolute = File::expand_path(file)
    lines = File.readlines(absolute, :encoding => 'UTF-8')
    base_result[absolute] = lines.map do |l|
      l.strip!
      l.empty? || l =~ /^end$/ || l[0] == '#' ? nil : 0
    end
  end

  SimpleCov.at_exit do
    coverage_result = Coverage.result
    covered_files = coverage_result.keys
    covered_files.each do |covered_file|
      base_result.delete(covered_file)
    end
    merged = SimpleCov::Result.new(coverage_result).original_result.merge_resultset(base_result)
    result = SimpleCov::Result.new(merged)
    result.format!
  end
end

So, before the test runs, I create a "baseline" containing for all possible ruby files (might need to filter that later), a nil if the line is empty or the "ending end" of a file, and a zero otherwise. After the tests have run, I remove the covered files from the "baseline", and those are then merged with the result that get the final result.

Now maybe try to translate that into a pull request :)


Comments
Add comment

Recent comments

Tags

ruby on rails 34 ruby 26 rails3 17 rails 15 oracle 11 rspec 9 rspec2 7 jquery 7 ubuntu 5 javascript 5 windows 5 activerecord 3 refactoring 3 geoserver 3 gis 3 arrrrcamp 3 actionmailer 2 oracle spatial 2 tdd 2 postgis 2 routing 2 rvm 2 mongoid 2 csharp 2 thin 2 win32 2 gem 2 rails4 2 git 2 service 2 haml 2 cucumber 2 view testing 2 i18n 1 displaysleep 1 spatial 1 gemsets 1 wubi 1 oracle_enhanced_adapter 1 migrations 1 watchr 1 ci 1 plugins 1 coderetreat 1 ie8 1 ssl 1 oci 1 nested model form 1 wcf 1 11.04 1 jsonp 1 ruby-oci8 1 teamcity 1 engines 1 pgadmin 1 soap 1 content_for 1 word automation 1 plugin 1 capybara 1 xml 1 bootstrap 1 migrate to rails3 1 mvc 1 unity 1 rendering 1 word2007 1 x64 1 limited stock 1 fast tests 1 pl/sql 1 delayed_job 1 pdf 1 test coverage 1 optimization 1 processing 1 borland 1 method_missing 1 cross-browser 1 devise 1 schema_plus 1 mongo 1 mongrel 1 dual boot 1 usability 1 mongrel_service 1 dba 1 mission statement 1 model 1 metadata 1 rcov 1 exceptions 1 image_tag 1 attachments 1 bde 1 css 1 yield 1 ajax 1 generative art 1 rails-assets 1 coordinate systems 1 submodules 1 netzke 1 ora-01031 1 authlogic 1 postgresql 1 shopping cart 1 agile 1 fast_tagger 1 subjective 1 wice_grid 1 generators 1 nvidia 1 mongodb 1 etsyhacks 1 staleobjecterror 1 session 1 jeweler 1 wordpress hacked 1 jasmine 1 heroku 1 rjs 1 life 1 unobtrusive-javascript 1 render_anywhere 1 html5 1 rails31 1 json 1 cocoon 1 mingw32 1 observe_field 1 osx 1 actionwebservice 1 testing 1 debugging 1 strings 1