what did i learn today

In some part of my code I ended writing the following: [ruby] self.count_processed ||= 0 self.count_processed += 1 [/ruby] where self is some ActiveRecord model, and count_processed is an attribute of that model (and stored in the database). What am i trying to achieve (if it is not blatantly obvious):

  • if count_processed is not initialised, make it zero
  • increment count_processed Imho this code is clear and readable, but I had the feeling it could be more concise/prettier. So I asked that question on our campfire, to see if we could come up with something shorter. Very nice to be working in a team where you can just throw up questions like these and a very useful, educational discussion unfolds. In short we came up with the following solutions.

Solution 1: to_i

[ruby] self.count_processed = self.count_processed.to_i + 1 [/ruby] Nifty! Isn't it? Use to_i because it will handle the nil correctly. But for me this looked wrong. If I would return to this code after a few weeks, months, I would wonder why I did this way, and not just wrote self.count_processed += 1. So while the code is correct, the intent of the code is not clear.

Solution 2: concise!

[ruby] self.count_processed = (self.count_processed || 0) + 1 [/ruby] This is very beautiful, and the intent is also very clear. If it is not initialised, use the zero, else just use the value and add 1. Awesome.

Solution 3: change the getter

And alternative solution would be to overwrite the getter, like this [ruby] def count_processed self[:count_processed] ||= 0 end [/ruby] Note the notation we used: we use self[:count_processed] because this will fetch the value from the database column. If this was a normal getter, we would write @count_processed (but that does not work for an ActiveRecord model). After redefining the getter, we can just write: [ruby] self.count_processed += 1 [/ruby] While this will work always, does it express its intent more clearly or not? Actually you no longer have to worry about the initialisation, because it is handled, and we can focus on what we really want: increment the counter. I opted for this solution.

What about you?

Which version do you prefer? Do you have any alternative suggestions?

Since rails 3.2.4 the link_to_function is effectively deprecated (again). Everything keeps on working, but when running my specs I got a horseload of deprecation warnings. I know the optimal/recommended way is to use unobtrusive javascript, but the quickest way to fix this is really easy. Just perform the following translation: [ruby] # before link_to_function icon_tag('close.png'), '$(this).parent().hide()', :title => t('actions.close'), :class => 'close' # after link_to icon_tag('close.png'), '#', :onclick => '$(this).parent().hide()', :title => t('actions.close'), :class => 'close' [/ruby] Dead-easy :) In the next refactoring I will remove all onclick blocks and replace them with unobtrusive javascript. But for now I got rid of the deprecation warnings :)

logging the activerecord-session-store

I had an issue where I was not sure the ActiveRecord::SessionStore was actually working (in hindsight: it worked). But to make sure, I needed to know what was stored in the session or retrieved. Now all logging for the session-store is silenced, using Base.silence. Now I was very interested in that logging, and did not find another to unsilence the logging but to add an initializer with the following code. So in file config/initializers/unsilence_logging.rb write: [ruby] class ActiveRecord::Base def self.silence yield self end end [/ruby] This will unsilence the SessionStore logging. Your logging will look like this: [bash] ^[[1m^[[36mAREL (0.0ms)^[[0m ^[[1mUPDATE "sessions" SET "data" = 'BAh7DEkiFnF1aWN<<snipped to protect the inncocent>>iEi9mcC9kYXNoYm9hcmQ= ', "updated_at" = '2012-05-04 11:17:24.704491' WHERE "sessions"."id" = 33635^[[0m [/bash] This at least allows us to verify that the sessions are stored and retrieved correctly. But how can we see what is stored inside the session? To be able to read or inspect what is actually stored in the session, you can use the following line: [ruby] session_data = 'BAh7DEkiFnF1aWN<<snipped to protect the inncocent>>iEi9mcC9kYXNoYm9hcmQ= ' Marshal.load(ActiveSupport::Base64.decode64(session_data)) [/ruby] And this will present your session data in a readable format. This way I learned that a time-drift between our two servers caused a very obscure bug. I hope it can help you too.

using jasmine without rails

Assume you have, like I did, a ruby gem that contains some javascript you want to test standalone. First you need to install the jasmine-gem. You have two options:

  • either you use your gemspec to drive your bundler gemfile, so just add it to your developement dependencies
  • I am still using jeweler, so I use a normal Gemfile, which jeweler parses to populate my gemspec with. Personally I find this much easier, and my workflow is much closer to any ruby development for me, this way If you have a rails-project, starting with jasmine is easy, and takes three easy steps: [bash] # first add jasmine to Gemfile, and then bundle install rails g jasmine:install rake jasmine [/bash] Inside your gem or simple ruby-project it is equally simple, just type [bash] jasmine init rake jasmine [/bash] Now you can need to edit the jasmine.yml to make sure it is running your tests and your code, instead of the example code. In my case I had to change only one line: [ruby] src_files: - app/assets/javascripts/**/*.js [/ruby] Happy testing :) Some interesting links to help you with jasmine:
  • a very interesting slideshow
  • jasmine homepage

I really like generative art, so I have been playing with processing for a while. Processing is an open source language (on top of java), that gives the possibilty to create images, animations, with added interactivity. First I created a simulation of raindrops, and because I wanted it to be easily configurable I used processing.js: processing implemented on top of javascript. So that becomes native processing in the browser. Allowing to interact with HTML and javascript objects easily. Publishing the sketch is just: give people the link. Next-up I wanted to create a music visualisation. Interact with the music. I had the perfect piece of music in my head: "Endless Season" by Ken Ishii. Now processing.js does not interact with music. There is HTML5 audio, but it still is very experimental, and I did not find any API for processing music, reacting to and analysing music played. Processing (the java version) has an excellent library for this: Minim (actually more than one, but I ended up using that one). When the sketch was finished, I wanted to share it, convert it to a video. What were the options.

Use Processing itself

There are two ways to convert a processing sketch to a movie, from within processing itself:

  • use MovieMaker: this requires quicktime and unfortunately does not work on Ubuntu
  • when each frame is drawn, do saveFrame and then convert all frames to a movie afterwards. While in theory this should work, saving the images slowed down my sketch, and ultimately screwed up the sync with the audio. My frame-rate was not consistent enough. For straightforward stuff this does not matter, but I needed it to sync with the audio.
  • There is a third option: GSVideo, but frankly, that seemed to damn hard for me, so I skipped that. So I needed an alternative approach. If it runs correctly on my screen, couldn't I just record it on my screen?

Use some sort of screenrecording/screencast software

To record my desktop, on ubuntu, including the sound from processing proved to have some issues:

  • processing (i.e. java/JDK 6) does not use ALSA to create the sound, but address the hardware devices directly
  • I do not want to record my entire desktop, but a specific part, of a specific size
  • I want to share my video on vimeo, so it has to follow certain guidelines The first proved to be the hardest.

Recording the system audio out together with the video

On ubuntu, I found one approach to work very well for me:

  • use gtk-recordmydesktop
  • use PulseAudio mixer, it will allow to take the sound output as input to record
  • and record away! :) But, unfortunately, since java does not ALSA but uses the hardware devices directly, PulseAudio was unable to capture the sounds. However, by accident I found out that if you export your processing sketch to an applet, and run the applet in the browser, it does use ALSA and can be recorded perfectly. Awesome. Part one solved.

Recording a specific part of the screen

gtk-recordmydesktop allows to specify an area of the screen to record, but somewhat akwardly. Now, for exporting to vimeo, it had to follow certain fixed, optimal sizes. E.g. 640×480 for 4:3 SD video, 640×360 for 16:9 SD video, and 1280×720 or 1920×1080 for HD. And that is hard to do if you are trying to position the recording box manually. But, as I found out here, when using recordmydesktop from the commandline, you can send those options along: [bash] recordmydesktop -x=0 -y=0 --width=320 --height=240 [/bash] So, if you open the Advanced->Misc, and look for the Extra options field, there you can fill in the same options, and when you press record gtk-recordmydesktop will show the bounding box that is recorded.

Preparing your video for uploading to vimeo

To upload your video to vimeo, you have make sure two things are correct: - the screensize, which we discussed before - the video format gtk-recordmydesktop creates an Ogg Theora file, and unfortunately vimeo does not accept that format yet. Converting that to MP4 was hell, until I found Arista Transcoder. Using Arista to create an MP4 is easy (but you have to know it): - open Arista Transcode - create a new conversion - select your file, by default called out.ogv, as the source - select Sony Playstation - PSP as the device - press create! This will create a file called out.mp4 which is just perfect for vimeo, including the sound.

The result

generating pdf on heroku

I was investigating ways to generate pdf's in Ruby on Rails, but I had one enormous constraint: it had to deploy on heroku. There are two very different ways to generate pdf's in ruby:

  • use prawn: it is pure ruby, very powerful. It has it's own DSL, that unleaches all the power of building a PDF, but at the same time: it seems to be very hard and tedious.
  • use some sort of HTML to PDF conversion. In ruby there exists two gems: wicked_pdf and PDFKit, both use wkhtmltopdf under the covers. I dreamed of having a view magically converted to PDF. I went for the second option. Furthermore, I choose wicked_pdf over PDFKit, because I felt the rails integration was better. It allowed my to just render a view which would automatically be downloaded as a PDF.

Setting up wicked_pdf in Rails to run on heroku

Luckily, getting it running on heroku proved to be incredibly easy:just including the correct gem with the binaries that work on heroku. In my Gemfile I added the following: [ruby] gem "wicked_pdf" gem "wkhtmltopdf-heroku", :git => 'git://' [/ruby] And, then, inside a view you want to render as pdf, write something like [ruby] respond_to do |format| format.js format.pdf { render :pdf => "show", :header => { :font_size => '8', :right => '[page] of [toPage]' }, :footer => {:font_size => '8', :right => 'Generated by' } } end end [/ruby] Then, you will still have to create the view, show.pdf.erb. Just make sure your view renders HTML and it will be converted to PDF correctly. That is just awesome. Hope this helps.

The Problem

For jottinx I wrote a small piece of code that allowed to sort items using drag and drop. Of course, after writing it and making sure it works (manually), I want to make sure it keeps working. So I add a test, using cucumber. My scenario actually looks pretty straightforward: [ruby] @javascript Scenario: Dragging a book Given I authenticate as a "default user" And I add a new book with title "a-new-book" And I add a new book with title "another-book" And I add a new book with title "one-last-book" And I drag book "one-last-book" to the top Then book "one-last-book" is at the top of the list [/ruby] The difficult bit was: how do I implement the dragging?. Actually it seemed straightforward, because capybara has a method called drag_to. So I implemented the step like this: [ruby] When /^I drag book "([^"]*)" to the top$/ do |book_title| drop_place = page.find(:css, 'ul.sortable-books li:first') page.find(:xpath, "//a[@href='##{book_title.parameterize}']").drag_to(drop_place) end [/ruby] But, unfortunately, this did not work. I googled around a bit and found the following two similar questions:

  • drag_to with jquery sortable list
  • drag_to don't work on sortable elements The short conclusion: it does not work, and it is a combination of how jquery implemented the sortable element, and the fact that the selenium driver does not support it yet. So, refraining to using the selenium driver directly does not help either. After some more googling, I found a similar question on stackoverflow, and there I found the solution.

The solution

Enter jquery.simulate.drag-sortable.js. It is a script that will allow you to simulate dragging in a sortable object by issuing a simple javascript command: [javascript] // drag item down one position in the list $('#itemToDrag').simulateDragSortable({ move: 1 }); [/javascript] If move parameter is negative, it will move up. And down if positive. If you include the js inside your project, you can easily test that out inside your Chrome console. It just works. Awesome piece of work. To use that in a step-definition, just write: [ruby] When /^I drag book "([^"]*)" to the top$/ do |book_title| page.execute_script %Q{ $('.sortable-books li:last').simulateDragSortable({move: -4}); } end [/ruby] Hope this helps :)

jottinx 0.0.25

This release contains

  • paginate really large notebooks (unobtrusive, when reaching the end of the page, it will automatically fetch missing notes if available)
  • a user can new edit her own profile (email/password) and delete her account if she so desires
  • added more copy to the about/faq page What was in the previous (undocumented) releases?
  • 0.0.24: entered links are clickable after saving (they were clickable in the preview only)
  • 0.0.23: redesigned landing page + fixed an error in the forgot password handling
jottinx 0.0.22

This release contains:

  • tags are clickable and we also show a clickable tag-cloud
  • deleting of notebooks: after deleting now jumps to the first book
  • create a new book in a modal window
  • there were some problems with tags undefined popping up, I hope to have fixed that If you have suggestions or encounter any problems, please let me know.
jottinx 0.0.20

Released some small improvements:

  • somehow it was impossible to save tags: that is now fixed
  • links are now opened in a new tab/window
jottinx 0.0.19

This new version contains a new layout setup. I see some room still for further improvement, but for now we get

  • better use of complete screen estate
  • faster loading of first page (as we start loading book after rendering the rest first)
  • books only get loaded when they are needed Hope you like it. What is next:
  • showing a clickable tag-cloud
  • making sure the sidebar and header is always visible As always, I am happy to hear your suggestions and ideas.
jottinx 0.0.18

Released 0.0.18 includes:

  • improved Markdown editor with immediate preview
  • some smaller improvements: you can now click on links immediately, flash-messages disappear, notes show the creation-date Nextup I will most likely tackle the general layout. Replace the tabs by something smaller. Make more use of the screen in general.
jottinx 0.0.16

Release 0.0.16 includes:

  • added search functionality
  • after import, jump to imported book Note that you can always follow-up on the status and vote for changes on my ticket-board.
jottinx 0.0.15

Yesterday I released 0.0.15, nothing much new, just further improved the import from Google Notebook, after I received some feedback. I also started using Trello to keep track of "things to do". Check it out here. When you check the Trello-board, you will notice I plan to work on some search functionality next, and I want to do something about the layout. Too much space is lost now.

jottinx 0.0.14

Release 0.0.14:

  • fixed a problem when a Google Notebook contained links. In the atom xml they look like this:
\<link rel="related" href="" title="Some title"\>

These can now be imported correctly. Note that if you want to click links inside a note, it will immediately try to edit the note. Currently the work-around is to right-click on the link. I am thinking about a cleaner solution.

  • improved the explanation on the Import page a little Also, if you would be worried, you can export your Google Notebook for at least a few months.
jottinx 0.0.13

Release 0.0.13:

  • fixed a problem with line-endings when importing from Google Notebook
  • adding a new note now behaves as it should
  • added a link to this blog
  • various cosmetic improvements

Next up:

  • delete items to a trash-can, so you can always restore them later

  • allow exporting –if a users wants to leave her data/notes are not lost. But for now, I want the users to find it first :)

  • upon importing jump to the uploaded page

  • allow moving/reordering notes

  • some more cosmetic fixes:

    • remember me: label is not clickable
    • flashes should disappear or be closeable
    • the delete-note link should be styled/placed better If you have more suggestions, problems, ideas: I would be happy to hear them.
jottinx 0.0.12

I just released 0.0.12.

Google Notebook Importing

Importing from Google Notebook now is decent.

  • upon importing the layout is converted to Markdown, and this is pretty close. Google Notebook had some weird ways to store the layout, so some things are not completely as it was. If you are having troubles with it, please contact me.
  • from the Atom XML it seems impossible for me to deduce the original order, so I order them on the last updated date. So if you have been moving stuff around, that order is lost. I will make sure you can reorder the items soon.

Next steps

  • Improve robustness and looks
  • allow exporting --if a users wants to leave her data/notes are not lost. But for now, I want the users to find it first :)
  • upon importing jump to the uploaded page
  • allow moving/reordering notes
  • still not quite sure if the tabs is the best solution. For now it is ok. Can get messy with a lot of books (as I have) And I guess a lot more will come up later. If you have any suggestions, let me know.
Introducing Jottinx

Until recently I was an avid Google Notebook user. I liked the simplicity. I just used it to collect ideas, links, scraps, jottings, but also important stuff I should not forget. I never really used the formatting, and neither the tags. Plain and simple. When I heard it would be discontinued, I looked around for a plain and simple alternative, and decided to build my own. Jottinx is the result of this. Jottinx in short:

  • dead-easy
  • note-taking and nothing more
  • uses markdown for formatting
  • imports your data from Google Notebook
  • is and will always be free
  • you remain the owner of your data, so you can always export your data back out again For the moment it still very much is a work in progress, but I hope you will give it a try.

I normally do not do a lot of view specs, but at least I want to make sure that my view renders without errors. And sometimes I really need to make sure that some link is shown or hidden depending on e.g. the role of the user or linked objects. For example, [ruby] describe "posts/show.html.haml" do context "without any comments" do it "displays no comments" do @post = Factory(:post) render rendered.should contain(I18n.t('')) end end end [/ruby] So we check the rendered result, whether it contains a specific text. But what happens if your view is rendering different yield regions? Like a body (the default region) and a sidebar. Let's use a view like this : [ruby] =show_for @post do |p| = p.attribute :title = p.attribute :content - if @post.comments = render :partial => 'comments' - else = t('') =content_for :sidebar do = link_to 'Edit', edit_post(@post) if is_allowed_to?(:edit) [/ruby] It renders the attributes using the show_for gem, and then renders inside the sidebar a link if the current user is allowed to edit it. Now I want to test what is rendered into the sidebar. To my dismay I found that neither content_for or content_for?worked at all inside rspec. And rendered does not contain the data for the other regions. So somehow I would want to get to the content for :sidebar. It appears that the different regions are actually stored inside an instance variable of the view. Once I figured that out, the rest was easy: [ruby] describe "posts/show.html.haml" do def rendered_content_for(name) view.instance_variable_get(:@_content_for)[name] end context "with enough rights" do it "displays a link to edit the post" do @post = Factory(:post) view.stub(:is_allowed_to?) { true } render rendered_content_for(:sidebar).should contain('Edit') end end context "with no rights" do it "does not display a link to edit the post" do @post = Factory(:post) view.stub(:is_allowed_to?) { false } render rendered_content_for(:sidebar).should_not contain('Edit') end end end [/ruby] Hope that this helps somebody. Or did you find a better way?

Speeding up rspec tests

We have a large test-suite that runs >2200 examples, that took 900 seconds to complete on my machine. Using a few optimisation techniques I was able to bring this time down to 650secs. Which is still long :) But if I run the tests in parallel it takes me down to 300secs (I have two cores). First off, I think it is crucial that your tests are as much as possible in isolation. You are only testing the code under test. Only the model, the controller, the view. This is not something we do consequently everywhere. Sometimes I stub the ActiveRecord finders, but more often I do not. I just make sure the correct data is available. But when a controller calls a method on a model, which I have tested in the model-test, I can safely stub that call. That is sometimes the hardest call: how much mocking and stubbing will you do. A lot of mocks and stubs will make your tests fast, but also brittle, as they are tied in too much with the implementation. If I am certain that the used modules are tested correctly, I will use mocks and stubs. Otherwise I test the used modules as well. So why test ActiveRecord as well? Because sometimes I use somewhat complicated queries and scopes, and want to make sure I did not make a mistake there. Aside of that, my tips to speed up rspec tests:

  • speed up your database: either use sqlite if possible, or tune your database for maximum speed. In my case I am using postgresql and i did the following to improve that.
    • set fsync=off in postgresql.conf
    • set shared_buffers to 28MB -- I even tried setting it to 128MB but that did not make any difference anymore. Using these settings will make postgresql almost behave as an in-memory database. You can even take it further, see here.
  • If you need a lot of data instantiated, using factories, use before(:all) instead of before(:each). Clean up the data in the after(:all). Note: we cannot use before(:all) for everything. Look that up :)
  • use tip from corey haines where applicable to not include spec_helper. For me this means you should take a good look at your spec_helper. We included a lot of helpers and support-methods in our spec_helper, which makes it every easy to write a test, but also makes running a test slower. Maybe it could be profitable to always use a lean spec_helper, and include what you need inside your spec-file. And for some files inside your lib you probably don't even need spec_helper at all.
  • Test smart! Use factories, but always try to create the minimum set needed to work. If you need more than 1 item, 2 items will suffice :)
  • Use rspec-prof to profile slow parts

Do you have any more tips to speed up tests?