Now what?

}

Stop. Think. Refactor.

Why stop?  To stop and embrace the evolutionary aspect of designing the architecture.   If I continue, I run the risk of building 10-20x more code on top of user paths with incomplete exception handling, lots of FIXME and TODOs, and quite frankly, various areas with fairly good but really just prototype code.   The existing code is not a loss.  Rather is was a series of experiments; ‘successful’ experiments.  Even the failures were a success.  And I refactored a lot along the way, so the current body of code is still quite valuable.  But it’s time to inspect.

Example:  Loading ruby classes.

I struggled a lot with LOADPATH visibility in the ruby runtime embedded inside vim.  If you ‘require’ a file in your ruby class, it tried to open that class file relative to the current directory where you’re generally editing – not your vim plugin lib path.   This actually turned out to be OK because I started to just ‘require’ all the files up front.  But I had to register the names manually, load base classes before subclasses, and I didn’t want to load the universe of classes in case the plugin failed to load (e.g. remote service not available).

Refactor: This weekend I was surfing the ruby Kernel API and was reminded of ‘autoload’ to simply register the files to load only if the Class/Module is actually referenced.  I turned about 20 lines of code, located in 2 files, with nested conditions, into the following solution that relies on the [Rails-like] convention where camel-case class names match underscore filenames:

 53   # Pre-register ALL ruby files to be loaded on-demand^
 54   # convention: underscore filename => camelized module^
 55   # example: update_task_todo_controller.rb --> :UpdateTaskTodoController^
 56   rbfiles = File.join(VIM::evaluate('g:rally_lib_dir'), "ruby", "**", "*.rb")^
 57   Dir.glob(rbfiles).each do |filename_abs|^
 58     filename_noext = File.basename(filename_abs).gsub!(/\.rb$/, '')  # extract name of file, no extension
 59     modname = filename_noext.gsub(/(^|_)(.)/) {$2.upcase}  # camelize filename^
 60     autoload(modname, filename_abs)^
 61   end^

Example: Global variables.

Yes.  I know.  I felt dirty at the mere thought of using a… glo… glob… global variable.   But after looking at the code, almost 90% of it used the same variable that was passed around EVERYWHERE.   It was the connection class to the remote service that also served as a cache… the whole plugin relies on this connection across the board.  Now the initial architecture is fairly MVC… but this means that the controller grabs the connection, passes it to the model, the model passes it to utility classes.  And only the handful of view classes are immune.   This also meant that automated testing would require… ugh… mocking a connection and passing to all test classes.

Refactor:  I created a single top-level global variable ($conn).  This cleaned up SO much of the API that using the global variable actually felt… clean.   Now I’m not advocating that it was the most pure approach, but from simplification of the APIs and code… it was a great!

68   $conn = ConnectionController.new.conn^

Example: Testable ruby.

In vim, if you want to envoke ruby code, you do this:

10 ruby << FOO
11 puts "bar"
12 FOO

That only makes your vimscript file get messy by having so much ruby code mixed with vimscript.  So instead, you can tuck those lines above into foo.rb and within vim do:

10 runtime /path/to/foo.rb

But that’s the problem!  Look at your ruby file.  The file is flanked by the vim/ruby load syntax “ruby << FOO …. FOO”.   So you can’t load that foo.rb file within ruby for independent… TESTING!   I had a much bigger problem writing tons of ruby classes that couldn’t be tested (easily) than dealing with a single global variable.

Refactor: Turns out that the first example above solved this issue entirely.  By using autoload() to preload the files, those files are written entirely in pure ruby.  Solved.  Now all classes can be loaded and independently tested.

Summary

I want to also add more exception handling for core activities,  support for trap’ping interrupts (e.g. user hits CTRL-C during network activity), and review the MVC flows that currently require 5 files (controller, model, view, formatter, display manager).

I’ll “lose” about 2 weeks doing this architecture refactor.    But I’m confident that it’ll make adding features 2-3x faster and the overall codebase will be maintainable.

Today I managed to crash Vim with a custom ruby-based plugin I’m writing.   I can’t find any solutions on the web and don’t have time for a full day of gdb coredump debugging but managed to narrow down the culprit.   Posting this in case someone else encounters a similar situation — and drop me a line if you do!

Here is the helpful information that Vim spit out:

Vim: Caught deadly signal SEGV
Vim: Finished.

This is the ruby code initiating the segfault (line 27):

26   c.each_index do |i|^
27     VIM::Buffer.current.append(i, c[i])^
28   end^

Here is the vimscript used to create a new buffer into which the above line tries to add new lines:

241   " launch new buffer^
242   exec "silent botright new"^
243   " create a 'scratch' buffer that cannot be saved^
244   setlocal nobuflisted^
245   setlocal noswapfile^
246   setlocal buftype=nofile^
247   setlocal bufhidden=wipe^
248   setlocal nolist^
249   setlocal nonumber^
250   setlocal foldcolumn=0 nofoldenable^
251   setlocal showbreak=>>^
252   setlocal wrap^
253   if has('syntax') | setlocal syntax=rally | endif^

Turns out that removing the bufhidden=wipe setting solves the segfault crash.  So something with the ruby VIM::Buffer.current is conflicting with that setting.   I’m not sure what… but at least i’m not choking anymore.

I’ll followup if I get clarity on this situation…

Tyler and I took a weekend trip from Santa Cruz to Pasadena to the annual JPL openhouse.

Event: $0.

Hotels, Gas, Food: $350.

Father & Son weekend: Priceless.

While the major attraction was the Mars Rover and various test versions, we had a blast in the fabrication facility looking at the tools and 3D printers.

Santa Cruz 1/2 marathon. round 3.

New personal record: 1hr 46min 48sec – 8:06/mi.  Nuts!  [results]

I’ve been working out regularly. Not just for this event, but to stay in shape and in good spirit. 15mi/wk on average. With 2 kids loving my attention, getting more than an hour away a day is tough so 5 mile runs will have to do.

Tip: I often need “mental time” at work to prepare for presentations or challenge value / time spent / waste on tasks de jour…. doing this while running [along the ocean] is very effective.

Kaizen.  Consistently improving your personal best record is a double edge sword.  One one hand, you conquered ‘it.’  On the other hand, you know that next year just got that much more serious. How serious?  Enter the 7min window.

Next year, the challenge is 7min/ 50sec per mile.  Game on.

David P Thomas - Costa Mesa - 115 attendees

Best of all, the 2nd half of each seminar is dedicated to demonstrating the latest software tools on the market to support Agile teams.

As a software engineer by trade, I’m always looking for reusable patterns.   With a focus on software development (and given a 20-min timebox ;) I discussed and demonstrated the following three (3) patterns suited for Agile teams.

Task-based Development – When it comes to tracking software changes, traditionally we focused on specific file versions, revision numbers, or commit transactions.  In this manner, you have to track the sets of files or the non-sequential, often multiple, commit transaction IDs, respectively.  This is horribly manual, fraught with error and even when using BugId in the commit comment, is only a reporting tool at best.  With Task-based development, you want to abstract those details and work at a higher level — a level where you don’t care how many files, how many commit transactions, or even how many people worked on a task (but you do care -who- did -what-).  Rather, you want to track changes as a logical unit of work, or a Task.   AccuRev’s change package facility ties directly to your agile project management (e.g. Rally) or issue tracking system (e.g. Jira, Bugzilla, etc) and associates the tasks/issues/defects to a working set of files.  You can promote as many files, as many times, from as many people and AccuRev tracks the working set.  Furthermore, beyond just reporting, the change package can be delivered as a whole unit to other baselines (e.g. bugfix releases, customer specific releases, future development lines, etc).

Workflow and Multi-stage Continuous Integration – The most important aspect of workflow (IMO) for Agile teams is ‘just enough’ stabilility.  While agile teams innately desire to integrate with other teams, early and often, you still need to provide a buffer – especially if you have follow-the-sun development.  For example, without a Team-level buffer, a single commit from one contributor can destabilize multiple teams.  With a good workflow in place, teams can co-integrate on the order of hours or days and, most critically, know that the integrated changes are “good.”

David P Thomas - Costa Mesa - Sharing Experiences

The idea is to organize teams such that if someone breaks the build, they only break it for their team — not other teams.   If a team deems their changes worthy, they can share with all other teams.   Determining worthiness of a change is the job of Continuous Integration (CI).  You want to be running CI not only at the shared integration area for all teams, but also at the individual team level.  The value of multi-stage CI is that changes are validated at a lower level before they progress to a higher level.  Also, the amount or rigor of testing at a lower level may be less than at a higher level, especially if test executions take hours or days to complete (e.g. full regression or performance tests).

The concept of moving changes along a workflow isn’t new.  Traditionally….changes on a team branch get merged to a shared integration branch.  Those changes get merged to a QA branch, etc.   The root problem here is using  branches… as each branch incurs manual merging.   Instead, AccuRev uses inheritance-based streams that allow changes to implicitly flow from parent stream to child.   The parent-child nature of streams lets you build an n-level promotion hierarchy from child to parent where individuals are at the leaves of the tree, organized by team, combined at the team-of-teams level, then queued for end of iteration review.  As changes are promoted up the tree, they automatically flow down to corresponding children.    This is a beautiful combination of stability and continuous [team] integration.

Private Sandbox - With the spotlight on the team, we can’t entirely forget about supporting the individual contributors.  Private sandboxes are an individual’s cornerstone for sanity.  By tracking private commit of file and directory changes, savepoints are marked that can be compared over time or even reverted back to if a current task is deemed out of scope.   The sandbox is also tracking refactoring and namespace changes.  For example, in Java, if you move and/or rename a class, that will likely impact an entire set of files that import the original (unless you are lazy and use import * :)   It is important to track these changes privately especially if you ever need to revert back —the entire set— to a previous state… all without impacting anyone else.  Private sandboxes should also store those changes on the server thus minimizing the requirement to manage, backup, or lose sleep over a given development machine.

It was hard choosing only three (3) patterns but those are the ones I see frequently in practice.  Of the set, I am particularly fond of Task-based development as very few tools support this and it is vital to tracking, delivering, and reporting on logical units of work, rather than just sets of files.

I really enjoy participating in these seminars and especially meeting new folks.   I encourage anyone involved in software development to attend one of these seminars.  They are a great networking event and you get to see some of the latest software in action!

“Silly Dad. chevrons (<) are for kids!” That’s what my son would say.

bad code:
6 from “From User <from@domain.com>“
good code:
6 from “From User (from@domain.com)”

HTH _ dave

30 if ($cart->count_contents() > 100000){ $cart->reset(); }^

Did you see the code comments? I pasted those too. ;(

I just ran into a problem starting postgres 8.4 in my development environment (this worked just fine on ubuntu 9.10).

dthomas@dthomas-laptop:~$ sudo /etc/init.d/postgresql-8.4 start
[sudo] password for dthomas:
* Starting PostgreSQL 8.4 database server * The PostgreSQL server failed to start. Please check the log output:
2010-05-02 21:30:24 PDT FATAL: could not create shared memory segment: Invalid argument
2010-05-02 21:30:24 PDT DETAIL: Failed system call was shmget(key=5432001, size=36880384, 03600).
2010-05-02 21:30:24 PDT HINT: This error usually means that PostgreSQL's request for a shared memory segment exceeded your kernel's SHMMAX parameter. You can either reduce the request size or reconfigure the kernel with larger SHMMAX. To reduce the request size (currently 36880384 bytes), reduce PostgreSQL's shared_buffers parameter (currently 4096) and/or its max_connections parameter (currently 103).
If the request size is already small, it's possible that it is less than your kernel's SHMMIN parameter, in which case raising the request size or reconfiguring SHMMIN is called for.
The PostgreSQL documentation contains more information about shared memory configuration.
[fail]


I was able to fix this by changing the following in: /etc/postgresql/8.4/main/postgresql.conf


106 #ORIG 10.04 UPGRADE FAILED shared_buffers = 32MB # min 128kB^
107 shared_buffers = 16MB # min 128kB^


Boy would I love to get the adult version of this bed ;)