Code in the hole (Entries tagged as commandlinefu)

Auto-generating an FAQ with Prototype

nospam@example.com (David Winterbottom) — Mon, 25 May 2009 20:34:01 +0000

Have just been writing an FAQ page for commandlinefu.com. Documenting is always tiresome, but FAQs particularly so when hand-coding the HTML links between each question and the summary table at the top of the page.

Javascript to the rescue: I cobbled together a quick Prototype script which automatically generates the FAQ summary links by parsing the DOM for the appropriate links:

document.observe('dom:loaded', function(){
    $$('a.question').each(function(ele){
        var id = ele.innerHTML.unescapeHTML().gsub(/[^\w- ]/, '').gsub(/[\s-]+/, '-').toLowerCase();
        ele.writeAttribute({id: id});
        var link = new Element('a', {href: '#'+id})
            .update(ele.innerHTML)
            .observe('click', function(event){
                event.stop();
                Effect.ScrollTo(ele); 
            });
        $('questions').insert(new Element('li').insert(link));
    });
});

Now, all I have to do is write my FAQ ensuring the questions live in anchor tags with the class "question" and there is an empty list tag at the top of the page to house the summary block.

Here's two snapshots the relevant section of the DOM before and after the dom:loaded event has fired. Before:

<ul id="questions"></ul>
<dl>
    <dt><a class="question">What is this site?</a></dt>
    <dd>...</dd>
    <dt><a class="question">Who created this site?</a></dt>
    <dd>...</dd>
</dl>

and after:

<ul id="questions">
    <li><a href="#what-is-this-site">What is this site?</a></li>
    <li><a href="#who-created-this-site">Who created this site?</a></li>
</ul>
<dl>
    <dt><a class="question" id="what-is-this-site">What is this site?</a></dt>
    <dd>...</dd>
    <dt><a class="question" id="who-created-this-site">Who created this site?</a></dt>
    <dd>...</dd>
</dl>

As you can see, the javascript simply extracts the relevant content from the questions and inserts the appropriate identities and links into the DOM to form the quick-links. Using the Effect.ScrollTo function also gives a pleasant user experience.

Using a Phing filter to flush browser caches

nospam@example.com (David Winterbottom) — Sun, 15 Mar 2009 19:01:13 +0000

A quick Phing tip that's made my life easier when deploying new versions of commandlinefu.com.

One of the key performance recommendations from Steve Souders' excellent "High Performance Websites" is to use Expires HTTP headers to set far-future expiration dates for your site components (such as images, Javascript files and CSS stylesheets). This way, browsers can cache the files between requests giving a performance boost to your site. Assuming you're using Apache for serving, the following settings can be used to set these headers for all Javascript and CSS files (there are a few alternative ways of achieving the same result):

<FilesMatch "\.(ico|pdf|flv|jpg|jpeg|png|gif|js|css|swf)$">
    Header set Expires "Thu, 15 Apr 2010 20:00:00 GMT"
</FilesMatch>

The main issue to be aware of using this technique is that, when your components change, you need to ensure your visitors are forced to download the latest version rather than using the one cached by their browser. The only way to ensure this happens is to use a different URL for the assets in question. One option might be to rename the files themselves but a more convenient alternative is to include a query string as part of the request URL (eg <script src="/js/behaviour.js?2009-03-15" type="text/javascript" />). Then changing the query string component is sufficient to force browsers to make a full request for the new component.

This works well but is an easy-to-forget overhead for deployment. However, this substitution can be automated by making use of the Filters that Phing provides. Doing so is trivial: simply insert a tokenised string as the query string after your asset URLs. That is:

<link rel="stylesheet" href="http://codeinthehole.com/css/styles.css?~~CACHEBUSTER~~" type="text/css" />

where the '~~' delimiter indicates the token. Then include something like the following snippet in your Phing deployment script.

<tstamp>
    <format property="build.datetimestring" pattern="%Y-%m-%d-%H-%M" />
</tstamp>
...
<target name="create-temp-build" description="Creates a temporary copy of the source files">	
    <echo msg="Copying deployment files into temporary directory" />
    <copy todir="${dev.folder.temp}">
        <filterchain>
            <replacetokens begintoken="~~" endtoken="~~">
                <token key="CACHEBUSTER" value="${build.datetimestring}" />
            </replacetokens>
        </filterchain>
	<fileset refid="deployment-files" />
    </copy>
</target>

The filterchain component of the copy task parses the given fileset for matching tokens that match and replaces them with the given value. In this example, I'm using timestamps as the replacements as these will ensure a different query string on each deployment. Doing so ensures that the deployed HTML includes the lines:

<link rel="stylesheet" href="http://codeinthehole.com/css/styles.css?2008-03-15-21-51" type="text/css" />
<script type="text/javascript" src="http://codeinthehole.com/js/site-behaviour.js?2008-03-15-21-51">

which then ensure that all subsequent visitors download the latest versions of the CSS and javascript files. The above target is taken from a deployment script I use which creates a temporary snapshot of the codebase that I want to deploy, but the basic principle of using the replacetokens filter is easily transferable to any deployment script.

The most important command-line tip - incremental history searching with .inputrc

nospam@example.com (David Winterbottom) — Tue, 03 Feb 2009 23:40:00 +0000

Getting www.commandlinefu.com off the ground has renewed my interest in Bash, UNIX and all things command-line. Powerful one-liners are things of beauty and are worth collecting; however what I consider to be the most influential command-line tip I know covers four:

"\e[A": history-search-backward
"\e[B": history-search-forward
"\e[C": forward-char
"\e[D": backward-char

These lines need to be placed in your ~/.inputrc file, the start-up script for the Readline utility used by Bash (as well as several other applications) and others). The important commands here are the first two, which bind your up and down cursor keys to incrementally search your history. (The second two ensure that left and right continue to work correctly).

This is incredibly useful for retrieving commands you've used previously and makes a huge difference to your productivity. For instance, to find a previous SSH command from a few days ago, simply type "ss" and press up a few times. This will allow you to browse through all your previous ssh… commands until you find the right one - you never need to use more than 4 or 5 keystrokes to retrieve any previous command. If your cycling through too many commands to find the right one, type in a few more characters to refine the search.

As indicated above, this functionality is available in all applications that use Readline including MySQL, Python, IRB (interactive Ruby shell) and others. Once you're used to this feature, it's hard to live without - the first thing I do once I've been set up as a user on a new server is update my .inputrc file to contain these settings. The one place where I sorely wish this functionality existed is the Firebug Javascript commandline in Firefox (a ticket already exists requesting a similar feature).

Another way of searching your history is to use CTRL+R, which essentially performs a full-text search on your history (keep pressing CTRL+R to cycle through results). In this case, searching for "ssh" will locate all commands that feature this string anywhere in the command. Although this is actually a more powerful feature than the incremental history search described above, I don't often use it as: (a) the incremental search generally lets me jump to the desired command in fewer key-presses and (b) I find "ssh", CTRL+R, CTRL+R slightly awkward to type and less intuitive than "ssh", UP, UP. Horses for courses really - you could probably be just as efficient with either one.

» Hat-tip to the place where I first learnt this: Power Shell Usage by Simon Myers

One extra thing: this functionality can be neatly complimented by some choice history settings in your ~/.bashrc file:

export HISTSIZE=1000000
export HISTFILESIZE=1000000000

These simply set your history to be very large so that you have a huge bank of commands to search.

Current pet project: Command-Line-Fu

nospam@example.com (David Winterbottom) — Thu, 22 Jan 2009 23:16:22 +0000

If you're anything like me, you spend a lot of time at the UNIX command-line manipulating the filesystem, configuring Linux, playing with services and so forth. As any UNIX user knows, tremendous power can be wielded through judicious function selection, piping and output redirection. It's often quite staggering what can be achieved in a single line given a rudimentary knowledge of sed, grep, awk, cut…

Indeed, when I stumble upon a line of particular elegance or usefulness, I generally log them to a Tomboy note (fired up in a flash using Gnome-do). This has proved extremely useful as I am often returning to the list to recall how to, say, rsync a fileset with an exclude list - it's generally faster than Googling or going to the man pages. As time has progressed though, this small repository has grown into a sizable collection and finding a particular command quickly is now a problem. Motivated by this problem and taking inspiration from my daily reading (Hacker News, Programming Reddit and Stack Overflow), I'm in the process of putting together a lightweight web-app for cataloguing and ranking notable UNIX one liners.

The basic idea is for users to be able to store their useful one liners on the site for (a) retrieval in the future and (b) sharing with others who undoubtedly will need to save the same problem. Once created, these commands are parsed for the functions used and any relevant tags to provide good navigational props. The individual commands also be discussed and commented on. Further, users can vote on each others commands allowing simple leaderboards to be constructed. After a few months, it will be interesting to see what the top 10 most useful awk commands are.

It's only been a week (working piecemeal after dinner) but the site's nearly ready for a beta release. It's constructed using a combination of CodeIgniter and Zend Framework, making use of the fast and lightweight nature of CodeIgniter coupled with the extensive range of components that ZF provides. The site's called http://www.commandlinefu.com and should be live in about a week assuming I find some spare time over the weekend. Watch this space.