Welcome to Nerdville

So then I discovered that the functionality I was trying to add to bsflite was kinda, sorta, maybe already there. Way to read the entire help output, smart guy.

That left only a couple of features I really wish bsflite had. One is more robust tab completion, such that screennames could be partially completed out to the next difference, the way bash does it. Another is deleting the last word on the line with Ctrl-W, also the way bash does it. I’m in love with bash, you know.

Since I’m about as good at C programming as Stevie Wonder would be at tennis, I thought that if I wanted to tackle one of those features, I’d have to go with the easier one. So, Ctrl-W word deletion it is!

I jumped the gun on the thing and had a semi-working prototype up and running, which I posted to the project page on Sourceforge just before realizing that it is full of bugs. Back to the drawing board I went, writing a test harness and experimenting with pointers and strings until I was really pretty darn sure it would work.

At which point it did not work.

As it turns out, the third time is the charm, and I really think I’ve gotten it to a workable stage. I was able to make it segfault once, but I haven’t yet figured out what caused it to do that, so for what it’s worth, here it is. Without further ado…

The bsflite 0.82 word delete patch

Long has it been since a post was made here; my apologies. Last week I totally snapped and bought an iPhone despite the modest litany of reasons I thought I shouldn’t (locked to one carrier, not standard GSM, battery life qualms, questionable extensibility) and my overall reaction is do it. If you are a smart and resourceful person, as I am sure you are since you are reading this blog, you will have no trouble bending the iPhone to your whims. This is one such report.

Lately, I’ve been Twittering (yes, it’s a verb now, too). For those of you who aren’t in the know, Twitter is a service that allows you to send a text message reporting, simply, what you are doing. That text message is then displayed on Twitter’s site (if you so desire), but is also relayed via SMS to any other Twitter users who are “following” you. Likewise, you receive text messages from Twitter when people you are “following” send in their updates.

The whole thing is a little bit silly, but the reported effect is to have a better peripheral idea of your friends’ daily lives and circumstances without having to hassle them synchronously (a little Web 2.0 lingo for you there). Thus, when you run into so-and-so, you may remember that they recently crashed their car or that they saw some movie you wanted to see, and so on. It’s not important information, but it’s neat.

So there I was, Twittering my heart out, when I realized that more of my friends read Facebook on a daily basis than participate in Twitter, and that Facebook provides an interface for updating your “status,” which is printed next to your picture on your profile page and disseminated through the “news feed.” Obviously, I could not stand idly by and let Twitter be updated but not Facebook! Ruby to the rescue.

The first step was to build an interface to Facebook’s status updating system, which is not supported in their public API (for shame). Because there is no programming interface for it, I had to make my script access the page through normal HTTP methods, forge the login security of the browser and cookies, and post the update that way. Here is the code:

Feel free to pick through it. The important part to notice is the handling of the authentication cookies. I used the “Live HTTP Headers” Firefox Add-In to trace the headers during a Facebook login, took note of the cookies required, and selected only those cookies to be transmitted. Facebook has some pretty slick security measures as well, including a “challenge” code that is generated on the login form and checked along with your authentication information, and a “post_form_id” value that appears within every link and form post on the front page of the site, presumably to further validate the session integrity. There are a couple of neat .match() lines to capture those.

Step two was to access Twitter. Twitter does have an API, which is accessed through REST (for Twitter’s interface, that’s just a fancy shmancy term for “post a form”). Here is the Ruby function for updating Twitter:

Then there is bootstrap code that calls those two functions. Hooray. So what about the iPhone, then?

Well, thanks to the iPhone’s nearly complete and rather robust support for e-mail, I decided to create a way for my home mail server to receive a message and use it to update the status on those sites. From there, the possibilities would be endless. All I needed was a little procmail recipe to check that the message was addressed to my status-update address and a little script to process the full e-mail message and run the update. Here is the procmail destination file that bootstraps the updates:

I just set procmail to deliver the message to that script if it matched my updater address (by using :0, no flags). That means I never receive e-mail to that address in my actual mail client, which is good. The script also returns an e-mail to my phone to let me know it succeeded because it would take me too long to figure out if it did or not through mobile Safari.

One of the cool things that this script also does is to capitalize the first letter of the update for the Twitter version and lowercase the first letter of the Facebook version. This is done because Facebook builds a sentence like “Aaron is “, while Twitter simply posts the sentence it is entirety. Being a slut for grammar, I require that capitalization and punctuation flow like a river. One also must keep in mind the point-of-view and temporal sense in which the update is written.

I have a plan to make the system handle Facebook updates even more cleverly by replacing “my” with “his” and things like that, though I might be biting off more than I can chew with that one.

Cool, huh?

Bash is my favorite shell by far. I have known an occasional csh or tcsh aficionado, and I’m sure that those shells have their own virtues, but for me, bash is the end-all be-all of command line interfaces. Come with me now as I wax nerdy all over bash’s pattern matching capabilities.

At work, I often have to make changes to a number of files that all have the same name, but that are stored in a variety of different folders. The challenge is quickly uploading all of these changed files to the server. Because they are in different folders, I would normally have to drag them, one at a time, into the FTP client.

My solution was to write a bash script that would pull these files together into a “staging” folder on my local drive, mirroring the directory structure they live in so that I can drag the entire tree into my FTP client and let it do its work. I know ahead of time what the directory names will be; they are numbers representing the sites. Here is what a typical folder structure might look like:

What I want to be able to do is simply indicate that I want all of the file.txt files to be staged onto my hard drive and have bash go and find the 660, 671, etc. folders based on their numbers and copy them. This is probably more background information than you need to appreciate the bash tricks I’ve used, but it helps to know what on Earth the script is supposed to do.

The concept of the script is to take a list of files (with relative paths) and for each one try to find one of the preemptively known site numbers in our list. Once we know that number, we can loop through all of the numbers, replacing the original number with each new one and using those generated paths to copy the files to my local drive. That way, we end up with a copy of each file that lives in a structure containing each of the numbers in our list.

For me, the challenge was knowing how to evaluate the matching and stem the paths directly within bash. I wanted to avoid writing this in Ruby and I also wanted to avoid doing a lot of shell execution to run things like tr or awk. Here is the code (implemented as a bash function):

The first little bash gem that I discovered is the ability to use the [[ ... ]] syntax to perform a string matching test. When you use the == operator, the right side is used as a pattern to match against the left side using bash “glob” syntax. In this example, I am just checking each site number to see if it occurs within my original path. If and when I find one of them, I save it in FILESITE.

The next step is to loop through all of the site numbers. With each one, I replace all occurrences of the originally found number with the current one, use that path to go find a file and then copy it to my local drive using cp’s parents switch, which causes cp to create directories as necessary to duplicate the structure.

The trick to parents is that the destination must be a directory, so I also use bash replacement to trim off the filename from the path given (as well as the trailing forward slash).

The replacement trick I used is ${file//$FILESITE/$site}, which will take the contents of $file and replace all occurrences of $FILESITE with the value of $site. The syntax is reminiscent of good old fashioned s//, and the first double slash simply means “replace all.” A single slash there would only replace the first occurrence (from left to right).

Then, to remove the file from the end of the path, I used ${NEWFILE%/*}, which takes the contents of the variable $NEWFILE and removes the part that matches /*. There are two forms of this replacement operation: the first uses a percent sign (as in my example) and searches from the end of the string, whereas the second uses a number (or pound or hash) sign and searches from the beginning. Using a single symbol is non-greedy and using two of them is greedy. So, to summarize, %/* searches from the end of the string and non-greedily removes characters. The pattern starts with a forward slash, so it stops matching when it comes to the last forward slash in the string.

I can demonstrate some of these techniques using a little bash interactive example:

I think this replacement functionality is one of the least-known and most commonly useful tricks in anyone’s bash toolbox and I’m surprised at how little coverage it receives in the “bash scripting primers” out there. Most useful is the ability to make sure that a given path does or doesn’t end with a trailing slash, which is instrumental in concatenating paths, especially when dealing with user-supplied arguments.

Let’s say you will receive a path fragment as an argument and you need to append a filename to the end of it, but you don’t know whether the fragment will end with a slash. Here’s how you might do it:

Of course, this function will not work correctly if the user-supplied path ends with a whole string of forward slashes, but it handles the two main use cases: one slash or no slashes. The replacement simply says “remove a slash from the end of the variable called 1,” which does nothing if the slash isn’t there. Then you can confidently add your own slash and the filename. Give it a try!

I try very hard not to be a buzzkill; my cynical personality makes it hard for me not to trash talk things that offend my sense of aesthetics, but if I did that whenever I felt the urge, I fear I’d never do anything else. As a web developer in 2007, as much as it pains me to say so, there is still an awful lot to feel queasy about.

You don’t see a lot of “under construction” graphics around these days, let alone animated ones, so at least we have that to be thankful for. We have certainly come a long way since:

www.geocities.com/city/alley/dumpster/MYHOMEP~1.HTM

Still, the Internet—as most other places as well, unfortunately—is riddled with tasteless, colorblind people. Fortunately for those of us who are total hardcore nerds, even other people’s lack of taste doesn’t have to affect our web browsing experience thanks to a little something called (or at least, what I call) per-site styles. Continue reading ‘Per-Site Styles in Firefox’