You can kill the background for speed, if you wish.[x]

Friday, December 18, 2009

What I learned from the xkcd effect

It is a well-known phenomenon by now that whenever Randall Munroe mentions anything obscure in xkcd, searches for it spike tremendously. To this point, as far as I'm aware, he hasn't wielded this fact for evil, but still...the power that Randall Munroe holds over the internet is terrifying. I was reminded of this when one of my friends referenced one of these comics on Twitter. I then, with a bit of Googling, was unable to find a good list of examples of the XKCD effect, and decided that I would endeavor to create one.

Of course, I decided to write a script to do this - there are 676 comics as of this writing, and fortunately, transcriptions are available via OhNoRobot, so I don't even have to deal with the images. After a little poking around, I found a post on the XKCD forums (thanks, philip!) that gave the url http://www.ohnorobot.com/transcribe.pl?comicid=apKHvCCc66NMg&url=http:%2F%2Fxkcd.com%2F[comic number]%2F to get the transcription for a given comic. Perfect! I of course lean towards Perl for these kinds of things, and I was tempted to go with Python because of the images. But then I remembered that a simple gnome-open [image file] would do more than I needed, so Perl it was.

Additionally, I remembered that I had already written an xkcd download script that I had stashed away in my gMail, so I had a leg up already. So, after a mere four hours of hacking, I present the xkcd effectalyzer. It's pretty self-explanatory, the only parameter is "-r", if you want to go through the comics in reverse. The script goes through your specified comics, presents you with the OhNoRobots text, and gives you the option to view the image of the comic. Once that's all done, it asks you for a phrase to search on Google Trends. It then (with the credentials you provided at the beginning) grabs the necessary CSVs from Google Trends to get the trend data for the 5 days around and including the publication date of the comic (which it gets from xkcd's archives page). It then tells you what the indexes are, and lets you decide whether or not to save that result to a the output CSV file. It continues to ask you for phrases until you don't give it one, and will write the first three to the CSV file.

It might do more than that, but like I said - it should be pretty self-explanatory. It's got all kinds of nifty features like saving your Google session, grabbing neighboring months if necessary, that are mostly what took so long. But anyway, I went through backwards from the current comic (676) back to 600, and besides revisiting some fantastic comics (including my favorite of the more recent ones), I found over 60 noticeable spikes in Google searches because of an episode of xkcd - that's in just 75 comics. Some comics spiked for multiple phrases, of course, and some none at all. But this also includes 20 terms that hit Google's Hot Trends page. I put them together in a graph that shows the spikes collected around the release of the comic, and there are a couple of interesting anomalies: Obviously, there are a couple ponts off to the right that need explaining. The two that are shifted one to the right are both from "Locke and Demosthenes", which was released on Friday, October 11 of this year. So why the discrepancy? Well, my script gathers the dates from the recommended source - the alt-text on the xkcd archive page. But for "Locke and Demosthenes", the alt-text is off by one day, and says "9-10-09". Since the previous comics were published 9-7-09 and 9-9-09, and Randall only publishes on Thursdays in the event of a five-day series - not to mention the Google results, I'm willing to bet that the archives page is in error, and it was actually published on Friday. The other anomaly on the right-hand side is just because the fifth day is in the next month, which screws up the relative numbers. It disappears in this chart based on the fixed data, which also, handily enough, highlights an anomaly on the right side: "github" spiked on the day that Munroe put out Branding, but was climbing in popularity before that. Why that is, I haven't the foggiest.

Now, pretty charts and things aside, it's also interesting, of course, to look at which terms spiked the most. So, here's a list of all the terms, sorted by the severity of their spikes:

As I looked through these, the thing I was most surprised by were some of the things that people Googled, presumably because they didn't know what it was about. I mean, some, like SMBC, Hofstad, Peter Wiggin, Demosthenes - I understand those. But classic stuff like "The only winning move is not to play", "the cake is a lie", and stuff like sampling bias, Q.E.D., Carl Sagan, 2038, or the debacle with the brontosaurus, demonstrates that xkcd readership does indeed include many that are not part of the normal geek crowd - such as liberal-arts majors. Also, Stephen Douglas? The Bull Moose party? What have history classes been teaching that people had to look those up?

But the takeaway from this, I think, is that Randall Munroe, as of late, anyway, has a better than 50% chance (41 times out of 76) of noticeably affecting the Google searches for whatever he happens to mention in his comic. It takes a whole lot of readers (which of course we know xkcd has) to do that with a single webcomic, and this illustrates quite clearly that Randall has them.

The other thing this demonstrates is that I have too much time on my hands, but I just finished finals, and it's Christmas break, so I don't want to hear about it.

Posted by Joel Bradshaw at 8:26 PM 0 comments

Wednesday, December 9, 2009

Metropolis for the win!

This isn't so much for anyone else's sake as for me to chronicle it while I remember it, but I was playing a game of SettlersXplorers over on AsoBrain Games, and was up 13-11-12 in a 15-point game. Come my turn, I had six wood, two brick, two cloth, and five paper (I already had the paper metropolis). I had the brick port, and level one coin and cloth. I could have just stolen the merchant, put it on wood, and built a settlement easily, but I figured out I could do much better...barely. So this is what happened:

Merchant to wood, for 14 points
Traded 6 wood and 2 brick for 4 cloth (6 cloth)
Upgraded from Market to Trading House and Trading House to Merchant Guild with 5 of the 6 cloth I now had. (1 cloth, level 3)
With my new ability to 2-for-1 any commodities, I traded 4 paper for 2 cloth. (3 cloth)
Using my Crane card that I happened to have, upgraded to Bank, grabbing metropolis for two points and finishing 16-11-11, much more dramatically than just building a settlement.

I think that's the first time I've gone from level 1 to 4 in one turn. So much fun.
Posted by Joel Bradshaw at 5:32 PM 0 comments

Monday, November 9, 2009

Getting Adobe Air to Recognize Your Default Browser

Among the issues I have re-encountered in reinstalling my Ubuntu fresh is that links from TweetDeck open up in Firefox, instead of Chrome, my default browser. Chrome on Linux, by the way, is still only available from the dev channel. It's readily available, but still has its share of issues - currently, my dropdown boxes are blank when I try to select an option. They get fixed - that particular isse has been taken care of already and will be fixed in the next release - but it's still iffy. That said, it is way zippier in Linux than Firefox, which is pretty much dog-slow at times, in my experience.

Anyway, for these reasons, I wanted TweetDeck to respect my default browser, but it stubbornly insisted on opening links in Firefox. Since I generally don't have it open, it takes a few seconds to open Firefox and load the page, instead of popping up Chrome with my page already displayed pretty much instantly. It didn't take long before it got on my nerves, and I remembered that I had fixed this before. A quick trip to the (one guess) Ubuntu forums pointed me to a very helpful post by Andrea Olivato, detailing how to fix this without resorting to really nasty hacks like symlinking Firefox to Chrome.

Turns out that Adobe, in all its wisdom, decided to hardcode Firefox into their binary for opening links. Brilliant. Which means that you have to get your hands dirty doing a little bit of hex editing and, yes, symlinking. Because for the hex editing, the string for the browser has to have the same number of characters as "Firefox", so you don't screw everything up. Andrea had the clever idea of symlinking "browser" to your desired browser, which is good semantically, and has the same number of characters as "Firefox". Perfect! His commenters further improved upon the fix by suggesting that "browser" be symlinked to either x-www-browser or xdg-open. Both are generic "default browser" options. After reading up a bit, I still don't really know which is "better". I think I like xdg-open though. So, here's how to Fix It Right, based on Andrea's instructions, with the commenters' suggestions integrated in:

Before anything, do a sudo su to get into a root prompt. Then, symlink the desired default-browser alias (xdg-open or x-www-browser) to "browser" in /usr/bin:

root@geekmobile# ln -s /usr/bin/xdg-open /usr/bin/browser
Then find your libCore.so file with a locate:
root@geekmobile# locate libCore.so
/opt/Adobe AIR/Versions/1.0/libCore.so
With your hex-capable editor of choice (Vim in my case, gEdit isn't going to do it this time), open the file found via the locate command:
root@geekmobile# vi /opt/Adobe\ AIR/Versions/1.0/libCore.so
Find where it says "openURL" in the file. In Vim this is a simple as a /openURL<enter>, but of course this depends on your editor. Then replace the "firefox" right after that point with the word "browser" (which you symlinked above to your default browser). Save the file, restart TweetDeck (or whatever Air app you're using), and enjoy quicker, better link-opening! Hurrah.

And again, many thanks to Andrea for the tips!

Posted by Joel Bradshaw at 10:18 AM 0 comments

Monday, November 2, 2009

Wordpress, Plugins, Paragraphs, and wpautop

Just a quick tip, because nothing I found in a quick Googling was helpful. I was writing a plugin for Wordpress recently that functioned as a filter on the_content, like many, many plugins do. As I was applying the CSS/HTML from a design mockup that I had sliced up from a PSD, I noticed that some of the spacing was funny for no apparent reason. Thanks to Firebug, I quickly discerned that Wordpress was randomly* sprinkling <p> and <br/> tags all over the place, in ways that very much broke my layout. I could fix it by smashing all my tags up onto one line, which seemed to dissuade Wordpress from throwing tags in, but that was ugly and made my code rather unreadable.

After a bit of research, I found out that this was the doing of the potentially useful, but unintentionally nefarious wpautop filter. After some more digging, I realized that if I could get my filter to happen after wpautop had worked its "magic," all of my problems were solved. Fortunately, the add_filter function provides just such an option. And upon investigation, the previous author (I'm revising an existing plugin) had set it to be priority 0 (aka before everything). I switched that up to priority 99, and the random paragraphs and breaks disappeared, and my layout fixed itself.

*Okay, there was some rhyme and reason, such as after images, or before divs, or something...but the point is, it was putting them places where I didn't tell it to, and didn't want them. End of story.

Posted by Joel Bradshaw at 5:48 PM 0 comments

Friday, October 23, 2009

xmodmap: fixing the mistakes of others

For the past few weeks, I've been using my girlfriend's old Toshiba Satellite while wrestling with Dell to get them to fix my Vostro (it's currently somewhere in Texas, I think). Which has been great - I just put my hard drive into her computer (fortunately they were both SATA) and off we go (well, Windows isn't too happy about it, but that's not really of consequence). However, one quirk has bugged me: it suffers from the random placement of "unimportant" keys that many laptop keyboards do. Unfortunately, I, being a programmer and power user, find such keys rather important. Here's the problem, in a picture:

As you can see, whoever designed the keyboard was so overly concerned with the gigantic media button panel that they had to displace the tilde/grave key. Granted, this is one of the lesser used keys on the keyboard, but I still use it for one of my favorite Vim shortcuts, `., which is a magic marker that hops to the last line you edited. But it gets worse. They decided to put it down next to the space key, and moved the Windows key out of the way to make room for it. This also wouldn't be a huge deal - no one uses the Windows key, right? Except that I also run Compiz and Gnome-Do, and have the Windows key mapped to my Gnome-Do summon, as well as most of my multiple-desktop management. Win+Arrows moves between desktops, Win+Tab switches between all desktops, Ctrl+Win+Arrows moves the active window between desktops, Alt+Win moves a window by grabbing angwhere. Obviously all of these become very difficult when the windows key is six miles northeast of anything else.

The solution to this problem, of course, is to remap my keys where I want them. And since this is Linux, if I try hard enough, I can probably do whatever I want, oftentimes without downloading anything. This is the case today. Optimally I would like to use the Fn key as the Win key, but it's a really weird key, and evidently not touchable by xmodmap. So I settled for this: make the tilde key (next to the spacebar) my new Win key. I could just switch the two, but that puts my grave six miles away from everything as well. Instead, I decided to replace my Caps Lock with the grave, because like most programmers, I never, ever, ever use my Caps Lock. This makes it decently closeby, and pretty close to where it was. And for those times when Caps Lock goes haywire, I might as well move the Caps Lock up to the now-unused Windows Key. So, to summarize:

After some trial and error with xmodmap, I came to the following conclusions:

  • add/remove is to add/remove modifiers from key assignments (Caps, Numlock, Windows key, etc)
  • keycode ## = functionone functiontwo is the best way to simply map one key to the other
  • If you remap keys, but don't want the modifiers to come with them, you have to remove the modifiers, then add them back
  • If you want shiftable keys like the grave/tilde key to work properly with shift, you have to assign both functions (e.g. "grave" and "tilde") to the same keycode
  • xev, xmodmap -pm, and xmodmap -pke are relentlessley helpful in finding out what the keycodes are and what the keywords for the various modifiers and key functions are.
With these newfound conclusions, I went to work. Using xev, I found out that my winkey is keycode 133, grave/tilde is 49, and Caps Lock is 66. Additionally, xmodmap -pm told me that the Win key modifier is mod4 and the Caps modifier is lock. Finally, xmodmap -pke (in combination with the previous command) fleshed out the function names for the keys: Win is Super_L, Caps is Caps_Lock, and grave/tilde is grave asciitilde. From there it was just a hop, a skip, and a jump to come up with this xmodmap config file:
remove lock = Caps_Lock
remove mod4 = Super_L

keycode 49 = Super_L
keycode 133 = Caps_Lock
keycode 66 = grave asciitilde

add lock = Caps_Lock
add mod4 = Super_L
Which simply removes the two modifiers from their mappings so we can move stuff around, reassigns the keycodes to the functions I want them to perform (notice the combination grave asciitilde to make Shift function correctly), and then reassign the modifiers now that the functions are where I want them to be.

The sites I found regarding xmodmap were moderately helpful, but none of them quite spelled out what was happening, so I figured I would. This method should be pretty flexible, and plenty sufficient for any of your key-remapping needs. If you're playing around with it, xmodmap -e "[command]" will allow you to run commands (e.g. any one of the lines above) on a trial basis, just for this session. It was invaluable in getting this stuff working correctly. If you're having trouble, give a shout out in the comments, and I'll see what I can figure out.

Posted by Joel Bradshaw at 2:41 AM 0 comments

Sunday, October 4, 2009

Dear NASA: Thank you.

The other day, I was walking down Queen Anne and noticed a certain office complex that had a sandwich board advertising their space. What I noticed about it was the picture that was emblazoned across the top of the board. Of note was that it, like so many other specimens of posterage across this country, was terribly pixellated. But this particular instance irked me more than most, simply because of the subject. It was Saturn:

Pretty standard sandwich board, right?

But check out the zoom of that top image: Phone number and website have been erased to protect the not-so-innocent. And the picture is taken with a crappy camphone, but that pixellation is in no way exaggerated in the photo.

Now I'll give you a second to figure out what's wrong with this picture. Really, you have as long as you want, because you have to move onto the next sentence, and I can't do anything to force you along. But now that you've wasted a good few seconds reading this explanation of that fact, I'll enlighten you if you haven't already figured it out. The problem is simply this: over a decade ago, our government, via one of its more useful and fruitful arms known affectionately (and officially) as NASA, spent two and a half billion dollars (with another half-billion from Europe) to send a probe through more than two billion miles of mostly empty space, in order to find out more about arguably the most beautiful planet in the solar system.

Nearly seven years later, it pulled up next to Saturn and began taking pictures, among other things. Lots and lots and lots of pictures. Thousands upon thousands, if not millions, of photographs. Staffers at NASA then stitched some of these pictures together to create truly gargantuan portraits of our celestial neighbor that in their full glory can only be described as indescribable, although truly stunning is a first stab at it. Then NASA, being a government agency, gave these portraits away to the public, free for use by anyone, including office buildings that want to spice up their sandwich boards. I even double-checked, and basically, as long as you're not implying NASA's endorsement of your product, you're free to use them as you please.

With images upwards of 25 megapixels readily available, there is no reason that a pixellated image of Saturn should appear anywhere, much less on a 3x2 foot printed sign. But then again, had it not appeared, I wouldn't have gone digging through NASA's archives, which has some pretty fantastic stuff to say the least. I found something as simple as a glimpse at Saturn's atmosphere fascinating. All in all, I came away with a renewed awe and gratitude for the incredible gift NASA has given, and continues to give, our country, and indeed, humanity at large. And hey, they have a sense of humor to boot.

Edit 12/18/09: Updated with pictures of the planetary carnage

Posted by Joel Bradshaw at 3:36 AM 0 comments

Wednesday, August 26, 2009

It's Jeopardy! Time!

For my stance on the strange punctuation of Jeopardy!, see the sidenote on a previous post. But this post is not about that. Yesterday, I took the online College Jeopardy! Test, the first step in being on Jeopardy! The aftermath was fun to watch on Twitter, and I contributed my fair share. And thanks to Twitter, I was able to find the answers and questions listed on the Jeopardy! forums, and compare what I remembered of my answers. And because I like stats so much, here's some stats:

24/50 correct (not too bad)
Longest right streak: 7 ("1812 Overture" to "Wifi")
Longest wrong streak: 4 ("Hemingway" to "Mamet")

Overall, it was lots of fun - reminds me of Knowledge Bowl, which I miss sorely. Stuff like "Equine, Feline...wolf is...lupus?...Lupine!" under a time constraint...I just love it. The feeling of figuring it out is just awesome, especially when there's something tangible you're going for. Dang I miss Knowledge Bowl.

Some of my favorites were ones that I knew for some random reason - I recently discovered that Yosemite was in California, so I knew that Yellowstone was the one in Wyoming. Crime and Punishment is one of my top 5 favorite books (thanks, Mrs. Ogden!). I was way into genetics in high school, so chromosomes were easy (thanks, Mr. Weeks!). I knew all about Athens and Sparta thanks to UScholars and my girlfriend. I watched Milk a couple months back, so I knew Sean Penn's second Academy Award. That movie, by the way, is highly recommended. You should watch it. And 30-Rock? I owe that entirely to stumbling across a blog post about the muppets several weeks ago. I remembered just enough to vaguely recognize the names.

Also fun were haphazardly educated guesses that paid of - Time magazine, Matron of Honor, Merril Lynch, the Rhine. And even that middle school knowledge that paid off - Eli Whitney, Deltoid, Eon. Man, I love Jeopardy. I need to watch it more often.

For anyone interested, by the way, the questions (and the answers) are over on Jeopardy.com, and I put a full list of my responses in a Google Spreadsheet.

Posted by Joel Bradshaw at 9:31 AM 0 comments
Subscribe to: Posts (Atom)