Tumble Roast

A 2004-era tumbleblog that somehow predates Tumblr and outlived it too.
15 years of commits. 15 years of mass casualties.
Every link is real. We couldn't make this up.

๐Ÿ’€ The 2021 Era: Security Is Optional ๐Ÿ’€

After 4 years of mass silence (2016-2020), someone finally shows up... to actively make things worse.

"TLS is a nice-to-have"

December 7, 2021

commit 83e9bfc

    Disable SSL verification for tumble links

    TLS is a nice-to-have.
ssl_opts => { verify_hostname => 0 }

The commit message that launched a thousand security audits. TLS is a nice-to-have. Just like seatbelts. Or parachutes. Or not shitting where you eat.

Bonus: The branch was named disable-verify-hostanme. They typo'd "hostname" in the branch name, and it's immortalized in the merge commit.

The User Agent Time Capsule

January 24, 2021

-    $agent->agent( 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.17)
                      Gecko/20110422 Ubuntu/8.04 (hardy) Firefox/3.6.17' );
+    $agent->agent( 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:84.0)
                      Gecko/20100101 Firefox/84.0' );

For 10 years (2011-2021), production traffic pretended to be Firefox 3.6 on Ubuntu 8.04 Hardy Heron — an OS released in 2008 and EOL'd in 2013.

The original was added to bypass "bots forbidden" errors. Nothing says "I'm a real human" like a user agent from an OS that predates the Obama administration.

๐Ÿบ The 2013 Era: Drunk Bot Rewrites ๐Ÿคฎ

The Stone 13 Incident

September 10, 2013 — 1:38 AM

commit 9198b74

    standings rewrite

The commit message doesn't tell the whole story. James White has stated in a chat session:

jameswhite   1:59 AM

"I rewrote our perl irc bot so it would support google chat, xmpp, irc and then I drank 8 pints of stone 13 anniversary ale and a shot jaegermeister"

Stone 13th Anniversary Ale is 9.5% ABV. Eight pints is roughly 10 standard drinks. Plus Jรคger. At 2 AM. After shipping a multi-protocol bot rewrite.

The Aftermath:

The commit is clean. The man was not. Lara's car was not.

โšฐ๏ธ The 2014 Era: Failed Rewrites & Language Wars โšฐ๏ธ

The 39-Minute Go Experiment

April 13, 2014

06:30  42ffced  "Render twitter links as actual twitter links"
                  (introduces Go utility, 45 lines)

06:49  671f63c  "Minor fixup to fall back to old behavior if the
                  twit-link utility fails in any way"

07:09  c187c59  "Remove golang dep - back to perl"
                  (rewrites it in 6 lines of Perl)

The commit message reads: "Since freyr is on squeeze, golang isn't readily available. Oh well."

39 minutes. That's how long the Go rewrite lasted before the team said "Oh well" and went back to Perl. The Go binary is still in scripts/twit-link.go, forever unused.

Remove Ruby Thing That Never Worked

February 5, 2014

commit f31411f

    Remove ruby thing that never worked

 32 files changed, 1485 deletions(-)

Three years of work (2011-2014). 32 files. CouchDB integration. A full test suite. All summarized with: "that never worked."

The Ruby app included models, specs, views, and a Rakefile. It had more infrastructure than some Y Combinator startups. Three years of someone's life. Gone. Reduced to atoms. Press F to pay respects.

Fix Scott's Typo

February 5, 2014

-sub selectrow_array { return( shift->['dbi'}->selectrow_array( @_ ) ); }
+sub selectrow_array { return( shift->{'dbi'}->selectrow_array( @_ ) ); }

Scott used ['dbi'} instead of {'dbi'}. Mismatched brackets in Perl — a square bracket opening with a curly brace closing.

The commit message: "Fix Scott's typo". Scott knows what he did.

The Twitter API v1 Fossil

Still in production code

my $tw_uri = "https://api.twitter.com/1/statuses/oembed.json?id=" . $id;

Twitter API v1 was deprecated in 2012 and fully killed in 2013. This code has been dead for over a decade but still runs on every page load, silently failing.

The next unless $tw_j; line catches the failure. No logging. No alerting. Just vibes.

The Unicorn Placeholder

Still in production code

my $string = 'unicorn';
return unless $string;

Someone set a variable to 'unicorn' and then immediately checks if it exists. It's never used for anything else. This is placeholder code that was never finished and has been running in production for over a decade.

The unicorn is eternal. The unicorn does nothing.

๐Ÿคฌ Production Code: Say What You Mean ๐Ÿคฌ

These are in the actual codebase. Serving actual users. Right now. Today.

The Greg Buchanan Clause

In production since 2010

<span>PPS - If your name is Greg Buchanan and you just
read the above postscript, you can suck my ass.</span>

This is in htdocs/buttons/button.cgi. A customer-facing page. For 15 years.

Greg, if you're out there: they haven't forgotten.

The Helpful Search Error

Still in production

Your search for '$search' did not return any results.
Perhaps the following tips can help aid you on your quest:
<ul>
    <li>Searches must be done using four or more characters.
    <li>MySQL fulltext-searching is the magic behind this.
        Stop blaming scott.
    <li>Try not to be such a fucking idiot.
</ul>

When your search returns no results, the app helpfully suggests you "Try not to be such a fucking idiot."

Customer service excellence. Somebody shipped this. Somebody reviewed this. Somebody said "yes, this is fine." Incredible.

The Daily Kitten Infrastructure

Critical production system

my $content = get( 'http://dailykitten.com/feed/' );

XML::Twig->new(
    twig_handlers => {
        item => sub {
            $map->{$_->field( 'link' )}->{$a} = $_->field( $a );
        }
    }
)->parse( $content );

Production infrastructure that depends on dailykitten.com staying online. This man scraped cat RSS feeds and called it a feature. Absolute legend.

The Disabled Script

#!/usr/bin/perl

# until you have your own flickr feed, let's just exit 1
exit 1;


my $content = get(
    'http://www.flickr.com/services/feeds/...'
);

Line 13: exit 1;. The entire script is disabled. But the 60+ lines of Flickr integration code below it remain, waiting patiently for a day that will never come.

The SQL Injection Special

Still in production

my $url = $dbh->selectrow_array( qq{
    SELECT url FROM ircLink
    WHERE ircLinkID = '$id'
} );

Raw string interpolation in a SQL query. The $id comes directly from $ENV{'QUERY_STRING'}. This is a textbook SQL injection vulnerability that's been live for 15 years.

Bobby Tables would be proud.

"How do I install this crap??"

Customer-facing since 2010

<span class="tumble_item_quote_quote">So how do I install this crap??</span>

The sample text on a user-facing help page. Nothing says "professional software" like calling your own product "crap" in the UI.

"MSIE sucks. Stop using it."

Customer-facing since 2010

<span>PS - Unfortunately, tumblebuttons don't work with
Microsoft Internet Explorer. MSIE sucks. Stop using it.</span>

Public browser shaming on a customer-facing page. To be fair, it was IE. But still — in production.

๐Ÿฆ• 2010: The Origin Story ๐Ÿฆ•

Where it all began. Where it all went wrong.

Initial Git Insert

March 2, 2010 — The beginning

commit b53cd478f0fc373f466bd35ccc8df48c6a00c9cf
Author: James White <whitejs@websages.com>
Date:   Tue Mar 2 12:07:59 2010 -0600

    initial git insert for tumble.wcyd.org

"git insert." INSERT. This isn't SQL, James. Did you learn git that morning from a man at a bus stop?

The Swap File

Committed in 2010, removed in 2014

b0VIM 7.0
schnesa
fapestniegd
/var/www/sites/tumble.loserfish.org/htdocs/thtml/tumble_item_top5.thtml

James committed a vim swap file containing his server's hostname. He doxxed his dev box AND immortalized his vim session. This .swp haunted the repo for 4 years until someone finally noticed.

"...ugh"

May 20, 2010

commit f04b1b2

    hard-coding the new db server in the 5 places it's buried, ugh

The database connection string was hardcoded in 5 different files. The commit message's trailing "ugh" conveys the exact energy of maintaining this codebase.

The Binary Situation

March 2, 2010

+ htdocs/2202/2202.swf    | Bin 0 -> 1113251 bytes

A 1.1MB Flash file. Raw-dogged straight into git. Steve Jobs had already killed Flash by this point, but James said "not on my watch."

๐Ÿ† Commit Message Hall of Fame ๐Ÿ†

Messages that will echo through eternity. Click any hash to witness the horror yourself.

816923c - "in the middele of breaking shit"

Note the typo in "middele" for extra authenticity. Committed on a Sunday.

deaece6 - "nonworking commit to work around"

The commit that admits its own defeat in the message.

951b050 - "fixed the libraries again..."

That trailing ellipsis conveys volumes of resignation.

078ea9a - "pictures work!"

The exclamation point of someone who wasn't sure it would.

4ab33c9 - "changing the useragent because of 'bots forbidden' errors"

The beginning of the decade-long Ubuntu Hardy masquerade.

๐ŸŽญ The Cast of Characters ๐ŸŽญ

The architects of this disaster. The legends. The warnings.

Michael Stahnke - 57 commits

The hero who tried to clean up everyone else's mess. Deleted the Ruby thing, tried Go for 39 minutes, fixed Scott's typo. Carried the team in 2014.

James White - 13 commits

The origin. The "git insert." The man who committed a vim swap file and a 1.1MB Flash binary. Patient zero.

Aziz Shamim - 12 commits

Architect of the Ruby rewrite that never worked. Author of "in the middele of breaking shit" and "nonworking commit to work around."

Stephen Yeargin - 6 commits

Emerged from 4 years of silence to declare "TLS is a nice-to-have." Typo'd "hostname" as "hostanme" in a branch name.

Scott Schneider - 4 commits

Made the typo. You know the one. ['dbi'}. Never forget.

๐Ÿ“… The Timeline of Neglect ๐Ÿ“…

A visual representation of giving up.

2010 โ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ Origin story, Flash files, swap files
2011 โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ Ruby rewrite attempt begins
2012 โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ (2 commits)
2013 โ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ James rewrites bot after 8 Stone 13s + Jรคger
2014 โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ Michael's heroic cleanup, Go experiment, Ruby deletion
2015 โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ (2 commits)
2016 โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ nothing
2017 โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ nothing
2018 โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ nothing
2019 โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ nothing
2020 โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ nothing
2021 โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ Stephen disables SSL verification
2022 โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ nothing
2023 โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ nothing
2024 โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ nothing
2025 โ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ This roast

Pattern: years of silence, then someone shows up to make security worse.

"I've seen things you people wouldn't believe. Swap files committed on repos off the shoulder of Orion. I watched kitten feeds glitter in the dark near the Tannhäuser Gate. All those moments will be lost in time, like tears in rain... Time to git push."