I signed an NDA with my company, so we're going to have to use an analogy for this situation. We'll make this fun. Here's the deal. You may have heard of the
Space Marines. Space Marines are the most baddass of all fighting forces both real and imaginary. In fact they are more baddass than Chuck Norris and all of the United States special forces
combined. Let me describe a typical day of a space marine: They get up. They get into a drop pod and fall from outer space all the way down to the planet surface, "landing" with a big explosion in the middle of a battle. Then they burst out of the new destroyed drop pod, guns ("bolters") blazing against an enemy with superior numbers. Then they each lunch.
Here is an image of a typical space marine, courtesy of some guy named
Neoshin:
Anyway, wow. This has like...nothing to do with work.
Ok. So.
I work in the...ok, lets say that space marine commanders use an online web interface to order these drop pods of marines down onto the battlefield. Does that sound ridiculous to you? Do you think I'm making a joke? If so, you haven't worked in the defense industry. But I digress.
Definitions
So I work in the place where we satisfy these requests. We'll call them "orders." A commander "orders" a hundred space marines. Lets assume that a space marine drop pod carries a maximum of ten marines, because I'm not looking that up. A hundred marines thus requires ten drop pods. We'll call them shipments. Every order of marines has one or more shipments of drop pods that they land in.
The system that I technically no longer work on, keeps track of the state of these orders and shipments (technically, thats not true, but we're pretending so to keep this explanation simple). For example, the marines have to like, put their helmets on, and then they have to walk down to the...to the...drop pod deck (?) and then they have to get in, and put their seatbelt on, maybe a quick trip to the restroom, and then we lock the drop pod doors, and send them out. There are a lot of steps there.
Mystery
Man, this is going to be so anticlimactic when I can finally explain what I fucked up. Ok, so, anyway, we track this shit using something called a "workflow." We have a workflow for sending out the shipment (drop pod), and, if a commander changes his mind, we have a workflow for canceling the shipment (drop pod). You know, canceling a shipment means opening the door, and getting them all out, maybe hosing down the inside because it problem smells, etc.
Obviously, if a shipment sending workflow is running (in the middle of loading up a drop pod) and we want to run the shipment canceling workflow, we have to stop the shipment sending workflow. I.E. in order to get all the space marines back out of the drop pod, you first have to stop sending them in.
Ok. So. Also, we fulfill a lot of drop pod orders. So many, in fact, that we need to use multiple databases to store the workflows in. This makes no sense in the space marine universe because a space marine chapter never has more than 1000 members* but just go with it. Also, space marine orders and these drop pod shipments both have these numbers, which I will call MarineIDs and PodIDs. These numbers are used to easily identify the total group of marines, and each drop pod they go out in, respectively.
So, for reasons that you don't need to know about, I...
improved...a script that sends cancel requests. Before, you needed to know the MarineID in order to cancel a drop pod shipment using this script. I improved the script by making it look up the damn id itself. Thats the kind of thing these stupid computers are supposed to be doing for us. I also did one other itty bitty change: I modified the script so that it could read in the ...PodIDs from
STDIN. For those of you programmers that don't know what STDIN is....please fall in a well and die.
So, that sounds awesome right? I thought so to, and I named my new script "start_ship_cancel_awesome" in honor of my awesomeness. And here's the key: it worked for me.
Then my buddy tried to use it...on...a different continent. Yeah, I know. Marines were fighting Orks all over the planet, or whatever. I am sick of this analogy. Anyway, so my buddy here uses my awesome script for a small number of shipments. The drop pod shipment workflow got canceled, and the record of the drop pod was deleted. Thats normal. Except...there was no record of the cancel workflow running. It just...it ran, but it was like invisible. This is highly unusual because we designed our system to create a record of the workflow before it runs the workflow. In other words, you can't run a workflow without having a record of it. I started thinking maybe I should rename my script to "stealth ship cancel." A nearly identical script--the version without my improvements--did leave a record of the workflow.
Hours, and a number of database sessions later, we discovered the culprit: The script was written in Ruby. Ruby is the scripting language here that everyone loves, because its like great or something. If you want to know why, I can introduce you to some geeks I met at a party this one time. Anyway, when you read from STDIN in Ruby, Ruby does not strip off the newline character at the end of the line, like every other fucking language on the planet does. Normally, when you say "give me the next
line" to a language, the fact that the line ended is kind of implied. You don't also need a newline character to remind you that the end of the line you're looking at is the end of the line you're looking at. Its not like you're going to say "give me the next line" and suddenly get three newline characters. It doesnt happen!
Thats like...thats like going to the grocery store, and putting your groceries on the belt, and then someone comes in behind you and puts down that little plastic divider, and then puts their groceries on the belt. And then the stoned out cashier rings you up and bags your food and
also bags the little plastic divider.
Long story short, that extra divider, which we here like to call "\n" got included on the id field. For example, "pod123" suddenly became "pod123\n." This changed something magical called a "hash value," which in turn made it get stuck in the wrong database partition, so that no one could find it. The workflow still ran, though; the extraneous new line apparently didn't bother anyone else.
So the missing workflow was found. Case Closed.
Whats the moral of the story? Chuck Norris is a wimp.
*I've just realized how pathetic it is that I know so much about a game I've never played.
!! how are u?
are u alone too?
