I was pretty amazed two days ago when I was prompted to visit Yahoo! Pipes only to find my name on the main page because my first pipe, "Search Google & Yahoo Simultaneously" had been run 19,000+ times; making the top 5 most run pipes. It's really a pretty simple pipe, so it'd be foolish of me to take any credit as it should really go to Ben Hammersley for his "Google to RSS" service that the pipe employs. (Ben; if you're reading this... I'd like to apologize for the heavy use on your Google API key. I didn't notice the option to provide your own, nor did I ever expect this pipe to be used so much)
Inspired by the "success" of my front page pipe, I decided to mess around on Yahoo! Pipes a bit more to see what else I could come up with. By the end of yesterday evening, I had only the idea to implement recursion with pipes. Was it a good idea? That’s questionable. Was the implementation good? No. Actually, I didn’t get much further than an incomplete Pipe named, “Recurse.” The recursion experiment was cut short when I gave the pipe a test spin prior to adding the base case logic; thus resulting in an infinite loop. No big deal right? The pipe execution will just timeout, and I’ll continue editing the pipe. Well… the pipe did timeout in a sense. It just stopped, so I clicked on “Edit this Pipe” to continue work. This is when I realized that it wasn’t just my “Recurse” pipe that timed out, but all of Yahoo! Pipes had taken a break. This happened around 9:20PM CST, on Monday, March 7, 2007. Pipes was down for the rest of my evening. I was a little concerned that “Recurse” had been the cause.
The next day, I noticed the Yahoo! Pipes Blog had posted a couple entries about unplanned downtime and a kind reminder that Pipes is in Beta. I don’t know what I had expected, but there was nothing about the outage being caused by a user created pipe; less the user that caused it.
Last night I logged on to Pipes, created a Google Search pipe (sorry Ben, I really didn’t notice your API key option until writing this), played around with the idea of employing Google Translate to translate feeds (a bust), and then finally… had another look at “Recurse.” I ran it. It worked. Yahoo! Pipes was still up. It was obvious that “Recurse” hadn’t caused the outage the night before. If, by chance, it had – it would have been a quick fix (or so says the guy not working on the project); put an limit on the number of times a Pipe can invoke itself within the context of the original execution. The easiest way to do that (again, from the guy not working on the project) would be to catch it at the “pipe definition level.” What about catching the problem at the HTTP request level by checking the referrer? Curiosity got the best of me, and I wanted to know. “Recurse” was modified to do the same as before, but instead of invoking itself, it would invoke a new pipe, “InvokeRecurse” which took the same input as “Recurse,” but invoked “Recurse” using the Yahoo! Pipes, “Fetch” pipe that takes a URL – In this case the “Recurse” URL with its query parameter. I ran the revised pipe around 9:30PM CST on Tuesday, March 7, 2007 and the exact same thing happened. The “Recurse” pipe just timed out. I clicked edit and that timed out too. Yahoo! Pipes was down again, but only and hour or so this time. Either Pipes is experiencing problems with some 9:30PM CST scheduled program (orsomethinglikethat) or it’s not prepared to handle infinite loops. You decide.
I do want to thank the Yahoo! Pipes team for a fun and innovative product. They’ve done an excellent job and deserve much credit. I’ll try to quit breaking my toys. I’m sure the best is yet to come as it’s only in Beta.
Comments(3) 
