YuviSense: Codin Kid

The blog is now at a new host.  Some things may be a bit flaky - Yuvi is sleeping, and I’ll let things go as they are until he wakes up!

Digg this story here: http://digg.com/tech_news/Engadget_Statistics_an_Analysis_by_Yuvi. Thanks to Rob for submitting!

Engadget is like the place where you go to when you’re bored to drool over gadgets which you might/could never buy any day. It’s on most people’s reading list, but almost never gets read(if most people are like me, that is). So, when I was sitting horrified over the spaghetti that produced the data for the analysis of Robert Scoble’s Blog(and linkblog) along with that of several others (Raymond Chen, Matt Cutts, Kamla Bhatt and Rob La Gesse), I needed something big to experiment with. Really big. Big as in, like, the most linked to blog on earth?

So, here is the first part of the analysis of Engadget. Mostly stuff about when they post, how they are growing, how the comments are growing, when they post, which categories are getting the most comments, etc,.  Next part would be about whom they are linking to and other outbound linking related things. Want just the keynotes? Head to the bottom of the post. Any specific requests? Head over to the bottom of the post!

Sheer Size

Engadget is huge. At least, my dataset is huge (So huge, that my bandwidth constraints (a mere 2.5 gigabytes a month) and constantly dropping line would have made this analysis impossible to run on my computer. It is only because Rob La Gesse (aka the Psychic Mind Reader) helped me with a generous (really generous) amount of bandwidth that you are able to read this. Thanks Rob!). Starting from 2^nd March 2004 and running more than 3 years (1,169 days, if you want to be exact) till 16^th May 2007, the dataset contained 27, 953 posts! That is around 24 posts a day, a post an hour, which, last time I checked, is about half the output of Robert Scoble’s Link Blog.

If it were on paper

What if that much material were to be on printed paper? Trying to produce an HTML file with just the Title, Author Name and content produced a 26 megabyte file which Word was unable to open. So, it had to go into a Text File (which was still 26 megabytes), which Word took 5 minutes to open. Then I proceeded to format them as 12 point Cambria with 1.5 line spacing in Letter sized pages. That made 11,947 letter sized pages. Note that this estimate is quite conservative, since on a real newspaper, you would have more whitespace, more formatting, bigger titles etc.

You saved quite a bit of trees there people.

Posting Frequency

Are they posting more? Less?

As you can see, it has steadily grown since launch, and dipped a bit in September 2006, probably due to their relaunch. Since then, it’s grown, and keeps growing.

And here is a more interesting graph, showing the absolute number of posts per day.

Today, most days have 10-40 posts, up from 0-25 in May 2004. And also, look at the tall buildings occurring every January: CES. In fact, January 7^th and 8^th 2007 had a whopping 256 posts, almost double that of CES 2006! So, Engadget at CES is getting bigger every year; let’s see what happens at CES 2008.

Let’s look at it month-by-month:

The January spike is for CES, the March spike is for CeBIT. Vacations around July and December. February is not that hot either.

And when do they post the most? Is Friday slow news day?

Looking at it relatively, Friday does have lesser posts than the other weekdays. Thursday and Wednesday are the busiest days, with the weekends being understandably slow…

And, at which time of the day are they really active?

Serious blogging starts at 7 AM, heats up at 8 AM, and goes on till 11 AM after which they probably take a lunch break From there, it’s a downward slope, with number of posts decreasing as it gets darker. Also, different Time Zones(did I capitalize that right?) might play a hand here. Oh, and my best guess about that burst of posts at midnight is that they’re automated posts and lifted press embargos…  

And just because I can, here is a chart showing you at which minute of the hour each post was posted.

Pointless as it might seem, look at the long bar at 00 minute and the long bars at 30, 15, and 45 minute marks. Automated posts:D

Post Size

4,485,413 words were used in those 1,169 days and 27,953 posts, at an average of 3,836 words a day, and 160 words per post. That makes the average Engadget post about the size of a small essay we write at school, but I digress…

If posting frequency is growing, how about the length of the posts?

That’s growing as well. The average number of words per post is now 160, up from just a 100 in May 2004. More words, more posts, more people, more news.

And, how is each day going?

Sunday posts are the longer than posts on the other days. So, while Sunday may have a lot less posts than the other days, those posts are longer. The same is not the case for Saturday, which doesn’t differ from the other days much with respect to the length of the post. My best guess is that the Sunday posts are longer editorial type ones and less frequent, while the Saturday posts were normal posts, but less frequent.

Forget it. I was just trying to tell you that they do less work on Saturdays than on Sundays. That was it.

Comments

Engadget has a total of 615,212 comments, at an average of 526 comments a day and 22 comments per post. That’s huge! Let’s look at how it grew:

It started off rather very slowly, but went very fast once it picked up. Look at that big dent in September ’06! Let’s start from its humble beginnings though: here is a close up view of March-04 to July-04:

As you can see, the comment count is extremely low until the start of June, and then picked up. In fact, for their first three months, they had gotten exactly 25 comments. So, anybody out to build the next Engadget, don’t despair!

 Here is a zoomed out version:

As you can see, it gently sloped up from July 04, and suddenly jumped up in September 06. In fact, it grew by around 100,000 comments in a single month! How? The Engadget Relaunch giveaways! They were giving away lots of prizes, and to enter, you had to leave a comment. The response was quite tremendous, resulting in 1 post(about a Zune, none the less!) with more than 15,000 comments, two more with more than 10,000 comments, 6 posts with more than 5,000 comments, and 21 posts with more than 1,000 comments. Also, the only posts which had more than 1,000 comments and were NOT about the giveaways were the Apple iPhone announcement, the announcement of the Nintendo Revolution (heck, without even the controller!) and a flame war between PSP & DS, and the PS3 launch. And, the number of comments for the Nintendo announcement (~2,000) was almost double the number of comments for the Apple iPhone(~1,000). The Apple iPhone barely made the cut for the greater-than-thousand-comments club, with only 1064 comments.

Oh, and btw, did anybody notice that the largest number of comments was to win a Zune, which was incidentally also voted the worst gadget of the year by the same readers? I wrote something here, but my editor cut it out.

And here is our Comments by Day graph:

Notice those really really tall buildings around September 2006? Those are the giveaways. Compare that to the tiny pike seen in Jan 07 for CES and Apple iPhone combined, and the tinier one for CeBIT 07.

The takeaway being, if you want to really boost traffic and community engagement, then just give away a Zune!

Hot Comments

And here is the top ten posts with the most comments which are NOT giveaways of any sort:

So, out of the ten, 2 are related to the Apple iPhone, 2 are related to the PS3, two to the PSP, 2 to the XBox, and 2 about the Nintendo Revolution (Wii). The second one (PSP vs DS) is a classic flame war! And am I the only one to notice that games are all the rage here?

Categories

Here’s a chart of the categories containing more than thousand posts:

Cellphones come out first, with around 5,000 posts, followed by Portable Audio with around 4,400 posts. Gaming takes a distant 5^th place, with about half the number of posts in the Cellphones category. But, hey, wait a second! Didn’t I just say that gaming was all the rage just a few minutes ago? If so, why aren’t there more gaming posts? Well, here is another chart, showing the number of comments per post in the categories:

So, featured Engadget stories get the most number of comments, and next are gaming related stories. In fact, Portable Audio, which has about two times the number of posts Gaming has, gets only a 6^th place, receiving on average half the amount of comments that Gaming has. Heck, Cellphones, which have the most number of posts, have an average of only 15 comments per post.

What this basically means is that while Engadget (and the industry in general) are churning out more posts (and products) about Cellphones and Portable Audio, Engadget readers are more interested in Gaming and informative Feature posts. I think the huge bars for Features and Gaming speak for themselves.

Takeaways

So, here are the major takeaways from today’s post, besides the raw numbers:

• Engadget is very big. Damn big, infact. I use too many infacts.
• They got more about 100,000 comments in a single month by giving away, among other things, a Zune.
• Cellphones and Portable Audio devices are the most posted about (and maybe most produced (and purchased)) things around
• But, Engadget readers prefer to comment more on articles about gaming and featured editorials written by experienced industry veterans.
•  Of the top ten most commented articles (excluding the giveaways), 8 are about gaming. The remaining two are about the iPhone.
• For the first three months of their existence, they got exactly 25 comments!
• They post more about transportation than about HDTV;)

What’s next?

This is Part 1. Part 2 would contain link analysis (who they link to, when, etc), and then there would be a Part 3 that would have some “fun stuff” in it, like, say, how many times the words “Microsoft”, “Vista”, “Windows”, “XBox”, and “Zune” appeared against “Apple”, “iPod”, “Mac” and “iPhone”. Maybe, we could also start internal political wars, like, uhm, say, who posts more with the word Wii on it? Peter Rojas or Ryan Block? Stuff like that. Things might change depending on what happens after I post this though(School is starting next week, and this year, being the 12^th(and last) is going to be hectic).

Have a question?

Have a question you’d like answered? Cool! A simple question, based on complexity, would cost you between $2 and $5, and a chart, based on complexity, would cost you between $5-$10 per two series.

Some examples here:

“How many times does the word televascetomy appear?” – This would cost you $2

“How many times does the word televascetomy appear in posts by Ryan Block?” – This would cost you $3

“How many times does the word televascetomy appear in the same post with the word apple in posts by Peter Rojas?” – This would cost you $4.

Hope you get the idea. I fix the price, but it’ll be in the range above.

And for charts, here’re some samples:

“How many comments did Ryan Block get per Post he has written, compared to Peter Rojas?” – This would cost you $5, and would still cost you only $5 even if you hadn’t asked to include Peter Rojas in the chart. That’s because the cost is $5-$10 per two series, and in this case, Peter Rojas is one series while Ryan Block is another.

Payment via paypal(yuvipanda@gmail.com)

Any clarifications? Leave a comment, email me at yuvipanda@gmail.com or IM me at yuvipanda@msn.com!

But why are you doing this?

Why am I doing this? My current hosting(which I choose primarily because it is dirt cheap(I paid around $75 for two years!)) sucks. It’s extremely slow, gives me only 500 megs of storage(which I am about to overrun soon) and offers me only PHP4 without much customization or advanced options available (I can’t run Python or Perl or RoR even if I want to). So, the money raised via this would go towards getting me better hosting. And, you would have the pickup line of, “Hey, didja know that Engadget spoke more about Nokia than about LG?”

Oh, and I might turn down an offer, but I’ll offer an explanation why. Oh, and if you need anything more custom, email me(yuvipanda@gmail.com), and we’ll work something out.

Though changing themes is trivial in Wordpress, it isn’t trivial once you factor in Yuvi’s pseudo-law: “Anything within reach of Yuvi inevitable suffers a screw up”. Recent screwups include using the word inevitable instead of the correcter inevitable, and then using the non-existant word “correcter”, breaking the comments when upgrading to Wordpress 2.2, and then screwing up a lot of things while trying to activate the newer Freshy 2.0 theme.

The new theme is up now, though I had to a use non-widgetized sidebar. I have no Idea who took the photo used in the header (It came with the theme), but it is a fantastic shot, with a razor thin Depth of Field. If you look carefully, the left portion of the fruit is in focus, while the right part isn’t. Wonderful.

Also, I changed the name of my blog from Yuvisense: Kid in Tech to Yuvisense: Codin Kid. I have a blog name changing fetish, as observed by the fact that the name of this poor blog has changed some 6 or 7 times since it started.

Oh, and Part I of StatBot: Analysis of Engadget is almost complete. Expect to see it soon!

Any bugs? Contact me!

a Little bIt of heAven.., by mR.bLind43>.

Zooomr Mark III has been running into walls again and again. They’ve been down for almost a week now. But, I’ll be with them. When Thomas Hawk can take a second mortage on his house so that I can share/upload photos for free, what more can I do but wait with hope?

Tried to update to Freshy 2.0 theme. Went awry (as usualy). I can never seem to upgrade anything without breaking something. This time, I was unable to use any widgets: Accessing the widgets page and hitting “Save” gives me just a 404. Shitty. I’ll try again tomorrow.

Just thought you guys might like to know.

You get spam. I too get spam. Most of us just ignore it. A small minority think it’s true and end up losing money. And then there’s the even smaller minority that goes up against those scammers. Some do it by building better spam filters. Some do it by building anti-phishing technology into the default browser of millions of people.

And, some do it by frustrating the heck out of those scammers. Even sending them in safaris miles across Africa. Wasting their time so someone innocent isn’t scammed. These are the scam-baiters. Read more about them here. Read why this is not non-ethical here.

And,yes, I am a scam baiter too. I am not as active now as I used to be, but I’ve gotten myself a death-threat as well as a passport to show off

I upgraded to Wordpress 2.2 a few days ago and thought I didn’t break anything. Rob, however, pointed out that I broke something(as usual): I broke my comments. Thankfully, without having to get me to tear out a lot of hair so as to why the heck on earth I was getting duplicate primary key fields in the database, Rob also pointed me tothis articlee to fix it. Works like a charm.

So, if you’ve tried to leave a comment over the past few days, I’m sorry for the abusive page containing obscure words that you encountered. It should be fixed now.

So, well, uhm, Thanks Rob!

Alfred Thompson says(with a kickass story) we should comment our code. I know. We all know. I also know I haven’t been doing it. Sigh. I remember the number of times I had to go back to Scoble’s StatBot code to have a look at a bug that I solved in a weird way that propped up again in Engadget’s StatBot code. A comment here or there would have helped me a lot. Hopefully, I’ll have more comments in Engadget’s StatBot.
This is why I am still calling myself a wannabe geek…

Found this while chatting with Rob. How exactly do you save 150% off a book?  They give you half the price of the book?

Funny:

Still waiting for Mark III, though I think it’s worth the wait. It probably will be up by the time I get up tomorrow morning, because Kristopher’s gotta demo it today evening Kinda weird when you write today evening in the future tense at 8 pm, but that’s the Internet for ya!

I Think Flickr Just Sent us some Pizzas!, by Kristopher Tate.

Cool! Kristopher was working for 3 days straight without any sleep, and he got this from flickr. Thomas Hawk ate most of it though (Atleast, I saw him eat some of it, and Kris was sleeping at that time, so I ignorantly assume that Thomas ate most of it. Like I said, all people, including me, are retarted)

Ars Technica has the scantily available details here. I haven’t hit by a virus in all the years I’ve been using Windows, and I don’t think I will. Remember, common sense is the best antivirus ever.

Also, remember, all people are Retarded, in some way or the other.

I know most of you have seen this cartoon too many times, but to those of you who haven’t had the enlightening experience of reading XKCD, I’d highly reccomend you put them in your feed reader.

P.S. This applies only if you understood what the title meant after you had seen the comic. Otherwise, stay away from it, for it might cause damage to certain parts of your cerebral neocortex.

Upgraded to Wordpress 2.2 a few days ago. Made a backup of the database just in case something went wrong, and just replaced the files. Everything works fine. Wordpress Rocks! (Oh, and if you happen to hit any problems, do mail me)

Nothing much radically new for me, since I don’t actually go to the web admin interface much(Windows Live Writer rocks as well!). Oh, and btw, they’re teasing us with a lot of yummy screenshots, but holding back for some reason they won’t disclose

BTW, any ideas why Google categorizes most Windows Live Spaces Blogs as an Unrecognized File Format?