Saturday, November 01, 2014

Macho-ism in Computer Science

It's common for me to see blog posts by companies talking about the high traffic volume in terms of QPS/RPS they handle and the amount of data they process, and that's super cool. But then there's another class of facts I see floating around a lot, and they tend to talk about the size of their hadoop or serving cluster, and things like "dozens or hundreds" of machines in your fleet seem to be something to be proud of. I don't understand this way of thinking or at least don't see the point of it. As I see things, it's nicer if you can get more done with fewer machines, and not more machines in your fleet.


Vishnu S Iyengar said...

I always read that as a matter of pride that we are able to maintain so MANY machines (and implied is that we maintain so many because we need to). I dont think its meant to imply that we chose more machines rather than fewer (unless that trade off was for developer time or head room or something)

Dhruv Matani said...

It used to be the case that people would add machines because they needed them to handle the workload, but now it seems fashionable to have more machines than you need. It's just a disturbing trend I'm seeing. There isn't enough emphasis on utilizing existing hardware in the most efficient was possible as there is in being able to manage a large fleet of machines. People aren't afraid to add more machines, and cost doesn't seem to bother them too much. Plus, AWS isn't helping fix that mentality either.