Archive for the ‘scale’ Tag

Atrophy by Abstraction

In 1957, Isaac Asimov wrote a short story entitled “A Feeling of Power” about a future society where humans had lost to atrophy the ability to perform even simple arithmetic due to a total dependence on computers.  Now I could log volumes on the abysmal state of math knowledge in the humans wandering around in today’s society, but this piece isn’t about that.  It is however along parallel lines for software engineering.

I’ve been doing a fair amount of recruiting for my Engineering team this year and I’m happy to say I’ve hired great people, but it wasn’t easy.  One of the things I like about the process is that I learn a lot.  One of the things I hate is what I sometimes learn, like how many software engineers who have been working in object-oriented technologies for many years who can’t give a lucid explanation of encapsulation or inheritance; and abandon all hopes of polymorphism – log that knowledge as an exception.  These are the core pillars of object orientation and if you can’t at least describe them (much less define them), you can’t use them correctly.

For the most part, I’m not talking about young engineers right out of school, although you’d think they would have forgotten less.  I’m talking about the 5-10 year senior engineer who stares as still as a fallen log hoping that the gods of seniority will suddenly inspire unto them the fundamentals.  And on the subject of “senior” in the title, calling a squirrel a duck doesn’t make it quack.  Admittedly Shakespeare put it more eloquently, “A rose by any other name would smell as sweet.

Another favorite question of mine as my colleagues well know is the binary search.  I often ask engineering candidates, especially server-side and database types, to describe it relative to any other search technique.  Half the time the answer starts by naming certain Java classes – nope pull up, this is not a Java question.  Overall, about 1 in 5 does pretty well.  For the rest, I usually resort to playing a game.

I’m thinking of a number from 1 to 100.  You need to guess it in the fewest number of tries and after each, I will tell you that you’re either correct, too low, or too high.

Almost everyone figures out that the logical first guess is 50.  It has no more of a chance of being right than any other guess, but at least you’re reliably cutting the space in half.  If I say “too high”, then guess 25.  If I then say “too low”, then guess 37, and so on.  That’s a binary search!  Start with a sorted collection and find what you need by successively dividing the search space by 2.

Only once was someone’s first answer not 50 – they guessed 70 and my head exploded scattering debris for miles and miles.

I ask this question because knowing how things work matters.  If you don’t understand the binary search, for example, then you have no idea how an index helps a SQL select or why database inserts and updates are costly when indices are overused.  You may never have to code a binary search ever again thanks to abstraction and reuse, but just because something has been abstracted away from daily view doesn’t mean it isn’t executing at runtime.  Understanding this principle is crucial to being able to play in the high-scale league.

Folding a little math back into my binary search question, I usually ask the following just for fun since only about 1 in 50 come close.  Given a collection of size N, what is the worst case number of tries before you’re sure to win?  More blank fallen log stares as they try to play out the guessing game in their heads, so I lead them down the path.  If N = 2x (i.e., multiplying by 2, x times), then what is the inverse function x = f(N) (i.e., how many times can N be divided by 2)?  What is the inverse of exponent?  But this only helps 1 or 2 more out of 50.

If the many occurrences of the word “log” so far in this post weren’t enough of a clue…

If N = 2x, then x = log2 N

Stealing a bit from Bill Maher and his HBO series, I think we need some New Rules:

  1. All programmers are not engineers.  Programmers write computer programs, maybe even really good ones.  But to be an engineer, one has to know how things work or at least possess the intellectual curiosity to want to know.
  2. Calendars and titles do not make engineers senior.  Few things raise my resume red flags higher than seeing every position since kindergarten as Senior this, Lead that, or Super-Duper something else.  Take the time to learn your craft.  That will distinguish you.
  3. Abstraction without fundamentals is unstable.  It can cause us to mistake tight source code for code that performs well, not thinking about the mass of code sitting in base classes, libraries, and frameworks.  We can write resource-sloppy code and assume the garbage collector will magically clean up behind us.  Try that at scale.

Summing up, abstraction is good.  It has marked the forward movement of software engineering for many decades.  It drives productivity and at least potentially drives better designs, testability, extensibility, maintainability, and lots of other good characteristics.  But it can also give us permission to be lazy.  With or without title qualifiers, good engineers do not let this happen.  They are self motivated to learn and they learn deep, not just broad.

Well, I’ve given away a few of my favorite interview questions here, but if my upcoming candidates suddenly know these answers, I can at least give them credit for reading the interviewer’s blog as preparation.

Performance != Scalability

Performance and scalability are often conflated, sometimes equated, but only somewhat related.  In web services platforms, both characteristics are business critical.  But while each exerts influence over the other, they are highly distinct and have profoundly different business impacts.

Performance & Scalability Revealed

Using a web services platform as context, performance and scalability can be defined as follows:

particlePerformance is a measure of speed from a caller’s perspective; the time it takes for the platform to accept a request, process it, and return a response.  We measure response time at the platform’s front door (i.e., excluding Internet latencies, etc.).  From a platform perspective, performance is also a measure of throughput; the number of operations that can be completed per unit time.  This is a function both of response time and concurrency.

Scalability is an observation about the trend in performance under increasing demands.  For example scalability can be characterized as the rate of response time degradation relative to increasing platform traffic.  Alternately, scalability can be viewed as the increase in throughput attained via an increase in system resources (e.g., if I double my server capacity, will I actually get twice the throughput?).

Note that this definition of scalability is irrespective of absolute performance.  For example, a slow operation may be considered scalable if a 10X increase in load results in only a 1.2X increase in response time.  Conversely, a lightning fast operation may require a resource that can only be accessed serially, thereby causing a concurrency bottleneck and thus may not be scalable.

Clearly performance and scalability are related, but are not equivalent.  In practice however, the faster an operation’s response time, the less system resources it consumes.  This means that it reduces load on other operations and is less affected by the load of others.  Both effects can positively impact throughput and thus scalability; doing more per unit system resource.


In our practical universe, scalability and performance are highly related through economics.  Per the previous point, the faster an operation’s response time, the less system resources it consumes.  Generally speaking, this translates into higher throughput by a given set of physical resources.  Thus, the higher the performance, the lower the rate of capital expenditures required to handle a given rate of traffic growth.  Therefore, scalability and performance both contribute to economic scalability; doing more per unit cost.

Put another way, both higher performance and higher scalability reduce the cost of scaling.

So Who Cares?

Performance as response time is business critical because it directly impacts user experience, without which there is no business.  Our users and our business partners care.

Performance as throughput is business critical because it directly impacts system expenditures and the ongoing total cost of ownership.  I just made the CFO happy.

Scalability is business critical for avoiding infrastructure meltdowns in the face of market success and is one of technology’s key contributions to profitability.  Clearly we all care about this one.

Finally, an early understanding of scalability characteristics is critical to aligning engineering investments with company growth.  Unlike performance tuning, scalability limitations are rarely fixed quickly since they are often a function of architecture rather than implementation.  Many resource constrained startups choose functionality over scalability for good and obvious reasons.  However, the sooner an engineering team can target some serious focus on scalability, the fewer techies will be seeing 3am from the office.