The definition of simplicity

Tue, Mar 31, 2020 ❝An attempt at defining simplicity, the word 'simple'.❞

Contents

2021-10-24: a later post fine-tuned the definition. However, further insight revealed that these corrections would take the definition in the wrong direction. The post refined definition of simplicity and its revised definition can best be skipped altogether.

This is an attempt at a general definition of simplicity that should be broadly applicable.

There exist many definitions of simplicity that make soft statements that are ambiguous and therefore easy to interpret in different ways. I’ve attempted a definition such that misinterpretation and therefore discussion should be limited.

I have chosen the terms for this definition such that three parts make up simplicity, each covering a different dimension of the answer. In this context, dimension means the full range of a certain property. The dimensions should complement each other, preferably to the extent that the three properties together are complete. That is to say, together the properties cover all aspects of ‘simplicity’.

NOTE for this article, I choose to use a very specific definition for the term ‘optimization’. It may not match with your own definition, so be warned.

The definition

simple = specialized & deoptimized & reduced

Simplicity is characterized by 3 properties that all need to hold for something to be called simple. Each of the properties evaluates a different dimension of the statement that is evaluated.

One may notice that this definition does not require that only a single answer is valid. Multiple answers may be applicable. Whether or not this is the case depends largely on the problem statement.

Properties

The definition consists of three properties: specialized, deoptimized and reduced. These properties are symbolic in nature, i.e. they do not represent a prescribed value, and are listed in definite order. Each property should be resolved in order, such that one can move towards the simple answer. Each property can be imagined as a range upon which we strive to move towards the idealistic extreme.

We discuss the following aspects for each of the properties:

• the dimension that it represents
• intuitive understanding of the property, the nature of its influence
• the benefits of moving within this dimension

Specialized

Implement an as-narrow-as-possible definition of a problem, using narrow, specific terms.

Dimension: the number of free variables describing the problem. (Fewer is better.)

Intuition: Specializing benefits simplicity by removing uncertainty and narrowing down both the problem and (consequently) its solution(s).
The problem is rephrased such that only that which is directly relevant needs to be solved. Instead of “How do I compute arbitrary powers for arbitrary numbers?”, solve “How do I square an arbitrary number?”. There is variation as is desirable for the problem. However, there is no more variation than is necessary. Consider that further specializing the problem to “How can I compute 2x2?” does not make sense, as this is a one-time calculation that can be evaluated.

Benefits: easily identifiable as solving the problem, given understanding of the problem. Problem characteristics (usually) identifiable in the solution.

Deoptimized

Avoid tricks and shortcuts that are available in the problem domain and/or the solution domain.

Dimension:

• the (size of the) expression language, (Smaller is better)
• the extent to which internal knowledge is required to comprehend the solution. (problem domain and/or solution domain)
  Orthogonality eases comprehension of the interplay of language parts.

Intuition: Given any problem, if the solution requires internal knowledge of the problem domain and/or solution domain, explaining the solution becomes significantly more complicated. The original problem statement does not need to be expressed using internal knowledge. The original question may still be trivial, e.g. “How can I calculate x²?”. It is only in expressing the solution that one may identify extra constraints that are present. These constraints are subsequently taken into account in defining the solution. The solution itself may be optimized further through the use of idiosyncrasies of the domain in which the solution is expressed.
All of this adds complication. Removing all applied optimizations leads to a simpler solution. One that is understood in the same terms in which the problem is expressed. No other knowledge is expected.

Benefits: Free of internal knowledge from the problem domain and/or solution domain. Free of “hacks” and “cutting corners” that aren’t part of the problem statement.
For example: given an example of a cryptographic problem, one can cut corners in the problem domain using a technique called “Karatsuba multiplication”. This technique provides additional performance when multiplying large numbers but incurs overhead for small numbers. The technique is suitable for cryptography as it works with large numbers. However, the technique itself uses clever mathematics trick to optimize multiplication making the solution hard to understand for the uninitiated. A simple multiplication expresses the solution equally fine, even though execution is not as fast. Similarly, in the solution domain: given that computers represent numbers in a binary system, one can apply bit-manipulations for calculations in the class of powers-of-2 values. For example, by applying bit-shifting for squarings.

Examples of (de)optimization

For the deoptimized property, examples are best provided as examples of optimization. The consequence thereof serve to illustrate why optimization is detrimental to simplicity.

Example: mathematics joke

“There are 10 types of people, those who understand binary and those who don’t.”

To understand this joke, one needs to interpret 10 as a binary number. However, given that the statement is expressed in natural language this is not what is first assumed by the reader. The number 2 becomes clear from the expression in natural language as the sentence progresses. (Note: deoptimizing the joke by substituting 2 for 10 kills the joke.)

Example: cryptographic implementations in programming languages

Cryptography is notoriously difficult to understand for the uninitiated. This is not helped by the fact that implementations are highly optimized for pure performance at the cost of all comprehensibility. For example, many important cryptographic implementations of the programming language Go are implemented in assembly code. Understanding this implementation requires understanding of assembly language, hardware such as processors and registers, memory, binary mathematics, all in addition to the specialized mathematics already required for understanding cryptography in general. Ironically, these optimizations provide room for bugs themselves. Such as forgetting to carry or excessively carrying a bit when it is not supposed to be carried.

Example: following the path

Lastly, let’s consider an example outdoors. By default, when taking a walk, one would assume to follow a path. The path, for example a side-walk, outlines the routes one can follow without making further assumptions of the area. One can consider cutting corners, which is fine if you want to cross a grassy area, but not if you attempt to cross a mountain range. Optimization only works given specific circumstances.

2020-03-31 a very illustrative example of the benefits and issues with a dirty hack in the Quake III Arena source code.

Reduced

Express the solution as succinctly as possible.

Dimension: the number of parts composing the answer. (Fewer is better)

Intuition: given an expression of the solution, write it in the most succinct form possible.

Benefits: easy to read, comprehend. Needs little interpretation to uncover the underlying intention due to succinctness.

Examples of reducibility

Example: statements in natural languages

“Taking one step forwards and two steps back.”

It is sometimes said that one takes “one step forward, two steps back”, which is a complicated way of saying that a little bit of progress is undone by more setbacks and problems. The net effect is taking one step backward, hence the former statement can be reduced to the latter statement.

Example: over-complicated mechanical contraptions

Rube Goldberg machines: a complicated solution to achieve a relatively simple goal. Many of the components in this solution can be replaced in order to reduce the number of components, and – most likely – significantly reduce the risks of failure during operation.
An obviously over-complicated example of this nature: passing the salt

The Incredible Machine (video game) is the video-game embodiment of overly complicated contraptions. However, given the ridiculous constraints one should note that little simplification is possible.

Simplifying

Let’s evaluate an example, the mathematical concept of squaring, with 2³.

Moving towards simplicity

Given a few examples of valid solutions that can be further simplified.

• Needs specialization: power(2, 3)
  Given some function power(a, b) - either self-defined or provided by the language - for raising to a specified power. The function power is suitable for a broad range of calculations for many initial values a and many powers b to raise to.
• Needs “de-optimization”: 1 << 3
  All bits are shifted by a number of positions such that it effectively doubles the original number with each shift.
  Knowledge is required about this new (lower) level of abstraction, such as how the bits represent the number, the binary number system, which bits are most and least significant, which operations are available in the platform.
  Assumptions are made about the representation in which the calculation is performed, possibly introducing classes of problems, such as deviating hardware, overflowing, and risk at failures due to, for example, a sign bit dropping in the process. In addition, there may be conditions and restrictions that apply specifically to this level of abstraction.
  An optimized solution will work given that the problem domain is restricted in a certain way – in this example: powers of two, and represented in the binary system – and the solution is appropriately adjusted to fit the domain.
  In an ironic twist of fate, simplicity failed me in formulating this example.
• Needs reduction: 2 + 2 + 2 + 2 (4 additions of 2)
  A valid, but less succinct expression of the solution. The solution, though correct, is less obvious as there are more parts to it. It can be written more succinctly. Note that, in an imaginary world with a language that does not support multiplication, 4 additions of 2 may be the best option available.

Respecting definite order of properties

2021-10-24: in the post concluding simplicity, we consider the dimensions orthogonal. Therefore, there would not be a definite order for these properties.

There is a definite order for the properties. Processing properties in a different order will result in wasted effort as the improvement is undone by an improvement with a larger/broader impact.

• “de-optimizing” before specializing: power(x, 1 << 1)
1 << 1 = 2, hence we can substitute for 2, however specializing to x * x would remove the need for the value 2 altogether.
• reducing before specializing: power(x, 1 + 1)
1 + 1 = 2, hence we can substitute for 2, however the less general solution x * x does not require a value 2 to be present at all.
• reducing before “de-optimizing”: x << (0 + 1)
0 + 1 = 1, hence we can substitute for 1, however after “de-optimizing” x << 1 the value 1 is no longer needed.

Note that each of these examples is trivial to the point of meaningless. Everyone will - given normal circumstances - immediately recognizes that 0 + 1 is a senseless way of writing the value 1. More important is the type of pattern it represents, namely that of an expression that itself can be evaluated. However complicated an expression, if parts may be statically evaluated, then the expression as a whole can be reduced. The chosen examples are basic as not to distract from the underlying issue which is the order of evaluation.

Result: simplified

Result: 2*2*2

Such simplified results are easy to read and understand, provided that the problem itself is comprehensible.

Considering programming languages, compilers and interpreters are typically very well suited to optimize such expressions by themselves. All of the properties, such as minimizing inputs, reducing expressions, etc. make it such that there is little variation possible. Variation is what prevents compilers and interpreters from performing optimization with certainty.

Apart from benefits in technical domains, little to no variation benefits comprehension significantly. The more variation there is, the more unknowns and uncertainties exist. For solid comprehension of a topic or domain, one needs to be certain of the extent of influence. That is why it is important to simplify as much as possible, but no more.

“Everything should be made as simple as possible, but not simpler” – Albert Einstein

Note that, provided there is a language construct ** to calculate powers, 2*2*2 = 2**3 (i.e. 2³ expressed in notation of the language), meaning there is a second answer that is equally valid and simple. We assume that, if the language provides an operator for this type of calculation, it would likely be aware of suitable optimizations. Therefore the user does not have the apply optimizations manually, hence benefiting simplicity.

Parting words

This article is a way of exploring an infamously vague concept: “simplicity”. What does the word “simple” actually represent. I believe I’ve come to a reasonable description and classification. Writing it down helps to me put my thoughts in order en helps in discovering holes in the description.

“Simple” as defined above, seems to be suitable for many instances both inside and outside the technical domain. It would be interesting to discover in which are the definition is lacking/unsuitable.

Changelog

• 2021-10-24 Final fixes to this article while finalizing the article “Concluding the definition of simplicity”.
• 2021-10-03 Fixed ironically bad error in the not-quite-simplest example possible. See “Bad example demonstrates failure in complexity”.
• 2020-07-16 A newer article that refines the definition is referenced at the top of this post.
• 2020-04-11 Use “deoptimized” instead of “unoptimized”. Misc fine-tuning of semantics.
• 2020-03-31 Added a reference to a blog post with a beautiful example of optimization benefits and issues as found in the Quake III Arena source code.
• 2020-03-26 Initial version.