Object Oriented Programming: Expectations - The Silent Third Characteristic

In object oriented programming, objects always consist of 2 parts.

1. State - the data that together describes the object.
2. Behavior - the ways in which you can apply the internal state of an object.

This makes sense from a theoretical perspective as these are represented through different parts of the programming language (syntax). Data are represented by fields and store/reference values. Behavior is expressed through the implementation, the program’s logic.

However, when looking at from the perspective of implementation, then you will notice that there are additional subtleties. For example, for state we focus on mutable state as this is the thing that contributes to the variations in outcome of method executions. Immutable state, static and non-static final fields, do not change and therefore the outcome does not change for successive method calls. Additionally, even though (interface) methods are predefined, there is still the notion of polymorphism (dynamic dispatching) which may determine as late as at runtime which method actually gets executed.

Therefore, in execution hides a third characteristic which is determined by the environment rather than the implementation:

The environment’s expectations of the object are defined beforehand through hardcoded constants, and just-in-time through the initialization of immutable fields by provisioning values and concrete implementations at runtime for the object to use.

This third, silent, characteristic is not “as physical” as State and Behavior. State and behavior are easily identifiable as they enjoy first-class representation in the programming language itself. The third is an external (immutable) part which are an implicit logical outcome by virtue of the use of objects, within and determined by the surrounding environment. Here I will call this the Expectations.

Describing Expectations

Expectations are a characteristic that expresses itself in either immutable state or the determination of concrete implementation of behavior, such as an implementation of an interface. It is the reason why statics get in the way and the reason why objects tend to get polluted with unnecessary state or behavior. Expectation is the “external” state of an object. The external state is dictated by the environment - the over-all application - onto the object. Objects should have behavior that matches behavior of (nearby) other objects such the object’s execution matches the intentions of the environment. This is a significant part of programming and program structure. So what are expectations in object oriented programming?

In functional programming, there exist no side-effects. Every function is pure and therefore you can predict 100% of its behavior just by looking at the function arguments when a function is called.

In object oriented programming, you do not have this luxury, as there is internal state that influences the behavior. Looking solely at a method’s arguments predicts part but not all of its behavior.

However, sometimes you get to the point where certain parts of the code use an object and expect it to behave in a certain way. Let’s take, for example, a simple calculation: 1/3 (i.e. divide 1 by 3). Furthermore, we’ll state that we do not want infinite precision or lazy computation. We want to calculate given a certain level of precision.

In a (simplistic) functional-esque approach, one would provide everything as an argument. The implementation is a function that accesses no other state. (The example is in Java, using a static method without class variables).

package thirdcomponent;

import java.math.BigDecimal;
import java.math.MathContext;

public class UtilNoState {

  public static void main(String[] args) {
    System.out.println(calculate(BigDecimal.ONE, BigDecimal.valueOf(3L), MathContext.DECIMAL32));
    // prints 0.3333333
  }

  public static BigDecimal calculate(BigDecimal x, BigDecimal y, MathContext c) {
    return x.divide(y, c);
  }
}

Now, taking purely an object-oriented view, this will become quite bothersome in large applications, as you would need to provide every argument every time. We understand that x and y often need to be provided explicitly because these may change with each use, but c is often a given in an implementation. It is “common knowledge” and expected to be known. So you make it a constant …

package thirdcomponent;

import java.math.BigDecimal;
import java.math.MathContext;

public class UtilWithStatics {

  // Predefined math context for use in calculation.
  private final static MathContext C = MathContext.DECIMAL32;

  public static void main(String[] args) {
    System.out.println(calculate(BigDecimal.ONE, BigDecimal.valueOf(3L)));
    // prints 0.3333333
  }

  public static BigDecimal calculate(BigDecimal x, BigDecimal y) {
    return x.divide(y, C);
  }
}

Declaring these things as constants is often not the solution, though. The reason for this is that these choices are not absolute. The choice is made beforehand, however the choice is only relevant for the given context, application or part thereof, or even for a single execution trace. The very next time the application executes, some values could be different. There are numerous examples to think of that have the same type of expected behavior that cannot be hardcoded, such as database access, connection management, open files, logging settings and configurations.

For this reason, it is not possible to implement this as constants and static methods. We can identify variations in the code, i.e. parts of the implementations are variable, although not variable enough to incorporate it as part of the internal state of an object and thus making it the responsibility of the object. In between these two extremes, there is another option.

What do expectations look like?

Let’s first look at the following code example. The calculation is implemented as a instance method. Static implementations are not sufficient. The decision for the expected behavior is provided upon construction of the object. The object is then used.

package thirdcomponent;

import java.math.BigDecimal;
import java.math.MathContext;

public class ExpectationsExample {

  public static void main(String[] args) {
    final BigDecimal one = BigDecimal.ONE;
    final BigDecimal three = BigDecimal.valueOf(3L);
    final ExpectationsExample comp = new ExpectationsExample(MathContext.DECIMAL32);
    System.out.println(comp.calculate(one, three));
    // prints 0.3333333
  }

  // provided, external choice
  // final as we don't allow ad hoc changes
  // value is dictated by environment, not managed by instance
  private final MathContext c;

  public ExpectationsExample(MathContext mc) {
    this.c = mc;
  }

  public BigDecimal calculate(BigDecimal x, BigDecimal y) {
    return x.divide(y, c);
  }
}

These expectations manifest themselves in 2 different ways:

1. As immutable fields (such as final fields in Java) that at construction get determined and supplement the object’s behavior. The field itself is constant/immutable. It does not need to change throughout the lifetime of the object. The mere presence of the provided value is sufficient to influence the logic such that it behaves according to expectations.
2. As an (alternative) implementation of an interface. The nature of the chosen implementation matches the expectations of the rest of the application. The application instantiates the class that matches its current requirements (the expected behavior) to fit in with the rest of the program. (Note that as a reference for option 2 needs to be stored somewhere, it essentially manifests itself as option 1 again.)

Option 1 is flexible. It is used in cases where the implementation itself is mainly fixed, except for some parameters. These parameters must be decided upon during implementation. This decision could be based on company parameters or yearly updated calculation rules or components in calculation rules, for example.

Option 2 is used in case of significantly different behavior. The (alternative) implementation of an interface can be vastly different, except for the semantics as defined by the interface which is the contract. As noted, option 2 is also captured in option 1, as we need to store a reference to the concrete implementation for which we again use an immutable field.

Ideally, one would provide the choices upon creation of the object, i.e. when calling the constructor. The decisions are incorporated into the implementation as immutable data and can be called upon when necessary. Note that there is no need for this data to be mutable, as it is dictated by the environment rather than the inner workings of the object.

There is one well-known manifestation of expectations: Dependency Injection. Dependency injection is used as the mechanism for providing instances that match the current set up of the application, i.e. the way the application expects an object to behave in certain situations, use certain objects. Unfortunately, due to the way most dependency injection is used, it does not allow for these dependencies to be immutable, even though they are not intended to be changed. (Note that there is a variation of dependency injection that inject dependencies via the object’s constructor, in which case they can be stored in final fields in the object.)

Arguably, one could call all of these expectations “dependencies to be injected”. It depends mostly on your definition of dependency. If single values, or references to simple objects for that matter, already qualify as dependency, then you could say that dependency injection is the provisioning of the expectations to your object.

Expectations are different from mutable field initialization

Now that we’ve seen what I consider to be expectations. I’d like to emphasize that initializing mutable fields of an object at construction is not considered to be part of the expectations. First, conceptually the expectations are dictated by the environment. Mutable fields are managed by the object itself, thus this contradicts the concept. Second, as these mutable fields are managed by the object itself, the object may change the field at any moment. The provided initial value is only that: the initial value. The value for the object to start with. Ultimately, the field is the responsibility of the object. Which is not the case for expectations.

Setting expectations separately from construction

So far we have only talked about setting expectations at construction time. The object is constructed and expectations set (i.e. dependencies injected and other operational parameters provided). What if we want to separate the two activities? If I defined my notion of expectations decently enough, you might already suspect what I am going for.

A well-known example of separating setting expectations from construction is the Factory. In this pattern, a “factory” instance is first constructed. During the construction of the factory we can already declare all expectations. Then at a later time, the factory instance is used to construct objects as requested with declared expectations. The factory itself has internal knowledge of the expectations of the environment which need not be known by the caller. The caller has no knowledge of expectations and does not need to know to correctly create the necessary objects, as the factory handles the actual construction of objects. The objects as produced by the factory contain the expectations as dictated by the environment and when used by the ignorant caller will still “fit in” with the environment. Furthermore, an Abstract factory pattern can be used if another level of abstraction is required.

A factory is quite remarkable in that it can cleanly and fully separate setting expectations from the remaining initialization-step. It follows quite naturally from the fact that expectations are known in advance, while initialization of the object happens at the point of construction, as exact arguments might only be known as late as right before the moment of creation (of the object).

But what if expectations change?

One might remark that the requirements of the environment might change and therefore the expectations (injected dependencies, if you like) cannot be immutable. This is a fair remark - I agree that this needs to be a possibility, although it may not be as simple as this.

For this use case, I believe that there is actually one additional requirement. The change is initiated from the environment, rather than the object keeping the reference. Even though we would like our object to conform to changed (or rather changing) expectations, we also require that all “users” (objects keeping a reference for use) conform to the changed expectations. Furthermore, even though we want to change expectations as required by the environment, we do not want the object to change its own expectations, because either:

1. the object knows about necessary expectations and thus is knowledgeable about more than its own purpose. Specifically, knowledge of application state besides its own state (mutable fields). Or …
2. the object might unintentionally incorrectly modify its expectations because of lack of knowledge of the environment. Or if publicly accessible, using code may make incorrect modifications.

Both cases are undesirable. What you would actually want to do in this particular case, is to create a suitable abstraction.

In case of “just a changing reference” one might use a basic (thread-safe if needed) container to store a changing expectation. A read-only container instance can be shared among many instances. The container reference can be stored in an immutable field. The read-only container ensures that using objects cannot modify the expectation. In case of changes, the reference inside the container can be updated accordingly by the environment, which should hold a write-enabled instance.

In case of a more complicated requirement, where the actual reference must be determined “just-in-time”, one can use a “proxy” object that delegates the operation to a suitable instance upon request. Again, this allows the proxy reference to be immutable. The proxy itself, which could be shared among many objects having the same expectations, can then intelligently select the expectations, i.e. the instance.

Are values defined in specifications considered expectations?

A manifestation of data that is similar to the described notion of “expectations” (at least conceptually) is data values that are defined in specifications. So how do we handle these values? The way we handle specification values is actually quite straight-forward and I believe this is predominantly handled in the correct (and obvious) way already.

Predefined values from specifications are modeled in your code as constants. That is, like expectations, this data is immutable. This is obvious as the spec does not change and you would want your library to match the spec without extra knowledge from the user. Unlike expectations though, the user of the object should not need to provide the values, as they are already defined in the specifications. Therefore, we incorporate specification values as constants.

Note that as soon as the implementation gets split up into multiple classes, then it may be necessary to pass on specified values to objects that are used inside the implementation. At that point, we are back at the original scenario where we have an “over-all application” that has expectations over the objects it will be using.

This also nicely illustrates that there is conceptually a boundary between the protocol implementation and code that uses the protocol implementation. At some point, there is a class that contains the specified constants. It does not expect to receive this data at construction time, as the user should not need to know this information. On the other hand, it sets the expectations for underlying code and does not (necessarily) expect used code to know these specification values. I say “necessarily” because complex specification may be implemented in parts/layers and some used objects may contain parts of the specified values that are relevant only to that particular part in the implementation.

Characteristics of expectations

Expectations have a number of distinguishing characteristics. There should be a clear difference between an object’s internal state, which is managed by the object itself, and other values, expectations, which are dictated by the environment.

1. Expectations are used to make an object “fit in” with the rest of the application. They do not “describe” (part of) the object, and instead are merely used by the object.
2. Expectations are not part of managed mutable (internal) state of the object. They are not the responsibility of the object. Instead they are dictated by the environment, the over-all application.
3. Expectations are references that, at the latest, are known at construction time. (They may be known from the start when defined as constants.)
4. Expectations references are final, they are intentionally immutable.
5. As we consciously decide which pieces of data are expectations and these are defined as immutable, we reduce the surface for null pointer exceptions, as well as eliminate the chance to accidentally modify data that should not be touched.
6. Expectations are still allowed to be null (or Optional if that’s the approach to optional values) if it is not a strict requirement for operation. This is less of a problem than expected as 1. it should not happen that often, and 2. it is easy to reason about possibility of null when we know that after construction the definitive value of the field is known.
7. Expectations may be transient. That is, you may not need to keep a reference to an expectation if it is only used during construction. This may be the case, for example, when constructing objects from a provided factory instance. The factory may lose relevance as soon as objects are constructed. Only the results are stored (as expectations thus immutable) but the factory instance itself is not preserved.
8. Clearly distinguishing between expectations and object state, makes it easier to reason about an object’s single purpose. By classifying provided data as expectations, you can differentiate between true object state and other data.
9. Expectations are determined by the application, instead of the object itself. Therefore, you can use expectations where static methods are not sufficiently flexible.

Furthermore, some additional comments:

1. Expectations must always be immutable. It might be the case that the expectation’s target may change. That requires a level of indirection, such as a container or a proxy object. The expectation as provided to the object should be immutable.

2. In some cases expectations are already lost by the time the object is constructed. There is a temporal separation between the two moments. A factory or abstract factory can help to bridge the gap between the moment when expectations are declared and the moment when objects are actually constructed. In that case, the caller need not even be aware of the expectations of the environment.

3. There is a concept known as “behavioral objects”. These are objects that have no state, only a number of (utility-style) methods. Behavioral objects are created in those cases where static utility methods are not feasible because of expectations w.r.t. the implementations. Even so these objects have no state to manage. Behavioral objects naturally emerge when distinguishing between internal state and expectations. Objects which operate only on expectations but have no data to manage, such as the example shown earlier of calculations that are performed within a predefined math context, automatically become behavioral objects as the only dependence is on expectations. All methods on the object are part of a collection of methods that operate with same set of expectations. These behavioral objects typically have a common topic such as the target class or interface on which the methods operate.

Now, do not fall into the trap of assuming that every immutable field is automatically an “expectation”. Not everything that is immutable is. Say you have an object that maintains a list of users. You would make the field referencing the list immutable, as you would always need the list itself to exist. (null is not a valid list.) The list may be empty though, which is valid state for an object maintaining a list of users. Note that, because the object’s purpose is to maintain a list of users, the list field is actually (part of) the purpose of the object.

Furthermore, would this list not be part of the object’s purpose, then the list itself would ideally also be read-only/unmodifiable, which means that we are not allowed/able to add or remove items. This is, of course, only relevant if the reference is provided for informational purposes. If you intend to provide a reference meant for updating, then it should (still) be update-enabled.

Conclusion

The silent third characteristic is about embracing the state in a manner that emphasizes a clear distinction between object’s internal state and other external state as dictated by the application/environment, which I here called expectations.

The internal state of an object is quite a clear and obvious concept. The “external” state that is used to express the expectations of the application for the object in order to “fit in” with the surrounding logic is less obvious and often not recognized as such. By recognizing the third characteristic, we can better leverage the features of the programming language to support/enforce the implementation and simplify reasoning and predictability about source code, and in particular about object implementations. We also help to ensure/improve the concept of single purpose of an object, also known as Single Responsibility Principle.

I am hoping programmers will be able to more clearly distinguish between these variants of state. With recognizing external state we know what to expect from these references, and what guarantees we get. This will ultimately contribute to better understandable and more maintainable software. This article is very much intended as an attempt to better understand software engineering and to share the insights with the world.

References

• Wikipedia: Dynamic dispatch
• Wikipedia: Factory (object-oriented programming)
• Wikipedia: Abstract factory pattern
• Wikipedia: Proxy pattern
• Wikipedia: Single Responsibility Principe
• Oracle: What is an object?