Rereading an old networking textbook this afternoon, I ran across this passage:

There is frequently a trade-off between some sort of optimality and scalability. When hierarchy is introduced, information is hidden from some nodes in the network, hindering their ability to make perfectly optimal decisions. However, information hiding is essential to scalability, since it saves all nodes from having global knowledge. It is invariably true in large networks that scalability is a more pressing design goal than perfect optimality.

Computer Networks, A Systems Approach, 3rd ed., p. 318

Larry L. Peterson and Bruce S. Davie

This reminds me of an idea I and a friend have been kicking around lately: software can be optimal only within its own domain, where a domain amounts to a layer in some software stack. Kernels optimize, say, the mappings between virtual and physical pages. Compilers optimize register allocation. Application writers optimize data structures and algorithms. And that's about as good as it gets. As a counterexample, if an application writer were also to have control over and knowledge of register allocation, page tables, i/o buffering, and so on down the stack, she might be able to create a maximally optimal application, but it would come at the cost of scalability of her own productivity. It would take her longer. So efficient use of programmer time requires layering.

At first glance I thought that layering also requires software reuse, but perhaps it doesn't. An ISV, for example, could ship an application for which it wrote a one-off operating system and compiler, with layering dictated by the fact that the application, operating system, and compiler teams were completely separate and communicated only by design specs. But that's inefficient as well. So the spirit of the law of layering requires software reuse, which in turn requires that each layer be as general as possible.

All of which reminds me of the argument David Clark made back in the 80s about layering in networks. I suppose there are equivalent papers about layering in software architecture as well, the most likely (canonical) one being Douglas McIlroy's Mass Produced Software Components.