The World as a State Machine

A state machine is basically a model of computation which helps one analyze the effects of input on a system. This system can remain in different states throughout its life cycle though in only one state at a time. It can transition from one state to another depending on some input. Every state machine has a start state and it progresses from there to other states eventually leading to an end state. Note that it is possible to reach the end state from any intermediate state as well as the start state. It depends on the system being modeled. Also, the output of each state may depend on the current state as well the inputs to that state.  Thus state machines model reactive systems i.e. systems which react. A good description of state machines can be found here. Note that the description there is related to finite state machines, which are so called because they have finite number of states. State machines are used in different fields of study not just electrical or computer engineering. They are used in biology, mathematics, linguistics etc. They also have different variants each trying to capture some additional parameters of a system which I would not go into. You can read about them at the link mentioned earlier.

I was wondering if the world can be modeled as a state machine. I think that the world in fact is a state machine except that its end state is unknown. Those with absolute faith in cosmological physics would state that the “Big Bang” can be considered as the start  state. Those with religious views might consider something else as the start state.  The beauty of this world being considered as a state machine lies in the fact that it does not matter whether you believe in science or not. It does not matter whether you have more of a religious bent of mind and would like to see the world from a religious or theological perspective or whether you want to see it only from a scientific standpoint. Either way, the world can be modeled as a state machine. You get to choose the start state depending on which viewpoint you are more comfortable with. In both the cases, the world is in fact a reactive system. It can even be considered as an aggregation of interacting state machines where each state machine can represent the economic, social, political, religious and scientific state of the world. And nobody would deny that all these concepts influence each other. Every electrical or computer engineering student studies about Moore and Mealy state machines. To them, the world is probably a Mealy state machine though not strictly so: the outputs in any state that this world resides in is dependent not only on the current inputs but also on the current state. If we look around us, it sounds so true,   does it not? However, this state machine is extremely complex!

What is optimization?

Perhaps optimization is the most abused word in all of research, engineering work or any task that seeks to maximize some intent. The Merriam-Webster dictionary defines it as “an act, process, or methodology of making something (as a design, system, or decision) as fully perfect, functional, or effective as possible; specifically : the mathematical procedures (as finding the maximum of a function) involved in this”. We can hear about optimizing power, area, a  performance metric like latency etc. Many people pass of every design decision as an optimization strategy. While such decisions may contribute to local optimization, they may fail in achieving global optimization. In fact such optimizations may actually degrade performance when the software or the design is used in a context which was not anticipated or thought of by the original developers. Read here about some insight into optimizing a memory allocator in C++.  You will find another debatable example of optimization to make software run faster here. And here is a nice article on efficiency versus intent. Typically optimization is associated with increasing the efficiency of a design (hardware or software) in some aspect. But such optimizations should not destroy the intent of the design. This requires a bit more analysis on part of the designer/developer to ensure that the intent is not lost. Here is another example.

The field of mathematical optimization, which is related to selecting the most appropriate choice (that satisfies a given set of criteria) from a set of alternatives, is vast and varied. There are numerous techniques suitable for different kinds of problems. You can see the list here. Frankly, it is a tough job to recommend one of these techniques for non-trivial problems. One needs to understand the nature of the problem in detail to make a just recommendation. Understanding the nature of such problems or modeling such problems in a way which is suitable for any of these techniques to be applicable is a non-trivial task. It requires a lot of theoretical and practical insight.

Relearning addition

Alvin Toffler in his book “Future Shock” says that  “The illiterate of the 21st century will not be those who cannot read or write; they will be those who cannot learn, unlearn, and relearn“. Taking this quote a little out of context in which Alvin used it, I would say that the process of learning, unlearning and relearning basically embodies the principles of evolution and adaptation. And these are equally applicable to education. Are these emphasized enough in universities and schools? Can they be taught? May be yes, may be no. I will give one simple example here. Every electronics or computer engineer would have done some basic C programming. To add two numbers, A and B, one just needs to use the expression ‘A+B’. Does it always work? Not in the world of computers where one has to deal with overflows and underflows. And there is always a limit to the biggest number that a computer or a computing platform can support.

So, how are we going to add two arbitrarily sized positive integers. Examples of positive integers are 123456, 90913456 etc. I will use positive integers to illustrate ‘learn, unlearn and relearn’. The example can easily be extended to other data types. In C language, the integer data type can only support a maximum value of 2,147,483,647 when adding two numbers. So there is an overflow if sum exceeds this value and addition is not possible if either A or B is bigger than this value. To avoid this, one can use other data types supporting greater number of bits until one hits yet another ceiling. After a point, you hit the final ceiling. If the numbers are really so big, one way to deal with them is to go back to our old school days when we learned to add numbers: 2 digits at a time with a carry propagated. Yes, that is all you need to do! And this does not require in-depth of knowledge of various IEEE methods to represent numbers. It is simple and good old school method. Of course, the old school method may not have a very wide application, but it does help where possible and makes it clear that  the symbol for addition “+” (or the add operator as it is referred to in programming languages) should not make us forget how addition is done. We “learn” to add 2 digits at a time in school, then we learn to use the “+” operator in programming languages. Thereafter we have to unlearn this concept to relearn (or recall) the school method.  I have written a reference implementation in C which you can find here. You can also find its link under the software tools tab here.

Numerical Stability in Calculations

I did not have any course on algorithms in my undergraduate education. I studied about them (their properties, design etc.) during my research work. I now realize why their study is important for anyone who wants to be really good at designing algorithms or implementing them. After all, algorithms solve problems. I recently came across the subject of numerical stability of algorithms, numerical algorithms to be precise. While algorithms help solve problems, they need to be implemented on a digital machine (a computer for example) which has limited precision. Whatever number system we use, they cannot cover all the numbers present in exact  mathematics. This leads to approximations as well as upper and lower bounds on the numbers that can be represented. Also, approximations can be the source of errors and deviations from the exact numerical answer. For instance, on a machine with only 3 digit precision, numbers like 22, 2.22, 0.110, 100, 177 can be represented. Now if you try to add 2 and 1000 instances of 0.11 , your sum would be 112 on this machine and this matches with the exact answer. Similarly, if you try to add 9 and 9 instances of 0.11, the answer on this machine would be 9.99, which matches with the exact answer. However, if you try to add 10 and 9 instances of 0.11 in that order i.e 10+0.11+0.11…., the machine would return 10 as answer because the moment you try to add 0.11 to 10, you are going to exceed the precision of the machine. Now imagine doing the same calculation in the reverse order i.e adding all the nine 0.11’s first and then 10 i.e. 0.11+0.11+….+10, the machine would return an answer of 0.99 which is far off from the actual answer 10.99 and far worse than the previous approximation of 10 (for the other order of addition). This means that the way you arrange your numbers( in memory, for instance an array) to be added also may influence the sum!! I wish that embedded systems engineers read more on this subject so that the numerical errors that we see cropping up in such systems get reduced. A nice introduction is at wikipedia.

Velocity, Displacement & Acceleration: Science vs. Engineering

One often encounters the question: What is the difference between science and engineering? An oft quoted answer is that engineering involves, roughly speaking, an application of science or scientific results borne out of investigation into the nature of matter and its interaction with its surroundings. Science is about acquiring more knowledge and understanding about existing phenomena whereas engineering involves solving problems by applying that knowledge. Therefore, many also hold the view that it is applied science. Well, I won’t get into the debate of engineering vs. science or put before you an essay on this topic in this post. I would just like to highlight an example of where engineering takes over from science. Every student studies the concepts of velocity, acceleration and displacement in elementary Physics classes. These concepts are very simple: velocity is the derivative of displacement with respect to time while acceleration is the derivative of velocity with respect to time. Therefore to get displacement from velocity , one needs to integrate the former with respect to time over a given time period. Similarly, velocity at a certain point in time is the result of integration of acceleration over a given time interval. Now, if one is asked to apply these principles to calculate velocity and displacement using the acceleration data obtained from a transducer mounted on an engine, how would one do it? In this case, the engine vibrates and there is no physical noticeable movement of engine body from one place to another in the traditional sense (like a ball traveling from place A to place B in a field). This is where engineering comes in. An engine is a complex system and its vibrations need not be linear or constant in time. There can be vibrations with low frequencies as well as high frequencies and there can be periods of no vibration at all. In these cases, calculation of displacement or velocity is not straight forward and requires greater insight into the mechanism of vibration as well as the nature of acceleration signal. I would recommend reading up 1, 2 and 3 to get an idea of how interesting and insightful it can become! These are links to articles by Prosig  which works in the area of noise and vibration analysis. Understanding these mechanisms is important for any embedded designer who writes code to measure such parameters using microcontrollers etc.

Can a computer do envy-free divison?

We have all studied division. In the world of simple mathematics, 8 divided by 2 is always 4.  But what about dividing a cake into 2 equal pieces? A computer program can always divide 8 by 2 and give 4 as answer, but can a computer program divide a cake into 2 equal pieces? Let us make it a bit more complicated. Say the cake has to be divided between persons A and B and in such a way that neither of them feels that the other person got more. This means that neither A or B will envy the share received by the other. So here the notion of equal division has to be understood in the context of the result leading to an envy-free solution. This is the subject of “Fair Division” also known as cake cutting problem. It is studied in politics, mathematics, economics and the like. Methods and algorithms have been proposed to achieve fair division but all require inputs from the parties involved in the division at different stages of the procedure. Note that these inputs need not be disclosed as these could be the feelings/assumptions/conclusions running in the minds of the parties involved. This means that different inputs at different stages can lead to different outcomes. Does it remind of “Observer Effect” in Physics? Yes. The inputs(observation of a current state of division) by a party affects the outcome of division (phenomenon being observed). It is impossible (?) for a computer to solve a problem of this type entirely on its own. Such problems arise routinely in allocation of goods, dispute resolution, negotiation of  treaties etc.

Borrowing terms from economics, a number can be treated as ‘a homogeneous good’ while a cake is essentially ‘a heterogeneous good’ as different parts of it can taste different. Hence, its envy-free division is far more complicated. If you are interested, try to read “Fair Division-From cake-cutting to dispute resolution“, an excellent book by Steven J. Brams (political scientist) and Alan D. Taylor (mathematician).