## What is optimization?

In Design Methodologies, Embedded Systems, Engineering Principles, Mathematics on April 15, 2013 at 12:04 AM

Perhaps optimization is the most abused word in all of research, engineering work or any task that seeks to maximize some intent. The Merriam-Webster dictionary defines it as “an act, process, or methodology of making something (as a design, system, or decision) as fully perfect, functional, or effective as possible; specifically : the mathematical procedures (as finding the maximum of a function) involved in this”. We can hear about optimizing power, area, a  performance metric like latency etc. Many people pass of every design decision as an optimization strategy. While such decisions may contribute to local optimization, they may fail in achieving global optimization. In fact such optimizations may actually degrade performance when the software or the design is used in a context which was not anticipated or thought of by the original developers. Read here about some insight into optimizing a memory allocator in C++.  You will find another debatable example of optimization to make software run faster here. And here is a nice article on efficiency versus intent. Typically optimization is associated with increasing the efficiency of a design (hardware or software) in some aspect. But such optimizations should not destroy the intent of the design. This requires a bit more analysis on part of the designer/developer to ensure that the intent is not lost. Here is another example.

The field of mathematical optimization, which is related to selecting the most appropriate choice (that satisfies a given set of criteria) from a set of alternatives, is vast and varied. There are numerous techniques suitable for different kinds of problems. You can see the list here. Frankly, it is a tough job to recommend one of these techniques for non-trivial problems. One needs to understand the nature of the problem in detail to make a just recommendation. Understanding the nature of such problems or modeling such problems in a way which is suitable for any of these techniques to be applicable is a non-trivial task. It requires a lot of theoretical and practical insight.

In Design Methodologies, Education, Embedded Systems, Mathematics on March 22, 2013 at 7:13 PM

Alvin Toffler in his book “Future Shock” says that  “The illiterate of the 21st century will not be those who cannot read or write; they will be those who cannot learn, unlearn, and relearn“. Taking this quote a little out of context in which Alvin used it, I would say that the process of learning, unlearning and relearning basically embodies the principles of evolution and adaptation. And these are equally applicable to education. Are these emphasized enough in universities and schools? Can they be taught? May be yes, may be no. I will give one simple example here. Every electronics or computer engineer would have done some basic C programming. To add two numbers, A and B, one just needs to use the expression ‘A+B’. Does it always work? Not in the world of computers where one has to deal with overflows and underflows. And there is always a limit to the biggest number that a computer or a computing platform can support.

So, how are we going to add two arbitrarily sized positive integers. Examples of positive integers are 123456, 90913456 etc. I will use positive integers to illustrate ‘learn, unlearn and relearn’. The example can easily be extended to other data types. In C language, the integer data type can only support a maximum value of 2,147,483,647 when adding two numbers. So there is an overflow if sum exceeds this value and addition is not possible if either A or B is bigger than this value. To avoid this, one can use other data types supporting greater number of bits until one hits yet another ceiling. After a point, you hit the final ceiling. If the numbers are really so big, one way to deal with them is to go back to our old school days when we learned to add numbers: 2 digits at a time with a carry propagated. Yes, that is all you need to do! And this does not require in-depth of knowledge of various IEEE methods to represent numbers. It is simple and good old school method. Of course, the old school method may not have a very wide application, but it does help where possible and makes it clear that  the symbol for addition “+” (or the add operator as it is referred to in programming languages) should not make us forget how addition is done. We “learn” to add 2 digits at a time in school, then we learn to use the “+” operator in programming languages. Thereafter we have to unlearn this concept to relearn (or recall) the school method.  I have written a reference implementation in C which you can find here. You can also find its link under the software tools tab here.

## Numerical Stability in Calculations

In Design Methodologies, Embedded Systems, Engineering Principles, Mathematics on January 24, 2013 at 11:44 PM

## Velocity, Displacement & Acceleration: Science vs. Engineering

In Design Methodologies, Education, Engineering Principles on December 24, 2012 at 7:04 PM

One often encounters the question: What is the difference between science and engineering? An oft quoted answer is that engineering involves, roughly speaking, an application of science or scientific results borne out of investigation into the nature of matter and its interaction with its surroundings. Science is about acquiring more knowledge and understanding about existing phenomena whereas engineering involves solving problems by applying that knowledge. Therefore, many also hold the view that it is applied science. Well, I won’t get into the debate of engineering vs. science or put before you an essay on this topic in this post. I would just like to highlight an example of where engineering takes over from science. Every student studies the concepts of velocity, acceleration and displacement in elementary Physics classes. These concepts are very simple: velocity is the derivative of displacement with respect to time while acceleration is the derivative of velocity with respect to time. Therefore to get displacement from velocity , one needs to integrate the former with respect to time over a given time period. Similarly, velocity at a certain point in time is the result of integration of acceleration over a given time interval. Now, if one is asked to apply these principles to calculate velocity and displacement using the acceleration data obtained from a transducer mounted on an engine, how would one do it? In this case, the engine vibrates and there is no physical noticeable movement of engine body from one place to another in the traditional sense (like a ball traveling from place A to place B in a field). This is where engineering comes in. An engine is a complex system and its vibrations need not be linear or constant in time. There can be vibrations with low frequencies as well as high frequencies and there can be periods of no vibration at all. In these cases, calculation of displacement or velocity is not straight forward and requires greater insight into the mechanism of vibration as well as the nature of acceleration signal. I would recommend reading up 1, 2 and 3 to get an idea of how interesting and insightful it can become! These are links to articles by Prosig  which works in the area of noise and vibration analysis. Understanding these mechanisms is important for any embedded designer who writes code to measure such parameters using microcontrollers etc.

## Can a computer do envy-free divison?

In Education, Interdisciplinary Science on July 28, 2012 at 10:15 PM

We have all studied division. In the world of simple mathematics, 8 divided by 2 is always 4.  But what about dividing a cake into 2 equal pieces? A computer program can always divide 8 by 2 and give 4 as answer, but can a computer program divide a cake into 2 equal pieces? Let us make it a bit more complicated. Say the cake has to be divided between persons A and B and in such a way that neither of them feels that the other person got more. This means that neither A or B will envy the share received by the other. So here the notion of equal division has to be understood in the context of the result leading to an envy-free solution. This is the subject of “Fair Division” also known as cake cutting problem. It is studied in politics, mathematics, economics and the like. Methods and algorithms have been proposed to achieve fair division but all require inputs from the parties involved in the division at different stages of the procedure. Note that these inputs need not be disclosed as these could be the feelings/assumptions/conclusions running in the minds of the parties involved. This means that different inputs at different stages can lead to different outcomes. Does it remind of “Observer Effect” in Physics? Yes. The inputs(observation of a current state of division) by a party affects the outcome of division (phenomenon being observed). It is impossible (?) for a computer to solve a problem of this type entirely on its own. Such problems arise routinely in allocation of goods, dispute resolution, negotiation of  treaties etc.

Borrowing terms from economics, a number can be treated as ‘a homogeneous good’ while a cake is essentially ‘a heterogeneous good’ as different parts of it can taste different. Hence, its envy-free division is far more complicated. If you are interested, try to read “Fair Division-From cake-cutting to dispute resolution“, an excellent book by Steven J. Brams (political scientist) and Alan D. Taylor (mathematician).