Numerical Stability in Calculations

I did not have any course on algorithms in my undergraduate education. I studied about them (their properties, design etc.) during my research work. I now realize why their study is important for anyone who wants to be really good at designing algorithms or implementing them. After all, algorithms solve problems. I recently came across the subject of numerical stability of algorithms, numerical algorithms to be precise. While algorithms help solve problems, they need to be implemented on a digital machine (a computer for example) which has limited precision. Whatever number system we use, they cannot cover all the numbers present in exact  mathematics. This leads to approximations as well as upper and lower bounds on the numbers that can be represented. Also, approximations can be the source of errors and deviations from the exact numerical answer. For instance, on a machine with only 3 digit precision, numbers like 22, 2.22, 0.110, 100, 177 can be represented. Now if you try to add 2 and 1000 instances of 0.11 , your sum would be 112 on this machine and this matches with the exact answer. Similarly, if you try to add 9 and 9 instances of 0.11, the answer on this machine would be 9.99, which matches with the exact answer. However, if you try to add 10 and 9 instances of 0.11 in that order i.e 10+0.11+0.11…., the machine would return 10 as answer because the moment you try to add 0.11 to 10, you are going to exceed the precision of the machine. Now imagine doing the same calculation in the reverse order i.e adding all the nine 0.11’s first and then 10 i.e. 0.11+0.11+….+10, the machine would return an answer of 0.99 which is far off from the actual answer 10.99 and far worse than the previous approximation of 10 (for the other order of addition). This means that the way you arrange your numbers( in memory, for instance an array) to be added also may influence the sum!! I wish that embedded systems engineers read more on this subject so that the numerical errors that we see cropping up in such systems get reduced. A nice introduction is at wikipedia.

Role of Discipline in Graduate Research

I recently came across a blog post titled “The Disciplined Pursuit of Less” on Harvard Business Review blog network. The author, Greg McKeown, describes in this article what distinguishes successful people from very successful people. There are many people who are good at doing many things and are willing to do them. They excel in almost everything that they do, no matter how varied those tasks are. Some of them, at one point in time, begin to ask themselves if they want to keep on doing things this way or they want to be remembered for just a few things where they excelled to astronomical heights. The HBR blog post is for those people. There are many who are just happy doing different kinds of things and never bothering themselves with that introspective question. They are happy with the many significant contributions, no matter small or big,  that they have made. But some of the more ambitious ones who are not satisfied with whatever they do, no matter how good they are at it, might want to read that blog post. The author has described in a very nice way the importance of priorities, the need for conscious decision making about what we choose to do with our skills.

It happens in graduate research also. Often, graduate students have the option to try many different options in their research. But in order to make the 4-5 years of their graduate research meaningful, they need to carefully consider which options would make them feel more satisfied. A tightly knit story can be made around a few related research questions that have been pursued in detail compared to a loosely knit story that would result from the pursuit of many research questions which often lack an assigned order of priority.