Projects for Goa Assembly Elections 2022

Press coverage of Shri Kunal, Chief Electoral Officer, Goa (during Goa Assembly Elections 2022) felicitating IIT Goa students for IT applications (Voterdost chatbot, AI based webcasting and absentee voter system) developed by them under my guidance for Goa State Elections 2022. The students (Aditi Saxena, Sakar Salunke, Shivom Aghalte, Uppala Sumana, Shivam Jaiswal and Yuvraj Agarwal) designed, developed and deployed these solutions with the help of Dept. of Information Technology, Govt. of Goa. The ideas for these projects were floated by Shri Kunal to bring about more IT enabled services and governance during elections. The chatbot was also released during the visit by the Hon’ble Election Commission of India in January 2022.

The various related tweets can be found at the links: link1, link2, link3, link4

Translational Research: What I learned doing (seemingly) mundane task of video annotation

In the recent past I have been doing some work related to automatic video annotation. Videos that you and I take can be annotated with data about the contents of the video. The contents of the video can mean: objects, their types, their shape, background scene (moving or static), number of objects, static and in-motion objects, color of objects etc. One would like to keep a track of objects as the video progresses. Tracking helps in knowing when an object appeared in the scene and when it disappeared. All of the prior work on automatic video annotation is not really completely automatic [1], [2] etc.. They are semi-automatic at best and manual input and control is still required when annotating using these methods.

While doing this work, I developed a better understanding of some of the so called “automatic object tracking for surveillance” solutions out there in the market.  None of these solutions can ensure a complete hands-off scenario for humans. Humans still need to be involved and there are reasons for that.  At the same time, it is also possible to do everything in cloud (including human interaction) and claim it as “hands off for a user”. In this case, it is simply that the client is paying someone else to provide the service. It is not a stand-alone autopilot kind of system installed in a user’s premises. Real automatic video annotation is extremely hard, especially when the scene can change without any guarantees. If we add “video analytics” i.e. ability to analyse the video automatically to detect a certain set of activities, it again becomes very difficult to propose a general solution. So, assumptions are again made and these can be based on user requirements or can be domain specific (say tennis video analytics at Wimbledon). Here is a system which may be of interest to you: IBM’s Digital Video Surveillance Service and a few others described in the paper titled “Automated visual surveillance in realistic scenarios“.

Most of the research work makes certain assumptions either about the scenes or about the methods they use. These assumptions simply fail in real world scenarios. These methods may work under a “restricted real world view” made using a set of assumptions, but when assumptions fail, these methods become limited in applicability.

I believe this is a critical issue that many researchers who want to translate their work into usable products have to understand. This is where both strong theoretical and practical foundations in a discipline are needed: theory gives the methods and the tools, engineering tells you what can/cannot be done and the two can interact back and forth.

ChatBot: Cost Cutting at the cost of User Experience

Many of you may be familiar with chatbots. For those who aren’t, a chatbot is a computer program designed to have conversation with a human being (wikipedia). So, instead of talking to a real person, you talk to a computer program. The chatbot responds using artificial intelligence methods which can also include using databases. For instance, you can ask a chatbot on a merchant website to show you “shoes of size 5, blue in color, for sports and within 50 dollars”. You don’t have to search using a filter and set various thresholds. The chatbot will process your “textual/verbal (assuming there is speech recognition) input” and get you the results.

This technology is being promoted as the next major innovation to improve efficiency. The problem is that “efficiency” is itself a much abused term: I will go into the details some other time. Companies are being told that by using chatbots they can increase customer satisfaction, reduce manpower, automate customer interaction etc. This to me appears far-fetched conclusions. Human beings like to chat with human beings. That is one reason why Honda’s humanoid robot ASIMO and other similar attempts have failed to make a cut as care takers. They simply can’t replace nurses as of today. Artificial intelligence and care taker robots are too hyped as of now.

I agree that in certain circumstances where a conversation revolves around very structured data and can be very specific, chatbots may be useful. However, if we examine how humans search for something, we will find plenty of randomness in that. Most of the music videos that I have liked, I have bumped into them accidentally. This may not be the case with music aficionados but it is with me and others like me who explore certain things randomly and out of curiosity.

The reason I am writing this post is because I had a recent experience with a chatbot which was contrary to the selling point of chatbot providers or those who buy chatbot technology to improve customer engagement. Read the hilarious conversation below. Let me call the chatbot CB though it actually had a name on the service provider’s website.

Me: I wanted to know something.

CB: I am here to help (This is actually a standard beginning response by CB to every conversation that is started)

Me: I wanted to know how I can register for user authentication.

CB: I have found the following links that may be helpful: link-1, link-2     ( a sequence of hyperlinks)

Me (the hyperlinks were not helpful as I had already seen them under the FAQ): The website says that I will be auto-registered for authentication by March 2016. But this is August 2016. How will I be auto-registered now? What should I do?

CB: I have found the following links that may be helpful: link-1, link-2     ( a sequence of hyperlinks; the exact same answer as earlier)

Clearly, CB had no idea what I was talking about. The service provider had initiated some ad-hoc measures for some time to register users for authentication but had not updated whatever provided data to CB. The service provider had also failed to address the discrepancy in time. I understand that business requirements can lead to such temporary measures but it also means that the client support system must be accordingly updated. Otherwise, it makes little sense.  Apparently, CB also had no mechanism to learn about new business measures on its own either. Needless to say that I was not satisfied with the service. This example demonstrated to me not only some of the limits of chatbot technology but also the carelessness with which businesses go about buying and integrating chatbot technology thinking that it is a good alternative to manpower based customer interaction in order to cut cost and increase customer engagement. On the contrary, approaches like this result in customer dissatisfaction and duplication of work and efforts somewhere else. And this experience was with a well known service provider of citizen services!

Economic Cost of Badly Designed Software Systems

The goal of every design activity, whether in computing or in some other field, is to come up with a system that serves some economic purpose. So, there are software and hardware systems that fly an airplane, that run our cars and power grids etc. In the past, people were distantly connected with these systems. They were mostly passive users with these systems being used for very specific purposes. However, there has been growing emphasis on using these systems, especially software systems, in governance and delivery of public services to citizenry. A lot of these public services are routine in nature and not particularly associated with life threatening risks (unlike power grids, cars etc.). Perhaps this is one reason why so many software systems for the delivery of public services are so poorly designed. Not only the design itself can be poor, but also the testing and validation for these systems is taken very lightly. I also feel that the testing and validation of these systems have to sync with the general life style and attitudes of the citizenry they serve. However, this is perhaps asking for the famous Swiss chocolates when not even a basic candy is available. 😛

Software systems that are used in industrial systems undergo rigorous testing and validation and still they can fail, crash, malfunction and give erroneous results. Studies conducted on the economic cost of such badly designed systems have reported losses of billions of dollars (see here and here). However, if badly designed software is used to provide citizen services, I am not aware of any report that analyzes the associated economic loss. You may be wondering what triggered this post or this conclusion. Well, in India, the government has mandated booking of cooking gas via dedicated hotline numbers which connect to a software system that manages the booking request, generation of customer invoice etc. However, during a recent such exercise, my father received a SMS that the booking has been cancelled (with an even funnier reason stated in the SMS: “Reason: Cancelled Booking”). He did not apply for cancellation. So, he had to drive to the vendor to inquire about this because a number of these vendors are not responsive enough to answer such questions on phone. The vendor replied that it is a software glitch and the booking will be processed shortly; the SMS can be ignored. Not only all this put stress on a citizen but also resulted in precious petrol going down the drain. Now multiply this one incident with another one lakh (a hundred thousand; a very conservative estimate) such cases a month and you get the picture. By the way, there are around 15 crore (i.e. 15 million) consumers of liquefied petroleum gas (LPG, the primary cooking gas in India) (see here).

Apart from the economic cost (whether big or small), such incident create friction and distrust in the system. This is a bigger danger as it cannot be put in monetary terms. Citizens begin to suspect service providers and begin to complain. All of this can be avoided if these social software systems are properly designed and the service providers educated about their proper usage. Unfortunately, this last part seems to be the least of concerns for many people involved in such exercises.

User Interface (UI) Design for Computer Systems

I believe that a proper User Interface (UI) design for computer systems is a must. All the technical, scientific and engineering wizardry that engineers may do while writing code and developing the system comes to a naught if the user interface is not human centric. There are countless examples of poor UI designs and one can find it at even those places which excel in research and development. Will it not be surprising if you happen to visit a renowned research lab or university where it takes time for a user to figure out how to use a machine to update some data on a card? It can be a bewildering experience.

When you go to an ATM machine to withdraw money, you are actually interacting with the machine through a user interface (UI). You insert your card, provide security details and choose options from the on-screen menu. This is all fine as long as you understand the languages used by the machine. These and similar other machines like queue number dispensers, ticket vending machines etc. are often used these days.

Among other things, I consider the choice of language as the most important decision that a user should be allowed to make before he provides other inputs to the machine for processing. If the user does not understand the current language and it takes a while to figure out how to set the language, it leaves the user with a bad experience.

The very-first view on the screen of such a machine should be related to the selection of a language. Now, the message there could be “Choose a language and a list of language is also shown simultaneously”. Of course, this assumes that  the user would understand the message “Choose a language” written in one of the supported languages. But I think a better option is to simply show all the supported languages without any message.  The user can then simply select one and thereafter the usual process follows. Such a design would work best with ATM machines, ticket vending machines etc. These are machines with which a user interacts instead of simply relying on it for information. For instance, the speedometer display of your car just provides you with information; you do not interact with it. For such interfaces, other UI designs will be suitable.

The problem with UI design in many systems is that it is done by engineers and managed by managers who have little training in this sphere or simply do not care to think as much as they would while doing software and hardware design for the system. This results in a clunky and sometimes dangerous user interface. Here are some examples of poor UI designs and their effects.

So, the next time when you do a UI design, please have some consideration for the poor users and let them have an easy life! 😉

Do you read User Guides?

I am a member of LinkedIn and like many of you am also a member of quite a few LinkedIn groups. The good thing about LinkedIn groups is that the discussions remain professional in tone and content. This is why I like them compared to discussions on other social media platforms where they can vary in tone and content from the most professional to the most ridiculous. In a discussion on such a LinkedIn forum meant for engineers, someone admitted that very few engineers or users of tech tools read the user guides. This is not far from reality. I have seen this when I interacted with practicing engineers on a more regular basis than now. I also see it in academic life.

Personally, I find user guides of development boards, software and hardware tools extremely useful. Reading them once gives me enough confidence in extracting the best out of these tools. For instance, user guides of FPGA vendor providers are very helpful and I am more confident about my design after having referred to the user guide at least once though often these guides can be voluminous. I guess the verbosity of these guides is one main reason why people don’t feel like reading them. The other reason, I think, is the propensity of many practicing engineers, graduate students and others to get  their hands dirty as soon as possible. They want to write code, design a circuit, run simulations etc. without getting bored reading these guides. While this enthusiasm to start working is worth appreciation, ignoring the “reading” part leads to problems later on in the product development process, research methods and has the potential to creep into the results. Basically, this haste leaves one vulnerable to questioning at a later stage. Sometimes this can prove very costly as well especially if it is related to product development. Of course one can always talk about pressure for results from managers, supervisors, customers etc.; this is not a very good excuse. Good managers etc. also understand the importance of being abreast with background information.

Is this issue observed more in the engineering industry than say banking or insurance sectors or for that matter safety critical engineering domains? Perhaps. Engineers take great pride in fixing things. They can use patches for software, make new releases, change components or simply replace the product.  However, bankers and insurers cannot do much once money is gone. The fear of losing money is too great to sustain the dislike for reading guides, whitepapers etc. Similarly those involved with safety critical engineering domains are more mindful about liability issues that aversion to poring over thick user guides is probably a non-issue.

One can also argue that  the presentation style of many user guides is quite boring. I agree when you compare with things that provide “instant thrill” thus leading to a desire to know more. User guides do not provide that thrill but writing code, experimenting with a development board etc. does give a lot of thrill to many engineers. Nevertheless, when it comes to getting a job done properly, there is no other choice but to sweat it out! 🙂

Communication Skills for User Interaction

I recently used the IVR (Interactive Voice Response) system of an organization tasked with issuing identity cards to citizens. An IVR system is supposed to improve customer experience besides helping the organization in managing complaints,requests etc. Therefore,it plays a very important role. An IVR system comprises one or multiple menus which are read out to a caller who then has to select one of the options. Interestingly, sometimes there are just so many options that one just loses tracks. It also happens when the “menu items” do not sound similar to what the called user has in mind. So what do you do? You just navigate to the one that sounds closest  to what you had in mind and hope that it will solve your problem or you wait for the option to talk to a staff on the other side!

The IVR system that I referred to earlier had peculiar issues. If you selected the option that said something similar to “I would like to know if I need to reapply”, you would expect it to prompt you to give some information based on which you would be told “whether or not” you should reapply. However, this IVR system would give the response similar to “Please do not reapply as it is not desirable to have two identity numbers”. Now how on earth is that helpful?

The IVR system of a prominent smartphone company would give some even more hilarious responses. When you call the number hoping to find a relevant menu or speak to someone, it would tell you something similar to “Please visit our website to resolve your issue”. Now imagine that for some reason you do not have access to internet, then is that response of any help? Absolutely not.

This begs the question about the people (engineers, manager, UI guys etc.) involved in designing IVR systems. Do they really understand how people use a language to communicate? Do they spend some time understanding the common phrases that people use to refer to their issues and then distill a subset that they can use in their system? Do they spend time brainstorming proper responses to different kinds of questions? A good IVR system is not just a software development exercise. It involves understanding about communication and is affected by the communication skills of the team doing the design. Similarly, an IVR system with multiple menus and sub-menus can get difficult to navigate especially for old people. Does the design team understand who the end users are and what kind of communication skills they have? I think these are important questions that should be considered. An IVR system is supposed to provide an easy solution to a user. It should be simple, straight and elegant.

Automation & Your Skills

During my undergraduate studies, when I first went to a workshop of carpentry, sheet-metal etc. where they teach you how to work with wood and metal and how to make different shapes and objects with them, I did not understand why an Electronics & Communication engineering major was supposed to learn those things. I am sure that there are many who will question that way and this debate will probably never end.  Probably, the best learning outcome of such an exercise is the improvement in our abilities to focus, concentrate, be precise and measure accurately. Also, it helps develop a sense for working with limited resources. After all the quantity of wood or metal that each student is given can be limited and one has to ensure that one gets the work done with that limited quantity. Perhaps such learning exercises should be promoted by also focusing on these learning outcomes.

In today’s world of complex manufacturing, computer aided design tools and computer aided manufacturing have taken over such manual tasks. Once can define and draw any shape using sophisticated design tools and have it carved out by a computer aided manufacturing machine. While these computer aided tools came into existence to deal with complex shapes as well as with the increasing scale of manufacturing, they do not let you have the experience of working with your own hands. So it is quite possible that someone adept at using these computer tools, will fail to bend a sheet of metal at a perfect 90 degrees. So what can be a not so promising consequence of this? Excessive reliance on automation even for simple tasks may lead to a loss of such basic skills. At the same time you run the risk of being unproductive when such computer tools shut down for some reason even if the work does not really require them. Automation is intended to reduce time and manage complexity and scale of operations. It is not supposed to replace acquiring skills by hand where possible.

Another example would be tools like Maple and Mathematica. These are extremely powerful tools to solve mathematical problems. Would you stop learning how to solve a differential equation or how to calculate the area of a triangle by pen and paper method because these tools can do it for you? I guess your answer would be “no” because if your answer is “yes”, you risk a future where people would have forgotten all such knowledge and lost such skills which would instead be built into computer/software systems. A breakdown of such systems would leave you with no option to get back on track!

P.S. The idea for this post came after reading Are We Losing the Secrets of the Masters? This article also mentions about old books which described in detail design of magnets and electromagnets, making neon signs, silver printing, building a forge, blacksmithing etc. Some of these books can now be found here, here and here.  A PDF version of a 1896 book titled “Five Hundred and Seven Mechanical Movements” digitized by Google can be found here and associated websites on animation of engines are this and this.

A Tale of Two Samsung Galaxy S4s

When you are in school or college, you are taught about the best ways to do things. It is generally about a point solution. Alternatives are rarely discussed in detail. One almost always looks for the best answer, the best method, the best algorithm. When you begin to work  for a company, you almost always realize that the best solution is not what one is always looking for. Time and market pressures play a role in choosing solutions. You can choose a solution that suits the “taste of the target market“. When you serve more than one market, then it becomes interesting. Would you want to choose two different solutions for two different markets for the same product? This is one of the reasons that analysts cite regarding what Samsung has done with its Galaxy S4 smart phone. While the US and the Korean versions appear identical on the outside, they use quite a number of different components. Their processors, wireless and image processing architectures are different. Supposedly, the Korean version is faster and has a longer battery life because it uses  Samsung’s Octacore Exynos 5 processor which has an architecture (read here) that helps to attain a balance of power efficient and performance more than the Qualcomm Snapdragon processor in the US version. iSuppli’s IHS Teardown Service reveals all the component level differences between the two designs here.

A more plausible reason for the difference in the two architectures is the fact that the LTE bands supported by mobile operators in US and Korea are different (see here). The two processors (essentially system on chips in this case) may not support both the LTE bands. However, it does illustrate an important point related to engineering product design. It shows that you can design the same product with different architectures. While not related to S4, this analysis reminds me of regulations in certain countries which make it compulsory for a manufacturer to source components from local suppliers for products to be sold in the local market.  An example is here. Therefore, as a manufacturer you can end up with different components in different markets for the same product.

I used to think that a consumer electronic item sold in different countries used the same components. That myth now stands broken! While you can easily spot the differences in software, prominent being the language used in user interface, it is not easy to spot differences in hardware.

The World as a State Machine

A state machine is basically a model of computation which helps one analyze the effects of input on a system. This system can remain in different states throughout its life cycle though in only one state at a time. It can transition from one state to another depending on some input. Every state machine has a start state and it progresses from there to other states eventually leading to an end state. Note that it is possible to reach the end state from any intermediate state as well as the start state. It depends on the system being modeled. Also, the output of each state may depend on the current state as well the inputs to that state.  Thus state machines model reactive systems i.e. systems which react. A good description of state machines can be found here. Note that the description there is related to finite state machines, which are so called because they have finite number of states. State machines are used in different fields of study not just electrical or computer engineering. They are used in biology, mathematics, linguistics etc. They also have different variants each trying to capture some additional parameters of a system which I would not go into. You can read about them at the link mentioned earlier.

I was wondering if the world can be modeled as a state machine. I think that the world in fact is a state machine except that its end state is unknown. Those with absolute faith in cosmological physics would state that the “Big Bang” can be considered as the start  state. Those with religious views might consider something else as the start state.  The beauty of this world being considered as a state machine lies in the fact that it does not matter whether you believe in science or not. It does not matter whether you have more of a religious bent of mind and would like to see the world from a religious or theological perspective or whether you want to see it only from a scientific standpoint. Either way, the world can be modeled as a state machine. You get to choose the start state depending on which viewpoint you are more comfortable with. In both the cases, the world is in fact a reactive system. It can even be considered as an aggregation of interacting state machines where each state machine can represent the economic, social, political, religious and scientific state of the world. And nobody would deny that all these concepts influence each other. Every electrical or computer engineering student studies about Moore and Mealy state machines. To them, the world is probably a Mealy state machine though not strictly so: the outputs in any state that this world resides in is dependent not only on the current inputs but also on the current state. If we look around us, it sounds so true,   does it not? However, this state machine is extremely complex!