sharadsinha

Translational Research: What I learned doing (seemingly) mundane task of video annotation

In Design Methodologies, Education, Embedded Systems, Engineering Principles, Research and Development on November 27, 2016 at 3:04 PM

In the recent past I have been doing some work related to automatic video annotation. Videos that you and I take can be annotated with data about the contents of the video. The contents of the video can mean: objects, their types, their shape, background scene (moving or static), number of objects, static and in-motion objects, color of objects etc. One would like to keep a track of objects as the video progresses. Tracking helps in knowing when an object appeared in the scene and when it disappeared. All of the prior work on automatic video annotation is not really completely automatic [1], [2] etc.. They are semi-automatic at best and manual input and control is still required when annotating using these methods.

While doing this work, I developed a better understanding of some of the so called “automatic object tracking for surveillance” solutions out there in the market.  None of these solutions can ensure a complete hands-off scenario for humans. Humans still need to be involved and there are reasons for that.  At the same time, it is also possible to do everything in cloud (including human interaction) and claim it as “hands off for a user”. In this case, it is simply that the client is paying someone else to provide the service. It is not a stand-alone autopilot kind of system installed in a user’s premises. Real automatic video annotation is extremely hard, especially when the scene can change without any guarantees. If we add “video analytics” i.e. ability to analyse the video automatically to detect a certain set of activities, it again becomes very difficult to propose a general solution. So, assumptions are again made and these can be based on user requirements or can be domain specific (say tennis video analytics at Wimbledon). Here is a system which may be of interest to you: IBM’s Digital Video Surveillance Service and a few others described in the paper titled “Automated visual surveillance in realistic scenarios“.

Most of the research work makes certain assumptions either about the scenes or about the methods they use. These assumptions simply fail in real world scenarios. These methods may work under a “restricted real world view” made using a set of assumptions, but when assumptions fail, these methods become limited in applicability.

I believe this is a critical issue that many researchers who want to translate their work into usable products have to understand. This is where both strong theoretical and practical foundations in a discipline are needed: theory gives the methods and the tools, engineering tells you what can/cannot be done and the two can interact back and forth.

ChatBot: Cost Cutting at the cost of User Experience

In Design Methodologies, Education, Embedded Systems, Science & Technology Promotion and Public Policy on August 31, 2016 at 4:09 PM

Many of you may be familiar with chatbots. For those who aren’t, a chatbot is a computer program designed to have conversation with a human being (wikipedia). So, instead of talking to a real person, you talk to a computer program. The chatbot responds using artificial intelligence methods which can also include using databases. For instance, you can ask a chatbot on a merchant website to show you “shoes of size 5, blue in color, for sports and within 50 dollars”. You don’t have to search using a filter and set various thresholds. The chatbot will process your “textual/verbal (assuming there is speech recognition) input” and get you the results.

This technology is being promoted as the next major innovation to improve efficiency. The problem is that “efficiency” is itself a much abused term: I will go into the details some other time. Companies are being told that by using chatbots they can increase customer satisfaction, reduce manpower, automate customer interaction etc. This to me appears far-fetched conclusions. Human beings like to chat with human beings. That is one reason why Honda’s humanoid robot ASIMO and other similar attempts have failed to make a cut as care takers. They simply can’t replace nurses as of today. Artificial intelligence and care taker robots are too hyped as of now.

I agree that in certain circumstances where a conversation revolves around very structured data and can be very specific, chatbots may be useful. However, if we examine how humans search for something, we will find plenty of randomness in that. Most of the music videos that I have liked, I have bumped into them accidentally. This may not be the case with music aficionados but it is with me and others like me who explore certain things randomly and out of curiosity.

The reason I am writing this post is because I had a recent experience with a chatbot which was contrary to the selling point of chatbot providers or those who buy chatbot technology to improve customer engagement. Read the hilarious conversation below. Let me call the chatbot CB though it actually had a name on the service provider’s website.

Me: I wanted to know something.

CB: I am here to help (This is actually a standard beginning response by CB to every conversation that is started)

Me: I wanted to know how I can register for user authentication.

CB: I have found the following links that may be helpful: link-1, link-2     ( a sequence of hyperlinks)

Me (the hyperlinks were not helpful as I had already seen them under the FAQ): The website says that I will be auto-registered for authentication by March 2016. But this is August 2016. How will I be auto-registered now? What should I do?

CB: I have found the following links that may be helpful: link-1, link-2     ( a sequence of hyperlinks; the exact same answer as earlier)

Clearly, CB had no idea what I was talking about. The service provider had initiated some ad-hoc measures for some time to register users for authentication but had not updated whatever provided data to CB. The service provider had also failed to address the discrepancy in time. I understand that business requirements can lead to such temporary measures but it also means that the client support system must be accordingly updated. Otherwise, it makes little sense.  Apparently, CB also had no mechanism to learn about new business measures on its own either. Needless to say that I was not satisfied with the service. This example demonstrated to me not only some of the limits of chatbot technology but also the carelessness with which businesses go about buying and integrating chatbot technology thinking that it is a good alternative to manpower based customer interaction in order to cut cost and increase customer engagement. On the contrary, approaches like this result in customer dissatisfaction and duplication of work and efforts somewhere else. And this experience was with a well known service provider of citizen services!

Economic Cost of Badly Designed Software Systems

In Design Methodologies, Education, Embedded Systems, Engineering Principles on July 18, 2016 at 10:47 PM

The goal of every design activity, whether in computing or in some other field, is to come up with a system that serves some economic purpose. So, there are software and hardware systems that fly an airplane, that run our cars and power grids etc. In the past, people were distantly connected with these systems. They were mostly passive users with these systems being used for very specific purposes. However, there has been growing emphasis on using these systems, especially software systems, in governance and delivery of public services to citizenry. A lot of these public services are routine in nature and not particularly associated with life threatening risks (unlike power grids, cars etc.). Perhaps this is one reason why so many software systems for the delivery of public services are so poorly designed. Not only the design itself can be poor, but also the testing and validation for these systems is taken very lightly. I also feel that the testing and validation of these systems have to sync with the general life style and attitudes of the citizenry they serve. However, this is perhaps asking for the famous Swiss chocolates when not even a basic candy is available. 😛

Software systems that are used in industrial systems undergo rigorous testing and validation and still they can fail, crash, malfunction and give erroneous results. Studies conducted on the economic cost of such badly designed systems have reported losses of billions of dollars (see here and here). However, if badly designed software is used to provide citizen services, I am not aware of any report that analyzes the associated economic loss. You may be wondering what triggered this post or this conclusion. Well, in India, the government has mandated booking of cooking gas via dedicated hotline numbers which connect to a software system that manages the booking request, generation of customer invoice etc. However, during a recent such exercise, my father received a SMS that the booking has been cancelled (with an even funnier reason stated in the SMS: “Reason: Cancelled Booking”). He did not apply for cancellation. So, he had to drive to the vendor to inquire about this because a number of these vendors are not responsive enough to answer such questions on phone. The vendor replied that it is a software glitch and the booking will be processed shortly; the SMS can be ignored. Not only all this put stress on a citizen but also resulted in precious petrol going down the drain. Now multiply this one incident with another one lakh (a hundred thousand; a very conservative estimate) such cases a month and you get the picture. By the way, there are around 15 crore (i.e. 15 million) consumers of liquefied petroleum gas (LPG, the primary cooking gas in India) (see here).

Apart from the economic cost (whether big or small), such incident create friction and distrust in the system. This is a bigger danger as it cannot be put in monetary terms. Citizens begin to suspect service providers and begin to complain. All of this can be avoided if these social software systems are properly designed and the service providers educated about their proper usage. Unfortunately, this last part seems to be the least of concerns for many people involved in such exercises.