sharadsinha

Posts Tagged ‘Academia’

Bias in Peer Review of Research

In Education, Intellectual Property, Research and Development on June 30, 2015 at 6:38 PM

Reviewing research papers for conferences and journals is a part of my job. When I review journal papers, I get to see the comments of other reviewers when the review is complete and comments are sent out to the authors. In the last few months, I have seen review comments which were obviously influenced by the author’s affiliations and past work. I am not saying that the author had any role to play in it because I know it is not possible in those journals. To me, those reviewers sounded as if  they were fans of the authors. Sometimes they adopted a condescending  approach but their comments never reflected the depth of academic rigor. They were more like “Yeah, I know it is a difficult problem. But you guys are well known. So, let me just say yes to your work without concerning myself too much with all that you have stated”. Mind you that mostly it is the decision of the majority that actually counts in a review process. So, if you are reviewer who reviews papers based only on its content, quality, novelty and such other parameters, without caring for author affiliations etc., you might be surprised with such biased reviews. This is one way in which an undeserving paper  gets published successfully. I think that is one reason why most conferences (at least in my field) insist on a blind-review process. The author names and affiliations are not available to reviewers during the review process. This is an example where peer review fails. There are a number of studies and commentaries on the strengths and the weaknesses of peer review which I won’t go into in this post. You can read some sample examples here and here.

The kind of bias that I just mentioned is akin to the culture of fan following in entertainment industry or in sports. You are a fan of someone, you will always support him/her. I have not yet figured out why journals have not adopted a blind review process. I guess if they do, they can reduce the effect of such biased reviews. I am interested in knowing about the review process in your fields and your experiences as a writer, reviewer editor etc. Please feel free to comment.

Advertisements

Research Attitude- What is it?

In Education, Research and Development, Science & Technology Promotion and Public Policy on May 31, 2015 at 1:24 PM

One of the tasks that a faculty member has to perform is to recruit new research students for his or her research group. At most of the institutions, it is entirely up to the faculty member to decide on who should be recruited. Of course, the application may be examined at the department or school level, but the principal responsibility is with the faculty member. In some institutions in some countries, entrance examinations are conducted which are followed by an interview before a student is admitted as a research candidate. I will not go into the pros and cons of these processes but will concentrate on a few characteristics that I think are very important for a student to be admitted as a research candidate. Everybody knows about grades, test scores, recommendations etc. So, I won’t talk about them. Instead I will focus on “research attitude”, which I learned more about (sometimes painfully) when I was involved in hiring students for my current research group. It is difficult to gauge attitude towards research based on grades, test scores etc. These can be used to gauge “potential for research” which I think is different from “research attitude“.

Students with good grades and recommendation letters tend to perform quite well during the discussions. They will talk about their past experiences with pride and would try to convince you on every issue or question that you may ask of them. Sometimes they would try to convince you so much that they tend to forget that the people they are talking to have already been through that process and have at least a couple of years of experience post-PhD. This, I usually treat as a symptom of over-confidence and lack of humility. It can also mean that they have a very high opinion of himself. While this might still be acceptable if they have to work alone, that is not the case these days. Research students typically work in a group and they need to interact with other members. This interaction will inevitably happen during the research program because no one knows everything. A student may need to seek assistance of another student to make progress on his research work. High degree of self-pride and lack of humility do not allow such interactions to be smooth. These characteristics also affect interactions with faculty members, especially those who are more gentle in approach to their students.

Another thing that I have noticed among such students is the lack of patience to study a subject matter in depth. They want to “finish research work” as soon as possible. Unfortunately, they forget or do not appreciate the fact that research work is not the same as doing some other task where all that matters at the end is the output (for instance designing an electric oven). A research work is not valued just based on its output but also based on the methodology, logic and reasoning used to arrive at that output. A lot of time and effort can be wasted when such students present their work to their research advisors.

Therefore, I think that patience, humility and willingness to learn are very important characteristics that a student should possess in order to perform good research.

P.S:This post contains only some of my ideas and in no way represents a comprehensive write up on this subject.

Presentation as a Sales Pitch?

In Education, Engineering Principles, Research and Development on March 25, 2015 at 5:19 PM

It is not uncommon to hear these days: “make your presentation to sell your ideas”, “a presentation is a sales pitch” etc. What was earlier confined mostly to marketing and sales departments is now making its way to other places as well, including academia. Imagine going to attend a talk titled “Truth and Lies About XX” and after spending some time there one realizes that the presentation has no relation to the title at all! The catchy title was just meant to attract people but it lacked substance. Over the past few years, I have come across quite a number of such presentations where the title and the content are very unrelated. The sad part is that most of these presenters walk away with impunity without any member of the audience ever making a remark with respect to the gap between the title and the content. I find this practice not only misleading but also unethical. Most of the time, people come to attend a presentation with a certain notion of it based on its synopsis, speaker’s bio-data  and the title. The title plays a very crucial role in creating excitement. However, I don’t think that it should go so far as to end up unrelated to the content.

I agree with the view that one needs to polish and shape one’s presentation to help the audience follow it; that one needs to choose words and phrases carefully to highlight the main points, one’s contributions etc. However, I don’t agree with a blatant disregard for the audience’s intellect that becomes visible when such titles are chosen. The presenters may say that it was unintentional and that they were only concerned with making it more fancy. However, the fact  that it was unintentional itself says that the presenters did not think deep enough about their target audience.

When people , who are not sales professionals, like engineers, scientists, doctors, lawyers etc. try to become like them, they often forget that there are both good and bad salesmen. That is why they teach sales and marketing in business schools. If it were just a matter of catchy title and pompous claims, business schools would not need to teach the subject. In their effort to sell their ideas, the presenters also forget that the audience has its own mind. In most cases, it won’t simply buy whatever is presented to it no matter how charismatic or fancy the salesman is. Of course, if a presenter knows that a certain audience has a bias, he can use all the tricks to impress the people. In general, I don’t think it is a good idea to keep emphasizing the “sales pitch” version of non-sales related presentations. Instead, what should be emphasized more is to connect truthfully with the audience.

When Economic Forces Influence Universities

In Education, Research and Development, Science & Technology Promotion and Public Policy on January 31, 2015 at 9:28 PM

That universities are being increasingly subjected to economic forces is no longer a surprising news. Many articles have been written about the utility of research done at universities, transforming them into products, restricting funding to research in areas of less economic importance etc. I won’t discuss these in this post as this subject is vast. However, I will highlight one important development that I learned about only recently. I was talking to a professor and we discussed faculty appointments, research areas at his university etc. It came to me as a surprise that most students in his department were opting for courses that led to jobs in companies in a few prominent industries in the region. As a result, the university and the department were increasingly more interested in hiring faculty who had experience in those subjects. This was not always the case with those students. Five to ten years ago, the student population was not skewed this way. As a result, the department had faculty in almost all areas of study/research. Now that the student population had become so skewed, a number of faculty members have very reduced teaching load. In effect, these faculty members are now becoming “surplus faculty”. Needless to say that their areas of research and scholarship are only remotely related or unrelated to areas in which students are getting placed. Consequently, there is little hiring of faculty members in these areas and it may also have an impact on the number of faculty members who get tenure. Is this good for education and research? What should a university do in such a case? I would say that such an effect of economic forces is not good for education and research. However, in a world that increasingly wants to relate every human activity to some sort of economic force, it can be difficult to make a convincing case for hiring/retaining scholars in those disciplines. As far as what a university should do is concerned, it is not an easy question to answer. It requires administration with vision, foresight and strength to deal with such a scenario. Whatever be the case, it seems that the concept of a university is undergoing evolution and there is a need to choose a path that is least damaging to all/most stakeholders.

Chasing Numbers

In Education, Engineering Principles, Mathematics, Research and Development on September 28, 2014 at 8:35 PM

In his book The Tyranny of Numbers: Mismeasurement and MisruleNicholas Eberstadt says, “Although he may not always recognize his bondage, modern man lives under a tyranny of numbers.” Other writers have also commented on how and why numbers alone cannot make us happy and how numbers can be both enlightening as well as confusing if not presented with the right kind of background information. This is very true with research literature, specially those pertaining to engineering and science disciplines where measurement plays a very important role in conveying one’s ideas to convince someone of their importance. I see this everyday when I read research papers. Sometimes I even see numbers and graphs which seemingly do not have any major relation to the central idea of the paper. Such numbers, graphs and tables are byproducts of primary measurement but are probably included with the hope that more numbers, graphs etc. make the papers not only look good but also appear convincing. Given the very short amount of time that most reviewers spend on a paper, it is only sometimes that one finds reviewers commenting on the unnecessary usage of such secondary artifacts. However, a cursory glance does make the paper look good and does give the impression that the authors have spent time analyzing their results (though this may not be the case).

When I see such papers I am reminded of Eberstadt’s statement. It makes me wonder if engineering and science people read papers and books from the field of social science or history or say English literature. Research is conducted even in these disciplines and data is also collected and analysed where needed. However, the force of the argument generally comes from rigorous analysis and reasoning. It is not always driven by the logic that since this paper achieves number X compared to number Y (where say Y is less than X), the proposed methodology is better than the one related to number Y. I have read Diffusion of Innovation by Everett M. Rogers and I have found it to be immensely enlightening. It not only uses numbers but also the force of reasoning. This is so strong that you begin to see what the author is trying to say. I wonder how, say a computer engineering scientist would review a sociology research paper.

Have you ever tried reviewing a paper or a book outside of your major discipline and trying to understand its logical progression?

What is the purpose of a lab?

In Education, Embedded Systems on July 22, 2014 at 9:22 PM

Laboratory sessions at universities form an integral part of curriculum. This is specially the case with science and engineering disciplines. While different disciplines have different requirements regarding what will actually be done in these sessions, a basic question to ask is – what is their purpose? I will discuss with respect to labs for computer engineering curriculum. These lab sessions are meant to give hands on experience to students in working with devices like micro-controllers, microprocessors, field programmable gate arrays (FPGA) etc. Often times, students are given codes (programs in a programming language) written by a teaching assistant (TA) which they are expected to use to program the device. They are expected to program the device using some Integrated Development Environment (IDE). The students may be required to modify these programs based on the lab exercises.

Among other things to learn, I have realized that there is too much emphasis on learning how to use the IDEs. This is not peculiar to one country or university. It seems to be the norm at many places if you look at the lab descriptions available online. It is true that different IDEs look dissimilar (obviously!) and the options that they provide to a user can be in different parts of the graphical user interface (GUI) and under different menus. However, they all follow a basic flow which is essential and relevant to the system or device that they target. Good IDEs are similar in layout and are easy to navigate. Therefore, it should be easier for students to move from one IDE to another after they have learned at least one properly. Besides, it is not so much the IDEs themselves but the different steps in the flow which are more essential to learn. After all, IDEs package different steps, necessary to program such systems and devices, into one nice coherent click-and-run flow.

I believe that lab sessions are meant to complement lecture based learning. How the different steps , algorithms, methods etc. taught in a class come together in a coherent manner in order to enable the programming of such systems, is an important learning outcome. Besides, when working with development boards and evaluation kits, students can learn to navigate through user guides, reference designs, schematics, bill of materials (BOM) files etc. These will seldom be taught in class room, but they form a very important part of an engineer’s life in industry. Lab sessions provide an opportunity for students to relate and expand their class room based learning to what actually goes into designing, building and testing real world systems. I think that should be one of the most important guiding factor for faculty members when designing lab sessions.

Automation & Your Skills

In Design Methodologies, Education on August 12, 2013 at 12:24 AM

During my undergraduate studies, when I first went to a workshop of carpentry, sheet-metal etc. where they teach you how to work with wood and metal and how to make different shapes and objects with them, I did not understand why an Electronics & Communication engineering major was supposed to learn those things. I am sure that there are many who will question that way and this debate will probably never end.  Probably, the best learning outcome of such an exercise is the improvement in our abilities to focus, concentrate, be precise and measure accurately. Also, it helps develop a sense for working with limited resources. After all the quantity of wood or metal that each student is given can be limited and one has to ensure that one gets the work done with that limited quantity. Perhaps such learning exercises should be promoted by also focusing on these learning outcomes.

In today’s world of complex manufacturing, computer aided design tools and computer aided manufacturing have taken over such manual tasks. Once can define and draw any shape using sophisticated design tools and have it carved out by a computer aided manufacturing machine. While these computer aided tools came into existence to deal with complex shapes as well as with the increasing scale of manufacturing, they do not let you have the experience of working with your own hands. So it is quite possible that someone adept at using these computer tools, will fail to bend a sheet of metal at a perfect 90 degrees. So what can be a not so promising consequence of this? Excessive reliance on automation even for simple tasks may lead to a loss of such basic skills. At the same time you run the risk of being unproductive when such computer tools shut down for some reason even if the work does not really require them. Automation is intended to reduce time and manage complexity and scale of operations. It is not supposed to replace acquiring skills by hand where possible.

Another example would be tools like Maple and Mathematica. These are extremely powerful tools to solve mathematical problems. Would you stop learning how to solve a differential equation or how to calculate the area of a triangle by pen and paper method because these tools can do it for you? I guess your answer would be “no” because if your answer is “yes”, you risk a future where people would have forgotten all such knowledge and lost such skills which would instead be built into computer/software systems. A breakdown of such systems would leave you with no option to get back on track!

P.S. The idea for this post came after reading Are We Losing the Secrets of the Masters? This article also mentions about old books which described in detail design of magnets and electromagnets, making neon signs, silver printing, building a forge, blacksmithing etc. Some of these books can now be found here, here and here.  A PDF version of a 1896 book titled “Five Hundred and Seven Mechanical Movements” digitized by Google can be found here and associated websites on animation of engines are this and this.

On Mentors and Mentees

In Education, Research and Development, Science & Technology Promotion and Public Policy on July 5, 2013 at 4:58 PM

The traditional Indian education system holds a teacher with the highest regard. A teacher is supposed to not only educate a student but also shape his/her character given  the fact that a significant amount of time is spent by a student in a school. A teacher with profound knowledge and insight and the ability to inspire students is referred to as “Guru”. Of course, times have changed and the education system (schools to universities) is as much a producer for economic forces as much a product of it. By economic forces, I don’t just mean the market forces but also any force that controls the funds and the resources to be allocated. While this interaction is important and probably unavoidable, it has also opened up the system to the vagaries and at times unreasonable demands of these forces on the education system. The effect can be seen on students as well as teachers. Probably, it is those genuinely interested in a well formed education system who experience more the push of those forces and the pull of their innate desire and commitment to the highest standards of teaching and mentorship. And the outcome is not always pleasant or holistically fruitful.

While the term “Guru” has religious origins, its use has permeated every sphere of knowledge and workmanship; now often used in a diluted sense to refer to someone who possesses exceptional knowledge and skills in a particular area.  The English word “Mentor” is the closest in meaning and spirit. A mentor these days faces the same challenges and being a really good mentor takes a lot of effort. But what exactly constitutes good mentorship? Well, there is no one single answer but an ensemble of insights, challenges and skills which forms the answer. Nature, a world renowned science journal, published an article  in 2007 on what all constitute true mentorship. It is based on an award program, named Nature Awards for Mentoring in Science, that it runs to recognize exceptional mentorship in the field of science. It is an excellent read for those interested in this subject. While it is based on inputs from people associated with science, they are so general that anyone in any other field will find them useful.

PhD vs Work Experience: The Perennial Debate

In Education, Engineering Principles, Intellectual Property, Research and Development on March 9, 2013 at 11:35 PM

Those of you who have ever considered doing a PhD or getting a higher technical degree would have definitely come across this debate on PhD vs work experience. One can find so many articles and opinion posts on this subject. Many of us tend to evaluate PhD and work experience by replacing one with the other. Setting aside financial considerations, we tend to evaluate these two experiences by examining the worth of each when replaced by the other. I think that this approach is improper. PhD  and work experience can be/made to be complimentary to each other. Not all work experiences are of high quality and same is the case with PhD granting institutions. Not all companies are alike just the way standards differ across institutions of higher learning. I would not be debating the pros and cons of PhD or of work experience in this post as that subject merits far greater analysis than what I can put in a blog post. However, taking a broader view, I would say that a PhD program lets you get out of your comfort zone and explore complex, unbounded problems which could be fundamental or applied in nature. It teaches you to learn, examine (and re-examine), critique, argue and persuade using facts and figures. Its not that there are no corporate jobs where one cannot learn these very things. But they are far and few and the degree to which you need to exercise your brain varies across them. As an example, you can be a great lawyer, corporate, civil or criminal, but being a great lawyer is different from being able to comment, analyse, contribute to the very subject of jurisprudence which gives rise to all judicial activities. Another example: you can be an excellent system on chip architect, but being able to get into the depth of power integrity analysis is a different story. Of course you can be a great power integrity analysis engineer too who can apply all sorts of engineering tricks to perform clean power integrity analysis but you need not be able to comment, analyse or examine the principles on which power integrity analysis is based to the same depth as a typical  PhD degree holder would do. The point I am trying to make is that “there is space and need for both kinds of experiences“. They need not be present  to the same degree in one single person. The utility of a PhD and that of work experience depends on many factors. At the end of the day, you do a PhD because you want to explore, find new things or just sit back and critically reflect on the existing things because other people are  busy meeting the demands of the market which has its own challenges!

Value of Scientific Work

In Education, Interdisciplinary Science, Research and Development on February 21, 2013 at 11:03 PM

We, researchers and non-researchers alike, often come across this question: what is the value of scientific work? Is it about publishing papers that expound upon novel ideas? Is it about work that can be commercialized? Is theoretical scientific work more valuable than practical scientific work? Are hands on abilities more valuable than critical thinking? These are perplexing questions with no straight forward answer. Probably, the best way to think of these is to understand that there is space in this world for all. Different approaches have their pros and cons. I think that it is even better to answer them in some context rather than after untying them from any context.

I have seen graduate students, especially PhD students, grappling with the value of their work in the beginning and then towards the end of their PhD. Many find their work of not much value compared to work that translates into something tangible, something that you can touch or see or hear; something that people can use; something that makes them feel that they have created something that did not exist.

Nevertheless, the use of an idea and the idea itself are two different things. They each deserve their own attention. It is also possible that a use case for an idea may not be realized in the immediate future once the idea was formulated. It may take time for the use case to appear and it may not be the same person who developed the idea. For instance, the inventor of laser would have probably never imagined that one day it would be used in laser pointers which we often use during presentations.

I think that when we question ourselves in this way, we need to go back in time and see what the savants of the field have said about these. One great piece is titled “The Value of Science” by Richard P.  Feynman, Physics Nobel Laureate (1965) and both a researcher and a teacher par excellence.  It is so rare to find an excellent researcher who is also an excellent teacher these days. The following paragraph from his public speech will probably be of immense interest to young researchers:

“I would now like to turn to a third value that science has.  It is a little less direct, but not much.  The scientist has a lot of experience with ignorance and doubt and uncertainty, and this experience is of very great importance, I think. …………….  We have found it of paramount importance that in order to progress we must recognize our ignorance and leave room for doubt.  Scientific knowledge is a body of statements of varying degrees of certainty – some most unsure, some nearly sure, but none absolutely certain.”

For a funny and amusing reflection on the different kinds of researchers today ;), see this by Daniel Lemire, a  computer scientist with a big following.