Standing on the shoulders of… nobody? – Connected Research, Related Work and Discussion sections in Human Computer Interaction

Disclaimer: this is purely personal opinion

This is a great time for Human-Computer Interaction. It seems that society and computer science’s awareness of how important it is to find new ways of interacting with the real world, engaging with our data, and finding our way in virtual communities keeps increasing. However, there seems to be an increasing discontent about some aspects of the community and the work it is generating. Some people complain about how difficult it is to publish real systems that perform real tasks and evaluate real scenarios. Others have suggested that forcing research to include empirical studies can kill new promising ideas in our field, but also that empirical research rarely replicates previous results, sometimes because it is considered incremental.

I agree with most of these concerns. I believe that our role as researchers in this community is to find the new, disruptive technologies, as well as to establish true and tried ways to improve people’s way of creating and interacting with their new interactive space. In my opinion, however, one of the main problems of this community is that work is disconnected. Probably one of the reasons why our empirical knowledge does not seem lead anywhere is because so many authors have no interest in taking others’ research further, but rather to publish their bit as fast as possible, or find some new original bit, without regard for how this advances the overall field.

I believe this is reflected in much related work sections of new works (particularly of interaction technique papers), probably as a symptom of how papers are written. I am tired of reviewing papers that have very weak related work sections that go like this:

  • Our work relates to [very famous early reference]
  • Some people have done work in the general area [X,Y,Z]
  • Author P did AB (superficial discussion of what they did), but not C, and their work is therefore
  • Author Q did BC (same problem), but not A
  • Author R did AC (same problem), but not B
  • We did ABC (and therefore our work is worth publishing – and better than P,Q, and R’s)

Many related work sections like this get published every day (and much worse). I myself might be even guilty of some of these. The main problem is that this is very superficial. These related work sections are often written only after the research is done, and they are written only because reviewers will not accept a paper if it does not mention certain basic references, and sometimes their own work.

However, this is not enough. The problem is that these connections with previous work are very weak. It should not only be about connecting nodes in the networks of publications. It should be about the quality of these connections. These related work sections mention other research, but they do not connect to other research meaningfully; they do not build knowledge. We are trying to find our little hole, the only clear snow where nobody has stepped before (and step on it), but not to build something upon each others’ work. How are we supposed to build a strong discipline, to stand on the shoulders of giants?

Perhaps one of the ways of solving this problem is to ask more of our related work sections. Related work sections should include all relevant work, and help the reader understand how the presented work builds upon it; moreover, related work sections should be paired with a part of the discussion, where the connection from previous work and current work is explained and developed. How does our work contradict previous findings? How does it support previous speculation? What are its possible limitations?

A related work section like this probably needs to be written before much of the research of the paper is done. In my opinion, this would prevent of bad papers to even come to life, will increase the connectedness of our work, and save a lot of time to researchers and students that find (to their dismay) that after all the research and a study, what they have done is a (bad) replication of previous work, which does not teach us much. I am also inclined to think that better connection with previous research would even increase our incentive as a community to replicate research: if we are to stand on each others’ shoulders, we will want to know that the fundamentals are solid, and this will require replication and an improvement of our methods.

I’m I crazy? Is this only a rant? Too prescriptive, not accurately descriptive? Will this all lead to disaster? or does people even care? Any examples of good related work sections?

Advertisements

4 thoughts on “Standing on the shoulders of… nobody? – Connected Research, Related Work and Discussion sections in Human Computer Interaction

  1. I agree but . . . if all papers need 2-3 pages listing all related work and how the work fits in, a full description of the proposed technique, an empirical study comparing to other techniques with a proper description of all the factors/choices/decisions made within the study so that it is repeatable, plus the highly motivation introduction, and full a page for all those references suddenly our conference articles need to be at least 12-14 pages long and I don’t think either reviewers or authors really want that.

    I wonder if the problem is more that we rely too heavily upon (and expect too much from) conference papers and don’t often enough take the time to write the proper, in-depth journal articles to back the ideas up.

    Reply
  2. I agree with John. This is a sensible idea but impractical for conferences. But I would expect no less for journal submissions. Perhaps this is a good time for me to publicise our forthcoming special issue of the IJHCS? 🙂

    Reply
  3. Hey Miguel. Nice ideas. The challenge I see here is actually in reviewing. Too often, we get back reviews that say, “what about X’s work on Y”. I think it’s very rare that I get back a set of reviews that don’t add 2-3 new references to my work, despite what I thought had been a relatively exhaustive literature review. In fact, I recently got that in a paper for which I had cherry-picked my previous work from my 200-reference comprehensive paper.

    It’s hard to go for depth and quality when we know that reviewers are simply looking to see if we’ve cited papers X, Y, and Z, not if we’ve understood the context of our work.

    Perhaps we should be giving different instructions to our reviewers in this context?

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s