You have worked hard on your project. You searched the literature, learned about the methods, painstakingly designed an experiment, and have almost finished implementing the software, but… what about the logging?

Most students think that logging is easy. Just write some lines on a text file. A couple of hours on the software should do, right? I don’t think so.

Experience has shown me that logging is extremely error-prone, and that paying little attention to it results in incredible loss of valuable information and time, and that most students do not realize how important doing good logging is.

Before I go on let me qualify a little bit what kind of logs I’m talking about. I’m referring mostly to the kind of experiment that you often see in HCI or Experimental Psychology, were there are many participants, and each participant performs many trials, possibly in multiple conditions. This is usually information that is suitably recorded in a simple format like comma separated values (CSV).

Let me state then my five fundamental laws of logging:

1. Log everything you can

Disk space is cheap, your time is expensive. Probably the most common mistake here is not to record enough information in each record because you think it is redundant and a waste of space. For example, why record today’s date in each record of each trial, if they are all the same? I tend to record the same information in each trial anyway, because it is always easy to discard info, but it takes a long time to recover data from different sources (including the name of the file, the creation date that the OS stores etc.). Don’t assume that you will remember where you are storing all that information when the time for analysis comes. Things that I tend to save in each trial record: userId, all condition values for all factors, the number of the trial with respect to the condition, the number of the trial with respect to the cell, the absolute number of the trial within the experiment (and the phase) and, of course, all the dependent variables.

Perhaps the only caveat to this is that all this recording should not negatively affect the performance of your program or the accuracy of the time measurements. If performance and timing accuracy is important, good strategy is to write first to memory, and only save to disk in between trials (or when timing is not an issue).

2. Make your logs self-contained

Name your variables wisely, and always include names on top of the file. This should be quite explanatory, and most analysis programs (e.g., SPSS) will allow you to name the variables automatically from the file. Handy and convenient. A good complementary practice is to have some comments (or a separate file) that provides an explanation of how each variable is recorded, but this requires discipline to maintain, because the logger program tends to evolve. Best to keep your measures simple.

3. Debug, debug, debug

Never assume that your code is recording properly. Simple visual inspection won’t cut it. I have experienced many problems that only became visible after all the experiments were recorded. The best way to avoid problems here is not only to debug, but also to use your pilots to gather realistic data, and analyze it in the same way that you will analyze the overall results from the finished study. This is not only good for your logs, it is also helpful to avoid possible flaws in your statistics (e.g., I do not believe in a posteriori power analysis).

4. Backup, backup, backup

Don’t trust your hard-drive, don’t trust your experimental software. Within your program, save the data to drive as soon as you can (but take into account the comments in point 1). This will allow you to recover from failures in your software. It is actually kind of nice to code your experimental software so that you can restart it again at any given trial within the session. When the experiment is finished, the first experimenter action should be to verify that the data is in the right place, and perhaps making a copy (or send yourself the data to your gmail account – if your data is properly anonymized, of course).

5. Protect yourself against confusion

If something can go wrong during the experiment, it probably will. It is good practice to save the date hour and second of experimental recording in the name of the file that your program saves. This will help you prevent accidental overwrites. Similarly, try to leave as little as possible human intervention for the actual session. For example, I never trust the experimenter -often myself- to select the right name for multiple files depending on the condition. Have the software do something reasonable for you. The only thing that I often make configurable is the participant identifier, so that I can separate real trials from debugging logs.

Hopefully these might be useful to you some time. Write a message below if you agree/disagree or want to add some more advice!