Posting from the lab today, running my 2nd (out of 3) experiment. Feedback from initial subjects and pilot testing over last weekend and yesterday resulted in the following changes for today:

• Increased the annotation interval from 10 to 20 seconds - We will get fewer data points (120 vs 240), but 10 seconds was found to lengthen the experiment to the point that subjects became bored and agitated.
• Removed the need of spontaneous annotations from the first part of the experiment - It was found that it distracted subjects from the actual game if they were instructed to give spontaneous feedback.
• Instruct the subject to minimise movement to increase recording accuracy

As far as data analysis goes, I plan to use weka (http://www.cs.waikato.ac.nz/ml/weka/) as a tool to analyse my data.

Still technically running on schedule, but I would like to get most of the experiments out of the way and start analysis this week, as I am relatively unfamiliar with the machine learning aspect of things and may need some time to read into this subject. I'll start to analyse these 3 experiments this week and see if i need to find more participants next week.

Running the pilot study

The LATTE group has software that I need to run the second part of my experiment (annotation of the video after physiological recording). Thus I will adapt my original prototype to be used in viewing the media during physiological recording to record spontaneous emotional response.

I'll be running a pilot study this week on myself first. The total experiment time should take no longer than 2 hours.

I've found 3 participants so far, and those tests will be run next week (mid-semester break).

Progress on prototype

Development of the prototype has been making progress. Some code from the Interactive Tutoring video annotation program has been adapted into my prototype. I plan to have a basic interface done by the end of this weekend. This puts me on track to finalise the prototype (week 5, which will be on schedule).

I have also found 3 or 4 subjects suitable for using NRL game footage as stimulus. At this point I will have to start looking into how I will acquire footage for an entire match.

Currently the plan is to use a 40 minute video segment (half a game) followed by 40 minutes of annotation, with 10 minutes changeover and setup. 90 minutes seems reasonable, but there may not be enough data for all different classifier combinations.

If there aren't enough data points, I might see if I lengthen each experiment session or do physiological recording and self-annotation stages in separate sessions. There are issues with either decision:

• By lengthening the recording sessions, I will gain more data points, which will probably increase the experiment's accuracy. On the other hand, if the session is too long (as I experienced in the siento experiments I participated in earlier) the subject can lose interest/concentration which could yield varying results.
• By splitting two stages up, I gain more data points and the subject will be 'fresh' for each session. However, the subject's emotional state may change from each session, especially if they cannot remember their emotional response during the session when annotating footage.

I will finalise a decision after testing the video annotation software against half a match of an NRL game and seeing how many data points are recorded.

Creating a prototype

Over the next few weeks a prototype will be developed for the experiment. Also, the experimental outcome and procedure has been finalised:

1. Pre-annotate a 40 minute video with 20 second intervals between annotations of valence and arousal (3 levels each, for a total of 6 classes. We need at least 20 samples per class, so 120 samples all up) *To ensure we have enough samples, an interval of 10 or 15 seconds may have to be used instead*
2. Record (physiological signals) a subject viewing the video.
3. Have the subject self-annotate the video after first viewing.

Total experiment time for the subject should be around 100 minutes in total (10 minutes setup, 40 minutes initial viewing, 50 minutes self-annotation).

The user interface for the self-annotation phase will have to be very intuitive (1 click to record valence and arousal in under 3 seconds) to shorten the time of the experiment and prevent the subject from losing interest.

The project will be coded up in MATLAB, with the code following the protocols used by other projects in the LATTE group. Schedule for the next few weeks:

• This week (week 3) - Review code for video annotation and start on own prototype
• Next week (week 4) - Work on prototype
• Following week (week 5) - Finish basic functioning prototype

Hopefully I can have a fully functioning prototype by week 7, run the experiments from weeks 8 through 10, analyse data from weeks 9 through 11 and write up the final report in weeks 12 and 13.

Finalising experimental procedure

A basic outline of the experimental procedure has been drawn up:

1. Annotate sports footage with expected levels of valency and arousal
2. Expose test subject to the same footage whilst gathering their feedback on valency and arousal
3. Measure test subject's physiological response whilst watching footage
4. By comparing these results to the pre-annotated results, we can see how well we can predict levels of valency and arousal from this media.
5. If there is enough time, we can use steps 1-4 on other forms of sports footage to see any correspondence in valency and arousal levels between different types of sports (e.g. if a test subject's team scores in any sport, we would at least expect positive valency)

The stimulus to be used will either be 1 half (45 minutes) of a UEFA Europa Cup game (football/soccer), or 1 half of a locally broadcast NRL game (rugby league). This will be finalised by Tuesday (2 days from now), as I'm contacting potential test subjects for their availability with this experiment.

A possible change in stimulus?

I've asked around for possible test subjects, and with the 2010 FIFA world cup happening soon, I might consider either using that or one of the European leagues currently playing as stimulus. Advantages with this stimulus is that it harder to watch entire games live due to the time difference (hence, it is more likely that test participants will not know the outcome of given stimulus). However, as previously mentioned, there aren't too many definite moments in a football match (e.g. scoring a goal) where affect can be annotated.

Another stimulus i could use are UFC (Ultimate Fighting Championship) or Boxing matches. The format of UFC matches (3 or 5 rounds of 5 minutes each with 1 minute of rest between each round) aligns closely with previous studies carried out using video stimulus. Also, the sport is not widely broadcast locally, which makes it easier to find subjects who won't know the result of a match (so their recordings will be more 'authentic'). The downside is that fights are very unpredictable and short; Most fights last for less than 2 or 3 rounds.

Analysing psychophysiological signals

After a week off last week due to a few assignments, I've been making an effort to catch up on work, especially with the progress report due next week. Reading this week was on a text book recommended by Rafa called "Psychophysiological Recording" (Stern, 2001). Currently I am familiarising myself with the equipment used in physiological recording and the analysis of EEG and EMG signals.