Progress on prototype

Development of the prototype has been making progress. Some code from the Interactive Tutoring video annotation program has been adapted into my prototype. I plan to have a basic interface done by the end of this weekend. This puts me on track to finalise the prototype (week 5, which will be on schedule).

I have also found 3 or 4 subjects suitable for using NRL game footage as stimulus. At this point I will have to start looking into how I will acquire footage for an entire match.

Currently the plan is to use a 40 minute video segment (half a game) followed by 40 minutes of annotation, with 10 minutes changeover and setup. 90 minutes seems reasonable, but there may not be enough data for all different classifier combinations.

If there aren't enough data points, I might see if I lengthen each experiment session or do physiological recording and self-annotation stages in separate sessions. There are issues with either decision:

• By lengthening the recording sessions, I will gain more data points, which will probably increase the experiment's accuracy. On the other hand, if the session is too long (as I experienced in the siento experiments I participated in earlier) the subject can lose interest/concentration which could yield varying results.
• By splitting two stages up, I gain more data points and the subject will be 'fresh' for each session. However, the subject's emotional state may change from each session, especially if they cannot remember their emotional response during the session when annotating footage.

I will finalise a decision after testing the video annotation software against half a match of an NRL game and seeing how many data points are recorded.

Creating a prototype

Over the next few weeks a prototype will be developed for the experiment. Also, the experimental outcome and procedure has been finalised:

1. Pre-annotate a 40 minute video with 20 second intervals between annotations of valence and arousal (3 levels each, for a total of 6 classes. We need at least 20 samples per class, so 120 samples all up) *To ensure we have enough samples, an interval of 10 or 15 seconds may have to be used instead*
2. Record (physiological signals) a subject viewing the video.
3. Have the subject self-annotate the video after first viewing.

Total experiment time for the subject should be around 100 minutes in total (10 minutes setup, 40 minutes initial viewing, 50 minutes self-annotation).

The user interface for the self-annotation phase will have to be very intuitive (1 click to record valence and arousal in under 3 seconds) to shorten the time of the experiment and prevent the subject from losing interest.

The project will be coded up in MATLAB, with the code following the protocols used by other projects in the LATTE group. Schedule for the next few weeks:

• This week (week 3) - Review code for video annotation and start on own prototype
• Next week (week 4) - Work on prototype
• Following week (week 5) - Finish basic functioning prototype

Hopefully I can have a fully functioning prototype by week 7, run the experiments from weeks 8 through 10, analyse data from weeks 9 through 11 and write up the final report in weeks 12 and 13.

Finalising experimental procedure

A basic outline of the experimental procedure has been drawn up:

1. Annotate sports footage with expected levels of valency and arousal
2. Expose test subject to the same footage whilst gathering their feedback on valency and arousal
3. Measure test subject's physiological response whilst watching footage
4. By comparing these results to the pre-annotated results, we can see how well we can predict levels of valency and arousal from this media.
5. If there is enough time, we can use steps 1-4 on other forms of sports footage to see any correspondence in valency and arousal levels between different types of sports (e.g. if a test subject's team scores in any sport, we would at least expect positive valency)

The stimulus to be used will either be 1 half (45 minutes) of a UEFA Europa Cup game (football/soccer), or 1 half of a locally broadcast NRL game (rugby league). This will be finalised by Tuesday (2 days from now), as I'm contacting potential test subjects for their availability with this experiment.