Connontational video streams as representations of on-line discussion contributions


What does it look like when you are in a discussion with 5 (or 500) people connected electronically?

    It is not realistic to have a discussion with 500 people. In my opinion, a real time discussion is only possible with maximum 5 to 8 people. If there are more people, the prevalent mode of communication will be one-to-many, which I wouldn't call interactive anymore, because it means that  most of the people have to listen most of the time. If everybody wants to be active, a larger group will have the strong urge to break down to several sub groups. This means, the foveal attention of a single person in a discussion will be on only 5 to 8 people. Everything else is peripheral. If the discussion is not real time, it is of course possible to be part of more than one discussion. Out of these reasons, my focus is primarily on discussion groups of 5 to 8 people. (And it doesn't matter if we look at it on a normal screen or on a very big screen.)

Intuitive, non-textual representation of discussion content

    To browse such discussion groups, I would like to propose the following: All textual information which each participant writes or has written (say, the email messages) is transformed to a continuous stream of video. The semantic content of the text messages is mapped onto a sequence of videos and pictures. The mapping is realized on the side of the receiver, because a simple text line like "I love cats" means different things to different people. Some would associate this with an ad for cat food, some other would be more negative because they dislike the mentality of cats and would therefore see pictures like a dog chasing a cat. For this purpose, each participant has a personal database of semantic descriptions of pictures and videos. If the participant scans the messages of a group, this textual information is transformed automatically to a user specific multiple stream of video. These video snippets do have mainly connotative meanings, not denotative! And because it is a triadic relation, the video stream generated from a certain collection of email messages is not visualized in the same manner for different users. It might even change over time for a single user, because her/his own personal database of semantic descriptions changes over time too. An example: If I move from Switzerland to Boston, this change of environment will certainly influence my visual connotations to words like "MIT", "Boston", etc. (How to update the database is a not yet solved problem.)

    The background to this idea is the theory behind the semantic differential (mainly by Osgood and Ertel) as well as the semiotic ecological approach of my Psychology Professor at the University of Bern, Alfred Lang.

So if we first look at an ongoing discussion, it might look like that:


Each picture (actually, video stream) represents the transformed collection of already written email messages in the form of a video clip. Of course not only the actual video pictures contribute to the connotative impact of the stream, but also the length of each scene as well as the combination of clips and pictures, means, post production and editing rules apply as well.

This is a single discussion or thread. But as mentioned above, we can participate in more than one group, as long as real time presence is not required. For this purpose, we have a remote control which allows us to zap between discussions like channels on TV. There is an up and down button, and we zap until we see something interesting, until a combination of video streams catches our interest. We can also pre-program our remote control for certain channels, which means, bookmarks to certain threads, and jump directly to a specific group.

On each "channel" = discussion, we can rewind and replay the discussion, which means, previous email messages are transformed to video streams. We have a complete non-linear access to the history of the thread. For this purpose, we have the equivalent buttons of a VCR: play (real time), fast rewind, fast forward (only if we are not already in real time). We also have a counter which shows us where we are, like the potential controls of a time machine:


One of these buttons allows us to jump to the beginning of the discussion, another one let us jump to real time. With the fast forward and fast rewind buttons, we can skim all parts of the discussion backwards and forwards.

The positions of the video windows on the screen are dynamic: they float on the desktop according to which person has replied to which other person recently. The closer two persons are, the more they have replied to or cited each other. This leads to an additional grouping of person representations.


There is also the possibility of layering. The video stream of the most active person is on top, overlaying those of people which have posted messages less recently. Additionally, the video of the person with the most recent contribution at all has a green border.

By clicking on a video, the actual email message pops up, depending on where in the time continuum we are located. Underneath the text boxes, the connotational video interpretations of the email content goes on.



Send me some comments! Stefan Marti Last updated Mar 9 1998.