This is one of the EyesWeb patches used for research at Casa Paganini - InfoMus on the analysis of non-verbal social signals in joint music action.
See papers in ftp.infomus.org/pub/Staff/AntonioCamurri for more details.
This patch is an example of synchronous playback of a multimodal recording, recorded by EyesWeb controlling two videocameras, the Qualisys mocap system, and a 6-channel audio input (one for each instrument and a stereo environmental audio).
This patch enables the user to choose in real-time one or more of the following multimodal channels, to play them synchronous:
- up to six audio files,
- one video file,
- the Qualisys TSV Motion-Capture data of the four musicians.
Further, the patch computes in real-time the following features extracted from the MoCap data:
- the direction of the heads of the four musicians,
- the barycenters of each pair of musicians and the overall barycenter
- the polygon formed by the barycenters of the 4 heads of the musicians
The user can
- switch on/off each audio file of musicians and the environmental audio with the 'Audio Channels selection' buttons,
- display or hide the barycenter and the four lines of the directions of the head, by pressing the buttons in the 'Display elements' area.
We thank the Quartetto di Cremona for the sample music performance.
See also: www.siempre.infomus.org
This Patch can also be found in the "nightly" EyesWeb-5.5 SetUp builds, at the following FTP address:
>>> ftp://ftp.infomus.org/Evaluate/EyesWeb/ ... tly/trunk/