Web-based video annotation tool in real-time

A web-based tool to annotate videos involving human motion with multiple annotation track timelines, including text, images and touch-pen.

BACKGROUND

Creative processes of choreographers, or other authors in the performing arts, involve several rehearsal iterations where video-based annotation can significantly enhance the process.

The MotionNotes video annotator was conceived as an auxiliary tool both for the creator’s task in real time compositions and for their archiving intentions.

This technology can be easily transposed to other fields, namely to the sports industry.

TECHNOLOGY OVERVIEW

MotionNotes is a web-based tool to annotate videos involving human motion. It features multiple annotation track timelines, including text, images and touch-pen or mouse drawings, as well as voice annotations (Fig. 1).

Fig. 1 – MotionNotes interface

The proposed software is a single page application (SPA). This tool is being developed in a client server architecture, which is a mandatory requirement, in order to allow users to share their videos and annotations. Web browser is working as the application client since it is compatible with almost every modern device with internet access. This software is implemented using HTML5, CSS3 and JavaScript (ES6) for client side. Regarding the server side, node.js and framework express.js are the technologies in use, as it is illustrated in Fig. 2.

Fig. 2 – Overview of used technologies

FURTHER DETAILS

• Currently it is being developed as a web-based app for the EU project CultureMoves: https://portal.culturemoves.eu/explore
• GOUVEIA, J., FERNANDES, C., AND CORREIA, N. (2016). “The Creation-Tool: Augmenting the Annotation of Performing Arts Rehearsals.” In: FERNANDES, C. (Ed.) Multimodality and Performance. Newcastle upon Tyne: Cambridge Scholars Publishing
• RIBEIRO C., KUFFNER DOS ANJOS, R., FERNANDES C. (2016). “3D Annotation in Contemporary Dance: Enhancing the Creation-Tool Video Annotator.” 3rd International Symposium on Movement & Computing (MOCO’16 London)
• Latest publication (in print): Rodriges, Madeira, Ribeiro, Fernandes and Correia (2019) Multimodal Web based Video Annotator with Real-Time Human Pose Estimation. SSIDEAL.

STAGE OF DEVELOPMENT

TLR 4 – Technology validated in laboratory.

BENEFITS

• Compatible with any device able to run a modern web browser.

APPLICATIONS

This tool was planned to assist any creative processes where human motion is involved, working as a digital notebook for personal annotations. Although it was designed at first to support the work of choreographers, it can of course be of use to several other purposes, such as in competitive sports, social and anthropological field work, creative video and educational settings.

OPPORTUNITY

Universidade NOVA de Lisboa would like to find funding entities for the monetization of this tool (currently in Beta version).

INTELLECTUAL PROPERTY

• Copyright.

SEEKING

• Commercial partner;
• Seeking investment;
• Development partner.

NOVA Inventors

Carla Fernandes