Demonstrators

This video shows a novel egocentric solution for visual-inertial upper-body motion tracking based on recursive filtering and model-based sensor fusion. Visual detections of the wrists in the images of a chest-mounted camera are used as substitute for the commonly used magnetometer measurements.
In contrast to currently available inertial motion capturing systems, the new method enables successful operation in real industrial environments, which often suffer from severe magnetic disturbances.
Hence, this approach is particularly interesting with regard to a smart user assistance system for industrial manipulation tasks, such as the one developed in the COGNITO project.

This video shows the user tests which were conducted in the SmartFactory for the FP7-Project COGNITO in June 2011. The aim of the tests were to evaluate the capabilities of the COGNITO concept as well as the acceptance and the applicability of Head-mounted-Displays (HMDs) for industrial usage. The results of the studies help the COGNITO consortium to identify the strengths and weaknesses of the first developments, and they will support the next development stage in the COGNITO project.

This demonstrator shows how a network of inertial sensors can be used to track the upper body movements of the wearer.

The user of the Upper Body Tracker wears a network of five interconnected miniature inertial measurement units (IMUs). The IMUs comprise micro-electro-mechanical system (MEMS) accelerometers, gyroscopes, and magnetometers contained in a 18 gram 3x3x1.5 cm casing.

The measurements from the sensor network contains information about the relative motion of the sensors. With IMUs strategically attached to the body, information about the pose and motion of the upper limbs can be obtained. The available information is extracted by comparing the measurements to predictions based on a biomechanical model of the body. The model consists of rigid bodies and restricted joints. This is a simplified but sufficient description of the human body for the purpose. The fusion of the measurements and the model is done in an extended Kalman filter (EKF) which produces joint angles and kinematics to describe the pose and motion of monitored parts of the body.

Video 1

Figure 1

Video 2

The aim of UTC-CNRS research unit in the COGNITO project is to develop a musculoskeletal model of the hand and forearm in order to carry out a biomechanical off-line analysis of the data collected with the on-body sensor-networks. The model is based on the motion capture of real manual movements frequently used in industrial manual tasks (see Video 1).

Then, kinematical data from the motion capture session are used to guide a musculoskeletal model composed of 21 segments, 28 muscles and 20 joints providing 24 degrees of freedom (see Figure 1)

Finally, joint loads and muscle-tendon forces are computed by using a inverse-to-direct dynamics method during the simulation. An example of result is given in Video 2.

Thus, this model will be used not only to document the motion performed by the worker but also to monitor which muscle functions are used and how many joints are loaded. It allows providing major information to optimize and ensure the ergonomy and safety of industrial tasks by subsequently providing an adequate feedback.

In this video we demonstrate how a system can track a workers actions and show them how to complete the current construction of maintenance task.

By observing the positions of both the hands and tools, and classifying these configurations into recognised actions, a workflow can be automatically formed from repeated examples. By demonstrating the sequence and nature of the necessary component actions we can build a model that is able to adapt to a new user, completing the same task, and make explicit a previously implicit workflow sequence. By formalising the demonstrated knowledge of experienced workers a new user can be trained by the system, while existing users can be monitored to prevent safety critical deviations from the task at hand.

Current work at the University of Leeds activity analysis group aims to develop techniques that allow the automatic formation of more advanced workflow models and to increase the detection accuracy of a variety of industrial activities.