A vision-based postural assessment approach for the unobtrusive detection of physical strain and the prevention of Musculoskeletal Disorders in industrial environments

Assessment and prevention of work-related musculoskeletal disorders (WMSD) is considered a common and critical task towards occupational safety and well-being in working environments ¹. Especially in the manufacturing industry, labor-intensive, repetitive assembly tasks often attribute prolonged, suboptimal working postures. These may lead to physical strain, thus increased risk for WMSD related to various body parts or joints, such as the low or upper back, the neck, the shoulders and the knees. The conditions of WMSD can severely affect the quality of life, especially for adults and ageing workers, related to significant productivity drops and increasing healthcare costs for industrial organizations and the public sector ².

A common practice for organizations is to use conventional ergonomic risk assessment methods, which are based on observations by experts and self-reports by workers using worksheets or checklists to evaluate the working postures, indicators for mental and physical stress and other risk factors related to WMSD and fatigue, making the process subjective, time-consuming, and labor-intensive ⁴. The effective detection of risk factors for WMSD and physical fatigue, in real-time during work tasks or offline, can significantly assist in supporting relevant preventive actions and promoting the occupational safety of workers ⁵. In the context of sustAGE, we are exploring and developing innovative unobtrusive methods for the automatic assessment of physical strain of workers by combining visual and heart rate data, detecting abnormal postures and worker’s fatigue.

Method

The sustAGE case study supports the car manufacturing industry, where workers often work in shifts on an assembly line, while the conveyor belt moves slowly at a constant speed. Each worker is responsible for a specific car assembly task cycle (e.g., welding, assembling), of duration of 4 to 5 minutes and which is repeated during the shift. To detect events causing physical strain and to support preventive actions, we rely on visual information acquired by stereo cameras placed along the production line supporting the automatic assessment of abnormal body postures and the ergonomic risk level of each such occurrence MURI risk analysis approach ⁶.

Specifically, we introduce a non-obtrusive method for automatic postural assessment for each line worker during assembly tasks using visual data acquired by a stereo camera, thus exempting the requirement of body-worn sensors by the workers ⁷. Firstly, we use state-of-the-art 2D and 3D body pose estimation and tracking methods, the OpenPose ⁸ and FORTH MocapNet ⁹ methods, respectively, to compute the body-centered positions and orientations of 16 body joints of each worker in the presence of severe body occlusions during work tasks. Then, we extract view-invariant, 3D skeletal body features ¹⁰ to be used as input to the proposed deep learning-based approach for postural analysis and classification.

The proposed approach supports four types of time-varying, abnormal postures based on the MURI risk analysis method¹¹, commonly used in ergonomic risk analysis in manufacturing.

The selected abnormal postures indicate physical strain imposed on specific body joints: (a) rotation angle of the waist, (b) flexion, stretching angle of the knees, (c) flexion angle of the waist, (d) height of working arms. The risk level was quantified as high, medium or low based on the body joint positions or angles. To classify the type and risk level of an abnormal body posture in a trimmed video clip, we propose a novel approach for postural classification using Spatio-temporal Graph Convolutional Networks ¹², to effectively model the correlation between body parts and assess their motion variations against a support set of prototypical postures.

Integration in the sustAGE system

The high-level semantic information related to the detected body postures and the associated risk for each worker is continuously monitored and aggregated by the system modules to support the vision-based user behaviour monitoring during assembly activities. The proposed vision-based module provides the sustAGE system with information on the detected postural events during or after every assembly task cycle performed by a line worker. This information is further aggregated over consecutive assembly task cycles performed by the same line worker to assess the short- and long-term ergonomic risk for physical strain. The final step is the generation and issue of personalized recommendations to the workers, i.e. to correct postural performance if medium strain condition is detected or take a short break in case of high strain condition, aiming to enhance occupational safety in a preventive manner.

VIDEO