Pose Detection for squat with count using ML Kit

Question

How can I get real time exercise count and angle using ML kit? Here, I check https://ai.googleblog.com/2020/08/on-device-real-time-body-pose-tracking.html for push up and squat exercise count.

I am getting angle by following method :

fun getAngle(firstPoint: PoseLandmark, midPoint: PoseLandmark, lastPoint: PoseLandmark): Double {
    var result = Math.toDegrees(atan2(lastPoint.getPosition().y - midPoint.getPosition().y,
            lastPoint.getPosition().x - midPoint.getPosition().x)
            - atan2(firstPoint.getPosition().y - midPoint.getPosition().y,
            firstPoint.getPosition().x - midPoint.getPosition().x))
    result = Math.abs(result) // Angle should never be negative
    if (result > 180) {
        result = 360.0 - result // Always get the acute representation of the angle
    }
    return result
}

I have added logic from my side but still want help if any proper way I got. What I am doing checking angle every time.

I want to display count and feedback based on user doing exercise.

Answer 1

Here are several things you could try:

(1) You need to ask your users to face the camera in a certain way, e.g. side way might be the easiest for detecting squat and frontal would be the hardiest. You could try something in-between. Also how high the camera is (on the ground, head level, etc..) could also affect the angle.

(2) Then you can calculate and track the angle between body and thigh and the angle between thigh and calf to determine whether a squat is done.

(3) About feedback, you may set some expected angles, and if the user's angle is smaller than that, you could say "squat deeper"...

(4) To get the expected angles, you would need to find some sample images and run the detector on it to get them.

Answer 2

I made a simple demo about squat count https://www.youtube.com/watch?v=XKrZV864rEQ

I just made three simple logical judgments

1. The height of the elbow is higher than the shoulder, otherwise the prompt "please hold your hands behind your head"
2. Standing straight is judged by the angle of the thigh and calf, and the effect is currently not good.
3. Compare the distance between the legs and the shoulders, and the legs should have a certain proportion of the shoulder width, otherwise the prompt "Please spread your feet and shoulder width" Both standing and squatting are judged by taking the human leg length/5 as the minimum movement unit, and the minimum distance from the last coordinate. Because the distance between the person standing and the camera will affect the coordinate ratio

My english is poor, most sentences tanslated by google translate