Python Module for customizing Drawing Styles for MediaPipe Hand Solution API

Task 2 (GSoC 2021)

Aniket Ray's photo
Aniket Ray
·Aug 18, 2021·

2 min read

MediaPipe Hands is a high-fidelity hand and finger tracking solution. It employs machine learning (ML) to infer 21 3D landmarks of a hand from just a single frame. (Read More)

Until now, the Drawing styles (Landmarks and Annotations) could be modified and replaced with custom styles, available only for Mediapipe Hands JS. Now we've brought the same feature to the Mediapipe Hands Python.

hand_landmarks.png

import mediapipe as mp
mp_styles = mp.solutions.drawing_styles

landmark_annotations = mp_styles.get_default_hand_landmark_style()
connection_annotations = mp_styles.get_default_hand_connection_style()

The dictionary landmark_annotations has the landmarks shown in the figure and DrawingSpec as the (key, value) pair.

Similarly, the dictionary connection_annotations has a tuple of landmark connections and DrawingSpec as the (key, value) pair.


Modifying landmark annotations

For modifying the Wrist Landmark (Point 0 in the diagram) with color as (B,G,R) values, thickness as 10, and radius as 5:

custom_color_1 = (B,G,R)
landmark_annotations[
    mp_hands.HandLandmark.WRIST] = mp_drawing.DrawingSpec(
    color=custom_color_1, thickness=10, circle_radius=5)

Modifying connection annotations

For modifying the connection between PINKY_DIP and PINKY_TIP (the line between Point 19 and Point 20 shown in the diagram) with color as (B,G,R) values, thickness as 2,

custom_color_2 = (B,G,R)
connection_annotations[(
    mp_hands.HandLandmark.PINKY_DIP,
    mp_hands.HandLandmark.PINKY_TIP)] = mp_drawing.DrawingSpec(
        color=custom_color_2, thickness=2)

Example

Example

import cv2
import mediapipe as mp

mp_drawing = mp.solutions.drawing_utils
mp_hands = mp.solutions.hands
mp_styles = mp.solutions.drawing_styles

landmark_annotations = mp_styles.get_default_hand_landmark_style()
connection_annotations = mp_styles.get_default_hand_connection_style()

# Modify Annotations as:

custom_color_1 = (102, 22, 21)
connection_annotations[(
    mp_hands.HandLandmark.PINKY_DIP,
    mp_hands.HandLandmark.PINKY_TIP)] = mp_drawing.DrawingSpec(
        color=custom_color_1, thickness=10)

custom_color_2 = (121, 111, 22)
landmark_annotations[mp_hands.HandLandmark.WRIST] = mp_drawing.DrawingSpec(
    color=custom_color_2, thickness=10, circle_radius=5)


# For webcam input:
cap = cv2.VideoCapture(0)
with mp_hands.Hands(min_detection_confidence=0.7,
                    min_tracking_confidence=0.5) as hands:
  while cap.isOpened():
    success, image = cap.read()
    if not success:
      print("Ignoring empty camera frame.")
      # If loading a video, use 'break' instead of 'continue'.
      continue

    # Flip the image horizontally for a later selfie-view display, and convert
    # the BGR image to RGB.
    image = cv2.cvtColor(cv2.flip(image, 1), cv2.COLOR_BGR2RGB)
    # To improve performance, optionally mark the image as not writeable to
    # pass by reference.
    image.flags.writeable = False
    results = hands.process(image)

    # Draw the hand annotations on the image.
    image.flags.writeable = True
    image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)
    if results.multi_hand_landmarks:
      for hand_landmarks in results.multi_hand_landmarks:
        mp_drawing.draw_landmarks(
            image=image,
            landmark_list=hand_landmarks,
            connections=mp_hands.HAND_CONNECTIONS,
            landmark_drawing_spec=landmark_annotations,
            connection_drawing_spec=connection_annotations)
    cv2.imshow('MediaPipe Hands', image)
    if cv2.waitKey(5) & 0xFF == 27:
      break
cap.release()

Github

 
Share this