npm install @mediapipe/tasks-vision

The face plugin uses MediaPipe’s Face Landmarker and exposes ShaderPad-specific uniforms, helper functions, and events for face-driven effects.

import face from 'shaderpad/plugins/face';

const shader = new ShaderPad(fragmentShaderSrc, {
	plugins: [face({ textureName: 'u_webcam', options: { maxFaces: 3 } })],
});

The plugin reads from the textureName texture. Initialize and update that exact ShaderPad texture name, or the detector will have no source to read from.

Options

Events

Subscribe with shader.on(name, callback).

shader.on('face:result', result => {
	console.log(`${result.faceLandmarks.length} faces detected`);
});

Uniforms

Most shaders should use the helper functions below instead of sampling u_faceLandmarksTex or u_faceMask directly.

Helper Functions

If history is enabled, every helper below also has a a trailing int framesAgo argument. 0 means the current analyzed frame, 1 means the previous stored frame, and so on.

nFacesAt

int nFacesAt(int framesAgo)

Returns the number of faces stored for the specified frame. This is useful when history is enabled and you want loop bounds that match an older frame.

faceLandmark

vec4 faceLandmark(int faceIndex, int landmarkIndex)
vec4 faceLandmark(int faceIndex, int landmarkIndex, int framesAgo)

Returns vec4(x, y, z, visibility).

• x, y: normalized landmark position in ShaderPad UV space
• z: MediaPipe landmark depth value
• w: landmark visibility / confidence

Use vec2(faceLandmark(...)) when you only need the screen position.

vec2 nosePos = vec2(faceLandmark(0, FACE_LANDMARK_NOSE_TIP));

leftEyeAt

vec2 leftEyeAt(vec2 pos)
vec2 leftEyeAt(vec2 pos, int framesAgo)

Returns vec2(hit, faceIndex).

• x: 1.0 when pos is inside the left eye region, otherwise 0.0
• y: the matching faceIndex, or -1.0 when no face matched

rightEyeAt

vec2 rightEyeAt(vec2 pos)
vec2 rightEyeAt(vec2 pos, int framesAgo)

Returns the same vec2(hit, faceIndex) tuple as leftEyeAt(), but for the right eye region.

eyeAt

vec2 eyeAt(vec2 pos)
vec2 eyeAt(vec2 pos, int framesAgo)

Checks both eyes and returns vec2(hit, faceIndex). This is the easiest entry point when you only care whether a pixel belongs to either eye.

leftEyebrowAt

vec2 leftEyebrowAt(vec2 pos)
vec2 leftEyebrowAt(vec2 pos, int framesAgo)

Returns vec2(hit, faceIndex) for the left eyebrow region.

rightEyebrowAt

vec2 rightEyebrowAt(vec2 pos)
vec2 rightEyebrowAt(vec2 pos, int framesAgo)

Returns vec2(hit, faceIndex) for the right eyebrow region.

eyebrowAt

vec2 eyebrowAt(vec2 pos)
vec2 eyebrowAt(vec2 pos, int framesAgo)

Checks both eyebrows and returns vec2(hit, faceIndex).

lipsAt

vec2 lipsAt(vec2 pos)
vec2 lipsAt(vec2 pos, int framesAgo)

Returns vec2(hit, faceIndex) for the lip ring only. Unlike mouthAt(), this does not include the inner mouth opening.

mouthAt

vec2 mouthAt(vec2 pos)
vec2 mouthAt(vec2 pos, int framesAgo)

Returns vec2(hit, faceIndex) for the full mouth area, including the lips and inner-mouth hole.

innerMouthAt

vec2 innerMouthAt(vec2 pos)
vec2 innerMouthAt(vec2 pos, int framesAgo)

Returns vec2(hit, faceIndex) for the inner mouth opening only.

faceOvalAt

vec2 faceOvalAt(vec2 pos)
vec2 faceOvalAt(vec2 pos, int framesAgo)

Returns vec2(hit, faceIndex) for pixels that fall inside the face-oval mask. This can be useful for directional effects, but might not cover the entire face in profile.

faceAt

vec2 faceAt(vec2 pos)
vec2 faceAt(vec2 pos, int framesAgo)

Returns vec2(hit, faceIndex) for pixels inside the full face area. Use this when you want a thorough “is this part of a face?” check.

vec2 faceHit = faceAt(v_uv);
if (faceHit.x > 0.0) {
  int i = int(faceHit.y);
  vec2 center = vec2(faceLandmark(i, FACE_LANDMARK_FACE_CENTER));
  color.rgb = mix(color.rgb, vec3(center, 1.0), 0.25);
}

inFace

float inFace(vec2 pos)
float inFace(vec2 pos, int framesAgo)

Returns 1.0 when the point is inside the full face area, otherwise 0.0.

inEye

float inEye(vec2 pos)
float inEye(vec2 pos, int framesAgo)

Returns 1.0 when pos is inside either eye, otherwise 0.0.

inEyebrow

float inEyebrow(vec2 pos)
float inEyebrow(vec2 pos, int framesAgo)

Returns 1.0 when pos is inside either eyebrow, otherwise 0.0.

inMouth

float inMouth(vec2 pos)
float inMouth(vec2 pos, int framesAgo)

Returns 1.0 when pos is inside the mouth area, otherwise 0.0.

inInnerMouth

float inInnerMouth(vec2 pos)
float inInnerMouth(vec2 pos, int framesAgo)

Returns 1.0 when pos is inside the inner mouth opening, otherwise 0.0.

inLips

float inLips(vec2 pos)
float inLips(vec2 pos, int framesAgo)

Returns 1.0 when pos is inside the lip ring, otherwise 0.0.

Landmark Layout

The plugin exposes MediaPipe’s standard 478 face landmarks plus two derived landmarks:

• FACE_LANDMARK_FACE_CENTER: center of the overall face bounds
• FACE_LANDMARK_MOUTH_CENTER: center of the inner-mouth bounds

The most commonly used named constants are:

• FACE_LANDMARK_L_EYE_CENTER
• FACE_LANDMARK_R_EYE_CENTER
• FACE_LANDMARK_NOSE_TIP

For the full MediaPipe landmark index map, use the upstream Face Landmarker model reference.