region_maps

feat.utils.region_maps

Facial AU / ARKit-blendshape region overlays on the MediaPipe-478 mesh.

This is the dense-mesh, non-overlapping successor to the dlib-68 muscle polygons in feat.plotting.draw_muscles and the overlapping muscle map in feat.utils.muscle_to_landmark. It ships two maps:

• au_region_map.json — the 20 FACS AUs, left+right merged per AU.
• blendshape_region_map.json — the spatially-distinct ARKit blendshapes, Left/Right kept independent (ARKit pre-splits ...Left/...Right).

Both are built by a winner-take-all geodesic-Voronoi partition of the mesh: each vertex is assigned to the single nearest region seed by walking the mesh surface (Dijkstra over the triangle-edge graph, capped at TIGHTNESS of the face span), with the eye/mouth aperture rings walling growth off. The partition is non-overlapping by construction.

The per-vertex partition is then resolved to a per-triangle assignment and cleaned so the region silhouettes are whole rather than ragged, while staying strictly on the 478-mesh triangle edges (no subdivision, no curve fitting — the boundaries never become smoother than the mesh itself):

1. each triangle takes the region of its majority vertex (the eye/mouth aperture triangles are permanent holes);
2. boundary-length (perimeter) descent flips boundary triangles to shorten the zigzag region↔region edges, bounded so no region's area drifts past CAP and tiny regions (<= SMALL_REGION triangles) are frozen;
3. the partition is L/R symmetrized with a geometric (triangle-centroid) mirror map;
4. small disconnected specks are removed symmetrically (each speck and its mirror reassigned together), killing stray "flipped" triangles.

The per-triangle assignment is fixed in index space (triangle indices into the bundled canonical tessellation), so the same map overlays a live detected 478-mesh — only the vertex positions change per frame. See feat.plotting.plot_face_regions.

The AU <-> muscle <-> blendshape correspondence is grounded in: * FACS (Ekman & Friesen) — AU -> muscle. * Melinda Ozel's ARKit-to-FACS cheat sheet (melindaozel.com/arkit-to-facs-cheat-sheet). * pooyadeperson's "Ultimate Guide to ARKit's 52 Facial Blendshapes" (anatomy refs).

aperture_triangles(tris, aperture)

Triangles wholly inside an eye/mouth aperture (all 3 verts on a ring) — the OPENINGS. They stay permanent holes (never gap-filled or relabeled).

Source code in feat/utils/region_maps.py

def aperture_triangles(tris, aperture):
    """Triangles wholly inside an eye/mouth aperture (all 3 verts on a ring) —
    the OPENINGS. They stay permanent holes (never gap-filled or relabeled)."""
    return {ti for ti, t in enumerate(tris) if all(int(v) in aperture for v in t)}

build_adjacency(tris, n)

Vertex adjacency over the triangle-edge graph.

Source code in feat/utils/region_maps.py

def build_adjacency(tris: np.ndarray, n: int) -> dict[int, list[int]]:
    """Vertex adjacency over the triangle-edge graph."""
    adj: dict[int, set] = defaultdict(set)
    for a, b, c in tris:
        for u, w in ((a, b), (b, c), (a, c)):
            adj[int(u)].add(int(w))
            adj[int(w)].add(int(u))
    return {k: sorted(v) for k, v in adj.items()}

build_au_region_map()

{AU: {muscles, triangles, mp478_vertices, n_triangles, n_vertices}} — muscle partition merged to AU (left+right share an AU).

Source code in feat/utils/region_maps.py

def build_au_region_map():
    """``{AU: {muscles, triangles, mp478_vertices, n_triangles, n_vertices}}`` —
    muscle partition merged to AU (left+right share an AU)."""
    p = _triangle_partition("au")
    treg, tris = p["treg"], p["tris"]
    region_tris = _region_triangles(treg)
    muscles_by_au: dict[str, set] = defaultdict(set)
    for muscle, (au, _seeds) in MUSCLE_SEEDS.items():
        muscles_by_au[au].add(muscle)
    out = {}
    for au in sorted(region_tris):
        ti = sorted(region_tris[au])
        verts = sorted({int(v) for t in ti for v in tris[t]})
        out[au] = dict(muscles=sorted(muscles_by_au[au]),
                       triangles=ti, mp478_vertices=verts,
                       n_triangles=len(ti), n_vertices=len(verts))
    return out

build_blendshape_region_map()

{blendshape: {au, muscle, side, triangles, mp478_vertices, n_triangles, n_vertices}}.

L/R pairs share one symmetric mesh FEATURE (their seeds merged) grown by the geodesic-Voronoi partition; the per-triangle cleanup divides each sided pair cleanly down the midline and forces exact L/R symmetry. Center shapes keep the whole bilateral feature. Non-overlapping by construction.

Source code in feat/utils/region_maps.py

def build_blendshape_region_map():
    """``{blendshape: {au, muscle, side, triangles, mp478_vertices,
    n_triangles, n_vertices}}``.

    L/R pairs share one symmetric mesh FEATURE (their seeds merged) grown by the
    geodesic-Voronoi partition; the per-triangle cleanup divides each sided pair
    cleanly down the midline and forces exact L/R symmetry. Center shapes keep
    the whole bilateral feature. Non-overlapping by construction."""
    p = _triangle_partition("blendshape")
    treg, tris = p["treg"], p["tris"]
    region_tris = _region_triangles(treg)
    out = {}
    for bs, (au, muscle, side, _seeds) in BLENDSHAPE_SEEDS.items():
        ti = sorted(region_tris.get(bs, []))
        verts = sorted({int(v) for t in ti for v in tris[t]})
        out[bs] = dict(au=au, muscle=muscle, side=side,
                       triangles=ti, mp478_vertices=verts,
                       n_triangles=len(ti), n_vertices=len(verts))
    return out

geodesic_voronoi(seeds_by_region, adj, xy, aperture, max_geo)

Winner-take-all surface partition: assign each vertex to the nearest seed's region by Dijkstra over the edge graph (euclidean edge weights on xy), capped at max_geo, never crossing aperture verts.

Returns {vertex_index: region_name}. Non-overlapping by construction — a vertex is claimed once, by whichever region reaches it first/closest.

Source code in feat/utils/region_maps.py

def geodesic_voronoi(seeds_by_region, adj, xy, aperture, max_geo):
    """Winner-take-all surface partition: assign each vertex to the nearest
    seed's region by Dijkstra over the edge graph (euclidean edge weights on
    ``xy``), capped at ``max_geo``, never crossing ``aperture`` verts.

    Returns ``{vertex_index: region_name}``. Non-overlapping by construction —
    a vertex is claimed once, by whichever region reaches it first/closest."""
    label: dict[int, str] = {}
    dist: dict[int, float] = {}
    pq: list = []
    for region, seeds in seeds_by_region.items():
        for s in seeds:
            if s in aperture or s >= len(xy):
                continue
            heapq.heappush(pq, (0.0, int(s), region))
    while pq:
        d, v, region = heapq.heappop(pq)
        if v in label and dist.get(v, np.inf) <= d:
            continue
        label[v] = region
        dist[v] = d
        for w in adj.get(v, []):
            if w in aperture:
                continue
            nd = d + float(np.linalg.norm(xy[v] - xy[w]))
            if nd <= max_geo and (w not in dist or nd < dist[w]):
                dist[w] = nd
                heapq.heappush(pq, (nd, w, region))
    return label

load_au_region_map()

Bundled non-overlapping AU region map (20 AUs, L+R merged). Each value: {"muscles", "triangles", "mp478_vertices", "n_triangles", "n_vertices"}.

Source code in feat/utils/region_maps.py

def load_au_region_map() -> dict:
    """Bundled non-overlapping AU region map (20 AUs, L+R merged). Each value:
    ``{"muscles", "triangles", "mp478_vertices", "n_triangles", "n_vertices"}``."""
    return _load(AU_MAP_FILENAME)

load_blendshape_region_map()

Bundled non-overlapping blendshape region map (L/R independent). Each value: {"au", "muscle", "side", "triangles", "mp478_vertices", "n_triangles", "n_vertices"}.

Source code in feat/utils/region_maps.py

def load_blendshape_region_map() -> dict:
    """Bundled non-overlapping blendshape region map (L/R independent). Each
    value: ``{"au", "muscle", "side", "triangles", "mp478_vertices",
    "n_triangles", "n_vertices"}``."""
    return _load(BLENDSHAPE_MAP_FILENAME)

project_xy(V)

Frontal x/y projection, flipped so the forehead (v10) is above the chin (v152). V may be the canonical mesh or a detected mesh (N>=153, x/y in cols 0/1). Used both for geodesic distances and for plotting.

Source code in feat/utils/region_maps.py

def project_xy(V: np.ndarray) -> np.ndarray:
    """Frontal x/y projection, flipped so the forehead (v10) is above the chin
    (v152). ``V`` may be the canonical mesh or a detected mesh (N>=153, x/y in
    cols 0/1). Used both for geodesic distances and for plotting."""
    xy = np.asarray(V, dtype=np.float64)[:, :2].copy()
    if xy[10, 1] < xy[152, 1]:
        xy[:, 1] = -xy[:, 1]
    return xy

render_assets(kind='au')

Cached rendering assets for kind in {"au", "blendshape"}:

{"tris": int[M,3], "region_tris": {region: [triangle indices]}, "n_base": 468}

tris is the bundled canonical tessellation; region_tris maps each final region name (an AU, or a sided/center blendshape) to the triangle indices it owns after the cleanup pipeline. Fixed in index space, so a deformed 478-mesh just supplies new vertex positions. See feat.plotting.plot_face_regions.

Source code in feat/utils/region_maps.py

def render_assets(kind: str = "au") -> dict:
    """Cached rendering assets for ``kind`` in {"au", "blendshape"}:

    ``{"tris": int[M,3], "region_tris": {region: [triangle indices]},
       "n_base": 468}``

    ``tris`` is the bundled canonical tessellation; ``region_tris`` maps each
    final region name (an AU, or a sided/center blendshape) to the triangle
    indices it owns after the cleanup pipeline. Fixed in index space, so a
    deformed 478-mesh just supplies new vertex positions. See
    ``feat.plotting.plot_face_regions``."""
    if kind not in _RENDER_ASSETS:
        p = _triangle_partition(kind)
        _RENDER_ASSETS[kind] = {"tris": p["tris"],
                                "region_tris": _region_triangles(p["treg"]),
                                "n_base": p["n_base"]}
    a = _RENDER_ASSETS[kind]
    return {"tris": a["tris"].copy(),
            "region_tris": {k: list(v) for k, v in a["region_tris"].items()},
            "n_base": a["n_base"]}

triangle_adjacency(tris)

{triangle_index: [neighbour triangle indices]} — triangles sharing an edge (two vertices).

Source code in feat/utils/region_maps.py

def triangle_adjacency(tris):
    """``{triangle_index: [neighbour triangle indices]}`` — triangles sharing an
    edge (two vertices)."""
    edge_tris: dict[tuple[int, int], list[int]] = defaultdict(list)
    for ti, t in enumerate(tris):
        a, b, c = int(t[0]), int(t[1]), int(t[2])
        for u, w in ((a, b), (b, c), (a, c)):
            edge_tris[(u, w) if u < w else (w, u)].append(ti)
    nb: dict[int, set] = defaultdict(set)
    for ts in edge_tris.values():
        for i in ts:
            for j in ts:
                if i != j:
                    nb[i].add(j)
    return {k: sorted(v) for k, v in nb.items()}