More Works from Our Lab

Face Re-morphing

Differential morphing attack detection via feature-space similarity changes.

IJCB 2026

XSA-MAD: Cross-modal Semantic Alignment for Morphing Attack Detection

Jie Jin, Mahiro Tokumasu, Yu Makino, Masakatsu Nishigaki, Tetsushi Ohki
Shizuoka University and RIKEN AIP ICIP 2026
Paper Supplementary arXiv BibTeX
Overview of the XSA-MAD framework

XSA-MAD builds an attribute-aware textual semantic space and aligns visual features to detect morphing attacks across unseen generation methods.

Abstract

Morphing attacks pose a serious threat to face recognition systems. Existing image-based morphing attack detection methods often generalize poorly to unseen generation techniques because they rely solely on visual cues.

XSA-MAD is a CLIP-based multimodal framework that explicitly models semantic inconsistencies between bona-fide and morphed faces. Morphing concepts are decomposed into four interpretable attributes: identity, facial geometry, texture, and consistency. The image encoder is progressively aligned with this discriminative textual space, resulting in a unified semantic representation that captures generation-invariant, concept-level discrepancies.

Experiments on MAD22 and MorDIFF, following training on SMDD, demonstrate strong generalization across diverse morphing principles. XSA-MAD achieves an equal error rate of 2.92% on GAN-based morphs and consistently outperforms existing methods under high-fidelity generative attacks.

Method

Stage 1: Textual Semantic Space

The CLIP text encoder is adapted with LoRA to construct discriminative class prototypes from structured prompts.

Stage 2: Visual Alignment

The learned prototypes are fixed, and the CLIP image encoder is adapted so image features align with the established textual semantic space.

The class prototype for class \(c\) is defined as the mean direction of normalized textual features:

\[ \mathbf{p}_c = \frac{1}{N_c} \sum_{i=1}^{N_c} \frac{\mathbf{t}_i}{\lVert \mathbf{t}_i \rVert} \]

Attribute anchors and prototype margins structure the textual space before visual alignment:

\[ \mathcal{L}_{\mathrm{attr}} = \sum_{k,c,\mathbf{x}} [m_{\mathrm{attr}} + \Delta\cos_k(\mathbf{x})]_+ + \sum_k [m_{\mathrm{attr}} + \cos(\mathbf{a}_{c'}^k, \mathbf{a}_c^k)]_+ \] \[ \Delta\cos_k(\mathbf{x}) = \cos(\mathbf{x}, \mathbf{a}_{c'}^k) - \cos(\mathbf{x}, \mathbf{a}_c^k), \qquad \mathcal{L}_{\mathrm{margin}} = \max(0, \cos(\mathbf{p}_0, \mathbf{p}_1) + m_{\mathrm{proto}}) \] \[ \mathcal{L}_{\mathrm{ca}} = \bigl(1 - \cos(\mathbf{p}_0, \bar{\mathbf{v}}_{\mathrm{i}})\bigr) + \bigl(1 - \cos(\mathbf{p}_1, \bar{\mathbf{v}}_{\mathrm{i}})\bigr) \]

During image-to-text alignment, image features are scored against the fixed class prototypes and optimized with a CosFace-style objective:

\[ z_{i,c} = \cos(\mathbf{v}_i, \mathbf{p}_c) \] \[ \mathcal{L}_{\mathrm{cls}} = -\log \frac{ \exp\bigl(s \cdot (z_{i,c} - m_{\mathrm{img}})\bigr) }{ \exp\bigl(s \cdot (z_{i,c} - m_{\mathrm{img}})\bigr) + \exp\bigl(s \cdot z_{i,1-c}\bigr) } \]
Component Bona-fide Morph
Base Context A face captured from a live subject. A generative morph spoof.
Identity Person-specific identity without overlap. Blends signatures from distinct individuals.
Geometry Plausible human alignment across all regions. Geometric ambiguity across key features.
Texture Fine-structure texture persists realistically. Texture resolution suppressed by blending.
Consistency All biometric signals support one unified subject. Identity mapping appears contradictory.

Key Results

6.13%

Overall EER

Best overall EER among the evaluated stage-wise and joint optimization variants.

2.92%

MIPGAN-II EER

Strong performance on high-fidelity GAN-based morphs.

1.88%

MorDIFF EER

Low error on diffusion-based morphs with strict operating points.

Semantic Space Analysis

Default CLIP textual semantic space
Default CLIP configuration.
Stage 1 optimized textual semantic space
Stage 1 optimized configuration.
Textual semantic space without the identity attribute
Without identity.
Textual semantic space without the texture attribute
Without texture.

Paper

Supplementary material: static/pdfs/supplementary.pdf

BibTeX

@inproceedings{jin2026xsamad,
  title = {XSA-MAD: Cross-modal Semantic Alignment for Morphing Attack Detection},
  author = {Jin, Jie and Tokumasu, Mahiro and Makino, Yu and Nishigaki, Masakatsu and Ohki, Tetsushi},
  booktitle = {Proceedings of the IEEE International Conference on Image Processing (ICIP)},
  year = {2026}
}