Need help with facemoji-kit?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

facemoji
151 Stars 5 Forks 7 Commits 2 Opened issues

Description

Face tracker with blend shapes coefficients, 3D head pose and dense mesh in real-time on iOS, Android, Mac, PC and Linux.

Services available

!
?

Need anything else?

Contributors list

No Data

Facemoji Kit

Bring Facemoji into your app and let users express themselves more comfortably—or their alter ego—in fun, new ways with privacy. Unlike lenses or AR filters that are one-size-fits-all, Facemoji avatars can be personalized.

Facemoji Kit provides an accurate 3D head pose in world space, 51 facial expressions, and optionally a dense 3D mesh of a person’s face. Facemoji Kit uses blend shapes/morph targets (FACS and ARKit-compatible) to animate a 2D or 3D character in ways that follow the user’s facial expressions in real-time.

How it works

RGB Input → Magic NN ✨ → Blend shapes weights and pose → Avatar modelling and rendering

Key features

  • Scalable facial tracking
    • A configurable trade-off between accuracy and speed
      • e.g. low-fidelity 2D vs. high-fidelity 3D avatars
  • 1-2MB
    ML model size
  • 3D reprojection to the input photo/video
  • No avatar mode similar to Snap's Lenses or Instagram's AR filters
  • Supported platforms
    • Unified iOS/Mac OS X Kit with internal platform optimizations
    • Platform suited API and packaging
    • Basic Windows and Android support, full Windows and Android Kits coming soon

Input

RGB image or video

Output

  • FACS and ARKit-compatible blendshapes
  • Gaze tracking
  • Head position in 2D and 3D
  • Head rotation and scale in world coordinates
  • On desktop or high-end phones:
    • Dense mesh (up to
      50 000
      keypoints)
    • Face segmentation (per pixel)
    • UV and depth estimation ## Tracking speed

Higher accuracy

  • 60 FPS
    on iPhone SE
  • 90 FPS
    on iPhone X or newer

Lower accuracy

  • 90 FPS
    on iPhone SE
  • 120 FPS
    on iPhone X or newer # Examples
  • Facemoji, our flagship app using the Kit :)
  • Facemoji for Mac, bring your Facemoji to Zoom
  • Squad, video chat and screen share with friends
  • One Word, a word game for friends
  • New: Trivia Royale, #1 trivia game in the world # Demo and download

Get the TestFlight demo app or try one of the apps above. If you’re interested in licensing Facemoji Kit, have a question and want to chat, feel free to reach out to us directly via email or join our chat room on Gitter.

3D Content

Facemoji Kit includes all the 3D accessories, models, textures, and shading assets to render Facemoji avatars in your app for your users. There are over 1,000 items, including hair styles, hats, sunglasses, face masks, animated accessories or even face tattoos. Accessories could be white-labeled.

Accessories

No avatar mode

Optional mode with no Facemoji and only 3D accessories with occluders.

Diversity

| | | | ---------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | | | | | | |

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.