Need help with WeChat-MiniProgram-WebAR?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

sanyuered
149 Stars 33 Forks BSD 3-Clause "New" or "Revised" License 108 Commits 2 Opened issues

Description

A WeChat MiniProgram AR with JavaScript Computer Vision Libraries.

Services available

!
?

Need anything else?

Contributors list

No Data

  1. Chinese README

  2. Chinese Source Code Analysis

Updated

| Date   | Update | | -- | -- | | 2019-08-16 | Update: The project structure has been modified. The color tracker and object tracker are removed. Fix "access a camera" mode that does not work properly on Android. | | 2019-08-06 | Fix Issue: When function "wx.canvasToTempFilePath" is called frequently on Android Wechat, WeChat will be crashed. | | 2019-08-01 | Update: The perspective transform is achieved. | | 2019-07-15 | Update: The NFT(Natural Feature Tracking) is achieved. | | 2019-07-08 | New: The affine transform is achieved. |

Introduction of WeChat MiniProgram Web AR

This is a WeChat Web AR Demo. On July 5, 2019, WeChat miniprogram supports AR. It has been added a new API named "CameraFrameListener".

CameraFrameListener API

We can create AR effects with the new API. This demo demonstrates a color tracker effect using "tracking.js" and "jsfeat" library.

The "tracking.js" brings computer vision algorithms and techniques into browser environment. We can do real-time color tracking, face detection and much more.

tracking.js

The "jsfeat" is a JavaScript computer vision library.

JSFeat

The Demo includes color tracker, face tracker, image tracker and object tracker. There are two modes for each item. It includes 'taking a photo' and 'access a camera'.

avatar

Face Tracker

Use the Demo to scan people's face.

avatar

Expect the effect below.

Use "Take a Photo" mode.

avatar

When face recognition is not exact, the result is not good.

avatar

When face recognition is exact, the result is good.

avatar

Image Tracker

a sample pattern picture is below.

avatar

Use the Demo to scan a rotate picture below.

avatar

Expected:

avatar

Use the Demo to scan a skew picture below.

avatar

Expected:

avatar

Use the Demo to scan a translate and scale picture below.

avatar

Expected:

avatar

Use the Demo to scan a perspective picture below.

avatar

Expected:

avatar

Color Tracker (Removed)

Use the Demo to scan the picture below.

avatar

Expect the effect below.

Use "Take a Photo" mode.

avatar

Use "Access a camera" mode.

avatar

Object Tracker (Removed)

There will be a rect aboves people's mouth. It is slow and not better than face tracker.

How to improve performance

Face tracker is slow, but we can do some thing to improve speed. For example, we can blur image, grayscale image, sobel image, compress image and so on. After testing, we found that reducing image size is more proper. When image size is reduced, the parameters of tracker needs to be updated.

frame size of camera: the image size is smaller, the tracker's speed is faster.

javascript
const frameWidth = 150;

face tracker parameter:the "initialScale" is bigger, the face tracker's speed is faster.

javascript
const initialScale = 2;

color tracker parameter:the "minDimension" is bigger, the color tracker's speed is faster.

javascript
const minDimension = 4;

interval time: interval time should be greater than cost time. The unit is milliseconds.

javascript
const intervalTime = 350;

Known Issues

Image tracker and face tracker are very slow on iOS WeChat.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.