Figment API

How to work with Figment.

figmentApp.activate()

Activates Figment and renders the output to the #figment <div>

To ensure that all parts of Figment are loaded before using the interface, .activate() return a Promise. The intended way to activate Figment is as follows:

figmentApp.activate().then(() => {
    // Interact with Figment
});

If Figment is activated without first executing setInputMediaStream(), Figment will read a MediaStream by itself usingmediaDevices.getUserMedia() with default parameters.

returns: Promise

figmentApp.deactivate()

Deactivates Figment. All rendering and computations are stopped.

returns: Promise

figmentApp.setInputMediaStream(mediaStream: MediaStream)

Lets you set the MediaStream that Figment uses as video input.

Usage:

figmentApp.setInputMediaStream(mediaStream);

figmentApp.activate().then(() => {
  // Interact with Figment 
});

This function should be executed before Figment is activated. If this function is executed while Figment is running, the app will be restarted, requiring to reinitialize the output MediaStream (see getOutputMediaStream()).

figmentApp.setMediaTrackConstraints(constraints: object)

Allows to set the MediaTrackConstraints for the input video. In this case, the input video is read by Figment using:

mediaStream = await navigator.mediaDevices.getUserMedia({video: constraints})

Usage:

figmentApp.setMediaTrackConstraints(constraints);

figmentApp.activate().then(() => {
  // Interact with Figment 
});

This function should be executed before Figment is activated. If this function is executed while Figment is running, the app will be restarted, requiring to reinitialize the output MediaStream (see getOutputMediaStream()).

figmentApp.getOutputMediaStream()

Lets you receive Figment's output MediaStream containing the final segmentation result. This function can only be executed if Figment is activated.

Usage:

figmentApp.activate().then(() => {
    let outputMediaStream = figmentApp.getOutputMediaStream()
    
    // Futher actions

});

returns: MediaStream

To aggregate the segmentation mask with the input video and apply important post-processing steps, we need to render the output into a <canvas> element using WebGL. This rendered output is intended to be the view for the local user running Figment client-side. The MediaStream from getOutputMediaStream() is intended to be sent over a peer connection.

figmentApp.setOption(option: string, value: string)

General function to change the internal state of Figment. This function may be executed before or while Figment is running.

option: Option to be set. Available options are: set_background_img | blur_background | activate_quality_mode | activate_performance_mode | show_hide_performance | set_mask_behavior

• set_background_img — sets the current background image. value: path_to_image.

• blur_background — blurs the background. value: ''

• activate_quality_mode (default) — uses the model with emphasis on segmentation quality. Slight drop in FPS is expected. value: ''

• activate_performance_mode — uses the model with emphasis on performance. Slight decrease in segmentation quality expected. value: ''

• show_hide_performance — allows to show and hide a transparent performance bar showing frames per second (FPS) and inference times in milliseconds. value: show | hide

• set_mask_behavior — Allows switching between a tight or a slightly looser mask. Both masks aim to accurately segment the person. The loose mask shows small fractions of the background in order to preserve the whole person, while the tight mask aims to show as little background as possible with the possibility that the mask may cut into the person at times. value: loose (default) | tight

value: value specifying the set option.

Usage examples:

// Change to the model with emphasis on performance
figmentApp.setOption('activate_performance_mode', '');

figmentApp.activate().then(() => {

    // Futher actions

});

or

<!-- Change Background -->
<button value="link/to/image" onclick="figmentApp.setOption('set_background_img', this.value);"></button>