Figment API

How to work with Figment.

General Functions

figment.activate(configs? : object)

Activates Figment and returns a promise. Accepts an optional config object with the field app and the possible values 'VB' and 'AR', which activate the virtual background and augmented reality functionality, respectively. If no configs are specified the virtual backgrounds feature is activated.

If the set_render_video option is enabled, the output is rendered to the #figment <div>.

Example:

// "VB" for activating virtual background, "AR" for activating augmented reality
await figment.activate({app: "VB"})    

// Interact with Figment    

If Figment is activated without first executing setInputMediaStream(), Figment will read a MediaStream by itself usingmediaDevices.getUserMedia() with default parameters.

returns: Promise

figment.deactivate()

Deactivates Figment. All rendering and computations are stopped.

Figment also stops the input MediaStream when deactivated. So, before reactivating, a new input MediaStream must be set, otherwise a MediaStream with default parameters is read.

returns: Promise

figment.setInputMediaStream(mediaStream: MediaStream)

Lets you set the MediaStream that Figment uses as video input.

Usage:

figment.setInputMediaStream(mediaStream);

await figment.activate()

This function should be executed before Figment is activated. If this function is executed while Figment is running, the output MediaStream is reinitialized and theFigmentOutputUpdatedEvent fires again.

figment.setMediaTrackConstraints(constraints: object)

Allows to set the MediaTrackConstraints for the input video. In this case, the input video is read by Figment using:

mediaStream = await navigator.mediaDevices.getUserMedia({video: constraints})

Usage:

figmentApp.setMediaTrackConstraints(constraints);

await figment.activate()

This function should be executed before Figment is activated. If this function is executed while Figment is running, the output MediaStream is reinitialized and theFigmentOutputUpdated Event fires again.

figment.getOutputMediaStream()

Lets you receive Figment's output MediaStream containing the final segmentation result. This function should only be executed after Figment fires its FigmentOutputUpdated event.

Usage:

// App should be activated

window.addEventListener('FigmentOutputUpdated', () => {
  
  let outputMediaStream = window.figment.getOutputMediaStream()
  
  // Further actions on your side

});

returns: MediaStream

figment.setOption(option: string, value: string | boolean)

General function to change the internal state of Figment. This function may be executed before or while Figment is running.

option: Option to be set. Available options are: set_active_app | set_render_video | show_hide_performance | set_input_frequency

• set_active_app — Lets you switch between the virtual background (VB) and augmented reality (AR) functionality. value: VB (default) | AR

• set_render_video — Option to control whether Figment should render its output into the #figment <div> . value: true (default) | false

• show_hide_performance — allows to show and hide a transparent performance bar showing frames per second (FPS) and inference times in milliseconds. value: show (default) | hide

• set_input_frequency — Lets you steer the frequency of video frames entering the machine learning model. To give you maximum control, this function lets you specify the exact interval between frames in milliseconds. The higher the interval, the lower Figment's CPU load and output FPS. To specify the input frames per second (FPS) you can set the value to 1000/FPS. Note: Only the input FPS is specified here, but not Figment's final output FPS. This is because the output FPS always depends on the inference time of the machine learning model, which varies depending on the performance of the local computer. value: 1 - Number.MAX_SAFE_INTEGER, default: 1000/30

value: value specifying the set option.

Virtual Background

figment.setOption(option: string, value: string)

Setting options for the virtual background feature.

option: Option to be set. Available options are: set_background_img | blur_background | activate_quality_mode | activate_performance_mode | set_mask_behavior

• set_background_img — sets the current background image. value: path_to_image.

• blur_background — blurs the background. We offer three different blur strengths: weak, balanced and strong. value: weak | balanced | strong

• activate_quality_mode (default) — uses the model with emphasis on segmentation quality. Slight drop in FPS is expected. value: ''

• activate_performance_mode — uses the model with emphasis on performance. Slight decrease in segmentation quality expected. value: ''

• set_mask_behavior — Allows switching between a tight or a slightly looser mask. Both masks aim to accurately segment the person. The loose mask shows small fractions of the background in order to preserve the whole person, while the tight mask aims to show as little background as possible with the possibility that the mask may cut into the person at times. value: tight (default) | loose

value: value specifying the set option.

Usage examples:

// Change to the model with emphasis on performance
figment.setOption('activate_performance_mode', '');

figment.activate().then(() => {

    // Futher actions

});

<!-- Change Background -->
<button value="link/to/image" onclick="window.figment.setOption('set_background_img', this.value);"></button>

Augmented Reality

figment.setOption(option: string, value: string | object)

Setting options for the augmented reality feature.

option: Option to be set. Available options are: set_ar_filter

• set_ar_filter — lets you set the augmented reality filter. value: The value can either be a string specifying the arFilterId, or an object with the fields arFilterId and animateFilters . animateFilters is an object that allows specifying animationType and objectCount for our animated filters.

We offer the following filters:

On face

classic_sunglasses | retro_sunglasses | golden_sunglasses | mustache | devil_horns | clown_nose | crown | floral_crown | birthday_hat | pirate_hat | joker_mask

Above Face

mario_question_box | sims_icon

Around Face (animated)

{
    arFilterId: "exclamation_mark" | "light_bulb" | "question_mark"
    animateFilters: {
        animationType: "rotate_360" | "rotate_180" | "floating" | ""
        objectCount : number
    }
}

value: value specifying the set option.

Usage examples:

window.figment.setOption('set_ar_filter', "classic_sunglasses");

window.figment.setOption(
    'set_ar_filter', 
    {
        arFilterId: 'question_mark',
        animateFilters: {animationType: 'floating', objectCount: 3}
});