WebRTC Low Latency Streaming

Note: Not a live demo, server configuration is just for example. For live working example try the Millicast Publisher and Subscriber example.

Example publisher using hidden player managed settings menu. Max device dimensions is disabled using maxDeviceDimensions , the maximum capable dimensions of the device camera will be used which could be 4K which could be too large for publishing. If disabled maxWidth and maxHeight can be used to restrict dimensions if no dimension dropdown is configured.

The WebRTC Subscriber Player is used for subscribing to the stream. If there is a transcoding option enabled in the server a HLS source can be used instead.

If publishing to the server as RTMP using an encoder and restreaming as WebRTC, the subscriber player can be used for subscribing to the stream.

An audio meter display can be used to display microphone activity and volume.

Introduction

The WebRTC is a browser based low latency peering streaming solution with only the requirement of a signal server.

While WebRTC can allow for full 2 way peering, the intended solution is for one to many low latency live broadasting with basic conferencing functions.

With Kurento, Millicast, Ant Media Server and Wowza Media Server conferencing solutions is provided however with Wowza the streams are pulled from the server as WebRTC or HLS. Demo java backend projects are provided.

With Millicast conferencing a demo node websocket service is provided to signal users when a publish stream begins. The WebRTC plugin provides an integrated browser based encoder publishing application, with media device, resolution, codec, bitrate controls. And a subscriber player.

Media device selection can use whatever device the browser detects including video and audio inputs. Audio output control for the publisher is possible for browsers that support it.

Screensharing capabilities with browsers that support it are possible using screen sharing apis. The screen, browser, browser tab can be chosen , with an audio input for audio. This may be suitable for live broadcasting instruction videos or even gaming.

Switching between camera and screenshare video tracks is possible while peering. Retaining the original microphone audio stream. Recording of the media device is possible using browser based media recorder api's for browsers that support it. With a download capability of the recorded data.

Device toggling can turn the video / audio device off before and while publishing.

Supported servers for now is Wowza, Kurento, Millicast WebRTC backends. Wowza only supports one to many peering and no data channels. Provided is a Kurento one to many signal server.

WebRTC settings can be configured as a player overlay,

Volume control of the audio input is made possible using the audio api. This also makes the possibility of an audio meter display

Ant Media Server

Wowza Media Server

Millicast

Kurento Media Server

Virtual Meeting Background Removal

With the use of BodyPix and Tensorflow. An included virtual background plugin. Support for dynamic background removal rendering is provided.

This rendering view can be used as the main video stream on the publisher and will be automatically detected.

The auto background uses an AI BodyPix feature that doesn't require a Chromakey Green Screen render. It's not perfect but with fine tuned settings it's enough to make a transparent background to show a background image behind the video.

A Chromakey rendering option for Green Screens is supported with option to configure the flat background colour or choose an auto option for choosing any background of a solid colour.

Audio Meter

An audio meter plugin can be configured to display microphone activity and volume. An external container with a peak-meter selector can be setup

<div class="peak-meter absolute right-4 -bottom-1.5 w-auto" style="max-width: 80px; height: 300px;"></div>

Features

Low latency streaming via the browser.
One to many live streaming via WebRTC
Group call conferencing as WebRTC or HLS.
Group conferencing features for Wowza, Millicast and Kurento.
Ability to feature a participant in a conference by an admin.
Media device video and audio selection.
Encoder application selections of bitrate, resolution, framerate, audio input, audio output and video sources.
Support for external UI control if managed select menus is not suitabled.
Recording of local video stream support with a download option.
Toggling of audio and video device before and while publishing.
Screensharing video capabilities using screensharing apis instead of camera and microphone.
Screensharing toggling support between screen capture and camera. While peering the tracks are replaced in realtime. Audio is retained while screen capturing.
Microphone Audio peak meter display option.
Full peering events support with debug capabilities.
Audio volume control of the microphone input and audio meter display functionality.
Support for dynamic background removal for virtual meetings without Green Screens using Bodypix AI.
Support for Chromakey Green Screen background removal rendering.

    <div class="max-w-screen-2xl h-auto my-auto">
          <div id="rtcstreaming" class="has-settings"></div>
  </div>

  <script type="text/javascript">

  	var player = jwplayer("rtcstreaming", {
    "aspectratio": "16:9",
    "playbackRateControls": true,
    "plugins": {
        "js/webrtcpeakmeter-8.20.0.js": {},
        "js/webrtcpublisher-8.20.0.js": {
            "applicationName": "webrtc",
            "autoStartDevice": true,
            "debug": true,
            "floatedControls": true,
            "maxDeviceDimensions": false,
            "maxHeight": 720,
            "maxWidth": 1280,
            "publisher": true,
            "recording": {
                "codec": "VP9",
                "mimeType": "video/webm",
                "name": "recording1"
            },
            "server": "wowza",
            "serverURL": "rtc.electroteque.org",
            "settings": true,
            "toggleScreen": true,
            "userData": {
                "param1": "value1"
            }
        }
    },
    "sources": [
        {
            "appName": "webrtc",
            "file": "myStream",
            "live": true,
            "publisher": true,
            "type": "mp4"
        }
    ],
    "width": "100%"
});

player.on("ready", function() {
player.on("devices", (devicesMap, deviceInfos) => {
//get available devices here for building your own UI
console.log("devices", devicesMap);
}).on("mediastart", (info, stream) => {
console.log("Device Start ", info.deviceInfo);
console.log("Updated Device Info ", info.newDeviceInfo);
console.log("Video Constraints ", info.videoConstraints);
console.log("Capabilities ", info.capabilities);
}).on("mediastop", (e) => {
console.log("Device Stop");
}).on("startpublish", (supportsParams) => {
console.log("Publishing Started");
//use this for enabling / disabling bandwidth select menus while publishing
//browsers that don"t support it cannot update bitrate controls while publishing
//console.log("Browser Supports peer setParameters ", supportsParams);
}).on("stoppublish", (e) => {
console.log("Publishing Stopped");
}).on("recordstart", (e) => {
console.log("Recording Started");
}).on("recordstop", (e) => {
console.log("Recording Stopped");
}).on("sendmessage", (message) => {
console.log("Signal Server Send Message: ",message);
}).on("gotmessage", (message) => {
console.log("Signal Server Receive Message: ", message);
}).on("offer", (offer) => {
console.log("WebRTC Offer ", offer.sdp);
}).on("answer", (answer) => {
console.log("WebRTC Answer ", answer.sdp);
}).on("bitratechanged", (params) => {
console.log("WebRTC Bitrate Changed ", params);
}).on("outputsuccess", (sinkId) => {
console.log("Success, audio output device attached:" + sinkId);
}).on("outputerror", (message) => {
console.log(message);
}).on("ready", function(eo) {
// console.log("READY", video);
}).on("screensharestart", (e) => {
console.log("screen share started");
}).on("screensharestop", (e) => {
console.log("screen share stopped");
});
});

  </script>