WebRTC 1.0: Real-time Communication Between Browsers

W3C Working Draft 27 October 2011

This version:

http://www.w3.org/TR/2011/WD-webrtc-20111027/

Latest published version:

http://www.w3.org/TR/webrtc/

Latest editor's draft:

http://dev.w3.org/2011/webrtc/editor/webrtc.html

Previous version:

none

Editors:

Adam Bergkvist, Ericsson

Daniel C. Burnett, Voxeo

Cullen Jennings, Cisco

Anant Narayanan, Mozilla

Initial Author of this Specification was Ian Hickson, Google Inc., with the following copyright statement:
© Copyright 2004-2011 Apple Computer, Inc., Mozilla Foundation, and Opera Software ASA. You are granted a license to use, reproduce and create derivative works of this document.

All subsequent changes since 26 July 2011 done by the W3C WebRTC Working Group are under the following Copyright:
© 2011 W3C^® (MIT, ERCIM, Keio), All Rights Reserved. Document use rules apply.

For the entire publication on the W3C site the liability and trademark rules apply.

Abstract

This document defines a set of APIs that allow local media, including audio and video, to be requested from a platform, media to be sent over the network to another browser or device implementing the appropriate set of real-time protocols, and media received from another browser or device to be processed and displayed locally. This specification is being developed in conjunction with a protocol specification developed by the IETF RTCWEB group.

Status of This Document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.

This document is a First Public Working Draft and is not complete. It is subject to major changes and, while early experimentations are encouraged, it is therefore not intended for implementation. The API is based on preliminary work done in the WHATWG. The Web Real-Time Communications Working Group expects this specification to evolve significantly based on:

• The outcomes of ongoing exchanges in the companion RTCWEB group at IETF to define the set of protocols that, together with this document, will enable real-time communications in Web browsers.
• Privacy issues that arise when exposing local capabilities and local streams.
• Technical discussions within the group, on the data channel in particular.
• Experience gained through early experimentations.
• Feedback received from other groups and individuals.

As the specification matures, the group hopes to strike the right balance between a low-level API that would enable interested parties to tweak potentially complex system parameters, and a more high-level API that Web developers can use without a priori technical knowledge about real-time communications.

This document was published by the Web Real-Time Communications Working Group as a Working Draft. This document is intended to become a W3C Recommendation. If you wish to make comments regarding this document, please send them to public-webrtc@w3.org (subscribe, archives). All feedback is welcome.

Publication as a Working Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

This document was produced by a group operating under the 5 February 2004 W3C Patent Policy. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.

1. Introduction

This section is non-normative.

There are a number of facets to video-conferencing in HTML:

• Getting a multimedia stream (video, audio, or both) from local devices (video cameras, microphones, Web cams) or from prerecorded files provided by the user.
• Recording such streams locally.
• Connecting to remote peers using NAT-traversal technologies such as ICE, STUN, and TURN.
• Sending the locally-produced streams to remote peers and receiving streams from remote peers.
• Displaying such streams (both the locally-produced ones and the remotely-obtained ones) locally using the video or audio elements.
• Sending arbitrary data to remote peers.

This document defines the APIs used for these features. This specification is being developed in conjunction with a protocol specification developed by the IETF RTCWEB group.

2. Obtaining local multimedia content

<script>
 navigator.getUserMedia('audio', gotAudio);
 function gotAudio(stream) {
   // ... use 'stream' ...
 }
</script>

<script>
 function beginCall() {
   navigator.getUserMedia('audio,video user', gotStream);
 }
 function gotStream(stream) {
   // ... use 'stream' ...
 }
</script>

3. Stream API

[Callback=FunctionOnly, NoInterfaceObject]
interface BlobCallback {
    void handleEvent (Blob blob);
};

interface URL {
    static DOMString createObjectURL (MediaStream stream);
};

<input type="button" value="Start" onclick="start()" id="startBtn">
<script>
 var startBtn = document.getElementById('startBtn');
 function start() {
   navigator.getUserMedia('audio,video', gotStream);
   startBtn.disabled = true;
 }
 function gotStream(stream) {
   stream.onended = function () {
     startBtn.disabled = false;
   }
 }
</script>

<input type="button" value="⚫" onclick="msgRecord()" id="recBtn">
<input type="button" value="◼" onclick="msgStop()" id="stopBtn" disabled>
<p id="status">To start recording, press the ⚫ button.</p>
<script>
 var recBtn = document.getElementById('recBtn');
 var stopBtn = document.getElementById('stopBtn');
 function report(s) {
   document.getElementById('status').textContent = s;
 }
 function msgRecord() {
   report('Attempting to access microphone...');
   navigator.getUserMedia('audio', gotStream, noStream);
   recBtn.disabled = true;
 }
 var msgStream, msgStreamRecorder;
 function gotStream(stream) {
   report('Recording... To stop, press to ◼ button.');
   msgStream = stream;
   msgStreamRecorder = stream.record();
   stopBtn.disabled = false;
   stream.onended = function () {
     msgStop();     
   }
 }
 function msgStop() {
   report('Creating file...');
   stopBtn.disabled = true;
   msgStream.onended = null;
   msgStream.stop();
   msgStreamRecorder.getRecordedData(msgSave);
 }
 function msgSave(blob) {
   report('Uploading file...');
   var x = new XMLHttpRequest();
   x.open('POST', 'uploadMessage');
   x.send(blob);
   x.onload = function () {
     report('Done! To record a new message, press the ⚫ button.');
     recBtn.disabled = false;
   };
   x.onerror = function () {
     report('Failed to upload message. To try recording a message again, press the ⚫ button.');
     recBtn.disabled = false;
   };
 }
 function noStream() {
   report('Could not obtain access to your microphone. To try again, press the ⚫ button.');
   recBtn.disabled = false;
 }
</script>

<article>
 <style scoped>
  video { transform: scaleX(-1); }
  p { text-align: center; }
 </style>
 <h1>Snapshot Kiosk</h1>
 <section id="splash">
  <p id="errorMessage">Loading...</p>
 </section>
 <section id="app" hidden>
  <p><video id="monitor" autoplay></video> <canvas id="photo"></canvas>
  <p><input type=button value="&#x1F4F7;" onclick="snapshot()">
 </section>
 <script>
  navigator.getUserMedia('video user', gotStream, noStream);
  var video = document.getElementById('monitor');
  var canvas = document.getElementById('photo');
  function gotStream(stream) {
    video.src = URL.getObjectURL(stream);
    video.onerror = function () {
      stream.stop();
    };
    stream.onended = noStream;
    video.onloadedmetadata = function () {
      canvas.width = video.videoWidth;
      canvas.height = video.videoHeight;
      document.getElementById('splash').hidden = true;
      document.getElementById('app').hidden = false;
    };
  }
  function noStream() {
    document.getElementById('errorMessage').textContent = 'No camera available.';
  }
  function snapshot() {
    canvas.getContext('2d').drawImage(video, 0, 0);
  }
 </script>
</article>

4. Peer-to-peer connections

A PeerConnection allows two users to communicate directly, browser-to-browser. Communications are coordinated via a signaling channel provided by script in the page via the server, e.g. using XMLHttpRequest.

Calling "new PeerConnection(configuration, signalingCallback)" creates a PeerConnection object.

The configuration string gives the address of a STUN or TURN server to use to establish the connection. [STUN] [TURN]

The allowed formats for this string are:

"TYPE 203.0.113.2:3478"

Indicates a specific IP address and port for the server.

"TYPE relay.example.net:3478""

Indicates a specific host and port for the server; the user agent will look up the IP address in DNS.

"TYPE example.net""

Indicates a specific domain for the server; the user agent will look up the IP address and port in DNS.

The "TYPE" is one of:

STUN

Indicates a STUN server

STUNS

Indicates a STUN server that is to be contacted using a TLS session.

TURN

Indicates a TURN server

TURNS

Indicates a TURN server that is to be contacted using a TLS session.

The signalingCallback argument is a method that will be invoked when the user agent needs to send a message to the other host over the signaling channel. When the callback is invoked, convey its first argument (a string) to the other peer using whatever method is being used by the Web application to relay signaling messages. (Messages returned from the other peer are provided back to the user agent using the processSignalingMessage() method.)

A PeerConnection object has an associated PeerConnection signaling callback, a PeerConnection ICE Agent, a PeerConnection readiness state and an SDP Agent. These are initialized when the object is created.

When the PeerConnection() constructor is invoked, the user agent must run the following steps. This algorithm has a synchronous section (which is triggered as part of the event loop algorithm). Steps in the synchronous section are marked with ⌛.

1. Let serverConfiguration be the constructor's first argument.

2. Let signalingCallback be the constructor's second argument.

3. Let connection be a newly created PeerConnection object.

4. Create an ICE Agent and let connection's PeerConnection ICE Agent be that ICE Agent. [ICE]

5. If serverConfiguration contains a U+000A LINE FEED (LF) character or a U+000D CARRIAGE RETURN (CR) character (or both), remove all characters from serverConfiguration after the first such character.

6. Split serverConfiguration on spaces to obtain configuration components.

7. If configuration components has two or more components, and the first component is a case-sensitive match for one of the following strings:

   • "STUN"
   • "STUNS"
   • "TURN"
   • "TURNS"

   ...then run the following substeps:

   1. Let server type be STUN if the first component of configuration components is 'STUN' or 'STUNS', and TURN otherwise (the first component of configuration components is "TURN" or "TURNS").

   2. Let secure be true if the first component of configuration components is "STUNS" or "TURNS", and false otherwise.

   3. Let host be the contents of the second component of configuration components up to the character before the first U+003A COLON character (:), if any, or the entire string otherwise.

   4. Let port be the contents of the second component of configuration components from the character after the first U+003A COLON character (:) up to the end, if any, or the empty string otherwise.

   5. Configure the PeerConnection ICE Agent's STUN or TURN server as follows:

      • If server type is STUN, the server is a STUN server. Otherwise, server type is TURN and the server is a TURN server.
      • If secure is true, the server is to be contacted using TLS-over-TCP, otherwise, it is to be contacted using UDP.
      • The IP address, host name, or domain name of the server is host.
      • The port to use is port. If this is the empty string, then only a domain name is configured (and the ICE Agent will use DNS SRV requests to determine the IP address and port).
      • The long-term username for the STUN or TURN server is the ASCII serialization of the entry script's origin; the long-term password is the empty string.

      If the given IP address, host name, domain name, or port are invalid, then the user agent must act as if no STUN or TURN server is configured.

8. Let the connection's PeerConnection signaling callback be signalingCallback.

9. Set connection's PeerConnection readiness state to NEW (0).

10. Set connection's PeerConnection ice state to NEW (0).

11. Set connection's PeerConnection sdp state to NEW (0).

12. Let connection's localStreams attribute be an empty read-only MediaStream array. [WEBIDL]

13. Let connection's remoteStreams attribute be an empty read-only MediaStream array. [WEBIDL]

14. Return connection, but continue these steps asynchronously.

15. Await a stable state. The synchronous section consists of the remaining steps of this algorithm. (Steps in synchronous sections are marked with ⌛.)

16. ⌛ If the ice state is set to NEW, it must queue a task to start gathering ICE address and set the ice state to ICEGATHERING.

17. ⌛ Once the ICE address gathering is complete, if there are any streams in localStreams, the SDP Agent will send the initial the SDP offer. The initial SDP offer must contain both the ICE candidate information as well as the SDP to represent the media descriptions for all the streams in localStreams.

During the lifetime of the peerConnection object, the following procedures are followed:

1. If a local media stream has been added and an SDP offer needs to be sent, and the ICE state is not NEW or ICEGATHERING, and the SDP Agent state is NEW or SDPIDLE, then send and queue a task to send an SDP offer and change the SPD state to SDP Waiting.

2. If an SDP offer has been received, and the SDP state is NEW or SDPIDLE, pass the ICE candidates from the SDP offer to the ICE Agent and change it state to ICECHECKING. Construct an appropriate SDP answer, update the remote streams, queue a task to send the SDP offer, and set the SDPAgent state to SDPIDLE.

3. At the point the sdpState changes from NEW to some other state, the readyState changes to NEGOTIATING.

4. If the ICE Agent finds a candidates that froms a valid connection, the ICE state is changed to ICECONNECTED

5. If the ICE Agent finishes checking all candidates, if a connection has been found, the ice state is changed to ICECOMPLETED and if not connection has been found it is changed to ICEFAILED.

6. If the iceState is ICECONNECTED or ICECOMPLETED and the SDP stat is SDPIDLE, the readyState is set to ACTIVE.

7. If the iceState is ICEFAILED, a task is queued to calls the close method.

8. The close method will cause the system to wait until the sdpStat is SDPIDLE then it will send an SDP offer terminating all media and change the readyState to CLOSING as well as stop all ICE process and change the iceState to ICE_CLOSED. Once an SDP anser to this offer is received, the readyState will be changed to CLOSED.

User agents may negotiate any codec and any resolution, bitrate, or other quality metric. User agents are encouraged to initially negotiate for the native resolution of the stream. For streams that are then rendered (using a video element), user agents are encouraged to renegotiate for a resolution that matches the rendered display size.

Starting with the native resolution means that if the Web application notifies its peer of the native resolution as it starts sending data, and the peer prepares its video element accordingly, there will be no need for a renegotiation once the stream is flowing.

All SDP media descriptions for streams represented by MediaStream objects must include a label attribute ("a=label:") whose value is the value of the MediaStream object's label attribute. [SDP] [SDPLABEL]

PeerConnections must not generate any candidates for media streams whose media descriptions do not have a label attribute ("a=label:"). [ICE] [SDP] [SDPLABEL]

When a user agent starts receiving media for a component and a candidate was provided for that component by a PeerConnection, the user agent must follow these steps:

1. Let connection be the PeerConnection expecting this media.

2. If there is already a MediaStream object for the media stream to which this component belongs, then associate the component with that media stream and abort these steps. (Some media streams have multiple components; this API does not expose the role of these individual components in ICE.)

3. Create a MediaStream object to represent the media stream. Set its label attribute to the value of the SDP Label attribute for that component's media stream.

4. Queue a task to run the following substeps:

   1. If the connection's PeerConnection readiness state is CLOSED (3), abort these steps.

   2. Add the newly created MediaStream object to the end of connection's remoteStreams array.

   3. Fire a stream event named addstream with the newly created MediaStream object at the connection object.

When a PeerConnection finds that a stream from the remote peer has been removed (its port has been set to zero in a media description sent on the signaling channel), the user agent must follow these steps:

1. Let connection be the PeerConnection associated with the stream being removed.

2. Let stream be the MediaStream object that represents the media stream being removed, if any. If there isn't one, then abort these steps.

3. By definition, stream is now finished.

   A task is thus queued to update stream and fire an event.

4. Queue a task to run the following substeps:

   1. If the connection's PeerConnection readiness state is CLOSED (3), abort these steps.

   2. Remove stream from connection's remoteStreams array.

   3. Fire a stream event named removestream with stream at the connection object.

The task source for the tasks listed in this section is the networking task source.

To prevent network sniffing from allowing a fourth party to establish a connection to a peer using the information sent out-of-band to the other peer and thus spoofing the client, the configuration information should always be transmitted using an encrypted connection.

[Constructor (in DOMString configuration, in SignalingCallback signalingCallback)]
interface PeerConnection {
    void processSignalingMessage (DOMString message);
    const unsigned short NEW = 0;
    const unsigned short NEGOTIATING = 1;
    const unsigned short ACTIVE = 2;
    const unsigned short CLOSING = 4;
    const unsigned short CLOSED = 3;
    readonly attribute unsigned short readyState;
    const unsigned short ICE_GATHERING = 0x100;
    const unsigned short ICE_WAITING = 0x200;
    const unsigned short ICE_CHECKING = 0x300;
    const unsigned short ICE_CONNECTED = 0x400;
    const unsigned short ICE_COMPLETED = 0x500;
    const unsigned short ICE_FAILED = 0x600;
    const unsigned short ICE_CLOSED = 00x700;
    readonly attribute unsigned short iceState;
    const unsigned short SDP_IDLE = 0x1000;
    const unsigned short SDP_WAITING = 0x2000;
    const unsigned short SDP_GLARE = 0x3000;
    readonly attribute unsigned short sdpState;
    void addStream (MediaStream stream, MediaStreamHints hints);
    void removeStream (MediaStream stream);
    readonly attribute MediaStream[]  localStreams;
    readonly attribute MediaStream[]  remoteStreams;
    void close ();
             attribute Function?      onconnecting;
             attribute Function?      onopen;
             attribute Function?      onstatechange;
             attribute Function?      onaddstream;
             attribute Function?      onremovestream;
};

PeerConnection implements EventTarget;

[Callback=FunctionOnly, NoInterfaceObject]
interface SignalingCallback {
    void handleEvent (DOMString message, PeerConnection source);
};

// the first argument describes the STUN/TURN server configuration
var local = new PeerConnection('TURNS example.net', sendSignalingChannel);
local.signalingChannel(...); // if we have a message from the other side, pass it along here

// (aLocalStream is some LocalMediaStream object)
local.addStream(aLocalStream); // start sending video

function sendSignalingChannel(message) {
  ... // send message to the other side via the signaling channel
}

function receiveSignalingChannel (message) {
  // call this whenever we get a message on the signaling channel
  local.signalingChannel(message);
}

local.onaddstream = function (event) {
  // (videoElement is some <video> element)
  videoElement.src = URL.getObjectURL(event.stream);
};

5. The data stream

Although progress is being made, there is currently not enough agreement on the data channel to write it up. This section will be filled in as rough consensus is reached.

6. Garbage collection

A Window object has a strong reference to any PeerConnection objects created from the constructor whose global object is that Window object.

8. Event summary

This section is non-normative.

The following event fires on MediaStream objects:

The following events fire on PeerConnection objects:

9. application/html-peer-connection-data

This registration is for community review and will be submitted to the IESG for review, approval, and registration with IANA.

Type name:

application

Subtype name:

html-peer-connection-data

Required parameters:

No required parameters

Optional parameters:

No optional parameters

Encoding considerations:

This MIME type defines a binary protocol format which uses UTF-8 for text encoding.

Security considerations:

This format is used for encoding UDP packets transmitted by potentially hostile Web page content via a trusted user agent to a destination selected by a potentially hostile remote server. To prevent this mechanism from being abused for cross-protocol attacks, all the data in these packets is masked so as to appear to be random noise. The intent of this masking is to reduce the potential attack scenarios to those already possible previously.

However, this feature still allows random data to be sent to destinations that might not normally have been able to receive them, such as to hosts within the victim's intranet. If a service within such an intranet cannot handle receiving UDP packets containing random noise, it might be vulnerable to attack from this feature.

Interoperability considerations:

Rules for processing both conforming and non-conforming content are defined in this specification.

Published specification:

This document is the relevant specification.

Applications that use this media type:

This type is only intended for use with SDP. [SDP]

Additional information:

Magic number(s):

No sequence of bytes can uniquely identify data in this format, as all data in this format is intentionally masked to avoid cross-protocol attacks.

File extension(s):

This format is not for use with files.

Macintosh file type code(s):

This format is not for use with files.

Person & email address to contact for further information:

Daniel C. Burnett <dburnett@voxeo.com>

Intended usage:

Common

Restrictions on usage:

No restrictions apply.

Author:

Daniel C. Burnett <dburnett@voxeo.com>

Change controller:

W3C

Fragment identifiers cannot be used with application/html-peer-connection-data as URLs cannot be used to identify streams that use this format.

A. Acknowledgements

The editors wish to thank the Working Group chairs, Harald Alvestrand and Stefan Håkansson, for their support.

B. References