[Pkg-voip-commits] [janus] 43/163: Aligned the VP9 SVC branch to the simulcast stuff merged in master

Jonas Smedegaard dr at jones.dk
Sat Oct 28 01:22:08 UTC 2017


This is an automated email from the git hooks/post-receive script.

js pushed a commit to annotated tag debian/0.2.5-1
in repository janus.

commit 44d6d66d9fa322410171ca2793eac59a678f37c7
Merge: fd8d681 211c2c2
Author: Lorenzo Miniero <lminiero at gmail.com>
Date:   Fri Jul 28 11:49:42 2017 +0200

    Aligned the VP9 SVC branch to the simulcast stuff merged in master

 README.md                                 |   2 +-
 conf/janus.plugin.streaming.cfg.sample.in |   3 +
 configure.ac                              |   2 +-
 docs/janus-doxygen.cfg                    |   2 +-
 html/devicetest.html                      |   8 +-
 html/devicetest.js                        | 155 ++++++-
 html/echotest.html                        |  10 +-
 html/echotest.js                          | 155 +++++++
 html/janus.js                             | 245 +++++++++-
 html/janus.nojquery.js                    | 245 +++++++++-
 html/streamingtest.html                   |   8 +-
 html/streamingtest.js                     | 185 ++++++++
 html/videocalltest.html                   |  14 +
 html/videocalltest.js                     | 152 +++++++
 html/videoroomtest.html                   |  32 +-
 html/videoroomtest.js                     | 167 +++++++
 html/vp9svctest.html                      |   7 +-
 ice.c                                     |  53 ++-
 ice.h                                     |   6 +
 janus.c                                   |  46 +-
 janus.ggo                                 |   2 +-
 plugins/janus_echotest.c                  | 342 +++++++++++++-
 plugins/janus_recordplay.c                |  21 +-
 plugins/janus_sip.c                       |  29 +-
 plugins/janus_streaming.c                 | 731 ++++++++++++++++++------------
 plugins/janus_videocall.c                 | 257 ++++++++++-
 plugins/janus_videoroom.c                 | 381 ++++++++++++++--
 postprocessing/pp-webm.c                  |   2 +-
 rtcp.c                                    |  16 +-
 rtcp.h                                    |  12 +-
 rtp.c                                     |  33 +-
 rtp.h                                     |  13 +
 sdp.c                                     | 130 +++++-
 sdp.h                                     |   8 +
 utils.c                                   | 304 +++++++++++++
 utils.h                                   |  49 +-
 36 files changed, 3424 insertions(+), 403 deletions(-)

diff --cc html/vp9svctest.html
index 8a29a42,0000000..ffb51a8
mode 100644,000000..100644
--- a/html/vp9svctest.html
+++ b/html/vp9svctest.html
@@@ -1,166 -1,0 +1,171 @@@
 +<!DOCTYPE html>
 +<html xmlns="http://www.w3.org/1999/xhtml">
 +<head>
 +<meta charset="utf-8">
 +<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
 +<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
 +<title>Janus WebRTC Gateway: VP9-SVC Video Room Demo</title>
 +<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/webrtc-adapter/3.4.3/adapter.min.js" ></script>
 +<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/jquery/1.7.2/jquery.min.js" ></script>
 +<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/jquery.blockUI/2.70/jquery.blockUI.min.js" ></script>
 +<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.0.2/js/bootstrap.min.js"></script>
 +<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/bootbox.js/4.1.0/bootbox.min.js"></script>
 +<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/spin.js/2.3.2/spin.min.js"></script>
 +<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/toastr.js/2.1.3/toastr.min.js"></script>
 +<script type="text/javascript" src="janus.js" ></script>
 +<script type="text/javascript" src="vp9svctest.js"></script>
 +<script>
 +	$(function() {
 +		$(".navbar-static-top").load("navbar.html", function() {
 +			$(".navbar-static-top li.dropdown").addClass("active");
 +			$(".navbar-static-top a[href='vp9svctest.html']").parent().addClass("active");
 +		});
 +		$(".footer").load("footer.html");
 +	});
 +</script>
 +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/bootswatch/3.3.7/cerulean/bootstrap.min.css" type="text/css"/>
 +<link rel="stylesheet" href="css/demo.css" type="text/css"/>
 +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.6.2/css/font-awesome.min.css"/>
 +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/toastr.js/2.1.3/toastr.css"/>
 +</head>
 +<body>
 +
 +<a href="https://github.com/meetecho/janus-gateway"><img style="position: absolute; top: 0; left: 0; border: 0; z-index: 1001;" src="https://s3.amazonaws.com/github/ribbons/forkme_left_darkblue_121621.png" alt="Fork me on GitHub"></a>
 +
 +<nav class="navbar navbar-default navbar-static-top">
 +</nav>
 +
 +<div class="container">
 +	<div class="row">
 +		<div class="col-md-12">
 +			<div class="page-header">
 +				<h1>Plugin Demo: VP9-SVC Video Room
 +					<button class="btn btn-default" autocomplete="off" id="start">Start</button>
 +				</h1>
 +			</div>
 +			<div class="container" id="details">
 +				<div class="row">
 +					<div class="col-md-12">
 +						<h3>Demo details</h3>
 +						<p>This is basically a clone of the plain <a href="videoroomtest.hmtl">Video Room</a>
 +						demo, but with a key difference: it forces VP9 on all publishers, and supports
 +						the VP9 SVC layer selection (if you don't know what this means, check this
 +						<a target="_blank" href="https://webrtchacks.com/chrome-vp9-svc/">excellent blog post</a>).
 +						As such, it will allow viewers to select which layer, spatial or temporal, to receive
 +						from the publishers, in order to receive more or less data according to what they
- 						can/want to get.</p>
++						can/want to get. Notice that this is fundamentally different from the concept of
++						simulcasting, and so must not be confused with the VP8 simulcasting some of the
++						plugins (including the VideoRoom itself) support: with simulcasting, different
++						streams with a different resolution/bitrate are sent at the same time; with SVC,
++						you have a single stream with multiple layers, which makes it more efficient,
++						especially because it consumes much less bandwidth.</p>
 +						<p>Notice that this only works if the publishers joining the room will use a recent
 +						Chrome version that has been started with the following flag:</p>
 +						<p><div class="alert alert-info"><code>--force-fieldtrials=WebRTC-SupportVP9SVC/EnabledByFlag_2SL3TL/</code></div></p>
 +						<p>If you join with a Chrome that doesn't have that option set, or with
 +						another browser, you'll be able to select a layer, but all request to set
 +						a custom layer on your video will fail, meaning you'll act as a plain VP9 client.
 +						If you ask for a layer from a publisher that doesn't support VP9 SVC either,
 +						then your request will be ignored as well.</p>
 +						<p>In case some publishers do support the VP9 SVC features, selecting a different
 +						layer from them should result in variations of the video quality, and a notable
 +						difference in the bandwidth used to receive their video as well.</p>
 +						<p>Press the <code>Start</code> button above to launch the demo.</p>
 +					</div>
 +				</div>
 +			</div>
 +			<div class="container hide" id="videojoin">
 +				<div class="row">
 +					<span class="label label-info" id="you"></span>
 +					<div class="col-md-12" id="controls">
 +						<div class="input-group margin-bottom-md hide" id="registernow">
 +							<span class="input-group-addon">@</span>
 +							<input autocomplete="off" class="form-control" autocomplete="off" type="text" placeholder="Choose a display name" id="username" onkeypress="return checkEnter(this, event);"></input>
 +							<span class="input-group-btn">
 +								<button class="btn btn-success" autocomplete="off" id="register">Join the room</button>
 +							</span>
 +						</div>
 +					</div>
 +				</div>
 +			</div>
 +			<div class="container hide" id="videos">
 +				<div class="row">
 +					<div class="col-md-4">
 +						<div class="panel panel-default">
 +							<div class="panel-heading">
 +								<h3 class="panel-title">Local Video <span class="label label-primary hide" id="publisher"></span>
 +									<div class="btn-group btn-group-xs pull-right hide">
 +										<div class="btn-group btn-group-xs">
 +											<button id="bitrateset" autocomplete="off" class="btn btn-primary dropdown-toggle" data-toggle="dropdown">
 +												Bandwidth<span class="caret"></span>
 +											</button>
 +											<ul id="bitrate" class="dropdown-menu" role="menu">
 +												<li><a href="#" id="0">No limit</a></li>
 +												<li><a href="#" id="128">Cap to 128kbit</a></li>
 +												<li><a href="#" id="256">Cap to 256kbit</a></li>
 +												<li><a href="#" id="512">Cap to 512kbit</a></li>
 +												<li><a href="#" id="1024">Cap to 1mbit</a></li>
 +												<li><a href="#" id="1500">Cap to 1.5mbit</a></li>
 +												<li><a href="#" id="2000">Cap to 2mbit</a></li>
 +											</ul>
 +										</div>
 +									</div>
 +								</h3>
 +							</div>
 +							<div class="panel-body" id="videolocal"></div>
 +						</div>
 +					</div>
 +					<div class="col-md-4">
 +						<div class="panel panel-default">
 +							<div class="panel-heading">
 +								<h3 class="panel-title">Remote Video #1 <span class="label label-info hide" id="remote1"></span></h3>
 +							</div>
 +							<div class="panel-body relative" id="videoremote1"></div>
 +						</div>
 +					</div>
 +					<div class="col-md-4">
 +						<div class="panel panel-default">
 +							<div class="panel-heading">
 +								<h3 class="panel-title">Remote Video #2 <span class="label label-info hide" id="remote2"></span></h3>
 +							</div>
 +							<div class="panel-body relative" id="videoremote2"></div>
 +						</div>
 +					</div>
 +				</div>
 +				<div class="row">
 +					<div class="col-md-4">
 +						<div class="panel panel-default">
 +							<div class="panel-heading">
 +								<h3 class="panel-title">Remote Video #3 <span class="label label-info hide" id="remote3"></span></h3>
 +							</div>
 +							<div class="panel-body relative" id="videoremote3"></div>
 +						</div>
 +					</div>
 +					<div class="col-md-4">
 +						<div class="panel panel-default">
 +							<div class="panel-heading">
 +								<h3 class="panel-title">Remote Video #4 <span class="label label-info hide" id="remote4"></span></h3>
 +							</div>
 +							<div class="panel-body relative" id="videoremote4"></div>
 +						</div>
 +					</div>
 +					<div class="col-md-4">
 +						<div class="panel panel-default">
 +							<div class="panel-heading">
 +								<h3 class="panel-title">Remote Video #5 <span class="label label-info hide" id="remote5"></span></h3>
 +							</div>
 +							<div class="panel-body relative" id="videoremote5"></div>
 +						</div>
 +					</div>
 +				</div>
 +			</div>
 +		</div>
 +	</div>
 +
 +	<hr>
 +	<div class="footer">
 +	</div>
 +</div>
 +
 +</body>
 +</html>
diff --cc plugins/janus_videoroom.c
index 5080c1a,b5477d6..e8a63dd
--- a/plugins/janus_videoroom.c
+++ b/plugins/janus_videoroom.c
@@@ -303,8 -307,8 +309,12 @@@ static struct janus_json_parameter conf
  	{"audio", JANUS_JSON_BOOL, 0},
  	{"video", JANUS_JSON_BOOL, 0},
  	{"data", JANUS_JSON_BOOL, 0},
++	/* For VP8 simulcast */
+ 	{"substream", JSON_INTEGER, JANUS_JSON_PARAM_POSITIVE},
 -	{"temporal", JSON_INTEGER, JANUS_JSON_PARAM_POSITIVE}
++	{"temporal", JSON_INTEGER, JANUS_JSON_PARAM_POSITIVE},
++	/* For VP9 SVC */
 +	{"spatial_layer", JSON_INTEGER, JANUS_JSON_PARAM_POSITIVE},
 +	{"temporal_layer", JSON_INTEGER, JANUS_JSON_PARAM_POSITIVE}
  };
  static struct janus_json_parameter listener_parameters[] = {
  	{"feed", JSON_INTEGER, JANUS_JSON_PARAM_REQUIRED | JANUS_JSON_PARAM_POSITIVE},
@@@ -465,11 -468,10 +475,11 @@@ typedef struct janus_videoroom 
  	uint16_t fir_freq;			/* Regular FIR frequency (0=disabled) */
  	janus_videoroom_audiocodec acodec;	/* Audio codec to force on publishers*/
  	janus_videoroom_videocodec vcodec;	/* Video codec to force on publishers*/
 +	gboolean do_svc;			/* Whether SVC must be done for video (note: only available for VP9 right now) */
  	gboolean audiolevel_ext;	/* Whether the ssrc-audio-level extension must be negotiated or not for new publishers */
  	gboolean audiolevel_event;	/* Whether to emit event to other users about audiolevel */
- 	int audio_active_packets;	/* amount of packets with audio level for checkup */
- 	int audio_level_average;	/* average audio level */
+ 	int audio_active_packets;	/* Amount of packets with audio level for checkup */
+ 	int audio_level_average;	/* Average audio level */
  	gboolean videoorient_ext;	/* Whether the video-orientation extension must be negotiated or not for new publishers */
  	gboolean playoutdelay_ext;	/* Whether the playout-delay extension must be negotiated or not for new publishers */
  	gboolean record;			/* Whether the feeds from publishers in this room should be recorded */
@@@ -562,12 -568,15 +576,19 @@@ typedef struct janus_videoroom_listene
  	janus_videoroom_participant *feed;	/* Participant this listener is subscribed to */
  	guint32 pvt_id;		/* Private ID of the participant that is subscribing (if available/provided) */
  	janus_rtp_switching_context context;	/* Needed in case there are publisher switches on this listener */
 -	int substream;			/* Which simulcast substream we should forward, in case the publisher is simulcasting */
++	int substream;			/* Which VP8 simulcast substream we should forward, in case the publisher is simulcasting */
+ 	int substream_target;	/* As above, but to handle transitions (e.g., wait for keyframe) */
 -	int templayer;			/* Which simulcast temporal layer we should forward, in case the publisher is simulcasting */
++	int templayer;			/* Which VP8 simulcast temporal layer we should forward, in case the publisher is simulcasting */
+ 	int templayer_target;	/* As above, but to handle transitions (e.g., wait for keyframe) */
+ 	gint64 last_relayed;	/* When we relayed the last packet (used to detect when substreams become unavailable) */
+ 	janus_vp8_simulcast_context simulcast_context;
  	gboolean audio, video, data;		/* Whether audio, video and/or data must be sent to this publisher */
  	gboolean paused;
  	gboolean kicked;	/* Whether this subscription belongs to a participant that has been kicked */
- 	/* The following are only relevant if we're doing VP9 SVC*/
++	/* The following are only relevant if we're doing VP9 SVC, and are not to be confused with VP8
++	 * simulcast, which has similar info (substream/templayer) but in a completely different context */
 +	int spatial_layer, target_spatial_layer;
 +	int temporal_layer, target_temporal_layer;
  } janus_videoroom_listener;
  static void janus_videoroom_listener_free(janus_videoroom_listener *l);
  
@@@ -575,13 -584,9 +596,14 @@@ typedef struct janus_videoroom_rtp_rela
  	rtp_header *data;
  	gint length;
  	gboolean is_video;
+ 	uint32_t ssrc[3];
  	uint32_t timestamp;
  	uint16_t seq_number;
 +	/* The following are only relevant if we're doing VP9 SVC*/
 +	gboolean svc;
 +	int spatial_layer;
 +	int temporal_layer;
 +	uint8_t pbit, dbit, ubit, bbit, ebit;
  } janus_videoroom_rtp_relay_packet;
  
  
@@@ -1207,15 -1206,14 +1232,22 @@@ json_t *janus_videoroom_query_session(j
  				json_object_set_new(media, "audio", json_integer(participant->audio));
  				json_object_set_new(media, "video", json_integer(participant->video));
  				json_object_set_new(media, "data", json_integer(participant->data));
+ 				if(feed->ssrc[0] != 0) {
+ 					json_object_set_new(info, "simulcast", json_true());
+ 					json_object_set_new(info, "substream", json_integer(participant->substream));
+ 					json_object_set_new(info, "substream-target", json_integer(participant->substream_target));
+ 					json_object_set_new(info, "temporal-layer", json_integer(participant->templayer));
+ 					json_object_set_new(info, "temporal-layer-target", json_integer(participant->templayer_target));
+ 				}
  				json_object_set_new(info, "media", media);
 +				if(participant->room && participant->room->do_svc) {
 +					json_t *svc = json_object();
 +					json_object_set_new(svc, "spatial-layer", json_integer(participant->spatial_layer));
 +					json_object_set_new(svc, "target-spatial-layer", json_integer(participant->target_spatial_layer));
 +					json_object_set_new(svc, "temporal-layer", json_integer(participant->temporal_layer));
 +					json_object_set_new(svc, "target-temporal-layer", json_integer(participant->target_temporal_layer));
 +					json_object_set_new(info, "svc", svc);
 +				}
  			}
  		}
  	}
@@@ -2447,28 -2486,9 +2533,31 @@@ void janus_videoroom_incoming_rtp(janus
  		packet.data = rtp;
  		packet.length = len;
  		packet.is_video = video;
 +		packet.svc = FALSE;
 +		if(video && videoroom->do_svc) {
 +			/* We're doing SVC: let's parse this packet to see which layers are there */
 +			int plen = 0;
 +			char *payload = janus_rtp_payload(buf, len, &plen);
 +			if(payload == NULL)
 +				return;
 +			uint8_t pbit = 0, dbit = 0, ubit = 0, bbit = 0, ebit = 0;
 +			int found = 0, spatial_layer = 0, temporal_layer = 0;
 +			if(janus_vp9_parse_svc(payload, plen, &found, &spatial_layer, &temporal_layer, &pbit, &dbit, &ubit, &bbit, &ebit) == 0) {
 +				if(found) {
 +					packet.svc = TRUE;
 +					packet.spatial_layer = spatial_layer;
 +					packet.temporal_layer = temporal_layer;
 +					packet.pbit = pbit;
 +					packet.dbit = dbit;
 +					packet.ubit = ubit;
 +					packet.bbit = bbit;
 +					packet.ebit = ebit;
 +				}
 +			}
 +		}
+ 		packet.ssrc[0] = (sc != -1 ? participant->ssrc[0] : 0);
+ 		packet.ssrc[1] = (sc != -1 ? participant->ssrc[1] : 0);
+ 		packet.ssrc[2] = (sc != -1 ? participant->ssrc[2] : 0);
  		/* Backup the actual timestamp and sequence number set by the publisher, in case switching is involved */
  		packet.timestamp = ntohl(packet.data->timestamp);
  		packet.seq_number = ntohs(packet.data->seq_number);
@@@ -3168,15 -3188,13 +3257,21 @@@ static void *janus_videoroom_handler(vo
  					if(!publisher->data)
  						listener->data = FALSE;	/* ... unless the publisher isn't sending any data */
  					listener->paused = TRUE;	/* We need an explicit start from the listener */
+ 					listener->substream = -1;
+ 					listener->substream_target = 2;
+ 					listener->templayer = -1;
+ 					listener->templayer_target = 2;
+ 					listener->last_relayed = 0;
+ 					janus_vp8_simulcast_context_reset(&listener->simulcast_context);
  					session->participant = listener;
 +					if(videoroom->do_svc) {
 +						/* This listener belongs to a room where VP9 SVC has been enabled,
 +						 * let's assume we're interested in all layers for the time being */
 +						listener->spatial_layer = -1;
 +						listener->target_spatial_layer = 1;		/* FIXME Chrome sends 0 and 1 */
 +						listener->temporal_layer = -1;
 +						listener->target_temporal_layer = 2;	/* FIXME Chrome sends 0, 1 and 2 */
 +					}
  					janus_mutex_lock(&publisher->listeners_mutex);
  					publisher->listeners = g_slist_append(publisher->listeners, listener);
  					janus_mutex_unlock(&publisher->listeners_mutex);
@@@ -3465,8 -3483,20 +3560,22 @@@
  				json_t *audio = json_object_get(root, "audio");
  				json_t *video = json_object_get(root, "video");
  				json_t *data = json_object_get(root, "data");
 -				json_t *substream = json_object_get(root, "substream");
 -				if(json_integer_value(substream) > 2) {
 +				json_t *spatial = json_object_get(root, "spatial_layer");
 +				json_t *temporal = json_object_get(root, "temporal_layer");
++				json_t *sc_substream = json_object_get(root, "substream");
++				if(json_integer_value(sc_substream) > 2) {
+ 					JANUS_LOG(LOG_ERR, "Invalid element (substream should be 0, 1 or 2)\n");
+ 					error_code = JANUS_VIDEOROOM_ERROR_INVALID_ELEMENT;
+ 					g_snprintf(error_cause, 512, "Invalid value (substream should be 0, 1 or 2)");
+ 					goto error;
+ 				}
 -				json_t *temporal = json_object_get(root, "temporal");
 -				if(json_integer_value(temporal) > 2) {
++				json_t *sc_temporal = json_object_get(root, "temporal");
++				if(json_integer_value(sc_temporal) > 2) {
+ 					JANUS_LOG(LOG_ERR, "Invalid element (temporal should be 0, 1 or 2)\n");
+ 					error_code = JANUS_VIDEOROOM_ERROR_INVALID_ELEMENT;
+ 					g_snprintf(error_cause, 512, "Invalid value (temporal should be 0, 1 or 2)");
+ 					goto error;
+ 				}
  				/* Update the audio/video/data flags, if set */
  				janus_videoroom_participant *publisher = listener->feed;
  				if(publisher) {
@@@ -3476,52 -3506,55 +3585,101 @@@
  						listener->video = json_is_true(video);
  					if(data && publisher->data)
  						listener->data = json_is_true(data);
 -					if(substream) {
 -						listener->substream_target = json_integer_value(substream);
++					/* Check if a simulcasting-related request is involved */
++					if(sc_substream && publisher->ssrc[0] != 0) {
++						listener->substream_target = json_integer_value(sc_substream);
+ 						JANUS_LOG(LOG_VERB, "Setting video SSRC to let through (simulcast): %"SCNu32" (index %d, was %d)\n",
+ 							publisher->ssrc[listener->substream], listener->substream_target, listener->substream);
+ 						if(listener->substream_target == listener->substream) {
+ 							/* No need to do anything, we're already getting the right substream, so notify the user */
+ 							json_t *event = json_object();
+ 							json_object_set_new(event, "videoroom", json_string("event"));
+ 							json_object_set_new(event, "room", json_integer(listener->room->room_id));
+ 							json_object_set_new(event, "substream", json_integer(listener->substream));
+ 							gateway->push_event(msg->handle, &janus_videoroom_plugin, NULL, event, NULL);
+ 							json_decref(event);
+ 						} else {
+ 							/* Send a FIR */
+ 							char buf[20];
+ 							janus_rtcp_fir((char *)&buf, 20, &publisher->fir_seq);
+ 							JANUS_LOG(LOG_VERB, "Simulcasting substream change, sending FIR to %"SCNu64" (%s)\n", publisher->user_id, publisher->display ? publisher->display : "??");
+ 							gateway->relay_rtcp(publisher->session->handle, 1, buf, 20);
+ 							/* Send a PLI too, just in case... */
+ 							janus_rtcp_pli((char *)&buf, 12);
+ 							JANUS_LOG(LOG_VERB, "Simulcasting substream change, sending PLI to %"SCNu64" (%s)\n", publisher->user_id, publisher->display ? publisher->display : "??");
+ 							gateway->relay_rtcp(publisher->session->handle, 1, buf, 12);
+ 						}
+ 					}
 -					if(temporal) {
 -						listener->templayer_target = json_integer_value(temporal);
++					if(sc_temporal && publisher->ssrc[0] != 0) {
++						listener->templayer_target = json_integer_value(sc_temporal);
+ 						JANUS_LOG(LOG_VERB, "Setting video temporal layer to let through (simulcast): %d (was %d)\n",
+ 							listener->templayer_target, listener->templayer);
+ 						if(listener->templayer_target == listener->templayer) {
+ 							/* No need to do anything, we're already getting the right temporal, so notify the user */
+ 							json_t *event = json_object();
+ 							json_object_set_new(event, "videoroom", json_string("event"));
+ 							json_object_set_new(event, "room", json_integer(listener->room->room_id));
+ 							json_object_set_new(event, "temporal", json_integer(listener->templayer));
+ 							gateway->push_event(msg->handle, &janus_videoroom_plugin, NULL, event, NULL);
+ 							json_decref(event);
+ 						} else {
+ 							/* Send a FIR */
+ 							char buf[20];
+ 							janus_rtcp_fir((char *)&buf, 20, &publisher->fir_seq);
+ 							JANUS_LOG(LOG_VERB, "Simulcasting temporal layer change, sending FIR to %"SCNu64" (%s)\n", publisher->user_id, publisher->display ? publisher->display : "??");
+ 							gateway->relay_rtcp(publisher->session->handle, 1, buf, 20);
+ 							/* Send a PLI too, just in case... */
+ 							janus_rtcp_pli((char *)&buf, 12);
+ 							JANUS_LOG(LOG_VERB, "Simulcasting temporal layer change, sending PLI to %"SCNu64" (%s)\n", publisher->user_id, publisher->display ? publisher->display : "??");
+ 							gateway->relay_rtcp(publisher->session->handle, 1, buf, 12);
+ 						}
+ 					}
  				}
 +				if(listener->room->do_svc) {
 +					/* Also check if the viewer is trying to configure a layer change */
 +					if(spatial) {
 +						int spatial_layer = json_integer_value(spatial);
 +						if(spatial_layer > 1) {
 +							JANUS_LOG(LOG_WARN, "Spatial layer higher than 1, will probably be ignored\n");
 +						}
 +						if(spatial_layer == listener->spatial_layer) {
 +							/* No need to do anything, we're already getting the right spatial layer, so notify the user */
 +							json_t *event = json_object();
 +							json_object_set_new(event, "videoroom", json_string("event"));
 +							json_object_set_new(event, "room", json_integer(listener->room->room_id));
 +							json_object_set_new(event, "spatial_layer", json_integer(listener->spatial_layer));
 +							gateway->push_event(msg->handle, &janus_videoroom_plugin, NULL, event, NULL);
 +							json_decref(event);
 +						} else if(spatial_layer != listener->target_spatial_layer) {
 +							/* Send a FIR to the new RTP forward publisher */
 +							char buf[20];
 +							janus_rtcp_fir((char *)&buf, 20, &publisher->fir_seq);
 +							JANUS_LOG(LOG_VERB, "Need to downscale spatially, sending FIR to %"SCNu64" (%s)\n", publisher->user_id, publisher->display ? publisher->display : "??");
 +							gateway->relay_rtcp(publisher->session->handle, 1, buf, 20);
 +							/* Send a PLI too, just in case... */
 +							janus_rtcp_pli((char *)&buf, 12);
 +							JANUS_LOG(LOG_VERB, "Need to downscale spatially, sending PLI to %"SCNu64" (%s)\n", publisher->user_id, publisher->display ? publisher->display : "??");
 +							gateway->relay_rtcp(publisher->session->handle, 1, buf, 12);
 +						}
 +						listener->target_spatial_layer = spatial_layer;
 +					}
 +					if(temporal) {
 +						int temporal_layer = json_integer_value(temporal);
 +						if(temporal_layer > 2) {
 +							JANUS_LOG(LOG_WARN, "Temporal layer higher than 2, will probably be ignored\n");
 +						}
 +						if(temporal_layer == listener->temporal_layer) {
 +							/* No need to do anything, we're already getting the right temporal layer, so notify the user */
 +							json_t *event = json_object();
 +							json_object_set_new(event, "videoroom", json_string("event"));
 +							json_object_set_new(event, "room", json_integer(listener->room->room_id));
 +							json_object_set_new(event, "temporal_layer", json_integer(listener->temporal_layer));
 +							gateway->push_event(msg->handle, &janus_videoroom_plugin, NULL, event, NULL);
 +							json_decref(event);
 +						}
 +						listener->target_temporal_layer = temporal_layer;
 +					}
 +				}
  				event = json_object();
  				json_object_set_new(event, "videoroom", json_string("event"));
  				json_object_set_new(event, "room", json_integer(listener->room->room_id));
@@@ -3993,111 -4032,126 +4164,230 @@@ static void janus_videoroom_relay_rtp_p
  			/* Nope, don't relay */
  			return;
  		}
 -		if(packet->ssrc[0] != 0) {
 +		/* Check if there's any SVC info to take into account */
- 		gboolean override_mark_bit = FALSE, has_marker_bit = packet->data->markerbit;
 +		if(packet->svc) {
 +			/* There is: check if this is a layer that can be dropped for this viewer
 +			 * Note: Following core inspired by the excellent job done by Sergio Garcia Murillo here:
 +			 * https://github.com/medooze/media-server/blob/master/src/vp9/VP9LayerSelector.cpp */
++			gboolean override_mark_bit = FALSE, has_marker_bit = packet->data->markerbit;
 +			int temporal_layer = listener->temporal_layer;
 +			if(listener->target_temporal_layer > listener->temporal_layer) {
 +				/* We need to upscale */
 +				JANUS_LOG(LOG_HUGE, "We need to upscale temporally:\n");
 +				if(packet->ubit && packet->bbit && packet->temporal_layer <= listener->target_temporal_layer) {
 +					JANUS_LOG(LOG_HUGE, "  -- Upscaling temporal layer: %u --> %u\n",
 +						packet->temporal_layer, listener->target_temporal_layer);
 +					listener->temporal_layer = packet->temporal_layer;
 +					temporal_layer = listener->temporal_layer;
 +					/* Notify the viewer */
 +					json_t *event = json_object();
 +					json_object_set_new(event, "videoroom", json_string("event"));
 +					json_object_set_new(event, "room", json_integer(listener->room->room_id));
 +					json_object_set_new(event, "temporal_layer", json_integer(listener->temporal_layer));
 +					gateway->push_event(listener->session->handle, &janus_videoroom_plugin, NULL, event, NULL);
 +					json_decref(event);
 +				}
 +			} else if(listener->target_temporal_layer < listener->temporal_layer) {
 +				/* We need to downscale */
 +				JANUS_LOG(LOG_HUGE, "We need to downscale temporally:\n");
 +				if(packet->ebit) {
 +					JANUS_LOG(LOG_HUGE, "  -- Downscaling temporal layer: %u --> %u\n",
 +						listener->temporal_layer, listener->target_temporal_layer);
 +					listener->temporal_layer = listener->target_temporal_layer;
 +					/* Notify the viewer */
 +					json_t *event = json_object();
 +					json_object_set_new(event, "videoroom", json_string("event"));
 +					json_object_set_new(event, "room", json_integer(listener->room->room_id));
 +					json_object_set_new(event, "temporal_layer", json_integer(listener->temporal_layer));
 +					gateway->push_event(listener->session->handle, &janus_videoroom_plugin, NULL, event, NULL);
 +					json_decref(event);
 +				}
 +			}
 +			if(temporal_layer < packet->temporal_layer) {
 +				/* Drop the packet: update the context to make sure sequence number is increased normally later */
 +				JANUS_LOG(LOG_HUGE, "Dropping packet (temporal layer %d < %d)\n", temporal_layer, packet->temporal_layer);
 +				listener->context.v_base_seq++;
 +				return;
 +			}
 +			int spatial_layer = listener->spatial_layer;
 +			if(listener->target_spatial_layer > listener->spatial_layer) {
 +				JANUS_LOG(LOG_HUGE, "We need to upscale spatially:\n");
 +				/* We need to upscale */
 +				if(packet->pbit == 0 && packet->bbit && packet->spatial_layer == listener->spatial_layer+1) {
 +					JANUS_LOG(LOG_HUGE, "  -- Upscaling spatial layer: %u --> %u\n",
 +						packet->spatial_layer, listener->target_spatial_layer);
 +					listener->spatial_layer = packet->spatial_layer;
 +					spatial_layer = listener->spatial_layer;
 +					/* Notify the viewer */
 +					json_t *event = json_object();
 +					json_object_set_new(event, "videoroom", json_string("event"));
 +					json_object_set_new(event, "room", json_integer(listener->room->room_id));
 +					json_object_set_new(event, "spatial_layer", json_integer(listener->spatial_layer));
 +					gateway->push_event(listener->session->handle, &janus_videoroom_plugin, NULL, event, NULL);
 +					json_decref(event);
 +				}
 +			} else if(listener->target_spatial_layer < listener->spatial_layer) {
 +				/* We need to downscale */
 +				JANUS_LOG(LOG_HUGE, "We need to downscale spatially:\n");
 +				if(packet->ebit) {
 +					JANUS_LOG(LOG_HUGE, "  -- Downscaling spatial layer: %u --> %u\n",
 +						listener->spatial_layer, listener->target_spatial_layer);
 +					listener->spatial_layer = listener->target_spatial_layer;
 +					/* Notify the viewer */
 +					json_t *event = json_object();
 +					json_object_set_new(event, "videoroom", json_string("event"));
 +					json_object_set_new(event, "room", json_integer(listener->room->room_id));
 +					json_object_set_new(event, "spatial_layer", json_integer(listener->spatial_layer));
 +					gateway->push_event(listener->session->handle, &janus_videoroom_plugin, NULL, event, NULL);
 +					json_decref(event);
 +				}
 +			}
 +			if(spatial_layer < packet->spatial_layer) {
 +				/* Drop the packet: update the context to make sure sequence number is increased normally later */
 +				JANUS_LOG(LOG_HUGE, "Dropping packet (spatial layer %d < %d)\n", spatial_layer, packet->spatial_layer);
 +				listener->context.v_base_seq++;
 +				return;
 +			} else if(packet->ebit && spatial_layer == packet->spatial_layer) {
 +				/* If we stop at layer 0, we need a marker bit now, as the one from layer 1 will not be received */
 +				override_mark_bit = TRUE;
 +			}
 +			/* If we got here, we can send the frame: this doesn't necessarily mean it's
 +			 * one of the layers the user wants, as there may be dependencies involved */
 +			JANUS_LOG(LOG_HUGE, "Sending packet (spatial=%d, temporal=%d)\n",
 +				packet->spatial_layer, packet->temporal_layer);
++			/* Fix sequence number and timestamp (publisher switching may be involved) */
++			janus_rtp_header_update(packet->data, &listener->context, TRUE, 4500);
++			if(override_mark_bit && !has_marker_bit) {
++				packet->data->markerbit = 1;
++			}
++			if(gateway != NULL)
++				gateway->relay_rtp(session->handle, packet->is_video, (char *)packet->data, packet->length);
++			if(override_mark_bit && !has_marker_bit) {
++				packet->data->markerbit = 0;
++			}
++			/* Restore the timestamp and sequence number to what the publisher set them to */
++			packet->data->timestamp = htonl(packet->timestamp);
++			packet->data->seq_number = htons(packet->seq_number);
++		} else if(packet->ssrc[0] != 0) {
+ 			/* Handle simulcast: don't relay if it's not the SSRC we wanted to handle */
+ 			uint32_t ssrc = ntohl(packet->data->ssrc);
+ 			int plen = 0;
+ 			char *payload = janus_rtp_payload((char *)packet->data, packet->length, &plen);
+ 			if(payload == NULL)
+ 				return;
+ 			gboolean switched = FALSE;
+ 			if(listener->substream != listener->substream_target) {
+ 				/* There has been a change: let's wait for a keyframe on the target */
+ 				int step = (listener->substream < 1 && listener->substream_target == 2);
+ 				if(ssrc == packet->ssrc[listener->substream_target] || (step && ssrc == packet->ssrc[step])) {
+ 					//~ if(janus_vp8_is_keyframe(payload, plen)) {
+ 						uint32_t ssrc_old = 0;
+ 						if(listener->substream != -1)
+ 							ssrc_old = packet->ssrc[listener->substream];
+ 						JANUS_LOG(LOG_WARN, "Received keyframe on SSRC %"SCNu32", switching (was %"SCNu32")\n", ssrc, ssrc_old);
+ 						listener->substream = (ssrc == packet->ssrc[listener->substream_target] ? listener->substream_target : step);;
+ 						switched = TRUE;
+ 						/* Notify the viewer */
+ 						json_t *event = json_object();
+ 						json_object_set_new(event, "videoroom", json_string("event"));
+ 						json_object_set_new(event, "room", json_integer(listener->room->room_id));
+ 						json_object_set_new(event, "substream", json_integer(listener->substream));
+ 						gateway->push_event(listener->session->handle, &janus_videoroom_plugin, NULL, event, NULL);
+ 						json_decref(event);
+ 					//~ } else {
+ 						//~ JANUS_LOG(LOG_WARN, "Not a keyframe on SSRC %"SCNu32" yet, waiting before switching\n", ssrc);
+ 					//~ }
+ 				}
+ 			}
+ 			/* If we haven't received our desired substream yet, let's drop temporarily */
+ 			if(listener->last_relayed == 0) {
+ 				/* Let's start slow */
+ 				listener->last_relayed = janus_get_monotonic_time();
+ 			} else {
+ 				/* Check if 250ms went by with no packet relayed */
+ 				gint64 now = janus_get_monotonic_time();
+ 				if(now-listener->last_relayed >= 250000) {
+ 					listener->last_relayed = now;
+ 					int substream = listener->substream-1;
+ 					if(substream < 0)
+ 						substream = 0;
+ 					if(listener->substream != substream) {
+ 						JANUS_LOG(LOG_WARN, "No packet received on substream %d for a while, falling back to %d\n",
+ 							listener->substream, substream);
+ 						listener->substream = substream;
+ 						/* Send a PLI */
+ 						JANUS_LOG(LOG_VERB, "Just (re-)enabled video, sending a PLI to recover it\n");
+ 						char rtcpbuf[12];
+ 						memset(rtcpbuf, 0, 12);
+ 						janus_rtcp_pli((char *)&rtcpbuf, 12);
+ 						if(listener->feed && listener->feed->session && listener->feed->session->handle)
+ 							gateway->relay_rtcp(listener->feed->session->handle, 1, rtcpbuf, 12);
+ 						/* Notify the viewer */
+ 						json_t *event = json_object();
+ 						json_object_set_new(event, "videoroom", json_string("event"));
+ 						json_object_set_new(event, "room", json_integer(listener->room->room_id));
+ 						json_object_set_new(event, "substream", json_integer(listener->substream));
+ 						gateway->push_event(listener->session->handle, &janus_videoroom_plugin, NULL, event, NULL);
+ 						json_decref(event);
+ 					}
+ 				}
+ 			}
+ 			if(ssrc != packet->ssrc[listener->substream]) {
+ 				JANUS_LOG(LOG_HUGE, "Dropping packet (it's from SSRC %"SCNu32", but we're only relaying SSRC %"SCNu32" now\n",
+ 					ssrc, packet->ssrc[listener->substream]);
+ 				return;
+ 			}
+ 			listener->last_relayed = janus_get_monotonic_time();
+ 			/* Check if there's any temporal scalability to take into account */
+ 			uint16_t picid = 0;
+ 			uint8_t tlzi = 0;
+ 			uint8_t tid = 0;
+ 			uint8_t ybit = 0;
+ 			uint8_t keyidx = 0;
+ 			if(janus_vp8_parse_descriptor(payload, plen, &picid, &tlzi, &tid, &ybit, &keyidx) == 0) {
+ 				//~ JANUS_LOG(LOG_WARN, "%"SCNu16", %u, %u, %u, %u\n", picid, tlzi, tid, ybit, keyidx);
+ 				if(listener->templayer != listener->templayer_target) {
+ 					/* FIXME We should be smarter in deciding when to switch */
+ 					listener->templayer = listener->templayer_target;
+ 					/* Notify the user */
+ 					json_t *event = json_object();
+ 					json_object_set_new(event, "videoroom", json_string("event"));
+ 					json_object_set_new(event, "room", json_integer(listener->room->room_id));
+ 					json_object_set_new(event, "temporal", json_integer(listener->templayer));
+ 					gateway->push_event(listener->session->handle, &janus_videoroom_plugin, NULL, event, NULL);
+ 					json_decref(event);
+ 				}
+ 				if(tid > listener->templayer) {
+ 					JANUS_LOG(LOG_HUGE, "Dropping packet (it's temporal layer %d, but we're capping at %d)\n",
+ 						tid, listener->templayer);
+ 					/* We increase the base sequence number, or there will be gaps when delivering later */
+ 					listener->context.v_base_seq++;
+ 					return;
+ 				}
+ 			}
+ 			/* If we got here, update the RTP header and send the packet */
+ 			janus_rtp_header_update(packet->data, &listener->context, TRUE, 4500);
+ 			char vp8pd[6];
+ 			memcpy(vp8pd, payload, sizeof(vp8pd));
+ 			janus_vp8_simulcast_descriptor_update(payload, plen, &listener->simulcast_context, switched);
+ 			/* Send the packet */
+ 			if(gateway != NULL)
+ 				gateway->relay_rtp(session->handle, packet->is_video, (char *)packet->data, packet->length);
+ 			/* Restore the timestamp and sequence number to what the publisher set them to */
+ 			packet->data->timestamp = htonl(packet->timestamp);
+ 			packet->data->seq_number = htons(packet->seq_number);
+ 			/* Restore the original payload descriptor as well, as it will be needed by the next viewer */
+ 			memcpy(payload, vp8pd, sizeof(vp8pd));
+ 		} else {
+ 			/* Fix sequence number and timestamp (publisher switching may be involved) */
+ 			janus_rtp_header_update(packet->data, &listener->context, TRUE, 4500);
+ 			/* Send the packet */
+ 			if(gateway != NULL)
+ 				gateway->relay_rtp(session->handle, packet->is_video, (char *)packet->data, packet->length);
+ 			/* Restore the timestamp and sequence number to what the publisher set them to */
+ 			packet->data->timestamp = htonl(packet->timestamp);
+ 			packet->data->seq_number = htons(packet->seq_number);
  		}
- 		/* Fix sequence number and timestamp (publisher switching may be involved) */
- 		janus_rtp_header_update(packet->data, &listener->context, TRUE, 4500);
- 		if(override_mark_bit && !has_marker_bit) {
- 			packet->data->markerbit = 1;
- 		}
- 		if(gateway != NULL)
- 			gateway->relay_rtp(session->handle, packet->is_video, (char *)packet->data, packet->length);
- 		if(override_mark_bit && !has_marker_bit) {
- 			packet->data->markerbit = 0;
- 		}
- 		/* Restore the timestamp and sequence number to what the publisher set them to */
- 		packet->data->timestamp = htonl(packet->timestamp);
- 		packet->data->seq_number = htons(packet->seq_number);
  	} else {
  		/* Check if this listener is subscribed to this medium */
  		if(!listener->audio) {
diff --cc utils.c
index cf5091d,705dde5..d8bffbd
--- a/utils.c
+++ b/utils.c
@@@ -497,137 -497,306 +497,441 @@@ gboolean janus_json_is_valid(json_t *va
  	return is_valid;
  }
  
+ /* The following code is more related to codec specific helpers */
+ #if defined(__ppc__) || defined(__ppc64__)
+ 	# define swap2(d)  \
+ 	((d&0x000000ff)<<8) |  \
+ 	((d&0x0000ff00)>>8)
+ #else
+ 	# define swap2(d) d
+ #endif
+ 
+ gboolean janus_vp8_is_keyframe(char* buffer, int len) {
+ 	if(!buffer || len < 0)
+ 		return FALSE;
+ 	/* Parse VP8 header now */
+ 	uint8_t vp8pd = *buffer;
+ 	uint8_t xbit = (vp8pd & 0x80);
+ 	uint8_t sbit = (vp8pd & 0x10);
+ 	if(xbit) {
+ 		JANUS_LOG(LOG_HUGE, "  -- X bit is set!\n");
+ 		/* Read the Extended control bits octet */
+ 		buffer++;
+ 		vp8pd = *buffer;
+ 		uint8_t ibit = (vp8pd & 0x80);
+ 		uint8_t lbit = (vp8pd & 0x40);
+ 		uint8_t tbit = (vp8pd & 0x20);
+ 		uint8_t kbit = (vp8pd & 0x10);
+ 		if(ibit) {
+ 			JANUS_LOG(LOG_HUGE, "  -- I bit is set!\n");
+ 			/* Read the PictureID octet */
+ 			buffer++;
+ 			vp8pd = *buffer;
+ 			uint16_t picid = vp8pd, wholepicid = picid;
+ 			uint8_t mbit = (vp8pd & 0x80);
+ 			if(mbit) {
+ 				JANUS_LOG(LOG_HUGE, "  -- M bit is set!\n");
+ 				memcpy(&picid, buffer, sizeof(uint16_t));
+ 				wholepicid = ntohs(picid);
+ 				picid = (wholepicid & 0x7FFF);
+ 				buffer++;
+ 			}
+ 			JANUS_LOG(LOG_HUGE, "  -- -- PictureID: %"SCNu16"\n", picid);
+ 		}
+ 		if(lbit) {
+ 			JANUS_LOG(LOG_HUGE, "  -- L bit is set!\n");
+ 			/* Read the TL0PICIDX octet */
+ 			buffer++;
+ 			vp8pd = *buffer;
+ 		}
+ 		if(tbit || kbit) {
+ 			JANUS_LOG(LOG_HUGE, "  -- T/K bit is set!\n");
+ 			/* Read the TID/KEYIDX octet */
+ 			buffer++;
+ 			vp8pd = *buffer;
+ 		}
+ 		buffer++;	/* Now we're in the payload */
+ 		if(sbit) {
+ 			JANUS_LOG(LOG_HUGE, "  -- S bit is set!\n");
+ 			unsigned long int vp8ph = 0;
+ 			memcpy(&vp8ph, buffer, 4);
+ 			vp8ph = ntohl(vp8ph);
+ 			uint8_t pbit = ((vp8ph & 0x01000000) >> 24);
+ 			if(!pbit) {
+ 				JANUS_LOG(LOG_HUGE, "  -- P bit is NOT set!\n");
+ 				/* It is a key frame! Get resolution for debugging */
+ 				unsigned char *c = (unsigned char *)buffer+3;
+ 				/* vet via sync code */
+ 				if(c[0]!=0x9d||c[1]!=0x01||c[2]!=0x2a) {
+ 					JANUS_LOG(LOG_WARN, "First 3-bytes after header not what they're supposed to be?\n");
+ 				} else {
+ 					int vp8w = swap2(*(unsigned short*)(c+3))&0x3fff;
+ 					int vp8ws = swap2(*(unsigned short*)(c+3))>>14;
+ 					int vp8h = swap2(*(unsigned short*)(c+5))&0x3fff;
+ 					int vp8hs = swap2(*(unsigned short*)(c+5))>>14;
+ 					JANUS_LOG(LOG_HUGE, "Got a VP8 key frame: %dx%d (scale=%dx%d)\n", vp8w, vp8h, vp8ws, vp8hs);
+ 					return TRUE;
+ 				}
+ 			}
+ 		}
+ 	}
+ 	/* If we got here it's not a key frame */
+ 	return FALSE;
+ }
+ 
+ gboolean janus_vp9_is_keyframe(char* buffer, int len) {
+ 	if(!buffer || len < 0)
+ 		return FALSE;
+ 	/* Parse VP9 header now */
+ 	uint8_t vp9pd = *buffer;
+ 	uint8_t ibit = (vp9pd & 0x80);
+ 	uint8_t pbit = (vp9pd & 0x40);
+ 	uint8_t lbit = (vp9pd & 0x20);
+ 	uint8_t fbit = (vp9pd & 0x10);
+ 	uint8_t vbit = (vp9pd & 0x02);
+ 	buffer++;
+ 	if(ibit) {
+ 		/* Read the PictureID octet */
+ 		vp9pd = *buffer;
+ 		uint16_t picid = vp9pd, wholepicid = picid;
+ 		uint8_t mbit = (vp9pd & 0x80);
+ 		if(!mbit) {
+ 			buffer++;
+ 		} else {
+ 			memcpy(&picid, buffer, sizeof(uint16_t));
+ 			wholepicid = ntohs(picid);
+ 			picid = (wholepicid & 0x7FFF);
+ 			buffer += 2;
+ 		}
+ 	}
+ 	if(lbit) {
+ 		buffer++;
+ 		if(!fbit) {
+ 			/* Non-flexible mode, skip TL0PICIDX */
+ 			buffer++;
+ 		}
+ 	}
+ 	if(fbit && pbit) {
+ 		/* Skip reference indices */
+ 		uint8_t nbit = 1;
+ 		while(nbit) {
+ 			vp9pd = *buffer;
+ 			nbit = (vp9pd & 0x01);
+ 			buffer++;
+ 		}
+ 	}
+ 	if(vbit) {
+ 		/* Parse and skip SS */
+ 		vp9pd = *buffer;
+ 		uint n_s = (vp9pd & 0xE0) >> 5;
+ 		n_s++;
+ 		uint8_t ybit = (vp9pd & 0x10);
+ 		if(ybit) {
+ 			/* Iterate on all spatial layers and get resolution */
+ 			buffer++;
+ 			uint i=0;
+ 			for(i=0; i<n_s; i++) {
+ 				/* Width */
+ 				uint16_t *w = (uint16_t *)buffer;
+ 				int vp9w = ntohs(*w);
+ 				buffer += 2;
+ 				/* Height */
+ 				uint16_t *h = (uint16_t *)buffer;
+ 				int vp9h = ntohs(*h);
+ 				buffer += 2;
+ 				JANUS_LOG(LOG_WARN, "Got a VP9 key frame: %dx%d\n", vp9w, vp9h);
+ 				return TRUE;
+ 			}
+ 		}
+ 	}
+ 	/* If we got here it's not a key frame */
+ 	return FALSE;
+ }
+ 
+ gboolean janus_h264_is_keyframe(char* buffer, int len) {
+ 	if(!buffer || len < 0)
+ 		return FALSE;
+ 	/* Parse H264 header now */
+ 	uint8_t fragment = *buffer & 0x1F;
+ 	uint8_t nal = *(buffer+1) & 0x1F;
+ 	uint8_t start_bit = *(buffer+1) & 0x80;
+ 	JANUS_LOG(LOG_HUGE, "Fragment=%d, NAL=%d, Start=%d\n", fragment, nal, start_bit);
+ 	if(fragment == 5 ||
+ 			((fragment == 28 || fragment == 29) && nal == 5 && start_bit == 128)) {
+ 		JANUS_LOG(LOG_HUGE, "Got an H264 key frame\n");
+ 		return TRUE;
+ 	}
+ 	/* If we got here it's not a key frame */
+ 	return FALSE;
+ }
+ 
+ int janus_vp8_parse_descriptor(char *buffer, int len,
+ 		uint16_t *picid, uint8_t *tl0picidx, uint8_t *tid, uint8_t *y, uint8_t *keyidx) {
+ 	if(!buffer || len < 0)
+ 		return -1;
+ 	if(picid)
+ 		*picid = 0;
+ 	if(tl0picidx)
+ 		*tl0picidx = 0;
+ 	if(tid)
+ 		*tid = 0;
+ 	if(y)
+ 		*y = 0;
+ 	if(keyidx)
+ 		*keyidx = 0;
+ 	uint8_t vp8pd = *buffer;
+ 	uint8_t xbit = (vp8pd & 0x80);
+ 	/* Read the Extended control bits octet */
+ 	if(xbit) {
+ 		buffer++;
+ 		vp8pd = *buffer;
+ 		uint8_t ibit = (vp8pd & 0x80);
+ 		uint8_t lbit = (vp8pd & 0x40);
+ 		uint8_t tbit = (vp8pd & 0x20);
+ 		uint8_t kbit = (vp8pd & 0x10);
+ 		if(ibit) {
+ 			/* Read the PictureID octet */
+ 			buffer++;
+ 			vp8pd = *buffer;
+ 			uint16_t partpicid = vp8pd, wholepicid = partpicid;
+ 			uint8_t mbit = (vp8pd & 0x80);
+ 			if(mbit) {
+ 				memcpy(&partpicid, buffer, sizeof(uint16_t));
+ 				wholepicid = ntohs(partpicid);
+ 				partpicid = (wholepicid & 0x7FFF);
+ 				if(picid)
+ 					*picid = partpicid;
+ 				buffer++;
+ 			}
+ 		}
+ 		if(lbit) {
+ 			/* Read the TL0PICIDX octet */
+ 			buffer++;
+ 			vp8pd = *buffer;
+ 			if(tl0picidx)
+ 				*tl0picidx = vp8pd;
+ 		}
+ 		if(tbit || kbit) {
+ 			/* Read the TID/Y/KEYIDX octet */
+ 			buffer++;
+ 			vp8pd = *buffer;
+ 			if(tid)
+ 				*tid = (vp8pd & 0xC0) >> 6;
+ 			if(y)
+ 				*y = (vp8pd & 0x20) >> 5;
+ 			if(keyidx)
+ 				*keyidx = (vp8pd & 0x1F) >> 4;
+ 		}
+ 	}
+ 	return 0;
+ }
+ 
+ static int janus_vp8_replace_descriptor(char *buffer, int len, uint16_t picid, uint8_t tl0picidx) {
+ 	if(!buffer || len < 0)
+ 		return -1;
+ 	uint8_t vp8pd = *buffer;
+ 	uint8_t xbit = (vp8pd & 0x80);
+ 	/* Read the Extended control bits octet */
+ 	if(xbit) {
+ 		buffer++;
+ 		vp8pd = *buffer;
+ 		uint8_t ibit = (vp8pd & 0x80);
+ 		uint8_t lbit = (vp8pd & 0x40);
+ 		uint8_t tbit = (vp8pd & 0x20);
+ 		uint8_t kbit = (vp8pd & 0x10);
+ 		if(ibit) {
+ 			/* Overwrite the PictureID octet */
+ 			buffer++;
+ 			vp8pd = *buffer;
+ 			uint8_t mbit = (vp8pd & 0x80);
+ 			if(!mbit) {
+ 				*buffer = picid;
+ 			} else {
+ 				uint16_t wholepicid = htons(picid);
+ 				memcpy(buffer, &wholepicid, 2);
+ 				*buffer |= 0x80;
+ 				buffer++;
+ 			}
+ 		}
+ 		if(lbit) {
+ 			/* Overwrite the TL0PICIDX octet */
+ 			buffer++;
+ 			*buffer = tl0picidx;
+ 		}
+ 		if(tbit || kbit) {
+ 			/* Should we overwrite the TID/Y/KEYIDX octet? */
+ 			buffer++;
+ 		}
+ 	}
+ 	return 0;
+ }
+ 
+ void janus_vp8_simulcast_context_reset(janus_vp8_simulcast_context *context) {
+ 	if(context == NULL)
+ 		return;
+ 	/* Reset the context values */
+ 	context->last_picid = 0;
+ 	context->base_picid = 0;
+ 	context->base_picid_prev = 0;
+ 	context->last_tlzi = 0;
+ 	context->base_tlzi = 0;
+ 	context->base_tlzi_prev = 0;
+ }
+ 
+ void janus_vp8_simulcast_descriptor_update(char *buffer, int len, janus_vp8_simulcast_context *context, gboolean switched) {
+ 	if(!buffer || len < 0)
+ 		return;
+ 	uint16_t picid = 0;
+ 	uint8_t tlzi = 0;
+ 	uint8_t tid = 0;
+ 	uint8_t ybit = 0;
+ 	uint8_t keyidx = 0;
+ 	/* Parse the identifiers in the VP8 payload descriptor */
+ 	if(janus_vp8_parse_descriptor(buffer, len, &picid, &tlzi, &tid, &ybit, &keyidx) < 0)
+ 		return;
+ 	if(switched) {
+ 		context->base_picid_prev = context->last_picid;
+ 		context->base_picid = picid;
+ 		context->base_tlzi_prev = context->last_tlzi;
+ 		context->base_tlzi = tlzi;
+ 	}
+ 	context->last_picid = (picid-context->base_picid)+context->base_picid_prev+1;
+ 	context->last_tlzi = (tlzi-context->base_tlzi)+context->base_tlzi_prev+1;
+ 	/* Overwrite the values in the VP8 payload descriptors with the ones we have */
+ 	janus_vp8_replace_descriptor(buffer, len, context->last_picid, context->last_tlzi);
+ }
++
 +/* Helper method to parse a VP9 RTP video frame and get some SVC-related info:
 + * notice that this only works with VP9, right now, on an experimental basis */
 +int janus_vp9_parse_svc(char *buffer, int len, int *found,
 +		int *spatial_layer, int *temporal_layer,
 +		uint8_t *p, uint8_t *d, uint8_t *u, uint8_t *b, uint8_t *e) {
 +	if(!buffer || len < 0)
 +		return -1;
 +	/* VP9 depay: */
 +		/* https://tools.ietf.org/html/draft-ietf-payload-vp9-04 */
 +	/* Read the first octet (VP9 Payload Descriptor) */
 +	uint8_t vp9pd = *buffer;
 +	uint8_t ibit = (vp9pd & 0x80) >> 7;
 +	uint8_t pbit = (vp9pd & 0x40) >> 6;
 +	uint8_t lbit = (vp9pd & 0x20) >> 5;
 +	uint8_t fbit = (vp9pd & 0x10) >> 4;
 +	uint8_t bbit = (vp9pd & 0x08) >> 3;
 +	uint8_t ebit = (vp9pd & 0x04) >> 2;
 +	uint8_t vbit = (vp9pd & 0x02) >> 1;
 +	if(!lbit) {
 +		/* No Layer indices present, no need to go on */
 +		if(found)
 +			*found = 0;
 +		return 0;
 +	}
 +	/* Move to the next octet and see what's there */
 +	buffer++;
 +	len--;
 +	if(ibit) {
 +		/* Read the PictureID octet */
 +		vp9pd = *buffer;
 +		uint16_t picid = vp9pd, wholepicid = picid;
 +		uint8_t mbit = (vp9pd & 0x80);
 +		if(!mbit) {
 +			buffer++;
 +			len--;
 +		} else {
 +			memcpy(&picid, buffer, sizeof(uint16_t));
 +			wholepicid = ntohs(picid);
 +			picid = (wholepicid & 0x7FFF);
 +			buffer += 2;
 +			len -= 2;
 +		}
 +	}
 +	if(lbit) {
 +		/* Read the octet and parse the layer indices now */
 +		vp9pd = *buffer;
 +		int tlid = (vp9pd & 0xE0) >> 5;
 +		uint8_t ubit = (vp9pd & 0x10) >> 4;
 +		int slid = (vp9pd & 0x0E) >> 1;
 +		uint8_t dbit = (vp9pd & 0x01);
 +		JANUS_LOG(LOG_HUGE, "Parsed Layer indices: Temporal: %d (%u), Spatial: %d (%u)\n",
 +			tlid, ubit, slid, dbit);
 +		if(temporal_layer)
 +			*temporal_layer = tlid;
 +		if(spatial_layer)
 +			*spatial_layer = slid;
 +		if(p)
 +			*p = pbit;
 +		if(d)
 +			*d = dbit;
 +		if(u)
 +			*u = ubit;
 +		if(b)
 +			*b = bbit;
 +		if(e)
 +			*e = ebit;
 +		if(found)
 +			*found = 1;
 +		/* Go on, just to get to the SS, if available (which we currently ignore anyway) */
 +		buffer++;
 +		len--;
 +		if(!fbit) {
 +			/* Non-flexible mode, skip TL0PICIDX */
 +			buffer++;
 +			len--;
 +		}
 +	}
 +	if(fbit && pbit) {
 +		/* Skip reference indices */
 +		uint8_t nbit = 1;
 +		while(nbit) {
 +			vp9pd = *buffer;
 +			nbit = (vp9pd & 0x01);
 +			buffer++;
 +			len--;
 +		}
 +	}
 +	if(vbit) {
 +		/* Parse and skip SS */
 +		vp9pd = *buffer;
 +		int n_s = (vp9pd & 0xE0) >> 5;
 +		n_s++;
 +		JANUS_LOG(LOG_HUGE, "There are %d spatial layers\n", n_s);
 +		uint8_t ybit = (vp9pd & 0x10);
 +		uint8_t gbit = (vp9pd & 0x08);
 +		if(ybit) {
 +			/* Iterate on all spatial layers and get resolution */
 +			buffer++;
 +			len--;
 +			int i=0;
 +			for(i=0; i<n_s; i++) {
 +				/* Been there, done that: skip skip skip */
 +				buffer += 4;
 +				len -= 4;
 +			}
 +		}
 +		if(gbit) {
 +			if(!ybit) {
 +				buffer++;
 +				len--;
 +			}
 +			uint8_t n_g = *buffer;
 +			JANUS_LOG(LOG_HUGE, "There are %u frames in a GOF\n", n_g);
 +			buffer++;
 +			len--;
 +			if(n_g > 0) {
 +				int i=0;
 +				for(i=0; i<n_g; i++) {
 +					/* Read the R bits */
 +					vp9pd = *buffer;
 +					int r = (vp9pd & 0x0C) >> 2;
 +					if(r > 0) {
 +						/* Skip reference indices */
 +						buffer += r;
 +						len -= r;
 +					}
 +					buffer++;
 +					len--;
 +				}
 +			}
 +		}
 +	}
 +	return 0;
 +}
diff --cc utils.h
index edd4400,d19e764..3cbb37b
--- a/utils.h
+++ b/utils.h
@@@ -220,20 -220,51 +220,67 @@@ gboolean janus_json_is_valid(json_t *va
  		} \
  	} while(0)
  
- #endif
+ /*! \brief Helper method to check if a VP8 frame is a keyframe or not
+  * @param[in] buffer The RTP payload to process
+  * @param[in] len The length of the RTP payload
+  * @returns TRUE if it's a keyframe, FALSE otherwise */
+ gboolean janus_vp8_is_keyframe(char* buffer, int len);
+ 
+ /*! \brief Helper method to check if a VP9 frame is a keyframe or not
+  * @param[in] buffer The RTP payload to process
+  * @param[in] len The length of the RTP payload
+  * @returns TRUE if it's a keyframe, FALSE otherwise */
+ gboolean janus_vp9_is_keyframe(char* buffer, int len);
+ 
+ /*! \brief Helper method to check if an H.264 frame is a keyframe or not
+  * @param[in] buffer The RTP payload to process
+  * @param[in] len The length of the RTP payload
+  * @returns TRUE if it's a keyframe, FALSE otherwise */
+ gboolean janus_h264_is_keyframe(char* buffer, int len);
+ 
+ /*! \brief VP8 simulcasting context, in order to make sure SSRC changes result in coherent picid/temporal level increases */
+ typedef struct janus_vp8_simulcast_context {
+ 	uint16_t last_picid, base_picid, base_picid_prev;
+ 	uint8_t last_tlzi, base_tlzi, base_tlzi_prev;
+ } janus_vp8_simulcast_context;
+ 
+ /*! \brief Set (or reset) the context fields to their default values
+  * @param[in] context The context to (re)set */
+ void janus_vp8_simulcast_context_reset(janus_vp8_simulcast_context *context);
+ 
+ /*! \brief Helper method to parse a VP8 payload descriptor for useful info (e.g., when simulcasting)
+  * @param[in] buffer The RTP payload to process
+  * @param[in] len The length of the RTP payload
+  * @param[out] picid The Picture ID
+  * @param[out] tl0picidx Temporal level zero index
+  * @param[out] tid Temporal-layer index
+  * @param[out] y Layer sync bit
+  * @param[out] keyidx Temporal key frame index
+  * @returns 0 in case of success, a negative integer otherwise */
+ int janus_vp8_parse_descriptor(char *buffer, int len,
+ 		uint16_t *picid, uint8_t *tl0picidx, uint8_t *tid, uint8_t *y, uint8_t *keyidx);
+ 
+ /*! \brief Use the context info to update the RTP header of a packet, if needed
+  * @param[in] buffer The RTP payload to process
+  * @param[in] len The length of the RTP payload
+  * @param[in] context The context to use as a reference
+  * @param[in] switched Whether there has been a source switch or not (important to compute offsets) */
+ void janus_vp8_simulcast_descriptor_update(char *buffer, int len, janus_vp8_simulcast_context *context, gboolean switched);
  
 +/*! \brief Helper method to parse a VP9 payload descriptor for SVC-related info (e.g., when SVC is enabled)
 + * @param[in] buffer The RTP payload to process
 + * @param[in] len The length of the RTP payload
 + * @param[out] found Whether any SVC related info has been found or not
 + * @param[out] spatial_layer Spatial layer of the packet
 + * @param[out] temporal_layer Temporal layer of the packet
 + * @param[out] p Inter-picture predicted picture bit
 + * @param[out] d Inter-layer dependency used bit
 + * @param[out] u Switching up point bit
 + * @param[out] b Start of a frame bit
 + * @param[out] e End of a frame bit
 + * @returns 0 in case of success, a negative integer otherwise */
 +int janus_vp9_parse_svc(char *buffer, int len, int *found,
 +		int *spatial_layer, int *temporal_layer,
 +		uint8_t *p, uint8_t *d, uint8_t *u, uint8_t *b, uint8_t *e);
++
+ #endif

-- 
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/pkg-voip/janus.git



More information about the Pkg-voip-commits mailing list