[Pkg-voip-commits] [janus] 43/163: Aligned the VP9 SVC branch to the simulcast stuff merged in master
Jonas Smedegaard
dr at jones.dk
Sat Oct 28 01:22:08 UTC 2017
This is an automated email from the git hooks/post-receive script.
js pushed a commit to annotated tag debian/0.2.5-1
in repository janus.
commit 44d6d66d9fa322410171ca2793eac59a678f37c7
Merge: fd8d681 211c2c2
Author: Lorenzo Miniero <lminiero at gmail.com>
Date: Fri Jul 28 11:49:42 2017 +0200
Aligned the VP9 SVC branch to the simulcast stuff merged in master
README.md | 2 +-
conf/janus.plugin.streaming.cfg.sample.in | 3 +
configure.ac | 2 +-
docs/janus-doxygen.cfg | 2 +-
html/devicetest.html | 8 +-
html/devicetest.js | 155 ++++++-
html/echotest.html | 10 +-
html/echotest.js | 155 +++++++
html/janus.js | 245 +++++++++-
html/janus.nojquery.js | 245 +++++++++-
html/streamingtest.html | 8 +-
html/streamingtest.js | 185 ++++++++
html/videocalltest.html | 14 +
html/videocalltest.js | 152 +++++++
html/videoroomtest.html | 32 +-
html/videoroomtest.js | 167 +++++++
html/vp9svctest.html | 7 +-
ice.c | 53 ++-
ice.h | 6 +
janus.c | 46 +-
janus.ggo | 2 +-
plugins/janus_echotest.c | 342 +++++++++++++-
plugins/janus_recordplay.c | 21 +-
plugins/janus_sip.c | 29 +-
plugins/janus_streaming.c | 731 ++++++++++++++++++------------
plugins/janus_videocall.c | 257 ++++++++++-
plugins/janus_videoroom.c | 381 ++++++++++++++--
postprocessing/pp-webm.c | 2 +-
rtcp.c | 16 +-
rtcp.h | 12 +-
rtp.c | 33 +-
rtp.h | 13 +
sdp.c | 130 +++++-
sdp.h | 8 +
utils.c | 304 +++++++++++++
utils.h | 49 +-
36 files changed, 3424 insertions(+), 403 deletions(-)
diff --cc html/vp9svctest.html
index 8a29a42,0000000..ffb51a8
mode 100644,000000..100644
--- a/html/vp9svctest.html
+++ b/html/vp9svctest.html
@@@ -1,166 -1,0 +1,171 @@@
+<!DOCTYPE html>
+<html xmlns="http://www.w3.org/1999/xhtml">
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
+<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+<title>Janus WebRTC Gateway: VP9-SVC Video Room Demo</title>
+<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/webrtc-adapter/3.4.3/adapter.min.js" ></script>
+<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/jquery/1.7.2/jquery.min.js" ></script>
+<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/jquery.blockUI/2.70/jquery.blockUI.min.js" ></script>
+<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.0.2/js/bootstrap.min.js"></script>
+<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/bootbox.js/4.1.0/bootbox.min.js"></script>
+<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/spin.js/2.3.2/spin.min.js"></script>
+<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/toastr.js/2.1.3/toastr.min.js"></script>
+<script type="text/javascript" src="janus.js" ></script>
+<script type="text/javascript" src="vp9svctest.js"></script>
+<script>
+ $(function() {
+ $(".navbar-static-top").load("navbar.html", function() {
+ $(".navbar-static-top li.dropdown").addClass("active");
+ $(".navbar-static-top a[href='vp9svctest.html']").parent().addClass("active");
+ });
+ $(".footer").load("footer.html");
+ });
+</script>
+<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/bootswatch/3.3.7/cerulean/bootstrap.min.css" type="text/css"/>
+<link rel="stylesheet" href="css/demo.css" type="text/css"/>
+<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.6.2/css/font-awesome.min.css"/>
+<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/toastr.js/2.1.3/toastr.css"/>
+</head>
+<body>
+
+<a href="https://github.com/meetecho/janus-gateway"><img style="position: absolute; top: 0; left: 0; border: 0; z-index: 1001;" src="https://s3.amazonaws.com/github/ribbons/forkme_left_darkblue_121621.png" alt="Fork me on GitHub"></a>
+
+<nav class="navbar navbar-default navbar-static-top">
+</nav>
+
+<div class="container">
+ <div class="row">
+ <div class="col-md-12">
+ <div class="page-header">
+ <h1>Plugin Demo: VP9-SVC Video Room
+ <button class="btn btn-default" autocomplete="off" id="start">Start</button>
+ </h1>
+ </div>
+ <div class="container" id="details">
+ <div class="row">
+ <div class="col-md-12">
+ <h3>Demo details</h3>
+ <p>This is basically a clone of the plain <a href="videoroomtest.hmtl">Video Room</a>
+ demo, but with a key difference: it forces VP9 on all publishers, and supports
+ the VP9 SVC layer selection (if you don't know what this means, check this
+ <a target="_blank" href="https://webrtchacks.com/chrome-vp9-svc/">excellent blog post</a>).
+ As such, it will allow viewers to select which layer, spatial or temporal, to receive
+ from the publishers, in order to receive more or less data according to what they
- can/want to get.</p>
++ can/want to get. Notice that this is fundamentally different from the concept of
++ simulcasting, and so must not be confused with the VP8 simulcasting some of the
++ plugins (including the VideoRoom itself) support: with simulcasting, different
++ streams with a different resolution/bitrate are sent at the same time; with SVC,
++ you have a single stream with multiple layers, which makes it more efficient,
++ especially because it consumes much less bandwidth.</p>
+ <p>Notice that this only works if the publishers joining the room will use a recent
+ Chrome version that has been started with the following flag:</p>
+ <p><div class="alert alert-info"><code>--force-fieldtrials=WebRTC-SupportVP9SVC/EnabledByFlag_2SL3TL/</code></div></p>
+ <p>If you join with a Chrome that doesn't have that option set, or with
+ another browser, you'll be able to select a layer, but all request to set
+ a custom layer on your video will fail, meaning you'll act as a plain VP9 client.
+ If you ask for a layer from a publisher that doesn't support VP9 SVC either,
+ then your request will be ignored as well.</p>
+ <p>In case some publishers do support the VP9 SVC features, selecting a different
+ layer from them should result in variations of the video quality, and a notable
+ difference in the bandwidth used to receive their video as well.</p>
+ <p>Press the <code>Start</code> button above to launch the demo.</p>
+ </div>
+ </div>
+ </div>
+ <div class="container hide" id="videojoin">
+ <div class="row">
+ <span class="label label-info" id="you"></span>
+ <div class="col-md-12" id="controls">
+ <div class="input-group margin-bottom-md hide" id="registernow">
+ <span class="input-group-addon">@</span>
+ <input autocomplete="off" class="form-control" autocomplete="off" type="text" placeholder="Choose a display name" id="username" onkeypress="return checkEnter(this, event);"></input>
+ <span class="input-group-btn">
+ <button class="btn btn-success" autocomplete="off" id="register">Join the room</button>
+ </span>
+ </div>
+ </div>
+ </div>
+ </div>
+ <div class="container hide" id="videos">
+ <div class="row">
+ <div class="col-md-4">
+ <div class="panel panel-default">
+ <div class="panel-heading">
+ <h3 class="panel-title">Local Video <span class="label label-primary hide" id="publisher"></span>
+ <div class="btn-group btn-group-xs pull-right hide">
+ <div class="btn-group btn-group-xs">
+ <button id="bitrateset" autocomplete="off" class="btn btn-primary dropdown-toggle" data-toggle="dropdown">
+ Bandwidth<span class="caret"></span>
+ </button>
+ <ul id="bitrate" class="dropdown-menu" role="menu">
+ <li><a href="#" id="0">No limit</a></li>
+ <li><a href="#" id="128">Cap to 128kbit</a></li>
+ <li><a href="#" id="256">Cap to 256kbit</a></li>
+ <li><a href="#" id="512">Cap to 512kbit</a></li>
+ <li><a href="#" id="1024">Cap to 1mbit</a></li>
+ <li><a href="#" id="1500">Cap to 1.5mbit</a></li>
+ <li><a href="#" id="2000">Cap to 2mbit</a></li>
+ </ul>
+ </div>
+ </div>
+ </h3>
+ </div>
+ <div class="panel-body" id="videolocal"></div>
+ </div>
+ </div>
+ <div class="col-md-4">
+ <div class="panel panel-default">
+ <div class="panel-heading">
+ <h3 class="panel-title">Remote Video #1 <span class="label label-info hide" id="remote1"></span></h3>
+ </div>
+ <div class="panel-body relative" id="videoremote1"></div>
+ </div>
+ </div>
+ <div class="col-md-4">
+ <div class="panel panel-default">
+ <div class="panel-heading">
+ <h3 class="panel-title">Remote Video #2 <span class="label label-info hide" id="remote2"></span></h3>
+ </div>
+ <div class="panel-body relative" id="videoremote2"></div>
+ </div>
+ </div>
+ </div>
+ <div class="row">
+ <div class="col-md-4">
+ <div class="panel panel-default">
+ <div class="panel-heading">
+ <h3 class="panel-title">Remote Video #3 <span class="label label-info hide" id="remote3"></span></h3>
+ </div>
+ <div class="panel-body relative" id="videoremote3"></div>
+ </div>
+ </div>
+ <div class="col-md-4">
+ <div class="panel panel-default">
+ <div class="panel-heading">
+ <h3 class="panel-title">Remote Video #4 <span class="label label-info hide" id="remote4"></span></h3>
+ </div>
+ <div class="panel-body relative" id="videoremote4"></div>
+ </div>
+ </div>
+ <div class="col-md-4">
+ <div class="panel panel-default">
+ <div class="panel-heading">
+ <h3 class="panel-title">Remote Video #5 <span class="label label-info hide" id="remote5"></span></h3>
+ </div>
+ <div class="panel-body relative" id="videoremote5"></div>
+ </div>
+ </div>
+ </div>
+ </div>
+ </div>
+ </div>
+
+ <hr>
+ <div class="footer">
+ </div>
+</div>
+
+</body>
+</html>
diff --cc plugins/janus_videoroom.c
index 5080c1a,b5477d6..e8a63dd
--- a/plugins/janus_videoroom.c
+++ b/plugins/janus_videoroom.c
@@@ -303,8 -307,8 +309,12 @@@ static struct janus_json_parameter conf
{"audio", JANUS_JSON_BOOL, 0},
{"video", JANUS_JSON_BOOL, 0},
{"data", JANUS_JSON_BOOL, 0},
++ /* For VP8 simulcast */
+ {"substream", JSON_INTEGER, JANUS_JSON_PARAM_POSITIVE},
- {"temporal", JSON_INTEGER, JANUS_JSON_PARAM_POSITIVE}
++ {"temporal", JSON_INTEGER, JANUS_JSON_PARAM_POSITIVE},
++ /* For VP9 SVC */
+ {"spatial_layer", JSON_INTEGER, JANUS_JSON_PARAM_POSITIVE},
+ {"temporal_layer", JSON_INTEGER, JANUS_JSON_PARAM_POSITIVE}
};
static struct janus_json_parameter listener_parameters[] = {
{"feed", JSON_INTEGER, JANUS_JSON_PARAM_REQUIRED | JANUS_JSON_PARAM_POSITIVE},
@@@ -465,11 -468,10 +475,11 @@@ typedef struct janus_videoroom
uint16_t fir_freq; /* Regular FIR frequency (0=disabled) */
janus_videoroom_audiocodec acodec; /* Audio codec to force on publishers*/
janus_videoroom_videocodec vcodec; /* Video codec to force on publishers*/
+ gboolean do_svc; /* Whether SVC must be done for video (note: only available for VP9 right now) */
gboolean audiolevel_ext; /* Whether the ssrc-audio-level extension must be negotiated or not for new publishers */
gboolean audiolevel_event; /* Whether to emit event to other users about audiolevel */
- int audio_active_packets; /* amount of packets with audio level for checkup */
- int audio_level_average; /* average audio level */
+ int audio_active_packets; /* Amount of packets with audio level for checkup */
+ int audio_level_average; /* Average audio level */
gboolean videoorient_ext; /* Whether the video-orientation extension must be negotiated or not for new publishers */
gboolean playoutdelay_ext; /* Whether the playout-delay extension must be negotiated or not for new publishers */
gboolean record; /* Whether the feeds from publishers in this room should be recorded */
@@@ -562,12 -568,15 +576,19 @@@ typedef struct janus_videoroom_listene
janus_videoroom_participant *feed; /* Participant this listener is subscribed to */
guint32 pvt_id; /* Private ID of the participant that is subscribing (if available/provided) */
janus_rtp_switching_context context; /* Needed in case there are publisher switches on this listener */
- int substream; /* Which simulcast substream we should forward, in case the publisher is simulcasting */
++ int substream; /* Which VP8 simulcast substream we should forward, in case the publisher is simulcasting */
+ int substream_target; /* As above, but to handle transitions (e.g., wait for keyframe) */
- int templayer; /* Which simulcast temporal layer we should forward, in case the publisher is simulcasting */
++ int templayer; /* Which VP8 simulcast temporal layer we should forward, in case the publisher is simulcasting */
+ int templayer_target; /* As above, but to handle transitions (e.g., wait for keyframe) */
+ gint64 last_relayed; /* When we relayed the last packet (used to detect when substreams become unavailable) */
+ janus_vp8_simulcast_context simulcast_context;
gboolean audio, video, data; /* Whether audio, video and/or data must be sent to this publisher */
gboolean paused;
gboolean kicked; /* Whether this subscription belongs to a participant that has been kicked */
- /* The following are only relevant if we're doing VP9 SVC*/
++ /* The following are only relevant if we're doing VP9 SVC, and are not to be confused with VP8
++ * simulcast, which has similar info (substream/templayer) but in a completely different context */
+ int spatial_layer, target_spatial_layer;
+ int temporal_layer, target_temporal_layer;
} janus_videoroom_listener;
static void janus_videoroom_listener_free(janus_videoroom_listener *l);
@@@ -575,13 -584,9 +596,14 @@@ typedef struct janus_videoroom_rtp_rela
rtp_header *data;
gint length;
gboolean is_video;
+ uint32_t ssrc[3];
uint32_t timestamp;
uint16_t seq_number;
+ /* The following are only relevant if we're doing VP9 SVC*/
+ gboolean svc;
+ int spatial_layer;
+ int temporal_layer;
+ uint8_t pbit, dbit, ubit, bbit, ebit;
} janus_videoroom_rtp_relay_packet;
@@@ -1207,15 -1206,14 +1232,22 @@@ json_t *janus_videoroom_query_session(j
json_object_set_new(media, "audio", json_integer(participant->audio));
json_object_set_new(media, "video", json_integer(participant->video));
json_object_set_new(media, "data", json_integer(participant->data));
+ if(feed->ssrc[0] != 0) {
+ json_object_set_new(info, "simulcast", json_true());
+ json_object_set_new(info, "substream", json_integer(participant->substream));
+ json_object_set_new(info, "substream-target", json_integer(participant->substream_target));
+ json_object_set_new(info, "temporal-layer", json_integer(participant->templayer));
+ json_object_set_new(info, "temporal-layer-target", json_integer(participant->templayer_target));
+ }
json_object_set_new(info, "media", media);
+ if(participant->room && participant->room->do_svc) {
+ json_t *svc = json_object();
+ json_object_set_new(svc, "spatial-layer", json_integer(participant->spatial_layer));
+ json_object_set_new(svc, "target-spatial-layer", json_integer(participant->target_spatial_layer));
+ json_object_set_new(svc, "temporal-layer", json_integer(participant->temporal_layer));
+ json_object_set_new(svc, "target-temporal-layer", json_integer(participant->target_temporal_layer));
+ json_object_set_new(info, "svc", svc);
+ }
}
}
}
@@@ -2447,28 -2486,9 +2533,31 @@@ void janus_videoroom_incoming_rtp(janus
packet.data = rtp;
packet.length = len;
packet.is_video = video;
+ packet.svc = FALSE;
+ if(video && videoroom->do_svc) {
+ /* We're doing SVC: let's parse this packet to see which layers are there */
+ int plen = 0;
+ char *payload = janus_rtp_payload(buf, len, &plen);
+ if(payload == NULL)
+ return;
+ uint8_t pbit = 0, dbit = 0, ubit = 0, bbit = 0, ebit = 0;
+ int found = 0, spatial_layer = 0, temporal_layer = 0;
+ if(janus_vp9_parse_svc(payload, plen, &found, &spatial_layer, &temporal_layer, &pbit, &dbit, &ubit, &bbit, &ebit) == 0) {
+ if(found) {
+ packet.svc = TRUE;
+ packet.spatial_layer = spatial_layer;
+ packet.temporal_layer = temporal_layer;
+ packet.pbit = pbit;
+ packet.dbit = dbit;
+ packet.ubit = ubit;
+ packet.bbit = bbit;
+ packet.ebit = ebit;
+ }
+ }
+ }
+ packet.ssrc[0] = (sc != -1 ? participant->ssrc[0] : 0);
+ packet.ssrc[1] = (sc != -1 ? participant->ssrc[1] : 0);
+ packet.ssrc[2] = (sc != -1 ? participant->ssrc[2] : 0);
/* Backup the actual timestamp and sequence number set by the publisher, in case switching is involved */
packet.timestamp = ntohl(packet.data->timestamp);
packet.seq_number = ntohs(packet.data->seq_number);
@@@ -3168,15 -3188,13 +3257,21 @@@ static void *janus_videoroom_handler(vo
if(!publisher->data)
listener->data = FALSE; /* ... unless the publisher isn't sending any data */
listener->paused = TRUE; /* We need an explicit start from the listener */
+ listener->substream = -1;
+ listener->substream_target = 2;
+ listener->templayer = -1;
+ listener->templayer_target = 2;
+ listener->last_relayed = 0;
+ janus_vp8_simulcast_context_reset(&listener->simulcast_context);
session->participant = listener;
+ if(videoroom->do_svc) {
+ /* This listener belongs to a room where VP9 SVC has been enabled,
+ * let's assume we're interested in all layers for the time being */
+ listener->spatial_layer = -1;
+ listener->target_spatial_layer = 1; /* FIXME Chrome sends 0 and 1 */
+ listener->temporal_layer = -1;
+ listener->target_temporal_layer = 2; /* FIXME Chrome sends 0, 1 and 2 */
+ }
janus_mutex_lock(&publisher->listeners_mutex);
publisher->listeners = g_slist_append(publisher->listeners, listener);
janus_mutex_unlock(&publisher->listeners_mutex);
@@@ -3465,8 -3483,20 +3560,22 @@@
json_t *audio = json_object_get(root, "audio");
json_t *video = json_object_get(root, "video");
json_t *data = json_object_get(root, "data");
- json_t *substream = json_object_get(root, "substream");
- if(json_integer_value(substream) > 2) {
+ json_t *spatial = json_object_get(root, "spatial_layer");
+ json_t *temporal = json_object_get(root, "temporal_layer");
++ json_t *sc_substream = json_object_get(root, "substream");
++ if(json_integer_value(sc_substream) > 2) {
+ JANUS_LOG(LOG_ERR, "Invalid element (substream should be 0, 1 or 2)\n");
+ error_code = JANUS_VIDEOROOM_ERROR_INVALID_ELEMENT;
+ g_snprintf(error_cause, 512, "Invalid value (substream should be 0, 1 or 2)");
+ goto error;
+ }
- json_t *temporal = json_object_get(root, "temporal");
- if(json_integer_value(temporal) > 2) {
++ json_t *sc_temporal = json_object_get(root, "temporal");
++ if(json_integer_value(sc_temporal) > 2) {
+ JANUS_LOG(LOG_ERR, "Invalid element (temporal should be 0, 1 or 2)\n");
+ error_code = JANUS_VIDEOROOM_ERROR_INVALID_ELEMENT;
+ g_snprintf(error_cause, 512, "Invalid value (temporal should be 0, 1 or 2)");
+ goto error;
+ }
/* Update the audio/video/data flags, if set */
janus_videoroom_participant *publisher = listener->feed;
if(publisher) {
@@@ -3476,52 -3506,55 +3585,101 @@@
listener->video = json_is_true(video);
if(data && publisher->data)
listener->data = json_is_true(data);
- if(substream) {
- listener->substream_target = json_integer_value(substream);
++ /* Check if a simulcasting-related request is involved */
++ if(sc_substream && publisher->ssrc[0] != 0) {
++ listener->substream_target = json_integer_value(sc_substream);
+ JANUS_LOG(LOG_VERB, "Setting video SSRC to let through (simulcast): %"SCNu32" (index %d, was %d)\n",
+ publisher->ssrc[listener->substream], listener->substream_target, listener->substream);
+ if(listener->substream_target == listener->substream) {
+ /* No need to do anything, we're already getting the right substream, so notify the user */
+ json_t *event = json_object();
+ json_object_set_new(event, "videoroom", json_string("event"));
+ json_object_set_new(event, "room", json_integer(listener->room->room_id));
+ json_object_set_new(event, "substream", json_integer(listener->substream));
+ gateway->push_event(msg->handle, &janus_videoroom_plugin, NULL, event, NULL);
+ json_decref(event);
+ } else {
+ /* Send a FIR */
+ char buf[20];
+ janus_rtcp_fir((char *)&buf, 20, &publisher->fir_seq);
+ JANUS_LOG(LOG_VERB, "Simulcasting substream change, sending FIR to %"SCNu64" (%s)\n", publisher->user_id, publisher->display ? publisher->display : "??");
+ gateway->relay_rtcp(publisher->session->handle, 1, buf, 20);
+ /* Send a PLI too, just in case... */
+ janus_rtcp_pli((char *)&buf, 12);
+ JANUS_LOG(LOG_VERB, "Simulcasting substream change, sending PLI to %"SCNu64" (%s)\n", publisher->user_id, publisher->display ? publisher->display : "??");
+ gateway->relay_rtcp(publisher->session->handle, 1, buf, 12);
+ }
+ }
- if(temporal) {
- listener->templayer_target = json_integer_value(temporal);
++ if(sc_temporal && publisher->ssrc[0] != 0) {
++ listener->templayer_target = json_integer_value(sc_temporal);
+ JANUS_LOG(LOG_VERB, "Setting video temporal layer to let through (simulcast): %d (was %d)\n",
+ listener->templayer_target, listener->templayer);
+ if(listener->templayer_target == listener->templayer) {
+ /* No need to do anything, we're already getting the right temporal, so notify the user */
+ json_t *event = json_object();
+ json_object_set_new(event, "videoroom", json_string("event"));
+ json_object_set_new(event, "room", json_integer(listener->room->room_id));
+ json_object_set_new(event, "temporal", json_integer(listener->templayer));
+ gateway->push_event(msg->handle, &janus_videoroom_plugin, NULL, event, NULL);
+ json_decref(event);
+ } else {
+ /* Send a FIR */
+ char buf[20];
+ janus_rtcp_fir((char *)&buf, 20, &publisher->fir_seq);
+ JANUS_LOG(LOG_VERB, "Simulcasting temporal layer change, sending FIR to %"SCNu64" (%s)\n", publisher->user_id, publisher->display ? publisher->display : "??");
+ gateway->relay_rtcp(publisher->session->handle, 1, buf, 20);
+ /* Send a PLI too, just in case... */
+ janus_rtcp_pli((char *)&buf, 12);
+ JANUS_LOG(LOG_VERB, "Simulcasting temporal layer change, sending PLI to %"SCNu64" (%s)\n", publisher->user_id, publisher->display ? publisher->display : "??");
+ gateway->relay_rtcp(publisher->session->handle, 1, buf, 12);
+ }
+ }
}
+ if(listener->room->do_svc) {
+ /* Also check if the viewer is trying to configure a layer change */
+ if(spatial) {
+ int spatial_layer = json_integer_value(spatial);
+ if(spatial_layer > 1) {
+ JANUS_LOG(LOG_WARN, "Spatial layer higher than 1, will probably be ignored\n");
+ }
+ if(spatial_layer == listener->spatial_layer) {
+ /* No need to do anything, we're already getting the right spatial layer, so notify the user */
+ json_t *event = json_object();
+ json_object_set_new(event, "videoroom", json_string("event"));
+ json_object_set_new(event, "room", json_integer(listener->room->room_id));
+ json_object_set_new(event, "spatial_layer", json_integer(listener->spatial_layer));
+ gateway->push_event(msg->handle, &janus_videoroom_plugin, NULL, event, NULL);
+ json_decref(event);
+ } else if(spatial_layer != listener->target_spatial_layer) {
+ /* Send a FIR to the new RTP forward publisher */
+ char buf[20];
+ janus_rtcp_fir((char *)&buf, 20, &publisher->fir_seq);
+ JANUS_LOG(LOG_VERB, "Need to downscale spatially, sending FIR to %"SCNu64" (%s)\n", publisher->user_id, publisher->display ? publisher->display : "??");
+ gateway->relay_rtcp(publisher->session->handle, 1, buf, 20);
+ /* Send a PLI too, just in case... */
+ janus_rtcp_pli((char *)&buf, 12);
+ JANUS_LOG(LOG_VERB, "Need to downscale spatially, sending PLI to %"SCNu64" (%s)\n", publisher->user_id, publisher->display ? publisher->display : "??");
+ gateway->relay_rtcp(publisher->session->handle, 1, buf, 12);
+ }
+ listener->target_spatial_layer = spatial_layer;
+ }
+ if(temporal) {
+ int temporal_layer = json_integer_value(temporal);
+ if(temporal_layer > 2) {
+ JANUS_LOG(LOG_WARN, "Temporal layer higher than 2, will probably be ignored\n");
+ }
+ if(temporal_layer == listener->temporal_layer) {
+ /* No need to do anything, we're already getting the right temporal layer, so notify the user */
+ json_t *event = json_object();
+ json_object_set_new(event, "videoroom", json_string("event"));
+ json_object_set_new(event, "room", json_integer(listener->room->room_id));
+ json_object_set_new(event, "temporal_layer", json_integer(listener->temporal_layer));
+ gateway->push_event(msg->handle, &janus_videoroom_plugin, NULL, event, NULL);
+ json_decref(event);
+ }
+ listener->target_temporal_layer = temporal_layer;
+ }
+ }
event = json_object();
json_object_set_new(event, "videoroom", json_string("event"));
json_object_set_new(event, "room", json_integer(listener->room->room_id));
@@@ -3993,111 -4032,126 +4164,230 @@@ static void janus_videoroom_relay_rtp_p
/* Nope, don't relay */
return;
}
- if(packet->ssrc[0] != 0) {
+ /* Check if there's any SVC info to take into account */
- gboolean override_mark_bit = FALSE, has_marker_bit = packet->data->markerbit;
+ if(packet->svc) {
+ /* There is: check if this is a layer that can be dropped for this viewer
+ * Note: Following core inspired by the excellent job done by Sergio Garcia Murillo here:
+ * https://github.com/medooze/media-server/blob/master/src/vp9/VP9LayerSelector.cpp */
++ gboolean override_mark_bit = FALSE, has_marker_bit = packet->data->markerbit;
+ int temporal_layer = listener->temporal_layer;
+ if(listener->target_temporal_layer > listener->temporal_layer) {
+ /* We need to upscale */
+ JANUS_LOG(LOG_HUGE, "We need to upscale temporally:\n");
+ if(packet->ubit && packet->bbit && packet->temporal_layer <= listener->target_temporal_layer) {
+ JANUS_LOG(LOG_HUGE, " -- Upscaling temporal layer: %u --> %u\n",
+ packet->temporal_layer, listener->target_temporal_layer);
+ listener->temporal_layer = packet->temporal_layer;
+ temporal_layer = listener->temporal_layer;
+ /* Notify the viewer */
+ json_t *event = json_object();
+ json_object_set_new(event, "videoroom", json_string("event"));
+ json_object_set_new(event, "room", json_integer(listener->room->room_id));
+ json_object_set_new(event, "temporal_layer", json_integer(listener->temporal_layer));
+ gateway->push_event(listener->session->handle, &janus_videoroom_plugin, NULL, event, NULL);
+ json_decref(event);
+ }
+ } else if(listener->target_temporal_layer < listener->temporal_layer) {
+ /* We need to downscale */
+ JANUS_LOG(LOG_HUGE, "We need to downscale temporally:\n");
+ if(packet->ebit) {
+ JANUS_LOG(LOG_HUGE, " -- Downscaling temporal layer: %u --> %u\n",
+ listener->temporal_layer, listener->target_temporal_layer);
+ listener->temporal_layer = listener->target_temporal_layer;
+ /* Notify the viewer */
+ json_t *event = json_object();
+ json_object_set_new(event, "videoroom", json_string("event"));
+ json_object_set_new(event, "room", json_integer(listener->room->room_id));
+ json_object_set_new(event, "temporal_layer", json_integer(listener->temporal_layer));
+ gateway->push_event(listener->session->handle, &janus_videoroom_plugin, NULL, event, NULL);
+ json_decref(event);
+ }
+ }
+ if(temporal_layer < packet->temporal_layer) {
+ /* Drop the packet: update the context to make sure sequence number is increased normally later */
+ JANUS_LOG(LOG_HUGE, "Dropping packet (temporal layer %d < %d)\n", temporal_layer, packet->temporal_layer);
+ listener->context.v_base_seq++;
+ return;
+ }
+ int spatial_layer = listener->spatial_layer;
+ if(listener->target_spatial_layer > listener->spatial_layer) {
+ JANUS_LOG(LOG_HUGE, "We need to upscale spatially:\n");
+ /* We need to upscale */
+ if(packet->pbit == 0 && packet->bbit && packet->spatial_layer == listener->spatial_layer+1) {
+ JANUS_LOG(LOG_HUGE, " -- Upscaling spatial layer: %u --> %u\n",
+ packet->spatial_layer, listener->target_spatial_layer);
+ listener->spatial_layer = packet->spatial_layer;
+ spatial_layer = listener->spatial_layer;
+ /* Notify the viewer */
+ json_t *event = json_object();
+ json_object_set_new(event, "videoroom", json_string("event"));
+ json_object_set_new(event, "room", json_integer(listener->room->room_id));
+ json_object_set_new(event, "spatial_layer", json_integer(listener->spatial_layer));
+ gateway->push_event(listener->session->handle, &janus_videoroom_plugin, NULL, event, NULL);
+ json_decref(event);
+ }
+ } else if(listener->target_spatial_layer < listener->spatial_layer) {
+ /* We need to downscale */
+ JANUS_LOG(LOG_HUGE, "We need to downscale spatially:\n");
+ if(packet->ebit) {
+ JANUS_LOG(LOG_HUGE, " -- Downscaling spatial layer: %u --> %u\n",
+ listener->spatial_layer, listener->target_spatial_layer);
+ listener->spatial_layer = listener->target_spatial_layer;
+ /* Notify the viewer */
+ json_t *event = json_object();
+ json_object_set_new(event, "videoroom", json_string("event"));
+ json_object_set_new(event, "room", json_integer(listener->room->room_id));
+ json_object_set_new(event, "spatial_layer", json_integer(listener->spatial_layer));
+ gateway->push_event(listener->session->handle, &janus_videoroom_plugin, NULL, event, NULL);
+ json_decref(event);
+ }
+ }
+ if(spatial_layer < packet->spatial_layer) {
+ /* Drop the packet: update the context to make sure sequence number is increased normally later */
+ JANUS_LOG(LOG_HUGE, "Dropping packet (spatial layer %d < %d)\n", spatial_layer, packet->spatial_layer);
+ listener->context.v_base_seq++;
+ return;
+ } else if(packet->ebit && spatial_layer == packet->spatial_layer) {
+ /* If we stop at layer 0, we need a marker bit now, as the one from layer 1 will not be received */
+ override_mark_bit = TRUE;
+ }
+ /* If we got here, we can send the frame: this doesn't necessarily mean it's
+ * one of the layers the user wants, as there may be dependencies involved */
+ JANUS_LOG(LOG_HUGE, "Sending packet (spatial=%d, temporal=%d)\n",
+ packet->spatial_layer, packet->temporal_layer);
++ /* Fix sequence number and timestamp (publisher switching may be involved) */
++ janus_rtp_header_update(packet->data, &listener->context, TRUE, 4500);
++ if(override_mark_bit && !has_marker_bit) {
++ packet->data->markerbit = 1;
++ }
++ if(gateway != NULL)
++ gateway->relay_rtp(session->handle, packet->is_video, (char *)packet->data, packet->length);
++ if(override_mark_bit && !has_marker_bit) {
++ packet->data->markerbit = 0;
++ }
++ /* Restore the timestamp and sequence number to what the publisher set them to */
++ packet->data->timestamp = htonl(packet->timestamp);
++ packet->data->seq_number = htons(packet->seq_number);
++ } else if(packet->ssrc[0] != 0) {
+ /* Handle simulcast: don't relay if it's not the SSRC we wanted to handle */
+ uint32_t ssrc = ntohl(packet->data->ssrc);
+ int plen = 0;
+ char *payload = janus_rtp_payload((char *)packet->data, packet->length, &plen);
+ if(payload == NULL)
+ return;
+ gboolean switched = FALSE;
+ if(listener->substream != listener->substream_target) {
+ /* There has been a change: let's wait for a keyframe on the target */
+ int step = (listener->substream < 1 && listener->substream_target == 2);
+ if(ssrc == packet->ssrc[listener->substream_target] || (step && ssrc == packet->ssrc[step])) {
+ //~ if(janus_vp8_is_keyframe(payload, plen)) {
+ uint32_t ssrc_old = 0;
+ if(listener->substream != -1)
+ ssrc_old = packet->ssrc[listener->substream];
+ JANUS_LOG(LOG_WARN, "Received keyframe on SSRC %"SCNu32", switching (was %"SCNu32")\n", ssrc, ssrc_old);
+ listener->substream = (ssrc == packet->ssrc[listener->substream_target] ? listener->substream_target : step);;
+ switched = TRUE;
+ /* Notify the viewer */
+ json_t *event = json_object();
+ json_object_set_new(event, "videoroom", json_string("event"));
+ json_object_set_new(event, "room", json_integer(listener->room->room_id));
+ json_object_set_new(event, "substream", json_integer(listener->substream));
+ gateway->push_event(listener->session->handle, &janus_videoroom_plugin, NULL, event, NULL);
+ json_decref(event);
+ //~ } else {
+ //~ JANUS_LOG(LOG_WARN, "Not a keyframe on SSRC %"SCNu32" yet, waiting before switching\n", ssrc);
+ //~ }
+ }
+ }
+ /* If we haven't received our desired substream yet, let's drop temporarily */
+ if(listener->last_relayed == 0) {
+ /* Let's start slow */
+ listener->last_relayed = janus_get_monotonic_time();
+ } else {
+ /* Check if 250ms went by with no packet relayed */
+ gint64 now = janus_get_monotonic_time();
+ if(now-listener->last_relayed >= 250000) {
+ listener->last_relayed = now;
+ int substream = listener->substream-1;
+ if(substream < 0)
+ substream = 0;
+ if(listener->substream != substream) {
+ JANUS_LOG(LOG_WARN, "No packet received on substream %d for a while, falling back to %d\n",
+ listener->substream, substream);
+ listener->substream = substream;
+ /* Send a PLI */
+ JANUS_LOG(LOG_VERB, "Just (re-)enabled video, sending a PLI to recover it\n");
+ char rtcpbuf[12];
+ memset(rtcpbuf, 0, 12);
+ janus_rtcp_pli((char *)&rtcpbuf, 12);
+ if(listener->feed && listener->feed->session && listener->feed->session->handle)
+ gateway->relay_rtcp(listener->feed->session->handle, 1, rtcpbuf, 12);
+ /* Notify the viewer */
+ json_t *event = json_object();
+ json_object_set_new(event, "videoroom", json_string("event"));
+ json_object_set_new(event, "room", json_integer(listener->room->room_id));
+ json_object_set_new(event, "substream", json_integer(listener->substream));
+ gateway->push_event(listener->session->handle, &janus_videoroom_plugin, NULL, event, NULL);
+ json_decref(event);
+ }
+ }
+ }
+ if(ssrc != packet->ssrc[listener->substream]) {
+ JANUS_LOG(LOG_HUGE, "Dropping packet (it's from SSRC %"SCNu32", but we're only relaying SSRC %"SCNu32" now\n",
+ ssrc, packet->ssrc[listener->substream]);
+ return;
+ }
+ listener->last_relayed = janus_get_monotonic_time();
+ /* Check if there's any temporal scalability to take into account */
+ uint16_t picid = 0;
+ uint8_t tlzi = 0;
+ uint8_t tid = 0;
+ uint8_t ybit = 0;
+ uint8_t keyidx = 0;
+ if(janus_vp8_parse_descriptor(payload, plen, &picid, &tlzi, &tid, &ybit, &keyidx) == 0) {
+ //~ JANUS_LOG(LOG_WARN, "%"SCNu16", %u, %u, %u, %u\n", picid, tlzi, tid, ybit, keyidx);
+ if(listener->templayer != listener->templayer_target) {
+ /* FIXME We should be smarter in deciding when to switch */
+ listener->templayer = listener->templayer_target;
+ /* Notify the user */
+ json_t *event = json_object();
+ json_object_set_new(event, "videoroom", json_string("event"));
+ json_object_set_new(event, "room", json_integer(listener->room->room_id));
+ json_object_set_new(event, "temporal", json_integer(listener->templayer));
+ gateway->push_event(listener->session->handle, &janus_videoroom_plugin, NULL, event, NULL);
+ json_decref(event);
+ }
+ if(tid > listener->templayer) {
+ JANUS_LOG(LOG_HUGE, "Dropping packet (it's temporal layer %d, but we're capping at %d)\n",
+ tid, listener->templayer);
+ /* We increase the base sequence number, or there will be gaps when delivering later */
+ listener->context.v_base_seq++;
+ return;
+ }
+ }
+ /* If we got here, update the RTP header and send the packet */
+ janus_rtp_header_update(packet->data, &listener->context, TRUE, 4500);
+ char vp8pd[6];
+ memcpy(vp8pd, payload, sizeof(vp8pd));
+ janus_vp8_simulcast_descriptor_update(payload, plen, &listener->simulcast_context, switched);
+ /* Send the packet */
+ if(gateway != NULL)
+ gateway->relay_rtp(session->handle, packet->is_video, (char *)packet->data, packet->length);
+ /* Restore the timestamp and sequence number to what the publisher set them to */
+ packet->data->timestamp = htonl(packet->timestamp);
+ packet->data->seq_number = htons(packet->seq_number);
+ /* Restore the original payload descriptor as well, as it will be needed by the next viewer */
+ memcpy(payload, vp8pd, sizeof(vp8pd));
+ } else {
+ /* Fix sequence number and timestamp (publisher switching may be involved) */
+ janus_rtp_header_update(packet->data, &listener->context, TRUE, 4500);
+ /* Send the packet */
+ if(gateway != NULL)
+ gateway->relay_rtp(session->handle, packet->is_video, (char *)packet->data, packet->length);
+ /* Restore the timestamp and sequence number to what the publisher set them to */
+ packet->data->timestamp = htonl(packet->timestamp);
+ packet->data->seq_number = htons(packet->seq_number);
}
- /* Fix sequence number and timestamp (publisher switching may be involved) */
- janus_rtp_header_update(packet->data, &listener->context, TRUE, 4500);
- if(override_mark_bit && !has_marker_bit) {
- packet->data->markerbit = 1;
- }
- if(gateway != NULL)
- gateway->relay_rtp(session->handle, packet->is_video, (char *)packet->data, packet->length);
- if(override_mark_bit && !has_marker_bit) {
- packet->data->markerbit = 0;
- }
- /* Restore the timestamp and sequence number to what the publisher set them to */
- packet->data->timestamp = htonl(packet->timestamp);
- packet->data->seq_number = htons(packet->seq_number);
} else {
/* Check if this listener is subscribed to this medium */
if(!listener->audio) {
diff --cc utils.c
index cf5091d,705dde5..d8bffbd
--- a/utils.c
+++ b/utils.c
@@@ -497,137 -497,306 +497,441 @@@ gboolean janus_json_is_valid(json_t *va
return is_valid;
}
+ /* The following code is more related to codec specific helpers */
+ #if defined(__ppc__) || defined(__ppc64__)
+ # define swap2(d) \
+ ((d&0x000000ff)<<8) | \
+ ((d&0x0000ff00)>>8)
+ #else
+ # define swap2(d) d
+ #endif
+
+ gboolean janus_vp8_is_keyframe(char* buffer, int len) {
+ if(!buffer || len < 0)
+ return FALSE;
+ /* Parse VP8 header now */
+ uint8_t vp8pd = *buffer;
+ uint8_t xbit = (vp8pd & 0x80);
+ uint8_t sbit = (vp8pd & 0x10);
+ if(xbit) {
+ JANUS_LOG(LOG_HUGE, " -- X bit is set!\n");
+ /* Read the Extended control bits octet */
+ buffer++;
+ vp8pd = *buffer;
+ uint8_t ibit = (vp8pd & 0x80);
+ uint8_t lbit = (vp8pd & 0x40);
+ uint8_t tbit = (vp8pd & 0x20);
+ uint8_t kbit = (vp8pd & 0x10);
+ if(ibit) {
+ JANUS_LOG(LOG_HUGE, " -- I bit is set!\n");
+ /* Read the PictureID octet */
+ buffer++;
+ vp8pd = *buffer;
+ uint16_t picid = vp8pd, wholepicid = picid;
+ uint8_t mbit = (vp8pd & 0x80);
+ if(mbit) {
+ JANUS_LOG(LOG_HUGE, " -- M bit is set!\n");
+ memcpy(&picid, buffer, sizeof(uint16_t));
+ wholepicid = ntohs(picid);
+ picid = (wholepicid & 0x7FFF);
+ buffer++;
+ }
+ JANUS_LOG(LOG_HUGE, " -- -- PictureID: %"SCNu16"\n", picid);
+ }
+ if(lbit) {
+ JANUS_LOG(LOG_HUGE, " -- L bit is set!\n");
+ /* Read the TL0PICIDX octet */
+ buffer++;
+ vp8pd = *buffer;
+ }
+ if(tbit || kbit) {
+ JANUS_LOG(LOG_HUGE, " -- T/K bit is set!\n");
+ /* Read the TID/KEYIDX octet */
+ buffer++;
+ vp8pd = *buffer;
+ }
+ buffer++; /* Now we're in the payload */
+ if(sbit) {
+ JANUS_LOG(LOG_HUGE, " -- S bit is set!\n");
+ unsigned long int vp8ph = 0;
+ memcpy(&vp8ph, buffer, 4);
+ vp8ph = ntohl(vp8ph);
+ uint8_t pbit = ((vp8ph & 0x01000000) >> 24);
+ if(!pbit) {
+ JANUS_LOG(LOG_HUGE, " -- P bit is NOT set!\n");
+ /* It is a key frame! Get resolution for debugging */
+ unsigned char *c = (unsigned char *)buffer+3;
+ /* vet via sync code */
+ if(c[0]!=0x9d||c[1]!=0x01||c[2]!=0x2a) {
+ JANUS_LOG(LOG_WARN, "First 3-bytes after header not what they're supposed to be?\n");
+ } else {
+ int vp8w = swap2(*(unsigned short*)(c+3))&0x3fff;
+ int vp8ws = swap2(*(unsigned short*)(c+3))>>14;
+ int vp8h = swap2(*(unsigned short*)(c+5))&0x3fff;
+ int vp8hs = swap2(*(unsigned short*)(c+5))>>14;
+ JANUS_LOG(LOG_HUGE, "Got a VP8 key frame: %dx%d (scale=%dx%d)\n", vp8w, vp8h, vp8ws, vp8hs);
+ return TRUE;
+ }
+ }
+ }
+ }
+ /* If we got here it's not a key frame */
+ return FALSE;
+ }
+
+ gboolean janus_vp9_is_keyframe(char* buffer, int len) {
+ if(!buffer || len < 0)
+ return FALSE;
+ /* Parse VP9 header now */
+ uint8_t vp9pd = *buffer;
+ uint8_t ibit = (vp9pd & 0x80);
+ uint8_t pbit = (vp9pd & 0x40);
+ uint8_t lbit = (vp9pd & 0x20);
+ uint8_t fbit = (vp9pd & 0x10);
+ uint8_t vbit = (vp9pd & 0x02);
+ buffer++;
+ if(ibit) {
+ /* Read the PictureID octet */
+ vp9pd = *buffer;
+ uint16_t picid = vp9pd, wholepicid = picid;
+ uint8_t mbit = (vp9pd & 0x80);
+ if(!mbit) {
+ buffer++;
+ } else {
+ memcpy(&picid, buffer, sizeof(uint16_t));
+ wholepicid = ntohs(picid);
+ picid = (wholepicid & 0x7FFF);
+ buffer += 2;
+ }
+ }
+ if(lbit) {
+ buffer++;
+ if(!fbit) {
+ /* Non-flexible mode, skip TL0PICIDX */
+ buffer++;
+ }
+ }
+ if(fbit && pbit) {
+ /* Skip reference indices */
+ uint8_t nbit = 1;
+ while(nbit) {
+ vp9pd = *buffer;
+ nbit = (vp9pd & 0x01);
+ buffer++;
+ }
+ }
+ if(vbit) {
+ /* Parse and skip SS */
+ vp9pd = *buffer;
+ uint n_s = (vp9pd & 0xE0) >> 5;
+ n_s++;
+ uint8_t ybit = (vp9pd & 0x10);
+ if(ybit) {
+ /* Iterate on all spatial layers and get resolution */
+ buffer++;
+ uint i=0;
+ for(i=0; i<n_s; i++) {
+ /* Width */
+ uint16_t *w = (uint16_t *)buffer;
+ int vp9w = ntohs(*w);
+ buffer += 2;
+ /* Height */
+ uint16_t *h = (uint16_t *)buffer;
+ int vp9h = ntohs(*h);
+ buffer += 2;
+ JANUS_LOG(LOG_WARN, "Got a VP9 key frame: %dx%d\n", vp9w, vp9h);
+ return TRUE;
+ }
+ }
+ }
+ /* If we got here it's not a key frame */
+ return FALSE;
+ }
+
+ gboolean janus_h264_is_keyframe(char* buffer, int len) {
+ if(!buffer || len < 0)
+ return FALSE;
+ /* Parse H264 header now */
+ uint8_t fragment = *buffer & 0x1F;
+ uint8_t nal = *(buffer+1) & 0x1F;
+ uint8_t start_bit = *(buffer+1) & 0x80;
+ JANUS_LOG(LOG_HUGE, "Fragment=%d, NAL=%d, Start=%d\n", fragment, nal, start_bit);
+ if(fragment == 5 ||
+ ((fragment == 28 || fragment == 29) && nal == 5 && start_bit == 128)) {
+ JANUS_LOG(LOG_HUGE, "Got an H264 key frame\n");
+ return TRUE;
+ }
+ /* If we got here it's not a key frame */
+ return FALSE;
+ }
+
+ int janus_vp8_parse_descriptor(char *buffer, int len,
+ uint16_t *picid, uint8_t *tl0picidx, uint8_t *tid, uint8_t *y, uint8_t *keyidx) {
+ if(!buffer || len < 0)
+ return -1;
+ if(picid)
+ *picid = 0;
+ if(tl0picidx)
+ *tl0picidx = 0;
+ if(tid)
+ *tid = 0;
+ if(y)
+ *y = 0;
+ if(keyidx)
+ *keyidx = 0;
+ uint8_t vp8pd = *buffer;
+ uint8_t xbit = (vp8pd & 0x80);
+ /* Read the Extended control bits octet */
+ if(xbit) {
+ buffer++;
+ vp8pd = *buffer;
+ uint8_t ibit = (vp8pd & 0x80);
+ uint8_t lbit = (vp8pd & 0x40);
+ uint8_t tbit = (vp8pd & 0x20);
+ uint8_t kbit = (vp8pd & 0x10);
+ if(ibit) {
+ /* Read the PictureID octet */
+ buffer++;
+ vp8pd = *buffer;
+ uint16_t partpicid = vp8pd, wholepicid = partpicid;
+ uint8_t mbit = (vp8pd & 0x80);
+ if(mbit) {
+ memcpy(&partpicid, buffer, sizeof(uint16_t));
+ wholepicid = ntohs(partpicid);
+ partpicid = (wholepicid & 0x7FFF);
+ if(picid)
+ *picid = partpicid;
+ buffer++;
+ }
+ }
+ if(lbit) {
+ /* Read the TL0PICIDX octet */
+ buffer++;
+ vp8pd = *buffer;
+ if(tl0picidx)
+ *tl0picidx = vp8pd;
+ }
+ if(tbit || kbit) {
+ /* Read the TID/Y/KEYIDX octet */
+ buffer++;
+ vp8pd = *buffer;
+ if(tid)
+ *tid = (vp8pd & 0xC0) >> 6;
+ if(y)
+ *y = (vp8pd & 0x20) >> 5;
+ if(keyidx)
+ *keyidx = (vp8pd & 0x1F) >> 4;
+ }
+ }
+ return 0;
+ }
+
+ static int janus_vp8_replace_descriptor(char *buffer, int len, uint16_t picid, uint8_t tl0picidx) {
+ if(!buffer || len < 0)
+ return -1;
+ uint8_t vp8pd = *buffer;
+ uint8_t xbit = (vp8pd & 0x80);
+ /* Read the Extended control bits octet */
+ if(xbit) {
+ buffer++;
+ vp8pd = *buffer;
+ uint8_t ibit = (vp8pd & 0x80);
+ uint8_t lbit = (vp8pd & 0x40);
+ uint8_t tbit = (vp8pd & 0x20);
+ uint8_t kbit = (vp8pd & 0x10);
+ if(ibit) {
+ /* Overwrite the PictureID octet */
+ buffer++;
+ vp8pd = *buffer;
+ uint8_t mbit = (vp8pd & 0x80);
+ if(!mbit) {
+ *buffer = picid;
+ } else {
+ uint16_t wholepicid = htons(picid);
+ memcpy(buffer, &wholepicid, 2);
+ *buffer |= 0x80;
+ buffer++;
+ }
+ }
+ if(lbit) {
+ /* Overwrite the TL0PICIDX octet */
+ buffer++;
+ *buffer = tl0picidx;
+ }
+ if(tbit || kbit) {
+ /* Should we overwrite the TID/Y/KEYIDX octet? */
+ buffer++;
+ }
+ }
+ return 0;
+ }
+
+ void janus_vp8_simulcast_context_reset(janus_vp8_simulcast_context *context) {
+ if(context == NULL)
+ return;
+ /* Reset the context values */
+ context->last_picid = 0;
+ context->base_picid = 0;
+ context->base_picid_prev = 0;
+ context->last_tlzi = 0;
+ context->base_tlzi = 0;
+ context->base_tlzi_prev = 0;
+ }
+
+ void janus_vp8_simulcast_descriptor_update(char *buffer, int len, janus_vp8_simulcast_context *context, gboolean switched) {
+ if(!buffer || len < 0)
+ return;
+ uint16_t picid = 0;
+ uint8_t tlzi = 0;
+ uint8_t tid = 0;
+ uint8_t ybit = 0;
+ uint8_t keyidx = 0;
+ /* Parse the identifiers in the VP8 payload descriptor */
+ if(janus_vp8_parse_descriptor(buffer, len, &picid, &tlzi, &tid, &ybit, &keyidx) < 0)
+ return;
+ if(switched) {
+ context->base_picid_prev = context->last_picid;
+ context->base_picid = picid;
+ context->base_tlzi_prev = context->last_tlzi;
+ context->base_tlzi = tlzi;
+ }
+ context->last_picid = (picid-context->base_picid)+context->base_picid_prev+1;
+ context->last_tlzi = (tlzi-context->base_tlzi)+context->base_tlzi_prev+1;
+ /* Overwrite the values in the VP8 payload descriptors with the ones we have */
+ janus_vp8_replace_descriptor(buffer, len, context->last_picid, context->last_tlzi);
+ }
++
+/* Helper method to parse a VP9 RTP video frame and get some SVC-related info:
+ * notice that this only works with VP9, right now, on an experimental basis */
+int janus_vp9_parse_svc(char *buffer, int len, int *found,
+ int *spatial_layer, int *temporal_layer,
+ uint8_t *p, uint8_t *d, uint8_t *u, uint8_t *b, uint8_t *e) {
+ if(!buffer || len < 0)
+ return -1;
+ /* VP9 depay: */
+ /* https://tools.ietf.org/html/draft-ietf-payload-vp9-04 */
+ /* Read the first octet (VP9 Payload Descriptor) */
+ uint8_t vp9pd = *buffer;
+ uint8_t ibit = (vp9pd & 0x80) >> 7;
+ uint8_t pbit = (vp9pd & 0x40) >> 6;
+ uint8_t lbit = (vp9pd & 0x20) >> 5;
+ uint8_t fbit = (vp9pd & 0x10) >> 4;
+ uint8_t bbit = (vp9pd & 0x08) >> 3;
+ uint8_t ebit = (vp9pd & 0x04) >> 2;
+ uint8_t vbit = (vp9pd & 0x02) >> 1;
+ if(!lbit) {
+ /* No Layer indices present, no need to go on */
+ if(found)
+ *found = 0;
+ return 0;
+ }
+ /* Move to the next octet and see what's there */
+ buffer++;
+ len--;
+ if(ibit) {
+ /* Read the PictureID octet */
+ vp9pd = *buffer;
+ uint16_t picid = vp9pd, wholepicid = picid;
+ uint8_t mbit = (vp9pd & 0x80);
+ if(!mbit) {
+ buffer++;
+ len--;
+ } else {
+ memcpy(&picid, buffer, sizeof(uint16_t));
+ wholepicid = ntohs(picid);
+ picid = (wholepicid & 0x7FFF);
+ buffer += 2;
+ len -= 2;
+ }
+ }
+ if(lbit) {
+ /* Read the octet and parse the layer indices now */
+ vp9pd = *buffer;
+ int tlid = (vp9pd & 0xE0) >> 5;
+ uint8_t ubit = (vp9pd & 0x10) >> 4;
+ int slid = (vp9pd & 0x0E) >> 1;
+ uint8_t dbit = (vp9pd & 0x01);
+ JANUS_LOG(LOG_HUGE, "Parsed Layer indices: Temporal: %d (%u), Spatial: %d (%u)\n",
+ tlid, ubit, slid, dbit);
+ if(temporal_layer)
+ *temporal_layer = tlid;
+ if(spatial_layer)
+ *spatial_layer = slid;
+ if(p)
+ *p = pbit;
+ if(d)
+ *d = dbit;
+ if(u)
+ *u = ubit;
+ if(b)
+ *b = bbit;
+ if(e)
+ *e = ebit;
+ if(found)
+ *found = 1;
+ /* Go on, just to get to the SS, if available (which we currently ignore anyway) */
+ buffer++;
+ len--;
+ if(!fbit) {
+ /* Non-flexible mode, skip TL0PICIDX */
+ buffer++;
+ len--;
+ }
+ }
+ if(fbit && pbit) {
+ /* Skip reference indices */
+ uint8_t nbit = 1;
+ while(nbit) {
+ vp9pd = *buffer;
+ nbit = (vp9pd & 0x01);
+ buffer++;
+ len--;
+ }
+ }
+ if(vbit) {
+ /* Parse and skip SS */
+ vp9pd = *buffer;
+ int n_s = (vp9pd & 0xE0) >> 5;
+ n_s++;
+ JANUS_LOG(LOG_HUGE, "There are %d spatial layers\n", n_s);
+ uint8_t ybit = (vp9pd & 0x10);
+ uint8_t gbit = (vp9pd & 0x08);
+ if(ybit) {
+ /* Iterate on all spatial layers and get resolution */
+ buffer++;
+ len--;
+ int i=0;
+ for(i=0; i<n_s; i++) {
+ /* Been there, done that: skip skip skip */
+ buffer += 4;
+ len -= 4;
+ }
+ }
+ if(gbit) {
+ if(!ybit) {
+ buffer++;
+ len--;
+ }
+ uint8_t n_g = *buffer;
+ JANUS_LOG(LOG_HUGE, "There are %u frames in a GOF\n", n_g);
+ buffer++;
+ len--;
+ if(n_g > 0) {
+ int i=0;
+ for(i=0; i<n_g; i++) {
+ /* Read the R bits */
+ vp9pd = *buffer;
+ int r = (vp9pd & 0x0C) >> 2;
+ if(r > 0) {
+ /* Skip reference indices */
+ buffer += r;
+ len -= r;
+ }
+ buffer++;
+ len--;
+ }
+ }
+ }
+ }
+ return 0;
+}
diff --cc utils.h
index edd4400,d19e764..3cbb37b
--- a/utils.h
+++ b/utils.h
@@@ -220,20 -220,51 +220,67 @@@ gboolean janus_json_is_valid(json_t *va
} \
} while(0)
- #endif
+ /*! \brief Helper method to check if a VP8 frame is a keyframe or not
+ * @param[in] buffer The RTP payload to process
+ * @param[in] len The length of the RTP payload
+ * @returns TRUE if it's a keyframe, FALSE otherwise */
+ gboolean janus_vp8_is_keyframe(char* buffer, int len);
+
+ /*! \brief Helper method to check if a VP9 frame is a keyframe or not
+ * @param[in] buffer The RTP payload to process
+ * @param[in] len The length of the RTP payload
+ * @returns TRUE if it's a keyframe, FALSE otherwise */
+ gboolean janus_vp9_is_keyframe(char* buffer, int len);
+
+ /*! \brief Helper method to check if an H.264 frame is a keyframe or not
+ * @param[in] buffer The RTP payload to process
+ * @param[in] len The length of the RTP payload
+ * @returns TRUE if it's a keyframe, FALSE otherwise */
+ gboolean janus_h264_is_keyframe(char* buffer, int len);
+
+ /*! \brief VP8 simulcasting context, in order to make sure SSRC changes result in coherent picid/temporal level increases */
+ typedef struct janus_vp8_simulcast_context {
+ uint16_t last_picid, base_picid, base_picid_prev;
+ uint8_t last_tlzi, base_tlzi, base_tlzi_prev;
+ } janus_vp8_simulcast_context;
+
+ /*! \brief Set (or reset) the context fields to their default values
+ * @param[in] context The context to (re)set */
+ void janus_vp8_simulcast_context_reset(janus_vp8_simulcast_context *context);
+
+ /*! \brief Helper method to parse a VP8 payload descriptor for useful info (e.g., when simulcasting)
+ * @param[in] buffer The RTP payload to process
+ * @param[in] len The length of the RTP payload
+ * @param[out] picid The Picture ID
+ * @param[out] tl0picidx Temporal level zero index
+ * @param[out] tid Temporal-layer index
+ * @param[out] y Layer sync bit
+ * @param[out] keyidx Temporal key frame index
+ * @returns 0 in case of success, a negative integer otherwise */
+ int janus_vp8_parse_descriptor(char *buffer, int len,
+ uint16_t *picid, uint8_t *tl0picidx, uint8_t *tid, uint8_t *y, uint8_t *keyidx);
+
+ /*! \brief Use the context info to update the RTP header of a packet, if needed
+ * @param[in] buffer The RTP payload to process
+ * @param[in] len The length of the RTP payload
+ * @param[in] context The context to use as a reference
+ * @param[in] switched Whether there has been a source switch or not (important to compute offsets) */
+ void janus_vp8_simulcast_descriptor_update(char *buffer, int len, janus_vp8_simulcast_context *context, gboolean switched);
+/*! \brief Helper method to parse a VP9 payload descriptor for SVC-related info (e.g., when SVC is enabled)
+ * @param[in] buffer The RTP payload to process
+ * @param[in] len The length of the RTP payload
+ * @param[out] found Whether any SVC related info has been found or not
+ * @param[out] spatial_layer Spatial layer of the packet
+ * @param[out] temporal_layer Temporal layer of the packet
+ * @param[out] p Inter-picture predicted picture bit
+ * @param[out] d Inter-layer dependency used bit
+ * @param[out] u Switching up point bit
+ * @param[out] b Start of a frame bit
+ * @param[out] e End of a frame bit
+ * @returns 0 in case of success, a negative integer otherwise */
+int janus_vp9_parse_svc(char *buffer, int len, int *found,
+ int *spatial_layer, int *temporal_layer,
+ uint8_t *p, uint8_t *d, uint8_t *u, uint8_t *b, uint8_t *e);
++
+ #endif
--
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/pkg-voip/janus.git
More information about the Pkg-voip-commits
mailing list