Background Reading

Submitted by aalhad on Sat, 09/05/2020 - 14:10

Languages you need to know

Java, Kotlin, Lua

WebRTC

WebRTC chapter from a book by Ilya Grigorik - https://hpbn.co/webrtc/
Actually the entire book is good https://hpbn.co/
https://webrtc.org/

XMPP

https://developer.ibm.com/technologies/messaging/tutorials/x-xmppintro/
https://www.blikoontech.com/tutorials/xmpp-made-simple-roster-and-presence-explained
https://medium.com/@thinkwik/web-sockets-vs-xmpp-which-is-better-for-chat-application-113e3520b327

This is the XMPP client library that jitsi uses - https://github.com/igniterealtime/Smack
And this is the XMPP server that jitsi uses - https://prosody.im/
https://prosody.im/doc/xmpp

TURN/STUN/ICE

Interactive Connectivity Establishment (ICE): A Protocol for Network Address Translator (NAT) Traversal

READ the RFCs
https://tools.ietf.org/html/rfc8445

https://www.html5rocks.com/en/tutorials/webrtc/infrastructure/
https://temasys.io/webrtc-ice-sorcery/

Understand and take notes of the ICE implementation that jitsi uses
https://github.com/jitsi/ice4j

Real world implementation of TURN - also, there was a reference to this on the jitsi forums
https://github.com/coturn/coturn

The coturn home page has a good list of RFCs

TURN specs:

RFC 5766 - base TURN specs
RFC 6062 - TCP relaying TURN extension
RFC 6156 - IPv6 extension for TURN
RFC 7443 - ALPN support for STUN & TURN
RFC 7635 - oAuth third-party TURN/STUN authorization
DTLS support (http://tools.ietf.org/html/draft-petithuguenin-tram-turn-dtls-00).
Mobile ICE (MICE) support (http://tools.ietf.org/html/draft-wing-tram-turn-mobility-02).
TURN REST API (http://tools.ietf.org/html/draft-uberti-behave-turn-rest-00)
Origin field in TURN (Multi-tenant TURN Server) (https://tools.ietf.org/html/draft-ietf-tram-stun-origin-06)
TURN Bandwidth draft specs (http://tools.ietf.org/html/draft-thomson-tram-turn-bandwidth-01)
TURN-bis (with dual allocation) draft specs (http://tools.ietf.org/html/draft-ietf-tram-turnbis-04).

STUN specs:

RFC 3489 - "classic" STUN
RFC 5389 - base "new" STUN specs
RFC 5769 - test vectors for STUN protocol testing
RFC 5780 - NAT behavior discovery support
RFC 7443 - ALPN support for STUN & TURN
RFC 7635 - oAuth third-party TURN/STUN authorization

Supported ICE and related specs:

RFC 5245 - ICE
RFC 5768 – ICE–SIP
RFC 6336 – ICE–IANA Registry
RFC 6544 – ICE–TCP
RFC 5928 - TURN Resolution Mechanism

The implementation fully supports the following client-to-TURN-server protocols:

UDP (per RFC 5766)
TCP (per RFC 5766 and RFC 6062)
TLS (per RFC 5766 and RFC 6062): TLS1.0/TLS1.1/TLS1.2; ECDHE is supported.
DTLS (http://tools.ietf.org/html/draft-petithuguenin-tram-turn-dtls-00): DTLS versions 1.0 and 1.2.
SCTP (experimental implementation).

Supported relay protocols:

UDP (per RFC 5766)
TCP (per RFC 6062)

Jingle

An XMPP protocol extension for initiating and managing peer-to-peer media sessions between two XMPP entities in a way that is interoperable with existing Internet standards.
https://xmpp.org/extensions/xep-0166.html

SIP -

a signaling protocol used for initiating, maintaining, and terminating real-time sessions that include voice, video and messaging applications.
https://en.wikipedia.org/wiki/Session_Initiation_Protocol

RTP - realtime transport protocol

https://en.wikipedia.org/wiki/Real-time_Transport_Protocol

RTSP

a network control protocol designed for use in entertainment and communications systems to control streaming media servers

SDP

a format for describing multimedia communication sessions for the purposes of session announcement and session invitation

 

Jibri

Jibri provides services for recording or streaming a Jitsi Meet conference.
https://github.com/jitsi/jibri

Colibri

an XMPP extension that allows real-time communications clients to discover and interact with conference bridges that provide conference mixing or relaying capabilities.
https://xmpp.org/extensions/xep-0340.html

SFUs, MCUs and performance

Multipoint Control Units (MCUs), Selective Forwarding Units (SFUs)
https://blog.developer.atlassian.com/video-conferencing/

Overall flow of Jitsi

Overall steps in the communication between client, jicofo and jvb.

A client connects to the xmpp, and sends an IQ to the jicofo component address.
Jicofo creates a room, joins the room, and make the one who requested to create the room an owner in the room, if he is the first one.
The client receives the succesfull response from jicofo and joins the room.
Nothing happens, here as there is just one participant.
A second participant joins, jicofo sees that and starts the process of connecting both clients sends an IQ to the jvb component address to allocate channels for both participants.
uses the allocated channels data to create jingle session-initiate, which then sends to both clients

Clients accept it and start sending and they are now connected using jvb.
From https://community.jitsi.org/t/how-jitsi-work/14482/2

A nice diagram showing the flow
https://go.gliffy.com/go/publish/image/7649541/L.png