Building scalable video conferencing

 

By Simon Woodhead

We announced last week how we’d spun up https://meet.simwood.com to help customers and the community through these times.

Others had a similar idea (a week later) and some claim to have “created” it. We, like them, are using Jitsi Meet which is an open source project we’ve liked for years and highly recommend. It is lead by our old friend Emil Ivov, and we’ve tried to support the project over the years – we knew we’d get opportunity to play with it one day. We didn’t invent it, it is open source. As has been amply demonstrated, any spanner can spin up a container, put their logo on it and give absolutely nothing back. I think that is pretty low but I digress.

So, why the post?

Well, firstly, we’ve been slightly blown away by how much it has been used. Those of you who tried https://meet.simwood.com last week might have found intermittent connectivity. From Universities doing remote learning, through Governments, through healthcare, including of course ourselves and many other SMEs, it has been hit pretty hard! But that’s a great thing!

We very quickly needed to scale it and that proved hard. We soon learned that video is a completely different animal to voice in so many ways! But my amazing colleagues knocked it out the park, as always, although have been burning the midnight oil to do so.

What we’ve got now is more of a clustered solution with core components like the Prosody XMPP server running HA, and a minimum of two nodes up. As things get busier we’re dynamically adding new video-bridge nodes, joining them to the cluster and then removing them when things quieten down. New conferences go to the least-busy node, until a trigger where we add more. It is awesome and completely scalable!

We’ve also added a stats module so we can monitor just how much you’re using it! I think we hold the record for conference size so far (but try it!) when we had a most-of-team social isolation check-in earlier in the week. The average call seems to be 3-4 parties with quite a lot of 2 people. The most pleasing stat is 0 failed conferences and 0 partially failed conferences since we moved to the clustered solution. We’re not logging anything remotely sensitive as privacy and security are one of the big USPs with this.

We’ve also dramatically simplified multi-branding so we can use it across our business, but also for those of you who have asked for your own version!

What started out as a quick solution to a problem has become something of a pre-occupation. We have two really exciting things in the pipeline:

  • a native app!
  • global proximity

The latter is really exciting and enables us to scale a conference beyond a single server in the cluster. In fact, it positions every user entering the conference on their closest media server, so two users in the USA talking to two users in Asia will respectively find their media local, and thus higher quality / throughput to each other than it would be with a single central host. It also means we can control the backbone between those hosts rather than rely on each participant to reach a potentially very distant host.

So although something we’re giving away for free, and an amazing open source project you can try yourselves, we’re taking deploying this pretty seriously. We need to given some of the uses you guys tell us you’re putting it to – we’ve no idea otherwise because that is private!

And that brings me to an important point. We’ve had the odd person question why we bothered when there’s Zoom. Well three reasons really.

  1. Zoom’s A Lifeline During COVID-19: This Is Why It’s Also A Privacy Risk” and “Beware of ‘ZoomBombing:’ screensharing filth to video calls” and “Zoom used for Cabinet meetings despite MoD ban over security concerns
  2. With annual pricing and essential add-ons it gets very expensive very quick. We know we’ve paid for it!
  3. Jeremy Corbyn uses it!

Enjoy, and thanks again to Emil and the whole Jitsi team!