creek
Somewhat-interactive Lecture Streaming SoftwareHome | Documentation | Code
Documentation
Table of Contents
01) Use Cases and Decisions
02) Server Setup
02.1) Install Necessary Software
02.2) Configure NGINX
02.3) Configure creek
02.4) (Optional) Configure WebRTC Streams
02.5) Start Creek
03) Streaming
03.1) OBS Setup
03.2) Web Interface: Student View
03.3) Web Interface: Admin View
04) params.js Reference
05) Hacking
06) The Name
1) Use Cases and Decisions
TODO: discuss different methods (HLS, WebRTC, Jitsi Meet, etc) and pros/cons. Maybe the catsoop variant also?
2) Server Setup
If you want to use the server at streaming.catsoop.org to run your
live-streams, send me a note (hz@mit.edu), and I may be able to set you up to
use it (subject to timesharing so as to avoid overloading the server). If it
works, I can set you up with a stream name and a secret key to use. There
should be no additional setup necessary, and you can skip ahead to the section
on Streaming below.
If you want to host this infrastructure on your own (GNU/Linux) machine, there
are a few steps, but nothing too complicated (at least on Debian Stretch, for
which these instructions were written). In all steps below, replace
EXAMPLE.HOST.COM with your hostname.
2.1) Install Necessary Software
To start, we'll need to install a few packages. NGINX, the NGINX-RTMP module, and FFMPEG can be installed from the Debian repositories, but Node will need to be installed from elsewhere since the version in Debian's repo is too old:
$ sudo apt -y install nginx curl git libnginx-mod-rtmp ffmpeg build-essential $ curl -sL https://deb.nodesource.com/setup_14.x | sudo -E bash - $ sudo apt -y install nodejs
You will also need to clone the creek repository and set it up:
$ git clone git://hz.mit.edu/creek.git $ cd creek $ npm install
You will also need SSL certificates. If this server is running at MIT, you can follow the relevant instructions from IS&T (or, if you're at CSAIL, follow the relevant instructions from TIG). If not, you may want to use Letsencrypt:
$ sudo apt -y install certbot python3-certbot-nginx $ sudo certbot --nginx -d EXAMPLE.HOST.COM
You will need to answer a few questions, but, after a short while, you should be shown some congratulatory text saying your certificates have been installed.
2.2) Configure NGINX
We'll need a couple of different pieces in the NGINX configuration: one to set up the RTMP server and set it up to produce a multi-bitrate HLS stream, and another to set up the web/websockets interface. The first is always necessary, but the second is only necessary if you are streaming video via HLS using creek.
Configure Web Interface
First, we'll configure NGINX to route HTTP(s) requests to creek, possibly for a subpath rather than for the root domain . If you want to put creek at my.computer.com/ with no subpath, you could use a block like the following:
location / {
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_cache_bypass $http_upgrade;
proxy_pass http://localhost:6009/;
}
For a subpath install, just change location / to be location /streaming/ or
similar, and add a rewrite rule so that the request is made to creek without
the subpath included (which would confuse it), e.g.:
rewrite /streaming/?(.*) /$1 break;
Regardless, if you are serving video using HLS, you will also want to set the following in your NGINX config, to make sure that requests for the HLS stream files are handled by NGINX directly (rather than being sent to creek):
location /hls/ {
try_files $uri $uri/ =404;
}
(Optional) Configure RTMP/HLS Streams
If you want to stream using RTMP/HLS using this server, add the following (or
something like it) to the very bottom of /etc/nginx/nginx.conf. We'll set up
the "live" stream as the place we push, and we'll use ffmpeg to generate
alternative streams (both RTMP and HLS). This setup creates multiple RTMP
streams at different quality levels, as well as a single HLS stream containing
the multiple different quality levels. Feel free to tweak for your purposes.
rtmp {
server {
listen 1935;
chunk_size 4096;
application live {
live on;
record off;
on_publish http://localhost:6009/auth;
on_publish_done http://localhost:6009/unauth;
allow play 127.0.0.1;
deny play all;
exec ffmpeg -i rtmp://localhost/live/$name -threads 1 -vn -c:a aac -ab 64k -f flv rtmp://localhost/broadcast/$name_audioonly;
exec ffmpeg -i rtmp://localhost/live/$name
-c:v libx264 -sc_threshold 0 -g 60 -crf 10 -maxrate 1.5M -bufsize 4M -s 1280x720 -f flv -c:a aac -b:a 1024k rtmp://localhost/broadcast/$name_hi
-c:v libx264 -sc_threshold 0 -g 60 -crf 20 -maxrate 512k -bufsize 1M -s 640x360 -f flv -c:a aac -b:a 128k rtmp://localhost/broadcast/$name_mid
-c:v libx264 -sc_threshold 0 -g 60 -crf 40 -maxrate 128k -bufsize 256k -s 640x360 -f flv -c:a aac -b:a 32k rtmp://localhost/broadcast/$name_low
-c:v libx264 -sc_threshold 0 -g 60 -crf 40 -maxrate 32k -bufsize 64k -s 384x216 -f flv -c:a aac -b:a 16k rtmp://localhost/broadcast/$name_reallylow
;
}
application broadcast {
live on;
hls on;
hls_fragment 2s;
hls_playlist_length 20s;
hls_path /var/www/html/hls;
hls_nested on;
hls_continuous off;
allow publish 127.0.0.1;
deny publish all;
allow play all;
hls_variant _reallylow BANDWIDTH=50000;
hls_variant _low BANDWIDTH=160000;
hls_variant _mid BANDWIDTH=640000;
hls_variant _hi BANDWIDTH=2500000;
}
}
}
2.3) Configure creek
TODO: document params.js more carefully.
As of right now, there is actually not much to configure in creek itself. But
you will want to make a file called params.js in creek's root directory (next
to index.js) that contains information about the streams that should be
available through this creek instance, as well as shared secret keys that
broadcasters can use to authenticate.
The keys can be whatever you like, but short random sequences of alphanumeric characters are a good idea. Here is an example file:
module.exports = {
'streams': {
'my_awesome_stream': {psk: 'deadbeef1234', title: 'The Best Stream'},
}
}
Each person broadcasting to a given stream will need to know that stream's PSK, which not only allows broadcast access to the streams, but also allows access to the admin interface (to see questions and live poll results in the web interface).
2.4) (Optional) Configure WebRTC Streams
TODO: write these docs.
2.5) Start Creek
Make sure you're in the right directory, and run:
$ node index.js
(I always start this inside of GNU Screen so that I can leave the server with no problems.)
3) Streaming
3.1) OBS Setup
Any streaming software capable of speaking RTMP should work fine for a server set up as described above, but OBS is probably the thing to use unless you have something else in mind. The first step is to set things up so that we're sending to the right place.
Assuming OBS is cool for you, you should download it and install it, and when starting it for the first time, say yes you want to use the wizard.
Choose to "optimize for streaming, recording is secondary". For the next question, I choose to force 30 fps (we have absolutely no need to stream at 60fps). Then under stream type, choose "Custom Streaming Server". Enter the following information (substituting your own hostname, stream name and key):
- Server:
rtmp://EXAMPLE.HOST.COM/live - Stream Key:
YOUR_STREAM_NAME?psk=YOUR_KEY
It should be fine to let it test for optimal bandwidth, which will take a little while. You can also tweak quality settings later on from the settings menu (happy to help with that). If possible, Ethernet is going to mean that you can push a better quality stream vs Wifi.
Assuming that all worked, you're (almost) ready to stream. The remaining piece is to set up the audio and video sources you want to send. OBS splits these into "scenes," each of which can be comprised of several "sources." To start, you'll want to make sure you have some kind of video feed (screen capture and/or video from a webcam) and that your mic is working properly (you should see the audio levels bounce around; if not, you can add another audio source for your mic).
For my setup, I have 4 scenes: the "test" screen (which will be replaced with a "here's what we talked about recently and here's what we're talking about today" screen), one pure screen capture filling the whole screen, one pure webcam capture filling the whole screen, and one with the screen taking most of the view and my webcam scaled down in the bottom-right corner (staff have indicated that seeing a face is nice, so I plan to use this one a lot). You certainly don't need anything fancy, though.
If you "Start Streaming," then after a few seconds, going to your stream's URL
(https://EXAMPLE.HOST.COM/?stream=YOUR_STREAM) should show your stream (a few
seconds delayed). "Start Recording" should save a local copy to disk. You can
run both of those in parallel. It's also possible to tweak the quality of the
recorded video separately from the quality of the stream (under Settings).
3.2) Web Interface: Student View
Navigating to https://EXAMPLE.HOST.COM/?stream=YOUR_STREAM should show you a
view that students will see when connecting to the stream. At the top, they
see your video stream (a few seconds delayed). Then, they see a place to send
questions to you during the stream, as well as a place to "react" to things in
the stream by changing remporarily changing their emoji to a different image
(other students see these 'reactions,' but not any questions).
3.3) Web Interface: Admin View
While streaming, you can get to an admin view by navigating to the following address:
https://EXAMPLE.HOST.COM/?stream=YOUR_STREAM&video=no&psk=YOUR_KEY
This view allows you see students' questions and reactions, and also to pose multiple-choice "concept questions" to the students.
As of now, everything in the web view is totally ephemeral. Messages get passed around, but nothing is stored on the server (and no one is authenticated in any way other than the PSK). This is worth mentioning because it means that if you refresh the admin view, any questions that you had received from students will no longer be accessible. I'm not sure this is the best design decision, but it's the one I've made for now.
4) params.js Reference
TODO: list all parameters and their usage here, along with an example params.js file.
5) Hacking
All of the source code is at https://hz.mit.edu/cgit/creek.git/. Apologies for the code being ugly.
You can clone a copy for yourself with the following command:
$ git://hz.mit.edu/creek.git
If you have questions, suggestions, or patches, please send them via e-mail, to
hz@mit.edu.
For a list of things that I may work on in the future (or that would be welcome
contributions), see the TODO
file.
6) The Name
creek (krēk, krĭk)
n. A stream of water smaller than a river and larger than a brook.
Hilarious, I know.