Self-hosted videos with HLS
Vincent Bernat
Hosting videos on YouTube is convenient for several reasons:
pretty good player, free bandwidth, mobile-friendly, network effect
and, at your discretion, no ads.1 On the other hand, this is one
of the less privacy-friendly solution. Most other providers
share the same characteristics—except the ability to disable ads for
free.
With the <video>
tag, self-hosting a video is
simple:2
<video controls> <source src="../videos/big_buck_bunny.webm" type="video/webm"> <source src="../videos/big_buck_bunny.mp4" type="video/mp4"> </video>
However, while it is possible to provide a different videos depending on the screen width, adapting the video to the available bandwidth is trickier. There are two solutions:
- HLS (RFC 8216); and
- MPEG-DASH (ISO/IEC 23009-1:2014).
They are both adaptive bitrate streaming protocols: the video is sliced in small segments and made available at a variety of different bitrates. Depending on current network conditions, the player automatically selects the appropriate bitrate to download the next segment.
HLS was initially implemented by Apple but is now also supported natively by Microsoft Edge and Chrome on Android. hls.js is a JavaScript library bringing HLS support to other browsers. MPEG-DASH is technically superior (codec-agnostic) but only works through a JavaScript library, like dash.js. In both cases, support of the Media Source Extensions is needed when native support is absent. Safari on iOS doesn’t have this feature and cannot use MPEG-DASH. Consequently, the most compatible solution is currently HLS.
Encoding#
To serve HLS videos, you need three kinds of files:
- the media segments (encoded with different bitrates/resolutions);
- a media playlist for each variant, listing the media segments; and
- a master playlist listing the media playlists.
Media segments can come in two formats:
- MPEG-2 Transport Streams (TS); or
- Fragmented MP4.
Fragmented MP4 media segments are supported since iOS 10. They are a bit more efficient and can be reused to serve the same content as MPEG-DASH (only the playlists are different). Also, they can be served from the same file with range requests. However, if you want to target older versions of iOS, you need to stick with MPEG-2 TS.3
FFmpeg is able to convert a video to media segments and generate the associated media playlists. Peer5’s documentation explains the suitable commands. I have put together a handy (Python 3.6) script, video2hls, stitching together all the steps. After executing it on your target video, you get a directory containing:
- media segments for each resolution (
1080p_1_001.ts
,720p_2_001.ts
, …) - media playlists for each resolution (
1080p_1.m3u8
,720p_2.m3u8
, …) - master playlist (
index.m3u8
) - progressive (streamable) MP4 version of your video (
progressive.mp4
) - poster (
poster.jpg
)
The script accepts a lot of options for customization. Use the
--help
flag to discover them. Run it with --debug
to get the
ffmpeg
commands executed with an explanation for each flag. For
example, the poster is built with this command:
ffmpeg \ `# seek to the given position (5%)` \ -ss 4 \ `# load input file` \ -i ../2018-self-hosted-videos.mp4 \ `# take only one frame` \ -frames:v 1 \ `# filter to select an I-frame and scale` \ -vf 'select=eq(pict_type\,I),scale=1280:720' \ `# request a JPEG quality ~ 10` \ -qscale:v 28 \ `# output file` \ poster.jpg
Serving#
We got a bunch of static files we can upload anywhere. Yet two details are important:
- When serving from another domain, CORS needs to be configured
to allow
GET
requests. AddingAccess-Control-Allow-Origin: *
to response headers is enough.4 - Some clients may be picky about the MIME types. Ensure files are served with the ones in the table below.
Kind | Extension | MIME type |
---|---|---|
Playlists | .m3u8 |
application/vnd.apple.mpegurl |
MPEG2-TS segments | .ts |
video/mp2t |
fMP4 segments | .mp4 |
video/mp4 |
Progressive MP4 | .mp4 |
video/mp4 |
Poster | .jpg |
image/jpeg |
Let’s host our files on Exoscale’s Object Storage which is compatible with S3 and located in Switzerland. As an example, the Caminandes 3: Llamigos video is about 213 MiB (five sizes for HLS and one progressive MP4). It would cost us less than €0.01 per month for storage and €1.42 for bandwidth if 1000 people watch the 1080p version from beginning to end—unlikely.5
We use s3cmd
to upload files. First, you need to recover your API
credentials from the portal and put them in ~/.s3cfg
:
[default] host_base = sos-ch-dk-2.exo.io host_bucket = %(bucket)s.sos-ch-dk-2.exo.io access_key = EXO..... secret_key = .... use_https = True bucket_location = ch-dk-2
The second step is to create a bucket:
$ s3cmd mb s3://hls-videos Bucket 's3://hls-videos/' created
You need to configure the CORS policy for this bucket. First, define
the policy in a cors.xml
file (you may want to restrict the allowed
origin):
<CORSConfiguration> <CORSRule> <AllowedOrigin>*</AllowedOrigin> <AllowedMethod>GET</AllowedMethod> </CORSRule> </CORSConfiguration>
Then, apply it to the bucket:
$ s3cmd setcors cors.xml s3://hls-videos
The last step is to copy the static files. Playlists are served compressed to save a bit of bandwidth. For each video, inside the directory containing all the generated files, use the following command:
while read extension mime gz; do [ -z "$gz" ] || { # gzip compression (if not already done) for f in *.${extension}; do ! gunzip -t $f 2> /dev/null || continue gzip $f mv $f.gz $f done } s3cmd --no-preserve -F -P \ ${gz:+--add-header=Content-Encoding:gzip} \ --mime-type=${mime} \ --encoding=UTF-8 \ --exclude=* --include=*.${extension} \ --delete-removed \ sync . s3://hls-videos/video1/ done <<EOF m3u8 application/vnd.apple.mpegurl true jpg image/jpeg mp4 video/mp4 ts video/mp2t EOF
The files are now available at
https://hls-videos.sos-ch-dk-2.exo.io/video1/
.
HTML#
We can insert our video in a document with the following markup:
<video poster="https://hls-videos.sos-ch-dk-2.exo.io/video1/poster.jpg" controls preload="none"> <source src="https://hls-videos.sos-ch-dk-2.exo.io/video1/index.m3u8" type="application/vnd.apple.mpegurl"> <source src="https://hls-videos.sos-ch-dk-2.exo.io/video1/progressive.mp4" type='video/mp4; codecs="avc1.4d401f, mp4a.40.2"'> </video>
Browsers with native support use the HLS version while others would fall back to the progressive MP4 version. However, with the help of hls.js, we can ensure most browsers benefit from the HLS version too:
<script src="https://cdn.jsdelivr.net/npm/hls.js@latest"></script> <script> if(Hls.isSupported()) { var selector = "video source[type='application/vnd.apple.mpegurl']", videoSources = document.querySelectorAll(selector); videoSources.forEach(function(videoSource) { var m3u8 = videoSource.src, once = false; // Clone the video to remove any source var oldVideo = videoSource.parentNode, newVideo = oldVideo.cloneNode(false); // Replace video tag with our clone. oldVideo.parentNode.replaceChild(newVideo, oldVideo); // Add an empty source (enable play event on Chromium 72+) newVideo.src = "about:blank"; // On play, initialize hls.js, once. newVideo.addEventListener('play',function() { if (once) return; once = true; var hls = new Hls({ capLevelToPlayerSize: true }); hls.loadSource(m3u8); hls.attachMedia(newVideo); hls.on(Hls.Events.MANIFEST_PARSED, function() { newVideo.play(); }); }, false); }); } </script>
Update (2018-12)
Starting from Chromium 72, a video without a
source (like built with the snippet above) cannot trigger a play event
anymore. Therefore, we use a dummy source using about:blank
.
Here is the result, featuring Caminandes 3: Llamigos, a video created by Pablo Vasquez, produced by the Blender Foundation and released under the Creative Commons Attribution 3.0 license:
Most JavaScript attributes, methods and events work just like with a
plain <video>
element. For example, you can seek to an arbitrary
position, like 1:00 or 2:00—but you would need to enable JavaScript to
test.
The player is different from one browser to another but provides the basic needs. You can upgrade to a more advanced player, like video.js or MediaElements.js. They also handle HLS videos through hls.js.
Update
For subtitles, have a look at “Self-hosted videos with HLS: subtitles.”
Hosting your videos on YouTube is not unavoidable: serving them yourself while offering quality delivery is technically affordable. If bandwidth requirements are modest and the network effect not important, self-hosting makes it possible to regain control of the published content and not to turn over readers to Google. In the same spirit, PeerTube offers a video sharing platform. Decentralized and federated, it relies on BitTorrent to reduce bandwidth requirements.
Addendum#
Preloading#
In the above example, preload="none"
was used for two reasons:
- Most readers won’t play the video as it is an addon to the main content. Therefore, bandwidth is not wasted by downloading a few segments of video, at the expense of slightly increased latency on play.
- We do not want non-native HLS clients to start downloading the
non-HLS version while hls.js is loading and taking over the
video. This could also be done by declaring the progressive MP4
fallback from JavaScript, but this would make the video unplayable
for users without JavaScript. If preloading is important, you can
remove the
preload
attribute from JavaScript—and not wait for the play event to initialize hls.js.
CSP#
Setting up CSP correctly can be quite a pain. For browsers with native HLS support, you need the following policy, in addition to your existing policy:
image-src https://hls-videos.sos-ch-dk-2.exo.io
for the posters; andmedia-src https://hls-videos.sos-ch-dk-2.exo.io
for the playlists and media segments.
With hls.js, things are more complex. Ideally, the following policy should also be applied:
worker-src blob:
for the transmuxing web worker;media-src blob: about:
for the transmuxed segments; andconnect-src https://hls-videos.sos-ch-dk-2.exo.io
to fetch playlists and media segments from JavaScript.
However, worker-src
is quite recent. The expected
fallbacks are child-src
(deprecated), script-src
(but
not everywhere) and then default-src
. Therefore, for
broader compatibility, you also need to append blob:
to
default-src
as well as to script-src
and child-src
if you
already have them. Here is an example policy—assuming the original
policy was just default-src 'self'
and media, XHR and workers were
not needed:
HTTP/1.0 200 OK Content-Security-Policy: default-src 'self' blob:; image-src 'self' https://hls-videos.sos-ch-dk-2.exo.io; media-src blob: about: https://hls-videos.sos-ch-dk-2.exo.io; connect-src https://hls-videos.sos-ch-dk-2.exo.io; worker-src blob:;
-
YouTube will now show ads on all videos even if creators don’t want them. ↩︎
-
Nowadays, everything supports MP4/H.264. It usually also brings hardware acceleration, which improves battery life on mobile devices. WebM/VP9 provides a better quality at the same bitrate. ↩︎
-
You could generate both formats and use them as variants in the master playlist. However, a limitation in hls.js prevents this option. ↩︎
-
Use
https://example.org
instead of the wildcard character to restrict access to your own domain. ↩︎ -
There is no need to host these files behind a (costly) CDN. Latency doesn’t matter much as long as you can sustain the appropriate bandwidth. ↩︎