LibrePlanet: Conference/2015/Streaming
Contents
Equipment Template — 3-Room Simultaneous Desktop + In-Room Streaming
Camera
- (3×) Elphel NC353L Camera (w/ '12-36V DC/DC' / "12V" option) + C/CS lens adapter ring in-place
- (3×) Tycon POE-INJ-LED-S Shielded POE Inserter with Power & Current LEDS
- (3×) Elphel power adapter, stock [TODO: identify 19V recommendation for long cable runs]
- (3×) 6' Belkin Premium Cat5e Networking Cable (flat/snagless)
- (3×) Universal lithium-ion power pack (generic) + charging cables (camera+router battery backup) [TODO: discontinued, find replacement item]
- (3×) One of the following lenses:
- Fujinon HF12.5SA-1 2/3" 12.5mm fixed focal lens (5 megapixel, c-mount)
- Fujinon DV3.4x3.8SA-1 3.8-13mm vari-focal lens (3 megapixel, c-mount)
- Computar E3Z4518CS-MPIR 5MP Full HD IR Varifocal Lens, 4.5-13.2mm (5 megapixel, c*s*-mount ) (boderline unsuitable for low-lighting conditions)
- C-mount <-> Canon EF/EF-S Adapter + manual or manual-option Canon lens of your choice (slight loss in pixel definition)
- (1×) Lens cloth + protective case
- (3×) Dolica Proline AX620B100 Tripod –or– (3×) SystemPro "The Clamper" Jr.
Laptop
- (6×) Lenovo x200, Libreboot'd + Open Video Reference Build
- Non-ancient layer of thermal paste under CPU heatsink, well-dusted exhaust fan
- 4GB of fast memory, matching modules. Memtest86+ testing recommended.
- >60GB hard drive, SSD preferred (8 hrs = ~50GB video file)
- Non-dead battery, no loose power jack connections or similar problems
- (3-6×) Kensington Combination Laptop Lock K64673US
- Note that the laptop HDD is easily removable and should be secured when streaming is inactive
- (3-6×) Lenovo 90W Ultraslim AC/DC Combo Adapter, Lenovo P/N 41R4493 (optional)
Audio
- (3×) Ifrogz EarPollution Plugz earbuds + single-use alcohol cleansing pads
One of the following:
- (3×) Pyle PAD10MXU 2-Channel Mini-Mixer
- (3×) Shure X2u XLR-to-USB Signal Adapter
- (3×) Behringer QXENYX1002 USB sound board [TODO: evaluate alternative multi-channel boards]
- (6×) 4' USB 2.0 A/B cable
- (3×) TRS (tip ring sleeve) <-> "male XLR" adapters
- (3×) Dolica Proline AX620B100 Tripod (mic stand – unscrew ball head)
- (3x) Ultimate Support ULTI-BOOM-TB Telescopic Mic Boom
- (3×) [TODO: Research suitable devices for digital-audio XLR-out from podium laptops]
- (3×) 15' XLR cables for balanced-audio out from podium laptop
- (3×) [TODO: Add podium mic stand model numbers]
Some combination of the following:
- (6-9×) Audio-Technica MB-1K Handheld Cardioid Dynamic Vocal Microphone –or– Audio Technica P735 Hypercardioid Dynamic Mic (high-end)
- (3×) XLR<->XLR "male-to-male" adapter
- (3×) XLR<->XLR "female-to-female" adapter
- (6×) 15' XLR cables, good condition
- (3×) Audio-Technica AT8202 Adjustable Inline Attenuator ("padders") (optional -- convert from line-level to mic-level)
- (3×) Shure MX393/O - Omni-Directional Boundary Microphone (ambient-sound pickup -- suitable for group conversations)
Semi-Portable, High-Power In-Room Projection
- (3×) QSC GX3, GX5, GX7 power amplifier
- (6×) QSC Audio AD-S82 –or– QSC I-282H ISIS Surface Mount Loudspeaker
- (6×) 15' Speakon or speaker-wire cables
- (3×) Dolica Proline AX620B100 Tripod (speaker supports)
- To convert these tripods to moderate-duty speaker supports: extend center column several inches, tighten locking collar, widen each support leg to the 2nd position using leg angle adjustment locks, remove the entirety of the ball head assembly (grip firmly and unscrew counter-clockwise), secure mic stand adapter nuts w/ Allen key, then twist-in loudspeakers to the adapter nuts
- (3×) 5/8”-27 -- M20 mic stand adapters with locking set screw [TODO: Verify]
- (1×) Allen key, small
General
- (3×) 14' Belkin Premium Cat5e Networking Cable (flat/snagless cable, black) (front-of-room to encoding station)
- (3×) 5-Port Gigabit Ethernet Switch [TODO: find model number]
- (3×) 15' Heavy-guage extension cord, orange
- (3×) Power Sentry 5-Outlet Powersquid
- (3x) Equipment transport bins, clear plastic, snap-shut lid, ~20" x ~15"
- (3×) 9" LED-lighted 3-prong extenion cord (for tight spaces)
- (1×) Label maker
- (1×) Victorinox 54525 pocketknife, black
- (1×) Energizer Aluminum Alloy Waterproof Lithium LED Flashlight
- (1×) 30 yd. Gaffers Tape, black ("tape everything!")
- (1×) Cooper Lighting PQS2504IN1 250W Halogen 4-in-1 Worklight (w/ 150w halogen bulb) (makeshift high-intensity room light)
Software Used
Streaming Server
- IceCast2 2.4.1
Streaming Clients
The following software was compiled and installed via the 'Open Video Reference Build' project on Trisquel 7 GNU/Linux:git clone git://code.openvideo.pro/openvideo-build.git
- GStreamer
- Jack Audio Server
- [TODO: add the rest]
Set up
General
- Unpack the gigabit switch
- Plug the switch into the wall-power outlet
- Use an Ethernet cable to connect the switch to the venue LAN
- Unpack the Elphel camera and attach it to the tripod
- Attach an Ethernet cable into the RJ45 connector on the camera
- Plug the other end of the cable into the passive-PoE injector
- Plug the passive-PoE injector into the switch
- Connect the passive-PoE injector to the wall-power outlet using the power adapter
At this point you should have have the Elphel 353 camera attached to a tripod, powered on, with the network connection physically setup.
Connecting to the camera
Additional parameters are needed with recently-released SSH clients as camera-supported crypto methods are being deprecated:ssh -c aes128-cbc -m hmac-sha1 root@192.168.0.9 scp -c aes128-cbc -o MACs=hmac-sha1 root@192.168.0.9:/any/file .
Use the above additional parameters if you receive the following error message when attempting to connect:debug1: expecting SSH2_MSG_KEXDH_REPLYRead from socket failed: Connection reset by peer
The default password is the four-letter word 'pass'; 'root' is the only account. Connecting to the camera in this way is only necessary for persistent configuration changes. Note that the camera requires approximately 60 seconds to boot after power is supplied.
Changing Imaging Defaults on the Elphel 353
It's helpful to modify parameter defaults to set the preferred capture mode automatically upon each boot -- note that if the image format provided by the camera is mismatched to the GStreamer pipeline the pipeline will fail. An example that changes the window-of-interest to match our "extra-widescreen" pipelines, increases the default JPEG image quality, and disables camera autoexposure in favor of manual control is as follows:
Start by obtaining a copy of the default parameter set:scp -c aes128-cbc -o MACs=hmac-sha1 root@192.168.0.9:/etc/autocampars.xml .
Copy the following patch into a text editor and save as 'elphel-widescreen.patch' :
--- autocampars.xml.original 2015-09-11 00:45:09.479139493 -0400
+++ autocampars.xml 2015-09-11 01:13:09.405220797 -0400
@@ -532,10 +532,10 @@
<TRIG>"0"</TRIG>
<EXPOS>"10000"</EXPOS>
<VIRT_KEEP>"0"</VIRT_KEEP>
- <WOI_LEFT>"0"</WOI_LEFT>
- <WOI_TOP>"0"</WOI_TOP>
- <WOI_WIDTH>"10000"</WOI_WIDTH>
- <WOI_HEIGHT>"10000"</WOI_HEIGHT>
+ <WOI_LEFT>"336"</WOI_LEFT>
+ <WOI_TOP>"428"</WOI_TOP>
+ <WOI_WIDTH>"2592"</WOI_WIDTH>
+ <WOI_HEIGHT>"1120"</WOI_HEIGHT>
<FLIPH>"0"</FLIPH>
<FLIPV>"0"</FLIPV>
<FPSFLAGS>"0"</FPSFLAGS>
@@ -563,7 +563,7 @@
<RSCALE_CTL>"0"</RSCALE_CTL>
<GSCALE_CTL>"0"</GSCALE_CTL>
<BSCALE_CTL>"0"</BSCALE_CTL>
- <QUALITY>"80"</QUALITY>
+ <QUALITY>"90"</QUALITY>
<PORTRAIT>"0"</PORTRAIT>
<CORING_INDEX>"327685"</CORING_INDEX>
<COLOR_SATURATION_BLUE>"200"</COLOR_SATURATION_BLUE>
@@ -581,7 +581,7 @@
<DGAINGB>"32768"</DGAINGB>
<DGAINB>"32768"</DGAINB>
<CORING_PAGE>"0"</CORING_PAGE>
- <AUTOEXP_ON>"1"</AUTOEXP_ON>
+ <AUTOEXP_ON>"0"</AUTOEXP_ON>
<HISTWND_RWIDTH>"32768"</HISTWND_RWIDTH>
<HISTWND_RHEIGHT>"32768"</HISTWND_RHEIGHT>
<HISTWND_RLEFT>"32768"</HISTWND_RLEFT>
@@ -614,7 +614,7 @@
<COMPMOD_DCSUB>"0"</COMPMOD_DCSUB>
<SENSOR_REGS>""</SENSOR_REGS>
<DAEMON_EN>"0"</DAEMON_EN>
- <DAEMON_EN_AUTOEXPOSURE>"1"</DAEMON_EN_AUTOEXPOSURE>
+ <DAEMON_EN_AUTOEXPOSURE>"0"</DAEMON_EN_AUTOEXPOSURE>
<DAEMON_EN_STREAMER>"1"</DAEMON_EN_STREAMER>
<DAEMON_EN_CCAMFTP>"0"</DAEMON_EN_CCAMFTP>
<DAEMON_EN_CAMOGM>"0"</DAEMON_EN_CAMOGM>
Then apply the patch we just created to the local copy of the camera parameter set:patch autocampars.xml widescreen_patch.patch
Finally, upload the patched file to the camera:ssh -c aes128-cbc -m hmac-sha1 root@192.168.0.9 'cat - > /etc/autocampars.xml; sync' < autocampars.xml
Power-cycle the camera to check that the changes were properly applied.
Image Quality: FPS & Color Balance
Start by setting brightness ("white sun" icon) in the 'Camera GUI (camvc) Controls' to a high value (~90% of max) then slowly decrease the value until 18FPS is achieved, regardless of final streaming framerate. If the resulting image is too dark enlarge the lens aperture and ambient lighting; if the image remains too dark increase gain ("green sun" icon) and gamma ("black/white" marble with vertical divider). Alternatively, 16-17FPS (~4FPS in completely-dark rooms) is acceptable at the cost of noticeable motion blur.
Then, slowly increase the black level ("black/white marble" with horizontal divider) until black objects start to transition into true black in the image. Click the gain icon ("green sun") once to change color control to manual, increase color saturation slightly ("red|green|blue" striped icon) then proceed to tweak the remaining color controls. Clicking on any slider from the bottom will provide coarse adjustment; clicking from the top will provide fine ajustment.
After initial camera values have been set resize the GUI window to hide the in-browser preview image and monitor focus using the GStreamer live preview window.
Understanding the Pyle PAD10MXU Mini-Mixer
The Pyle PAD10MXU ("mini-board") is a small form-factor audio capture/playback device useful for live-streaming applications. The mini-board has two capture channels as follows:
- Ch 1 is a mono-only channel that provides a balanced-audio capture interface suitable for use with XLR or 'Tip-Ring-Sleeve (TRS)' male plugs – simply push the plug in after aligning pins (XLR only, TRS is center-plug). This mono channel ("Line 1") is split/duplicated into stereo tracks as necessary (i.e., 2-channel USB capture, headphone monitoring, Main Out). Additionally, the push-button directly under the XLR/TRS combo jack toggles the impedance of the jack and should be set to Line (a.k.a. Line Level / Low-Z) for most purposes and can be connected directly to many, but not all, microphones.
- Channel 2 is comprised of a) Line 2/3, which provide two unbalanced-audio, capture-only RCA connectors – left and right stereo, respectively, and b) digital-audio playback when the USB Assign to Line push-button is depressed (via additive mixing). That is to say it's possible to combine audio from a live microphone (CH 1), output from a portable music player (Line 2/3), and audio from a web browser (also Line 2/3) simultaneously into Main Out.
Ch 1 has a dedicated gain control knob that provides +10dB signal/volume boost. A corresponding Peak red LED indicator lights if the gain and/or signal level is high enough to cause undesirable sound distortion; the indicator, which flickers in realtime according to level, is a warning light and should remain unlit during normal operation.
Line 2/3 Level is a dedicated control knob that provides 0 - 10dB gain to Channel 2. The individual levels for each line, including USB playback, are not controlled individually but rather globally. For USB-audio-playback-only via the USB to Computer jack simply disconnect the RCA input jacks and depress the USB Assign to Line button.
Monitor is a dedicated control knob that provides 0 - 10dB gain for the 1/4" headphone jack labelled Phones Monitor. It is provided simply for listening convenience and does not affect any other aspect of the mini-mixer.
Master is a dedicated control knob that provides 0 - 10dB gain for Main Out and USB capture. Master has an associated Clip LED indicator which is akin to the Peak indicator with the exception that green coloration indicates "active sound in 80% of maximum range" (good) and red coloration when output is being actively clipped (bad).
USB to Computer is a bi-directional USB Type B jack located on the back of the mini-mixer that either a) converts Master to digital samples suitable for capture by applications such as Audacity, and/or b) mixes digital samples received to Line 2/3 depending on the state of the USB Assign to Line button. The mini-mixer is powered directly by the host computer when the jack is used as USB to Computer – Ext. DC In is the term used when the mixer is disconnected from a host computer and is powered via any USB power adapter or hub rated 500mA or higher.
Phantom Power is a depressible button that provides 18V of XLR-delivered phantom power (17.3V actual) for microphones that require it for operation. It will have no effect on microphones that do not require it. The associated red LED indicator will slowly fade-to-unlit after the button has been disabled or if the mini-mixer has lost power.
Power is a green LED that remains lit if the mini-mixer is powered.
Quirks/Caveats
- If 'lsusb |grep -i 'PCM2900' does not return any results after plugging in the mini-mixer try using another USB port. [TODO: File kernel bug report.]
- Some professional microphones require 48V of phantom power and are unable to be powered directly by the mini-mixer. When using any microphone set Ch 1 gain to +10dB/Max to boost the signal from mic-level to line-level.
- Excessive clipping on Ch 1 may cause Phones Monitor to go silent presumably due to overall device power limitations. If this happens re-plugging the Phones Monitor jack will resolve this issue.
- Where possible, use only balanced-audio to prevent audio quality issues common to RCA and minijack connections. Specifically, use Ch 1 only with a balanced-XLR source, set Line 2/3 Level to 0dB/Min when not in use, and leave Main Out disconnected.
- Where possible, use a headset with a full-sized 1/4" TRS jack—and not an 1/4" to minijack adapter—to prevent loose connections from interfering with audio monitoring.
- The mini-mixer is marketed as both 'Pyle' and 'Pyle Pro' with minor labelling differences.
Our GStreamer Pipeline
In this example please replace the following with your own information:
- 192.168.48.2 - IP Address of camera
- live.example.org - IceCast2 server address
- examplepassword - IceCast2 password
- speaker_mountpoint - IceCast2 mountpoint and local sink filename for the speaker audio/video feed
- slides_mountpoint - IceCast2 mountpoint for the slides
Additional pipelines are listed in the GST Cookbook.
Audio/Video
gst-launch-1.0 -e rtspsrc location=rtsp://192.168.48.2:554 latency=100 ! queue max-size-bytes=100000000 max-size-time=0 ! rtpjpegdepay ! queue max-size-bytes=100000000 max-size-time=0 ! jpegdec max-errors=-1 ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw,framerate=14/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw, width=1296, height=560 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=halfres ! queue max-size-bytes=100000000 max-size-time=0 ! jpegenc idct-method=2 ! queue max-size-bytes=100000000 max-size-time=0 ! matroskamux name=mux jackaudiosrc connect=1 client-name="GStreamer Input" ! audio/x-raw,rate=48000,channels=1 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=jackaudio ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=vorbisaudio ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0 ! filesink location=speaker_mountpoint`date +%s`.mkv sync=false halfres. ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw,width=648, height=280 ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=400 speed-level=1 ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=livestream vorbisaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! livestream. livestream. ! queue max-size-bytes=0 max-size-time=0 ! shout2send ip=live.example.org port=80 password=examplepassword mount=/speaker_mountpoint.ogv halfres. ! videoscale add-borders=true ! queue max-size-bytes=100000000 max-size-time=0 ! videoconvert ! queue max-size-bytes=100000000 max-size-time=0 ! xvimagesink sync=false jackaudio. ! queue max-size-bytes=100000000 max-size-time=0 jackaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! audioconvert ! queue max-size-bytes=100000000 max-size-time=0 ! alsasink
Slides/Screencast
gst-launch-1.0 --eos-on-shutdown ximagesrc use_damage=false ! capsfilter caps=video/x-raw,framerate=4/1,width=1280,height=800 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw, width=1056, height=660 ! queue max-size-bytes=100000000 max-size-time=0 ! videoconvert ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=400 keyframe-auto=false keyframe-force=12 keyframe-freq=12 speed-level=1 drop-frames=false ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=live.example.com port=80 password=examplepassword mount=/slides-mountpoint.ogv
Web Frontend
The live stream web client was written using MithrilJS. This interface depends on Icecast >= 2.4.0 for the JSON statistics API. Source code.
Additionally, a LibreJS-compatible fork of the KiwiIRC client was embedded onto the page. Source code.
Future work
- The in-room streaming X200's are operating at near-100% CPU utilization, limiting the final streaming framerate to 14FPS (18FPS preferred). Evaluate stream stability operating at 18FPS after building GCC from source; alternatively, port X201-series laptops to Libreboot.
- Provide Vorbis-only and/or Opus-only audio streams, CPU overhead permitting.
- Provide instructions for Icecast relays and bandwidth sharing.
- Evaluate Icecast stability under load with HTTPS and low-FPS Theora streams; bare metal hardware might be necessary.
- Integrate <video> element repositioning via jQuery Draggable
- Across-the-board security improvements; development of a Python + openFrameworks streaming app
- Consider 12-hr-delayed re-streaming for across-the-world viewer convenience
- File and bug reports for in-browser playback (esp. Mozilla Firefox-based browsers); watch for regressions and provide patches
- [TODO: list specific bugs here for tracking]
- Document different types of approaches for camera network-addressibility (single-camera use case and multi-room)
- Document development strategy of "next generation" live-streaming cameras
This page was a featured resource in May 2015.
