UPDATE 10/01/2007 found out that if the input from the cam is 6FPS and the output to SWF via FFSERVER is 4FPS - there's no lag in camera controls. The scripts below have been changed to pull the MJPEG images from the camera at 6FPS and keep the flash SWF output as 4FPS.

The controllable IP Network cameras form Canon are great. They helped start an odd yet somewhat accepted precedent on the web facilitating user-controlled cameras through the web.

Naturally all the IP Network cameras' UI Module are written in Java - and have remained unchanged for a some time. This is the point of contention for this write-up and all the digging I have done to find an alternative to this Java UI module.

The crux of these cameras - in a high-availability scenario - is that they offer no proxy of sorts. There are several products that facilitate the replication of a single stream from said IP cameras and duplicate them as part of a streaming server. Again - all of these offerings only support re-use of the java UI Module from either Canon of Sony - a kludge of software and very "closed" in the sense that it's not very reusable.

With the crux being known, I set out to find an alternative route to interface with the cameras. I must disclaim that this is a proof of concept - yet is not far from being a production-quality piece (similar to Perl-ish kludge-hackery fashion).

My first milestone was to be able to stream the live feed from a Canon VB-C10R Network Camera into Flash Video without relying on the custom Canon UI module so affectionately referred to as "LiveView".

My second milestone was to be able to "proxy" the live stream as pulled from the camera. This means we'll only be pulling the stream from the camera once and the proxy server handling all the external requests - naturally a large number of requests.

The first milestone was to figure out how to pull the live video stream from the Canon VB-C10R. To begin - I "sniffed" incoming requests from the LiveApplet example using tools such as tcpdump, ngrep, and snort.

More background information: The Canon VB-C10R (and the VB-C family) offer two methods of accessing the live stream from the camera, via a specialized port, or via HTTP. I didn't want to setout to try and use the HTTP port because it is actually a slower protocol because it adds a HTTP header for every request. The only advantage of the HTTP protocol to access the stream is so that it's more accessible over popular firewall configurations. In this case - it's a direct unprotected link to the camera (private network). This also makes pulling the images as fast as possible. The proprietary port in question for the live stream is 65310. The format for the stream is simple and widely accepted MJPEG (Motion JPEG) video format.

In order to figure out how to communicate in this proprietary format, I used the following snort command (snort worked the best out of all the sniffers to figure out the communication method...) :

# snort -dvq host 10.0.0.1

With the following return set (10.0.0.1 = the server 10.0.0.20 = the camera):

Do not forget that 10.0.0.1 = the server 10.0.0.20 = the camera.

I dissected the protocol and highlighted the requests and acknowledgments in red. The outline of the pulling the stream is a simple process (pseudo code):

• open socket to camera server via TCP
• send MODE 0\0AVERSION\0ASTART\0A
• wait for response: WebView/Livescope Video Capture...
• first send \0A\0A\0A\0A
• wait for header response
• loop send \0A\0A\0A\0A
• loop wait for response for next frame in MJPEG

Before the MJPEG "header" is send over this protocol - there is a send "\0A\0A\0A\0A" with a response of "100" - which is meaningless as far as I can tell. I still write it to my MJPEG stream in the end without fail.

I wrote the following PHP script to pull the MJPEG stream directly from the CB-V10 camera on it's native 65310 streaming port:

<?php
// Set time limit to indefinite execution
set_time_limit(0);
// create TCP socket
$sock = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
// connect to the camera's native stream protocol IP:Port
socket_connect($sock,"10.0.0.20", 65310);
// handshake with the camera
socket_write($sock,"START\r\n");
// ignore the handshake response
socket_read($sock, 12000);
// daemonize this script
while (true) {
    // simulate 4 escape sequences "\0A" with "\r\n" - or chr(0)
    socket_write($sock,"\r\n\r\n\r\n\r\n");
    // read the image from the stream (max 80k)
    print socket_read($sock, 80000);
    // limit to ~6 frames per second
    usleep(166666);
}
?>

This writes the stream as pulled directly from the VB-C10 through its native video streaming port and outputs it directly to STDOUT. PHP was chosen because of simplicity - and is simple enough to port to a more robust C application. That's on the list of things todo. I would not fear using this in a live production environment so long as there were protective measures in place to ensure this would be daemonized indefinitely.

To output the VB-C10 video stream directly to STDOUT, run this PHP script through PHP-CLI via:

# php -f pull_stream.php

This will flood your terminal if you do not direct STDOUT, but is good for testing to make sure all is well. Don't forget to reset your terminal if it has been "flashed" - i.e. all the characters are now "funky".

As per the code - you can change the socket_read() function to attempt to pull a specific amount of data as per the configuration of your camera. Depending on the quality of your images as setup in the camera settings - each frame pulled via this script will be that size. If you get corrupted images in your stream, increase this number. If you want a better framerate than 4 frames per second, as I have chosen, minimize the usleep() function as per how many micro-seconds between frames.

I must disclaim that this PHP script is merely a proof-of-concept and will not pull the exact FPS desired, nor will it be a completely reliable proxy to pull the camera's stream. Not only that, it cannot gracefully handle disconnects of the camera or the like.

We now have a way to capture the live MJPEG stream from the camera - now what to do with it...? My second milestone was to take the MJPEG stream and convert into native flash streaming video format. FFMPEG and FFSERVER opensource programs are absolutely amazing pieces of work. Not only that, but they manifest my second milestone perfectly.

More detail on the milestone: I want to dynamically take the MJPEG stream and convert it to FLV on the fly. I also want to make the product FLV something that can be streamed from countless clients. FFMPEG does the on the fly conversion and FFSERVER provides streaming functionality.

In sum - I have tweaked and configured FFMPEG and FFSERVER to produce a replicated proxy server that takes the VB-C10 MJPEG stream and duplicates it in FLV real-time to be served to a much wider audience.

FFMPEG and FFSERVER must be configure to work in concert and here are the following commands and configuration I gave to produce the final product:

FFMPEG

# php -f pull_stream.php | ffmpeg -an -f mjpeg -maxrate 500 \
-r 6 -i - -r 6 http://localhost:8090/feed1.ffm

NOTES: I run the "daemonized" pull_stream.php script, pipe STDOUT directly to ffmpeg via the "... -i - ..." arguments. The -i denotes an input file and - for a filename denotes STDIN . The -f denotes forcing of the MJPEG format for the input stream seeing as it will not be obvious to ffmpeg coming from STDIN. The -maxrate 500 limits the bitrate for the input file to 500kb/s. The -r 4 limits both the input and output frame-rate to 4fps (before the -i - affects the input stream and after affects the output stream). Then the output is the last argument is a URI which points to the FFSERVER's configured default stream feed running on the same server.

FFSERVER - /etc/ffserver.conf snippet

Port 8090
# bind to all IPs aliased or not
BindAddress 0.0.0.0
# max number of simultaneous clients
MaxClients 1000
# max bandwidth per-client (kb/s)
MaxBandwidth 10000
<Feed feed1.ffm>
	File /tmp/feed1.ffm
	FileMaxSize 5M
</Feed>
# SWF output - great for testing
<Stream test.swf>
	# the source feed
	Feed feed1.ffm
	# the output stream format - SWF = flash
	Format swf
	# this must match the ffmpeg -r argument
	VideoFrameRate 4
	# generally leave this is a large number
	VideoBufferSize 80000
	# another quality tweak
	VideoBitRate 500
	# quality ranges - 1-31 (1 = best, 31 = worst)
	VideoQMin 1
	VideoQMax 5
	VideoSize 320x240
	# this sets how many seconds in past to start
	PreRoll 0
	# wecams don't have audio
	NoAudio
</Stream>
# FLV output - good for streaming
<Stream test.flv>
	# the source feed
	Feed feed1.ffm
	# the output stream format - FLV = FLash Video
	Format flv
	VideoCodev flv
	# this must match the ffmpeg -r argument
	VideoFrameRate 4
	# generally leave this is a large number
	VideoBufferSize 80000
	# another quality tweak
	VideoBitRate 500
	# quality ranges - 1-31 (1 = best, 31 = worst)
	VideoQMin 1
	VideoQMax 5
	VideoSize 320x240
	# this sets how many seconds in past to start
	PreRoll 0
	# wecams dont have audio
	NoAudio
</Stream>
<Stream stat.html>
	Format status
</Stream>
<Redirect index.html>
	# credits!
	URL http://ffmpeg.sourceforge.net/
</Redirect>

This is a limited yet functional ffserver.conf file.

To prove Milestone 1 and Milestone 2 working in concert and to get a stream working, ffserver must be running before running the ffmpeg command.

The next two projects will be:

• Encapsulating all this functionality to make it high-availability.
• Reverse engineering the VB-C10/C50 camera controls via a Flash Remoting Application. Started this work here: http://www.overset.com/2007/01/24/canon-network-camera-control-protocol-hack-vb-c50i-control-port-communication/

As always -- More to come...