How to Upload Images to Sensr.net

Sensr.net’s server architecture is organized into two main parts.  There are “Front-End” webservers that handle browser and API requests and a large number of “Back-End” Camera processors that are dedicated to receiving images through a variety of mechanisms including

  • FTP Image Upload,
  • HTTP Image Upload, and
  • HTTP MJPEG Stream Push.

If you’re a developer, it may help you to think of Sensr.net as a service that allocates inexpensive FTP accounts optimized for image uploading, with a front-end for viewing, cataloging and sharing these images.

Sensr.net Architecture

The front-end and back-end implement different authorization mechanisms.  The front-end authenticates a Sensr User with their email and a password.  The back-ends have separate logins for each Camera.  The front-end can allocate and view logins on the back-ends, but not vice-versa.

Every time you create a new camera using the web interface, Sensr.net allocates a new login on a back-end processor.  A Sensr.net user may have many cameras, each having their own upload account.  These accounts are not very powerful: they only allow uploading images to a back-end host through FTP and HTTP.  (In other words, you can’t login to the front-end Sensr website with them.)  This is a good thing:  if you want to embed these logins in code on cameras and servers you don’t need to worry about someone being able to do something malicious with them.

The back-end FTP login is what you see on the “Create New Camera” page.  Three pieces of information identify a back-end login.  These are the HOST name (or FTP Server), the USERNAME and the PASSWORD.  The back-end login credentials for a camera can also be observed by opening up the “Settings” control of your camera.

Using the API, you can retrieve the back-end login creditials by retrieving the camera object.  If you are the owner of the camera, you will find these in the JSON representation of your camera object.  Here is an example:

{ "name":"Front-Door Camera",
    ...,
    "host":"f7.sensr.net",
    "username":"cam28",
    "password":"l4hqpy0ywm",
    ... }

Uploading using FTP

FTP Upload is the method Sensr.net recommends for most of the Netcams currently available.  When configuring a camera, you set the HOST, USERNAME and PASSWORD to the credentials obtained from the Sensr website.  For each frame, a netcam creates a JPG file and issues an FTP PUT command to transfer it to your Sensr.net upload host.

Commercially available Netcams vary widely in their capabilities, but there are a few guidelines that can help.  These guidelines apply to FTP upload applications you may be writing as a developer too.

  • You must use PASV (passive) FTP mode.
  • Your upload file should be written to the “/” (root) directory if possible.  Do not give the upload directory a name like “/MyCamera/”.
  • If possible, give each uploaded file a unique name.  Appending a timestamp included in the uploaded file name is great.  Use something like “/SensrCam2010102011073001.jpg” for your uploaded image names if possible.  (An increment number can suffice instead of a timestamp.)  This helps avoid race conditions on the server.
  • Avoid FTP commands that rename or move files around on the server. (Some cameras like the Panasonic BL-C140A – when in “overwrite” mode – attempt to rename files from “imagenew.jpg” to “imageold.jpg” in an effort to maintain a buffered image.  This technique causes errors.)
  • Re-use the FTP connection for successive image uploads if possible.  The camera-upload program EvoCam has a “Stay Connected” option, for example.

FTP: A Linux Example

You can experiment with FTP upload by building your own FTP-based “Netcam” on a Linux machine by using an external webcam, the program ffmpeg and a our shell script. After allocating a new camera on Sensr.net, you should gather the HOST, USERNAME and PASSWORD parameters and hard-code them into your copy of the script.

The script uses dov4l to set image capture parameters for the webcam.  The script then enters a loop where it uses ffmpeg to grab several frames from the camera and save them as JPG files.  One of these frames is selected for upload by FTP, and then the loop repeats.  You can interrupt the script by hitting `^C`.

Uploading using HTTP

HTTP Uploading avoids many problems that occur with the FTP Upload method.  First and foremost: HTTP gets through firewalls and proxies better than FTP.  Some cameras, like the Axis M1011 Network Camera support HTTP upload directly.  If you’re a programmer creating an App on a mobile device like an iPhone or Android, there are simple ways to upload JPGs using HTTP in the appropriate SDK. HTTP Upload is a very simple protocol.  There is a single connection and the request contains all of the upload information.  FTP in contrast, opens and closes TCP connections, uses multiple ports, offers passive and active variants, and has pitfalls related to file naming.

Sensr.net’s HTTP Upload protocol implements Basic Authentication with the same USERNAME and PASSWORD credentials as the FTP Upload option.  The single-photo upload URL for your HOST, USERNAME, PASSWORD credentials shown below.  Notice how the USERNAME parameter occurs twice: it is the name of the account and also identifies the camera.

http://${USERNAME}:${PASSWORD}@${HOST}/photo/${USERNAME}

Sensr.net’s HTTP Basic Authentication requires the “Authorization:” header in the request.  A challenge will be returned to the client if the header is not present and it is not valid.  (See http://en.wikipedia.org/wiki/Basic_access_authentication for an overview of this method.)

HTTP Request in Detail

A full HTTP request with authentication information is shown below.  The body of the request is the binary data of the uploaded JPEG image.  JPEG files begin with the bytes 0xFF, 0xD8 and end with the bytes 0xFF, 0xD9.

POST /photo/cam28 HTTP/1.0
User-Agent: Wget/1.11.4
Accept: */*
Authorization: Basic Y2FtMTE6d2x6bzNwaGgxZQ==
Host: localhost:3000
Connection: Keep-Alive
Content-Type: image/jpeg
Content-Length: 484881

<FF><D8><FF><E0>^@^PJFIF^@^A^A^@^@^A^@^A^@^@
...
...
^NNh<A3><FF><D9>

 

HTTP Status Codes

An HTTP Photo request will return one of the following status codes.

  • 200 – OK.  Upload stucceeded.
  • 404 – Not found.  Camera name does not exist.
  • 401 – Unauthorized.  Password does not match camera.

 

HTTP: A Linux Example

You can upload images from your computer to a Sensr.net camera using command line tools curl and wget.  The examples below show how to upload a single image called “Rocketship.jpg” to cam28. The curl program accepts the USERNAME and PASSWORD as part of the upload URL.  By placing the authorization parameters in the URL, curl includes the Basic Authorization headers in the request.   The other parameters that we need to give are the upload filename and the Content-Type header.

curl -T Rocketship.jpg -H "Content-Type: image/jpeg" http://cam28:lfhapy07wm@f7.sensr.net/photo/cam28

The wget program requires that the USERNAME and PASSWORD are given as separate command line arguments.   Wget differs from curl in that it does not present the authorization headers until it receives a challenge from the server.  We can override this behavior by including the “–auth-no-challenge” flag so that wget puts the “Authorization:” header in the request on the first go-around.

wget --auth-no-challenge --user=cam28 --password=l4hqpy0ywm  --post-file=Rocketship.jpg --header "Content-Type: image/jpeg" http://f7.sensr.net/photo/cam28

Streaming Upload using HTTP and MJPEG

Motion-JPEG (MJPEG) is a simple video format that consists of a stream of JPEG images, one after another.  MJPEG does not include time-code information.  Sensr accepts MJPEG Upload streams over HTTP.   Streaming uploads require HTTP Basic Authorization just the same as Sensr.net’s Photo uploads:

http://${USERNAME}:${PASSWORD}@${HOST}/upload/${USERNAME}

MJPEG upload is very similar to the HTTP Photo upload method described earlier, but instead of the request body consisting of a single JPEG image, the request body contains a sequence of JPEG frames.  As Sensr reads the request body through an Internet Socket, it splits the body into frames as each JPEG image is detected.  The Sensr timestamp of the arriving image is the time at which Sensr processes that portion of the request. JPEG files begin with the bytes 0xFF, 0xD8 and end with the bytes 0xFF, xD9.  An MJPEG streaming upload request looks something like the following.

POST /upload/cam28 HTTP/1.0
User-Agent: Wget/1.11.4
Accept: */*
Authorization: Basic Y2FtMTE6d2x6bzNwaGgxZQ==

<FF><D8><FF><E0>^@^PJFIF^@^A^A^@^@^A^@^A^@^@
...
...
^NNh<A3><FF><D9>
<FF><D8><FF><E0>^@^PJFIF^@^B^B^@^@^B^@^B^@^@
...
...
^NNh<A3><FF><D9>
<FF><D8><FF><E0>^@^PJFIF^@^C^C^@^@^C^@^C^@^@
...
...
^NNh<A3><FF><D9>

Streams uploaded this way can keep connections open for a very long time.

MJPEG: a Linux Example

You can experiment with HTTP MJPEG streaming uploads on your Linux computer with the ever-versatile ffmpeg and an external webcam.  The ffmpeg program knows about MJPEG URLs, making uploading trivial.  Use the script below and customize it with your own USERNAME and PASSWORD to upload streams to Sensr.

Conclusion

Sensr.net is a versatile upload service for JPEG images from all kinds of image sources.  The examples here show just how simple image upload clients can be.  If you are a developer, perhaps some of the resources shown here will inspire you use Sensr.net as an image repository for your own projects.