Xbox Live webcam on Linux

[JamesF]

A quick summary for those who haven't been following elsewhere:

I want to use an Xbox Live camera to make an all(-ish)-sky camera and an analemma camera, driven by a SheevaPlug computer, which is an ARM-based unit about the size of two fag-packets put back-to-back and draws about 5W. The one I'm testing with is running a recent Debian release.

I've run into a couple of problems, firstly that the camera driver incorrectly reports the manual exposure setting range, which appears to be because the camera reports the wrong value, and secondly that I get lots of timeouts during communication with the camera over USB.

I now have a fairly tidy patch to the Linux UVC driver that fixes these problems and the camera now works without any obvious problems. If anyone else would like to get an Xbox Live camera working with Linux, I've documented it here:

Kernel Patch for Xbox Live Camera on Linux | www.tanstaafl.co.uk

Right now, it's not really for the faint-hearted. You'll need to be comfortable using the "patch" utility and rebuilding kernel modules. I'll endeavour to write up some more instructions on that shortly, but unless/until the UVC driver maintainers feel that I've done a sufficiently good enough job that they'll incorporate the fix into the driver permanently, it's not really something for the Linux newbie.

My next step is to finish off a little command line application I've written that will grab one or more images from the camera whilst allowing the gain, exposure and other camera control values to be set, allowing the creation of time-lapse sequences from the images.

James

[JamesF]

I now have a command-line program working that will capture images at fixed intervals and allow me to feed it exposure settings and I've run it for six hours taking a frame every 30 seconds allowing me to create a time-lapse video of the view from my office window.

The bad news is that the camera doesn't appear to respond to V4L2 requests to change the gain in any way at all. It does work in SharpCap (although it behaves somewhat oddly even there) so clearly it's possible to change the gain, but I have no idea what I am doing wrong nor who to make it work. Unless I can find out then night-time use for the camera with Linux appears to be a dead end.

James

[JamesF]

The mere fact of writing the above sparked off the possibility of an idea in my head that turns out to look like there may be a solution.

Whilst it is possible to disable auto-exposures and set the current exposure time on the camera before starting to stream data from it, it looks like the gain can only be changed after starting to stream data and that any gain settings made beforehand are reset once streaming starts. I don't know if that's an artefact of the v4l2 library I'm using to pick the images apart or something specific to the camera. I'll try to work it out later. Meantime I'm still testing my new code in the hope of getting out tonight and capturing some real data.

James

[JamesF]

Well, I'm now feeling fairly positive about this camera again, though I'm in no position to test things much further given the last two days of horizontal rain. However, I appear to have the gain issue sorted, and as a result of further twiddling with the driver I have discovered that it's possible to push the exposure setting up to about five seconds. As a result I've had to tweak my driver patch again, and have put the new version up here:

Xbox Live Camera on Linux (Part 3) | www.tanstaafl.co.uk

I have also written up some instructions for rebuilding the UVC driver module once patched:

Xbox Live Camera on Linux (Part 4) | www.tanstaafl.co.uk

James

[DrRobin]

Hi James,

I am doing something similar to you only under Windows. I have noticed some strange behaviour as follows:

With the Xbox camera connected to my laptop running Windows 7, I have ver 1.2 of the driver loaded. The exposure control goes from -13 to +3 and under manual control I can get the frame rate down to 0.1fps in Sharpcap. With a dark frame I can see lots of noise, fairly bright coloured. Testing outside at night it picks up a fair amount of details.

With the camera connected to my XP machine (the one I actually want to use) it has a driver version 1.1. The exposure control goes from -13 to 0, but zero isn't any longer than about -2 (some fold back on the camera?). Anyway try as I might I can't get the same amount of readout noise under a dark frame and the frame rate never goes below 5fps.

So same camera, different PC and completely different performance.

I can offset the problem to some extent using frame integration and stacking in AstroVideo but I was hoping to get the best starting point possible.

Have you any experience of this frame rate/exposure using Windows or Linux?

[JamesF]

Not really, no. I don't understand why the Windows drivers do this range translation thing. It really makes no sense to me. With the UVC-compliant cameras at least, they are supposed to report the controls they support and the range of values for those controls to the driver. The Linux driver passes those on to the application verbatim, but in the case of the Xbox camera there was a sign problem on the exposure setting that caused trouble. Regardless, the range of exposure settings that can be sent to the camera that do something useful are from 1 to about 35,000. Why these get remapped to a completely different range of figures in Windows, and what that range actually means, is beyond me.

It might be that the application and/or driver work to ensure a fixed frame rate. I wrote my own code to take the next frame whenever it was ready and not worry about how many were available per second. I think some Windows capture applications get stressy if they can't achieve the intended frame rate however.

In some ways, knowing what I now know makes things worse because at least before (under Windows) I thought I knew what I was doing, whereas no I know that I don't

James

[DrRobin]

Hi James,

I have some experience of programming under Windows but hardly any experience of programming under Linux. I do work with embedded Linux, but it's just command line stuff and scripts for me.

I am more of a hardware engineer, but unless I went in to the software for the camera driver, like you I couldn't really tell what is going on. I a m guessing they mapped the exposure range in to an easier to understand universal range, like 1:10 or greater as is the case here, pity they weren't consistent.

It's annoying that the camera works differently on Win 7 and XP, but I don't think I am going to find an easy solution. Anyway my camera is outside in a dome so if we get some clear skies I will try running a few stacks to see if it is sensitive enough.

My ultimate aim is to run automatically exposure and upload images, using 1 frame during the day and as many frames as required through the night. I currently have a 1.9mm lens attached, which gives 140 degrees, might be too wide without a lot of frames. Unfortunately, the lens has an IR filter built in. I also have the stock lens, gives 40 degrees and a 3.6mm lens, gives 70 degrees, so might test all of them to see what works best.

[JamesF]

I wanted to use the Xbox camera as an all-sky camera, but I'm now fairly convinced that's not possible. With such a short focal length I think the signal from most stars will get drowned out by noise (unfortunately it is a fairly noisy sensor). With a 170 degree lens in I couldn't even pick up Venus.

With the stock lens I have been able to pick up stars and it may be interesting to see how stacking a number of five second exposures does, but the field of view is small, as you point out.

I'm going to revert to my original plan of LX-modding an SPC900. The longer exposures possible, lower sensor noise and increased sensitivity are a better bet, I think. Even then I'm not sure how just how well it will do with the wider angle lens, though yesyes has posted some images from his own setup.

The inspiration for writing my own capture software came from wanting to run a command line program that would just grab a few frames on a regular basis. There don't seem to be that many command-line based capture programs that work that way and allow the camera settings to be changed. FTP uploads and such I can do fairly easily with a bit of perl.

James

[DrRobin]

Hi James,

All points noted. I started with an ExView video board, but there was quite high pixel noise and the digitiser board didn't cope too well with them, the result, when I did dark frame subtraction there was still a fair bit of noise.

You could see stars on it, the Plough and Gemini were both visible when around 100 frames were stacked, with a 3.6mm lens.

There was however, no long exposure mod for the camera. There are other boards for sale which do have a long exposure mod, I might try one of those.

Sadly, I gave my SPC900 away a couple of months ago and now realise that the cost of buying a new one is quite high.

The other way to go is to buy a dmk21/31, HAD sensor and long exposure.

[DrRobin]

Hi James,

I have had a fairly extensive test of my Xbox all sky camera with a 1.9mm lens and even integrating 1000 frames with contrast stretch the stars just don't appear out of the noise.

After spending ages fiddling with settings, I finally found out that my 1.9mm is IR blocked (it wasn't supposed to be) so if I replaced it with a lens which lets IR light through I might have stood a better chance.

Now that I reckon that the Xbox camera is just not sensitive enough I went back to my Sony exView video board to have another go. I went off the video board and video capture unit as the results I was getting were fairly mixed.

Since the Xbox camera is in my dome on the summerhouse roof, I just placed the exView on the bench with a 3.6mm lens (no IR blocking) and used an integration of 250 frames, 500 frames and 1000 frames to boost the image. I used a dark subtraction on the 500 frames (not a very good one though) and a better one on the 1000 frames.

Looking at the 1000 frames, I reckon that's just too many. Looking at the screen during capture (real time) I could see the brighter stars in the real time view.

[JamesF]

Those are looking much more like it

Interesting point about the IR coatings on the lenses. I'd not though of that. None of mine say they have one, but that doesn't mean they don't, I guess.

James

[DrRobin]

Hi,

Yes getting there. Looking at the images I reckon integrating 250 to 500 frames is looking the best. These were a single image (of the specified frames) so the next thing into stack a few images to improve the detail and reduce the noise. Astrovideo can be used to automate this using a script.

I thought this camera wasn't working very well and generally has high readout noise and too many hot pixels for my liking so I ordered 1004xa from rfconcepts, it hasn't arrived yet, but may be waiting at the post office.

I wanted the 1004xa so I can do the LX mod and hopefully get away from having to integrate frames.

I didn't realise that my 1.9mm lens was IR blocked until I removed the IR filter from the Xbox camera and the image didn't change. When I now look at the base of the lens and catch reflected light it looks red so it must have an IR filter in. I am contacting the supplier to see if it supposed to have an IR and will then see about either removing it or order another.

Given my success with a 3.6mm lens and your concerns about super wide angles I might try a 2.5 as a compromise.

[DrRobin]

Hi James,

My 1004xa turned up today, so I did a quick comparison using 100 frames integrated in to one dark frame. The noise is much lower and there are hardly any hot pixels. Another triumph for RFConcepts.

I posted the results on the other thread.....

http://stargazerslounge.com/diy-astronomer/160629-diy-all-sky-cam-8.html#post2273826

Regarding my IR lens, it isn't IR blocked, but is multi-coated to reduce reflection and the coating also attenuated IR. Seems you have to order a day/night or day & IR lens, not as easy as I was thinking it would be.

Stock Optics sell a 2.6-6mm varifocal day/night at F1.6 (quite fast) and this will give me a number of options to get the focal length versus camera sensitivity right. They also do fixed focal lenses but the smallest they carry in day/night is 4mm.

Now all I need is a break in the weather to get my dome down from the shed roof and replace the Xbox camera with the new 1004xa.