Capture software for Linux

[JamesF]

That's just very poor driver coding by whoever wrote the driver.

I've been writing camera software for PC's for years now, and nearly every single time their are problems with cameras (quite often) it's always down to very poor driver coding, at least when it comes to windows drivers, manufactures are very poor at writing driver code and have no pride in their work/coding

The Linux SPC900 driver (well, a shared driver for a whole set of Philips-based cameras) is something of a nightmare because the person who originally wrote it provided some of it as a binary because he was under NDA to Philips.  Someone else then reverse-engineered it and there was a bit of a row, people got hacked off, the world moved on and no-one other than astronomers then really had an interest in it.  It's been left in something of a limbo state for perhaps ten years whilst the kernel has been re-worked, internal interfaces have changed and in fact a whole new video device interface was created.  Hence my initial surprise that it worked at all.

So, it may be not be specifically poor coding, but that it's been unsupported for so long and isn't behaving in the way that current kernels require.

I'd actually be prepared to take it on and update it, but for the fact that kernel drivers (that are intended to go into the distributed source) now appear to have to go through quite a rigorous code review and learning all the whys and wherefores is something I'd really struggle to find time for at the moment (or if I could, I'd spend it doing something else).  Fifteen years ago or more I found some bugs in the driver implementation for a NIC I wanted to use in a Linux box, fixed them, emailed a patch off to Alan Cox (who was "owner" of all things network-related at the time) and within a day received a reply along the lines of "Oh, yeah, I'll put that in for the next release".  Sadly it's not quite like that any more

James

[Martinus]

Just wanted to say thanks for taking the time to work on this, James. I'll be watching your progress with much interest.

[JamesF]

Just wanted to say thanks for taking the time to work on this, James. I'll be watching your progress with much interest.

Thank you.  I feel as if I'm making significant progress now.  I've spent a little time today doing "tidying up" work -- fixing hacks in the code that I put in just to see if it work get things working and suchlike, but when it is next cloudy (tomorrow night, probably I shall get on to integrating the ASI interface.  It feels like ages since I had this much fun writing code

James

[JamesF]

Been looking at the ASI library this whilst I wait for the children to make dinner this evening (under supervision

The OpenCV stuff looks like it might be a bit of a red herring.  The demo appears to use it just for processing and displaying the grabbed image, but the actual library interface looks really simple compared to the Linux V4L2 interface.  The forecast for this evening is pretty good up until midnight however, so I don't think I'm going to get too much coding done this evening

James

[NickK]

OpenCV is about image processing with SSE based algorithms. The GPGPU portion is dire.. although nVidia have added lots of CUDA but OpenCL is lacking..

[JamesF]

OpenCV is about image processing with SSE based algorithms. The GPGPU portion is dire.. although nVidia have added lots of CUDA but OpenCL is lacking..

Yes, but in this case the issue was more that the ASI SDK examples appear to use openCV and the WIndows DLL does in fact appear to be rolled up with openCV somehow.  I was concerned that perhaps the camera control library was somehow using openCV for processing the data from the camera and may rely on it for delivering data to the client application, but in fact that doesn't appear to be the case at all for Linux and the user interface appears to be very simple.

There are unfortunately other niggles with the SDK.  For instance the library appears to be in C, but the header file will only compile in C++.  I've fixed that one fairly trivially.  More awkward to fix is the apparent inability of the SDK to support more than one camera at any one time.

That's a bridge to cross another day however.  Given the relative simplicity of the interface I just want to crack on and get it working for my ASI120MC.  If I can do  that this week (and there's no real reason to believe that shouldn't be possible as long as I have enough spare time) I could potentially be imaging next weekend's triple shadow transit of Jupiter with my own imaging application which would be exceptionally pleasing.

James

[JamesF]

Actually there are one or two tricky problems to solve, though they're more UI issues than application logic ones.

Some cameras appear to allow for arbitrary ROIs to be set.  The ASI120 allows ROIs, but only a fixed set of sizes.  Other cameras allow different resolutions to be used, but still show the full field of view (a bit like binning, I guess).  Combining all of those into a tidy yet meaningful presentation in the UI is going to need some thought.

The camera also support 2x binning, but only at a single frame size.  That is, you can't set an ROI and use binning on that.  Which means the binning option will have to interact with frame size selection.  I'd not considered the need for that before.  Hardly the end of the world though.

James

[JamesF]

Been working on ASI support since it turned cloudy this evening.

It's just occurred to me however that I also need support for a histogram before I can really use the application for imaging.  I shall see if I can get that done as well.

James

[NickK]

I'd do a simple pipeline that's extendable with a common interface between each processing function/class so it becomes plug and play.. 

1. Capture & store raw

2. Serial algorithm processing (ie mean/min/max and histo)

3. Parallel algorithm processing (ie do SSE/OpenCL etc)

4. Display processed / store processed

[Cath]

And definitely make good use of any multi-core CPU's. A bit of a must when doing any sort of real time image processing etc.

I also nearly always use openGL for image display, it saves a lot of CPU cycles which can be used where they are needed.

[NickK]

And definitely make good use of any multi-core CPU's. A bit of a must when doing any sort of real time image processing etc.

I also nearly always use openGL for image display, it saves a lot of CPU cycles which can be used where they are needed.

 With OpenCL you could simply render the ouput texture direct to OpenGL that makes it even faster (all on GPU processing & display)

[JamesF]

It might be useful to look at that one day, but for the purposes of getting the application working in the first place I'm not going to worry about it right now.  I hand the bitmap to Qt and let it decide how best to deal with it and that's been ok thus far.  If necessary it's an area I can look at optimising in the future.

My main concern regarding performance right now is if I should have a separate thread for writing the data to disk.  I can see some justification for that, but again it's not been a major performance issue so far (and I have been doing some testing on an old Aspire One netbook), so  I think time is better spent on getting a usable application rather than tuning something that may not actually need tuning.

James

[cgarry]

I have to agree with you James, early optimisation is a bad idea.  I say get your design working first and then only optimise areas when the profiling tools say that it is worth doing.

Cheers,

Chris

[Cath]

Yes I agree too. If you've writing code for an area you've not done before then for sure leave your optimization till later.

The main priority is to get those image frames off your camera and into memory without dropping any. Getting them saved to disk is another high priority. Displaying the frames on screen is a low priority.

[JamesF]

Laydeez an gennelmun! I present to you a screenshot of data captured from my ASI120MC

This is using the 2.1mm lens that was originally supplied with it.

I still have some way to go with support for this camera, but given that the only documentation is a few comments in the header file for ZWO's library and that I've probably only put five or six hours into integrating it with my existing application I'm feeling rather pleased with myself right now.  I'm actually surprised how relatively easy it's been given that I didn't really look at the ZW API in detail before I started coding the application.  Hopefully that's a reasonable indication that I have pretty much the right design for the interface.

Of the bits still to do, I particularly need to sort out changing the ROI and look into the exposure times as the limits for that control returned by the library look quite odd.  I always intended to have a separate pop-up window that exposed all the available camera controls rather than just those that are probably immediately useful for imaging and I think I might need to bring that forward as there's a very blue cast to the image that can probably be fixed using the blue and red balance controls.

I also need to think about how I'm going to hook the binning into resolution changes, but that's lower priority.

Oh, and there's clearly a bug there the preview pane doesn't get resized when I rescale the image to be smaller so the scroll bars are left in place.  I shall have to fix that (the image from the camera is showing scaled to 50% in this capture).

James

[cgarry]

Nice work sir!

[Gina]

Very nice   I like the UI

[JamesF]

Very nice   I like the UI

Thank you.  I have to admit to pinching some ideas from FireCapture for the UI, but imitation is the sincerest form of flattery .  My plan is eventually to allow the controls to be either at the top or the side.  I think with a netbook it might fit better on the screen if the controls are alongside rather than above the preview window.

James

[JamesF]

A quick tinker over lunchtime and I have the window resize on rescaling problem fixed.

I think I've also tracked down the problem with changing the ROI though it's going to take a little longer to fix.  I discovered that some V4L2 cameras need partially restarting when the resolution is changed and added functionality to do that.  It appears that's not enough for the ASI camera though.  Looks like I need to completely disconnect from the camera and reinitialise it from scratch otherwise it dumps core in the library's debayering code

I'm not entirely sure that all the frames are being returned correctly either.  There's no demo code for anything other than 8-bit monochrome and next to no documentation, so I may be doing something wrong.  Further investigation required, but it looks like it could be related to debayering too.

James

[JamesF]

I now have an inkling that the frame returned by the SDK library is a plane of red, then green, then blue whereas everywhere else I'm using 8 bits red, 8 bits green and 8 bits blue, repeated over and over again.  The image I'm getting at the moment looks like this:

I'll reorder the data and see if that helps.

James

[JamesF]

Well, I'm still no further forward

I'm actually having my doubts about the reliability of the ASI library at the moment.  I built their demo program, which basically just grabs a frame and throws it on the screen repeatedly.  At times it "jumps" and I see an image that has, say, the left-hand  two thirds apparently not debayered, whilst the right hand third is debayered, but is actually the part of the image that should be on the left-hand side of the image.  It looks a bit as though the buffers for the debayered and raw images are getting overlapped for some reason, or the data for the image isn't being copied from the correct place.

James

[JamesF]

Here's an example of the problem, this time from my capture program:

I think the fact that this happens in both my capture code and in their demo program which share no common code above the C/C++ and USB libraries strongly suggests to me that the likely cause is their library.

James

[JamesF]

Oh, also the fact that I get part of the raw colour data.  I don't at any point have direct access to the raw data.

James

[Gina]

I think you're right James.

[Cath]

Welcome to the nightmarish world of camera drivers and their libraries lol

I'm sure you'll work out what the problem is eventually, it does sometimes take a while.