Capture software for Linux

[JamesF]

Just when it was going so well, someone throws a great big spanner in the works

I discovered that to write raw non-debayered avi files I can't use the ffmpeg library I have been working with (v1.2.x) because there's no support for that format.  The earliest release I can find support in is v2.1.x.  Having built v2.1.1 and installed it I've discovered that my existing code won't compile against it

James

[JamesF]

Haha!  I found a web page giving details of how to translate the old style stuff into the new.  Back on track again!  Not much chance of any more work being done tonight if the sky stays clear though.

James

[JamesF]

Well, I just can't help myself

The atmosphere was pretty soggy so I packed up an hour or so ago.  Whilst AS!2 chunters through my capture runs I did a little more coding and now have imges from a TIS colour camera displaying on-screen as an non-debayered greyscale image and being written to a file which I believe is raw BGGR8 data.  Unfortunately all the tools installed from my Mint 15 distribution are built against a much earlier release of ffmpeg, so I can't actually test playback of the saved file yet.  I might have to push it over to Win7 and see what AS!2 makes of it.

I should probably also revisit the ASI colour cameras now as at least some allow non-debayered capture.  It's added to my list of things to do for this release

I've been thinking about debayered colour images from cameras that support non-debayered output.  I think if the camera will do RGB and the user wants debayered data I'll just push the camera into RGB mode.  If the user wants a debayered image in the preview but to save the raw format then I'll debayer on the fly using a function from a separate library.  There seem to be many different algorithms for debayering, some of which are slow and computationally expensive but produce better results.  A library would allow me to also use the code a debayering application that could be used to try various different algorithms and then use the one producing the best output.

James

[JamesF]

Oooh, some days...

I've spent this evening working on picking apart the bayer matrix, which is a little hairy even doing just the simple "nearest neighbour" algorithm.  The camera driver announces it as BGGR, which I assume means that the pixels are arranged thus:

BGBGBG

GRGRGR

BGBGBG

GRGRGR

But that clearly wasn't working even though I'd tested it as functioning correctly.  So I added an RGGB handler, but that didn't work either.  Searching the web in desperation it appears the camera lies about its colour mask and that it may in fact be GBRG.  I guess I'm writing yet another decoder then.  Never mind, I'd have had to have done it at some point.

I was hoping that I might be able to handle the layout just from the data provided by the drivers and not have to ask the user for the mask arrangement, but it looks like that isn't going to work

James

[JamesF]

And just before I went to bed last night I managed to get everything working.  With GBRG and GRBG handlers written and some config options to switch between them I found that the camera is GBRG and I have a nice colour image.  Big step forward

This actually makes my "todo" list for the next release longer, because it means I now have little reason not to add handling for the ASI120MC 8-bit raw mode.  I might leave the 16-bit raw mode for another release as I don't think much of the additional data is actually very useful.

James

[JamesF]

Well, I've now added a bilinear debayer method and support for the 8-bit raw mode of the ASI120MC (so hopefully all the ZWO colour cameras).  That pretty much leaves support for the QHY5 as the only major thing I wanted to do for the next release.

The bad news is that whilst I was doing the additional support for the ASI120 I had one of those moments where you think "If I'd known then what I do now, I wouldn't have done things this way"  A rather neater way to handle setting the various camera controls and features has occurred to me.  I think that can wait for a while though.  I was half-expecting that it would happen anyway as the greater complexity of implementing support for different features of such different ranges of camera functionality became more obvious.

James

[JamesF]

I'm making progress with the QHY5 support now.  It's a hideous mess though  I can't find any documentation for the QHY5 firmware despite the fact that a number of open source applications using the QHY5 exist and therefore the documentation must presumably be around somewhere.  The person who wrote the firmware seems to have disappeared off the face of the planet as well, so I'm pulling things together by comparing the various applications and trying to extract information from them which is fine apart from where they disagree

Still, all I have to do now is to actually grab the image from the camera and pick it up to return to the calling application, which kind of sounds like it ought to be pretty much everything the code does, but in reality I think (hope it's going to work out as about twenty lines of code from a total of around a thousand.

The QHY family of cameras seem to have interfaces that are almost, but not entirely completely different which is making for fun in terms of structuring the code.  I really don't want a QHY5 interface, another for the QHY5v, another for the QHY5-II etc. etc. if I can avoid it.  That's meant I've actually ended up structuring the code closer to the way I should have done it to start with, so perhaps it's not as bad as it could have been.  I can see a re-write of the main camera library interface coming fairly shortly though, which should lead to a simpler, cleaner and more efficient system.  At least until I find the next camera that doesn't quite fit

James

[JamesF]

Hah!  I knew I wasn't too far off  There are still plenty of niggles to sort, but I have now captured a raw greyscale avi from the QHY5 at 1280x1024 using the CCTV lens from my ASI120 (which doesn't have enough inwards focus nor a large enough image disc):

Seems about the ideal time to call it a night

James

[JamesF]

I've been having a bit of a break from "hobby" coding since Christmas.  Work has been fairly demanding and doing a full day's coding only to start again on some more in the evening just didn't have any appeal.  And I wanted to spend some time sorting my C9.25 so I could do some imaging of Jupiter.

Anyhow, I'm now back in the chair and finishing off the remaining items before making another release.  This evening I think I've finished support for SER format output though it isn't properly tested yet.  The remaining things could probably wait for the next release, so I'm going to try to get something out before the end of the month.

One thing I was hoping to have in but probably won't be is support for the ASI034MC.  Apparently that requires an SDK update from ZWO which isn't get available.  Not that I have one to test, but I think if the SDK were available it would probably work.

James

[JamesF]

Lots done this evening.  Finished tidying up some small niggles with the SER support, added additional text to make some of the configuration options clearer and finished all the items on my "todo" list for the next release other than a couple that I've pushed back into the next one.

I've also set up a website on which I'll put together some manual pages and so on, but I think that can perhaps wait until after I've actually got the next release packaged up and ready to go.

James

[Eddy_J1]

I'm really interested in this James as I'd love to try the combo of my ASI120mc with a raspberry pi which obviously has to run some sort of Linux, and the aim would to be to make it a low power 24/7 all sky cam. Any ideas if your program will run on a distro compatible with the pi?

Obviously high frame rate wouldn't work on the pi but for an all sky cam that isn't an issue.

Cheers

[JamesF]

I'm really interested in this James as I'd love to try the combo of my ASI120mc with a raspberry pi which obviously has to run some sort of Linux, and the aim would to be to make it a low power 24/7 all sky cam. Any ideas if your program will run on a distro compatible with the pi?

Obviously high frame rate wouldn't work on the pi but for an all sky cam that isn't an issue.

Cheers

There is one outstanding issue with the current RPi version of the ASI SDK that prevents the application linking, but I believe that has been corrected and should work in the next release, if there's not a beta test version out there already.

This is in fact one of the purposes to which I want to put the software.  Not necessarily this capture program though I see no reason it wouldn't work.  However the libraries on which it is built would be entirely suitable for writing, say, a CLI-based application for capture that wouldn't have all the GUI overheads.  One of the other things I want to do at some point is to make an "analemma cam", to take a picture facing south every day at solar noon for a year.  I envisage a similar approach for that.  Once I get a bit further with the GUI capture application then I'll start thinking about that.  Or perhaps someone else will pick up the baton on that one, who knows?

James

[Dave_D]

I've been having a bit of a break from "hobby" coding since Christmas.  Work has been fairly demanding and doing a full day's coding only to start again on some more in the evening just didn't have any appeal

James

Been there, done that, know exactly where you're coming from.

Except I haven't written a line of code for over 5 years thanks to burning myself out to line the pockets of corporate morons who didn't appreciate a bit (ok, a lot) of quality code...

[Eddy_J1]

Thanks for the reply James, I will see if i can get it running on some sort of fedora for the pi, then report back for you. If it works you'll be a hero As its something I've wanted to do ever since I got my ASI120, then I just need to somehow send the stream to my TV or something like uploading a frame every 30 seconds to a webpage.

[JamesF]

Thanks for the reply James, I will see if i can get it running on some sort of fedora for the pi, then report back for you. If it works you'll be a hero As its something I've wanted to do ever since I got my ASI120, then I just need to somehow send the stream to my TV or something like uploading a frame every 30 seconds to a webpage.

I didn't know there was a Fedora distribution for the Pi.  I might investigate that at some point.  I've been using the Debian-based one so far.  I've just been hunting around and found this post: http://zwoug.org/viewtopic.php?f=21&t=1441&p=2028#p2028 which has a reference to an RPi version of the SDK that works.

James

[JamesF]

The next release has been delayed slightly as I've been trying it out against the planned next release of the SDK and there appears to be a problem somewhere.  If it looks like it's my application then I'll fix it, otherwise I'll get the release sorted anyhow and do a further one if necessary for the new SDK.

James

[Eddy_J1]

Hi James, i'm planning on trying it on the pi tonight, I had a fear of doing it during the week incase it wasn't working and I ended up spending all night trying to make it work hehe, I'll try Pidora and if that doesn't work i'll give Raspbian a go, is that YAACA thing basically the same as what your making? Either way i'm happy to try yours on the pi, providing I can get it installed

[JamesF]

YAACA is similar but different, I think.

My capture application should be standalone, but I intend that it should provide a set of tools upon which other applications can be built.  YAACA appears to be more of a single-purpose thing targeted at a small set of cameras.  Or was, last time I looked.  I do hope I haven't maligned the author there.

James

[JamesF]

Oh, there's a new "testing" ASI SDK available here: http://zwoptical.com/software/asilinuxsdk.tar as of today.  I don't think Sam will mind me mentioning that as he's posted the location to a public mailing list.  I've not even tried them yet, though I did have to make some small tweaks to get the previous testing version to compile with my code, or rather, to get my code to compile with it.

I really do want to get the next release out asap though.  There are so many fixes and lots of support for new stuff that it will make quite a difference I think.

James

[Eddy_J1]

Well I gave it a go, and well as i'm not a linux user I didn't get too far haha, i managed to install a couple of things(from cd the readme) but no idea how to install the rPi SDK that you've linked.

I guess i'm maybe trying to jump in too early as I've no idea how to install all this stuff :S

[JamesF]

I'll see what I can do with the next release to make things easier.  Perhaps even make up an RPi binary release.  We've definitely found a few issues with that new SDK so once I'm done writing documentation I'll make a new release using the existing SDK version.

James

[Eddy_J1]

Wow thanks for the rapid reply I'm fine with windows but when it comes to linux I'm a bit rusty to say the least

Look forward to the release, even if you aren't 100% sure it's working, feel free to post it and i'll try it out.

Ed

[NickK]

James - what version of Linux are you using? I'm assuming you're using libusb?

[JamesF]

James - what version of Linux are you using? I'm assuming you're using libusb?

I'm mainly using Mint 15 and Mint 16 on my 64-bit and 32-bit Intel architectures, so effectively a relatively recent Ubuntu releases without the horrors of Unity or Gnome3  For anything on the RPi I'm using the latest Raspian release which is based on Debian Wheezy.

I make use of libusb for scanning for connected cameras and where I've had to write my own user-space driver for a camera (eg. the QHY5).  If the camera vendor supplies a Linux interface then I use that (just the ZWO cameras at the moment, and I'm fairly sure their SDK is layered on top of libusb anyhow).  For webcams such as the SPC900 and Microsoft's Lifecam and Xbox Live cam, as well as Imaging Source cameras, I go straight to the V4L2 kernel interface as there's no obvious good reason to reinvent the wheel.

I did find when writing the QHY5 interface that I needed a very recent version of libusb -- rather later than is included with many current Linux distributions -- so my build/install scripts create a private copy just for oaCapture.  In fact I started with libusbx 1.0.17 when I found problems with the QHY5 initially, but at the start of 2014 libusbx and libusb "unforked" and there's now a 1.0.18 release for the merged project so I'm going with that.

James

[JamesF]

I have written some documentation now, and tested another beta release of the ZWO SDK which appears to work fine.  I've also got the code to build and install on 32-bit and 64-bit Mint 15/16 and on the RPi.  Just need to do some testing of cameras now and package the files up and I'll be ready for another release

James