Homebrew code - Lodestar Live update

[Paul81]

Hi All,

I posted a while back that I was working on a homebrew project to create an application to turn lower end astro CCD cameras into a live view camera functioning rather like the Watec and Mallincam devices on the market. I have seen some great images from other users using the SX Lodestar, but thought it would be great to combine it with a simple to use program which handles the live stacking of short exposures, with the ability to perform some simple display image processing. Apart from the interest, the other reason for creating something was I run Mac OSX and do not want to run MS Windows. There doesn't seem to be anything out there so thought I'd try and change that!

I settled on the SX Lodestar as I needed a camera whereby the USB command set was available. SX publish theirs, where as other manufacturers (Atik, Brightstar, ASI) do not.

I have posted some screenshots. So far I have sorted the display processing, so you can real-time view the image histogram, stretch it, apply gamma, contrast and brightness etc. The program allows you to chose an exposure (intention is 5-60 sec), it will talk to the camera etc and then real-time dark frame subtract, align and stack the images. You can view the last raw, or latest stacked image. It will also export raw images to FITS, and allow you to save a copy of the processed image. Other modes include focus and GOTO alignment assist, and dark frame capture. The program uses libUSB for low level USB and the QT framework for the UI, so is cross platform compatible (thus Windows and Linux versions can be built).

Progress was quite good, but recently been doing loads of overtime at work so had very little energy for my astro side project :-(. I am currently working on the image registration (alignment) and auto frame rejection algorithm.

Will keep the forum up to date with progress - I really do see things like this being very useful as the results are way more instance that hard core imaging, and it allows you to see more detail than what you could visually with the same scope! My target is to finish by the end of the year - when I can afford a mount for my telescope so I can do some video imaging

!

[Daniel-K]

wow that actually looks very good!!! would this work with OSC low end CCD'S?

[Paul81]

Thank you for the kind comment!

Yes I was intending on supporting OSC in the future. My own plan is to use a Lodestar Mono (for reasons of sensitivity and ability to use narrowband), but later add filter wheel integration and thus update all the internal algorithms to handle colour. At the same time I can add OSC support - my thought is to debayer the raw OSC frame as it arrives from the camera, and then process the RGB as if it were captured using separate R,G and B exposures. Keeps most of the data flow common.

I'm really itching to finish this project - damn work getting in the way! Haha!

[shirva]

Got to put bread on the table and a roof over your head first Paul. ..lot of hard work you have put into it so far and looks good...I've not went into tech side of capture progs I'm just looking to point and shoot so any program that kis all the better for me...good luck saving for your mount...Davy

[Spacedout]

Hi Paul,

I admit as a beginner I'm lost on the jargon but this looks pretty impressive to me! I shall be keeping an eye on your progress with interest.

Cheers

Karl

[nytecam]

Brilliant Paul - as a longtime Lodestar imager with both M and OSC C versions this is great news. Hope darks/flats included for these uncooled cams and you get around to Windows version too. My M13 etc pics from Friday night below with auto dark/flat/register/stack via the old/cranky SX s/ware. It doesn't work with colour cam due to registers pixel shift. Good luck

[DoctorD]

Hi Paul

Looks like the start of a great program. Your approach to OSC sounds like a good idea - reuse as much code as possible.

I look forward to seeing how this develops.

Clear skies

Paul

[steppenwolf]

This looks really good - I'd be interested in something like this for outreach purposes. Good luck with the project.

[Paul81]

Nytecam, I have to admit it was your excellent posts on this forum that really engaged my interest in using the Lodestar and creating this program. It's amazing how much detail you pick up in such a short time. My goal is to keep the program nice and simple, point and shoot and keep life easy. I think using astro CCD rather than the analogue integrating cameras can offer much greater flexibility and simplicity given some suitable work. I want to keep it easy as I love the simplicity I currently have my with my manual dob, but I must admit I want to 'see more', so want to augment my visual observing with some short exposure style imaging.

There will be dark frame support, and a few people have mentioned flats so I can probably work that in at some point (the math is easy for flats, and the UI elements are similar to the dark frame stuff).

Thank you all again for the interest and kind words!

[shirva]

Evening gaffer (Steve) that you turning from dark side.lol.glad to see you visiting the video forum...Davy

[steppenwolf]

I'll have a go at most things if it involves the night sky and imaging - although planets require somewhat more focal length than I can currently muster ....!

[shirva]

Good excuse for a second obs. .tell the Mrs its a donation for doing your reviews lol...still clouded out up here gaffer. ...Davy

[Paul81]

Just a small update..

I received my Lodestar last week and have been playing around in the limited time I have had. The good news is I have written the driver code for it (tested Mac and Linux). My next job is to integrate the code into the main application. I can then start coding the main modes of operation (framing, focus and alignment, dark frame capture, image capture). I'll post some screenshots when I have them.

I do not have a mount for my scope yet so I have been using a 12mm C-mount TV lens on the Lodestar - should make for some cool widefield shots!

[shirva]

Great news I'm Shure there will be a few waiting on the program getting up and running ...Davy

[simonct]

Hi Paul,

Your project sounds extremely interesting, keep us updated ! I've been working on my own Mac SX software and have been pondering what you could do with CoreImage/OpenCL especially with relatively small frame CCDs like the Lodestar/Superstar...

All the source code's here - feel free to grab anything you find useful

Simon

[NickK]

Just to cover Simon's point:

OpenCL - is very suited to image processing. My background is parallel/distributed systems and I've used GPGPU since 2005 (Radeon's CTM and OpenGL). It works very effectively but you'll need to design your system to work in a parallel paradigm rather than a serial.

CoreImage - I would avoid like the plague for non-display. For example using affine transforms results in 8 bit RGB output even if you put in 16 bit mono.

[shirva]

Wooooosh right over my head....keep up the good work. .Davy

[simonct]

OpenCL - is very suited to image processing. My background is parallel/distributed systems and I've used GPGPU since 2005 (Radeon's CTM and OpenGL). It works very effectively but you'll need to design your system to work in a parallel paradigm rather than a serial.

Very true. I had the opportunity to do some OpenCL a few years back and the problem has to fit the technology. I was thinking along the lines of feeding video frames into the GPU and then calibrating, stacking and possibly registering them all into an OpenGL buffer. The first tasks are per-pixel/workgroup so should be fine, not so sure about the registration but it could probably work.

CoreImage - I would avoid like the plague for non-display. For example using affine transforms results in 8 bit RGB output even if you put in 16 bit mono.

Hmm, sounds like a bug. Which interfaces are you using ? - there are so many ways to convert between image formats these days there may well be a better option.

Usually for offline processing I usually just convert to floating point once the raw pixels are archived and throw vImage at it. Something like vImageAffineWarp_PlanarF is usually plenty fast enough without sending data back and forth to the GPU.

[Paul81]

So far I have been coding the image processing completely from scratch (at work I design high end real time video processing systems so used to the weird and wonderful maths of image processing), rather than using any libraries. In this way I can tailor things exactly as I want them (and keep all data 16-bit until display). So far the code is straight C++, but parts will be moved to SIMD assembler for speed. I know there are some libraries out there, but creating it all from scratch is part of the fun and challenge!

Likewise as you guys I have plenty of GPGPU experience and agree the GPU would be excellent to accelerate the image processing... however I took the decision to avoid using the GPU for this application and use just the CPU instead. The main though is that when people image in the field they either use a low end laptop / netbook which has a lousy GPU but normally a fairly decent CPU (1.5-2 GHz with a couple of cores), or (in my case) I have a Macbook Pro with a dual GPU set-up. Although the NVIDIA 650M in the Macbook would eat the data processing for the Lodestar for breakfast, the battery life on my laptop reduces somewhat when the discrete GPU is active. By sticking to the in-built GPU (and keep the code running on the CPU) the battery life is circa 10 hours. However, I have kept good encapsulation within the code so should I want to do a GPGPU version then it's not a major re-write or headache (in fact I think its easier than the SSE assembler). The code is multi-threaded, the hard core tasks (such as feature detection, feature matching and homography estimation) will run in the background, and the display side processing (DNR, gamma etc) and the camera handling run in their own threads too. Keeps the design easier as well.

In terms of progress, I tweaked the display processing and have further prototyped the feature detection, feature matching and image registration in Matlab (well GNU Octave - Matlab is too expensive for home use). I am thinking of allowing the selection between using bilinear and bicubic sampling when the image registration transforms take place. People using short exposures and lower end computers probably wont have the horsepower for bicubic sampling, but longer exposures and hot CPUs should be fine.

So much to do... so little time...

Nick - I saw your video - your app is looking good!

[NickK]

Very true. I had the opportunity to do some OpenCL a few years back and the problem has to fit the technology. I was thinking along the lines of feeding video frames into the GPU and then calibrating, stacking and possibly registering them all into an OpenGL buffer. The first tasks are per-pixel/workgroup so should be fine, not so sure about the registration but it could probably work.

The performance killer for GPGPU has always been the transfer over the PCI-E.

Hmm, sounds like a bug. Which interfaces are you using ? - there are so many ways to convert between image formats these days there may well be a better option.

Yup - it appears to be a known bug that's been around for a while in CIImage. Using CImage was a stopgap for prototyping but makes no sense long term.

[NickK]

Fast scope, sensitive CCD, high gain, .. and then you get to the time required to collect enough photons - which is enough time to register/stack an image usually. I just elected to just to keep a flat plane for live view and leave the frames to PI which is not much slower.

It'll be good to have more mac apps - I'll butt out of your thread but the app looks good

[simonct]

So far I have been coding the image processing completely from scratch (at work I design high end real time video processing systems so used to the weird and wonderful maths of image processing), rather than using any libraries.

Fascinating field - I worked in broadcast R&D for a while but mostly on the workflow side.

I’ve mainly been thinking about using the GPU for planetary work. Do you think your registration code would deal with that kind of imaging ?

My own app can capture subframes to QuickTime which, via ffmpeg to AVI, can then be stacked in the usual Windows programs running on Wine - not ideal. Would be great to (a) do it all on a Mac in one program and ( do it in real time !

Btw, I’ve got a few SX cameras including a Lodestar so just shout if you need any beta testing.

Simon

[Paul81]

The current registration code would not work for planetary imaging in its current form. It is a feature based registration and the feature being detected is the stars in the image.

I was having some thought to a subsequent app after this one for live planetary stacking - I agree you would definitely need to use the GPU in order to process the frames at a fast enough rate. I would need to adapt my registration algorithm to pick planetary features and match those but I think it would be do-able.

I have some very nice 60Hz high resolution GigE vision cameras at work (which I am sure I could borrow over a weekend..). They could make for some nice planetary images.

One project at a time though! Lol!

Also not sure if you have come across it but there is a Mac based planetary stacking software app out there - Lynkeos - although I have never used it.

[simonct]

Thanks for reminding me about Lynkeos. I gave it a go a while back but hadn't stuck with it for some reason, I'll take another look.

Btw, have you any experience/opinions on the OpenCV library ? I appreciate you're rolling your own code but this seems to have some interesting alignment capabilities. My own app has a fairly primitive offline automatic alignment feature which is fast but only deals with translations so far which limits its usefulness. I've dabbled with OpenCV before and it seems pretty quick, especially with the boost threadpool library. Looks like it's starting to support OpenCL as well.

[Paul81]

I have used OpenCV for some projects in the past and I think it is an excellent library. I have often used it for prototyping some image processing algorithms and then found I have needed to write my own code as it had to run on very low power or FPGA based hardware, but it does serve as an excellent tool to get you up and running.

OpenCV has some good homography estimation which you can use to get the translation, rotation, scaling and shear transformation matrix of one image with respect to another to yield registration.