Capture software for Linux

[JamesF]

With standard AVI libraries I could not figure out how to generate mono (DIB) AVI files that were not 3x the size the needed to be.  That is why I wrote my own class.  For the Ut Video codec (and MPEG-4) I just wrote a wrapper for ffmpeg and I am very pleased with how that worked.

Chris

I had shied away from ffmpeg because the interface seems to change on such a regular basis.  I've downloaded  the demo code from the ffmpeg website and none of the different version actually compile against the libraries I have installed.  If I get it to work I might have to static link it.

James

[JamesF]

Unexpected side-effects of the US government shutdown.  I can't get to NASA's copy of the FITS specs.

http://heasarc.gsfc.nasa.gov/fitsio/CCfits/html/index.html

James

[Robny]

This is looking very good and something I will watch closely - be interested how it goes.  On another note I've been curious to know what's available for Linux, but I'll take th?at to a different thread so not to hijack this one.

Stella work

Rob

[cgarry]

I had shied away from ffmpeg because the interface seems to change on such a regular basis.  I've downloaded  the demo code from the ffmpeg website and none of the different version actually compile against the libraries I have installed.  If I get it to work I might have to static link it.

James

It took me ages to get ffmpeg working on Windows, there are so many out of date examples out there on the internet.  Once I got it building all was fine as I distribute the DLLs with PIPP.  Ironically I build the Windows' ffmpeg DLLs on my Linux box because it is much easier.

How would this would this work with Linux?  Would you build and distribute the ffmpeg libraries with your application or does it not work that way?

Also, are you planning to make your program open source?

Cheers,

Chris

[JamesF]

It took me ages to get ffmpeg working on Windows, there are so many out of date examples out there on the internet.  Once I got it building all was fine as I distribute the DLLs with PIPP.  Ironically I build the Windows' ffmpeg DLLs on my Linux box because it is much easier.

How would this would this work with Linux?  Would you build and distribute the ffmpeg libraries with your application or does it not work that way?

Also, are you planning to make your program open source?

Cheers,

Chris

I'm finding a huge number of out-of-date examples myself  Even the example code that is shipped with the source code for the version I have installed doesn't actually build without lots of warnings

I'd probably opt for static-linking the ffmpeg libraries into the executable on Linux if I have to go down that route.  It's far easier to keep things under control that way and the cost in terms of size is hardly huge.

It will definitely be open source.  I've earned my living from working with open source software for fifteen years or more and been a user of it in one guise or another for not far off twice that.  It would feel disrespectful not to give a little back.

James

[JamesF]

I'm going to have another look at the ffmpeg stuff today.  If I can't get things to work then I'll leave it for the moment and do SER and FITS as they look far simpler.

James

[cgarry]

I'm finding a huge number of out-of-date examples myself  Even the example code that is shipped with the source code for the version I have installed doesn't actually build without lots of warnings

I'd probably opt for static-linking the ffmpeg libraries into the executable on Linux if I have to go down that route.  It's far easier to keep things under control that way and the cost in terms of size is hardly huge.

It will definitely be open source.  I've earned my living from working with open source software for fifteen years or more and been a user of it in one guise or another for not far off twice that.  It would feel disrespectful not to give a little back.

James

I am planning to open source PIPP once I have reworked it to be a C++ QT application (rather than a C# GUI round a C++ application), plus tidied up some of my scruffy code!  Though finding the time to do this is the hard part.

I have sent you a PM that you may or may not find useful!

Cheers,

Chris

[JamesF]

I am planning to open source PIPP once I have reworked it to be a C++ QT application (rather than a C# GUI round a C++ application), plus tidied up some of my scruffy code!  Though finding the time to do this is the hard part.

I have sent you a PM that you may or may not find useful!

Thanks for the PM.

Though I shudder to admit it, the awful weather has been something of a blessing in terms of getting coding done 

Once I've made more progress with this I'm more than happy to help with the Qt port of PIPP if you need it, but we can discuss that further down the line if you've not already done it by then

James

[perfrej]

One important thing with astro stuff, regardless of operating system, is USB. It is a very tricky thing to get stable, and the fact that the connectors are Rubbish also adds to the bonfire. I solved this with a hardware device that has an on-board 4-port USB hub with three of the ports switchable, both 5V and data leads. With that I can "physically unplug/replug" any USB device without doing anything to the computer. I have incorporated device drivers for ASCOM, but the protocol to talk to the thing is so simple that it would be a close to no-brainer to write Linuxdrivers for it.

General specs:

5 switchable 12V supply outputs

3 switchable 12V supply outputs that are switched together with...

3 switchable USB-ports

2 PWM 12V outputs for roof motors

4 digital inputs for roof sensors (opto isolated)

4 general inputs (opto isolated)

1 analog input for voltage check on batteries

4 general digital outputs (opto isolated)

1 sky quality sensor input (sensor is $5, same as in the commercial SQMs)

1 stepper motor driver for focuser

ATMEGA 328P processor with all "firmware" in Arduino code

As you can see, it is an entire observatory bar the astro stuff. You could easily run it from Linux via a serial port and get focuser, roof control and what have you. The focus driver, by the way, is a hardware stepper driver that can do unipolar and bipolar, and that micro-steps up to x8 if you want to run the motor without gear-box. There is software controlled hold/no hold also - good if you run naked steppers.

The coolest feature is the switchable USB ports. They use analog switches specific for USB switching use.

Great DYI project, indeed! Prototype (1 of 6) is running on my desk and will be integrated into the balcony setup this weekend. Since it has an Arduino boot-loader in the processor, you can write your own firmware with free tools on Windows (that is how it is spelled), Linux and OSX.

/per

Edited October 3, 2013 by perfrej

[Cath]

Ah, so this is your new capture program James. Looks good! .. keep at it

Theirs nothing like having your own software doing just what it is you want it to do rather than having to use someone elses software doing what it is they wanted it to do.

I think there might be a linux SDK for talking too the fireflys, although not 100% sure as I only used there Windows SDK.

[JamesF]

Ah, so this is your new capture program James. Looks good! .. keep at it

Theirs nothing like having your own software doing just what it is you want it to do rather than having to use someone elses software doing what it is they wanted it to do.

I think there might be a linux SDK for talking too the fireflys, although not 100% sure as I only used there Windows SDK.

Indeed it is, Cath.  I shall have a look for a Firefly SDK.  It would be quite handy to have those cameras running with this.

James

[JamesF]

And woohoo!

It's been a slog and a half and more than once I was considering abandoning AVI formats altogether, but as of five minutes ago I have this:

$ ls -l oaCapture-20131004-011601.avi -rw-r--r-- 1 james james 47297118 Oct  4 01:16 oaCapture-20131004-011601.avi$ file oaCapture-20131004-011601.avi oaCapture-20131004-011601.avi: RIFF (little-endian) data, AVI, 640 x 480, ~15 fps, video:

I had to resort to building my own copy of ffmpeg, to get their Ut Video encoder as the default Ubuntu/Mint version doesn't have much to offer in the way of codecs suitable for AVI files that can also handle RGB format frames that I could get to work.

This is a quite a big step forward and I'm feeling rather pleased with things right now

James

[knro]

Has anyone tried Ekos in KStars? Ekos already relies on INDI, so you have all the V4L2 cams, SBIG, QHY, QSI, FLI, Starlight Xpress, ATIK...etc CCDs already well supported. When I first got my astrophotography gear a couple of years ago, I was also furstrated by the lack of decent capturing software for a typical amatur astrophotograhy session. I wanted to be able to perform a decent polar alignment, focus my CCD, guide it, and capture images with different filter wheels while I am away, and this is why I developed Ekos, I wanted to do everything from one place.

Here is a Youtube video I made on Ekos which hopefully should shed light on some of its primary features:

[JamesF]

Has anyone tried Ekos in KStars? Ekos already relies on INDI, so you have all the V4L2 cams, SBIG, QHY, QSI, FLI, Starlight Xpress, ATIK...etc CCDs already well supported. When I first got my astrophotography gear a couple of years ago, I was also furstrated by the lack of decent capturing software for a typical amatur astrophotograhy session. I wanted to be able to perform a decent polar alignment, focus my CCD, guide it, and capture images with different filter wheels while I am away, and this is why I developed Ekos, I wanted to do everything from one place.

I'd not want to discourage others from trying it, but for me this isn't just about creating another capture program.  My longer-term goal is to create a suite of imaging tools I can do all sorts of other things with, such as running them on different architectures and even on headless servers or ones without any kind of graphics display at all (though they are not my only goals).  Writing a capture program is just one step along the way.  Fortunately it's a step that has very useful output

James

[knro]

I'd not want to discourage others from trying it, but for me this isn't just about creating another capture program.  My longer-term goal is to create a suite of imaging tools I can do all sorts of other things with, such as running them on different architectures and even on headless servers or ones without any kind of graphics display at all (though they are not my only goals).  Writing a capture program is just one step along the way.  Fortunately it's a step that has very useful output

James

The backend is INDI which already runs on headless servers and even Rasperry PI. Ekos is just a front-end to INDI.

[JamesF]

Yes.  Sadly INDI has a few other niggles that irritate me and I can't find a way to do within its existing architecture some things I want.  If I can find a way of fixing them then I shall do so.  If I can't then, err, I'll think about that when I get that far.  Which is probably going to be a while yet.  I have other fish to fry first.

James

[knro]

Yes.  Sadly INDI has a few other niggles that irritate me and I can't find a way to do within its existing architecture some things I want.  If I can find a way of fixing them then I shall do so.  If I can't then, err, I'll think about that when I get that far.  Which is probably going to be a while yet.  I have other fish to fry first.

James

I'm also the maintaner of INDI, what issues are you facing with INDI exactly?

[JamesF]

I'm also the maintaner of INDI, what issues are you facing with INDI exactly?

When I have some time, I'll make a list

James

[JamesF]

I've spent this evening making different bits of the application hang together (getting the captured frame count to show in the status bar, disabling some controls when the application is capturing to disk etc.) and doing some tidying up before I embark on getting my ASI120 to work.

Thus far I've been testing with an SPC900, my modded Lifecam and a cheap old webcam I bought to turn into an IR cam.  Mostly they work on my desktop box, though there is some issue with the brightness control on the Lifecam.  I reckon that might be something weird with the Lifecam though.

I thought I'd also give it a go on my laptop and netbook, both of which have built-in cameras.  The netbook is still building the ffmpeg library, but on the laptop the built-in camera works fine (but highlighted a bug with showing some unsupported camera controls  When I plugged in the other cameras however, the SPC900 would not work, throwing an error in the kernel.  I might have put this down to software, but bar a few patches that I am now installing, the laptop is set up almost exactly the same as my desktop.  The logs show the camera throws an error in the kernel which is certainly the same.  This happens whether the camera is directly-connected or in a hub.  Most odd.  I'm starting to wonder if it might not be something to do with the USB hardware itself.  Hopefully tomorrow I can try it in the netbook.

I think I shall ignore the problem for the time being if it continues to fail in favour of getting the ASI120 working as that would mean I can actually go out and capture some real data with it.  Perhaps at a later date I will need to revisit the user-space USB driver for the Philips cameras after all.

James

[JamesF]

Just tried the SPC900 on the netbook and it works fine.  I think therefore that it must be some sort of hardware compatibility issue with my laptop, or the Linux drivers for the laptop implementation on my laptop are iffy.  Irritating

James

[JamesF]

Aha!  Just discovered that the kernel driver is returning an ENOSPC error which means it can't allocate sufficient bandwidth on the USB bus.  No idea why that should be, but I'll have to look into it over the weekend.

James

[NickK]

Aha!  Just discovered that the kernel driver is returning an ENOSPC error which means it can't allocate sufficient bandwidth on the USB bus.  No idea why that should be, but I'll have to look into it over the weekend.

James

I see the driver is using interrupt transfers - the downside is they fail if the requested reserved bandwidth isn't available.

Check that the hub isn't multiplexed with internal devices.

[JamesF]

I see the driver is using interrupt transfers - the downside is they fail if the requested reserved bandwidth isn't available.

Check that the hub isn't multiplexed with internal devices.

Indeed.  There's nothing obvious at the moment, but in a similar vein I did have a mouse plugged in.  Removing the mouse allows the camera to work, but if the mouse is plugged in after the camera is running then the mouse doesn't start up, presumably for the same reason.

So, there's always the possibility that it's an issue with the mouse.  I shall  try a different one later and see what else I can find out about the hardware.  Unfortunately this morning I have the far more exciting task of visiting another potential secondary school for my son.

James

[JamesF]

Well, it looks like any other device plugged into the same USB bus with the camera means the camera has insufficient bandwidth on the bus.  I think for the time being I'm happy to leave that there.  It's clearly not a problem with my code.  It looks like the same problem exists with other cameras that use the same driver having had a quick google.  Ironic really, that a USB1.1 device should generate an error due to lack of bandwidth on a USB2 bus

James

[Cath]

That's just very poor driver coding by whoever wrote the driver.

I've been writing camera software for PC's for years now, and nearly every single time their are problems with cameras (quite often) it's always down to very poor driver coding, at least when it comes to windows drivers, manufactures are very poor at writing driver code and have no pride in their work/coding