Too much data that put my computer on its knees

[Herra Kuulapaa]

For two days I most likely need to scale it down before AS3 processing, or upgrade my gear. Still working with my trusted i7-870.

Anyway, this was captured 19th of April.

[Davey-T]

Nice image Herra, I capture all my solar to an external USB3 drive, also process on it then just import the finished tiff onto the main HD for further processing.

Dave

[Herra Kuulapaa]

How much data do you normally take Dave? I’m thinking to switch from 14bit to 12bit capture and bin as well. 

[Macavity]

I would like (some time) to have better data filtering ! Basically, to process
the images as quickly as possible to work out if the SEEING is any good? lol
That would of course need a new Laptop, rather than my little Netbooks!

TBH, I note that the benchmark for the FASTEST possible machine now is
only SIX times my modest i5 (4 core) 3.3Ghz. lol. It's also the data transfer
(Notebook to Desktop) that takes time here. I must investigate that too!

Intel Core i9 7980XE - 2.6GHz 18 Core Socket 2066 Processor
A positive "steal" at a mere £2075 (sterling) on Amazon UK etc. 

[Davey-T]

2 hours ago, Herra Kuulapaa said:

How much data do you normally take Dave? I’m thinking to switch from 14bit to 12bit capture and bin as well. 

I use Firecapture in 16bit mode with PGBfly camera, 3000 ser file  frames is nearly 6GB so can quickly mount up to 300GB to 500GB on a good summers day imaging.

The camera runs at 30fps on full frame and data transfers to 1tb USB drive with no trouble.

Dave

 

[Herra Kuulapaa]

I'm using 500GB SSD, but it still feels like it fills in a mere moment with ASI178.

3000 frame SER file takes 18GB captured 12bit and 14bit takes just a insane amount of disk space. 20 stacks 3000 frames 3000x2000 pixel each just makes my computer crawl. It helps a bit downsizing files to 1500x1000 pixels to reduce over sampling, but my flats tend sometimes get out of sync then. Hardware binning could help this and I did some experimenting last season even though didn't draw any final conclusions on quality.

I'm also dreaming of better computer, but it sits 99% of time idle so it feels a bit unnecessary to put 1500€ into more cores. Or does it...

[Macavity]

Interesting thread! I really must try using .ser files - Now I have the Chameleon3...
I am surprised that I can get full frame1280x960  at 30fps even via (still!) USB2.0
I will have to work on my judicious use of ROI now solar features are small(er)!

Any projected new Laptop would have SSD as temporary storage. But using 16bit
along with ImPPG has reduced by need for BOTH "disk" and "proms" data. I now
extract both from the one data file... So an extra few BITS (via .ser) may help?

[Herra Kuulapaa]

I tend to think that the benefit of 14bit vs 12bit is not that big. In fact I have failed to notice any difference, but I can only speak on my behalf and it doesn’t mean the benefit isn’t there.

For me it makes sense trying to capture best seeing with less bit depth and more disk space. I’ve also understood it doesn’t increase dynamic range to use larger ”colour” space - it only makes shade transformation smoother but we stretch the data like crazy persons in the end anyway

[Davey-T]

Cheapest solution maybe to sell the ASI178 and get a less demanding camera that produces smaller files I don't think the difference in image quality will be very noticeable.

You can always mosaic for larger images.

Dave

 

[Herra Kuulapaa]

Or just start binning 2x2 during the capture because I use that same camera in OAG winter time. Either way can’t keep processing data resulting 10.000 x 10.000 pixel end images with pointless over sampling.

[johnfosteruk]

Nicely done Herra, very sharp.