Is it the Data... or is it the Computer.. Luna Image Help!

[Fozzie]

Morning all!

I've a problem, and it's starting to really get me in respect to my lunar imaging... I seem to be unable to complete the work flow due to either my PC at home (which is not the best) or my laptop which is an i3 4gb system freezing or taking for ever.. or my data sets being so poor that the computers/programmes cant do anything with it.. I need to know which!!

As a bit of background, my work flow is fairly standard..

1. Shoot images in CR2 files, each is about 22mb, usually a 300 - 500
2. Crop and centre in Pipp, using the standard lunar full frame settings, in mono
3. Registax 6 or Autostakkert to stack images
4. Registax 6 to wavelets
5. cannon photopro 4 for a final sharpen.

Now I've had good results in the past on this but for the last 8 months, I've been hitting obstacle after obstacle.. Pipp is not an issue although it does take some time (hours in some cases), however I am now falling down on the staking stage.

I am 99% convinced its a "hardware" issue and I need to be upgrading the PC by £1000  which if i'm totally honest I can't begin to justify.. but then Im from Yorkshire and it takes me 3 months to persuade myself to by a new pair of jeans!  I'm also convinced it's hardware as a took a 6000 frame avi which the steam engine computer had no issues with (it just doesn't have the level of detail I would like, when single frames on the canon 750d have so much more life in them.

So anyway I thought I would just check that there is nothing wrong with the data collected in the cr2 format, if anyone has time on there hands to give it a spin through pipp and stack them.

The drop box link is the data I collected at 05:00 on sunday morning after getting up at 03:30 for a session.. So i'm defo putting the effort in in that sense, but it would be nice to know for sure where the issue is..

If any one fancies a spin, the link is here.. theres just under 12gb..

https://www.dropbox.com/sh/h46p5okw4u9fr1h/AAAwQ1kPzcHtFzceRklGz08la?dl=0

files 726 - 1082 are one shoot, 1083 - 1258 are another with slightly different ISO etc..

Focus looked to be ok on the CR2 files in photo pro4..

Thanks for any help.. be nice if anyone does get some song out of the data to see it posted below.

Thanks again, in advance..

A frustrated Fozzie x

 

EDIT - The Data has even maxed out my drop box now... only 99 files uploaded, and I cant seem to upgrade... I'm reeeeaaaallllly bad at this stuff!  Will try and sort shortly.. but feel free to use the 99 image file.

EDIT1 - Just uploaded files 1083-1258 in to the drop box..

[xtreemchaos]

hi fozzie, it would take me a week to download 12 gb, do you have any staxed images or didn't you get that far. when you sorted with Pipp what type of file did you save as ?. charl.

[michael.h.f.wilkinson]

I have had issues with Registax freezing on larger files. AutoStakkert!2 cannot handle more very large images, I have found, but 64-bit AutoStakkert!3 does (although you may have to change some default settings for maximum width and height of the images). AS!3 can be found here:

http://www.astrokraai.nl/software/latest.php

[Robp]

You can easily check computer resource in Windows Task Manager. Open Windows task manager on the computer and have a look at the performance tab. Start the stack and check the performance to see what is maxing out (CPU, Memory or both). Next try to stack a smaller stack, say 10 images. Check the performance again and see if you notice a difference.

The bigger the size of the data set, the longer and more taxing it will be on the application. I was able to get by with an ancient AMD cpu and 4GB Ram, you might just need to drop the amount of data you push through the applications down.

I'll have a go at the data later today if no one else can.

[Fozzie]

44 minutes ago, xtreemchaos said:

hi fozzie, it would take me a week to download 12 gb, do you have any staxed images or didn't you get that far. when you sorted with Pipp what type of file did you save as ?. charl.

HI Charl, I've got passed the Pipp stage, but haven't got as far as stacking.. It's almost as if the computer has done too much and then slows right down.. I usually save the files as TIF to feed into registax or autostakkert..

What's frustrating is that I've managed this before, with reasonable results.. but now....?

35 minutes ago, michael.h.f.wilkinson said:

I have had issues with Registax freezing on larger files. AutoStakkert!2 cannot handle more very large images, I have found, but 64-bit AutoStakkert!3 does (although you may have to change some default settings for maximum width and height of the images). AS!3 can be found here:

http://www.astrokraai.nl/software/latest.php

I've had more joy out of AS than RS6.. In fact last time I got a reasonable set out of a selection but the alignment was off on a few frames, throwing an ghosting image.. despite using a motorized eq mount that was quite well on the PA.. I know this because I managed this unguided 30 sec sub of the Orion Nebula as a test shot, waiting for the moon to come up.. That was with the F11 scope too..

Will try AS3 on the Pipp files later..

30 minutes ago, Robp said:

You can easily check computer resource in Windows Task Manager. Open Windows task manager on the computer and have a look at the performance tab. Start the stack and check the performance to see what is maxing out (CPU, Memory or both). Next try to stack a smaller stack, say 10 images. Check the performance again and see if you notice a difference.

The bigger the size of the data set, the longer and more taxing it will be on the application. I was able to get by with an ancient AMD cpu and 4GB Ram, you might just need to drop the amount of data you push through the applications down.

I'll have a go at the data later today if no one else can.

Will give that a try.. But historically the wee duel core Pentium has stacked a good 150 images before, cant figure why theres a'n issue now.

Many thanks for the responses!

Ta

Fozzie

[michael.h.f.wilkinson]

BTW, I have a laptop with a Core-M processor, and just 4GB of RAM, and I manage to stack large data sets quite regularly. When making big lunar mosaics with my ASI178MM and C8, I typically gather 250-odd GB of data (40-50 1000-frame SER files), which I then stack through either AS!2 or AS!3 overnight. In the past I really struggled stacking raw images from my DSLR, but I do manage now (with AS!3).

[xtreemchaos]

thanks Fozzie, I don't like raw for the moon and sol, I only shoot in Jpeg or avi, let the camera do the work mate, its a lot easyer to stax and process. I still get good shots. I used to do the the long way round doing RAW cropping ect but it just wasn't needed, I think the last time I used raw was about 3 years ago. raws and tiff hold there own with deep sky images where you need to get every last bit of detail but its overkill where the moon and sol is concerned. charl.

[Fozzie]

31 minutes ago, xtreemchaos said:

thanks Fozzie, I don't like raw for the moon and sol, I only shoot in Jpeg or avi, let the camera do the work mate.

Thanks Charl, I'll give JPeg a try... could probably use Canon Photo pro4 to convert the raw in to Jpeg before I try to Workflow them..  Not a big fan of the avi on the canon 750d, I just don't get the detail out..

 

[xtreemchaos]

thanks fozzie, I in pipp save my avis after sorting as ser files, it staxs lovely. charl.

[Robp]

Converting them to JPG should also reduce the filesize as would reducing the image dimensions.

When you have done this before successfully, did you use RAW files or Jpgs?

Raw files are quite large so it might be a combination of individual file size (dimensions of the image, and on disk size) and amount of files that is giving you problems? It would explain the AVI working as PIPP splits that up into small jpgs if I remember correctly and Avis are usually lower dimensions than a full RAW file from a canon. (been a while since I did this ).

 

 

 

[Fozzie]

47 minutes ago, Robp said:

Converting them to JPG should also reduce the filesize as would reducing the image dimensions.

When you have done this before successfully, did you use RAW files or Jpgs?

Raw files are quite large so it might be a combination of individual file size (dimensions of the image, and on disk size) and amount of files that is giving you problems? It would explain the AVI working as PIPP splits that up into small jpgs if I remember correctly and Avis are usually lower dimensions than a full RAW file from a canon. (been a while since I did this ).

 

 

 

Hi Rob, I've always worked with RAW files, and had some reasonable results for full disk work..  As said the AVI didn't seem to struggle, took a bit of time, but did the work, the only issue is the spec on video mode for the 750d.. 1920 x 1080 (29.97, 25, 23.98 fps) just doesn't pull the detail out, for me, against a series of stills.

I will give converting to JPEG or use pip to output a SER or AVI.. see if I get anywhere with that..

as an example below, these are all over 100 frames, from RAW (CR2) files.. kind of what i'm looking for.. as I know I can do this!!

Ta

Fozzie

[Fozzie]

I've deleted the Drop box files... will give the above advice a run through the pc tonight!

Thanks everyone..

Ta

Fozzie

[Fozzie]

So iv'e managed to run through the process, converting the CR2 to SER using Pipp, registax and wavelets as a quick test..

I feel like I'm loosing some definition but actually for a stack of 350 or so.. this isn't too bad..

Many thanks to @xtreemchaos, @Robp and @michael.h.f.wilkinson for the advice..

Here's what I've got thus far.. It's not had a final sharpen but as a starter its not bad

Thanks everyone

Ta

Fozzie

 

 

[Robp]

Nice one. 

Ive never heard of a SER format, your maybe loosing some quality from the conversion but you'd get the same from converting to JPG.

What is the stack size on disk before and after conversion?

What version of AS! are you using and does that support more than 4GB RAM?

If you check system resources do you max out RAM when stacking?

if the last two questions are yes, have a look and see if you can drop some more RAM in? check your OS supports it (64bit), borrow something compatible from a friend, or another computer you might have, and see if it makes a difference. it might be a cheap fix.

[cgarry]

17 hours ago, Robp said:

Ive never heard of a SER format, your maybe loosing some quality from the conversion but you'd get the same from converting to JPG.

There will be no loss of quality when converting from CR2 to SER with PIPP.  Both formats are lossless and SER can hold 16-bit data.

Cheers,
Chris

[Fozzie]

On ‎17‎/‎10‎/‎2017 at 23:02, Robp said:

Nice one. 

Ive never heard of a SER format, your maybe loosing some quality from the conversion but you'd get the same from converting to JPG.

What is the stack size on disk before and after conversion?

What version of AS! are you using and does that support more than 4GB RAM?

If you check system resources do you max out RAM when stacking?

if the last two questions are yes, have a look and see if you can drop some more RAM in? check your OS supports it (64bit), borrow something compatible from a friend, or another computer you might have, and see if it makes a difference. it might be a cheap fix.

I was using AS2 and/or RS6, both my systems are 4mb and 64bit.. I have looked at increasing the RAM on the home computer, but it's one of those all in one jobs my wife used for simple emails and invoicing (did the job), and despite the fact I have built my own PC's in the past (from parts salvaged from old/broken office pc's) I don't fancy the swap.. something tells me not to mess with it.. The other is my works laptop, so that would be hand slapping if I broke it!

I'll see how I get on shooting in Jpeg instead of RAW..

Ultimately I will need to invest in new, but if I can get the SER file format to work then I might be on to a short term fix.

Hopefully this will be the end of the frustration for a while.

Thanks again

Fozzie

[Fozzie]

20 hours ago, cgarry said:

There will be no loss of quality when converting from CR2 to SER with PIPP.  Both formats are lossless and SER can hold 16-bit data.

Cheers,
Chris

Thanks Chris, good to know... hopefully I can get back to the imaging a bit more frequently now, and not spend over 6 hours trying to process it all..