Struggling with APP + GIMP

[Anthonyexmouth]

Since spending all my money on my my new camera i didnt leave enough to upgrade to the full version of Pixinsight which i was getting on with in a very basic way. This lack of money forced me to go for a rental of APP and also trying GIMP. 

With the learning curve of the new cooled camera and new processing software im really struggling to get anywhere. Everything is too over processed or a black screen.

Any advice would be appreciated

my latest attempt, the soul nebula (poorly framed) struggling with framing as well with the new camera. previews dont seem as bright as the old canon.

included the stacked tiff too if anyone wants to play with it

Autosave.tif

45x240s 

50 darks

50 flats

[vlaiv]

You mention APP for stacking, but Autosave shows DSS as software used to create it.

What sort of stacking did you use? This Autosave.tif when stretched in Gimp can provide image like this:

For some reason it feels that something was not done properly in stacking phase, there is too much noise and gradients are very strange - it might be due to improper darks or maybe due to flats calibration - don't know, but like I said, I suspect that something might be optimal in calibration and stacking ...

[Anthonyexmouth]

yeah i used DSS because i was getting frustrated with APP. 

Heres one of my darks and flats

D_2019-07-21_12-45-14_Bin1x1_240s__-10C.fit

F_2019-07-28_08-34-29_Bin1x1_0.00547s__-7C.fit

 

[vlaiv]

Everything seems to be in order with these (you've got a strong flat panel - very short flat exposure, but that is fine, on my setup I also have flats of a few milliseconds).

Only thing that I would "change" is maybe getting additional 50 or so flat darks to see if that will help (it might). Same settings as flats, just cover scope (read off exposure length from flat file name and use unity gain settings like you already did for flats).

Oh, btw - what was your offset (it is set in ASCOM driver together with gain)?

[Anthonyexmouth]

i think the offset is 30. havent played with it as i read somewhere that its fixed now with the latest drivers. 

flats were done with a t-shirt outside and i let APT work it out with the CCD flats aid. 

[vlaiv]

2 minutes ago, Anthonyexmouth said:

i think the offset is 30. havent played with it as i read somewhere that its fixed now with the latest drivers. 

flats were done with a t-shirt outside and i let APT work it out with the CCD flats aid. 

Guess offset of 30 should be ok and even if it did cause trouble - it would be different kind of trouble, so that is fine.

Flats are fine too, regardless of method. Try adding flat darks - it should take only a minute to run 50 subs at 5ms or so, and it might help with calibration. If you want, maybe upload all subs somewhere (like google drive or what ever) and I can try calibration and stacking in ImageJ to compare results?

[Anthonyexmouth]

thanks, i'll start making a google drive folder. 

any idea why its so noisy? is that something with the stacking or the raw images?

[vlaiv]

I'm suspecting it might be something with stacking - not 100% sure, but something tells me it could be a problem with stacking.

I just checked out values in red channel (nebula is mostly Ha - so red), and background is at 0.042-0.043 while nebulosity is about 0.045 (in 0-1 range). For some reason it is very low in signal and it's just above background noise - this is why it is so difficult to process. That and fairly strange gradients, which might be quite legitimate LP are causing trouble. I suspect that there might be something wrong with stacking because signal is so low and maybe background equalization went wrong or selection of stacking method was wrong ... Maybe I'm wrong with thinking this way

 

[Anthonyexmouth]

I just followed the suggested settings in DSS for stacking. K/sig for lights and median K/Sig for flats/darks. 

only had the trial of PixinSight but really miss that kinda workflow. its intimidating to start but after watching astrodudes tutorials it really is nice to work with. 

must start saving to buy it and not get distracted with toys 

[vlaiv]

I made another attempt with above tif that you posted. First split channels, then did x2 binning to see if I can recover some SNR, then did some weird background removal algorithm of mine and did basic processing in Gimp - stretch + denoise.

This is looking better now - there is a "green" zone in center which might be due to missing flat darks, and strange gradients were mix of LP and issue with framing - bottom right are outer parts of Heart.

[Anthonyexmouth]

heres the lights/darks/flats

not had time to do the dark flats yet.

https://drive.google.com/drive/folders/1nIk1YOAIJnij2MAqulXBJj9WEZWj7oEw?usp=sharing

[wornish]

Been following this thread and just thought I would try your data in APP to see what happens.

All the lights you shared are Grayscale,  I thought the camera was one shot colour?

As they are mono images there is nothing to tell which channel they are, red, green or blue.  Not sure if I am missing something here.

 

 

[vlaiv]

3 minutes ago, wornish said:

Been following this thread and just thought I would try your data in APP to see what happens.

All the lights you shared are Grayscale,  I thought the camera was one shot colour?

As they are mono images there is nothing to tell which channel they are, red, green or blue.  Not sure if I am missing something here.

 

 

They are raw images with bayer matrix, you need to calibrate them and then debayer to get color out of them. Bayer matrix setting for this camera model is RGGB if I'm not mistaken.

[Anthonyexmouth]

23 minutes ago, vlaiv said:

They are raw images with bayer matrix, you need to calibrate them and then debayer to get color out of them. Bayer matrix setting for this camera model is RGGB if I'm not mistaken.

correct

[sloz1664]

Just done a quick APP & Pix process with your images. APP threw up a a warning of no bias frames, which you should gather for your camera:-

Steve

Edited August 6, 2019 by sloz1664

[wornish]

4 hours ago, vlaiv said:

They are raw images with bayer matrix, you need to calibrate them and then debayer to get color out of them. Bayer matrix setting for this camera model is RGGB if I'm not mistaken.

Thanks my bad...    as usual🤐

I use a mono ASI1600 so not come across this before.

I guess its best I just follow the thread and not embarrass myself going forward. - Sorry to intervene.

 

 

[vlaiv]

2 minutes ago, wornish said:

Thanks my bad...    as usual🤐

I use a mono ASI1600 so not come across this before.

I guess its best I just follow the thread and not embarrass myself going forward. - Sorry to intervene.

 

 

Why don't you give it a go anyway? APP should have option to debayer images taken with OSC sensor - after that it's pretty much processing as usual ...

Here is a little thread how to work with OSC fits (APP does automatic job on DSLR raws but does not recognize OSC fits):

https://www.astropixelprocessor.com/community/main-forum/how-to-debayer/

[Anthonyexmouth]

ok, a little confused now. a few say i don't need bias frames and some say i do. Also, whats the difference between bias and dark flats? 

[wimvb]

The light frames contain the following "data":

Photons x (vignetting + shadows) + dark current + read pattern

Dark frames contain:

Dark current + read pattern

Flats:

(Vignetting + shadows) + dark current* + read pattern

Bias:

Read pattern

The dark current in flats is different from that in lights because of the different (shorter) exposure time. If the flats exposure time is very short, "dark current*" is basically 0.

Calibration of light frames has to eliminate anything but "photons".

Because "read pattern" is in every type of frame, you can calibrate your lights with just darks, as long as you don't alter those darks (a process called scaling, eg when the exposure time of your darks doesn't match that of your lights).

Lights - darks leaves you with

Photons x (vignetting + shadows)

You still need to divide by (vignetting + shadows). That has to come from your calibrated flats. To calibrate flats you either need dark flats, which contain "dark current* + read pattern", or assuming that the term "dark current*" is zero, just "read pattern" from bias frames.

Long story short, if you don't scale your darks, and take dark flats, you don't need bias. Otherwise you do.

I hope this makes sense.

Edited August 7, 2019 by wimvb

[Anthonyexmouth]

4 minutes ago, wimvb said:

 

Long story short, if you don't scale your darks, and take dark flats, you don't need bias. Otherwise you do.

I hope this makes sense.

so as long as i have a library of darks that correspond to the various exposure times of any subs i take then a single library of dark flats is all i need, and dark flats are the same exposure length as my flats?

[wimvb]

1 minute ago, Anthonyexmouth said:

so as long as i have a library of darks that correspond to the various exposure times of any subs i take then a single library of dark flats is all i need, and dark flats are the same exposure length as my flats?

Yes

[Anthonyexmouth]

1 minute ago, wimvb said:

Yes

cool, i'll get to work on the dark flats tonight. forecast is promising so i'll uncover the scope and maybe get some imaging done too. 

[wimvb]

Because of possible amp glow, even in short exposure flats, flat frames taken with (cooled) cmos cameras should, in my opinion, be calibrated with dark flats, and not just bias frames. It doesn't take much longer to shoot dark flats than bias frames, and assuming that your light source for flats is fairly constant, you can reuse dark flats. If flats aren't calibrated properly, and contain residual "dark current*", your lights won't be properly calibrated either, leaving an uneven background in the stacked image.

[vlaiv]

59 minutes ago, Anthonyexmouth said:

cool, i'll get to work on the dark flats tonight. forecast is promising so i'll uncover the scope and maybe get some imaging done too. 

Thing with flat darks or dark flats (I've seen both orders used ) is that they are so close to bias in some circumstances - when you have short flat exposures. It is not always the case - some sensors need longer flats and dimmer flat source like CCD sensors with mechanical shutter where you have to be careful not to capture motion of the shutter in single frame (flat exposure needs to be much longer than shutter time).

In principle, you should always avoid bias frames with CMOS sensors - I think there is no CMOS sensor currently that has OK bias frames - every CMOS sensor that I've seen suffers from some sort of problem with bias frames. There fore it is best to think of flat darks (or dark flats) as being just that - darks and not bias - meaning they need to be taken at exact same settings as flats (gain, offset, exposure, temperature, ....) with only difference being covered scope.

Wim explained the math behind it - all calibration is aimed at producing only photon response and removing all other signals related to electronics stuff (and shadows).

Flat dark library is good thing to have provided you have "controlled" environment - same way as with regular darks (where you control temperature). This means that you need to have regulated light source for flats and always use that flat source for every filter (or in your case just sensor because it's OSC). If your method of getting flats is T-shirt over aperture and bright sky in sunset / sunrise, then I'm afraid you won't be able to use flat dark library - exposure length for flats will depend on how bright daylight is at time you take your flats and also how thick is T-shirt, and how many times you folded it over aperture of the scope.

Best approach in that case is to take set of flat darks right after taking flats - just cover your scope after you've taken flats and do another set with same settings - exposure length also being the same. Flats / flat darks don't take much time to complete, you should be done with them in about 10 minutes or so (depends on download speed, exposure length and number of subs you take).

[Anthonyexmouth]

6 hours ago, vlaiv said:

 

Best approach in that case is to take set of flat darks right after taking flats - just cover your scope after you've taken flats and do another set with same settings - exposure length also being the same. Flats / flat darks don't take much time to complete, you should be done with them in about 10 minutes or so (depends on download speed, exposure length and number of subs you take).

As a flat field generator is out of the question at the moment, motorised focuser is next on the shopping list. Would a flat field app on my tablet be a better way to take uniform flats? am i right in thinking i can lower the light to take slightly longer flats and then take the flat darks right after with the same settings?

what would be a good number of frames to take?