Whats going on with my Jellyfish..Help!

[Craig123]

Hi all

I have been trying to use calibration frames on a few hours of  lights I obtained of IC443 recently.

It's the first time I've tried to use  calibration frames and also I'm using a new cmos camera  and fairly new to PI. After  a couple of sessions trying in PI I'm getting a little wobbly and can't work out what is happening in the final result.

Each time the finished calibrated lights come out looking  darker and a STF shows no sign of the nebula that is evident in a single uncalibrated sub with stf.

I've followed Light vortex tutorials and some cloudy nights forums on calibration frames with the camera ..ASi1600 mono cooled using master dark and Flats calibrated with dark flats.

Please would anyone be so good as to have a look at my fits files and advise if they look ok before it all slides right off my dessert plate on to the floor in a big mess. 🤑

Many thanks

 

integrationcalibratedmasterflat.fit integrationmasterdarkic443.fit jellyfishhalights2_00019.fits

Edited December 11, 2019 by Craig123

[Laurin Dave]

I've had a look .. your flat looks ok (but read later) ... but your Dark shows evidence of a light leak and consequently has a high average value which when you subtract it from the light leaves nothing except the star centres.

Did you take your darks and flat darks on the scope?  The best way to take darks and flat darks is to take the camera off the scope put its cap on and put in face down on a bench..  that way no light can get in and mess things up.  You'll need to figure out what exposures to take for your flat darks as you wont want to take the camera off to do them after you've taken your flats.   Once done they'll last a while (Mine are14 months old and still work perfectly) .  I've attached one of my 10 minute Darks so yo can see what it looks like stretched... unstretched its average value is 875 (16 bit basis) yours are 8300 … the Light has an average value of about 1500.  I suspect your darkflats are also messed up so you'll need to re-do them... they look like they've worked as the average for the flat is so high but if done I the same way as the Darks they won't because they'll also suffer from the light leak. 

HTH

Dave

Mine

Yours

Edited December 11, 2019 by Laurin Dave

[Craig123]

11 minutes ago, Laurin Dave said:

I've had a look .. your flat looks ok (but read later) ... but your Dark shows evidence of a light leak and consequently has a high average value which when you subtract it from the light leaves nothing except the star centres.

Did you take your darks and flat darks on the scope?  The best way to take darks and flat darks is to take the camera off the scope put its cap on and put in face down on a bench..  that way no light can get in and mess things up.  You'll need to figure out what exposures to take for your flat darks as you wont want to take the camera off to do them after you've taken your flats.   Once done they'll last a while (Mine are14 months old and still work perfectly) .  I've attached one of my 0 minute Darks so yo can see what it looks like stretched... unstretched its average value is 875 (16 bit basis) yours are 8300 … the Light has an average value of about 1500.

HTH

Dave

Mine

Yours

Yes thanks Dave. I did do them with  the camera attached to the scope. I did 50 frames with the cap on the scope  using sharpcap capture darks which automatically stacked them. Just as a precaution I re-did them next day as individual frames and that's what I've just run through pixinsight and uploaded with similar results. I guess I can still do the dark flats again as the orientation of camera and focus is only relevant to the flat frames and not the dark flats. Does that sound right? 

Many thanks

Craig

[Laurin Dave]

Yes if you re do the dark flats and apply them to your existing flat frames it  should be fine..  you'll then need to do another set of flats once the camera is back on.. 

Dave

[wimvb]

1 hour ago, Laurin Dave said:

The best way to take darks and flat darks is to take the camera off the scope put its cap on and put in face down on a bench.. 

And for good measure you can also wrap the front of the camera (with cap on) with double layers of aluminium foil. Plastic caps are not light proof, especially infra red light can make it through the plastic. 

[vlaiv]

There are couple of issues here.

One might be light leak. It looks like dark is suffering from it but it looks like flat dark is also (there is negative imprint of that pattern on master flat as well):

However that is not main issue with darks - master dark is wrongly created. If you have followed some tutorial there is a good chance that you missed a step or confused steps between flat and dark parts.

Range of values in master dark are 0.1 - 0.2 ADU and that simply cannot be for proper master - it looks like it has been scaled like when creating master flat. Same seems to be the case with flat darks  - as such a small bright patch could not make imprint on flats that are exposed properly, yet your master flat shows it clearly.

Master flat is also scaled and again - it's not scaled properly - it is in range 0.28 - 0.64 and it should be scaled so that max is around 1.0 value (in principle it does not matter what range flat is in, but if you are going to scale it, one would expect that scaling to be done to unity range - brightest part to be at 1.0 - or 100% light).

My recommendation would be to first redo everything that you already have to make sure that you did not mess up processing workflow. Start by creating master dark - create master dark and post it here together with single dark sub for inspection.

[Craig123]

13 hours ago, wimvb said:

And for good measure you can also wrap the front of the camera (with cap on) with double layers of aluminium foil. Plastic caps are not light proof, especially infra red light can make it through the plastic. 

I will do that thanks. A handy tip. 

I have a feeling alot of it could be the  manual filter wheel but I need to be more carefully all round about taking these frames 👍

[Craig123]

3 hours ago, vlaiv said:

There are couple of issues here.

One might be light leak. It looks like dark is suffering from it but it looks like flat dark is also (there is negative imprint of that pattern on master flat as well):

However that is not main issue with darks - master dark is wrongly created. If you have followed some tutorial there is a good chance that you missed a step or confused steps between flat and dark parts.

Range of values in master dark are 0.1 - 0.2 ADU and that simply cannot be for proper master - it looks like it has been scaled like when creating master flat. Same seems to be the case with flat darks  - as such a small bright patch could not make imprint on flats that are exposed properly, yet your master flat shows it clearly.

Master flat is also scaled and again - it's not scaled properly - it is in range 0.28 - 0.64 and it should be scaled so that max is around 1.0 value (in principle it does not matter what range flat is in, but if you are going to scale it, one would expect that scaling to be done to unity range - brightest part to be at 1.0 - or 100% light).

My recommendation would be to first redo everything that you already have to make sure that you did not mess up processing workflow. Start by creating master dark - create master dark and post it here together with single dark sub for inspection.

Thanks for looking at these Vlad. Much appreciated. When i get back I will have a look at the subs again. It makes sense what you are pointing out but I don't know anything about scaling.. 

I tried to come up with a workflow for flat darks while following the light vortex tutorials on Calibration and its obviously gone pear shaped. I will post up what I come up with. I believe some light will be getting in at least  via the manual filter wheel. 

Cheers 👍👍

[wimvb]

Just remember that with a CMOS camera, you should never scale darks. Leave that option unchecked in the calibration process. You can of course, create a dark library and do so on a cloudy night in a darkened room. Less risk of light leaks that way.

[Craig123]

22 hours ago, vlaiv said:

There are couple of issues here.

One might be light leak. It looks like dark is suffering from it but it looks like flat dark is also (there is negative imprint of that pattern on master flat as well):

 

BTW Vlad. I thought the light in the top left corner was the Ampglow ? It's on the lights. There is no sign (yet) of the star burst ampglow that I've seen in some posts. 

Craig

[Craig123]

16 hours ago, wimvb said:

Just remember that with a CMOS camera, you should never scale darks. Leave that option unchecked in the calibration process. You can of course, create a dark library and do so on a cloudy night in a darkened room. Less risk of light leaks that way.

Thanks. I don't remember seeing the scaling option but I will look out for it nxt time I load image calibration. 

[Craig123]

3 minutes ago, Craig123 said:

BTW Vlad. I  thought the light in the top left corner was the Ampglow ? It's on the lights. There is no sign (yet) of the star burst ampglow that I've seen in some posts. 

Craig

 

[vlaiv]

Just now, Craig123 said:

BTW Vlad. I thought the light in the top left corner was the Ampglow ? It's on the lights. There is no sign (yet) of the star burst ampglow that I've seen in some posts. 

Craig

There is some amp glow in your subs, but I don't think that all of top left corner is due to amp glow (could be wrong though - we need proper master dark to establish that).

You have image of what dark should look like above, and here is another example of it (my master dark really stretched):

Amp glow pattern in ASI1600 goes like that - two sections at right side (top/bottom) and a bit less to the left side rather "undefined" in shape following edges.

Here is your light frame very stretched to show "features":

Now, I marked what could be a light leak, but could also be some sort of gradient from LP? Dark sub matches position of bright spot, so that reinforces probability of it being light leak, but it's not matching "shape" fully to rule out any other explanation.

[Craig123]

10 minutes ago, vlaiv said:

There is some amp glow in your subs, but I don't think that all of top left corner is due to amp glow (could be wrong though - we need proper master dark to establish that).

You have image of what dark should look like above, and here is another example of it (my master dark really stretched):

Amp glow pattern in ASI1600 goes like that - two sections at right side (top/bottom) and a bit less to the left side rather "undefined" in shape following edges.

Here is your light frame very stretched to show "features":

Now, I marked what could be a light leak, but could also be some sort of gradient from LP? Dark sub matches position of bright spot, so that reinforces probability of it being light leak, but it's not matching "shape" fully to rule out any other explanation.

Thanks. That helps, alot. 

The light in the top left is worse in earlier  subs than in the later ones  from that night. I wondered if it was the laptop screen as i dimmed it right down later on but then  moved my scope inside for darks..so then  it was in a different position to the camera. 

[david_taurus83]

You dont need to leave the camera on the scope to take flat darks. Just take it off and shoot them the same time as your darks. If you have a metal cap for the camera, great. If only a plastic one, you could put tin foil on the inside before you push it on. Just remember, darks need to be the same exposure, gain and temperature as your lights. Flat darks need to be the same exposure, gain and optionally same temperature as your flats.

[vlaiv]

Any sort of light leak on scope needs to be dealt with regardless if darks are taken off scope or not - because it will impact lights and flats (to some extent, depends how strong light leak is).

[Craig123]

On 13/12/2019 at 09:13, vlaiv said:

Any sort of light leak on scope needs to be dealt with regardless if darks are taken off scope or not - because it will impact lights and flats (to some extent, depends how strong light leak is).

Hi Vlad, Hi all.. 

I had a go at IC405 the other night and tried to eliminate the light leak you have highlighted above.

I think it was the laptop near the manual filter wheel. Anyway I tried a few calibration frames  which I have added here if you have a moment to have a look and see if they are closer to what is required. They are single frames dark, flat dark, flat and light frames before I try and stack them.

Also is an automatic averaged stack of 50 darks using Sharpcap capture dark which I tried for comparison.

Seems like I'm getting closer but I didn't remove the camera from the scope yet incase I need to re do anything. The light panel I'm using isn't too bright (Gerd Nuemann which got slated on SGL the other day ) so the flats are  all the way up to 5 seconds which seems very long but the Histogram analysis for flats  in sharp cap says its acceptable and ADU is around 25000.

Many thanks

Craig

 

ic405ha.fit IC405darkframesingle.fit ic405flatsdarks5secsingle.fit ic405flats5sec_00006.fit autostackdark_50_frames_-20.0C_2019-12-19T21_53_38.fit

Edited December 20, 2019 by Craig123

[vlaiv]

21 minutes ago, Craig123 said:

Hi Vlad. Hope your well...

I had a go at IC405 the other night and tried to eliminate the light leak you have highlighted above.

I think it was the laptop near the manual filter wheel. Anyway I tried a few calibration frames  which I have added here if you have a moment to have a look and see if they are closer to what is required. They are single frames dark, flat dark, flat and light frames before I try and stack them.

Also is an automatic averaged stack of 50 darks using Sharpcap capture dark which I tried for comparison.

Seems like I'm getting closer but I didn't remove the camera from the scope yet incase I need to re do anything. The light panel I'm using isn't too bright (Gerd Nuemann which got slated on SGL the other day ) so the flats are  all the way up to 5 seconds which seems very long but the Histogram analysis in sharp cap says its acceptable and ADU is around 25000...I hope ??

Many thanks

Craig

 

ic405ha.fit 31.38 MB · 0 downloads IC405darkframesingle.fit 31.38 MB · 0 downloads ic405flatsdarks5secsingle.fit 31.38 MB · 0 downloads ic405flats5sec_00006.fit 31.38 MB · 0 downloads autostackdark_50_frames_-20.0C_2019-12-19T21_53_38.fit 31.38 MB · 0 downloads

Not sure what is going on with your subs, but here are my findings (or rather what I feel is not right):

1. There is massive gradient / light leak in dark sub - visible here:

2. I can't see details of file capture because these files are not straight out of the camera - their fits header shows that they were loaded in PixInsight and saved from there (can't tell if they have been manipulated in any way):

I have no idea what was set point temperature and what was actual temperature and don't know what gain and offset were used and if they match across the subs (lights and darks; flats and flat darks).

3. Mean value of dark sub is higher than mean value of light sub - this should not be so.

This can happen if:

- you have had your gain set on higher when taking darks, or maybe messed with offset (I don't believe it is offset because it can't make such a difference)

- you have a light leak that was worse when doing darks - you had some additional ambient light when doing darks or it was not fully dark yet

- subs were altered in some way

4. Flat and flat dark look ok as far as can tell, but I can't apply them to see if they work properly - dark calibration (first step in calibration) fails because darks are not suitable because of higher mean value (point 3).

5. Don't do sharpcap auto stacking of anything - it simply produces very different results:

How can you stack subs that have average value of ~5400 and get average of 1014?

If one stacks 30 subs - each being 16bit, you want result to be 32bit floating point - don't use sharpcap auto stacking as it produces 16bit result (and you have no idea what it did with the data)

6. It would be good if you could do following:

- if you are using sharpcap to capture your subs - use ASCOM driver instead of native driver (in case you are not already using it) - but do consider changing capture software as Sharpcap is better suited for planetary and fast video applications than it is to long exposure imaging

- try using files directly from capture software and not loading them in another application (pixinsight) and then saving them from there

- try doing darks with camera removed from scope, covered with plastic cap and set "face down" on thick wooden table (or plank or whatever you have), just to eliminate any possible light leak and see if you get proper darks. Just a few test darks will be enough.

[Craig123]

37 minutes ago, vlaiv said:

Not sure what is going on with your subs, but here are my findings (or rather what I feel is not right):

1. There is massive gradient / light leak in dark sub - visible here:

2. I can't see details of file capture because these files are not straight out of the camera - their fits header shows that they were loaded in PixInsight and saved from there (can't tell if they have been manipulated in any way):

I have no idea what was set point temperature and what was actual temperature and don't know what gain and offset were used and if they match across the subs (lights and darks; flats and flat darks).

3. Mean value of dark sub is higher than mean value of light sub - this should not be so.

This can happen if:

- you have had your gain set on higher when taking darks, or maybe messed with offset (I don't believe it is offset because it can't make such a difference)

- you have a light leak that was worse when doing darks - you had some additional ambient light when doing darks or it was not fully dark yet

- subs were altered in some way

4. Flat and flat dark look ok as far as can tell, but I can't apply them to see if they work properly - dark calibration (first step in calibration) fails because darks are not suitable because of higher mean value (point 3).

5. Don't do sharpcap auto stacking of anything - it simply produces very different results:

How can you stack subs that have average value of ~5400 and get average of 1014?

If one stacks 30 subs - each being 16bit, you want result to be 32bit floating point - don't use sharpcap auto stacking as it produces 16bit result (and you have no idea what it did with the data)

6. It would be good if you could do following:

- if you are using sharpcap to capture your subs - use ASCOM driver instead of native driver (in case you are not already using it) - but do consider changing capture software as Sharpcap is better suited for planetary and fast video applications than it is to long exposure imaging

- try using files directly from capture software and not loading them in another application (pixinsight) and then saving them from there

- try doing darks with camera removed from scope, covered with plastic cap and set "face down" on thick wooden table (or plank or whatever you have), just to eliminate any possible light leak and see if you get proper darks. Just a few test darks will be enough.maybe 

Mmmm.. Need to check dark subs as maybe light got switched on next door. That's all I can think. I need to remove camera for that but looks like flats worked so might be OK. The settings were all the same at minus 20.

[Craig123]

5 hours ago, vlaiv said:

Not sure what is going on with your subs, but here are my findings (or rather what I feel is not right):

1. There is massive gradient / light leak in dark sub - visible here:

2. I can't see details of file capture because these files are not straight out of the camera - their fits header shows that they were loaded in PixInsight and saved from there (can't tell if they have been manipulated in any way):

I have no idea what was set point temperature and what was actual temperature and don't know what gain and offset were used and if they match across the subs (lights and darks; flats and flat darks).

3. Mean value of dark sub is higher than mean value of light sub - this should not be so.

This can happen if:

- you have had your gain set on higher when taking darks, or maybe messed with offset (I don't believe it is offset because it can't make such a difference)

- you have a light leak that was worse when doing darks - you had some additional ambient light when doing darks or it was not fully dark yet

- subs were altered in some way

4. Flat and flat dark look ok as far as can tell, but I can't apply them to see if they work properly - dark calibration (first step in calibration) fails because darks are not suitable because of higher mean value (point 3).

5. Don't do sharpcap auto stacking of anything - it simply produces very different results:

How can you stack subs that have average value of ~5400 and get average of 1014?

If one stacks 30 subs - each being 16bit, you want result to be 32bit floating point - don't use sharpcap auto stacking as it produces 16bit result (and you have no idea what it did with the data)

6. It would be good if you could do following:

- if you are using sharpcap to capture your subs - use ASCOM driver instead of native driver (in case you are not already using it) - but do consider changing capture software as Sharpcap is better suited for planetary and fast video applications than it is to long exposure imaging

- try using files directly from capture software and not loading them in another application (pixinsight) and then saving them from there

- try doing darks with camera removed from scope, covered with plastic cap and set "face down" on thick wooden table (or plank or whatever you have), just to eliminate any possible light leak and see if you get proper darks. Just a few test darks will be enough.

Thanks for the info on this. Here is a 5 minute dark  i've just done with camera off the scope and wrapped in foil aswell ... first time i have tried this and it seems better.

darkoffscope_00002.fits darkoffscope_00005.fits

Edited December 20, 2019 by Craig123

[vlaiv]

5 hours ago, Craig123 said:

Thanks for the info on this. Here is a 5 minute dark  i've just done with camera off the scope and wrapped in foil aswell ... first time i have tried this and it seems better.

darkoffscope_00002.fits 31.26 MB · 1 download darkoffscope_00005.fits 31.26 MB · 0 downloads

Yes, this now looks much better - look at it being stretched:

Also note something:

That is comparison of mean values between your old flat dark and this new dark of 300s - one is 1022 and other is 1068 ADU. That is what you would expect from normal dark, very little change because dark current is low.

Now that you found that you have a light leak - that is good first step. You can do darks like this and that is guaranteed to make you good darks, but problem is that you still have light leak on your scope and your lights and flats will be polluted with this light leak. That is something you need to sort out if you want decent images.

 

[Craig123]

18 minutes ago, vlaiv said:

Yes, this now looks much better - look at it being stretched:

Also note something:

That is comparison of mean values between your old flat dark and this new dark of 300s - one is 1022 and other is 1068 ADU. That is what you would expect from normal dark, very little change because dark current is low.

Now that you found that you have a light leak - that is good first step. You can do darks like this and that is guaranteed to make you good darks, but problem is that you still have light leak on your scope and your lights and flats will be polluted with this light leak. That is something you need to sort out if you want decent images.

 

Well thanks for checking them over. I know what I need to do with the darks. One problem down.

Would you say that sub I uploaded of flaming star looked to be suffering from light leak. To me the gradient on the left  seems far less on this  last try than  was evident on the original jellyfish sub. 

If its still there I'm stumped as to why.. 

Many thanks 

[Craig123]

On 11/12/2019 at 22:30, Laurin Dave said:

I've had a look .. your flat looks ok (but read later) ... but your Dark shows evidence of a light leak and consequently has a high average value which when you subtract it from the light leaves nothing except the star centres.

Did you take your darks and flat darks on the scope?  The best way to take darks and flat darks is to take the camera off the scope put its cap on and put in face down on a bench..  that way no light can get in and mess things up.  You'll need to figure out what exposures to take for your flat darks as you wont want to take the camera off to do them after you've taken your flats.   Once done they'll last a while (Mine are14 months old and still work perfectly) .  I've attached one of my 10 minute Darks so yo can see what it looks like stretched... unstretched its average value is 875 (16 bit basis) yours are 8300 … the Light has an average value of about 1500.  I suspect your darkflats are also messed up so you'll need to re-do them... they look like they've worked as the average for the flat is so high but if done I the same way as the Darks they won't because they'll also suffer from the light leak. 

HTH

Dave

Mine

Yours

The camera off the scope with some foil on the  cap worked Dave. Thanks for the tip. 👍👍

I've done the darks.. Now need flat darks the  same way. The flats I've just done are 5 seconds . Does it matter if they are that long if the adu is in the right ballpark.. Adu 2590 on my last try but the light  panel is not adjustable. 

Many thanks

Craig 

[Laurin Dave]

Hi Craig 

not sure what scale you are using but on a 16bit basis (0-65000) the ADU needs to be about 30000..  5 seconds is ok. 
 

Dave

[Craig123]

Hi Dave

Thanks. It's 16 bit I think. Was just setting the readout options in PI yesterday. Maybe I need a better light panel to get it up to the right levels. Got a s/h gerd Nueman but they get less than glowing reviews on here. 

Craig

Edited December 21, 2019 by Craig123