Crescent Nebula DSLR Woes

[TiffsAndAstro]

Just now, alacant said:

?

The method is given in the link I sent. It builds in under a minute. On Linux at least.

The idea of open source is that we all get involved. Report back, make suggestions, test new stuff, have a say...

Cheers 

I don't have Linux installed to run it on even if I did manage to compile it. 

Guess I'll wait for a win64 release.

Thanks for your help though

 

[alacant]

9 minutes ago, TiffsAndAstro said:

don't have Linux installed

?

You can build it on whatever platform you like. There are instructions for Linux, Mac and Windows in the link I provided.

 

[TiffsAndAstro]

4 minutes ago, alacant said:

?

You can build it on whatever platform you like. There are instructions for Linux, Mac and Windows in the link I provided.

 

Will give it a go ty

[imakebeer]

Hello, me again... 🤭

@alacant (or anyone else for that matter) can you offer any suggestions why my flats seem not to be working for M31.....

The Crescent Nebula is sorted now - thank you again everyone for your help. But the other night I imaged:

• Veil Nebula NGC6992
• Andromeda Galaxy M31 (2 panel mosaic)
• California Nebula NGC1499

The Veil Nebula worked out fine, Cali Nebula is still in progress, but on M31 it seems like my flats haven't worked again ☹️ The processed images have very obvious dust motes which plagued my previous images of the Crescent Nebula until we solved the issue.

• I've used the same set of flats for all targets since nothing changed between them
• I took 20 x 3s flats, then also 20 x 5s. In both cases the "level" (whatever that statistic Siril gives is) is about 4000-5000, i.e. well below the ~8000 we discussed for half well depth.
• Issue is present with both sets of flats, but seems worse with the 5s ones

Any ideas?

M31 with 5s flats

Stats for a typical 5s flat

Edited October 8 by imakebeer

[TiffsAndAstro]

To my noob eyes looks perfectly fine. It might need more hours on it though.

.my latest m31 is 4.3hrs panel1 and 2 hours panel 2 and I think I need 12 hours on both panels

Edited October 8 by TiffsAndAstro

[alacant]

Post -links to- a sample each light and flat frames unprocessed.

[imakebeer]

59 minutes ago, alacant said:

Post -links to- a sample each light and flat frames unprocessed.

Thanks in advance dude:

• 3 x lights of M31 - https://drive.google.com/file/d/1-H0yU6_BEnHInaSsNt3stXHrz43FuXVe/view?usp=sharing
• 3 x flats @ 3s exposure - https://drive.google.com/file/d/1-Cf8jh_l0xRcDNGXy3HgNN9pL73Gc6vG/view?usp=sharing
• 3 x flats @ 5s exposure - https://drive.google.com/file/d/1-ER6TmXu5YO4R9_h4x9hVJFId4ABuK_2/view?usp=sharing

[alacant]

4 hours ago, imakebeer said:

ideas?

The flats are overcorrecting. That usually means the offset is wrong and/or the flat frames were taken under a differing optical train. A filter perhaps?
Please post a link to a frame taken in darkness with an exposure of 1/4000s

Edited October 8 by alacant

[imakebeer]

11 hours ago, alacant said:

The flats are overcorrecting. That usually means the offset is wrong and/or the flat frames were taken under a differing optical train. A filter perhaps?
Please post a link to a frame taken in darkness with an exposure of 1/4000s

3 x "bias" frames @ 1/4000s exposure - https://drive.google.com/file/d/1-qH-pOViRufmsaWUsvPDqh4s-H5Xzyby/view?usp=sharing

The optical train literally hasn't moved since I took my successful images of the Crescent Nebula - so yes there is a filter in there (Optolong L-eNhance) but I didn't add it specifically for this session. So no new lenses or filters, didn't even rotate the camera. The only thing that could have changed between sessions was the focus, and that was just a slight tweak between sessions to check it was still in focus.

TIA 👍

Edit: Is the offset the sigma value shown here? If so, close, but not exactly equal to the 1024 I'm using (and doesn't seem to vary between frames) 🤔

Edited October 9 by imakebeer

[alacant]

1 hour ago, imakebeer said:

Is the offset the sigma value

No. The offset according to your dark frames is around 4100, which is why they do not correct at 1024 (the value hard coded by Canon):

And as we can see, the flats work fine with an offset of 4100:

Which stacks to a respecatable image, even with only three frames.

As there's no way to set this on a 450d, then the sensor must have developed some extra noise in the form of hot pixels.
Was this taken in a hot climate? Any other clues? How are you powering the camera? The only time I can recall anything similar was on hot nights where the sensor was showing ~35º, when recording to an in-camera sd card or with a dummy battery and poor quality power supply. Or a combination of all the above.

I know you like explanations, but I'm struggling with this one. A solution I can give. An explanation... Still thinking. This is astrophotography afterall!

Cheers and HTH

Edited October 9 by alacant

[imakebeer]

5 hours ago, alacant said:

As there's no way to set this on a 450d, then the sensor must have developed some extra noise in the form of hot pixels.
Was this taken in a hot climate? Any other clues? How are you powering the camera? The only time I can recall anything similar was on hot nights where the sensor was showing ~35º, when recording to an in-camera sd card or with a dummy battery and poor quality power supply. Or a combination of all the above.

Many thanks @alacant - some thoughts/comments on the above..... (sorry, bit long, but please bear with me...)

• OK, so in the statistics from Siril the "Mean" value is the offset, right?
• I'm not imaging in a hot climate - South East of UK which is mild to cold at this time of year. I recall that night was fairly mild.
• I'm powering the camera from a mains power supply with a dummy batttery, but I've always used that with this camera.
• I'm not recording to the in-camera SD-card, the images go straight to a USB plugged into my ASIAIR Mini (could be a clue here.....)
• I took a break from AP between May 2023-August 2024. The camera was stored on the scope in the garage during this time, which can get warm in the summer and cold in the winter. Possible damage here?
• I had the camera modified in March 2023.
• But I went back to the bias frames (1/4000s) captured in April & May 2023 (i.e. after modification) - loading these into Siril they show a "Mean" value of around 1024, exactly as expected.
• This was long before I had an ASIAIR - back then I was using BackyardEOS to control the camera (using the same power supply and data cable as now)
• Lately I haven't been taking bias frames, not since you told me about this 1024 offset instead - but I did take some before I knew about this, with my first session with the ASIAIR in Auguest 2024. And the Siril statistics of those also show a "Mean" of about 4100!

So now I'm wondering either:

1. Did the camera get damaged somehow while not in use for ~18 months?
2. Is the ASIAIR doing something to the files? I note that the ASIAIR always gives me *.fit files whereas when I was using BackyardEOS I always got Canon's native RAW *.cr2 files. Could it be the ASIAIR messing something up when it converts the files??? Easy enough to check this!

6 hours ago, alacant said:

I know you like explanations, but I'm struggling with this one. A solution I can give. An explanation... Still thinking. This is astrophotography afterall!

Hahaha - you're absolutely right! 🤣 But I can work with a solution for now, and think about explanations in the meantime. Thank you so much for your continued help, I really do appreciate it 👍

[imakebeer]

OK, more thoughts about the ASIAIR and offset after a bit of Googling.....

• From CN - "ZWO knows what offset works for it's cmos cameras (same with color balance), but for DSLR's maybe it just uses a fixed 'safe' value"
• From a ZWO tecchie on their forum - "Offset is not visible in ASIAIR in order to make things easy as our slogan says "as easy as 1,2,3" "
• And from right here on SGL - "IIRC others have commented that ASIAir does not provide a user-settable offset."

I bet ZWO have hardcoded into the ASIAIR an offset of 4096 (approx. = 4100!!!) when they convert DSLR RAW files to *.fit (or something like that) haven't they!

So maybe this wouldn't matter if I was using real bias frames but it will impact of using a constant "1024" offset in Siril.

[Elp]

You can't set the offset in the asiair, never have been able to. When I used a 600D with one I used @alacants advice of using the fixed bias in Siril, worked as expected. Of more note was the control of taking flats consistently in the first place, since Ive used a very dimmed LED tracer panel and diffused with perspex sheets and the auto exposure flats in autorun mode on the air, no issues with flats since. I believe a DSLR body needs to be in AV mode when you take the flats.

Edited October 9 by Elp

[alacant]

2 hours ago, imakebeer said:

maybe this wouldn't matter if I was using real bias frames but it will impact of using a constant "1024" offset in Siril.

Doesn't follow.

If it's the air messing up, then it would be consistently messing up, whereupon the synthetic offset should be set to 4100 each time. But it isn't as you have one set of flats which work with 1024 and another set which work with 4100. 

Your first set of flats were over exposed. The two sets you posted more recently consist of varying light levels frame to frame. That needs addressing as a priority. .

Next, I'd go with new everything, replacing one item at a time until a dark frame gives you an offset of 1024. Each time, every time. Perhaps begin with the PSU to the dummy; just power it with an eos battery of known functionality.

Camera-wise I have a 450d on a mini pc under Ubuntu and can report that INDI (as pirated by zwo) consistently gives an offset of around 1024.  Borrow another eos and check the offset -old models 1024, 2048 from (I think) the 60/600 18mp series- on the air.

Edited October 9 by alacant

[imakebeer]

3 hours ago, alacant said:

But it isn't as you have one set of flats which work with 1024 and another set which work with 4100.

Umm... no, that's not quite right. In 1 night I imaged 3 targets, so 3 sets of light frames. But I only took 1 set of flats, i.e. use the same set of flats for all 3 targets. Nothing changed on the camera or imaging train, no change in focus, so as far as I'm aware 1 set of flats should be OK here - right?

But what's odd is I could process The Veil Nebula fine with an offset of 1024. But when I tried the same with M31 I got dust spots - I'm reprocessing now with an offset of 4096 but I'm not totally convinced it's working right. I think the dust spots are gone but there still seems to be banding and noise (or something), even after removing banding in Siril and then de-noising in GraXpert.

 

But in other news..... I've been experimenting taking bias frames at 1/4000s:

• Using just the DSLR (i.e. ASIAIR powered off) I got 5 x Canon RAW *.CR2 files - they all have a "Mean" of about 1024
• Repeat with ASIAIR on but also switch on option to save files direct to DSLR SD card - very interesting, the FIT files from the ASIAIR all have a mean of about 4127 whereas the corresponding *.CR2 files from the SD card are all about 1024.

Interesting....

Edit: Further observations:

The Canon vs. ASIAIR bias frames are different "brightness" (different offset?), but when viewed in Siril using "Histogram" mode they are near as dammit identical with just two key distinctions (in addition to the brightness above):

1. The ASIAIR FIT files seem to be mirrored about the horizontal axis. Flip it round and then they match with the original Canon RAW files
2. The ASIAIR FIT files have a mysterious line of white pixels along the top of the image (or bottom if you flip it). I think it might just be a line 1 pixel high, but this is not present in the original Canon files.

I'm curious now to see how Canon vs. ASIAIR flat frames compare.....

Edited October 9 by imakebeer

[alacant]

23 minutes ago, imakebeer said:

In 1 night I imaged 3 targets,

I have two sets of flats. One set as in this thread and one set from the crescent nebula. Neither worked.

Assuming the camera is ok, and as I have consistent correct calibration under a pc running INDI using the same camera, I can only conclude that the inconsistency you are seeing is due to the asiair. I will not have use of an air until the weekend.

A systematic approach based upon elimination I think is the only way forward. 

You are close to cracking it. I'm certain.

 

[imakebeer]

12 minutes ago, alacant said:

I can only conclude that the inconsistency you are seeing is due to the asiair. I will not have use of an air until the weekend.

A systematic approach based upon elimination I think is the only way forward. 

I agree, and my experiemnt above suggests this may well be the case.

If you happen to have the opportunity at the weekend to repeat my experiment I'd be interested in the outcome just to be sure I'm not being a total bonehead, i.e. select the option on the ASIAIR to write files to both the SD card and the Air, then compare...

[catburglar]

Perhaps the ASI air is scaling the 14bit raw files into a 16bit fit - explains why the offset is multiplied by 4…Doesn’t explain why the flats correct some images and not others.

[alacant]

14 minutes ago, catburglar said:

air is scaling the 14bit raw files

... But only sometimes?

Anyway...

 

[TiffsAndAstro]

How new is your asi air? I'm assuming it's updated to latest version of it's software? 

[imakebeer]

36 minutes ago, catburglar said:

Perhaps the ASI air is scaling the 14bit raw files into a 16bit fit - explains why the offset is multiplied by 4…Doesn’t explain why the flats correct some images and not others.

Yes I think it may well be. I think Siril is doing same/similar too.

I noticed after my previous post that the FIT files from both the ASIAIR, and when Siril converts CR2 to FIT, the FIT files have roughly 1.5x as many pixels in each direction compared with the original RAW CR2 files. 

23 minutes ago, alacant said:

... But only sometimes?

Anyway...

 

Maybe it's always doing it, just that for some reason with the Crescent Nebula it wasn't such a big deal??m

15 minutes ago, TiffsAndAstro said:

How new is your asi air? I'm assuming it's updated to latest version of it's software? 

Yep, up to date - TBH it seems it won't let you proceed unless you agree to install the latest update every time one is released!

[Elp]

2 hours ago, imakebeer said:

In 1 night I imaged 3 targets, so 3 sets of light frames. But I only took 1 set of flats, i.e. use the same set of flats for all 3 targets. Nothing changed on the camera or imaging train, no change in focus, so as far as I'm aware 1 set of flats should be OK here - right?

But your camera temperature is changing throughout the night in between imaging different targets.

Just an observation, I've only ever imaged one target a night with a DSLR, so flats taken at the end of the session normally worked.

But as I've mentioned before, with experience your flats should all come out with the same general exposure time if you use the auto exposure mode on the air (EG my OSC with a luminence filter normally come out between 1-3s or so each, my mono narrowband usually are always 10s long, I can achieve this because my panel is always set at the same brightness and the same number of diffusing materials are always the same, IE the conditions when taking the flats are repeatable).

 

[imakebeer]

13 hours ago, Elp said:

IE the conditions when taking the flats are repeatable

I think you're right, I think there is value in making flat production more rigourous and repeatable. My current method of a white T-shirt and a grey wall in a room with varying ambient light is not going to be ideal in this regard. Now where is my credit card.....

 

14 hours ago, catburglar said:

Perhaps the ASI air is scaling the 14bit raw files into a 16bit fit - explains why the offset is multiplied by 4…Doesn’t explain why the flats correct some images and not others.

I've been pondering this some more. I think you're right, I think both the ASIAIR and Siril are scaling the raw files to 16-bit. I'm going to re-process my decent-ish images of both the Crescent and Veil nebulas with an offset of 4096 and see what comes of it - when I processed them initially with an offset of 1024 maybe I just got lucky but for some reason this luck doesn't extend as far as M31???

[TiffsAndAstro]

1 hour ago, imakebeer said:

I think you're right, I think there is value in making flat production more rigourous and repeatable. My current method of a white T-shirt and a grey wall in a room with varying ambient light is not going to be ideal in this regard. Now where is my credit card.....

 

I've been pondering this some more. I think you're right, I think both the ASIAIR and Siril are scaling the raw files to 16-bit. I'm going to re-process my decent-ish images of both the Crescent and Veil nebulas with an offset of 4096 and see what comes of it - when I processed them initially with an offset of 1024 maybe I just got lucky but for some reason this luck doesn't extend as far as M31???

No need for credit card for flats.

Rubber band, scrap of white t-shirt cloth and a tablet and Nina flat wizard.

Use flats wizard to point at zenith. Apply t-shirt and rubber band. Two layers of t-shirt might be required.

Plonk tablet showing a white screen on top. 

Press play in wizard.

If all goes well great. If not see what Nina says and increase panel brightness or remove one layer of t-shirt. Remove tablet before parking

I've just ordered a NASA spec A4 tracing panel powered by 5v usb for £10 just to try as my tablet is on its last legs

[Elp]

The issue with using screens is that their colour balance or natural tone and even refresh rate may cause issues, some don't even display a uniform white from edge to edge.