Long Exp. v's loads of stacked subs?

[MartinB]

Here here!

[Psychobilly]

Yipppeee!!!!!.....

Most important when you do go out an"experiment" try and have some "structure" to it...

I like to use a "bi-section" method similar to that used in "fault" finding...

start at the extremes say 60s and 360s or longer if your mount tracking can take it... then try one around 3 mins... i know this isn't the middle but your going to need to record "something" so you cant start at 0 and if you sky background has already washed you out at 60s then your going to need to sort out some filtering....

Get to know your gear and your skies... the only way to do this is by getting out there and doing it...

I have quite a range of "test" that I perform on any new camera (DSLR) to enable me to decide how I am going to use it...

Peter....

Peter.....

[ngc2403]

120 minute single sub? or the stack of multiple 5minute subs? very important to deciding how all this theory applies in practice.

Bashing scientific theory is less productive then talking about the theory on an astronomy website. i beleive that this fine site has a imaging discusion part perhaps we could have this fireside talk there?

or if you would like another opinion of the subject matter then perhaps the CCDtech group might interest you or further a field is the yahoo CCD tech group home to some of the best CCD imagers around. they don't post every image but talk about the ways to improve image quality, something that seems not to happen here.

Martin, having also read most of Ron Wodaski's book on CCD's in astronomy, i know that it is more to do with the processing of the images not the source or science behind it. You say that he talks about the sub length being related to the sky glow, but i think that he saies that you should image passed that point because you can remove the sky glow signal later in post processing but you need the large signal to over come the combined signal and skyglow shot noise.

in the short web page site you linked there are some issues with the terms he is using to describe the processes involved how the noise changes during stacking. he doesn't go into it and it can be read as if the noise is added like the signal which is incorrect. He also talks about the read noise being gaussian and that the average of this gaussian distribution would be zero or that stacking cause the read noise to tend to zero, hence it would get removed from the image.

Roundycat, i never said that you subtracte the flats, darks and bias only that by properly using these frames to calibrate an image, (something that has been talked about alot recently on this form) with dark current, fixed pattern noise and offset noise all removed. you are left with only shot noise and read noise on each sub.

Question If people see this sort of talk as too must science then why do you question every statement about it? either you have read about it or you don't think its important so you have not bothered to learn it. i have no problem with people not wanting to know about the science behind everything, but to question the science and not want to talk about it is going to get us no where.

[RobH]

Hello all. This is how I do things based on the scienbce and practicality.

I judge my sub length by a couple of different parameters.

1. I know that too short a sub will be full of noise produced by the fact I'm using imperfect technology, so unless it's a pretty bright object (most of my targets aren't, I go for more than 1 minute.

2. I have pushed my guiding up to 20 minutes at one point, so I know I don't have too much trouble there, but most of the time the maximum sub length is dictated by this, and background sky glow.

3. How many subs do I want? If the subs are all very long, I won't get many to the hour, and stand to lose a lot more if I have to scrap a 20 minute sub, than I do if I scrap a 3 minute one.

I do want a lot of subs though, to give me a good S/N ratio.

Are there clouds and/or gusty conditions? These aren't good for long subs.

I find that these factors mean that I tend to work with subs that are between 3 minutes when shooting colour in 2x2 binned mode, and 10 minutes max.

This gives a good balance between the various factors above.

Cheers

Rob

[narrowbandpaul]

i posted about noise so that gary, who did express a lack of understanding, could learn more and improve his imaging.

i am a firm believer that the theoretical underpinning of CCD images is essential to good images. Sure, you can go out, shot an hour of subs, come back in, stack them, and then find that your image is full of noise.

With an understanding of the science behind noise, one can instantly pick out which form of noise is affecting their image, and consequently the reduction or removal of the noise can be accomplished.

The way I see it, there are several ways to improve your imaging.

One is to build a remote observatory in arizona, with great steady skies. This is both expensive and impractical.

another is to have a very good telescope, with quality optics. I doubt if any telescopes are the limiting factor in image quality.

The other is to understand the limitations imposed by our camera. Fundamental to this is the understanding and minimisation of noise. It is also important to understand to some level what influence the camera manufacturer has on the end product. They dont just put it in a box. Voltages need to be set, read noise minimised, clock timings aligned. a camera manufacturer can make a CCD imager fantastic or very bad.

The easisest way to improve imaging, according to the discussion above is to understand the CCD sensor.

Thats my view, and its part of the reason why I like CCD technology so much.

Yeah, the textbook, Scientific Charge Coupled Devices, is big. But I love it. The reason is simple. Its written by a guy who is in industry, not a lecturer. His style is good, and for a book which goes in to detail, it is very well explained.

Imaging and science do go hand in hand, there is a lot more technology involved in imaging than when observing.

And it is my view, and I feel strongly about this, that understanding the science and technology behind the CCD is crucial in maximising image quality.

Thats my opinion. Some may disagree, but thats life!

Paul

[bazza]

To be honest, I always get lost in the science aspect of this particular topic, but I can see that the maths behind it, is important.

My belief was/is that the optimum sub exposure time depended on several things:

The gain of one's camera

Read noise & Shot noise (I think the gain of the camera is part of the calculation for the read noise)

Sky flux - ie. How dark are your skies?

The calculator and instructions at the link Martin posted:

Welcome to CCDWare

seems to make sense to me, although, as I said, I sometimes get lost in the maths!

Of course, a non, anti-blooming camera would also limit your exposure time.

I probably haven't added anything constructive to this thread, but suffice to say that the calculator above works well for me.

Barry.

[Psychobilly]

Rob I like your style ... and no one can argue that you methods certainly produce the results...

Paul... you have chosen to get deeply involved with the technology and good for you...as this seems to be where your heading in future so go for it...

Some of you guys use really expensive kit so don't suffer from some the limits of those people imaging on a budget making do with the kit they have and trying to get the best value for money out of it and making the most of whatever skies they have at home...

I started with an ETX and an LPI then a DSI... soon found i was "mount" limited...

Sorted those out and then found i was LP limited so i got hold of some basic filters ..

I was then fortunate enough to be able to upgrade the scope to a CPC800 (alt az) and was able to start getting better tracking - all be it in alt/az.. so I made a basic wooden wedge... sub Lent went up so i needed better LP filters... which allowed much longer subs which needed better tracking so I made a better wedge and started autoguiding... the decided i needed a more rigid mount with better polar alignment so I designed and built a pier.... then i decide I needed protection from the elements...so built the obs... then I decide my cameras weren't sensitive enough to Ha so I bought a Canon and modded it... then i decide being older generation I needed a newer "less" noisy camera....

The point is decide what you want to achieve how much you are prepared to spend... and just get on with it...

If we all had mega bucks we'd all have the best kit available under the best skies avaiable.... and still be discussing how to make things better...

Gone along way from Gary's original post haven't we....

Peter....

I

[CELESCOPE]

no its great that we are getting a very broad view of this ,and its up to one to take on board all has been said , then do what you want from the info that has been passed on , some may take the tech route ,while others will take the practical route , what ever we decide can only improve what we do.

Rog

[gary1968]

It has gone all techie hasn't it? Its good to get this kind of advice from those in the know but most of the techie stuff has gone over my head. I think after a good few nights out with the kit and seeing what results I get might just make the topic of 'noise' a bit easier for me to understand, once I can actually see it in images.

I think this has been a great and very imformative thread and would like to thank all for their contribution.

Now, some clear skies please to get out and try to make all this advice work.........

Cheers,

Gary

[gary1968]

So, this will be my starting point then.............

Experiment to find the LX point at which the skyglow starts to infringe on the image, then, back of the exposure time a little to the point where it is not apparent. This will depend on the LP in the area and any filters used.

Take as many images at this setting as time / weather / equipment will allow and stack them in DSS.

I think, based on the advice given this will be the best starting point for me, yes?

Thanks again.

Gary

[Psychobilly]

provided your not seeing any significant trailing in the image then that sounds about right...

If your getting trailing and can turn up the "gain" on the camera then do that and shorten the subs...

Peter....

[roundycat]

OK, so maybe there is something wrong with my reading skills after all. First, a recent quote from ngc2403;

‘Roundycat, i never said that you subtracte the flats, darks and bias only that by properly using these frames to calibrate an image, (something that has been talked about alot recently on this form) with dark current, fixed pattern noise and offset noise all removed. you are left with only shot noise and read noise on each sub’.

This is what you actually said:

‘Stacking frames averages read noise to zero but this only helps improve the S/N after the darks, flats and most important of all for short exposures the bias frames have been subtrated off’. (See post #9 above).

To my way of thinking that is straightforward enough. The second quote above came first in the chronology and clearly says ‘darks, flats and bias have been subtrated (sic) off. This is what I said was wrong and it is wrong. One of the problems with a complex subject is that we must be certain of what has been written and what we have written. Any time you say something on a technical subject in an authoritative manner you will be held up and measured against it.

I omitted to mention when posting the Rosette images above that one was a single 5m sub and the other was a stack of 12 x 10m subs. The point I was trying to make is that if you are once-upon-a-time blessed with superb conditions a single sub can knock your socks off. If you want that sort of result all the time go and live in the Atacama desert.

A quote from Gary, he who started this off…….

‘It has gone all techie hasn't it? Its good to get this kind of advice from those in the know but most of the techie stuff has gone over my head. I think after a good few nights out with the kit and seeing what results I get might just make the topic of 'noise' a bit easier for me to understand, once I can actually see it in images’.

If I might be so bold Gary this is exactly what I meant when I said we all need to keep our feet on the ground and not get too tied up in academia as it often leads nowhere in a practical sense.

There has been talk here and elsewhere about taking flats, darks and bias by the hundred. I never take more than 30 darks or bias and never more than 20 flats and often only 10. I am prepared to put my work up for you to judge if this is good enough for what you are trying to achieve. Accordingly I have posted below a composite of five images of the Rosette, from a single frame which is noisy mainly because of uncertainty in the signal, and 3, 5, 7 and 10 frames combined using Maxim’s SD Mask. Put more frames in the stack and it will get even better, you set your own limit based on your own aesthetic requirements.

The answer to your original question by the way is still no! Aim at 3-8 minutes for extended objects shooting at f5 to f8 and you will not go far wrong. Just get plenty of them.

Dennis

[centroid]

A very interesting 'thread' and discussion indeed.

For what its worth, here's my 'take' on it.

Firstly, I'm a great believer in that "the proof of pudding, is in the eating". Having spent 20 years of a 40 year technical career, in an R&D environment, I found on many occasions, practice contradicted theory.

However, I'm not going to agree or disagree with any of the comments made in this 'thread', but simply state what my own practical experience has shown.

CDD imaging has a number of limitations, some related to equipment, and others to the degree of 'sky glow' at any particular location.

For simplicity, I will ignore telescopes and apertures, refer only to cameras and sky conditions.

Also, again for simplicity, I'm not going to get into 'shot' or 'read' noise, but only refer to 'dark current'.

With the camera, there are two significant factors, the sensitivity/quantum efficiency (sorry, I did want to avoid technical jargon) of the sensor, and the degree of 'dark current' generated in the image.

For the location, ignoring the 'seeing', sky glow/light pollution is a limiting factor on exposure length.

With some cameras, in my case the SXVF-H9C, 'dark current' is very low indeed, and I rarely if ever, bother with 'dark frame' subtraction.

Other cameras, may well not be as good in this respect, and as such the longer the exposure time, the more significant the presence of 'dark current' will be. This can of course be addressed in processing, but not at 'zero expense'. I'll explain later.

Sky Glow, is something over which we have no control, and either have to live with, or move to a less polluted location. Easier said than done of course.

So, with a CCD camera, having good sensitivity, low dark current characteristics, and a dark sky, which is best, long or short exposures?

From my point of view and experience, the long exposure wins every time.

We know that a 'long' (long without being extreme of course), exsposure, will show a better SNR in the image, than will a 'short' exposure. We also know that 'stacking' several 'short' exposures, will also improve the SNR in the final image. Albeit, in my experience not as good as an equivalent single long exposure image.

What, in this 'thread', we seem to be losing site of, is feint detail in the object being imaged.

Here, we need to understand, that the most information/ detail contained in an image, is in the orginal image as delivered by the camera.

Every thing we do to that original image, applying 'darks', 'flats', stretching, and filtering etc etc, destroys some of that original data.

Bear in mind, you cannot put into an image, what isn't there in the original.

If a short exposure, not matter how many, doesn't contain certain detail, then that detail is not going to appear by 'stacking' and processing.

I have taken subs, 12 or 15 min exposure time, and then again at 5 mins exposure time. Fine detail captured in the longer subs, cannot be seen in the shorter exposures. It just isn't there.

So, for me, the 'bottom line', is go for as long an exposure as your camera, sky conditions, and guiding will allow.

Without doubt, 4x15 min subs, will contain more information/detail, than will 12x5 min subs.

Yes, I know that this is dependent upon how bright on dim the object is, but as a general rule, it applies.

Dave

[CELESCOPE]

Spot on Dave , last night i gave mine a try , i always image at 6 7 or 8 mins, when switching to GALAXIESi noticed a lot of noise in the back drop. so last night i ran 4x12 min subs, conditions were the same as my previous nights ,

What difference those extra 4 0r 5 mins made, with just a very small amount of noise in my subs .

So its proved to me , i suffer ye not with LP like i thought , but lacking purely in exposure time , the next thing of course is to push just a wee bit more to see if i exceed those boundries, but for sure IN MY CASE, the longer exposure wins.

Rog:headbang:

[MartinB]

I think your plan is a good one Gary!

Dave, I obsess (on the few occasions I manage to get out) on skyglow and sub exposure length. Because I get so few imaging opportunities getting sub exposure time is critical otherwise I'm wasting time. My EM200 allows my to get an extremely accurate polar alignment very quickly and I can guide for ever. So a decision has to be made on sub length. I guess that you make your decisions on experience and intuition and I bet you get it about right. Personally I like to calculate exposures based on sky glow and chip characteristics. My exposure times vary greatly depending on the set up I'm using, the direction and dec of the target and the conditions on that night but with my QSI I aim for a skyglow of 500-600ADUs, for my old H9 it used to be 1500 ADUs (higher read noise). I have found that this works pretty well.

[ngc2403]

i don't think i need explain this in more detail than i have already,

as can be seen from several independent sources now, is that less longer subs totalling the same overall exposure time give better snr than more short subs.

Practical experience shows that at the extremes of this there are differences, example roundycat's images all have a good SNR and so longer subs would not show much more detail. however we have all seen images with too little signal and these are the ones had do benefit from longer subs.

Roundycat, you caught me, there i was drunk typing away at my pc, and when i said subtract off the frames i meant use them to subtract off the noise that each corresponds to, but i think you knew that right?

friendly advice is subject to the reader's opinion on what they want to do and what they think is relevant, also this thread has been read by more than just gary who i know is in the early stages of learning about imaging DSO, however has you have read rog has taken from this a new understanding and has got better images as a result.

What have your posts against my opinions done for this thread other than show a complete lack of respect for science.

if you would like a better description of the noise and the effects of stacking on calibrated images then i would point you in the direction of the book i mentioned before, otherwise i would suggest that you stop posting negatively about some harmless science background to the processes being described.

It only services to prove that you find my views on the subject, and the views of other who seek to get the most out of their imaging time by looking into the maths of it all, less important than your own.

As always willing to answer any questions on the topic as best i can

alasdair

[MartinB]

Alasdair, no one is trying to pick a fight with you, far from it. Certainly, I don't have a problem with people posting detailed scientific analyses, the last thing I want is for SGL to be dumbed down. My problem was/is your analysis, specifically that shot noise isn't removed by stacking. We have a difference of opinion, that is all.

I think there is plenty in here to answer Gary's original question but the question goes deep and it is quite appropriate to address these more techinical and scientific areas. Many people on SGL, myself included, work in scientific professions and are quite comfortable to delve deeper into things

[blinky]

Well I have to agree 100% with what Dave has said! In my (limited) experience longer subs are ALWAYS better than shorter ones. If not enough light/Photons have hit the CCD to triger a response then no matter how many subs you stack you wont get that detail. The longer the sub the more faint detail and the better the S/N ratio, this makes post processing much easier.

[RobH]

I agree with Dave.

In practice, I find that the longer the subs are, the better, all other things being equal.

If it was possible, I'd be shooting lots of 20-30 minute subs all the time.

Ally, I don't think anyone here is bashing the science. As Martin has pointed out, many of us are in, or have been in, technical professions, and most people with an interest in astroimaging are interested in science generally...I certainly am.

Hard fact can't be argued with, but there are also many other factors are involved in this less than perfect world.

Cheers

Rob

[gary1968]

Billy,

I have not found the 'gain' control on the software that came with the camera, it is pretty basic as far as I can see. (Maxim DL Essentials)

I have just installed Maxim DL 5 and am working my way through it to see if there is control on there for 'gain', it does seem to be more complex than 'Essentials' It might be that I will have to use some other software to control the camera.

I will be working on polar alignment next time the weather clears. Up till now all I did was center polaris in the ep and then carry out a 2 star alignment from there, which has been fine for visual and the basic imaging I was doing up till now, I will be using the polar alignment routine in the mount and drift aligning with WCS from there. So hopefully I will be able to get much longer subs than before.

This thread has indeed been very informative and as the threads are stored for a while I will no doubt be going back to it for info as time passes and my experiance increases.

Thanks to all who posted, and please 'nae fighting guys' :-)

Gary

[MartinB]

Here is an interesting paper. The author shows the basis for calculating the minimum exposure time required to overwhelm the CCD readout noise by the sky glow. This has been the traditional "ideal" sub exposure time to aim for. He then compares the effect on SNR of using shorter subs and looks at how much longer overall exposure time is required.

He makes a good argument for using half of the traditionally used optimum exposure time. This is very dependent on suppressing the noise using dithering. Principle advantages of this technique are better statistical data rejection during stacking, less subs lost to random adverse events such as the odd passing cloud or gusts of wind, and less demands on the tracking set up.

Very interesting stuff.

http://hiddenloft.com/notes/SubExposures.pdf

[ngc2403]

Martin, Rob, was not referring to either off you. i am all for expressing opinions and sharing knowledge and for the most part that is what happens on the site with great success.

i feel that the posts of the few are dragging down the image of the site. anyway the less said about that the better.

Martin, that is a nice paper on noise. i can see where you are coming from with the read noise and sky background stuff.

I believe that the amount of read noise that a camera has to have to mean that it is better to take shorter frames of and object is not the norm for DSO imagers.

My reason is that modern cameras even DSLR's have very low read noise compared to just a few years ago and because read noise reduces by the root of the number of images, you might find the random noise (read noise) is small compared to the shot noise is a stack of anymore than five images.

Also as the camera exposes for longer the limiting noise changes, short exposure it is read, then shot then fixed pattern, however if you go really long and then calibrate the images to remove the dark current fixed pattern noise and the bias then you get to the point were you are limited by the sky shot noise. this is the limiting magnitude of the imaging system and subject to lots of external factors.

i can't say for sure, but cause it would have to go away and work through the maths but what is in that paper is a more focussed version of what is in the Janesick book, in that his is more general.

Oh and a common misconception, so i was told it that the gain has no effect the SNR for a fixed exposure time?

[centroid]

Ok, I'm sure that we have now filled Gary's head with so much info, that his head is probably 'spinning' a bit now .

I think what you can draw from this Gary, is that there isn't a 'one answer suits all' to your original question.

How an invidual goes about his image aquisition, is very dependent on his or her, situation. By this I mean equipment and sky conditions.

What works very well for me in my relatively dark rural location, won't work for say Martin, in his somewhat light polluted location.

Its all down to achieving the best you can, with what you have available to you.

Theory is good, and is the basis of all things technical. Without the theory, we wouldn't have the technology. However, theory often needs to be balanced against practice. Sometimes the two concur, and at other times, they will contradict. Why?, because theory is based on the 'perfect case', where all the parameters are precisely defined and met.

In the 'real world', the are so many variations to these parameters, and the solution often becomes a mixture of theory and practice.

In my career, I met and worked with a great many 'well read' people (Master's, and PhDs). Some of these people were blessed with a superb understanding of the theory, while also being able to put it into practice. Others unfortunatley, fell 'flat on their face', when it came to the practical application of their extensive theoretical knowledge. Why?, because they coudn't see beyond the pure theoretical case.

Understanding some of theory, helps with its practical application, as you will have 'more of a feel' for what you are trying to achieve, and a better understanding of why things aren't as good as you had expected.

There is however, no substitute for experimentation, and from this, finding what works best for you. This, I think, is the real answer to your question.

Finally, I do despair when someone like Gary, asks a question, which then leads people getting 'up tight' because someone disagrees with their view.

Different views on a subject, are all part of healthy discussion, from which we all learn. Criticising someones comments/input, or being pedantic, can lead to a 'tit for tat' situation, which serves no purpose whatsoever.

Challenging someones input, based on sound reasoning, is good. Provided it is done in the right manner. This is NOT the same as being critical or pedantic.

So, come on guys, cut the 'point scoring', and keep on making your valued inputs to this great resource.

Dave

[narrowbandpaul]

when I read that Mr QuiHY stated that SNR varied with gain, I almost had a heart attack.

The only thing that determines SNR is the number of electrons in the pixel. This situation occurs before the ADC comes in.

so yes ally, gain is a misconception.

I checked with rdc...he agreed.

paul

[ngc2403]

nicely summed up Dave,

i think that in the end a good discussion was had on the topic though, and perhaps it should be extended in the future but for now i think that what has been posted is more than enough to keep everyone reading for quiet a while.

my apologies to SGL for perhaps lowering the opinion of the site in eyes of guess or new members, over the tit for tat comments.

ally