han59

8 hours ago, Rustang said:

Thanks, I need a little clarification, the ASTAP went into windows c:\programe files then the database downloaded as a zip folder in the downloads folder. Where do you then un zip that folder so that it goes into the ASTAP programe directory properly? 

Rustang,

Steve has nicely show what should be in the program directory. The star database can be downloaded as installer (g17_star_database_mag17.exe) or zipped. The zip file you have to unpack but Windows doesn't allow that directly in the c:\program files\astap  So you have to unpack it somewhere else and then move the 290 files to c:\program files\astap   Windows will protest once again but you can go ahead accepting the risk. Easiest is probably to delete the zipped file and download the star database installer and run that installer which will do al the work for you.

 

Steve,

Each of the database files contains a section of the the sky. You probably could skip the files for below declination -40 degrees but when running a blind solve ASTAP could complain about it and exit with a run time error. I have not limited the search based on latitude of the FITS file. Maybe I should but for a successful solve you will never go beyond -40 degrees declination assuming you live around 50 degrees north.

Han

 

The star database file is an installer. So you if you execute it it will unpack 290 files in your ASTAP program directory. Assuming you have Windows, in c:\program files\astap you should have:

astap.exe

*.290 files

 

Han

Yes Windows has become much more restrictive to programs trying to control the Windows colours. For a computer operated in the dark, you could either use a different Windows theme or try the Windows night colours.

I personally have moved to a red or grey layer film on the screen but newer displays are probably better in dimming to a low intensity.

Han

 

 

>>But how do you take into account duration of the flat and mean flat signal level?

The flat exposure was missing. But it has no real influence. I have added it.  See attached spreadsheet.

>If you for example have camera with 7k pixel well and you have another camera with 57k full well capacity.

>Both of these have their flats taken at 75% histogram peak.

>Single flat of first camera will have SNR of ~72.45, while single flat of second camera will have SNR of ~206.76.

>Scale both flats so that signal is 1, noise will be very small.  In case of first camera, although bigger, noise will be ~0.0138.

That done with the input B8, enter here 12, 14 bit. With this value the number of electrons is calculated.   75% * 65535/2^(16-nrbits)

>This value is very small, but how does it impact SNR of image?

The spreadsheet should give the answer.

>Let's say that signal in the image is 1e and it is perfect signal - no noise. We need to correct it with above flat. We will have:

>1 / 1 +- 0.0138 => 1/0.9862 to 1/1.0138 => 1.014 to 0.9864

>This is 0.014 above 1 and 0.0136 below 1. I'm showing you this because this is no longer symmetrical distribution - it is no longer Gaussian and you can't add it like Gaussian (or Poisson) >distributions if you are not sure it will work like that.

?? If you add an AC (alternating) signal to a DC signal the signal shape will not change.

------------------------------

There is an other problem with the simulation is that combining flats doesnt help after about 50 flats. The noise doesn't go done anymore.

Standard deviation as function of the number of combined flats:

 

 

SNR-Calculator 2020-8-12.zip

It was late and I made a typing error, I will correct. The summation sign could be confusing since you averaging the flats to the same level as one flat. It should be:

noise_flats := (flats_noise^2 + biases_noise^2)^0.5

 

The vignetting was not take into account. The noise level is for the center of the image.

First step in modeling is for lights and dark frames:

signal_lights := (1/n)*Σ (Lights- masterdark)

noise_lights^2 := (Σ(light_noise^2 + dark_noise^2))^2

noise_lights := (lights_noise^2 + darks_noise^2)^0.5

----------------------------------------------------------

same for master flat and bias frames

signal_flats := (1/n)*Σ (flats - masterbias)

noise_flats := (flats_noise^2 + biases_noise^2)^0.5

---------------------------------------------------------

The flat is normalized to 1 and applied as divisor to the signal_lights

total_signal:=signal_lights /  signal_flats

 

For the noise without normalisation to 1:

total_noise^2 := (noise_lights^2 + (noise_flats * signal_lights/signal_flats)^2)/signal_flats^2

or since the flat is normalized to 1

total_noise^2 := noise_lights^2 + (noise_flats* signal_lights)^2

 

No further binning of blur of the flats was assumed.

-----------------------------------------------------------

 

 

I have updated the spreadsheet calculation with flats and bias frames.

The input parameters are the camera noise parameters, binning, number of darks, flats, bias frames, filter bandwidth and sky SQM value

If the model is correct, only a few darks are required but the number of flats should be near the number of lights.

Han

Attachment is updated. See later posting.

The simulation is ready.

The influence of the darks seems almost steady. The noise from the master dark goes further down due to the dithering effect. So the master dark is each time shifted while stacking resulting in a lower noise addition then would be expected from the master dark itself.  The dithering seems to reduce the master dark noise with a factor (2/nr_lights)^0.5

• For imaging with no filtering only one darks seems sufficient.
• For imaging with a H-alpha filter only a few darks seem required.

 

It is possible to enter the SQM value and object  brightness to calculate the SNR value

Han

Attachment updated. See next post.

The simulation is almost ready. But I noticed an interesting phenomena. Even using a single dark and many lights the final noise level is much lower then a single dark. This must be caused by the dithering. The noise in the single dark is averaged out while stacking. It is like a Gaussian blur is applied.

Yes it is CPU intensive:

• Keep the number of pixels low something like 1280x960 pixels. (disconnect the ASCOM "sky camera"  first before changing)
• The artificial sky option works often faster then the deepsky images downloads.
• Keep the exposure time at few seconds or more.

Han

Using new dark images taken last night, the dark noise values measured and calculated from the dark current match very accurately. See below. So I assume the math is now good and I can make the spreadsheet to calculate the influence of the darks noise on the final image.

Han

 

New data ASI1600MM-Cool:

dark, bin 1x1, 2 sec -15 Celsius, mean value 272 ADU, σ = 26.9 ADU

dark, bin 1x1,  600 sec, -15 Celsius,  mean value is 357 ADU,  σ = 45.3 ADU

 

The σ was measured by subtracting two darks of the same exposure divided by sqrt(2)

A exposure of 2 seconds was taken as bias since the mean values can drift  in the first second(s)

 

Using the mean values:

The dark signal increases with 357-272 is 85 ADU or 85/16 is 5.31 electron. The dark current is then 5.31 electrons/600sec is 0.00885 electrons/sec.

The noise calculated is then sqrt(5.31 e)= 2.3 electrons or 2.3*16 is 36.7 ADU

(dark noise follows a Poisson relationship to dark current, and is equivalent to the square-root of the number of thermal electrons generated within the image exposure time.)

 

Using the measured standard deviation values:

The measured noise should be

totalnoise^2 := readnoise^2 + darknoise^2

45.3^2 : = 26.9^2 + darknoise^2 =>  the darknoise measured is then 36.4 ADU. 

Software binning for me is averaging both in drivers and software. You do the same for master darks and flats otherwise the levels are wrong.

--------------------------------------------------------------------------------------------------------------------------------------------------------------

Yes there should distinction between dark current and dark noise. My measurement where noise measurements. Lets do it again:

Data:

2x2 bin, bias -15 Celsius, mean value 304 ADU, σ = 17.5 ADU

2x2 dark 200 sec, -15 Celsius,  mean value is 320 ADU,  σ = 21.9 ADU

 

Using the mean values:

The dark signal increases with 320-304 is 16 ADU or 16/16 is 1 electron. This also valid for a single pixel, so the dark current is then  1 electrons /200sec is 0.005 electrons/sec.

The noise is then sqrt(1 e)= 1 electrons or 16 ADU. For bin 2x2 the noise should be half or 8 ADU

 

Using the measured standard deviation values:

The measured noise should be 21.9^2=17.5^2+darknoise^2 =>  the darknoise measured is 16 ADU. 

 

 

The measured noise 16 ADU seems higher then predicted 8 ADU.  Random telegraph noise? It could also be the sudden increase of the mean in the first seconds as reported here. That would indicate that the delta mean is 1 electron too low.

I will do some checks at bin 1x1 when it is colder. At the moment I can't reach -15 Celsius.

------------------------------------------------------------------------------------------------------------------------------------------------------------------

Quote

it will be much higher than master dark resulting noise.

The idea is to calculate it. It seems to matter for H-alpha images.

Quote

Your reasoning is somewhat flawed here.

I don't see your point. What you write is the same as what I write.

 

Quote

I would expect 2x2 binned from 0 seconds exposure if bias signal is removed to have about 3.4 as a result at unity gain (if we take 1.7e to be read noise per pixel).

As you bin the noise reduces.  So 4 pixels give combined 1.7e/sqrt(4)=0.85e noise. In the binning the 4 pixel values are added together and then divided by 4. I just did as test with the ASI1600 at 0 Celsius:

bias unity gain at 0 Celsius, 1x1 binned - master bias=>  σ = 28 ADU measured or 28/16= 1.75 electron

bias unity gain at 0 Celsius, 2x2 binned - master bias=>  σ = 14 ADU measured or 14/16= 0.88 electron

or your method:

bias 1x1 - bias 1x1 =>  σ = 40 measured, so 40/sqrt(2)=28 ADU

bias 2x2 - bias 2x2 =>  σ = 20 measured, so 20/sqrt(2)=14 ADU

Quote

 

dark_current_200 = (15^2 - 13^2)/200 = 56/200 = 0.28ADU/s/px

dark current_400 = (18^2 - 13^2)/400 = 155/400 = 0.3875ADU/s/px

Dark current at 400s is higher by about 50% than that at 200s - that can't be right, can it? Here we did not do any comparison to other values - just checked your measurements one against another.

 

?? These where quick inaccurate local spot measurements. The 15 could 15.5 or so. In your calculation the sqrt is missing.

Dark current noise for 200 second bin2x2 is (15^2-13^2)^0.5 /(200sec*16)= 0.002339 electrons/sec. The dark current noise for single pixel is sqrt(4) larger equals 0.004677 electrons/sec..

Dark current noise for 400 second bin2x2 is (18^2-13^2)^0.5 /(400sec*16) = 0.001945 electrons/sec. The dark current noise for single pixel is sqrt(4) larger equals 0.003891 electrons/sec.

The dark current should be the same for 200 seconds and 400 seconds. This offset is just because of the inaccurate measurement. Next time I will measure noise in (dark - master dark)  or your method  noise in (dark -dark)/1.414.

Han

 

 

The perfect guiding doesn't exist as far I'm aware.

--------------------------------------------------------------------------------------------------------------------------------------

I tried to answer the question how many darks you will need to take to be sure the noise addition by the dark is minimal, lets say 10%.  Since you have to add noise by take the power you have to use this formula:  Noise^2 :=LightNoise^2+DarkNoise^2,  so 1.1^2:=1^2+darknoise^2. Then dark noise is should be sqrt(1.1^2-1^2)= 45% of the light noise maximum.  I did that wrong in the first post

In the lights the most noise comes from the incoming fotons, the so called sky glow background.  If you analyse one dark and one light you can estimate how many darks you need to reduce the dark noise influence to 10%.  So lets do the math correctly:

---------------------------------------------------------------------------------------------------------------------------------------

DARK calculation:

The total noise in a dark is:

• Noise^2 :=ReadNoise^2+DarkNoise^2

I measure for the ASI1600, -15C  in unity gain these dark noise values in ADU's:

• 0 seconds binned 2x2,  σ = 13 ADU, this is 13/16= 0.82 electrons since the 12 bit is converted to 16 bit equals x16. That is 0.82*sqrt(4)=1.64 electrons for a single unbinned pixel
• 200 seconds binned 2x2,  σ = 15 ADU
• 400 seconds binned 2x2,  σ = 18 ADU, using above formula 18^2=13^2+ DarkNoise^2. Then Darknoise is 12.4 ADU or  (12.4/16)/200= 0.0039 electrons/seconds. For unbinned this is then 0.0078 electrons/seconds

This matches nicely with reported values.

---------------------------------------------------------------------------------------------------------------------------------------

LIGHT calculation and number of darks required:

Now for the light:

• Noise^2 :=SkyNoise^2+ ReadNoise^2+DarkNoise^2

The skyglow in a ASI1600 behind a F/5.8 telescope, unity gain for a single light 200 seconds binned 2x2, unfiltered taken at SQM 20 is 6800 ADU and the measured 

σ =168 ADU. 

So 168^2 :=SkyNoise^2+ 13^2+12.4^2. The SkyNoise  is then 167 ADU or  (167/16)/200 elektron/seconde.  The shotnoise should be about sqrt(6800)=82 ADU and the remainder PRNL noise

If 100 lights are stacked the resulting noise is about 168/sqrt(100) is 16.8 ADU. If I want the darks adding only 10 % noise to the result, then (16.8*1.1)^2=16.8^2+ DarkNoise^2.  Then total dark noise should be then 7.7 ADU maximum. That would require only 4 darks since 15/sqrt(4)=7.5. This is a different result then before. So the number of darks required are much less.

-------------------------------------------------------------------------------------------------------------------------------------------

LIGHT H-alpha calculation and number of darks required:

If I repeat the above calculation for a light using a H-alpha filter 7nm, then the  sky noise is about 34 ADU. A stack of 100  lights will result in 3.4 ADU noise. This requires darks with a total 1.5 ADU noise to keep the influence below 10%, so  for H-alpha it requires 100 darks since 15/sqrt(100)=1.5.

 

Note this 7 nm H-alpha reduces the light with a factor 7nm/280nm= 0.025. The noise is then sqrt(0.025 ) is 0.158 time lower. This matches with 34 ADU/168 ADU is 0.2

----------------------------------------------------------------------------------------------------------------------------------------------

I better put this in a spreadsheet. Please correct me for any error.

 

Han

 

 

 

 

That is a interesting method you describe. I will look into tomorrow. In principle it should be possible to calculate the required ratio  nrdarks /nnrlights   from  the local SQM value, focal ratio and camera noise figures and exposure time. This minimum ratio will indicate how many dark are required to keep the dark noise below 10% of total noise.

Probably an old discussion but lets review it with some measurements:

 

The dark noise should only have a small influence on the total noise of the final image. Most noise is generated by the sky background. Under good conditions SQM = 20.4, I measure using my ASI1600MM-Cool the following noise (standard deviation) in a dark and in a light for an area where no stars are visible (local measurement using ASTAP):

Dark 1 x 200sec, σ = 15 (range 0..65535)
Light 1 x 200sec, σ = 130

The noise in the dark is roughly 12% of the light, which seems acceptable to me. That would argue for about the same amount of darks as lights. With a worse SQM, you can probably do 2.5 times less darks for each (magnitude) step. So under light polluted sky you can do with much less darks than lights.

If you are going to photograph with the H-alpha filter, it will be super dark. In a single H-alpha (7nm) light I measure a σ = 25r. Of these, 15 are self-noise and 10 of the incoming light. In good conditions and using an H-alpha filter, this is an argument to make much more darks than lights

Above for a monochrome camera. To measure with an OSC (color) sensor I think it is better to first split the 4 Bayer pixels into 4 files and then measure them separately.

Some measurements with my ASI1600MM-Cool, monochrome:

DARKS noise:
1 x 200 seconds, σ = 16
1 x 200 seconds - master dark, σ = 15
4 x 200 seconds combined - master dark, σ = 6.8    This is approximately 15 / square root (4)
41 x 200 seconds combined, σ = 5
90 x 200 seconds combined, σ = 3.8   This is a limit value that arises mainly from unevenness of the pixels. The noise will be smaller, approximately 15 / square root (90) is 1.6

 

STACKED LIGHTS noise (lights corrected with darks and flats):
11x200 seconds, σ = 70   (measured at a star free area, standard deviation in 0..65535 range, sky conditions could have been different)
18x200 seconds, σ = 36
18x200 seconds, σ = 40
40x200 seconds, σ = 26
42x200 seconds, σ = 30
44x200 seconds, σ = 25
58x200 seconds, σ = 20
95x200 seconds, σ = 16

Apparently the light noise decreases considerably while stacking more lights and I reach σ values up to 16 a 20. You do not want to stack these images with a single dark having a σ = 15. If you want to keep the dark noise added below 10% of σ = 16 then you need 100 darks because they give: 15 / square root (100) = 1.5 noise.

So this confirms for a good suburban site (SQM=20.4) you will need about the same amount (or more) darks then lights. For a more light polluted area you can take less darks since the noise from the skybackground will be abundant. For H-alpha work you better take more darks then lights.

Han

You can retrieve DSS1, DSS2 images online using RA,DEC coordinates and scale. I'm not aware if you can retrieve directly them based on name. ESO also allows to retrieve the images as GIF of JEPG Here some hints:

 

 

 

22 hours ago, Aramcheck said:

I'm not sure if other Plate Solving software also has the PGC catalog (eg ASTAP, PlateSolve2 or All Sky Plate Solver).

Below a Hyperleda (PGC) annotation of the image using ASTAP. Just download the Hyperleda database installer from the ASTAP webpage :

Han

 

15 hours ago, soundwave said:

Astrometry.net

 

When uploading the image to Astrometry.net's website, I get the correct coordinates (202.367, 47.281) 
But when uploading the same image to Astrometry.net using N.I.N.A, I get the wrong coordinates (13.491, 47.281)

An RA of 202.367 is in degrees [0..360] while 13.491 is in hours [0..24]

They are the same since 202.367*24/360=13.491

21 hours ago, calorno said:

.... are likely to offer the best bang for buck for stacking.

Interesting test, but since imaging takes hours, the best bang for bug is some patience.

Normally a huge amount of mbytes are processed. I'm surpriced HDD vs SSD doesn't makes a real difference.

You could also have a look to VM Ware player.  Virtual box or VM Player both will work well.  I use VM player to run Linux in my observatory under Win7 for testing and can control the mount and cameras. I use also VM player to run WinXP for an old scanner and several Linux flavors under Win10. Running a Mac  worked  for me under Virtual box.

The nice thing is that you can keep several virtual machines and copy/modify them as you like.

Just try and see how good Virtual box or VM Ware player works for you.  What's important is that you get the copy & paste working between the virtual machine and the host system.  You can copy also astronomical files to a disk and unmount it again so it becomes accessible to the host system but that's more cumbersone. You can also have a shared directory, but I never managed to get that running in either Virtual box or VM player.

An other problem can be the resolution. Some Linux distributions have problems to display the full resolution in the virtual machine. The host machine will require 4 gbyte minimum but 8 gbyte will works much better. The speed is slower but still reasonable good. Stability is good.

I assume you want run Linux under Win10.  I would suggest to use Lubuntu since it uses less resources then Ubuntu and the interface is more similar as Windows.

Han

 

 

Thanks for the solution. I was already a little puzzled. Abell1809 is missing from my database. Now it is clear.

While processing some old image series, I had a serie of a few galaxies. Nothing special. Exposure  9x200 seconds with a 100 mm APO F/5.8 and an ASI1600 camera. But then I noticed a large amount of unsharp stars near NGC5365. Further investigation indicated they where all galaxies. The cluster has as a far I know no name.

Han

 

A small part of above image around NGC5365. De unsharp stars are galaxies.

 

Annotated and the cluster marked with a rec circle.

Still I wouldn't recommend X-trans sensor  cameras for astronomy.  Getting a good star colour with X-trans pattern is more difficult unless your image is oversampled. A star has to illuminate 3x3 pixels equally for a good colour rather then 2x2 pixels. Secondly the important red is 2/9 of the pixels compared with 1/4 for a Bayer matrix.

Han

Use method "Sigma clip average" for stacking. This will detect that the hot pixels are outliers and remove them.