Debayering a DSLR's Bayer matrix.

[Herra Kuulapaa]

Sensors are different and so are the resins between glass and sensor frame. I've noticed a trend of harder and more heat resistant resins when moving towards most recent models in Nikon line up. Glass bond is much more heat resistant.

I discussed with one chip engineer and he suspected that despite and because of good thermal conductivity of silicon the sensor internal temperature may not rise too high even during relatively heavy local heating. As long as the whole sensor is not baked in oven.

I'm currently fighting one chip with heat trying to guess how far I can go. And therefore, does anyone have info how much heat sensors can actually take?

[Herra Kuulapaa]

Small break due to heavy workload, but back again.

There seems to be remains of RGB matrix still on sensor. This layer is really hard in Nikon D5100.

[Unassorted]

Time for an update! After finally getting a clear night last night I went out to get a first light test done on the camera(despite the almost full moon).

Forewarning, I was in a rush to set up and break down so the tracking on my mount wasn't near as good as I would have wanted it to be.

Here is the flat that I took after some more removal that I ended up being "okay" with - http://i.imgur.com/noJpfqV.jpg

I wanted to give people a bit of a comparison to go off of on why this is a good idea so thats exactly what my plan was when I started this. Back in October during a new moon phase, I took a 7 min exposure of the Crescent Nebula(no reason back then but Im glad I did). The subs for the image were 7 x 60. Last night I took a 7 min exposure of the Crescent Nebula with the debayered camera for comparison. The subs for the image were 14 x 30(tracking issues). I took one sub from each image, a 30 sec and a 60 sec and placed them side by side. This is the result - http://i.imgur.com/8GV3pxN.png

This just blew me away. The fact that the 30 sec exposure under a FULL MOON has more data than a 60 sec exposure under a new moon phase is staggering. 

I, of course, have the two images for you and here is the 7 min exposure of the nebula before debayering - http://i.imgur.com/94OzHof.jpg

Here is the 7 min exposure after debayering - http://i.imgur.com/86BFdTZ.jpg

The debayered image IS slightly out of focus(like I said, I was in a rush) and neigher of the images used darks or flats(comparison people).

But of course, I did take darks and flats to go along with the images from last night and this is what I have after about 10 min of post processing - http://i.imgur.com/70hc0xL.jpg

Lights: 14 x 30 at 1600 ISO

Darks: 10 x 30 at 1600 ISO

Flats: 15

Despite the full moon being 100% on saturday, I am going back out again since its a clear night and I will have all night to properly set up and properly focus.

[Herra Kuulapaa]

Interesting, thanks J

You seem to suffer from extensive IR exposure (blob stars). Do you have a naked sensor?

[Unassorted]

Interesting, thanks J

You seem to suffer from extensive IR exposure (blob stars). Do you have a naked sensor?

I believe this was due to the tracking errors on my images. Here is a new image - http://i.imgur.com/yFQmtz2.jpg

6 x 60 at ISO 400 taken on 12/06/2014 during the full moon. Scope is a Celestron Omni 102 XLT on an AVX mount.

The tracking on this one is a much better and the blob stars are non-existent.

I believe that the blob stars from the previous post are also due to my scope and the CA that it suffers but I could be wrong. More testing is needed to make sure.

[Herra Kuulapaa]

Here is a small report of my D5100 project.

http://kuulapaa.com/home/astrophotography/nikond5100mod.html

Ps. Some people may find the other material on the page interesting as well.

[Unassorted]

I was able to get some time outside tonight and I have an hour of imaging the Iris Nebula - http://i.imgur.com/ttPBJDH.jpg

Going to need to get a new sensor to get rid of the scratches since my flat frames are messing up for some reason.

[Herra Kuulapaa]

I have been struggling with that checkerboard pattern with my Nikon D5100. So far I've thought it to be some kind of filter remains but today I made an interesting discovery.

This is what I've been seeing in every file:

But with Nikon hacker true dark current mod the pattern is gone:

This is very good news for D5100!

https://landingfield.wordpress.com/2014/05/13/teaser-nikon-dslr-black-point-hack-for-astrophotography/

[Herra Kuulapaa]

One sample frame with some dust on it:

This was taken under relatively bad sky with IDAS LPS 5min ISO800 D5100.

[Chris]

That's a great result! congrats

[Herra Kuulapaa]

Small comparison about the pixel grid and how it behaves when stacked. 5 frames seem to be enough to avarage it out.

[Herra Kuulapaa]

average...

[D4N]

I wonder if the formerly green pixels are given a lower rating by the camera to compensate for the fact that there's twice as many of them? It seems that in any group of four pixels two of them are dark and two light and that the light / dark pixels tend not to be next to each other. This would coincide with the bayer matrix.

If so possibly this could be corrected for in processing so no data is lost.

I guess the stack will correct the problem so long as there is at least one pixel of dither.

TSED70Q, iOptron Smart EQ pro, ASI-120MM, Finepix S5 pro.

[Herra Kuulapaa]

Some microscopy images. There are so many "do not touch"-structures existing.

The almost invisible gold wires:

The blue areas:

The golden lining:

[Fred_76]

Hi everybody !

I finally jumped on the dark side of the moon, and made my Canon 500D debayered (by Astroghost, in France). I asked him to do one additional thing : adding a copper blade under the sensor with a radiator and a fan to extract the heat out of the box. Astroghost found a way to move the internal thermal sensor of the 500D close to the imaging sensor so that I will now have the right temperature in the Exif. Don't ask me how he did !!!

I am still waiting for the camera who is on its way back home, but Astroghost sent me some RAW images and I could start making some measurements.

Noise and resolution

The picture on the left shows an extract before the CFA matrix removal, and on the right, after removal.

One can see that the noise drops significantly, and there is also a real gain in resolution.

Sensitivity

I analysed areas where the CFA pattern was not removed, in the corners or on the sensor's borders. Nine areas were big enough to be able to do some stats.

I compared, for each chanel (red, green, blue) the level of the debayered photosites (area 2) and the levels of the intact photosites (area 1). The average gain is shown below for each chanel :

- red : approx +30%

- green : approx +30%

- blue : approx +75%

This explains why the signal to noise ratio increased :

- each debayered photosite is receiving more light, so the signal increases

- there is no more colour equilibration that multiplies at least the noise of the blue channel by a big number, so the noise decreases

=> therefore S/N increases.

I will post more info when I receive my 500D.

Fred

[Herra Kuulapaa]

The weather has been exceptionally bad here. I was out yesterday evening and waited three hours for clouds to clear but it never happened. Despite of the constant roll of thick clouds I was able to get single ISO800 / 110 sec HA frame during one brief opening. Other holes lasted only 30 sec so they were practically useless. Ground is covered in snow however, so it will be a white Christmas!

Due to the limited exposure I had to stretch the frame rather hard. It serves as a proof of concept for the D5100 Mono-HA project though.

[Unassorted]

I finally had a nice clear night and was able to get a decent test with my modded 1000D.

M45 - The Pleiades - http://i.imgur.com/lRyALpe.jpg

30 x 1min Lights

9 Darks

15 Flats

20 Bias

Williams Optics GTF-81

Celestron AVX

Canon 1000D(mono modded)

Processed in PixInsight

The amount of detail collected in just 30 min is astounding to me. You can even see the galaxy PGC 13696 in the image(red circle is where it is located - http://i.imgur.com/pJhsGZc.jpg)

I can't wait to do this again since I have a few friends of mine asking me to do it for them as well!

[Gina]

That's amazing   Is this with Luminance filter?

Edited December 22, 2014 by Gina

[Unassorted]

That's amazing   Is this with Luminance filter?

Nope! Just 100% camera.

[Fred_76]

Hi!

More test report from my brand new monomodded 500D.

First of all a reminder on the global characteristics of the sensor :

EOS 500D/T1i Sensors characteristics


Photosites: 4.7 µm (width x height: 4752 x 3168)
 

ISO    Gain e-/DN    Gain DN/e-    RON e-        RON DN        Dynamic (bits)100    2.13            0.47          15.9           7.5           11.0200    1.10            0.91           8.7           7.9           10.9400    0.53            1.89           5.0           9.5           10.7800    0.27            3.75           3.5          13.1           10.21600   0.13            7.52           2.8          20.8            9.53200   0.07           14.71           2.8          40.6            8.6

These figures are unchanged by the monomod.

Thermal noise

An average thermal level is substracted from the value measured on the "darken photosites" around the sensor. This operation is performed before the RAWs are saved so that it is not possible to directly measure the dark current. However, only an average level - constant - is substracted from all the pixels. Each pixel has its own behaviour vs dark current and its specific respons to the temperature remains. We can therefore measure the remaining variations as a "thermal noise" (no more a dark current).

The following graph shows the thermal noise in DN vs temperature before and after the monomod.

The temperature here is the real sensor's temperature - even if read in the EXIF - as Astroghost changed the position of the internal 500D's thermal probe from it original location (close to the sensor's processor) to the sensor itself.

On can see that it is better not to allow the sensor to heat above 10-15°C. This is why I asked Astroghost to insert a copper plate under the sensor with a fan to extract the heat.

Test on cooling

Freezer: I placed the 500D in the freezer at -24°C. After about 1 hour, to allow the sensor to cool down, I started shooting photos of 5 min with a delay of 5 s in between, at 800 ISO. The temperature of the sensor was always between -19 and -21°C, therefore about 5°C about ambient temperature.

Fridge: I then placed the 500D in the fridge at +4°C. The temperature of the sensor rapidly increased and stabilised at +10°C, therefore about 6°C about ambient temperature.

Ambient: And I placed the 500D outside in a box at +24°C. Again, the temperature increased rapidly and stabilised at +29°C, 5°C above ambient.

Therefore I can conclude that the copper+fan allows the sensor to stabilise at 5°C above ambient temperature in about 30 minutes.

Before the mod, the difference was about 12-15°C and the temperature was longer to stabilise (more than 1 hour).

Test on daylight

As the weather is horrible since a long time here in Normandy, I can't test my modded 500D on the night sky. I therefore tested it on daylight with an Astronomik Proplant 807 nm infrared pass filter. This is more an "artistic" photo as I wanted to obtain the look and feel of the old days photos with harsh grain.

Canon EOS 500D + Canon EF 55-200mm f/4.5-5.6 II USM at 55 mm F/11. 1/100 s @ 3200 ISO.

Darks

I did a comparison of the darks for 3 temperatures for 300s at 800 ISO :

Before CFA removal is above, after is below. Offset has been removed.

Offset

Again, before CFA removal is above, after is below.

One can see that the offset has quite changed. This is mainly due to two things :

- the CFA removal is quite brutal for the sensor, it has an effect of the structure and induces some defects

- to insert the copper plate, François had to remove the rear sensors's shielding

However the offset is easily removed in post processing.

Merry Xmas to all !

Fred

[Gina]

Very interesting information - thank you Fred   I arranged Peltier TEC cooling on my DSLRs to -15C but your thermal noise figures agree with others in showing that nothing is achieved by reducing the temperature below -5C and only a little below 0C.  In view of dewing problems at low temperatures I think I shall aim at around 0C for future experiments and final versions.

[StuartJPP]

One thing I am not sure of is does the quantum efficiency decrease with temperature drop or does it increase?

I know that the noise decreases with a decrease in temperature, but does the QE also decrease or any other criteria related to wanted signal? Removing as much unwanted noise is good, removing signal is not good...I accept that we are wanting to improve the S/N ratio which we are doing by decreasing the noise, but sometimes that is not the whole picture.

I have seen dark images of 60 minute exposures where there is very little noise...now please point that camera to the stars and show us some results, preferably alongside a stock camera that hasn't been cooled. I am sure that the cooled camera will win hands down, but by how much?

[Fred_76]

Hi Stuart,

There is no reason that the QE decreases with the temperature. In fact,  the signal to noise ratio is increasing as the noise decreases. And this is exactly what we are looking for

Merry Xmas to all !

Fred

[D4N]

The loss would come from the removal of the micro lenses, how much of an effect it has will depend on the size of the pixels and the amount of space between them. Judging by the images people post with a debate red camera it's not a big problem.

Sent from my iPad using Tapatalk

[Gina]

My experiments showed a net gain in sensitivity - there was more gain from removing the CFA than loss from removing the micro lenses.  This was with a couple of different models of Canon EOS DSLRs.