Debayering a DSLR's Bayer matrix.

[Stub Mandrel]

1 hour ago, D4N said:

Theoretically yes, however the camera and subject would need to be identically placed between the two images.  Since you need to remove the UV blocking filter I think this would be impractical for most purposes.

Sounds like a challenge!

[Herra Kuulapaa]

Hmmmm., It can be very difficult in real life due to the fact that the sensor response is quite low in UVA range compared to VIS. This results UVA signal being just a tiny fraction of total signal - UVA gets swamped heavily by VIS.

Interesting idea though and UVA should be there, if one just can separate it

[Stub Mandrel]

I remember that with film, even using a UV or Skyulight filter I used to have to underexpose things like buttercups by at least a stop to avoid the UV bleaching out on the honey guides.

[Thalestris24]

16 hours ago, Stub Mandrel said:

I wonder if I could get UV photos by subtracting a v-blocked frame from a full spectrum one.

Hi

Maybe easier with a uv-pass filter. I have one but haven't tried it for astro - not sure if there are any targets it might be useful for...

Louise

[zfedoran]

Thought I should share my results, managed to remove the bayer matrix off my Canon 20D with scotch tape

It didn't come off without a fight though. I could not get the tape to work at all and began using a wooden tool like many others. After some frustration, I decided to use some tape to clean the dust from the sensor. It seems once the CFA surface has been broken the tape works a lot better.

I'd recommend the tape method over other methods for older sensors.

 

Edited July 19, 2016 by zfedoran

[zfedoran]

The following is a comparison between a stock 20D and a debayered canon 20D using the same lens, both at the same distance and same settings. The color image has been desaturated, but the levels/curves have not been changed. 

It looks like there is a minor loss in QE and a gain in resolution sensitivity. This would make sense because the micro lenses have been removed.

[RAC]

8 hours ago, zfedoran said:

The following is a comparison between a stock 20D and a debayered canon 20D using the same lens, both at the same distance and same settings. The color image has been desaturated, but the levels/curves have not been changed. 

It looks like there is a minor loss in QE and a gain in resolution sensitivity. This would make sense because the micro lenses have been removed.

 

Well that is very interesting, the color one is horrible to look at compared to the mono one. The color one looks brighter but there is a lot better data in the mono image and way less noise?

[Herra Kuulapaa]

On 21.7.2016 at 01:34, zfedoran said:

The following is a comparison between a stock 20D and a debayered canon 20D using the same lens, both at the same distance and same settings. The color image has been desaturated, but the levels/curves have not been changed. 

It looks like there is a minor loss in QE and a gain in resolution sensitivity. This would make sense because the micro lenses have been removed.

Good job! The comparison however looks a bit soft. I wonder if the focus was spot on?

Did you process the images without demosaicing? There is some sort of pattern in mono image, but it should be flat and uniform.

Left image below is a true raw converted RGB image (not demosaiced and showing the RGB filters) and the right one is mono, also true raw converted. It doesn't show any pattern because there is no filtering anymore.

[zfedoran]

Yeah I noticed that too. it could be a result of imaging a monitor, the pattern could be from pixels on the screen. I wonder if it could be a moiré pattern.

Regardless, I was very careful to focus both. Also, I'm not sure if this is worth mentioning, but these are two separate cameras not a before and after.

The mono image was processed using dcraw.

Edited July 25, 2016 by zfedoran

[Marf1]

I  use this

https://www.dpreview.com/forums/post/58349979

 

dcraw + photoshop for remove pattern

dcraw.exe -v -r 1 1 1 1 -d -6 -T *.nef

https://www.dropbox.com/s/pw57s6skewtyctg/NoCfa.atn?dl=0

 

 

 

[zfedoran]

@Marf1 I just tried running dcraw with "-v -r 1 1 1 1 -d -6 -T" and it seems to be processing my CR2 files in an odd way. It is possible that these flags work for you but not for me.

I have a few images that are exposed to the right of the histogram, when I open the CR2 files directly in photoshop, I am able to see all data without clipping. However, after running dcraw with your settings, it seems like the highlights are clipped quite a bit.

I'm not sure if this will work for you, but after reviewing the source code for dcraw, I found an unlisted flag, "-E", when used with the following flags, it seems to do the right thing (that is, give me the data without applying a bayer transform, AND don't do any weird scaling on the data). I could be wrong though

> dcraw -E -4 -T *.CR2

Here is a sample (zoomed in 1200% on a small chunk of remaining CFA, note how the hot-pixel is not there on the color version):

Let me know if you see improved results running with "-E -4 -T" instead.

[Marf1]

Thanks zfedoran... I did not know -E

 

For recovery highlights  i use -H 3

dcraw.exe -v -r 1 1 1 1 -k 100 -H 3 -d -b 2.0 -6 -T

-k=black point

-b=exposure

[Herra Kuulapaa]

Interesting, did you debayer a D750? How was the sensor cover glass?

[Adam J]

Hey all, would like to show you my first test photo from my debayered Canon 1000D, having read this thread I opted not to even try to push to the edges of the sensor, ill happily crop it.
 

I removed the tougher micro lenses first using a cocktail stick and then the filters themselves using car plastic polish. This is a practice sensor and I will be doing another 1000D soon taking into account my experience. One scratch just to the left of the penguins foot caused by me pressing to hard, did not know how hard i needed to press on at first and way over did it. Everything else baring the edge is dust or loose bits of the filter layer that will vanish with a little more cleaning. Over all nothing that wont disappear with a little dithering and some flats. Clearly focus is not perfect due to the filter having being removed.

P.S I know that many people say that you can remove the cover glass on the 1000D without heating but I found that this was much easier when taken to a mild 60c using a hair dryer.

[Adam J]

Cleaned and cropped flat frame, not too shabby if you ask me. Its a testament to the work you guys have done on here that i can get this result from my first attempt. Comes out at 7.2 mega pixels after cropping the rubbish out.

Edited to add a picture of my scope and DSLR cooler taken with the mono 1000D.

Edited October 23, 2016 by Adam J

[StuartJPP]

Good job Adam.

I have got a Canon 1000D camera and a spare 1000D sensor but haven't gotten around to attempting this mod...not sure if I will or not.

What plastic polish did you use?

[Adam J]

4 hours ago, StuartJPP said:

Good job Adam.

I have got a Canon 1000D camera and a spare 1000D sensor but haven't gotten around to attempting this mod...not sure if I will or not.

What plastic polish did you use?

In all seriousness I am looking for another sensor as I think that I can get it perfectly flat without scratches on my second attempt so if you dont get around to it and want to sell it then I would be happy to buy it from you.

I used "Meguiars Plast.Rx" Polish. White plastic bottle with a black cap with a pointed felt polishing tool for a Dremil but by hand only.  I did not mess with the wires with epoxy, or even attempt to get close to the blue area around the edge of the active sensor region. From reading the entire thread I think that 95% of the problems have arisen from trying to get too close and using the wrong type of epoxy on the gold wires. If you dont have patience and a steady hand I would save yourself some money and not bother lol. I am a perfectionist and so it hurts not to do the whole sensor but I have just told myself that I am going to have to crop it out of the final image.

Good Luck.

[Gina]

Lack of a steady hand nowadays was my reason for giving up.  It's very time consuming as well as needing extreme patience and I simply had better things to do with my time.  There comes a time in your life when you can no longer do the things you used to do and need to prioritise to make the best of your increasingly valuable time.  I did give it a pretty good go though

[Adam J]

1 hour ago, Gina said:

Lack of a steady hand nowadays was my reason for giving up.  It's very time consuming as well as needing extreme patience and I simply had better things to do with my time.  There comes a time in your life when you can no longer do the things you used to do and need to prioritise to make the best of your increasingly valuable time.  I did give it a pretty good go though

Your contributions to the thread where very valuable in helping me get a good result.

I am going to make a video on you tube of the process as I think my chosen method was quite effective for this sensor anyway. I just need to find another 1000D at a reasonable price or I may try a 450D.

I have done a little analysis of the results:

1) I set a custom white balance in the camera to balance out the contribution of each of the red green and blue channels....not sure if this is the correct method. Should I be doing this?

2) I used my PC monitor to make some individual separate Red Green and Blue flat frames. So MS Paint custom Color and 0, 0, 255....etc.

3) I then used PixInsight LE to take a look at the average pixel readings for the area center area incomparison to a clean section of the border (with the filters and micro lenses still in place)

???Should I be setting the camera to Black and White mode???

Looking at just the pixels of the relevant color in both areas of the image I found that the lack of micro lenses reduced sensitivity as follows:

Red: With Filters and Lenses = 0.669 , Removed = 0.479 (71.5% of original value)

Green: With Filters and Lenses = 0.843, Removed = 0.662 (78.5% of original value)

Blue: With Filters and Lenses = 0.651, Removed = 0.524 (80.4% of original value)

So for each sub pixel type there has been a reduction in sensitivity of between 20% and 30% due to the micro lenses being removed. At least if my methodology is sound.

Clearly from the attached picture the green is brighter then the red and blue on average....not sure if this is due to double the number of green pixels or due to double the number of green pixels on the monitor or a bit of both. But this does make sense....

 

This is when my knowledge runs out and I could use a little help. How should I be processing this? Clearly when I use my H-a filter I am going to now get 4 x active sub pixels instead of 1 x active sub pixel. However, each sub pixel will only be ~70% as effective due to the removal of the micro lens. So......the bit I dont get it this, I suspect that 4  x sub pixels together to not equal one single pixel with 4 times the collecting area. Am I gaining 4 times the sensitivity or am I just effectively getting an average value of the 4 sub pixels, similar to stacking 4 frames? Should I be converting to gray scale in processing? I am a little confused. Or by doing this am I only effectively gaining resolution at the expense of some sensitivity????????????

 

Some help would be appreciated.

 

[Adam J]

If I look at the sum lum of all channels in Pix Insight LE then I am getting a very slight reduction for green, a slight increase for red and a big increase for blue.....I am not totally convinced that this mod is going to help sensitivity a hug amount as things stand in may have a positive effect on signal to noise however.

I am not sure how the quantum efficiency of OIII, H-a and SII is effected by the transmission at those wavelength of the bayer matrix alone....it is possible that at those specific wavelength rather than the broader bands in my test the RGB matrix may also be less than 100% transmission. If this is the case its possible that some performance will be gained at these wavelengths.

Finally I would say that despite my best efforts the test colors are not pure and as such this may result in the performance of the mono area of the sensor being overs estimated.  Although the area I selected around the edge for comparison looks clean, there is no grantee that the micro lens is completely undamaged.

Could anyone comment?

The three shots below are RED GREEN AND BLUE (in that order) flats shot with the camera in mono (b/w) mode. The brightness variation is likely the variation inherent in the color balance of my monitor.

Edited October 24, 2016 by Adam J

[StuartJPP]

20 hours ago, Adam J said:

In all seriousness I am looking for another sensor as I think that I can get it perfectly flat without scratches on my second attempt so if you dont get around to it and want to sell it then I would be happy to buy it from you.

I used "Meguiars Plast.Rx" Polish. White plastic bottle with a black cap with a pointed felt polishing tool for a Dremil but by hand only.  I did not mess with the wires with epoxy, or even attempt to get close to the blue area around the edge of the active sensor region. From reading the entire thread I think that 95% of the problems have arisen from trying to get too close and using the wrong type of epoxy on the gold wires. If you dont have patience and a steady hand I would save yourself some money and not bother lol. I am a perfectionist and so it hurts not to do the whole sensor but I have just told myself that I am going to have to crop it out of the final image.

Good Luck.

Thanks for the info Adam.

Regarding the sensors, I am not 100% sure if the "spare" one works as it came with a faulty camera, I think the person plugged in the wrong power supply and fried the camera main board. The other camera is working fine, it has had the filters removed. I was/am going to have a go as a Winter project, but am still undecided if the effort is going to be worth it. I want to try but then again I am not sure I will use it if I do get it done...will let you know.

With regards to shooting in mono, I 'd definitely say no. Shoot in RAW and then use DCRAW with the parameters elsewhere in this thread to extract the image without demosaicing.

I would suspect that where the microlenses have been removed you will see a quite a reduction in sensitivity (from what I have read about microlenses), but then a gain since the CFA/Bayer matrix has been removed.

[Adam J]

1 hour ago, StuartJPP said:

Thanks for the info Adam.

Regarding the sensors, I am not 100% sure if the "spare" one works as it came with a faulty camera, I think the person plugged in the wrong power supply and fried the camera main board. The other camera is working fine, it has had the filters removed. I was/am going to have a go as a Winter project, but am still undecided if the effort is going to be worth it. I want to try but then again I am not sure I will use it if I do get it done...will let you know.

With regards to shooting in mono, I 'd definitely say no. Shoot in RAW and then use DCRAW with the parameters elsewhere in this thread to extract the image without demosaicing.

I would suspect that where the microlenses have been removed you will see a quite a reduction in sensitivity (from what I have read about microlenses), but then a gain since the CFA/Bayer matrix has been removed.

Cheers, Ill have a look at DCRAW I have never used it to be honest. Could you be more specific on the advantages of this method? See below for my analysis of the effects of sensitivity for the 1000D.

Edited October 24, 2016 by Adam J

[Adam J]

Ok, so I decided the best way to look at the sensitivity at a given wavelengths was just to use my 2 inch H-a and OIII filters to create a pair of flat frames and compare pixel values on the Bebrayered section to the edge with both micro lens and filters still intact. I have summerised my finding in the table below.

The real shocker for me was that the OIII seemed to increase in sensitivity where as the H-a decreased in sensitivity on a per sub pixel basis. Honestly not sure what this means in terms of real world imaging....i suspect it means that ill have to increase the lenght of my H-a subs but that noise will be greatly reduced leading to a lower total integration time????

The totals for each 4 sub pixel group are significantly higher, however as I noted before I am not sure how this will translate to the real world, i am reasonably sure its not the same as having the equivalent increase from a single pixel though.

Btw I have no idea why the luminance reported by Pix Insight is higher than the 4 sub pixel average...

Sorry if this is becoming something of a monologue lol.

Edited October 24, 2016 by Adam J

[iapa]

On 24 October 2016 at 10:10, Gina said:

Lack of a steady hand nowadays was my reason for giving up.  It's very time consuming as well as needing extreme patience and I simply had better things to do with my time.  There comes a time in your life when you can no longer do the things you used to do and need to prioritise to make the best of your increasingly valuable time.  I did give it a pretty good go though

I know where you are coming from.

ant fine work, I get some one (often well overpaid) to do it nowadays.

 

Just need to find some one local(ish) to debater a 700D for me

[astroghost-francois]

As you can see on my personnal website on the page talking about sensor debayer techniques , there is a tutorial made for a french photo magazine.

here :

http://translate.google.fr/translate?u=http://astronomie-astrophotographie.fr/TUTO-DEBAYERING.htm&sl=fr&tl=en&hl=&ie=UTF-8

 

and here :

http://www.focus-numerique.com/test-2336/interview-defiltre-bayer-canon-350D-noir-blanc-1.html

 

 

Francois

Edited November 5, 2016 by astroghost-francois