Demosaic or not with DSLR? Confusion reigns!

[The Admiral]

Up to now I've been quite happily using AstroArt to pre-process my Fuji RAWs. The program is set to de-mosaic on load. So far as I can tell, it seems to work.

BUT, odd snippets I've read here and there seem to suggest that calibration should not take place with demosaiced images. The ever-unhelpful users' guide says about configuring RAW:

"DCRAW demosaic on load. .....If you enable this option (not recommended) you won't be able to correct the dark frame and the hot pixels. Instead, disabling the option you will be able to correct the defects and you will demosaic later with the command Demosaic."

I really have no idea what this means and why I should need to do this. It may seem a moot point, but there are knock-on rammifications to this, because the Fuji doesn't use a Bayer filter pattern, and although DCRAW will demosaic  the RAWs, I cannot demosaic later with the Demosaic command because it only offers the Bayer pattern choices.

I'm not sure how DSS deals with this situation either, though I must say it doesn't seem to work with the Fuji RAWs (but will if I pre-convert to DNG), even though DCRAW (which DSS uses) should demosaic.

It's all a bit of a mystery. Can any of you gurus shed some light on this please?

Ian

[The Admiral]

Anyone?

Let me put the question another way. I understand why it is necessary to demosaic before aligning, but it isn't at all clear to me why calibration should necessarily be performed on the RAW images. After all, we stack the demosaiced images, so what is the problem with subtracting bias and darks and dividing by the flat after demosaicing as well? What is the downside of doing this? Does it invalidate the process by using demosaiced files?

Any thoughts?

Ian

[steppenwolf]

The correct flow is to calibrate - de-Bayer (de-mosaic) - align - stack. The Fuji sensor uses a non-Bayer matrix as you say so this does complicate things a little.

The reason that calibration is carried out first is because we are only interested in adjusting pixels levels at calibration stage so that we have a correctly adjusted matrix ready for de-mosaicing - this ensures that the necessary interpolation is carried out accurately.

[rotatux]

22 hours ago, The Admiral said:

I really have no idea what this means and why I should need to do this. It may seem a moot point, but there are knock-on rammifications to this, because the Fuji doesn't use a Bayer filter pattern, and although DCRAW will demosaic  the RAWs, I cannot demosaic later with the Demosaic command because it only offers the Bayer pattern choices.

Hi Ian

That looks like an option to use DCRAW demosaicing when AA doesn't know how to do it.

I don't understand, though, why it would not allow you to apply calibration frames. Those are pixel-level processing techniques which would apply correctly to either a grey frame or sub-component frames of a RGB image. In my early days with Regim I have used color darks (= debayered, and applied after demosaicing the lights) and they were giving good results; I now have reverted to the more traditional "before debayer" way of doing calibration, not because it was wrong but because processing 1/3 smaller images is much faster and takes less space on my storage.

IMO the only drawback I could see, is a pixel level precision / resolution one. After demosaicing, some discrete (= integer) math has been applied to produce 3 image channels from one, so a bit of resolution is lost in combining adjacent pixel values -- actually I think not completely lost but rather distributed around in unpredictable ways. Typically a single hot raw pixel will give a few color-sticky pixels as output, but roughly the same distribution will occur in all images, so a dark should remain effective. Since that distribution of error/precision depends partly on input channel level and surroundings, the demosaicing algorithm cannot guarantee a value would be processed the same in a light and in a dark or flat. Thus your calibration frames, though applicable, won't give exactly the same result as would the traditional way, and you may in theory need a bit more frames to achieve the same level of precision as others (seen that way, the distribution I spoke of would look like another kind of noise). However we are talking of off-by-one cases, so I don't think that would be high enough compared to noises of other natures, that one would easily notice it.

[The Admiral]

3 hours ago, steppenwolf said:

The correct flow is to calibrate - de-Bayer (de-mosaic) - align - stack. The Fuji sensor uses a non-Bayer matrix as you say so this does complicate things a little.

The reason that calibration is carried out first is because we are only interested in adjusting pixels levels at calibration stage so that we have a correctly adjusted matrix ready for de-mosaicing - this ensures that the necessary interpolation is carried out accurately.

Thanks Steve, indeed that workflow is the one which I'm just becoming aware of. The Fuji sensor does complicate things, but rather more than a little unfortunately. DCRAW will demosaic the Fuji RAW files, but once the calibration is done the files are no longer in RAW format, or so I am under the impression. AA gives one the opportunity to demosaic afterwards but its only options are for the standard Bayer arrays. So the question really remains, what is the detriment in demosaicing at the outset and calibrating using the demosaiced RAWs?

1 hour ago, rotatux said:

Hi Ian

That looks like an option to use DCRAW demosaicing when AA doesn't know how to do it.

I don't understand, though, why it would not allow you to apply calibration frames. Those are pixel-level processing techniques which would apply correctly to either a grey frame or sub-component frames of a RGB image. In my early days with Regim I have used color darks (= debayered, and applied after demosaicing the lights) and they were giving good results; I now have reverted to the more traditional "before debayer" way of doing calibration, not because it was wrong but because processing 1/3 smaller images is much faster and takes less space on my storage.

IMO the only drawback I could see, is a pixel level precision / resolution one. After demosaicing, some discrete (= integer) math has been applied to produce 3 image channels from one, so a bit of resolution is lost in combining adjacent pixel values -- actually I think not completely lost but rather distributed around in unpredictable ways. Typically a single hot raw pixel will give a few color-sticky pixels as output, but roughly the same distribution will occur in all images, so a dark should remain effective. Since that distribution of error/precision depends partly on input channel level and surroundings, the demosaicing algorithm cannot guarantee a value would be processed the same in a light and in a dark or flat. Thus your calibration frames, though applicable, won't give exactly the same result as would the traditional way, and you may in theory need a bit more frames to achieve the same level of precision as others (seen that way, the distribution I spoke of would look like another kind of noise). However we are talking of off-by-one cases, so I don't think that would be high enough compared to noises of other natures, that one would easily notice it.

Thanks for your thoughts Fabien. Sorry, I think that you may have misinterpreted the way AA works. Basically, AA doesn't prevent you from doing anything! You can set up the way RAWs are treated on import - either demosaic or not, and without a great appreciation of the finer points I've been demosaicing (which as I mentioned above, is not the preferred option according to the manual). And when it comes to what it calls Preprocessing there is an options page where one can set how the resultant stacked image is created, how the alignment is done, etc., and whether to demosaic RGB. Now I've assumed, as there is no explicit mention of this in the manual, that this demosaic option specifically refers to demosaicing before aligning and stacking. The only problem here is that there are only the options for the various Bayer arrays, not the Fuji of course, which is non-standard. I can only surmise that by the time the files have been calibrated they are not in the original RAW format* that can be demosaiced by DCRAW, but I guess in some sort of bitmap files. This is why I have demosaiced from the outset.

This practice does not seem to have been too detrimental to the final output so far as I can tell, and I'm heartened to hear that this way of working was the same as your original approach. What you say about demosaicing imprecision does make sense, and I can see it probably would have greatest impact on the way hot pixels (or strictly, sensels I suppose) are dealt with. Really, without buying another camera with a standard Bayer array I will have continue as I'm doing, but it is helpful to know that it won't be too detrimental to my final image.

Ian

*Edit. If the intermediate files are saved they are saved as FITS.