Looking for some FITS files fresh from your camera

[Ags]

Neutralize background function implemented!

[Ags]

Playing with some denoising techniques I found on an internet:

The method is too slow, I need to offload this to a web worker, multithread, and add a progress bar.

Edited August 4 by Ags

[Xilman]

1 hour ago, Ags said:

Playing with some denoising techniques I found on an internet:

The method is too slow, I need to offload this to a web worker, multithread, and add a progress bar.

I found FABADA to be very effective at noise removal.  If you haven't yet tried it, a simple search will find it for you.

[Ags]

Ok, I have found a FABADA Python script I can convert to JS. Just glancing at it suggests if might be a bit slow. But I will play with it...

[Xilman]

21 hours ago, Ags said:

Ok, I have found a FABADA Python script I can convert to JS. Just glancing at it suggests if might be a bit slow. But I will play with it...

It's not the fastest but I have found it to be very effective indeed. To see what I mean, try going to

A shame you need to translate to JS. If you could use C it would be much faster.

Edited August 5 by Xilman
Add link.

[Ags]

Here is a single sub from my latest project loaded into the page. The white box on the right is the beginnings of a histogram view.

 

[Ags]

Got the histogram working, mostly

[Ags]

Very small update, I handle odd shaped sensors like the 533 (tall images dont lead to scrolling):

Edited August 12 by Ags

[Skipper Billy]

Here is a single 600 second Ha image from an Atik 16200 mono CCD.

 

LIGHT_600.00_1x1_0001_Ha.fit

[Ags]

Thanks, looks like my script can handle Atik mono files too. Nice image @Skipper Billy

I am currently trying to load stacked images from Gimp and DSS, but getting some ugly results.

[Ags]

Aaaaaa-haaaaaa! Although the cameras seem to be producing SIGNED 16-bit integers, Gimp seems to produce UNSIGNED 16 bit integers. Actually it produces 32 bit data, but I am reducing precision as a baby step - need to add support for more numerical formats now. There's still something wrong with my numerical conversion, but at least it looks like a picture now. Despite the raw and processed FITS having different numerical formats, I don't see a FITS header that can guide me, although perhaps I can rely on the DATAMAX header.

Edited August 15 by Ags

[symmetal]

@Ags To conform to the fits standard, integers must always be stored in signed format, (except 8 bit integers). For unsigned integers an offset must be applied to the data before being stored in the fits file to make it conform to a signed integer format. The BZERO keyword holds this offset and for 16 bit data where BITPIX = 16, BZERO is set to 32768 (which is the same as hex 8000). When the fits file is then read, the value of BZERO (if present) is added to the fits data to convert it back to an unsigned format.

For integer data BITPIX is either 8, 16, 32 or 64.

For floating point data BITPIX is -32 or -64 corresponding to 32 or 64 bit floating point.

If the BITPIX keyword is written as ZBITPIX, then it means the data is stored in compressed format and the keyword ZCMPTYPE specifies the compression algorithm, usually GZIP_2.

You could write unsigned data directly to the fits file and set BZERO = 0 or omit it, but the reading software would need to be aware of this and that the file doesn't conform to the fits standard. Unaware reading software would treat the data as signed in this case. 😬 Setting the SIMPLE keyword to F rather than the normal T indicates that the file doesn't conform to the fits standard, but that's risky, and relies on a possible comment entry being read to find out why.

If signed data is treated as unsigned or vice versa, the image usually looks OK at first glance, but the clipped stars will have black centres. 😉

Definition of the Flexible Image Transport System

DATAMAX is the maximum data value stored and is written as a floating point number even if the data is integer, so doesn't reliably indicate the data format, though can infer it. 

The 53 keywords currently defined in the fits standard

Alan

Edited August 16 by symmetal

[Ags]

@symmetal thanks for the links and suggestions.

What I don't understand is DATAMAX is set to 65535 (16 bit unsigned) but like you say star cores are black suggesting loading it as 16 bit unsigned is wrong. BZERO=0 by the way.

[symmetal]

Have you tried loading these problem files into Fits Liberator and see if the stars have holes. This uses NASA's cfitsio library and will load any fits file. I was wanting to display the GOES Solar UV image files in fits format on NASA's website but none of the commonly used Astro Imaging software would display them except Fits Liberator. The image data wasn't in the first data block immediately after the header and was also zip compressed in 32 bit floating point. I ended up compiling the cfitsio library myself along with the gzip library and it made loading fits image data or header info easy with just a function call, automatically calling gzip if necessary. I used C++ though.

I use SGP for imaging and it always sets BZERO to 32768 so stores it as signed data. The DATAMAX keyword isn't present. I would assume this keyword is just added by the program writing the fits file and isn't automatically generated by a fits library writing routine. It could therefore be set to 65535 irrespective to whether it's written as signed or unsigned as that's the original data maximum value.

Is it one of the files posted in the thread which is causing the stars with holes issue? If you say which one I'll see what my program displays it as.

If you have a star image with no bright stars and so the highest ADU value doesn't exceed 32767, the unsigned data can be written as unsigned with BZERO set to 0 and be fully fits complient as signed and unsigned 16 bit data less than 32768 is of course the same.

I remember that Registar, an earlier star alignment program would sometimes display holed stars when reading fits files as it seems it didn't read the BZERO keyword and you had to set an option in the program as to whether the fits data was signed or unsigned. Liberties are taken with software writing fits files which aren't fits complient causing these issues.

Alan

[Ags]

Looking at using Web Assembly (WASM) to speed up the program. There are compilers of C to WASM, so porting (parts of) CFITSIO to the browser is not impossible.

[symmetal]

On 16/08/2024 at 19:36, symmetal said:

Good idea. Emscripten looks like a good GUI interface for doing that. Good luck. 😊

Alan

 

[Ags]

Actually I am enjoying the project as is and learning a lot of useful new stuff, so I think I will persist with my own code. I think cross compiling cfitsio to WebAssembly would require modifying the C code in many places in order to comply with the browser sandbox and the WASM VM. C is not my favorite to be honest, and it would be long road before seeing any returns, while my current approach keeps me interested with frequent tangible progress 😄

[Ags]

On 30/07/2024 at 01:30, CCD-Freak said:

Here is a stacked fit file of M45 taken with an ASI533MC through a SS15028HNT captured and processed with AstroArt

M45 QD-7.fit 101.97 MB · 13 downloads

 

On 30/07/2024 at 12:37, Ags said:

It is an interesting challenge. AstroArt is saving the data as 32-bit floating point (I think) and the color info is structured differently from a raw FITS from a camera that includes the bayer matrix. That's a lot of stuff I don't currently handle, so I'll come back to this one. I see astroart is setting the SIMPLE header to NaN - oopsy!

Well, I finally got there. I can load most (?) FITS files produced by most image processing software I think. I haven't tested all the possible numerical types yet though, and I do assume the data is in the first data unit. @CCD-Freak here is your picture duly loaded.

Edited August 20 by Ags