Jocular: a tool for astronomical observing with a camera

[Martin Meredith]

4 hours ago, Steve in Boulder said:

Martin, does LRGB mode work with the watched folder, and if so is there a naming convention for the files?

Yes. The names should start with 'red', 'green', 'blue' or 'ha', or end with '_red', '_green', etc  (the rest are treated as lights). There are some special cases for programs like StarlightLive that append various prefixes such as 'image_' e.g. 'image_red...'

However, if the information is available in the FITS header that will be used instead. In this latter case the FITS keyword is 'FILTER' or 'FILT' and the value can be in various forms, including 'r', 'R', 'red'.

It is easy to add more if other programs use different conventions.

[Steve in Boulder]

5 hours ago, Martin Meredith said:

Yes. The names should start with 'red', 'green', 'blue' or 'ha', or end with '_red', '_green', etc  (the rest are treated as lights). There are some special cases for programs like StarlightLive that append various prefixes such as 'image_' e.g. 'image_red...'

However, if the information is available in the FITS header that will be used instead. In this latter case the FITS keyword is 'FILTER' or 'FILT' and the value can be in various forms, including 'r', 'R', 'red'.

It is easy to add more if other programs use different conventions.

I’ll soon find out what the ASIAir does, though I can use a script to rename them and/or revise the FITS header appropriately for Jocular. 

[Martin Meredith]

Prior to incorporating it in a normal release, there's a dev version that can be installed with

pip install jocular==0.5.4.dev5

which contains a couple of new sliders

'sharp' -- unsharp masking

'tnr' --  'Tony's Noise Reduction' (based on an algorithm developed by AKB -- thanks!)

Both have parameters that can be set under conf/monochrome for altering things like radii/kernel size. Both are switched off at the '-' end of the slider. Note that if you choose a large value for the radius or kernel size the algorithm can be quite slow (but I find it usable for up to 2M pixels).

Feel free to have a play. I'll document them properly in the upcoming release (and most likely shift the sliders around a bit) 

Here's an example of sharpening a recent Messier 88 (7m15s in 15s subs; Lodestar X2 mono + filters, LRGB)

The dev version also fixes a few bugs and reads the temperature directly from the sensor if you're using an ASI camera.

Any comments/examples of the sharpening/noise reduction welcome as I'm happy to incorporate any changes before the official release.

[AKB]

1 hour ago, Martin Meredith said:

pip install jocular==0.5.4.dev5

Perhap slightly less cryptically, thanks to Martin's software expertise, and possibly egged-on by several users, the latest development version of Jocular sports two what-might-be-called "post-processing" features: sharpening, and noise reduction.

These two contrasting [sorry for the pun] and somewhat complementary features are operated from sliders on the ring, and static settings under the configuration's Monochrome page (not sure about the name, perhaps Martin will explain.)

• Slider sharp, configuration parameter radius, this is an implementation of an unsharp mask.
• Slider TNR, parameters noise reduction kernel and kernel size in pixels.

The noise parameters probably need a separate post to fully describe their effect, although I'm sure that Martin is working on the documentation as we speak 😉.

For larger images, the current implementation of the noise reduction can make the response times a bit sluggish.  Best practice, at this time, is probably to set both sliders to their minimum values, effectively switching them off, until final cosmetic tweaking of the image is required.  (Perhaps this should happen on every image reload??)

I've already posted one example of this processing here:

https://stargazerslounge.com/topic/349834-cancer-arp-82-shk-185-hickson-37/#comment-4226745

But I attach another below:

1. no processing
2. sharpening only
3. sharpening + noise reduction

Tony

 

 

 

 

Edited March 25, 2022 by AKB

[Martin Meredith]

I don't have such a large FOV but here's the same processing applied to Arp 16 (ie, 2 of the galaxies in Tony's shot)

No processing

Sharpening (about 60% of the way)

Sharpening plus TNR (about 70% of the way)

 

Monochrome: that's all the monochromatic processing; there's also Multispectral (LAB Colour on the config screen) which handles all the colour/narrowband stuff.

[Martin Meredith]

BTW Note how Tony's noise reduction also deals with the dust bunny! Small sensor people perhaps no longer need flats...

[Steve in Boulder]

Very nice results from sharpening plus noise reduction, in both instances.  

[Martin Meredith]

I should add that my original noise reduction slider (labelled 'nr') is still present. This has a slightly different and more modest purpose when compared to TNR. All it does is smooth out the very low end of the intensity range (and as such doesn't produce artefacts). The rationale is that some of the more powerful stretch functions tend to behave badly for very low intensities (often having gradients tending to infinity), so extremely low level noise is amplified. The 'nr' slider attenuates low intensity levels in a smooth way. I tend to have it on max so there may be a case for removing this slider and allowing any user that prefers something other than max to set it on a configuration panel. My only concern is not to remove any genuine faint stuff. However, moving the 'bg' (background) slider to let in more background (which might actually occasionally contain 'target' faint stuff) can have a similar effect as backing off the noise suppression.

[AKB]

14 minutes ago, Steve in Boulder said:

Very nice results from sharpening plus noise reduction

I believe you're to blame for the sharpening addition!   So thanks for that suggestion.

Tony

[alphatripleplus]

I too am looking forward to trying out these new tools once  the dev version makes it to prod.😀

[Steve in Boulder]

Well, four images by three users is certainly enough testing, right?😎

[catburglar]

32 minutes ago, Steve in Boulder said:

Well, four images by three users is certainly enough testing, right?😎

Couldn't agree more- there's always pip if I need to downgrade 😉

On a slightly more relevant note...I had a session with the dev version last night and had no issues. Here's a pre and post image showing the impact of the sharpening and noise reduction tools.

 

I've posted a couple more from the same session in this thread. You need to be a bit careful sharpening data with lower SNR, and if you push it too far you'll see some ring artefacts and "blotchiness" in the sky background, but that's the same with any of these tools. Overall I think they're a really nice addition to the toolset, and on my low pixel count images it doesn't slow down the processing so I can still do it live- running the application on a cheap compute stick with a Celeron J4125 processor. 

 

[Martin Meredith]

1 hour ago, alphatripleplus said:

I too am looking forward to trying out these new tools once  the dev version makes it to prod.😀

Done! 

The version is 0.5.5

I've incorporated Tony's suggestion to reset the TNR and sharpness controls to 'off' at the start of each new object.

[AKB]

@catburglar was kind enough to share with me the raw stack of M51 (and the other images on his "Showpieces" thread, results of which I've shown there.)

Here's a couple of attempts I've made, and the slider settings I used.

Note that similar results are achievable with a variety of stretches (here using asinh and gamma) but with vastly different positions for the W and S sliders.  The auto setting for B invariably makes an excellent choice.  In this case, a very light touch on the NR slider gives good visibility of the dimmer parts of the image.  Modest amounts of sharp and TNR settings generally suffice.  I've used a median noise reduction kernel of size 16, and an unsharp mask radius of 9.

Hope this helps.

Tony

 

[Mike JW]

I too have been playing with sharpening, NR and the TNR sliders.

My feeling is to not sharpen more than about a third of the way along the slider, to keep something of the soft view that would be seen visually. For the TNR I have pushed it to about half way.

NR - varies all the time as to how much to use - just depends on the amount of noise, faintness of the DSO/feature. On faint DSOs/features too much sharpening/TNR and features get lost. These sliders do seem to work well on bright DSOs.

Good idea to have the default value of the sliders to off.

Tony - I have no imaging background so if could you explain in simple, layman's terms as to what 'median noise reduction kernal 16 and unsharp mask radius of 9' mean. All a mystery to me.

Mike

[AKB]

1 hour ago, Mike JW said:

could you explain in simple, layman's terms as to what 'median noise reduction kernal 16 and unsharp mask radius of 9' mean

Both smoothing (noise reduction) and sharpening an image requires the selection of an image ‘scale’ – essentially a length over which to average.

Remarkably, unsharp masking is achieved by subtracting a smoothed version of an image from itself.  All the more amazing because the technique was developed for analogue processing of image plates in a darkroom.  So the unsharp mask radius is simply the half width of a window (probably a Gaussian function) used to smooth the data. 

The noise reduction approach I suggested that Martin implement, has a sound theoretical background in Bayesian statistics but has been much simplified and approximated to make it tractable for use in an interactive environment.  It essentially adaptively mixes a smoothed version of the image (again!), an approximation to the background, with the image itself.  There are two choices of smoothing – a Gaussian average, or a median – both of which require a width parameter.  This is the noise reduction kernel size.

The sharp and TNR (an acronym chosen by Martin to denote  Tony’s Noise Reduction) sliders control the strength of the filtering in both cases.  The scale parameters, somewhat less critical, are defined on the Configuration Monochrome settings page.

Modern noise suppression and sharpening methods are almost all multi-scale implementations, with many more parameters, and much, much more computation time.  The somewhat simplistic approaches here are probably as good a fit as any to the EEVA ethos, remarkably effective, and as far as many of us would care to go in the post-processing direction.

 

[Mike JW]

Thanks Tony,

That has given me a better feel for what is going on but to be honest I am not sure I understand - I had better do some homework on the internet. Most the time I just move the sliders until I get what I think looks nice with no idea what I am doing or why they work.

Mike

[AKB]

4 minutes ago, Mike JW said:

I just move the sliders until I get what I think looks nice

Frankly, I think that’s what most of us do!

Wikipedia has a good page on unsharp masking: https://en.wikipedia.org/wiki/Unsharp_masking

Edited March 27, 2022 by AKB

[Mike JW]

That is good to know. Just found a useful article which whilst way too technical complements your explanation so currently playing with the Jocular settings - all good fun but not as exciting as getting out there observing.

Mike

[catburglar]

This has been a useful illustration for me. Whilst I generally take the approach of adjusting the sliders until I’ve got something that looks half decent, I think I have a tendency to keep the stretch near to it’s maximum and the adjust the white point to make the image bright enough and then to reduce the background using the bg and nr sliders. Tony’s example with the asinh stretch shows a different approach- reducing the degree of stretch and leaving in a bit more bg. 
I’m guessing this is effective because the asinh stretch is more aggressive across the brightness range than say the gamma function, so any small tweak on the bg slider has a more significant effect on the resultant image. Reducing the degree of stretch mitigates this and provides a bit more latitude in which to work.

I also note I’d left the config parameters at the default settings which I think were smaller than those Tony used. I’m guessing this has the effect sharpening ‘noise’ and reducing the impact of the noise reduction algorithm because the I was averaging over a smaller region…

[AKB]

18 minutes ago, catburglar said:

I’m guessing this has the effect sharpening ‘noise’ and reducing the impact of the noise reduction algorithm because the I was averaging over a smaller region…

Yes.  A smaller unsharp mask width increases the high frequencies in the image... ie. noisier.

For the noise reduction, the window effectively defines the scale and amplitude of the background mottling that you can see if you raise the background level too high.

[Martin Meredith]

An issue Mike ran into using the latest version of Jocular with StarlightLive (SLL) is that the latest release of the package I'm using for reading FITs (astropy) doesn't like SLL FITs files (this is a known issue and will no doubt be fixed in the next release of astropy). The solution is to downgrade to the previous release of astropy immediately after installing Jocular, by typing:

pip install -U astropy==4.2

I'll ensure that the next Jocular release only depends on v4.2 of astropy if the issue isn't fixed before then.

[PatG]

In preparation for downloading Jocular, I have checked my Windows 10 laptop and don't have Python installed. I see from the Jocular web site that I need to download a Python V3.6-3.9 but not V3.10

On the Python website, I'm looking at Python download 3.9.12 which is latest version 3.9. I'm not sure which batch of files to download as there seem many and I want to make sure I am downloading the correct pack. I'm looking at this webpage.

Python Releases for Windows | Python.org

I'm also unsure whether I need to download a 32 bit or 64 bit version - 

Advice/guidance much appreciated.

Thanks 

Pat

[catburglar]

I haven’t upgraded python in a while (my version is 3.9.9) running on Windows10. If I was setting up now I’d download the 64 bit installer…

[Mike JW]

I use python 3.7 64 bit on my windows 10 laptop.

Mike

Edited April 4, 2022 by Mike JW