Any Python image processors on here?

[NickK]

I've been away for too long   as it happens I've been made redundant again.. but not before spending 6 months playing with python and quantum computing as part of work.

So that got me thinking .. and playing. I've started looking at using Python with astropy (astropy.org) that can load FITS images. Coupled with Jupyter notebook seems to work.

As my MBP finally died, I've now switched to a Mac Mini i7 12core with 32GB ram which seems happy as Larry. So my intent is to move the GPU/c++ code I had for processing images into python and continue working on it whilst looking for a new job. I also have some new ideas for processing.

 

[han59]

I'm sorry for you loosing your job. 

Python is an interpreter. For speed a compiler solution would be better. But if your comfortable with Python at least it is muli-platform. What are the ideas?

Han

 

[pete_l]

The CV2 library is a good place to start for python image processing.

[AKB]

Check out the Jocular application written in Python by @Martin Meredith

Actually a real-time acquisition/  processing / visualisation tool, it's amazing.  Probably not what you were thinking of doing yourself, but perhaps there are some useful Python libraries that he can point you towards.

Edited December 6, 2019 by AKB

[BabyPepper]

pick up a basler camera, they are python right of the box and include the SDK.

 

I love image processing and would love to see what you add to the field.  I recently discovered a different way to process solar images, so i know that there are tons of tricks waiting to be unlocked.  Especially at the camera data output before it hits the SSD.      

 

The reality of the situation, is that technology is here already that can instantly process the video signal on the fly     , and the neural network thing paired with an nvidia gpu for parallel processing. Well this can delete the blurry pixels  instantly to only record and rebuild the best data.  Its powerful enough and smart enough to stack on the fly as well..

 

 

Edited December 6, 2019 by BabyPepper

[NickK]

25 minutes ago, han59 said:

I'm sorry for you loosing your job. 

Python is an interpreter. For speed a compiler solution would be better. But if your comfortable with Python at least it is muli-platform. What are the ideas?

Han

 

The annoying piece is I keep having to re-write the code - first Grand Central, then OpenCL, then C++ because OpenCL was unsupported and Metal is now the in-thing however I need wider precision and Metal is single precision floating point. So.. it's either write for Octave (a free R clone) or Python.

I understand Python is interpreted, as is Octave, however it seems more scalable with a move to arrays with pre-coded libraries.

The old code did phase correlation with the PSF from the guide star in the Z axis to rebuild saturated stars. This works well but expands the dynamic range (hence needing larger range).

I did realtime GPU based alignment and stacking too but this is more processing offline concentrating on the maths rather than re-writing based on Apple's whims.

The reason I've picked something Linux friendly (python) is that I have a ODroid 4 core arm that runs INDI + kstars that performs plat solving all onboard.

Edited December 6, 2019 by NickK

[NickK]

18 minutes ago, BabyPepper said:

pick up a basler camera, they are python right of the box and include the SDK.

 

I love image processing and would love to see what you add to the field.  I recently discovered a different way to process solar images, so i know that there are tons of tricks waiting to be unlocked.  Especially at the camera data output before it hits the SSD.      

 

The reality of the situation, is that technology is here already that can instantly process the video signal on the fly     , and the neural network thing paired with an nvidia gpu for parallel processing. Well this can delete the blurry pixels  instantly to only record and rebuild the best data.  Its powerful enough and smart enough to stack on the fly as well..

 

 

Yes, it's possible to use the GPU to inspect the noise - you don't need a neural network per-say as you could simply use a a kernel to detect the high value vs local values but better still (and a method I've used in the original code for this) is to use the guider image to detect if the noise is a remote object by phase correlation - if there's no correlation then it doesn't look like light that has come through the atmosphere.

This was from 5 years ago.. using the GPU in the laptop with FFT phase correlation to align the image (you'll note the image stays reasonably steady but the white borders move):

 

However this is post processing as the GPU in most laptops/cards doesn't have the required floating point range. I could use a eGPU with the mini (original plan) but at the moment.. this will do.

 

Oh - if you like solar processing there was a nVidia deconvolution using swarms to create the PSF to help improve the image.

 

 

Edited December 6, 2019 by NickK

[NickK]

My thinking is if you can use some maths to better estimate the noise levels.

For example

Here they are using gems and thieves, however the same could be viewed as a pixel and the composition of noise - how much is signal, signal overlapping from elsewhere (PSF) and the various noise.

 

[han59]

4 hours ago, NickK said:

Metal is single precision floating point. So.. it's either write for Octave (a free R clone) or Python.

The old code did phase correlation with the PSF from the guide star in the Z axis to rebuild saturated stars. This works well but expands the dynamic range (hence needing larger range).

I'm surprised you need more then single precision. In my program I define the arrays as single precision by purpose since memory requirements quickly increase if you have a few images in memory.

If your looking for a new challenge, develop and document a de-mosaic routine for astro OSC images.  Especially when stars are saturated in the center and the stars are small, HFD/FWHM value 2 to 3 pixels. 

 

4 hours ago, NickK said:

My thinking is if you can use some maths to better estimate the noise levels.

Here they are using gems and thieves, however the same could be viewed as a pixel and the composition of noise - how much is signal, signal overlapping from elsewhere (PSF) and the various noise.

 

For me it is just simple statistics, average value, standard deviation....

[NickK]

47 minutes ago, han59 said:

I'm surprised you need more then single precision. In my program I define the arrays as single precision by purpose since memory requirements quickly increase if you have a few images in memory.

If your looking for a new challenge, develop and document a de-mosaic routine for astro OSC images.  Especially when stars are saturated in the center and the stars are small, HFD/FWHM value 2 to 3 pixels. 

 

For me it is just simple statistics, average value, standard deviation....

 

The issues come when the psf for an extended saturated star is deconvoluted. As you're now forced to either rescale  or expand the range. I want to keep the detail.

On the OSC, it's probably better running a video with a constant movement across the matrix. The frames and offsets from the movement should give a better image akin to super resolution filling in the gaps of missing colour data.

I could say average value removing the outliers. It may be faster, however you have information that is not being used.

[NickK]

For the record - it appears R and Octave (Matlab clone) are single threaded. The libraries that perform functions may be multi-threaded in the function if they support it (unlikely for most) and Octave provides ndpar_arrayfun() and para_arrayfun() for parallelisation with n-dimensional arrays. Python you still have to essentially manage the parallel processes operating on the data as you would in c++ or objective-c++.