Collaborative imaging

[vernmid]

I appreciate that total integration time is key for good astro images and that that time can be spread over multiple nights.

Would it be possible to produce a collaborative image whereby the same target is imaged by multiple people at the same instant but in different locations? For example it could be arranged that a group targets M42 over a particular night (or set of nights). Could the resulting subs be combined to produce an image? Would very long integration time images be able to be produced over a shorted elapsed time scale?

Would it be worthwhile or too complex?

Just a thought :-)

[johnfosteruk]

There have been a few images posted on here with data from more than one imager. It's easy enough to do because all the data is plate solved and registered, so FOV isn't a problem. It doesn't even have to be the same scale, I believe Registar takes care of that, and other tools probably as well.

@ollypenrice @gnomus @wimvb @gorann are some of the people you want to talk to I think.

 

[ollypenrice]

Yes, it is a great idea and there is no reason for it to be done at the same time. Registar will co-register and combine images from wildly different instruments, pixel sizes, field curvatures image scales, etc etc. There is nothing it cannot do in the regard. Astro Pixel Processor can also do it, I think. As John says, quite a few of us like to work in this way.

Olly

[alexbb]

Astro Pixel Processor is advertised with the LNC (local normalization correction) and MBB (multi band blending) being ones of its great features. Check out the features and the example:
https://www.astropixelprocessor.com/registration-normalization-integration-using-ddc-lnc-mbb/

[vlaiv]

Both very simple and very complex topic, depending how you look at it.

In simple terms, you agree with someone to shoot a target, it does not even have to be on a same night (things don't change that much between different nights ), and then you combine your data. Simple

Complex version:

For effective combination of different data you need really sophisticated algorithms. In general case, data will be obtained in different conditions, using different equipment, at different resolution, with different response curves for different wavelengths and with different aberrations present in the systems.

In general case it is not exactly solvable problem. Warning, a bit of technical discussion ahead, I'll give you an example:

Even for data set taken on a single night, by single system / imager, current algorithms can't deliver optimum solution, and I don't think it is even possible to deliver optimum solution at all. One shoots target over the course of the night. At one point target will be at alt 60 while at some other point it might be at zenith - alt 90 degrees. Just simple fact that there is atmospheric extinction and that it depends on height above horizon means that two different subs will have different signal to noise ratios. Normal stacking assumes that SNRs of subs are the same, but they are not. Throw in changing conditions over the course of the night, like transparency, changing LP levels (people turn on/off their lights), etc ...

In reality for any set of data there are ideal weights that one can assign in order to combine them to the highest SNR, but we have no means to determine SNR of each frame so no way to properly calculate those weights. Only thing that we can do is some sort of SNR estimation and based on that assign weights to each sub.

Second problem is of course resolution of the system combined with seeing. In many cases people ended up discarding frames simply because seeing got worse at some point during the night and stars were bloated. At the moment we don't even have implementation to deal with different FWHM of stars when combining subs.

There are however algorithms that can deal with all problems involved in combining different data sets. Unfortunately there is no software implementation that combines all of the algorithms needed.

I've spent some time thinking and developing algorithms that can deal with such scenarios. Ideally one would like to have a software tool, similar to bit torrent, so people fire up their app, load their subs, do "local" stacking but also decide if they want to publish their work to a collaborative effort, and just leave app to do it's business. After some time each participant would have stacked and processed collaborative data on their computer .... Maybe it is simple after all

 

[wimvb]

Many astro image processing packages can handle data from different sources (fl, pixel size, fov). For a colaboration, you can either team up with someone who has the same fov and resolution, to get images that make the best use of a certain framing. (This way you increase integration time.) Or you can combine a wider fov with a more detailed, smaller fov to give localised higher resolution. For example a wide field image with a detailed galaxy. To practise, you could combine your own data with that from a public data pool. (Good for practising, but I would not publish these images unless I had a permission from the data provider to use and publish his/her data, and then only with full credit given.)

Pooling data basically gives you access to a dual scope rig for the price of one.

[DarkAntimatter]

So if we pool 1000's of amateur scope data, can we eventually get more seeing power than Hubble?

[vlaiv]

5 minutes ago, DarkAntimatter said:

So if we pool 1000's of amateur scope data, can we eventually get more seeing power than Hubble?

No way we can match Hubble in resolution, that is simply impossible.

If we use readily available software that does not account for different PSF, you end up with resolution somewhere between best and worst. So If someone submits subs in poor seeing with small aperture scope it will impact overall result.

There is however way to "correct" each sub to certain resolution, it has to be above diffraction limit of smallest aperture subs submitted, but such corrected frames have their SNR lowered, and will contribute considerably less to overall image.

We are limited by laws of physics to create sharp high resolution image no matter how many small telescopes we use.

Could we reach deep field at lower resolution? Maybe, but I think that we would need much more than 1000 people submitting. If we take on average that 4h data will be submitted per person, that is something like 4000 hours. We can also take an average of 150mm aperture, which is 16 times less than Hubble by diameter, so we need x256 more exposure time to approximately match "depth" of image. So 4000 hours is something like 166.666 days, and Hubble would match that in 15-16 hours.

Deep field exposures by Hubble measure something like couple of weeks worth of exposure, so there is significant difference.

But I do think that we can go as deep with 1000 people as very large ground based telescopes - they are also limited to single night imaging and need to gather as much data possible, so I don't think they spend much time on single target.

[DarkAntimatter]

2 minutes ago, vlaiv said:

No way we can match Hubble in resolution, that is simply impossible.

If we use readily available software that does not account for different PSF, you end up with resolution somewhere between best and worst. So If someone submits subs in poor seeing with small aperture scope it will impact overall result.

There is however way to "correct" each sub to certain resolution, it has to be above diffraction limit of smallest aperture subs submitted, but such corrected frames have their SNR lowered, and will contribute considerably less to overall image.

We are limited by laws of physics to create sharp high resolution image no matter how many small telescopes we use.

Could we reach deep field at lower resolution? Maybe, but I think that we would need much more than 1000 people submitting. If we take on average that 4h data will be submitted per person, that is something like 4000 hours. We can also take an average of 150mm aperture, which is 16 times less than Hubble by diameter, so we need x256 more exposure time to approximately match "depth" of image. So 4000 hours is something like 166.666 days, and Hubble would match that in 15-16 hours.

Deep field exposures by Hubble measure something like couple of weeks worth of exposure, so there is significant difference.

But I do think that we can go as deep with 1000 people as very large ground based telescopes - they are also limited to single night imaging and need to gather as much data possible, so I don't think they spend much time on single target.

Interesting, thank you.  

So just to compute some silly numbers, using your 166 2/3 amateur days = 15.5 Hubble hours, we'd need (166 2/3 * 24) / 15.5 x 1000 = 250065 amateurs of average 150mm aperature to match what Hubble can do, hour per hour.  Neglecting the problem of combining all those images with something like good efficiency.  

So I guess they should proceed with the Jack Web after all...

[vlaiv]

10 hours ago, DarkAntimatter said:

Interesting, thank you.  

So just to compute some silly numbers, using your 166 2/3 amateur days = 15.5 Hubble hours, we'd need (166 2/3 * 24) / 15.5 x 1000 = 250065 amateurs of average 150mm aperature to match what Hubble can do, hour per hour.  Neglecting the problem of combining all those images with something like good efficiency.  

So I guess they should proceed with the Jack Web after all...

Not really, if you want to calculate hour for hour, then you must not assume 4h limit per amateur. It is simple matter of aperture area.

So Hubble has 2.4m diameter mirror, and we are going to oversimplify things and neglect secondary obstruction, fact that SNR is not quite linear in time, but just to get rough estimate of amateurs per Hubble . We can say that average amateur has 150mm of aperture. That is 16 times difference by diameter or 256 times difference by area. So 256 amateurs working around the clock would be sort of "equal" to Hubble in collecting power. But problem of course is that Hubble can collect 24/7 and amateurs can collect only during nighttime, and let's assume that amateurs are spread around the world so at any given time someone will be able to collect data, and on average amateur is able to collect 4h of data daily. That means that for such scenario 8 times as many amateurs is needed. Now we have 2048 amateurs/Hubble. You can push this even further, and estimate on average how many clear nights you have per year, ....

Original estimate assumed that any given amateur will spend total 4 hours on target on average and stop collecting data, so it is not usable for hour per hour comparison.

 

[wimvb]

11 hours ago, DarkAntimatter said:

So if we pool 1000's of amateur scope data, can we eventually get more seeing power than Hubble?

Never. The atmosphere is in the way for all those terrestrial amateur scopes. And the large Hubble mirror has far better resolving power. You can't match that combination with any terrestrial telescope. But 20+ meters aperture and adaptive optics, placed on a high plateau in Chile, comes close. As in real estate, astrophotography is all about location, location, and location.

[DarkAntimatter]

6 hours ago, vlaiv said:

Not really, if you want to calculate hour for hour, then you must not assume 4h limit per amateur. It is simple matter of aperture area.

So Hubble has 2.4m diameter mirror, and we are going to oversimplify things and neglect secondary obstruction, fact that SNR is not quite linear in time, but just to get rough estimate of amateurs per Hubble . We can say that average amateur has 150mm of aperture. That is 16 times difference by diameter or 256 times difference by area. So 256 amateurs working around the clock would be sort of "equal" to Hubble in collecting power. But problem of course is that Hubble can collect 24/7 and amateurs can collect only during nighttime, and let's assume that amateurs are spread around the world so at any given time someone will be able to collect data, and on average amateur is able to collect 4h of data daily. That means that for such scenario 8 times as many amateurs is needed. Now we have 2048 amateurs/Hubble. You can push this even further, and estimate on average how many clear nights you have per year, ....

Original estimate assumed that any given amateur will spend total 4 hours on target on average and stop collecting data, so it is not usable for hour per hour comparison.

 

OK, so let's pretend we multiply by 0.5 to account for clear nights.  For me it is much less than 0.5, but for those in, say Spain or Arizona, it is more.  We we could get 10000 amateurs to contribute, we could see deeper into space than Hubble (at worse resolution) for a given number of hours real-time imaging?  

[vlaiv]

11 minutes ago, DarkAntimatter said:

OK, so let's pretend we multiply by 0.5 to account for clear nights.  For me it is much less than 0.5, but for those in, say Spain or Arizona, it is more.  We we could get 10000 amateurs to contribute, we could see deeper into space than Hubble (at worse resolution) for a given number of hours real-time imaging?  

Depends on total integration time.

We will neglect differences in equipment now, and other sources of noise like light pollution and such and just take into account aperture.

So for 2h of integration time, 256 amateurs will gather same amount of light as Hubble (given above assumptions like 150mm average amateur aperture).

If we assume that Hubble images for 8 hours, and that on average amateurs can have 4h on a given night, and that we have two consecutive days of clear skies and everyone is able to do both nights, then either:

256 amateurs will gather same amount of light in two nights, or 512 amateurs will gather that much light on a single night.

The more time you throw at imaging, more important becomes the fact that people are limited in imaging time, while Hubble can do it 24/7.

So if Hubble images for 16 hours, 256 amateurs is going to need 4 days (4 nights, 4h each night, provided that they have 4 consecutive nights clear skies), or if we want to do it in a single night - 1024 amateurs.

In reality, at least going by my average number of sessions per year, 1024 amateurs would take 1 month to match 16h of Hubble imaging time - if you account for moonlight (1/3 - 1/2 of month is no fly zone, depending on target), weather ....

And in reality even small exposures will need more than stated number of amateurs, simply because this approximation only considers light gathering, and that is only small part in overall SNR equation.

So, going deeper than Hubble is not trivial task, because to go "deeper than Hubble" - Hubble needs to just add some more imaging time, and that translates into lots of amateurs and lots of nights just waiting for clear skies

 

 

[DarkAntimatter]

3 minutes ago, vlaiv said:

Depends on total integration time.

We will neglect differences in equipment now, and other sources of noise like light pollution and such and just take into account aperture.

So for 2h of integration time, 256 amateurs will gather same amount of light as Hubble (given above assumptions like 150mm average amateur aperture).

If we assume that Hubble images for 8 hours, and that on average amateurs can have 4h on a given night, and that we have two consecutive days of clear skies and everyone is able to do both nights, then either:

256 amateurs will gather same amount of light in two nights, or 512 amateurs will gather that much light on a single night.

The more time you throw at imaging, more important becomes the fact that people are limited in imaging time, while Hubble can do it 24/7.

So if Hubble images for 16 hours, 256 amateurs is going to need 4 days (4 nights, 4h each night, provided that they have 4 consecutive nights clear skies), or if we want to do it in a single night - 1024 amateurs.

In reality, at least going by my average number of sessions per year, 1024 amateurs would take 1 month to match 16h of Hubble imaging time - if you account for moonlight (1/3 - 1/2 of month is no fly zone, depending on target), weather ....

And in reality even small exposures will need more than stated number of amateurs, simply because this approximation only considers light gathering, and that is only small part in overall SNR equation.

So, going deeper than Hubble is not trivial task, because to go "deeper than Hubble" - Hubble needs to just add some more imaging time, and that translates into lots of amateurs and lots of nights just waiting for clear skies

 

 

Thanks for replying.  I was just wondering how many amateurs it would take to start rivaling Hubble, but sounds like it's more like 1000's not 100's, in practice. 

It would be fun to have lots of amateurs image the same (dim) object some time just to see what could be done collaboratively.  But probably not enough active imagers out there to rival Hubble.  

[vlaiv]

57 minutes ago, DarkAntimatter said:

Thanks for replying.  I was just wondering how many amateurs it would take to start rivaling Hubble, but sounds like it's more like 1000's not 100's, in practice. 

It would be fun to have lots of amateurs image the same (dim) object some time just to see what could be done collaboratively.  But probably not enough active imagers out there to rival Hubble.  

Astrophotography seems to be on a rise. Equipment is getting more affordable, and a lot of people is passing on the knowledge. Also with the raising levels of light pollution more people seem to enjoy hobby in "electronically assisted" way. So I suspect that there will be more opportunity for collaboration in the future as the software for something like that gets developed.

[gorann]

I do not see Hubble as a major competitor. It has many users and they are mostly after very specific data and are very rarely aiming for pretty pictures. We amateurs are often targeting more wide field objects than Hubble, and there we will win every time with our short focal lengths. We would be beaten every time on distant galaxies but in wide field we could probaly achieve some unprecedented images if we combined our data - we could go for 1000 hours at FL 500mm on NGC7000, for example (although someone is likely to burn his/her CPU trying to stack that data)