Anyone tried running PixInsight or equivalent on a cloud-based virtual machine?

[BrendanC]

Hi all,

I've been considering getting a better machine for running PixInsight. When I start running through the specs and totting up how much it would cost, I'm well north of £3K. This, for a machine that will be average within 2-3 years, and getting long in the tooth after five.

So I was looking at the possibility of cloud services ie basically a Windows virtual machine that I can install my software on, plug my OneDrive into, and process much more quickly. The major providers such as Microsoft/Amazon/Google seem very full-on, but I do like the look of Shadow (https://eu.shadow.tech/shop/en-GB). Seems much easier to set up, and even at £30-£45 per month that's going to work out much cheaper than a new machine for at least a few months while trialling.

Any thoughts?

Thanks
Brendan

[scotty38]

As an aside I built a 7950x based machine a couple of months ago and even though I went overboard on cooling, case and power it still cost me under £2k. I could have got away with a cheaper MB too. Over £3k, must be a decent bit of kit, what's the spec?

[ampleamp]

Instead of renewing my pc, I’ve kept my 5yr old machine but recently upgraded to the extent the motherboard could take. This has meant two new m.2 drives (one as c drive and one as swop and storage). I’ve maxed out the RAM (very cheaply as it is now deemed old) and I also swopped out the video card for a second hand NVIDIA cuda card (RTX4000).  
The speed of general processes and scripts like WBPP which use processor and RAM have noticably improved, but the current whizzkids BlurX, StarX and NoiseX are transformed from minutes to seconds with the NVIDIA.

I was going down the rabbit hole of chasing PI benchmark numbers, but as the ££££ mounted I decided upgrading a known machine would keep me going a while yet. Benchmarking the renewed machine suggests it is decidedly average according to their lists but I don’t care. One day maybe I will transfer to PI on a Linux computer, maybe even this one 😁  

 

[Ouroboros]

11 hours ago, BrendanC said:

Hi all,

I've been considering getting a better machine for running PixInsight. When I start running through the specs and totting up how much it would cost, I'm well north of £3K. This, for a machine that will be average within 2-3 years, and getting long in the tooth after five.

So I was looking at the possibility of cloud services ie basically a Windows virtual machine that I can install my software on, plug my OneDrive into, and process much more quickly. The major providers such as Microsoft/Amazon/Google seem very full-on, but I do like the look of Shadow (https://eu.shadow.tech/shop/en-GB). Seems much easier to set up, and even at £30-£45 per month that's going to work out much cheaper than a new machine for at least a few months while trialling.

Any thoughts?

Thanks
Brendan

It’s an interesting thought. Regardless of any other considerations I’d be concerned about the upload time of all the data. That needs to be factored in when thinking about processing time. 

Have you thought of leasing a machine?  About the same cost I think. 
 

 

 

[BrendanC]

11 hours ago, scotty38 said:

As an aside I built a 7950x based machine a couple of months ago and even though I went overboard on cooling, case and power it still cost me under £2k

I'm after a laptop - I really don't want to have to sit at a workstation. I do a lot of my processing just sitting wherever I want. I know, I know, this means less bang for the buck, but it's how I work.

I have to say I'm not absolutely sure any more where I came across that £3K figure but I think it was this as a starting point: https://www.pcspecialist.co.uk/form-view/recoil-VII-17/Recoil-Ultra-17/

10 hours ago, ampleamp said:

Instead of renewing my pc, I’ve kept my 5yr old machine but recently upgraded to the extent the motherboard could take

Ditto. I could revamp my old machine too, but I want a laptop.

2 hours ago, Ouroboros said:

I’d be concerned about the upload time of all the data.

That's why it would need to interface directly with OneDrive, where I have all my data stored. I know Azure would do it, but I don't know if Shadow would. They say they work with it, but I'd need direct transfer, not download/upload.

2 hours ago, Ouroboros said:

Have you thought of leasing a machine?

That's a very interesting idea actually! Never even thought of that. Problem is, I expect most lease-hire operations just do fairly standard machines whereas I want something blazingly fast. Also, from what I've seen, cloud-based virtual machines absolutely outstrip local processing so there would still be better value going the cloud route.

Edited April 11, 2023 by BrendanC

[scotty38]

1 hour ago, BrendanC said:

I'm after a laptop - I really don't want to have to sit at a workstation. I do a lot of my processing just sitting wherever I want. I know, I know, this means less bang for the buck, but it's how I work.

Ah ok, must admit I didn't think laptop... That's a good spec though and includes a 4090 😍 Hope you're a gamer too :-)

[BrendanC]

Nope. Just want a blazingly fast machine that won't take hours to stack images and process them!

 

[scotty38]

Okey dokey but you do realise that 99% of PI won't use the GPU? I have an AMD GPU in my machine (not part of the sub £2k cost btw)  which doesn't get used by 100% of PI but my machine absolutely flies. Even the bits like Starnet that would use an Nvidia GPU are still blazingly fast.

Just a thought as a 4090 will be adding a load to the price I'd have thought.

[scotty38]

Much as you probably have found, it's WBPP, for example, that can take ages. I now run that for a laugh it's so fast. What would have taken, 5,6,7 or more hours are now done in under 20 minutes. Add on the caching and it's no issue to add a few more files or even change settings. GPU not involved no matter what you have 🙂

[BrendanC]

Interesting. I was kind of hoping I could run CUDA with an nVidia card to speed up things like BlurXTerminator and suchlike.

As you say though, it's WBPP that hurts most. 

[scotty38]

Yes you can run CUDA for the *terminator tools. Bit of config work needed but yes they're part of the 1% (made up number) I was on about. Even so you probably don't need a 4090 just for that.