What to do with 500gb of data

[Nigella Bryant]

Hi all, I need to do something with 500gb of data from files accumulated from camera. Do I just delete or transfer to separate disk drive? What do others do with old data? Sometimes I've revisited the data, mostly Avi files and redone the data as my data processing has improved. Be nice to know what others do. I'm running a 1Tb laptop for processing and acquisition but it's nearly full. 

[fifeskies]

Cheap and cheerful external HDD as an archive is what I do.

Connect it once in a while to offload data I no longer work with but "may" just want to come back to.

Doesn't need to be a fast high performance (read expensive) drive , just data dump overnight.

[Stuart1971]

Transfer to an external SSD hard drive, fairly cheap these days, with USB 3 connection...👍

[Davey-T]

I've got a stack of 1tb USB3 external drives to put stuff on, not that I can ever find it 😂

Dave

[fifeskies]

I got myself a usb connected dock for hard drives I had collected from my old PCs

these are easily dropped into the dock and used as cheap and cheerful backup devices

But like Dave above , finding what your looking for can be an issue , so clear labelling of folders helps , in particular date references.

[Nigella Bryant]

Thanks everyone, I'll get usb 3 external drive, fairly cheap these days. I'll probably never look at them again 🤣

[JamesF]

This is a problem I've been pondering on myself of late.  Perhaps it's down to my age :)

There's lots of stuff I have in electronic form -- photos, useful documents and suchlike, that I'd like to be able to save somewhere for the people who will inherit my debts when I finally shuffle off.  There doesn't seem to be a modern equivalent of the shoebox full of old family photos on top of the wardrobe though.  We need some reliable high-volume storage medium that is pretty much guaranteed to be readable (ie it won't have degraded and the technology will still be easily obtainable) in, say, fifty or one hundred years time.

It's an odd thought that we still have access to the original Magna Carta, now over 800 years old, but, say, an ebook published now and available to far more people may completely disappear without record within a few tens of years.

James

[Nigella Bryant]

2 minutes ago, JamesF said:

This is a problem I've been pondering on myself of late.  Perhaps it's down to my age

There's lots of stuff I have in electronic form -- photos, useful documents and suchlike, that I'd like to be able to save somewhere for the people who will inherit my debts when I finally shuffle off.  There doesn't seem to be a modern equivalent of the shoebox full of old family photos on top of the wardrobe though.  We need some reliable high-volume storage medium that is pretty much guaranteed to be readable (ie it won't have degraded and the technology will still be easily obtainable) in, say, fifty or one hundred years time.

It's an odd thought that we still have access to the original Magna Carta, now over 800 years old, but, say, an ebook published now and available to far more people may completely disappear without record within a few tens of years.

James

Very true James. Perhaps drawings of observations will last longer than the digital observations we make. Makes me wonder.

[Nigella Bryant]

Case in point, just processed some Mars data from this September 2020. I'd taken several data files and just looked at this today. 

[gilesco]

You don't need to get an SSD drive, I use one of these:

Seagate BarraCuda 8TB 3.5" SATA HDD/Hard Drive

Sorry- cut and paste looks a bit funny, but it's a spindle 8TB disk, for around £165, it's in my desktop system, mounted as an "archive" drive, I don't need fast access to my old data, but nice to have it, and cheaper than a SSD drive.

[Steenamaroo]

Impossible to say what will happen in 50 to 100 years but there's hardly a digital data, or analog a/v, format from the past that can't be accessed by someone, somewhere.
Enthusiasts keep old computer platforms alive and there are always public services for the restoration/recovery/backup of obsolete a/v formats.

Your job, everyone here, is to have at least two copies of everything!
That big hard drive you keep all your stuff on is great until it breaks, and they don't always need a reason to break.
Spinning media (physical disc) is a lot more volatile than SSD, particularly where movement, shock, changes in temperature, etc, are concerned, but anything can break without obvious reason.

Cost used to be an issue but you could get multiple terrabytes of storage now for not a lot of money.

For extra protection, and some amount of convenience, you can get a NAS and regularly update it with important data. That could be your shoebox.

Edited November 10, 2020 by Steenamaroo

[Luke]

I deleted most of my solar and lunar captures after stacking them and checking that they look okay, with a quick stretch and sharpen. I see your point about Mars though! There's always the paint tool, nobody will notice!

With solar I was getting so much data that it didn't seem realistic to keep all the movie files in the long run.

I recently picked up a 14TB internal drive that should keep me going for a while! Not fancy pants SSD, as it is just for storage.

I backup onto an external USB 3 12TB. And I have some smaller drives I take with me out and about, just in case something happens to the two main drives! I do wonder though about storing some stuff online, I think 1TB would do for the most important stuff.

[fifeskies]

Only trouble with cloud storage  (and I do use some myself) is that astro files are so big it takes ages to upload or download significant amounts of it.

[JamesF]

1 minute ago, fifeskies said:

Only trouble with cloud storage  (and I do use some myself) is that astro files are so big it takes ages to upload or download significant amounts of it.

Not the "only" trouble, perhaps.  Personally I don't trust any third party not to turn around one day and say "Oh, sorry, we lost all your data and we can't get it back".  Privacy is also an issue, but one that is perhaps a little easier to keep some control of.

James

[rnobleeddy]

I already used Google cloud. I've found that it works very well if I mirror a folder on my desktop with the cloud, with syncing being done behind the scenes.

 

Aside from any ethical concerns or technical constraints like slow internet, the cloud seems like an obvious and sensible choice.

[fifeskies]

yes anything I have in the cloud is

a) backed up locally

b) Not anything I need kept private

 

[DaveS]

I've started to save my image files directly to my NAS, 4 x 6TB NAS spec drives in RAID 5, a bit over 16TB nett. The files I want to work on are copied into a 480GB PCI  NVMe.