Jump to content

SkySurveyBanner.jpg.21855908fce40597655603b6c9af720d.jpg

Data organisational tools


MattJenko

Recommended Posts

Hi Imagers,

How do you manage all the files generated by cumulative imaging and the associated information?

Do you save all raw + calibrated images, all the intermediate saved processed images, all the metadata (what equipment, what settings) and how do you organise your directory structures to best manage all the data? Does anyone run images though plate solving to get metadata on the images in the same way AstroBin does and do you store that in any logical database? Do you attach notes to imaging sessions so you have an idea of conditions when you revisit old data etc etc?

I am a beginner, but even so, it is getting to the point where things are a 'little congested and disorderly' and I am thinking of writing a basic metadata management system of my own to help gain a degree of control over it all, but wondered if there was something akin to AstroPlanner for observing, but aimed at imaging.

I would like to eventually make the move to some Spectroscopy and Photometry, and I am expecting to have to be considerably more organised - wondered if there were pre-existing tools to assist.

Thanks as always,

Matt

Link to comment
Share on other sites

In short, yes to all the above, unless what you got in the can is utter rubbish.  I save everything.  A night's imaging collects, what, a 500Megs to a Gig or so of data?  Man, you can by huge 2Tb hard disks now for £50 (enough to store 200 nights of imaging on - like we get that many in the UK!) and external disks for the same, so storage is dirt cheap.  Store it all in a folder called M45, Horsehead, Rossette folder, or whatever, and have a subfolder with the date of the subs and calibration data.  As long as the imaging set is decent you may want to combine it with subs you take a few years or so down the line.  I have still got my DSLR data even though I use a CCD now.  The DSLR may come in handy at some point.

Link to comment
Share on other sites

I keep all my "Raw" data and stacked images etc - Hard discs are cheap!  I also use A4 hardbound notebooks (usually two per year) where I write down all the details of what I have been doing, that way I can go back to old data easily.  It takes some discipline to note everything on every occasion but it is well worth it.  

Link to comment
Share on other sites

 I also use A4 hardbound notebooks (usually two per year) where I write down all the details of what I have been doing, that way I can go back to old data easily.  It takes some discipline to note everything on every occasion but it is well worth it.  

Great idea Roger!  And my New Year's resolution is.....? ;)

Link to comment
Share on other sites

I'd love to be able to tell you that I write down details of each imaging session but the truth is, I just rely on my FITS header data which includes platesolve information. I keep all my data files so I end up with:-

1. original FITS subframes

2. calibration frames (these are normally just flats as I use library BIAS and DARK frames)

3. calibrated and stacked master frames as FITS files

4. 16bit TIF versions 3. above

5. Multi-layered PDF file from PS3 (although frequently there is more than one version of this!)

Keeping these files allows me to re-visit and re-process from any stage in the future in the light of learning a new technique.

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue. By using this site, you agree to our Terms of Use.