Working with Star Tools – From Nikon D7100 to DSS to Star Tools – Advice?

[Oddsocks]

I have been following this thread for a while as I occasionally use a Nikon D90 but have not experienced the problems that you are having although I use MaximDL or Pixinsight for stacking, not DSS, and I use the Nikon ViewNX-2 utility to open and convert my raw images to 16 bit TIF with no LZW compression, an updated version was released last week by the way, v2.10.5

I downloaded your stacked file but couldn't do much with it in Startools and moved it over to Photoshop CS6 for a look see.

The first problem with the image is the bit depth is too small, only 8 bit with a maximum 255 levels, which seems to be why Startools is struggling with it, you need 16 bit images to get the best results.

The second problem is looking at the histogram for the RGB channels the main peaks are beginning way over to the right, all of the data seems to exist between the 190 to 225 levels.

I don't know if this is a problem with the DSS stacking, the original exposure was too long or you were imaging with very bad light pollution but I would expect the background level on an 8 bit image to be around the 20 mark so that when you look at the histograms in PS the left side of the main peak should be approx 1/4 to 1/3 distance from the left side of the histogram window.

If you get a chance and are able to upload to dropbox I would be interested in seeing the raw images, say just 4 or 5 of the lights only, no darks bias or flats, and I will run them through my version of ViewNX-2 and Maxim DL/PI/Startools to see what difference it makes.

I might try DSS as well but I don't think it will run with your big files on my iMac - Parallels desktop - Windows Vista virtual machine, never seems to be enough memory free for the virtual partition.

William.

[redarmy27]

I have been following this thread for a while as I occasionally use a Nikon D90 but have not experienced the problems that you are having although I use MaximDL or Pixinsight for stacking, not DSS, and I use the Nikon ViewNX-2 utility to open and convert my raw images to 16 bit TIF with no LZW compression, an updated version was released last week by the way, v2.10.5

I downloaded your stacked file but couldn't do much with it in Startools and moved it over to Photoshop CS6 for a look see.

The first problem with the image is the bit depth is too small, only 8 bit with a maximum 255 levels, which seems to be why Startools is struggling with it, you need 16 bit images to get the best results.

The second problem is looking at the histogram for the RGB channels the main peaks are beginning way over to the right, all of the data seems to exist between the 190 to 225 levels.

I don't know if this is a problem with the DSS stacking, the original exposure was too long or you were imaging with very bad light pollution but I would expect the background level on an 8 bit image to be around the 20 mark so that when you look at the histograms in PS the left side of the main peak should be approx 1/4 to 1/3 distance from the left side of the histogram window.

If you get a chance and are able to upload to dropbox I would be interested in seeing the raw images, say just 4 or 5 of the lights only, no darks bias or flats, and I will run them through my version of ViewNX-2 and Maxim DL/PI/Startools to see what difference it makes.

I might try DSS as well but I don't think it will run with your big files on my iMac - Parallels desktop - Windows Vista virtual machine, never seems to be enough memory free for the virtual partition.

William.

Sounds good.  The developer had me redo an image and send it to him; I can provide you with that one and send it or redo another one for sure and send it to you.  I've been using Capture NX-D to resize and convert my images.  I was shooting in some intense light pollution that night.

How do you like Maxim DL or PI for stacking?

I appreciate the help and insight.

Jake

[Oddsocks]

I prefer Pixinsight Jake because it works well on my iMac (27" 5k retina i7 with16gb ram, OS X Yosemite).

I wanted to leave Windows behind having lost fifteen years of astro and family pictures when my old Windows 7 machine died and the new one refused to recover the windows back up I had been making religiously every week, seems Windows backup didn't like striped raid arrays! Swore I'd never have a Windows machine again after that...

I'm not certain but I think PI have ended support for 32 bit OS, as long as you have a 64 bit platform and reasonable processor and memory then PI produces great images but it is steep learning curve, the user interface is very mathematical but if you put the effort in to learn then it is very rewarding. Harry's tutorials on the web were very helpful when I began with PI

It is interesting that sometimes when I do get good source data and process the same source image in both PI and Startools then the results are virtually identical but in Startools all the maths is hidden away from view, but given marginal data then PI has the edge as it is possible to execute a finer level of control.

Maxim DL I still have for "old times sake" running on a virtual Windows partition on the Mac as it used to run my observatory but a few years ago I switched to a Paramount MX mount and the Sky X software to run the observatory together with a QSI camera, all of which run natively on the Mac OS and Maxim has been left behind, I'm sure if I were still Windows based then Maxim would be my choice for observatory control and acquisition but PI beats Maxim for post processing by a long way now.

Sometimes if I do have a difficult set of subs to stack I do return to Maxim on the virtual Windows partition, I feel more comfortable with the stacking options and process in Maxim but probably that is just years of familiarity since I began using Maxim way back with version 4 and I've only been using PI for a little over a year.

If you were shooting your imges with heavy light pollution that probably explains the high background levels, the exposure needed to be shorter, and many more of them, to avoid pushing all the data to the top of the pixel values.

If you can dropbox three or four of the unprocessed RAW light frames then I can try and look to see why your stacked result is only 8 bit, I have got a copy of DSS on the virtual windows partition that I can use as an experiment.

William

[lensman57]

Pi for sure gives you more control through the thousands of variables available to the user but these are only of use if the user can understand their functions and do call me an idiot but unless one is a mathematician the function of most of these controls are totally alien to most people. I use Pi more often than ST but I can tell you that at least for me using Decon and NR is a lot easier and controllable with ST than with Pi. The ace up the Pi sleeve is the DBE tool which if used properly is superior to the vague Wipe module of the ST and I feel that Pi colour calibration routine gives a slightly more consistent result than ST but this is a personal opinion and not fact based. I do believe that the developer of ST, Ivo Jager , is working constantly to improve the software as are the team of Pi so these are all evolving at great pace. Both are very competent AP software and considering the cost of lets say PS both look like a bargain, ST in particular.

Regards,

A.G

[redarmy27]

I prefer Pixinsight Jake because it works well on my iMac (27" 5k retina i7 with16gb ram, OS X Yosemite).

I wanted to leave Windows behind having lost fifteen years of astro and family pictures when my old Windows 7 machine died and the new one refused to recover the windows back up I had been making religiously every week, seems Windows backup didn't like striped raid arrays! Swore I'd never have a Windows machine again after that...

I'm not certain but I think PI have ended support for 32 bit OS, as long as you have a 64 bit platform and reasonable processor and memory then PI produces great images but it is steep learning curve, the user interface is very mathematical but if you put the effort in to learn then it is very rewarding. Harry's tutorials on the web were very helpful when I began with PI

It is interesting that sometimes when I do get good source data and process the same source image in both PI and Startools then the results are virtually identical but in Startools all the maths is hidden away from view, but given marginal data then PI has the edge as it is possible to execute a finer level of control.

Maxim DL I still have for "old times sake" running on a virtual Windows partition on the Mac as it used to run my observatory but a few years ago I switched to a Paramount MX mount and the Sky X software to run the observatory together with a QSI camera, all of which run natively on the Mac OS and Maxim has been left behind, I'm sure if I were still Windows based then Maxim would be my choice for observatory control and acquisition but PI beats Maxim for post processing by a long way now.

Sometimes if I do have a difficult set of subs to stack I do return to Maxim on the virtual Windows partition, I feel more comfortable with the stacking options and process in Maxim but probably that is just years of familiarity since I began using Maxim way back with version 4 and I've only been using PI for a little over a year.

If you were shooting your imges with heavy light pollution that probably explains the high background levels, the exposure needed to be shorter, and many more of them, to avoid pushing all the data to the top of the pixel values.

If you can dropbox three or four of the unprocessed RAW light frames then I can try and look to see why your stacked result is only 8 bit, I have got a copy of DSS on the virtual windows partition that I can use as an experiment.

William

Pi for sure gives you more control through the thousands of variables available to the user but these are only of use if the user can understand their functions and do call me an idiot but unless one is a mathematician the function of most of these controls are totally alien to most people. I use Pi more often than ST but I can tell you that at least for me using Decon and NR is a lot easier and controllable with ST than with Pi. The ace up the Pi sleeve is the DBE tool which if used properly is superior to the vague Wipe module of the ST and I feel that Pi colour calibration routine gives a slightly more consistent result than ST but this is a personal opinion and not fact based. I do believe that the developer of ST, Ivo Jager , is working constantly to improve the software as are the team of Pi so these are all evolving at great pace. Both are very competent AP software and considering the cost of lets say PS both look like a bargain, ST in particular.

Regards,

A.G

Thanks for the replies guys; I really appreciate them.

I'm going to keep on messing with ST a bit even though I'm not able to get good "virgin data"; I still need to convert the files to a Tiff and then also resize the file for it to be stacked correctly (at least with DSS).

The funny part is that I already own "normal" photography processing programs so I'm pretty familiar with PS and how to edit "normal" portraits, landscapes, ect.  I can light a whole photography shoot, give readings out the wazoo and create dynamic images, but the process of capturing a celestial object thousands of light years away from us and then processing it is a completely new learning experience.

I almost want to stay in PS and editing my photos that way, but from what I've read of AP processing programs, PS is mainly meant for normal photos; specialized programs are much more sensitive to the data in an AP photo.  Thankfully there are trial versions out there for me to try, but as you've said, the learning curve is pretty steep with some.

Through formatting my images ahead of upload, DSS does work, but I've thought about trying out Nebulosity.  I've noticed the usage/learning curve is a bit higher with that program as well.

So many programs, so little time!

I'll keep tinkering around.  I've been under the weather the past few days due to a serious sinus infection...argh.

Jake

[Christopher Davenport]

Thanks for the replies guys; I really appreciate them.

I'm going to keep on messing with ST a bit even though I'm not able to get good "virgin data"; I still need to convert the files to a Tiff and then also resize the file for it to be stacked correctly (at least with DSS).

The funny part is that I already own "normal" photography processing programs so I'm pretty familiar with PS and how to edit "normal" portraits, landscapes, ect. I can light a whole photography shoot, give readings out the wazoo and create dynamic images, but the process of capturing a celestial object thousands of light years away from us and then processing it is a completely new learning experience.

I almost want to stay in PS and editing my photos that way, but from what I've read of AP processing programs, PS is mainly meant for normal photos; specialized programs are much more sensitive to the data in an AP photo. Thankfully there are trial versions out there for me to try, but as you've said, the learning curve is pretty steep with some.

Through formatting my images ahead of upload, DSS does work, but I've thought about trying out Nebulosity. I've noticed the usage/learning curve is a bit higher with that program as well.

So many programs, so little time!

I'll keep tinkering around. I've been under the weather the past few days due to a serious sinus infection...argh.

Jake

Hi Jake,

PS is absolutely top notch for astro. Startools has a couple of good functions, but I would not be able to do anything without PS.

Google Noels tools set of actions for PS.

I have not tried pix insight yet but if you serious about this game it is the bees knees.

Check the link in my SIG and look at the processing tab, I made a short how to in PS if you interested.

[Oddsocks]

Replying to Jake (redarmy27) ....et al...

First of all, this is a long post so apologies in advance.

Note that all the images in the post were processed from only a very small number of Jake's Nikon DSLR NEF RAW light frames with no calibration frames and are compressed to jpg for direct view in this post, for that reason the final processed images displayed here do not represent what the image processing packages are truly capable of.

Jake posted originally that he needed help with Startools to process frames from his Nikon D7100.

Jake had converted his stack of frames using the Nikon's own NEF RAW conversion software, loaded them into the free stacking software package DSS and then loaded the finished, calibrated image into Startools for final processing, besides having computing issues actually handling the large image files he was also unable to produce any kind of useable result during Startools processing.

Jake sent me five of his Nikon NEF RAW light frames to see if I could determine what the problem was.

After experimenting with the raw frames in my own copy of the Nikon RAW converter software ViewNX-2 the problem has been found that the Nikon software is applying a compression algorithm to the NEF to TIFF conversion process. Even though 16 Bit Tiff with no compression is selected for the conversion the resultant TIFF image is non-linear stretched and is variable depending on the value of the average background pixel value for the whole frame.

In Jake's light frames he had a very high background light pollution level and a very small image signal so the resulting Nikon NEF RAW to TIFF conversion was over stretched and unsuitable after stacking for use by Startools.

This first image is the result of five RAW NEFs' converted by Nikon ViewNX-2 into 16 bit Tiff  and stacked in DSS.  

In the accompanying histogram notice that even though the file should be non compressed and non-stretched that the main RGB channel peaks are stretched and pushed to the right, there is a high background level and there is no real black in the image.

 

 

This second image is the result of five RAW NEFs' converted by Nikon ViewNX-2 into 16 bit TIFF and stacked in Pixinsight, the image is almost identical to the DSS version, ignore the slight registration error, I did not spend to much time fine tuning the PixInsight stacking parameters. Again the histogram shows a non linear image with the RGB peaks to the right and with a high background level.

 

 

The third image is the result of five RAW NEF's converted and stacked entirely within PixInsight and outputted as a 16 bit TIFF, notice now the image is linear, the background is black and the RGB histogram peaks are well to the left.

 

 

The fourth image is the result of five RAW NEF's converted and stacked entirely within MAXIM DL6, again the image is linear, the background is black and the RGB histogram peaks are well to the left although MAXIM seems to have set a different black point during the NEF conversion and stack process and lost some data as a result.

 

 

The result of this experiment shows that for Nikon camera's using Nikon's own NEF converter software the output TIFF file is never linear.

If the original RAW image was compromised by high background light pollution and the signal to background level very low then the resulting TIFF's, even after stacking and calibrating with darks, flats and bias would be virtually unusable by most astronomy image processing packages although Photoshop could make a good picture from the final stacked TIFF as it doesn't care whether the image is linear or not.

DSS is doing it's job properly it is not altering the incoming data during stacking, the problem occurs within Nikons ViewNX RAW NEF to TIFF conversion process.

It is interesting that the true astro image processing packages with built in NEF converters do keep the TIFF data linear where Nikon's ViewNX does not.

The only packages I know well are PixInsight and Maxim and they handled the NEF files well, I expect some of the other packages available do to but I don't have a copy to test those.

Jake could try a few of the other free NEF to TIFF converters that are available to see if any of them will produce a linear TIFF file.

In the absence of an obvious software package to suggest for linear NEF to TIFF conversion it might be that Jake would find PixInsight a good one-stop-shop since he can take the RAW NEF straight into PixInsight, stack debayer and process to final image in the one package with a little tweaking in Photoshop afterwards if necessary.

As A.G.(lensman57) mentioned in an earlier post Startools is also a good processing package but it can not convert or stack the RAW NEF files from Nikon.

If Jake does elect to try PixInsight then I recommend watching Harrys Tutorials to get off the mark quickly:

http://www.harrysastroshed.com/index.html

Finally after a long winded post just for fun I include a pair of final processed images from the five uncalibrated frames.

The top picture is the result of the full sequence within Pixinsight of RAW NEF stack and process, following the beginners recommended path as laid out by Harry's tutorials.

The bottom picture is the Pixinsight RAW NEF stack converted to TIFF, exported to and processed in Startools.

Just a reminder that these images are low quality because the source data was rather sparse, only five frames of Jakes heavily polluted lights with no calibration frames and I did not spend an awful lot of time on the process so they do not represent the image quality that can be achieved normally by either PixInsight or Startools.

I think I made a slightly better job with the PixInsight version but then I use it much more than Startools and did not really try to vary the Startools default settings during processing too much, given a little more time (and patience) I think I could have improved the Startools version significantly.

(the images are a small crop of the original stack as I wanted to keep the processing time short and not spend too much time with star masks etc!)

 

 

William.

[lensman57]

Replying to Jake (redarmy27) ....et al...

First of all, this is a long post so apologies in advance.

Note that all the images in the post were processed from only a very small number of Jake's Nikon DSLR NEF RAW light frames with no calibration frames and are compressed to jpg for direct view in this post, for that reason the final processed images displayed here do not represent what the image processing packages are truly capable of.

Jake posted originally that he needed help with Startools to process frames from his Nikon D7100.

Jake had converted his stack of frames using the Nikon's own NEF RAW conversion software, loaded them into the free stacking software package DSS and then loaded the finished, calibrated image into Startools for final processing, besides having computing issues actually handling the large image files he was also unable to produce any kind of useable result during Startools processing.

Jake sent me five of his Nikon NEF RAW light frames to see if I could determine what the problem was.

After experimenting with the raw frames in my own copy of the Nikon RAW converter software ViewNX-2 the problem has been found that the Nikon software is applying a compression algorithm to the NEF to TIFF conversion process. Even though 16 Bit Tiff with no compression is selected for the conversion the resultant TIFF image is non-linear stretched and is variable depending on the value of the average background pixel value for the whole frame.

In Jake's light frames he had a very high background light pollution level and a very small image signal so the resulting Nikon NEF RAW to TIFF conversion was over stretched and unsuitable after stacking for use by Startools.

This first image is the result of five RAW NEFs' converted by Nikon ViewNX-2 into 16 bit Tiff  and stacked in DSS.  

In the accompanying histogram notice that even though the file should be non compressed and non-stretched that the main RGB channel peaks are stretched and pushed to the right, there is a high background level and there is no real black in the image.

Raw_Stack_NEF_Nik_NX2_Tiff_16bit_DSS.jpg

Histogram_NEF_Nik_NX2_Tiff_16bit_DSS.jpg

This second image is the result of five RAW NEFs' converted by Nikon ViewNX-2 into 16 bit TIFF and stacked in Pixinsight, the image is almost identical to the DSS version, ignore the slight registration error, I did not spend to much time fine tuning the PixInsight stacking parameters. Again the histogram shows a non linear image with the RGB peaks to the right and with a high background level.

Raw_Stack_NEF_Nik_NX2_Tiff_16bit_Pixinsight.jpg

Histogram_NEF_Nik_NX2_Tiff_16bit_Pixinsight.jpg

The third image is the result of five RAW NEF's converted and stacked entirely within PixInsight and outputted as a 16 bit TIFF, notice now the image is linear, the background is black and the RGB histogram peaks are well to the left.

RAW_Stack_NEF_Pixinsight_16bit.jpg

Histogram_NEF_Pixinsight_16bit.jpg

The fourth image is the result of five RAW NEF's converted and stacked entirely within MAXIM DL6, again the image is linear, the background is black and the RGB histogram peaks are well to the left although MAXIM seems to have set a different black point during the NEF conversion and stack process and lost some data as a result.

Raw_Stack_NEF_Maxim_16Bit.jpg

Histogram_NEF_Maxim_16bit.jpg

The result of this experiment shows that for Nikon camera's using Nikon's own NEF converter software the output TIFF file is never linear.

If the original RAW image was compromised by high background light pollution and the signal to background level very low then the resulting TIFF's, even after stacking and calibrating with darks, flats and bias would be virtually unusable by most astronomy image processing packages although Photoshop could make a good picture from the final stacked TIFF as it doesn't care whether the image is linear or not.

DSS is doing it's job properly it is not altering the incoming data during stacking, the problem occurs within Nikons ViewNX RAW NEF to TIFF conversion process.

It is interesting that the true astro image processing packages with built in NEF converters do keep the TIFF data linear where Nikon's ViewNX does not.

The only packages I know well are PixInsight and Maxim and they handled the NEF files well, I expect some of the other packages available do to but I don't have a copy to test those.

Jake could try a few of the other free NEF to TIFF converters that are available to see if any of them will produce a linear TIFF file.

In the absence of an obvious software package to suggest for linear NEF to TIFF conversion it might be that Jake would find PixInsight a good one-stop-shop since he can take the RAW NEF straight into PixInsight, stack debayer and process to final image in the one package with a little tweaking in Photoshop afterwards if necessary.

As A.G.(lensman57) mentioned in an earlier post Startools is also a good processing package but it can not convert or stack the RAW NEF files from Nikon.

If Jake does elect to try PixInsight then I recommend watching Harrys Tutorials to get off the mark quickly:

http://www.harrysastroshed.com/index.html

Finally after a long winded post just for fun I include a pair of final processed images from the five uncalibrated frames.

The top picture is the result of the full sequence within Pixinsight of RAW NEF stack and process, following the beginners recommended path as laid out by Harry's tutorials.

The bottom picture is the Pixinsight RAW NEF stack converted to TIFF, exported to and processed in Startools.

Just a reminder that these images are low quality because the source data was rather sparse, only five frames of Jakes heavily polluted lights with no calibration frames and I did not spend an awful lot of time on the process so they do not represent the image quality that can be achieved normally by either PixInsight or Startools.

I think I made a slightly better job with the PixInsight version but then I use it much more than Startools and did not really try to vary the Startools default settings during processing too much, given a little more time (and patience) I think I could have improved the Startools version significantly.

(the images are a small crop of the original stack as I wanted to keep the processing time short and not spend too much time with star masks etc!)

M42_Pixinsight_NEF_Converted_Processed.jpg

M42__Pixinsight_NEF_Converted_Startools_Proc.jpg

William.

All said and done, these are digital cameras designed for very short exposure terrestrial photography. What we expect them to do is well beyond their design envelope so do not be surprised if Canon apply some clever noise reduction to the Raw data before they are written to CR2 or Nikon apply star eating or clever compression to the NEF files. If you want a proper unadulterated raw file then get a CCD and save in fits format.

A.G

[Christopher Davenport]

@William,

Superb post, with some clean up this should really be put as a sticky in the camera section. Under using a Nikon for Astro.

@A.G.,

Stating the obvious really CCD always wins hands down.

I have always been a cannon fanboy, so switch to Astro I stuck to what I know. But I have mates who claim that Nikon are more sensitive, which may be true, but they don't exactly promote for astro usage the way cannon does - witness the 60Da.

I am really impressed with what has been achieved with Startools, maybe I should give it a second chance.

Although if I am going to learn something new should it just be pixinsight.

Been holding off until I can afford a CCD.

[redarmy27]

Good morning guys!
 

This post has been wonderful in helping me with addressing the issues I’ve been having.

I know I’ve been talking to William back and forth through messages, but again thank you so much for going into such detail with that post!  After reading it, I’m very tempted to give Pixinsight a run, especially since I can take the images straight from my camera and dump them into the program to be stacked and then processed. 

My main point of contention is would it be worth it though?  And furthermore, would the program struggle with my NEF images if I imported dozens of them in there at a time?

As I’m only a couple months into this hobby, and I’m not sure if I’d ever dive in and get a full on scope with a CCD camera.  I think eventually I’d like to, but for now I’m bound to using my camera for both terrestrial footage and whatever I can get with my tripod and tracker. (I just bought an engagement ring –wow, did my wallet cry and am getting ready for the trip to Hawaii where I plan on popping the question and get some great photos!) I’m considering PI because it appears to be a one stop shop for my needs.

Upon waking up after being in my medicine induced slumber, I did some research regarding other file converters and I couldn’t land any that didn’t have some sort of compression that it ran.  As much as I’m gritting my teeth to spend over $200 on a processing software, if it works with what I currently have and I can also expand on it, I think it might be worth it.

I did try Nebulosity a little bit last night and it seemed to do alright, although I’ve heard/read that PI is a bit harder to learn but much better.   I guess I’ll have to give the PI a trial run and see what I can do with it.

This thread has been a big help.  I really need to improve on my capturing methods.  I have a Bahtinov mask coming in today for my lens and I’m looking around for a good light pollution filter on my 77mm lens.  In the meantime, what could I do with my Nikon to improve my odds of getting decent captures?  Keep the exposures shorter?  Could I take my light frames or would be adding more than 40 or so be diminishing returns?  I live in a metro area but I can get to a decent spot about 35-45 minutes away and to an even better spot about an hour away.  I will be looking at the PS information that Chris provided to make the most of what I currently have.
 

Thank you guys so much!

Jake

[lensman57]

Good morning guys!

This post has been wonderful in helping me with addressing the issues I’ve been having.

I know I’ve been talking to William back and forth through messages, but again thank you so much for going into such detail with that post!  After reading it, I’m very tempted to give Pixinsight a run, especially since I can take the images straight from my camera and dump them into the program to be stacked and then processed. 

My main point of contention is would it be worth it though?  And furthermore, would the program struggle with my NEF images if I imported dozens of them in there at a time?

As I’m only a couple months into this hobby, and I’m not sure if I’d ever dive in and get a full on scope with a CCD camera.  I think eventually I’d like to, but for now I’m bound to using my camera for both terrestrial footage and whatever I can get with my tripod and tracker. (I just bought an engagement ring –wow, did my wallet cry and am getting ready for the trip to Hawaii where I plan on popping the question and get some great photos!) I’m considering PI because it appears to be a one stop shop for my needs.

Upon waking up after being in my medicine induced slumber, I did some research regarding other file converters and I couldn’t land any that didn’t have some sort of compression that it ran.  As much as I’m gritting my teeth to spend over $200 on a processing software, if it works with what I currently have and I can also expand on it, I think it might be worth it.

I did try Nebulosity a little bit last night and it seemed to do alright, although I’ve heard/read that PI is a bit harder to learn but much better.   I guess I’ll have to give the PI a trial run and see what I can do with it.

This thread has been a big help.  I really need to improve on my capturing methods.  I have a Bahtinov mask coming in today for my lens and I’m looking around for a good light pollution filter on my 77mm lens.  In the meantime, what could I do with my Nikon to improve my odds of getting decent captures?  Keep the exposures shorter?  Could I take my light frames or would be adding more than 40 or so be diminishing returns?  I live in a metro area but I can get to a decent spot about 35-45 minutes away and to an even better spot about an hour away.  I will be looking at the PS information that Chris provided to make the most of what I currently have.

Thank you guys so much!

Jake

Pi can be as straight forward as u like but you won't get the best out of it. To do Pi properly a lot of time and effort has to be put into learning the hundreds of functions that it has to offer. I only use about 20% of the available functions as I like to keep things simple but in depth Pi is anything but.

A.G

[redarmy27]

Pi can be as straight forward as u like but you won't get the best out of it. To do Pi properly a lot of time and effort has to be put into learning the hundreds of functions that it has to offer. I only use about 20% of the available functions as I like to keep things simple but in depth Pi is anything but.

A.G

I don't mind learning at all as I've had to do the same with PS and learning the various functions needed for processing photos.  I enjoy learning new things; I just want to make the most of what I currently have.

I'll keep looking at it!

Jake

[lensman57]

I don't mind learning at all as I've had to do the same with PS and learning the various functions needed for processing photos.  I enjoy learning new things; I just want to make the most of what I currently have.

I'll keep looking at it!

Jake

You will enjoy the experience for sure.

A.G

[Willi1972]

Another thought on this is that there are settings that can be applied to RAW file settings on the D7100. My RAWs work in DSS and my settings are as follows :

Shooting menu, NEF (RAW) recording, type - lossless compressed, NEF bit depth = 14 bit.

If these settings are the same with your images, I can see no reason why dss did not work with them. Feel free to send me a few NEFs, and I'll see if they work on my laptop.

[knobby]

If I remember correctly, there is a tick box in NX2 (apply camera settings/ settings file) this should be unticked to get a fully raw image

[lensman57]

If I remember correctly, there is a tick box in NX2 (apply camera settings/ settings file) this should be unticked to get a fully raw image

Nikon NEF files are not RAW files as we understand them. A lot goes on behind the scene that I do not understand for one. A look into the Nikon Hackers Project might help one to  understand this. There are firmware patches available along the lines of Canon's Magic Lantern and a it is  worthwhile to see if these patches make any difference to the written files.

A.G

[Oddsocks]

I have been looking at DSS and the way it stacks and registers images and think the basic issue revolves around just how much memory your PC has and whether you have a 32bit or 64bit OS.

The RAW NEF file that you import into DSS has to be decompressed and debayered before it can be aligned and stacked.

The RAW NEF from a Nikon D7100 is around 30Mb in size but decompressed internally within DSS to a TIFF or FIT format, which it would have to be for alignment and stacking, the file become a whopping 150Mb.

Now imagine you are importing to DSS, RAW NEF's, twenty lights, twenty darks, twenty bias and twenty flats, DSS would need to have 12,000Mb of RAM available, that is 12Gb!

If you only have a 32bit OS you're in trouble from the start since a 32Bit OS can only address a maximum of 4Gb RAM and would need to start using a page file on the hard drive as virtual memory and the processing speed will drop to a crawl.

When I processed Jake's five lights on my primary computer I was using an iMac with 16Gb of RAM, and four core hyper-threaded processor, albeit using a virtual copy of windows via Parallels desktop which can not use all of the host machines memory, but even so the stacking process took only around three minutes.

Yesterday I borrowed a 32Bit Windows Xp machine with 2Gb of RAM and a two core non hyper-threading processor and loaded Jake's five NEF RAW's into DSS on that and stacking and alignment took 12 minutes with almost constant disk activity showing that the page file was being used extensively on the HD.

Then I tried it with some NEF's from my own Nikon D90 and imported just ten lights, ten darks and ten bias but no flats, now the D90 NEF's are only half the size of the D7100's that decompressed to TIF become around the 70Mb size, even so DSS took 26 minutes to process the final image and once again the HD was permanently active showing the page file was in use because thirty decompressed 70Mb files inside DSS would need 2.1Gb of RAM but the machine I was using only had 2Gb total!

So the conclusion I came to is the experience of using DSS with large DSLR RAW files is going to be different for everybody and depends on so many variables that one size will not fit all...

In the case of having to process such large files in a limited working space the only answer is to break down the stacking process into smaller steps by working on smaller groups of files at any one time.

If you have twenty RAW NEF bias frames then import just five and stack them, ensure the output file selected is FIT and rename the autosave file on the desk top to BIAS1, do the same for each of the remaining bias files so you end up with four saved files on the desk top named BIAS 1,...2....3...4 etc and then reimport these files and stack them again to make a final file named master bias.

Do the same with the flats and darks.

For the lights it is a little more complicated when working on a relatively small number of very large files since it is pointless asking DSS to apply quality rules otherwise you may only end up stacking 2 files from five inputted to DSS, I would probably set the Nikon to save NEF RAW + Jpg and evaluate the Jpg's first, choosing just the best frames for stacking and then importing the corresponding NEF's five at time to DSS with the quality rejection set at 100%, or in other words use and stack all the files loaded.

The end of this rather laborious process is that you end up with just four FIT files on the desktop, master bias, flat, dark and light then you reimport these four files back into DSS for the last time and produce the calibrated image, autosaved as FIT for further processing in Startools, Nebulosity or whatever and manually saved as TIF for use in Photoshop.

To sum up, if you have a 64bit OS, 16Gb Ram and multicore hyper threading CPU then you would be fine loading forty or so NEF RAW's from a DX-format cropped sensor DSLR direct into DSS in one go.

Anything smaller then you would have to adapt the processing steps to suit the hardware you are operating with.

William.

[Willi1972]

I think the key thing is whether it is a 64 bit os or not. My win 7 with 4GB copes quite happily with D7100 RAWs despite being a 5 year old i3. IIRC, win 7 home is only 32 bit and so, as you say, will spend a lot of time with the page file.

[redarmy27]

That's true. I'm running on a 64 bit 8gig mulitcore. I'll have to mess with the settings or take the images into stacking in smaller chunks.

[Willi1972]

It *should * easily handle 50 RAWs without breaking a sweat

[redarmy27]

True! Hence my frustration. I looked at my Nikon's settings and am running at the same as yours (raw, uncompressed, 14bit).

[redarmy27]

I'll send you some of my Neffs when I get to my place in a bit, William