star repair in PixInsight - part 2

While reprocessing old data, I came across a very instructive incidence of star over exposure. When imaging under light polluted skies, sky glow is added to any sky signal (stars, nebulae, etc). If the pollution is strong enough to over expose stars, the true star colour data is destroyed. When the sky glow is removed during post processing, this will lead to bright stars having the wrong colour.
For example, say a bright star is slightly blue in colour. Without any sky glow, it would register as RGB 0.8, 0.8, 0.95. Sky glow gives an added 0.3 in blue. This will put the star colour on sensor as RGB 0.8, 0.8, 1.0, since the maximum value for a pixel is 1.0.
When the sky glow is removed during background extraction, the value 0.2 will be subtracted, and we end up with a star's RGB values of 0.8, 0.8, 0.7. The star core has suddenly turned yellow. However, away from the core, the star will still be blue, since these areas weren't over exposed. Therefore the need to correct star colours.
During normal stretching of an image, most of the bright stars will be maximised, and have a final colour of RGB 1.0, 1.0, 1.0. However, if masked stretch is used, the stars will not be saturated, and will have a funny looking core.
Here's an example of an unstretched star, with (left) and without (right) colour repair.

When should star repair be implemented in the workflow?
Of course, before any stretch is applied, the colour must be corrected. However, there is one earlier instance where star colour matters in processing, and that is during colour calibration.
Colour calibration tries to set a white point by looking at all the stars in an image, and apply a calibration scheme that is determined by the average star colour. When the star colour is wrong due to over exposure and background extraction, the white balance after colour calibration will be off. Therefore, there is an argument for applying HSV repair prior to colour calibration. The workflow then becomes as follows:

• cropping of edges
• background extraction (ABE or DBE)
• background neutralisation
• HSV repair
• colour calibration

This image shows the effect of doing HSV repair before (left) or after (right) colour calibration. Due to the skyglow adding mainly blue to the image, several star cores had a warmer colour after DBE. This resulted in the colour calibration routine making the blue stars more intense blue. By doing the HSV repair process prior to colour calibration, stars got a more natural blue colour after stretching.

Raspberry Pi DSLR trigger

Here's a small, simple project, a remote trigger for a DSLR.
I control my telescope mount through INDI, but unfortunately I can't control my old DSLR that way. The only way I can do "automated" exposures is if I connect an intervallometer to the camera. The problem with intervallometers is however, that they only run on small batteries, and will stop working when it's cold.
Another problem is, that it's impossible to use dithering with an intervallometer and simultaneous guiding.
Here's a partial solution to these problems. I wrote a simple Python script that will run on the Raspberry Pi, The script sends exposure signals through the RPi's GPIO bus to the remote port of my camera.
To connect the Raspberry Pi to the camera, I made a small optocoupler circuit, that will isolate the camera electronics from the RPi.

The input (left) will connect to a GPIO pin (pin 18, GPIO 24) and ground (pin 20), while the ouput (right) connects to the remote port of the camera.
Here's the code.
_____________________________

#!/usr/bin/python

import RPi.GPIO as GPIO
import time
import sys

NrExposures = 1
ExposureTime = 30
TimeBetweenExposures = 6
print ' '
print 'Make sure that the camera remote port is connected to pins 18 (signal) and 20 (ground).'
print ' '
if len(sys.argv) == 1 :
   print 'No arguments provided. Will use single 30 sec exposure.'
elif len(sys.argv) == 2 :
   ExposureTime = int(sys.argv[1], 10)
   print 'Single', ExposureTime, 'seconds exposure.'
elif len(sys.argv) == 3 :
   ExposureTime = int(sys.argv[1], 10)
   NrExposures = int(sys.argv[2], 10)
   print NrExposures, 'x', ExposureTime, 'seconds exposures.'
else :
   ExposureTime = int(sys.argv[1], 10)
   NrExposures = int(sys.argv[2], 10)
   TimeBetweenExposures = int(sys.argv[3], 10)
   print NrExposures, 'x', ExposureTime, 'seconds exposures, with', TimeBetweenExposures, 'seconds delay.'

TriggerPin = 24  # Broadcom pin 24 (P1 pin 18)

GPIO.setmode(GPIO.BCM)  # Broadcom pin-numbering scheme
GPIO.setup(TriggerPin, GPIO.OUT)  # trigger pin as output

GPIO.output (TriggerPin, GPIO.LOW)
time.sleep(1)

counter = NrExposures
print ' '
print 'Start : %s' % time.ctime()
while (counter > 0):
   print '  Exposure nr', counter, 'started'
   GPIO.output(TriggerPin, GPIO.HIGH)
   time.sleep(ExposureTime)
   GPIO.output(TriggerPin, GPIO.LOW)
   print '  Exposure nr', counter, 'ended'
   counter = counter - 1
   time.sleep(TimeBetweenExposures)
   print ' '

print 'End :  %s' % time.ctime()
GPIO.output(TriggerPin, GPIO.LOW)
GPIO.cleanup()
print ' '
print 'Goodbye.'
______________________________

The script is saved as Trigger.py and is made executable:

chmod +x Trigger.py

The script takes up to three command line arguments. The first argument is the single frame exposure time in seconds. The second argument is the number of exposures to take, and the third argument is the wait time between exposures, also in seconds.
For example

./Trigger.py 1 2 3

will take 2 exposures of 1 second each with a 3 seconds wait time after each exposure. The script will take 8 seconds to run.

If only two arguments are given, the wait time will be set to a default value of 6 seconds.
If only one argument is given, this is interpreted as exposure time. Only one exposure will be taken.
If no arguments are given, the script will do a single 30 seconds exposure.

Eventually, I may try to rewrite the INDI CCD driver to control my camera, as this will give me the possibility to dither between exposures. But for now this simple script will do the job.

star repair in PixInsight

When stars start to get over exposed during data collection, their core may become a different colour than the normally exposed outer halo. When the stacked image is stretched using MaskedStretch in PixInsight, stars are stretched less than dimmer parts. This means that stars in the stretched image are not saturated, and can display an odd colour. PixInsight has a script that can correct star cores of partially saturated stars. It is called "Repaired HSV Separation", and can be found under Scripts -> Utilities. The script should be applied just before the first stretch. It will decompose the colour image into H, Sv, and V colour components, and repair the colour values of the stars.
The different components are then to be combined using the ChannelCombination process (which is under ColorSpaces).

The upper part of the dialog box is used to determine which images are to be created, while the lower part is used for repair of the H, Sv, and V channels.For best results, mainly the Repair level parameter needs to be adjusted.
Alejandro Tombolini (who brought this script to my attention), recommends to use the unrepaired V channel when combining the channels. But experiment to find out if the repaired or unrepaired V channel works best. Be sure to select the HSV colour space.

Here's how it affected one of my most recent images, a before and after shot of the Pleiades (M45). Look at the core of the brightes stars. In the original image, the cores arepink in colour, while in the repaired image, they are blue, the same colour as the outer haloes.

Correcting dark lines in dslr astro images

My Pentax DSLR suffers from dark horizontal lines when I photograph bright stars. I'm not sure about the cause of this, but it may be some reverse blooming or ADC related issue. This issue isn't uncommon for digital cameras, but it sure is a nuisance.

Alejandro Tombolini showed in one of his processing examples how to deal with these lines. Here's my adaptation of his process.

I will use the CanonBandingReduction script to correct the lines. This script works only on entire images, and can introduce an uneven background and other artefacts when used on images which do not have bands across the entire width.
I therefore start with making a preview that contains the area I want to correct. I leave some margin, because later on I will clone the preview and shrink the clone.
By dragging the preview onto the workspace, I create a new image, which I apply the CBR script on.
The next step is to make this preview the same size as the original image. For this I use the crop tool.

Set the margins such that the image becomes the correct size, with the corrected image now in the same place as the preview in the original. If the result is ok, the image is saved as xisf file.

Next I shrink the preview in the original image and create a hole in the image where the preview is. For this I use pixelmath.

A new image is created with a black patch where the (smaller) preview was.
This image is also saved.

Finally the two saved images (the corrected preview, and the uncorrected image with the black patch) are merged using GradientMergeMosaic.

And this is the corrected image

Removing hot pixels in a stacked image

Sometimes even an aggressive hot pixel filter won't remove all hot pixels. Here's a technique that can remove any residual hot pixels in a final stacked image. I use PixInsight's Morphological Transformation with a starmask to remove these nuisances.
Here's a crop of an image, showing what I'm talking about. The image was taken with a DSLR and consists of a stack of 10 sub frames exposed for 15 minutes each at ISO 800. My camera, a Pentax K20D, is getting old, and I always have lots of hot pixels in my images. Calibration removes most, but frequently a number remain after image integration. The technique which I describe here will dim the remaining pixels.

hot pixels after stacking

I start with making a Luminance copy of the image in its linear state, and apply STF to this grayscale image. Then I use the StarMask tool with a low value for Scale (typically 3 works ok) and a noise threshold of 0.5 (to be experimented with). I decrease large-scale, small-scale and compensation (1, 0, 1) and smoothness (about 6 - 8). Then apply the mask tool to the luminance copy. It may be necessary to tweak the parameters. No stars should be in the "Star-Mask" that is created.
When I'm satisfied, I apply the mask to the original colour image.
For pixel removal I use Morphological Transformation with Morphological Median as operator. Amount to about 0.5, iterations to 4 - 5, and Structuring Element to 9 pixels with a circular pattern.
Apply the tool to the image. If hot pixels of a certain colour remain, I split the RGB channels and use the channel that has the remaining hot pixels to repeat the process. The result is this.

Same crop after hot pixel removal

Further tweaking of the star mask and morphology parameters can improve this result even more, of course.

First steps in guiding

Finally I have taken the plunge and invested in a guiding setup. I decided for the SkyWatcher ST80 scope with ZWO ASI120MM camera. The camera is the older USB2 version.
As I don't want to take my laptop out in the field, I intend to use a RaspberryPi as a guiding computer.
The last couple of days and nights, I have been trying to get this to work. My configuration at the moment is this:
ASI120MM connected to RaspberryPi, running Ubuntu Mate as an operating system.
The Pi also holds an INDI server and the lin_guider software. The camera connects to the Pi and receives guiding pulses which it sends on to the mount (SW AZ-EQ6 GT) via the ST4 port.
Installation was quite straightforward, despite warnings that the camera driver may not be stable. Setting the exposure time to 1 sec in Lin_guider seems to work fine though.
Last night, despite partial cloud cover, I was able to test the guiding, and it worked fine.
Lin_guider connected to the camera, and frames started to flow in. Focussing was a bit of a hassle. I had to take my laptop out (despite the dew), and because there is no live view, it took a while to get focus right. In the end I had my setup guiding on Vega (which was grossly overexposed at any gain setting), and later on a nearby much fainter star. This worked fine until the stars disappeared behind my neighbour's trees and clouds rolled in.
I haven't tried imaging yet, and I still have to figure out the best settings for PID gain, but so far so good.

Creating a customized "Batch Process" in PixInsight

Some processes in PixInsight are adapted for large batches of images. But sometimes you want to do a sequence of process steps, for which there is no batch process, on several images. Opening each image and applying a number of processes is quite tedious.
Fortunately, PixInsight has a solution for this. It involves an image container and a process container.
For any process in PI, if you drag the small triangle in the lower left corner to an image, it will apply that process to the image. This can also be applied to a collection of images, if these are in an image container. And the process doesn't have to be a single process, it can be any number of processes that are in a process container. How is this done?

Prepare the process container.

Open an image and apply the processes you want to batch to that and other images.
Now open the image's history explorer, which should be located on the left edge of the workspace. Drag the small triangle at the bottom left to an open area in the workspace. This will create an instance of the process history of that image as a process container in the workspace.

Now you can close the image without saving.

Create an image container

 Next create an image container by right clicking anywhere in the workspace, or press Ctrl+Alt+I.

This will create an image container in the workspace. Open the container and add the image files you want to batch process. Also supply a name for the output directory where you want the processed images to be saved. Finish by dragging the small triangle to an empty spot in the workspace. This will create a new instance of your image container, with all the images in it.

Apply the processes in the process container by simply dragging the process container onto the image container that contains the images.
That's it. You've just applied several process to a batch of images.