Astronomical Programming II (IRAF/IRIS) PDF

AST08 - Astronomical Programming II (IRAF/IRIS) IRAF Overview Image Reduction and Analysis Facility (IRAF) is an image processing tool that astronomers use to reduce and analyze imaging and spectroscopic data. It was developed at the National Optical Astronomy Observatories in Tucson, Arizona. Whi...

AST08 - Astronomical Programming II (IRAF/IRIS) IRAF Overview Image Reduction and Analysis Facility (IRAF) is an image processing tool that astronomers use to reduce and analyze imaging and spectroscopic data. It was developed at the National Optical Astronomy Observatories in Tucson, Arizona. While the system has mainly been designed for image data, and for astronomical image data in particular, IRAF also has other general facilities that can be applied to many other classes of data. Some of the functions that are provided are quite specialized, dealing as they do with the characteristics of specific instruments, but others are generalized functions for plotting data, computing statistics, processing lists, and performing other functions that are common to data processing tasks in many other fields. IRAF are mainly used for cleaning astronomical images or data that are captured by telescopes turning the image into something more pleasing to the eye and highlighting important details that allows researchers to analyze and understand the data. IRAF also has a wide range of image processing tools although IRAF provides a wide range of image processing tools for the user, and is mainly used for processing images, it also has an open system that can be extended to add new analysis capabilities and that can support user customization of the existing facilities. COMPONENTS OF IRAF Command Language - default mode, it acts like a notepad where you can write commands and is used for most terminal interactions, it acts as an interface between the user and the IRAF program.  Application Packages - acts like a folder where a collection of data analysis algorithms are compiled per different scientific application programs. Virtual Operating System - it is a portable operating system that can provide foundation for all the higher level functions, and allows us to operate multiple operating systems in a device. Example of this is VirtualBox Host System Interface (HSI) - acts as a communication bridge that allows different parts of a computer system to communicate and to function smoothly. This ensures that data can be sent and received correctly allowing your device to function properly. Such as bootstrap utilities that initiate and set up the systems, applications, etc. and kernel  primitives that allow the operating system to support application processes, ensuring the system to operate properly. Based on the book provided by Peter MB Shames, STSI and Doug Tody 1986, the basic function of the Command Language is to provide a clean, consistent interface between the user and the various packages of functions that complete the IRAF environment. The CL provides an interface between the user and all applications programs, giving the user complete control over the parameters, data, and system resources (graphics devices, etc.) used by IRAF programs. Many features have been incorporated into the CL to provide on-line support for users, whether they are old hands or new to the system. The CL and the IRAF system present the user with a complete data analysis environment which is independent of the underlying operating system. Users running IRAF under UNIX, VMS, AOS, or some other operating system have the same analysis environment available to them and can type exactly the same commands while in the CL. COMMAND LANGUAGE FEATURES The CL is designed to serve both as a command language and as an interpreted programming language. The following are the characteristics of the CL: provides a uniform environment on all host systems package structure for organization and extensibility menus and extensive online help facilities concise command syntax similar to UNIX cshell i/o redirection and pipes; aggregate commands minimum match abbreviations for task and parameter names both local and global parameters, hidden parameters direct access to host system; foreign task interface parameter set editor; command history editor (edt, emacs, vi) background job submission (including queuing) logfile facility for recording all task invocations graphics and image display cursor mode facilities virtual filename facility; unix style path names to files programmable: procedures, C style expressions and control constructs PROCESS STRUCTURE The CL handles all communications to the user Terminal. A single application process usually contains tasks or executable programs. The graphics i/o (i/o means input/output) device is usually connected to CL as a subprocess. As explained by Doug Tody of the National Optical Astronomy Observatories the process cache always contains the process "x_system.e", which contains all the tasks in the system package. The system subprocess is locked in the cache by default. The remaining process cache slots (typically 2 or 3 slots) are dynamically assigned as tasks are run by the user. Up to 3 graphics kernels may be simultaneously connected. The entire process structure is duplicated when a background job is submitted by the user from an interactive CL. IRAF BASIC STRUCTURE OTHER IMAGE REDUCTION SOFTWARE Image processing is the process that turns an image from a science image to something more pleasing to the eye or highlights a structure of interest. Many image processing techniques render the image unsuitable for photometry. There are multiple image reduction software that can be used: MaximDL - an astronomical software created by Cyanogen Imaging for the intended purpose of astronomical imaging. It is available for Windows operating systems, however it has been reported correct operation for both imaging and processing. (http://www.company7.com/cyanogen/products/maximdl.html#Features) MIRAMETRICS - provides off-the-shelf and custom image processing, data analysis, and data visualization solutions for researchers, educators, and students in science and engineering. (https://www.mirametrics.com/index.php) IRIS - a powerful image processing utility with a command-line editor and a graphical interface (GUI). IRAF - one of the historic ways that astronomers reduce and analyze imaging and spectroscopic data.  Useful IRAF Commands COMMANDS USE Help Gives instruction on using the commands package Lists command packages loaded epar Edits the parameters for the command Lpar Lists the parameters for the command display Displays an image in the image window Implot Plots lines or columns of an image in a graphic window Imstat Gives statistics for an image delete Deletes a non image file Imcopy Copy an image Imrename Rename an image Imdelete Deletes an image zerocomb Processes and combines zero images darkcomb Processes and combines dark images flatcomb Processes and combines flat images ccdproc Processes images of objects combine Adds together images imshift Shifts images imexamine Handy quick measurement of star positions and magnitudes apphot Interactive aperture photometry of star images hedit Edit image header ccdlist List images by type imhead View image header Ls, dir Two different ways of listing the directory IRAF Installation INSTALLATION OF VIRTUAL OPERATING SYSTEM (Applicable for Windows user) Installation of virtual operating system such as virtualbox is required in windows in order to run and install IRAF, since it relies on Unix-like operating system such as ubuntu and other compatibility issues and thus virtual operating system is needed in order to emulate Unix-like operating system like ubuntu Step 1: Install VirtualBox in Google and download the latest version for their website (make sure to select the right platform packages you’re using, so if you’re operating on windows select the windows hosts). Step 2: Click the file you install then run the install by clicking it make sure to read every step, click next then install and finish. Step 3: Then the installation of VirtualBox is finished INSTALLATION OF UNIX-LIKE OPERATING SYSTEM Now in order to run IRAF we need to install a unix-like operating system such as ubuntu and we need to put it in the VirtualBox in order to run the ubuntu operating system. Step 1: Install the Ubuntu desktop from the Ubuntu official website Step 2: Open the VirtualBox and click the new button, input the name for the virtual machine, then choose Linux as the type and Ubuntu (64-bit) for the version. Step 3: Select the amount of memory (RAM) and processors you want to allocate to the virtual machine (set the right amount based on the specification of your device). Step 4: Create a virtual hard disk and select the amount of hard disk size based on how much you want to allocate and how much available space you have. Step 5: After that you will proceed to main menu of the VirtualBox on that press settings Step 6: On the settings, under the General tab, select Advanced and change the Shared Clipboard and Drag’n’Drop to Bidirectional? Step 7: Then go to the System tab, deselect the floppy disk by clicking the checkbox after that press the arrow down until the optical disk and hard disk is above it. Step 8: After that go to the Display tab, enable the 3D acceleration and set the video memory allocated for the display based on your preference. Step 9: Then go to the Storage tab, from the storage select the empty disk and click the drop down disk located at the right side of the optical drive and select the ISO file of ubuntu. Step 10: After that click OK, this will bring you back to the main menu of the VirtualBox, from there launch Ubuntu, then start the installation of Ubuntu. Step 11: After that you will now go to the Ubuntu OS and from there select your language, keyboard layout, installation type, etc. You will also need to fill up important details such as your name, computer’s name and password when opening Ubuntu. Step 12: Wait until the installation is complete, when the installation is complete a tab will appear to restart, restart the computer, then login, after that we will now proceed with the installation of IRAF. INSTALLATION OF IRAF Before we start with the installation, download first the needed files in installing IRAF. First download the Anaconda for Linux and the SAOImage DS9 to http://ds9.si.edu/site/Download.html and select Ubuntu 22 ARM64 under Linux, so that we can easily access it later when needed. INSTALLATION OF IRAF DEPENDENCIES Step 1: Open the terminal (ctrl + alt + t) then install the dependencies on ubuntu, make sure to always add sudo before the command when installing, adding or getting applications so that you run it as an administrator. Run this command below in order to install IRAF dependencies  - sudo dpkg --add-architecture i386 - sudo apt-get update - sudo apt-get install libc6:i386 libz1:i386 libncurses5:i386 libbz2-1.0:i386 libuuid1:i386 libxcb1:i386 libxmu6:i386 CREATING ANACONDA VIRTUAL ENVIRONMENT Step 2: After step 1, proceed creating the anaconda virtual environment. We can do that by accessing the Anaconda file for Linux. Follow the needed commands - ls - cd Downloads (you can double check if the file is in the download folder, through accessing the file explorer in Ubuntu) - bash Anaconda3-2023.07-2-Linux-x86_64.sh (make sure that it matches the name of the anaconda file that you installed) - yes (It will ask you to accept the license terms at this part) (OPTIONAL) If you want to configure virtual environment to stop autostart from startup use the command - conda config --set auto_activate_base false CREATING THE IRAF ENVIRONMENT Step 3: After installation of the Anaconda virtual environment we will now proceed on creating the IRAF environment. Simply type the commands - conda config --add channels http://ssb.stsci.edu/astroconda (Use to add astroconda channel o conda and install IRAF) - conda create -n iraf27 python=2.7 iraf-all pyraf-all stsci (It will download packages needed for IRAF) - y - conda update -n base -c defaults conda On this part, the commands will download all the needed packages and updates needed in creating the IRAF environment. INSTALLATION OF SAOIMAGE DS9 SAOImage DS9 is an astronomical application that allows us to view astronomical images as it supports FITS images which allows us to view and analyze astronomical images which makes it very essential. Step 4: To install SAOImage DS9 we need to access the file for SAOImage DS9 we downloaded earlier. Follow the command below to install SAOImage DS9 - cd Downloads (to open Download folder) - tar xvzf [FILENAME] (to extract file) - Go to Downloads folder and copy the extracted file to sudo cp /home/ubuntix/Downloads/ds9 usr  - Then use the same command sudo cp /home/iraf/Downloads/ds9 usr and at the end add: /usr/bin /anaconda3/envs/iraf27/bin /anaconda3/envs/iraf27/lib /anaconda3/envs/iraf/lib32 Double check the directories such as the user and folder name. STARTING IRAF Step 5: Activate the IRAF environment with conda use command - conda activate iraf27 or source activate iraf27 (After activating you will notice from base it will become iraf27, which means that IRAF environment is activated) Step 6: Then we create a folder within a new folder to organize work, use this command to do it. - mkdir [folder name] (Use to create directories) - cd [folder name] (Use to open and navigate through directories) Step 7: To change to your working directory you can also use command - cd [name of the folder or directory] Step 8: After that we create a login.cl file, creating login.cl file is essential since it sets up the IRAF environment that is necessary before starting an IRAF session. Use this following command to create a login.cl file - mkiraf - xterm Step 9: We can now start IRAF by logging in cl and SAOImage ds9 using ds9 & Step 10: For exiting IRAF use logout then conda deactivate or source deactivate Pre-Processing Pre-processing image calibration on IRAF refers to initial steps in preparing the raw images which have imperfections in it, the pre-processing aims to remove or correct specific imperfections in the raw images such as bad pixels, through the use of bias correction, flat correction and dark correction. This process is essential since it produces calibrated images that accurately reflect the astronomical object captured, making them a lot more detailed and pleasing to the eye. Through these processes it also makes scientific analysis and observation possible. LIGHT FRAME - Contains the actual or raw image data of the object. - Reduce the random noise in the light frames with stacking - Thermal noise, vignetting, dust, bias signal, hot pixels - Fixed by Dark, Flat and Bias Frames BIAS FRAME - Used to remove readout signal from the camera sensor - Can be subtracted from Light, Dark, Flats, Dark flats to remove variation - Pictures taken at Zero exposure length (as close as possible) and with the lens cap on - Not dependent on temp but should be taken with the same ISO FLAT FRAME - Taken when the focus of lens is blurred and with an empty evenly illuminated surface or smooth sky in the background during dawn and dusk - Displays the dust, fungus, marks and damages/scratches on the surface of the lens or/and sensor - Flat frames are intended to correct optical imperfection since most telescopes do not evenly distribute light across the camera sensor. This causes images to be brighter in the center, and darker towards the edges. DARK FRAME - Displays the dead pixels in the image (Either vertical or horizontal line(s)) - Taken when the shutter/lens is closed or the image captured is in a dark room - Displays noise produced by the sensor and subtracted from subsequent images to correct for fixed -pattern noise - Used for removing amp glow, a portion of sensor lighting up due to internal heat sources. - Dark-frame subtraction has been done for some time in scientific imaging COSMIC RAY CORRECTIONS - Charged particles which can penetrate through telescope domes and produce a large charge in one of your pixels. - Can be corrected using sophisticated cosmic ray detection algorithms FRINGE CORRECTION - Can appear due to the interference of light due to the thin film on the optics - Can be corrected by analyzing flat field images BAD PIXELS - Some pixels die, some misbehave which results in dead and hot pixels - Can be corrected by marking them and asking a suitable interpolation algorithm to use the knowledge of neighboring pixels to estimate the likely value of the pixel READ NOISE - Added by the read out circuit while converting charges into digital numbers - Cannot be corrected, only quantified Bias Correction Before starting the process for bias correction make sure that you have all the needed file, prepare the file for the object you captured, organize them by naming if it is a bias, flat or dark frames. Then open the IRAF through activating the anaconda environment, starting the IRAF environment then logging in by command cl just like we did earlier. After starting the IRAF proceed with the set up by opening correct application packages in doing bias correction proceed with these commands: - ecl > noao - noao > imred - imred > ccdred After we open the ccdred application package, we proceed by making a bias.list which compiles all the bias files or frames into a list. - ls (This command calls out the list of your files, it is needed so that we can check the files that we will rename). - display [filename.fits] (Display image just to check what the current bias image looks like, ds9 should pop out and present the file or image you open when you do this command) - ls b*.fits > bias.list (This command basically creates a list file for all bias fits files) - cat bias.list (This command calls out the bias.list file and display all the bias fits files in it) MAKING MASTER BIAS Master bias basically combines all the bias files that are in the list you created earlier. To create a master bias use this command - epar zerocomb (epar command allows you to edit parameters in the zerocomb, zerocomb opens the zerocombine which allows you to combine bias files) This open the zerocombine task, only change and put these in the zerocombine task: - input: @bias.list (the name depends on the name of the list you created) - output: Zero (output name of the combined bias images, the name depends on you) - combine: average - ccdtype: (space+enter) (this basically leave the ccdtype as blank) - process: no - :g (to execute command) One you type :g the master bias will be created check by typing the command ls CORRECTION OF BIAS FRAMES Correction of bias frame correction basically removes or at least reduces electronic noises such as readout signals. In doing bias correction follow these steps. - epar ccdproc (edit parameters, in the ccdproc task) After opening the ccdproc task change or put this on the ccdproc task - images: @filename of the list you created.lists - output: @filename of the list for corrected frames.list - ccdtype: (space+enter) - fixpix: no - overscan: no - trim: no - zerocor: yes - darkcor: no - flatcor: no - zero: (filename of your master bias.fits) - minrepl: -50 - :g (to execute command) After this a bias frames should be corrected To view the corrected bias frame in full details and in ds9 use the following: - ccdlist @filename of any corrected image - imhead @filename of any corrected image lo+ - display “filename of any corrected image”.fits (frame no.) Dark Correction In doing the dark correction, it is somewhat similar in doing the bias correction, make a list for the dark.fits files, create a master dark using the command epar darkcomb and correction of dark frames using the ccdproc just like in the bias correction earlier. We make a list for a dark.fits files using this commands - ls d*.fits > dark.list - cat dark.list (to view all the dark.fits files in the list) MAKING MASTER DARK After that we make a master dark through this process - epar darkcomb (this command allows you to edit the parameters in the darkcombine task In the darkcomb task change and put these: - input: @dark.list (the list for the dark files you created earlier) - output: Dark (you can name it whatever you want, this is the name of the master dark that you will create) - combine: average - reject: avsigclip - ccdtype: (space+enter) - process: yes - scale: exposure - :g (to execute the command) After executing the command type the command ls to check the master dark you created CORRECTION OF DARK FRAMES Correction of dark frames removes thermal signals inherent in CCD detectors such as amp glow. Almost the same process in correction of bias frames applies in dark frames, enter this command in correction of dark frames. - epar ccdproc In the ccdproc task change or put these:  - images: @filename of the list you created.lists - output: @filename of the list for corrected frames.list - ccdtype: (space+enter) - fixpix: no - overscan: no - trim: no - zerocor: yes - darkcor: yes - dark: file name of the master dark.fits - minrepl: -50 - :g (to execute command) To view the corrected bias frame in full details and in ds9 use the following: - ccdlist @filename of any corrected image - imhead @filename of any corrected image lo+ - display “filename of any corrected image”.fits (frame no.) Flat Correction Flat correction process is also somewhat similar to the bias and dark correction we have done earlier. Make a list for flat.fits files - ls f*.fits > flat.list - cat flat.list (to view all the flat.fits files in the list) MAKING MASTER FLAT After creating the list for flat.fits files we will combine them using: - epar flatcomb In the flatcomb task, change or put these: - input: @flat.list (the list for the flat files you created earlier) - output: Flat (you can name it whatever you want, this is the name of the master dark that you will create) - combine: average - ccdtype: (space+enter) - subset: yes - process: yes - scale: mode - :g (to execute command) After executing the command type the command ls to check the master flat you created CORRECTION OF FLAT FRAMES Correction of flat frames are used to correct optical imperfections in the image, since most telescopes do not evenly distribute light across the camera sensor. This causes images to be brighter in the center, and darker towards the edges. In the correction of flat frames, use the command: - epar ccdproc In the ccdproc task, change or put these: - images: @filename of the list you created.lists - output: @filename of the list for corrected frames.list - ccdtype: (space+enter) - fixpix: no - overscan: no - trim: no  - zerocor: no - darkcor: no - flatcor: yes - zero: (space+enter) - dark: (space+enter) - flat: file name of the master flat.fits - minrepl: -50 - :g (to execute command) To view the corrected bias frame in full details and in ds9 use the following: - ccdlist @filename of any corrected image - imhead @filename of any corrected image lo+ - display “filename of any corrected image”.fits (frame no.) Post-Processing Post-processing is one of the most crucial steps as it aims to enhance, analyze and prepare the image for any other scientific interpretation or investigations. It includes several steps such as image aligning, image combining, aperture photometry, and cloudfield photometry enhancing the image quality and allows us to conduct photometric measurements of celestial objects. IMAGE ALIGNING Image aligning is vital in post processing because different frames are taken from different time which allows objects in the night sky to shift, image aligning allows us to align the different images through the use of coordinate on an image, and also by comparing the reference image and the shifted image to identify how drastic the shift is. In aligning image use this steps Step 1: Display two images, one as your reference image and the other is the shifted image see how the image has shifted. Step 2: Establish the coordinate file, use this command: - ccdred > rimcur > ref.coo Choose at least 10-20 unsaturated and isolated stars not near the edge, spacebar to register star and ctrl+D to quit - cat ref.coo (use to display the coordinates information of the registered stars) Step 3: Use command epar tvmark, and change the certain parameters: - frame: 1 - coords: ref.coo - mark: circle - radii: 15 - :g (execute command) After executing the command you can see that the stars that you mark have shifted from your reference image. Step 4: To align the images use the command epar imalign, and change or put these parameters: - input: filename of your shifted image.fits - reference: filename of your reference image.fits - coords: ref.coo - output: filename of your output image.fits - boxsize: 25 - bigbox: 75  - niterat: 25 - :g (execute command) After executing the command, display the output image that you created and you can see that it now matches the coordinates of the registered stars in the reference image. Make sure to align all the images before combining them. IMAGE COMBINING The image combining is used to combine the images or frames, it helps improve the details and clarity of the image especially when there are multiple frames or images combined. Before combining image, first create a list of images you want to combine - ls image*.fits > image.list (Create a list of files for all image.fits file) - cat image.list (Display the image.fits files in the image.list) In combining images use this command. - epar imcomb In imcomb, change or put these parameters. - input: @file name of the list of images that you created earlier - output: name of the output image - combine: average - reject: avsigclip - scale: mode - statsec: [*,*] - expname: EXPTIME - verbose: yes - :g (execute command) After executing the command the images are now combined and the name of the file of the combined images depends on the name you put in the output parameters in the imcomb. APERTURE PHOTOMETRY AND CLOUDFIELD PHOTOMETRY The aperture photometry technique is used to measure the brightness of a certain or selected point source such as stars. While a cloudfield photometry is used to identify and determine the coordinates of the stars or other celestial objects, it involves tasks such as measuring position and calculating the magnitude of objects. Before we do aperture photometry, make sure that you are in the right application packages follow these before we do aperture photometry - ccdred > digiphot - digiphot > apphot - apphot > daophot after entering the daophot, use command display [filename of your combined image.fits] to display the combined image. After opening the image use the command imexam “l” “r” *to get the Full Width Half Maximum (FWHM) “c” “m” *to get the STDDEV of the background In the combined image presented in ds9 - Choose 5-10 stars and get the average of the FWHM. (add all the values of FWHM and divide it on the number of counts) - Get the average of the STDDEV of the sky background. (add all the values of STDDEV and divide it on the number of counts) PRODUCING.COO FILE A.coo file is a file that contains the coordinates of the objects. It contains the X and Y pixel coordinates of the objects. To produce.coo file use these command - epar daofind In daofind change or put this in the parameters - image: filename of the combined image - output: default  - datapar: :e (used to enter the parameter, when you execute this command you will go to the datapars task) - fwhmpsf: 5.83 (Average of FWMH that we got earlier) - sigma: 15.41 (Average of STDDEV that we got earlier) - gain: 1.43 - readnoise: 13.6 - exposure: EXPTIME - filter: SUBSET - obstime: DATE-OBS - :q (quit the parameter or task) - findpar: :e (used to enter the parameter, when you execute this command you will go to the findpars task) - thresho: 5 - :q (quit the parameter or task) - verbose: yes (this is in the daofind task) - :g (execute command) After executing the command, a.coo file will be produced. - cat file name of the.coo file (use to view or display the details on the file) - tvmark 1 file name of the.coo file (use to mark positions on the image displayed in ds9 which is the combined images) After that: - Visit aavso.org to get the finding chart of AE Cas. - Find the variable star and 3 standard stars (comparison star and check stars) Note: - Press “space bar” to register the chosen star - Press “q” to quit - Press “n” to proceed to the next image PRODUCING.MAG FILE A.mag file is a file that contains photometric measurements of celestial objects such as stars, including their magnitudes and associated parameters. A photometric measurements refer to the quantitative measurement of the brightness or intensity of astronomical objects, typically performed in specific wavelength bands or filters. To produce.mag file use the command - epar qphot In qphot, change or put this in the parameters - image: filename of the combined image - cbox: 8.75 (~1.5 FWHM) - annulus: 29.15 (~5-6 FWHM) - dannulus: 10 (~10 pixels)  - aperture: 23.32 (~4-5 FWHM) - coords: file name of the.coo file created earlier - output: default - exposure: EXPTIME - filter: SUBSET - epadu: 1.43 (Gain) - obstime: DATE-OBS - interac: yes - verbose: yes - :g After executing the command a.mag file will be created Use a cat file name of the.mag file produced (to inspect the.mag file). EXTRACTING.MAG FILE To extract use command - epar pdump In pdump, change or enter these parameter - fields: id,xcent,ycent,xairmass,ifilter,otime,mag,merr - :g da > pdump aecas-070s.mag.1 > aecas-070s.dat Can be opened in TextEditor or Excel, open in excel or text editor to view Note: *ID is based on the sequence of choosing the stars. *ID #1 is AE Cas *ID #2, #3 and #4 are Standard stars S1, S2 and S3, respectively.