Overview Of The Aqua Satellite Project, Update 1 Features

In this post we're going to take a look at the three new shell scripts and new executable that come in Update 1 of the Aqua Satellite Project.

New Scripts
All three of the new shell scripts are for working with HDF files en masse. They all expect a directory structure similar to the one shown in the picture above. In this structure there are a series of folders that all contain HDF files.

The hdf_to_text script will walk this structure looking for HDF binary (.hdf) files in the immediate subdirectories. When it finds them, it creates a text subfolder in the folder containing the HDF files, converts the HDF files to text and places them in the text subfolder. At the end of the script, each of the folders shown above would have a text subfolder containing text versions of their HDF files.

The amsu_qa script is similar. It walks the subfolders and converts .hdf binaries to text in a text folder just like hdf_to_text does, and then it runs AMSUQA on the resulting text file. This saves you the trouble of having to run both scripts. The amsu_qa script is also smart enough not to convert an .hdf file to text if a text version already exists. The output of the amsu_qa script is up to three files in the text folder. These files are named passed, failed, and suspect, and contain the QA results for the files that NASA's QA marked as passed, failed, or suspect in the automaticQualityFlag of the file.

The amsu_extract script is just like the amsu_qa script, except that it runs the new AMSUExtract program rather than AMSUQA. AMSUExtract allows you to provide several different command line switches to get the data you want, and only the data you want. The amsu_extract script lets you specify these switches and it passes them on to AMSUExtract. Generally, you'll be interested in the -s, -c, and -F switches, which allow you to specify the scan lines, channels, and footprints that are extracted.

AMSUExtract
The AMSUExtract program allows you to pull scan information from a Level 1B AMSU text file. Output is in csv format and is printed to standard output, allowing you to redirect it as needed. AMSUExtract has a number of command line switches that control its functionality. These are shown below as displayed by the command's help text.

AMSUExtract Extracts scan information from an AMSU text file and sends results to standard output. Usage:
AMSUExtract [-h] -f AMSU_Text_File [-s Scanlines] [-c Channels] [-F footprints] [-a] [-H]
f The location of the AMSU text file to be QAed.
s An optional comma seperated list of scanlines to extract. Ranges can be specified using a dash. Example -s 1-5,7,10
c An optional comma seperated list of channels to extract. Ranges can be specified using a dash. Example -c 1-5,7,10
F An optional comma seperated list of footprints to extract. Ranges can be specified using a dash. Example -F 1-5,7,10
a Use this switch to extract antenna temperatures. The default is to extract brightness temperatures.
H Do not display headers. The default is to display headers.

The -f switch is required. It specifies the location of the HDF text file to process.

The -s, -c, and -F switches allow you to specify the scan lines, channels, and footprints to extract. You can specify a single number, a collection of comma separated numbers, or a range of numbers with a dash in between the start and end of the range. The comma separated numbers and ranges can be used together. The command to pull all 45 scan lines of channel 5 with footprints 15 and 16 looks like this:

AMSUExtract -f filename -c 5 -F 15,16

The first few lines of the results of that command look like this:

Beginning Date,Clock,Scan Line,Channel 5 Footprint 15 Brightness,Channel 5 Footprint 16 Brightness
2009-12-31,536457572,1,260.328,260.598
2009-12-31,536457580,2,260.262,260.116
2009-12-31,536457588,3,259.891,259.744

The output shows the date, clock time, and scan line number, followed by the channel and footprint values requested. Note that the -s, -c, and -F switches are all optional. If you don't use them, you'll extract all scan lines, all channels, and all footprints.

The csv output format of AMSUExtract makes it easy to build graphs or load values into spreadsheets. The image below shows a graph of scans for channel 5, footprint 15 for all 333,450 scan lines of January, 2010. The data was produced by the amsu_extract script and graphed using an application named DataGraph.

Requirements
All the scripts and programs work on text versions of HDF files. To convert the binary HDF files from NASA into text, use ncdump. To get ncdump, follow this instructions in this post.

The code itself is written in standard C++ and built using the free gcc compiler. It has only been tested on a Macintosh, but in theory should compile and run on any system.

The bash scripts naturally require the free bash shell.

Previous Posts In This Series:
Aqua Satellite Project, Update 1 Released
Spot Checking The Spot Check
NASA, UAH Notified Of QA Spot Check Findings
About The Aqua Satellite Project
UAH January Raw Data Spot Check
So, About That January UAH Anomaly
A Note On UAH's High January Temperature

See Also:
AMSR-E And AMSU HDF-EOS C++ Readers Are Done

References:
Looking At The Aqua Satellite Data
DataGraph 2.1.1
gcc compiler
bash shell