magicjava

Aqua Satellite Data Processing

A Note On The UAH And RSS Raw Data

How UAH And RSS Temperatures Are Measured

Overview Of The Aqua Satellite

Looking At The Aqua Satellite Data

UAH Satellite Data

Dangit! More Climate Stuff. UAH and RSS Raw Data

Prepping The Data File
The AMSR code reads un-compacted ASCII files, and HDF file are compacted binary. So you'll need to change that before reading them with the code. To do that, just use the ncdump tool. The only argument you supply to ncdump in this case is the filename. Redirect the output to a file and use that file as input to the code. And example command line is shown below:

ncdump AIRS.2010.01.01.240.L1B.AMSU_Rad.v5.0.0.0.G10005063822.hdf > MyFile.txt

If you don't yet have ncdump on your computer, see this post for instructions on downloading and using it.

The Code
The underlying code is fairy complex, so we'll skip discussing it. The code is well commented, so you can look there for specific details. Here we'll just discuss how to use the code to read data.

The code sample below shows how to create the reader, declare memory to hold data you'll get from the reader, and read the raw antenna temperatures.

// Supply location of HDF file.

HDFReader oHDFReader(string("./MyFile.txt"));

// 45 scan lines, 30 footprints, 15 channels.

float aData[45][30][15];       



// Load data.

oHDFReader.getVariableData(string("antenna_temp"), aData);

Yup. That's it. Of course, with this version of the code, you need to know that the name of the antenna temperature data is antenna_temp and that it's an array of floats 45 by 30 by 15. Fear not. You can open the ASCII version of the HDF file and see all the variables listed in the variables section near the beginning of the file. These variables will have dimensions listed as constants and those constants are defined in the dimensions section at the beginning of the file.

There's also some hierarchical data in the HDF file and that data is stored in a node structure in HDFReader. To get the root of the node structure, use the getRootNode() method.

Future versions of the code will build on what's here now. Specifically, I want to add methods to access the variables without forcing the programmer to dig through the data file to see what variables are there. Code to do that should be ready soon.

Also coming soon is a look at some of the raw UAH data that gets extracted from these files.

References:
Some Useful Climate Code

Summary Of Aqua Satellite Data, Computer Code, And Broken Equipment

Aqua Satellite Raw AMSR-E Data

Aqua Satellite Data Processing

A Note On The UAH And RSS Raw Data

How UAH And RSS Temperatures Are Measured

Overview Of The Aqua Satellite

Looking At The Aqua Satellite Data

UAH Satellite Data

Dangit! More Climate Stuff. UAH and RSS Raw Data

HDFReader.h
HDFReader.cpp

Monday, February 1, 2010

Aqua Satellite Raw AMSR-E Data

In this post we take a look at the AMSR-E Level 2A data. Specifically, we now want to be able to write pseudocode that reads the file, checks for errors, and grabs the raw temperature data. In order to get to this level of understanding of the data, we're going to use the Level 2A Data Fields web page. It describes the fields in the Level 2A data file for the AMSR-E instrument.

Additional Posts In This Series

Aqua Satellite Data Processing
A Note On The UAH And RSS Raw Data
How UAH And RSS Temperatures Are Measured
Overview Of The Aqua Satellite
Looking At The Aqua Satellite Data
UAH Satellite Data
Dangit! More Climate Stuff. UAH and RSS Raw Data

Scan Information

Each AMSR-E Level 2A data file contains a singe scan at low resolution and a single scan at high resolution.

For low resolution, the AMSR-E scans 1994 times in the same direction as the satellite is moving. Each of these 1994 scans contains 243 readings perpendicular to the movement of the satellite (called Footprints), and each of these 243 readings contains 5 channels, each at a different frequency.

This gives a total of 2,422,710 readings in a single Level 2A file. This data has been "smoothed" as part of its processing from Level 1B to Level 2A.

For high resolution scans the AMSR-E scans 1994 times in the same direction as the satellite is moving. Each of these 1994 scans contains 486 readings perpendicular to the movement of the satellite, and each of these 486 readings contains 1 channel at a frequency of 89.0 GHz.

This gives a total of 969,084 readings in a single Level 2A file. This data has not been smoothed. It's in its original Level 1B form.

NOTE: The high resolution scans were designed to be collected by two "horns" on the AMSR-E instrument, called Horn A and Horn B. However, recall from this post that Horn A failed on November 11th, 2004. A software change was made to gather all high resolution data from Horn B. Horn A still has data fields in the level 2A file, but they are all set to -32,768.

NOTE: Both the low resolution and high resolution temperatures are stored in a scaled offset format. To obtain the actual temperature in degrees Kelvin multiply the stored values by 0.01 and add 327.68 to the result.

Data Quality

There are several fields in the AMSR-E Level 2A data file that contain the results of Quality Assurance (QA) checks. These fields tell if various pieces of data in the file can be used with confidence. Here are the important QA fields to check when processing temperature data:

For Low Resolution Scans

Scan Quality Flag A series of 1994 flags indicating the quality of each scan line. If a flag is not zero, the corresponding scan line should be rejected.
Channel Quality Flag 6 To 52 A series of 1994 arrays containing 12 flags each. There are two flags for channels 6.9, 10.7, 18.7, 23.8, and 36.5. The last two fields are not used as there is no channel 52. These flags indicate the horizontal and vertical scan validity for each scan footprint. If either flag for a footprint is not zero, the corresponding footprint should be rejected.

For High Resolution Scans

There are two high resolutions scans in the Level 2A file, one for Horn A and one for Horn B. Horn A is broken and its data should be ignored.

Scan Quality Flag 89B A series of 1994 flags indicating the quality of each scan line for Horn B. If a flag is not zero, the corresponding scan line should be rejected.
Channel Quality Flag 89B A series of 1994 arrays containing 2 flags each. These flags indicate the horizontal and vertical scan validity for each scan footprint. If either flag for a footprint is not zero, the corresponding footprint should be rejected.

Pseudocode For Reading Raw AMSR-E Data

The following pseudocode shows how to process a AMSR-E Level 2A file, checking for QA errors, and grabbing the temperature data. Recall that temperatures are provided in scaled offset degrees Kelvin and that Horn A is always bad.

Open File

Read Low Resolution Record Into Memory



Foreach Of The 1994 Scanlines

   If Scan Quality Flag Is Not 0 Then Reject Scanline



   Foreach Of The 5 Footprints

      If Either Of The Channel Quality Flags Are Not 0 Then Reject Footprint



      Read Each Of The 243 Footprints For the Channel

   End Foreach

End Foreach



Read High Resolution Horn B Record Into Memory



Foreach Of The 1994 Scanlines

   If Scan Quality Flag Is Not 0 Then Reject Scanline



   If Either Of The Channel Quality Flags Are Not 0 Then Reject Footprint



   Read Each Of The 486 Footprints For the Channel

End Foreach

References:

Sunday, January 31, 2010

Aqua Satellite Raw UAH Data, Part 2

In this post we take a more detailed look at the AMSU Level 1B data that serves as the raw data for UAH. Specifically, we now want to be able to write pseudocode that reads the file, checks for errors, and grabs the raw temperature data. In order to get to this level of understanding of the data, we're going to use the AIRS Version 5.0 Released Files Description PDF. It contains a section for each AIRS instrument on the Aqua satellite and each section describes every field in the Level 1B data file for that instrument.

Additional Posts In This Series

Scan Information

Each AMSU Level 1B data file contains a a collection of scans that take place over 6 minutes.

In these 6 minutes, the AMSU scans 45 times in the same direction as the satellite is moving. Each of these 45 scans contains 30 readings perpendicular to the movement of the satellite (called Footprints), and each of these 30 readings contains 15 channels, each at a different height above the Earth's surface.

NOTE: These values are collected by three different pieces of hardware. Receiver A11 handles channels 6, 7, and 9 through 15. Receiver A12 handles channels 3, 4, 5, and 8. Receiver A2 handles channels 1 and 2. Also, recall from Part 1 that channel 7 is always bad.

This gives a total of 20,250 readings in a single Level 1B file. These 20,250 readings are stored as a contiguous block of data in the file that, in the parlance of C programmers, can be read into computer memory as a three dimensional array.

There are actually two sets of this 20,250 piece block of information in the AMSU Level 1B files. The first set is called antenna temperatures and is the raw data in degrees Kelvin as read by the satellite modified by calibration information (specifically, antenna temperatures = calibration coefficient/instrument readings²).

The second set is called brightness temperatures consists of slightly modified antenna data.The brightness temperatures have a companion set of data, 20,250 error estimates called brightness temperature errors that estimate the accuracy of the brightness temperatures.

Either the antenna temperatures or the brightness temperatures can be considered to be the raw data used to create the UAH data.

Data Quality

There are several fields in the AMSU Level 1B data file that contain the results of Quality Assurance (QA) checks. In fact, much of the file is QA data. These fields tell if various pieces of data in the file can be used with confidence. Here are the important QA fields to check when processing temperature data:

State 1 Indicates the state of the AMSU-A1 unit for each scan line. If this value is anything other than 0, the data in the scan line should be rejected.
QA Receiver A11 and QA Receiver A12 These flags indicate the quality of the AMSU temperature receivers for each scan line. If either are non-zero, the data in the scan line should be rejected.
SATGEO QA, GLINTGEO QA, and MOONGEO QA These fields contain QA information for each scan line. If a non-zero value is present, the data in the corresponding field should be rejected.
FTPTGEO QA, ZENGEO QA, and DEMGEO QA These are satellite position QA flags and occur for each of the 30 field of view footprints for each of the 45 scan lines. If a non-zero value is present, the data in the corresponding field should be rejected.
QA Channel This is a series of 15 flags indicating the reliability of each channel of the AMSU. Channels with non-zero values should not be used. Recall from Part 1 that channel 7 is always bad.

Pseudocode For Reading Raw UAH Data

The following pseudocode shows how to process a AMSU Level 1B file, checking for QA errors, and grabbing the temperature data. Recall that temperatures are provided in degrees Kelvin and that channel 7 is always bad.

Open File

Read AMSU Record Into Memory



Foreach Of The 45 Scanlines

   If State_1 Not Equal To 0 Then Reject Scanline

   If QA_Receiver_A11 Not Equal To 0 Then Reject Scanline

   If QA_Receiver_A12 Not Equal To 0 Then Reject Scanline

   If SATGEO_QA Not Equal To 0 Then Reject Scanline

   If GLINTGEO_QA Not Equal To 0 Then Reject Scanline

   If MOONGEO_QA Not Equal To 0 Then Reject Scanline



   Foreach Of The 30 Footprints

      If FTPTGEO_QA Not Equal To 0 Then Reject Footprint

      If ZENGEO_QA Not Equal To 0 Then Reject Footprint

      If DEMGEO_QA Not Equal To 0 Then Reject Footprint



      Foreach Of The 15 Channels

         If QA_CHANNEL Not Equal To 0 Then Reject Channel



         Read Antenna Temperature for this Scanline/Footprint/Channel

         Read Brightness Temperature for this Scanline/Footprint/Channel

         Read Brightness Temperature Error for this Scanline/Footprint/Channel

      End Foreach

   End Foreach

End Foreach

References:
Aqua Satellite Raw UAH Data, Part 1