The C++ source code used to run these tests has been added to the Downloads section of the blog under the name Aqua Satellite Project (BETA).
Update:
While I said words to this effect at the end of the post, I wanted to make sure it's clear: These findings are preliminary. Although nearly two months of research has gone into them more needs to be done.
In the meantime, folks wanting to see the data for themselves can download it from NASA by following the steps here: UAH Satellite Data and downloading the viewers by following the steps here: Looking At The Aqua Satellite Data.
======
Killing two birds with one stone, I decided to give my AMSU C++ code a bit of a workout by doing a Quality Control spot check on the January UAH raw data. I downloaded all the January Level 1B data for the AMSU from NASA. There are 7410 files for the month of January. Here's what I found:
No Files Passed NASA's QA. None.
Of these 7410 files, 7386 of them had their automaticQualityFlag marked "suspect" and another 24 of them had the flag marked "failed". Add that up and it's all 7410 files for January were either suspect or failed. No a single file was marked as "passed".
Of the 7386 suspect files, all of them had a "Good Data" percentage of 93.33334 percent, meaning that's the percentage of data in each file that's considered reliable. There's no way such a number would actually be identical out to one one hundred thousandths of a percentage 7386 times out of 7386 tries, so I figured my code that was reading the wrong data for the good/bad/missing/special values.
Nope, my code was good and manually checking a random selection of about 20 files by opening them up in an editor revealed they all had the identical numbers in them for these fields. In other words, the good/bad/missing/special data flags seem to have hard-coded values, at least for "suspect" data.
Channel 4 Never Works (Which Means Channel 5 Never Works Either)
Each of the the files contains 45 readings from each channel. Of these 45 readings, channel 4 failed 45 times every time. The only exception was in the files that had been marked as "failed". So there was never a good reading from channel 4 for the entire month of January.
This is significant because the statistics engine used to correct the satellite readings for channel 5, the channel UAH uses to get it's anomaly information, requires data from channels 4 and 6.
So this means they're either using a non-published algorithm to calculate the anomalies, or the anomalies are wrong.
Screen Shoot Of QA Spot Check Results
(Click for larger image)
Disclaimer
Here are a few things I think of that may affect the validity of this spot check:
- I'm not a scientist, I'm a software engineer.
- Prior to working on the Aqua satellite as documented in this blog, I had no experience on working with satellite data.
- Despite having checked my code, it is beta code, and I may have missed errors.
- It's possible that data files are not given "passed" status (as opposed to "suspect" and "failed") until some QA procedure that takes place after Level 1B occurs.
- It's possible the order of channel information in the data file does not represent the actual channel number. In other words, the fourth channel in the data file may not be channel 4.
- UAH uses other satellites in addition to the Aqua AMSU.
Still, further research needs to be done to triple check these things, and that's what I'm working on now. If I don't come across anything that changes my views, I'll forward these results to Dr. Spencer and Dr. Christy for their input.
Conclusion
At this point, while admitting the possibility of an error on my part, I'd have to say it seems to me the January AMSU data has no validity at all. Nor does anything derived from it. The hard-coded quality values and the complete failure of channel 4 both look like show-stoppers to me.
Previous Posts In this Series
So, About That January UAH Anomaly
A Note On UAH's High January Temperature
References:
The Limb Adjustment of AMSU-A Observations: Methodology and Validation
Posts
Dear magicjava,
ReplyDeletethese are interesting statements and observations. But it would be even nicer if I could easily get to the files that you analyzed, and some description how the properties of the files were determined.
In your form, it's a black box.
You shouldn't underestimate the number of confusing conventions in their files etc. This is usually meant to be understandable by the owners only.
GISS counted January 2010 as the 2nd warmest on record, but only 0.13 deg C or so warmer than December 2009.
Best wishes
Lubos
Thanks for taking the time to look at this Lumo. I really appreciate it.
ReplyDeleteI have a post on getting tools to view the data here:
http://magicjava.blogspot.com/2010/01/looking-at-aqua-satellite-data.html
And how to download the data from NASA here:
http://magicjava.blogspot.com/2010/01/uah-satellite-data.html
For more info, click the Aqua Satellite group in my sidebar. There's lots of background there and lots of links to official papers describing the satellite. Of particular interest is this one which describes errors and corrections:
The Limb Adjustment of AMSU-A Observations: Methodology and Validation
Additionally, I'll be posting the source code I used for this test either today or tomorrow.
Sorry, the link to the Limb Adjustment paper is:
ReplyDeletehttp://ams.allenpress.com/archive/1520-0450/40/1/pdf/i1520-0450-40-1-70.pdf
Good work.
ReplyDeleteI'm promoting your work on my FB.
What does the good Dr. Spencer have to say about this?
ReplyDelete- MattN
@Anonymous
ReplyDeleteGood question - this was my first thought, too
At the risk of being obvious, what results do you get when you use Decembers data, or Novembers, or any month other than January?
ReplyDeleteIf you bypass the bad scan flags - hard code it to 'good' does what comes out look like data opr random noise?
Thats assuming that the data doesn't normally look like random noise...:)
I seem to remember something satellites and dead sensors - i.e. individual sensors die on these satellites but they have enough that it is not critical. The QA checks would pick up the bad sensors but someone with knowledge of which ones are already 'known to be bad' has to check the data to make sure no new sensors have gone bad.
ReplyDeleteAs I noted in the post, I haven't yet alerted Dr. Spencer or Dr. Christy to these results. I'm currently making a final effort to shoot down the results myself. Failing that, I'll notify both UAH and NASA before this week is up. This isn't something that should linger without professional input.
ReplyDeleteChuckles -
I'm planning on doing additional testing even if it turns out there's an explanation for these results.
Ultimately, there's information in these raw files that never makes it to the final UAH product, and I'd like to be able to produce products that have that information. To do that, I need to first be assured that the data is worth using. So additional testing is needed no matter what.
P.S. The source code I used to run the test is up on SourceForge. A link to it is provided in the "Downloads" section of this blog under the name Aqua Satellite Project (BETA).
ReplyDeleteThe Aqua Satellite Project will be ongoing, but this gets the QA source I used up and available to the public.