Winding Down

Just a short post to announce I'll be wrapping up this blog in the near future. This is mostly due to the complete collapse of the environmental movement's credibility. While the movement still exists and there are still battles to be fought, there are plenty of people capable of doing it without my help. This allows me to turn my attention to matters in my personal world.

I hope to get the Climate Scientist version 2.0 out and check NASA numbers from the FOIA stuff, and wrap up any other loose ends that come to mind. But these updates will come only as time permits and once they're done, so too will this blog be done.

NASA Responds To FOIA Request

NASA has responded to my FOIA Request. To recap, I requested two items from NASA: 1) the height in the atmosphere scanned by each footprint for each scan of the Aqua satellite, and 2) the Ai and Theta Bar i used by NASA to calculate the synthetic values for Aqua's channel 4.

Their response was that there are no records for the height of the footprint scans on the Aqua satellite and that the values for Ai and Theta Bar i are as follows:

A(i) = 0.38648412E-1, -0.13865858, 0.60949275, 0., 2.0037199, -0.95993573, 0., 0.28063577, -0.9307997E-1, 0.37032401E-1, -0.15795620E-2, -0.49943969E-2, -0.11815043E-1, 0.70557197E-1, -0.33432437E-1

thetabar(i) = A_mean(i) * amsu_noise_tr(i) where:

A_mean(i) = 1200.2806, 901.08093, 949.21509, 0., 1242.5841, 1382.2343, 0., 1359.1735, 1323.4860, 975.67328, 920.14246, 654.80048, 499.30478, 313.854318, 1955.0244

amsu_noise_tr(i) = 0.17, 0.22, 0.25, 0.14, 0.20, 0.17, 0.14, 0.16, 0.16, 0.22, 0.24, 0.35, 0.48, 0.80, 0.12

Notice that each of these collections of numbers contains 15 numbers. These correspond to the 15 channels on the Aqua satellite.

I still need to check that the above values, when used in NASA's formula for creating synthetic channel 4 data, actually produce the published values for that channel.

I'm out of ideas on how to obtain the height in the atmosphere each footprint scans.

Previous Posts In This Series
Appealing NASA's FOIA Decision
NASA Responds To FOIA Request
Quick FOIA Update
The Government Way
FOIA Request Filed With NASA

Appealing NASA's FOIA Decision

I've decided to appeal the FOIA decision from NASA, JPL. To do this I have to write out a hard copy letter and snail mail it to the appeals office. Here's the copy of that letter:

Dear Sirs,

I'm writing this letter to appeal a FOIA decision involving NASA JPL. The decision involved 3 issues, the first of which has been resolved. For the second and third decision, I've included relevant background information as well as the reason I'm appealing.

Issue 1: Radiative Transfer Algorithm Used By AIRS.
Status: Resolved.

Issue 2: Atmospheric Scan Depth Of Aqua Satellite AMSU Channel 5 Footprints.

Background
The Aqua satellite has an AMSU instrument that scans the atmosphere at 15 different channels. Each of these channels scans 30 different locations in the atmosphere. These locations are sometimes referred to as "footprints". Each footprint scans at a different height in the atmosphere.

NASA Discussion On How This Data Is Used
Where reliable sensor data is available, it is applied directly to the appropriate portion of the atmosphere, taking into account the angle of the observation.
- Steven Friedman, NASA JPL, Personal Correspondance (Included)

My FOIA Request
(2) Atmospheric scan depth for each footprint on channel 5 of the AQUA AMSU.

FOIA Response
This is to advise that NASA has no responsive Government records at JPL for parts (2) and (3) of your request.

Why I'm Appealing This Decision
NASA JPL has indicated that they apply each scan to the appropriate portion of the atmosphere. This is impossible to do if it is unknown at what depth of the atmosphere a given footprint is scanning.

Issue 3: The 230000 Scan Readings And/Or The Values Of Vectors Ai And Theta Bar i Used To Synthesis Aqua AMSU Channel 4 Data

Background
Aqua's AMSU channel 4 failed in late 2007, as described in AMSU-A Channel 4 NeDT Update: 20 December 2007 at the document archive. The data for channel 4 is now artificially generated. The generation process uses two vectors, Ai and Theta Bar i. The values for these vectors is not available to the public. The data for the vectors was itself generated from 230000 scans taken from the AMSU. This data is also not available to the public.

NASA Discussion On How This Data Is Used
See AIRS/AMSU/HSB Version 5 Modification of Algorithm to Account for Increased NeDT in AMSU Channel 4 at http://disc.sci.gsfc.nasa.gov/AIRS/documentation/v5_docs/AIRS_V5_Release_User_Docs/V5-Modification-for-AMSU-Ch-4-NeDT.pdf for a detailed discussion on this data. In particular, see equation (1) for the use of Ai and Theta Bar i.

My FOIA Request
(3) In references to the creation of synthetic readings for the AQUA AMSU channel 4, the 230000 cases used to create the values for the vectors Ai and Theta Bar i, or the values of vectors Ai and Theta Bar i themselves if the 230000 readings are no longer available. These values and readings are referenced but not actually provided in the document AIRS/AMSU/HSB Version 5 Modification of Algorithm to Account for Increased NeDT in AMSU Channel 4 available online at http://disc.sci.gsfc.nasa.gov/AIRS/documentation/v5_docs/AIRS_V5_Release_User_Docs/V5-Modification-for-AMSU-Ch-4-NeDT.pdf..."

FOIA Response
This is to advise that NASA has no responsive Government records at JPL for parts (2) and (3) of your request.

Why I'm Appealing This Decision
NASA JPL has indicated that this data is used to generate the synthesized channel 4 data. This would be impossible to do if the values for vectors Ai and Theta Bar i were not known to them, as these values are two of the four values used to generate the synthesized data.

Note that NASA JPL also indicated that NASA GSFC may have responsive records for issue 3. I am in contact with them to see if this is the case, but as my ability to appeal this decision is limited to 30 days, I am simultaneously appealing.

Thank you for you time and help in this matter.

...and the e-mail sent to NASA, GSFC for the data used to generate synthetic channel 4 values:

I was informed by Dennis B. Mahon of NASA, JPL that NASA GSFC may have information regarding data used to generate synthetic channel 4 data for Aqua's AMSU channel 4. Specifically, I am looking for he 230000 cases used to create the values for the vectors Ai and Theta Bar i, or the values of vectors Ai and Theta Bar i themselves if the 230000 readings are no longer available. These values and readings are referenced but not actually provided in the document AIRS/AMSU/HSB Version 5 Modification of Algorithm to Account for Increased NeDT in AMSU Channel 4 available online at http://disc.sci.gsfc.nasa.gov/AIRS/documentation/v5_docs/AIRS_V5_Release_User_Docs/V5-Modification-for-AMSU-Ch-4-NeDT.pdf..."

See equation (1) of the referenced PDF for specific details.

Please note that this request is associated with a FOIA involving NASA, JPL that is currently under appeal.

Previous Posts In This Series
NASA Responds To FOIA Request
Quick FOIA Update
The Government Way
FOIA Request Filed With NASA

Aqua Satellite Project, Update 10 Released.

Update 10 for the Aqua Satellite Project is ready. You can download it here. This update is part 2 of a 2 part release that adds global griding of the data.

This release adds the ISCCPSummarize program that summarizes ISCCP grid-style data generated by AMSUToGrid and the ISCCPNormalize program that normalizes output files from ISCCPSummarize on a scale of 0.0 to 1.0.

This release also fixed a bug in the AMSUToGrid class DegreeGrid that was passing grid edge rather than grid center to weights, replaced custom code in DegreeGrid to calculate grid locations with code that uses official ISCCP values, and added mask images and generated grid data to MapToGrid folder. Masks are in the Masks subfolder, generated grid data is in the Generated Grid Data subfolder. There are Masks and generated data for Africa, Antarctica, Arctic Ocean, Asia, Atlantic Ocean, Australia, Europe, Indian Ocean, Land and Water, North America, Pacific Ocean, South America, Southern Ocean, and US Lower 48 States. Generated data is in text, XML, JavaScript, and C code formats.

Regional Data
In addition to summarizing data for the entire globe, the ISCCPSummarize program provides summaries for the following regions:

Africa

Antarctica

Asia

Australia

Europe

North America

South America

Arctic Ocean

Atlantic Ocean

Indian Ocean

Pacific Ocean

Southern Ocean

...and the following UAH-like regions:

● Global Land
● Global Ocean
● Northern Hemisphere
● Northern Hemisphere Land
● Northern Hemisphere Ocean
● Southern Hemisphere
● Southern Hemisphere Land
● Southern Hemisphere Ocean
● Northern Extra Tropics
● Northern Extra Tropics Land
● Northern Extra Tropics Ocean
● Southern Extra Tropics
● Southern Extra Tropics Land
● Southern Extra Tropics Ocean
● North Pole
● North Pole Land
● North Pole Ocean
● South Pole
● South Pole Land
● South Pole Ocean
● U.S.A. Lower 48 States

These UAH regions use the same latitude boundaries as genuine UAH regions do. My thanks to Dr. Christy of UAH for providing me with the UAH region boundary information.

Three-Valued Logic And Irreproducible Results In Science, Part II

Introduction
In the previous post for this series we discussed how Two-Valued and Three-Valued Logic works. In this post we take a look at how Three-Valued Logic (3VL) interacts with the results of science. We'll be using the properties of 3VL's three truth values, True, False, and Unknown, to do this.

In particular we'll be examining how the truth value Unknown of 3VL spreads through science like a virus. In fact, we'll make the claim that scientific results that have unverifiable claims that cannot be independently reproduced are viruses. Any further work that builds upon them is infected and becomes unverifiable and irreproducible itself.

Scientific Claims Must Be Verified

In science, it's not enough to simply make a claim. Claims must be independently verifiable before they are accepted.

When the procedures of the claim are followed and the results of the claim are reproduced, the claim is said to be True. When the procedures of the claim are followed and the results of the claim are not reproduced, the claim is said to be False. An example of a scientific claim being shown False is the original Cold Fusion hypothesis by Martin Fleischmann and Stanley Pons.

However, in situations where the procedures of the claim cannot be followed, we cannot show the claim to be True or False. It is in these cases that the need for 3VL arises, and the truth of the claim is Unknown.

Technical Aside: Cold fusion research has continued since the days of Fleischmann and Pons. Today it is funded by several governments around the world and many researchers in the field believe that the original claims of Fleischmann and Pons have been vindicated. However, several mainstream scientific organizations still disagree, most notably the U.S. Department of Energy and Science magazine. For more information on the ongoing cold fusion research, see the LENR-CANR website.

Example: Aqua Satellite Channel 4 Virus
It helps to have an example, so we'll be using channel 4 of the AMSU on the Aqua satellite. Channel 4 failed completely around December, 2007. In response to this, NASA created a new algorithm and has used it to synthetically create channel 4 data from October 1st, 2007 onward.

While NASA publishes the algorithm used to create synthetic channel 4 values, that algorithm requires certain data that is not available to anyone outside of NASA. Even the folks at NASA's JPL, who are in charge of the Aqua satellite, have said they don't have access to the data.

Without this data it's impossible to verify if the algorithm for synthesizing channel 4 data is correct, even though the algorithm itself is published. Similarly, we cannot demonstrate that the algorithm fails to correctly synthesis channel 4 data. Therefore, the ability of the algorithm to correctly synthesis data must be classified as Unknown because the statement that the algorithm is accurate cannot independently be shown to be True or False.

How The Virus Spreads
To qualify as a virus, the Unknown values must be capable of spreading to other works. To see how this occurs, let's first take a look at how False research is capable of spreading.

We take the example of research attempting to build upon claims that have been demonstrated False, in this case Cold Fusion. The diagram above shows new research that is correct being combined with the results of Cold Fusion. Because Cold Fusion has been shown to be False, the overall conclusions of the research must be False because they require Cold Fusion.

Logically, this situation is captured by a simple predicate: True AND False = False.

The same situation occurs in 3VL when using Unknown, rather than False, values. Because the creation of synthetic data for Aqua's AMSU Channel 4 cannot be shown to be True or False, it is Unknown.  Any research combined with it, no matter how good it is, produces a final result that is also Unknown.

This too is captured by a simple 3VL predicate: True AND Unknown = Unknown.

A concrete example of the spreading of Unknown results in published research is provided by NASA's claims of increased yield due to synthetic channel 4 data. We'll assume that these claims are True and that yields are in fact increasing. However, even with this assumption, we cannot demonstrate that yields should be increasing. Because it cannot be verified that the synthetic channel 4 data is valid, we cannot verify that the synthetic data causes bad data to pass QA or good data to fail QA. The quality of the data in these increased yields is Unknown. This is because the quality of the synthetic data is Unknown.

This cascading of the Unknown value continues through anything that uses the data from these increased yields. In practice, it turns out that all processes referred to by NASA as "Level 2" or higher that use Aqua AMSU data will be infected by the Unknown values. That is, all such data sets have an Unknown truth value themselves due to their dependence upon the increased yield data. These "Level 2" products include:

● Temperature profile from 3 mbar (45 km) to the surface.
● Water vapor profiles.
● Snow and ice coverage.
● Cloud liquid water.
● Cloud-cleared IR radiances.
● Rain Rate.
● Ozone.
● Carbon Dioxide Support Products.

Other Examples Of The Virus Spreading
GHCN Data

A while back Willis Eschenbach made the claim that GHCN data at Darwin station was being manipulated to show a warming where none existed in the raw data. Pro-AGW bloggers jumped on this claiming the adjustments were valid.

The problem is the Australian CSIRO Atmospheric Research Center provided no reason why the adjustments were made, stating only that Darwin is a urban site (which should make adjustments go down, not up).

This is an illustrative example of the problem with GHCN adjustments. Even though GHCN provides its raw data and describes its adjustment procedures, it's adjustments cannot be replicated when reasons for the adjustments aren't given.

For this reason, the validity of GHCN data must be classified as Unknown. This Unknown value spreads to anything using GHCN data. This includes the Intergovernmental Panel on Climate Change 4th Assessment Report.

Deep Impact

Deep Impact was a NASA mission to probe a comet by slamming a probe into the comet Tempel I and analyzing the impact. The Deep Impact team at NASA JPL released a photo of the comet with water photoshopped onto its surface (seen at left), a series of medium resolution images of the event, and a chart of the thermal emission spectra of the debris. The chart is shown below.

However, the chart is made of data that's been modified by NASA and the raw data used to generate the Tempel I spectra has never been released. This makes it impossible to verify that the scanners actually produced the results shown in the chart. The resulting unverifiable claims are therefore Unknown.

Curing The Virus
As far-reaching as the consequences of the examples provided here are, we've covered only a small handful of examples. Many more could be provided. Their flow through related work could be tracked and we'd discover that a significant portion of modern science rests upon unverifiable claims.

I think most people wouldn't consider such science to be science at all, but as a problem that stands in the way of science. Fortunately, it's a problem that's easy to fix.

Simply make the claims verifiable.

By making the raw data and computer code used to generate the claims publicly available, and by noting why changes are made to raw data, claims that are currently unverifiable can be demonstrated to be True or False.

And that is the whole reason science exists in the first place.

References
Three-Valued Logic And Irreproducible Results In Science, Part I
Cold Fusion claims by Martin Fleischmann and Stanley Pons - Wikipedia Entry
NASA Responds To FOIA Request
AIRS/AMSU/HSB Version 5 Modification of Algorithm to Account for Increased NeDT in AMSU Channel 4
AMSU - Wikipedia Entry
AIRS/Aqua Level 2 Carbon Dioxide Support Products
The Smoking Gun At Darwin Zero
Willis Eschenbach caught lying about temperature trends
Updating Australia’s high-quality annual temperature dataset
GHCN V.2 Raw Data
GHCN Quality Control, Homogeneity Testing, and Adjustment Procedures
GHCN-Monthly Version 2 Introduction
Deep Impact
LENR-CANR website

Aqua Satellite Project, Update 9 Released.

Update:
According to stats at SourceForge, this is the most popular release of the Aqua Satellite Project yet. To the folks downloading the software: feel free to tell me of any bugs you may find or improvements you'd like to see.
============


Update 9 for the Aqua Satellite Project is ready. You can download it here. This update is part 1 of a 2 part release that adds global griding of the data. This release adds the AMSUToGrid program for converting extracted csv files to a globally gridded format. The csv files must be extracted with longitude and latitude data. The output format is identical to the format used by the ISCCP project for 2.5 degree equal area grid data in ASCII format. The following weighting options are available as command line switches:

● DropInTheBucket
● InverseDistance
● LinearInterpolation
● SinXOverX
● CubicConvolution

Three-Valued Logic And Irreproducible Results In Science, Part I

Introduction
This is a short series of two posts that discusses Three-Valued Logic (3VL) and uses it to demonstrate how irreproducible results corrupt not only the scientific work in which they appear, but also spread that corruption to any related work.

In short, this series of posts demonstrates that irreproducible results are viral. My current work on the Aqua satellite is used as an illustrative example in the second of these posts.

In this post we look at Two-Valued Logic and Three-Valued Logic and briefly discuss how they work.

Two-Valued Logic
To start off, let's take a quick look at the more common two valued logic system. This system gets its name from the fact that there are only two possible values for any statement: True or False. Two-Valued Logic is also called Boolean logic.

In regards to this post, we're most concerned with how these logic values of True and False can move through a system of logical predicates. For example, if we have a True statement and a False statement and a logical predicate requiring at least one true statement, than that predicate transforms the one True statement and one False statement into a single statement that is True. This is because the requirement of at least one True statement has been met and the predicate is therefore True.

Such a predicate is commonly referred to as an OR statement. For an OR statement to be True, either statement 1 or statement 2 must be True. There are several types of predicates in Boolean logic. The most common are OR, AND, and NOT. One way to define predicates is using a true table. A truth table shows all possible inputs to the predicate and what the output of the predicate is. The truth tables for OR, AND, and NOT in Boolean logic are shown below.

OR Truth Table.

AND Truth Table

NOT Truth Table.

In a nutshell, the OR predicate accepts two inputs and converts them to a single output. That output will be True if either of its inputs are True, or False if both inputs are False.

The AND predicate also accepts two inputs, but returns TRUE only if both inputs are True. In all other cases it returns False.

The NOT predicate simply flips True inputs to False outputs and False inputs to True Outputs.

TECHNICAL ASIDE: Truth tables are one way to define logical predicates. They use "case analysis", a method of listing every possible case and the corresponding outcome. However, there are other, more elegant ways to define predicates. In his book Predicate Calculus And Programming Semantics, the late Edsger Dijkstra defined the AND predicate using what is known as The Golden Rule, shown below, with the equivalence operator (==) having the lowest binding and meaning "Is The Same As":

      The Golden Rule: p AND q == p == q == p OR q

This type of definition is considered superior to case analysis. Once the properties of the various predicates are captured in this manner, they can be used as building blocks to far more sophisticated theorems. Case analysis, on the other hand, can never demonstrate anything more than what is provided by the cases themselves.

Three-Valued Logic
Three-Valued Logic (3VL) builds upon Boolean Logic. There are several conventions for 3VL. We'll be looking at a particular convention, one in which the third truth value is Unknown. So 3VL, as discussed here, has three truth values: True, False, and Unknown.

TECHNICAL ASIDE: Relational database products make heavy use of 3VL, and refer to the Unknown truth value as NULL. The late E.F. Codd invented relational databases while working for IBM, He introduced the convention of using the word NULL to represent Unknown, and assigned NULL the symbol of the Greek lowercase omega, shown in the image at the beginning of this post.


Like Boolean Logic, 3VL has predicates that can be defined using truth tables. The truth tables for 3VL AND, OR, and NOT predicates are shown below.

When the only values in use are True and False, 3VL gives the exact same answers as Boolean logic. However, when an Unknown truth value is used as inputs it can also show up as the outputs for a predicate. The result of the predicate "False OR Unknown" is Unknown. The result of the predicate "NOT Unknown" is always Unknown. The result of an AND predicate with an Unknown input is always Unknown or False, never True.

The Point Of All This
It is this last result that interests us here. The result of the predicate "X AND Unknown" can never be True, no matter what the value of X is.

This is our stopping point for now. The next post in this series will discuss the relationship between Three-Valued Logic and science.

References
Boolean Logic - Wikipedia Entry
Ternary Logic - Wikipedia Entry
SQL NULL - Wikipedia Entry
Predicate Calculus And Programming Semantics - Edsger Dijkstra
A Relational Model of Data for Large Shared Data Banks - Edgar F. Codd