Three-Valued Logic And Irreproducible Results In Science, Part II

Introduction
In the previous post for this series we discussed how Two-Valued and Three-Valued Logic works. In this post we take a look at how Three-Valued Logic (3VL) interacts with the results of science. We'll be using the properties of 3VL's three truth values, True, False, and Unknown, to do this.

In particular we'll be examining how the truth value Unknown of 3VL spreads through science like a virus. In fact, we'll make the claim that scientific results that have unverifiable claims that cannot be independently reproduced are viruses. Any further work that builds upon them is infected and becomes unverifiable and irreproducible itself.

Scientific Claims Must Be Verified

In science, it's not enough to simply make a claim. Claims must be independently verifiable before they are accepted.

When the procedures of the claim are followed and the results of the claim are reproduced, the claim is said to be True. When the procedures of the claim are followed and the results of the claim are not reproduced, the claim is said to be False. An example of a scientific claim being shown False is the original Cold Fusion hypothesis by Martin Fleischmann and Stanley Pons.

However, in situations where the procedures of the claim cannot be followed, we cannot show the claim to be True or False. It is in these cases that the need for 3VL arises, and the truth of the claim is Unknown.

Technical Aside: Cold fusion research has continued since the days of Fleischmann and Pons. Today it is funded by several governments around the world and many researchers in the field believe that the original claims of Fleischmann and Pons have been vindicated. However, several mainstream scientific organizations still disagree, most notably the U.S. Department of Energy and Science magazine. For more information on the ongoing cold fusion research, see the LENR-CANR website.

Example: Aqua Satellite Channel 4 Virus
It helps to have an example, so we'll be using channel 4 of the AMSU on the Aqua satellite. Channel 4 failed completely around December, 2007. In response to this, NASA created a new algorithm and has used it to synthetically create channel 4 data from October 1st, 2007 onward.

While NASA publishes the algorithm used to create synthetic channel 4 values, that algorithm requires certain data that is not available to anyone outside of NASA. Even the folks at NASA's JPL, who are in charge of the Aqua satellite, have said they don't have access to the data.

Without this data it's impossible to verify if the algorithm for synthesizing channel 4 data is correct, even though the algorithm itself is published. Similarly, we cannot demonstrate that the algorithm fails to correctly synthesis channel 4 data. Therefore, the ability of the algorithm to correctly synthesis data must be classified as Unknown because the statement that the algorithm is accurate cannot independently be shown to be True or False.

How The Virus Spreads
To qualify as a virus, the Unknown values must be capable of spreading to other works. To see how this occurs, let's first take a look at how False research is capable of spreading.

We take the example of research attempting to build upon claims that have been demonstrated False, in this case Cold Fusion. The diagram above shows new research that is correct being combined with the results of Cold Fusion. Because Cold Fusion has been shown to be False, the overall conclusions of the research must be False because they require Cold Fusion.

Logically, this situation is captured by a simple predicate: True AND False = False.

The same situation occurs in 3VL when using Unknown, rather than False, values. Because the creation of synthetic data for Aqua's AMSU Channel 4 cannot be shown to be True or False, it is Unknown.  Any research combined with it, no matter how good it is, produces a final result that is also Unknown.

This too is captured by a simple 3VL predicate: True AND Unknown = Unknown.

A concrete example of the spreading of Unknown results in published research is provided by NASA's claims of increased yield due to synthetic channel 4 data. We'll assume that these claims are True and that yields are in fact increasing. However, even with this assumption, we cannot demonstrate that yields should be increasing. Because it cannot be verified that the synthetic channel 4 data is valid, we cannot verify that the synthetic data causes bad data to pass QA or good data to fail QA. The quality of the data in these increased yields is Unknown. This is because the quality of the synthetic data is Unknown.

This cascading of the Unknown value continues through anything that uses the data from these increased yields. In practice, it turns out that all processes referred to by NASA as "Level 2" or higher that use Aqua AMSU data will be infected by the Unknown values. That is, all such data sets have an Unknown truth value themselves due to their dependence upon the increased yield data. These "Level 2" products include:

● Temperature profile from 3 mbar (45 km) to the surface.
● Water vapor profiles.
● Snow and ice coverage.
● Cloud liquid water.
● Cloud-cleared IR radiances.
● Rain Rate.
● Ozone.
● Carbon Dioxide Support Products.

Other Examples Of The Virus Spreading
GHCN Data

A while back Willis Eschenbach made the claim that GHCN data at Darwin station was being manipulated to show a warming where none existed in the raw data. Pro-AGW bloggers jumped on this claiming the adjustments were valid.

The problem is the Australian CSIRO Atmospheric Research Center provided no reason why the adjustments were made, stating only that Darwin is a urban site (which should make adjustments go down, not up).

This is an illustrative example of the problem with GHCN adjustments. Even though GHCN provides its raw data and describes its adjustment procedures, it's adjustments cannot be replicated when reasons for the adjustments aren't given.

For this reason, the validity of GHCN data must be classified as Unknown. This Unknown value spreads to anything using GHCN data. This includes the Intergovernmental Panel on Climate Change 4th Assessment Report.

Deep Impact

Deep Impact was a NASA mission to probe a comet by slamming a probe into the comet Tempel I and analyzing the impact. The Deep Impact team at NASA JPL released a photo of the comet with water photoshopped onto its surface (seen at left), a series of medium resolution images of the event, and a chart of the thermal emission spectra of the debris. The chart is shown below.

However, the chart is made of data that's been modified by NASA and the raw data used to generate the Tempel I spectra has never been released. This makes it impossible to verify that the scanners actually produced the results shown in the chart. The resulting unverifiable claims are therefore Unknown.

Curing The Virus
As far-reaching as the consequences of the examples provided here are, we've covered only a small handful of examples. Many more could be provided. Their flow through related work could be tracked and we'd discover that a significant portion of modern science rests upon unverifiable claims.

I think most people wouldn't consider such science to be science at all, but as a problem that stands in the way of science. Fortunately, it's a problem that's easy to fix.

Simply make the claims verifiable.

By making the raw data and computer code used to generate the claims publicly available, and by noting why changes are made to raw data, claims that are currently unverifiable can be demonstrated to be True or False.

And that is the whole reason science exists in the first place.

References
Three-Valued Logic And Irreproducible Results In Science, Part I
Cold Fusion claims by Martin Fleischmann and Stanley Pons - Wikipedia Entry
NASA Responds To FOIA Request
AIRS/AMSU/HSB Version 5 Modification of Algorithm to Account for Increased NeDT in AMSU Channel 4
AMSU - Wikipedia Entry
AIRS/Aqua Level 2 Carbon Dioxide Support Products
The Smoking Gun At Darwin Zero
Willis Eschenbach caught lying about temperature trends
Updating Australia’s high-quality annual temperature dataset
GHCN V.2 Raw Data
GHCN Quality Control, Homogeneity Testing, and Adjustment Procedures
GHCN-Monthly Version 2 Introduction
Deep Impact
LENR-CANR website

Aqua Satellite Project, Update 9 Released.

Update:
According to stats at SourceForge, this is the most popular release of the Aqua Satellite Project yet. To the folks downloading the software: feel free to tell me of any bugs you may find or improvements you'd like to see.
============


Update 9 for the Aqua Satellite Project is ready. You can download it here. This update is part 1 of a 2 part release that adds global griding of the data. This release adds the AMSUToGrid program for converting extracted csv files to a globally gridded format. The csv files must be extracted with longitude and latitude data. The output format is identical to the format used by the ISCCP project for 2.5 degree equal area grid data in ASCII format. The following weighting options are available as command line switches:

● DropInTheBucket
● InverseDistance
● LinearInterpolation
● SinXOverX
● CubicConvolution

Three-Valued Logic And Irreproducible Results In Science, Part I

Introduction
This is a short series of two posts that discusses Three-Valued Logic (3VL) and uses it to demonstrate how irreproducible results corrupt not only the scientific work in which they appear, but also spread that corruption to any related work.

In short, this series of posts demonstrates that irreproducible results are viral. My current work on the Aqua satellite is used as an illustrative example in the second of these posts.

In this post we look at Two-Valued Logic and Three-Valued Logic and briefly discuss how they work.

Two-Valued Logic
To start off, let's take a quick look at the more common two valued logic system. This system gets its name from the fact that there are only two possible values for any statement: True or False. Two-Valued Logic is also called Boolean logic.

In regards to this post, we're most concerned with how these logic values of True and False can move through a system of logical predicates. For example, if we have a True statement and a False statement and a logical predicate requiring at least one true statement, than that predicate transforms the one True statement and one False statement into a single statement that is True. This is because the requirement of at least one True statement has been met and the predicate is therefore True.

Such a predicate is commonly referred to as an OR statement. For an OR statement to be True, either statement 1 or statement 2 must be True. There are several types of predicates in Boolean logic. The most common are OR, AND, and NOT. One way to define predicates is using a true table. A truth table shows all possible inputs to the predicate and what the output of the predicate is. The truth tables for OR, AND, and NOT in Boolean logic are shown below.

OR Truth Table.

AND Truth Table

NOT Truth Table.

In a nutshell, the OR predicate accepts two inputs and converts them to a single output. That output will be True if either of its inputs are True, or False if both inputs are False.

The AND predicate also accepts two inputs, but returns TRUE only if both inputs are True. In all other cases it returns False.

The NOT predicate simply flips True inputs to False outputs and False inputs to True Outputs.

TECHNICAL ASIDE: Truth tables are one way to define logical predicates. They use "case analysis", a method of listing every possible case and the corresponding outcome. However, there are other, more elegant ways to define predicates. In his book Predicate Calculus And Programming Semantics, the late Edsger Dijkstra defined the AND predicate using what is known as The Golden Rule, shown below, with the equivalence operator (==) having the lowest binding and meaning "Is The Same As":

      The Golden Rule: p AND q == p == q == p OR q

This type of definition is considered superior to case analysis. Once the properties of the various predicates are captured in this manner, they can be used as building blocks to far more sophisticated theorems. Case analysis, on the other hand, can never demonstrate anything more than what is provided by the cases themselves.

Three-Valued Logic
Three-Valued Logic (3VL) builds upon Boolean Logic. There are several conventions for 3VL. We'll be looking at a particular convention, one in which the third truth value is Unknown. So 3VL, as discussed here, has three truth values: True, False, and Unknown.

TECHNICAL ASIDE: Relational database products make heavy use of 3VL, and refer to the Unknown truth value as NULL. The late E.F. Codd invented relational databases while working for IBM, He introduced the convention of using the word NULL to represent Unknown, and assigned NULL the symbol of the Greek lowercase omega, shown in the image at the beginning of this post.


Like Boolean Logic, 3VL has predicates that can be defined using truth tables. The truth tables for 3VL AND, OR, and NOT predicates are shown below.

When the only values in use are True and False, 3VL gives the exact same answers as Boolean logic. However, when an Unknown truth value is used as inputs it can also show up as the outputs for a predicate. The result of the predicate "False OR Unknown" is Unknown. The result of the predicate "NOT Unknown" is always Unknown. The result of an AND predicate with an Unknown input is always Unknown or False, never True.

The Point Of All This
It is this last result that interests us here. The result of the predicate "X AND Unknown" can never be True, no matter what the value of X is.

This is our stopping point for now. The next post in this series will discuss the relationship between Three-Valued Logic and science.

References
Boolean Logic - Wikipedia Entry
Ternary Logic - Wikipedia Entry
SQL NULL - Wikipedia Entry
Predicate Calculus And Programming Semantics - Edsger Dijkstra
A Relational Model of Data for Large Shared Data Banks - Edgar F. Codd

SEC Watched Porn As Economy Crashed

I try to stay clear of politics on this blog, but every once in a while something happens that I just can't let slip by. This is one of those times.

The AP is reporting that the SEC's inspector general is conducting probes into 31 of its employee's charged with downloading and viewing porn on government time, using government equipment, while the economy crashed.

Here are a few highlights from the report:

● A senior attorney at the SEC's Washington headquarters spent up to eight hours a day looking at and downloading pornography. When he ran out of hard drive space, he burned the files to CDs or DVDs, which he kept in boxes around his office. He agreed to resign, an earlier watchdog report said.

● An accountant was blocked more than 16,000 times in a month from visiting websites classified as "Sex" or "Pornography." Yet he still managed to amass a collection of "very graphic" material on his hard drive by using Google images to bypass the SEC's internal filter, according to an earlier report from the inspector general.

● Seventeen of the employees were "at a senior level," earning salaries of up to $222,418.

For more details, see the original story here.

 

Aqua Satellite Project, Update 8 Released. Now With More She-Hulk!

Update 8 for the Aqua Satellite Project is ready. You can download it here. This update is part 2 of a 2 part release that adds some spit and polish to the graphics, charting, and platform-independant font engines. Highlights for this release include:

● Added event handling to all shapes classes. These classes can now respond to mouse, keyboard, touch, and command events.
● Added anti-aliasing capabilities to shapes and FontManager. Anti-aliasing is turned on by default. It can be turned off by setting a flag in the various graphics and text calls.
● Added font scaling capabilities to the FontManager. The built-in fonts can now be used at any desired font size.
● Added histogram, bi-linear interpolation, equalize, threshold, grayscale, edge detection, translation, rotation, scaling, and erosion filters for use with bitmaps. These filters are located in the new Filters.h and Filters.cpp files in the Graphics folder.
● Made significant enhancements to the DrawingContext2D and Color classes.
● Fixed several bugs in the Chart class.

The code has been placed in the public domain.

Samples

Edge Detection

Equalize

Grayscale

Threshold

Rotate

Translate

Scale

Scaled, Anti-Aliased, Platform-Independant Text

Erosion

Bi-Linear Interpolation

Original Text Image (For Comparison)

Anti-Aliasing

Area Chart

Bar Chart

Line Chart

Pie Chart

Point Chart

NASA Responds To FOIA Request

NASA has responded to my FOIA request.

In a nutshell, they provided me with a link to the AMSU-A Radiative Transfer Algorithm documentation and said they had no information on the scan depths for the footprints of channel 5 on the AMSU and didn't have the vector data they use to synthesis AMSU channel 4.

Both of the claims of "no information" seem impossible to me, as they're needed to process AMSU data the way NASA JPL does.

While I mull over my next move, here's their response.

Your Freedom of Information Act (FOIA) request for release of information from the files of the National Aeronautics and Space Administration (NASA) received in FOIA processing at the NASA Management Office-Jet Propulsion Laboratory on April 8, 2010. You requested the following:

"...(1) Documentation on how the AQUA AMSU-A Radiative Transfer Algorithm works. (2) Atmospheric scan depth for each footprint on channel 5 of the AQUA AMSU. (3) In references to the creation of synthetic readings for the AQUA AMSU channel 4, the 230000 cases used to create the values for the vectors Ai and Theta Bar i, or the values of vectors Ai and Theta Bar i themselves if the 230000 readings are no longer available. These values and readings are referenced but not actually provided in the document AIRS/AMSU/HSB Version 5 Modification of Algorithm to Account for Increased NeDT in AMSU Channel 4 available online at http://disc.sci.gsfc.nasa.gov/AIRS/documentation/v5_docs/AIRS_V5_Release_User_Docs/V5-Modification-for-AMSU-Ch-4-NeDT.pdf..."

This is to advise you that responsive records to part (1) of your request may be found at the following web site:

http://eospso.gsfc.nasa.gov/eos_homepage/for_scientists/atbd/docs/AIRS/atbd-airs-L2.pdf

This is to advise that NASA has no responsive Government records at JPL for parts (2) and (3) of your request. Additionally, pertaining to part (3) of your request, we have no responsive records because the specific research and development was performed by the NASA Goddard Space Flight Center (GSFC). Therefore, it is possible that GSFC may have responsive records.

You have appeal rights concerning these actions.

And I do want to thank them for not charging me for this request. :)

Previous Posts In This Series
Quick FOIA Update
The Government Way
FOIA Request Filed With NASA

Aqua Satellite Project, Update 7 Released

Update 7 for the Aqua Satellite Project is ready. You can download it here. This update is part 1 of a 2 part release that adds some spit and polish to the graphics, charting, and platform-independant font engines.

This release adds all the functionality that wasn't yet written for the Chart class and removes the need for external files from the font engine (although you can still use such files to add your own fonts, they are no longer required for the fonts shipped with the font engine). It adds a FontManger that manages the built-in fonts, using less memory, less disk space, and runs faster than the old method. As before, the built-in fonts are Arial, Courier, Helvetica, Times New Roman, TrebuchetMS, and Verdana in sizes of 10, 12, 18, 24, 36, 48, and 72. A bug in the Courier font bitmaps was also fixed.

The LiveGraph application no longer has a -font parameter as it can now use the FontManager to produce the fonts it needs.

All code has been placed in the public domain.

Draw2D V1.1 JavaScript Graphics Engine Update Released

Version 1.1 of the Draw2D JavaScript Graphics Engine has been released. This is a minor update that adds speed improvements to all the curve drawing classes (Quadradic Curves, Cubic Curves, Round Rectangles, Ovals And Arcs) and fixes a bug in the rendering of Cubic Curves.

You can get the source code, several example HTML pages, and a Programmer's Manual here.

The code has been placed in the public domain.

Previous Posts In This Series
Draw2D JavaScript Graphics Engine Released

Quick FOIA Update

Just dropping a note to update the status of my FOIA. In the previous post I noted that NASA JPL wanted me to state up front how much I was willing to pay for the information.

Having no idea what these fees are for, I figured the next step is to ask why they charge these fees. That way I can give some sort of reasonable answer on what I'd pay. Here's my resonse:

Dear Mr. Mahon,

As I mentioned in my previous e-mail, this is my first FOIA request, So, if I may, I'd like to ask the purpose of the fees. Are they to cover mailing costs? Will I be actually paying the salary of the person looking up the information? Are they arbitrary fees designed to prevent "fishing expeditions? The reason I ask is so that I can provide you with an answer that is sensible for the purpose of the fees. If possible, providing an "average fee" for obtaining one document would be useful.

Previous Posts In This Series
The Government Way
FOIA Request Filed With NASA

The Government Way

I got a response back from NASA JPL regarding my FOIA request. They want to know up front just how much I'm willing to pay for the information. Gotta love how our government operates. :)

Here's the text (emphasis mine):

Greetings:

Please respond with your complete postal mailing address, phone and fax numbers. Also, you must state your willingness to pay fees that may be assessed in processing your request. Please reply stating your willingness to pay fees and the amount you are willing to pay so we may proceed with your request. Thank you.

So now I'll have to mull over just how much I'm willing to pay for information my taxes already paid for.

FOIA Request Filed With NASA

About three weeks ago, as I noted in this post, I sent off a request to NASA JPL for information needed to reproduce their synthetic creation of AMSU channel 4 data and for the atmospheric scan depths of each footprint for channel 5 on the AMSU.

I've not received a response back, and 3 weeks is one week longer than my usual waiting period of two weeks. So today I filed a FOIA with NASA JPL for the information. A copy of the request is shown below.

Dear Mr. Mahon,

This is my first FOIA request, so allow me to apologize in advance for any procedural mistakes I may make.

About three weeks ago (on or about March, 13th, 2010) I requested several pieces of information from NASA JPL via the "Ask AIRS" web interface and have received no response. So I am now requesting that information via the FOIA. The requested information is:

● Documentation on how the AQUA AMSU-A Radiative Transfer Algorithm works.

● Atmospheric scan depth for each footprint on channel 5 of the AQUA AMSU.

● In references to the creation of synthetic readings for the AQUA AMSU channel 4, the 230000 cases used to create the values for the vectors Ai and Theta Bar i, or the values of vectors Ai and Theta Bar i themselves if the 230000 readings are no longer available. These values and readings are referenced but not actually provided in the document AIRS/AMSU/HSB Version 5 Modification of Algorithm to Account for Increased NeDT in AMSU Channel 4 available online at
http://disc.sci.gsfc.nasa.gov/AIRS/documentation/v5_docs/AIRS_V5_Release_User_Docs/V5-Modification-for-AMSU-Ch-4-NeDT.pdf

Thank you for your time.

P.S.

Just in case you're wondering where my normal two-week waiting period came from, two weeks is the amount of time it took Stephen Hawking to respond to a question I had regarding black holes and quantum physics. Given Dr. Hawking's well-known medical issues, this should be a reasonable time frame for any reply from anyone else.

Aqua Satellite Project, Update 6 Released

Update 6 for the Aqua Satellite Project is ready. You can download it here. The big item is the new LiveGraph program which creates a movie of a graph. This movie can be overlaid on top of GridToMap movies.

A sample LiveGraph movie is shown below. It graphs cloud and surface temperature data over a map of clouds.

Movie made from LiveGraph and GridToMap.

In support to LiveGraph, a graphics engine, a charting engine, and a platform-independant font engine have been added to the code. There was also some reworking of the MathUtils.h file to make it a lot smaller. Most of the code was moved to a new Math folder.

All code has been placed in the public domain.