REE Solar Terrestrial Probe Science Application Team (REE/STP)

FINAL REPORT

1998-2001

Artist's depiction of a Geospace constellation of constellations at dawn.

Artist's depiction of a multi scale Geospace constellation at dawn.

OBJECTIVE

The objective of the REE Solar-Terrestrial Probe Science Applications Team (REE/STP) was to generate proofs-of-concept that prepare the way for performing computationally demanding yet fundamentally important data analyses on board spacecraft associated with NASA's Solar-Terrestrial Probe Line and Sun-Earth Connections theme.

Space is an information-rich environment that can place extreme demands on the systems that transport information from the spacecraft sensor to the data's users on the surface of the Earth. Communication rate and latency limits impose severe costs on mission performance. In addition, to make progress Science missions are being driven towards multiple spacecraft missions involving five to tens to a hundred or more platforms. By automating and moving some data analyses onboard the spacecraft, dramatic reductions in communications requirements can be achieved while simultaneously enabling more sophisticated and more focused Science missions.

APPROACH

To examine the benefits mentioned above, two important space science data processing methods, plasma moment calculation and cross-correlative interferometry, were modeled and implemented for multi CPU operation. Subordinate to these calculations are linear algebraic calculations, which we also studied in the space science context. The goals of REE/STP have been substantially reached, namely that initial model space science applications using REE techniques were developed.

To move the STP applications closer to deployment, REE/STP began to study how REE enabled software applications might be applied to upcoming missions in the STP line and the Sun-Earth Connections theme. In particular, application to Magnetospheric Multi Scale (MMS, launch 2008), a multi spacecraft mission to the dynamic boundary layers in the Earth's magnetosphere, was determined to be exceptionally promising (figure 1). Therefore REE/STP shifted focus towards specific capabilities that will enhance the science return of MMS. A critical objective was the incorporation of real data into the calculations we performed. This incorporation is an important step in demonstrating the Technological Readiness of this technology for space flight.

Finally, automation and autonomy are becoming more important to reduce mission operations costs and to reduce the impact of long communication latencies inherent in many space missions. For most REE test mission scenarios, the resource requirements of REE must be minimal, including communications and operations costs. Therefore, REE/STP studied ways to automate some data analysis or processing.

Magnetospheric Multi Scale Themes

Figure 1. Magnetospheric Multi Scale. A Solar-Terrestrial Probe mission of the Sun-Earth Connections Theme to study how the microphysics of Geospace boundaries affects energy and matter transport at micro- and meso-scopic time and length scales.

Model Applications

Two model space science calculations were developed for the REE testbed and environment. These two applications are Radio Astronomical Imaging (RAI) and Plasma Moment analysis (PMA). RAI works with a demanding signal processing problem that arises in plasma wave analysis and synthetic aperture imaging. PMA was developed to study the analysis required by plasma particle detector data that provides important information about the plasma state in the vicinity of the spacecraft.

Model Applications: RAI

REE/STP implemented the Radio Astronomical Imaging application (RAI) as an example of the use of correlative interferometry. Correlative interferometry is the practice of combining wave measurements obtained at different spatial locations to obtain information about the spatial and spectral structure of those waves. Interferometers designed to respond to radio frequency plane waves along a given direction form the basis for Radio Astronomical Imagers such as the Very Large Array. Only limited portions of the spatial Fourier space are measured, which leads to inadequately constrained images when these measurements are converted back to configuration (real or sky) space images (figure 2). For the space science application of RAI we consider, the Solar Imaging Radio Array, a large number of spacecraft are required to generate acceptable images in real-time with a time scale relevant to solar radio burst evolution (cf K. Weiler, Low Frequency Radio Astronomy From Space. For situations where the waves are not simple plane waves, interferometers are invaluable for constructing spectra and dispersion relatia. These analyses aid the identification of waves, their sources, and provide important information about the physics of the waves.

Radio snapshot point spread function for 25 spacecraft array. Radio snapshot point spread function for 97 spacecraft array.

Figure 2. Left: RAI model for a point source observed with 25 spacecraft. Right: RAI model for a point source observed with 97 spacecraft.

Details of the RAI application include:

Model Applications: PMA

REE/STP implemented the Plasma Moment Application (PMA) as an example of the moment based analysis of plasma spectrometry (figure 3). Particle spectrometers measure the number of particles that enter the instrument from a given spatial direction within a given velocity range. From many of these measurements a detailed picture of the particle distribution of the surrounding space plasma can be generated. In fact, the particle velocity distributions produced this way are much too detailed for many scientific purposes. For these analyses, a key data reduction is to form statistical moments of the particle distributions. Moments that have physical significance include plasma density, mass and heat flow, and pressure. Other moments retain information about particle distribution structure including anisotropies due to particle beaming, ring or fan distributions, magnetic fields, and particle-shock interactions.

PMA model showing irregular tetrahedral gridding of velocity space.

Figure 3. Right: The PMA model showing the use of an irregular tetrahedral grid which fits the underlying measurement scheme better than commonly used cube-based grids.

Details of the PMA application include:

Part of REE/STP development involved porting PMA from the parallel linear algebra package ScaLAPACK to PLAPACK as PLAPACK became the preferred library on REE machines.

Studies of High Performance Computing on Magnetospheric Multi Scale

A top-level analysis of MMS was performed to understand how REE might be of benefit. MMS science focuses on the small scale plasma physics that defines the roles of magnetic reconnection, particle acceleration, and turbulence. Very imprecise is our advance knowledge of regions where small scale phenomena are important. The REE enabled capability to perform some data reductions on orbit to produce high quality summary parameters was considered. Possibilities for on-board data interpretation, feature recognition, and data selection were also considered within the context of MMS as opening a path towards more autonomous subsystem or instrument autonomy.

In addition to top level studies, gaining experience and confidence working within realistic environments and working with real data are important for demonstrating technological readiness. To move towards realistic analogs of the MMS instrument data stream, we sought to incorporate archived data from a relevant mission. The HAWKEYE (1974-1978) mission featured a large, polar orbit that sampled nearly all regions important for the MMS mission (figures 4 and 5). Furthermore, HAWKEYE's coverage of Geospace is essentially unique and therefore analyses of its data are still scientifically important. The particle data from the LEPEDEA instrument is two-dimensional and therefore somewhat limited compared to newer instruments, but two-dimensions and the relatively low data production rate of the instrument ease aspects of our software development and analysis. Once we have gained experience with HAWKEYE data we plan to move to more complicated, larger, three dimensional phase-space data.

Line drawing of HAWKEYE spacecraft.

Figure 4. HAWKEYE 1974 - 1978.

HAWKEYE drawings from the National Space Science Data Center.

Hawkeye encountering magnetic reconnection region.

Figure 5. HAWKEYE orbit and possible encounter with magnetic reconnection region.

To analyze HAWKEYE data, we implemented functions to import the data into the PMA application. Once the data are within PMA, moments are calculated and a model fit is performed so that we can obtain performance measurements with this data set. We also can manipulate the data set so that it resembles more the MMS particle data sets: in this way we can obtain more reasonable estimates on the computing requirements of MMS particle instrumentation. Finally, archived data sets spanning years such as HAWKEYE provide an excellent opportunity to study and hone techniques of automated data reduction and analysis. In addition to the natural signatures that are HAWKEYE's main object, the data show all of the blemishes that one comes to expect from such instrumentation: dropouts, sun pulses, radiation belt contamination, etc. We aimed to develop techniques to first spot signatures of good data and worse, and then to develop techniques to either capture, modify, or discard data as appropriate.

Finally, the level of on board computation that REE enables opens up possibilities for a variety of levels of automation and autonomy. To better understand the possibilities and tradeoffs in the context of multi platform space science missions, we have entered into collaborations with researchers in the fields of spacecraft science instrumentation, autonomous systems, and various spacecraft subsystem specialties.

ACCOMPLISHMENTS

REE/STP reviewed three existing space science applications: (1) Plasma Moment Analysis (PMA), (2) Radio Astronomical Imaging (RAI), and (3) Magnetometry Analysis (MA). These three applications touch on themes important for space science. All three applications involve aspects of in situ measurement of particles or electromagnetic fields. PMA and RAI touch on remote sensing applications as well, for example, in neutral atom imagers and solar or deep space radio imaging.

We obtained data analysis software that provided the capabilities we wish to use on orbit. In particular, we examined the AIPS++ Astronomical Image Processing System from National Radio Astronomical Observatory and plasma diagnostic software developed to analyze spacecraft observations for the International Solar Terrestrial Physics program. We assessed these existing applications to determine their suitability for use as prototypical onboard data analysis software. Though each program has strengths and weaknesses, neither package was written for efficient or parallel computer use. Because our prototypes are to be used for benchmarking and testing the prototype REE Flight Processor, we reimplemented the key and limiting functionality of these packages in more efficient and portable forms.

Model Science Applications: RAI and PMA

For PMA, three dimensional density functions need to be fitted to plasma models and reduced to plasma (fluid) moments. For RAI, the most expensive computations involve the cross correlation of radio signals from multiple antennae/receiver elements. MA, which deals with magnetic field data, provides important information that is definitely useful for onboard calibration and data analysis and is not computationally expensive to analyze. Therefore we developed two software applications that illustrated critical areas of performance for PMA and RAI.

Development accomplishments include:

Performance accomplishments: PMA

Efficiency of PMA. Scalability of PMA. Speed up of PMA.

Figure 6. Performance measurements for PMA on multiple processors.

Performance accomplishments: RAI

Speed up of RAI.

Figure 7. Performance measurements showing speed up of RAI.

MMS Science Enhancement Research

A strategy for enhancing MMS science return with REE technology was developed and documented. Development along a path of steps of increasing difficulty and reward was proposed. The first step on the path consists of the return of summary statistics, including plasma moments, of the highest resolution data available on board to a proposed REE High Performance Computing System (HPCS). The second step involves a basic ability to select important data, but mainly centers on the aggressive compression of kinetic or small scale plasma data. The third, and most ambitious step, involves the implementation a phenomena-recognition function that instructs the operation of an autonomous science manager that resides within the HPCS. The science manager seeks to achieve goals provided by the ground to minimize upload use and optimize the value of the data downlinked by the HPCS. This strategy was presented at the 51st International Astronautical Conference of the International Astronautical Federation, 2000 and subsequently at Meetings of the American Geophysical Union.

MMS Science Enhancment Accomplishments

MMS Analyses using real data

To improve the technological readiness of REE/STP elements applicable to MMS, we moved to study archived data from the HAWKEYE mission. The entire HAWKEYE data set was obtained, and functions allowing import of HAWKEYE science and engineering data into PMA were implemented (figure 8). Plasma moments were calculated from HAWKEYE data and results were verified by comparison with an existing HAWKEYE analysis code. In parallel with this initial importation into PMA, a technique for identifying plasma suffering various amounts entropy production within Geospace was studied in collaboration with researchers at the University of California, Berkeley. Thus we first applied PMA to real data, and implemented our first Geospace plasma diagnostic. For the 2000 Spring Meeting of the American Geophysical Union, we applied the entropy-based plasma diagnostic to data from a possible site of magnetic reconnection (figure 9); such sites are of prime importance to MMS.

Figure 8. HAWKEYE LEPEDEA Plasma Phase-Space (Velocity) Distributions showing oppositely directed flow as expected in the vicinity of sites of magnetic reconnection (see schematic in figure 5 above). PMA's numerical grid is visible as the triangulation in these images.

Plot of log-density vs. specific entropy.

Figure 9. A plot of the plasma density (vertical) vs. the plasma specific entropy (horizontal) in the vicinity of a possible site of magnetic reconnection. This graph shows a possible diagnostic for discriminating between plasma with different histories. The magnetosheath plasma which is the shocked solar wind is seen in the low entropy measurements on the left of the graph. The higher entropy boundary layer plasma is to the right.

Electromagnetic field data reduction

In addition to the entropy-based diagnostic applied to particle data described above, we felt that techniques for plasma wave data reduction and analysis should be studied as well. Therefore, in collaboration with researchers from the Radio Plasma Imager (RPI) project we studied techniques for detecting features in radio receiver data (figure 10). RPI has flown a radar on the IMAGE spacecraft, launched in March 2000. RPI is perhaps the most sensitive radio receiver yet flown in space, and provides an excellent view of stimulated and natural radio emissions in the Earth's magnetosphere. Our work consisted of two parts: (1) an ad hoc part that identifies and labels high signal-to-noise ratio echoes and (2) a likelihood method based within a Bayesian framework that searches for evidence of echoes within very noisy data. This work was performed with model data and completed before IMAGE's launch.

Detection of an echo in noisy data.

Figure 10. Detection of a weak radar echo in a noisy background. The triangles mark the identification of a high signal-to-noise-ratio echo, whereas the box shows a position where evidence for a weak signal exists between the box and triangle along the black line. The black line represents a reasonably likely detection of a radar echo. Radio frequencies (horizontal) are in kHz and ranges (vertical) are in Earth radii. Red-to-blue coloring denotes descending received power.

Autonomous subsystem operation and Science data processing

Numerical computation is only one aspect of constructing reliable scientific results from raw sensor data. Other aspects include the management of sensor data processing and of the sensor itself. By management we mean the choice of modes and processes that affect data, sensor hardware, software, and science processing software during operations. In many operational scenarios where REE would be put to use, data processing must be performed without direct ground-based command and control. Autonomy then becomes necessary in cases where faults, transient opportunities to obtain science, or methods of non-trivial data processing are required. Accomplishments involving science data reduction have been described above, below we discuss faults, automation and parallelization of data reduction, region identification, and subsystem autonomy.

REE/STP studied the effect of faults on the PMA application. The main finding of this study is that random, single-bit corruption to data and code affect the results of the analysis only ~10% of the time. Of that 10%, most lead to program crashes or hangs. This is due to the statistical reductions that are being performed on the data, the use of floating point by PMA, and the inherent variability of the data. Most single-bit faults to program code and data lead to acceptable results or no result. The effect of faults occuring in system code or in logic, registers, or other hardware-based elements, was not studied.

Also developed were a set of programs and shell-scripts to automate the analysis of HAWKEYE particle detector data. These routines were fairly modular and could be used to drive similar data processing on computer clusters with little modification. Here PMA was used to calculate moments and to fit a model plasma distribution to the data. Special routines were constructed to retrieve, distribute, read, check, and format HAWKEYE data. Part of this process involved the automated generation of scripts that partitioned data sets and ran the PMA code. Good speedup was obtained, with fifty hours of spacecraft data analyzed in 1/2 an hour on 25 nodes of a computer cluster. Most of the run-time was spent extracting and reformatting HAWKEYE data, a task which was also run in parallel.

A rule based filter that determines the region of Geospace a spacecraft inhabits was prototyped (figure 11). The regions that the identifier can currently identify are the magnetosphere, the magnetosheath, and the solar wind. This region identifier provides location information to the onboard science analysis control software to determine the importance of any particular set of measurements. Particle detector and magnetometer data are among the science instruments used, along with supporting engineering data. Future development of this identifier will include functions to detect sun-pulse or other interference as well as the extension of this identifier to high-bandwidth electromagnetic field data. These recognition functions are an important part of autonomous data analysis.

Figure 11. Results of REE/STP magnetospheric region identifier. White: magnetosphere; Blue: magnetosheath; Green: solar wind.

Moving from science data processing to instrument control, REE/STP collaborated with GSFC Nano-satellite researchers to outline an approach to realizing electrical power subsystem autonomy within the stringent limits of the nano-satellite resource envelope (Johnson et al, 1999). Though mainly concerned with hardware and some hybrid hardware/software approaches to making undesirable states physically inaccessible to the subsystem, an important component of that study centered on the use of onboard intelligence, e.g. implemented onboard a high performance computer, to spot trends and adapt subsystem operation to maximize performance or lifetime. This effort led to continued research on how to reliably and safely control space science instruments, in particular, those with high voltage power supplies (Johnson et al, 2001a,b). This work has led to an ongoing separate effort to develop COTS-based software tools to enable instrument and subsystem developers to construct instrument and process models for use in implementing autonomous control systems (Bailin et al, 2002). These prototypical software tools provide model editing and checking functions, and are to provide code generation facilities that implement functions important for autonomous control.

SIGNIFICANCE

Model Applications

The significance of our accomplishments with the model science applications is that we have demonstrated that computations exist within the realm of space science data analysis that can make use of parallel computation. Furthermore, to obtain reduced data in real-time requires advanced computational speed and algorithms that can monitor the quality of the produced data products. As an aside, we note that for many mission hard real-time may not be required, but requirements on average throughput still point to advanced computation. Missions that involve radiowave interferometry for aperture synthesis or "snapshot" imaging are enabled by high performance on-board computing. Furthermore, we feel that advanced computation on-board enables more flexible, more ambitious particle instrumentation that have greater sensitivity and higher resolution than is currently practical. On-board data reduction has the potential to provide effective reductions in communication data volumes measured by factors of thousands, before standard compression techniques are applied. As our confidence in our ability to perform on-board data reduction and selection improves, we will be able to reap the benefits of these aggressive on-board data reductions and selections.

Magnetospheric Multi Scale Applicability Studies

Study into the application of REE technology to MMS suggests a significant enhancement to the possible Science return is possible. A minimal scientific enhancement made possible by REE is the calculation of plasma moments from the highest resolution data available on board the spacecraft. This would provide 100% coverage of the entire orbit with high time resolution, higher order moments, and would only cost less than an average of 100 Megabits per day for communication downlink (mission total for four spacecraft scenario). Because the highest resolution, "burst mode", possible on board the MMS baseline is about a factor of seven times greater than "normal mode" resolution, high quality moments computed on board would compare to a mission with seven times the data production capacity of MMS. Currently, MMS is baselined at a mission downlink capacity of an average of 8 Gigabits per day, therefore with REE on board, one may obtain 100 Mb/day of moments generated from 56 Gb/day of data. Standard compression techniques will further reduce the 100 Mb/day downlink cost. The quality of these moments should approach that achievable via post-processing on the ground.

The implementation of a basic plasma diagnostic, the radiowave analysis code and the entropy-based plasma diagnostic mentioned above, point out that we may soon be able to have software systems on orbit that can soon distinguish between important and unimportant features in the data. This capability is the beginning of a data selection function that may allow a science manager operating on the HPCS to sift high resolution data on board MMS for nuggets of science opportunities. Note that HPCS can flag only a small fraction of the high resolution data for downlink via its small communication downlink budget. However, the HPCS will return high resolution data that would otherwise be discarded by as a matter of routine by MMS. The quality of this data is limited only by the performance of the data selection schemes we may implement.

Finally, we began work with archived data that is relevant to MMS mission goals. This is a key step along the way to flight qualification and proving that an REE based system can add value to the downlink stream.

STATUS

At the time of this writing development of the PMA and RAI applications have been frozen and the software has been archived. Having already shown that space science data processing provides ample opportunities for parallel processing, at the conclusion of REE, our focus was principally on the elaboration of MMS relevant data processing. Our aim was to develop our techniques to the point where they might readily be applied to MMS instrumentation. Work was progressing at several levels and reached points described by the following. Some of the work we pursued was not novel from the point of view of space science data analysis. However, attempting to develop a framework in which science products involving non-trivial onboard computation is novel. By non-trivial we mean computations involving recursive or iterative algorithms that often involve an assessment step that determines the quality of a final or intermediate data product. Most of our work, and that of previous researchers, has involved what amount to filters. These filters transform data, producing results that are reviewed by a human data analyst who varies the transformation as needed to obtain desirable results. At the project close, we were implementing functions for assessing data, data products, and their importance and quality. These assessment functions, coupled with a rule base for driving management actions were to form the foundation of an autonomous science management system.

REFERENCES

Papers

Conference Presentations

Visuals

POINT OF CONTACT

Dr. S. A. Curtis, NASA/GSFC Code 695 Greenbelt MD, 20771.

Dr. M. L. Rilee, Emergent-IT NASA/GSFC Code 931 Greenbelt MD, 20771.

Dr. S. A. Boardsen, Dr. M. K. Bhat, RAYTHEON and EMERGENT IT