[GEM] THE GEM MESSENGER, Volume 28, Number 40

Newsletter Editor editor at igpp.ucla.edu
Thu Aug 16 18:32:58 PDT 2018


***************************
     THE GEM MESSENGER
***************************

Volume 28, Number 40
Aug.16,2018

Announcement submission website: http://aten.igpp.ucla.edu/gem/messenger_form/

============================================================
Table of Contents

1. GEM Modeling Methods and Validation Focus Group: 2018 Workshop Report

============================================================

------------------------------------------------------------
1. GEM Modeling Methods and Validation Focus Group: 2018 Workshop Report
------------------------------------------------------------
From: Katherine Garcia-Sage, Mike Liemohn, Lutz Rastaetter, and Rob Redmon (katherine.garcia-sage at nasa.gov)

The Modeling Methods and Validation Focus Group held two individual sessions at the 2018 summer workshop, as well as joint sessions with the ULF Wave Modeling and Dayside Kinetics focus groups.

On Tuesday afternoon, we discussed the Dayside Challenges (2015 November 18 southward IMF event and 2015 December 25 northward IMF event) in two joint sessions with the Dayside Kinetics focus group. The report for these sessions is issued by Dayside Kinetics.

On Wednesday morning, we discussed the ULF Wave Challenge (2017 May 27 storm event) in a joint session with the ULF Wave Modeling, Effects, and Applications (UMEA) focus group. The report for this session is issued by UMEA. 

On Wednesday afternoon, we held a general validation session with 6 talks, followed by a presentation and panel discussion on the role and future of validation at GEM.

Mike Liemohn presented a talk on metrics for modeling geomagnetic indices. He presented progress of the iCCMC-LWS team that has been tasked with developing standard metrics to be used for validation of modeled geomagnetic indices.

Adam Kellerman presented a talk on "Application Usability Levels: A framework for objectively measuring a projects progress towards specific applications." The talk focused on the metrics developed by the iCCMC-LWS team for Assessment of Understanding and Quantifying Progress Toward Science Understanding and Operational Readiness. He presented Application Usability Levels (AULs) as a method for tracking progress of a particular project from basic research to continuous operational use.

Adam Kellerman also presented a talk on "Verification of the UCLA real-time data assimilative VERB code over the 2016-2018 period of operation."

Lutz Rastaetter presented new visualizations and metrics that are under development by the CCMC. These new tools will enhance the ability of CCMC users to analyze magnetospheric model results.

Misha Sitnov presented “Global MHD validation of substorms and implications for kinetic simulations.” He showed how statistical magnetospheric data can be used to test the ability of global MHD to correctly model magnetospheric configuration during substorms.

Nathaniel Frissell presented "Modeling Amateur Radio Soundings of the Ionospheric Response to the 2017 Great American Eclipse." He showed a citizen science effort during the eclipse that was used to validate ionospheric models and their response to the solar eclipse.

Katherine Garcia-Sage presented slides on “GEM Validation - Issues and Ideas,” followed by a panel discussion on how to enhance model validation efforts at GEM.

Panel Participants: H Singer, H. Hietala, A. Kellerman, M. Liemohn

The panel discussed the role and importance of validation, emphasizing the importance of asking the right questions. They highlighted that models reflect current knowledge, and so validation is needed to see if we capture observed phenomena. They pointed out that end users rely on models for interpretation, so there is a need for connecting with end users. Users need to be confident in models through validation, which requires standardization of tests and metrics. SWPC operations require confidence levels.

The discussion highlighted the need to consider the full array of models for the prediction and validation. The models with best validation may not be scientific. Neural net or heavy computation vs. knowledge and physics-based models to get efficient prediction. For the purposes of validation, consider system science - ways to organize observations. e.g., super index of magnetosphere state combining 11 indices.

Next the panel and other participants in the session discussed ideas for how to implement improvements in GEM validation. H. Hietala talked about having a validation steering committee member, as well as a tutorial talk dedicated to challenge status updates. H. Singer challenged the premise of a validation focus group not being science oriented - current validation efforts are science oriented. He pointed to synergy between MMV and CCMC. MMV may not have expertise to reach out to all groups. H. Hietala emphasized that the dayside validation study benefited from MMV expertise.

Several possibilities were discussed, including keeping the focus group as it currently is. H. Singer pointed out the usefulness for MMV studies to serve as an example. S. Morley stated that validation as a separate activity may have value, but an alternative is for every focus group to have a MMV plan and convener/liaison person responsible. A. Glocer pointed out that Focus groups may exist that don’t advance models. The validation group is an outlet for studies that does not fit in current science groups.

There was discussion on MMV’s role as a focus group to support other groups validation and metrics studies. K. Garcia-Sage discussed the possibility of having an overarching coordinator (focus group is currently doing this to some extent) - matchmaking between other groups. The was discussion on the idea of a SC member supporting validation activities. H Hietala pointed out a need to keep records of what was successful from past challenges. K. Garcia-Sage suggested a coordinator who can take bird’s eye view, pass on knowledge to a staggered co-coordinator, and advocate validation needs in the Steering Committee. M. Liemohn pointed out area coordinators each cover a wider area than validation. Validation coordinators would provide dedicated voice in SC.

P. Cassak pointed out that best practices are already on the GEM-Wiki, so a coordinator may not be needed. K. Garcia-Sage the focus group structure means validation efforts last 5 years and then time is up, relying on the hope that someone else steps up. An SC member would shows the dedication of GEM community to keep maintaining validation best practices and having a dedicated point of contact. P. Cassak had a concern that SC members are usually not benefitting from their role as members of the SC. There may be concern that SC members would be contributing to an effort that they would personally benefit from. H. Singer suggested that proposals from focus groups should include validation deliverables. C. Gabrielse suggested a proposal for MMV to be permanent and have a mechanism to rotate leadership in that group. MMV positions advertised by SC.

On Thursday afternoon we held a session on the Conductance Challenge. Events up for discussion were:
2016 Oct 13-15
2013 March 17
2015 Jun 21-24
2012 March 9

Steve Kaeppler had a talk on “Poker Flat Incoherent Scatter Radar Observations of Conductance” (presented by K. Garcia-Sage). He showed E region conductances from PFISR, which makes nearly continuous observations of E and F region plasma. He showed limitations of the data, including the localized nature of the observations, so conductance enhancements driven by localized phenomena (e.g. discrete auroral arcs) are only locally valid. Average conductivity may be useful. Ground truth data is available for all 4 events. March 2013 Event was shown, with two different calculations of ion temperature that produce results in good agreement with each other.

George Khazanov presented “Contributions of M-I Coupling Processes to Electron Precipitation and Ionospheric Conductivity.” He showed that with a primary electron precipitation flux from ECH Whistler Waves, some particles are reflected back off the ionosphere and pass through waves in the tail and then precipitate into the conjugate hemisphere. His calculations account for multiple reflections. He showed results from the October 2016 storm from LFM conductances with multiple reflections and without (i.e. Robinson formula only), resulting in a factor of 2 difference in the Pedersen conductivity.

Bob Robinson had a talk on “Auroral Precipitation: AuroraPHILE team” (presented by K. Garcia-Sage) The goal of the team is to establish quantitative means to measure the accuracy and reliability of modeled properties including precipitation, conductivity, E-fields, neutral winds, currents and Joule heating. For the following events: 9 March 2012, 17 March 2013, 21-24 June 2015, gather all available data and develop best estimates of ground truth from data. Next step will be to run models and develop a skill score for each parameter based on comparison to ground truth. He showed AMPERE, AMIE and SuperDARN for the March 2013 event.

Margaret Chen presented a talk on RCM-E (Aerospace Version). The conductance model included EUV conductance from IRI 2007, precipitating electron contribution from Robinson, and precipitating ions contribution from Galand and Richmond. Ion precipitation due to Field Line Curvature (FLC) scattering (Schulz) occurs where you get stretching of the field lines. The contribution of precipitating ions from FLC to total conductivity is very small and localized - possibly important locally, not globally.
Conductance low at night pre-storm (as expected), increasing as storm develops. Comparison to PFISR is not good during the main part of storm (possibly due to discrete aurora), but better during recovery. Future steps include improving the calculation of conductance using B3c auroral transport model (Strickland et al., 1993).

Agnit Mukhopadhyay presented “Can accurate conductivity lead to better predictions of dB/dt during extreme events?” The SWMF conductance calculations were originally developed against limited coverage in FAC and were missing extreme events (i.e. only 1 month of AMIE in 1997. This led to underprediction of nightside conductance and overprediction of dayside conductance. A new Empirical Conductance Model (ECM2018) is fitted to 1 year (2003) of AMIE. This model includes extreme events, but no precipitating physics is currently included. Preliminary MHD results for SWPC Event 6: May 8, 2011 and SWPC Events 3, 5, 6 show dB/dt skill scores have improved.


========================================
The Geospace Environment Modeling (GEM) program is sponsored by the Division of Atmospheric and Geospace Sciences (AGS) of the National Science Foundation (NSF).

To broadcast announcements to the GEM community, please fill out the online request form at:

http://aten.igpp.ucla.edu/gem/messenger_form/

To subscribe to the newsletter, please go to the web page at:
http://lists.igpp.ucla.edu/mailman/listinfo/gem
(IMPORTANT: Do not use this web page to post announcements.)

For any other questions, please contact Peter Chi, GEM Communications Coordinator, at <pchi at igpp.ucla.edu>

URL of GEM Home Page:  http://aten.igpp.ucla.edu/gemwiki
Workshop Information:  http://www.cpe.vt.edu/gem/index.html
========================================



More information about the Gem mailing list