CEDAR-GEM Challenge for Systematic Assessment of Ionosphere/Thermosphere Models in Predicting TEC During the 2006 December Storm Event

J. S. Shim, L. Rastätter, M. Kuznetsova, D. Bilitza, M. Codrescu, A. J. Coster, B. A. Emery, M. Fedrizzi, M. Förster, T. J. Fuller-Rowell, L. C. Gardner, Larisa Petrovna Goncharenko, J. Huba, S. E. McDonald, A. J. Mannucci, A. A. Namgaladze, X. Pi, B. E. Prokhorov, A. J. Ridley, L. ScherliessR. W. Schunk, J. J. Sojka, L. Zhu

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

In order to assess current modeling capability of reproducing storm impacts on total electron content (TEC), we considered quantities such as TEC, TEC changes compared to quiet time values, and the maximum value of the TEC and TEC changes during a storm. We compared the quantities obtained from ionospheric models against ground-based GPS TEC measurements during the 2006 AGU storm event (14–15 December 2006) in the selected eight longitude sectors. We used 15 simulations obtained from eight ionospheric models, including empirical, physics-based, coupled ionosphere-thermosphere, and data assimilation models. To quantitatively evaluate performance of the models in TEC prediction during the storm, we calculated skill scores such as RMS error, Normalized RMS error (NRMSE), ratio of the modeled to observed maximum increase (Yield), and the difference between the modeled peak time and observed peak time. Furthermore, to investigate latitudinal dependence of the performance of the models, the skill scores were calculated for five latitude regions. Our study shows that RMSE of TEC and TEC changes of the model simulations range from about 3 TECU (total electron content unit, 1 TECU = 1016 el m−2) (in high latitudes) to about 13 TECU (in low latitudes), which is larger than latitudinal average GPS TEC error of about 2 TECU. Most model simulations predict TEC better than TEC changes in terms of NRMSE and the difference in peak time, while the opposite holds true in terms of Yield. Model performance strongly depends on the quantities considered, the type of metrics used, and the latitude considered.

Original languageEnglish
Pages (from-to)1238-1256
Number of pages19
JournalSpace Weather
Volume15
Issue number10
DOIs
Publication statusPublished - Oct 1 2017

Fingerprint

thermosphere
ionosphere
total electron content
GPS
simulation
data assimilation
physics

All Science Journal Classification (ASJC) codes

  • Atmospheric Science

Cite this

Shim, J. S., Rastätter, L., Kuznetsova, M., Bilitza, D., Codrescu, M., Coster, A. J., ... Zhu, L. (2017). CEDAR-GEM Challenge for Systematic Assessment of Ionosphere/Thermosphere Models in Predicting TEC During the 2006 December Storm Event. Space Weather, 15(10), 1238-1256. https://doi.org/10.1002/2017SW001649

CEDAR-GEM Challenge for Systematic Assessment of Ionosphere/Thermosphere Models in Predicting TEC During the 2006 December Storm Event. / Shim, J. S.; Rastätter, L.; Kuznetsova, M.; Bilitza, D.; Codrescu, M.; Coster, A. J.; Emery, B. A.; Fedrizzi, M.; Förster, M.; Fuller-Rowell, T. J.; Gardner, L. C.; Goncharenko, Larisa Petrovna; Huba, J.; McDonald, S. E.; Mannucci, A. J.; Namgaladze, A. A.; Pi, X.; Prokhorov, B. E.; Ridley, A. J.; Scherliess, L.; Schunk, R. W.; Sojka, J. J.; Zhu, L.

In: Space Weather, Vol. 15, No. 10, 01.10.2017, p. 1238-1256.

Research output: Contribution to journalArticle

Shim, JS, Rastätter, L, Kuznetsova, M, Bilitza, D, Codrescu, M, Coster, AJ, Emery, BA, Fedrizzi, M, Förster, M, Fuller-Rowell, TJ, Gardner, LC, Goncharenko, LP, Huba, J, McDonald, SE, Mannucci, AJ, Namgaladze, AA, Pi, X, Prokhorov, BE, Ridley, AJ, Scherliess, L, Schunk, RW, Sojka, JJ & Zhu, L 2017, 'CEDAR-GEM Challenge for Systematic Assessment of Ionosphere/Thermosphere Models in Predicting TEC During the 2006 December Storm Event', Space Weather, vol. 15, no. 10, pp. 1238-1256. https://doi.org/10.1002/2017SW001649
Shim, J. S. ; Rastätter, L. ; Kuznetsova, M. ; Bilitza, D. ; Codrescu, M. ; Coster, A. J. ; Emery, B. A. ; Fedrizzi, M. ; Förster, M. ; Fuller-Rowell, T. J. ; Gardner, L. C. ; Goncharenko, Larisa Petrovna ; Huba, J. ; McDonald, S. E. ; Mannucci, A. J. ; Namgaladze, A. A. ; Pi, X. ; Prokhorov, B. E. ; Ridley, A. J. ; Scherliess, L. ; Schunk, R. W. ; Sojka, J. J. ; Zhu, L. / CEDAR-GEM Challenge for Systematic Assessment of Ionosphere/Thermosphere Models in Predicting TEC During the 2006 December Storm Event. In: Space Weather. 2017 ; Vol. 15, No. 10. pp. 1238-1256.
@article{6aec59102dae417cb0f5090ae601d929,
title = "CEDAR-GEM Challenge for Systematic Assessment of Ionosphere/Thermosphere Models in Predicting TEC During the 2006 December Storm Event",
abstract = "In order to assess current modeling capability of reproducing storm impacts on total electron content (TEC), we considered quantities such as TEC, TEC changes compared to quiet time values, and the maximum value of the TEC and TEC changes during a storm. We compared the quantities obtained from ionospheric models against ground-based GPS TEC measurements during the 2006 AGU storm event (14–15 December 2006) in the selected eight longitude sectors. We used 15 simulations obtained from eight ionospheric models, including empirical, physics-based, coupled ionosphere-thermosphere, and data assimilation models. To quantitatively evaluate performance of the models in TEC prediction during the storm, we calculated skill scores such as RMS error, Normalized RMS error (NRMSE), ratio of the modeled to observed maximum increase (Yield), and the difference between the modeled peak time and observed peak time. Furthermore, to investigate latitudinal dependence of the performance of the models, the skill scores were calculated for five latitude regions. Our study shows that RMSE of TEC and TEC changes of the model simulations range from about 3 TECU (total electron content unit, 1 TECU = 1016 el m−2) (in high latitudes) to about 13 TECU (in low latitudes), which is larger than latitudinal average GPS TEC error of about 2 TECU. Most model simulations predict TEC better than TEC changes in terms of NRMSE and the difference in peak time, while the opposite holds true in terms of Yield. Model performance strongly depends on the quantities considered, the type of metrics used, and the latitude considered.",
author = "Shim, {J. S.} and L. Rast{\"a}tter and M. Kuznetsova and D. Bilitza and M. Codrescu and Coster, {A. J.} and Emery, {B. A.} and M. Fedrizzi and M. F{\"o}rster and Fuller-Rowell, {T. J.} and Gardner, {L. C.} and Goncharenko, {Larisa Petrovna} and J. Huba and McDonald, {S. E.} and Mannucci, {A. J.} and Namgaladze, {A. A.} and X. Pi and Prokhorov, {B. E.} and Ridley, {A. J.} and L. Scherliess and Schunk, {R. W.} and Sojka, {J. J.} and L. Zhu",
year = "2017",
month = "10",
day = "1",
doi = "10.1002/2017SW001649",
language = "English",
volume = "15",
pages = "1238--1256",
journal = "Space Weather",
issn = "1542-7390",
publisher = "American Geophysical Union",
number = "10",

}

TY - JOUR

T1 - CEDAR-GEM Challenge for Systematic Assessment of Ionosphere/Thermosphere Models in Predicting TEC During the 2006 December Storm Event

AU - Shim, J. S.

AU - Rastätter, L.

AU - Kuznetsova, M.

AU - Bilitza, D.

AU - Codrescu, M.

AU - Coster, A. J.

AU - Emery, B. A.

AU - Fedrizzi, M.

AU - Förster, M.

AU - Fuller-Rowell, T. J.

AU - Gardner, L. C.

AU - Goncharenko, Larisa Petrovna

AU - Huba, J.

AU - McDonald, S. E.

AU - Mannucci, A. J.

AU - Namgaladze, A. A.

AU - Pi, X.

AU - Prokhorov, B. E.

AU - Ridley, A. J.

AU - Scherliess, L.

AU - Schunk, R. W.

AU - Sojka, J. J.

AU - Zhu, L.

PY - 2017/10/1

Y1 - 2017/10/1

N2 - In order to assess current modeling capability of reproducing storm impacts on total electron content (TEC), we considered quantities such as TEC, TEC changes compared to quiet time values, and the maximum value of the TEC and TEC changes during a storm. We compared the quantities obtained from ionospheric models against ground-based GPS TEC measurements during the 2006 AGU storm event (14–15 December 2006) in the selected eight longitude sectors. We used 15 simulations obtained from eight ionospheric models, including empirical, physics-based, coupled ionosphere-thermosphere, and data assimilation models. To quantitatively evaluate performance of the models in TEC prediction during the storm, we calculated skill scores such as RMS error, Normalized RMS error (NRMSE), ratio of the modeled to observed maximum increase (Yield), and the difference between the modeled peak time and observed peak time. Furthermore, to investigate latitudinal dependence of the performance of the models, the skill scores were calculated for five latitude regions. Our study shows that RMSE of TEC and TEC changes of the model simulations range from about 3 TECU (total electron content unit, 1 TECU = 1016 el m−2) (in high latitudes) to about 13 TECU (in low latitudes), which is larger than latitudinal average GPS TEC error of about 2 TECU. Most model simulations predict TEC better than TEC changes in terms of NRMSE and the difference in peak time, while the opposite holds true in terms of Yield. Model performance strongly depends on the quantities considered, the type of metrics used, and the latitude considered.

AB - In order to assess current modeling capability of reproducing storm impacts on total electron content (TEC), we considered quantities such as TEC, TEC changes compared to quiet time values, and the maximum value of the TEC and TEC changes during a storm. We compared the quantities obtained from ionospheric models against ground-based GPS TEC measurements during the 2006 AGU storm event (14–15 December 2006) in the selected eight longitude sectors. We used 15 simulations obtained from eight ionospheric models, including empirical, physics-based, coupled ionosphere-thermosphere, and data assimilation models. To quantitatively evaluate performance of the models in TEC prediction during the storm, we calculated skill scores such as RMS error, Normalized RMS error (NRMSE), ratio of the modeled to observed maximum increase (Yield), and the difference between the modeled peak time and observed peak time. Furthermore, to investigate latitudinal dependence of the performance of the models, the skill scores were calculated for five latitude regions. Our study shows that RMSE of TEC and TEC changes of the model simulations range from about 3 TECU (total electron content unit, 1 TECU = 1016 el m−2) (in high latitudes) to about 13 TECU (in low latitudes), which is larger than latitudinal average GPS TEC error of about 2 TECU. Most model simulations predict TEC better than TEC changes in terms of NRMSE and the difference in peak time, while the opposite holds true in terms of Yield. Model performance strongly depends on the quantities considered, the type of metrics used, and the latitude considered.

UR - http://www.scopus.com/inward/record.url?scp=85034425777&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85034425777&partnerID=8YFLogxK

U2 - 10.1002/2017SW001649

DO - 10.1002/2017SW001649

M3 - Article

AN - SCOPUS:85034425777

VL - 15

SP - 1238

EP - 1256

JO - Space Weather

JF - Space Weather

SN - 1542-7390

IS - 10

ER -