2006-03-14: Ispra IT Meeting
From Datafedwiki
Backlinks from :2006-03-14: Ispra IT Meeting
[edit] Work Group Meeting on
IT Techniques, Tools and Philosophies for Model Intercomparison
IT Techniques, Tools and Philosophies for Model Intercomparison
[edit] Background
F. Dentener
There is an increasing number of model intercomparison studies
These cover different modeling scales (global, regional, local), pollutants (gases, aerosols, metals, radioactivity) and comparison methodologies
Intercomparison participants may not be familiar with similar work elsewhere
A common problem for all these studies is the intense use of information technologies for bringing together, homogenizing and comparing the models
[edit] Meeting Goals
- Present the major intercomparison studies with emphasis on IT issues
- Discuss IT issues and solutions, (e.g. common formats, conventions)
- Recommend actions for joint IT-related efforts
Attendees:
Frank Dentener, JRC, Ispra
Thunis/Cuvelier, JRC, Ispra
Michael Schulz, JRC, Ispra(?)
Stefan Kinne, Max Plank Institute, Hamburg
Charles Doutriaux,
Rudolf Husar, Washington University
Christiane Textor,
Stefano Galmarini, JRC, Ispra
Agenda:
09:00-09:15 Frank Dentener: Welcome/Background - (PPT)
09:15-10:00 Thunis/Cuvelier: The Euro-and City-Delta Model Intercomparison - (PPT)
10:00-10:45 Michael Schulz/Stefan Kinne: AeroCom – Aerosol Model Comparison - (PPT)
10:45-11:00 Coffee
11:00-11:45 Charles Doutriaux: Towards Automated Model Output Analysis - (PPT)
11:45-12:30 Rudolf Husar: Datared: Federated Data System for Air Quality - (PPT)
12:30-14:00 Lunch + discussion
14:00-14:30 Christiane Textor: Global Earth-system Monitoring using Space and in-situ data (GEMS) - (PPT)
14:30-15:15 Stefano Galmarini: ENSEMBLE: Reconciliation of Disparate National Medium/Long Range Dispersion Forecasts - (PPT)
15:15-15:30 Tea
15:30-17:00 Discussion+ summarizing recommendations
[edit] Meeting Report
Rudolf Husar
This brief meeting report is to inform the US HTAP program participants who could not attend the meeting. It contains the presentations and discussions from the meeting. A formal meeting report for broader distribution is being prepared by Frank Dentener and the IT group.
The meeting of the small group of (9) IT 'professionals' was well organized and accomplished its stated goals. It was an effective way to share ideas on IT issues related to model intercomparison studies. This report is intended to continue that communication process.
Below is a summary of workshop presentations. For more detail, the individual presentations can be accessed from the Agenda table. Each presentation summary contains subjective statements in three categories: General Comments, IT Issues and HTAP Relevance.
The last item is the outline of the proposed "Atmospheric Tracer Model Intercomparison Tools Initiative" by Michael Schulz. It was distributed but not discussed in detail.
The key recommendations by the IT Group the are still under review. However, one outstanding unanimous agreement among the participants was need to adopt the netCDF data formats that follows the CF convention. Specific proposed action items arising from that agreement are:
- Set up a small workgroup to study the usage and possible augmentation of netCDF CF protocol
- Use a set of test datasets to demonstrate the interoperability among the networked participants
- Have an interoperability demo working by this summer (?!)
These are clear candidate technical recommendations for consideration by broader HTAP Task Force. Notable was the focus of this ad hoc IT work group on model intercomparisons. Observation-based model evaluation was declared to be beyond the scope of this group activity.
[edit] Summary of Presentations
[edit] Past Model Intercomparison Studies
Frank Dentener
ACCENT-Photo Com
EuroDelta
AEROCOM
MICS-Asia
Mercury Transport
General Comments: After introductions and stating the meeting goals, Frank briefly described his initial catalog of model intercomparison studies. He intends to maintain an updated catalog of such studies.
IT Issues: In the ACCENT project there was a plan to use networked model data sharing, however they did not use the Storage Resource Broker (SRB). Instead they reverted to standard manual internet transfer. All the analyses were performed using standard procedural programs.
[edit] The Euro-and City-Delta Model Intercomparison
P. Thunis, K. Cuvelier
|
General Comments:
The presentation described the procedures and software tools used in Euro-Delta and City-Delta model evaluation and intercomparison studies.
IT Issues: For the study separate tools were developed for monitoring data, emissions, model-validation, mapping, etc. These tools are desktop programs that operate on specifically formatted observational and model datasets. From the discussion it was evident that there was no convenient way to use the data and the tools independently.
HTAP Relevance: Euro-Delta and City-Delta are focusing on European domain, addressing European issues and the IT system is designed specifically to support that goal. The broader applicability of the developed IT to HTAP appears to be limited.
[edit] AeroCom – Aerosol Model Comparison
C. Textor, S. Guibert, S. Kinne, J. Penner, M. Schulz, F. Dentener
|
General Comments: AeroCom is probably the most extensive effort sofar of intercomparing global chemical models (?!). The AeroCom project also includes many homogenized observational datasets (satellite & surface). Michael Schulz has focused his AeroCom presentation on the IT aspects of the project. Further on AeroCom.
IT Issues: The AeroCom model intercomparison was very labor intensive due to lack of automation. Each model dataset was reformatted and homogenized using a common structure and naming conventions. For future AeroCom studies, Michael proposed the development of tools and procedures that would automate many of the IT tasks. He is teaming up with Charles Doutriaux to develop the automated model diagnostic facility in the ESG/CDAT IT infrastructure framework (see below).
HTAP Relevance: The work of Michael Schulz and co. is highly relevant to HTAP. Their proposed "Atomospheric Tracer Model Intercomparison Tools Initiative" is a key European effort for advancing the IT support for model comparison and evaluation studies. Coordination and collaboration with this project would seem highly beneficial for HTAP. Since the ESG/CDAT IT infrastrucure is for low-level storage/computation support for application projects, it is a natural complement to the more open web-services-based systems such as DataFed.
[edit] GEMS Global Earth-system Monitoring using Space and in-situ data
C. Textor
|
General Comments: GEMS main goal is to build a validated, operational, assimilation system for atmospheric composition. Christiane presented the general goals and methods of the GEMS project, which is still in the formulation phase.
IT Issues: Evidently the GEMS project is supported by a wide array of IT tools. Most of the IT is to be provided by the 31 consortium members.
HTAP Relevance: Based on its very wothy (and very ambitious) goal of an assimilating chemical model, the GEMS project is highly relevant to HTAP. Its specific IT contributions will become clearer, when the project evolves.
[edit] AutoMOD: Towards Automated Model Output Analysis
Charles Doutriaux
|
General Comments:
Charles has outlined the Earth System Grid(ESG) and Climate Data Analysis Tools (CDAT). Both are being developed as part of cyberinfrastructure to support international Earth Science (mostly weather and climate) research. The concept of AutoMOD is to automate many of the model comparison tasks using ESG/CDAT. It seems to be closely linked to AeroCOM, the sequel.
IT Issues: The automated model diagnostic facility (AutoMod) is an application project designed specifically to automate model intercomparison and evaluation studies. The key feature of the system is the reliance on standard netCDF format and CF compliant naming conventions. Parts of AutoMOD are evidently in beta phase (?).
HTAP Relevance: Since the AeroCom and possibly other European projects will use the ESG/CDAT infrastructure, this project is highly relevant to HTAP. In fact, interoperable networking between American data federations (e.g. DataFed) and this project could serve as the initial IT backbone for a networked HTAP system. However, there is also a need to include the third major 'node' of the HTAP network: the Asian modeling hub.
[edit] ENSEMBLE: Reconciliation of Disparate National Medium/Long Range Dispersion Forecasts
Stefano Galmarini
|
General Comments:
Stefano presented the European modeling and analysis system that forcasts the dispersion of accidental releases to the atmosphere.
IT Issues: The IT for the project is a highly integrated closed system. There are no provisions for exposing the IT system for general use.
HTAP Relevance: The European multi-model system for dispersion forecasting is an interesting example of event-driven forecast modeling. I was really impressed by the readyness and functionality of the multi-national ENSEMBLE forecast system! It looks like an important new branch of modeling. Conceivably, the idea of ENSEMBLE and some of the methods could be applicable to similar forecast modeling of hemispheric pollution events such as forest fires (e.g. Siberia, Canada, Central America), dust storms (Gobi, Sahara) or other more nasty stuff.
[edit] DataFed: Federated Data System for Air Quality
R. B. Husar, Washington University
|
General Comments:
This was a general review of the DataFed federated data system technologies with some illustration of distributed model access and processing.
IT Issues: DataFed is a general purpose system for federating diseparte, observational, and modeling datasets. DataFed is a general connection mechanism for tying together and homogenizing for processing distributed datasets relevant to HTAP, including emissions, observations and models.
HTAP Relevance: Being a mediator facility, DataFed would be a natural complement to the supercomputer-based Earth System Grid for data storage and processing. In fact, in disussing this with Charles Doutriaux, we concluded, that pursuing the link between ESG/CDAT and DatFed would be an important near-term activity. The main feature of that link is the use of the suitably adopted netCDF CF interoprability protocol.
[edit] Proposal for cooperation: Atmospheric Tracer Model Intercomparison Tools Initiative
Michael Schulz
[edit] General Goals
- Accelerate Analysis of Models and Feedback to model participants
- Develop jointly intercomparison tools through transformation, integration, adaptation and development of tools
- Allow participants to join more easily into the analysis of an intercomparison
[edit] Specific Objectives to which the Network should contribute for any intercomparison
- Model documentation
- Model quality control
- Model comparison
- Model benchmarking
- Model improvement
- Scientific understanding
[edit] Procedure
In the first place tutorial institutions are identified, which provide general support for the planning and implementation of the initiative (JRC, LSCE/IPSL + ??) A steering committee is put into place to develop the initiative and report to the tutorial institutions. A work plan is elaborated until early summer 2006 to structure and prioritise the actions to be undertaken.
A pilot project is the support of the planned 1st phase of the intercomparison under the HTAP convention, by reusing and integrating tools prepared for EuroDelta, ACCENT and AeroCom. Present the initiative at forthcoming meetings (eg AeroCom 17-19 October HTAP workshops etc) and promote its existence through publication (IGAC newsletter? +?)
[edit] Workplan components
The Workplan aims to identify a detailed implementation plan On What steps are taken, Who is doing them, and When they are ready.
- Identify the formatting standards needed > units, CF or less strict??
- Identify standard names for specific tracer variables > report to CF
- Identify the standard interface specifications between any of the tools and any of the data bases (see slide 5) so that different tools can call and address each databases
- To develop a standard protocol form to be used for different intercomparisons and subparts of them ( reference+compliance w check tools)
- Define standards for file names and image names for databases
- Develop compliance test tools to test whether data and tools fulfill the standards set under 1-5
- Implement a pilot database based on automod for testing if different existing tools (AeroCom catalogues etc)
- Build a web based repository for users to find tools to prepare CF compliant model output (FORTRAN routines) to rename and reformat files (nco examples)to do specific diagnostics (region budgets, aerosol size fractions etc)to do regridding compliant to the standards to handle/replace/identify missing data
- To develop check tools for physical meaning of data (budgets, units, order of magnitude, ocean/land contrast)
- Develop observational data comparison tools which interface to the model data as defined under 1-4
- Develop tools to develop higher order analysis results (ensemble averages, spatial correlation)
- Documentation tool for keeping track of model version characteristics
[edit] DataFed Participation in HTAP Model Intercomparison
Terry, Frank & Ispra Workgroup
I have read Michaels's March 14 Ispra workshop summary with great deal of satisfaction. The amount of information exchanged, the degree of harmony in our collective thinking and the excellent summary made the meeting valuable to me personally and hopefully for the entire HTAP process. Of particular value are the lists of to-do items and assignments.
Below please find three particular areas in which our group could contribute to this HTAP model intercomparison effort. These entries are added at this late phase of the report preparation and have not been specifically discussed at the meeting. However, each proposed items fits into one of the entries in the to-do list.
o Demonstrate data access/transfer interoperability ( e.g. between OCMIP-AutoMod - DataFed) using standard protocol-based interfaces (e.g. OGC-WCS), common data models and standard data formats ( e.g. netCDF-CF)
o to present such progress at future workshops [ e.g. ACCENT model benchmarking workshop (Thessaloniki May 2006); IGARSS06/GEOSS Meeting (Denver, July 2006), IGBP-IGAC/WCRP-SPARC Atmospheric Chemistry and Climate Initiative (Boulder, August 2006), AeroCom workshop (Virginia Beach, October 2006); WMO workshop on IT (Geneva. Jun 2007)].
This third item is a recommendation to the Ispra group regarding interoperability: it would be beneficial to have members of this model intercomparison group participate in on-going netCDF-CF/OGC-WCS interoperability experiments conducted by the broader community. Our group is already actively participates in GALEON (Geo-interface to Atmosphere, Land, Earth, Ocean, NetCDF, http://www.ogcnetwork.net/galeon ) which is now entering Phase II. It would seem desirable for someone else from this group (e.g. Charles) to join this active hands-on interoperability experiment. Incidentally, GALEON has already a strong European membership through U. Florence/CNR and also NERC in the UK with outstanding technical expertise in this area. If someone from this group would like to participate in GALEON I would be happy to make the necessary introductions ....
We are very much looking forward to the next, active collaborative phase of this HTAP sub-project. I am done with the finals for the semester and over the next few months I am planning to devote considerable attention to this and similar collaborative efforts with focus on interoperability. Also, as we collect relevant materials, I will add this to the wiki page of my March Ispra meeting report to Terry Keating: http://datafedwiki.wustl.edu/index.php/Mar_14%2C_2006:_Ispra_IT_Meeting
Looking forward for your response,
Cheers, Rudy STL, May 4, 2006
Categories: Activities | Done | 060314 | Atomic | DataFed
