CHIMERE - XIOS

XIOS notes : https://www.lmd.polytechnique.fr/~rpennel/chimere/XIOS-formation-dev.pdf

Philosophy

  1. client-server paradigm , i.e the client (chimere) sends data to the server(s) (xios) and xios deals with those data to produce outputs
  2. the data flux from chimere to netcdf outputs is therefore asynchronous
  3. the client sends all data every time step (needs to be constant time step) and xios decides if the data is usefull or not for output
  4. xios is driven with xml files (xios : xml input output system) - most of the xml tags have a fortran counterpart (i.e possible to give instructions to xios via xml or fortran code)

Implementation in CHIMERE

Compilation

  1. In the architecture file add :
    • my_netcdfCinc= # Path to NETCDF-C (linked with HDF5 parallel) include directory
    • configure_xios_file_name= arch-[gfortran/ifort].fcm # Makefile header (*.fcm) to compile XIOS in config_xios directory
  2. ./build-chimere.sh compile oasis -> xios -> chimere
  3. xios compilation can be long, if xios is already compiled build-chimere will not recompile it
  4. oasis compilation is required before xios compilation
  5. xios compilation is mandatory to use chimere , even if it is not used
  6. ./build-xios.sh is called by build-chimere.sh there should be no reason to use it directly

Switch XIOS use on/off

in chimere.par 2 new tags :

content_copy
[$nproc_xios]       Number of xios processors    : 2
[$io_nc4]           Use classic NC4 I/O          : 1
  1. if [$nproc_xios] : 0 xios is not used
  2. the number of xios servers [$nproc_xios] is up to you and can act on performance (has been tested with 2, but if lots of data can be increased)
  3. [$io_nc4] switch on/off the use of the classical chimere output system
  4. for now both systems can be used at the same time to check correct behaviour of xios

XML files

a new directory : xml
contains several files :

  1. iodef.xml : general information for xios (memory management/ log management )
    • using_oasis : dynamically set if chimere online or offline (in scripts/chimere-init.sh)
  2. context_chimere.xml : everythinhg , i.e fields, params, files (those 3 are described in seprate files), grids
  3. param_def_chimere.xml : list of constants like molar masses used by xios to perform units changes :
    • ex : <variable id="mm_APINEN" type="float"> 136.0 </variable>
  4. field_def_chimere.xml : list of all the fields (i.e conc, meteo variables) that are potentially used during a chimere run, and that the user would like to output
    • a field have an id, this id is used on the fotran side to send data to xios call xios_send_field('O3', conc(indx, 1:nzonal, 1:nmerid, ks))
    • operations can be performed by xios on this field, here change units to ppb:
    content_copy
     <field id="fppb" field_ref="airm" long_name="" standard_name="" unit="who knows !"> 1e9/this</field>
     <field id="O3" long_name="O3" standard_name="O3" unit="ppb vol" grid_ref="grid_T_3D">  this*fppb  </field>
    • fields can be gathered into field_group
    • 2 specials field groups are defined and set empty in the xml :
    content_copy
    <field_group id="run_conc" enabled=".true." />        
    <field_group id="run_meteo" enabled=".true." />
    • those 2 groups are dynamically filled on the fortran side with the concs and meteo vars necessary to produce the classical out file
  5. file_def.xml: list of output files
    • <file id="out" name_suffix="_out" output_freq="1h" sync_freq="1ts" enabled=".true.">
    • file can be enabled (enabled=".true.") or disabled (enabled=".false.")
    • output_freq : 1 record every output freq
    • sync_freq : file updated on disk every sync freq (ts is time step)
    • fields that we want in the output are listed in the file tag, we put here our 2 special field groups :
    content_copy
     <file id="out" name_suffix="_out" output_freq="1h" sync_freq="1ts" enabled=".true.">
          <variable  id="Title"                     type="string">  CHIMERE        </variable>
          <variable  id="Sub-title"                 type="string">  'hourly data'  </variable>
          <variable  id="Date_origin"               type="string">  'Date_origin'  </variable>
          <variable  id="CHIMERE_version"           type="string">  v2020r1        </variable>
          <variable  id="Domain"                    type="string">  domain         </variable>
          <variable  id="CHIMERE_type"              type="string">  out            </variable>
          <variable  id="Online"                    type="string">  online         </variable>
          <variable  id="Cpl_case"                  type="string">  cpl_case       </variable>
          <variable  id="Physical_timestep"         type="string">  physstep       </variable>
          <variable  id="Chemical_timestep"         type="string">  chemstep       </variable>
          <variable  id="Labchem"                   type="string">  labchem        </variable>
     
          <field  field_ref="cut_off_diameters"  />
          <field  field_ref="a_vcoord"           />
          <field  field_ref="b_vcoord"           />
          <field  field_ref="mmd"                />
          <field  field_ref="wavelength"         />
          <field  field_ref="hgt"                />
                <!-- All fields will be auto completed on the fortan side   -->
                <!-- DO NOT MODIFY - for custom outputs add a new file below -->
                 <field_group id="auto_out"  enabled=".true.">
                         <field_group group_ref="run_conc"  />
                         <field_group group_ref="run_meteo" />
                 </field_group>
      </file>
    • 2 special files : start and out , start is the initial condition , out the classical out chimere file (without the first time step)
    • 4 example files : ex_0[1-4] disabled by default
      • ex_01 : statistics on a field (min, max, average)
      • ex_02 : operation on a field (sum, test)
      • ex_03 : extract first levels of a field (using a special grid) every time step
      • ex_04 : extract a profil every time step
    • content_copy
        -rw-r--r-- 1 rpennel lmdpol   2138343 Dec  8 13:25 chimere_xios_force_IDF5_2017010100_2017010200_extract.nc
        -rw-r--r-- 1 rpennel lmdpol 545775954 Dec  8 13:25 chimere_xios_force_IDF5_2017010100_2017010200_out.nc
        -rw-r--r-- 1 rpennel lmdpol   3797231 Dec  8 13:25 chimere_xios_force_IDF5_2017010100_2017010200_peaks.nc
        -rw-r--r-- 1 rpennel lmdpol     89048 Dec  8 13:25 chimere_xios_force_IDF5_2017010100_2017010200_prof.nc
        -rw-r--r-- 1 rpennel lmdpol  23682461 Dec  8 13:25 chimere_xios_force_IDF5_2017010100_2017010200_start.nc
        -rw-r--r-- 1 rpennel lmdpol   5675991 Dec  8 13:25 chimere_xios_force_IDF5_2017010100_2017010200_stats.nc

Fortran side

  1. everything in a new module ciom_mod no call of xios anywhere else (except in chisis but to be changed)
  2. ciom used in 3 files :
    • chimere.F90 : to init mpi, xios and finalize
    • iniworker.F90 : to call chisis-endef
    • worker.F90 : update xios calendar, send fields, deal with screen outputs

Check if outputs are ok

How to add a new variable in chimere

  1. define a new field in field_def_chimere, if it is a new conc id should be the name in output_species
  2. if it is a new conc that's it
  3. if it is another variable :
    • either add the field in the fortran side to the 2 special groups like in ciom_init_meteo or put it directly in the xml in the start and out section
    • add a send of this variable to xios like in ciom_send_meteo