Introduction
FastEddy is a resident-GPU large eddy simulation (LES) model owned by the National Center for Atmospheric Research (NCAR) Research Applications Laboratory (RAL). It is designed for future turbulence-resolving numerical weather prediction.
This is a tutorial designed so that a user can learn how to execute FastEddy. Four test cases are described:
Dry neutral boundary layer
Dry convective boundary layer
Dry stable boundary layer
Moist cloud-topped boundary layer
Required tutorial resources including input files, python utilities and Jupyter Notebooks are provided in https://github.com/NCAR/FastEddy-tutorials. All test cases are ideal setups over zero terrain. For each case, the user will set up the input parameter file, execute FastEddy, visualize the output using a Jupyter notebook, and perform some basic analysis of the output. After examining the test cases, the user will carry out some sensitivity tests by changing various input parameters. The purpose of these tests are for the user to become more familiar with the input parameters, and how changes to those parameters affect the output. After the tutorial, the user is expected to have basic knowledge to carry out LES using FastEddy.
Software and computing requirements
Computing resources with at least four general purpose graphics processing units are recommended to carry out the test cases. System must be enabled with python and Jupyter notebook packages. Add other requirements (compilers, libraries, etc).
Instructions on how to build and run FastEddy on NCAR’s Casper architecture https://github.com/NCAR/FastEddy-model/blob/main/README.md.
Dry neutral boundary layer
Background
This is a canonical neutral boundary layer scenario. The case is broadly based upon Sauer and Munoz-Esparza (2020) but is not identical. A geostrophic wind is prescribed over ground with a set aerodynamic roughness length under a neutrally stratified boundary layer. The purpose of this test case is to visualize and analyze the resultant flow and turbulence characteristics that develop when the LES reaches statistical steady-state.
Input parameters
Number of grid points: \([N_x,N_y,N_z]=[640,634,58]\)
Isotropic grid spacings in the horizontal directions: \([dx,dy]=[15,15]\) m, vertical grid is \(dz=15\) m at the surface and stretched with verticalDeformFactor \(=0.75\)
Domain size: \([9.6 \times 9.51 \times 1.08]\) km
Model time step: \(0.04\) s
Advection scheme: 5th-order upwind
Time scheme: 3rd-order Runge Kutta
Geostrophic wind: \([U_g,V_g]=[10,0]\) m/s
Latitude: \(54.0^{\circ}\) N
Surface potential temperature: \(300\) K
Potential temperature profile:
Surface heat flux: \(0.0\) Km/s
Surface roughness length: \(z_0=0.1\) m
Rayleigh damping layer: uppermost \(400\) m of the domain
Initial perturbations: \(\pm 0.25\) K
Depth of perturbations: \(375\) m
Top boundary condition: free slip
Lateral boundary conditions: periodic
Time period: \(7\) h
Execute FastEddy
Run FastEddy using the input parameters file /examples/Example01_NBL.in. To execute FastEddy, follow the instructions here: https://github.com/NCAR/FastEddy-model/blob/main/README.md.
Visualize the output
Open the Jupyter notebook entitled “MAKE_FE_TUTORIAL_PLOTS.ipynb” and execute it using setting: case = ‘neutral’.
XY-plane views of instantaneous velocity components at \(t=7\) h (FE_NBL.630000):
XZ-plane views of instantaneous velocity components at \(t=7\) h (FE_NBL.630000):
Mean (domain horizontal average) vertical profiles of state variables at \(t=7\) h (FE_NBL.630000):
Horizontally-averaged vertical profiles of turbulence quantities at \(t=6-7\) h [perturbations are computed at each time instance from horizontal-slab means, then averaged horitontally and over the previous 1-hour mean]:
Analyze the output
Using the XY and XZ cross sections, discuss the characteristics (scale and magnitude) of the resolved turbulence.
What is the boundary layer height in the neutral case?
Using the vertical profile plots, explain why the boundary layer is neutral.
Dry convective boundary layer
This is the convective boundary layer scenario described by Sauer and Munoz-Esparza (2020). This case represents the boundary layer conditions at the SWiFT facility near Lubbock, Texas at 4 July 2012 during the period of 18Z-20Z (12:00–14:00 local time), the strongest period of convection on the day.
Input parameters
Number of grid points: \([N_x,N_y,N_z]=[600,594,122]\)
Isotropic grid spacings in the horizontal directions: \([dx,dy]=[20,20]\) m, vertical grid is \(dz=20\) m at the surface and stretched with verticalDeformFactor \(=0.80\)
Domain size: \([12.0 \times 11.9 \times 3.0]\) km
Model time step: \(0.05\) s
Geostrophic wind: \([U_g,V_g]=[9,0]\) m/s
Advection scheme: Hybrid 5th order upwind
Time scheme: 3rd-order Runge Kutta
Latitude: \(33.5^{\circ}\) N
Surface potential temperature: \(309\) K
Potential temperature profile:
Surface heat flux: \(0.35\) Km/s
Surface roughness length: \(z_0=0.05\) m
Rayleigh damping layer: uppermost \(400\) m of the domain
Initial perturbations: \(\pm 0.25\) K
Depth of perturbations: \(400\) m
Top boundary condition: free slip
Lateral boundary conditions: periodic
Time period: \(4\) h
Execute FastEddy
Run FastEddy using the input parameters file /examples/Example02_CBL.in. To execute FastEddy, follow the instructions here: https://github.com/NCAR/FastEddy-model/blob/main/README.md.
Visualize the output
Open the Jupyter notebook entitled “MAKE_FE_TUTORIAL_PLOTS.ipynb” and execute it using setting: case = ‘convective’.
XY-plane views of instantaneous velocity components at \(t=4\) h (FE_CBL.288000):
XZ-plane views of instantaneous velocity components at \(t=4\) h (FE_CBL.288000):
Mean (domain horizontal average) vertical profiles of state variables at \(t=4\) h (FE_CBL.288000):
Horizontally-averaged vertical profiles of turbulence quantities \(t=3-4\) h [perturbations are computed at each point relative to the previous 1-hour mean, and then horizontally averaged]:
Analyze the output
Using the XY and XZ cross sections, discuss the characteristics (scale and magnitude) of the resolved turbulence.
What is the boundary layer height in the convective case?
Using the vertical profile plots, explain why the boundary layer is unstable.
Dry stable boundary layer
Background
This is the stable boundary layer scenario described by Sauer and Munoz-Esparza (2020). This the stable boundary layer scenario outlined in Kosovic and Curry (2000).
Input parameters
Number of grid points: \([N_x,N_y,N_z]=[128,126,122]\)
Isotropic grid spacings: \([dx,dy,dz]=[3.125,3.125,3.125]\) m
Domain size: \([0.40 \times 0.39 \times 0.38]\) km
Model time step: \(0.005\) s
Geostrophic wind: \([U_g,V_g]=[8,0]\) m/s
Advection scheme: 5th-order upwind
Time scheme: 3rd-order Runge Kutta
Latitude: \(73^{\circ}\) N
Surface potential temperature: \(265\) K
Potential temperature profile:
Surface heat flux: \(-0.25\) K/h
Surface roughness length: \(z_0=0.1\) m
Rayleigh damping layer: uppermost \(75\) m of the domain
Initial perturbations: \(\pm 0.25\) K
Top boundary condition: free slip
Lateral boundary conditions: periodic
Time period: \(12\) h
Execute FastEddy
Run FastEddy using the input parameters file /examples/Example03_SBL.in. To execute FastEddy, follow the instructions here: https://github.com/NCAR/FastEddy-model/blob/main/README.md.
Visualize the output
Open the Jupyter notebook entitled “MAKE_FE_TUTORIAL_PLOTS.ipynb” and execute it using setting: case = ‘stable’.
XY-plane views of instantaneous velocity components at \(t=12\) h (FE_SBL.8640000):
XZ-plane views of instantaneous velocity components at \(t=12\) h (FE_SBL.8640000):
Mean (domain horizontal average) vertical profiles of state variables at \(t=12\) h (FE_SBL.8640000):
Horizontally-averaged vertical profiles of turbulence quantities at \(t=11-12\) h (FE_TEST.8640000) [perturbations are computed at each point relative to the previous 1-hour mean, and then horizontally averaged]:
Analyze the output
Using the XY and XZ cross sections, discuss the characteristics (scale and magnitude) of the resolved turbulence.
What is the boundary layer height in the stable case?
Using the vertical profile plots, explain why the boundary layer is stable.
Moist cloud-topped boundary layer
This tutorial case is the BOMEX LES intercomparison study from Siebesma et al. (2003), corresponding to a non-precipitating shallow cumulus cloud case informed by the Barbados Oceanographic and Meteorological Experiment (BOMEX, Holland & Rasmusson, 1973). The forcing consists of two different sources including prescribed kinematic surface fluxes of sensible and latent heat and large-scale forcing (LSF) tendencies due to mesoscale horizontal advection of water vapor mixing ratio, liquid potential temperature and horizontal momentum. The LSF includes subsidence to compensate the integrated effect of surface fluxes and advection tendencies, formulated as a prescribed time-invariant subsidence profile multiplied by the vertical gradient of horizontally averaged fields accros the domain. The main settings of this case are listed below and are further detailed in Munoz-Esparza et al. (2022).
Input parameters
Number of grid points: \([N_x,N_y,N_z]=[152,146,122]\)
Isotropic grid spacings: \([dx,dy,dz]=[100,100,40]\) m
Domain size: \([15.2 \times 14.6 \times 4.9]\) km
Model time step: \(0.075\) s
Geostrophic wind: \([U_g,V_g]=[10,0]\) m \(\mbox{s}^{-1}\)
Advection schemes: 5th-order upwind (dry dynamics), 3rd-order upwind (water vapor), and 3rd-order WENO (liquid water)
Time scheme: 3rd-order Runge Kutta
Latitude: \(14.94^{\circ}\) N
Surface potential temperature: \(299.1\) K
Surface sensible heat flux: \(8 \times 10^{-3}\) K m \(\mbox{s}^{-1}\)
Surface latent heat flux: \(5.2 \times 10^{-5}\) m \(\mbox{s}^{-1}\)
Surface roughness length: \(z_0=0.0002\) m
Rayleigh damping layer: uppermost \(500\) m of the domain
Initial perturbations: \(\pm 0.1\) K
Depth of perturbations: \(1600\) m
Top boundary condition: free slip
Lateral boundary conditions: periodic
Time period: \(6\) h
Initital conditions: vertical profiles of \(u\), \(q_v\), and SGSTKE as specified in Siebesma et al. (2003)
Large-scale forcings: vertical profiles of subsidence and horizontal advection of potential temperature and water vapor as specified in Siebesma et al. (2003)
Execute FastEddy
Run FastEddy using the input parameters file /examples/Example04_BOMEX.in. Note that the BOMEX case requires an initial condition provided in /datasets/BOMEX_IC/FE_BOMEX.0. To execute FastEddy, follow the instructions here: https://github.com/NCAR/FastEddy-model/blob/main/README.md.
Visualize the output
Open the Jupyter notebook entitled “FE_Postrocessing_Example04_BOMEX.ipynb” and execute it.
Time evolution of domain averaged total cloud cover (\(\alpha_c\)) and liquid water path (LWP):
Vertical profiles of potential temperature (\(\theta\)), water vapor (\(q_v\)), horizontal velocity components (\(u\), \(v\)), and liquid content (\(q_l\)). Thin black dashed lines correspond to the initial conditions. Profiles are averaged for the last 3 hr (\(t = 180-360\) min) and over horizontal domain slabs:
Vertical profiles of turbulence kinetic energy (TKE), vertical velocity variance (\(\sigma^2_w\)), and vertical turbulent fluxes of zonal momentum (\(\langle u'w' \rangle\)), water vapor (\(\langle w'q_v' \rangle\)), liquid cloud, and virtual potential temperature (\(\langle w'q_l' \rangle\)). Profiles are averaged for the last 3 hr (\(t = 180–360\) min) and perturbations are computed as the departure from horizontal slab averages. These turbulence quantities are the sum of resolved and subgrid-scale components.
Analyze the output
Using the time series of cloud properties, could you identify when the simulated shallow cumulus cloud deck has reached quasi-eqilibrium?
What is the effect of boundary-layer turbulence to the mean profiles of momentum?
Identify the vertical extent of the cloud layer.
Which of the turbulent vertical transport terms is responsible for the resulting vertical liquid cloud distribution?
Instructions
Re-run the neutral case with \([N_x,N_y,N_z]=[400,400,122]\) and isotropic grid spacings of \([dx,dy,dz]=[10,10,10]\). Adjust the model time step accordingly. Re-make all plots and discuss the differences between the control case. How much longer did it take to complete the simulation?
Re-run the convective case with a surface heat flux of \(=+0.70\) Km/s. Re-make all plots and discuss the differences between the control case.
Re-run the neutral case with \(z_0=0.3\) m. Re-make all plots and discuss the differences between the control case.
Re-run the neutral case with the first order upwind advection scheme. Re-make all plots and discuss the differences between the control case. Why is the first order scheme a bad choice?
Re-run the stable case with a surface cooling rate of \(-0.5\) K/h. Re-make all plots and discuss the differences between the control case.
Re-run the stable case using half of the GPUs used in the control simulation. How much slower does the case run?
Re-run the BOMEX case with a higher-order advection for water vapor (moistureAdvSelectorQv = 3). What is the impact of the increased effective resolution on dynamical, thermodynamical and microphysical quantities, along with turbulence variability and fluxes? How does that change influce the comparison to the other BOMEX LES models?