Running RAPID

Tutorial

Step 1: Initialize the RAPID manager class.

  • First, add the path to you rapid executable location.
  • Next, you need to either tell it how many processors to use using the num_processors input variable or to use all available processors set use_all_processors to true.
  • After that, add any other parameters you would like to use that would normally be in the rapid namelist file (this is case sensitive).
class RAPIDpy.rapid.RAPID(rapid_executable_location='', num_processors=1, use_all_processors=False, cygwin_bin_location='', mpiexec_command='mpiexec', ksp_type='richardson', **kwargs)[source]

This class is designed to prepare the rapid_namelist file and run the RAPID program. There are also other utilities added.

rapid_executable_location

Optional[str] – Path to the RAPID executable location.

num_processors

Optional[int] – Number of procesors to use. Default is 1. Overridden if use_all_processors is True.

use_all_processors

Optional[bool] – If set to True, the RAPID program will use all available processors. Default is False.

cygwin_bin_location

Optional[str] – If using Windows, this is the path to the Cygwin ‘bin’ directory.

mpiexec_command

Optional[str] – This is the mpi execute commmand. Default is “mpiexec”.

ksp_type

Optional[str] – This is the solver type. Default is “richardson”.

**kwargs

Optional[str] – Keyword arguments matching the input parameters in the RAPID namelist.

Linux Example:

from RAPIDpy import RAPID

rapid_manager = RAPID(rapid_executable_location='~/work/rapid/run/rapid'
                      use_all_processors=True,
                      ZS_TauR=24*3600, #duration of routing procedure (time step of runoff data)
                      ZS_dtR=15*60, #internal routing time step
                      ZS_TauM=365*24*3600, #total simulation time
                      ZS_dtM=24*3600 #input time step
                     )

Windows with Cygwin Example:

from RAPIDpy import RAPID

rapid_manager = RAPID(rapid_executable_location='C:/cygwin64/home/username/work/rapid/run/rapid',
                      cygwin_bin_location='C:/cygwin64/bin',
                      use_all_processors=True,
                      ZS_TauR=24*3600, #duration of routing procedure (time step of runoff data)
                      ZS_dtR=15*60, #internal routing time step
                      ZS_TauM=365*24*3600, #total simulation time
                      ZS_dtM=24*3600 #input time step
                     )

Step 2 (optional): Add/update additional namelist parameters later

RAPID.update_parameters(**kwargs)[source]

You can add or update rapid namelist parameters by using the name of the variable in the rapid namelist file (this is case sensitive).

Parameters:**kwargs (Optional[str]) – Keyword arguments matching the input parameters in the RAPID namelist.

Example:

from RAPIDpy import RAPID

rapid_manager = RAPID(
                        #ADD PARAMETERS
                     )

rapid_manager.update_parameters(rapid_connect_file='../rapid-io/input/rapid_connect.csv',
                                Vlat_file='../rapid-io/input/m3_riv.nc',
                                riv_bas_id_file='../rapid-io/input/riv_bas_id.csv',
                                k_file='../rapid-io/input/k.csv',
                                x_file='../rapid-io/input/x.csv',
                                Qout_file='../rapid-io/output/Qout.nc',
                                )

Step 3 (optional): Update reach number data

RAPID.update_reach_number_data()[source]

Update the reach number data for the namelist based on input files.

Warning

You need to make sure you set rapid_connect_file and riv_bas_id_file before running this function.

Example:

from RAPIDpy import RAPID

rapid_manager = RAPID(
                      #ADD PARAMETERS
                      rapid_connect_file='../rapid-io/input/rapid_connect.csv',
                      riv_bas_id_file='../rapid-io/input/riv_bas_id.csv',
                     )

rapid_manager.update_reach_number_data()

Step 4 (optional): Update simulation runtime data

RAPID.update_simulation_runtime()[source]

Updates the total simulation duration from the m3 file (Vlat_file) and the time step (ZS_TauR).

Warning

You need to set the m3 file (Vlat_file) and the time step (ZS_TauR) before runnning this function.

Example:

from RAPIDpy import RAPID

rapid_manager = RAPID(
                      #ADD PARAMETERS
                      Vlat_file='../rapid-io/input/m3_riv.csv',
                      ZS_TauR=3*3600,
                     )

rapid_manager.update_simulation_runtime()

Step 5: Run RAPID

RAPID.run(rapid_namelist_file='')[source]

Run RAPID program and generate file based on inputs This will generate your rapid_namelist file and run RAPID from wherever you call this script (your working directory).

Parameters:rapid_namelist_file (Optional(str)) – Path of namelist file to use in the simulation. It will be updated with any parameters added to the RAPID manager.

Linux Example:

from RAPIDpy import RAPID

rapid_manager = RAPID(rapid_executable_location='~/work/rapid/src/rapid'
                      use_all_processors=True,
                     )

rapid_manager.update_parameters(rapid_connect_file='../rapid-io/input/rapid_connect.csv',
                                Vlat_file='../rapid-io/input/m3_riv.nc',
                                riv_bas_id_file='../rapid-io/input/riv_bas_id.csv',
                                k_file='../rapid-io/input/k.csv',
                                x_file='../rapid-io/input/x.csv',
                                Qout_file='../rapid-io/output/Qout.nc',
                                )

rapid_manager.update_reach_number_data()
rapid_manager.update_simulation_runtime()
rapid_manager.run(rapid_namelist_file='../rapid-io/input/rapid_namelist')

Linux Reservoir Forcing Flows Example:

from RAPIDpy import RAPID

rapid_manager = RAPID(rapid_executable_location='~/work/rapid/src/rapid',
                      num_processors=4,
                      IS_for_tot=4,
                      IS_for_use=4,
                      for_tot_id_file='../rapid-io/input/dam_id.csv',
                      for_use_id_file='../rapid-io/input/dam_id.csv',
                      Qfor_file='../rapid-io/input/qout_dams.csv',
                      ZS_dtF=86400,
                      BS_opt_for=True,
                      )

rapid_manager.run(rapid_namelist_file='../rapid-io/input/rapid_namelist_regular_run')

Windows with Cygwin Example:

from RAPIDpy import RAPID
from os import path

rapid_manager = RAPID(rapid_executable_location='C:/cygwin64/home/username/work/rapid/run/rapid',
                      cygwin_bin_location='C:/cygwin64/bin',
                      use_all_processors=True,
                      ZS_TauR=24*3600,
                      ZS_dtR=15*60,
                      ZS_TauM=365*24*3600,
                      ZS_dtM=24*3600
                     )

rapid_input_folder = 'C:/cygwin64/home/username/work/rapid-io/input'
rapid_output_folder = 'C:/cygwin64/home/username/work/rapid-io/output'
rapid_manager.update_parameters(rapid_connect_file=path.join(rapid_input_folder, 'rapid_connect.csv'),
                                Vlat_file=path.join(rapid_input_folder, 'm3_riv.nc'),
                                riv_bas_id_file=path.join(rapid_input_folder, 'riv_bas_id.csv'),
                                k_file=path.join(rapid_input_folder, 'k.csv'),
                                x_file=path.join(rapid_input_folder, 'x.csv'),
                                Qout_file=path.join(rapid_output_folder, 'Qout.nc'),
                                )

rapid_manager.update_reach_number_data()
rapid_manager.update_simulation_runtime()
rapid_manager.run()

Step 6 (optional): Convert RAPID output to be CF Compliant

RAPID.make_output_CF_compliant(simulation_start_datetime, comid_lat_lon_z_file='', project_name='Normal RAPID project')[source]

This function converts the RAPID output to be CF compliant. This will require a comid_lat_lon_z.csv file (See: FlowlineToPoint() to generate the file).

Note

It prepends time an initial flow to your simulation from the qinit_file. If no qinit file is given, an initial value of zero is added.

Warning

This will delete your original Qout file.

Parameters:
  • simulation_start_datetime (datetime) – Datetime object with the start date of the simulation.
  • comid_lat_lon_z_file (Optional[str]) – Path to the comid_lat_lon_z.csv file. If none given, spatial information will be skipped.
  • project_name (Optional[str]) – Name of project to add to the RAPID output file.

Example:

from RAPIDpy import RAPID

rapid_manager = RAPID(rapid_executable_location='~/work/rapid/run/rapid'
                      use_all_processors=True,
                      ZS_TauR=24*3600,
                      ZS_dtR=15*60,
                      ZS_TauM=365*24*3600,
                      ZS_dtM=24*3600
                      rapid_connect_file='../rapid-io/input/rapid_connect.csv',
                      Vlat_file='../rapid-io/input/m3_riv.nc',
                      riv_bas_id_file='../rapid-io/input/riv_bas_id.csv',
                      k_file='../rapid-io/input/k.csv',
                      x_file='../rapid-io/input/x.csv',
                      Qout_file='../rapid-io/output/Qout.nc',
                     )

rapid_manager.run()

rapid_manager.make_output_CF_compliant(simulation_start_datetime=datetime.datetime(1980, 1, 1),
                                       comid_lat_lon_z_file='../rapid-io/input/comid_lat_lon_z.csv',
                                       project_name="ERA Interim Historical flows by US Army ERDC")

Full API Description

class RAPIDpy.rapid.RAPID(rapid_executable_location='', num_processors=1, use_all_processors=False, cygwin_bin_location='', mpiexec_command='mpiexec', ksp_type='richardson', **kwargs)[source]

This class is designed to prepare the rapid_namelist file and run the RAPID program. There are also other utilities added.

rapid_executable_location

Optional[str] – Path to the RAPID executable location.

num_processors

Optional[int] – Number of procesors to use. Default is 1. Overridden if use_all_processors is True.

use_all_processors

Optional[bool] – If set to True, the RAPID program will use all available processors. Default is False.

cygwin_bin_location

Optional[str] – If using Windows, this is the path to the Cygwin ‘bin’ directory.

mpiexec_command

Optional[str] – This is the mpi execute commmand. Default is “mpiexec”.

ksp_type

Optional[str] – This is the solver type. Default is “richardson”.

**kwargs

Optional[str] – Keyword arguments matching the input parameters in the RAPID namelist.

Linux Example:

from RAPIDpy import RAPID

rapid_manager = RAPID(rapid_executable_location='~/work/rapid/run/rapid'
                      use_all_processors=True,
                      ZS_TauR=24*3600, #duration of routing procedure (time step of runoff data)
                      ZS_dtR=15*60, #internal routing time step
                      ZS_TauM=365*24*3600, #total simulation time
                      ZS_dtM=24*3600 #input time step
                     )

Windows with Cygwin Example:

from RAPIDpy import RAPID

rapid_manager = RAPID(rapid_executable_location='C:/cygwin64/home/username/work/rapid/run/rapid',
                      cygwin_bin_location='C:/cygwin64/bin',
                      use_all_processors=True,
                      ZS_TauR=24*3600, #duration of routing procedure (time step of runoff data)
                      ZS_dtR=15*60, #internal routing time step
                      ZS_TauM=365*24*3600, #total simulation time
                      ZS_dtM=24*3600 #input time step
                     )
generate_namelist_file(rapid_namelist_file)[source]

Generate rapid_namelist file.

Parameters:rapid_namelist_file (str) – Path of namelist file to generate from parameters added to the RAPID manager.
generate_qinit_from_past_qout(qinit_file, time_index=-1, out_datetime=None)[source]

Generate qinit from a RAPID qout file

Parameters:
  • qinit_file (str) – Path to output qinit_file.
  • time_index (Optional[int]) – Index of simulation to generate initial flow file. Default is the end.
  • out_datetime (Optional[datetime]) – Datetime object containing time of initialization.

Example:

from RAPIDpy import RAPID

rapid_manager = RAPID(Qout_file='/output_mississippi-nfie/Qout_k2v1_2005to2009.nc',
                      rapid_connect_file='/input_mississippi_nfie/rapid_connect_ECMWF.csv'
                     )

rapid_manager.generate_qinit_from_past_qout(qinit_file='/input_mississippi_nfie/Qinit_2008_flood.csv',
                                            time_index=10162)
generate_seasonal_intitialization(qinit_file, datetime_start_initialization=datetime.datetime(2017, 7, 6, 18, 35, 48, 418889))[source]

This creates a seasonal qinit file from a RAPID qout file. This requires a simulation Qout file with a longer time period of record and to be CF compliant. It takes the average of the current date +- 3 days and goes back as far as possible.

Parameters:
  • qinit_file (str) – Path to output qinit_file.
  • datetime_start_initialization (Optional[datetime]) – Datetime object with date of simulation to go back through the years and get a running average to generate streamflow initialization. Default is utcnow.

This example shows how to use it:

from RAPIDpy.rapid import RAPID

rapid_manager = RAPID(Qout_file='/output_mississippi-nfie/Qout_2000to2015.nc',
                      rapid_connect_file='/input_mississippi_nfie/rapid_connect_ECMWF.csv'
                      )

rapid_manager.generate_seasonal_intitialization(qinit_file='/input_mississippi_nfie/Qinit_seasonal_avg_jan_1.csv')
generate_usgs_avg_daily_flows_opt(reach_id_gage_id_file, start_datetime, end_datetime, out_streamflow_file, out_stream_id_file)[source]

Generate daily streamflow file and stream id file required for calibration or for substituting flows based on USGS gage ids associated with stream ids.

Parameters:
  • reach_id_gage_id_file (str) – Path to reach_id_gage_id file.
  • start_datetime (datetime) – A datetime object with the start date to download data.
  • end_datetime (datetime) – A datetime object with the end date to download data.
  • out_streamflow_file (str) – The path to output the streamflow file for RAPID.
  • out_stream_id_file (str) – The path to output the stream ID file associated with the streamflow file for RAPID.

Example reach_id_gage_id_file:

COMID, USGS_GAGE_ID
2000, 503944
...

Warning

Overuse will get you blocked from downloading data from USGS.

Warning

This code does not clean the data in any way. Thus, you are likely to run into issues if you simply use the raw data.

Warning

The code skips gages that do not have data for the entire time period.

Simple Example:

import datetime
from os.path import join
from RAPIDpy import RAPID

main_path = "/home/username/data"

rapid_manager = RAPID()
rapid_manager.generate_usgs_avg_daily_flows_opt(reach_id_gage_id_file=join(main_path,"mississippi_usgsgage_id_comid.csv"),
                                                start_datetime=datetime.datetime(2000,1,1),
                                                end_datetime=datetime.datetime(2014,12,31),
                                                out_streamflow_file=join(main_path,"streamflow_2000_2014.csv"),
                                                out_stream_id_file=join(main_path,"streamid_2000_2014.csv"))

Complex Example:

import datetime
from os.path import join
from RAPIDpy import RAPID

main_path = "/home/username/data"

rapid_manager = RAPID(rapid_executable_location='~/work/rapid/run/rapid'
                      use_all_processors=True,
                      ZS_TauR=24*3600,
                      ZS_dtR=15*60,
                      ZS_TauM=365*24*3600,
                      ZS_dtM=24*3600
                     )

rapid_manager.update_parameters(rapid_connect_file='../rapid-io/input/rapid_connect.csv',
                                Vlat_file='../rapid-io/input/m3_riv.nc',
                                riv_bas_id_file='../rapid-io/input/riv_bas_id.csv',
                                k_file='../rapid-io/input/k.csv',
                                x_file='../rapid-io/input/x.csv',
                                Qout_file='../rapid-io/output/Qout.nc',
                                )

rapid_manager.update_reach_number_data()
rapid_manager.update_simulation_runtime()
rapid_manager.generate_usgs_avg_daily_flows_opt(reach_id_gage_id_file=join(main_path,"mississippi_usgsgage_id_comid.csv"),
                                                start_datetime=datetime.datetime(2000,1,1),
                                                end_datetime=datetime.datetime(2014,12,31),
                                                out_streamflow_file=join(main_path,"streamflow_2000_2014.csv"),
                                                out_stream_id_file=join(main_path,"streamid_2000_2014.csv"))
rapid_manager.run()
make_output_CF_compliant(simulation_start_datetime, comid_lat_lon_z_file='', project_name='Normal RAPID project')[source]

This function converts the RAPID output to be CF compliant. This will require a comid_lat_lon_z.csv file (See: FlowlineToPoint() to generate the file).

Note

It prepends time an initial flow to your simulation from the qinit_file. If no qinit file is given, an initial value of zero is added.

Warning

This will delete your original Qout file.

Parameters:
  • simulation_start_datetime (datetime) – Datetime object with the start date of the simulation.
  • comid_lat_lon_z_file (Optional[str]) – Path to the comid_lat_lon_z.csv file. If none given, spatial information will be skipped.
  • project_name (Optional[str]) – Name of project to add to the RAPID output file.

Example:

from RAPIDpy import RAPID

rapid_manager = RAPID(rapid_executable_location='~/work/rapid/run/rapid'
                      use_all_processors=True,
                      ZS_TauR=24*3600,
                      ZS_dtR=15*60,
                      ZS_TauM=365*24*3600,
                      ZS_dtM=24*3600
                      rapid_connect_file='../rapid-io/input/rapid_connect.csv',
                      Vlat_file='../rapid-io/input/m3_riv.nc',
                      riv_bas_id_file='../rapid-io/input/riv_bas_id.csv',
                      k_file='../rapid-io/input/k.csv',
                      x_file='../rapid-io/input/x.csv',
                      Qout_file='../rapid-io/output/Qout.nc',
                     )

rapid_manager.run()

rapid_manager.make_output_CF_compliant(simulation_start_datetime=datetime.datetime(1980, 1, 1),
                                       comid_lat_lon_z_file='../rapid-io/input/comid_lat_lon_z.csv',
                                       project_name="ERA Interim Historical flows by US Army ERDC")
run(rapid_namelist_file='')[source]

Run RAPID program and generate file based on inputs This will generate your rapid_namelist file and run RAPID from wherever you call this script (your working directory).

Parameters:rapid_namelist_file (Optional(str)) – Path of namelist file to use in the simulation. It will be updated with any parameters added to the RAPID manager.

Linux Example:

from RAPIDpy import RAPID

rapid_manager = RAPID(rapid_executable_location='~/work/rapid/src/rapid'
                      use_all_processors=True,
                     )

rapid_manager.update_parameters(rapid_connect_file='../rapid-io/input/rapid_connect.csv',
                                Vlat_file='../rapid-io/input/m3_riv.nc',
                                riv_bas_id_file='../rapid-io/input/riv_bas_id.csv',
                                k_file='../rapid-io/input/k.csv',
                                x_file='../rapid-io/input/x.csv',
                                Qout_file='../rapid-io/output/Qout.nc',
                                )

rapid_manager.update_reach_number_data()
rapid_manager.update_simulation_runtime()
rapid_manager.run(rapid_namelist_file='../rapid-io/input/rapid_namelist')

Linux Reservoir Forcing Flows Example:

from RAPIDpy import RAPID

rapid_manager = RAPID(rapid_executable_location='~/work/rapid/src/rapid',
                      num_processors=4,
                      IS_for_tot=4,
                      IS_for_use=4,
                      for_tot_id_file='../rapid-io/input/dam_id.csv',
                      for_use_id_file='../rapid-io/input/dam_id.csv',
                      Qfor_file='../rapid-io/input/qout_dams.csv',
                      ZS_dtF=86400,
                      BS_opt_for=True,
                      )

rapid_manager.run(rapid_namelist_file='../rapid-io/input/rapid_namelist_regular_run')

Windows with Cygwin Example:

from RAPIDpy import RAPID
from os import path

rapid_manager = RAPID(rapid_executable_location='C:/cygwin64/home/username/work/rapid/run/rapid',
                      cygwin_bin_location='C:/cygwin64/bin',
                      use_all_processors=True,
                      ZS_TauR=24*3600,
                      ZS_dtR=15*60,
                      ZS_TauM=365*24*3600,
                      ZS_dtM=24*3600
                     )

rapid_input_folder = 'C:/cygwin64/home/username/work/rapid-io/input'
rapid_output_folder = 'C:/cygwin64/home/username/work/rapid-io/output'
rapid_manager.update_parameters(rapid_connect_file=path.join(rapid_input_folder, 'rapid_connect.csv'),
                                Vlat_file=path.join(rapid_input_folder, 'm3_riv.nc'),
                                riv_bas_id_file=path.join(rapid_input_folder, 'riv_bas_id.csv'),
                                k_file=path.join(rapid_input_folder, 'k.csv'),
                                x_file=path.join(rapid_input_folder, 'x.csv'),
                                Qout_file=path.join(rapid_output_folder, 'Qout.nc'),
                                )

rapid_manager.update_reach_number_data()
rapid_manager.update_simulation_runtime()
rapid_manager.run()
update_namelist_file(rapid_namelist_file, new_namelist_file=None)[source]

Update existing namelist file with new parameters

Parameters:
  • rapid_namelist_file (str) – Path of namelist file to use in the simulation. It will be updated with any parameters added to the RAPID manager.
  • new_namelist_file (Optional[str]) – Path to output the updated namelist file.
update_parameters(**kwargs)[source]

You can add or update rapid namelist parameters by using the name of the variable in the rapid namelist file (this is case sensitive).

Parameters:**kwargs (Optional[str]) – Keyword arguments matching the input parameters in the RAPID namelist.

Example:

from RAPIDpy import RAPID

rapid_manager = RAPID(
                        #ADD PARAMETERS
                     )

rapid_manager.update_parameters(rapid_connect_file='../rapid-io/input/rapid_connect.csv',
                                Vlat_file='../rapid-io/input/m3_riv.nc',
                                riv_bas_id_file='../rapid-io/input/riv_bas_id.csv',
                                k_file='../rapid-io/input/k.csv',
                                x_file='../rapid-io/input/x.csv',
                                Qout_file='../rapid-io/output/Qout.nc',
                                )
update_reach_number_data()[source]

Update the reach number data for the namelist based on input files.

Warning

You need to make sure you set rapid_connect_file and riv_bas_id_file before running this function.

Example:

from RAPIDpy import RAPID

rapid_manager = RAPID(
                      #ADD PARAMETERS
                      rapid_connect_file='../rapid-io/input/rapid_connect.csv',
                      riv_bas_id_file='../rapid-io/input/riv_bas_id.csv',
                     )

rapid_manager.update_reach_number_data()
update_simulation_runtime()[source]

Updates the total simulation duration from the m3 file (Vlat_file) and the time step (ZS_TauR).

Warning

You need to set the m3 file (Vlat_file) and the time step (ZS_TauR) before runnning this function.

Example:

from RAPIDpy import RAPID

rapid_manager = RAPID(
                      #ADD PARAMETERS
                      Vlat_file='../rapid-io/input/m3_riv.csv',
                      ZS_TauR=3*3600,
                     )

rapid_manager.update_simulation_runtime()