Running RAPID¶
Tutorial¶
Step 1: Initialize the RAPID manager class.¶
- First, add the path to you rapid executable location.
- Next, you need to either tell it how many processors to use using the num_processors input variable or to use all available processors set use_all_processors to true.
- After that, add any other parameters you would like to use that would normally be in the rapid namelist file (this is case sensitive).
-
class
RAPIDpy.rapid.
RAPID
(rapid_executable_location='', num_processors=1, use_all_processors=False, cygwin_bin_location='', mpiexec_command='mpiexec', ksp_type='richardson', **kwargs)[source]¶ This class is designed to prepare the rapid_namelist file and run the RAPID program. There are also other utilities added.
-
rapid_executable_location
¶ Optional[str] – Path to the RAPID executable location.
-
num_processors
¶ Optional[int] – Number of procesors to use. Default is 1. Overridden if use_all_processors is True.
-
use_all_processors
¶ Optional[bool] – If set to True, the RAPID program will use all available processors. Default is False.
-
cygwin_bin_location
¶ Optional[str] – If using Windows, this is the path to the Cygwin ‘bin’ directory.
-
mpiexec_command
¶ Optional[str] – This is the mpi execute commmand. Default is “mpiexec”.
-
ksp_type
¶ Optional[str] – This is the solver type. Default is “richardson”.
-
**kwargs
Optional[str] – Keyword arguments matching the input parameters in the RAPID namelist.
Linux Example:
from RAPIDpy import RAPID rapid_manager = RAPID(rapid_executable_location='~/work/rapid/run/rapid' use_all_processors=True, ZS_TauR=24*3600, #duration of routing procedure (time step of runoff data) ZS_dtR=15*60, #internal routing time step ZS_TauM=365*24*3600, #total simulation time ZS_dtM=24*3600 #input time step )
Windows with Cygwin Example:
from RAPIDpy import RAPID rapid_manager = RAPID(rapid_executable_location='C:/cygwin64/home/username/work/rapid/run/rapid', cygwin_bin_location='C:/cygwin64/bin', use_all_processors=True, ZS_TauR=24*3600, #duration of routing procedure (time step of runoff data) ZS_dtR=15*60, #internal routing time step ZS_TauM=365*24*3600, #total simulation time ZS_dtM=24*3600 #input time step )
-
Step 2 (optional): Add/update additional namelist parameters later¶
-
RAPID.
update_parameters
(**kwargs)[source]¶ You can add or update rapid namelist parameters by using the name of the variable in the rapid namelist file (this is case sensitive).
Parameters: **kwargs (Optional[str]) – Keyword arguments matching the input parameters in the RAPID namelist. Example:
from RAPIDpy import RAPID rapid_manager = RAPID( #ADD PARAMETERS ) rapid_manager.update_parameters(rapid_connect_file='../rapid-io/input/rapid_connect.csv', Vlat_file='../rapid-io/input/m3_riv.nc', riv_bas_id_file='../rapid-io/input/riv_bas_id.csv', k_file='../rapid-io/input/k.csv', x_file='../rapid-io/input/x.csv', Qout_file='../rapid-io/output/Qout.nc', )
Step 3 (optional): Update reach number data¶
-
RAPID.
update_reach_number_data
()[source]¶ Update the reach number data for the namelist based on input files.
Warning
You need to make sure you set rapid_connect_file and riv_bas_id_file before running this function.
Example:
from RAPIDpy import RAPID rapid_manager = RAPID( #ADD PARAMETERS rapid_connect_file='../rapid-io/input/rapid_connect.csv', riv_bas_id_file='../rapid-io/input/riv_bas_id.csv', ) rapid_manager.update_reach_number_data()
Step 4 (optional): Update simulation runtime data¶
-
RAPID.
update_simulation_runtime
()[source]¶ Updates the total simulation duration from the m3 file (Vlat_file) and the time step (ZS_TauR).
Warning
You need to set the m3 file (Vlat_file) and the time step (ZS_TauR) before runnning this function.
Example:
from RAPIDpy import RAPID rapid_manager = RAPID( #ADD PARAMETERS Vlat_file='../rapid-io/input/m3_riv.csv', ZS_TauR=3*3600, ) rapid_manager.update_simulation_runtime()
Step 5: Run RAPID¶
-
RAPID.
run
(rapid_namelist_file='')[source]¶ Run RAPID program and generate file based on inputs This will generate your rapid_namelist file and run RAPID from wherever you call this script (your working directory).
Parameters: rapid_namelist_file (Optional(str)) – Path of namelist file to use in the simulation. It will be updated with any parameters added to the RAPID manager. Linux Example:
from RAPIDpy import RAPID rapid_manager = RAPID(rapid_executable_location='~/work/rapid/src/rapid' use_all_processors=True, ) rapid_manager.update_parameters(rapid_connect_file='../rapid-io/input/rapid_connect.csv', Vlat_file='../rapid-io/input/m3_riv.nc', riv_bas_id_file='../rapid-io/input/riv_bas_id.csv', k_file='../rapid-io/input/k.csv', x_file='../rapid-io/input/x.csv', Qout_file='../rapid-io/output/Qout.nc', ) rapid_manager.update_reach_number_data() rapid_manager.update_simulation_runtime() rapid_manager.run(rapid_namelist_file='../rapid-io/input/rapid_namelist')
Linux Reservoir Forcing Flows Example:
from RAPIDpy import RAPID rapid_manager = RAPID(rapid_executable_location='~/work/rapid/src/rapid', num_processors=4, IS_for_tot=4, IS_for_use=4, for_tot_id_file='../rapid-io/input/dam_id.csv', for_use_id_file='../rapid-io/input/dam_id.csv', Qfor_file='../rapid-io/input/qout_dams.csv', ZS_dtF=86400, BS_opt_for=True, ) rapid_manager.run(rapid_namelist_file='../rapid-io/input/rapid_namelist_regular_run')
Windows with Cygwin Example:
from RAPIDpy import RAPID from os import path rapid_manager = RAPID(rapid_executable_location='C:/cygwin64/home/username/work/rapid/run/rapid', cygwin_bin_location='C:/cygwin64/bin', use_all_processors=True, ZS_TauR=24*3600, ZS_dtR=15*60, ZS_TauM=365*24*3600, ZS_dtM=24*3600 ) rapid_input_folder = 'C:/cygwin64/home/username/work/rapid-io/input' rapid_output_folder = 'C:/cygwin64/home/username/work/rapid-io/output' rapid_manager.update_parameters(rapid_connect_file=path.join(rapid_input_folder, 'rapid_connect.csv'), Vlat_file=path.join(rapid_input_folder, 'm3_riv.nc'), riv_bas_id_file=path.join(rapid_input_folder, 'riv_bas_id.csv'), k_file=path.join(rapid_input_folder, 'k.csv'), x_file=path.join(rapid_input_folder, 'x.csv'), Qout_file=path.join(rapid_output_folder, 'Qout.nc'), ) rapid_manager.update_reach_number_data() rapid_manager.update_simulation_runtime() rapid_manager.run()
Step 6 (optional): Convert RAPID output to be CF Compliant¶
-
RAPID.
make_output_CF_compliant
(simulation_start_datetime, comid_lat_lon_z_file='', project_name='Normal RAPID project')[source]¶ This function converts the RAPID output to be CF compliant. This will require a comid_lat_lon_z.csv file (See:
FlowlineToPoint()
to generate the file).Note
It prepends time an initial flow to your simulation from the qinit_file. If no qinit file is given, an initial value of zero is added.
Warning
This will delete your original Qout file.
Parameters: - simulation_start_datetime (datetime) – Datetime object with the start date of the simulation.
- comid_lat_lon_z_file (Optional[str]) – Path to the comid_lat_lon_z.csv file. If none given, spatial information will be skipped.
- project_name (Optional[str]) – Name of project to add to the RAPID output file.
Example:
from RAPIDpy import RAPID rapid_manager = RAPID(rapid_executable_location='~/work/rapid/run/rapid' use_all_processors=True, ZS_TauR=24*3600, ZS_dtR=15*60, ZS_TauM=365*24*3600, ZS_dtM=24*3600 rapid_connect_file='../rapid-io/input/rapid_connect.csv', Vlat_file='../rapid-io/input/m3_riv.nc', riv_bas_id_file='../rapid-io/input/riv_bas_id.csv', k_file='../rapid-io/input/k.csv', x_file='../rapid-io/input/x.csv', Qout_file='../rapid-io/output/Qout.nc', ) rapid_manager.run() rapid_manager.make_output_CF_compliant(simulation_start_datetime=datetime.datetime(1980, 1, 1), comid_lat_lon_z_file='../rapid-io/input/comid_lat_lon_z.csv', project_name="ERA Interim Historical flows by US Army ERDC")
Full API Description¶
-
class
RAPIDpy.rapid.
RAPID
(rapid_executable_location='', num_processors=1, use_all_processors=False, cygwin_bin_location='', mpiexec_command='mpiexec', ksp_type='richardson', **kwargs)[source] This class is designed to prepare the rapid_namelist file and run the RAPID program. There are also other utilities added.
-
rapid_executable_location
Optional[str] – Path to the RAPID executable location.
-
num_processors
Optional[int] – Number of procesors to use. Default is 1. Overridden if use_all_processors is True.
-
use_all_processors
Optional[bool] – If set to True, the RAPID program will use all available processors. Default is False.
-
cygwin_bin_location
Optional[str] – If using Windows, this is the path to the Cygwin ‘bin’ directory.
-
mpiexec_command
Optional[str] – This is the mpi execute commmand. Default is “mpiexec”.
-
ksp_type
Optional[str] – This is the solver type. Default is “richardson”.
-
**kwargs
Optional[str] – Keyword arguments matching the input parameters in the RAPID namelist.
Linux Example:
from RAPIDpy import RAPID rapid_manager = RAPID(rapid_executable_location='~/work/rapid/run/rapid' use_all_processors=True, ZS_TauR=24*3600, #duration of routing procedure (time step of runoff data) ZS_dtR=15*60, #internal routing time step ZS_TauM=365*24*3600, #total simulation time ZS_dtM=24*3600 #input time step )
Windows with Cygwin Example:
from RAPIDpy import RAPID rapid_manager = RAPID(rapid_executable_location='C:/cygwin64/home/username/work/rapid/run/rapid', cygwin_bin_location='C:/cygwin64/bin', use_all_processors=True, ZS_TauR=24*3600, #duration of routing procedure (time step of runoff data) ZS_dtR=15*60, #internal routing time step ZS_TauM=365*24*3600, #total simulation time ZS_dtM=24*3600 #input time step )
-
generate_namelist_file
(rapid_namelist_file)[source]¶ Generate rapid_namelist file.
Parameters: rapid_namelist_file (str) – Path of namelist file to generate from parameters added to the RAPID manager.
-
generate_qinit_from_past_qout
(qinit_file, time_index=-1, out_datetime=None)[source]¶ Generate qinit from a RAPID qout file
Parameters: - qinit_file (str) – Path to output qinit_file.
- time_index (Optional[int]) – Index of simulation to generate initial flow file. Default is the end.
- out_datetime (Optional[datetime]) – Datetime object containing time of initialization.
Example:
from RAPIDpy import RAPID rapid_manager = RAPID(Qout_file='/output_mississippi-nfie/Qout_k2v1_2005to2009.nc', rapid_connect_file='/input_mississippi_nfie/rapid_connect_ECMWF.csv' ) rapid_manager.generate_qinit_from_past_qout(qinit_file='/input_mississippi_nfie/Qinit_2008_flood.csv', time_index=10162)
-
generate_seasonal_intitialization
(qinit_file, datetime_start_initialization=datetime.datetime(2017, 7, 6, 18, 35, 48, 418889))[source]¶ This creates a seasonal qinit file from a RAPID qout file. This requires a simulation Qout file with a longer time period of record and to be CF compliant. It takes the average of the current date +- 3 days and goes back as far as possible.
Parameters: - qinit_file (str) – Path to output qinit_file.
- datetime_start_initialization (Optional[datetime]) – Datetime object with date of simulation to go back through the years and get a running average to generate streamflow initialization. Default is utcnow.
This example shows how to use it:
from RAPIDpy.rapid import RAPID rapid_manager = RAPID(Qout_file='/output_mississippi-nfie/Qout_2000to2015.nc', rapid_connect_file='/input_mississippi_nfie/rapid_connect_ECMWF.csv' ) rapid_manager.generate_seasonal_intitialization(qinit_file='/input_mississippi_nfie/Qinit_seasonal_avg_jan_1.csv')
-
generate_usgs_avg_daily_flows_opt
(reach_id_gage_id_file, start_datetime, end_datetime, out_streamflow_file, out_stream_id_file)[source]¶ Generate daily streamflow file and stream id file required for calibration or for substituting flows based on USGS gage ids associated with stream ids.
Parameters: - reach_id_gage_id_file (str) – Path to reach_id_gage_id file.
- start_datetime (datetime) – A datetime object with the start date to download data.
- end_datetime (datetime) – A datetime object with the end date to download data.
- out_streamflow_file (str) – The path to output the streamflow file for RAPID.
- out_stream_id_file (str) – The path to output the stream ID file associated with the streamflow file for RAPID.
Example reach_id_gage_id_file:
COMID, USGS_GAGE_ID 2000, 503944 ...
Warning
Overuse will get you blocked from downloading data from USGS.
Warning
This code does not clean the data in any way. Thus, you are likely to run into issues if you simply use the raw data.
Warning
The code skips gages that do not have data for the entire time period.
Simple Example:
import datetime from os.path import join from RAPIDpy import RAPID main_path = "/home/username/data" rapid_manager = RAPID() rapid_manager.generate_usgs_avg_daily_flows_opt(reach_id_gage_id_file=join(main_path,"mississippi_usgsgage_id_comid.csv"), start_datetime=datetime.datetime(2000,1,1), end_datetime=datetime.datetime(2014,12,31), out_streamflow_file=join(main_path,"streamflow_2000_2014.csv"), out_stream_id_file=join(main_path,"streamid_2000_2014.csv"))
Complex Example:
import datetime from os.path import join from RAPIDpy import RAPID main_path = "/home/username/data" rapid_manager = RAPID(rapid_executable_location='~/work/rapid/run/rapid' use_all_processors=True, ZS_TauR=24*3600, ZS_dtR=15*60, ZS_TauM=365*24*3600, ZS_dtM=24*3600 ) rapid_manager.update_parameters(rapid_connect_file='../rapid-io/input/rapid_connect.csv', Vlat_file='../rapid-io/input/m3_riv.nc', riv_bas_id_file='../rapid-io/input/riv_bas_id.csv', k_file='../rapid-io/input/k.csv', x_file='../rapid-io/input/x.csv', Qout_file='../rapid-io/output/Qout.nc', ) rapid_manager.update_reach_number_data() rapid_manager.update_simulation_runtime() rapid_manager.generate_usgs_avg_daily_flows_opt(reach_id_gage_id_file=join(main_path,"mississippi_usgsgage_id_comid.csv"), start_datetime=datetime.datetime(2000,1,1), end_datetime=datetime.datetime(2014,12,31), out_streamflow_file=join(main_path,"streamflow_2000_2014.csv"), out_stream_id_file=join(main_path,"streamid_2000_2014.csv")) rapid_manager.run()
-
make_output_CF_compliant
(simulation_start_datetime, comid_lat_lon_z_file='', project_name='Normal RAPID project')[source] This function converts the RAPID output to be CF compliant. This will require a comid_lat_lon_z.csv file (See:
FlowlineToPoint()
to generate the file).Note
It prepends time an initial flow to your simulation from the qinit_file. If no qinit file is given, an initial value of zero is added.
Warning
This will delete your original Qout file.
Parameters: - simulation_start_datetime (datetime) – Datetime object with the start date of the simulation.
- comid_lat_lon_z_file (Optional[str]) – Path to the comid_lat_lon_z.csv file. If none given, spatial information will be skipped.
- project_name (Optional[str]) – Name of project to add to the RAPID output file.
Example:
from RAPIDpy import RAPID rapid_manager = RAPID(rapid_executable_location='~/work/rapid/run/rapid' use_all_processors=True, ZS_TauR=24*3600, ZS_dtR=15*60, ZS_TauM=365*24*3600, ZS_dtM=24*3600 rapid_connect_file='../rapid-io/input/rapid_connect.csv', Vlat_file='../rapid-io/input/m3_riv.nc', riv_bas_id_file='../rapid-io/input/riv_bas_id.csv', k_file='../rapid-io/input/k.csv', x_file='../rapid-io/input/x.csv', Qout_file='../rapid-io/output/Qout.nc', ) rapid_manager.run() rapid_manager.make_output_CF_compliant(simulation_start_datetime=datetime.datetime(1980, 1, 1), comid_lat_lon_z_file='../rapid-io/input/comid_lat_lon_z.csv', project_name="ERA Interim Historical flows by US Army ERDC")
-
run
(rapid_namelist_file='')[source] Run RAPID program and generate file based on inputs This will generate your rapid_namelist file and run RAPID from wherever you call this script (your working directory).
Parameters: rapid_namelist_file (Optional(str)) – Path of namelist file to use in the simulation. It will be updated with any parameters added to the RAPID manager. Linux Example:
from RAPIDpy import RAPID rapid_manager = RAPID(rapid_executable_location='~/work/rapid/src/rapid' use_all_processors=True, ) rapid_manager.update_parameters(rapid_connect_file='../rapid-io/input/rapid_connect.csv', Vlat_file='../rapid-io/input/m3_riv.nc', riv_bas_id_file='../rapid-io/input/riv_bas_id.csv', k_file='../rapid-io/input/k.csv', x_file='../rapid-io/input/x.csv', Qout_file='../rapid-io/output/Qout.nc', ) rapid_manager.update_reach_number_data() rapid_manager.update_simulation_runtime() rapid_manager.run(rapid_namelist_file='../rapid-io/input/rapid_namelist')
Linux Reservoir Forcing Flows Example:
from RAPIDpy import RAPID rapid_manager = RAPID(rapid_executable_location='~/work/rapid/src/rapid', num_processors=4, IS_for_tot=4, IS_for_use=4, for_tot_id_file='../rapid-io/input/dam_id.csv', for_use_id_file='../rapid-io/input/dam_id.csv', Qfor_file='../rapid-io/input/qout_dams.csv', ZS_dtF=86400, BS_opt_for=True, ) rapid_manager.run(rapid_namelist_file='../rapid-io/input/rapid_namelist_regular_run')
Windows with Cygwin Example:
from RAPIDpy import RAPID from os import path rapid_manager = RAPID(rapid_executable_location='C:/cygwin64/home/username/work/rapid/run/rapid', cygwin_bin_location='C:/cygwin64/bin', use_all_processors=True, ZS_TauR=24*3600, ZS_dtR=15*60, ZS_TauM=365*24*3600, ZS_dtM=24*3600 ) rapid_input_folder = 'C:/cygwin64/home/username/work/rapid-io/input' rapid_output_folder = 'C:/cygwin64/home/username/work/rapid-io/output' rapid_manager.update_parameters(rapid_connect_file=path.join(rapid_input_folder, 'rapid_connect.csv'), Vlat_file=path.join(rapid_input_folder, 'm3_riv.nc'), riv_bas_id_file=path.join(rapid_input_folder, 'riv_bas_id.csv'), k_file=path.join(rapid_input_folder, 'k.csv'), x_file=path.join(rapid_input_folder, 'x.csv'), Qout_file=path.join(rapid_output_folder, 'Qout.nc'), ) rapid_manager.update_reach_number_data() rapid_manager.update_simulation_runtime() rapid_manager.run()
-
update_namelist_file
(rapid_namelist_file, new_namelist_file=None)[source]¶ Update existing namelist file with new parameters
Parameters: - rapid_namelist_file (str) – Path of namelist file to use in the simulation. It will be updated with any parameters added to the RAPID manager.
- new_namelist_file (Optional[str]) – Path to output the updated namelist file.
-
update_parameters
(**kwargs)[source] You can add or update rapid namelist parameters by using the name of the variable in the rapid namelist file (this is case sensitive).
Parameters: **kwargs (Optional[str]) – Keyword arguments matching the input parameters in the RAPID namelist. Example:
from RAPIDpy import RAPID rapid_manager = RAPID( #ADD PARAMETERS ) rapid_manager.update_parameters(rapid_connect_file='../rapid-io/input/rapid_connect.csv', Vlat_file='../rapid-io/input/m3_riv.nc', riv_bas_id_file='../rapid-io/input/riv_bas_id.csv', k_file='../rapid-io/input/k.csv', x_file='../rapid-io/input/x.csv', Qout_file='../rapid-io/output/Qout.nc', )
-
update_reach_number_data
()[source] Update the reach number data for the namelist based on input files.
Warning
You need to make sure you set rapid_connect_file and riv_bas_id_file before running this function.
Example:
from RAPIDpy import RAPID rapid_manager = RAPID( #ADD PARAMETERS rapid_connect_file='../rapid-io/input/rapid_connect.csv', riv_bas_id_file='../rapid-io/input/riv_bas_id.csv', ) rapid_manager.update_reach_number_data()
-
update_simulation_runtime
()[source] Updates the total simulation duration from the m3 file (Vlat_file) and the time step (ZS_TauR).
Warning
You need to set the m3 file (Vlat_file) and the time step (ZS_TauR) before runnning this function.
Example:
from RAPIDpy import RAPID rapid_manager = RAPID( #ADD PARAMETERS Vlat_file='../rapid-io/input/m3_riv.csv', ZS_TauR=3*3600, ) rapid_manager.update_simulation_runtime()
-