processing flow control (pyrad.flow
)¶
Functions to control the Pyrad data processing flow
main (cfgfile[, starttime, endtime, …]) |
Main flow control. |
main_rt (cfgfile_list[, starttime, endtime, …]) |
main flow control. |
-
pyrad.flow.
main
(cfgfile, starttime=None, endtime=None, trajfile='', trajtype='plane', flashnr=0, infostr='', MULTIPROCESSING_DSET=False, MULTIPROCESSING_PROD=False, PROFILE_MULTIPROCESSING=False, USE_CHILD_PROCESS=False)[source]¶ Main flow control. Processes radar data off-line over a period of time given either by the user, a trajectory file, or determined by the last volume processed and the current time. Multiple radars can be processed simultaneously
Parameters: - cfgfile : str
path of the main config file
- starttime, endtime : datetime object
start and end time of the data to be processed
- trajfile : str
path to file describing the trajectory
- trajtype : str
type of trajectory file. Can be either ‘plane’, ‘lightning’ or ‘proc_periods’
- flashnr : int
If larger than 0 will select a flash in a lightning trajectory file. If 0 the data corresponding to the trajectory of all flashes will be plotted
- infostr : str
Information string about the actual data processing (e.g. ‘RUN57’). This string is added to product files.
- MULTIPROCESSING_DSET : Bool
If true the generation of datasets at the same processing level will be parallelized
- MULTIPROCESSING_PROD : Bool
If true the generation of products from each dataset will be parallelized
- PROFILE_MULTIPROCESSING : Bool
If true and code parallelized the multiprocessing is profiled
- USE_CHILD_PROCESS : Bool
If true the reading and processing of the data will be performed by a child process controlled by dask. This is done to make sure all memory used is released.
-
pyrad.flow.
main_rt
(cfgfile_list, starttime=None, endtime=None, infostr_list=None, proc_period=60, proc_finish=None)[source]¶ main flow control. Processes radar data in real time. The start and end processing times can be determined by the user. This function is inteded for a single radar
Parameters: - cfgfile_list : list of str
path of the main config files
- starttime, endtime : datetime object
start and end time of the data to be processed
- infostr_list : list of str
Information string about the actual data processing (e.g. ‘RUN57’). This string is added to product files.
- proc_period : int
period of time before starting a new processing round (seconds)
- proc_finish : int or None
if set to a value the program will be forced to shut down after the value (in seconds) from start time has been exceeded
Returns: - end_proc : Boolean
If true the program has ended successfully