processing flow control (pyrad.flow)

Functions to control the Pyrad data processing flow

main(cfgfile[, starttime, endtime, …]) Main flow control.
main_rt(cfgfile_list[, starttime, endtime, …]) main flow control.
pyrad.flow.main(cfgfile, starttime=None, endtime=None, trajfile='', trajtype='plane', flashnr=0, infostr='', MULTIPROCESSING_DSET=False, MULTIPROCESSING_PROD=False, PROFILE_MULTIPROCESSING=False, USE_CHILD_PROCESS=False)[source]

Main flow control. Processes radar data off-line over a period of time given either by the user, a trajectory file, or determined by the last volume processed and the current time. Multiple radars can be processed simultaneously

Parameters:
cfgfile : str

path of the main config file

starttime, endtime : datetime object

start and end time of the data to be processed

trajfile : str

path to file describing the trajectory

trajtype : str

type of trajectory file. Can be either ‘plane’, ‘lightning’ or ‘proc_periods’

flashnr : int

If larger than 0 will select a flash in a lightning trajectory file. If 0 the data corresponding to the trajectory of all flashes will be plotted

infostr : str

Information string about the actual data processing (e.g. ‘RUN57’). This string is added to product files.

MULTIPROCESSING_DSET : Bool

If true the generation of datasets at the same processing level will be parallelized

MULTIPROCESSING_PROD : Bool

If true the generation of products from each dataset will be parallelized

PROFILE_MULTIPROCESSING : Bool

If true and code parallelized the multiprocessing is profiled

USE_CHILD_PROCESS : Bool

If true the reading and processing of the data will be performed by a child process controlled by dask. This is done to make sure all memory used is released.

pyrad.flow.main_rt(cfgfile_list, starttime=None, endtime=None, infostr_list=None, proc_period=60, proc_finish=None)[source]

main flow control. Processes radar data in real time. The start and end processing times can be determined by the user. This function is inteded for a single radar

Parameters:
cfgfile_list : list of str

path of the main config files

starttime, endtime : datetime object

start and end time of the data to be processed

infostr_list : list of str

Information string about the actual data processing (e.g. ‘RUN57’). This string is added to product files.

proc_period : int

period of time before starting a new processing round (seconds)

proc_finish : int or None

if set to a value the program will be forced to shut down after the value (in seconds) from start time has been exceeded

Returns:
end_proc : Boolean

If true the program has ended successfully