smrf.utils package

smrf.utils.queue module

Create classes for running on multiple threads

20160323 Scott Havens

class smrf.utils.queue.DateQueue_Threading(maxsize=0, timeout=None)

Bases: queue.Queue

DateQueue extends Queue.Queue module Stores the items in a dictionary with date_time keys When values are retrieved, it will not remove them and will require cleaning at the end to not have to many values

20160323 Scott Havens

clean(index)

Need to clean it out so mimic the original get

get(index, block=True, timeout=None)

Remove and return an item from the queue.

If optional args ‘block’ is true and ‘timeout’ is None (the default), block if necessary until an item is available. If ‘timeout’ is a non-negative number, it blocks at most ‘timeout’ seconds and raises the Empty exception if no item was available within that time. Otherwise (‘block’ is false), return an item if one is immediately available, else raise the Empty exception (‘timeout’ is ignored in that case).

This is from queue.Queue but with modifcation for supplying what to get

put(item, block=True, timeout=None)

Put an item into the queue.

If optional args ‘block’ is true and ‘timeout’ is None (the default), block if necessary until a free slot is available. If ‘timeout’ is a non-negative number, it blocks at most ‘timeout’ seconds and raises the Full exception if no free slot was available within that time. Otherwise (‘block’ is false), put an item on the queue if a free slot is immediately available, else raise the Full exception (‘timeout’ is ignored in that case).

class smrf.utils.queue.QueueCleaner(date_time, queue)

Bases: threading.Thread

QueueCleaner that will go through all the queues and check if they all have a date in common. When this occurs, all the threads will have processed that time step and it’s not longer needed

run()

Go through the date times and look for when all the queues have that date_time

class smrf.utils.queue.QueueOutput(queue, date_time, out_func, out_frequency, nx, ny)

Bases: threading.Thread

Takes values from the queue and outputs using ‘out_func’

run()

Output the desired variables to a file.

Go through the date times and look for when all the queues have that date_time

smrf.utils.utils module

20160104 Scott Havens

Collection of utility functions

class smrf.utils.utils.CheckStation(**kwargs)

Bases: inicheck.checkers.CheckType

Custom check for ensuring our stations are always capitalized

cast()

smrf.utils.utils.backup_input(data, config_obj)

Backs up input data files so a user can rerun a run with the exact data used for a run.

Parameters:

• data – Pandas dataframe containing the station data
• config_obj – The config object produced by inicheck

smrf.utils.utils.check_station_colocation(metadata_csv=None, metadata=None)

Takes in a data frame representing the metadata for the weather stations as produced by smrf.framework.model_framework.SMRF.loadData and check to see if any stations have the same location.

Parameters:

• metadata_csv – CSV containing the metdata for weather stations
• metadata – Pandas Dataframe containing the metdata for weather stations

Returns:

list of station primary_id that are colocated

Return type:

repeat_sta

smrf.utils.utils.find_configs(directory)

Searches through a directory and returns all the .ini fulll filenames.

Parameters:directory – string path to directory. Returns:list of paths pointing to the config file. Return type:configs

smrf.utils.utils.getConfigHeader()

Generates string for inicheck to add to config files

Returns:string for cfg headers Return type:cfg_str

smrf.utils.utils.get_asc_stats(fp)

Returns header of ascii dem file

smrf.utils.utils.get_config_doc_section_hdr()

Returns the header dictionary for linking modules in smrf to the documentation generated by inicheck auto doc functions

smrf.utils.utils.getgitinfo()

gitignored file that contains specific SMRF version and path

Returns:git version from ‘git describe’ Return type:str

smrf.utils.utils.getqotw()

smrf.utils.utils.grid_interpolate(values, vtx, wts, shp, fill_value=nan)

Broken out gridded interpolation from scipy.interpolate.griddata that takes the vertices and wts from interp_weights function

Parameters:

• values – flattened WindNinja wind speeds
• vtx – vertices for interpolation
• wts – weights for interpolation
• shape – shape of SMRF grid
• fill_value – value for extrapolated points

Returns:

interpolated values

Return type:

ret

smrf.utils.utils.grid_interpolate_deconstructed(tri, values, grid_points, method='linear')

Underlying methods from scipy grid_data broken out to pass in the tri values returned from qhull.Delaunay. This is done to improve the speed of using grid_data

Parameters:

• tri – values returned from qhull.Delaunay
• values – values at HRRR stations generally
• grid_points – tuple of vectors for X,Y coords of grid stations
• method – either linear or cubic

Returns:

result of interpolation to gridded points

smrf.utils.utils.handle_run_script_options(config_option)

Handle function for dealing with args in the SMRF run script

Parameters:config_option – string path to a directory or a specific config file. Returns:Full path to an existing config file. Return type:configFile

smrf.utils.utils.interp_weights(xy, uv, d=2)

Find vertices and weights of LINEAR interpolation for gridded interp. This routine follows the methods of scipy.interpolate.griddata as outlined here: https://stackoverflow.com/questions/20915502/speedup-scipy-griddata-for-multiple-interpolations-between-two-irregular-grids This function finds the vertices and weights which is the most computationally expensive part of the routine. The interpolateion can then be done quickly.

Parameters:

• xy – n by 2 array of flattened meshgrid x and y coords of WindNinja grid
• uv – n by 2 array of flattened meshgrid x and y coords of SMRF grid
• d – dimensions of array (i.e. 2 for our purposes)

Returns:

wts:

Return type:

vertices

smrf.utils.utils.is_leap_year(year)

smrf.utils.utils.nan_helper(y)

Helper to handle indices and logical indices of NaNs.

Example

>>> # linear interpolation of NaNs
>>> nans, x= nan_helper(y)
>>> y[nans]= np.interp(x(nans), x(~nans), y[~nans])

Parameters:y – 1d numpy array with possible NaNs Returns:nans - logical indices of NaNs index - a function, with signature

indices=index(logical_indices) to convert logical indices of NaNs to ‘equivalent’ indices

Return type:tuple

smrf.utils.utils.set_min_max(data, min_val, max_val)

Ensure that the data is in the bounds of min and max

Parameters:

• data – numpy array of data to be min/maxed
• min_val – minimum threshold to trim data
• max_val – Maximum threshold to trim data

Returns:

numpy array of data trimmed at min_val and max_val

Return type:

data

smrf.utils.utils.water_day(indate)

Determine the decimal day in the water year

Parameters:indate – datetime object Returns:dd - decimal day from start of water year wy - Water year Return type:tuple

20160105 Scott Havens

smrf.utils.wind package

• smrf.utils.wind_model package
  • smrf.utils.wind.model module
  • smrf.utils.wind.wind_c module

Module contents