smrf.data package

Submodules

smrf.data.loadData module

class smrf.data.loadData.wxdata(dataConfig, start_date, end_date, time_zone='UTC', dataType=None)[source]

Bases: object

Class for loading and storing the data, either from - CSV file - MySQL database - Add other sources here

Inputs to data() are: - dataConfig, either the [csv] or [mysql] section - start_date, datetime object - end_date, datetime object - dataType, either ‘csv’ or ‘mysql’

The data will be loaded into a Pandas dataframe

db_config_vars = ['user', 'password', 'host', 'database', 'port', 'metadata', 'data_table', 'station_table', 'stations', 'client']
load_from_csv()[source]

Load the data from a csv file Fields that are operated on - metadata -> dictionary, one for each station, must have at least the following: primary_id, X, Y, elevation - csv data files -> dictionary, one for each time step, must have at least the following columns: date_time, column names matching metadata.primary_id

load_from_mysql()[source]

Load the data from a mysql database

variables = ['air_temp', 'vapor_pressure', 'precip', 'wind_speed', 'wind_direction', 'cloud_factor']

smrf.data.loadGrid module

smrf.data.loadGrid.apply_utm(s, force_zone_number)[source]

Calculate the utm from lat/lon for a series

Parameters

  • s – pandas series with fields latitude and longitude

  • force_zone_number – default None, zone number to force to

Returns

pandas series with fields ‘X’ and ‘Y’ filled

Return type

s

class smrf.data.loadGrid.grid(dataConfig, topo, start_date, end_date, time_zone='UTC', dataType='wrf', tempDir=None, forecast_flag=False, day_hour=0, n_forecast_hours=18)[source]

Bases: object

Class for loading and storing the data, either from a gridded dataset in: - NetCDF format - other format

Inputs to data() are: - dataConfig, from the [gridded] section - start_date, datetime object - end_date, datetime object

get_latlon(utm_x, utm_y)[source]

Convert UTM coords to Latitude and longitude

Parameters

  • utm_x – UTM easting in meters in the same zone/letter as the topo

  • utm_y – UTM Northing in meters in the same zone/letter as the topo

Returns

(lat,lon) latitude and longitude conversion from the UTM

coordinates

Return type

tuple

load_from_hrrr()[source]

Load the data from the High Resolution Rapid Refresh (HRRR) model The variables returned from the HRRR class in dataframes are

  • metadata

  • air_temp

  • relative_humidity

  • precip_int

  • cloud_factor

  • wind_u

  • wind_v

The function will take the keys and load them into the appropriate objects within the grid class. The vapor pressure will be calculated from the air_temp and relative_humidity. The wind_speed and wind_direction will be calculated from wind_u and wind_v

load_from_netcdf()[source]

Load the data from a generic netcdf file

Parameters

  • lat – latitude field in file, 1D array

  • lon – longitude field in file, 1D array

  • elev – elevation field in file, 2D array

  • variable – variable name in file, 3D array

load_from_wrf()[source]

Load the data from a netcdf file. This was setup to work with a WRF output file, i.e. wrf_out so it’s going to look for the following variables: - Times - XLAT - XLONG - HGT - T2 - DWPT - GLW - RAINNC - CLDFRA - UGRD - VGRD

Each cell will be identified by grid_IX_IY

model_domain_grid()[source]

Retrieve the bounding box for the gridded data by adding a buffer to the extents of the topo domain.

Returns

(dlat, dlon) Domain latitude and longitude extents

Return type

tuple

smrf.data.loadTopo module

class smrf.data.loadTopo.Topo(topoConfig, calcInput=True, tempDir=None)[source]

Bases: object

Class for topo images and processing those images. Images are: - DEM - Mask - veg type - veg height - veg k - veg tau

Inputs to topo are the topo section of the config file topo will guess the location of the WORKDIR env variable and should work for unix systems.

topoConfig

configuration for topo

tempDir

location of temporary working directory

dem

numpy array for the DEM

mask

numpy array for the mask

veg_type

numpy array for the veg type

veg_height

numpy array for the veg height

veg_k

numpy array for the veg K

veg_tau

numpy array for the veg transmissivity

sky_view
ny

number of columns in DEM

nx

number of rows in DEM

u,v

location of upper left corner

du, dv

step size of grid

unit

geo header units of grid

coord_sys_ID

coordinate syste,

x,y

position vectors

X,Y

position grid

stoporad_in

numpy array for the sky view factor

IMAGES = ['dem', 'mask', 'veg_type', 'veg_height', 'veg_k', 'veg_tau']
get_center(ds, mask_name=None)[source]

Function returns the basin center in the native coordinates of the a netcdf object.

The incoming data set must contain at least and x, y and optionally whatever mask name the user would like to use for calculating . If no mask name is provided then the entire domain is used.

Parameters

  • ds – netCDF4.Dataset object containing at least x,y, optionally a mask variable name

  • mask_name – variable name in the dataset that is a mask where 1 is in the mask

Returns

x,y of the data center in the datas native coordinates

Return type

tuple

gradient(gfile)[source]

Calculate the gradient and aspect

Parameters

gfile – IPW file to write the results to

readNetCDF()[source]

Read in the images from the config file where the file listed is in netcdf format

stoporadInput()[source]

Calculate the necessary input file for stoporad The IPW and WORKDIR environment variables must be set

smrf.data.mysql_data module

Created on Dec 22, 2015

Read in metadata and data from a MySQL database The table columns will most likely be hardcoded for ease of development and users will require the specific table setup

class smrf.data.mysql_data.database(user, password, host, db, port)[source]

Bases: object

Database class for querying metadata and station data

get_data(table, station_ids, start_date, end_date, variables, time_zone='UTC')[source]

Get data from the database, either for the specified stations or for the specific group of stations in client

Parameters

  • table – table to load data from

  • station_ids – list of station ids to get

  • start_date – start of time period

  • end_date – end of time period

  • variable – string for variable to get

  • time_zone – String timezone to set the data in

metadata(table, station_ids=None, client=None, station_table=None)[source]

Similar to the CorrectWxData database call Get the metadata from the database for either the specified stations or for the specific group of stations in client

Parameters

  • table – metadata table in the database

  • station_id – list of stations to read, default None

  • client – client to read from the station_table, default None

  • station_table – table name that contains the clients and list of stations, default None

Returns

Pandas DataFrame of station information

Return type

d

query(query, params)[source]
smrf.data.mysql_data.date_range(start_date, end_date, increment)[source]

Calculate a list between start and end date with an increment

Module contents

API Documentation smrf.distribute package