Another use case would be the same as above, except that the tuples in the MultiArray do not represent a regular grid but just arbitrary lon/lat tuples of single measurement locations, for example.
Probably doesn't change anything for the implementation, but I thought I'd just add it here.

What I'm looking for is something like

def DataFrame.grid(bins, method):
    """
    bins : e.g. tuple of arrays; see the bins parameter of numpy.histogram2d

    method : either 'mean' or 'std' or any function which will work on the data which will go into one new grid cell

    returns a pandas.DataFrame with the columns being the same as the original DataFrame, and the index being a MultiIndex which has the centers of the bins as values
    """

I know a lot of this should be more general (e.g. index.ndim > 2), but this here would be all I need for now.

As for the averaging of the grid cells, I recently did something similar using numpy.histogram2d:

def _histnd(lons, lats, lonbins, latbins, data=None):
    if data == None:
        summed = np.histogram2d(lons, lats, (lonbins, latbins))[0]
    elif data.ndim == 1:
        summed = np.histogram2d(lons, lats, (lonbins, latbins),
                                weights=data)[0]
    elif data.ndim == 2:
        summed = np.empty(np.r_[data.shape[0], lonbins.shape[0] - 1,
                                latbins.shape[0] - 1])
        for i in xrange(summed.shape[0]):
            summed[i] = np.histogram2d(lons, lats, (lonbins, latbins),
                                       weights=data[i])[0]
    else:
        raise ValueError("constraint ``data.ndim in [1, 2]`` violated!")

    summed = ma.masked_invalid(summed)
    return summed


def bin_data(data, lons, lats, lonbins, latbins, mask_zero=False,
             calc_std=False):
    """
    Bin data by longitude and latitude, using mean()

    data : np.ndarray
       The data to be binned, dimension (d, n)

    lons : np.ndarray
       the longitudes of the data points, dimension (n)

    lats : np.ndarray
       the latitudes of the data points, dimension (n)

    lonbins : np.ndarray
       the edges of the longitude bins

    latbins : np.ndarray
       the edges of the latitude bins

    mask_zero : bool
       if True, do not count data points with a value of 0. Defaults to False

    calc_std : bool
       if True, also calculate the standard deviation within each bin.

    returns mean, std (if calc_std == True)


    The idea to this function is taken from

       http://stackoverflow.com/questions/6163334/

    For calculating the stds on the grid, the formula was taken from

       http://stackoverflow.com/questions/1174984/1175084#1175084

    """
    summed = _histnd(lons, lats, lonbins, latbins, data)

    count_weights = np.where(data != 0., 1, 0) if mask_zero else None
    count = _histnd(lons, lats, lonbins, latbins, count_weights)
    count = ma.masked_invalid(count)

    # in the 3d data case, we need to set the count to the highest count of the
    # appropriate lon/lat pair
    if data.ndim == 2:
        count = np.tile(count.max(axis=0).copy(), (count.shape[0], 1, 1))

    if mask_zero:
        summed = ma.masked_where(summed==0., summed)
        count = ma.masked_where(count==0., count)

    binned = summed / count

    if calc_std:
        if data.ndim == 3:
            raise NotImplementedError("Calculation of std not implemented for \
3d data yet")
        sum_squares = np.histogram2d(lons, lats, (lonbins, latbins),
                                     weights=data**2)[0]
        return binned, np.sqrt((sum_squares / count) - (binned **2))
    else:
        return binned

Uh oh!

Implement custom "grouper" class #153

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions