3
���h�M � @ s� d dl Z d dlZd dlZddlmZ ddlmZ ddl m
Z
m Z m Z ddl
mZ ddlmZmZmZ ddlmZmZmZ d d
� Zd d � Zd
d� ZG dd� de�ZG dd� dee
�Z ej e � ej e � G dd� dee �Z ej e � ej e � dS )� N� )�ops)�concat)�ImplementsArrayReduce�ImplementsDatasetReduce�_maybe_promote)�zip)�peek_at�maybe_wrap_array�safe_cast_to_index)� as_variable�Variable�
Coordinatec C s\ t j| dd�\}}dd� tt|��D �}x*t|�D ] \}}|dkr2|| j|� q2W ||fS )a� Group an array by its unique values.
Parameters
----------
ar : array-like
Input array. This will be flattened if it is not already 1-D.
Returns
-------
values : np.ndarray
Sorted, unique values as returned by `np.unique`.
indices : list of lists of int
Each element provides the integer indices in `ar` with values given by
the corresponding value in `unique_values`.
T)�sortc S s g | ]}g �qS � r )�.0�_r r �3/tmp/pip-build-5_djhm0z/xray/xarray/core/groupby.py�
<listcomp> s z'unique_value_groups.<locals>.<listcomp>r )�pdZ factorize�range�len� enumerate�append)�arZinverse�values�groups�n�gr r r �unique_value_groups s r c C s t | �\} }|S )z\Return a fill value that appropriately promotes types when used with
np.concatenate
)r )�dtypeZ
fill_valuer r r �_get_fill_value( s r! c s� ddl m} ddlm} t� |�r^|tdd� � jj� D ��t� fdd�� jj� D ��� j �}nBt� |�r�|t
� j �t� fdd�� jj� D ��� j � j d�}nt
�|S ) Nr )�Dataset)� DataArrayc s s | ]\}}|t |j�fV qd S )N)r! r )r �k�vr r r � <genexpr>4 s z _dummy_copy.<locals>.<genexpr>c 3 s* | ]"\}}|� j kr|t|j�fV qd S )N)�dimsr! r )r r$ r% )�
xarray_objr r r& 6 s c 3 s* | ]"\}}|� j kr|t|j�fV qd S )N)r' r! r )r r$ r% )r( r r r& < s )�name�attrs)�datasetr" Z dataarrayr# �
isinstance�dictZ data_vars�itemsZcoordsr* r! r r) �AssertionError)r( r" r# �resr )r( r � _dummy_copy0 s
r1 c @ s� e Zd ZdZd#dd�Zedd� �Zdd � Zd
d � Zd d
� Z dd� Z
e d$dd��Z dd� Z
dd� Zdd� Zdd� Zdd� Zd%d d �Zd&d d �Zd!d"� ZdS )'�GroupBya
A object that implements the split-apply-combine pattern.
Modeled after `pandas.GroupBy`. The `GroupBy` object can be iterated over
(unique_value, grouped_array) pairs, but the main way to interact with a
groupby object are with the `apply` or `reduce` methods. You can also
directly call numpy methods like `mean` or `std`.
You should create a GroupBy object by using the `DataArray.groupby` or
`Dataset.groupby` methods.
See Also
--------
Dataset.groupby
DataArray.groupby
FNc C s� ddl m} |jdkr td��t|dd�dkr6td��t|d�sHtd��|j\}y|j| }W n$ tk
r� |j|j |� }Y nX |j
|kr�td ��d}|dk �r@t |�} | j s�td
��t
jtj| j
�| �}
|
j|�j� } | j� j� r�| j}| j� } | j} d d � t| dd� | dd� �D �t| d d�g }
t|j| j�}nd|j|jk�r�|j|jfk�rftd
��tj|j
�}
|�s�dd � |
D �}
|}nt |�\}}
t|j|�}|| _ || _ || _ |
| _ || _!d| _"|| _#dS )a: Create a GroupBy object
Parameters
----------
obj : Dataset or DataArray
Object to group.
group : DataArray or Coordinate
1-dimensional array with the group values.
squeeze : boolean, optional
If "group" is a coordinate of object, `squeeze` controls whether
the subarrays have a dimension of length 1 along that coordinate or
if the dimension is squeezed out.
grouper : pd.Grouper, optional
Used for grouping values along the `group` array.
r )�
as_datasetz `group` must be 1 dimensionalr) Nz`group` must have a namer' z$`group` must have a 'dims' attributezZthe group variable's length does not match the length of this variable along its dimensionz&index must be monotonic for resamplingc S s g | ]\}}t ||��qS r )�slice)r �i�jr r r r � s z$GroupBy.__init__.<locals>.<listcomp>zN`group` is required to be a coordinate if `group.name` is a dimension in `obj`c S s g | ]}t ||d ��qS )r )r4 )r r5 r r r r � s ���r7 )$r+ r3 �ndim�
ValueError�getattr�hasattrr' � TypeError�shape� get_axis_num�sizer Z is_monotonicr ZSeries�npZarange�groupby�firstZisnull�any�indexZdropnar r r4 r r) r �obj�group� group_dim�
group_indices� unique_coord�_groups� _full_index)�selfrE rF ZsqueezeZgrouperr3 rG Z
expected_sizeZ
full_indexrD �sZ first_itemsZbinsrH rI Z
unique_valuesr r r �__init__V sX
" zGroupBy.__init__c C s& | j d kr tt| jj| j��| _ | j S )N)rJ r- r rI r rH )rL r r r r � s
zGroupBy.groupsc C s | j jS )N)rI r? )rL r r r �__len__� s zGroupBy.__len__c C s t | jj| j� �S )N)r rI r �
_iter_grouped)rL r r r �__iter__� s zGroupBy.__iter__c c s* x$| j D ]}| jjf | j|i�V qW dS )z'Iterate over each element in this groupN)rH rE ZiselrG )rL �indicesr r r rP � s zGroupBy._iter_groupedc C s, | j |jkr| j}| j}n
| j}d }||fS )N)rG r' rF rH rI )rL �applied_example�
concat_dim� positionsr r r �_infer_concat_args� s zGroupBy._infer_concat_argsc s t j� �� �fdd��}|S )Nc s. �s� n
� fdd�}| j ||�}| j|�}|S )Nc s
� || �S )Nr )�x�y)�fr r �<lambda>� s z2GroupBy._binary_op.<locals>.func.<locals>.<lambda>)�_yield_binary_applied�_concat)rL �otherr �applied�combined)rY � reflexiver r �func� s
z GroupBy._binary_op.<locals>.func)� functools�wraps)rY r` Zignored_kwargsra r )rY r` r �
_binary_op� s zGroupBy._binary_opc c s� d }d}x�| D ]x\}}y |j f | jj|i�}d}W nB tk
rN td��Y n( tk
rt |d krlt|�}|}Y nX |||�}|V qW |s�td| jj ��d S )NFTzYGroupBy objects only support binary ops when the other argument is a Dataset or DataArrayz"no overlapping labels %r dimension)�selrF r) �AttributeErrorr< �KeyErrorr1 r9 ) rL ra r] �dummyZfound_some_valuesZ group_valuerE Z other_sel�resultr r r r[ � s"
z GroupBy._yield_binary_appliedc C s6 | j dk r2| jj|jkr2| jj| j i}|jf |�}|S )z�Our index contained empty groups (e.g., from a resampling). If we
reduced on that dimension, we want to restore the full index.
N)rK rF r) r' Zreindex)rL r_ Zindexersr r r �_maybe_restore_empty_groups� s z#GroupBy._maybe_restore_empty_groupsc C s
| j |�S )a� Fill missing values in this object by group.
This operation follows the normal broadcasting and alignment rules that
xarray uses for binary arithmetic, except the result is aligned to this
object (``join='left'``) instead of aligned to the intersection of
index coordinates (``join='inner'``).
Parameters
----------
value : valid type for the grouped object's fillna method
Used to fill all matching missing values by group.
Returns
-------
same type as the grouped object
See also
--------
Dataset.fillna
DataArray.fillna
)Z_fillna)rL �valuer r r �fillna� s zGroupBy.fillnac C s
| j |�S )a� Return an object of the same shape with all entries where cond is
True and all other entries masked.
This operation follows the normal broadcasting and alignment rules that
xarray uses for binary arithmetic.
Parameters
----------
cond : DataArray or Dataset
Returns
-------
same type as the grouped object
See also
--------
Dataset.where
)Z_where)rL Zcondr r r �where s z
GroupBy.wherec C s2 t | jd ttjf�r | jS | j|| j||dd�S )Nr T)�skipna�
keep_attrsZ
allow_lazy)r, rH �intr@ �integerrE �reducerG )rL �oprn ro r r r �_first_or_last s zGroupBy._first_or_lastTc C s | j tj||�S )zIReturn the first element of each group along the group dimension
)rt r rB )rL rn ro r r r rB s z
GroupBy.firstc C s | j tj||�S )zHReturn the last element of each group along the group dimension
)rt r �last)rL rn ro r r r ru # s z GroupBy.lastc s | j � fdd��S )zfAssign coordinates by group.
See also
--------
Dataset.assign_coords
c s | j f � �S )N)�
assign_coords)�ds)�kwargsr r rZ / s z'GroupBy.assign_coords.<locals>.<lambda>)�apply)rL rx r )rx r rv ( s zGroupBy.assign_coords)FN)F)NT)NT)�__name__�
__module__� __qualname__�__doc__rN �propertyr rO rQ rP rV � staticmethodrd r[ rj rl rm rt rB ru rv r r r r r2 F s"
L
r2 c @ sF e Zd ZdZdd� Zdd� Zdd� Zdd d
�Zdd d �Zddd�Z d
S )�DataArrayGroupByz=GroupBy object specialized to grouping DataArray objects
c c s, | j j}x | jD ]}|| j|i V qW dS )zWFast version of `_iter_grouped` that yields Variables without
metadata
N)rE �variablerH rG )rL �varrR r r r �_iter_grouped_shortcut5 s z'DataArrayGroupBy._iter_grouped_shortcutc C sF t j|||dd�}|jj| jj� | jj|�}t|dd�|j|j< |S )NT)�shortcut)�copy) r
r r* �updaterE Z_replace_maybe_drop_dimsr Z_coordsr) )rL r^ rT rU �stackedri r r r �_concat_shortcut= s z!DataArrayGroupBy._concat_shortcutc s$ � fdd�}t |j|d�}|j|� S )Nc s8 | � j jkr� j j\} | � jjkr0� jj| �}nd}|S )Ng ��.A)rF r) r' rE r> )Z dimension�axis)rL r r � lookup_orderJ s
z9DataArrayGroupBy._restore_dim_order.<locals>.lookup_order)�key)�sortedr' Z transpose)rL r� r� Z new_orderr )rL r �_restore_dim_orderI s z#DataArrayGroupBy._restore_dim_orderFc sF |r| j � }n| j� }� �fdd�|D �}| j||d�}| j|�}|S )aW Apply a function over each array in the group and concatenate them
together into a new array.
`func` is called like `func(ar, *args, **kwargs)` for each array `ar`
in this group.
Apply uses heuristics (like `pandas.GroupBy.apply`) to figure out how
to stack together the array. The rule is:
1. If the dimension along which the group coordinate is defined is
still in the first grouped array after applying `func`, then stack
over this dimension.
2. Otherwise, stack over the new dimension given by name of this
grouping (the argument to the `groupby` function).
Parameters
----------
func : function
Callable to apply to each array.
shortcut : bool, optional
Whether or not to shortcut evaluation under the assumptions that:
(1) The action of `func` does not depend on any of the array
metadata (attributes or coordinates) but only on the data and
dimensions.
(2) The action of `func` creates arrays with homogeneous metadata,
that is, with the same dimensions and attributes.
If these conditions are satisfied `shortcut` provides significant
speedup. This should be the case for many common groupby operations
(e.g., applying numpy ufuncs).
**kwargs
Used to call `func(ar, **kwargs)` for each array `ar`.
Returns
-------
applied : DataArray
The result of splitting, applying and combining this array.
c 3 s | ]}t |� |f���V qd S )N)r
)r Zarr)ra rx r r r& s z)DataArrayGroupBy.apply.<locals>.<genexpr>)r� )r� rP r\ rj )rL ra r� rx Zgroupedr^ r_ ri r )ra rx r ry V s %
zDataArrayGroupBy.applyc C sZ t |�\}}| j|�\}}|r.| j|||�}nt|||d�}t|t| j��rV| j|�}|S )N)rU )r rV r� r r, �typerE r� )rL r^ r� rS rT rU r_ r r r r\ � s
zDataArrayGroupBy._concatNTc s" � ����fdd�}| j ||d�S )a� Reduce the items in this group by applying `func` along some
dimension(s).
Parameters
----------
func : function
Function which can be called in the form
`func(x, axis=axis, **kwargs)` to return the result of collapsing an
np.ndarray over an integer valued axis.
dim : str or sequence of str, optional
Dimension(s) over which to apply `func`.
axis : int or sequence of int, optional
Axis(es) over which to apply `func`. Only one of the 'dimension'
and 'axis' arguments can be supplied. If neither are supplied, then
`func` is calculated over all dimension for each group item.
keep_attrs : bool, optional
If True, the datasets's attributes (`attrs`) will be copied from
the original object to the new one. If False (default), the new
object will be returned without attributes.
**kwargs : dict
Additional keyword arguments passed on to `func`.
Returns
-------
reduced : Array
Array with summarized data and the indicated dimension(s)
removed.
c s | j ��� fd�i���S )Nro )rr )r )r� �dimra ro rx r r � reduce_array� s z-DataArrayGroupBy.reduce.<locals>.reduce_array)r� )ry )rL ra r� r� ro r� rx r� r )r� r� ra ro rx r rr � s zDataArrayGroupBy.reduce)F)F)NNFT)
rz r{ r| r} r� r� r� ry r\ rr r r r r r� 2 s
.
r� c @ s. e Zd Zdd� Zdd� Zd dd�Zd d
� ZdS ) �DatasetGroupByc s< �j dd� � �fdd�| j� D �}| j|�}| j|�}|S )a� Apply a function over each Dataset in the group and concatenate them
together into a new Dataset.
`func` is called like `func(ds, *args, **kwargs)` for each dataset `ds`
in this group.
Apply uses heuristics (like `pandas.GroupBy.apply`) to figure out how
to stack together the datasets. The rule is:
1. If the dimension along which the group coordinate is defined is
still in the first grouped item after applying `func`, then stack
over this dimension.
2. Otherwise, stack over the new dimension given by name of this
grouping (the argument to the `groupby` function).
Parameters
----------
func : function
Callable to apply to each sub-dataset.
**kwargs
Used to call `func(ds, **kwargs)` for each sub-dataset `ar`.
Returns
-------
applied : Dataset
The result of splitting, applying and combining this dataset.
r� Nc 3 s | ]}� |f��V qd S )Nr )r rw )ra rx r r r& � s z'DatasetGroupBy.apply.<locals>.<genexpr>)�poprP r\ rj )rL ra rx r^ r_ ri r )ra rx r ry � s
zDatasetGroupBy.applyc C s, t |�\}}| j|�\}}t|||d�}|S )N)rU )r rV r )rL r^ rS rT rU r_ r r r r\ � s zDatasetGroupBy._concatNFc s � ���fdd�}| j |�S )a� Reduce the items in this group by applying `func` along some
dimension(s).
Parameters
----------
func : function
Function which can be called in the form
`func(x, axis=axis, **kwargs)` to return the result of collapsing an
np.ndarray over an integer valued axis.
dim : str or sequence of str, optional
Dimension(s) over which to apply `func`.
axis : int or sequence of int, optional
Axis(es) over which to apply `func`. Only one of the 'dimension'
and 'axis' arguments can be supplied. If neither are supplied, then
`func` is calculated over all dimension for each group item.
keep_attrs : bool, optional
If True, the datasets's attributes (`attrs`) will be copied from
the original object to the new one. If False (default), the new
object will be returned without attributes.
**kwargs : dict
Additional keyword arguments passed on to `func`.
Returns
-------
reduced : Array
Array with summarized data and the indicated dimension(s)
removed.
c s | j �� �f��S )N)rr )rw )r� ra ro rx r r �reduce_dataset� s z-DatasetGroupBy.reduce.<locals>.reduce_dataset)ry )rL ra r� ro rx r� r )r� ra ro rx r rr � s zDatasetGroupBy.reducec s | j � fdd��S )zbAssign data variables by group.
See also
--------
Dataset.assign
c s | j f � �S )N)�assign)rw )rx r r rZ s z'DatasetGroupBy.assign.<locals>.<lambda>)ry )rL rx r )rx r r� s zDatasetGroupBy.assign)NF)rz r{ r| ry r\ rr r� r r r r r� � s !
!r� ) rb Znumpyr@ Zpandasr � r �combiner �commonr r r Zpycompatr �utilsr r
r r� r r
r r r! r1 �objectr2 r� Zinject_reduce_methodsZinject_binary_opsr� r r r r �<module> s( m
R
|