3
���h1 � @ s� d Z ddlmZ ddlmZ ddlmZmZmZm Z m
Z
m Z m Z m
Z
mZ ddlmZmZ ddlmZmZ ddlmZ ddljjZdd lm Z dd
l m Z m Z e e e
e e e e e
e! e e
e" f d �d d
�Z#e d�dd�Z$e%e
e%e
e! e
e" f d�dd�Z&e e
eef e e
eef d�dd�Z'e e e e d�dd�Z(eed�dd �Z)e e e!e
eee!f f e
e
e" e ee f d �d d �Z*e%e
e
e! e
ee!ed ef f f d�d!d"�Z+dS )#z�
aggregation.py contains utility functions to handle multiple named and lambda
kwarg aggregations in groupby and DataFrame/Series aggregation
� )� defaultdict)�partial) �Any�Callable� DefaultDict�Dict�List�Optional�Sequence�Tuple�Union)� AggFuncType�Label)� is_dict_like� is_list_like)�SpecificationErrorN)�Index)�FrameOrSeriesUnion�Series)�func�returnc K st | dkot f |�}d}d}|sVt| t�rFt| �tt| ��krFtd��n| dkrVtd��|rht|�\} }}|| ||fS )a�
This is the internal function to reconstruct func given if there is relabeling
or not and also normalize the keyword to get new order of columns.
If named aggregation is applied, `func` will be None, and kwargs contains the
column and aggregation function information to be parsed;
If named aggregation is not applied, `func` is either string (e.g. 'min') or
Callable, or list of them (e.g. ['min', np.max]), or the dictionary of column name
and str/Callable/list of them (e.g. {'A': 'min'}, or {'A': [np.min, lambda x: x]})
If relabeling is True, will return relabeling, reconstructed func, column
names, and the reconstructed order of columns.
If relabeling is False, the columns and order will be None.
Parameters
----------
func: agg function (e.g. 'min' or Callable) or list of agg functions
(e.g. ['min', np.max]) or dictionary (e.g. {'A': ['min', np.max]}).
**kwargs: dict, kwargs used in is_multi_agg_with_relabel and
normalize_keyword_aggregation function for relabelling
Returns
-------
relabelling: bool, if there is relabelling or not
func: normalized and mangled func
columns: list of column names
order: list of columns indices
Examples
--------
>>> reconstruct_func(None, **{"foo": ("col", "min")})
(True, defaultdict(<class 'list'>, {'col': ['min']}), ('foo',), array([0]))
>>> reconstruct_func("min")
(False, 'min', None, None)
NzFFunction names must be unique if there is no new column names assignedz4Must provide 'func' or tuples of '(column, aggfunc).)�is_multi_agg_with_relabel�
isinstance�list�len�setr � TypeError� normalize_keyword_aggregation)r �kwargsZ
relabeling�columns�order� r! �9/tmp/pip-build-5_djhm0z/pandas/pandas/core/aggregation.py�reconstruct_func s ) r# )r c K s" t dd� | j� D ��o t| �dkS )ax
Check whether kwargs passed to .agg look like multi-agg with relabeling.
Parameters
----------
**kwargs : dict
Returns
-------
bool
Examples
--------
>>> is_multi_agg_with_relabel(a="max")
False
>>> is_multi_agg_with_relabel(a_max=("a", "max"), a_min=("a", "min"))
True
>>> is_multi_agg_with_relabel()
False
c s s$ | ] }t |t�ot|�d kV qdS )� N)r �tupler )�.0�vr! r! r" � <genexpr>s s z,is_multi_agg_with_relabel.<locals>.<genexpr>r )�all�valuesr )r r! r! r" r ^ s r )r r c C s� t t�}g }tt| j� � �\}}x@t||�D ]2\}\}}|| j|� |j|tj|�pX|f� q,W t|�}dd� |j� D �} t| �}
t|
�j |�} ||| fS )aL
Normalize user-provided "named aggregation" kwargs.
Transforms from the new ``Mapping[str, NamedAgg]`` style kwargs
to the old Dict[str, List[scalar]]].
Parameters
----------
kwargs : dict
Returns
-------
aggspec : dict
The transformed kwargs.
columns : List[str]
The user-provided keys.
col_idx_order : List[int]
List of columns indices.
Examples
--------
>>> normalize_keyword_aggregation({"output": ("input", "sum")})
(defaultdict(<class 'list'>, {'input': ['sum']}), ('output',), array([0]))
c S s, g | ]$\}}|D ]}|t j|�p"|f�qqS r! )�com�get_callable_name)r& �column�aggfuncs�aggfuncr! r! r" �
<listcomp>� s z1normalize_keyword_aggregation.<locals>.<listcomp>)
r r �zip�items�appendr+ r, �_make_unique_kwarg_listr � get_indexer) r Zaggspecr r �pairs�namer- r/ Zuniquified_orderZ
aggspec_orderZuniquified_aggspec�
col_idx_orderr! r! r" r x s r )�seqr c s � fdd�t � �D �S )a
Uniquify aggfunc name of the pairs in the order list
Examples:
--------
>>> kwarg_list = [('a', '<lambda>'), ('a', '<lambda>'), ('b', '<lambda>')]
>>> _make_unique_kwarg_list(kwarg_list)
[('a', '<lambda>_0'), ('a', '<lambda>_1'), ('b', '<lambda>')]
c sN g | ]F\}}� j |�d krF|d dj|d t� d|� j |��g�fn|�qS )� r �_N)�count�join�str)r& �i�pair)r9 r! r" r0 � s z+_make_unique_kwarg_list.<locals>.<listcomp>)� enumerate)r9 r! )r9 r" r4 � s
r4 )r. r c C s` t | �dkr| S d}g }xB| D ]:}tj|�dkrNt|�}d|� d�|_|d7 }|j|� q W |S )aJ
Possibly mangle a list of aggfuncs.
Parameters
----------
aggfuncs : Sequence
Returns
-------
mangled: list-like
A new AggSpec sequence, where lambdas have been converted
to have unique names.
Notes
-----
If just one aggfunc is passed, the name will not be mangled.
r: r z<lambda>z<lambda_�>)r r+ r, r �__name__r3 )r. r? �mangled_aggfuncsr/ r! r! r" �_managle_lambda_list� s
rE )�agg_specr c C sr t | �}|pt| �s| S t| �� }|rfxF| j� D ]0\}}t|�rTt |� rTt|�}n|}|||< q0W nt| �}|S )aZ
Make new lambdas with unique names.
Parameters
----------
agg_spec : Any
An argument to GroupBy.agg.
Non-dict-like `agg_spec` are pass through as is.
For dict-like `agg_spec` a new spec is returned
with name-mangled lambdas.
Returns
-------
mangled : Any
Same type as the input.
Examples
--------
>>> maybe_mangle_lambdas('sum')
'sum'
>>> maybe_mangle_lambdas([lambda: 1, lambda: 2]) # doctest: +SKIP
[<function __main__.<lambda_0>,
<function pandas...._make_lambda.<locals>.f(*args, **kwargs)>]
)r r �typer2 rE )rF Zis_dictZmangled_aggspec�keyr. rD r! r! r" �maybe_mangle_lambdas� s
rI )�resultr r r r c C s� dd� t t||�dd� d�D �}i }d}t| t� o@t| j�dk}x~|j� D ]r\}} | | j� }
|r�dd� | D �} t|
j �j
| �} |
| }
|||t| � � |
_ |
j |d d
�||< |t| � }qLW |S ) a� Internal function to reorder result if relabelling is True for
dataframe.agg, and return the reordered result in dict.
Parameters:
----------
result: Result from aggregation
func: Dict of (column name, funcs)
columns: New columns name for relabelling
order: New order for relabelling
Examples:
---------
>>> result = DataFrame({"A": [np.nan, 2, np.nan],
... "C": [6, np.nan, np.nan], "B": [np.nan, 4, 2.5]}) # doctest: +SKIP
>>> funcs = {"A": ["max"], "C": ["max"], "B": ["mean", "min"]}
>>> columns = ("foo", "aab", "bar", "dat")
>>> order = [0, 1, 2, 3]
>>> _relabel_result(result, func, columns, order) # doctest: +SKIP
dict(A=Series([2.0, NaN, NaN, NaN], index=["foo", "aab", "bar", "dat"]),
C=Series([NaN, 6.0, NaN, NaN], index=["foo", "aab", "bar", "dat"]),
B=Series([NaN, NaN, 2.5, 4.0], index=["foo", "aab", "bar", "dat"]))
c S s g | ] }|d �qS )r r! )r& r@ r! r! r" r0 4 s z"relabel_result.<locals>.<listcomp>c S s | d S )Nr: r! )�tr! r! r" �<lambda>4 s z relabel_result.<locals>.<lambda>)rH r r: c S s$ g | ] }t |t�s tj|�n|�qS r! )r r> r+ r, )r& �fr! r! r" r0 S s F)�copy) �sortedr1 r r r r r2 Zdropnar �indexr5 Zreindex) rJ r r r Zreordered_indexesZreordered_result_in_dict�idxZ reorder_mask�colZfun�sr8 r! r! r" �relabel_result s rT .c C sj d}d}t | �}g }x@| j� D ]4}t|t�p2t|�sHt|jt|�j���|j |� q W |sbt|��||fS )a�
Validates types of user-provided "named aggregation" kwargs.
`TypeError` is raised if aggfunc is not `str` or callable.
Parameters
----------
kwargs : dict
Returns
-------
columns : List[str]
List of user-provied keys.
func : List[Union[str, callable[...,Any]]]
List of user-provided aggfuncs
Examples
--------
>>> validate_func_kwargs({'one': 'min', 'two': 'max'})
(['one', 'two'], ['min', 'max'])
z2Must provide 'func' or named aggregation **kwargs.z-func is expected but recieved {} in **kwargs.)
r r* r r> �callabler �formatrG rC r3 )r Zno_arg_messageZtuple_given_messager r Zcol_funcr! r! r" �validate_func_kwargs` s rW ),�__doc__� collectionsr � functoolsr Ztypingr r r r r r r
r r Zpandas._typingr
r Zpandas.core.dtypes.commonr r Zpandas.core.baser Zpandas.core.common�core�commonr+ Zpandas.core.indexes.apir Zpandas.core.seriesr r �boolr> �intr# r �dictr r4 rE rI rT rW r! r! r! r" �<module> s2 , ,="8!-E |