3
���ho� � @ sl d dl Z d dlZd dlZd dlmZmZmZmZmZ d dl Z
d dl m Z d dl
mZmZmZmZmZ d dlmZmZmZmZmZ d dlmZ d dlm Z m Z d dl m Z m Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z- d d l.m/Z/ d d
l0m1Z1m2Z2m3Z3 d d l4m5Z5 ed d
dd�Z6e6dk Z7d
a8d�e9dd�dd�Z:e:e d�� G dd� d�Z;G dd� d�Z<ee=e9d�dd�Z>e9d �d d �Z?d�eee d �d d!�Z@e
jAe9ee
jA ee
jA d"�d#d$�ZBd�e
jAe9eee= ee
jA ee
jAee
jA e
jCe
jCef d%�d&d'�ZDee9d(�d)d*�ZEd�ed+�d,d-�ZFe
jAeeG eee
jAf d.�d/d0�ZHd�e
jAeeG e9ee
jA e9d1�d2d3�ZId�e
jAeeG e9ee
jA e9d1�d4d5�ZJe;d6�d�e
jAeeG e9eGee
jA eKd7�d8d9��ZLe;e/�e<� d�e
jAeeG e9ee
jA eKd1�d:d;���ZMe<� d�d<d=��ZNeKfeeG ee
jA eeG eGeeeeGe
jAf eeGe
jAf f d>�d?d@�ZOe;d6�e<dAdB�d�dCdD���ZPe;d6dE�e<dAdB�d�dFdG���ZQe;d6dE�d�e
jAeeG e9eGee
jA eKdH�dIdJ��ZRdKdL� ZSeSdMdNdO�ZTeSdPdQdO�ZUe;dR�d�e
jAeeG e9ee
jA eeGe
jAf d1�dSdT��ZVe;dR�d�e
jAeeG e9ee
jA eeGe
jAf d1�dUdV��ZWe;d6dE�d�e
jAeeG e9ee
jA eKd1�dWdX��ZXe;d6dE�d�e
jAeeG e9ee
jA eKd1�dYdZ��ZYe;d6dE�d�e
jAeeG e9eGee
jA eKd7�d[d\��ZZe
jAeeG ee
jA e9ee
jAeGf d]�d^d_�Z[eKfeeGd`f ee
jA eeG eeeGeKe
jAf da�dbdc�Z\d�e
jAeeG ee
jA eeGd`f eGeKdd�dedf�Z]eeGd`f ee
jA eGe9dg�dhdi�Z^djdk� Z_e;d6dE�d�e
jAe
jAeeG dm�dndo��Z`dpdq� Zae;d6dE�d�e
jAe
jAeeG eeG dr�dsdt��Zbdudv� Zcdwdx� Zdedeje�Zfedejg�Zhedeji�Zjedejk�Zledejm�Znedejo�Zpe
jAe
jAeeee
jAf dy�dzd{�Zqe
jAeGe
jAeGd|�d}d~�Zree9ed�d�d��ZsdS )�� N)�Any�Optional�Tuple�Union�cast)�
get_option)�NaT� Timedelta� Timestamp�iNaT�lib)� ArrayLike�Dtype�DtypeObj�F�Scalar)�import_optional_dependency)�
_int64_max�maybe_upcast_putmask)�
_get_dtype�is_any_int_dtype�
is_bool_dtype�
is_complex�is_datetime64_any_dtype�is_float�is_float_dtype�
is_integer�is_integer_dtype�is_numeric_dtype�is_object_dtype� is_scalar�is_timedelta64_dtype�needs_i8_conversion� pandas_dtype)� PeriodDtype)�isna�na_value_for_dtype�notna)�
extract_arrayZ
bottleneckF�warn)Zraise_on_missingZ
on_versionT)�v�returnc C s t r| ad S )N)�_BOTTLENECK_INSTALLED�_USE_BOTTLENECK)r* � r. �4/tmp/pip-build-5_djhm0z/pandas/pandas/core/nanops.py�set_use_bottleneck* s r0 zcompute.use_bottleneckc s: e Zd Z� fdd�Zed�dd�Zeed�dd�Z� ZS ) �disallowc s" t � j� tdd� |D ��| _d S )Nc s s | ]}t |�jV qd S )N)r# �type)�.0�dtyper. r. r/ � <genexpr>7 s z$disallow.__init__.<locals>.<genexpr>)�super�__init__�tuple�dtypes)�selfr9 )� __class__r. r/ r7 5 s
zdisallow.__init__)r+ c C s t |d�ot|jj| j�S )Nr4 )�hasattr�
issubclassr4 r2 r9 )r: �objr. r. r/ �check9 s zdisallow.check)�fr+ c s" t j� �� �fdd��}tt|�S )Nc s� t j| |j� �}t�fdd�|D ��rD� jjdd�}td|� d���y"tjdd�� � | |�S Q R X W n: t k
r� } z t
| d �r�t|�|�� W Y d d }~X nX d S )
Nc 3 s | ]}� j |�V qd S )N)r? )r3 r> )r: r. r/ r5 @ s z0disallow.__call__.<locals>._f.<locals>.<genexpr>�nan� zreduction operation 'z ' not allowed for this dtype�ignore)�invalidr ) � itertools�chain�values�any�__name__�replace� TypeError�np�errstate�
ValueErrorr )�args�kwargsZobj_iterZf_name�e)r@ r: r. r/ �_f= s
z disallow.__call__.<locals>._f)� functools�wrapsr r )r: r@ rR r. )r@ r: r/ �__call__< s zdisallow.__call__) rI �
__module__� __qualname__r7 �boolr? r rU �
__classcell__r. r. )r; r/ r1 4 s r1 c @ s& e Zd Zddd�Zeed�dd�ZdS )�bottleneck_switchNc K s || _ || _d S )N)�namerP )r: r[ rP r. r. r/ r7 U s zbottleneck_switch.__init__)�altr+ c sp �j p
� j�ytt���W n ttfk
r6 d �Y nX tj� �dtj t
t t d�� ���fdd�
�}t
t|�S )NT)rG �axis�skipnac s� t �j�dkr6x&�jj� D ]\}}||kr|||< qW | jdkrX|jd�d krXt| |�S tr�|r�t| j��r�|jdd �d kr�|j dd � �| fd|i|��}t
|�r҈ | f||d�|��}q� | f||d�|��}n� | f||d�|��}|S )Nr � min_count�maskr] )r] r^ ) �lenrP �items�size�get�_na_for_min_countr- � _bn_ok_dtyper4 �pop� _has_infs)rG r] r^ �kwds�kr* �result)r\ �bn_func�bn_namer: r. r/ r@ a s
z%bottleneck_switch.__call__.<locals>.f)NT)r[ rI �getattr�bn�AttributeError� NameErrorrS rT rL �ndarrayr �intrX r r )r: r\ r@ r. )r\ rl rm r: r/ rU Y s
&"zbottleneck_switch.__call__)N)rI rV rW r7 r rU r. r. r. r/ rZ T s
rZ )r4 r[ r+ c C s( t | � r$t| � r$|dkr dS dS dS )N�nansum�nanprodFT)rt ru )r r" )r4 r[ r. r. r/ rf � s
rf )r+ c C sl t | tj�r@| jdkr&tj| jd��S | jdkr@tj| jd��S ytj| �j � S t
t fk
rf dS X d S )N�f8�KZf4F) �
isinstancerL rr r4 r Z has_infs_f8�ravelZ has_infs_f4�isinfrH rK �NotImplementedError)rk r. r. r/ rh � s
rh )r4 �
fill_valuec C sZ |dk r |S t | �r:|dkr"tjS |dkr0tjS tj S n |dkrFtS |dkrRtS tS dS )z; return the correct fill value for the dtype of the values Nz+inf)� _na_ok_dtyperL rA �infr r )r4 r| �fill_value_typr. r. r/ �_get_fill_value� s
r� )rG r^ r` r+ c C s0 |dkr,t | j�s t| j�r dS |r,t| �}|S )a�
Compute a mask if and only if necessary.
This function will compute a mask iff it is necessary. Otherwise,
return the provided mask (potentially None) when a mask does not need to be
computed.
A mask is never necessary if the values array is of boolean or integer
dtypes, as these are incapable of storing NaNs. If passing a NaN-capable
dtype that is interpretable as either boolean or integer data (eg,
timedelta64), a mask must be provided.
If the skipna parameter is False, a new mask will not be computed.
The mask is computed using isna() by default. Setting invert=True selects
notna() as the masking function.
Parameters
----------
values : ndarray
input array to potentially compute mask for
skipna : bool
boolean for whether NaNs should be skipped
mask : Optional[ndarray]
nan-mask if known
Returns
-------
Optional[np.ndarray]
N)r r4 r r% )rG r^ r` r. r. r/ �_maybe_get_mask� s !r� )rG r^ r| r r` r+ c C s� t |�s t�t| dd�} t| ||�}| j}t| j�rDtj| jd��} t |�}t
|||d�}|r�|dk r�|dk r�| j � } |r�|j � r�tj
| ||� nt| ||�\} }|}t|�s�t|�r�tjtj�}nt|�r�tjtj�}| ||||fS )a1
Utility to get the values view, mask, dtype, dtype_max, and fill_value.
If both mask and fill_value/fill_value_typ are not None and skipna is True,
the values array will be copied.
For input arrays of boolean or integer dtypes, copies will only occur if a
precomputed mask, a fill_value/fill_value_typ, and skipna=True are
provided.
Parameters
----------
values : ndarray
input array to potentially compute mask for
skipna : bool
boolean for whether NaNs should be skipped
fill_value : Any
value to fill NaNs with
fill_value_typ : str
Set to '+inf' or '-inf' to handle dtype-specific infinities
mask : Optional[np.ndarray]
nan-mask if known
Returns
-------
values : ndarray
Potential copy of input value array
mask : Optional[ndarray[bool]]
Mask for values, if deemed necessary to compute
dtype : np.dtype
dtype for values
dtype_max : np.dtype
platform independent dtype
fill_value : Any
fill value used
T)�
extract_numpy�i8)r| r N)r �AssertionErrorr( r� r4 r" rL �asarray�viewr} r� �copyrH �putmaskr r r �int64r �float64) rG r^ r| r r` r4 Zdtype_ok�_� dtype_maxr. r. r/ � _get_values� s( .
r� )r4 r+ c C s t | �r dS t| jtj� S )NF)r" r= r2 rL �integer)r4 r. r. r/ r} B s r} )r4 c C s� t |�rf|dkrt}t| tj�sZt|dd�}t|� s>td��| |krLtj} t | |d�} q�| j
|�} nVt |�r�t| tj�s�| |kr�tj} tj | �t
kr�td��t| dd�} n| j
d�j|�} | S ) z wrap our results if needed N�tzz Expected non-null fill_value)r� z overflow in timedelta operation�ns)�unitzm8[ns])r r rx rL rr rn r% r� rA r
�astyper! �fabsr rN r r� )rk r4 r| r� r. r. r/ �
_wrap_resultsH s& r� )rG r] r+ c C s� t | �r| jd�} t| j�}| jdkr*|S |dk s6t�| jd|� | j|d d� }ytj||| jd�}W n t k
r� tj||�}Y nX |S dS )a�
Return the missing value for `values`.
Parameters
----------
values : ndarray
axis : int or None
axis for the reduction, required if values.ndim > 1.
Returns
-------
result : scalar or ndarray
For 1-D values, returns a scalar of the correct missing type.
For 2-D values, returns a 1-D array where each element is missing.
r� � N)r4 )
r r� r& r4 �ndimr� �shaperL �fullrN )rG r] r| Z result_shaperk r. r. r/ re g s
re )rG r] r^ r` r+ c C s$ t | |d|d�\} }}}}| j|�S )a
Check if any elements along an axis evaluate to True.
Parameters
----------
values : ndarray
axis : int, optional
skipna : bool, default True
mask : ndarray[bool], optional
nan-mask if known
Returns
-------
result : bool
Examples
--------
>>> import pandas.core.nanops as nanops
>>> s = pd.Series([1, 2])
>>> nanops.nanany(s)
True
>>> import pandas.core.nanops as nanops
>>> s = pd.Series([np.nan])
>>> nanops.nanany(s)
False
F)r| r` )r� rH )rG r] r^ r` r� r. r. r/ �nanany� s !r� c C s$ t | |d|d�\} }}}}| j|�S )a
Check if all elements along an axis evaluate to True.
Parameters
----------
values : ndarray
axis: int, optional
skipna : bool, default True
mask : ndarray[bool], optional
nan-mask if known
Returns
-------
result : bool
Examples
--------
>>> import pandas.core.nanops as nanops
>>> s = pd.Series([1, 2, np.nan])
>>> nanops.nanall(s)
True
>>> import pandas.core.nanops as nanops
>>> s = pd.Series([1, 0])
>>> nanops.nanall(s)
False
T)r| r` )r� �all)rG r] r^ r` r� r. r. r/ �nanall� s !r� ZM8)rG r] r^ r_ r` r+ c
C sf t | |d|d�\} }}}}|}t|�r,|}nt|�r:tj}| j||d�} t| ||| j|d�} t| |�S )a�
Sum the elements along an axis ignoring NaNs
Parameters
----------
values : ndarray[dtype]
axis: int, optional
skipna : bool, default True
min_count: int, default 0
mask : ndarray[bool], optional
nan-mask if known
Returns
-------
result : dtype
Examples
--------
>>> import pandas.core.nanops as nanops
>>> s = pd.Series([1, 2, np.nan])
>>> nanops.nansum(s)
3.0
r )r| r` )r4 )r_ ) r� r r! rL r� �sum�_maybe_null_outr� r� )
rG r] r^ r_ r` r4 r� r� � dtype_sum�the_sumr. r. r/ rt � s rt c
C s� t | |d|d�\} }}}}|}tj}t|�s<t|�s<t|�rDtj}nt|�rT|}|}t| j|||d�} t | j
||d��}
|dk r�t |
dd�r�t tj
| �} tjdd�� |
| } W dQ R X | dk} | j� r�tj| | < n| dkr�|
| ntj} t| |�S ) a
Compute the mean of the element along an axis ignoring NaNs
Parameters
----------
values : ndarray
axis: int, optional
skipna : bool, default True
mask : ndarray[bool], optional
nan-mask if known
Returns
-------
float
Unless input is a float array, in which case use the same
precision as the input array.
Examples
--------
>>> import pandas.core.nanops as nanops
>>> s = pd.Series([1, 2, np.nan])
>>> nanops.nanmean(s)
1.5
r )r| r` )r4 Nr� FrC )r� )r� rL r� r r r! r � _get_countsr� �_ensure_numericr� rn r rr rM rH rA r� )
rG r] r^ r` r4 r� r� r� Z dtype_count�countr� Zthe_meanZct_maskr. r. r/ �nanmean s, r� c s$ � fdd�}t | � |d�\} }}}}t| j�sxy| jd�} W n( tk
rd } z t|�W Y dd}~X nX |dk rxtj| |< |dkr�| jd�} | j }| j
dk�r
|rʈ s�t tj ||| �|�S t tj
| |�|�S tj| j�} tj| j
�}
tj| |
|k �} | jtj� t | |�S t |�r|| �ntj|�S )a�
Parameters
----------
values : ndarray
axis: int, optional
skipna : bool, default True
mask : ndarray[bool], optional
nan-mask if known
Returns
-------
result : float
Unless input is a float array, in which case use the same
precision as the input array.
Examples
--------
>>> import pandas.core.nanops as nanops
>>> s = pd.Series([1, np.nan, 2, 2])
>>> nanops.nanmedian(s)
2.0
c s, t | �}� r |j� r tjS tj| | �S )N)r' r� rL rA � nanmedian)�xr` )r^ r. r/ �
get_median\ s z nanmedian.<locals>.get_median)r` rv Nrw r� )r� r r4 r� rN rK rL rA ry rc r� r� Zapply_along_axisr� �arrayr� Zarange�empty�fill) rG r] r^ r` r� r4 r� �errZnotemptyZshpZdims�retr. )r^ r/ r� C s0
r� )� value_countsr` r] �ddofr4 r+ c C s| t |�}t| |||d�}||j|� }t|�rD||krttj}tj}n0||k}|j� rttj||tj� tj||tj� ||fS )a!
Get the count of non-null values along an axis, accounting
for degrees of freedom.
Parameters
----------
values_shape : Tuple[int]
shape tuple from values ndarray, used if mask is None
mask : Optional[ndarray[bool]]
locations in values that should be considered missing
axis : Optional[int]
axis to count along
ddof : int
degrees of freedom
dtype : type, optional
type to use for count
Returns
-------
count : scalar or array
d : scalar or array
)r4 )r r� r2 r rL rA rH r� )r� r` r] r� r4 r� �dZmask2r. r. r/ �_get_counts_nanvar� s r� r� )r� c C s@ | j }t| ||d�\} }}}}tjt| ||||d��}t||�S )a�
Compute the standard deviation along given axis while ignoring NaNs
Parameters
----------
values : ndarray
axis: int, optional
skipna : bool, default True
ddof : int, default 1
Delta Degrees of Freedom. The divisor used in calculations is N - ddof,
where N represents the number of elements.
mask : ndarray[bool], optional
nan-mask if known
Returns
-------
result : float
Unless input is a float array, in which case use the same
precision as the input array.
Examples
--------
>>> import pandas.core.nanops as nanops
>>> s = pd.Series([1, np.nan, 2, 3])
>>> nanops.nanstd(s)
1.0
)r` )r] r^ r� r` )r4 r� rL �sqrt�nanvarr� )rG r] r^ r� r` �
orig_dtyper� rk r. r. r/ �nanstd� s r� Zm8c C s" t | dd�} | j}t| ||�}t|�rB| jd�} |dk rBtj| |< t| j�rft| j |||| j�\}}nt| j |||�\}}|r�|dk r�| j
� } tj | |d� t | j
|tjd��| }|dk r�tj||�}t || d �} |dk r�tj | |d� | j
|tjd�| }
t|��r|
j|�}
t|
| j�S )a�
Compute the variance along given axis while ignoring NaNs
Parameters
----------
values : ndarray
axis: int, optional
skipna : bool, default True
ddof : int, default 1
Delta Degrees of Freedom. The divisor used in calculations is N - ddof,
where N represents the number of elements.
mask : ndarray[bool], optional
nan-mask if known
Returns
-------
result : float
Unless input is a float array, in which case use the same
precision as the input array.
Examples
--------
>>> import pandas.core.nanops as nanops
>>> s = pd.Series([1, np.nan, 2, 3])
>>> nanops.nanvar(s)
1.0
T)r� rv Nr )r] r4 � )r( r4 r� r r� rL rA r r� r� r� r� r� r� r� � expand_dimsr� ) rG r] r^ r� r` r4 r� r� �avgZsqrrk r. r. r/ r� � s.
r� )rG r] r^ r� r` r+ c C sn t | ||||d� t| ||�}t| j�s2| jd�} t| j|||| j�\}}t | |||d�}tj|�tj|� S )a�
Compute the standard error in the mean along given axis while ignoring NaNs
Parameters
----------
values : ndarray
axis: int, optional
skipna : bool, default True
ddof : int, default 1
Delta Degrees of Freedom. The divisor used in calculations is N - ddof,
where N represents the number of elements.
mask : ndarray[bool], optional
nan-mask if known
Returns
-------
result : float64
Unless input is a float array, in which case use the same
precision as the input array.
Examples
--------
>>> import pandas.core.nanops as nanops
>>> s = pd.Series([1, np.nan, 2, 3])
>>> nanops.nansem(s)
0.5773502691896258
)r� r` rv )r� ) r� r� r r4 r� r� r� rL r� )rG r] r^ r� r` r� r� �varr. r. r/ �nansem$ s %
r� c s< t d� d�dtjtt tttj td�� �fdd�
�}|S )NrA )r[ T)rG r] r^ r` r+ c
s� t | |� |d�\} }}}}|d k r0| j| dks:| jdkr�y"t| ��||d�}|jtj� W q� ttt fk
r| tj}Y q�X nt| ��|�}t
|||�}t |||| j�S )N)r r` r )r4 ) r� r� rc rn r� rL rA rp rK rN r� r� )rG r] r^ r` r4 r� r| rk )r �methr. r/ � reductionV s z _nanminmax.<locals>.reduction)NTN)rZ rL rr r rs rX r )r� r r� r. )r r� r/ �
_nanminmaxU s r� �minz+inf)r �maxz-inf�Oc C s6 t | dd|d�\} }}}}| j|�}t||||�}|S )a�
Parameters
----------
values : ndarray
axis: int, optional
skipna : bool, default True
mask : ndarray[bool], optional
nan-mask if known
Returns
-------
result : int or ndarray[int]
The index/indices of max value in specified axis or -1 in the NA case
Examples
--------
>>> import pandas.core.nanops as nanops
>>> arr = np.array([1, 2, 3, np.nan, 4])
>>> nanops.nanargmax(arr)
4
>>> arr = np.array(range(12), dtype=np.float64).reshape(4, 3)
>>> arr[2:, 2] = np.nan
>>> arr
array([[ 0., 1., 2.],
[ 3., 4., 5.],
[ 6., 7., nan],
[ 9., 10., nan]])
>>> nanops.nanargmax(arr, axis=1)
array([2, 2, 1, 1], dtype=int64)
Tz-inf)r r` )r� Zargmax�_maybe_arg_null_out)rG r] r^ r` r� rk r. r. r/ � nanargmaxu s &
r� c C s6 t | dd|d�\} }}}}| j|�}t||||�}|S )a�
Parameters
----------
values : ndarray
axis: int, optional
skipna : bool, default True
mask : ndarray[bool], optional
nan-mask if known
Returns
-------
result : int or ndarray[int]
The index/indices of min value in specified axis or -1 in the NA case
Examples
--------
>>> import pandas.core.nanops as nanops
>>> arr = np.array([1, 2, 3, np.nan, 4])
>>> nanops.nanargmin(arr)
0
>>> arr = np.array(range(12), dtype=np.float64).reshape(4, 3)
>>> arr[2:, 0] = np.nan
>>> arr
array([[ 0., 1., 2.],
[ 3., 4., 5.],
[nan, 7., 8.],
[nan, 10., 11.]])
>>> nanops.nanargmin(arr, axis=1)
array([0, 0, 1, 1], dtype=int64)
Tz+inf)r r` )r� Zargminr� )rG r] r^ r` r� rk r. r. r/ � nanargmin� s &
r� c
C s� t | dd�} t| ||�}t| j�s<| jd�} t| j||�}nt| j||| jd�}|rr|dk rr| j� } tj | |d� | j
|tj d�| }|dk r�tj ||�}| | }|r�|dk r�tj ||d� |d }|| }|j
|tj d�} |j
|tj d�}
t
| �} t
|
�}
tjddd ��* ||d
d |d |
| d } W dQ R X | j} t| ��rT| j| �} t| tj��r�tj| dkd| �} tj| |d
k < | S | dk�r�dn| } |d
k �r�tjS | S dS )a�
Compute the sample skewness.
The statistic computed here is the adjusted Fisher-Pearson standardized
moment coefficient G1. The algorithm computes this coefficient directly
from the second and third central moment.
Parameters
----------
values : ndarray
axis: int, optional
skipna : bool, default True
mask : ndarray[bool], optional
nan-mask if known
Returns
-------
result : float64
Unless input is a float array, in which case use the same
precision as the input array.
Examples
--------
>>> import pandas.core.nanops as nanops
>>> s = pd.Series([1, np.nan, 1, 2])
>>> nanops.nanskew(s)
1.7320508075688787
T)r� rv )r4 Nr r� rC )rD �divider� g �?g �?� )r( r� r r4 r� r� r� r� rL r� r� r� r� �_zero_out_fperrrM rx rr �whererA )
rG r] r^ r` r� �mean�adjusted� adjusted2Z adjusted3�m2Zm3rk r4 r. r. r/ �nanskew� sD #
.
r� c C s t | dd�} t| ||�}t| j�s<| jd�} t| j||�}nt| j||| jd�}|rr|dk rr| j� } tj | |d� | j
|tj d�| }|dk r�tj ||�}| | }|r�|dk r�tj ||d� |d }|d }|j
|tj d�} |j
|tj d�}
tj
ddd ��V d
|d d |d |d
} ||d |d |
} |d |d
| d }
W dQ R X t| �} t|
�}
t|
tj��s�|d k �r�tjS |
dk�r�dS tj
ddd �� | |
| }W dQ R X | j}t|��r�|j|�}t|tj��r tj|
dkd|�}tj||d k < |S )
a�
Compute the sample excess kurtosis
The statistic computed here is the adjusted Fisher-Pearson standardized
moment coefficient G2, computed directly from the second and fourth
central moment.
Parameters
----------
values : ndarray
axis: int, optional
skipna : bool, default True
mask : ndarray[bool], optional
nan-mask if known
Returns
-------
result : float64
Unless input is a float array, in which case use the same
precision as the input array.
Examples
--------
>>> import pandas.core.nanops as nanops
>>> s = pd.Series([1, np.nan, 1, 3, 2])
>>> nanops.nankurt(s)
-1.2892561983471076
T)r� rv )r4 Nr r� rC )rD r� r� r� � )r( r� r r4 r� r� r� r� rL r� r� r� r� rM r� rx rr rA r� )rG r] r^ r` r� r� r� r� Z adjusted4r� Zm4ZadjZnumerZdenomrk r4 r. r. r/ �nankurt! sN #
"
r� c C sF t | ||�}|r(|dk r(| j� } d| |< | j|�}t|||| j|d�S )a�
Parameters
----------
values : ndarray[dtype]
axis: int, optional
skipna : bool, default True
min_count: int, default 0
mask : ndarray[bool], optional
nan-mask if known
Returns
-------
Dtype
The product of all elements on a given axis. ( NaNs are treated as 1)
Examples
--------
>>> import pandas.core.nanops as nanops
>>> s = pd.Series([1, 2, 3, np.nan])
>>> nanops.nanprod(s)
6.0
Nr� )r_ )r� r� �prodr� r� )rG r] r^ r_ r` rk r. r. r/ ru ~ s
ru )rk r] r` r^ r+ c C sp |d kr | S |d ks"t | dd� rB|r4|j� r@d} ql|j� rld} n*|rR|j|�}n
|j|�}|j� rld| |< | S )Nr� Fr� ���r� r� )rn r� rH )rk r] r` r^ Zna_maskr. r. r/ r� � s
r� .)� values_shaper` r] r4 r+ c C s� t |�}|dkr<|dk r(|j|j� }n
tj| �}|j|�S |dk rZ|j| |j|� }n| | }t|�rt|j|�S y
|j|�S t k
r� tj
||d�S X dS )a�
Get the count of non-null values along an axis
Parameters
----------
values_shape : tuple of int
shape tuple from values ndarray, used if mask is None
mask : Optional[ndarray[bool]]
locations in values that should be considered missing
axis : Optional[int]
axis to count along
dtype : type, optional
type to use for count
Returns
-------
count : scalar or array
N)r4 ) r rc r� rL r� r2 r� r r� rp r� )r� r` r] r4 �nr� r. r. r/ r� � s
r� )rk r] r` r� r_ r+ c C s� |dk r�|dk r�t | dd�r�|j| |j|� | dk }tj|�r�t| �rvtj| �r`| jd�} n
| jd�} tj| |< q�d| |< n| t k r�t
|||�r�tj} | S )zu
Returns
-------
Dtype
The product of all elements on a given axis. ( NaNs are treated as 1)
Nr� Fr Zc16rv ) rn r� r� rL rH r Z iscomplexobjr� rA r �check_below_min_count)rk r] r` r� r_ Z null_maskr. r. r/ r� � s
r� )r� r` r_ r+ c C s: |dkr6|dkr t j| �}n|j|j� }||k r6dS dS )a�
Check for the `min_count` keyword. Returns True if below `min_count` (when
missing value should be returned from the reduction).
Parameters
----------
shape : tuple
The shape of the values (`values.shape`).
mask : ndarray or None
Boolean numpy array (typically of same shape as `shape`) or None.
min_count : int
Keyword passed through from sum/prod call.
Returns
-------
bool
r NTF)rL r� rc r� )r� r` r_ Z non_nullsr. r. r/ r�
s r� c C s\ t | tj�r:tjdd�� tjtj| �dk d| �S Q R X n tj| �dk rT| jjd�S | S d S )NrC )rD g�+����=r )rx rL rr rM r� �absr4 r2 )�argr. r. r/ r� ) s r� �pearson)�a�b� min_periodsc C sp t | �t |�krtd��|dkr$d}t| �t|�@ }|j� sL| | } || }t | �|k r^tjS t|�}|| |�S )z
a, b: ndarrays
z'Operands to nancorr must have same sizeNr� )ra r� r' r� rL rA �
get_corr_func)r� r� �methodr� �validr@ r. r. r/ �nancorr2 s r� c s| | dkr$ddl m� � fdd�}|S | dkrHddl m� �fdd�}|S | d kr\d
d� }|S t| �rh| S td | � d ���d S )
NZkendallr )�
kendalltauc s � | |�d S )Nr r. )r� r� )r� r. r/ �funcO s zget_corr_func.<locals>.funcZspearman)� spearmanrc s � | |�d S )Nr r. )r� r� )r� r. r/ r� V s r� c S s t j| |�d S )Nr r� )r r� )rL Zcorrcoef)r� r� r. r. r/ r� \ s zUnknown method 'z@', expected one of 'kendall', 'spearman', 'pearson', or callable)Z scipy.statsr� r� �callablerN )r� r� r. )r� r� r/ r� K s r� )r� r� r� r� c C sr t | �t |�krtd��|d kr$d}t| �t|�@ }|j� sL| | } || }t | �|k r^tjS tj| ||d�d S )Nz&Operands to nancov must have same sizer� )r� r )r r� )ra r� r' r� rL rA Zcov)r� r� r� r� r� r. r. r/ �nancovi s r� c C s@ t | tj�r�t| �s t| �r*| jtj�} n�t| �r�y| jtj�} W n^ t t
fk
r� y| jtj�} W n4 t
k
r� } zt d| � d��|�W Y d d }~X nX Y nX tj tj | ��s�| j
} n�t| �p�t| �p�t| ��s<y t| �} W nZ t
k
�r: y t| �} W n6 t
k
�r4 } zt d| � d��|�W Y d d }~X nX Y nX | S )NzCould not convert z to numeric)rx rL rr r r r� r� r Z
complex128rK rN rH �imag�realr r r �float�complex)r� r� r. r. r/ r� � s, * *r� c s � fdd�}|S )Nc sh t | �}t |�}||B }tjdd�� � | |�}W d Q R X |j� rdt|�rT|jd�}tj||tj� |S )NrC )r� r� )r% rL rM rH r r� r� rA )r� �yZxmaskZymaskr` rk )�opr. r/ r@ � s
zmake_nancomp.<locals>.fr. )r� r@ r. )r� r/ � make_nancomp� s r� )rG r` �na_valuer+ c C sN | | } t | �dkr>tj|�r$|S tj|gt |� | jd�S tj| ||d�S )a�
Wrapper for np.percentile that skips missing values, specialized to
1-dimensional case.
Parameters
----------
values : array over which to find quantiles
mask : ndarray[bool]
locations in values that should be considered missing
q : scalar or array of quantile indices to find
na_value : scalar
value to return for empty or all-null values
interpolation : str
Returns
-------
quantiles : scalar or array
r )r4 )�
interpolation)ra r r rL r� r4 �
percentile)rG r` �qr� r� r. r. r/ �_nanpercentile_1d� s
r� )rG r] r` r� c s� | j jd
kr8t| jd��|�jd�||� �}|j| j �S tj|� r�|j� r�|dkrft| |��� d�S |j | j k r~|j
| j �}|dkr�| j } |j }� ��fdd�t
t| �t|��D �}tj|| j d d
�j }|S ntj| �|� d �S d S )a�
Wrapper for np.percentile that skips missing values.
Parameters
----------
values : array over which to find quantiles
q : scalar or array of quantile indices to find
axis : {0, 1}
na_value : scalar
value to return for empty or all-null values
mask : ndarray[bool]
locations in values that should be considered missing
ndim : {1, 2}
interpolation : str
Returns
-------
quantiles : scalar or array
�m�Mr� r� )r� r c s" g | ]\}}t ||��� d ��qS ))r� )r� )r3 �valr� )r� r� r� r. r/ �
<listcomp> s z!nanpercentile.<locals>.<listcomp>F)r4 r� )r] r� N)r� r� )r4 �kind�
nanpercentiler� r� r r rH r� r� Zreshaper� �T�zip�listrL r� r� )rG r� r] r� r` r� r� rk r. )r� r� r� r/ r� � s$ r� )rG r^ r+ c C s� t jdt jft jjt j t jft jdt jft jjt jt jfi| \}}| jj d k�r,| j}t
| �}|t jjkr�| j d�}t j t j
�j||< d}n| }d}||j d�dd �} |r�t| |< n8|t jjkr�t j|� j� d }
t|
�r�t| d
|
d �< |� r�t||< t| t j��r| j |�} nt| �j| |d �} nZ|�rzt| jjt jt jf� �rz| j� } t
| �}|| |< || dd �} || |< n || dd �} | S )
a
Cumulative function with skipna support.
Parameters
----------
values : np.ndarray or ExtensionArray
accum_func : {np.cumprod, np.maximum.accumulate, np.cumsum, np.minimum.accumulate}
skipna : bool
Returns
-------
np.ndarray or ExtensionArray
g �?g r� r� r� TFr )r] N)r4 )r� r� )rL ZcumprodrA �maximum�
accumulater~ ZcumsumZminimumr4 r� r% r� Ziinfor� r� r r� Znonzerora rx rr r2 Z_from_sequencer= r� Zbool_r� ) rG Z
accum_funcr^ Zmask_aZmask_br� r` r� �changedrk Znz�valsr. r. r/ �
na_accum_func sD
r� )T)NN)NNN)N)NTN)NTN)NTr N)NTN)NTN)NTr� N)NTr� N)NTr� N)NTN)NTN)NTN)NTN)NTr N)r� )r� N)Nr� )trS rE �operatorZtypingr r r r r ZnumpyrL Zpandas._configr Z pandas._libsr r r
r r Zpandas._typingr
r r r r Zpandas.compat._optionalr Zpandas.core.dtypes.castr r Zpandas.core.dtypes.commonr r r r r r r r r r r r r! r" r# Zpandas.core.dtypes.dtypesr$ Zpandas.core.dtypes.missingr% r& r' Zpandas.core.constructionr( ro r, r- rX r0 r1 rZ �strrf rh r� rr r� r4 r� r} r� rs re r� r� r� rt r� r� r� r� r� r� r� ZnanminZnanmaxr� r� r� r� ru r� r� r� r� r� r� r� r� r� r� �gtZnangt�geZnange�ltZnanlt�leZnanle�eqZnaneq�neZnanner� r� r� r. r. r. r/ �<module> s D 4- (O
% " & 9 M *(#C * & & N W ! - $
6 |