3
���h� � - @ s� U d Z ddlZddlmZ ddlmZmZmZmZm Z ddl
m Z m Z m
Z
ddlZddlmZmZmZmZmZ ddlmZmZmZ ddlm Z dd l m Z m Z m Z dd
l!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3m4Z4m5Z5m6Z6m7Z7m8Z8m9Z9m:Z: dd l;m<Z<m=Z=m>Z>m?Z?m@Z@mAZAmBZB dd lCmDZDmEZE dd
lFmGZGmHZH ddlImJZJ e�rpddlKmLZL i ZMeeNeNf Mdxee eejOef d�dd�ZPeeeed�dd�ZQdd� ZRejSejTejUejVejWd�ZXdd� ZYdd � ZZd d � Z[eNd �d d!�Z\d"d#� Z]e]Z^eeejOd$�d%d&�Z_dze`eejOejOf d(�d)d*�Zae ed+�ed,�ed-�d.�d|ebee` ee` eejOe ejOe>f f d0�d1d2��Zcd}ebebebebd4d5�d6d7�Zdebd8�d9d:�Zed~ejOd �d<d=�Zfdebd4d>�d?d@�Zgd�e`eNeNebebdC�dDdE�Zhd�dFdG�Zid�dIdJ�ZjG dKdL� dL�ZkG dMdN� dNek�ZlG dOdP� dPek�Zmd�dQdR�ZndSdT� ZodUdV� Zpe`dW�dXdY�Zqd�ejrd�ejsd�ejtd�ejud�ejvd�ejwd�ejxd�ejyd�ejzd�ej{d�ej|d�ej}d�ej~d�ejd�ej�d�ej�d�ej�d�enej�ej�ej��d�enej�ej�d�d�enej}ej�ej�ej��iZ�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�enej�ej�ej��d�enej�ej�d�d�enej�ej�ej�ej�dc�iZ�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�enej�ej�ej��d�enej�ej�d�d�enej�ej�ej�ej�dc�iZ�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�ej�d�enej�ej�ej��d�enej�ej�d�d�enej�ej�ej�ej�dc�iZ�d�e`e`dd�dedf�Z�d�e`ebdg�dhdi�Z�ddej�d3fe`ebdg�djdk�Z�e�Z�ej�fdldm�Z�d�dodp�Z�d]d_d\d[d^dZhZ�d�e`e`dr�dsdt�Z�d�e`ebebe ejOeejOejOf f du�dvdw�Z�dS )�zl
Generic data algorithms. This module is experimental at the moment and not
intended for public consumption
� N)�dedent)�
TYPE_CHECKING�Dict�Optional�Tuple�Union)�catch_warnings� simplefilter�warn)� Timestamp�algos� hashtable�iNaT�lib)� AnyArrayLike� ArrayLike�DtypeObj)�doc)�'construct_1d_object_array_from_listlike�infer_dtype_from_array�
maybe_promote)�ensure_float64� ensure_int64�
ensure_object�ensure_platform_int�
ensure_uint64�
is_array_like�
is_bool_dtype�is_categorical_dtype�is_complex_dtype�is_datetime64_dtype�is_datetime64_ns_dtype�is_extension_array_dtype�is_float_dtype�
is_integer�is_integer_dtype� is_list_like�is_numeric_dtype�is_object_dtype�is_period_dtype� is_scalar�is_signed_integer_dtype�is_timedelta64_dtype�is_unsigned_integer_dtype�needs_i8_conversion� pandas_dtype)�ABCDatetimeArray�ABCExtensionArray�ABCIndex�
ABCIndexClass�
ABCMultiIndex� ABCSeries�ABCTimedeltaArray)�isna�na_value_for_dtype)�array�
extract_array)�validate_indices)�Series)�dtype�returnc C s� t | t�st| dd�} t|�r6ttj| ��tjd�fS t| �r^|dkr^ttj| ��tjd�fS y�t| �spt|�r�tj| �j d�tjd�fS t
| �s�t
|�r�t | �tjd�fS t | �s�t |�r�t
| �tjd�fS t| �s�t|�r�t| �tjd�fS t| ��st|��r8t� � td tj� t| �} W dQ R X | tjd�fS W n* tttfk
�rd t| �tjd�fS X t| d
d�}t|��s�t|��r8t|��s�t|��r�d d lm} || �} | j}nxt |��s�t |��r�d d
lm } || �} | j}nH| j dk�rt |��r| j d�}| j}||fS d dlm!} || �} | j}| j"|fS t#|��rtt#|��sV|dk�rt| j$} t%d�}t | �} | |fS tj| t&d�} t| �tjd�fS )a
routine to ensure that our data is of the correct
input dtype for lower-level routines
This will coerce:
- ints -> int64
- uint -> uint64
- bool -> uint64 (TODO this should be uint8)
- datetimelike -> i8
- datetime64tz -> i8 (in local tz)
- categorical -> codes
Parameters
----------
values : array-like
dtype : pandas_dtype, optional
coerce to this dtype
Returns
-------
values : ndarray
pandas_dtype : np.dtype or ExtensionDtype
T)�
extract_numpy�objectN�uint64�bool�int64�float64�ignorer= r )� PeriodIndex)�TimedeltaIndex� �i8)�
DatetimeIndex�category)r= )'�
isinstancer4 r: r( r �np�asarrayr= r �astyper+ r r- r r# r r r r ZComplexWarning� TypeError�
ValueError�
OverflowError�getattrr. r) �pandasrF r, rG �ndimr! �viewrJ �asi8r �codesr/ r@ )�valuesr= Z
vals_dtyperF rG rW rJ � rZ �8/tmp/pip-build-5_djhm0z/pandas/pandas/core/algorithms.py� _ensure_dataG s`
r\ )rY r= �originalr> c C s� t | t�r| j|kr| S t|�rL|j� }t | |�r@| j|kr@| S |j| �} n`t|�r|| j|dd�} t |t�r�| jt dd�} n0|dk r�t
|�r�d}n t |�r�d}| j|dd�} | S )z�
reverse of _ensure_data
Parameters
----------
values : np.ndarray or ExtensionArray
dtype : np.ndtype or ExtensionDtype
original : AnyArrayLike
Returns
-------
ExtensionArray or np.ndarray
F)�copyNzdatetime64[ns]ztimedelta64[ns]) rL r1 r= r" Zconstruct_array_typeZ_from_sequencer rO r3 r@ r r, )rY r= r] �clsrZ rZ r[ �_reconstruct_data� s$
r` c C sH t | �sDtj| dd�}|dkr:t| t�r0t| �} t| �} n
tj| �} | S )z5
ensure that we are arraylike if not already
F)�skipna�mixed�string)rb rc ) r r � infer_dtyperL �tuple�listr rM rN )rY �inferredrZ rZ r[ �_ensure_arraylike� s
rh )rD rA rC rc r@ c C s$ t | �\} }t| �}t| }|| fS )z�
Parameters
----------
values : arraylike
Returns
-------
htable : HashTable subclass
values : ndarray
)r\ �_check_object_for_strings� _hashtables)rY �_�ndtype�htablerZ rZ r[ �_get_hashtable_algo� s rn c C s t | �r| j� } t| �\} }| S )N)r Z_values_for_rankr\ )rY rk rZ rZ r[ �_get_values_for_rank s ro c C s( t | �} t| �}tj|td �}|| fS )Nr@ )ro ri rj �get)rY rl rm rZ rZ r[ �_get_data_algo s rq )r> c C s* | j j}|dkr&tj| dd�dkr&d}|S )z�
Check if we can use string hashtable instead of object hashtable.
Parameters
----------
values : ndarray
ndtype : str
Returns
-------
str
r@ F)ra rc )rc )r= �namer rd )rY rl rZ rZ r[ ri s
ri c C sP t | �} t| �r| j� S | }t| �\}} |t| ��}|j| �}t||j|�}|S )ai
Hash table-based unique. Uniques are returned in order
of appearance. This does NOT sort.
Significantly faster than numpy.unique. Includes NA values.
Parameters
----------
values : 1d array-like
Returns
-------
numpy.ndarray or ExtensionArray
The return can be:
* Index : when the input is an Index
* Categorical : when the input is a Categorical dtype
* ndarray : when the input is a Series/ndarray
Return numpy.ndarray or ExtensionArray.
See Also
--------
Index.unique : Return unique values from an Index.
Series.unique : Return unique values of Series object.
Examples
--------
>>> pd.unique(pd.Series([2, 1, 3, 3]))
array([2, 1, 3])
>>> pd.unique(pd.Series([2] + [1] * 5))
array([2, 1])
>>> pd.unique(pd.Series([pd.Timestamp('20160101'),
... pd.Timestamp('20160101')]))
array(['2016-01-01T00:00:00.000000000'], dtype='datetime64[ns]')
>>> pd.unique(pd.Series([pd.Timestamp('20160101', tz='US/Eastern'),
... pd.Timestamp('20160101', tz='US/Eastern')]))
array([Timestamp('2016-01-01 00:00:00-0500', tz='US/Eastern')],
dtype=object)
>>> pd.unique(pd.Index([pd.Timestamp('20160101', tz='US/Eastern'),
... pd.Timestamp('20160101', tz='US/Eastern')]))
DatetimeIndex(['2016-01-01 00:00:00-05:00'],
... dtype='datetime64[ns, US/Eastern]', freq=None)
>>> pd.unique(list('baabc'))
array(['b', 'a', 'c'], dtype=object)
An unordered Categorical will return categories in the
order of appearance.
>>> pd.unique(pd.Series(pd.Categorical(list('baabc'))))
[b, a, c]
Categories (3, object): [b, a, c]
>>> pd.unique(pd.Series(pd.Categorical(list('baabc'),
... categories=list('abc'))))
[b, a, c]
Categories (3, object): [b, a, c]
An ordered Categorical preserves the category ordering.
>>> pd.unique(pd.Series(pd.Categorical(list('baabc'),
... categories=list('abc'),
... ordered=True)))
[b, a, c]
Categories (3, object): [a < b < c]
An array of tuples
>>> pd.unique([('a', 'b'), ('b', 'a'), ('a', 'c'), ('b', 'a')])
array([('a', 'b'), ('b', 'a'), ('a', 'c')], dtype=object)
)rh r" �uniquern �lenr` r= )rY r] rm �table�uniquesrZ rZ r[ rs 7 s N
rs )�compsrY r> c
C s� t | �s tdt| �j� d���t |�s<tdt|�j� d���t|ttttj f�s\t
t |��}t | dd�} t
| �rz| j|�S t| �\} }t||d�\}}tj}t| �dkr�t| � r�t|�j� r�dd� }ntj}n�t| ��r4y&|jd d
d �}| jd d
d �} tj}W n0 tttfk
�r0 |jt �}| jt �} Y nX n`t | ��r�y&|jd d
d �}| jd d
d �} tj }W n. ttfk
�r� |jt �}| jt �} Y nX || |�S )
z�
Compute the isin boolean array.
Parameters
----------
comps : array-like
values : array-like
Returns
-------
ndarray[bool]
Same length as `comps`.
zIonly list-like objects are allowed to be passed to isin(), you passed a [�]T)r? )r= i@B c S s t jt j| |�t j| ��S )N)rM �
logical_or�in1d�isnan)�c�vrZ rZ r[ �<lambda>� s zisin.<locals>.<lambda>rC F)r^ rD ) r& rP �type�__name__rL r2 r5 r1 rM �ndarrayr rf r: r �isinr\ rm Zismember_objectrt r( r7 �anyrz r% rO Zismember_int64rQ rR r@ r# Zismember_float64)rw rY r= rk �frZ rZ r[ r� � sF
r� rH )� na_sentinelr> c C sB t | �\}} ||pt| ��}|j| |||d�\}}t|�}||fS )aR
Factorize an array-like to codes and uniques.
This doesn't do any coercion of types or unboxing before factorization.
Parameters
----------
values : ndarray
na_sentinel : int, default -1
size_hint : int, optional
Passed through to the hashtable's 'get_labels' method
na_value : object, optional
A value in `values` to consider missing. Note: only use this
parameter when you know that you don't have any values pandas would
consider missing in the array (NaN for float data, iNaT for
datetimes, etc.).
mask : ndarray[bool], optional
If not None, the mask is used as indicator for missing values
(True = missing, False = valid) instead of `na_value` or
condition "val != val".
Returns
-------
codes : ndarray
uniques : ndarray
)r� �na_value�mask)rq rt � factorizer ) rY r� � size_hintr� r� �
hash_klassru rv rX rZ rZ r[ �_factorize_array� s r� z� values : sequence
A 1-D sequence. Sequences that aren't pandas objects are
coerced to ndarrays before factorization.
zt sort : bool, default False
Sort `uniques` and shuffle `codes` to maintain the
relationship.
zG size_hint : int, optional
Hint to the hashtable sizer.
)rY �sortr� F)r� r� r� r> c C s� t | �} | }t| t�s"t| dd�} d}|dkr6d}d}t| ttf�r�| jdk r�| j|d�\}}t|t�rx|j |dd�}n t|t
�r�dd l m } ||�}||fS t
| j�r�| j|d
�\}}|j} n<t| �\} } |jjdkr�t|j�}
nd}
t| |||
d
�\}}|�r&t|�dk�r&t|||ddd�\}}||k} | �rr| j� �rrt|jdd�}
tj||
g�}tj| t|�d |�}t|| |�}t|t��r�|j |dd�}n t|t
��r�dd l m } ||�}||fS )a
Encode the object as an enumerated type or categorical variable.
This method is useful for obtaining a numeric representation of an
array when all that matters is identifying distinct values. `factorize`
is available as both a top-level function :func:`pandas.factorize`,
and as a method :meth:`Series.factorize` and :meth:`Index.factorize`.
Parameters
----------
{values}{sort}
na_sentinel : int or None, default -1
Value to mark "not found". If None, will not drop the NaN
from the uniques of the values.
.. versionchanged:: 1.1.2
{size_hint}
Returns
-------
codes : ndarray
An integer ndarray that's an indexer into `uniques`.
``uniques.take(codes)`` will have the same values as `values`.
uniques : ndarray, Index, or Categorical
The unique valid values. When `values` is Categorical, `uniques`
is a Categorical. When `values` is some other pandas object, an
`Index` is returned. Otherwise, a 1-D ndarray is returned.
.. note ::
Even if there's a missing value in `values`, `uniques` will
*not* contain an entry for it.
See Also
--------
cut : Discretize continuous-valued array.
unique : Find the unique value in an array.
Examples
--------
These examples all show factorize as a top-level method like
``pd.factorize(values)``. The results are identical for methods like
:meth:`Series.factorize`.
>>> codes, uniques = pd.factorize(['b', 'b', 'a', 'c', 'b'])
>>> codes
array([0, 0, 1, 2, 0]...)
>>> uniques
array(['b', 'a', 'c'], dtype=object)
With ``sort=True``, the `uniques` will be sorted, and `codes` will be
shuffled so that the relationship is the maintained.
>>> codes, uniques = pd.factorize(['b', 'b', 'a', 'c', 'b'], sort=True)
>>> codes
array([1, 1, 0, 2, 1]...)
>>> uniques
array(['a', 'b', 'c'], dtype=object)
Missing values are indicated in `codes` with `na_sentinel`
(``-1`` by default). Note that missing values are never
included in `uniques`.
>>> codes, uniques = pd.factorize(['b', None, 'a', 'c', 'b'])
>>> codes
array([ 0, -1, 1, 2, 0]...)
>>> uniques
array(['b', 'a', 'c'], dtype=object)
Thus far, we've only factorized lists (which are internally coerced to
NumPy arrays). When factorizing pandas objects, the type of `uniques`
will differ. For Categoricals, a `Categorical` is returned.
>>> cat = pd.Categorical(['a', 'a', 'c'], categories=['a', 'b', 'c'])
>>> codes, uniques = pd.factorize(cat)
>>> codes
array([0, 0, 1]...)
>>> uniques
['a', 'c']
Categories (3, object): ['a', 'b', 'c']
Notice that ``'b'`` is in ``uniques.categories``, despite not being
present in ``cat.values``.
For all other pandas objects, an Index of the appropriate type is
returned.
>>> cat = pd.Series(['a', 'a', 'c'])
>>> codes, uniques = pd.factorize(cat)
>>> codes
array([0, 0, 1]...)
>>> uniques
Index(['a', 'c'], dtype='object')
If NaN is in the values, and we want to include NaN in the uniques of the
values, it can be achieved by setting ``na_sentinel=None``.
>>> values = np.array([1, 2, 1, np.nan])
>>> codes, uniques = pd.factorize(values) # default: na_sentinel=-1
>>> codes
array([ 0, 1, 0, -1])
>>> uniques
array([1., 2.])
>>> codes, uniques = pd.factorize(values, na_sentinel=None)
>>> codes
array([0, 1, 0, 2])
>>> uniques
array([ 1., 2., nan])
T)r? NrH F)r� )rr r )�Index)r� �m�M)r� r� r� )r� �
assume_unique�verify)�compat���)r� r� )rh rL r4 r: r0 r6 �freqr� r3 Z
_shallow_copyr5 rT r� r" r= r\ �kindr8 r� rt � safe_sortr� rM �append�wherer` ) rY r� r� r� r] �dropnarX rv r� r= r� Z
code_is_narZ rZ r[ r� sT
r� Tr< )r� � ascending� normalizer� r> c C sF ddl m} t| dd�}|dk r�ddlm} || �} y|| |dd�} W n, tk
rr }
ztd�|
�W Y dd}
~
X nX | j|d �} | | jj� } | jj d
�| _| j
� } |r�| j dkj � r�| j
dd� } tjt| �g�} nDt| �r�|| �j j|d �} || _| j } n t| |�\}
} || |
|d �} |�r,| j|d �} |�rB| t| j� � } | S )
aK
Compute a histogram of the counts of non-null values.
Parameters
----------
values : ndarray (1-d)
sort : bool, default True
Sort by values
ascending : bool, default False
Sort in ascending order
normalize: bool, default False
If True then compute a relative histogram
bins : integer, optional
Rather than count values, group them into half-open bins,
convenience for pd.cut, only works with numeric data
dropna : bool, default True
Don't include counts of NaN
Returns
-------
Series
r )r< rr N)�cutT)Zinclude_lowestz+bins argument only works with numeric data.)r� �interval)�indexrr )r� )Zpandas.core.seriesr< rS Zpandas.core.reshape.tiler� rP � value_countsr� ZnotnarO Z
sort_index�_values�all�ilocrM r9 rt r" rr �_value_counts_arraylike� sort_values�float�sum)rY r� r� r� Zbinsr� r< rr r� �ii�err�result�counts�keysrZ rZ r[ r� � s6 r� )r� c
C s� t | �} | }t| �\} }| jj}t|j�rZtj| |�\}}|r�|tk}|| || }}nbttd|� ��}|| |�\}}t | �} | r�| j
� r�t |�j
� s�t j |dt j
�}t j |d| j� �}t||j|�}||fS )z�
Parameters
----------
values : arraylike
dropna : bool
Returns
-------
uniques : np.ndarray or ExtensionArray
counts : np.ndarray
Z value_count_r )rh r\ r= rr r. rm Zvalue_count_int64r rS r7 r� rM �insert�NaNr� r` )
rY r� r] rk rl r� r� Zmskr� r� rZ rZ r[ r� + s$
r� �firstc C s0 t | �\} }| jj}ttd|� ��}|| |d�S )a�
Return boolean ndarray denoting duplicate values.
Parameters
----------
values : ndarray-like
Array over which to check for duplicate values.
keep : {'first', 'last', False}, default 'first'
- ``first`` : Mark duplicates as ``True`` except for the first
occurrence.
- ``last`` : Mark duplicates as ``True`` except for the last
occurrence.
- False : Mark all duplicates as ``True``.
Returns
-------
duplicated : ndarray
Z duplicated_)�keep)r\ r= rr rS rm )rY r� rk rl r� rZ rZ r[ �
duplicatedW s r� )r� r> c
C s� ddl m} t| �} | }t| �rNt| |�rB|| jj|d�| jd�S | j|d�S |rnt| j �rn| j
� }| | } t | �\} }| j j}t t
d|� ��}|| |d�}ytj|�}W n0 tk
r� } ztd| � �� W Y dd} ~ X nX t||j |�}||�S )a3
Returns the mode(s) of an array.
Parameters
----------
values : array-like
Array over which to check for duplicate values.
dropna : boolean, default True
Don't consider counts of NaN/NaT.
.. versionadded:: 0.24.0
Returns
-------
mode : Series
r )r< )r� )rr Zmode_zUnable to sort modes: N)rT r< rh r rL r� �moderr r. r= Zisnullr\ rS rm rM r� rP r
r` )
rY r� r< r] r� rk rl r� r� r� rZ rZ r[ r� p s(
r� �averager� )�axis�method� na_optionr� �pctc C s^ | j dkr(t| �} tj| ||||d�}n2| j dkrRt| �} tj| |||||d�}ntd��|S )a�
Rank the values along a given axis.
Parameters
----------
values : array-like
Array whose values will be ranked. The number of dimensions in this
array must not exceed 2.
axis : int, default 0
Axis over which to perform rankings.
method : {'average', 'min', 'max', 'first', 'dense'}, default 'average'
The method by which tiebreaks are broken during the ranking.
na_option : {'keep', 'top'}, default 'keep'
The method by which NaNs are placed in the ranking.
- ``keep``: rank each NaN value with a NaN ranking
- ``top``: replace each NaN with either +/- inf so that they
there are ranked at the top
ascending : boolean, default True
Whether or not the elements should be ranked in ascending order.
pct : boolean, default False
Whether or not to the display the returned rankings in integer form
(e.g. 1, 2, 3) or in percentile form (e.g. 0.333..., 0.666..., 1).
rH )� ties_methodr� r� r� � )r� r� r� r� r� z&Array with ndim > 2 are not supported.)rU ro r Zrank_1dZrank_2drP )rY r� r� r� r� r� ZranksrZ rZ r[ �rank� s&
r� c
C sX t j|| j�}|dk r&t j|| j�}nd}|dk rJ|dk rJt j||B �}nB|dk r^t j|�}n.|dk rrt j|�}nt j| jtd�}|jd� |dk}|dk }|j� s�t jt j �j
| | k|@ j� } n~|j� s�t jt j �j | | k |@ j� } nVt jt j �j || | | k || @ j� �p@t jt j �j
|| | | k|| @ j� } | �rPt d��| | S )a�
Perform array addition that checks for underflow and overflow.
Performs the addition of an int64 array and an int64 integer (or array)
but checks that they do not result in overflow first. For elements that
are indicated to be NaN, whether or not there is overflow for that element
is automatically ignored.
Parameters
----------
arr : array addend.
b : array or scalar addend.
arr_mask : boolean array or None
array indicating which elements to exclude from checking
b_mask : boolean array or boolean or None
array or scalar indicating which element(s) to exclude from checking
Returns
-------
sum : An array for elements x + b for each element x in arr if b is
a scalar or an array for elements x + y for each element pair
(x, y) in (arr, b).
Raises
------
OverflowError if any x + y exceeds the maximum or minimum int64 value.
N)r= Tr zOverflow in int64 addition)
rM Z broadcast_to�shapeZ logical_not�emptyrB �fillr� �iinforC �min�maxrR )
�arr�bZarr_maskZb_maskZb2Zb2_maskZnot_nanZmask1Zmask2Zto_raiserZ rZ r[ �checked_add_with_arr� s.
,*r� �fractionc s� t j| �} t| �}| | } t j| ��dd� ����fdd�� t|�rN� |�S t j|t j�}� fdd�|D �}t j|t jd�}|S dS ) a�
Compute sample quantile or quantiles of the input array. For example, q=0.5
computes the median.
The `interpolation_method` parameter supports three values, namely
`fraction` (default), `lower` and `higher`. Interpolation is done only,
if the desired quantile lies between two data points `i` and `j`. For
`fraction`, the result is an interpolated value between `i` and `j`;
for `lower`, the result is `i`, for `higher` the result is `j`.
Parameters
----------
x : ndarray
Values from which to extract score.
q : scalar or array
Percentile at which to extract score.
interpolation_method : {'fraction', 'lower', 'higher'}, optional
This optional parameter specifies the interpolation method to use,
when the desired quantile lies between two data points `i` and `j`:
- fraction: `i + (j - i)*fraction`, where `fraction` is the
fractional part of the index surrounded by `i` and `j`.
-lower: `i`.
- higher: `j`.
Returns
-------
score : float
Score at percentile.
Examples
--------
>>> from scipy import stats
>>> a = np.arange(100)
>>> stats.scoreatpercentile(a, 50)
49.5
c S s | || | S )z|
Returns the point at the given fraction between a and b, where
'fraction' must be between 0 and 1.
rZ )�ar� r� rZ rZ r[ � _interpolateQ s z quantile.<locals>._interpolatec s� t ��dkrtjS | t ��d }|d dkr<�t|� }nf�dkrj� �t|� �t|�d |d �}n8�dkr��tj|� }n �dkr��tj|� }ntd��|S )Nr rH r� �lowerZhigherzAinterpolation_method can only be 'fraction' , 'lower' or 'higher')rt rM �nan�int�floor�ceilrQ )Zat�idxZscore)r� �interpolation_methodrY rZ r[ �
_get_scoreX s &z quantile.<locals>._get_scorec s g | ] }� |��qS rZ rZ )�.0�x)r� rZ r[ �
<listcomp>r s z quantile.<locals>.<listcomp>)r= N)rM rN r7 r� r* rD r9 )r� �qr� r� r� rZ )r� r� r� rY r[ �quantile# s '
r� c @ s@ e Zd Zeed�dd�Zdd� Zdd� Zee e
d�d d
��Z d S ) �SelectN)�nr� c C s( || _ || _|| _| jdkr$td��d S )Nr� �lastr� z,keep must be either "first", "last" or "all")r� r� r� )�objr� r� rQ )�selfr� r� r� rZ rZ r[ �__init__} s
zSelectN.__init__c C s
| j d�S )N�nlargest)�compute)r� rZ rZ r[ r� � s zSelectN.nlargestc C s
| j d�S )N� nsmallest)r� )r� rZ rZ r[ r� � s zSelectN.nsmallest)r= r> c C s t | �rt| � pt| �S )zg
Helper function to determine if dtype is valid for
nsmallest/nlargest methods
)r' r r. )r= rZ rZ r[ �is_valid_dtype_n_method� s z SelectN.is_valid_dtype_n_methodN) r� �
__module__� __qualname__r� �strr� r� r� � staticmethodr rB r� rZ rZ rZ r[ r� | s
r� c @ s e Zd ZdZdd� ZdS )�
SelectNSeriesz�
Implement n largest/smallest for Series
Parameters
----------
obj : Series
n : int
keep : {'first', 'last'}, default 'first'
Returns
-------
nordered : Series
c C s� | j }| jj}| j|�s,td|� d|� ���|dkr>| jg S | jj� }|t| j�kr�| jdkpf|dk}|dk}|r�tj d d d � n tj d d � }|| j
|d�j |�S t |j
�\}} |dkr�| }t| �r�|d8 }nt| �r�d| }| jdk�r|d d d
� }t|�}
t||
�}tj|j� |d �} tj|| k�\} | || jd d
� }
| jd k�rd|
d |� }
| jdk�r||
d |
}
|j|
S )NzCannot use method 'z
' with dtype r r� r� r� rH )r� � mergesort)r� r� r� r� )r� r� r= r� rP r� rt r� rM Zs_r� �headr\ rY r% r r� r Z kth_smallestr^ Znonzero�argsortr� )r� r� r� r= ZdroppedZ
reverse_itr� Zslcr� r/ ZnarrZkth_val�nsZindsrZ rZ r[ r� � s>
"
zSelectNSeries.computeN)r� r� r� �__doc__r� rZ rZ rZ r[ r� � s
r� c s0 e Zd ZdZeed�� fdd� Zdd� Z� ZS )� SelectNFramez�
Implement n largest/smallest for DataFrame
Parameters
----------
obj : DataFrame
n : int
keep : {'first', 'last'}, default 'first'
columns : list or str
Returns
-------
nordered : DataFrame
)r� r� c s< t � j|||� t|� s$t|t�r*|g}t|�}|| _d S )N)�superr� r& rL re rf �columns)r� r� r� r� r� )� __class__rZ r[ r� � s
zSelectNFrame.__init__c s� ddl m} | j}| j}| j}xD|D ]<}|| j}| j|�s$tdt|�� d|� dt� �� d���q$W � fdd�}|j } |j
d d
� }
}|} |g �} x�t |�D ]�\}
}|
| }t |�d |
k}t
|� �| |r�| jnd d
�}|s�t |�| kr�|| |j �} P |||j d k}|| }|| }|| |j �} |
j|j }
|t | � } q�W |j| �}| j| �|_ t |�d k�rj|S � dk}|j||dd�S )Nr )�
Int64IndexzColumn z has dtype z, cannot use method z with this dtypec s � dkr| j |�S |j | �S dS )z{
Helper function to concat `current_indexer` and `other_indexer`
depending on `method`
r� N)r� )Zcurrent_indexerZ
other_indexer)r� rZ r[ � get_indexer� s
z)SelectNFrame.compute.<locals>.get_indexerT)ZdroprH r� )r� r� r� )r� r� r� )rT r� r� r� r� r= r� rP �reprr� Z reset_index� enumeratert rS r� �loc�taker� )r� r� r� r� �framer� �columnr= r� Zoriginal_indexZ cur_frameZcur_n�indexer�iZseriesZis_last_columnrY Z border_valueZ
unsafe_valuesZ safe_valuesr� rZ )r� r[ r� � sD
&
zSelectNFrame.compute) r� r� r� r� r� r� r� r� �
__classcell__rZ rZ )r� r[ r� � s r� c s t jf� ���fdd� }|S )Nc sH � d k r| j � �} �d k r$|j ��}�d k r4�|�}�| |||d� d S )N)�
fill_value)rV )r� r� �outr� )� arr_dtyper� � fill_wrap� out_dtyperZ r[ �wrapperG s
z _view_wrapper.<locals>.wrapper)rM r� )r� r� r� r� r� rZ )r� r� r� r� r[ �
_view_wrapperF s r� c s t jf� �fdd� }|S )Nc s | j � �} �| |||d� d S )N)r� )rO )r� r� r� r� )�
conv_dtyper� rZ r[ r� T s
z!_convert_wrapper.<locals>.wrapper)rM r� )r� r� r� rZ )r� r� r[ �_convert_wrapperS s r� c C s� |\}}|d k r"|\\}}\} }
n |dk}|dk}|j � } |j � }
|d k rr| r^|||d d �f< |
rr||d d �|f< xNtt|��D ]>} || } x0tt|��D ] }
||
}| | |f || |
f< q�W q�W d S )NrH r� r� )r� �rangert )r� r� r� r� � mask_info�row_idx�col_idx�row_mask�col_mask� row_needs� col_needsr� Zu_�jr} rZ rZ r[ �_take_2d_multi_object[ s" r )r� c C s� |d k r|\}}n|dk}|j � }| j|jkr:| j|j�} | j| dkr\| jt|�||d� |r�td �g| j }|||< ||t|�< d S )NrH r )r� r� r� ) r� r= rO r� r� r �slicerU re ) r� r� r� r� r� r r� �
needs_maskingZ
outindexerrZ rZ r[ �_take_nd_objectr s
r �int8�int32rC rD �int16�float32r@ rB �datetime64[ns])r� )rU r� c s� | dkr�|j |j f}| dkr*tj|d �}n*| dkrT� dkrHtj|d �}n tj|d �}|d k r`|S |j |j f}| dkr�tj|d �}n*| dkr�� dkr�tj|d �}n tj|d �}|d k r�t||�}|S tjf� �fdd� }|S )Nr� rH r c s t |�}t| ||� |�d� d S )N)r� r� r )r r )r� r� r� r� )r� r rZ r[ �func2 s z$_get_take_nd_function.<locals>.func2)rr �
_take_1d_dictrp �_take_2d_axis0_dict�_take_2d_axis1_dictr� rM r� )rU r� r� r� r �tup�funcr rZ )r� r r[ �_get_take_nd_function� s,
r )r� �
allow_fillc C s\ t | �stj| �} tj|tjd�}|rJt|| j| � t| ||d|d�}n| j||d�}|S )a�
Take elements from an array.
.. versionadded:: 0.23.0
Parameters
----------
arr : sequence
Non array-likes (sequences without a dtype) are coerced
to an ndarray.
indices : sequence of integers
Indices to be taken.
axis : int, default 0
The axis over which to select values.
allow_fill : bool, default False
How to handle negative values in `indices`.
* False: negative values in `indices` indicate positional indices
from the right (the default). This is similar to :func:`numpy.take`.
* True: negative values in `indices` indicate
missing values. These values are set to `fill_value`. Any other
other negative values raise a ``ValueError``.
fill_value : any, optional
Fill value to use for NA-indices when `allow_fill` is True.
This may be ``None``, in which case the default NA value for
the type (``self.dtype.na_value``) is used.
For multi-dimensional `arr`, each *element* is filled with
`fill_value`.
Returns
-------
ndarray or ExtensionArray
Same type as the input.
Raises
------
IndexError
When `indices` is out of bounds for the array.
ValueError
When the indexer contains negative values other than ``-1``
and `allow_fill` is True.
Notes
-----
When `allow_fill` is False, `indices` may be whatever dimensionality
is accepted by NumPy for `arr`.
When `allow_fill` is True, `indices` should be 1-D.
See Also
--------
numpy.take : Take elements from an array along an axis.
Examples
--------
>>> from pandas.api.extensions import take
With the default ``allow_fill=False``, negative numbers indicate
positional indices from the right.
>>> take(np.array([10, 20, 30]), [0, 0, -1])
array([10, 10, 30])
Setting ``allow_fill=True`` will place `fill_value` in those positions.
>>> take(np.array([10, 20, 30]), [0, 0, -1], allow_fill=True)
array([10., 10., nan])
>>> take(np.array([10, 20, 30]), [0, 0, -1], allow_fill=True,
... fill_value=-10)
array([ 10, 10, -10])
)r= T)r� r r� )r� )r rM rN Zintpr; r� �take_1dr� )r� �indicesr� r r� r� rZ rZ r[ r� s L
r� c C s� d}t | �r | j|||d�S t| �} tj| �} |dkr`tj| j| tjd�}| j| jj � }}n�t
|dd�}|s�| j| jj � }}d
}nxt | j|�\}}|| jko�|dks�|j|k�r |dk}|j � } || f}| r�|dk r�|j|kr�t
d��n| j| jj � }}d}
| jdk�r | jj�r d }
|
�rH| j} | j| d }|dk �rH|j}|dk�r�t| j�} t|�| |< t| �} | jj�r�|| jd k�r�tj| |d
d �}ntj| |d�}t| j| j|j||d �}
|
| |||� |
�r�|j}|S )a�
Specialized Cython take which sets NaN values in one pass
This dispatches to ``take`` defined on ExtensionArrays. It does not
currently dispatch to ``SparseArray.take`` for sparse ``arr``.
Parameters
----------
arr : array-like
Input array.
indexer : ndarray
1-D array of indices to take, subarrays corresponding to -1 value
indices are filed with fill_value
axis : int, default 0
Axis to take from
out : ndarray or None, default None
Optional output array, must be appropriate type to hold input and
fill_value together, if indexer has any -1 value entries; call
maybe_promote to determine this type for any fill_value
fill_value : any, default np.nan
Fill value to replace -1 values with
allow_fill : boolean, default True
If False, indexer is assumed to contain no -1 values so no filling
will be done. This short-circuits computation of a mask. Result is
undefined if allow_fill == False and -1 is present in indexer.
Returns
-------
subarray : array-like
May be the same type as the input, or cast to an ndarray.
N)r� r )r= F)r^ rH z Incompatible type for fill_valuer� T�F)r= �order)r� r )NFr� )r" r� r: rM rN �aranger� rC r= r r r r� rP rU �flags� f_contiguous�Trf rt re r� r )r� r� r� r� r� r r r= r� r
Z
flip_orderZ
out_shape_� out_shaper rZ rZ r[ �take_ndh sV "
r" c
s` |dk s t �|d dk s t �|d dk s,t �|\}}t|�}t|�}||f}d� t| j|�\}}|| jkr�|dk}|d k}|j� }|j� } ||f|| ff� |p�| s�| j| jj� }}t|�t|�f}
tj|
|d�} t j
| jj | jj fd�} | dk�r0| j| jk�r0t j
| jj | jj fd�} | dk �r0t | | j�} | dk�rLtj
f� fdd� } | | || |d� | S )
zD
Specialized Cython take which sets NaN values in one pass.
Nr rH )r= c s t | |||� d� d S )N)r� r )r )r� r� r� r� )r rZ r[ r s ztake_2d_multi.<locals>.func)r� r� r� r� )�AssertionErrorr r r= r� r rt rM r� �_take_2d_multi_dictrp rr r� r� )
r� r� r� r r r= r r r r r! r� r rZ )r r[ �
take_2d_multi� s8
r% �leftc C s( |dk rt |�}t| tj�r�t| �r�t|�s4t|�r�tj| jj�}t |�rVtj
|g�ntj
|�}||j kj � r�||j
kj � r�| j}n|j}t |�r�|j|�}n t
||d�}nht| �p�t| �p�t| ��st |�r�t
|g�nt
|�}t |�r�|d n|}t|t��r|jdk�r|j� }| j|||d�}|S )a�
Find indices where elements should be inserted to maintain order.
.. versionadded:: 0.25.0
Find the indices into a sorted array `arr` (a) such that, if the
corresponding elements in `value` were inserted before the indices,
the order of `arr` would be preserved.
Assuming that `arr` is sorted:
====== ================================
`side` returned index `i` satisfies
====== ================================
left ``arr[i-1] < value <= self[i]``
right ``arr[i-1] <= value < self[i]``
====== ================================
Parameters
----------
arr: array-like
Input array. If `sorter` is None, then it must be sorted in
ascending order, otherwise `sorter` must be an array of indices
that sort it.
value : array_like
Values to insert into `arr`.
side : {'left', 'right'}, optional
If 'left', the index of the first suitable location found is given.
If 'right', return the last such index. If there is no suitable
index, return either 0 or N (where N is the length of `self`).
sorter : 1-D array_like, optional
Optional array of integer indices that sort array a into ascending
order. They are typically the result of argsort.
Returns
-------
array of ints
Array of insertion points with the same shape as `value`.
See Also
--------
numpy.searchsorted : Similar method from NumPy.
N)r= r )�side�sorter)r rL rM r� r% r$ r� r= r r* r9 r� r� r� r( r' r r �tzinfoZ
to_datetime64� searchsorted) r� �valuer' r( r� Z value_arrr= Z value_serr� rZ rZ r[ r* s( , r* � )r� r� c C s� ddl m} t|�}tj}| j}|jdkr2tj}ntj }t
||�rRtj | �} | j}t |�r�t
| d|j� d��r~|| | j|��S tdt|d� tj | �} | j}d}d} t| j�r�tj}| jd�} t}d }n"t|�r�tj}d } nt|�r�tj}tj|�}tj| j|d
�}
td �g| j } |dk�r(td |�nt|d �| |< ||
t | �< | j d k�rp| jj t k�rpt j!| |
||� �ntd �g| j } |dk�r�t|d �ntd |�| |< t | �}
td �g| j }|dk�r�td | �n
t| d �||< t |�}|�rZ| |
}| | }| |
|k| | |kB }|j"� �r@|j#� }d||< |j#� }d||< || }|||< ||
|
< n0| �rv| |
| | A |
|
< n| |
| | |
|
< |�r�|
j$d
�jd�}
|
S )a1
difference of n between self,
analogous to s-s.shift(n)
Parameters
----------
arr : ndarray
n : int
number of periods
axis : int
axis to shift on
stacklevel : int
The stacklevel for the lost dtype warning.
Returns
-------
shifted
r )� PandasDtyper� �__zwdtype lost in 'diff()'. In the future this will raise a TypeError. Convert to a suitable dtype prior to calling 'diff'.)�
stacklevelFrI T)r= Nr� rC ztimedelta64[ns])%Zpandas.core.arraysr- r� rM r� r= r� �operator�xor�subrL rN r" �hasattrr� �shiftr
�
FutureWarningr. rD rV r r Zobject_r% r� r� r rU re rr �
_diff_specialr Zdiff_2dr� r^ rO )r� r� r� r/ r- �nar= �opZ is_timedeltaZis_boolZout_arrZ
na_indexerZ _res_indexerZ res_indexerZ _lag_indexerZ lag_indexer�resZlagr� r� rZ rZ r[ �diffi s|
" "&
r: )r� r� r� r> c
C s t | �std��t| tj� rBt| � rBt| �\}}tj| |d�} dd� }d}t| � rttj | dd�dkrt|| �} n4y| j
� }| j |�} W n tk
r� || �} Y nX |dkr�| S t |�s�td ��t tj|��}| r�t
t| ��t
| �k r�td
��|dk�r0t| �\}
} |
t
| ��} | j| � t | j| ��}|dk�rx|j
� } t| |dd �}
|�rr|t
| � k |t
| �kB }nd}ndtjt
|�tjd�}|j|tjt
|��� |j |d
d�}
||k}|�r�||t
| � k B |t
| �kB }|dk �r�tj|
||� | t |
�fS )a�
Sort ``values`` and reorder corresponding ``codes``.
``values`` should be unique if ``codes`` is not None.
Safe for use with mixed types (int, str), orders ints before strs.
Parameters
----------
values : list-like
Sequence; must be unique if ``codes`` is not None.
codes : list_like, optional
Indices to ``values``. All out of bound indices are treated as
"not found" and will be masked with ``na_sentinel``.
na_sentinel : int, default -1
Value in ``codes`` to mark "not found".
Ignored when ``codes`` is None.
assume_unique : bool, default False
When True, ``values`` are assumed to be unique, which can speed up
the calculation. Ignored when ``codes`` is None.
verify : bool, default True
Check if codes are out of bound for the values and put out of bound
codes equal to na_sentinel. If ``verify=False``, it is assumed there
are no out of bound codes. Ignored when ``codes`` is None.
.. versionadded:: 0.25.0
Returns
-------
ordered : ndarray
Sorted ``values``
new_codes : ndarray
Reordered ``codes``; returned when ``codes`` is not None.
Raises
------
TypeError
* If ``values`` is not list-like or if ``codes`` is neither None
nor list-like
* If ``values`` cannot be sorted
ValueError
* If ``codes`` is not None and ``values`` contain duplicates.
zFOnly list-like objects are allowed to be passed to safe_sort as values)r= c S sN t jdd� | D �td�}t j| | �}t j| | �}t j|t j|td�g�S )Nc S s g | ]}t |t��qS rZ )rL r� )r� r� rZ rZ r[ r� s z1safe_sort.<locals>.sort_mixed.<locals>.<listcomp>)r= )rM r9 rB r� Z concatenaterN r@ )rY Zstr_pos�nums�strsrZ rZ r[ �
sort_mixed s z safe_sort.<locals>.sort_mixedNF)ra z
mixed-integerzMOnly list-like objects or None are allowed to be passed to safe_sort as codesz,values should be unique if codes is not NonerH )r� �wrap)r� r� r� )r& rP rL rM r� r" r rN r rd r� r� r rt rs rQ rq Z
map_locations�lookupr r� �int_�putr Zputmask)rY rX r� r� r� r= rk r= r( �orderedr� �tZorder2Z new_codesr� Zreverse_indexerrZ rZ r[ r� � sX 1
r� )Nr� )r� NNNr� )Fr� N)TFFNT)r� )T)r r� r� TF)NN)r� )NNN)r r )r r
)r rC )r rD )r r )r r
)r rC )r rD )r
r
)r
rC )r
rD )rC rC )rC rD )r r )r rD )rD rD )r@ r@ )rB rB )rB r@ )r r )r r )r r
)r rC )r rD )r r )r r
)r rC )r rD )r
r
)r
rC )r
rD )rC rC )rC rD )r r )r rD )rD rD )r@ r@ )rB rB )rB r@ )r r )r r )r r
)r rC )r rD )r r )r r
)r rC )r rD )r
r
)r
rC )r
rD )rC rC )rC rD )r r )r rD )rD rD )r@ r@ )rB rB )rB r@ )r r )r r )r r
)r rC )r rD )r r )r r
)r rC )r rD )r
r
)r
rC )r
rD )rC rC )rC rD )r r )r rD )rD rD )r@ r@ )rB rB )rB r@ )r r )r N)r FN)r& N)r r, r� )Nr� FT)�r� r0 �textwrapr Ztypingr r r r r �warningsr r r
ZnumpyrM Z pandas._libsr r r
rm r r Zpandas._typingr r r Zpandas.util._decoratorsr Zpandas.core.dtypes.castr r r Zpandas.core.dtypes.commonr r r r r r r r r r r! r" r# r$ r% r& r' r( r) r* r+ r, r- r. r/ Zpandas.core.dtypes.genericr0 r1 r2 r3 r4 r5 r6 Zpandas.core.dtypes.missingr7 r8 Zpandas.core.constructionr9 r: Zpandas.core.indexersr; rT r< Z _shared_docsr� r� r\ r` rh ZFloat64HashTableZUInt64HashTableZInt64HashTableZStringHashTableZPyObjectHashTablerj rn ro rq ri rs Zunique1dr� r� r� rB r� r� r� r� r� r� r� r� r� r� r� r� r� r r Ztake_1d_int8_int8Ztake_1d_int8_int32Ztake_1d_int8_int64Ztake_1d_int8_float64Ztake_1d_int16_int16Ztake_1d_int16_int32Ztake_1d_int16_int64Ztake_1d_int16_float64Ztake_1d_int32_int32Ztake_1d_int32_int64Ztake_1d_int32_float64Ztake_1d_int64_int64Ztake_1d_int64_float64Ztake_1d_float32_float32Ztake_1d_float32_float64Ztake_1d_float64_float64Ztake_1d_object_objectZtake_1d_bool_boolZuint8Ztake_1d_bool_objectrC r Ztake_2d_axis0_int8_int8Ztake_2d_axis0_int8_int32Ztake_2d_axis0_int8_int64Ztake_2d_axis0_int8_float64Ztake_2d_axis0_int16_int16Ztake_2d_axis0_int16_int32Ztake_2d_axis0_int16_int64Ztake_2d_axis0_int16_float64Ztake_2d_axis0_int32_int32Ztake_2d_axis0_int32_int64Ztake_2d_axis0_int32_float64Ztake_2d_axis0_int64_int64Ztake_2d_axis0_int64_float64Z take_2d_axis0_float32_float32Z take_2d_axis0_float32_float64Z take_2d_axis0_float64_float64Ztake_2d_axis0_object_objectZtake_2d_axis0_bool_boolZtake_2d_axis0_bool_objectr Ztake_2d_axis1_int8_int8Ztake_2d_axis1_int8_int32Ztake_2d_axis1_int8_int64Ztake_2d_axis1_int8_float64Ztake_2d_axis1_int16_int16Ztake_2d_axis1_int16_int32Ztake_2d_axis1_int16_int64Ztake_2d_axis1_int16_float64Ztake_2d_axis1_int32_int32Ztake_2d_axis1_int32_int64Ztake_2d_axis1_int32_float64Ztake_2d_axis1_int64_int64Ztake_2d_axis1_int64_float64Z take_2d_axis1_float32_float32Z take_2d_axis1_float32_float64Z take_2d_axis1_float64_float64Ztake_2d_axis1_object_objectZtake_2d_axis1_bool_boolZtake_2d_axis1_bool_objectr Ztake_2d_multi_int8_int8Ztake_2d_multi_int8_int32Ztake_2d_multi_int8_int64Ztake_2d_multi_int8_float64Ztake_2d_multi_int16_int16Ztake_2d_multi_int16_int32Ztake_2d_multi_int16_int64Ztake_2d_multi_int16_float64Ztake_2d_multi_int32_int32Ztake_2d_multi_int32_int64Ztake_2d_multi_int32_float64Ztake_2d_multi_int64_int64Ztake_2d_multi_int64_float64Z take_2d_multi_float32_float32Z take_2d_multi_float32_float64Z take_2d_multi_float64_float64Ztake_2d_multi_object_objectZtake_2d_multi_bool_boolZtake_2d_multi_bool_objectr$ r r� r� r" r r% r* r6 r: r� rZ rZ rZ r[ �<module> sP l$ l)
]H& $ =
I,1 2
L
YBn
#^ g<
Wx |