3
���h�Y � @ sh U d dl Z d dlmZmZmZmZmZmZmZ d dl Z d dl
Z d dl m
Z
mZ d dlmZmZ d dlmZ d dlmZ d dlmZ d dlmZ d d l m Z m Z m Z m Z m!Z!m"Z"m#Z#m$Z$m%Z% d d
l&m'Z' d d l(m)Z) d d l*m+Z+ d d
l,m-Z- d dl.m/Z/ d dl0m1Z1 ddl2m3Z3m4Z4 e�r d dl5Z5G dd� de4�Z6d1e7dd�dd�Z8e7d�dd�Z9d2e7ee j:e j:f d�d d �Z;G d d� de3�Z<e<j=� e<j>� d Z?eG d d!� d!e6��Z@eG d"d#� d#e6��ZAeG d$d%� d%e6��ZBeG d&d'� d'e6��ZCeG d(d)� d)e6��ZDeG d*d+� d+e6��ZEeG d,d-� d-e6��ZFeG d.d/� d/e6��ZGe@� eA� eB� eC� eD� eE� eF� eG� d0�ZHeeIe6f HdS )3� N)�
TYPE_CHECKING�Dict�List�Optional�Tuple�Type�Union)�lib�missing)� ArrayLike�DtypeObj)�set_function_name)�function)�cache_readonly)�register_extension_dtype) �
is_bool_dtype�is_datetime64_dtype�is_float�is_float_dtype�
is_integer�is_integer_dtype� is_list_like�is_object_dtype� pandas_dtype)�isna)�ops)�masked_reductions)�invalid_comparison)�unpack_zerodim_and_defer)�
to_numeric� )�BaseMaskedArray�BaseMaskedDtypec @ s� e Zd ZU dZedZeed�dd�Z e
e d�dd��Z e
e d�dd ��Z
ee d�d
d ��Ze
ejd�d d
��Ze
ed�dd��Ze
ed�dd��Zeed d�dd��Zee ee d�dd�Ze d dd�dd �Z dS ) �
_IntegerDtypea'
An ExtensionDtype to hold a single size & kind of integer dtype.
These specific implementations are subclasses of the non-public
_IntegerDtype. For example we have Int8Dtype to represent signed int 8s.
The attributes name & type are set when these subclasses are created.
N)�returnc C s$ | j r
dnd}|� dd| j � d�S )N�U� �Int� zDtype())�is_unsigned_integer�itemsize)�self�sign� r- �</tmp/pip-build-5_djhm0z/pandas/pandas/core/arrays/integer.py�__repr__5 s z_IntegerDtype.__repr__c C s
| j dkS )N�i)�kind)r+ r- r- r. �is_signed_integer9 s z _IntegerDtype.is_signed_integerc C s
| j dkS )N�u)r1 )r+ r- r- r. r) = s z!_IntegerDtype.is_unsigned_integerc C s dS )NTr- )r+ r- r- r. � _is_numericA s z_IntegerDtype._is_numericc C s t j| j�S )z' Return an instance of our numpy dtype )�np�dtype�type)r+ r- r- r. � numpy_dtypeE s z_IntegerDtype.numpy_dtypec C s | j jS )N)r8 r1 )r+ r- r- r. r1 J s z_IntegerDtype.kindc C s | j jS )z* Return the number of bytes in this dtype )r8 r* )r+ r- r- r. r* N s z_IntegerDtype.itemsize� IntegerArrayc C s t S )zq
Return the array type associated with this dtype.
Returns
-------
type
)r9 )�clsr- r- r. �construct_array_typeS s z"_IntegerDtype.construct_array_type)�dtypesr$ c C sJ t dd� |D ��sd S tjdd� |D �g �}tj|tj�rFtt|� S d S )Nc s s@ | ]8}t |t�p6t |tj�o6tj|tj�p6tj|tj�V qd S )N)�
isinstancer" r5 r6 �
issubdtype�numberZbool_)�.0�tr- r- r. � <genexpr>a s z2_IntegerDtype._get_common_dtype.<locals>.<genexpr>c S s g | ]}t |t�r|jn|�qS r- )r= r" r8 )r@ rA r- r- r. �
<listcomp>j s z3_IntegerDtype._get_common_dtype.<locals>.<listcomp>)�allr5 Zfind_common_typer> �integer�_dtypes�str)r+ r< Znp_dtyper- r- r. �_get_common_dtype^ s
z _IntegerDtype._get_common_dtype�
pyarrow.Array�pyarrow.ChunkedArray)�arrayr$ c C s� ddl }ddlm} |j| j�}|jj|�s6|j|�}t||j�rJ|g}n|j }g }x<|D ]4}||| jd�\}} t
|j � | dd�}
|j |
� qZW t
j
|�S )zI
Construct IntegerArray from pyarrow Array/ChunkedArray.
r N)� pyarrow_array_to_numpy_and_mask)r6 F)�copy)�pyarrowZ pandas.core.arrays._arrow_utilsrL Zfrom_numpy_dtyper7 �equals�castr= �Array�chunksr9 rM �appendZ_concat_same_type) r+ rK rN rL Z pyarrow_typerR �resultsZarr�data�maskZint_arrr- r- r. �__from_arrow__p s
z _IntegerDtype.__from_arrow__)rI rJ ) �__name__�
__module__� __qualname__�__doc__rG �name�baser r7 r/ r �boolr2 r) �propertyr4 r5 r6 r8 r1 �intr* � classmethodr; r r r rH r rW r- r- r- r. r# ' s,
r# Fr9 )rM r$ c C s t | ||d�\} }t| |�S )a+
Infer and return an integer array of the values.
Parameters
----------
values : 1D list-like
dtype : dtype, optional
dtype to coerce
copy : bool, default False
Returns
-------
IntegerArray
Raises
------
TypeError if incompatible types
)r6 rM )�coerce_to_arrayr9 )�valuesr6 rM rV r- r- r. �
integer_array� s rd )rM c C st y| j |d|d�S tk
rn } zB| j ||d�}|| kj� r@|S td| j� dtj|�� ��|�W Y dd}~X nX dS )z}
Safely cast the values to the dtype if they
are equivalent, meaning floats must be equivalent to the
ints.
�safe)ZcastingrM )rM z"cannot safely cast non-equivalent z to N)�astype� TypeErrorrD r6 r5 )rc r6 rM �errZcastedr- r- r. � safe_cast� s ri c C sT |dkr"t | d�r"t| j�r"| j}|dk r�t|t�rP|jd�sH|jd�rP|j� }tt|�t �s�yt
tt j|�� }W n2 t k
r� } zt
d|� ��|�W Y dd}~X nX t| t�r�| j| j } }|dk r�| j|jdd�} |r�| j� } |j� }| |fS t j| |d�} t| ��r\tj| dd �}|d
k�r@t jt| ��} | jt j� n|dk�r�t | j� d���nJt | ��r�t|��r�t j| t |d�} n$t| ��p�t | ��s�t | j� d���|dk�r�t | �}nt|�t| �k�s�t!�| j"dk�s�t d��|j"dk�s�t d��|dk�rt jd�}n|j}|j#� �r>| j� } d| |< t$| |dd�} nt$| |dd�} | |fS )a-
Coerce the input values array to numpy arrays with a mask
Parameters
----------
values : 1D list-like
dtype : integer dtype
mask : bool 1D array, optional
copy : bool, default False
if True, copy the input
Returns
-------
tuple of (values, mask)
Nr6 r' ZUIntzinvalid dtype specified F)rM T)�skipna�empty�floatingrE �
mixed-integer�
integer-na�mixed-integer-floatz' cannot be converted to an IntegerDtype)r6 rM r z values must be a 1D list-likezmask must be a 1D list-like�int64)rl rE rm rn ro )%�hasattrr r6 r= rG �
startswith�lower�
issubclassr7 r# rF r5 �KeyError�
ValueErrorr9 �_data�_maskrf r8 rM rK r r Z infer_dtyperk �len�fill�nanrg r r` r r �AssertionError�ndim�anyri )rc r6 rV rM rh Z
inferred_typer- r- r. rb � sj
"
rb c s e Zd ZdZdZeed�dd��Zd*ej ej e
d�� fdd �
Z d
d � Z d d
� Z
dd� Zed+e
d d�dd��Zed,e
d d�dd��Zej ejfZed�dd�Zeej ej f d�dd�Zd-e
ed�d d �Zej d�d d �Zed d!� �Z d.d#d$�Z ed%�d&d'�Z ed(d)� �Z � Z S )/r9 a\
Array of integer (optional missing) values.
.. versionadded:: 0.24.0
.. versionchanged:: 1.0.0
Now uses :attr:`pandas.NA` as the missing value rather
than :attr:`numpy.nan`.
.. warning::
IntegerArray is currently experimental, and its API or internal
implementation may change without warning.
We represent an IntegerArray with 2 numpy arrays:
- data: contains a numpy integer array of the appropriate dtype
- mask: a boolean array holding a mask on the data, True is missing
To construct an IntegerArray from generic array-like input, use
:func:`pandas.array` with one of the integer dtypes (see examples).
See :ref:`integer_na` for more.
Parameters
----------
values : numpy.ndarray
A 1-d integer-dtype array.
mask : numpy.ndarray
A 1-d boolean-dtype array indicating missing values.
copy : bool, default False
Whether to copy the `values` and `mask`.
Attributes
----------
None
Methods
-------
None
Returns
-------
IntegerArray
Examples
--------
Create an IntegerArray with :func:`pandas.array`.
>>> int_array = pd.array([1, None, 3], dtype=pd.Int32Dtype())
>>> int_array
<IntegerArray>
[1, <NA>, 3]
Length: 3, dtype: Int32
String aliases for the dtypes are also available. They are capitalized.
>>> pd.array([1, None, 3], dtype='Int32')
<IntegerArray>
[1, <NA>, 3]
Length: 3, dtype: Int32
>>> pd.array([1, None, 3], dtype='UInt16')
<IntegerArray>
[1, <NA>, 3]
Length: 3, dtype: UInt16
r )r$ c C s t t| jj� S )N)rF rG rw r6 )r+ r- r- r. r6 c s zIntegerArray.dtypeF)rc rV rM c s6 t |tj�o|jjdks td��t� j|||d� d S )Nr0 r3 zIvalues should be integer numpy array. Use the 'pd.array' function instead)rM )r0 r3 )r= r5 �ndarrayr6 r1 rg �super�__init__)r+ rc rV rM )� __class__r- r. r� g s zIntegerArray.__init__c C s t | �| j | j�S )N)r7 rw rx )r+ r- r- r. �__neg__o s zIntegerArray.__neg__c C s | S )Nr- )r+ r- r- r. �__pos__r s zIntegerArray.__pos__c C s t | �tj| j�| j�S )N)r7 r5 �absrw rx )r+ r- r- r. �__abs__u s zIntegerArray.__abs__N)rM r$ c C s t |||d�S )N)r6 rM )rd )r: �scalarsr6 rM r- r- r. �_from_sequencex s zIntegerArray._from_sequencec C s t |dd�}| j|||�S )N�raise)�errors)r r� )r: �stringsr6 rM r� r- r- r. �_from_sequence_of_strings| s z&IntegerArray._from_sequence_of_strings)�methodc s |dkrt d��|jdf �}x&|| D ]}t|| jtf �s&tS q&W tj| ||f|�|�}|tk rf|S tj t
| �t d�� g }x8|D ]0}t|t�r�� |j O � |j
|j� q�|j
|� q�W � fdd��t||�||�}t|t�r�t�fdd�|D �� n�|�S d S ) N�reducez%The 'reduce' method is not supported.�out)r6 c s* t | j�r � j� }t| |�S tj| � < | S )N)r r6 rM r9 r5 r{ )�x�m)rV r- r. � reconstruct� s
z1IntegerArray.__array_ufunc__.<locals>.reconstructc 3 s | ]}� |�V qd S )Nr- )r@ r� )r� r- r. rB � s z/IntegerArray.__array_ufunc__.<locals>.<genexpr>)�NotImplementedError�getr= �_HANDLED_TYPESr9 �NotImplementedr Z!maybe_dispatch_ufunc_to_dunder_opr5 �zerosry r^ rx rS rw �getattr�tuple) r+ Zufuncr� �inputs�kwargsr� r� �resultZinputs2r- )rV r� r. �__array_ufunc__� s,
z IntegerArray.__array_ufunc__c C s t || jd�S )N)r6 )rb r6 )r+ �valuer- r- r. �_coerce_to_array� s z IntegerArray._coerce_to_arrayTc C s� ddl m} ddlm} t|�}| j|kr:|r6| j� S | S t||�r�| jj |j
|d�}|| jkrf| j n| j j� }|j � ||dd�S t||�r�|j � j
| dd�S t|�r�tj}nt|�r�tjd�}ntj}| j||dd�S )a�
Cast to a NumPy array or ExtensionArray with 'dtype'.
Parameters
----------
dtype : str or dtype
Typecode or data-type to which the array is cast.
copy : bool, default True
Whether to copy the data, even if not necessary. If False,
a copy is made only if the old dtype does not match the
new dtype.
Returns
-------
ndarray or ExtensionArray
NumPy ndarray, BooleanArray or IntegerArray with 'dtype' for its dtype.
Raises
------
TypeError
if incompatible type with an IntegerDtype, equivalent of same_kind
casting
r )r" )� StringDtype)rM FZNaT)r6 �na_valuerM )Zpandas.core.arrays.maskedr" Zpandas.core.arrays.string_r� r r6 rM r= rw rf r8 rx r; r� r r5 r{ r Z
datetime64r Z
no_defaultZto_numpy)r+ r6 rM r" r� rU rV r� r- r- r. rf � s"
zIntegerArray.astypec C s* | j j� }| jj� r&|j� d || j< |S )a
Return values for sorting.
Returns
-------
ndarray
The transformed values should maintain the ordering between values
within the array.
See Also
--------
ExtensionArray.argsort
r )rw rM rx r~ �min)r+ rU r- r- r. �_values_for_argsort� s
z IntegerArray._values_for_argsortc s8 � j �t� j �� �fdd��}d� j � d�}t||| �S )Nc s0 ddl m} d }t||tf�r.|j|j }}n<t|�rjtj|�}|j dkrRt
d��t | �t |�krjt d��|t
jkr�tj| jjdd�}tj| jjdd�}nltj� �^ tjdd t� tjdd
��" t| jd �� d ��}||�}W d Q R X |tkr�t| j|� �}W d Q R X |d k�r | jj� }n
| j|B }|||�S ) Nr )� BooleanArrayr z(can only perform ops with 1-d structuresz Lengths must match to comparer^ )r6 �ignoreZ elementwise)rD �__)Z
pandas.arraysr� r= r9 rw rx r r5 �asarrayr} r� ry rv �
libmissing�NAr� �shapeZones�warnings�catch_warnings�filterwarnings�
FutureWarning�errstater� r� r rM )r+ �otherr� rV r� r� )�op�op_namer- r. �
cmp_method s2
z:IntegerArray._create_comparison_method.<locals>.cmp_methodr� )rX r r
)r: r� r� r\ r- )r� r� r. �_create_comparison_method� s 0z&IntegerArray._create_comparison_methodr c K s& t jf |� tj| j| j||d�}|S )N)rc rV rj � min_count)�nvZ validate_sumr �sumrw rx )r+ rj r� r� r� r- r- r. r� 5 s zIntegerArray.sum)r� c C s8 t |�st|�s|dkr&tj||< |S t| �||dd�S )z�
Parameters
----------
result : array-like
mask : array-like bool
other : scalar or array-like
op_name : str
�rtruediv�truedivF)rM )r� r� )r r r5 r{ r7 )r+ r� rV r� r� r- r- r. �_maybe_mask_result< s
z IntegerArray._maybe_mask_resultc s8 � j �t� j �� �fdd��}d� j � d�}t||| �S )Nc
s0 d }t |dd�dkr td��t|t�r6|j|j }}nxt|�r�tj|�}|j dkrZtd��t
| �t
|�krrt d��t |�p�t
|�s�td��n"t|�p�t|�p�|tjks�td��|d kr�| jj� }|tjkr�|dO }n
| j|B }�dk�rHtj| jdk| j @ d |�}|d k �r(tj|dk| @ d |�}n |tjk �r�tj|dkd |�}nh�d
k�r�|d k �rvtj|dk| @ d |�}n |tjk �r�tj|dkd |�}tj| jdk| j @ d |�}|tjk�r�tj| j�}n$tjd d �� � | j|�}W d Q R X �d
k�r |\}}| j|||d�| j|||d�fS | j|||��S )Nr} r r z(can only perform ops with 1-d structureszLengths must matchz(can only perform ops with numeric valuesT�powFZrpowr� )rD �divmod�floordiv�mod)r� r� r= r9 rw rx r r5 r� r} ry rv r r rg r r r� r� rM �whereZ ones_liker� r� )r+ r� ZomaskrV r� �divr� )r� r� r- r. �integer_arithmetic_methodT sT
zIIntegerArray._create_arithmetic_method.<locals>.integer_arithmetic_methodr� )rX r r
)r: r� r� r\ r- )r� r� r. �_create_arithmetic_methodP s Cz&IntegerArray._create_arithmetic_method)F)NF)NF)T)Tr )!rX rY rZ r[ Z_internal_fill_valuer r# r6 r5 r r^ r� r� r� r� ra r� r� �numbers�Numberr� rG r� r r� r rf r� r� r� r� r� �
__classcell__r- r- )r� r. r9 s, D -6 7
z�
An ExtensionDtype for {dtype} integer data.
.. versionchanged:: 1.0.0
Now uses :attr:`pandas.NA` as its missing value,
rather than :attr:`numpy.nan`.
Attributes
----------
None
Methods
-------
None
c @ s" e Zd ZejZdZejdd�Z dS )� Int8DtypeZInt8�int8)r6 N)
rX rY rZ r5 r� r7 r\ �_dtype_docstring�formatr[ r- r- r- r. r� � s r� c @ s" e Zd ZejZdZejdd�Z dS )�
Int16DtypeZInt16�int16)r6 N)
rX rY rZ r5 r� r7 r\ r� r� r[ r- r- r- r. r� � s r� c @ s" e Zd ZejZdZejdd�Z dS )�
Int32DtypeZInt32�int32)r6 N)
rX rY rZ r5 r� r7 r\ r� r� r[ r- r- r- r. r� � s r� c @ s" e Zd ZejZdZejdd�Z dS )�
Int64DtypeZInt64rp )r6 N)
rX rY rZ r5 rp r7 r\ r� r� r[ r- r- r- r. r� � s r� c @ s" e Zd ZejZdZejdd�Z dS )�
UInt8DtypeZUInt8�uint8)r6 N)
rX rY rZ r5 r� r7 r\ r� r� r[ r- r- r- r. r� � s r� c @ s" e Zd ZejZdZejdd�Z dS )� UInt16DtypeZUInt16�uint16)r6 N)
rX rY rZ r5 r� r7 r\ r� r� r[ r- r- r- r. r� � s r� c @ s" e Zd ZejZdZejdd�Z dS )� UInt32DtypeZUInt32�uint32)r6 N)
rX rY rZ r5 r� r7 r\ r� r� r[ r- r- r- r. r� � s r� c @ s" e Zd ZejZdZejdd�Z dS )� UInt64DtypeZUInt64�uint64)r6 N)
rX rY rZ r5 r� r7 r\ r� r� r[ r- r- r- r. r� � s r� )r� r� r� rp r� r� r� r� )NF)NF)Jr� Ztypingr r r r r r r r� Znumpyr5 Z pandas._libsr r
r� Zpandas._typingr r Z
pandas.compatr
Zpandas.compat.numpyr r� Zpandas.util._decoratorsr Zpandas.core.dtypes.baser Zpandas.core.dtypes.commonr r r r r r r r r Zpandas.core.dtypes.missingr Z pandas.corer Zpandas.core.array_algosr Zpandas.core.opsr Zpandas.core.ops.commonr Zpandas.core.tools.numericr Zmaskedr! r" rN r# r^ rd ri r rb r9 Z_add_arithmetic_opsZ_add_comparison_opsr� r� r� r� r� r� r� r� r� rF rG r- r- r- r. �<module> sp $ , f` |