3
���h>t � @ sf d dl m Z mZ d dlZd dlZd dlmZmZmZmZ d dl Z d dl
Z d dl m
Z
mZmZmZ d dljjZd dlmZ d dlmZ d dlmZ d dlm Z d d l m Z d d
l m Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z* d d l+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3m4Z4m5Z5m6Z6m7Z7m8Z8m9Z9m:Z:m;Z;m<Z<m=Z=m>Z>m?Z?m@Z@ d d lAmBZB d d
lCmDZDmEZEmFZFmGZG d dlHmIZImJZJmKZK d dlLjMjNZd dlOmPZP d dlQmRZRmSZSmTZTmUZUmVZVmWZW d dlXmYZY d dlZjMj[Z\d dl]m^Z^ d dl_m`Z`maZambZb d dlcjMjdZdd dlemfZf e�r
d dlgmhZh G dd� deY�ZiG dd� dei�ZjG dd� dej�ZkG d d � d ei�ZlG d d � d el�ZmG d d!� d!em�ZnG d"d#� d#em�ZoG d$d%� d%el�ZpG d&d'� d'�ZqG d(d)� d)eqei�ZrG d*d+� d+ejer�ZsG d,d-� d-eqep�ZtG d.d/� d/el�ZuG d0d1� d1ei�ZvG d2d3� d3ej�ZwdFd4d5�ZxdGd6d7�ZydHd8d9�ZzdIe e{e d;�d<d=�Z|d>d?� Z}e j~e j~e j~d@�dAdB�Ze e j~dC�dDdE�Z�dS )J� )�datetime� timedeltaN)�
TYPE_CHECKING�Any�List�Optional)�NaT�algos�lib�writers)�BlockPlacement)�
conversion)�
tz_compare)� ArrayLike)�validate_bool_kwarg) �astype_nansafe� convert_scalar_for_putitemlike�find_common_type�infer_dtype_from�infer_dtype_from_scalar�maybe_downcast_numeric�maybe_downcast_to_dtype�maybe_infer_dtype_type�
maybe_promote� maybe_upcast�soft_convert_objects)� DT64NS_DTYPE� TD64NS_DTYPE�
is_bool_dtype�is_categorical_dtype�is_datetime64_dtype�is_datetime64tz_dtype�is_dtype_equal�is_extension_array_dtype�is_float�is_float_dtype�
is_integer�is_integer_dtype�is_interval_dtype� is_list_like�is_object_dtype�is_period_dtype�is_re�is_re_compilable� is_sparse�is_timedelta64_dtype� pandas_dtype)�ExtensionDtype)� ABCDataFrame�
ABCIndexClass�ABCPandasArray� ABCSeries)� _isna_compat�is_valid_nat_for_dtype�isna)�shift)� Categorical�
DatetimeArray�ExtensionArray� PandasArray� PandasDtype�TimedeltaArray)� PandasObject)�
extract_array)�check_setitem_lengths�is_empty_indexer�is_scalar_indexer)�
nanpercentile)�Indexc @ s� e Zd ZdZdddgZdZdZdZdZdZ dZ
dZ dZ dZ
dZdZdZdZdZeeeed d�dd ��Zd�d d �Zd
d� Zedd� �Zedd� �Z ee d�dd��Z ee d�dd��Z dd� Z dd� Z!e"d�d d �Z#d�d d �Z$e%j&d�d d!�Z'ed"d#� �Z(ed$d%� �Z)e)j*d&d%� �Z)d�d d�d'd(�Z+d�d)d*�Z,e-d�d+d,�Z.ed�d-d.�Z/d/d0� Z0d1d2� Z1d3d4� Z2d�d5d6�Z3ed7d8� �Z4ed9d:� �Z5d;d<� Z6d=d>� Z7d
d�d?d@�Z8e9d d�dAdB�Z:e9d d�dCdD�Z;d�e e9d dE�dFdG�Z<e e9d dE�dHdI�Z=d�e9d e9d dJ�dKdL�Z>d�dMdN�Z?d�e e-dP�dQdR�Z@d�e e e e e dS�dTdU�ZAeBe dV�dWdX�ZCee dY�dZd[�ZDd�d]d^�ZEd�e d_�d`da�ZFd�e e e db�dcdd�ZGdedf� ZHdgdh� ZId�e ee e9d dj�dkdl�ZJdmdn� ZKd�e-eeLdq e eLe e-eLe- eLeB e eLe- dr�
dsdt�ZMd�e-ee eLe eLeB e eLe- e9d du�dvdw�ZNd�e-dqeLeB eeLe e-eLe- e eLe- e9d dx�
dydz�ZOd
ePjQfed{�d|d}�ZRd�eee9d d�d�d��ZSd�eed��d�d��ZTd�e ee9d d��d�d��ZUd�d�� ZVd�ed{�d�d��ZWd�e e e db�d�d��ZXd
S )��Blockz�
Canonical n-dimensional unit of homogeneous dtype contained in a pandas
data structure
Index-ignorant; let the container take care of that
� _mgr_locs�values�ndimFT)rI � placementrJ �returnc C s t j| �}||_||_||_|S )z<
Fastpath constructor, does *no* validation
)�object�__new__rJ rI rH )�clsrI rK rJ �obj� rQ �>/tmp/pip-build-5_djhm0z/pandas/pandas/core/internals/blocks.py� _simple_newo s
zBlock._simple_newNc C s^ | j ||�| _|| _|| _| jrZ| jrZt| j�t| j�krZtdt| j�� dt| j�� ���d S )Nz Wrong number of items passed z, placement implies )� _check_ndimrJ �mgr_locsrI �_validate_ndim�len�
ValueError)�selfrI rK rJ rQ rQ rR �__init__| s zBlock.__init__c C s: |dkr|j }| jr6|j |kr6td|j � d|� d���|S )a�
ndim inference and validation.
Infers ndim from 'values' if not provided to __init__.
Validates that values.ndim and ndim are consistent if and only if
the class variable '_validate_ndim' is True.
Parameters
----------
values : array-like
ndim : int or None
Returns
-------
ndim : int
Raises
------
ValueError : the number of dimensions do not match
Nz1Wrong number of dimensions. values.ndim != ndim [z != �])rJ rV rX )rY rI rJ rQ rQ rR rT � s zBlock._check_ndimc C s dS )z�
The array-like that can hold the underlying values.
None for 'Block', overridden by subclasses that don't
use an ndarray.
NrQ )rY rQ rQ rR �_holder� s z
Block._holderc C s | j | jjfS )N)�_can_consolidate�dtype�name)rY rQ rQ rR �_consolidate_key� s zBlock._consolidate_key)rL c C s | j jdk S )z* return a boolean if I am possibly a view N)rI �base)rY rQ rQ rR �is_view� s z
Block.is_viewc C s | j p
| jS )z$ return True if I am a non-datelike )� is_datetime� is_timedelta)rY rQ rQ rR � is_datelike� s zBlock.is_datelikec C s | j S )aC
The array that Series.values returns (public attribute).
This has some historical constraints, and is overridden in block
subclasses to return the correct array (e.g. period returns
object ndarray and datetimetz a datetime64[ns] ndarray instead of
proper extension array).
)rI )rY rQ rQ rR �external_values� s zBlock.external_valuesc C s | j S )zJ
The array that Series._values returns (internal values).
)rI )rY rQ rQ rR �internal_values� s zBlock.internal_valuesc C s
t | j�S )zP
The array that Series.array returns. Always an ExtensionArray.
)r= rI )rY rQ rQ rR � array_values� s zBlock.array_valuesc C s t |�r| jjt�S | jS )z�
return an internal format, currently just the ndarray
this is often overridden to handle to_dense like operations
)r* rI �astyperM )rY r^ rQ rQ rR �
get_values� s zBlock.get_valuesc C s t j| j�j| j�S )z2
This is used in the JSON C code.
)�np�asarrayrI �reshape�shape)rY rQ rQ rR �get_block_values_for_json� s z Block.get_block_values_for_jsonc C s t jS )N)rk �nan)rY rQ rQ rR �
fill_value� s zBlock.fill_valuec C s | j S )N)rH )rY rQ rQ rR rU � s zBlock.mgr_locsc C s t |tj�stj|�}|| _d S )N)�
isinstance� libinternalsr rH )rY � new_mgr_locsrQ rQ rR rU � s
c C s2 |dkr| j }| jr"t|| jd�}t||| jd�S )zm
Create a new block, with type inference propagate any values that are
not specified
N)rJ )rK rJ )rU � is_extension� _block_shaperJ �
make_block)rY rI rK rQ rQ rR rw � s
zBlock.make_blockc C s. |dkr| j }|dkr | j}t| �|||d�S )z4 Wrap given values in a block of same type as self. N)rK rJ )rU rJ �type)rY rI rK rJ rQ rQ rR �make_block_same_class s
zBlock.make_block_same_classc C sj t | �j}| jdkr0|� dt| �� d| j� �}n6djdd� | jD ��}|� d| jj� d|� d| j� �}|S ) N� z: z dtype: z x c s s | ]}t |�V qd S )N)�str)�.0�srQ rQ rR � <genexpr> s z!Block.__repr__.<locals>.<genexpr>z, z , dtype: ) rx �__name__rJ rW r^ �joinrn rU �indexer)rY r_ �resultrn rQ rQ rR �__repr__ s
zBlock.__repr__c C s
t | j�S )N)rW rI )rY rQ rQ rR �__len__ s z
Block.__len__c C s | j j| jfS )N)rU r� rI )rY rQ rQ rR � __getstate__ s zBlock.__getstate__c C s( t j|d �| _|d | _| jj| _d S )Nr rz )rs r rU rI rJ )rY �staterQ rQ rR � __setstate__ s
zBlock.__setstate__c C s
| j | S )z return a slice of my values )rI )rY �slicerrQ rQ rR �_slice s z Block._slicec C st |dkr*t |t�r|d n|}| j| }nt |t�s<t|�}| j|�}| jr`|j| jkr`td��t| �j ||| j�S )z�
Perform __getitem__-like, return result as block.
As of now, only supports slices that preserve dimensionality.
Nr z Only same dim slicing is allowed)
rr �tuplerU r r� rV rJ rX rx rS )rY r� rt Z axis0_slicer�
new_valuesrQ rQ rR �
getitem_block$ s
zBlock.getitem_blockc C s | j jS )N)rI rn )rY rQ rQ rR rn 7 s z Block.shapec C s | j jS )N)rI r^ )rY rQ rQ rR r^ ; s z Block.dtypec C s
| j | S )N)rI )rY �irQ rQ rR �iget? s z
Block.igetc C s || j |< dS )z�
Modify block values in-place with new item value.
Notes
-----
`set` never creates a new array or new Block, whereas `setitem` _may_
create a new array and always creates a new Block.
N)rI )rY �locsrI rQ rQ rR �setB s z Block.setc C s$ t j| j|d�| _| jj|�| _dS )z;
Delete given loc(-s) from block in-place.
r N)rk �deleterI rU )rY �locrQ rQ rR r� M s z Block.deletec
K s0 t jdd�� || jf|�}W dQ R X | j|�S )z[
apply the function to my values; return a block if we are not
one
�ignore)�allN)rk ZerrstaterI �_split_op_result)rY �func�kwargsr� rQ rQ rR �applyT s z Block.applyc C sn t |�rT|jdkrTg }x8t| j�D ]*\}}|| }| j||gd�}|j|� q"W |S t|t�sh| j|�}|gS )Nrz )rI rK )r# rJ � enumeraterU rw �appendrr rG )rY r� Znbsr� r� �vals�blockrQ rQ rR r� ^ s
zBlock._split_op_result)�inplacerL c s� t � d�� t�j�}�dk rBtjd�d��d||j�jd ��k< �js\� rR�gS �j� gS �j ��r��j
|�� d�}�j ||�S |j � s�� r��gS �j� gS � ���fdd�}�j
d|� �S ) zs
fillna on the block with the value. If we fail, then convert to
ObjectBlock and try again
r� N)�limitFrz )r� c s8 �j ��}|d k r&|jt||d ��}|j��� d d�S )Nrz )r� r� �downcast)�coerce_to_target_dtyper� �slice�fillna)�mask�val�idxr� )r� r� rY �valuerQ rR �f� s
zBlock.fillna.<locals>.f)r r8 rI �libalgosZ_validate_limitZcumsumrJ � _can_hold_na�copy�_can_hold_element�putmask�_maybe_downcast�any�split_and_operate)rY r� r� r� r� r� �blocksr� rQ )r� r� rY r� rR r� o s
z Block.fillnac
s� |dkrt jd� jd�}� j}� fdd�}� jdkrl|j� rJ|||d�}n|rR|n|j� }||� j�}|gS g }xbt� j�D ]T\} }
|| } || } | j� r�|| | | �}n|r�| n| j� }|||
g�}|j |� q|W |S )a�
split the block per-column, and apply the callable f
per-column, return a new block for each. Handle
masking which will not change a block unless needed.
Parameters
----------
mask : 2-d boolean mask
f : callable accepting (1d-mask, 1d values, indexer)
inplace : boolean
Returns
-------
list of blocks
NT)rn c sZ t | t�r:t| �dks t| ��t | d t�s0t�| d }n t| � jd�} � j| |d�}|S )Nrz r )rJ )rI rK )rr �listrW �AssertionErrorrG rv rJ rw )�nv�ref_locr� )rY rQ rR � make_a_block� s
z-Block.split_and_operate.<locals>.make_a_blockrz )
rk Z broadcast_torn rI rJ r� r� rU r� r� )
rY r� r� r� r� r� r� r� �
new_blocksr� r� �m�vrQ )rY rR r� � s(
zBlock.split_and_operate)r� rL c s4 � d kr | j s| js| jr |S t� fdd�|D ��S )Nc s g | ]}|j � ��qS rQ )r� )r| �b)r� rQ rR �
<listcomp>� s z)Block._maybe_downcast.<locals>.<listcomp>)r$ rd rc �_extend_blocks)rY r� r� rQ )r� rR r� � s zBlock._maybe_downcastc C s� |dkr | S | j }| jdkr<|dkr(d}t||�}| j|�S |dkrH| S |dkpXt|t�sdtd��n|dkrttd��dd� }| jd|d�S ) z< try to downcast each item to the dict of dtypes if present Frz N�inferz:downcast must have a dictionary or 'infer' as its argumentz#dtypes as dict is not supported yetc S s t |dd�}|S )Nr� )r^ )r )r� r� r� rQ rQ rR r� � s zBlock.downcast.<locals>.f) rI rJ r rw rr �dictrX r� r� )rY ZdtypesrI r� r� rQ rQ rR r� � s"
zBlock.downcast�raise)r� �errorsc
C s d}||kr*dt |�� d|� d�}t|��tj|�rTt|t�rTd|j� d�}t|��|dk rdt|�}t |�r�t | j
j �r�| j | j
j
||d ��S | j t| j
|d
��S t|�}t| j |�r�|r�| j� S | S | j�ry| j
j
|�}W n. ttfk
�r |dk�r| j
}n� Y nX n�t|jt��r8| j�r0| j� }n| j
}n | j|d
�}|j� }yt||d d �}W n8 ttfk
�r� |dk�r�� |�r�| j� n| } | S X t|tj��r�|j | j �}t || j | j d �} | j �r| j �r| j | j k�rtd
|� d| j j!� d| j � d| j j!� d| j � d� ��| S )a�
Coerce to the new dtype.
Parameters
----------
dtype : str, dtype convertible
copy : bool, default False
copy if indicated
errors : str, {'raise', 'ignore'}, default 'ignore'
- ``raise`` : allow exceptions to be raised
- ``ignore`` : suppress exceptions. On error return original object
Returns
-------
Block
r� r� z.Expected value of kwarg 'errors' to be one of z. Supplied value is '�'zExpected an instance of z7, but got the class instead. Try instantiating 'dtype'.N)r� )r^ T)rK rJ z cannot set astype for copy = [z
] for dtype (z [z]) to different shape (z]))r� r� )"r� rX �inspectZisclass�
issubclassr1 r � TypeErrorr0 r rI r^ rw ri r: r" r� ru rx r{ re �to_native_typesrj �ravelr rr rk �ndarrayrm rn rU rJ �
is_numericr_ )
rY r^ r� r� Zerrors_legal_valuesZ invalid_arg�msgrI Zvals1d�newbrQ rQ rR ri s\
2z Block.astype)r� r �numericr �coercec C s |r | j � S | S )z�
attempt to coerce any object types to better types return a copy
of the block (if copy = True) by definition we are not an ObjectBlock
here!
)r� )rY r� r r� r r� rQ rQ rR �convertk s
z
Block.convert)�elementrL c C s0 | j jj}t|�}|dk r&t|j|�S t||�S )z% require the same dtype as ourselves N)rI r^ rx r r� rr )rY r� r^ �tiporQ rQ rR r� z s
zBlock._can_hold_element)r� rL c C s t |j| j�S )z�
Should we set self.values[indexer] = value inplace or do we need to cast?
Parameters
----------
value : np.ndarray or ExtensionArray
Returns
-------
bool
)r" r^ )rY r� rQ rQ rR � should_store� s zBlock.should_storerp c K sx | j }t|�}tj|�}| j r^| r^|r^|jt�}|jjt jd�j |k rl|jd|� ��}nt j
|dd�}|||< |S )z$ convert to our native types format ZU1z<UrM )r^ ) rI r8 r Zword_len� is_objectri r{ r^ �itemsizerk �array)rY �na_rep�quotingr� rI r� r� rQ rQ rR r� � s
zBlock.to_native_types)�deepc C s" | j }|r|j� }| j|| jd�S )z copy constructor )rJ )rI r� ry rJ )rY r� rI rQ rQ rR r� � s z
Block.copy)r� �regexr� c
sv t � d�� |}�j|�s�t|t�s6� r,�gS �j� gS �fdd�|D �}t|�sd� rZ�gS �j� gS t|�dkr��j|d |� ||d�S t��r�t��j t
�}|j||� ||d�S �j }t j
|�r�t|tj�r�t||j�}tj||�} y�j| |� d�}
W n^ ttfk
�rX t���r � �j�s:�j|� �s:t|���j t
�}|j||� ||d�S X |�rr� fd d�|
D �}
|
S )
z�
replace the to_replace value with value, possible to create new
blocks here this is just a call to putmask. regex is not used here.
It is used in ObjectBlocks. It is here for API compatibility.
r� c s g | ]}� j |�r|�qS rQ )r� )r| �x)rY rQ rR r� � s z!Block.replace.<locals>.<listcomp>rz r )r� r� r� )�
to_replacer� r� r� r� )r� c s g | ]}|j d � d��qS )F)r� r� )r� )r| r� )r� rQ rR r� s )r r� rr r� r� rW �replacer* r� ri rM rI r
� is_scalarrk r� r r^ �missingZ mask_missingr� r� rX ru ) rY r� r� r� r� r� Zoriginal_to_replacer� rI r� r� rQ )r� rY rR r� � sZ
z
Block.replacec O s |d r | S | j � S )z no-op on a non-ObjectBlock r� )r� )rY �argsr� rQ rQ rR �_replace_single
s zBlock._replace_singlec C s | j dk}t|tj�r2|j | j kr2td| j � ���|dkrF| jrFtj}| j}| j|�rnt j
|�r�t ||j �}n�t
|d�r�|j }nLt j
|�r�t|� r�t|dd�\}}n&ttj|�j �\}}| j|�j||�S t|j |g�}t| j |� s�t�| j|�j||�S tt|dd���rd}|}nd}tj|�}|�r4|j}t|||� t|j ��ol|j d |j d k�ol|j |j k} t ||��r|n�t || j ��r�|||< nn| �r�t |j ��r�|||< | j!t"| j|j d ��S | �r�|�r�| j!|�S | �r�|||< |j|j dd
�}n|||< |�r|j}| j!|�}
|
S ) a�
Attempt self.values[indexer] = value, possibly creating a new array.
Parameters
----------
indexer : tuple, list-like, array-like, slice
The subset of self.values to set
value : object
The value being set
Returns
-------
Block
Notes
-----
`indexer` is a direct slice/positional indexer. `value` must
be a compatible shape.
� z Cannot set values with ndim > Nr^ T)r0 Fr )r^ )r� )#rJ rr rk r� rX r� rp rI r� r
r� r r^ �hasattrr8 r r r� ri �setitemr r" r� r# �getattr�TrB rW rn �sizerC rD r rw r: ) rY r� r� � transposerI r^ �_Z is_ea_valueZ arr_valueZ exact_matchr� rQ rQ rR r� s^
z
Block.setitemr )r� �axisr� rL c
sV t |�}t� tttf� s t�| j}t� � rDt� �rD| j rD| j
� | j � ��rtt j
� �rht� | jj�� |rr|j}t� dd�dkr�| jd � jkr�|dkr�tj� |jd �j| j�� � j|j�� || jkr�| r�|j� }t� �o�tj|| �o�t� dd�dk�rd|| jd t� �k�r(tj||� � n:|jd
t� �k�sJt� �dk�rZtj ||� � nt d��ntj ||� � n�|j� �r|�r�|j}t� tj ��r�� j� |j| d }t� dd�dk�r�| jd � jk�r�t � j�}|j |d� � jt!|��� � fdd�}| j"|||�} | S |�r | gS |�rJ|dk�rD|�r:| jn| jj� }|j}| j#|�gS ) a
putmask the data to the block; it is possible that we may create a
new dtype of block
Return the resulting block(s).
Parameters
----------
mask : np.ndarray[bool], SparseArray[bool], or BooleanArray
new : a ndarray/object
inplace : bool, default False
Perform inplace modification.
axis : int
transpose : bool, default False
Set to True if self is stored with axes reversed.
Returns
-------
List[Block]
rJ r rz z-cannot assign mismatch length to masked arrayc sf |d kr� }nHt � tj�r4tj� |� jd �}n
tj� �}t|j�\}}|j|�}t || |�}|S )Nr )
rr rk r� Zsqueezern r� r r^ ri �_putmask_smart)r� r� r� �nr^ r� r� )�newrQ rR r� � s
zBlock.putmask.<locals>.fN���r� r� )$�_extract_bool_arrayrr r3 r5 r2 r� rI r) r8 r� rq r� r
r� r r^ r� r� rJ rk �repeatrn rm ri r� r� rW �placer� rX r� r� �insertr� r� rw )
rY r� r� r� r� r� r� � new_shaper� r� rQ )r� rR r� � s\
"
z
Block.putmaskc
C sJ t |dd�\}}t| j|�r | S | js6t|�s6t|�r@| jt�S | jsL| j r`t
|�s\t |�r`| S | j svt
|�svt|�r�t
|�s�t|�o�| j s�| jt�S t| jdd�}t|dd�}t||�s�| jt�S td| � d|� ���n>| js�t|��rt|�o�| j�s| jt�S td| � d|� ���y
| j|�S tttfk
�rD | jt�S X dS )z�
coerce the current block to a dtype compat for other
we will return a block, possibly object, and not raise
we can also safely try to coerce to the same dtype
and will receive the same block
T)r0 �tzNz.possible recursion in coerce_to_target_dtype: � )r r" r^ �is_boolr* r ri rM r$ �
is_complexr' r% rc r r! r� r r� rd r/ rX r� �
OverflowError)rY �otherr^ r� ZmytzZothertzrQ rQ rR r� s:
z Block.coerce_to_target_dtype�pad�forwardrF )
�methodr� �indexr� r� �limit_direction�
limit_arearq r� r� c
K s� t |d�}| js| jr.| j r.|r&| S | j� S ytj|�} W n tk
rT d } Y nX | d k rv| j| ||||| |
d�S tj |f| �} |d k s�t
�| j f | ||||||||
d� | ��S )Nr� )r� r� r� r� rq r� r� ) r� r� r� r� r� r� rq r� r� ) r r� r& rd r� r� Zclean_fill_methodrX �_interpolate_with_fillZclean_interp_methodr� � _interpolate)
rY r� r� r� r� r� r� r� rq r� r� r� r� rQ rQ rR � interpolate@ s:
zBlock.interpolate)r� r� r� r� rq r� r� rL c
C s� t |d�}|r(| js(|r | gS | j� gS |r2| jn| jj� }t|| jj�}tj|||||| jd�}| j|| j d�g} | j
| |�S )z, fillna but using the interpolate machinery r� )r� r� r� rq r^ )rJ ) r r� r� rI r r^ r� Zinterpolate_2dry rJ r� )
rY r� r� r� r� rq r� r� rI r� rQ rQ rR r� w s"
z Block._interpolate_with_fill)
r� r� rq r� r� r� r� r� r� rL c
s� t |d�}|r| jn| jj� } | js<| js0| gS | jtj�} � dkrJ| j� �d
krf�j sft
�� d���tj tj d�� ������fdd � } tj | || �}
| j
|
�g}| j|| �S ) z" interpolate using scipy wrappers r� N�krogh�piecewise_polynomial�pchipz4 interpolation requires that the index be monotonic.)�yvaluesrL c s" t jf �| ����� dd����S )NF)Zxvaluesr r� r� r� r� rq Z bounds_error)r� Zinterpolate_1d)r )rq r� r� r� r� r� r� rQ rR r� � s z Block._interpolate.<locals>.func)r� r r )r rI r� r$ r& ri rk Zfloat64rq Z is_monotonicrX r� Zapply_along_axisry r� )rY r� r� rq r� r� r� r� r� r� r� �datar� Z
interp_valuesr� rQ )rq r� r� r� r� r� r� rR r� � s
$ zBlock._interpolate)r� c C s� | j }|tjkr | j}d}nd}tj|||||d�}|dkoB|dk sJt�|dkrX| j}t|j | j �sr| j
||�S | j ||�S dS )zQ
Take values according to indexer and return them as a block.bb
FT)r� �
allow_fillrq r N) rI r
�
no_defaultrq r �take_ndr� rU r" r^ rw ry )rY r� r� rt rq rI r r� rQ rQ rR r � s
z
Block.take_ndrz )r� r� rL c C s" t j| j||dd�}| j|d�gS )z) return block for the diff of the values � )r� �
stacklevel)rI )r �diffrI rw )rY r� r� r� rQ rQ rR r � s z
Block.diff)�periodsr� c C s* t | j|�\}}t||||�}| j|�gS )z- shift the block by periods, possibly upcast )r rI r9 rw )rY r
r� rq r� rQ rQ rR r9 � s z Block.shift)�try_castr� rL c
s ddl jjj�t� �� t|tttf� s,t �|dks8t ��j
dk}�j }|}|rV|j }t
|dd�dkr�|j
d |j
kr�|dkr�|jt|jd ��}n|r�|j
�j
d kr�� j � t� d�s�td ����fd
d �} � jd �j� r�|}
nNy| � ||�}
W n< tk
�r. �j|�} | j|� |||d
�} �j| d�S X �j�sD�j
dk�r\|�rP|
j }
�j|
�gS � j
d }� j|d�� t j � fdd�t � jd �D �t d�}
g }x`|
|
gD ]R}|j � �r�|
j!|j"� d |d�}t#|�j$�}�j|j �j%| d�}|j&|� �q�W |S )a�
evaluate the block; return result block(s) from the result
Parameters
----------
other : a ndarray/object
cond : np.ndarray[bool], SparseArray[bool], or BooleanArray
errors : str, {'raise', 'ignore'}, default 'raise'
- ``raise`` : allow exceptions to be raised
- ``ignore`` : suppress exceptions. On error return original object
axis : int, default 0
Returns
-------
List[Block]
r Nr� r� r� rJ rz rn z0where must have a condition that is ndarray likec sb �j s �jo tj|�o tj|�sP�j|�s.t�tj|�rPt |tj
�rPt ||j �}� j
| ||�}|S )N)r& r� r
r$ rk �isnanr� r� r� rr r� r r^ �where)�condrI r� Zfastres)� expressionsrY rQ rR �
where_func0 s
z Block.where.<locals>.where_func�K)r� r r� r� c s g | ]}� | j � �qS rQ )r� )r| r� )r rQ rR r� ^ s z Block.where.<locals>.<listcomp>)r^ )r� )rK )r� r� )rz )'Z#pandas.core.computation.expressions�coreZ computationr r� rr r3 r5 r2 r� rJ rI r� r� rm r� rn r� rX r� r� r� r� r
r� r� rw Zswapaxesrk r� �range�boolr� �takeZnonzeror r^ rU r� )rY r� r r� r r� r� rI Z
orig_otherr r� r� r� r� �
result_blocksr� Ztaken�r�nbrQ )r r rY rR r
sR
&
z Block.wherec C sH |j | jj|d�\}}|jd�}|j| }|| }t||d�g}||fS )a�
Return a list of unstacked blocks of self
Parameters
----------
unstacker : reshape._Unstacker
fill_value : int
Only used in ExtensionBlock._unstack
Returns
-------
blocks : list of Block
New blocks of unstacked values.
mask : array_like of bool
The mask of columns of `blocks` we should keep.
)rq r )rK )�get_new_valuesrI r� r� rw )rY � unstackerrq �
new_placementr� r� r� rQ rQ rR �_unstackj s
zBlock._unstack�linearc
C s | j dkst�| j� }|j| dk}t|� }|r8|g}|rptjtj| jgt |� �t |��j
t |�t |��}nFtj t |��}t
|tj|�d || j||j |d�}tj|dd�}|j}|r�tj|� r�|jd
dks�t|j��|d }tj|�}tj |�} t|tjt |��| d �S ) a)
compute the quantiles of the
Parameters
----------
qs: a scalar or list of the quantiles to be computed
interpolation: type of interpolation, default 'linear'
axis: axis to compute, default 0
Returns
-------
Block
r� r �d )r� �na_valuer� rJ �
interpolationF)r� rz .)rK rJ r� ).r )rJ r� rj rn r) rk r� r� rq rW rm rl r8 rE r� r
r� Zitem_from_zerodimrw �arange)
rY �qsr r� rI Zis_emptyZ orig_scalarr� r� rJ rQ rQ rR �quantile� s6
zBlock.quantilec C s@ |j � r<|s&| j|�} | j|||d�S | j||||||d�S | S )a
Replace value corresponding to the given boolean array with another
value.
Parameters
----------
to_replace : object or pattern
Scalar to replace or regular expression to match.
value : object
Replacement object.
inplace : bool, default True
Perform inplace modification.
regex : bool, default False
If true, perform regular expression substitution.
convert : bool, default True
If true, try to coerce any object types to better types.
mask : array-like of bool, optional
True indicate corresponding element is ignored.
Returns
-------
A new block if there is anything to replace or the original block.
)r� )r� r� r� r� )r� r� r� r� )rY r� r� r� r� r� r� rQ rQ rR �_replace_coerce� s
zBlock._replace_coerce)N)N)N)NN)N)NFN)N)N)Fr� )TTTTF)rp N)T)FFT)Fr F)
r� r NFNr� NNFN)r� r FNNFN)Nr Nr� NFN)rz )r N)r� Fr )r r )TFFN)Yr �
__module__� __qualname__�__doc__� __slots__r� r$ r& r� rc �
is_datetimetzrd r� r� �is_categoricalru r� r] rV � classmethodr r �intrS rZ rT �propertyr\ r` r rb re rf rg r< rh rj rk r� ro rq rU �setterrw ry r{ r� r� r� r� r� r� rn r^ r� r� r� r r� r� r� r� r� r� ri r� r r� r� r� r� r� r� r� r� r� r r� r� r� r
r r r r9 r
r r# r$ rQ rQ rQ rR rG W s�
';
$k
Zu}A 6. " $, c < rG c s� e Zd ZdZdZdZdZdE� fdd� Zedd� �Z d d
� Z
e e d �d d
�Z
dd� ZdFe ee ed d�dd�Zdd� Zedd� �Zedd� �Zedd � �Zee d �d d ��Zed d!� �Zd"d#� ZdGd$d%�Zed �d&d'�ZdHd)d*�Z dde j fed+�d,d-�Z e e d.�d/d0�Z!d1d2� Z"dId3d4�Z#dJd6d7�Z$dKeeed d9�� fd:d;�
Z%dLeee ed d<�d=d>�Z&dMe eed d@�dAdB�Z'dCdD� Z(� Z)S )N�ExtensionBlocka
Block for holding extension types.
Notes
-----
This holds all 3rd-party extension array types. It's also the immediate
parent class for our internal extension types' blocks, CategoricalBlock.
ExtensionArrays are limited to 1-D.
FTNc st | j |�}t|tj�s tj|�}|dkr>t|�dkr:d}nd}t� j|||d� | jdkrpt| j�dkrpt d��dS )z�
Initialize a non-consolidatable block.
'ndim' may be inferred from 'placement'.
This will call continue to call __init__ for the other base
classes mixed in with this Mixin.
Nrz r� )rJ zblock.size != values.size)
�_maybe_coerce_valuesrr rs r rW �superrZ rJ rU r� )rY rI rK rJ )� __class__rQ rR rZ s
zExtensionBlock.__init__c C s* | j dkrt| j�fS t| j�t| j�fS )Nrz )rJ rW rI rU )rY rQ rQ rR rn s
zExtensionBlock.shapec C s� | j dkrtt|t�rt|\}}tj|� r@|dkr@t| � d���n*t|t�rj|td �kr^t|��| j|g S | j| S |dkr�t| � d���| jS d S )Nr� r z only contains one item) rJ rr r� �comZ
is_null_slice�
IndexErrorr� �NotImplementedErrorrI )rY �colr� rQ rQ rR r� # s
zExtensionBlock.iget)r� rL c C s t || j�S )z@
Can we set the given array-like value inplace?
)rr r\ )rY r� rQ rQ rR r� 4 s zExtensionBlock.should_storec C s |j � dgkst�|| _d S )Nr )�tolistr� rI )rY r� rI rQ rQ rR r� : s zExtensionBlock.setr rG )r� r� r� rL c C sp t |d�}t|�}|r | jn| jj� }t|tjtf�rNt|�t|�krN|| }t ||j
�}|||< | j |d�gS )z+
See Block.putmask.__doc__
r� )rI ) r r� rI r� rr rk r� r< rW �
_safe_reshapern rw )rY r� r� r� r� r� r� rQ rQ rR r� > s
zExtensionBlock.putmaskc C s t |�S )aF
Unbox to an extension array.
This will unbox an ExtensionArray stored in an Index or Series.
ExtensionArrays pass through. No dtype coercion is done.
Parameters
----------
values : Index, Series, ExtensionArray
Returns
-------
ExtensionArray
)rA )rY rI rQ rQ rR r0 R s z#ExtensionBlock._maybe_coerce_valuesc C s
t | j�S )N)rx rI )rY rQ rQ rR r\ c s zExtensionBlock._holderc C s
| j jjS )N)rI r^ r )rY rQ rQ rR rq h s zExtensionBlock.fill_valuec C s | j jS )N)r\ r� )rY rQ rQ rR r� m s zExtensionBlock._can_hold_na)rL c C s dS )z,Extension arrays are never treated as views.FrQ )rY rQ rQ rR rb r s zExtensionBlock.is_viewc C s
| j jjS )N)rI r^ Z _is_numeric)rY rQ rQ rR r� w s zExtensionBlock.is_numericc C s. t |t�r|d }t||| j� || j|< | S )a2
Attempt self.values[indexer] = value, possibly creating a new array.
This differs from Block.setitem by not allowing setitem to change
the dtype of the Block.
Parameters
----------
indexer : tuple, list-like, array-like, slice
The subset of self.values to set
value : object
The value being set
Returns
-------
Block
Notes
-----
`indexer` is a direct slice/positional indexer. `value` must
be a compatible shape.
r )rr r� rB rI )rY r� r� rQ rQ rR r� { s
zExtensionBlock.setitemc C s t j| j�j| j�S )N)rk rl rI rm rn )rY r^ rQ rQ rR rj � s zExtensionBlock.get_valuesc C s | j S )N)rI )rY rQ rQ rR rh � s zExtensionBlock.array_valuesrp c K s6 | j }t|�}tj|jt��}|||< |jdt|��S )z8override to use ExtensionArray astype for the conversionrz )rI r8 rk rl ri rM rm rW )rY r� r� r� rI r� rQ rQ rR r� � s
z ExtensionBlock.to_native_types)r� c C sR |t jkrd}| jj||dd�}| jdko0|dk s8t�|dkrF| j}| j||�S )zN
Take values according to indexer and return them as a block.
NT)rq r rz )r
r rI r rJ r� rU ry )rY r� r� rt rq r� rQ rQ rR r � s
zExtensionBlock.take_nd)r� rL c C s dS )NTrQ )rY r� rQ rQ rR r� � s z ExtensionBlock._can_hold_elementc C s� t |t� r"| jdkr"|td�f}t |t�rzt|�dkrz|d }t |t�sTtd|��| j| }t|�rp|d }n
td|��| j| S )a
Return a slice of my values.
Parameters
----------
slicer : slice, ndarray[int], or a tuple of these
Valid (non-reducing) indexer for self.values.
Returns
-------
np.ndarray or ExtensionArray
r� Nr z+invalid slicing for a 1-ndim ExtensionArrayrz )rr r� rJ r� rW r� rU rI )rY r� �firstZnew_locsrQ rQ rR r� � s
zExtensionBlock._slicec C s8 |r
| j n| j j� }|j||d�}| j|| j| jd�gS )N)r� r� )rI rK rJ )rI r� r� ry rU rJ )rY r� r� r� r� rI rQ rQ rR r� � s zExtensionBlock.fillnar� c K s0 |r
| j n| j j� }| j|j|||d�| jd�S )N)r� r� r� )rI rK )rI r� ry r� rU )rY r� r� r� r� rq r� rI rQ rQ rR r� � s zExtensionBlock.interpolaterz )r� r� rL c s@ |dkr&|dkr&t � jt| j�dd�S |dkr2d}t � j||�S )Nr )r� rz )r1 r rW rI )rY r� r� )r2 rQ rR r s
zExtensionBlock.diff)r
r� rq rL c C s" | j | jj||d�| j| jd�gS )z�
Shift the block by `periods`.
Dispatches to underlying ExtensionArray and re-boxes in an
ExtensionBlock.
)r
rq )rK rJ )ry rI r9 rU rJ )rY r
r� rq rQ rQ rR r9 s
zExtensionBlock.shiftr� )r r� rL c
C s< t |�}t|tttf� s t�t|tj�rV|jdkrV|j d dksFt�|d d �df }t|tj�r�|jdkr�|j d dks~t�|d d �df }t
j |�r�t |�r�| j
j}t| j�r�d }n| j
}| jj� }| }t
j |�r�|} n|| } y | ||< W n6 ttfk
�r( | jjtj|| j|�|d�}Y nX | j|| jd�gS )Nr� rz r )r^ )rK )r� rr r3 r5 r2 r� rk r� rJ rn r
r� r8 r^ r r. rI r� r5 r� r\ �_from_sequencer
ry rU )
rY r� r r� r r� r^ r� ZicondZ set_otherrQ rQ rR r
" s0
zExtensionBlock.wherec sT �j d }tj|�}|j|dd�\}}|jd�}� �fdd�t|j|�D �}||fS )Nrz )rq r c s, g | ]$\}}�j �jj|d � d�|g��qS )T)r rq )ry rI r )r| �indicesr� )rq rY rQ rR r� b s z+ExtensionBlock._unstack.<locals>.<listcomp>r� r� )rn rk r! r r� �zipr� ) rY r rq r Zn_rowsZ dummy_arrr� r� r� rQ )rq rY rR r S s
zExtensionBlock._unstack)N)Fr F)N)rp N)NFN)r� r FNN)rz )r N)r� Fr )*r r% r& r' r] rV ru rZ r- rn r� r r r� r� r, r r� r0 r\ rq r� rb r� r� rj r< rh r� r
r r r r� r� r� r� r r9 r
r �
__classcell__rQ rQ )r2 rR r/ � sB
&
/r/ c @ s e Zd ZdZdd� ZdS )�ObjectValuesExtensionBlockz�
Block providing backwards-compatibility for `.values`.
Used by PeriodArray and IntervalArray to ensure that
Series[T].values is an ndarray of objects.
c C s | j jt�S )N)rI ri rM )rY rQ rQ rR rf s s z*ObjectValuesExtensionBlock.external_valuesN)r r% r& r' rf rQ rQ rQ rR r> k s r> c @ s e Zd Zf ZdZdZdS )� NumericBlockTN)r r% r& r( r� r� rQ rQ rQ rR r? w s r? c @ s e Zd Zf ZdS )�FloatOrComplexBlockN)r r% r& r( rQ rQ rQ rR r@ } s r@ c @ s. e Zd Zf ZdZeed�dd�Zd
dd �ZdS ) �
FloatBlockT)r� rL c C sn t |�}|d k r:t|jtjtjf�o8t|jtjtjf� S t|t t
tjtj f�olt|t tj
tttjtjf� S )N)r r� rx rk �floating�integer�
datetime64� timedelta64rr �floatr, �int_r �bool_r r )rY r� r� rQ rQ rR r� � s z FloatBlock._can_hold_element� N�.c
K sp | j }|dkrH|dkrHt|�}|s.|jt�}ntj|dd�}|||< |S ddlm} ||||||dd�} | j� S ) z$ convert to our native types format NrJ rM )r^ r )�FloatArrayFormatterF)r� � float_format�decimalr� Z fixed_width) rI r8 ri r{ rk r� Zpandas.io.formats.formatrK Zget_result_as_array)
rY r� rL rM r� r� rI r� rK � formatterrQ rQ rR r� � s" zFloatBlock.to_native_types)rI NrJ N) r r% r& r( r$ r r r� r� rQ rQ rQ rR rA � s rA c @ s4 e Zd Zf ZdZeed�dd�Zeed�dd�Z dS ) � ComplexBlockT)r� rL c C sR t |�}|d k r(t|jtjtjtjf�S t|tt t
tj tj f�oPt|t
tjf� S )N)r r� rx rk rB rC �complexfloatingrr rF r, �complexZfloat_rG r rH )rY r� r� rQ rQ rR r� � s z ComplexBlock._can_hold_element)r� rL c C s t |jjtj�S )N)r� r^ rx rk rP )rY r� rQ rQ rR r� � s zComplexBlock.should_storeN)
r r% r& r( r� r r r� r r� rQ rQ rQ rR rO � s rO c @ s( e Zd Zf ZdZdZeed�dd�ZdS )�IntBlockTF)r� rL c C sZ t |�}|d k rBt|jtj�o@t|jtjtjf� o@| jj|jkS t |�pXt
|�oX|j � S )N) r r� rx rk rC rD rE r^ r� r& r$ )rY r� r� rQ rQ rR r� � s zIntBlock._can_hold_elementN) r r% r& r( r& r� r r r� rQ rQ rQ rR rR � s rR c @ sT e Zd ZdZedd� �Zedd� �Zddd�Zd d
� Zd d � Z d
d� Z
ddd�Z dS )�DatetimeLikeBlockMixinzCMixin class for DatetimeBlock, DatetimeTZBlock, and TimedeltaBlock.c C s t S )N)r; )rY rQ rQ rR r\ � s z DatetimeLikeBlockMixin._holderc C s t jdd�S )Nr �ns)rk rD )rY rQ rQ rR rq � s z!DatetimeLikeBlockMixin.fill_valueNc C s t |�r| j| j�jt�S | jS )zS
return object dtype as boxed values, such as Timestamps/Timedelta
)r* r\ rI ri rM )rY r^ rQ rQ rR rj � s z!DatetimeLikeBlockMixin.get_valuesc C s | j � S )N)rh )rY rQ rQ rR rg � s z&DatetimeLikeBlockMixin.internal_valuesc C s | j j| j�S )N)r\ rS rI )rY rQ rQ rR rh � s z#DatetimeLikeBlockMixin.array_valuesc C s | j � j| j�| S )N)rh rm rn )rY �keyrQ rQ rR r� � s zDatetimeLikeBlockMixin.igetr c C s" | j � }|j|||d�}| j|�S )N)rq r� )rh r9 ry )rY r
r� rq rI r� rQ rQ rR r9 � s z DatetimeLikeBlockMixin.shift)N)r N) r r% r& r' r- r\ rq rj rg rh r� r9 rQ rQ rQ rR rS � s
rS c sr e Zd Zf ZdZd� fdd� Zedd� �Zdd� Zde e
d �� fd d
�
Z e e d�dd�Z
ddd�Zdd� Z� ZS )�
DatetimeBlockTNc s | j |�}t� j|||d� d S )N)rK rJ )r0 r1 rZ )rY rI rK rJ )r2 rQ rR rZ s
zDatetimeBlock.__init__c C s dS )NTrQ )rY rQ rQ rR r� s zDatetimeBlock._can_hold_nac C s@ |j tkrtj|�}t|t�r$|j}t|tj�s<t t
|���|S )ac
Input validation for values passed to __init__. Ensure that
we have datetime64ns, coercing if necessary.
Parameters
----------
values : array-like
Must be convertible to datetime64
Returns
-------
values : ndarray[datetime64ns]
Overridden by DatetimeTZBlock.
) r^ r r
�ensure_datetime64nsrr r; �_datark r� r� rx )rY rI rQ rQ rR r0 s
z"DatetimeBlock._maybe_coerce_valuesFr� )r� r� c sh t |�}t|�rV| j}|r"|j� }t|dd�dkr@t|�jd�}|j|j�}| j |�S t
� j |||d�S )zr
these automatically copy, so copy=True has no effect
raise on an except if raise == True
r� N�UTC)r^ r� r� ) r0 r! rI r� r� r; Z tz_localizeZ
tz_convertr� rw r1 ri )rY r^ r� r� rI )r2 rQ rR ri ! s
zDatetimeBlock.astype)r� rL c C sx t |�}|d k r6| jr.t|| j�p,t|| j�S t|�S |tkrBdS t|t�rl| jrbt |j
| jj �S |j
d kS t|| j�S )NT) r r) r" r^ r7 r r rr r r �tzinfor� )rY r� r� rQ rQ rR r� 6 s
z DatetimeBlock._can_hold_elementr c K s( | j � }|jf ||d�|��}tj|�S )z$ convert to our native types format )r� � date_format)rh �_format_native_typesrk �
atleast_2d)rY r� r[ r� Zdtar� rQ rQ rR r� J s z DatetimeBlock.to_native_typesc C s t j|dd�}|| j|< dS )z'
See Block.set.__doc__
F)r� N)r
rW rI )rY r� rI rQ rQ rR r� S s zDatetimeBlock.set)N)Fr� )r N)r r% r& r( rc rZ r- r� r0 r r{ ri r r� r� r� r= rQ rQ )r2 rR rV � s
rV c s� e Zd ZdZf ZdZdZejZe j
Z
e j Z e j
dd�ZejZejZedd� �Zdd� Zeed �d
d ��Zd d
d�Zdd� Zd!eeed d�dd�Zd"� fdd� Z � fdd�Z d#d d �Z d d � Z � Z S )$�DatetimeTZBlockz2 implement a datetime64 block with a tz attribute Tr rT c C s t S )N)r; )rY rQ rQ rR r\ j s zDatetimeTZBlock._holderc C s, t || j�s| j|�}|jdkr(td��|S )a3
Input validation for values passed to __init__. Ensure that
we have datetime64TZ, coercing if necessary.
Parameters
----------
values : array-like
Must be convertible to datetime64
Returns
-------
values : DatetimeArray
Nz,cannot create a DatetimeTZBlock without a tz)rr r\ r� rX )rY rI rQ rQ rR r0 n s
z$DatetimeTZBlock._maybe_coerce_values)rL c C s | j jjdk S )z* return a boolean if I am possibly a view N)rI rX ra )rY rQ rQ rR rb � s zDatetimeTZBlock.is_viewNc C s* | j }t|�r|jt�}tj|�j| j�S )a9
Returns an ndarray of values.
Parameters
----------
dtype : np.dtype
Only `object`-like dtypes are respected here (not sure
why).
Returns
-------
values : ndarray
When ``dtype=object``, then and object-dtype ndarray of
boxed values is returned. Otherwise, an M8[ns] ndarray
is returned.
DatetimeArray is always 1-d. ``get_values`` will reshape
the return value to be the same dimensionality as the
block.
)rI r* ri rM rk rl rm rn )rY r^ rI rQ rQ rR rj � s
zDatetimeTZBlock.get_valuesc C s t j| jjddd��S )Nzdatetime64[ns]F)r� )rk rl rI ri )rY rQ rQ rR rf � s z DatetimeTZBlock.external_valuesr rG )r� r� rL c C sr |dkr t �|dkr*tj| jjtjd�}n | j| j||d�d j j}|jdt |��}|j
d�}t || j j
d�gS )a�
1st discrete difference.
Parameters
----------
n : int
Number of periods to diff.
axis : int, default 0
Axis to diff upon.
Returns
-------
A list with a new TimeDeltaBlock.
Notes
-----
The arguments here are mimicking shift so they are called correctly
by apply.
r )r^ )r� rz ztimedelta64[ns])rK )r5 rk �zerosrI rn Zint64r9 Zasi8rm rW ri �TimeDeltaBlockrU r� )rY r� r� r� rQ rQ rR r � s
zDatetimeTZBlock.diffFc s4 | j |�r t� j||||�S | jt�j||||d�S )N)r� r� r� )r� r1 r� ri rM )rY r� r� r� r� )r2 rQ rR r� � s
zDatetimeTZBlock.fillnac sZ | j |�s t|tj�r.|jdkr.t� j||�S | jjt �}t
|| j t | j
d�}|j||�S )Nr )rK �klassrJ )r� rr rk r� r� r1 r� rI ri rM rw rU � ObjectBlockrJ )rY r� r� Zobj_valsr� )r2 rQ rR r� � s
zDatetimeTZBlock.setitemr c C sX | j jd�}|j| j�}| j|�}|j|||d�}| j|j j� | jd�}| j ||j
d�S )NzM8[ns])r r� )r^ )rJ ) rI �viewrm rn rw r# r\ r� r^ ry rJ )rY r" r r� ZnaiveZblkZres_blkZawarerQ rQ rR r# � s
zDatetimeTZBlock.quantilec C s4 |dkr|j }|j |kr0td|j � d|� d���|S )a�
ndim inference and validation.
This is overriden by the DatetimeTZBlock to check the case of 2D
data (values.ndim == 2), which should only be allowed if ndim is
also 2.
The case of 1D array is still allowed with both ndim of 1 or 2, as
if the case for other EAs. Therefore, we are only checking
`values.ndim > ndim` instead of `values.ndim != ndim` as for
consolidated blocks.
Nz1Wrong number of dimensions. values.ndim != ndim [z != r[ )rJ rX )rY rI rJ rQ rQ rR rT � s
zDatetimeTZBlock._check_ndim)N)r )NFN)r r )!r r% r& r' r( r) ru rG rg rV r� r� rk rD rq r� r/ rh r- r\ r0 r rb rj rf r, r r r� r� r# rT r= rQ rQ )r2 rR r^ \ s*
%
r^ c sl e Zd Zf ZdZdZdZejdd�Z d� fdd� Z
e dd � �Z e
ed
�d d �Z� fd
d�Zddd�Z� ZS )r` TFr rT Nc sT |j tkrtj|�j}t|t�r&|j}t|tj�s>tt |���t
� j |||d� d S )N)rK rJ ) r^ r r? r: rX rr rk r� r� rx r1 rZ )rY rI rK rJ )r2 rQ rR rZ s
zTimeDeltaBlock.__init__c C s t S )N)r? )rY rQ rQ rR r\ ! s zTimeDeltaBlock._holder)r� rL c C sJ t |�}|d k r t|jtj�S |tkr*dS t|ttjf�r>dS t|| j �S )NT)
r r� rx rk rE r rr r r7 r^ )rY r� r� rQ rQ rR r� % s z TimeDeltaBlock._can_hold_elementc s t |�rtd��t� j|f|�S )Nz�Passing integers to fillna for timedelta64[ns] dtype is no longer supported. To obtain the old behavior, pass `pd.Timedelta(seconds=n)` instead.)r& r� r1 r� )rY r� r� )r2 rQ rR r� / s zTimeDeltaBlock.fillnac K s | j � }|j|f|�S )z$ convert to our native types format )rh r\ )rY r� r� ZtdarQ rQ rR r� < s z TimeDeltaBlock.to_native_types)N)r )r r% r& r( rd r� r� rk rE rq rZ r- r\ r r r� r� r� r= rQ rQ )r2 rR r` s
r` c s: e Zd Zf ZdZdZeed�dd�Zd� fdd� Z � Z
S ) � BoolBlockTF)r� rL c C s. t |�}|d k r t|jtj�S t|ttjf�S )N)r r� rx rk rH rr r )rY r� r� rQ rQ rR r� G s zBoolBlock._can_hold_elementc s: t |d�}tj|�}tj|t�s$| S t� j|||||d�S )Nr� )r� r� r� )r rk Z
atleast_1dZcan_castr r1 r� )rY r� r� r� r� r� Zto_replace_values)r2 rQ rR r� M s
zBoolBlock.replace)FFT) r r% r& r( r� r� r r r� r� r= rQ rQ )r2 rR rd B s
rd c s� e Zd Zf ZdZdZd� fdd� Zedd� �Zde e e e e d �d
d �Z
de d e d d
�dd�Z e
e d�dd�Zd � fdd� Zd � fdd� Zd � fdd� Z� ZS ) rb TNr� c s2 t |jjt�r tj|td�}t� j|||d� d S )N)r^ )rJ rK ) r� r^ rx r{ rk r� rM r1 rZ )rY rI rK rJ )r2 rQ rR rZ \ s zObjectBlock.__init__c C s t j| jjd��S )z]
we can be a bool if we have only bool values but are of type
object
r )r
Z
is_bool_arrayrI r� )rY rQ rQ rR r� b s zObjectBlock.is_boolF)r� r r� r r� c sX � ����fdd�}| j dkr.| jd|d�}n&|d| jj� d�}t|| j | jd�g}|S )z�
attempt to coerce any object types to better types return a copy of
the block (if copy = True) by definition we ARE an ObjectBlock!!!!!
can return multiple blocks!
c s8 |j }t|j� ���� �d�}t|tj�r4|j|�}|S )N)r r� r r� r� )rn r r� rr rk r� rm )r� r� r� rn rI )r� r� r r� r rQ rR r� y s
z ObjectBlock.convert.<locals>.fr� NF)rJ rK )rJ r� rI r� rw rU ) rY r� r r� r r� r� r� rI rQ )r� r� r r� r rR r� j s
zObjectBlock.convertrG )r� rL c C s |d k r |S t dd� |D ��S )Nc S s g | ]}|j d dd��qS )TF)r r� )r� )r| r� rQ rQ rR r� � s z/ObjectBlock._maybe_downcast.<locals>.<listcomp>)r� )rY r� r� rQ rQ rR r� � s zObjectBlock._maybe_downcast)r� rL c C s dS )NTrQ )rY r� rQ rQ rR r� � s z ObjectBlock._can_hold_elementc s" t |�}t |�}|o|}|p |} g }
| g} | rLt|�rL| j|||d|d�S | pR|sjt� j|||||d�S |r�xJt||�D ]<\} }
g }
x*| D ]"}|j| |
|||d�}t||
�}
q�W |
} qzW |
S |o�|�rx@|D ]8} g }
x*| D ]"}|j| ||||d�}t||
�}
q�W |
} q�W |
S | j|||||d�S )NT)r� r� r� )r� r� r� )r) r, r� r1 r� r< r� )rY r� r� r� r� r� Zto_rep_is_listZ
value_is_listZ
both_listsZ either_listr r� Zto_repr� r� r� )r2 rQ rR r� � s@
zObjectBlock.replacec s t |d�}|ot|�}t|�}|r.|r.td��|r6|}|p<|}t|�rN|j} n|} |rf| rftj|�� nt� j|�||d�S |r�| j n| j j
� }
t ��s�t �t
� r�� �fdd�} n� �fdd�} tj| | jgd�} |dkr�| |
�|
dd�< n| |
| �|
|< | j|
�}
|�r|
jd d
�}
|
S ) a�
Replace elements by the given value.
Parameters
----------
to_replace : object or pattern
Scalar to replace or regular expression to match.
value : object
Replacement object.
inplace : bool, default False
Perform inplace modification.
regex : bool, default False
If true, perform regular expression substitution.
convert : bool, default True
If true, try to coerce any object types to better types.
mask : array-like of bool, optional
True indicate corresponding element is ignored.
Returns
-------
a new block, the result after replacing
r� z8only one of to_replace and regex can be regex compilable)r� r� c s0 t � �r(t| t�r(� j| �d k r$�S | S | S d S )N)r, rr r{ �search)r} )�rxr� rQ rR � re_replacer
s z0ObjectBlock._replace_single.<locals>.re_replacerc s& t � �r t| t�r � j�| �S | S d S )N)r, rr r{ �sub)r} )rf r� rQ rR rg
s )ZotypesNF)r� )r r- r� r, �pattern�re�compiler1 r� rI r� r8 rr r{ rk Z vectorizer^ rw r� )rY r� r� r� r� r� r� Z to_rep_reZregex_reri r� rg r� r� )r2 )rf r� rR r� � s6
zObjectBlock._replace_singlec sN |j � r6t� j||||||d�}|r2dd� |D �}|S |rJ| jddd�gS | S )a
Replace value corresponding to the given boolean array with another
value.
Parameters
----------
to_replace : object or pattern
Scalar to replace or regular expression to match.
value : object
Replacement object.
inplace : bool, default False
Perform inplace modification.
regex : bool, default False
If true, perform regular expression substitution.
convert : bool, default True
If true, try to coerce any object types to better types.
mask : array-like of bool, optional
True indicate corresponding element is ignored.
Returns
-------
A new block if there is anything to replace or the original block.
)r� r� r� r� r� r� c S s g | ]}|j d dd��qS )FT)r� r� )r� )r| r� rQ rQ rR r� J
s z/ObjectBlock._replace_coerce.<locals>.<listcomp>FT)r� r� )r� r1 r$ r� )rY r� r� r� r� r� r� r� )r2 rQ rR r$ &
s zObjectBlock._replace_coerce)Nr� )TTTTF)N)FFT)FFTN)TFFN)r r% r& r( r� r� rZ r- r� r r� r r� r r� r� r� r$ r= rQ rQ )r2 rR rb W s
!,_rb c sP e Zd Zf ZdZdZejZd � fdd� Ze dd� �Z
d e e e d�d d
�Z � Z
S )
�CategoricalBlockTNc s4 t |�}t|t�s tt|���t� j|||d� d S )N)rK rJ )rA rr r: r� rx r1 rZ )rY rI rK rJ )r2 rQ rR rZ X
s zCategoricalBlock.__init__c C s t S )N)r: )rY rQ rQ rR r\ ^
s zCategoricalBlock._holderF)r� r� r� c C s0 t |d�}|r| n| j� }|jj||dd� |S )Nr� T)r� )r r� rI r� )rY r� r� r� r� r� r� rQ rQ rR r� b
s
zCategoricalBlock.replace)N)FFT)r r% r& r( r* r� rG r� rZ r- r\ r r� r= rQ rQ )r2 rR rl Q
s rl c C s� |p| j }|j}t|�r t}n�t| j �r.t}n�t|tj�rPt | j � sJt
�t }n�t | j �r`t }n�t
|�spt|�rvt}n|t| j �r�t}nlt|tj�r�t}nZt|tj�r�t|tj�s�t
�t}n8t|tj�r�t}n&t|tj�r�t}n|tjkr�t}nt}|S )z�
Find the appropriate Block subclass to use for the given values and dtype.
Parameters
----------
values : ndarray-like
dtype : numpy or pandas dtype
Returns
-------
cls : class, subclass of Block
) r^ rx r. r/ r rl r� rk rD r! r� rV r^ r( r+ r> r# rB rA rE rC r` rP rO rR rH rd rb )rI r^ ZvtyperO rQ rQ rR �get_block_typeu
s8
rm c C s� t | t�r(| j� } |r(|dkr(tj| �} t |t�r8|j}|d krV|pH| j}t| |�}n"|t krxt
| j� rxt j | |d�} || ||d�S )Nrz )r^ )rJ rK )
rr r4 Zto_numpyrk r] r> Z numpy_dtyper^ rm r^ r! r; rS )rI rK ra rJ r^ rQ rQ rR rw �
s
rw c C sj |dkr g }t | t�rFxN| D ]$}t |t�r6|j|� q |j|� q W n t | t�s\tt| ���|j| � |S )z0 return a new extended blocks, given the result N)rr r� �extendr� rG r� rx )r� r� r rQ rQ rR r� �
s
r� rz )rI rJ rL c C s0 | j |k r,| j}t| j�s,| jtd| ��} | S )z6 guarantee the shape of the values to be at least 1 d rz )rz )rJ rn r# r^ rm r� )rI rJ rn rQ rQ rR rv �
s
rv c C s. t | t�r| j} t| j�s*tj| �j|�} | S )a�
If possible, reshape `arr` to have shape `new_shape`,
with a couple of exceptions (see gh-13012):
1) If `arr` is a ExtensionArray or Index, `arr` will be
returned as is.
2) If `arr` is a Series, the `_values` attribute will
be reshaped and returned.
Parameters
----------
arr : array-like, object to be reshaped
new_shape : int or tuple of ints, the new shape
)rr r5 Z_valuesr# r^ rk rl rm )Zarrr� rQ rQ rR r8 �
s
r8 )r� r� rL c
s( t |�stj|t� ��}y |� }W n tk
r8 Y n�X t| |d �sJn�t|j�p\t|j�s`ntt| j�prt| j�svn^t j
dd�� t j dtj � |j
| j�}W dQ R X ||k}t |�r�|j� r�| j� }||� < |S tj|�}� fdd�}| jj|jjk�r|| |�S t|j�\}} | j
|�} || |�S )a�
Return a new ndarray, try to preserve dtype if possible.
Parameters
----------
v : np.ndarray
`values`, updated in-place.
mask : np.ndarray[bool]
Applies to both sides (array like).
n : `new values` either scalar or an array like aligned with `values`
Returns
-------
values : ndarray with updated values
this *may* be a copy of the original
See Also
--------
ndarray.putmask
r T)�recordr� Nc s6 y|� | � < W n t tfk
r0 || � < Y nX | S )N)r4 rX )r� r� )r� rQ rR �_putmask_preserve/ s
z)_putmask_smart.<locals>._putmask_preserve)r) rk r� rW r� r6 r% r^ r' �warnings�catch_warnings� simplefilterZComplexWarningri r� r� rl �kindr )
r� r� r� �nnZnn_at�compr� rp r^ r� rQ )r� rR r� �
s6
r� )r� rL c C sJ t | t�rtj| tjd�} t | tj�s2tt| ���| jt ksFt| j��| S )zP
If we have a SparseArray or BooleanArray, convert it to ndarray[bool].
)r^ )
rr r< rk rl rH r� r� rx r^ r )r� rQ rQ rR r� B s
r� )N)NNN)N)rz )�r r r� rj Ztypingr r r r rq Znumpyrk Z pandas._libsr r r� r
r Zpandas._libs.internalsZ_libsZ internalsrs r Zpandas._libs.tslibsr
Z pandas._libs.tslibs.timezonesr Zpandas._typingr Zpandas.util._validatorsr Zpandas.core.dtypes.castr r r r r r r r r r r Zpandas.core.dtypes.commonr r r r r r! r" r# r$ r% r& r' r( r) r* r+ r, r- r. r/ r0 Zpandas.core.dtypes.dtypesr1 Zpandas.core.dtypes.genericr2 r3 r4 r5 Zpandas.core.dtypes.missingr6 r7 r8 Zpandas.core.algorithmsr Z
algorithmsZ"pandas.core.array_algos.transformsr9 Zpandas.core.arraysr: r; r< r= r> r? Zpandas.core.baser@ Zpandas.core.common�commonr3 Zpandas.core.constructionrA Zpandas.core.indexersrB rC rD Zpandas.core.missingr� Zpandas.core.nanopsrE ZpandasrF rG r/ r> r? r@ rA rO rR rS rV r^ r` rd rb rl rm rw r� r, rv r8 r� r� r� rQ rQ rQ rR �<module> s� 4
\ $ } 2'` 61 {$
.
P |