3
���hPc � @ s� d dl mZ d dlmZ d dlZd dlmZ d dlmZm Z m
Z
m Z m Z m
Z
d dlmZ d dlZd dlmZ d dlmZmZmZ d d lmZm Z m Z m Z d d
l m Z! d d l"m#Z#m$Z$m%Z% d d l&m'Z'm(Z( d d
l)m*Z*m+Z+m,Z, d dl-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3m4Z4m5Z5m6Z6m7Z7m8Z8m9Z9m:Z: d dl;m<Z< d dl=m>Z>m?Z? d dl@mAZA d dlBmCZCmDZDmEZE d dlFmGZG d dlHmIZImJZJ d dlKjLjMZMd dlKmNZNmOZOmPZPmQZQ d dlRmSZS d dlTmUZUmVZV d dlWmXZXmYZYmZZZm[Z[ d dl\jLj]Z^d dl_m`Z`maZambZb d dlcmdZdmeZe d dlfmgZg d d lhmiZi d d ljmkZk d d llmmZm d d � Znd!d"� ZoG d#d$� d$eVeZ�ZpeJepd%d&gd'd(�eJepd)d*d+d,d-d.d/d0gd1d(�G d2d3� d3eIeZeY���Zqd4d5� Zrejsd6�d7d8�Ztd9d:� Zud;d<� ZvdS )=� )�QUOTE_NONNUMERIC)�partialN)�get_terminal_size)�Dict�Hashable�List�Type�Union�cast)�warn)�
get_option)�NaT�algos� hashtable)� ArrayLike�Dtype�Ordered�Scalar)�function)�cache_readonly�deprecate_kwarg�doc)�validate_bool_kwarg�validate_fillna_kwargs)�coerce_indexer_dtype� maybe_cast_to_extension_array�maybe_infer_to_datetimelike)
� ensure_int64�
ensure_object�is_categorical_dtype�is_datetime64_dtype� is_dict_like�is_dtype_equal�is_extension_array_dtype�is_integer_dtype� is_list_like�is_object_dtype� is_scalar�is_timedelta64_dtype�needs_i8_conversion)�CategoricalDtype)�
ABCIndexClass� ABCSeries)� is_hashable)�is_valid_nat_for_dtype�isna�notna)�ops)�PandasDelegate�delegate_names)�_get_data_algo� factorize�take_1d�unique1d)�shift)�_T�NDArrayBackedExtensionArray)�ExtensionArray�NoNewAttributesMixin� PandasObject� _shared_docs)�array�
extract_array�sanitize_array)�check_array_indexer�deprecate_ndim_indexing)�interpolate_2d)�unpack_zerodim_and_defer)�nargsort)�consolec s. d� j � d��t��� �fdd��}�|_ |S )N�__c sD t |�r t|�t| �kr td��| js6�dkr6td��t|t��rDd}t| j�t|j�krht|d ��n>| jr�| j|jkj� r�t|��n t | j�t |j�ks�t|��| j|jks�td ��| j r�| jj
|j� r�t || j�}n|j }t
| j ��}||�}| j dk|dkB }|j� �r@�d k�r8d || j dk|dk@ < nd
||< |S t|��r�|| jk�r�| jj|�}t
| j ��|�}�dk�r�| j dk}d
||< |S �dk�r�tjt| �td�S �d k�r�tjt| �td�S td�� d���n`�dk�rtd�� dt|�� d���t|t��r&t|j��r&� || �S t
tj| ���tj|��S d S ) NzLengths must match.�__lt__�__gt__�__le__�__ge__z7Unordered Categoricals can only compare equality or notz?Categoricals can only be compared if 'categories' are the same.z! Categories are different lengthsz:Categoricals can only be compared if 'ordered' is the same� �__ne__TF�__eq__)�dtypez$Cannot compare a Categorical for op z( with a scalar, which is not a category.z with type zB.
If you want to compare values, use 'np.asarray(cat) <op> other'.)rI rJ rK rL ���rQ rQ rQ > rO rJ rL rQ )rO rN )r% �len�
ValueError�ordered� TypeError�
isinstance� Categorical�
categories�all�set�equals�_get_codes_for_values�_codes�getattr�anyr' �get_loc�np�zeros�bool�ones�typer; r) rP r? )�self�other�msg� other_codes�f�ret�mask�i)�op�opname� �@/tmp/pip-build-5_djhm0z/pandas/pandas/core/arrays/categorical.py�funcD s`
z _cat_compare_op.<locals>.func)�__name__rE )rn rr rp )rn ro rq �_cat_compare_opA s Prt c s\ t |� y| jj|�}W n ttfk
r0 dS X t|�rB|� kS t� fdd�|D ��S dS )aO
Helper for membership check for ``key`` in ``cat``.
This is a helper method for :method:`__contains__`
and :class:`CategoricalIndex.__contains__`.
Returns True if ``key`` is in ``cat.categories`` and the
location of ``key`` in ``categories`` is in ``container``.
Parameters
----------
cat : :class:`Categorical`or :class:`categoricalIndex`
key : a hashable object
The key to check membership for.
container : Container (e.g. list-like or mapping)
The container to check for membership in.
Returns
-------
is_in : bool
True if ``key`` is in ``self.categories`` and location of
``key`` in ``categories`` is in ``container``, else False.
Notes
-----
This method does not check for NaN values. Do that separately
before calling this method.
Fc 3 s | ]}|� kV qd S )Nrp )�.0Zloc_)� containerrp rq � <genexpr>� s zcontains.<locals>.<genexpr>N)�hashrX r` �KeyErrorrU r' r_ )�cat�keyrv �locrp )rv rq �contains� s r} c sJ e Zd ZdZdZedd�Zeje dg�B ZdZ
d�dd �Z e d
d � �Z
e
jd d � �Z
e ed
�dd��Ze ed
�dd��Ze ed d
�dd��Zed�dd��Zd�dd�Zd�eeed�dd�Zee d
�d d ��Z e e d
�d d �Z e Z!ed�d d!��Z"ed�d"d#��Z#e e$j%d
�d$d%��Z&d�d&d'�Z'ed d(�d)d*�Z(d�d+d,�Z)d�d-d.�Z*d�d/d0�Z+d�d1d2�Z,d�d3d4�Z-d�d5d6�Z.d�d7d8�Z/d�d9d:�Z0d�d;d<�Z1d=d>� Z2e3e4j5�Z6e3e4j7�Z8e3e4j9�Z:e3e4j;�Z<e3e4j=�Z>e3e4j?�Z@d�d?d@�ZAdAdB� ZBd�e$j%d
�dCdD�ZCdEdF� ZDdGdH� ZEe dIdJ� �ZFd�dKdL�ZGeHeIdM d dN�d�dPdQ��ZJdRdS� ZKeKZLdTdU� ZMeMZNdVdW� ZOd�dXdY�ZPdZd[� ZQd\d]� ZRd^d_� ZSdʇ fdadb� ZTd�eeeUdd�dedf�ZVdgdh� ZWd�didj�ZXdkdl� ZYd�dmdn�ZZd�e[ee[do�dpdq�Z\e e$j%d
�drds��Z]e$j%d dt�dudv�Z^d�edw�dxdy�Z_dzd{� Z`ed
�d|d}�Zad�eUd
�dd��Zbd�d�� ZceUd
�d�d��ZdeUd
�d�d��Zed�eUd
�d�d��ZfeUd
�d�d��Zgd�d�� Zhd�d�� Zid�d�� Zjekele$j%f d
�d�d��Zmd�eUed��d�d��Zneod�d�d��d�d�d���Zpeod�d�d��d�d�d���Zqd�d�d��Zrd�d�� Zsd�d�� Zted�d�� �Zud�d�� Zvd�d�� Zwd�d�� Zxe d�d�� �Zyed�d�� �Zzd�d�� Z{d�ed��d�d��Z|� Z}S )�rW a
Represent a categorical variable in classic R / S-plus fashion.
`Categoricals` can only take on only a limited, and usually fixed, number
of possible values (`categories`). In contrast to statistical categorical
variables, a `Categorical` might have an order, but numerical operations
(additions, divisions, ...) are not possible.
All values of the `Categorical` are either in `categories` or `np.nan`.
Assigning values outside of `categories` will raise a `ValueError`. Order
is defined by the order of the `categories`, not lexical order of the
values.
Parameters
----------
values : list-like
The values of the categorical. If categories are given, values not in
categories will be replaced with NaN.
categories : Index-like (unique), optional
The unique categories for this categorical. If not given, the
categories are assumed to be the unique values of `values` (sorted, if
possible, otherwise in the order in which they appear).
ordered : bool, default False
Whether or not this categorical is treated as a ordered categorical.
If True, the resulting categorical will be ordered.
An ordered categorical respects, when sorted, the order of its
`categories` attribute (which in turn is the `categories` argument, if
provided).
dtype : CategoricalDtype
An instance of ``CategoricalDtype`` to use for this categorical.
Attributes
----------
categories : Index
The categories of this categorical
codes : ndarray
The codes (integer positions, which point to the categories) of this
categorical, read only.
ordered : bool
Whether or not this Categorical is ordered.
dtype : CategoricalDtype
The instance of ``CategoricalDtype`` storing the ``categories``
and ``ordered``.
Methods
-------
from_codes
__array__
Raises
------
ValueError
If the categories do not validate.
TypeError
If an explicit ``ordered=True`` is given but no `categories` and the
`values` are not sortable.
See Also
--------
CategoricalDtype : Type for categorical data.
CategoricalIndex : An Index with an underlying ``Categorical``.
Notes
-----
See the `user guide
<https://pandas.pydata.org/pandas-docs/stable/user_guide/categorical.html>`_
for more.
Examples
--------
>>> pd.Categorical([1, 2, 3, 1, 2, 3])
[1, 2, 3, 1, 2, 3]
Categories (3, int64): [1, 2, 3]
>>> pd.Categorical(['a', 'b', 'c', 'a', 'b', 'c'])
['a', 'b', 'c', 'a', 'b', 'c']
Categories (3, object): ['a', 'b', 'c']
Ordered `Categoricals` can be sorted according to the custom order
of the categories and can have a min and max value.
>>> c = pd.Categorical(['a', 'b', 'c', 'a', 'b', 'c'], ordered=True,
... categories=['c', 'b', 'a'])
>>> c
['a', 'b', 'c', 'a', 'b', 'c']
Categories (3, object): ['c' < 'b' < 'a']
>>> c.min()
'c'
i� F)rT �tolistZ categoricalNc s t j� |||�}|r4t� |j�| _| jj|�| _d S tjd�}t � �r`|jd kr�t � j|j
�}n�t � t t
f�s�t� dd�� t � tj�s�tj� �� t� �dkr�tjd�nd }t� �}|j� r؇ fdd�tj| �d D �� t� d |d�� |jd k�r�yt� dd �\}}W nr tk
�rL } z(t� dd �\}}|j
�r<td
�| �W Y d d } ~ X n. tk
�rx } ztd �| �W Y d d } ~ X nX t ||j
�}nFt � j��r�t � t
��r�� j j n� j }
t |
� jj|j�}n t � |j�}|j� �r�tj |j!|jd� } || | < | }| jj|�| _t||j�| _d S ) NFT)Z
convert_datesr �Oc s g | ] }� | �qS rp rp )ru �idx)�valuesrp rq �
<listcomp>R s z(Categorical.__init__.<locals>.<listcomp>)rP )�sortzl'values' is not ordered, please explicitly specify the categories order by passing in a categories argument.z3> 1 ndim Categorical are not supported at this time)"r* �_from_values_or_dtyper rX r] �_dtype� update_dtypera r? r rT rV r+ r, r �ndarray�comZconvert_to_list_likerR rP r/ r_ �whererA r5 rU rS �NotImplementedErrorZ_values�codes�recode_for_categoriesr\ rd �shape) rf r� rX rT rP �fastpath� null_maskZsanitize_dtyper� �errZ old_codesZ
full_codesrp )r� rq �__init__/ sX
zCategorical.__init__c C s | j jS )a�
The categories of this categorical.
Setting assigns new values to each category (effectively a rename of
each individual category).
The assigned value has to be a list-like object. All items must be
unique and the number of items in the new categories must be the same
as the number of items in the old categories.
Assigning to `categories` is a inplace operation!
Raises
------
ValueError
If the new categories do not validate as categories or if the
number of new categories is unequal the number of old categories
See Also
--------
rename_categories : Rename categories.
reorder_categories : Reorder categories.
add_categories : Add new categories.
remove_categories : Remove the specified categories.
remove_unused_categories : Remove categories which are not used.
set_categories : Set the categories to the specified ones.
)rP rX )rf rp rp rq rX � s zCategorical.categoriesc C sB t || jd�}| jjd k r8t| jj�t|j�kr8td��|| _d S )N)rT zKnew categories need to have the same number of items as the old categories!)r* rT rP rX rR rS r� )rf rX � new_dtyperp rp rq rX � s
)�returnc C s | j jS )zF
Whether the categories have an ordered relationship.
)rP rT )rf rp rp rq rT � s zCategorical.orderedc C s | j S )zT
The :class:`~pandas.api.types.CategoricalDtype` for this instance.
)r� )rf rp rp rq rP � s zCategorical.dtypec C s t S )N)rW )rf rp rp rq � _constructor� s zCategorical._constructorc C s t ||d�S )N)rP )rW )�clsZscalarsrP �copyrp rp rq �_from_sequence� s zCategorical._from_sequencec C s d S )Nrp )rf Zboxedrp rp rq �
_formatter� s zCategorical._formatterT)rP r� r� c C s� t |�rNttttf |�}| jj|�}|r2| j� n| } || jkrD| S | j|�S t |�rdt
| ||d�S t |�r�| j � j
� r�td��tj
| ||d�S )aZ
Coerce this type to another dtype
Parameters
----------
dtype : numpy dtype or pandas type
copy : bool, default True
By default, astype always returns a newly allocated object.
If copy is set to False and dtype is categorical, the original
object is returned.
)rP r� z#Cannot convert float NaN to integer)r r
r �strr* rP r� r� �
_set_dtyper# r? r$ r/ r_ rS ra )rf rP r� rp rp rq �astype� s
zCategorical.astypec C s | j jS )z6
return the size of a single category
)rX �itemsize)rf rp rp rq r� � s zCategorical.itemsizec C s t | �S )z�
Return a list of the values.
These are each a scalar type, which is a Python scalar
(for str, int, float) or a pandas scalar
(for Timestamp/Timedelta/Interval/Period)
)�list)rf rp rp rq r~ � s zCategorical.tolistc C s ddl m}m}m}m} ||�} t|t�o2|jdk }
|
r�|jj� rP||dd�} nVt |j�rh||dd�} n>t
|j�r�||dd�} n&|jj � r�|dkr�dddg}| j |�} |
r�|j} t
|| | �} n@| js�| j� }
| j� } t
||
| �} t| d d
�}nt| d d
�}|} | | |d d �S )
aA
Construct a Categorical from inferred values.
For inferred categories (`dtype` is None) the categories are sorted.
For explicit `dtype`, the `inferred_categories` are cast to the
appropriate type.
Parameters
----------
inferred_categories : Index
inferred_codes : Index
dtype : CategoricalDtype or 'category'
true_values : list, optional
If none are provided, the default ones are
"True", "TRUE", and "true."
Returns
-------
Categorical
r )�Index� to_datetime�
to_numeric� to_timedeltaNZcoerce)�errors�True�TRUE�trueF)rT T)rP r� )�pandasr� r� r� r� rV r* rX �
is_numericr r( Z
is_boolean�isinr� Zis_monotonic_increasingr� � sort_values)r� Zinferred_categoriesZinferred_codesrP Z true_valuesr� r� r� r� �catsZknown_categoriesrX r� Zunsortedrp rp rq �_from_inferred_categories� s2
z%Categorical._from_inferred_categoriesc C s� t j|||d�}|jdkr&d}t|��t|�rZt|�rZt|�j� rJtd��|jt j
d�}n
t j |�}t |�r~t|� r~td��t |�r�|j
� t |j�ks�|j� d k r�td��| ||d d
�S ) al
Make a Categorical type from codes and categories or dtype.
This constructor is useful if you already have codes and
categories/dtype and so do not need the (computation intensive)
factorization step, which is usually done on the constructor.
If your data does not follow this convention, please use the normal
constructor.
Parameters
----------
codes : array-like of int
An integer array, where each integer points to a category in
categories or dtype.categories, or else is -1 for NaN.
categories : index-like, optional
The categories for the categorical. Items need to be unique.
If the categories are not given here, then they must be provided
in `dtype`.
ordered : bool, optional
Whether or not this categorical is treated as an ordered
categorical. If not given here or in `dtype`, the resulting
categorical will be unordered.
dtype : CategoricalDtype or "category", optional
If :class:`CategoricalDtype`, cannot be used together with
`categories` or `ordered`.
.. versionadded:: 0.24.0
When `dtype` is provided, neither `categories` nor `ordered`
should be provided.
Returns
-------
Categorical
Examples
--------
>>> dtype = pd.CategoricalDtype(['a', 'b'], ordered=True)
>>> pd.Categorical.from_codes(codes=[0, 1, 0, 1], dtype=dtype)
['a', 'b', 'a', 'b']
Categories (2, object): ['a' < 'b']
)rX rT rP NzKThe categories must be provided in 'categories' or 'dtype'. Both were None.z codes cannot contain NA values)rP z$codes need to be array-like integersrM z1codes need to be between -1 and len(categories)-1T)rP r� rQ )r* r� rX rS r# r$ r/ r_ Zto_numpyra �int64�asarrayrR �max�min)r� r� rX rT rP rh rp rp rq �
from_codes1 s -
&zCategorical.from_codesc C s | j j� }d|j_|S )a�
The category codes of this categorical.
Codes are an array of integers which are the positions of the actual
values in the categories array.
There is no setter, use the other categorical methods and the normal item
setter to change values in the categorical.
Returns
-------
ndarray[int]
A non-writable view of the `codes` array.
F)r] �view�flagsZ writeable)rf �vrp rp rq r� w s
zCategorical.codesc C s\ |rt j|| j�}nt || jd�}| rR| jjdk rRt|j�t| jj�krRtd��|| _dS )a�
Sets new categories inplace
Parameters
----------
fastpath : bool, default False
Don't perform validation of the categories for uniqueness or nulls
Examples
--------
>>> c = pd.Categorical(['a', 'b'])
>>> c
['a', 'b']
Categories (2, object): ['a', 'b']
>>> c._set_categories(pd.Index(['a', 'c']))
>>> c
['a', 'c']
Categories (2, object): ['a', 'c']
)rT NzMnew categories need to have the same number of items than the old categories!)r* �_from_fastpathrT rP rX rR rS r� )rf rX r� r� rp rp rq �_set_categories� s zCategorical._set_categories)rP r� c C s$ t | j| j|j�}t| �||dd�S )a+
Internal method for directly updating the CategoricalDtype
Parameters
----------
dtype : CategoricalDtype
Notes
-----
We don't do any validation here. It's assumed that the dtype is
a (valid) instance of `CategoricalDtype`.
T)rP r� )r� r� rX re )rf rP r� rp rp rq r� � s
zCategorical._set_dtypec C s: t |d�}t| j|d�}|r | n| j� }||_|s6|S dS )a|
Set the ordered attribute to the boolean value.
Parameters
----------
value : bool
Set whether this categorical is ordered (True) or not (False).
inplace : bool, default False
Whether or not to set the ordered attribute in-place or return
a copy of this categorical with ordered set to the value.
�inplace)rT N)r r* rX r� r� )rf �valuer� r� rz rp rp rq � set_ordered� s
zCategorical.set_orderedc C s t |d�}| jd|d�S )aa
Set the Categorical to be ordered.
Parameters
----------
inplace : bool, default False
Whether or not to set the ordered attribute in-place or return
a copy of this categorical with ordered set to True.
Returns
-------
Categorical
Ordered Categorical.
r� T)r� )r r� )rf r� rp rp rq �
as_ordered� s
zCategorical.as_orderedc C s t |d�}| jd|d�S )af
Set the Categorical to be unordered.
Parameters
----------
inplace : bool, default False
Whether or not to set the ordered attribute in-place or return
a copy of this categorical with ordered set to False.
Returns
-------
Categorical
Unordered Categorical.
r� F)r� )r r� )rf r� rp rp rq � as_unordered� s
zCategorical.as_unorderedc C s� t |d�}|dkr| jj}t||d�}|r.| n| j� }|rt|jjdk r�t|j�t|jj�k r�d|j|jt|j�k< nt|j |j|j�}||_||_
|s�|S dS )ap
Set the categories to the specified new_categories.
`new_categories` can include new categories (which will result in
unused categories) or remove old categories (which results in values
set to NaN). If `rename==True`, the categories will simple be renamed
(less or more items than in old categories will result in values set to
NaN or in unused categories respectively).
This method can be used to perform more than one action of adding,
removing, and reordering simultaneously and is therefore faster than
performing the individual steps via the more specialised methods.
On the other hand this methods does not do checks (e.g., whether the
old categories are included in the new categories on a reorder), which
can result in surprising changes, for example when using special string
dtypes, which does not considers a S1 string equal to a single char
python string.
Parameters
----------
new_categories : Index-like
The categories in new order.
ordered : bool, default False
Whether or not the categorical is treated as a ordered categorical.
If not given, do not change the ordered information.
rename : bool, default False
Whether or not the new_categories should be considered as a rename
of the old categories or as reordered categories.
inplace : bool, default False
Whether or not to reorder the categories in-place or return a copy
of this categorical with reordered categories.
Returns
-------
Categorical with reordered categories or None if inplace.
Raises
------
ValueError
If new_categories does not validate as categories
See Also
--------
rename_categories : Rename categories.
reorder_categories : Reorder categories.
add_categories : Add new categories.
remove_categories : Remove the specified categories.
remove_unused_categories : Remove categories which are not used.
r� N)rT rM rQ ) r rP rT r* r� rX rR r] r� r� r� )rf �new_categoriesrT �renamer� r� rz r� rp rp rq �set_categories� s 3
zCategorical.set_categoriesc sl t |d�}|r| n| j� }t� �r:� fdd�|jD �|_n&t� �rZ� fdd�|jD �|_n� |_|sh|S dS )a�
Rename categories.
Parameters
----------
new_categories : list-like, dict-like or callable
New categories which will replace old categories.
* list-like: all items must be unique and the number of items in
the new categories must match the existing number of categories.
* dict-like: specifies a mapping from
old categories to new. Categories not contained in the mapping
are passed through and extra categories in the mapping are
ignored.
* callable : a callable that is called on all items in the old
categories and whose return values comprise the new categories.
.. versionadded:: 0.23.0.
inplace : bool, default False
Whether or not to rename the categories inplace or return a copy of
this categorical with renamed categories.
Returns
-------
cat : Categorical or None
With ``inplace=False``, the new categorical is returned.
With ``inplace=True``, there is no return value.
Raises
------
ValueError
If new categories are list-like and do not have the same number of
items than the current categories or do not validate as categories
See Also
--------
reorder_categories : Reorder categories.
add_categories : Add new categories.
remove_categories : Remove the specified categories.
remove_unused_categories : Remove categories which are not used.
set_categories : Set the categories to the specified ones.
Examples
--------
>>> c = pd.Categorical(['a', 'a', 'b'])
>>> c.rename_categories([0, 1])
[0, 0, 1]
Categories (2, int64): [0, 1]
For dict-like ``new_categories``, extra keys are ignored and
categories not in the dictionary are passed through
>>> c.rename_categories({'a': 'A', 'c': 'C'})
['A', 'A', 'b']
Categories (2, object): ['A', 'b']
You may also provide a callable to create the new categories
>>> c.rename_categories(lambda x: x.upper())
['A', 'A', 'B']
Categories (2, object): ['A', 'B']
r� c s g | ]}� j ||��qS rp )�get)ru �item)r� rp rq r� � s z1Categorical.rename_categories.<locals>.<listcomp>c s g | ] }� |��qS rp rp )ru r� )r� rp rq r� � s N)r r� r! rX �callable)rf r� r� rz rp )r� rq �rename_categories@ s C
z Categorical.rename_categoriesc C s6 t |d�}t| jj�t|�kr&td��| j|||d�S )a�
Reorder categories as specified in new_categories.
`new_categories` need to include all old categories and no new category
items.
Parameters
----------
new_categories : Index-like
The categories in new order.
ordered : bool, optional
Whether or not the categorical is treated as a ordered categorical.
If not given, do not change the ordered information.
inplace : bool, default False
Whether or not to reorder the categories inplace or return a copy of
this categorical with reordered categories.
Returns
-------
cat : Categorical with reordered categories or None if inplace.
Raises
------
ValueError
If the new categories do not contain all old category items or any
new ones
See Also
--------
rename_categories : Rename categories.
add_categories : Add new categories.
remove_categories : Remove the specified categories.
remove_unused_categories : Remove categories which are not used.
set_categories : Set the categories to the specified ones.
r� z=items in new_categories are not the same as in old categories)rT r� )r rZ rP rX rS r� )rf r� rT r� rp rp rq �reorder_categories� s
$
z Categorical.reorder_categoriesc C s� t |d�}t|�s|g}t|�t| jj�@ }t|�dkrFtd|� ���t| jj�t|� }t|| j �}|rn| n| j
� }||_ t |j
|j�|_
|s�|S dS )a3
Add new categories.
`new_categories` will be included at the last/highest place in the
categories and will be unused directly after this call.
Parameters
----------
new_categories : category or list-like of category
The new categories to be included.
inplace : bool, default False
Whether or not to add the categories inplace or return a copy of
this categorical with added categories.
Returns
-------
cat : Categorical with new categories added or None if inplace.
Raises
------
ValueError
If the new categories include old categories or do not validate as
categories
See Also
--------
rename_categories : Rename categories.
reorder_categories : Reorder categories.
remove_categories : Remove the specified categories.
remove_unused_categories : Remove categories which are not used.
set_categories : Set the categories to the specified ones.
r� r z0new categories must not include old categories: N)r r% rZ rP rX rR rS r� r* rT r� r� r r] )rf r� r� Zalready_includedr� rz rp rp rq �add_categories� s !
zCategorical.add_categoriesc s� t |d�}t|�s|g}t|�� � t| jj� }� fdd�| jjD �}tt|��rndd� |D �}dd� |D �}t|�dkr�td|� ���| j || j
d |d
�S ) a
Remove the specified categories.
`removals` must be included in the old categories. Values which were in
the removed categories will be set to NaN
Parameters
----------
removals : category or list of categories
The categories which should be removed.
inplace : bool, default False
Whether or not to remove the categories inplace or return a copy of
this categorical with removed categories.
Returns
-------
cat : Categorical with removed categories or None if inplace.
Raises
------
ValueError
If the removals are not contained in the categories
See Also
--------
rename_categories : Rename categories.
reorder_categories : Reorder categories.
add_categories : Add new categories.
remove_unused_categories : Remove categories which are not used.
set_categories : Set the categories to the specified ones.
r� c s g | ]}|� kr|�qS rp rp )ru �c)� removal_setrp rq r� s z1Categorical.remove_categories.<locals>.<listcomp>c S s h | ]}t |�r|�qS rp )r0 )ru �xrp rp rq � <setcomp> s z0Categorical.remove_categories.<locals>.<setcomp>c S s g | ]}t |�r|�qS rp )r0 )ru r� rp rp rq r� s r z(removals must all be in old categories: F)rT r� r� ) r r% rZ rP rX r_ r/ rR rS r� rT )rf Zremovalsr� Z not_includedr� rp )r� rq �remove_categories� s
z Categorical.remove_categoriesc C s� t |d�}|r| n| j� }tj|jdd�\}}|jdkrZ|d dkrZ|dd� |d }}|jjj|�}t j
|| j d�}||_ t
||j�|_|s�|S dS ) a�
Remove categories which are not used.
Parameters
----------
inplace : bool, default False
Whether or not to drop unused categories inplace or return a copy of
this categorical with unused categories dropped.
Returns
-------
cat : Categorical with unused categories dropped or None if inplace.
See Also
--------
rename_categories : Rename categories.
reorder_categories : Reorder categories.
add_categories : Add new categories.
remove_categories : Remove the specified categories.
set_categories : Set the categories to the specified ones.
r� T)Zreturn_inverser rM N)rT rQ )r r� ra �uniquer] �sizerP rX �taker* r� rT r� r )rf r� rz r� �invr� r� rp rp rq �remove_unused_categories s
z$Categorical.remove_unused_categoriesc C sj | j j|�}y| j| jj� || jd�S tk
rd tj| jdk�rV|j t
|�tj �}tj || j�S X dS )a�
Map categories using input correspondence (dict, Series, or function).
Maps the categories to new categories. If the mapping correspondence is
one-to-one the result is a :class:`~pandas.Categorical` which has the
same order property as the original, otherwise a :class:`~pandas.Index`
is returned. NaN values are unaffected.
If a `dict` or :class:`~pandas.Series` is used any unmapped category is
mapped to `NaN`. Note that if this happens an :class:`~pandas.Index`
will be returned.
Parameters
----------
mapper : function, dict, or Series
Mapping correspondence.
Returns
-------
pandas.Categorical or pandas.Index
Mapped categorical.
See Also
--------
CategoricalIndex.map : Apply a mapping correspondence on a
:class:`~pandas.CategoricalIndex`.
Index.map : Apply a mapping correspondence on an
:class:`~pandas.Index`.
Series.map : Apply a mapping correspondence on a
:class:`~pandas.Series`.
Series.apply : Apply more complex functions on a
:class:`~pandas.Series`.
Examples
--------
>>> cat = pd.Categorical(['a', 'b', 'c'])
>>> cat
['a', 'b', 'c']
Categories (3, object): ['a', 'b', 'c']
>>> cat.map(lambda x: x.upper())
['A', 'B', 'C']
Categories (3, object): ['A', 'B', 'C']
>>> cat.map({'a': 'first', 'b': 'second', 'c': 'third'})
['first', 'second', 'third']
Categories (3, object): ['first', 'second', 'third']
If the mapping is one-to-one the ordering of the categories is
preserved:
>>> cat = pd.Categorical(['a', 'b', 'c'], ordered=True)
>>> cat
['a', 'b', 'c']
Categories (3, object): ['a' < 'b' < 'c']
>>> cat.map({'a': 3, 'b': 2, 'c': 1})
[3, 2, 1]
Categories (3, int64): [3 < 2 < 1]
If the mapping is not one-to-one an :class:`~pandas.Index` is returned:
>>> cat.map({'a': 'first', 'b': 'second', 'c': 'first'})
Index(['first', 'second', 'first'], dtype='object')
If a `dict` is used, all unmapped categories are mapped to `NaN` and
the result is an :class:`~pandas.Index`:
>>> cat.map({'a': 'first', 'b': 'second'})
Index(['first', 'second', nan], dtype='object')
)rX rT rM NrQ )
rX �mapr� r] r� rT rS ra r_ �insertrR �nanr� )rf Zmapperr� rp rp rq r� G s E zCategorical.mapc C sD | j }|jdkrtd��| j|�}t||d|d�}| j|| jdd�S )a�
Shift Categorical by desired number of periods.
Parameters
----------
periods : int
Number of periods to move, can be positive or negative
fill_value : object, optional
The scalar value to use for newly introduced missing values.
.. versionadded:: 0.24.0
Returns
-------
shifted : Categorical
rM zCategorical with ndim > 1.r )�axis�
fill_valueT)rP r� )r� �ndimr� �_validate_fill_valuer8 r� rP )rf Zperiodsr� r� rp rp rq r8 � s
zCategorical.shiftc C s: t |�rd}n(|| jkr&| jj|�}ntd|� d���|S )aM
Convert a user-facing fill_value to a representation to use with our
underlying ndarray, raising ValueError if this is not possible.
Parameters
----------
fill_value : object
Returns
-------
fill_value : int
Raises
------
ValueError
rM z 'fill_value=z1' is not present in this Categorical's categoriesrQ )r/ rX r` rS )rf r� rp rp rq r� � s
z Categorical._validate_fill_valuec C sF t | jj| j�}|r0t|| jj� r0tj||�S t|�rBtj|�}|S )z�
The numpy array interface.
Returns
-------
numpy.array
A numpy array of either the specified dtype or,
if dtype==None (default), the same dtype as
categorical.categories.dtype.
) r6 rX r� r] r" rP ra r� r# )rf rP rk rp rp rq � __array__� s
zCategorical.__array__c O s> t j| ||f|�|�}|tk r"|S td| j� d|j� ���d S )NzObject with dtype z cannot perform the numpy op )r1 Z!maybe_dispatch_ufunc_to_dunder_op�NotImplementedrU rP rs )rf Zufunc�method�inputs�kwargs�resultrp rp rq �__array_ufunc__� s zCategorical.__array_ufunc__c C sV t |t�std��d|kr0t|d |d �|d< x |j� D ]\}}t| ||� q:W dS )z*Necessary for making this object picklablezinvalid pickle stater� Z _categoriesZ_orderedN)rV �dict� Exceptionr* �items�setattr)rf �state�kr� rp rp rq � __setstate__� s
zCategorical.__setstate__c C s | j j| jjjj S )N)r] �nbytesrP rX r� )rf rp rp rq r� s zCategorical.nbytesc C s | j j| jjj|d� S )a�
Memory usage of my values
Parameters
----------
deep : bool
Introspect the data deeply, interrogate
`object` dtypes for system-level memory consumption
Returns
-------
bytes used
Notes
-----
Memory usage does not include memory consumed by elements that
are not components of the array if deep=False
See Also
--------
numpy.ndarray.nbytes
)�deep)r] r� rP rX � memory_usage)rf r� rp rp rq r� s zCategorical.memory_usage� searchsorted)�klass�leftc sZ t |�r$� jj|�}� jjj|�}n$� fdd�|D �}tj|� jjd�}� jj|||d�S )Nc s g | ]}� j j|��qS rp )rX r` )ru r� )rf rp rq r� - s z,Categorical.searchsorted.<locals>.<listcomp>)rP )�side�sorter) r' rX r` r� rP re ra r? r� )rf r� r� r� r� Zlocsrp )rf rq r� % s zCategorical.searchsortedc C s | j dk}|S )aW
Detect missing values
Missing values (-1 in .codes) are detected.
Returns
-------
a boolean array of whether my values are null
See Also
--------
isna : Top-level isna.
isnull : Alias of isna.
Categorical.notna : Boolean inverse of Categorical.isna.
rM rQ )r] )rf rk rp rp rq r/ 1 s
zCategorical.isnac C s
| j � S )a�
Inverse of isna
Both missing values (-1 in .codes) and NA as a category are detected as
null.
Returns
-------
a boolean array of whether my values are not null
See Also
--------
notna : Top-level notna.
notnull : Alias of notna.
Categorical.isna : Boolean inverse of Categorical.notna.
)r/ )rf rp rp rq r0 G s zCategorical.notnac C s | | j � }|S )z�
Return the Categorical without null values.
Missing values (-1 in .codes) are detected.
Returns
-------
valid : Categorical
)r0 )rf r� rp rp rq �dropna] s
zCategorical.dropnac C s� ddl m}m} | j| j }}t|�d|k }}tj|�|j� }} |sL| rp| rT|n|| }
tj |
|phdd�} n tj tj
|||��} tj |d �}| j || j
dd�}|| ||�dd�S )
a{
Return a Series containing counts of each category.
Every category will have an entry, even those with a count of 0.
Parameters
----------
dropna : bool, default True
Don't include counts of NaN.
Returns
-------
counts : Series
See Also
--------
Series.value_counts
r )�CategoricalIndex�Series)Z minlengthrM T)rP r� r� )�indexrP rQ )r� r� r� r] rX rR ra �arangerY Zbincountr� �appendr� rP ) rf r� r� r� �coderz Zncatrl �ix�cleanZobs�countrp rp rq � value_countsk s zCategorical.value_countsc C sV t | jj�r | jj| jtd�S t| j�rLd| jkrL| jjd�j| jtj d�S tj
| �S )a
Return the values.
For internal compatibility with pandas formatting.
Returns
-------
np.ndarray or Index
A numpy array of the same dtype as categorical.categories.dtype or
Index if datetime / periods.
)r� rM �objectrQ ) r) rX rP r� r] r
r$ r� ra r� r? )rf rp rp rq �_internal_get_values� s
z Categorical._internal_get_valuesc C s | j std|� d���dS )z assert that we are ordered z)Categorical is not ordered for operation zG
you can use .as_ordered() to change the Categorical to an ordered one
N)rT rU )rf rn rp rp rq �check_for_ordered� s z Categorical.check_for_orderedc C s | j S )N)r] )rf rp rp rq �_values_for_argsort� s z Categorical._values_for_argsort� quicksortc s t � jf ||d�|��S )a�
Return the indices that would sort the Categorical.
.. versionchanged:: 0.25.0
Changed to sort missing values at the end.
Parameters
----------
ascending : bool, default True
Whether the indices should result in an ascending
or descending sort.
kind : {'quicksort', 'mergesort', 'heapsort'}, optional
Sorting algorithm.
**kwargs:
passed through to :func:`numpy.argsort`.
Returns
-------
numpy.array
See Also
--------
numpy.ndarray.argsort
Notes
-----
While an ordering is applied to the category values, arg-sorting
in this context refers more to organizing and grouping together
based on matching category values. Thus, this function can be
called on an unordered Categorical instance unlike the functions
'Categorical.min' and 'Categorical.max'.
Examples
--------
>>> pd.Categorical(['b', 'b', 'a', 'c']).argsort()
array([2, 0, 1, 3])
>>> cat = pd.Categorical(['b', 'b', 'a', 'c'],
... categories=['c', 'b', 'a'],
... ordered=True)
>>> cat.argsort()
array([3, 0, 1, 2])
Missing values are placed at the end
>>> cat = pd.Categorical([2, None, 1])
>>> cat.argsort()
array([2, 0, 1])
)� ascending�kind)�super�argsort)rf r� r� r� )� __class__rp rq r� � s 3zCategorical.argsort�last)r� r� � na_positionc C s` t |d�}|d kr$tdt|�� ���t| ||d�}|rD| j| | _n| j| j| | jdd�S dS )
a7
Sort the Categorical by category value returning a new
Categorical by default.
While an ordering is applied to the category values, sorting in this
context refers more to organizing and grouping together based on
matching category values. Thus, this function can be called on an
unordered Categorical instance unlike the functions 'Categorical.min'
and 'Categorical.max'.
Parameters
----------
inplace : bool, default False
Do operation in place.
ascending : bool, default True
Order ascending. Passing False orders descending. The
ordering parameter provides the method by which the
category values are organized.
na_position : {'first', 'last'} (optional, default='last')
'first' puts NaNs at the beginning
'last' puts NaNs at the end
Returns
-------
Categorical or None
See Also
--------
Categorical.sort
Series.sort_values
Examples
--------
>>> c = pd.Categorical([1, 2, 2, 1, 5])
>>> c
[1, 2, 2, 1, 5]
Categories (3, int64): [1, 2, 5]
>>> c.sort_values()
[1, 1, 2, 2, 5]
Categories (3, int64): [1, 2, 5]
>>> c.sort_values(ascending=False)
[5, 2, 2, 1, 1]
Categories (3, int64): [1, 2, 5]
Inplace sorting can be done as well:
>>> c.sort_values(inplace=True)
>>> c
[1, 1, 2, 2, 5]
Categories (3, int64): [1, 2, 5]
>>>
>>> c = pd.Categorical([1, 2, 2, 1, 5])
'sort_values' behaviour with NaNs. Note that 'na_position'
is independent of the 'ascending' parameter:
>>> c = pd.Categorical([np.nan, 2, 2, np.nan, 5])
>>> c
[NaN, 2, 2, NaN, 5]
Categories (2, int64): [2, 5]
>>> c.sort_values()
[2, 2, 5, NaN, NaN]
Categories (2, int64): [2, 5]
>>> c.sort_values(ascending=False)
[5, 2, 2, NaN, NaN]
Categories (2, int64): [2, 5]
>>> c.sort_values(na_position='first')
[NaN, NaN, 2, 2, 5]
Categories (2, int64): [2, 5]
>>> c.sort_values(ascending=False, na_position='first')
[NaN, NaN, 5, 2, 2]
Categories (2, int64): [2, 5]
r� r �firstzinvalid na_position: )r� r T)r� rP r� N)r r )r rS �reprrF r] r� rP )rf r� r� r Z
sorted_idxrp rp rq r� � s L
zCategorical.sort_valuesc C st ddl m} | jr>| j}|dk}|j� rp|jd�}tj||< n2| jj � rTtj
| �}n tj
| j || j�j � j
��}|S )z�
For correctly ranking ordered categorical data. See GH#15420
Ordered categorical data should be ranked on the basis of
codes with -1 translated to NaN.
Returns
-------
numpy.array
r )r� rM Zfloat64rQ )r� r� rT r� r_ r� ra r� rX r� r? r� Zrankr� )rf r� r� rl rp rp rq �_values_for_rank< s
z Categorical._values_for_rankc C s$ |d k rt |��| j| j| jdd�S )NT)r� rP r� )r� r� r] rP )rf rP rp rp rq r� Z s zCategorical.viewc C s t dtdd� tj| �S )z�
Return my 'dense' representation
For internal compatibility with numpy arrays.
Returns
-------
dense : array
ziCategorical.to_dense is deprecated and will be removed in a future version. Use np.asarray(cat) instead.� )�
stacklevel)r �
FutureWarningra r� )rf rp rp rq �to_dense_ s
zCategorical.to_densec C sz t ||dd�\}}|dkr tj}|dk r0td��| j}|dk r�tj| �jd t| ��}t||dd|�j | j
j �d }t || j
�}n�t
|tjttf�r�tj|| j
� }t|| �j� s�td��t || j
�}tj|d k�}|j� }|| ||< n~t|��rTt|� �r|| j
k�rtd��|d
k}|j� �rh|j� }t|��rBd||< n| j
j|�||< ntdt |�j � ���| j || j d d
�S )a(
Fill NA/NaN values using the specified method.
Parameters
----------
value : scalar, dict, Series
If a scalar value is passed it is used to fill all missing values.
Alternatively, a Series or dict can be used to fill in different
values for each index. The value should not be a list. The
value(s) passed should either be in the categories or should be
NaN.
method : {'backfill', 'bfill', 'pad', 'ffill', None}, default None
Method to use for filling holes in reindexed Series
pad / ffill: propagate last valid observation forward to next valid
backfill / bfill: use NEXT valid observation to fill gap
limit : int, default None
(Not implemented yet for Categorical!)
If method is specified, this is the maximum number of consecutive
NaN values to forward/backward fill. In other words, if there is
a gap with more than this number of consecutive NaNs, it will only
be partially filled. If method is not specified, this is the
maximum number of entries along the entire axis where NaNs will be
filled.
Returns
-------
filled : Categorical with NA/NaN filled
F)Zvalidate_scalar_dict_valueNz:specifying a limit for fillna has not been implemented yetrM r z fill value must be in categorieszE'value' parameter must be a scalar, dict or Series, but you passed a T)rP r� rQ rQ rQ rQ ) r ra r� r� r] r� ZreshaperR rD r� rX rP r\ rV r� rW r, �
algorithmsr� r/ rY rS r� r� r- r_ r` rU re rs r� ) rf r� r� �limitr� r� rl Z values_codes�indexerrp rp rq �fillnaq sD
zCategorical.fillna)rf �
allow_fillr� c C s t j| |||d�S )a
Take elements from the Categorical.
Parameters
----------
indexer : sequence of int
The indices in `self` to take. The meaning of negative values in
`indexer` depends on the value of `allow_fill`.
allow_fill : bool, default False
How to handle negative values in `indexer`.
* False: negative values in `indices` indicate positional indices
from the right. This is similar to
:func:`numpy.take`.
* True: negative values in `indices` indicate missing values
(the default). These values are set to `fill_value`. Any other
other negative values raise a ``ValueError``.
.. versionchanged:: 1.0.0
Default value changed from ``True`` to ``False``.
fill_value : object
The value to use for `indices` that are missing (-1), when
``allow_fill=True``. This should be the category, i.e. a value
in ``self.categories``, not a code.
Returns
-------
Categorical
This Categorical will have the same categories and ordered as
`self`.
See Also
--------
Series.take : Similar method for Series.
numpy.ndarray.take : Similar method for NumPy arrays.
Examples
--------
>>> cat = pd.Categorical(['a', 'a', 'b'])
>>> cat
['a', 'a', 'b']
Categories (2, object): ['a', 'b']
Specify ``allow_fill==False`` to have negative indices mean indexing
from the right.
>>> cat.take([0, -1, -2], allow_fill=False)
['a', 'b', 'a']
Categories (2, object): ['a', 'b']
With ``allow_fill=True``, indices equal to ``-1`` mean "missing"
values that should be filled with the `fill_value`, which is
``np.nan`` by default.
>>> cat.take([0, -1, -1], allow_fill=True)
['a', NaN, NaN]
Categories (2, object): ['a', 'b']
The fill value can be specified.
>>> cat.take([0, -1, -1], allow_fill=True, fill_value='a')
['a', 'a', 'a']
Categories (2, object): ['a', 'b']
Specifying a fill value that's not in ``self.categories``
will raise a ``ValueError``.
)r
r� )r: r� )rf r r
r� rp rp rq r� � s GzCategorical.takec C s | j S )N)r] )rf rp rp rq �_ndarray s zCategorical._ndarray)�arrr� c C s | j || jdd�S )NT)rP r� )r� rP )rf r rp rp rq �_from_backing_data s z Categorical._from_backing_data)r
c C s t dtdd� | j|||d�S )Nz?Categorical.take_nd is deprecated, use Categorical.take insteadr )r )r
r� )r r r� )rf r r
r� rp rp rq �take_nd" s
zCategorical.take_ndc C s t | j� j� �S )zJ
Returns an Iterator over the values of this Categorical.
)�iterr� r~ )rf rp rp rq �__iter__+ s zCategorical.__iter__c C s* t || jj�r| j� j� S t| || jd�S )z?
Returns True if `key` is in this Categorical.
)rv )r. rX rP r/ r_ r} r] )rf r{ rp rp rq � __contains__1 s zCategorical.__contains__�
c C sv |d }| d|� j ddd�}| || d� j ddd�}|dd� � d|dd� � �}|rn|� d| j� � �}t|�S ) zd
a short repr displaying only max_vals and an optional (but default
footer)
r NF)�length�footerrM z, ..., �
rQ )� _get_repr� _repr_footerr� )rf Zmax_valsr �num�head�tailr� rp rp rq �
_tidy_repr; s zCategorical._tidy_reprc C s� t d�dkrdnt d�}ddlm} t|jdtd�}t| j�|kr�|d }|| jd|� �}|| j| d� �}|dg | }n
|| j�}d d
� |D �}|S ) z9
return the base repr for the categories
zdisplay.max_categoriesr r )�formatN)� formatter�quotingr z...c S s g | ] }|j � �qS rp )�strip)ru r� rp rp rq r� a s z0Categorical._repr_categories.<locals>.<listcomp>)r �pandas.io.formatsr r � format_arrayr rR rX )rf Zmax_categories�fmtr$ r r r �
category_strsrp rp rq �_repr_categoriesJ s
z Categorical._repr_categoriesc C s | j � }t| jj�}dt| j�� d|� d�}t� \}}td�p@|}tj� rNd}d}d}t|�} | j rhdnd\}
} | j
� d } xr|D ]j}
|dkr�| |
t|
� |kr�|| d t|�d
7 }t|�d
} n|s�|| 7 }| t|
�7 } ||
7 }d}q�W |d |j dd� d S )z@
Returns a string representation of the footer.
z Categories (�, z): z
display.widthr � T� � < r r � rM F�[z < ... < z ... �])r* r+ )r r( ) r' r� rX rP rR r r rG Zin_ipython_frontendrT �rstrip�replace)rf r& rP Z levheader�width�height� max_widthZ levstring�startZ cur_col_lenZsep_len�sep�linesep�valrp rp rq �_repr_categories_infod s,
z!Categorical._repr_categories_infoc C s | j � }dt| �� d|� �S )NzLength: r )r8 rR )rf �inforp rp rq r � s zCategorical._repr_footer�NaNc C s. ddl m} |j| |||d�}|j� }t|�S )Nr )r )r �na_repr )r# r ZCategoricalFormatterZ to_stringr� )rf r r; r r% r r� rp rp rq r � s
zCategorical._get_reprc C sf d}t | j�|kr | j|�}nDt | j�dkrB| jt | �|kd�}n | jddd�jdd�}d |� �}|S )
z(
String representation.
r r )r FT)r r r z, z[], )rR r] r r r0 )rf �_maxlenr� rh rp rp rq �__repr__� s
zCategorical.__repr__c C s* t |tj�r&|jjdkr&|j| jj�}|S )z>
return an indexer coerced to the codes dtype
rm )rV ra r� rP r� r� r] )rf r rp rp rq �_maybe_coerce_indexer� s z!Categorical._maybe_coerce_indexerc C sn t |ttjf�r2| j| }|dkr(tjS | j| S t| |�}| j| }|jdkr\t |� |S | j
|| j dd�S )z!
Return an item.
rM T)rP r� rQ ) rV �intra �integerr] r� rX rB r� rC r� rP )rf r{ rm r� rp rp rq � __getitem__� s
zCategorical.__getitem__c C sJ t |dd�}t|t�rXt| |�s(td��| jj|j�sXt|j|j| j�}tj || j
d�}t |�rd|n|g}ddl m
} ||�j| j�}t|�r�t|�j� r�td��t|ttjf�r�nft|t��rt|�dkr�tj|d �s�td ��|d
}n t|�d
k�r|d }ntd ��n t|t��r | jj|�}| j|�}t | |�}|| j |< d S ) z�
Item assignment.
Raises
------
ValueError
If (one or more) Value is not in categories or if a assigned
`Categorical` does not have the same categories
T)Z
extract_numpyzCCannot set a Categorical with another, without identical categories)rP r )r� zMCannot setitem on a Categorical with a new category, set the categories firstr z(invalid slicing for a 1-ndim categoricalrM N) r@ rV rW r" rS rX r[ r� r� r� rP r% r� r� �
differencerR r/ rY r? ra r@ �tupler� Z
is_null_slice�AssertionError�slice� get_indexerr> rB r] )rf r{ r� � new_codesZrvaluer� Zto_addZlindexerrp rp rq � __setitem__� s<
zCategorical.__setitem__c sZ | j }tj| jjd�|j�\� }|j� }� fdd�t||dd� �D �}tt||��}|S )aw
Compute the inverse of a categorical, returning
a dict of categories -> indexers.
*This is an internal function*
Returns
-------
dict of categories -> indexers
Examples
--------
>>> c = pd.Categorical(list('aabca'))
>>> c
['a', 'a', 'b', 'c', 'a']
Categories (3, object): ['a', 'b', 'c']
>>> c.categories
Index(['a', 'b', 'c'], dtype='object')
>>> c.codes
array([0, 0, 1, 2, 0], dtype=int8)
>>> c._reverse_indexer()
{'a': array([0, 1, 4]), 'b': array([2]), 'c': array([3])}
r� c 3 s | ]\}}� ||� V qd S )Nrp )ru r4 �end)�rrp rq rw s z/Categorical._reverse_indexer.<locals>.<genexpr>rM N) rX �libalgosZgroupsort_indexerr� r� r� Zcumsum�zipr� )rf rX �countsZ_resultr� rp )rJ rq �_reverse_indexer� s z Categorical._reverse_indexer)�name�skipnac K s4 t | |d �}|d kr"td|� ���|f d|i|��S )Nz)Categorical cannot perform the operation rP )r^ rU )rf rO rP r� rr rp rp rq �_reduce s zCategorical._reduceZ numeric_onlyrP )Z old_arg_nameZ new_arg_namec K sr t jf |� | jd� t| j�s(| jjS | jdk}|j� s^|rV|j� rV| j| j � }qht
j S n
| jj � }| j | S )ao
The minimum value of the object.
Only ordered `Categoricals` have a minimum!
.. versionchanged:: 1.0.0
Returns an NA value on empty arrays
Raises
------
TypeError
If the `Categorical` is not `ordered`.
Returns
-------
min : the minimum of this `Categorical`
r� rM rQ )
�nvZ validate_minr� rR r] rP �na_valuerY r_ r� ra r� rX )rf rP r� �good�pointerrp rp rq r� % s
zCategorical.minc K sr t jf |� | jd� t| j�s(| jjS | jdk}|j� s^|rV|j� rV| j| j � }qht
j S n
| jj � }| j | S )ao
The maximum value of the object.
Only ordered `Categoricals` have a maximum!
.. versionchanged:: 1.0.0
Returns an NA value on empty arrays
Raises
------
TypeError
If the `Categorical` is not `ordered`.
Returns
-------
max : the maximum of this `Categorical`
r� rM rQ )
rR Z validate_maxr� rR r] rP rS rY r_ r� ra r� rX )rf rP r� rT rU rp rp rq r� I s
zCategorical.maxc C sD | j }|r | j dk}| j | }ttjt|�|��}| j|| jdd�S )a^
Returns the mode(s) of the Categorical.
Always returns `Categorical` even if only one value.
Parameters
----------
dropna : bool, default True
Don't consider counts of NaN/NaT.
.. versionadded:: 0.24.0
Returns
-------
modes : `Categorical` (sorted)
rM T)r� rP r� rQ )r] �sorted�htableZ
mode_int64r r� rP )rf r� r� rT rp rp rq �modem s
zCategorical.modec C sF t | j�}| j� }||_||dk }| jr4tj|�}|j|jj |��S )a�
Return the ``Categorical`` which ``categories`` and ``codes`` are
unique. Unused categories are NOT returned.
- unordered category: values and categories are sorted by appearance
order.
- ordered category: values are sorted by appearance order, categories
keeps existing order.
Returns
-------
unique values : ``Categorical``
See Also
--------
pandas.unique
CategoricalIndex.unique
Series.unique
Examples
--------
An unordered Categorical will return categories in the
order of appearance.
>>> pd.Categorical(list("baabc")).unique()
['b', 'a', 'c']
Categories (3, object): ['b', 'a', 'c']
>>> pd.Categorical(list("baabc"), categories=list("abc")).unique()
['b', 'a', 'c']
Categories (3, object): ['b', 'a', 'c']
An ordered Categorical preserves the category ordering.
>>> pd.Categorical(
... list("baabc"), categories=list("abc"), ordered=True
... ).unique()
['b', 'a', 'c']
Categories (3, object): ['a' < 'b' < 'c']
rM rQ )
r7 r� r� r] rT ra r� r� rX r� )rf Z unique_codesrz Z
take_codesrp rp rq r� � s *
zCategorical.uniquec C s | j jd�}|dfS )Nr� rM rQ )r� r� )rf r� rp rp rq �_values_for_factorize� s z!Categorical._values_for_factorizec C s |j |jj|�|jd�S )N)rP )r� rX r� rP )r� Zuniques�originalrp rp rq �_from_factorized� s z Categorical._from_factorizedc C sD | j |�r@| jj|j�r |j}nt|j|j| j�}tj| j|�S dS )z�
Returns True if categorical arrays are equal.
Parameters
----------
other : `Categorical`
Returns
-------
bool
F)r" rX r[ r] r� r� ra Z array_equal)rf rg ri rp rp rq r[ � s
zCategorical.equalsc C s2 yt | j�t |j�kS ttfk
r, dS X dS )z�
Returns True if categoricals are the same dtype
same categories, and same ordered
Parameters
----------
other : Categorical
Returns
-------
bool
FN)rx rP �AttributeErrorrU )rf rg rp rp rq r" � s
zCategorical.is_dtype_equalc C sN | j dd�}|t|j� � }ddlm} |||gdd�}ddg|_d |j_|S )
z�
Describes this Categorical
Returns
-------
description: `DataFrame`
A dataframe with frequency and counts by category.
F)r� r )�concatrM )r� rM �freqsrX )r� �float�sumZpandas.core.reshape.concatr] �columnsr� rO )rf rM r^ r] r� rp rp rq �describe� s
zCategorical.describec C s dS )NTrp )rf rp rp rq � _can_hold_na s zCategorical._can_hold_nac C s ddl m} ||�S )Nr )�union_categoricals)Zpandas.core.dtypes.concatrd )rf Z to_concatrd rp rp rq �_concat_same_type s z Categorical._concat_same_typec C sf t |�s"t|�j}td|� d���t|dd�}tjt|��}| jj |�}|||dkB }t
j | j |�S )a�
Check whether `values` are contained in Categorical.
Return a boolean NumPy Array showing whether each element in
the Categorical matches an element in the passed sequence of
`values` exactly.
Parameters
----------
values : set or list-like
The sequence of values to test. Passing in a single string will
raise a ``TypeError``. Instead, turn a single string into a
list of one element.
Returns
-------
isin : numpy.ndarray (bool dtype)
Raises
------
TypeError
* If `values` is not a set or list-like
See Also
--------
pandas.Series.isin : Equivalent method on Series.
Examples
--------
>>> s = pd.Categorical(['lama', 'cow', 'lama', 'beetle', 'lama',
... 'hippo'])
>>> s.isin(['cow', 'lama'])
array([ True, True, True, False, True, False])
Passing a single string as ``s.isin('lama')`` will raise an error. Use
a list of one element instead:
>>> s.isin(['lama'])
array([ True, False, True, False, True, False])
zIonly list-like objects are allowed to be passed to isin(), you passed a [r. Nr )
r% re rs rU rA ra r� r/ rX rF r r� r� )rf r� Z values_typer� Z code_valuesrp rp rq r�
s )
zCategorical.isin)r� c s� t |d�}|r| n| j� }t|�r6� fdd�|D �}n|� i}x�|j� D ]�\}}||krZqH||jkrHt|�r||j|dd� qH|jj� }|j|�} ||jkr�|j|�}
|
|j |j | k< |j|dd� qH||| < |j
|dd� qHW |s�|S dS )aM
Replaces all instances of one value with another
Parameters
----------
to_replace: object
The value to be replaced
value: object
The value to replace it with
inplace: bool
Whether the operation is done in-place
Returns
-------
None if inplace is True, otherwise the new Categorical after replacement
Examples
--------
>>> s = pd.Categorical([1, 2, 1, 3])
>>> s.replace(1, 3)
[3, 2, 3, 3]
Categories (2, int64): [2, 3]
r� c s i | ]
}� |�qS rp rp )ru �
replace_value)r� rp rq �
<dictcomp>c s z'Categorical.replace.<locals>.<dictcomp>T)r� N) r r� r% r� rX r/ r� r~ r� r] r� ) rf Z
to_replacer� r� rz Z replace_dictrf Z new_valuerX r� Z value_indexrp )r� rq r0 B s,
zCategorical.replace)NNNF)NF)F)T)N)NNN)F)F)F)F)NFF)F)NF)F)F)F)N)N)F)r� N)T)Tr� )FTr )N)NNN)FN)FN)r T)Tr: T)T)T)T)T)F)~rs �
__module__� __qualname__�__doc__Z__array_priority__r* r� r= Z
_deprecations� frozensetZ_typr� �propertyrX �setterr rT rP r r� � classmethodr� r� r rc r r� r r? r� r r r~ Zto_listr� r� ra r� r� r� r� r� r� r� r� r� r� r� r� r� r� rt �operator�eqrO �nerN �ltrI �gtrJ �lerK �gerL r8 r� r� r� r� r� r� r r> r� r/ Zisnullr0 Znotnullr� r� r� r� r� r� r� r� r r� r r r9 r� r r r r r r r' r8 r r r= r> rA rH r r rN rQ r r� r� rX r� rY r[ r[ r" rb rc re r� r0 �
__classcell__rp rp )r� rq rW � s� Y
P
< E
%
I
O
+
2
4
'Q
$ 6X
ZN
D#
#
#
6 5rW rX rT rl )ZdelegateZ accessors�typr� r� r� r� r� r� r� r� r� c @ sH e Zd ZdZdd� Zedd� �Zdd� Zdd � Ze d
d � �Z
d d
� Z dS )�CategoricalAccessora
Accessor object for categorical properties of the Series values.
Be aware that assigning to `categories` is a inplace operation, while all
methods return new categorical data per default (but can be called with
`inplace=True`).
Parameters
----------
data : Series or CategoricalIndex
Examples
--------
>>> s = pd.Series(list("abbccc")).astype("category")
>>> s
0 a
1 b
2 b
3 c
4 c
5 c
dtype: category
Categories (3, object): ['a', 'b', 'c']
>>> s.cat.categories
Index(['a', 'b', 'c'], dtype='object')
>>> s.cat.rename_categories(list("cba"))
0 c
1 b
2 b
3 a
4 a
5 a
dtype: category
Categories (3, object): ['c', 'b', 'a']
>>> s.cat.reorder_categories(list("cba"))
0 a
1 b
2 b
3 c
4 c
5 c
dtype: category
Categories (3, object): ['c', 'b', 'a']
>>> s.cat.add_categories(["d", "e"])
0 a
1 b
2 b
3 c
4 c
5 c
dtype: category
Categories (5, object): ['a', 'b', 'c', 'd', 'e']
>>> s.cat.remove_categories(["a", "c"])
0 NaN
1 b
2 b
3 NaN
4 NaN
5 NaN
dtype: category
Categories (1, object): ['b']
>>> s1 = s.cat.add_categories(["d", "e"])
>>> s1.cat.remove_unused_categories()
0 a
1 b
2 b
3 c
4 c
5 c
dtype: category
Categories (3, object): ['a', 'b', 'c']
>>> s.cat.set_categories(list("abcde"))
0 a
1 b
2 b
3 c
4 c
5 c
dtype: category
Categories (5, object): ['a', 'b', 'c', 'd', 'e']
>>> s.cat.as_ordered()
0 a
1 b
2 b
3 c
4 c
5 c
dtype: category
Categories (3, object): ['a' < 'b' < 'c']
>>> s.cat.as_unordered()
0 a
1 b
2 b
3 c
4 c
5 c
dtype: category
Categories (3, object): ['a', 'b', 'c']
c C s. | j |� |j| _|j| _|j| _| j� d S )N)� _validater� �_parentr� �_indexrO �_nameZ_freeze)rf �datarp rp rq r�
s
z CategoricalAccessor.__init__c C s t | j�std��d S )Nz2Can only use .cat accessor with a 'category' dtype)r rP r\ )r} rp rp rq ry
s
z CategoricalAccessor._validatec C s t | j|�S )N)r^ rz )rf rO rp rp rq �_delegate_property_get
s z*CategoricalAccessor._delegate_property_getc C s t | j||�S )N)r� rz )rf rO Z
new_valuesrp rp rq �_delegate_property_set
s z*CategoricalAccessor._delegate_property_setc C s ddl m} || jj| jd�S )z>
Return Series of codes as well as the index.
r )r� )r� )r� r� rz r� r{ )rf r� rp rp rq r�
s zCategoricalAccessor.codesc O s@ ddl m} t| j|�}|||�}|d k r<||| j| jd�S d S )Nr )r� )r� rO )r� r� r^ rz r{ r| )rf rO �argsr� r� r� �resrp rp rq �_delegate_method
s
z$CategoricalAccessor._delegate_methodN) rs rh ri rj r� � staticmethodry r~ r rl r� r� rp rp rp rq rx � s } rx c C s� t | j|j�}t|j�rPt| �rP|jj� }t|| �} t| |�sdt| �} t|�}n|sdt| �} t|�}t|t�r~t |j
| �|�S t | �\}}t |�\}}|t |��}|j
|� t |j|�|�S )zR
utility routine to turn values into codes given the specified categories
)r" rP r# r& Zconstruct_array_typer rV r r+ r Zget_indexer_forr4 rR Z
map_locations�lookup) r� rX Z dtype_equalr� Z
hash_klass�vals�_r� �trp rp rq r\ '
s"
r\ )r� c C sL t |�dkr| j� S |j|�r&| j� S t|j|�|�}t|| j� dd�}|S )a�
Convert a set of codes for to a new set of categories
Parameters
----------
codes : np.ndarray
old_categories, new_categories : Index
Returns
-------
new_codes : np.ndarray[np.int64]
Examples
--------
>>> old_cat = pd.Index(['b', 'a', 'c'])
>>> new_cat = pd.Index(['a', 'b'])
>>> codes = np.array([0, 1, 1, 2])
>>> recode_for_categories(codes, old_cat, new_cat)
array([ 1, 0, 0, -1], dtype=int8)
r rM )r� rQ )rR r� r[ r rF r6 )r� Zold_categoriesr� r rG rp rp rq r� G
s
r� c C sp t | �std��t| �rPt| �} tjt| j�| jj d�}t
j || j d�}| j}nt
| dd�}|j}|j}||fS )a�
Factorize an input `values` into `categories` and `codes`. Preserves
categorical dtype in `categories`.
*This is an internal function*
Parameters
----------
values : list-like
Returns
-------
codes : ndarray
categories : Index
If `values` has a categorical dtype, then `categories` is
a CategoricalIndex keeping the categories and order of `values`.
zInput must be list-like)rP F)rT ) r% rU r r@ ra r� rR rX r� rP rW r� )r� Z cat_codesrX r� rz rp rp rq �factorize_from_iterablei
s r� c C s, t | �dkrg g gS tttdd� | D �� �S )aR
A higher-level wrapper over `factorize_from_iterable`.
*This is an internal function*
Parameters
----------
iterables : list-like of list-likes
Returns
-------
codes_list : list of ndarrays
categories_list : list of Indexes
Notes
-----
See `factorize_from_iterable` for more info.
r c s s | ]}t |�V qd S )N)r� )ru �itrp rp rq rw �
s z+factorize_from_iterables.<locals>.<genexpr>)rR r� r� rL )� iterablesrp rp rq �factorize_from_iterables�
s r� )w�csvr � functoolsr ro �shutilr Ztypingr r r r r r
�warningsr Znumpyra Zpandas._configr Z pandas._libsr
r rK r rW Zpandas._typingr r r r Zpandas.compat.numpyr rR Zpandas.util._decoratorsr r r Zpandas.util._validatorsr r Zpandas.core.dtypes.castr r r Zpandas.core.dtypes.commonr r r r r! r" r# r$ r% r& r' r( r) Zpandas.core.dtypes.dtypesr* Zpandas.core.dtypes.genericr+ r, Z pandas.core.dtypes.inferencer- Zpandas.core.dtypes.missingr. r/ r0 Z pandas.corer1 Zpandas.core.accessorr2 r3 Zpandas.core.algorithms�corer r4 r5 r6 r7 Z"pandas.core.array_algos.transformsr8 Zpandas.core.arrays._mixinsr9 r: Zpandas.core.baser; r< r= r> Zpandas.core.common�commonr� Zpandas.core.constructionr? r@ rA Zpandas.core.indexersrB rC Zpandas.core.missingrD Zpandas.core.ops.commonrE Zpandas.core.sortingrF r# rG rt r} rW rx r\ r� r� r� r� rp rp rp rq �<module> s� < X3 F "& |