3
���h+[ � @ s� d Z ddlmZmZmZmZmZ ddlZddl m
Z
m Z ddl m
Z
ddlmZ ddlmZmZmZmZmZmZmZmZ ddlmZ d d
� ZdKd d
�Z ddddgZ ddddddddddd d d d gZ e e d �d!d"�Z e d#�d$d%�Z!dLej"ej"ee ee# e ee ee e$ee# d'� d(d)�Z%dMd*d+�Z&dNd,d-�Z'dOd.d/�Z(dPd1d2�Z)dQd4d5�Z*d6d7� Z+dRd8d9�Z,dSd:d;�Z-dTd<d=�Z.dUd>d?�Z/dVd@dA�Z0e-e.dB�Z1dCdD� Z2dEdF� Z3dGdH� Z4dIdJ� Z5dS )Wz$
Routines for filling missing data.
� )�Any�List�Optional�Set�UnionN)�algos�lib)�import_optional_dependency)�infer_dtype_from_array)�ensure_float64�is_datetime64_dtype�is_datetime64tz_dtype�is_integer_dtype�is_numeric_v_string_like� is_scalar�is_timedelta64_dtype�needs_i8_conversion)�isnac
C s t |�\}}ytj||d�}W n" tk
r@ tj|td�}Y nX t|�}|| }d}xf|D ]^}|dkr�t| |�rzd}n| |k}t|�r�tj| j t
d�}q^t| |�r�|dO }q^|| |kO }q^W |j � r�|dkr�t| �}n |t| �O }|dkr�tj| j t
d�}|S )z}
Return a masking array of same size/shape as arr
with entries equaling any member of values_to_mask set to True
)�dtypeNF) r
�np�array� Exception�objectr r r �zeros�shape�bool�any)ZarrZvalues_to_maskr Zna_maskZnonna�mask�x� r �5/tmp/pip-build-5_djhm0z/pandas/pandas/core/missing.py� mask_missing s2
r! Fc C sv | d kr d S t | t�r8| j� } | dkr,d} n | dkr8d} ddg}d}|rV|jd� d}| |krrtd |� d
| � ���| S ) N�asfreqZffill�padZbfill�backfillz pad (ffill) or backfill (bfill)�nearestz(pad (ffill), backfill (bfill) or nearestz Invalid fill method. Expecting z. Got )Nr" )�
isinstance�str�lower�append�
ValueError)�method�
allow_nearestZ
valid_methodsZ expectingr r r �clean_fill_methodK s
r- �linear�time�index�valuesr% �zero�slinear� quadratic�cubic� barycentric�krogh�spline�
polynomial�from_derivatives�piecewise_polynomial�pchip�akima� cubicspline)r+ �returnc K sL |j d�}| dkr"|d kr"td��tt }| |krHtd|� d| � d���| S ) N�orderr8 r9 z7You must specify the order of the spline or polynomial.zmethod must be one of z. Got 'z
' instead.)r8 r9 )�getr* �
NP_METHODS�
SP_METHODS)r+ �kwargsr@ �validr r r �clean_interp_methody s
rF )�howc C s� |dks t �t| �dkr dS t| � }| jdkr:|jd�}|dkrR|dd� j� }|dkrxt| �d |ddd� j� }|| }|s�dS |S ) a
Retrieves the index of the first valid value.
Parameters
----------
values : ndarray or ExtensionArray
how : {'first', 'last'}
Use this parameter to change between the first or last valid index.
Returns
-------
int or None
�first�lastr N� � )rH rI ���)�AssertionError�lenr �ndimr Zargmax)r1 rG Zis_validZidxposZ chk_notnar r r �find_valid_index� s
rP �forward) �xvalues�yvaluesr+ �limit�limit_direction�
limit_area�
fill_value� bounds_errorr@ c K s� t |�}
|
} | j� s<tjtj| �tjd�} | jtj� | S | j� rH|S |dkrht | dd�sdt
d��d}ddd g}
|j � }||
kr�t
d
|
� d |� d ���|dk r�d
dg}|j � }||kr�t
d|� d|� d���t j
d|d�}ttj|
��}ttt|d���}ttdt|d� t| ���}|| | }|dk�rD|tt|
|d��B }n0|dk�rd|tt|
d|��B }ntt|
||��}|d
k�r�|||B O }n|dk�r�||O }t|�}t |d|�}|j� } t | d| �} |dk�r�| }nDtj| �}t|j��r�|jtj�}|dk�r|jtjk�rtj |�}|t k�rZtj || �}tj ||
|| | || | �| |
< n.t || || ||
f||||d�| ��| |
< tj| |< | S )z�
Logic for the 1-d interpolation. The result should be 1-d, inputs
xvalues and yvalues will each be 1-d arrays of the same length.
Bounds_error is currently hardcoded to False since non-scipy ones don't
take it as an argument.
)r r/ � is_all_datesNzStime-weighted interpolation only works on Series or DataFrames with a DatetimeIndexr1 rQ ZbackwardZbothz*Invalid limit_direction: expecting one of z, got 'z'.ZinsideZoutsidez%Invalid limit_area: expecting one of z, got �.)ZnobsrT rH rK rI r r. r0 )r+ rW rX r@ )r1 r0 )!r r r Z
empty_like�asarrayZfloat64�fill�nan�all�getattrr* r( r Z_validate_limit�setZ flatnonzero�rangerP rN �
_interp_limit�sorted�copyr r �view�int64Zobject_r Zmaybe_convert_objectsrB ZargsortZinterp�_interpolate_scipy_wrapper)rR rS r+ rT rU rV rW rX r@ rD �invalidrE �resultZvalid_limit_directionsZvalid_limit_areasZall_nansZ
start_nansZend_nansZmid_nansZ
preserve_nansZindsZindexerr r r �interpolate_1d� s�
$
rj c K sx |� d�}t d|d� ddlm} tj|�}| j| jttd�}
t| dd�rb| j j
d �|j
d � } }|d
krv| j |
d
< n"|d kr�t |
d < n|d kr�t
|
d < d
dddddg} || kr�|dkr�|}| j| ||||d�} | |�}
n�|dk�r&t|�s�|dk�rtd|� ���| j| |fd|i|��} | |�}
nN| jj�s8| j� } |jj�sJ|j� }|jj�s\|j� }|
| }|| ||f|�}
|
S )z�
Passed off to scipy.interpolate.interp1d. method is scipy's kind.
Returns an array interpolated at new_x. Add any new methods to
the list in _clean_interp_method.
z interpolation requires SciPy.�scipy)�extrar )� interpolate)r6 r7 r: r; rY F�i8r< r= r> r% r2 r3 r4 r5 r9 )�kindrW rX r8 z;order needs to be specified and greater than 0; got order: �k)r rk rm r r[ Zbarycentric_interpolateZkrogh_interpolate�_from_derivativesr_ Z_values�astypeZpchip_interpolate�_akima_interpolate�_cubicspline_interpolateZinterp1dr r* ZUnivariateSpline�flagsZ writeablerd )r �yZnew_xr+ rW rX r@ rD rl rm Z alt_methodsZinterp1d_methodsZterpZnew_yr r r rg 6 sV
rg c C s4 ddl m} |jj}|| |jdd�||d�}||�S )a�
Convenience function for interpolate.BPoly.from_derivatives.
Construct a piecewise polynomial in the Bernstein basis, compatible
with the specified values and derivatives at breakpoints.
Parameters
----------
xi : array_like
sorted 1D array of x-coordinates
yi : array_like or list of array-likes
yi[i][j] is the j-th derivative known at xi[i]
order: None or int or array_like of ints. Default: None.
Specifies the degree of local polynomials. If not None, some
derivatives are ignored.
der : int or list
How many derivatives to extract; None for all potentially nonzero
derivatives (that is a number equal to the number of points), or a
list of derivatives to extract. This number includes the function
value as 0th derivative.
extrapolate : bool, optional
Whether to extrapolate to ouf-of-bounds points based on first and last
intervals, or to return NaNs. Default: True.
See Also
--------
scipy.interpolate.BPoly.from_derivatives
Returns
-------
y : scalar or array_like
The result, of length R or length M or M by R.
r )rm rK )Zorders� extrapolaterL )rk rm ZBPolyr: �reshape) �xi�yir r@ �derrw rm r+ �mr r r rq | s " rq c C s( ddl m} |j| ||d�}|||d�S )a[
Convenience function for akima interpolation.
xi and yi are arrays of values used to approximate some function f,
with ``yi = f(xi)``.
See `Akima1DInterpolator` for details.
Parameters
----------
xi : array_like
A sorted list of x-coordinates, of length N.
yi : array_like
A 1-D array of real values. `yi`'s length along the interpolation
axis must be equal to the length of `xi`. If N-D array, use axis
parameter to select correct axis.
x : scalar or array_like
Of length M.
der : int, optional
How many derivatives to extract; None for all potentially
nonzero derivatives (that is a number equal to the number
of points), or a list of derivatives to extract. This number
includes the function value as 0th derivative.
axis : int, optional
Axis in the yi array corresponding to the x-coordinate values.
See Also
--------
scipy.interpolate.Akima1DInterpolator
Returns
-------
y : scalar or array_like
The result, of length R or length M or M by R,
r )rm )�axis)�nu)rk rm ZAkima1DInterpolator)ry rz r r{ r} rm �Pr r r rs � s $ rs �
not-a-knotc C s( ddl m} |j| ||||d�}||�S )aq
Convenience function for cubic spline data interpolator.
See `scipy.interpolate.CubicSpline` for details.
Parameters
----------
xi : array_like, shape (n,)
1-d array containing values of the independent variable.
Values must be real, finite and in strictly increasing order.
yi : array_like
Array containing values of the dependent variable. It can have
arbitrary number of dimensions, but the length along ``axis``
(see below) must match the length of ``x``. Values must be finite.
x : scalar or array_like, shape (m,)
axis : int, optional
Axis along which `y` is assumed to be varying. Meaning that for
``x[i]`` the corresponding values are ``np.take(y, i, axis=axis)``.
Default is 0.
bc_type : string or 2-tuple, optional
Boundary condition type. Two additional equations, given by the
boundary conditions, are required to determine all coefficients of
polynomials on each segment [2]_.
If `bc_type` is a string, then the specified condition will be applied
at both ends of a spline. Available conditions are:
* 'not-a-knot' (default): The first and second segment at a curve end
are the same polynomial. It is a good default when there is no
information on boundary conditions.
* 'periodic': The interpolated functions is assumed to be periodic
of period ``x[-1] - x[0]``. The first and last value of `y` must be
identical: ``y[0] == y[-1]``. This boundary condition will result in
``y'[0] == y'[-1]`` and ``y''[0] == y''[-1]``.
* 'clamped': The first derivative at curves ends are zero. Assuming
a 1D `y`, ``bc_type=((1, 0.0), (1, 0.0))`` is the same condition.
* 'natural': The second derivative at curve ends are zero. Assuming
a 1D `y`, ``bc_type=((2, 0.0), (2, 0.0))`` is the same condition.
If `bc_type` is a 2-tuple, the first and the second value will be
applied at the curve start and end respectively. The tuple values can
be one of the previously mentioned strings (except 'periodic') or a
tuple `(order, deriv_values)` allowing to specify arbitrary
derivatives at curve ends:
* `order`: the derivative order, 1 or 2.
* `deriv_value`: array_like containing derivative values, shape must
be the same as `y`, excluding ``axis`` dimension. For example, if
`y` is 1D, then `deriv_value` must be a scalar. If `y` is 3D with
the shape (n0, n1, n2) and axis=2, then `deriv_value` must be 2D
and have the shape (n0, n1).
extrapolate : {bool, 'periodic', None}, optional
If bool, determines whether to extrapolate to out-of-bounds points
based on first and last intervals, or to return NaNs. If 'periodic',
periodic extrapolation is used. If None (default), ``extrapolate`` is
set to 'periodic' for ``bc_type='periodic'`` and to True otherwise.
See Also
--------
scipy.interpolate.CubicHermiteSpline
Returns
-------
y : scalar or array_like
The result, of shape (m,)
References
----------
.. [1] `Cubic Spline Interpolation
<https://en.wikiversity.org/wiki/Cubic_Spline_Interpolation>`_
on Wikiversity.
.. [2] Carl de Boor, "A Practical Guide to Splines", Springer-Verlag, 1978.
r )rm )r} �bc_typerw )rk rm Z CubicSpline)ry rz r r} r� rw rm r r r r rt � s F rt r# c
C s� | }|dkrdd� ndd� }| j }| j dkrP|dkr<td��| jtd | j ��} |dkr^d} nt|| �|�} t|�}|dkr�|t|| �|| |d ��} n|t|| �|| |d ��} |dkr�| d } |j j
d
kr�| j |j �} | S ) z}
Perform an actual interpolation of values, values will be make 2-d if
needed fills inplace, returns the result.
r c S s | S )Nr )r r r r �<lambda>* s z interpolate_2d.<locals>.<lambda>c S s | j S )N)�T)r r r r r� * s rK z0cannot interpolate on a ndim == 1 with axis != 0Nr# )rT r r �M)rK ) rO rM rx �tupler r! r- �pad_2d� backfill_2dr ro rr )
r1 r+ r} rT rW r Z orig_valuesZtransfrO r r r r �interpolate_2d! s(
r� c C s: t |�st|�st|�r&| jtj�} nt| �r6t| �} | S )zN
Cast values to a dtype that algos.pad and algos.backfill can handle.
)r r
r re r rf r r )r1 r r r r �_cast_values_for_fillnaK s r� c C s< |d kr| j }|d kr t| �}t| |�} |jtj�}| |fS )N)r r r� re r Zuint8)r1 r r r r r � _fillna_prep` s
r� c C s$ t | ||�\} }tj| ||d� | S )N)rT )r� r Z pad_inplace)r1 rT r r r r r �pad_1do s r� c C s$ t | ||�\} }tj| ||d� | S )N)rT )r� r Zbackfill_inplace)r1 rT r r r r r � backfill_1du s r� c C s2 t | ||�\} }tj| j�r.tj| ||d� n | S )N)rT )r� r r^ r r Zpad_2d_inplace)r1 rT r r r r r r� { s r� c C s2 t | ||�\} }tj| j�r.tj| ||d� n | S )N)rT )r� r r^ r r Zbackfill_2d_inplace)r1 rT r r r r r r� � s r� )r# r$ c C s t | �} t| S )N)r- �
_fill_methods)r+ r r r �
get_fill_func� s r� c C s t | dd�S )NT)r, )r- )r+ r r r �clean_reindex_fill_method� s r� c s� t | �� t� }t� }� fdd�}|dk rN|dkrDttj| �d �}n
|| |�}|dk r�|dkrb|S t|| ddd� |��}t� d tj|� �}|dkr�|S ||@ S )aj
Get indexers of values that won't be filled
because they exceed the limits.
Parameters
----------
invalid : boolean ndarray
fw_limit : int or None
forward limit to index
bw_limit : int or None
backward limit to index
Returns
-------
set of indexers
Notes
-----
This is equivalent to the more readable, but slower
.. code-block:: python
def _interp_limit(invalid, fw_limit, bw_limit):
for x in np.where(invalid)[0]:
if invalid[max(0, x - fw_limit):x + bw_limit + 1].all():
yield x
c s` t |� �}t| |d �jd�}ttj|�d | �ttj| d |d � j� dk�d �B }|S )NrK r )�min�_rolling_windowr^ r` r �whereZcumsum)rh rT Zwindowed�idx)�Nr r �inner� s
(z _interp_limit.<locals>.innerNr rK rL )rN r` r r� �listr[ )rh Zfw_limitZbw_limitZf_idxZb_idxr� r )r� r rb � s
rb c C sJ | j dd� | j d | d |f }| j| jd f }tjjj| ||d�S )z�
[True, True, False, True, False], 2 ->
[
[True, True],
[True, False],
[False, True],
[True, False],
]
NrK )r �stridesrL rL rL )r r� r r Z
stride_tricksZ
as_strided)�aZwindowr r� r r r r� � s $r� )F)r. NrQ NNFN)NFN)Nr F)r r )r r� N)r# r NNN)NN)NNN)NNN)NNN)NNN)6�__doc__Ztypingr r r r r Znumpyr Z pandas._libsr r Zpandas.compat._optionalr Zpandas.core.dtypes.castr
Zpandas.core.dtypes.commonr r r
r r r r r Zpandas.core.dtypes.missingr r! r- rB rC r' rF rP Zndarray�intr rj rg rq rs rt r� r� r� r� r� r� r� r� r� r� rb r� r r r r �<module> sd (
1
( 8
E
+
+
P
)
A |