3
���h� � @ s� d dl mZ d dlmZ d dlmZ d dlmZ d dlmZm Z m
Z
m Z m Z m
Z
mZmZ d dlZd dlZd dlmZmZ d dlmZmZmZ d d lmZm Z m Z d d
l m Z d d l m!Z!m"Z"m#Z# d d l$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z. d d
l/m0Z0m1Z1 d dl2m3Z3 d dl4m5Z5m6Z6 d dl7m8Z8 d dl9m:Z: d dl;m<Z<m=Z=m>Z> d dl?m@Z@ d dlAmBZB e�r~d dlCmDZD d dlEmFZF ee
e e!df ZGeeHeIeJf ZKe
deKe�ZLeeLeGf ZMdd� ZNdLeGeIe eH eOd �d d �ZPeGe eJ eOe dd �d d!�ZQdMe!e eO e"e@d"�d#d$�ZRdNeMde eJ d%d&�d'd(�ZSd)d*� ZTdOe eJ e"e e# e eJ e eJ e eO e eO e eO e eO d+� d,d-�ZUd.d/� ZVedPeLeJeOeOe eO e eJ eOe eJ eOeOeeLd1f d2� d3d4��ZWedQdeJeOeOe eO e eJ eOe eJ eOeOdd2� d5d4��ZWedRee
e f eJeOeOe eO e eJ eOe eJ eOeOeBd2� d6d4��ZWdSeMeJeOeOe eO e eJ eOe eJ eOeOeeBdeLd1f d2� d;d4�ZWd<d<d=d=d>d>d?d?d@d@dAdAdBdBdBdCdCdCdDdDdDdE�ZXdFdG� ZYdHdI� ZZdTdJdK�Z[dS )U� )�abc)�datetime)�partial)�islice)�
TYPE_CHECKING�Callable�List�Optional�Tuple�TypeVar�Union�overloadN)�tslib�tslibs)� Timestamp�
conversion�parsing)�DateParseError�_format_is_iso�_guess_datetime_format)�array_strptime)� ArrayLike�Label�Timezone)
�
ensure_object�is_datetime64_dtype�is_datetime64_ns_dtype�is_datetime64tz_dtype�is_float�
is_integer�is_integer_dtype� is_list_like�is_numeric_dtype� is_scalar)� ABCDataFrame� ABCSeries)�notna)�
DatetimeArray� IntegerArray)�
algorithms)�unique)�maybe_convert_dtype�objects_to_datetime64ns� tz_to_dtype)�Index)�
DatetimeIndex)�NaTType)�Seriesr1 �DatetimeScalarc K s0 t | �j� d }t|�r,t| |d f|�S d S )Nr )r& Znonzero�lenr )Zarr�kwargsZnon_nan_elements� r5 �=/tmp/pip-build-5_djhm0z/pandas/pandas/core/tools/datetimes.py� _guess_datetime_format_for_arrayH s r7 �ffffff�?)�arg� unique_share� check_count�returnc C s� d}|dkr@t | �dkr dS t | �dkr:tt | �d �}qpd}n0d| koVt | �kn sdtd ��|dkrpdS d| k o�d
k n s�td ��tt| |��}t |�|| kr�d}|S ) a
Decides whether to do caching.
If the percent of unique elements among `check_count` elements less
than `unique_share * 100` then we can do caching.
Parameters
----------
arg: listlike, tuple, 1-d array, Series
unique_share: float, default=0.7, optional
0 < unique_share < 1
check_count: int, optional
0 <= check_count <= len(arg)
Returns
-------
do_caching: bool
Notes
-----
By default for a sequence of less than 50 items in size, we don't do
caching; for the number of elements less than 5000, we take ten percent of
all elements to check for a uniqueness share; if the sequence size is more
than 5000, then we check only the first 500 elements.
All constants were chosen empirically by.
TN�2 Fi� g�������?i� r z1check_count must be in next bounds: [0; len(arg)]� z+unique_share must be in next bounds: (0; 1))r3 �int�AssertionError�setr )r9 r: r; Z
do_cachingZunique_elementsr5 r5 r6 � should_cacheO s rB )r9 �format�cache�convert_listliker< c C sX ddl m} |td�}|rTt| �s&|S t| �}t|�t| �k rT|||�}|||d�}|S )a�
Create a cache of unique dates from an array of dates
Parameters
----------
arg : listlike, tuple, 1-d array, Series
format : string
Strftime format to parse time
cache : boolean
True attempts to create a cache of converted values
convert_listlike : function
Conversion function to apply on dates
Returns
-------
cache_array : Series
Cache of converted, unique dates. Can be empty
r )r1 )�dtype)�index)�pandasr1 �objectrB r* r3 )r9 rC rD rE r1 � cache_arrayZ unique_datesZ cache_datesr5 r5 r6 � _maybe_cache� s
rK )�dt_array�utc�namer< c C s. t | �r"|rdnd}t| ||d�S t| |d�S )a�
Properly boxes the ndarray of datetimes to DatetimeIndex
if it is possible or to generic Index instead
Parameters
----------
dt_array: 1-d array
Array of datetimes to be wrapped in an Index.
tz : object
None or 'utc'
name : string, default None
Name for a resulting index
Returns
-------
result : datetime of converted dates
- DatetimeIndex if convertible to sole datetime64 type
- general Index otherwise
rM N)�tzrN )rN )r r/ r. )rL rM rN rO r5 r5 r6 �_box_as_indexlike� s rP r. )r9 rJ rN r< c C s( ddl m} || �j|�}t|d|d�S )a
Convert array of dates with a cache and wrap the result in an Index.
Parameters
----------
arg : integer, float, string, datetime, list, tuple, 1-d array, Series
cache_array : Series
Cache of converted, unique dates
name : string, default None
Name for a DatetimeIndex
Returns
-------
result : Index-like of converted dates
r )r1 N)rM rN )rH r1 �maprP )r9 rJ rN r1 �resultr5 r5 r6 �_convert_and_box_cache� s rS c sF t jdd� t| |�D ��}� dk r:t j� fdd�|D ��}t||d�S )a�
Return results from array_strptime if a %z or %Z directive was passed.
Parameters
----------
result : ndarray
int64 date representations of the dates
timezones : ndarray
pytz timezone objects
tz : object
None or pytz timezone object
name : string, default None
Name for a DatetimeIndex
Returns
-------
tz_result : Index-like of parsed dates with timezone
c S s g | ]\}}t |�j|��qS r5 )r � tz_localize)�.0�res�zoner5 r5 r6 �
<listcomp>� s z3_return_parsed_timezone_results.<locals>.<listcomp>Nc s g | ]}|j � ��qS r5 )�
tz_convert)rU Z tz_result)rO r5 r6 rX � s )rN )�np�array�zipr. )rR � timezonesrO rN Z
tz_resultsr5 )rO r6 � _return_parsed_timezone_results� s
r^ ) rC rN rO �unit�errors�infer_datetime_format�dayfirst� yearfirst�exactc
5 C s� t | ttf�r tj| dd�} t| dd�}
t|
�rht | ttf�sLt| ||d�S |dkrd| j d�j
|�} | S t |
�r�t | ttf�s�yt| ||d�S t k
r� Y q�X n|r�| j
|�S | S |dk �r�|dk r�t d��t| d| �} t | t
��r | jd |� d
��} d} ntj| ||d �\} } |d k�r,t| |d
�} n t| |d
�} y| j
d�j | �} W n tk
�rb | S X |dk �r�| jdk�r�| j
|�} n
| j |�} | S t| dd�dk�r�td��| }
yt| dd�\} }W nb tk
�r( |dk�rtjdgdd�jt| ��} t| |d
�S |d k�r"t| |d
�} | S � Y nX t| �} d}|�rR|dk�rRt| |d�}|dk �rtt|�}|�rt| }d}d} d} |dk �rX�y^|dk�r�yt|
�}
t|
|d �} W n6 t tt j fk
�r� } zt d�|�W Y dd}~X nX | dk�r�y:t | || |d�\} }d|k�sd |k�r&t | |||�S W n� t j k
�r� |d k�rJ� n6|dk�r|tj | j!d d�} | j"d �}|j#t j$� n| } Y n` t k
�r� |�s�|d k�r�� n6|dk�r�tj | j!d d�} | j"d �}|j#t j$� n| } Y nX W np t k
�rV } zRy,t%j&| �\}}t|t'|�d�}tj(||d
�S t tfk
�rD |�Y nX W Y dd}~X nX | dk�r�|dk�sv|�svt)�|dk}t*| |||||d d!�\} } | dk �r�t| t'| �d�}tj(||d
�S |dk}t+| ||d"�S )#aH
Helper function for to_datetime. Performs the conversions of 1D listlike
of dates
Parameters
----------
arg : list, tuple, ndarray, Series, Index
date to be parsed
name : object
None or string for the Index name
tz : object
None or 'utc'
unit : string
None or string of the frequency of the passed data
errors : string
error handing behaviors from to_datetime, 'raise', 'coerce', 'ignore'
infer_datetime_format : boolean
inferring format behavior from to_datetime
dayfirst : boolean
dayfirst parsing behavior from to_datetime
yearfirst : boolean
yearfirst parsing behavior from to_datetime
exact : boolean
exact format matching behavior from to_datetime
Returns
-------
Index-like of parsed dates
�O)rF rF N)rO rN rM z#cannot specify both format and unit�_valuesz datetime64[�])r` �ignore)rN �UTC�ndimr> zAarg must be a string, datetime, list, tuple, 1-d array, or SeriesF)�copy�coerceZNaTzdatetime64[ns])rb z%Y%m%dz0cannot convert the input to '%Y%m%d' date format)rd r` z%Zz%z�raisezM8[ns]�i8T)rb rc rM r` �require_iso8601Z allow_object)rM rN ),�
isinstance�list�tuplerZ r[ �getattrr r' r/ rY rT r �
ValueErrorr( �astyper Zarray_with_unit_to_datetimer. �AttributeErrorrO � TypeErrorr+ �repeatr3 r r7 r �_attempt_YYYYMMDDr �OutOfBoundsDatetimer r^ �empty�shape�view�fill�iNaTr Zdatetime_to_datetime64r- Z _simple_newr@ r, rP )r9 rC rN rO r_ r` ra rb rc rd Z arg_dtyperR Z tz_parsedZorig_arg�_ro Zformat_is_iso8601�errr] �iresult�e�valuesZdtarM r5 r5 r6 �_convert_listlike_datetimes s� *
r� c C s� |dkr�| }t d�j� }|dkr(td��y | | } W n, tk
r` } ztd�|�W Y dd}~X nX t jj� | }t jj� | }tj| |k�s�tj| |k �r�tj |� d����n$t
| �r�t | �p�t | �p�t
tj| ��s�td| � d |� d
���y t |�}W nn tj k
�r0 } ztj d |� d ��|�W Y dd}~X n6 tk
�rd } ztd |� d
��|�W Y dd}~X nX |jdk �r�td|� d���|t d�8 }|tjd|d� }t| ��r�t| tttjf� �r�tj| �} | | } | S )a�
Helper function for to_datetime.
Adjust input argument to the specified origin
Parameters
----------
arg : list, tuple, ndarray, Series, Index
date to be adjusted
origin : 'julian' or Timestamp
origin offset for the arg
unit : string
passed unit from to_datetime, must be 'D'
Returns
-------
ndarray or scalar of adjusted date(s)
Zjulianr �Dz$unit must be 'D' for origin='julian'z3incompatible 'arg' type for given 'origin'='julian'Nz% is Out of Bounds for origin='julian'�'z!' is not compatible with origin='z+'; it must be numeric with a unit specifiedzorigin z is Out of Boundsz# cannot be converted to a Timestampzorigin offset z must be tz-naiver> )r_ )r Zto_julian_datert rw �max�minrZ �anyr rz r# r r r" ZasarrayrO Z Timedeltar! rp r% r. Zndarray) r9 �originr_ �originalZj0r� Zj_maxZj_min�offsetr5 r5 r6 �_adjust_to_origin� sL
r� .r0 ) r9 r` rb rc rM rC rd r_ ra rD r< c C s d S )Nr5 ) r9 r` rb rc rM rC rd r_ ra r� rD r5 r5 r6 � to_datetime* s r� c C s d S )Nr5 ) r9 r` rb rc rM rC rd r_ ra r� rD r5 r5 r6 r� ; s c C s d S )Nr5 ) r9 r` rb rc rM rC rd r_ ra r� rD r5 r5 r6 r� L s rm FT�unixc C s� | dkr dS | dkr t | | |�} |r(dnd} tt| ||||||d�} t| t�r~| }
| dk rz| jdk rp|
j| �}
n
|
j| �}
�n@t| t�r�t | ||
| �}|j
s�| j |�}
n | | j |�}| j
|| j| jd�}
n�t| ttjf�r�t| || �}
n�t| t��r.t | ||
| �}|j
�s t| || jd�}
n| | || jd�}
n�t| ��r�yt | ||
| �}W n< tjk
�r� |dk�rj� dd lm} |g td
�}Y nX |j
�s�t| |�}
n
| | |�}
n| t j | g�|�d }
|
S ) aA
Convert argument to datetime.
Parameters
----------
arg : int, float, str, datetime, list, tuple, 1-d array, Series, DataFrame/dict-like
The object to convert to a datetime.
errors : {'ignore', 'raise', 'coerce'}, default 'raise'
- If 'raise', then invalid parsing will raise an exception.
- If 'coerce', then invalid parsing will be set as NaT.
- If 'ignore', then invalid parsing will return the input.
dayfirst : bool, default False
Specify a date parse order if `arg` is str or its list-likes.
If True, parses dates with the day first, eg 10/11/12 is parsed as
2012-11-10.
Warning: dayfirst=True is not strict, but will prefer to parse
with day first (this is a known bug, based on dateutil behavior).
yearfirst : bool, default False
Specify a date parse order if `arg` is str or its list-likes.
- If True parses dates with the year first, eg 10/11/12 is parsed as
2010-11-12.
- If both dayfirst and yearfirst are True, yearfirst is preceded (same
as dateutil).
Warning: yearfirst=True is not strict, but will prefer to parse
with year first (this is a known bug, based on dateutil behavior).
utc : bool, default None
Return UTC DatetimeIndex if True (converting any tz-aware
datetime.datetime objects as well).
format : str, default None
The strftime to parse time, eg "%d/%m/%Y", note that "%f" will parse
all the way up to nanoseconds.
See strftime documentation for more information on choices:
https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior.
exact : bool, True by default
Behaves as:
- If True, require an exact format match.
- If False, allow the format to match anywhere in the target string.
unit : str, default 'ns'
The unit of the arg (D,s,ms,us,ns) denote the unit, which is an
integer or float number. This will be based off the origin.
Example, with unit='ms' and origin='unix' (the default), this
would calculate the number of milliseconds to the unix epoch start.
infer_datetime_format : bool, default False
If True and no `format` is given, attempt to infer the format of the
datetime strings based on the first non-NaN element,
and if it can be inferred, switch to a faster method of parsing them.
In some cases this can increase the parsing speed by ~5-10x.
origin : scalar, default 'unix'
Define the reference date. The numeric values would be parsed as number
of units (defined by `unit`) since this reference date.
- If 'unix' (or POSIX) time; origin is set to 1970-01-01.
- If 'julian', unit must be 'D', and origin is set to beginning of
Julian Calendar. Julian day number 0 is assigned to the day starting
at noon on January 1, 4713 BC.
- If Timestamp convertible, origin is set to Timestamp identified by
origin.
cache : bool, default True
If True, use a cache of unique, converted dates to apply the datetime
conversion. May produce significant speed-up when parsing duplicate
date strings, especially ones with timezone offsets. The cache is only
used when there are at least 50 values. The presence of out-of-bounds
values will render the cache unusable and may slow down parsing.
.. versionadded:: 0.23.0
.. versionchanged:: 0.25.0
- changed default value from False to True.
Returns
-------
datetime
If parsing succeeded.
Return type depends on input:
- list-like: DatetimeIndex
- Series: Series of datetime64 dtype
- scalar: Timestamp
In case when it is not possible to return designated types (e.g. when
any element of input is before Timestamp.min or after Timestamp.max)
return will have datetime.datetime type (or corresponding
array/Series).
See Also
--------
DataFrame.astype : Cast argument to a specified dtype.
to_timedelta : Convert argument to timedelta.
convert_dtypes : Convert dtypes.
Examples
--------
Assembling a datetime from multiple columns of a DataFrame. The keys can be
common abbreviations like ['year', 'month', 'day', 'minute', 'second',
'ms', 'us', 'ns']) or plurals of the same
>>> df = pd.DataFrame({'year': [2015, 2016],
... 'month': [2, 3],
... 'day': [4, 5]})
>>> pd.to_datetime(df)
0 2015-02-04
1 2016-03-05
dtype: datetime64[ns]
If a date does not meet the `timestamp limitations
<https://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html
#timeseries-timestamp-limits>`_, passing errors='ignore'
will return the original input instead of raising any exception.
Passing errors='coerce' will force an out-of-bounds date to NaT,
in addition to forcing non-dates (or non-parseable dates) to NaT.
>>> pd.to_datetime('13000101', format='%Y%m%d', errors='ignore')
datetime.datetime(1300, 1, 1, 0, 0)
>>> pd.to_datetime('13000101', format='%Y%m%d', errors='coerce')
NaT
Passing infer_datetime_format=True can often-times speedup a parsing
if its not an ISO8601 format exactly, but in a regular format.
>>> s = pd.Series(['3/11/2000', '3/12/2000', '3/13/2000'] * 1000)
>>> s.head()
0 3/11/2000
1 3/12/2000
2 3/13/2000
3 3/11/2000
4 3/12/2000
dtype: object
>>> %timeit pd.to_datetime(s, infer_datetime_format=True) # doctest: +SKIP
100 loops, best of 3: 10.4 ms per loop
>>> %timeit pd.to_datetime(s, infer_datetime_format=False) # doctest: +SKIP
1 loop, best of 3: 471 ms per loop
Using a unix epoch time
>>> pd.to_datetime(1490195805, unit='s')
Timestamp('2017-03-22 15:16:45')
>>> pd.to_datetime(1490195805433502912, unit='ns')
Timestamp('2017-03-22 15:16:45.433502912')
.. warning:: For float arg, precision rounding might happen. To prevent
unexpected behavior use a fixed-width exact type.
Using a non-unix epoch origin
>>> pd.to_datetime([1, 2, 3], unit='D',
... origin=pd.Timestamp('1960-01-01'))
DatetimeIndex(['1960-01-02', '1960-01-03', '1960-01-04'], dtype='datetime64[ns]', freq=None)
Nr� rM )rO r_ rb rc r` rd ra )rG rN )rN rm r )r1 )rF ) r� r r� rp r rO rY rT r% rK r{ rQ rf Z _constructorrG rN r$ r �MutableMapping� _assemble_from_unit_mappingsr. rS r! r rz rH r1 rI rZ r[ )r9 r` rb rc rM rC rd r_ ra r� rD rO rE rR rJ r� r1 r5 r5 r6 r� ] s^ )
�year�month�day�h�m�s�ms�us�ns)r� Zyearsr� �monthsr� �days�hour�hours�minute�minutes�second�secondsr� Z millisecondZ millisecondsr� � microsecond� microsecondsr� Z
nanosecond� nanosecondsc s ddl m}m�m} || �} | jjs,td��dd� ��fdd�| j� D �}dd� |j� D �}d d
d g}t t
|�t
|j� � �}t |�r�d j |�} td
| � d���t t
|j� �t
t
j� � �}
t |
�r�d j |
�} td| � d���� �fdd�} | | |d �d | | |d
�d | | |d � }
yt|
d� |d�}
W n8 ttfk
�rp } ztd|� ��|�W Y dd}~X nX x�d"D ]�}|j|�}|dk �rx|| k�rxy |
|| | | �|� d �7 }
W n> ttfk
�r� } z td |� d!|� ��|�W Y dd}~X nX �qxW |
S )#a�
assemble the unit specified fields from the arg (DataFrame)
Return a Series for actual parsing
Parameters
----------
arg : DataFrame
errors : {'ignore', 'raise', 'coerce'}, default 'raise'
- If 'raise', then invalid parsing will raise an exception
- If 'coerce', then invalid parsing will be set as NaT
- If 'ignore', then invalid parsing will return the input
tz : None or 'utc'
Returns
-------
Series
r )� DataFrame�
to_numeric� to_timedeltaz#cannot assemble with duplicate keysc S s, | t krt | S | j� t kr(t | j� S | S )N)� _unit_map�lower)�valuer5 r5 r6 �fw s
z'_assemble_from_unit_mappings.<locals>.fc s i | ]}� |�|�qS r5 r5 )rU �k)r� r5 r6 �
<dictcomp>� s z0_assemble_from_unit_mappings.<locals>.<dictcomp>c S s i | ]\}}||�qS r5 r5 )rU r� �vr5 r5 r6 r� � s r� r� r� �,zNto assemble mappings requires at least that [year, month, day] be specified: [z ] is missingz9extra keys have been passed to the datetime assemblage: [rg c s& �| � d�} t | �r"| jddd�} | S )N)r` �int64F)rk )r ru )r� )r` r� r5 r6 rl � s z,_assemble_from_unit_mappings.<locals>.coercei' �d z%Y%m%d)rC r` rM z cannot assemble the datetimes: Nr� r� r� r� r� r� )r_ r` z cannot assemble the datetimes [z]: )r� r� r� r� r� r� )rH r� r� r� �columnsZ is_uniquert �keys�items�sortedrA r3 �joinr� r� r� rw �get)r9 r` rO r� r� r_ Zunit_rev�required�reqZ _requiredZexcessZ_excessrl r� r� �ur� r5 )r` r� r� r6 r� ] sD
8"
r� c s� �fdd�� � fdd�}y� | j tj��S tttfk
rB Y nX y| j tj�}||t|��S tttfk
rx Y nX y tj | t
t j �� }|| |�S tttfk
r� Y nX dS )a"
try to parse the YYYYMMDD/%Y%m%d format, try to deal with NaT-like,
arg is a passed in as an object dtype, but could really be ints/strings
with nan-like/or floats (e.g. with nan)
Parameters
----------
arg : passed value
errors : 'raise','ignore','coerce'
c s: | j t�} tj| d | d d | d �}tj|� d�d S )Ni' r� )r` r )ru rI r Ztry_parse_year_month_dayr Zarray_to_datetime)�carg�parsed)r` r5 r6 �calc� s
z _attempt_YYYYMMDD.<locals>.calcc sT t j| jdd�}|jd�}tj|| < � | | jt j�jt j��}|jd�||< |S )NzM8[ns])rF rn ) rZ r{ r| r} r r ru �float64r� )r� �maskrR r� Z
masked_result)r� r5 r6 �calc_with_mask� s
z)_attempt_YYYYMMDD.<locals>.calc_with_maskN)
ru rZ r� rt �
OverflowErrorrw r� r& r) �isinrq r Z nat_strings)r9 r` r� r� r� r5 )r� r` r6 ry � s"
ry c C s* t jdtdd� ddlm} || |||�S )Nz{`to_time` has been moved, should be imported from pandas.core.tools.times. This alias will be removed in a future version.� )�
stacklevelr )�to_time)�warnings�warn�
FutureWarningZpandas.core.tools.timesr� )r9 rC Zinfer_time_formatr` r� r5 r5 r6 r� � s r� )r8 N)NN)N)NNNNNNNN)
..........)
..........)
..........)
rm FFNNTNFr� T)NFrm )\� collectionsr r � functoolsr � itertoolsr Ztypingr r r r r
r r r
r� ZnumpyrZ Z pandas._libsr r Zpandas._libs.tslibsr r r Zpandas._libs.tslibs.parsingr r r Z pandas._libs.tslibs.strptimer Zpandas._typingr r r Zpandas.core.dtypes.commonr r r r r r r r! r" r# Zpandas.core.dtypes.genericr$ r% Zpandas.core.dtypes.missingr& Z
pandas.arraysr' r( Z pandas.corer) Zpandas.core.algorithmsr* Z pandas.core.arrays.datetimesr+ r, r- Zpandas.core.indexes.baser. Z pandas.core.indexes.datetimesr/ Zpandas._libs.tslibs.nattyper0 rH r1 ZArrayConvertibler? �float�strZScalarr2 Z DatetimeScalarOrArrayConvertibler7 �boolrB rK rP rS r^ r� r� r� r� r� ry r� r5 r5 r5 r6 �<module> s� (
0 7$ @ SK " " * " ]X4 |