HOME

sh-3ll 1.0
DIR:/usr/local/lib64/python3.6/site-packages/pandas/core/reshape/__pycache__/
Current File : //usr/local/lib64/python3.6/site-packages/pandas/core/reshape/__pycache__/merge.cpython-36.pyc
3

���h��
@sTdZddlZddlZddlmZddlZddlmZmZm	Z	m
Z
ddlZddlZ
ddlmZmZmZddljjZddlmZmZddlmZddlmZmZdd	lm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1dd
l2m3Z3m4Z4ddl5m6Z6m7Z7ddl8m9Z9m:Z:m;Z;dd
l<m=Z=ddl>j?j@ZAddlBmCZCddlDj?jEZFddlGmHZHddlImJZJddlKmLZLddlMmNZNe�r�ddl8mOZOmPZPed�eeJdd�d\eQeReReReReRdd�dd���ZSeJdeS_ddd�d d!�ZTd^eQdd#�d$d%�ZUd`eReReReQdd'�d(d)�ZVGd*d+�d+�ZWdaeReQd,�d-d.�ZXe;e;d�d/d0�ZYGd1d2�d2eW�ZZeQd3�d4d5�Z[eQd3�d6d7�Z\e!e e"d8�Z]d9d:�Z^Gd;d<�d<eZ�Z_e;eRd=�d>d?�Z`dbeRd@�dAdB�Zadce:e:eRdC�dDdE�ZbdFdG�ZcddeeeReQe	e
jde
jdeefdH�dIdJ�Zfe
jgdK�dLdM�ZheRd@�dNdO�ZieRdP�dQdR�ZjeRdP�dSdT�ZkeddU�dVdW�Zle:e:e	eQeQfdX�dYdZ�ZmdS)ez
SQL-style merge routines
�N)�partial)�
TYPE_CHECKING�Optional�Tuple�Union)�	Timedelta�	hashtable�lib)�	ArrayLike�
FrameOrSeries)�
MergeError)�Appender�Substitution)�ensure_float64�ensure_int64�
ensure_object�
is_array_like�is_bool�
is_bool_dtype�is_categorical_dtype�is_datetime64tz_dtype�is_dtype_equal�is_extension_array_dtype�is_float_dtype�
is_integer�is_integer_dtype�is_list_like�	is_number�is_numeric_dtype�is_object_dtype�needs_i8_conversion)�ABCDataFrame�	ABCSeries)�isna�na_value_for_dtype)�Categorical�Index�
MultiIndex)�groupby)�recode_for_categories)�
extract_array)�
_merge_doc)�concatenate_block_managers)�is_int64_overflow_possible)�	DataFrame�Seriesz
left : DataFrame)�indents�innerF�_x�_yTr.)�how�
left_index�right_index�sort�copy�	indicator�returnc
Cs*t||||||||||	|
||d�
}
|
j�S)N)r4�on�left_on�right_onr5r6r7�suffixesr8r9�validate)�_MergeOperation�
get_result)�left�rightr4r;r<r=r5r6r7r>r8r9r?�op�rE�;/tmp/pip-build-5_djhm0z/pandas/pandas/core/reshape/merge.py�merge9srG)rBrCcsJg}t|ttf�s|g}|j|dd�}d}y|j|dd�}Wntk
rPYnXx�|D]�\}}	|dkrn|}
nry|j|j|�}
Wn\tk
r�|	jj����fdd�|jD�}|	j	|d�}t
t|��|_|j
|�wXYnX||	|
�}x|D]}
|
|kr�|||
<q�W|j
|�qXWddlm}||d	d
�}|j	|djdd�}||fS)z�
    groupby & merge; we are always performing a left-by type operation

    Parameters
    ----------
    by: field to group
    on: duplicates field
    left: DataFrame
    right: DataFrame
    merge_pieces: function for merging
    F)r7Ncsg|]}|t��kr|�qSrE)�set)�.0�r)�lcolsrErF�
<listcomp>�sz&_groupby_and_merge.<locals>.<listcomp>)�columnsr)�concatT)Zignore_index)rMr8)�
isinstance�list�tupler(�KeyError�take�indicesrM�tolistZreindex�range�len�index�appendZpandas.core.reshape.concatrN)�byr;rBrCZmerge_pieces�piecesZlbyZrby�key�lhs�rhs�colsZmerged�krN�resultrE)rKrF�_groupby_and_merge`s<



rb�outer)r4r:c
s�������fdd��|dk	r0|dk	r0td��nV|dk	rVt|�||�fdd��\}
}n0|dk	r|t|�||�fdd��\}
}n
�||�}
|
S)a�
    Perform merge with optional filling/interpolation.

    Designed for ordered data like time series data. Optionally
    perform group-wise merge (see examples).

    Parameters
    ----------
    left : DataFrame
    right : DataFrame
    on : label or list
        Field names to join on. Must be found in both DataFrames.
    left_on : label or list, or array-like
        Field names to join on in left DataFrame. Can be a vector or list of
        vectors of the length of the DataFrame to use a particular vector as
        the join key instead of columns.
    right_on : label or list, or array-like
        Field names to join on in right DataFrame or vector/list of vectors per
        left_on docs.
    left_by : column name or list of column names
        Group left DataFrame by group columns and merge piece by piece with
        right DataFrame.
    right_by : column name or list of column names
        Group right DataFrame by group columns and merge piece by piece with
        left DataFrame.
    fill_method : {'ffill', None}, default None
        Interpolation method for data.
    suffixes : list-like, default is ("_x", "_y")
        A length-2 sequence where each element is optionally a string
        indicating the suffix to add to overlapping column names in
        `left` and `right` respectively. Pass a value of `None` instead
        of a string to indicate that the column name from `left` or
        `right` should be left as-is, with no suffix. At least one of the
        values must not be None.

        .. versionchanged:: 0.25.0
    how : {'left', 'right', 'outer', 'inner'}, default 'outer'
        * left: use only keys from left frame (SQL: left outer join)
        * right: use only keys from right frame (SQL: right outer join)
        * outer: use union of keys from both frames (SQL: full outer join)
        * inner: use intersection of keys from both frames (SQL: inner join).

    Returns
    -------
    DataFrame
        The merged DataFrame output type will the be same as
        'left', if it is a subclass of DataFrame.

    See Also
    --------
    merge : Merge with a database-style join.
    merge_asof : Merge on nearest keys.

    Examples
    --------
    >>> df1 = pd.DataFrame(
    ...     {
    ...         "key": ["a", "c", "e", "a", "c", "e"],
    ...         "lvalue": [1, 2, 3, 1, 2, 3],
    ...         "group": ["a", "a", "a", "b", "b", "b"]
    ...     }
    ... )
    >>> df1
          key  lvalue group
    0   a       1     a
    1   c       2     a
    2   e       3     a
    3   a       1     b
    4   c       2     b
    5   e       3     b

    >>> df2 = pd.DataFrame({"key": ["b", "c", "d"], "rvalue": [1, 2, 3]})
    >>> df2
          key  rvalue
    0   b       1
    1   c       2
    2   d       3

    >>> merge_ordered(df1, df2, fill_method="ffill", left_by="group")
      key  lvalue group  rvalue
    0   a       1     a     NaN
    1   b       1     a     1.0
    2   c       2     a     2.0
    3   d       2     a     3.0
    4   e       3     a     3.0
    5   a       1     b     NaN
    6   b       1     b     1.0
    7   c       2     b     2.0
    8   d       2     b     3.0
    9   e       3     b     3.0
    c
s t||������d�}|j�S)N)r;r<r=r>�fill_methodr4)�
_OrderedMergerA)�x�yrD)rdr4r<r;r=r>rErF�_mergerszmerge_ordered.<locals>._mergerNz*Can only group either left or right framescs
�||�S)NrE)rfrg)rhrErF�<lambda>szmerge_ordered.<locals>.<lambda>cs
�||�S)NrE)rfrg)rhrErFris)�
ValueErrorrb)rBrCr;r<r=�left_by�right_byrdr>r4ra�_rE)rhrdr4r<r;r=r>rF�
merge_ordered�sh

rn�backward)r5r6�allow_exact_matches�	directionr:cCs.t||||||||||	|
d|||
d�}|j�S)a#
    Perform an asof merge.

    This is similar to a left-join except that we match on nearest
    key rather than equal keys. Both DataFrames must be sorted by the key.

    For each row in the left DataFrame:

      - A "backward" search selects the last row in the right DataFrame whose
        'on' key is less than or equal to the left's key.

      - A "forward" search selects the first row in the right DataFrame whose
        'on' key is greater than or equal to the left's key.

      - A "nearest" search selects the row in the right DataFrame whose 'on'
        key is closest in absolute distance to the left's key.

    The default is "backward" and is compatible in versions below 0.20.0.
    The direction parameter was added in version 0.20.0 and introduces
    "forward" and "nearest".

    Optionally match on equivalent keys with 'by' before searching with 'on'.

    Parameters
    ----------
    left : DataFrame
    right : DataFrame
    on : label
        Field name to join on. Must be found in both DataFrames.
        The data MUST be ordered. Furthermore this must be a numeric column,
        such as datetimelike, integer, or float. On or left_on/right_on
        must be given.
    left_on : label
        Field name to join on in left DataFrame.
    right_on : label
        Field name to join on in right DataFrame.
    left_index : bool
        Use the index of the left DataFrame as the join key.
    right_index : bool
        Use the index of the right DataFrame as the join key.
    by : column name or list of column names
        Match on these columns before performing merge operation.
    left_by : column name
        Field names to match on in the left DataFrame.
    right_by : column name
        Field names to match on in the right DataFrame.
    suffixes : 2-length sequence (tuple, list, ...)
        Suffix to apply to overlapping column names in the left and right
        side, respectively.
    tolerance : int or Timedelta, optional, default None
        Select asof tolerance within this range; must be compatible
        with the merge index.
    allow_exact_matches : bool, default True

        - If True, allow matching with the same 'on' value
          (i.e. less-than-or-equal-to / greater-than-or-equal-to)
        - If False, don't match the same 'on' value
          (i.e., strictly less-than / strictly greater-than).

    direction : 'backward' (default), 'forward', or 'nearest'
        Whether to search for prior, subsequent, or closest matches.

    Returns
    -------
    merged : DataFrame

    See Also
    --------
    merge : Merge with a database-style join.
    merge_ordered : Merge with optional filling/interpolation.

    Examples
    --------
    >>> left = pd.DataFrame({"a": [1, 5, 10], "left_val": ["a", "b", "c"]})
    >>> left
        a left_val
    0   1        a
    1   5        b
    2  10        c

    >>> right = pd.DataFrame({"a": [1, 2, 3, 6, 7], "right_val": [1, 2, 3, 6, 7]})
    >>> right
       a  right_val
    0  1          1
    1  2          2
    2  3          3
    3  6          6
    4  7          7

    >>> pd.merge_asof(left, right, on="a")
        a left_val  right_val
    0   1        a          1
    1   5        b          3
    2  10        c          7

    >>> pd.merge_asof(left, right, on="a", allow_exact_matches=False)
        a left_val  right_val
    0   1        a        NaN
    1   5        b        3.0
    2  10        c        7.0

    >>> pd.merge_asof(left, right, on="a", direction="forward")
        a left_val  right_val
    0   1        a        1.0
    1   5        b        6.0
    2  10        c        NaN

    >>> pd.merge_asof(left, right, on="a", direction="nearest")
        a left_val  right_val
    0   1        a          1
    1   5        b          6
    2  10        c          7

    We can use indexed DataFrames as well.

    >>> left = pd.DataFrame({"left_val": ["a", "b", "c"]}, index=[1, 5, 10])
    >>> left
       left_val
    1         a
    5         b
    10        c

    >>> right = pd.DataFrame({"right_val": [1, 2, 3, 6, 7]}, index=[1, 2, 3, 6, 7])
    >>> right
       right_val
    1          1
    2          2
    3          3
    6          6
    7          7

    >>> pd.merge_asof(left, right, left_index=True, right_index=True)
       left_val  right_val
    1         a          1
    5         b          3
    10        c          7

    Here is a real-world times-series example

    >>> quotes = pd.DataFrame(
    ...     {
    ...         "time": [
    ...             pd.Timestamp("2016-05-25 13:30:00.023"),
    ...             pd.Timestamp("2016-05-25 13:30:00.023"),
    ...             pd.Timestamp("2016-05-25 13:30:00.030"),
    ...             pd.Timestamp("2016-05-25 13:30:00.041"),
    ...             pd.Timestamp("2016-05-25 13:30:00.048"),
    ...             pd.Timestamp("2016-05-25 13:30:00.049"),
    ...             pd.Timestamp("2016-05-25 13:30:00.072"),
    ...             pd.Timestamp("2016-05-25 13:30:00.075")
    ...         ],
    ...         "ticker": [
    ...                "GOOG",
    ...                "MSFT",
    ...                "MSFT",
    ...                "MSFT",
    ...                "GOOG",
    ...                "AAPL",
    ...                "GOOG",
    ...                "MSFT"
    ...            ],
    ...            "bid": [720.50, 51.95, 51.97, 51.99, 720.50, 97.99, 720.50, 52.01],
    ...            "ask": [720.93, 51.96, 51.98, 52.00, 720.93, 98.01, 720.88, 52.03]
    ...     }
    ... )
    >>> quotes
                         time ticker     bid     ask
    0 2016-05-25 13:30:00.023   GOOG  720.50  720.93
    1 2016-05-25 13:30:00.023   MSFT   51.95   51.96
    2 2016-05-25 13:30:00.030   MSFT   51.97   51.98
    3 2016-05-25 13:30:00.041   MSFT   51.99   52.00
    4 2016-05-25 13:30:00.048   GOOG  720.50  720.93
    5 2016-05-25 13:30:00.049   AAPL   97.99   98.01
    6 2016-05-25 13:30:00.072   GOOG  720.50  720.88
    7 2016-05-25 13:30:00.075   MSFT   52.01   52.03

    >>> trades = pd.DataFrame(
    ...        {
    ...            "time": [
    ...                pd.Timestamp("2016-05-25 13:30:00.023"),
    ...                pd.Timestamp("2016-05-25 13:30:00.038"),
    ...                pd.Timestamp("2016-05-25 13:30:00.048"),
    ...                pd.Timestamp("2016-05-25 13:30:00.048"),
    ...                pd.Timestamp("2016-05-25 13:30:00.048")
    ...            ],
    ...            "ticker": ["MSFT", "MSFT", "GOOG", "GOOG", "AAPL"],
    ...            "price": [51.95, 51.95, 720.77, 720.92, 98.0],
    ...            "quantity": [75, 155, 100, 100, 100]
    ...        }
    ...    )
    >>> trades
                         time ticker   price  quantity
    0 2016-05-25 13:30:00.023   MSFT   51.95        75
    1 2016-05-25 13:30:00.038   MSFT   51.95       155
    2 2016-05-25 13:30:00.048   GOOG  720.77       100
    3 2016-05-25 13:30:00.048   GOOG  720.92       100
    4 2016-05-25 13:30:00.048   AAPL   98.00       100

    By default we are taking the asof of the quotes

    >>> pd.merge_asof(trades, quotes, on="time", by="ticker")
                         time ticker   price  quantity     bid     ask
    0 2016-05-25 13:30:00.023   MSFT   51.95        75   51.95   51.96
    1 2016-05-25 13:30:00.038   MSFT   51.95       155   51.97   51.98
    2 2016-05-25 13:30:00.048   GOOG  720.77       100  720.50  720.93
    3 2016-05-25 13:30:00.048   GOOG  720.92       100  720.50  720.93
    4 2016-05-25 13:30:00.048   AAPL   98.00       100     NaN     NaN

    We only asof within 2ms between the quote time and the trade time

    >>> pd.merge_asof(
    ...     trades, quotes, on="time", by="ticker", tolerance=pd.Timedelta("2ms")
    ... )
                         time ticker   price  quantity     bid     ask
    0 2016-05-25 13:30:00.023   MSFT   51.95        75   51.95   51.96
    1 2016-05-25 13:30:00.038   MSFT   51.95       155     NaN     NaN
    2 2016-05-25 13:30:00.048   GOOG  720.77       100  720.50  720.93
    3 2016-05-25 13:30:00.048   GOOG  720.92       100  720.50  720.93
    4 2016-05-25 13:30:00.048   AAPL   98.00       100     NaN     NaN

    We only asof within 10ms between the quote time and the trade time
    and we exclude exact matches on time. However *prior* data will
    propagate forward

    >>> pd.merge_asof(
    ...     trades,
    ...     quotes,
    ...     on="time",
    ...     by="ticker",
    ...     tolerance=pd.Timedelta("10ms"),
    ...     allow_exact_matches=False
    ... )
                         time ticker   price  quantity     bid     ask
    0 2016-05-25 13:30:00.023   MSFT   51.95        75     NaN     NaN
    1 2016-05-25 13:30:00.038   MSFT   51.95       155   51.97   51.98
    2 2016-05-25 13:30:00.048   GOOG  720.77       100     NaN     NaN
    3 2016-05-25 13:30:00.048   GOOG  720.92       100     NaN     NaN
    4 2016-05-25 13:30:00.048   AAPL   98.00       100     NaN     NaN
    �asof)
r;r<r=r5r6rZrkrlr>r4�	tolerancerprq)�
_AsOfMergerA)rBrCr;r<r=r5r6rZrkrlr>rsrprqrDrErErF�
merge_asof#s&ruc@s�eZdZdZdZd,ed-ed.eeeeeed�d
d�Zdd�Z	dde
d/d�dd�Zdd�Zdd�Z
dd�Zdd�Zdd�Zd0eeed�d d!�Zd"d#�Zd$d%�Zd&d'�Zed(�d)d*�ZdS)1r@z�
    Perform a database (SQL) merge operation between two DataFrame or Series
    objects using either columns as keys or their row indexes
    rGr1N�FTr2r3r/r.)rBrCr4r5r6r7r8r9cCs�t|�}t|�}||_|_||_|_||_||_|jjdkrHd|nd|_t	j
|�|_t	j
|�|_t	j
|�|_
||_||_|
|_||_|	|_|
|_|t|jt�r�|j|_n&t|jt�r�|jr�dnd|_ntd��t|�s�tdt|�����t|	��stdt|	�����|jj|jjk�rDd|jj�d	|jj�d
�}tj|t�|j �|j!�\|_"|_#|_$|j%�|dk	�r||j&|�dS)N�rvrZ_mergez<indicator option can only accept boolean or string argumentsz/left_index parameter must be of type bool, not z0right_index parameter must be of type bool, not z@merging between different levels can give an unintended result (z levels on the left,z on the right))'�_validate_operandrB�	orig_leftrC�
orig_rightr4Zbm_axis�ndim�axis�comZmaybe_make_listr;r<r=r8r>r7r5r6r9rO�str�indicator_name�boolrjr�typerM�nlevels�warnings�warn�UserWarning�_validate_specification�_get_merge_keys�left_join_keys�right_join_keys�
join_names�_maybe_coerce_merge_keys�	_validate)�selfrBrCr4r;r<r=r|r5r6r7r>r8r9r?�_leftZ_right�msgrErErF�__init__@sJ


z_MergeOperation.__init__cCs�|jr|j|j|j�\|_|_|j�\}}}t|jj|jj|j�\}}|dk	rVd|ini}|dk	rjd|ini}t|jj	|f|jj	|fg|j
|�|gd|jd�}|jj}	|	|�j
||jd�}
|jr�|j|
�}
|j|
||�|j|
�|
S)Nrvr)�axes�concat_axisr8)�method)r9�_indicator_pre_mergerBrC�_get_join_info�_items_overlap_with_suffix�
_info_axisr>r,�_mgrrYr8�_constructor�__finalize__�_merge_type�_indicator_post_merge�_maybe_add_join_keys�_maybe_restore_index_levels)r��
join_index�left_indexer�
right_indexer�llabels�rlabels�	lindexers�	rindexers�result_data�typrarErErFrA�s&


z_MergeOperation.get_result)rBrCr:cCs�|jj|j�}x"dD]}||krtd|����qW|j|krDtd��|j�}|j�}d|d<|djd�|d<d|d<|djd�|d<||fS)	N�_left_indicator�_right_indicatorzECannot use `indicator=True` option when data contains a column named z:Cannot use name of an existing column for indicator columnrvZint8rw)r�r�)rM�unionrjrr8�astype)r�rBrCrM�irErErFr��s

z$_MergeOperation._indicator_pre_mergecCs||djd�|d<|djd�|d<t|d|ddddgd�||j<||jjjdd	d
g�||j<|jddgdd�}|S)Nr�rr�rvrw�)�
categoriesZ	left_onlyZ
right_onlyZboth)�labelsr|)Zfillnar%r�catZrename_categoriesZdrop)r�rarErErFr��sz%_MergeOperation._indicator_post_mergecCsjg}xNt|j|j|j�D]8\}}}|jj|�r|jj|�r||jjkr|j	|�qW|rf|j
|dd�dS)a
        Restore index levels specified as `on` parameters

        Here we check for cases where `self.left_on` and `self.right_on` pairs
        each reference an index level in their respective DataFrames. The
        joined columns corresponding to these pairs are then restored to the
        index of `result`.

        **Note:** This method has side effects. It modifies `result` in-place

        Parameters
        ----------
        result: DataFrame
            merge result

        Returns
        -------
        None
        T)�inplaceN)�zipr�r<r=ry�_is_level_referencerzrX�namesrY�	set_index)r�raZnames_to_restore�name�left_key�	right_keyrErErFr��sz+_MergeOperation._maybe_restore_index_levelscsld}d}t|j|j|j�}�xJt|�D�]<\}\�}}	t||	�sBq&d	\}
}��k�r|dk	ob|dk	�r>�|jkr�|dkr�|d
kj�}|r�|j|}t	��j
|j�j
�s�|j�j}
nV�|jk�r>|dkr�|dkj�}|�r>|j
|}
t	��j
|j�j
��s>|j�j}n.|dk	�r>t|j
|��r>|j
|}
|j|}|
dk	�sP|dk	r&|
dk�rf��j}nt|
j
�}
tj|
||
d�}|dk�r���j}nt|j
�}tj|||d�}|dk}|j��r�|�nt|�j||���j���r���<q&�j���rJt�jt��r:��_���fdd��jjD�}�j|dd�nt��d��_q&�j|��p^d|����q&WdS)
Nrv)�
fill_valuecs$g|]}|�kr�jj|�n��qSrE)rXZget_level_values)rIZ
level_name)�key_colr�rarErFrLPsz8_MergeOperation._maybe_add_join_keys.<locals>.<listcomp>T)r�)r��key_)NN���r�r�)r�r�r<r=�	enumerate�_should_fillrB�anyr�r�dtype�_valuesrCr�rr$�algosZtake_1d�allr&�whereZ_is_label_referencer�rOrXr'r�r�r��insert)r�rar�r�Zleft_has_missingZright_has_missing�keysr��lname�rnameZ	take_leftZ
take_rightZlvalsZlfillZrvalsZrfill�maskZidx_listrE)r�r�rarFr�sf












z$_MergeOperation._maybe_add_join_keyscCst|j|j|j|jd�S)z return the join indexers )r7r4)�_get_join_indexersr�r�r7r4)r�rErErFr�\sz"_MergeOperation._get_join_indexerscCs�|jj|j}|jj|j}|jrR|jrR|jdkrR|j||jd|jd�\}}}�n*|jr�|jdkr�t	|||j
|jd�\}}}�n�|jr�|jdkr�t	|||j|jd�\}}}n�|j�\}}|j�rt
|j�dkr�|j|jj|jj||dd�}n"|jjj|�}tjd
gt
|��}nl|j�rjt
|j�dk�rF|j|jj|jj||dd�}n"|jjj|�}tjdgt
|��}nttjt
|���}t
|�dk�r�|jt�}|||fS)NrrT)r4Zreturn_indexersr7rB)r7rCr)r4rvr�r�)rBr�r|rCr5r6r4�joinr7�_left_join_on_indexr�r�r�rW�_create_join_indexrXrS�np�arrayr&Zaranger��object)r��left_ax�right_axr�r�r�rErErFr�bsH


z_MergeOperation._get_join_inforB)rX�other_indexr4cCsT|j|dfkrJt|t�rJ|dk}tj|�rJt|jdd�}|jt|g��}|j	|�S)a�
        Create a join index by rearranging one index to match another

        Parameters
        ----------
        index: Index being rearranged
        other_index: Index used to supply values not found in index
        indexer: how to rearrange index
        how: replacement is only necessary if indexer based on other_index

        Returns
        -------
        join_index
        rcrvF)�compatr�)
r4rOr'r�r�r$r�rYr&rS)r�rXr��indexerZ
other_indexerr4r�r�rErErFr��s
z"_MergeOperation._create_join_indexcs�g}g}g}g}g}|j|j���fdd�}�fdd�}t|j�oLt|j��r��xtt|j|j�D�].\}}	||�r�|j|�||	�r�|j|	�|jd�n>|	dk	r�|j�j|	��|j|	�n|j�j�|j�jj	�qb||	��sH|	dk	�r|j�j|	��n|j�j�|dk	�rR||	k�rRt
��dk�r<|j|	�n
|j|�n
|j|	�|dk	�rx|j�j|��|j|�qb|j�j�|j�jj	�qbW�n.t|j��r2xJ|jD]@}
||
��r�|j|
�|jd�n|j�j|
��|j|
��q�Wt|jjt��r$dd�t|jjj
|jjj�D�}n|jjjg}n�t|j��r�xJ|jD]@}
||
��rj|j|
�|jd�n|j�j|
��|j|
��qFWt|jjt��r�dd�t|jjj
|jjj�D�}n|jjjg}|�r�|jj|�|_|�r�|jj|�|_|||fS)	z�
        Note: has side effects (copy/delete key columns)

        Parameters
        ----------
        left
        right
        on

        Returns
        -------
        left_keys, right_keys
        cst|�ot|�t��kS)N)rrW)rf)rBrErFri�sz1_MergeOperation._get_merge_keys.<locals>.<lambda>cst|�ot|�t��kS)N)rrW)rf)rCrErFri�sNrcSsg|]\}}|jj|��qSrE)r�rS)rI�lev�	lev_codesrErErFrL
sz3_MergeOperation._get_merge_keys.<locals>.<listcomp>cSsg|]\}}|jj|��qSrE)r�rS)rIr�r�rErErFrLs)rBrC�_anyr<r=r�rYZ_get_label_or_level_valuesrXr�rWrOr'�levels�codesr�Z_drop_labels_or_levels)r��	left_keys�
right_keysr�Z
right_dropZ	left_dropZis_lkeyZis_rkey�lk�rkr`rE)rBrCrFr��s~










z_MergeOperation._get_merge_keyscCs��xxt|j|j|j�D�]`\}}}t|�r4t|�st|�rHt|�rHqt|j�}t|j�}t|j�}t|j�}|r�|r�|j|�r�qn|s�|r�nt|j|j�r�qd|j�d|j�d�}t	|j�o�t	|j��r�|jj
|jj
kr�qn�t|j�o�t|j��r&||j
|j�ktj|�j�stjdt�qnht|j��rnt|j��rn||j
|j�ktj|�j�stjdt�qn tj|dd�tj|dd�k�r�q�nh|�r�t|j��s�t|j��r�|�r��n@|�r�t	|j��s�t	|j��r\|�r\tj|dd�}	tj|dd�}
ddd	d
g}ddd
dd
g}|	|k�r*|
|k�r*n0|	|k�r>|
|k�sR|
|k�r�|	|k�r�t|��n�t|j��r�t|j��r�t|��nzt|j��r�t|j��r�t|��nVt|j��r�t|j��r�t|��n2t|j��r�t|j��r�t|��n|�r�|�r�q||jjk�r:|�r|jjnt}
|jjf||j|j
|
�i�|_||jjkr|�rT|jjnt}
|jjf||j|j
|
�i�|_qWdS)NzYou are trying to merge on z and z9 columns. If you wish to proceed you should use pd.concatziYou are merging on int and float columns where the float values are not equal to their int representationF)Zskipna�integerz
mixed-integer�boolean�empty�string�unicode�mixed�bytes) r�r�r�r�rWrr�rrr�kindrrr�r��isnanr�r�r�r�r	Zinfer_dtyperrjr rrBrMr�r�ZassignrC)r�r�r�r�Z	lk_is_catZ	rk_is_catZlk_is_objectZrk_is_objectr�Z
inferred_leftZinferred_rightZ
bool_types�string_typesr�rErErFr�+s�$




  





	 z(_MergeOperation._maybe_coerce_merge_keysc	Cs�|jdko|jdko|jdk�r|jr<|jr<ff|_|_n�|jrV|jdkrTtd��n�|jrp|jdkrntd��n�|jj}|jj}|j	|�}t
|�dkr�td|j�d|j�d|j�d|j����|j|dd	�js�|j|dd	�jr�td
t
|�����||_|_n�|jdk	�r>|jdk	�s&|jdk	�r.td��|j|_|_n�|jdk	�r�t
|j�}|j�r�t
|j�|jjjk�rztd��dg||_nH|jdk	�r�t
|j�}|j�r�t
|j�|jjjk�r�td
��dg||_t
|j�t
|j�k�r�td��dS)Nz&Must pass right_on or right_index=Truez$Must pass left_on or left_index=Truerz>No common columns to perform merge on. Merge options: left_on=z, right_on=z
, left_index=z, right_index=r1)r4zData columns not unique: zSCan only pass argument "on" OR "left_on" and "right_on", not a combination of both.zDlen(left_on) must equal the number of levels in the index of "right"zDlen(right_on) must equal the number of levels in the index of "left"z%len(right_on) must equal len(left_on))r;r<r=r5r6rrBrMrC�intersectionrWr��	is_unique�reprrXr�rj)r�Z	left_colsZ
right_colsZcommon_cols�nrErErFr��sP 




&

z'_MergeOperation._validate_specification)r?cCs�|jr|jjj}ntj|j�j}|jr2|jjj}ntj|j	�j}|dkrz|r^|r^t
d��q�|slt
d��q�|s�t
d��n>|dkr�|s�t
d��n(|dkr�|s�t
d��n|dkr�ntd��dS)N�
one_to_one�1:1zQMerge keys are not unique in either left or right dataset; not a one-to-one mergezAMerge keys are not unique in left dataset; not a one-to-one mergezBMerge keys are not unique in right dataset; not a one-to-one merge�one_to_many�1:mzBMerge keys are not unique in left dataset; not a one-to-many merge�many_to_one�m:1zCMerge keys are not unique in right dataset; not a many-to-one merge�many_to_many�m:mz!Not a valid argument for validate)r�r�)r�r�)r�r�)r�r�)r5ryrXr�r'�from_arraysr�r6rzr�rrj)r�r?Zleft_uniqueZright_uniquerErErFr��s6z_MergeOperation._validate�r2r3)r1NNNrvFFTr�TFN)r/r.)r/r.)r.r.)rB)�__name__�
__module__�__qualname__�__doc__r�rr~r�r�rArr�r�r�r�r�r�r&r�r�r�r�r�rErErErFr@8s8&I!#U9t;r@)r7r4cs�t��t��kstd������fdd�tt���D�}t|�}dd�|D�\}}}	t|||	��\}
}t|
|��d�\}
}}tj|�}�dkr��|d<tjtj	t
tjd	��}
|
|
||f|�S)
aI

    Parameters
    ----------
    left_keys: ndarray, Index, Series
    right_keys: ndarray, Index, Series
    sort: bool, default False
    how: string {'inner', 'outer', 'left', 'right'}, default 'inner'

    Returns
    -------
    tuple of (left_indexer, right_indexer)
        indexers into the left_keys, right_keys

    z/left_key and right_keys must be the same lengthc3s&|]}t�|�|��d�VqdS))r7r4N)�_factorize_keys)rIr�)r4r�r�r7rErF�	<genexpr>2sz%_get_join_indexers.<locals>.<genexpr>cSsg|]}t|��qSrE)rP)rIrfrErErFrL6sz&_get_join_indexers.<locals>.<listcomp>)r7r4rBr7)r1rBrCrc)rW�AssertionErrorrVr��_get_join_keysr�r8�libjoinZ
inner_join�left_outer_join�_right_outer_joinZfull_outer_join)r�r�r7r4�kwargs�mapped�zipped�llab�rlab�shape�lkey�rkey�countZ	join_funcrE)r4r�r�r7rFr�s$

r�cCs�td�dd�}||�}|j}|j}|j}	|dkr:t|j�}|dkrLt|j�}xv|D]n}
|
|jkrj|}|}n|}|}|jj|
�}
|j|
}|j|
}tj||dd�}||g}||g}|	|
g}	qRW|||	fS)a�
    *this is an internal non-public method*

    Returns the levels, labels and names of a multi-index to multi-index join.
    Depending on the type of join, this method restores the appropriate
    dropped levels of the joined multi-index.
    The method relies on lidx, rindexer which hold the index positions of
    left and right, where a join was feasible

    Parameters
    ----------
    left : MultiIndex
        left index
    right : MultiIndex
        right index
    dropped_level_names : str array
        list of non-common level names
    join_index : MultiIndex
        the index of the join between the
        common levels of left and right
    lindexer : intp array
        left indexer
    rindexer : intp array
        right indexer

    Returns
    -------
    levels : list of Index
        levels of combined multiindexes
    labels : intp array
        labels of combined multiindexes
    names : str array
        names of combined multiindexes

    )r:cSs(t|t�r|Stj|jg|jgd�SdS)N)r�)rOr'r�r�r�)rXrErErF�_convert_to_multiindexzs
zA_restore_dropped_levels_multijoin.<locals>._convert_to_multiindexNrv)r�r�)	r'r�r�r�rV�sizerXr�Ztake_nd)rBrCZdropped_level_namesr�ZlindexerZrindexerrZjoin_levelsZ
join_codesr�Zdropped_level_name�idxr�Zname_idxZrestore_levelsr�Z
restore_codesrErErF�!_restore_dropped_levels_multijoinNs.,








rc
@s.eZdZdZdeeeed	�d
d�Zdd
�ZdS)reZ
ordered_mergeNFrvr2r3Trc)r5r6r8r4c

Cs,||_tj|||||||||||	dd�dS)NT)	r;r<r5r6r=r|r4r>r7)rdr@r�)
r�rBrCr;r<r=r5r6r|r>r8rdr4rErErFr��sz_OrderedMerge.__init__c
Cs�|j�\}}}t|jj|jj|j�\}}|jdkrHtj|�}tj|�}n|}|}|dk	r`d|ini}|dk	rtd|ini}	t	|jj
|f|jj
|	fg|j|�|gd|jd�}
|jj
}||
�j||jd�}|j|||�|S)NZffillrvr)r�r�r8)r�)r�r�rBr�rCr>rdr�Z
ffill_indexerr,r�rYr8r�r�r�r�)
r�r�r�r�r�r�Zleft_join_indexerZright_join_indexerr�r�r�r�rarErErFrA�s&


z_OrderedMerge.get_result�r2r3)
NNNFFrvrTNrc)r�r�r�r�r�r~r�rArErErErFre�sre)rqcCsd|��}tt|d�S)N�
asof_join_)�getattrr�)rqr�rErErF�_asof_function�s
rcCsd|�d�}tt|d�S)NrZ
_on_X_by_Y)rr�)rqr�rErErF�_asof_by_function�sr)�int64_t�doubler�cCs t|�rdSt|�rdSdSdS)z4 Upcast a dtype to 'int64_t', 'double', or 'object' rrr�N)rr)r�rErErF�_get_cython_type_upcasts
rcsNeZdZdZdeeeeeed
�dd�Z�fd
d�Z�fdd�Zdd�Z	�Z
S)rtZ
asof_mergeNFrvr2r3Trrro)r5r6r8r4rprqcCsJ||_|	|_|
|_||_||_||_tj||||||||||||d�dS)N)	r;r<r=r5r6r|r4r>rd)rZrkrlrsrprqrer�)r�rBrCr;r<r=r5r6rZrkrlr|r>r8rdr4rsrprqrErErFr�
s&z_AsOfMerge.__init__cs�t�j�t|j�dkr(|jr(td��t|j�dkrF|jrFtd��|jrbt|j	j
t�rbtd��|jr~t|jj
t�r~td��|j
dk	r�|jdk	s�|jdk	r�td��|j
|_|_|jdkr�|jdk	r�td��|jdk	r�|jdkr�td��|jdk	�rdt|j��s|jg|_t|j��s"|jg|_t|j�t|j�k�r@td	��|jt|j�|_|jt|j�|_|jdk�r�td
|j����dS)Nrvzcan only asof on a key for leftz can only asof on a key for rightzleft can only have one indexzright can only have one indexz(Can only pass by OR left_by and right_byzmissing left_byzmissing right_byz(left_by and right_by must be same lengthro�forward�nearestzdirection invalid: )rorr)�superr�rWr<r5rr=r6rOrBrXr'rCrZrkrlrrPrq)r�)�	__class__rErFr�9s8



z"_AsOfMerge._validate_specificationc	s�t�j�\}}}x�tt||��D]�\}\}}t|j|j�s t|j�rtt|j�rtd|�dt|j��dt|j��d�}n$d|�dt|j��dt|j��d�}t|��q W|j	dk	�r�|j
r�|jj}n|d
}d|j	�dt|j���}t
|��rt|j	tj��st|��|j	td	�k�r�td
��npt|��rPt|j	��s:t|��|j	d	k�r�td
��n<t|��r�t|j	��snt|��|j	d	k�r�td
��ntd��t|j��s�d|j��}t|��|||fS)Nzincompatible merge keys [z] z and z), both sides category, but not equal onesz, must be the same typervzincompatible tolerance z, must be compat with type rztolerance must be positivez'key must be integer, timestamp or floatz,allow_exact_matches must be boolean, passed r�)rr�r�r�rr�rr�rrsr5rBrXr rO�datetime�	timedeltarrrrrrrp)	r�r�r�r�r�r�r�r��lt)rrErFr�es@
&$






z_AsOfMerge._get_merge_keyscCs�tjd�dd�}|jr |jjjn|jd
}|jr:|jjjn|j	d}|j
}t|�js�d}t
|�j�rvtd|�d���nt|�d���t|�js�d	}t
|�j�r�td|�d���nt|�d���t|�r�|jd
�}|jd
�}|dk	r�t|�}|j}|jdk	�r�|j�r|j�r|j}|j	}n|jdd�}|j	dd�}t|�dk�rV|d}|d}n||�}||�}t|j�}t|}	|	|�}|	|�}t|j�}
|
|||||j|�St|j�}
|
|||j|�SdS)z return the join indexers )r:cSsTdd�|D�}ttjdt|���}dd�|D�}tt||��}tjtt|��|�S)z6 unlike np.transpose, this returns an array of tuples cSs$g|]}t|�s|n
t|�j��qSrE)rr*Z_values_for_argsort)rIrfrErErFrL�sz?_AsOfMerge._get_join_indexers.<locals>.flip.<locals>.<listcomp>NcSsg|]
}|j�qSrE)r�)rIrfrErErFrL�s)rPr��ascii_lowercaserWr�r�r�)�xsr�ZdtypesZlabeled_dtypesrErErF�flip�sz+_AsOfMerge._get_join_indexers.<locals>.fliprvrBz"Merge keys contain null values on z sidez keys must be sortedrC�i8Nrr�r�r�r�)r��ndarrayr5rBrXr�r�r6rCr�rsr&Zis_monotonicr#r�rjr �viewr�valuerkrWrr��
_type_castersrrqrpr)r�r Zleft_valuesZright_valuesrsZsideZleft_by_valuesZright_by_valuesZby_typeZby_type_caster�funcrErErFr��sZ







z_AsOfMerge._get_join_indexers�r2r3)NNNFFNNNrvr'TNrrNTro)r�r�r�r�r�r~r�r�r�r��
__classcell__rErE)rrFrt
s(,Hrt)rXr7cs*���fdd�t�j�D�}t|�}dd�|D�\}}}�rRtttj|�j��}ndd�}tt|�j��}x�tt���D]v}	�j|	d
k}
|
j	�rx�|	||	||	dk}|j
dks�|d|dkr�||	d7<||	d||	|
<qxWt|||��\}}
t||
�d	�\}}
}t
j||
|�d	�S)Nc3s&|]}t�j|�|�d�VqdS))r7N)r�r�)rIr�)rX�	join_keysr7rErFr�	sz*_get_multiindex_indexer.<locals>.<genexpr>cSsg|]}t|��qSrE)rP)rIrfrErErFrL
sz+_get_multiindex_indexer.<locals>.<listcomp>cSs|jdddd�S)Nr!FT)�subokr8)r�)�arErErFrisz)_get_multiindex_indexer.<locals>.<lambda>rvr)r7r�)rVr�r�rP�mapr�rSr�rWr�rr�r�r�r)r)rXr7rrZrcodesZlcodesrZi8copyr�r�r+rr	r
rE)rXr)r7rF�_get_multiindex_indexers$r-)r7cCs:t|||d�\}}}tjt|�t|�||d�\}}||fS)N)r7)r�r�rr)Zjoin_keyrXr7r�r�r
r�r�rErErF�_get_single_indexer)sr.)r�r�r7cCs�t|�dkr@t|t�o"t|�|jks,td��t|||d�\}}n|d}t|||d�\}}|snt|�t|�kr�|j|�}|||fS|d|fS)Nrvz�If more than one join key is given then 'right_ax' must be a MultiIndex and the number of join keys must be the number of levels in right_ax)r7r)rWrOr'r�r�r-r.rS)r�r�r)r7r�r�Zjkeyr�rErErFr�3s

r�cCstj|||�\}}||fS)N)r�r)rfrgZ
max_groupsr�r�rErErFrOsr)r�r�r7r4r:cCsLt|dd�}t|dd�}t|j�rFt|j�rF|j�\}}|j�\}}n�t|�r�t|�r�t||�r�t|t�snt�t|t�s|t�|j	j
|j	�r�|j}nt|j|j	|j	�}t
|j�}t
|�}n0t|j�r�t|j|j�r�|j�\}}|j�\}}t|�o�t|��rtj}t
tj|��}t
tj|��}nbt|j��rjt|j|j��rjtj}t
tj|tjd��}t
tj|tjd��}ntj}t|�}t|�}|tt|�t|���}|j|�}|j|�}|j�}	|�r�|jj�}
t|
||�\}}|dk}|j�}|dk}
|
j�}|�s�|�r.|�rtj |||	�|�r&tj ||
|	�|	d7}	|dk�rB|||	fS|||	fS)u#
    Encode left and right keys as enumerated types.

    This is used to get the join indexers to be used when merging DataFrames.

    Parameters
    ----------
    lk : array-like
        Left key.
    rk : array-like
        Right key.
    sort : bool, defaults to True
        If True, the encoding is done such that the unique elements in the
        keys are sorted.
    how : {‘left’, ‘right’, ‘outer’, ‘inner’}, default ‘inner’
        Type of merge.

    Returns
    -------
    array
        Left (resp. right if called with `key='right'`) labels, as enumerated type.
    array
        Right (resp. left if called with `key='right'`) labels, as enumerated type.
    int
        Number of unique elements in union of left and right labels.

    See Also
    --------
    merge : Merge DataFrame or named Series objects
        with a database-style join.
    algorithms.factorize : Encode the object as an enumerated type
        or categorical variable.

    Examples
    --------
    >>> lk = np.array(["a", "c", "b"])
    >>> rk = np.array(["a", "c"])

    Here, the unique values are `'a', 'b', 'c'`. With the default
    `sort=True`, the encoding will be `{0: 'a', 1: 'b', 2: 'c'}`:

    >>> pd.core.reshape.merge._factorize_keys(lk, rk)
    (array([0, 2, 1]), array([0, 2]), 3)

    With the `sort=False`, the encoding will correspond to the order
    in which the unique elements first appear: `{0: 'a', 1: 'c', 2: 'b'}`:

    >>> pd.core.reshape.merge._factorize_keys(lk, rk, sort=False)
    (array([0, 1, 2]), array([0, 1]), 3)
    T)Z
extract_numpy)r�rvrCr�r�)!r*rr�Z_values_for_factorizerrrOr%r�r��equalsr�r)rrr�libhashtableZInt64Factorizerr�Zasarrayr Zint64Z
Factorizerr�maxrWZ	factorizeZ	get_count�uniquesZto_array�_sort_labelsr�Zputmask)r�r�r7r4rm�klassZrizerrrr
r2ZlmaskZlanyZrmaskZranyrErErFr�Ts^6






r�)r2c	CsTt|�}tj||g�}tj||dd�\}}t|�}|d|�||d�}}||fS)Nrv)Zna_sentinelr�)rWr�Zconcatenater�Z	safe_sortr)	r2rBrCZllengthr�rmZ
new_labelsZnew_leftZ	new_rightrErErFr3�sr3cs0�fdd�}tt|tt��dd���}tj�d|�dd�}||djdddd�}||djdddd�}xVtd|�D]H}	tjd	d
��|�|	}WdQRX|||	|7}|||	|7}qxW|t��kr�||fSt|||d�\}}}
|g||d�}|g||d�}|
g�|d��t	||�|�S)
Ncst�d|��S)N)r-)r�)rrErFri�sz _get_join_keys.<locals>.<lambda>rrvr!)r�F)r*r8�ignore)�divide)r7r�)
�next�filterrVrWr��prodr�Zerrstater�r�)rrrr7�predZnlevZstriderr	r�r
rE)rrFr��s"r�)r:cCs$t|t�st|t�rdS||kS)NT)rOr~)r�r�rErErFr��sr�cCs|dk	otj|�S)N)r}Zany_not_none)rfrErErFr�sr�)�objr:cCsNt|t�r|St|t�r6|jdkr,td��qJ|j�Sntdt|��d���dS)Nz$Cannot merge a Series without a namez.Can only merge Series or DataFrame objects, a z was passed)rOr!r"r�rjZto_frame�	TypeErrorr�)r;rErErFrx	s




rx)rBrCr>cs�t|dd�s(tjdt|��d�tdd�|j|��t��dkrF||fS|\}}|rh|rhtd������fd	d
�}t||d�}t||d�}|j	|�|j	|�fS)z�
    Suffixes type validation.

    If two indices overlap, add suffixes to overlapping entries.

    If corresponding suffix is empty, the entry is simply converted to string.

    F)Z
allow_setszPassing 'suffixes' as a z�, is not supported and may give unexpected results. Provide 'suffixes' as a tuple instead. In the future a 'TypeError' will be raised.�)�
stacklevelrz)columns overlap but no suffix specified: cs |�kr|dk	r|�|��S|S)a?
        Rename the left and right indices.

        If there is overlap, and suffix is not None, add
        suffix, otherwise, leave it as-is.

        Parameters
        ----------
        x : original column name
        suffix : str or None

        Returns
        -------
        x : renamed column name
        NrE)rf�suffix)�	to_renamerErF�renamer2sz+_items_overlap_with_suffix.<locals>.renamer)r?)
rr�r�r��
FutureWarningr�rWrjrZ_transform_index)rBrCr>ZlsuffixZrsuffixrAZlrenamerZrrenamerrE)r@rFr�s	
r��r2r3)r1NNNFFFrCTFN�r2r3)NNNNNNrDrc�r2r3)NNNFFNNNrENTro)Fr1)F)F)Tr1)nr�r8r�	functoolsrr�Ztypingrrrrr�Znumpyr�Zpandas._libsrrr0r	Zpandas._libs.joinZ_libsr�r�Zpandas._typingr
rZ
pandas.errorsrZpandas.util._decoratorsr
rZpandas.core.dtypes.commonrrrrrrrrrrrrrrrrrr Zpandas.core.dtypes.genericr!r"Zpandas.core.dtypes.missingr#r$Zpandasr%r&r'Zpandas.corer(Zpandas.core.algorithms�coreZ
algorithmsr�Zpandas.core.arrays.categoricalr)Zpandas.core.common�commonr}Zpandas.core.constructionr*Zpandas.core.framer+Zpandas.core.internalsr,Zpandas.core.sortingr-r.r/r~r�rGrbrnrur@r�rrerrr%rrtr-r.r�rr��intr�r"r3r�r�r�rxr�rErErErF�<module>s�P

A}
h4\C
|$