HOME


sh-3ll 1.0
DIR:/usr/local/lib64/python3.6/site-packages/numpy/lib/__pycache__/
Upload File :
Current File : //usr/local/lib64/python3.6/site-packages/numpy/lib/__pycache__/recfunctions.cpython-36.pyc
3

���h���@s�dZddlZddlZddljZddlmZmZddlmZddl	m
Z
ddlmZddl
mZddlmZejjjZd	d
ddd
ddddddddddddddddgZdd�Zee�d d��Zd!d"�Zd#d�Zd$d�Zd%d�Zdpd'd(�Zdqd)d*�Zdrd+d�Zd,d-�Zd.d/�Zdsd1d2�Zdtd3d4�Z dud5d6�Z!dvd7d8�Z"ee"�dxd:d��Z#dyd;d<�Z$ee$�dzd=d��Z%d{d>d?�Z&d@dA�Z'ee'�dBd��Z(dCdD�Z)ee)�dEd��Z*d|dFdG�Z+ee+�d~dHd	��Z,ddIdJ�Z-ee-�d�dKd��Z.d�dLdM�Z/ee/�d�dNd��Z0d�dOdP�Z1d�dQdR�Z2ee2�d�dTd��Z3d�dUdV�Z4ee4�d�dWd��Z5dXdY�Z6ee6�dZd
��Z7d�d[d\�Z8ee8�d�d]d��Z9d^d_�Z:ee:�d`d��Z;d�dadb�Z<ee<�d�dcd��Z=d�ddde�Z>ee>�d�dfd
��Z?d�dgdh�Z@ee@�d�dld��ZAd�dmdn�ZBeeB�d�dod��ZCdS)�z�
Collection of utilities to manipulate structured arrays.

Most of these functions were initially implemented by John Hunter for
matplotlib.  They have been rewritten and extended for convenience.

�N)�ndarray�recarray)�MaskedArray)�
MaskedRecords)�array_function_dispatch)�_is_string_like)�suppress_warnings�
append_fields�apply_along_fields�assign_fields_by_name�drop_fields�find_duplicates�
flatten_descr�get_fieldstructure�	get_names�get_names_flat�join_by�merge_arrays�rec_append_fields�rec_drop_fields�rec_join�recursive_fill_fields�
rename_fields�
repack_fields�require_fields�stack_arrays�structured_to_unstructured�unstructured_to_structuredcCs||fS)N�)�input�outputrr�7/tmp/pip-build-5_djhm0z/numpy/numpy/lib/recfunctions.py�!_recursive_fill_fields_dispatcher sr"cCsn|j}xb|jD]X}y||}Wntk
r4wYnX|jjdk	rRt|||�q|||dt|��<qW|S)aj
    Fills fields from output with fields from input,
    with support for nested structures.

    Parameters
    ----------
    input : ndarray
        Input array.
    output : ndarray
        Output array.

    Notes
    -----
    * `output` should be at least the same size as `input`

    Examples
    --------
    >>> from numpy.lib import recfunctions as rfn
    >>> a = np.array([(1, 10.), (2, 20.)], dtype=[('A', np.int64), ('B', np.float64)])
    >>> b = np.zeros((3,), dtype=a.dtype)
    >>> rfn.recursive_fill_fields(a, b)
    array([(1, 10.), (2, 20.), (0,  0.)], dtype=[('A', '<i8'), ('B', '<f8')])

    N)�dtype�names�
ValueErrorr�len)rr �newdtype�field�currentrrr!r$scs:�jdkrd�fgS�fdd��jD�}dd�|D�SdS)aR
    Produce a list of name/dtype pairs corresponding to the dtype fields

    Similar to dtype.descr, but the second item of each tuple is a dtype, not a
    string. As a result, this handles subarray dtypes

    Can be passed to the dtype constructor to reconstruct the dtype, noting that
    this (deliberately) discards field offsets.

    Examples
    --------
    >>> dt = np.dtype([(('a', 'A'), np.int64), ('b', np.double, 3)])
    >>> dt.descr
    [(('a', 'A'), '<i8'), ('b', '<f8', (3,))]
    >>> _get_fieldspec(dt)
    [(('a', 'A'), dtype('int64')), ('b', dtype(('<f8', (3,))))]

    N�c3s|]}|�j|fVqdS)N)�fields)�.0�name)r#rr!�	<genexpr>bsz!_get_fieldspec.<locals>.<genexpr>cSs4g|],\}}t|�dkr|n
|d|f|df�qS)�r)r&)r,r-�frrr!�
<listcomp>esz"_get_fieldspec.<locals>.<listcomp>)r$)r#r+r)r#r!�_get_fieldspecKs


r2cCsTg}|j}x@|D]8}||}|jdk	r>|j|tt|��f�q|j|�qWt|�S)a�
    Returns the field names of the input datatype as a tuple.

    Parameters
    ----------
    adtype : dtype
        Input datatype

    Examples
    --------
    >>> from numpy.lib import recfunctions as rfn
    >>> rfn.get_names(np.empty((1,), dtype=int))
    Traceback (most recent call last):
        ...
    AttributeError: 'numpy.ndarray' object has no attribute 'names'

    >>> rfn.get_names(np.empty((1,), dtype=[('A',int), ('B', float)]))
    Traceback (most recent call last):
        ...
    AttributeError: 'numpy.ndarray' object has no attribute 'names'
    >>> adtype = np.dtype([('a', int), ('b', [('ba', int), ('bb', int)])])
    >>> rfn.get_names(adtype)
    ('a', ('b', ('ba', 'bb')))
    N)r$�append�tupler)�adtype�	listnamesr$r-r)rrr!rjs

cCsJg}|j}x6|D].}|j|�||}|jdk	r|jt|��qWt|�S)a	
    Returns the field names of the input datatype as a tuple. Nested structure
    are flattened beforehand.

    Parameters
    ----------
    adtype : dtype
        Input datatype

    Examples
    --------
    >>> from numpy.lib import recfunctions as rfn
    >>> rfn.get_names_flat(np.empty((1,), dtype=int)) is None
    Traceback (most recent call last):
        ...
    AttributeError: 'numpy.ndarray' object has no attribute 'names'
    >>> rfn.get_names_flat(np.empty((1,), dtype=[('A',int), ('B', float)]))
    Traceback (most recent call last):
        ...
    AttributeError: 'numpy.ndarray' object has no attribute 'names'
    >>> adtype = np.dtype([('a', int), ('b', [('ba', int), ('bb', int)])])
    >>> rfn.get_names_flat(adtype)
    ('a', 'b', 'ba', 'bb')
    N)r$r3�extendrr4)r5r6r$r-r)rrr!r�s


cCsl|j}|dkrd|ffSg}xB|D]:}|j|\}}|jdk	rN|jt|��q"|j||f�q"Wt|�SdS)aD
    Flatten a structured data-type description.

    Examples
    --------
    >>> from numpy.lib import recfunctions as rfn
    >>> ndtype = np.dtype([('a', '<i4'), ('b', [('ba', '<f8'), ('bb', '<i4')])])
    >>> rfn.flatten_descr(ndtype)
    (('a', dtype('int32')), ('ba', dtype('float64')), ('bb', dtype('int32')))

    Nr*)r$r+r7rr3r4)�ndtyper$�descrr(�typ�_rrr!r�s


FcCs|g}|r(xh|D]}|jt|j��qWnJxH|D]@}|j}|jdk	r`t|j�dkr`|jt|��q.|jd|f�q.Wtj|�S)N�r*)r7rr#r$r&r2r3�np)�	seqarrays�flattenr'�ar)rrr!�
_zip_dtype�s

rAcCst||d�jS)z�
    Combine the dtype description of a series of arrays.

    Parameters
    ----------
    seqarrays : sequence of arrays
        Sequence of arrays
    flatten : {boolean}, optional
        Whether to collapse nested descriptions.
    )r?)rAr9)r>r?rrr!�
_zip_descr�srBcCs�|dkri}|j}x�|D]�}||}|jdk	rZ|r>|g||<ng||<|jt|||��qdd�|j|g�pngD�}|r�|j|�n
|r�|g}|p�g||<qW|S)ab
    Returns a dictionary with fields indexing lists of their parent fields.

    This function is used to simplify access to fields nested in other fields.

    Parameters
    ----------
    adtype : np.dtype
        Input datatype
    lastname : optional
        Last processed field name (used internally during recursion).
    parents : dictionary
        Dictionary of parent fields (used interbally during recursion).

    Examples
    --------
    >>> from numpy.lib import recfunctions as rfn
    >>> ndtype =  np.dtype([('A', int),
    ...                     ('B', [('BA', int),
    ...                            ('BB', [('BBA', int), ('BBB', int)])])])
    >>> rfn.get_fieldstructure(ndtype)
    ... # XXX: possible regression, order of BBA and BBB is swapped
    {'A': [], 'B': [], 'BA': ['B'], 'BB': ['B'], 'BBA': ['B', 'BB'], 'BBB': ['B', 'BB']}

    NcSsg|]}|�qSrr)r,r;rrr!r1sz&get_fieldstructure.<locals>.<listcomp>)r$�updater�getr3)r5�lastname�parentsr$r-r)Z
lastparentrrr!r�s"

ccs8x2|D]*}t|tj�r*tt|��EdHq|VqWdS)zu
    Returns an iterator of concatenated fields from a sequence of arrays,
    collapsing any nested structure.

    N)�
isinstancer=�void�_izip_fields_flatr4)�iterable�elementrrr!rIs
rIccsjxd|D]\}t|d�r0t|t�r0t|�EdHqt|tj�r\tt|��dkr\t|�EdHq|VqWdS)zP
    Returns an iterator of concatenated fields from a sequence of arrays.

    �__iter__Nr<)�hasattrrG�str�_izip_fieldsr=rHr&r4)rJrKrrr!rO%s

rOTccs:|r
t}nt}x&tj|d|i�D]}t||��Vq WdS)a*
    Returns an iterator of concatenated items from a sequence of arrays.

    Parameters
    ----------
    seqarrays : sequence of arrays
        Sequence of arrays.
    fill_value : {None, integer}
        Value used to pad shorter iterables.
    flatten : {True, False},
        Whether to
    �	fillvalueN)rIrO�	itertools�zip_longestr4)r>�
fill_valuer?Zzipfunc�tuprrr!�
_izip_records5s
rUcCs>t|t�sd}|r"|r:|jt�}ntj|�}|r:|jt�}|S)z�
    Private function: return a recarray, a ndarray, a MaskedArray
    or a MaskedRecords depending on the input parameters
    F)rGr�viewr�ma�filledr)r �usemask�
asrecarrayrrr!�_fix_outputMs


r[cCs\|jj}|j|j|j}}}x8|p&ij�D](\}}||kr,|||<|||||<q,W|S)zp
    Update the fill_value and masked data of `output`
    from the default given in a dictionary defaults.
    )r#r$�data�maskrS�items)r �defaultsr$r\r]rS�k�vrrr!�
_fix_defaults^srbcCs|S)Nr)r>rSr?rYrZrrr!�_merge_arrays_dispatcherlsrcr<cCs�t|�dkrtj|d�}t|ttjf�r�|j}|jdkrJtjd|fg�}|sbt|fdd�|kr�|j	�}|r~|rxt
}q�t}n|r�t}nt}|j
||d�S|f}ndd	�|D�}td
d�|D��}t|�}t||d�}	g}
g}|�rx�t||�D]�\}}
||
}|j	�j�}tj|�j	�}|�r�t||j�}t|ttjf��r�t|j�dk�r`|j�d}d}n"tj||jdd�}tjd|jd
�}nd}d}|
jtj||g|��|jtj||g|��q�Wtt|
|d��}tjtj||	|d�tt||d��d�}|�r�|j
t
�}n�x�t||�D]�\}}
||
}|j	�j�}|�r�t||j�}t|ttjf��r�t|j�dk�rr|j�d}ntj||jdd�}nd}|
jtj||g|���qWtjtt|
|d��|	|d�}|�r�|j
t�}|S)aR
    Merge arrays field by field.

    Parameters
    ----------
    seqarrays : sequence of ndarrays
        Sequence of arrays
    fill_value : {float}, optional
        Filling value used to pad missing data on the shorter arrays.
    flatten : {False, True}, optional
        Whether to collapse nested fields.
    usemask : {False, True}, optional
        Whether to return a masked array or not.
    asrecarray : {False, True}, optional
        Whether to return a recarray (MaskedRecords) or not.

    Examples
    --------
    >>> from numpy.lib import recfunctions as rfn
    >>> rfn.merge_arrays((np.array([1, 2]), np.array([10., 20., 30.])))
    array([( 1, 10.), ( 2, 20.), (-1, 30.)],
          dtype=[('f0', '<i8'), ('f1', '<f8')])

    >>> rfn.merge_arrays((np.array([1, 2], dtype=np.int64),
    ...         np.array([10., 20., 30.])), usemask=False)
     array([(1, 10.0), (2, 20.0), (-1, 30.0)],
             dtype=[('f0', '<i8'), ('f1', '<f8')])
    >>> rfn.merge_arrays((np.array([1, 2]).view([('a', np.int64)]),
    ...               np.array([10., 20., 30.])),
    ...              usemask=False, asrecarray=True)
    rec.array([( 1, 10.), ( 2, 20.), (-1, 30.)],
              dtype=[('a', '<i8'), ('f1', '<f8')])

    Notes
    -----
    * Without a mask, the missing value will be filled with something,
      depending on what its corresponding type:

      * ``-1``      for integers
      * ``-1.0``    for floating point numbers
      * ``'-'``     for characters
      * ``'-1'``    for strings
      * ``True``    for boolean values
    * XXX: I just obtained these values empirically
    r<rNr*T)r?)r#�typecSsg|]}tj|��qSr)r=�
asanyarray)r,�_mrrr!r1�sz merge_arrays.<locals>.<listcomp>css|]}|jVqdS)N)�size)r,r@rrr!r.�szmerge_arrays.<locals>.<genexpr>)r#Zndmin)r#)r#�count)r])r<)r&r=rerGrrHr#r$rA�ravelrrrrVr4�max�zipZ	__array__rWZgetmaskarray�_check_fill_value�item�arrayZonesr3rQ�chainrUZfromiter�list)r>rSr?rYrZZseqdtypeZseqtype�sizesZ	maxlengthr'ZseqdataZseqmaskr@�nZ	nbmissingr\r]ZfvalZfmskr rrr!rqsz1


cCs|fS)Nr)�base�
drop_namesrYrZrrr!�_drop_fields_dispatcher�srucsXt|�r|g}nt|�}�fdd���|j|�}tj|j|d�}t||�}t|||d�S)a
    Return a new array with fields in `drop_names` dropped.

    Nested fields are supported.

    ..versionchanged: 1.18.0
        `drop_fields` returns an array with 0 fields if all fields are dropped,
        rather than returning ``None`` as it did previously.

    Parameters
    ----------
    base : array
        Input array
    drop_names : string or sequence
        String or sequence of strings corresponding to the names of the
        fields to drop.
    usemask : {False, True}, optional
        Whether to return a masked array or not.
    asrecarray : string or sequence, optional
        Whether to return a recarray or a mrecarray (`asrecarray=True`) or
        a plain ndarray or masked array with flexible dtype. The default
        is False.

    Examples
    --------
    >>> from numpy.lib import recfunctions as rfn
    >>> a = np.array([(1, (2, 3.0)), (4, (5, 6.0))],
    ...   dtype=[('a', np.int64), ('b', [('ba', np.double), ('bb', np.int64)])])
    >>> rfn.drop_fields(a, 'a')
    array([((2., 3),), ((5., 6),)],
          dtype=[('b', [('ba', '<f8'), ('bb', '<i8')])])
    >>> rfn.drop_fields(a, 'ba')
    array([(1, (3,)), (4, (6,))], dtype=[('a', '<i8'), ('b', [('bb', '<i8')])])
    >>> rfn.drop_fields(a, ['ba', 'bb'])
    array([(1,), (4,)], dtype=[('a', '<i8')])
    csd|j}g}xT|D]L}||}||kr&q|jdk	rN�||�}|r\|j||f�q|j||f�qW|S)N)r$r3)r8rtr$r'r-r)r9)�_drop_descrrr!rv(s


z drop_fields.<locals>._drop_descr)r#)rYrZ)r�setr#r=�empty�shaperr[)rsrtrYrZr'r r)rvr!r�s&
cs:�fdd�|D�}tj�j|d�}t�|�}t|||d�S)a�
    Return a new array keeping only the fields in `keep_names`,
    and preserving the order of those fields.

    Parameters
    ----------
    base : array
        Input array
    keep_names : string or sequence
        String or sequence of strings corresponding to the names of the
        fields to keep. Order of the names will be preserved.
    usemask : {False, True}, optional
        Whether to return a masked array or not.
    asrecarray : string or sequence, optional
        Whether to return a recarray or a mrecarray (`asrecarray=True`) or
        a plain ndarray or masked array with flexible dtype. The default
        is False.
    csg|]}|�j|f�qSr)r#)r,rr)rsrr!r1Qsz _keep_fields.<locals>.<listcomp>)r#)rYrZ)r=rxryrr[)rsZ
keep_namesrYrZr'r r)rsr!�_keep_fields>s
rzcCs|fS)Nr)rsrtrrr!�_rec_drop_fields_dispatcherWsr{cCst||ddd�S)zK
    Returns a new numpy.recarray with fields in `drop_names` dropped.
    FT)rYrZ)r)rsrtrrr!r[scCs|fS)Nr)rs�
namemapperrrr!�_rename_fields_dispatchercsr}cs"�fdd���|j|�}|j|�S)a�
    Rename the fields from a flexible-datatype ndarray or recarray.

    Nested fields are supported.

    Parameters
    ----------
    base : ndarray
        Input array whose fields must be modified.
    namemapper : dictionary
        Dictionary mapping old field names to their new version.

    Examples
    --------
    >>> from numpy.lib import recfunctions as rfn
    >>> a = np.array([(1, (2, [3.0, 30.])), (4, (5, [6.0, 60.]))],
    ...   dtype=[('a', int),('b', [('ba', float), ('bb', (float, 2))])])
    >>> rfn.rename_fields(a, {'a':'A', 'bb':'BB'})
    array([(1, (2., [ 3., 30.])), (4, (5., [ 6., 60.]))],
          dtype=[('A', '<i8'), ('b', [('ba', '<f8'), ('BB', '<f8', (2,))])])

    csZg}xP|jD]F}|j||�}||}|jdk	rD|j|�||�f�q|j||f�qW|S)N)r$rDr3)r8r|r'r-Znewnamer))�_recursive_rename_fieldsrr!r~s
z/rename_fields.<locals>._recursive_rename_fields)r#rV)rsr|r'r)r~r!rgsccs|V|EdHdS)Nr)rsr$r\�dtypesrSrYrZrrr!�_append_fields_dispatcher�sr�c	CsPt|ttf�r,t|�t|�krBd}t|��nt|t�rB|g}|g}|dkrndd�|D�}dd�t||�D�}n`t|ttf�s�|g}t|�t|�kr�t|�dkr�|t|�}nd}t|��dd�t|||�D�}t|||d	�}t|�dkr�t|d
||d�}n|j�}t	j
tt|�t|��t|j
�t|j
�d�}t||�}t||�}t|||d
�S)a

    Add new fields to an existing array.

    The names of the fields are given with the `names` arguments,
    the corresponding values with the `data` arguments.
    If a single field is appended, `names`, `data` and `dtypes` do not have
    to be lists but just values.

    Parameters
    ----------
    base : array
        Input array to extend.
    names : string, sequence
        String or sequence of strings corresponding to the names
        of the new fields.
    data : array or sequence of arrays
        Array or sequence of arrays storing the fields to add to the base.
    dtypes : sequence of datatypes, optional
        Datatype or sequence of datatypes.
        If None, the datatypes are estimated from the `data`.
    fill_value : {float}, optional
        Filling value used to pad missing data on the shorter arrays.
    usemask : {False, True}, optional
        Whether to return a masked array or not.
    asrecarray : {False, True}, optional
        Whether to return a recarray (MaskedRecords) or not.

    z7The number of arrays does not match the number of namesNcSsg|]}tj|ddd��qS)FT)�copy�subok)r=rn)r,r@rrr!r1�sz!append_fields.<locals>.<listcomp>cSs"g|]\}}|j||jfg��qSr)rVr#)r,r-r@rrr!r1�sr<z5The dtypes argument must be None, a dtype, or a list.cSs0g|](\}}}tj|dd|d�j||fg��qS)FT)r�r�r#)r=rnrV)r,r@rr�drrr!r1�s)rYrST)r?rYrS)r#)rYrZ)rGr4rpr&r%rNrkr�poprW�
masked_allrjr2r#rr[)	rsr$r\rrSrYrZ�msgr rrr!r	�s< 




ccs|V|EdHdS)Nr)rsr$r\rrrr!�_rec_append_fields_dispatcher�sr�cCst||||ddd�S)aM
    Add new fields to an existing array.

    The names of the fields are given with the `names` arguments,
    the corresponding values with the `data` arguments.
    If a single field is appended, `names`, `data` and `dtypes` do not have
    to be lists but just values.

    Parameters
    ----------
    base : array
        Input array to extend.
    names : string, sequence
        String or sequence of strings corresponding to the names
        of the new fields.
    data : array or sequence of arrays
        Array or sequence of arrays storing the fields to add to the base.
    dtypes : sequence of datatypes, optional
        Datatype or sequence of datatypes.
        If None, the datatypes are estimated from the `data`.

    See Also
    --------
    append_fields

    Returns
    -------
    appended_array : np.recarray
    TF)r\rrZrY)r	)rsr$r\rrrr!r�s
cCs|fS)Nr)r@�align�recurserrr!�_repack_fields_dispatchersr�cCs�t|tj�s*t|j||d�}|j|dd�S|jdkr8|Sg}x^|jD]T}|j|}|rjt|d|dd�}n|d}t|�dkr�|d|f}|j||f�qDWtj||d	�}tj|j	|f�S)
a�
    Re-pack the fields of a structured array or dtype in memory.

    The memory layout of structured datatypes allows fields at arbitrary
    byte offsets. This means the fields can be separated by padding bytes,
    their offsets can be non-monotonically increasing, and they can overlap.

    This method removes any overlaps and reorders the fields in memory so they
    have increasing byte offsets, and adds or removes padding bytes depending
    on the `align` option, which behaves like the `align` option to `np.dtype`.

    If `align=False`, this method produces a "packed" memory layout in which
    each field starts at the byte the previous field ended, and any padding
    bytes are removed.

    If `align=True`, this methods produces an "aligned" memory layout in which
    each field's offset is a multiple of its alignment, and the total itemsize
    is a multiple of the largest alignment, by adding padding bytes as needed.

    Parameters
    ----------
    a : ndarray or dtype
       array or dtype for which to repack the fields.
    align : boolean
       If true, use an "aligned" memory layout, otherwise use a "packed" layout.
    recurse : boolean
       If True, also repack nested structures.

    Returns
    -------
    repacked : ndarray or dtype
       Copy of `a` with fields repacked, or `a` itself if no repacking was
       needed.

    Examples
    --------

    >>> from numpy.lib import recfunctions as rfn
    >>> def print_offsets(d):
    ...     print("offsets:", [d.fields[name][1] for name in d.names])
    ...     print("itemsize:", d.itemsize)
    ...
    >>> dt = np.dtype('u1, <i8, <f8', align=True)
    >>> dt
    dtype({'names':['f0','f1','f2'], 'formats':['u1','<i8','<f8'], 'offsets':[0,8,16], 'itemsize':24}, align=True)
    >>> print_offsets(dt)
    offsets: [0, 8, 16]
    itemsize: 24
    >>> packed_dt = rfn.repack_fields(dt)
    >>> packed_dt
    dtype([('f0', 'u1'), ('f1', '<i8'), ('f2', '<f8')])
    >>> print_offsets(packed_dt)
    offsets: [0, 1, 9]
    itemsize: 17

    )r�r�F)r�NrT�r/)r�)
rGr=r#r�astyper$r+r&r3rd)r@r�r��dtZ	fieldinfor-rT�fmtrrr!rs :

c
s�dd�}g}x�|jD]�}|j|}|d|d}}||�\}}|jdkrl|jtj||ff�|||f�qt|||�}	|j�x>t|�D]2��dkr�|j|	�q�|j��fdd�|	D��q�WqW|S)z�
    Returns a flat list of (dtype, count, offset) tuples of all the
    scalar fields in the dtype "dt", including nested fields, in left
    to right order.
    cSs:d}x,|jfkr0x|jD]}||9}qW|j}qW||fS)Nr<)ryrs)r�rhrgrrr!�
count_elembs
z+_get_fields_and_offsets.<locals>.count_elemrr<Ncs$g|]\}}}|||��f�qSrr)r,r��c�o)�irgrr!r1{sz+_get_fields_and_offsets.<locals>.<listcomp>)	r$r+r3r=r#�_get_fields_and_offsets�itemsize�ranger7)
r��offsetr�r+r-r(Zf_dtZf_offsetrrZ	subfieldsr)r�rgr!r�Ys	

""r�cCs|fS)Nr)�arrr#r��castingrrr!�&_structured_to_unstructured_dispatchersr��unsafec
s|jjdkrtd��t|j�}t|�}|dkr@|dkr@td��n|dkrPtd��t|�\}}}dd�t|�D�}	|dkr�tj	dd�|D���n|�tj|	|||jj
d	��}
t��}|jt
d
�|j|
�}WdQRXtj|	�fdd�|D�d��}|j|||d
�}|j�t|�ff�S)ap
    Converts and n-D structured array into an (n+1)-D unstructured array.

    The new array will have a new last dimension equal in size to the
    number of field-elements of the input array. If not supplied, the output
    datatype is determined from the numpy type promotion rules applied to all
    the field datatypes.

    Nested fields, as well as each element of any subarray fields, all count
    as a single field-elements.

    Parameters
    ----------
    arr : ndarray
       Structured array or dtype to convert. Cannot contain object datatype.
    dtype : dtype, optional
       The dtype of the output unstructured array.
    copy : bool, optional
        See copy argument to `ndarray.astype`. If true, always return a copy.
        If false, and `dtype` requirements are satisfied, a view is returned.
    casting : {'no', 'equiv', 'safe', 'same_kind', 'unsafe'}, optional
        See casting argument of `ndarray.astype`. Controls what kind of data
        casting may occur.

    Returns
    -------
    unstructured : ndarray
       Unstructured array with one more dimension.

    Examples
    --------

    >>> from numpy.lib import recfunctions as rfn
    >>> a = np.zeros(4, dtype=[('a', 'i4'), ('b', 'f4,u2'), ('c', 'f4', 2)])
    >>> a
    array([(0, (0., 0), [0., 0.]), (0, (0., 0), [0., 0.]),
           (0, (0., 0), [0., 0.]), (0, (0., 0), [0., 0.])],
          dtype=[('a', '<i4'), ('b', [('f0', '<f4'), ('f1', '<u2')]), ('c', '<f4', (2,))])
    >>> rfn.structured_to_unstructured(a)
    array([[0., 0., 0., 0., 0.],
           [0., 0., 0., 0., 0.],
           [0., 0., 0., 0., 0.],
           [0., 0., 0., 0., 0.]])

    >>> b = np.array([(1, 2, 5), (4, 5, 7), (7, 8 ,11), (10, 11, 12)],
    ...              dtype=[('x', 'i4'), ('y', 'f4'), ('z', 'f8')])
    >>> np.mean(rfn.structured_to_unstructured(b[['x', 'z']]), axis=-1)
    array([ 3. ,  5.5,  9. , 11. ])

    Nzarr must be a structured arrayrz(arr has no fields. Unable to guess dtypez#arr with no fields is not supportedcSsg|]}dj|��qS)zf{})�format)r,rrrrr!r1�sz.structured_to_unstructured.<locals>.<listcomp>cSsg|]
}|j�qSr)rs)r,r�rrr!r1�s)r$�formats�offsetsr�zNumpy has detectedcsg|]}�|jf�qSr)ry)r,r�)�	out_dtyperr!r1�s)r$r�)r�r�)r#r$r%r�r&�NotImplementedErrorrkr�r=Zresult_typer�r�filter�
FutureWarningrVr��sum)
r�r#r�r�r+�n_fields�dts�countsr�r$�flattened_fields�sup�
packed_fieldsr)r�r!r�s04

cCs|fS)Nr)r�r#r$r�r�r�rrr!�&_unstructured_to_structured_dispatcher�sr�csj�jfkrtd���jd}|dkr,td��|dkr�|dkrNdd�t|�D�}tj�fdd�|D�|d	�}t|�}t|�\}	}
}np|dk	r�td
��t|�}t|�dkr�ggg}	}
}nt|�\}	}
}|t	|
�kr�td��|}|r�|j
r�td��d
d�tt|��D�}tj|�fdd�|	D�d��}tj��j|��tj||	||j
d��}
�j|
||d���j|�dS)a�
    Converts and n-D unstructured array into an (n-1)-D structured array.

    The last dimension of the input array is converted into a structure, with
    number of field-elements equal to the size of the last dimension of the
    input array. By default all output fields have the input array's dtype, but
    an output structured dtype with an equal number of fields-elements can be
    supplied instead.

    Nested fields, as well as each element of any subarray fields, all count
    towards the number of field-elements.

    Parameters
    ----------
    arr : ndarray
       Unstructured array or dtype to convert.
    dtype : dtype, optional
       The structured dtype of the output array
    names : list of strings, optional
       If dtype is not supplied, this specifies the field names for the output
       dtype, in order. The field dtypes will be the same as the input array.
    align : boolean, optional
       Whether to create an aligned memory layout.
    copy : bool, optional
        See copy argument to `ndarray.astype`. If true, always return a copy.
        If false, and `dtype` requirements are satisfied, a view is returned.
    casting : {'no', 'equiv', 'safe', 'same_kind', 'unsafe'}, optional
        See casting argument of `ndarray.astype`. Controls what kind of data
        casting may occur.

    Returns
    -------
    structured : ndarray
       Structured array with fewer dimensions.

    Examples
    --------

    >>> from numpy.lib import recfunctions as rfn
    >>> dt = np.dtype([('a', 'i4'), ('b', 'f4,u2'), ('c', 'f4', 2)])
    >>> a = np.arange(20).reshape((4,5))
    >>> a
    array([[ 0,  1,  2,  3,  4],
           [ 5,  6,  7,  8,  9],
           [10, 11, 12, 13, 14],
           [15, 16, 17, 18, 19]])
    >>> rfn.unstructured_to_structured(a, dt)
    array([( 0, ( 1.,  2), [ 3.,  4.]), ( 5, ( 6.,  7), [ 8.,  9.]),
           (10, (11., 12), [13., 14.]), (15, (16., 17), [18., 19.])],
          dtype=[('a', '<i4'), ('b', [('f0', '<f4'), ('f1', '<u2')]), ('c', '<f4', (2,))])

    z$arr must have at least one dimensionr<rz&last axis with size 0 is not supportedNcSsg|]}dj|��qS)zf{})r�)r,rrrrr!r1#sz.unstructured_to_structured.<locals>.<listcomp>csg|]}|�jf�qSr)r#)r,rr)r�rr!r1$s)r�z!don't supply both dtype and nameszVThe length of the last dimension of arr must be equal to the number of fields in dtypez'align was True but dtype is not alignedcSsg|]}dj|��qS)zf{})r�)r,rrrrr!r18scsg|]}�j|jf�qSr)r#ry)r,r�)r�rr!r1>s)r$r�)r$r�r�r�)r�r�.���).r)ryr%r�r�r=r#r�rkr&r�ZisalignedstructZascontiguousarrayrVr�r�)r�r#r$r�r�r�Zn_elemr�r+r�r�r�r�r�r)r�r!r�s@7

cCs|fS)Nr)�funcr�rrr!�_apply_along_fields_dispatcherKsr�cCs(|jjdkrtd��t|�}||dd�S)a@
    Apply function 'func' as a reduction across fields of a structured array.

    This is similar to `apply_along_axis`, but treats the fields of a
    structured array as an extra axis. The fields are all first cast to a
    common type following the type-promotion rules from `numpy.result_type`
    applied to the field's dtypes.

    Parameters
    ----------
    func : function
       Function to apply on the "field" dimension. This function must
       support an `axis` argument, like np.mean, np.sum, etc.
    arr : ndarray
       Structured array for which to apply func.

    Returns
    -------
    out : ndarray
       Result of the recution operation

    Examples
    --------

    >>> from numpy.lib import recfunctions as rfn
    >>> b = np.array([(1, 2, 5), (4, 5, 7), (7, 8 ,11), (10, 11, 12)],
    ...              dtype=[('x', 'i4'), ('y', 'f4'), ('z', 'f8')])
    >>> rfn.apply_along_fields(np.mean, b)
    array([ 2.66666667,  5.33333333,  8.66666667, 11.        ])
    >>> rfn.apply_along_fields(np.mean, b[['x', 'z']])
    array([ 3. ,  5.5,  9. , 11. ])

    Nzarr must be a structured arrayr<)Zaxisr�)r#r$r%r)r�r��uarrrrr!r
Ns#cCs||fS)Nr)�dst�src�zero_unassignedrrr!�!_assign_fields_by_name_dispatcherysr�cCs\|jjdkr||d<dSx>|jjD]2}||jjkr@|rTd||<q"t|||||�q"WdS)a�
    Assigns values from one structured array to another by field name.

    Normally in numpy >= 1.14, assignment of one structured array to another
    copies fields "by position", meaning that the first field from the src is
    copied to the first field of the dst, and so on, regardless of field name.

    This function instead copies "by field name", such that fields in the dst
    are assigned from the identically named field in the src. This applies
    recursively for nested structures. This is how structure assignment worked
    in numpy >= 1.6 to <= 1.13.

    Parameters
    ----------
    dst : ndarray
    src : ndarray
        The source and destination arrays during assignment.
    zero_unassigned : bool, optional
        If True, fields in the dst for which there was no matching
        field in the src are filled with the value 0 (zero). This
        was the behavior of numpy <= 1.13. If False, those fields
        are not modified.
    N.r)r#r$r)r�r�r�r-rrr!r|s
cCs|fS)Nr)rn�required_dtyperrr!�_require_fields_dispatcher�sr�cCstj|j|d�}t||�|S)a�
    Casts a structured array to a new dtype using assignment by field-name.

    This function assigns from the old to the new array by name, so the
    value of a field in the output array is the value of the field with the
    same name in the source array. This has the effect of creating a new
    ndarray containing only the fields "required" by the required_dtype.

    If a field name in the required_dtype does not exist in the
    input array, that field is created and set to 0 in the output array.

    Parameters
    ----------
    a : ndarray
       array to cast
    required_dtype : dtype
       datatype for output array

    Returns
    -------
    out : ndarray
        array with the new dtype, with field values copied from the fields in
        the input array with the same name

    Examples
    --------

    >>> from numpy.lib import recfunctions as rfn
    >>> a = np.ones(4, dtype=[('a', 'i4'), ('b', 'f8'), ('c', 'u1')])
    >>> rfn.require_fields(a, [('b', 'f4'), ('c', 'u1')])
    array([(1., 1), (1., 1), (1., 1), (1., 1)],
      dtype=[('b', '<f4'), ('c', 'u1')])
    >>> rfn.require_fields(a, [('b', 'f4'), ('newf', 'u1')])
    array([(1., 0), (1., 0), (1., 0), (1., 0)],
      dtype=[('b', '<f4'), ('newf', 'u1')])

    )r#)r=rxryr)rnr��outrrr!r�s'
cCs|S)Nr)�arraysr_rYrZ�autoconvertrrr!�_stack_arrays_dispatcher�sr�cCs�t|t�r|St|�dkr"|dSdd�|D�}dd�|D�}dd�|D�}dd�|D�}|d}	t|	�}
dd�|
D�}x�|dd	�D]�}x|t|�D]p\}
}|
|kr�|
j|
|f�|j|
�q�|j|
�}|
|\}}|r�|
t||�f|
|<q�||kr�td
||f��q�Wq�Wt|
�dk�r&tj	|�}n�tj
tj|�f|
�}tj
tjd|f�}g}x�t|||d	d
�|dd	��D]r\}}}}|jj}|d	k�r�||dt|�||�<n8x6|D].}||||||�<||k�r�|j|��q�W�qrWtt||�||d�S)a�
    Superposes arrays fields by fields

    Parameters
    ----------
    arrays : array or sequence
        Sequence of input arrays.
    defaults : dictionary, optional
        Dictionary mapping field names to the corresponding default values.
    usemask : {True, False}, optional
        Whether to return a MaskedArray (or MaskedRecords is
        `asrecarray==True`) or a ndarray.
    asrecarray : {False, True}, optional
        Whether to return a recarray (or MaskedRecords if `usemask==True`)
        or just a flexible-type ndarray.
    autoconvert : {False, True}, optional
        Whether automatically cast the type of the field to the maximum.

    Examples
    --------
    >>> from numpy.lib import recfunctions as rfn
    >>> x = np.array([1, 2,])
    >>> rfn.stack_arrays(x) is x
    True
    >>> z = np.array([('A', 1), ('B', 2)], dtype=[('A', '|S3'), ('B', float)])
    >>> zz = np.array([('a', 10., 100.), ('b', 20., 200.), ('c', 30., 300.)],
    ...   dtype=[('A', '|S3'), ('B', np.double), ('C', np.double)])
    >>> test = rfn.stack_arrays((z,zz))
    >>> test
    masked_array(data=[(b'A', 1.0, --), (b'B', 2.0, --), (b'a', 10.0, 100.0),
                       (b'b', 20.0, 200.0), (b'c', 30.0, 300.0)],
                 mask=[(False, False,  True), (False, False,  True),
                       (False, False, False), (False, False, False),
                       (False, False, False)],
           fill_value=(b'N/A', 1.e+20, 1.e+20),
                dtype=[('A', 'S3'), ('B', '<f8'), ('C', '<f8')])

    r<rcSsg|]}tj|�j��qSr)r=reri)r,r@rrr!r1sz stack_arrays.<locals>.<listcomp>cSsg|]}t|��qSr)r&)r,r@rrr!r1scSsg|]
}|j�qSr)r#)r,r@rrr!r1scSsg|]
}|j�qSr)r$)r,r�rrr!r1scSsg|]\}}|�qSrr)r,rrr�rrr!r1
sNzIncompatible type '%s' <> '%s'zf%i)rYrZr�)rGrr&r2r3�indexrj�	TypeErrorrW�concatenater�r=r�ZcumsumZr_rkr#r$r[rb)r�r_rYrZr�r>Znrecordsr8ZfldnamesZdtype_lZnewdescrr$Zdtype_n�fname�fdtype�nameidxr;�cdtyper r��seenr@rrr��jr-rrr!r�sL)

,



cCs|fS)Nr)r@�key�
ignoremask�return_indexrrr!�_find_duplicates_dispatcher.sr�c
Cs�tj|�j�}t|j�}|}|rBx||D]}||}q*W||}|j�}||}|j�}	|	dd�|	dd�k}
|r�|j}d|
|dd�<tjdg|
f�}
|
dd�|
dd�|
dd�<|||
}|r�|||
fS|SdS)a�
    Find the duplicates in a structured array along a given key

    Parameters
    ----------
    a : array-like
        Input array
    key : {string, None}, optional
        Name of the fields along which to check the duplicates.
        If None, the search is performed by records
    ignoremask : {True, False}, optional
        Whether masked data should be discarded or considered as duplicates.
    return_index : {False, True}, optional
        Whether to return the indices of the duplicated values.

    Examples
    --------
    >>> from numpy.lib import recfunctions as rfn
    >>> ndtype = [('a', int)]
    >>> a = np.ma.array([1, 1, 1, 2, 2, 3, 3],
    ...         mask=[0, 0, 1, 0, 0, 0, 1]).view(ndtype)
    >>> rfn.find_duplicates(a, ignoremask=True, return_index=True)
    (masked_array(data=[(1,), (1,), (2,), (2,)],
                 mask=[(False,), (False,), (False,), (False,)],
           fill_value=(999999,),
                dtype=[('a', '<i8')]), array([0, 1, 3, 4]))
    Nr<Fr�r�r�)	r=rerirr#�argsortrXZ
recordmaskr�)
r@r�r�r�r+rsr0ZsortidxZ
sortedbaseZ
sorteddata�flagZ
sortedmask�
duplicatesrrr!r
3s(
 c		Cs||fS)Nr)	r��r1�r2�jointype�	r1postfix�	r2postfixr_rYrZrrr!�_join_by_dispatchermsr��inner�1�2c	-s�|dkrtd|��t�t�r$�f�tt���t��kr^t�fdd�t��D��}	td|	��x<�D]4}
|
|jjkr�td|
��|
|jjkrdtd	|
��qdW|j	�}|j	�}t|�}|jj|jj}}
t|�t|
�@t��}|o�|p�|�rd
}|d7}t|���fdd
�|D�}t
||�}t
||�}tj||f�}|j
�d�}||}tjdg|dd�|dd�kf�}|dd�|dd�|dd�<||}|||k}|||k|}t|�t|�}}|dk�r�d\}}n�|dk�r8||}tj||||kf�}tj||||k|f�}t|�|t|�|}}n<|dk�rt||}tj||||kf�}t|�|d}}||||}}t|j�} x0t|j�D]"\}!}"|!�k�r�| j|!|"f��q�Wx�t|j�D]�\}!}"tdd�| D��}#y|#j|!�}$Wn$tk
�r| j|!|"f�YnPX| |$\}%}&|!�k�rF|!t|"|&�f| |$<n$|!||&f|!||"fg| |$|$d�<�q�Wtj| �} t||�}'tj|'||f| d�}(|(jj}#x�|D]z})||)}*|)|#k�s�|)|
k�r�|�r�|)�k�r�|)|7})|(|)}+|*d|�|+d|�<|dk�r�|*|d�|+|'|'|�<�q�Wx�|
D]~})||)}*|)|#k�s`|)|k�rh|�rh|)�k�rh|)|7})|(|)}+|*d|�|+d|�<|dk�r.|�r.|*|d�|+|d�<�q.W|(j�d�t||d�},tt|(|�f|,�S)a<
    Join arrays `r1` and `r2` on key `key`.

    The key should be either a string or a sequence of string corresponding
    to the fields used to join the array.  An exception is raised if the
    `key` field cannot be found in the two input arrays.  Neither `r1` nor
    `r2` should have any duplicates along `key`: the presence of duplicates
    will make the output quite unreliable. Note that duplicates are not
    looked for by the algorithm.

    Parameters
    ----------
    key : {string, sequence}
        A string or a sequence of strings corresponding to the fields used
        for comparison.
    r1, r2 : arrays
        Structured arrays.
    jointype : {'inner', 'outer', 'leftouter'}, optional
        If 'inner', returns the elements common to both r1 and r2.
        If 'outer', returns the common elements as well as the elements of
        r1 not in r2 and the elements of not in r2.
        If 'leftouter', returns the common elements and the elements of r1
        not in r2.
    r1postfix : string, optional
        String appended to the names of the fields of r1 that are present
        in r2 but absent of the key.
    r2postfix : string, optional
        String appended to the names of the fields of r2 that are present
        in r1 but absent of the key.
    defaults : {dictionary}, optional
        Dictionary mapping field names to the corresponding default values.
    usemask : {True, False}, optional
        Whether to return a MaskedArray (or MaskedRecords is
        `asrecarray==True`) or a ndarray.
    asrecarray : {False, True}, optional
        Whether to return a recarray (or MaskedRecords if `usemask==True`)
        or just a flexible-type ndarray.

    Notes
    -----
    * The output is sorted along the key.
    * A temporary array is formed by dropping the fields not in the key for
      the two arrays and concatenating the result. This array is then
      sorted, and the common entries selected. The output is constructed by
      filling the fields with the selected entries. Matching is not
      preserved if there are some duplicates...

    r��outer�	leftouterzWThe 'jointype' argument should be in 'inner', 'outer' or 'leftouter' (got '%s' instead)c3s*|]"\}}|�|dd�kr|VqdS)r<Nr)r,rr�x)r�rr!r.�szjoin_by.<locals>.<genexpr>zduplicate join key %rzr1 does not have key field %rzr2 does not have key field %rz8r1 and r2 contain common names, r1postfix and r2postfix zcan't both be emptycsg|]}|�kr|�qSrr)r,rr)r�rr!r1�szjoin_by.<locals>.<listcomp>)�orderFr<Nrcss|]\}}|VqdS)Nr)r,r-r#rrr!r.�s)r#)rYrZ)r�r�r�r�r�r�)rr)r�r�)r%rGrNr&rw�next�	enumerater#r$rirzrWr�r�r=r2r3rpr�rjr��sort�dictr[rb)-r�r�r�r�r�r�r_rYrZ�dupr-Znb1Zr1namesZr2namesZ
collisionsr��key1Zr1kZr2kZauxZidx_sortZflag_inZidx_inZidx_1Zidx_2Zr1cmnZr2cmnZr1spcZr2spcZidx_out�s1�s2r8r�r�r$r�r;r�Zcmnr r0�selectedr)�kwargsr)r�r!rss�4



$ 









 


&

&cCs||fS)Nr)r�r�r�r�r�r�r_rrr!�_rec_join_dispatcher'sr�cCs$t||||ddd�}t|||f|�S)z�
    Join arrays `r1` and `r2` on keys.
    Alternative to join_by, that always returns a np.recarray.

    See Also
    --------
    join_by : equivalent function
    FT)r�r�r�r_rYrZ)r�r)r�r�r�r�r�r�r_r�rrr!r-s)F)F)NN)NT)TF)N)NNNNr�)r�FFF)NN)TF)TF)NNNNr�)Nr�TF)N)N)NN)FF)r)NNN)NFr�)NNNNN)NNFFr�)N)T)NNNN)NTFF)NNN)NTF)NNNNNN)r�r�r�NTF)NNNN)r�r�r�N)D�__doc__rQZnumpyr=Znumpy.marWrrrZnumpy.ma.mrecordsrZnumpy.core.overridesrZnumpy.lib._iotoolsrZ
numpy.testingr�corerl�__all__r"rr2rrrrArBrrIrOrUr[rbrcrrurrzr{rr}rr�r	r�rr�rr�r�rr�rr�r
r�rr�rr�rr�r
r�rr�rrrrr!�<module>s�


'$#


/





@
(
E
"
P
&
[
f+
%,
W
:
4