3
���h:f � @ sT d dl Z d dlZd dlZd dlZd dlmZmZm Z m
Z
d dl m Z m
Z
ddlmZmZmZ ddlmZ d dlmZ d dlmZ d d lmZ d d
lmZ G d d � d e�Z G d
d� de�Z ej dd� �Z G dd� de�Z G dd� de�Z!eG dd� de��Z"eG dd� de��Z#G dd� dee�Z$G dd � d e j%�Z&d d � Z'eG d d � d ee��Z(dS )!� N)� conventions�Variable�Dataset� open_dataset)�utils�indexing� )�TestCase�requires_netCDF4�unittest)�CFEncodedDataTest)� iteritems)�InMemoryDataStore)�WritableCFDataStore)� decode_cfc @ s$ e Zd Zdd� Zdd� Zdd� ZdS )�TestMaskedAndScaledArrayc C s t jtjd�dd�}| j|jtjd�� | j|jd� | j|jd� | j|jd� | jt |�d� | j
tj ddg|� t jtjd�dd�}| j
tjd�d |� t jtjd�dd�}| j
dtjd� |� t jtj dddddg�dd
d�}tj tj d dd d
g�}| j
||� d S )N� r )�
fill_value�floatr � )�
add_offset)Z scale_factor�c g{�G�z�?g�G�z��?g)\��(�?gR� ��Q�?)r i�������i����)
r �MaskedAndScaledArray�np�arange� assertEqual�dtype�shape�size�ndim�len�assertArrayEqual�nan�array)�self�x�expected� r( �</tmp/pip-build-5_djhm0z/xray/xarray/test/test_conventions.py�test s
z TestMaskedAndScaledArray.testc C s` t jtjd�dd�}| jtj|�� | jtj|d �� t jtjd�dd�}| jd|d � d S )Nr )r .�
)r r r r$ �
assertTrue�isnanr )r% r& r( r( r) �test_0d% s
z TestMaskedAndScaledArray.test_0dc C s| t jtjd�tjddg�d�}| jtjtjddg|� t jtjd�tjddg�d�}| jtj|�� | jtj|d �� d S )N� r r )r r r .) r r r r r$ r" r# r, r- )r% r& r( r( r) �test_multiple_fill_value- s z1TestMaskedAndScaledArray.test_multiple_fill_valueN)�__name__�
__module__� __qualname__r* r. r0 r( r( r( r) r s r c @ s$ e Zd Zdd� Zdd� Zdd� ZdS )�TestCharToStringArrayc C s� t jtd�dd�}tj|�}t jddd�}| j|j|j� | j|j|j� | j|j|j� | j|j |j � | j
t �� t |� W d Q R X | j
||� | j
t�� |d d� W d Q R X | jt|�d� t jtd�td�gdd�}tj|�}t jddgdd�}| j|j|j� | j|j|j� | j|j|j� | j|j |j � | jt |�t |�� | j
||� | j
|d d� |d d� � | j
t�� |d d �d d�f W d Q R X d S )N�abc�S)r r Zcdfr )r r$ �listr ZCharToStringArrayr r r r r � assertRaises� TypeErrorr! r" �
IndexError�str)r% r$ �actualr' r( r( r) �test_wrapper_class9 s2
z(TestCharToStringArray.test_wrapper_classc C sj t jdddgdddgg�}t jddg�}tj|�}| j||� t jd d
d g�}tj|j�}| j||� d S ) N�a�b�c�d�e�fr5 �def�ad�beZcf)r r$ r Zchar_to_stringr" �T)r% r$ r' r< r( r( r) �test_char_to_stringU s
z)TestCharToStringArray.test_char_to_stringc C s� t jddgddgg�}t jddgddggd d
gd d ggg�}tj|�}| j||� t jddgd d
ggddgd d ggg�}tj|j�}| j||� d S )
N�abZcdZefZghr> r? r@ rA rB rC �g�h)r r$ r Zstring_to_charr" rG )r% r$ r' r< r( r( r) �test_string_to_char_ s
z)TestCharToStringArray.test_string_to_charN)r1 r2 r3 r= rH rL r( r( r( r) r4 8 s
r4 c C s t | d�r| jd d�S | S d S )N�tzinfo)rM )�hasattr�replace)�dtr( r( r) �_ensure_naive_tzl s
rQ c @ s� e Zd Zedd� �Zdd� Zdd� Zdd� Zd d
� Zed d � �Z ed
d� �Z
edd� �Z edd� �Z edd� �Z
edd� �Zdd� Zdd� Zdd � Zd d � Zd d � Zed!d"� �Zd#S )$� TestDatetimec
C s
dd l }�x�tjd�dftjd�jdd�dfdtjd� dfdtjd� dfdtjd� d fdtjd� d
fd!dgd fdggdfddgdftjd�dfd"dgd fdggd ftjd�d ftjdd
d�dfd#ddgdfd$d%gD �]&\}}�xd&D �]}t|j|||��}t|||� tj � � tj
dd� t j |||�}W d Q R X t
|tj��rjtj|jtj��rj|jd �}n|}| j||� t j|||�\}} } d |k� r�| j|tj|d �� t|d �� r�|jd k� r�d |k� r�t jtj|�||�\}} } | j|tj|d �� � q�W q�W d S )'Nr r+ �days since 2000-01-01r � i 0 z hours since 1680-01-01 00:00:00z hour since 1680-01-01 00:00:00z Hour since 1680-01-01 00:00:00z" Hour since 1680-01-01 00:00:00 zdaYs since 2000-01-01�days since 1000-01-01i�� i N zdays since 1900-01-01� �.MpA�hours since 1-1-1 00:00:0.0g �?g �?z hours since 1900-01-01T00:00:00�&milliseconds since 2000-01-01T00:00:00�µseconds since 2000-01-01T00:00:00�standard� gregorian�proleptic_gregorian�ignorezUnable to decode time axiszM8[us]z1-1-1r r Z1000)r+ rS )r rU )rV rW )r rX )r rY )rZ r[ r\ )�netCDF4r r Zreshaper$ rQ �num2date�print�warnings�catch_warnings�filterwarningsr �decode_cf_datetime�
isinstanceZndarrayZ
issubdtyper Z
datetime64Zastyper" Zencode_cf_datetimeZaroundrN r �pd�Index)
r% �nc4� num_dates�units�calendarr' r< Z
actual_cmp�encoded�_r( r( r) �test_cf_datetimeu sX
z TestDatetime.test_cf_datetimec C s� t jtjdddg�dd�}tjddd�j}| j|jtjd �� | j ||� t jtjdddg�d�}| j|jtjd �� | j ||� d S )
Nr r r zdays since 1900-01-01rZ z
1900-01-01r )�periodszdatetime64[ns])
r �DecodedCFDatetimeArrayr r$ rf �
date_range�valuesr r r" )r% r< r' r( r( r) � test_decoded_cf_datetime_array� s z+TestDatetime.test_decoded_cf_datetime_arrayc C s� t jtjdddg�dd�}tjddd�j}| j|jtjd �� | j |t
dd� |t
dd� � t jtjdddg�dd�}tjddd�j}| j|jtjd �� | j |ddg |ddg � d S )
Nr r r zdays since 1900-01-01rZ z
1900-01-01r )ro zdatetime64[ns]) r rp r r$ rf rq rr r r r" �slice)r% r< r' r( r( r) �$test_slice_decoded_cf_datetime_array� s z1TestDatetime.test_slice_decoded_cf_datetime_arrayc C s6 t jdddd�}d}tjtjd�|�}| j||� d S )N�d z
1970-01-01rK )ro �start�freqzhours since 1-1-1970)rf rq r rd r r r" )r% r' rj r< r( r( r) �*test_decode_cf_datetime_non_standard_units� s z7TestDatetime.test_decode_cf_datetime_non_standard_unitsc s6 t dgtjd�dddd��� | jtd� fdd �� d S )
N�tr+ �foobarr r )rj �
missing_value�
_FillValuez _FillValue and missing_valuec s
t j� �S )N)r �decode_cf_variabler( )�varr( r) �<lambda>� s zQTestDatetime.test_decode_cf_with_conflicting_fill_missing_value.<locals>.<lambda>)r r r �assertRaisesRegexp�
ValueError)r% r( )r r) �2test_decode_cf_with_conflicting_fill_missing_value� s
z?TestDatetime.test_decode_cf_with_conflicting_fill_missing_valuec C sf t jdddd�}tjd�dftjd�dftjd�dfg}x(|D ] \}}tj||�}| j||� q>W d S )Nrv z
2000-01-01rK )ro rw rx zhours since 2000-01-01 0zhours since 2000-1-1 0zhours since 2000-01-01 0:00)rf rq r r r rd r" )r% r' Zcasesri rj r< r( r( r) �'test_decode_cf_datetime_non_iso_strings� s z4TestDatetime.test_decode_cf_datetime_non_iso_stringsc
C s� dd l }x�dD ]�}d}tjd d
d d �}|j|j� ||d
�}|j}tj� �" tjdd� t j
|||d
�}W d Q R X | j |j t
j d�� t|| �}| j|t
jdd�kj� � qW d S )Nr �noleap�365_day�360_day�julian�all_leap�366_dayzdays since 0001-01-01z
2001-04-01-00z
2001-04-30-23�H)�endrx )rk r] zUnable to decode time axiszM8[ns]r �s)r� r� r� r� r� r� )r^ rf rq �date2num�
to_pydatetimerr ra rb rc r rd r r r �absr, � timedelta64�all) r% rh rk rj �timesZ noleap_timer' r< Zabs_diffr( r( r) �!test_decode_non_standard_calendar� s
z.TestDatetime.test_decode_non_standard_calendarc C st d}xjd
D ]b}x\ddgdgggD ]H}t j� �" t jd d
� tj|||d �}W d Q R X | j|jtjd �� q W q
W d S )Nzdays since 0001-01-01r� r� r� r� r� r� i�8 r] zUnable to decode time axis)rk zM8[ns])r� r� r� r� r� r� )ra rb rc r rd r r r )r% rj rk �num_timer< r( r( r) �0test_decode_non_standard_calendar_single_element s
z=TestDatetime.test_decode_non_standard_calendar_single_elementc
C s� dd l }d}|jjddd�}x�d D ]z}|j|||�}| jd �� tj|||d
�}W d Q R X tj|j |||��}t
||||� | j |j tj d �� | j ||� q"W d S )
Nr zdays since 0001-01-01i� r � r� r� r� zUnable to decode time axis)rk �O)r� r� r� )
r^ Z
netcdftime�datetimer� � assertWarnsr rd r Zasarrayr_ r` r r )r% rh rj rP rk r� r< r' r( r( r) �9test_decode_non_standard_calendar_single_element_fallback s
zFTestDatetime.test_decode_non_standard_calendar_single_element_fallbackc C s dd l }d}d}tjdddd�}tjdd dd�}|j|j� ||d
�}|j|j� ||d
�}tjt|�d f�}||d d �df< ||d d �d f< |j} |j}
t j
� �" t j d
d� t j
|||d
�} W d Q R X | j| jtjd�� | j| d d �df | � | j| d d �d f |
� d S )Nr r� zdays since 0001-01-01z
2001-04-01z
2001-04-05�D)r� rx z
2001-05-01z
2001-05-05)rk r r r] zUnable to decode time axiszM8[ns])r^ rf rq r� r� r �emptyr! rr ra rb rc r rd r r r" ) r% rh rk rj Ztimes1Ztimes2Z noleap_time1Z noleap_time2Z mdim_timeZ expected1Z expected2r< r( r( r) �/test_decode_non_standard_calendar_multidim_time s*
z<TestDatetime.test_decode_non_standard_calendar_multidim_timec C s� dd l }x�dD ]�}x�dD ]�}d}d
j|�}tjd �}|j|||�}tjd d
��F}tjd� tj |||d�}| j
t |�d� | j dt
|d j�� W d Q R X | j
|jtjd�� | j||� qW qW d S )Nr �� �� �� �� �� r� r� r� zdays since {0}-01-01rv T)�record�always)rk r zUnable to decode time axisr� )r� r� r� r� r� )r� r� r� )r^ �formatr r r_ ra rb � simplefilterr rd r r! �assertInr; �messager r" ) r% rh �yearrk rj Z num_timesr' �wr< r( r( r) �*test_decode_non_standard_calendar_fallback8 s
z7TestDatetime.test_decode_non_standard_calendar_fallbackc C s� x�t jgddgft jdgdddgft jddgddddgfgD ]P\}}}tj� � tjdd� tj||�}W d Q R X t j|d d
�}| j||� q<W d S ) Nzdays since 2000-01-01�NaTr z2000-01-01T00:00:00Zr z2000-01-02T00:00:00Zr] zAll-NaNzdatetime64[ns])r ) r r# ra rb rc r rd r$ r" )r% ri rj Z
expected_listr< r' r( r( r) �test_cf_datetime_nanN s
z!TestDatetime.test_cf_datetime_nanc C s� x�t jddd�dft jdddd�d fdd
d gd ft jdd
d
g�dft jddg�d ft jd
dg�dft jd
g�dfgD ]\}}| j|tj|�� qtW d S )Nz
1900-01-01rT )ro z days since 1900-01-01 00:00:00z1900-01-01 12:00:00r� r )rx ro z hours since 1900-01-01 12:00:00z
1900-01-02z1900-01-02 00:00:01z!seconds since 1900-01-01 00:00:00r� z1900-01-02T00:00:00.005z days since 1970-01-01 00:00:00)rf rq Z to_datetimer r Zinfer_datetime_units)r% �datesr' r( r( r) �test_infer_datetime_units\ s$
z&TestDatetime.test_infer_datetime_unitsc
C sx ddt jd�fdddgdt jdddgd�fd d
t jd�fd d t jd�fd
dt jd�fdddgd t jddgfddgd
ddgfg}tjdkr�|jt jdd�dt jfddgdt jt jgfg� x�|D ]�\}}}tj|dd�}t j|�}|}t j
||�\}}| j ||� | j |j
|j
� |d k r�|}t j||�}| j ||� | j |j
|j
� q�W t jdd�}t jt jt j�d�}| j ||� d S )NZ1D�daysr Z2DZ3Dr r �int64�1h�hoursZ1msZ millisecondsZ1us� microsecondsr� Z0sZ1sr Z30mZ60mg �?g �?z0.16�nsF)Zbox)r r� r$ r# rf � __version__�extendr� � to_timedeltar Zencode_cf_timedeltar" r r Zdecode_cf_timedelta)r% ZexamplesZ
timedeltasrj Znumbersr' r< rm r( r( r) �test_cf_timedeltaq s2
z TestDatetime.test_cf_timedeltac C sl xft jddg�dft jddg�dft jddtjg�d ft jd
d g�d fgD ]\}}| j|tj|�� qJW d S )
Nz1 dayz2 daysr� r� z 1 day 1 hourr� Z1mZ2m�minutesZ1m3sZ1m4s�seconds)rf r� r r# r r Zinfer_timedelta_units)r% Zdeltasr' r( r( r) �test_infer_timedelta_units� s z'TestDatetime.test_infer_timedelta_unitsc C s> t ddddgddifi�}| jtd�� t|� W d Q R X d S )N�timer r rj zfoobar since 123zunable to decode time)r r� r� r )r% �dsr( r( r) �!test_invalid_units_raises_eagerly� s z.TestDatetime.test_invalid_units_raises_eagerlyc
C s� ddd�}t j� �< t jdd� ttddddg|fi��}| jd t|�� W d Q R X d
d i}ttddddg|fi��}| jd t|�� d S )
Nzdays since 0001-01-01r� )rj rk r] zunable to decode timer� r r z
(time) objectrj zdays since 1900-01-01z(time) datetime64[ns])ra rb rc r r r� �repr)r% �attrsr� r( r( r) �(test_dataset_repr_with_netcdf4_datetimes� s
z5TestDatetime.test_dataset_repr_with_netcdf4_datetimesN)r1 r2 r3 r
rn rs ru ry r� r� r� r� r� r� r� r� r� r� r� r� r� r( r( r( r) rR t s" ?
"rR c @ s e Zd Zdd� ZdS )�TestNativeEndiannessArrayc C s^ t jddd�}t jddd�}tj|�}|j|jks6t�|j|d d � jksNt�| j||� d S )NrT z>i8)r r� )r r r ZNativeEndiannessArrayr �AssertionErrorr" )r% r& r' r> r( r( r) r* � s
z TestNativeEndiannessArray.testN)r1 r2 r3 r* r( r( r( r) r� � s r� c @ s e Zd Zdd� Zdd� ZdS )�TestEncodeCFVariablec
C s� t dgtjddd�ddi�t dgtjdg�ddi�t dgdd d
gd did d
i�t dgdd d
gd did d
i�g}x,|D ]$}| jt�� tj|� W d Q R X qtW d S )
Nrz z
2000-01-01r )ro rj r{ z1 dayr r r r r} )r rf rq r� r8 r� r �encode_cf_variable)r% Z invalid_varsr r( r( r) � test_incompatible_attributes� s
z1TestEncodeCFVariable.test_incompatible_attributesc C sJ t dgtjtjdddg��}ddi|_| jd�� tj|� W d Q R X d S )Nr& r r r r Zint16z!floating point data as an integer)r r r$ r# �encodingr� r r� )r% �vr( r( r) �test_missing_fillvalue� s
z+TestEncodeCFVariable.test_missing_fillvalueN)r1 r2 r3 r� r� r( r( r( r) r� � s r� c @ s< e Zd Zdd� Zdd� Zdd� Zdd� Zd d
� Zd d � Zd
S )� TestDecodeCFc C s� t ddddgddifddddgddd �fdd
d dgd
difd��}t dddddgddifitjddd�dddtjgfd��}tj|�}| j||� d S )Nrz r r r rj zdays since 2000-01-01�y�bar)� coordinatesrj rT r+ i� r} )rz �foor� r� z
2000-01-01r )ro g @g $@)rz r� i���i���)r rf rq r r# r r �assertDatasetIdentical)r% �originalr' r< r( r( r) � test_dataset� s
zTestDecodeCF.test_datasetc C s4 t ddddgddifi�}tj|�}| j||� d S )Nr� rz r r r� �invalid)r r r r� )r% r� r< r( r( r) �test_invalid_coordinates� s
z%TestDecodeCF.test_invalid_coordinatesc C sF t dddgddifdddgfd��}tj|�}| j|jjd d� d S ) Nrz r r r� r& r/ rT )r� r& )r r r r r� r� )r% r� r< r( r( r) �test_decode_coordinates� s
z$TestDecodeCF.test_decode_coordinatesc C sB t f tjd�ddid�}t f tjd��}tj|�}| j||� d S )Nr r r� )r� )r r Zint32r� r Zmaybe_encode_dtyper� )r% r� r' r< r( r( r) �test_0d_int32_encoding� s
z#TestDecodeCF.test_0d_int32_encodingc C s� t dgdddgdtjddg�i�}t dgtjtjdgi �}tjdd��2}tj|�}| j||� | j dt
|d j �� W d Q R X d S ) Nrz r r r r| T)r� zvariable has multiple fill) r r r$ r# ra rb r r~ r� r� r; r� )r% r� r' r� r< r( r( r) �+test_decode_cf_with_multiple_missing_values� s
z8TestDecodeCF.test_decode_cf_with_multiple_missing_valuesc C s� t ddddgddifddd d
gdd ifddddgdddgdddggdd ifdd
ddgddifd��}t tjddd�ddddgfddddgdddgdddggdd ifdd
dtjgfd��}tj|dd�}tj|dd�}| j||� | j||� d S )Nrz r r r rj zdays since 2000-01-01r& � � � �kmr� rT r+ i� r} )rz r& r� r� z
2000-01-01r )ro )Zdrop_variables)rz r& i���i���)rz r& )r& )r rf rq r r# r r r� )r% r� r' r< Zactual2r( r( r) �"test_decode_cf_with_drop_variables� s $ $ z/TestDecodeCF.test_decode_cf_with_drop_variablesN) r1 r2 r3 r� r� r� r� r� r� r( r( r( r) r� � s r� c @ s e Zd ZdS )�CFEncodedInMemoryStoreN)r1 r2 r3 r( r( r( r) r�
s r� c @ s e Zd ZdZdd� Zdd� ZdS )� NullWrapperzw
Just for testing, this lets us create a numpy array directly
but make it look like its not in memory yet.
c C s
|| _ d S )N)r$ )r% r$ r( r( r) �__init__ s zNullWrapper.__init__c C s | j tj|| j� S )N)r$ r Zorthogonal_indexerr )r% �keyr( r( r) � __getitem__ s zNullWrapper.__getitem__N)r1 r2 r3 �__doc__r� r� r( r( r( r) r� s r� c C s$ t dd� t| �D ��}t|| jd�S )zr
Given a data store this wraps each variable in a NullWrapper so that
it appears to be out of memory.
c s s, | ]$\}}|t |jt|j�|j�fV qd S )N)r Zdimsr� rr r� )�.0�kr� r( r( r) � <genexpr> s z null_wrap.<locals>.<genexpr>)� variables�
attributes)�dictr
r r� )r� r� r( r( r) � null_wrap s r� c @ sF e Zd Zejdd� �Zeji i fdd��Zdd� Zdd� Zd d
� Z d S ) �TestCFEncodedDataStorec c s t � V d S )N)r� )r% r( r( r) � create_store& s z#TestCFEncodedDataStore.create_storec c s& t � }|j|f|� t|f|�V d S )N)r� Z
dump_to_storer )r% �dataZ save_kwargsZ open_kwargs�storer( r( r) � roundtrip* s z TestCFEncodedDataStore.roundtripc C s t jd��d S )Nz;cannot roundtrip coordinates yet for CFEncodedInMemoryStore)r ZSkipTest)r% r( r( r) �test_roundtrip_coordinates0 s z1TestCFEncodedDataStore.test_roundtrip_coordinatesc C s d S )Nr( )r% r( r( r) �"test_invalid_dataarray_names_raise4 s z9TestCFEncodedDataStore.test_invalid_dataarray_names_raisec C s d S )Nr( )r% r( r( r) �test_encoding_kwarg7 s z*TestCFEncodedDataStore.test_encoding_kwargN)
r1 r2 r3 �
contextlib�contextmanagerr� r� r� r� r� r( r( r( r) r� $ s r� ))r� Znumpyr Zpandasrf ra Zxarrayr r r r Z xarray.corer r � r r
r Z
test_backendsr Zxarray.core.pycompatr
Zxarray.backends.memoryr Zxarray.backends.commonr Zxarray.conventionsr r r4 Z vectorizerQ rR r� r� r� r� Z NDArrayMixinr� r� r� r( r( r( r) �<module> s6 (4 <
<
|