3
���h�D � @ s� d dl Z d dlZd dlmZ d dlmZ d dlZd dlZ ddl
m Z ddl
m Z ddl
m
Z
ddl
mZ dd l
mZ dd
l
mZ dd l
mZ d d
l
mZ ddlmZmZ ddlmZmZ ddl
mZmZ ddlmZm Z ddl m Z m Z m Z m Z m!Z! ddl"m#Z#m$Z$m%Z%m&Z&m'Z' ddl(m)Z)m*Z*m+Z+m,Z, ddl-m.Z. ddddddd d d d d d!d"d#gZ/d$d%� Z0d&d'� Z1d5d)d*�Z2d+d,� Z3G d-d.� d.e�Z4G d/d0� d0e5�Z6G d1d2� d2eee�Z7e j8e7d3d4� dS )6� N)�Mapping)�Number� )�ops)�utils)�common)�groupby)�indexing)� alignment)�
formatting� )� conventions)�align�align_variables)�DatasetCoordinates�Indexes)�ImplementsDatasetReduce�BaseDataObject)�merge_datasets�expand_variables)�Frozen�SortedKeysDict�ChainMap�maybe_wrap_array�hashable)� as_variable�Variable�
Coordinate�broadcast_variables�default_index_coordinate)� iteritems�
basestring� OrderedDict�dask_array_type)�concat�year�month�day�hour�minute�second� microsecondZ
nanosecond�date�timeZ dayofyearZ
weekofyearZ dayofweekZquarterc
C s� t |t�st|��|jdd�}t|�dkr2t|��|\}}| | }|jdkrV|j� }n |jdkrntj|j �}nt|��|dkr�t
j dddd g�}|j }||d
d } n
t
||�} ||t|j| �fS ) zhGet a virtual variable (e.g., 'time.year') from a dict of
xarray.Variable objects (if possible)
�.r r r ZseasonZDJFZMAMZJJAZSON� � )�
isinstancer! �KeyError�split�len�ndim�to_index�pdZ Timestamp�values�np�arrayr& �getattrr �dims)
� variables�keyZ split_key�ref_name�var_nameZref_varr, Zseasonsr& �data� rB �3/tmp/pip-build-5_djhm0z/xray/xarray/core/dataset.py�_get_virtual_variable$ s$
rD c
C s� i }i }t dd� t| �D ��}x�t| �D ]|\}}xrt|j|j�D ]`\}}||kr\td| ��||krv|||< |||< q@|| |kr@td||||| || f ��q@W q(W |S )z�Calculate the dimensions corresponding to a set of variables.
Returns dictionary mapping from dimension names to sizes. Raises ValueError
if any of the dimension sizes conflict.
c s s | ]\}}|j s|V qd S )N)r< )�.0�k�vrB rB rC � <genexpr>J s z"_calculate_dims.<locals>.<genexpr>z0dimension %s already exists as a scalar variablezGconflicting sizes for dimension %r: length %s on %r and length %s on %r)�setr �zipr< �shape�
ValueError)r= r< Z last_usedZ scalar_varsrF �var�dim�sizerB rB rC �_calculate_dimsB s
"rP �%sc C s | rt || ��d S )N)rL )�args�msgrB rB rC �
_assert_emptyZ s rT c C s( t | d�r| j� } t| t�s$t| �} | S )z�Cast the given object to a Dataset.
Handles Datasets, DataArrays and dictionaries of variables. A new Dataset
object is only created if the provided object is not already one.
�
to_dataset)�hasattrrU r1 �Dataset)�objrB rB rC �
as_dataset_ s
rY c @ s< e Zd Zdd� Zdd� Zdd� Zdd� Zd d
� Zd d � Zd
S )�
DataVariablesc C s
|| _ d S )N)�_dataset)�self�datasetrB rB rC �__init__m s zDataVariables.__init__c s � fdd�� j jD �S )Nc 3 s | ]}|� j jkr|V qd S )N)r[ � _coord_names)rE r> )r\ rB rC rH q s z)DataVariables.__iter__.<locals>.<genexpr>)r[ �
_variables)r\ rB )r\ rC �__iter__p s zDataVariables.__iter__c C s t | jj�t | jj� S )N)r4 r[ r` r_ )r\ rB rB rC �__len__t s zDataVariables.__len__c C s || j jko|| j jkS )N)r[ r` r_ )r\ r> rB rB rC � __contains__w s zDataVariables.__contains__c C s" || j jkr| j | S t|��d S )N)r[ r_ r2 )r\ r> rB rB rC � __getitem__{ s
zDataVariables.__getitem__c C s
t j| �S )N)r Z vars_repr)r\ rB rB rC �__repr__� s zDataVariables.__repr__N) �__name__�
__module__� __qualname__r^ ra rb rc rd re rB rB rB rC rZ l s rZ c @ s e Zd Zdd� Zdd� ZdS )� _LocIndexerc C s
|| _ d S )N)r] )r\ r] rB rB rC r^ � s z_LocIndexer.__init__c C s t j|�std��| jjf |�S )Nz-can only lookup dictionaries from Dataset.loc)r � is_dict_like� TypeErrorr] �sel)r\ r> rB rB rC rd � s
z_LocIndexer.__getitem__N)rf rg rh r^ rd rB rB rB rC ri � s ri c @ s� e Zd ZdZejZd�dd�Zdd� Zd�d d
�Z d d � Z
e d�d
d��Z dd� Z
dd� Zdd� Zdd� Zedd� �Zdd� Zedd � �Zejd d � �Zed d � �Zd d!� Zd"d#� Ze d�d$d%��Ze� Zded&fd'd(�Z d�d)d*�Z d+d,� Z d-d.� Z d/d0� Z d1d2� Z!d�d3d4�Z"d5d6� Z#d7d8� Z$d9d:� Z%ed;d<� �Z&ed=d>� �Z'd?d@� Z(dAdB� Z)dCdD� Z*dZ+dEdF� Z,dGdH� Z-dIdJ� Z.dKdL� Z/edMdN� �Z0edOdP� �Z1edQdR� �Z2edSdT� �Z3d�dUdV�Z4d�dWdX�Z5d�dYdZ�Z6d�d\d]�Z7e8j9e7d^�Z:e8j9e7d_�Z;d`da� Z<edbdc� �Z=d�dedf�Z>dgdh� Z?d�didj�Z@d�dldm�ZAd�dndo�ZBd�dpdq�ZCd�drds�ZDd�dtdu�ZEd�dvdw�ZFdxdy� ZGdzd{� ZHd|d}� ZId�d~d�ZJd&eK� dd�fd�d��ZLd�d�d��ZMd�d�d��ZNd�d�� ZOd�d�� ZPd�d�� ZQed�d�� �ZRd�d�d��ZSd�d�d��ZTd�d�� ZUd�d�d��ZVd&f fd�d��ZWd�d�� ZXd�d�d��ZYd�d�� ZZd�d�� Z[e d�d�� �Z\e]d�d�d���Z^e]d�d�d���Z_e]d�d�� �Z`d�d�d��Zad�d�d��Zbd�d�� Zcd�d�� Zded�d�� �Zeed�d�� �ZfdS )�rW a� A multi-dimensional, in memory, array database.
A dataset resembles an in-memory representation of a NetCDF file, and
consists of variables, coordinates and attributes which together form a
self describing dataset.
Dataset implements the mapping interface with keys given by variable names
and values given by DataArray objects for each variable name.
One dimensional variables with name equal to their dimension are index
coordinates used for label based indexing.
N�broadcast_equalsc K s� t � | _t� | _i | _d| _d| _|r\d|krF|jd�}tj ddd� |r\t
dj |j � ���|dkrhi }|dkrvt� }|dk s�|dk r�| j
|||� |dk r�|| _d| _dS )a� To load data from a file or file-like object, use the `open_dataset`
function.
Parameters
----------
data_vars : dict-like, optional
A mapping from variable names to :py:class:`~xarray.DataArray`
objects, :py:class:`~xarray.Variable` objects or tuples of the
form ``(dims, data[, attrs])`` which can be used as arguments to
create a new ``Variable``. Each dimension must have the same length
in all variables in which it appears.
coords : dict-like, optional
Another mapping in the same form as the `variables` argument,
except the each item is saved on the dataset as a "coordinate".
These variables have an associated meaning: they describe
constant/fixed/independent quantities, unlike the
varying/measured/dependent quantities that belong in `variables`.
Coordinates values may be given by 1-dimensional arrays or scalars,
in which case `dims` do not need to be supplied: 1D arrays will be
assumed to give index values along the dimension with the same
name.
attrs : dict-like, optional
Global attributes to save on this dataset.
compat : {'broadcast_equals', 'equals', 'identical'}, optional
String indicating how to compare variables of the same name for
potential conflicts:
- 'broadcast_equals': all values must be equal when variables are
broadcast against each other to ensure common dimensions.
- 'equals': all values and dimensions must be the same.
- 'identical': all values, dimensions and attributes must be the
same.
Nr= z,Variables kwarg is deprecated. Use data_varsr )�
stacklevelz{0} are not valid kwargsT)r" r` rI r_ �_dims�_attrs� _file_obj�pop�warnings�warnrk �format�keys�_set_init_vars_and_dims�attrs� _initialized)r\ � data_vars�coordsrx �compat�kwargsrB rB rC r^ � s( #
zDataset.__init__c C s6 x0t | j�D ]"\}}|| jkr t||�| j|< q W dS )z3Add missing coordinates to self._variables
N)r r< r` r )r\ rN rO rB rB rC �_add_missing_coords_inplace� s
z#Dataset._add_missing_coords_inplaceTc st � dkr i � |r| j j� n| j }|r>t� fdd�| jD �d� |j|� t|�}|| _ || _| j� | jj� � dS )as Add a dictionary of new variables to this dataset.
Raises a ValueError if any dimensions have conflicting lengths in the
new dataset. Otherwise will update this dataset's _variables and
_dims attributes in-place.
Set `needs_copy=False` only if this dataset is brand-new and hence
can be thrown away if this method fails.
Nc s g | ]}|� kr|�qS rB rB )rE rF )�new_coord_namesrB rC �
<listcomp>� s z3Dataset._update_vars_and_coords.<locals>.<listcomp>z;coordinates with these names already exist as variables: %s) r` �copyrT rz �updaterP ro r~ r_ )r\ �
new_variablesr �
needs_copy�check_coord_namesr= r< rB )r rC �_update_vars_and_coords� s
z Dataset._update_vars_and_coordsc sZ t � fdd�|D �d� t|� �}t|�}t||d�\}}|j� � | j||ddd� dS )zBSet the initial value of Dataset variables and dimensions
c s g | ]}|� kr|�qS rB rB )rE rF )r{ rB rC r� � s z3Dataset._set_init_vars_and_dims.<locals>.<listcomp>z'redundant variables and coordinates: %s)r| F)r� r� N)rT r r r r� r� )r\ �varsr{ r| r= Zalignedr� r rB )r{ rC rw � s
z Dataset._set_init_vars_and_dimsc C s4 |j � \}}|r |||�\}}| ||d�}||_|S )zWCreate a new dataset from the contents of a backends.*DataStore
object
)rx )�loadrq )�cls�store�decoderr= �
attributesrX rB rB rC �
load_store
s zDataset.load_storec C s | j dk r| j j� d| _ dS )z/Close any files linked to this dataset
N)rq �close)r\ rB rB rC r� s
z
Dataset.closec C s | S )NrB )r\ rB rB rC � __enter__ s zDataset.__enter__c C s | j � d S )N)r� )r\ �exc_type� exc_value� tracebackrB rB rC �__exit__ s zDataset.__exit__c C s | j � | jj� }d|d< |S )z*Always load data in-memory before picklingNrq )r� �__dict__r� )r\ �staterB rB rC � __getstate__# s
zDataset.__getstate__c C s
t | j�S )z^Frozen dictionary of xarray.Variable objects constituting this
dataset's data
)r r` )r\ rB rB rC r= - s zDataset.variablesc C s | j d krd S t| j �S )N)rp r" )r\ rB rB rC � _attrs_copy4 s zDataset._attrs_copyc C s | j dkrt� | _ | j S )z8Dictionary of global attributes on this dataset
N)rp r" )r\ rB rB rC rx 7 s
z
Dataset.attrsc C s t |�| _d S )N)r" rp )r\ �valuerB rB rC rx ? s c C s t t| j��S )z�Mapping from dimension names to lengths.
This dictionary cannot be modified directly, but is updated when adding
new variables.
)r r ro )r\ rB rB rC r< C s z Dataset.dimsc C sr t dd� | jj� D ��}t dd� |j� D ��}|rnddlj}|j|j� � }x"t||�D ]\}}|| j| _qVW | S )a� Manually trigger loading of this dataset's data from disk or a
remote source into memory and return this dataset.
Normally, it should not be necessary to call this method in user code,
because all xarray functions should either work on deferred data or
load data automatically. However, this method can be necessary when
working with many file objects on disk.
c s s | ]\}}||j fV qd S )N)rA )rE rF rG rB rB rC rH V s z Dataset.load.<locals>.<genexpr>c s s$ | ] \}}t |t�r||fV qd S )N)r1 r# )rE rF rG rB rB rC rH W s r N) �dictr= �itemsZ
dask.arrayr: Zcomputer8 rJ rA )r\ Zall_dataZ lazy_data�daZevaluated_datarF rA rB rB rC r� L s
z Dataset.loadc C s t jdtdd� | j� S )NzFthe Dataset method `load_data` has been deprecated; use `load` insteadr )rn )rs rt �
FutureWarningr� )r\ rB rB rC � load_datad s
zDataset.load_datac C s2 t j| �}||_||_||_||_||_d|_|S )zaShortcut around __init__ for internal use when we want to skip
costly validation
T)�object�__new__r` r_ ro rp rq ry )r� r= � coord_namesr< rx Zfile_objrX rB rB rC �_construct_directj s
zDataset._construct_directFc C st t |�}|r<|| _|| _|dk r&|| _|| jk r6|| _| }n4|dkrN| jj� }|| jkr`| j� }| j||||�}|S )a* Fastpath constructor for internal use.
Preserves coord names and attributes; dimensions are recalculated from
the supplied variables.
The arguments are *not* copied when placed on the new dataset. It is up
to the caller to ensure that they have the right type and are not used
elsewhere.
Parameters
----------
variables : OrderedDict
coord_names : set or None, optional
attrs : OrderedDict or None, optional
Returns
-------
new : Dataset
N) rP ro r` r_ �_Dataset__default_attrsrp r� r� r� )r\ r= r� rx �inplacer< rX rB rB rC �_replace_vars_and_dims{ s
z Dataset._replace_vars_and_dimsc C sH |r t dd� t| j�D ��}n
| jj� }| j|| jj� | jj� | j� �S )z�Returns a copy of this dataset.
If `deep=True`, a deep copy is made of each of the component variables.
Otherwise, a shallow copy is made, so each variable in the new dataset
is also a variable in the original dataset.
c s s" | ]\}}||j d d�fV qdS )T)�deepN)r� )rE rF rG rB rB rC rH � s z Dataset.copy.<locals>.<genexpr>)r" r r` r� r� r_ ro r� )r\ r� r= rB rB rC r� � s
z Dataset.copyc s� t � }x |j� D ]}|j|j� qW x:� jD ]0}t � j| j�|kr,� j| ||< |j|� q,W t� fdd�|D ��}� j ||||�S )Nc 3 s | ]}|� j | fV qd S )N)ro )rE rF )r\ rB rC rH � s z8Dataset._subset_with_all_valid_coords.<locals>.<genexpr>)
rI r8 r� r< r_ r= r` �addr� r� )r\ r= r� rx � needed_dimsrG rF r< rB )r\ rC � _subset_with_all_valid_coords� s z%Dataset._subset_with_all_valid_coordsc C s� t � }t� }xb|D ]Z}y| j| ||< W q tk
rj t| j|�\}}}|||< || jkrf|j|� Y qX qW | j||| jj � d�S )z�Create a new Dataset with the listed variables from this dataset and
the all relevant coordinates. Skips all validation.
)rx )
r" rI r` r2 rD r_ r� r� rx r� )r\ �namesr= r� �namer? r@ rM rB rB rC � _copy_listed� s
zDataset._copy_listedc C s� ddl m} y| j| }W n& tk
r@ t| j|�\}}}Y nX t� }t|j�}x0| jD ]&}t| j | j�|krZ| j | ||< qZW ||||dd�S )z7Construct a DataArray by indexing this dataset
r )� DataArrayT)r� Zfastpath)
� dataarrayr� r` r2 rD r" rI r< r{ r= )r\ r� r� �variable�_r{ r� rF rB rB rC �_construct_dataarray� s
z Dataset._construct_dataarrayc C s | j dd�S )NF)r� )r� )r\ rB rB rC �__copy__� s zDataset.__copy__c C s | j dd�S )NT)r� )r� )r\ �memorB rB rC � __deepcopy__� s zDataset.__deepcopy__c C s
|| j kS )zzThe 'in' operator will return true or false depending on whether
'key' is an array in the dataset or not.
)r` )r\ r> rB rB rC rc � s zDataset.__contains__c C s
t | j�S )N)r4 r` )r\ rB rB rC rb � s zDataset.__len__c C s
t | j�S )N)�iterr` )r\ rB rB rC ra � s zDataset.__iter__c C s t dd� | jj� D ��S )Nc s s | ] }|j V qd S )N)�nbytes)rE rG rB rB rC rH � s z!Dataset.nbytes.<locals>.<genexpr>)�sumr= r8 )r\ rB rB rC r� � s zDataset.nbytesc C s t | �S )z�Attribute for location based indexing. Only supports __getitem__,
and only when the key is a dict of the form {dim: labels}.
)ri )r\ rB rB rC �loc� s z Dataset.locc C s< t j|�r| jf |�S t|�r(| j|�S | jtj|��S dS )z�Access variables or coordinates this dataset as a
:py:class:`~xarray.DataArray`.
Indexing with a list of names will return a new ``Dataset`` object.
N)r rj �iselr r� r� r9 �asarray)r\ r> rB rB rC rd s
zDataset.__getitem__c C s$ t j|�rtd��| j||i� dS )ac Add an array to this dataset.
If value is a `DataArray`, call its `select_vars()` method, rename it
to `key` and merge the contents of the resulting dataset into this
dataset.
If value is an `Variable` object (or tuple of form
``(dims, data[, attrs])``), add it to this dataset as a new
variable.
z:cannot yet use a dictionary as a key to set Dataset valuesN)r rj �NotImplementedErrorr� )r\ r> r� rB rB rC � __setitem__ s
zDataset.__setitem__c sX �fdd�}|� � � �j krT�j � = � fdd�t�j�D �}x|D ] � |� � qDW dS )z�Remove a variable from this dataset.
If this variable is a dimension, all variables containing this
dimension are also removed.
c s � j | = � jj| � d S )N)r` r_ �discard)rF )r\ rB rC �remove& s z#Dataset.__delitem__.<locals>.removec s g | ]\}}� |j kr|�qS rB )r< )rE rF rG )r> rB rC r� . s z'Dataset.__delitem__.<locals>.<listcomp>N)ro r r` )r\ r> r� Z also_deleterB )r> r\ rC � __delitem__ s
zDataset.__delitem__c s, � fdd�}| j |j ko*tj| j|j|d�S )z(Helper function for equals and identicalc s t | � �|�S )N)r; )�x�y)�
compat_strrB rC r| : s z#Dataset._all_compat.<locals>.compat)r| )r_ r �
dict_equivr` )r\ �otherr� r| rB )r� rC � _all_compat6 s zDataset._all_compatc C s* y | j |d�S ttfk
r$ dS X dS )a� Two Datasets are broadcast equal if they are equal after
broadcasting all variables against each other.
For example, variables that are scalar in one dataset but non-scalar in
the other dataset can still be broadcast equal if the the non-scalar
variable is a constant.
See Also
--------
Dataset.equals
Dataset.identical
rm FN)r� rk �AttributeError)r\ r� rB rB rC rm @ s
zDataset.broadcast_equalsc C s* y | j |d�S ttfk
r$ dS X dS )a� Two Datasets are equal if they have matching variables and
coordinates, all of which are equal.
Datasets can still be equal (like pandas objects) if they have NaN
values in the same locations.
This method is necessary because `v1 == v2` for ``Dataset``
does element-wise comparisons (like numpy.ndarrays).
See Also
--------
Dataset.broadcast_equals
Dataset.identical
�equalsFN)r� rk r� )r\ r� rB rB rC r� R s zDataset.equalsc C s: y t j| j|j�o | j|d�S ttfk
r4 dS X dS )z�Like equals, but also checks all dataset attributes and the
attributes on all variables and coordinates.
See Also
--------
Dataset.broadcast_equals
Dataset.equals
� identicalFN)r r� rx r� rk r� )r\ r� rB rB rC r� f s
zDataset.identicalc C s t | �S )zJOrderedDict of pandas.Index objects used for label based indexing
)r )r\ rB rB rC �indexesu s zDataset.indexesc C s t | �S )z]Dictionary of xarray.DataArray objects corresponding to coordinate
variables
)r )r\ rB rB rC r{ { s zDataset.coordsc C s t | �S )zODictionary of xarray.DataArray objects corresponding to data variables
)rZ )r\ rB rB rC rz � s zDataset.data_varsc C s t jdtdd� | jS )NzHthe Dataset property `vars` has been deprecated; use `data_vars` insteadr )rn )rs rt r� rz )r\ rB rB rC r� � s
z Dataset.varsc C s: t |t�r|g}| j|� |r"| n| j� }|jj|� |S )a� Given names of one or more variables, set them as coordinates
Parameters
----------
names : str or list of str
Name(s) of variables in this dataset to convert into coordinates.
inplace : bool, optional
If True, modify this dataset inplace. Otherwise, create a new
object.
Returns
-------
Dataset
)r1 r! �_assert_all_in_datasetr� r_ r� )r\ r� r� rX rB rB rC �
set_coords� s
zDataset.set_coordsc C s� |dkr| j t| j� }n2t|t�r*|g}| j|� tt|�t| j�@ d� |rT| n| j� }|j j|� |r�x|D ] }|j |= qrW |S )aW Given names of coordinates, reset them to become variables
Parameters
----------
names : str or list of str, optional
Name(s) of non-index coordinates in this dataset to reset into
variables. By default, all non-index coordinates are reset.
drop : bool, optional
If True, remove coordinates instead of converting them into
variables.
inplace : bool, optional
If True, modify this dataset inplace. Otherwise, create a new
object.
Returns
-------
Dataset
Nz5cannot remove index coordinates with reset_coords: %s)
r_ rI r< r1 r! r� rT r� �difference_updater` )r\ r� �dropr� rX r� rB rB rC � reset_coords� s
zDataset.reset_coordsc
C sz |dkr i }t j| �\}}t� }x(|j� D ] \}} | || _|j|� q*W |r\|||�\}}|j|||� |rv|j� dS )z7Store dataset contents to a backends.*DataStore object.N)r
Zencode_dataset_coordinatesrI r� �encodingr� r� �sync)
r\ r� �encoderr� r� r= rx Zcheck_encodingrF �encrB rB rC �
dump_to_store� s
zDataset.dump_to_store�wc C s. |dkr i }ddl m} || ||||||d�S )a� Write dataset contents to a netCDF file.
Parameters
----------
path : str, optional
Path to which to save this dataset. If no path is provided, this
function returns the resulting netCDF file as a bytes object; in
this case, we need to use scipy.io.netcdf, which does not support
netCDF version 4 (the default format becomes NETCDF3_64BIT).
mode : {'w', 'a'}, optional
Write ('w') or append ('a') mode. If mode='w', any existing file at
this location will be overwritten.
format : {'NETCDF4', 'NETCDF4_CLASSIC', 'NETCDF3_64BIT', 'NETCDF3_CLASSIC'}, optional
File format for the resulting netCDF file:
* NETCDF4: Data is stored in an HDF5 file, using netCDF4 API
features.
* NETCDF4_CLASSIC: Data is stored in an HDF5 file, using only
netCDF 3 compatible API features.
* NETCDF3_64BIT: 64-bit offset version of the netCDF 3 file format,
which fully supports 2+ GB files, but is only compatible with
clients linked against netCDF version 3.6.0 or later.
* NETCDF3_CLASSIC: The classic netCDF 3 file format. It does not
handle 2+ GB files very well.
All formats are supported by the netCDF4-python library.
scipy.io.netcdf only supports the last two formats.
The default format is NETCDF4 if you are saving a file to disk and
have the netCDF4-python library available. Otherwise, xarray falls
back to using scipy to write netCDF files and defaults to the
NETCDF3_64BIT format (scipy does not support netCDF4).
group : str, optional
Path to the netCDF4 group in the given file to open (only works for
format='NETCDF4'). The group(s) will be created if necessary.
engine : {'netcdf4', 'scipy', 'h5netcdf'}, optional
Engine to use when writing netCDF files. If not provided, the
default engine is chosen based on available dependencies, with a
preference for 'netcdf4' if writing to a file on disk.
encoding : dict, optional
Nested dictionary with variable names as keys and dictionaries of
variable specific encodings as values, e.g.,
``{'my_variable': {'dtype': 'int16', 'scale_factor': 0.1, 'zlib': True}, ...}``
Nr )� to_netcdf)ru �group�enginer� )Z backends.apir� )r\ �path�moderu r� r� r� r� rB rB rC r� � s
. zDataset.to_netcdf�dump�dumpsc C s
t j| �S )N)r Z dataset_repr)r\ rB rB rC re s zDataset.__repr__c sh i � xV| j j� D ]H}|jdk rtt|j|j��}t� fdd�|D ��rNtd��� j|� qW t t
� ��S )z[Block dimensions for this dataset's data or None if it's not a dask
array.
Nc 3 s& | ] \}}|� kr|� | kV qd S )NrB )rE �d�chunk)�chunksrB rC rH # s z!Dataset.chunks.<locals>.<genexpr>zinconsistent chunks) r= r8 r� �listrJ r< �anyrL r� r r )r\ rG Z
new_chunksrB )r� rC r� s
zDataset.chunks�xarray-c
s� yddl m� W n$ tk
r4 ddl}td��Y nX t� t�rNtj�j� �� � dk rx�fdd�� D �}|rxt d| ��dd � ������fd
d ��t
� �fd d��j j � D ��}�j
|�S )
a� Coerce all arrays in this dataset into dask arrays with the given
chunks.
Non-dask arrays in this dataset will be converted to dask arrays. Dask
arrays will be rechunked to the given chunk sizes.
If neither chunks is not provided for one or more dimensions, chunk
sizes along that dimension will not be updated; non-dask arrays will be
converted into dask arrays with a single block.
Parameters
----------
chunks : int or dict, optional
Chunk sizes along each dimension, e.g., ``5`` or
``{'x': 5, 'y': 5}``.
name_prefix : str, optional
Prefix for the name of any new dask arrays.
token : str, optional
Token uniquely identifying this dataset.
lock : optional
Passed on to :py:func:`dask.array.from_array`, if the array is not
already as dask array.
Returns
-------
chunked : xarray.Dataset
r )�tokenizeNz)xarray requires dask version 0.6 or newerc s g | ]}|� j kr|�qS rB )r< )rE r� )r\ rB rC r� P s z!Dataset.chunk.<locals>.<listcomp>z6some chunks keys are not dimensions on this object: %sc s" � d kr d S t � fdd�|D ��S )Nc 3 s" | ]}|� kr|� | fV qd S )NrB )rE r� )�dict_rB rC rH X s z1Dataset.chunk.<locals>.selkeys.<locals>.<genexpr>)r� )r� rv rB )r� rC �selkeysU s z Dataset.chunk.<locals>.selkeysc sX �||j �}|sd }|jdkrP�| �r*�n|j�}d�| |f }|j||� d�S |S d S )Nr z%s%s-%s)r� �lock)r< r5 �_datar� )r� rM r� Ztoken2Zname2)r� � name_prefixr� �tokenr� rB rC � maybe_chunkZ s
z"Dataset.chunk.<locals>.maybe_chunkc s g | ]\}}|�||� �f�qS rB rB )rE rF rG )r� r� rB rC r� e s )Z dask.baser� � ImportError�daskr1 r r� �fromkeysr< rL r" r= r� r� )r\ r� r� r� r� r� �bad_dimsr= rB )r� r� r� r� r\ r� r� r� rC r� ) s"
z
Dataset.chunkc s� � fdd�|D �}|r"t d| ��dd� t|�D �}t� }x<t� j�D ].\}�t�fdd�|D ��}�jf |�||< qFW � j|�S )a Returns a new dataset with each array indexed along the specified
dimension(s).
This method selects values from each array using its `__getitem__`
method, except this method does not require knowing the order of
each array's dimensions.
Parameters
----------
**indexers : {dim: indexer, ...}
Keyword arguments with names matching dimensions and values given
by integers, slice objects or arrays.
Returns
-------
obj : Dataset
A new Dataset with the same contents as this dataset, except each
array and dimension is indexed by the appropriate indexers. In
general, each array's data will be a view of the array's data
in this dataset, unless numpy fancy indexing was triggered by using
an array indexer, in which case the data will be a copy.
See Also
--------
Dataset.sel
Dataset.sel_points
Dataset.isel_points
DataArray.isel
c s g | ]}|� j kr|�qS rB )r< )rE rF )r\ rB rC r� � s z Dataset.isel.<locals>.<listcomp>zdimensions %r do not existc S s4 g | ],\}}|t |ttjtf�s*tj|�n|f�qS rB )r1 �intr9 �integer�slicer� )rE rF rG rB rB rC r� � s c 3 s$ | ] \}}|� j kr||fV qd S )N)r< )rE rF rG )rM rB rC rH � s z Dataset.isel.<locals>.<genexpr>)rL r r" r` r� r� r� )r\ �indexers�invalidr= r� Z var_indexersrB )r\ rM rC r� i s z Dataset.iselc K s | j f tj| |||d��S )a� Returns a new dataset with each array indexed by tick labels
along the specified dimension(s).
In contrast to `Dataset.isel`, indexers for this method should use
labels instead of integers.
Under the hood, this method is powered by using pandas's powerful Index
objects. This makes label based indexing essentially just as fast as
using integer indexing.
It also means this method uses pandas's (well documented) logic for
indexing. This means you can use string shortcuts for datetime indexes
(e.g., '2000-01' to select all values in January 2000). It also means
that slices are treated as inclusive of both the start and stop values,
unlike normal Python indexing.
Parameters
----------
method : {None, 'nearest', 'pad'/'ffill', 'backfill'/'bfill'}, optional
Method to use for inexact matches (requires pandas>=0.16):
* None (default): only exact matches
* pad / ffill: propgate last valid index value forward
* backfill / bfill: propagate next valid index value backward
* nearest: use nearest valid index value
tolerance : optional
Maximum distance between original and new labels for inexact
matches. The values of the index at the matching locations most
satisfy the equation ``abs(index[indexer] - target) <= tolerance``.
Requires pandas>=0.17.
**indexers : {dim: indexer, ...}
Keyword arguments with names matching dimensions and values given
by scalars, slices or arrays of tick labels.
Returns
-------
obj : Dataset
A new Dataset with the same contents as this dataset, except each
variable and dimension is indexed by the appropriate indexers. In
general, each variable's data will be a view of the variable's data
in this dataset, unless numpy fancy indexing was triggered by using
an array indexer, in which case the data will be a copy.
See Also
--------
Dataset.isel
Dataset.sel_points
Dataset.isel_points
DataArray.sel
)�method� tolerance)r� r �remap_label_indexers)r\ r� r� r� rB rB rC rl � s 3
z Dataset.sel�pointsc
sH t |�� � fdd�}|�j�}|�j�}|j� �dd� t|�D �}xL|D ]D\}}|�jkrftd| ��|jjdkrzt d��|j
dkrHtd ��qHW t d
d � |D ��}t |�dkr�td ��t |t
�r�|�jkr�td
��n:t|d�r�|j�jkr�td��nddlm} | |ddd�}t�fdd��fdd�tdd� |D �� D �D �|||d�S )a
Returns a new dataset with each array indexed pointwise along the
specified dimension(s).
This method selects pointwise values from each array and is akin to
the NumPy indexing behavior of `arr[[0, 1], [0, 1]]`, except this
method does not require knowing the order of each array's dimensions.
Parameters
----------
dim : str or DataArray or pandas.Index or other list-like object, optional
Name of the dimension to concatenate along. If dim is provided as a
string, it must be a new dimension name, in which case it is added
along axis=0. If dim is provided as a DataArray or Index or
list-like object, its name, which must not be present in the
dataset, is used as the dimension to concatenate along and the
values are added as a coordinate.
**indexers : {dim: indexer, ...}
Keyword arguments with names matching dimensions and values given
by array-like objects. All indexers must be the same length and
1 dimensional.
Returns
-------
obj : Dataset
A new Dataset with the same contents as this dataset, except each
array and dimension is indexed by the appropriate indexers. With
pointwise indexing, the new Dataset will always be a copy of the
original.
See Also
--------
Dataset.sel
Dataset.isel
Dataset.sel_points
DataArray.isel_points
c s � fdd�| j � D �S )Nc s, g | ]$\}}t � fd d�|jD ��r|�qS )c 3 s | ]}|� kV qd S )NrB )rE r� )� indexer_dimsrB rC rH � s zHDataset.isel_points.<locals>.relevant_keys.<locals>.<listcomp>.<genexpr>)r� r< )rE rF rG )r� rB rC r� � s z>Dataset.isel_points.<locals>.relevant_keys.<locals>.<listcomp>)r� )�mapping)r� rB rC �
relevant_keys� s z*Dataset.isel_points.<locals>.relevant_keysc S s g | ]\}}|t j|�f�qS rB )r9 r� )rE rF rG rB rB rC r� � s z'Dataset.isel_points.<locals>.<listcomp>zdimension %s does not exist�izIndexers must be integersr z Indexers must be 1 dimensionalc s s | ]\}}t |�V qd S )N)r4 )rE rF rG rB rB rC rH s z&Dataset.isel_points.<locals>.<genexpr>z$All indexers must be the same lengthzQExisting dimension names are not valid choices for the dim argument in sel_pointsr< zLExisting dimensions are not valid choices for the dim argument in sel_points)r� r� )r< r� c s g | ]}� j f |��qS rB )r� )rE r� )r\ rB rC r� s c s g | ]}t t� |���qS rB )r� rJ )rE Zinds)rv rB rC r� s c S s g | ] \}}|�qS rB rB )rE rF rG rB rB rC r� ! s )rN r{ rz )rI rz r{ rv r r< rL �dtype�kindrk r5 r4 r1 r! rV r� r� r� r$ rJ )
r\ rN r� r� rz r{ rF rG �lengthsr� rB )r� rv r\ rC � isel_points� s8 %
zDataset.isel_pointsc K s& t j| |||d�}| jf d|i|��S )a� Returns a new dataset with each array indexed pointwise by tick
labels along the specified dimension(s).
In contrast to `Dataset.isel_points`, indexers for this method should
use labels instead of integers.
In contrast to `Dataset.sel`, this method selects points along the
diagonal of multi-dimensional arrays, not the intersection.
Parameters
----------
dim : str or DataArray or pandas.Index or other list-like object, optional
Name of the dimension to concatenate along. If dim is provided as a
string, it must be a new dimension name, in which case it is added
along axis=0. If dim is provided as a DataArray or Index or
list-like object, its name, which must not be present in the
dataset, is used as the dimension to concatenate along and the
values are added as a coordinate.
method : {None, 'nearest', 'pad'/'ffill', 'backfill'/'bfill'}, optional
Method to use for inexact matches (requires pandas>=0.16):
* None (default): only exact matches
* pad / ffill: propagate last valid index value forward
* backfill / bfill: propagate next valid index value backward
* nearest: use nearest valid index value
tolerance : optional
Maximum distance between original and new labels for inexact
matches. The values of the index at the matching locations most
satisfy the equation ``abs(index[indexer] - target) <= tolerance``.
Requires pandas>=0.17.
**indexers : {dim: indexer, ...}
Keyword arguments with names matching dimensions and values given
by array-like objects. All indexers must be the same length and
1 dimensional.
Returns
-------
obj : Dataset
A new Dataset with the same contents as this dataset, except each
array and dimension is indexed by the appropriate indexers. With
pointwise indexing, the new Dataset will always be a copy of the
original.
See Also
--------
Dataset.sel
Dataset.isel
Dataset.isel_points
DataArray.sel_points
)r� r� rN )r r� r� )r\ rN r� r� r� Z pos_indexersrB rB rC �
sel_points$ s 4zDataset.sel_pointsc s4 t � fdd�|jj� D ��}� jf |||d�|��S )a; Conform this object onto the indexes of another object, filling
in missing values with NaN.
Parameters
----------
other : Dataset or DataArray
Object with an 'indexes' attribute giving a mapping from dimension
names to pandas.Index objects, which provides coordinates upon
which to index the variables in this dataset. The indexes on this
other object need not be the same as the indexes on this
dataset. Any mis-matched index values will be filled in with
NaN, and any mis-matched dimension names will simply be ignored.
method : {None, 'nearest', 'pad'/'ffill', 'backfill'/'bfill'}, optional
Method to use for filling index values from other not found in this
dataset:
* None (default): don't fill gaps
* pad / ffill: propagate last valid index value forward
* backfill / bfill: propagate next valid index value backward
* nearest: use nearest valid index value (requires pandas>=0.16)
tolerance : optional
Maximum distance between original and new labels for inexact
matches. The values of the index at the matching locations most
satisfy the equation ``abs(index[indexer] - target) <= tolerance``.
Requires pandas>=0.17.
copy : bool, optional
If `copy=True`, the returned dataset contains only copied
variables. If `copy=False` and no reindexing is required then
original variables from this dataset are returned.
Returns
-------
reindexed : Dataset
Another dataset, with this dataset's data but coordinates from the
other object.
See Also
--------
Dataset.reindex
align
c 3 s$ | ] \}}|� j kr||fV qd S )N)r< )rE rF rG )r\ rB rC rH � s z'Dataset.reindex_like.<locals>.<genexpr>)r� r� r� )r� r� r� �reindex)r\ r� r� r� r� r� rB )r\ rC � reindex_like\ s * zDataset.reindex_likec sl t j||d�}|s&|r"� jdd�S � S � fdd�|D �}|rHtd| ��tj� j� j||||d�}� j|�S )a� Conform this object onto a new set of indexes, filling in
missing values with NaN.
Parameters
----------
indexers : dict. optional
Dictionary with keys given by dimension names and values given by
arrays of coordinates tick labels. Any mis-matched coordinate values
will be filled in with NaN, and any mis-matched dimension names will
simply be ignored.
method : {None, 'nearest', 'pad'/'ffill', 'backfill'/'bfill'}, optional
Method to use for filling index values in ``indexers`` not found in
this dataset:
* None (default): don't fill gaps
* pad / ffill: propagate last valid index value forward
* backfill / bfill: propagate next valid index value backward
* nearest: use nearest valid index value (requires pandas>=0.16)
tolerance : optional
Maximum distance between original and new labels for inexact
matches. The values of the index at the matching locations most
satisfy the equation ``abs(index[indexer] - target) <= tolerance``.
Requires pandas>=0.17.
copy : bool, optional
If `copy=True`, the returned dataset contains only copied
variables. If `copy=False` and no reindexing is required then
original variables from this dataset are returned.
**kw_indexers : optional
Keyword arguments in the same form as ``indexers``.
Returns
-------
reindexed : Dataset
Another dataset, with this dataset's data but replaced coordinates.
See Also
--------
Dataset.reindex_like
align
pandas.Index.get_indexer
r� T)r� c s g | ]}|� j kr|�qS rB )r< )rE r� )r\ rB rC r� � s z#Dataset.reindex.<locals>.<listcomp>z invalid reindex dimensions: %s)r� ) r Zcombine_pos_and_kw_argsr� rL r
Zreindex_variablesr= r� r� )r\ r� r� r� r� Z kw_indexersr� r= rB )r\ rC r� � s * zDataset.reindexc
s� x<� j � D ]0\}}|| kr&td| ��|| kr
td| ��q
W t� }t� }xht| j�D ]Z\}}� j||�}t� fdd�|jD ��}|j dd�} || _| ||< || j
krV|j |� qVW | j |||d�S )a` Returns a new object with renamed variables and dimensions.
Parameters
----------
name_dict : dict-like
Dictionary whose keys are current variable or dimension names and
whose values are new names.
inplace : bool, optional
If True, rename variables and dimensions in-place. Otherwise,
return a new dataset object.
Returns
-------
renamed : Dataset
Dataset with renamed variables and dimensions.
See Also
--------
Dataset.swap_dims
DataArray.rename
z=cannot rename %r because it is not a variable in this datasetz the new name %r already existsc 3 s | ]}� j ||�V qd S )N)�get)rE rN )� name_dictrB rC rH � s z!Dataset.rename.<locals>.<genexpr>F)r� )r� )
r� rL r" rI r r` r �tupler< r� r_ r� r� )
r\ r r� rF rG r= r� r� r< rM rB )r rC �rename� s$
zDataset.renamec
s� xL� j � D ]@\}}|| jkr(td| ��| j| j|fkr
td||f ��q
W t� fdd�| jD ��}t� }| jj� }|j� j � � xTt
| j�D ]F\}}t � fdd�|jD ��}||kr�|j � n|j
� } || _| ||< q�W | j|||d�S )aj Returns a new object with swapped dimensions.
Parameters
----------
dims_dict : dict-like
Dictionary whose keys are current dimension names and whose values
are new names. Each value must already be a variable in the
dataset.
inplace : bool, optional
If True, swap dimensions in-place. Otherwise, return a new dataset
object.
Returns
-------
renamed : Dataset
Dataset with swapped dimensions.
See Also
--------
Dataset.rename
DataArray.swap_dims
zEcannot swap from dimension %r because it is not an existing dimensionzHreplacement dimension %r is not a 1D variable along the old dimension %rc 3 s | ]}� j ||�V qd S )N)r )rE rN )� dims_dictrB rC rH s z$Dataset.swap_dims.<locals>.<genexpr>c 3 s | ]}� j ||�V qd S )N)r )rE rN )r rB rC rH s )r� )r� r< rL r= rI r" r_ r� r� r8 r r Zto_coordZ to_variabler� )
r\ r r� rF rG Z result_dimsr= r� r< rM rB )r rC � swap_dims� s$
zDataset.swap_dimsc s� t � }x�� jj� D ]�\}�||krt�fdd�|D ��r��fdd�|D �}t�j�| }� fdd�|D �}�j||�}|jf ||i�} | ||< q�jdd�||< qW t j
j � fdd�|D �|d �}
t ||
�||< t
� j�t
|� t
|g�B } � j|| �S )
Nc 3 s | ]}|� j kV qd S )N)r< )rE r� )rM rB rC rH $ s z&Dataset._stack_once.<locals>.<genexpr>c s g | ]}|� j kr|�qS rB )r< )rE r� )rM rB rC r� % s z'Dataset._stack_once.<locals>.<listcomp>c s g | ]}� j | �qS rB )r< )rE r� )r\ rB rC r� ' s F)r� c s g | ]}� j | �qS rB )r� )rE r� )r\ rB rC r� . s )r� )r" r= r� r� r� r< � expand_dims�stackr� r7 �
MultiIndex� from_productr rI r_ r� ) r\ r< �new_dimr= r� Zadd_dimsZvdimsrK Zexp_varZ stacked_var�idxr� rB )r\ rM rC � _stack_once s
zDataset._stack_oncec K s* | }x |j � D ]\}}|j||�}qW |S )aS
Stack any number of existing dimensions into a single new dimension.
New dimensions will be added at the end, and the corresponding
coordinate variables will be combined into a MultiIndex.
Parameters
----------
**dimensions : keyword arguments of the form new_name=(dim1, dim2, ...)
Names of new dimensions, and the existing dimensions that they
replace.
Returns
-------
stacked : Dataset
Dataset with stacked data.
See also
--------
Dataset.unstack
)r� r
)r\ Z
dimensions�resultr r< rB rB rC r 6 s z
Dataset.stackc
C s< || j krtd| ��| j| }t|tj�s4td��tjj|j|jd�}| j f ddi||i��}|j}t
dd� |D ��r�td��d d
� |jD �}t � }xT|j j
� D ]F\}} ||kr�|| j kr�t t||��}
| jf ||
i�||< q�| ||< q�W x&t||j�D ]\}} t|| �||< q�W t| j�t|g� t|�B } | j|| �S ) a�
Unstack an existing dimension corresponding to a MultiIndex into
multiple new dimensions.
New dimensions will be added at the end.
Parameters
----------
dim : str
Name of the existing dimension to unstack.
Returns
-------
unstacked : Dataset
Dataset with unstacked data.
See also
--------
Dataset.stack
zinvalid dimension: %sz:cannot unstack a dimension that does not have a MultiIndex)r� r� Fc s s | ]}|d kV qd S )NrB )rE r� rB rB rC rH r s z"Dataset.unstack.<locals>.<genexpr>z,cannot unstack dimension with unnamed levelsc S s g | ]
}|j �qS rB )rO )rE �levrB rB rC r� u s z#Dataset.unstack.<locals>.<listcomp>)r< rL r� r1 r7 r r
�levelsr� r� r� r" r= r� rJ �unstackr rI r_ r� )
r\ rN �index�full_idxrX Z
new_dim_namesZ
new_dim_sizesr= r� rM Znew_dimsr r� rB rB rC r Q s,
zDataset.unstackc C s | j ||t|�dd�S )ak Update this dataset's variables with those from another dataset.
Parameters
----------
other : Dataset or castable to Dataset
Dataset or variables with which to update this dataset.
inplace : bool, optional
If True, merge the other dataset into this dataset in-place.
Otherwise, return a new dataset object.
Returns
-------
updated : Dataset
Updated dataset.
Raises
------
ValueError
If any dimensions would have inconsistent sizes in the updated
dataset.
�left)r� �overwrite_vars�join)�merger� )r\ r� r� rB rB rC r� � s zDataset.update�outerc C s6 t | ||||d�\}}|r | n| j� }|j||� |S )a& Merge the arrays of two datasets into a single dataset.
This method generally not allow for overriding data, with the exception
of attributes, which are ignored on the second dataset. Variables with
the same name are checked for conflicts via the equals or identical
methods.
Parameters
----------
other : Dataset or castable to Dataset
Dataset or variables to merge with this dataset.
inplace : bool, optional
If True, merge the other dataset into this dataset in-place.
Otherwise, return a new dataset object.
overwrite_vars : str or sequence, optional
If provided, update variables of these name(s) without checking for
conflicts in this dataset.
compat : {'broadcast_equals', 'equals', 'identical'}, optional
String indicating how to compare variables of the same name for
potential conflicts:
- 'broadcast_equals': all values must be equal when variables are
broadcast against each other to ensure common dimensions.
- 'equals': all values and dimensions must be the same.
- 'identical': all values, dimensions and attributes must be the
same.
join : {'outer', 'inner', 'left', 'right'}, optional
Method for joining ``self`` and ``other`` along shared dimensions:
- 'outer': use the union of the indexes
- 'inner': use the intersection of the indexes
- 'left': use indexes from ``self``
- 'right': use indexes from ``other``
Returns
-------
merged : Dataset
Merged dataset.
Raises
------
ValueError
If any variables conflict (see ``compat``).
)r| r )r r� r� ) r\ r� r� r r| r Z replace_varsr rX rB rB rC r � s
. z
Dataset.mergec C s0 t |�t | j� }|r || j8 }|r,td��d S )NzFOne or more of the specified variables cannot be found in this dataset)rI r` Zvirtual_variablesrL )r\ r� Z virtual_okayZ bad_namesrB rB rC r� � s
z Dataset._assert_all_in_datasetc C sD t j|�r|g}|dkr"| j|�S | j| j|�}| j||i S dS )a Drop variables or index labels from this dataset.
If a variable corresponding to a dimension is dropped, all variables
that use that dimension are also dropped.
Parameters
----------
labels : str
Names of variables or index labels to drop.
dim : None or str, optional
Dimension along which to drop index labels. By default (if
``dim is None``), drops variables rather than index labels.
Returns
-------
dropped : Dataset
N)r Z is_scalar�
_drop_varsr� r� r� )r\ �labelsrN Z new_indexrB rB rC r� � s
z Dataset.dropc sr | j �� t��� � t�fdd�t| j�D ��O � t� fdd�t| j�D ���t�fdd�| jD ��}| j�|�S )Nc 3 s, | ]$\}� t � fd d��D ��r|V qdS )c 3 s | ]}|� j kV qd S )N)r< )rE r� )rG rB rC rH � s z/Dataset._drop_vars.<locals>.<genexpr>.<genexpr>N)r� )rE rF )r� )rG rC rH � s z%Dataset._drop_vars.<locals>.<genexpr>c 3 s" | ]\}}|� kr||fV qd S )NrB )rE rF rG )r� rB rC rH � s c 3 s | ]}|� kr|V qd S )NrB )rE rF )r= rB rC rH � s )r� rI r r` r" r_ r� )r\ r� r� rB )r� r� r= rC r � s
zDataset._drop_varsc G s t jdtdd� | j|�S )NzFthe Dataset method `drop_vars` has been deprecated; use `drop` insteadr )rn )rs rt r� r� )r\ r� rB rB rC � drop_vars s
zDataset.drop_varsc sv |r,t |�t | j�A r,td|t| j�f ��| j� }x<t| j�D ].\}� t� fdd�|D ��}� j|� |j|< q@W |S )aK Return a new Dataset object with all array dimensions transposed.
Although the order of dimensions on each array will change, the dataset
dimensions themselves will remain in fixed (sorted) order.
Parameters
----------
*dims : str, optional
By default, reverse the dimensions on each array. Otherwise,
reorder the dimensions to this order.
Returns
-------
transposed : Dataset
Each array in the dataset (including) coordinates will be
transposed to the given order.
Notes
-----
Although this operation returns a view of each array's data, it
is not lazy -- the data will be fully loaded into memory.
See Also
--------
numpy.transpose
DataArray.transpose
zDarguments to transpose (%s) must be permuted dataset dimensions (%s)c 3 s | ]}|� j kr|V qd S )N)r< )rE rN )rM rB rC rH ) s z$Dataset.transpose.<locals>.<genexpr>)rI r< rL r r� r r` � transpose)r\ r< �dsr� Zvar_dimsrB )rM rC r s zDataset.transposec C s | j � S )N)r )r\ rB rB rC �T- s z Dataset.Tc C s t j| | j|�S )a� Returns a new dataset with squeezed data.
Parameters
----------
dim : None or str or tuple of str, optional
Selects a subset of the length one dimensions. If a dimension is
selected with length greater than one, an error is raised. If
None, all length one dimensions are squeezed.
Returns
-------
squeezed : Dataset
This dataset, but with with all or a subset of the dimensions of
length 1 removed.
Notes
-----
Although this operation returns a view of each variable's data, it is
not lazy -- all variable data will be fully loaded.
See Also
--------
numpy.squeeze
)r �squeezer< )r\ rN rB rB rC r 1 s zDataset.squeezer� c s � �j krtd� ��|dkr(t�j�}tj�j � tjd�}d}x^|D ]V}�j| }� |j krH� fdd�|j D �} ||j| �7 }|tj �fdd�| D ��7 }qHW |dk r�||k}
nB|dkr�||k}
n0|d kr�|dk}
n |dk r�td
| ��nt
d ���j f � |
i�S ) a Returns a new dataset with dropped labels for missing values along
the provided dimension.
Parameters
----------
dim : str
Dimension along which to drop missing values. Dropping along
multiple dimensions simultaneously is not yet supported.
how : {'any', 'all'}, optional
* any : if any NA values are present, drop that label
* all : if all values are NA, drop that label
thresh : int, default None
If supplied, require this many non-NA values.
subset : sequence, optional
Subset of variables to check for missing values. By default, all
variables in the dataset are checked.
Returns
-------
Dataset
z%%s must be a single dataset dimensionN)r� r c s g | ]}|� kr|�qS rB rB )rE r� )rN rB rC r� r s z"Dataset.dropna.<locals>.<listcomp>c s g | ]}� j | �qS rB )r< )rE r� )r\ rB rC r� t s r� �allzinvalid how option: %szmust specify how or thresh) r< rL r� rz r9 �zerosZint64r` �count�prodrk r� ) r\ rN �howZthreshZsubsetr" rO rF r: r< �maskrB )rN r\ rC �dropnaL s,
zDataset.dropnac C s
| j |�S )a' Fill missing values in this object.
This operation follows the normal broadcasting and alignment rules that
xarray uses for binary arithmetic, except the result is aligned to this
object (``join='left'``) instead of aligned to the intersection of
index coordinates (``join='inner'``).
Parameters
----------
value : scalar, ndarray, DataArray, dict or Dataset
Used to fill all matching missing values in this dataset's data
variables. Scalars, ndarrays or DataArrays arguments are used to
fill all data with aligned coordinates (for DataArrays).
Dictionaries or datasets match data variables and then align
coordinates if necessary.
Returns
-------
Dataset
)Z_fillna)r\ r� rB rB rC �fillna� s zDataset.fillnac s8 t |t�rt|g�� n |dkr*t�j�� nt|�� t�fdd�� D �d� t� �x�t�j�D ]�\}}� fdd�|jD �} | s�|j r�|�jkr�| s�t j
|j t j �s�|j t j
kr�t| �dkr�| \} nt| �|jkr�d} |j|f| ||d�|���|< q\|�|< q\W t�fdd ��jD ��}
|�r&�jnd} �j�|
| �S )
ao Reduce this dataset by applying `func` along some dimension(s).
Parameters
----------
func : function
Function which can be called in the form
`f(x, axis=axis, **kwargs)` to return the result of reducing an
np.ndarray over an integer valued axis.
dim : str or sequence of str, optional
Dimension(s) over which to apply `func`. By default `func` is
applied over all dimensions.
keep_attrs : bool, optional
If True, the dataset's attributes (`attrs`) will be copied from
the original object to the new one. If False (default), the new
object will be returned without attributes.
numeric_only : bool, optional
If True, only apply ``func`` to variables with a numeric dtype.
**kwargs : dict
Additional keyword arguments passed on to ``func``.
Returns
-------
reduced : Dataset
Dataset with this object's DataArrays replaced with new DataArrays
of summarized data and the indicated dimension(s) removed.
Nc s g | ]}|� j kr|�qS rB )r< )rE rN )r\ rB rC r� � s z"Dataset.reduce.<locals>.<listcomp>z+Dataset does not contain the dimensions: %sc s g | ]}|� kr|�qS rB rB )rE rN )r< rB rC r� � s r )rN �
keep_attrs�
allow_lazyc 3 s | ]}|� kr|V qd S )NrB )rE rF )r= rB rC rH � s z!Dataset.reduce.<locals>.<genexpr>)r1 r! rI r< rT r" r r` r{ r9 Z
issubdtyper� �numberZbool_r4 r5 �reducerx r� ) r\ �funcrN r( Z numeric_onlyr) r} r� rM Z reduce_dimsr� rx rB )r< r\ r= rC r+ � s6
zDataset.reducec s> t � ��fdd�t| j�D ��}|r*| jnd}t| �||d�S )a: Apply a function over the data variables in this dataset.
Parameters
----------
func : function
Function which can be called in the form `f(x, **kwargs)` to
transform each DataArray `x` in this dataset into another
DataArray.
keep_attrs : bool, optional
If True, the dataset's attributes (`attrs`) will be copied from
the original object to the new one. If False, the new object will
be returned without attributes.
args : tuple, optional
Positional arguments passed on to `func`.
**kwargs : dict
Keyword arguments passed on to `func`.
Returns
-------
applied : Dataset
Resulting dataset from applying ``func`` over each data variable.
c 3 s, | ]$\}}|t |�|f� ����fV qd S )N)r )rE rF rG )rR r, r} rB rC rH � s z Dataset.apply.<locals>.<genexpr>N)rx )r" r rz rx �type)r\ r, r( rR r} r= rx rB )rR r, r} rC �apply� s
z
Dataset.applyc K s | j � }|j|�}|j|� |S )a8 Assign new data variables to a Dataset, returning a new object
with all the original variables in addition to the new ones.
Parameters
----------
kwargs : keyword, value pairs
keywords are the variables names. If the values are callable, they
are computed on the Dataset and assigned to new data variables. If
the values are not callable, (e.g. a DataArray, scalar, or array),
they are simply assigned.
Returns
-------
ds : Dataset
A new Dataset with the new variables in addition to all the
existing variables.
Notes
-----
Since ``kwargs`` is a dictionary, the order of your arguments may not
be preserved, and so the order of the new variables is not well
defined. Assigning multiple variables within the same ``assign`` is
possible, but you cannot reference other variables created within the
same ``assign`` call.
See Also
--------
pandas.DataFrame.assign
)r� Z_calc_assign_resultsr� )r\ r} rA �resultsrB rB rC �assign� s
zDataset.assignr� c s| ddl m} � fdd�� jD �}t|� }tjdd� |D �dd�}t� j�}t� j�||< |f|d j }||||� j
|d�S ) a� Convert this dataset into an xarray.DataArray
The data variables of this dataset will be broadcast against each other
and stacked along the first axis of the new array. All coordinates of
this dataset will remain coordinates.
Parameters
----------
dim : str, optional
Name of the new dimension.
name : str, optional
Name of the new data array.
Returns
-------
array : xarray.DataArray
r )r� c s g | ]}� j | �qS rB )r= )rE rF )r\ rB rC r� 2 s z$Dataset.to_array.<locals>.<listcomp>c S s g | ]
}|j �qS rB )rA )rE �brB rB rC r� 4 s r )Zaxis)rx r� ) r� r� rz r r r r� r{ r� r< rx ) r\ rN r� r� rz Zbroadcast_varsrA r{ r< rB )r\ rC �to_array s
zDataset.to_arrayc sJ �fdd��D �}� �fdd�|D �}�j j� �}tjtt||��|d�S )Nc s g | ]}|� j kr|�qS rB )r< )rE rF )r\ rB rC r� > s z)Dataset._to_dataframe.<locals>.<listcomp>c s$ g | ] }�j | j� �jjd��qS )r ���)r` r r8 �reshape)rE rF )� ordered_dimsr\ rB rC r� ? s )r )r{ r6 r7 Z DataFramer" rJ )r\ r5 �columnsrA r rB )r5 r\ rC �
_to_dataframe= s
zDataset._to_dataframec C s | j | j�S )z�Convert this dataset into a pandas.DataFrame.
Non-index variables in this dataset form the columns of the
DataFrame. The DataFrame is be indexed by the Cartesian product of
this dataset's indices.
)r7 r< )r\ rB rB rC � to_dataframeD s zDataset.to_dataframec C s� |j jstd��|j}| � }t|d�r�tjj|j|j d�}|j
|�}dd� t |j �D �}x$t ||j�D ]\}}||f||< qfW dd� |jD �}n*|j
dk r�|j
ndf}||f||d < d }x,t|�D ] \} }
|
jj|�} || f|| < q�W |S ) a% Convert a pandas.DataFrame into an xarray.Dataset
Each column will be converted into an independent variable in the
Dataset. If the dataframe's index is a MultiIndex, it will be expanded
into a tensor product of one-dimensional indices (filling in missing
values with NaN). This method will produce a Dataset very similar to
that on which the 'to_dataframe' method was called, except with
possibly redundant dimensions (since all dataset variables will have
the same dimensionality).
z0cannot convert DataFrame with non-unique columnsr )r� c S s$ g | ] \}}|d k r|nd| �qS )Nzlevel_%irB )rE �nr� rB rB rC r� i s z*Dataset.from_dataframe.<locals>.<listcomp>c S s g | ]
}|j �qS rB )rO )rE r rB rB rC r� m s Nr r r r3 )r6 Z is_uniquerL r rV r7 r r
r r� r� � enumeraterJ r� r r8 r4 ) r� Z dataframer rX r r< rN r rK r� ZseriesrA rB rB rC �from_dataframeM s(
zDataset.from_dataframec s t j� �� �fdd��}|S )Nc sF | j j� }x*| jD ] }� | j| f|�|�|j|< qW �rB| j|_|S )N)r{ rU rz r` rp )r\ rR r} r rF )�fr( rB rC r, z s
z Dataset._unary_op.<locals>.func)� functools�wraps)r< r( r, rB )r< r( rC � _unary_opx s zDataset._unary_op�innerc s t j��� ���fdd��}|S )Nc sz t |tj�rtS t|d�rRt| |�dd�\} }dd� | jj� D �}|rRtd| ���sZ�n
�fdd�}| j ||� d �}|S )
Nr� F)r r� c S s g | ]\}}|d kr|�qS )r rB )rE r� �srB rB rC r� � s z4Dataset._binary_op.<locals>.func.<locals>.<listcomp>z-no overlapping labels for some dimensions: %sc s
� || �S )NrB )r� r� )r< rB rC �<lambda>� s z2Dataset._binary_op.<locals>.func.<locals>.<lambda>)� drop_na_vars)
r1 r �GroupBy�NotImplementedrV r r< r� rL �_calculate_binary_op)r\ r� Z
empty_indexes�gr )rC r< r � reflexiverB rC r, � s
z Dataset._binary_op.<locals>.func)r= r> )r< rH r rC r, rB )rC r< r rH rC �
_binary_op� s
zDataset._binary_opc s t j� �� fdd��}|S )Nc sb t |tj�rtd��t|d�r,|j| dd�}tj� �}| j||dd�}| j |j
|j |j dd� | S )NzLin-place operations between a Dataset and a grouped object are not permittedr� F)r� T)r� )
r1 r rD rk rV r r Zinplace_to_noninplace_oprF r� r` r_ rp )r\ r� rG r )r< rB rC r, � s
z(Dataset._inplace_binary_op.<locals>.func)r= r> )r< r, rB )r< rC �_inplace_binary_op� s
zDataset._inplace_binary_opc
s� � ��fdd�}t j|�r@t|t� r@|�j|�j|�}t|�S t|dd �}�jj|�}t|t�rz|�j|j�j|j�} n(t|d|��t ���fdd��jD ��} |j
j | � |S )Nc s� t � }d}x^| D ]V}||kr8�|| || �||< d}q�rVtdt| �t|�f ��q� s|| ||< qW |s�tdt| �t|�f ��|S )NFTzUdatasets must have the same data variables for in-place arithmetic operations: %s, %sz3datasets have no overlapping data variables: %s, %s)r" rL r� )Z
lhs_data_varsZ
rhs_data_varsZlhs_varsZrhs_varsZ dest_varsZ performed_oprF )rC r< r� rB rC �apply_over_both� s"
z5Dataset._calculate_binary_op.<locals>.apply_over_bothr{ r� c 3 s" | ]}|� �j | ��fV qd S )N)r= )rE rF )r< �other_variabler\ rB rC rH � s z/Dataset._calculate_binary_op.<locals>.<genexpr>) r rj r1 rW rz r; r{ r r= r" r` r� )
r\ r< r� r� rC rK Z
new_data_varsZ other_coordsr Znew_varsrB )rC r< r� rL r\ rC rF � s
z Dataset._calculate_binary_opr �upperc C s� |dkr | S |dk r"t dj|���|tdd�i}|tdd�i}|dkrL|}n|dkrZ|}nt d��t� }xbt| j�D ]T\}} || jkr�|| jkr�| jf |�| jf |� ||< q�| jf |�||< qt| ||< qtW | j |�}
|dkr�|
j
||d �S |
S dS ) a� Calculate the n-th order discrete difference along given axis.
Parameters
----------
dim : str, optional
Dimension over which to calculate the finite difference.
n : int, optional
The number of times values are differenced.
label : str, optional
The new coordinate in dimension ``dim`` will have the
values of either the minuend's or subtrahend's coordinate
for values 'upper' and 'lower', respectively. Other
values are not supported.
Returns
-------
difference : same type as caller
The n-th order finite difference of this object.
Examples
--------
>>> ds = xr.Dataset({'foo': ('x', [5, 5, 6, 6])})
>>> ds.diff('x')
<xarray.Dataset>
Dimensions: (x: 3)
Coordinates:
* x (x) int64 1 2 3
Data variables:
foo (x) int64 0 1 0
>>> ds.diff('x', 2)
<xarray.Dataset>
Dimensions: (x: 2)
Coordinates:
* x (x) int64 2 3
Data variables:
foo (x) int64 1 -1
r z*order `n` must be non-negative but got {0}Nr rM �lowerz8The 'label' argument has to be either 'upper' or 'lower'r3 ) rL ru r� r" r r= r< rz r� r� �diff) r\ rN r9 �labelZ kwargs_startZ
kwargs_endZ
kwargs_newr= r� rM �
differencerB rB rC rO � s0 '
z Dataset.diffc s� � fdd�|D �}|r"t d| ��t� }xTt� j�D ]F\}�|� jkrrt�fdd�|j� D ��}�jf |�||< q4�||< q4W � j|�S )a� Shift this dataset by an offset along one or more dimensions.
Only data variables are moved; coordinates stay in place. This is
consistent with the behavior of ``shift`` in pandas.
Parameters
----------
**shifts : keyword arguments of the form {dim: offset}
Integer offset to shift along each of the given dimensions.
Positive offsets shift to the right; negative offsets shift to the
left.
Returns
-------
shifted : Dataset
Dataset with the same coordinates and attributes but shifted data
variables.
See also
--------
roll
Examples
--------
>>> ds = xr.Dataset({'foo': ('x', list('abcde'))})
>>> ds.shift(x=2)
<xarray.Dataset>
Dimensions: (x: 5)
Coordinates:
* x (x) int64 0 1 2 3 4
Data variables:
foo (x) object nan nan 'a' 'b' 'c'
c s g | ]}|� j kr|�qS rB )r< )rE rF )r\ rB rC r� B s z!Dataset.shift.<locals>.<listcomp>zdimensions %r do not existc 3 s$ | ] \}}|� j kr||fV qd S )N)r< )rE rF rG )rM rB rC rH I s z Dataset.shift.<locals>.<genexpr>) rL r" r r= rz r� r� �shiftr� )r\ �shiftsr� r= r� �
var_shiftsrB )r\ rM rC rR s #
z
Dataset.shiftc st � fdd�|D �}|r"t d| ��t� }x@t� j�D ]2\}�t�fdd�|j� D ��}�jf |�||< q4W � j|�S )a� Roll this dataset by an offset along one or more dimensions.
Unlike shift, roll rotates all variables, including coordinates. The
direction of rotation is consistent with :py:func:`numpy.roll`.
Parameters
----------
**shifts : keyword arguments of the form {dim: offset}
Integer offset to rotate each of the given dimensions. Positive
offsets roll to the right; negative offsets roll to the left.
Returns
-------
rolled : Dataset
Dataset with the same coordinates and attributes but rolled
variables.
See also
--------
shift
Examples
--------
>>> ds = xr.Dataset({'foo': ('x', list('abcde'))})
>>> ds.roll(x=2)
<xarray.Dataset>
Dimensions: (x: 5)
Coordinates:
* x (x) int64 3 4 0 1 2
Data variables:
foo (x) object 'd' 'e' 'a' 'b' 'c'
c s g | ]}|� j kr|�qS rB )r< )rE rF )r\ rB rC r� s s z Dataset.roll.<locals>.<listcomp>zdimensions %r do not existc 3 s$ | ] \}}|� j kr||fV qd S )N)r< )rE rF rG )rM rB rC rH y s z Dataset.roll.<locals>.<genexpr>)rL r" r r= r� r� �rollr� )r\ rS r� r= r� rT rB )r\ rM rC rU Q s " z Dataset.rollc C s | j dd� dd�| �S )Nc S s | j S )N)�real)r� rB rB rC rB � s z Dataset.real.<locals>.<lambda>T)r( )r? )r\ rB rB rC rV s z Dataset.realc C s | j dd� dd�| �S )Nc S s | j S )N)�imag)r� rB rB rC rB � s z Dataset.imag.<locals>.<lambda>T)r( )r? )r\ rB rB rC rW � s z Dataset.imag)NNNrm )NTT)N)N)F)N)F)NFF)NTN)Nr� NNNN)Nr� NF)NN)r� )r� NN)NNT)NNNT)F)F)T)F)N)N)r� NN)NFFF)r� N)F)Fr@ T)FT)r rM )grf rg rh �__doc__r ZDatasetGroupByZ groupby_clsr^ r~ r� rw � classmethodr� r� r� r� r� �propertyr= r� rx �setterr< r� r� r� r� r� r� r� r� r� r� r� r� rc rb ra r� r� rd r� r� �__hash__r� rm r� r� r� r{ rz r� r� r� r� r� r Zfunction_aliasr� r� re r� r� r� rl r� r� r r� r r r
r r r� rI r r� r� r r r r r r& r' r+ r. r0 r2 r7 r8 r; � staticmethodr? rI rJ rF rO rR rU rV rW rB rB rB rC rW � s�
7
%
#
3
?.
6
W
8
/
8
,
16
3
'
7
A %
+
,
M2. rW F)Z
array_only)rQ )9r= rs � collectionsr Znumbersr Znumpyr9 Zpandasr7 � r r r r r r
r r
r r Z coordinatesr r r r r r r r r r r r r� r r r r r Zpycompatr r! r" r# �combiner$ Z_DATETIMEINDEX_COMPONENTSrD rP rT rY rZ r� ri rW Z!inject_all_ops_and_reduce_methodsrB rB rB rC �<module> sb
|