3
���h� � @ s> d dl mZ d dlmZ d dlZd dlZd dlZd dlZd dlZ d dl
Z
d dl Z d dl Z
d dl mZ e
jd�Ze
jd�Ze
jd�Ze
jd�ZdZdZdZej� Zd d
� Zd d � Zd
d� Zdd� Z dd� Z dd� Z dd� Z d)e e!d�dd�Z"dd � Z#d d � Z$d d � Z%d!d"� Z&e
j'j(d#dd$g�d%d&� �Z)e
j'j*G d'd(� d(��Z+dS )*� )� ExitStack)�datetimeN)� DataFramezgoogle.api_core.exceptionszgoogle.cloud.bigqueryz google.oauth2.service_account�
pandas_gbqc C s t � stjd� d S )Nz1Cannot run integration tests without a project id)�_get_project_id�pytest�skip� r r �:/tmp/pip-build-5_djhm0z/pandas/pandas/tests/io/test_gbq.py�_skip_if_no_project_id s r c C s t � stjd� d S )NzACannot run integration tests without a private key json file path)�_get_private_key_pathr r r r r r
� _skip_if_no_private_key_path s r
c C s dt jkodt jkS )N�TRAVIS_BUILD_DIR�GBQ_PROJECT_ID)�os�environr r r r
�_in_travis_environment% s r c C s" t � rtjjd�S tp tjjd�S )Nr )r r r �get�
PROJECT_IDr r r r
r ) s r c C s8 t � r tjjtjjd�ddg� S t} | s4tjjd�} | S )Nr Zciztravis_gbq.jsonZ"GBQ_GOOGLE_APPLICATION_CREDENTIALS)r r �path�joinr r �PRIVATE_KEY_JSON_PATH)�private_key_pathr r r
r / s r c C s t � } | rtjj| �S d S )N)r �service_accountZ CredentialsZfrom_service_account_file)r r r r
�_get_credentials; s r c C s t � } t� }tj| |d�S )N)�project� credentials)r r �bigqueryZClient)�
project_idr r r r
� _get_clientA s r �
)�length�returnc C s dj tjtj| d��S )N� )�k)r �random�choices�string�ascii_lowercase)r! r r r
�generate_rand_strG s r) c C s� t jjdd| fd�jt�}t jjd| �}t jjddd| fd�}t jjddd| fd�jt�}dd� t| �D �}t|d |d |d |d |d d�t| �d �S )
N� � )�sizer c S s g | ]}t jtjd ���qS )z
US/Arizona)r �now�pytz�timezone)�.0�tr r r
�
<listcomp>Q s z+make_mixed_dataframe_v2.<locals>.<listcomp>r )�bools�flts�ints�strs�times)�index) �npr% �randintZastype�boolZrandn�str�ranger )� test_sizer3 r4 r5 r6 r7 r r r
�make_mixed_dataframe_v2K s
r? c sB i � � fdd�}| j d|� tjd� d� ks2t�d� ks>t�d S )Nc s � j |� tdgg�S )Ng �?)�updater )�sql�kwargs)�captured_kwargsr r
�
mock_read_gbqa s
z>test_read_gbq_without_deprecated_kwargs.<locals>.mock_read_gbqzpandas_gbq.read_gbqzSELECT 1�verboseZ private_key)�setattr�pd�read_gbq�AssertionError)� monkeypatchrD r )rC r
�'test_read_gbq_without_deprecated_kwargs^ s
rK c sH i � � fdd�}| j d|� tjdddd� � d s8t�� d sDt�d S )
Nc s � j |� tdgg�S )Ng �?)r@ r )rA rB )rC r r
rD o s
z4test_read_gbq_with_new_kwargs.<locals>.mock_read_gbqzpandas_gbq.read_gbqzSELECT 1Tr+ )�use_bqstorage_api� max_resultsrL rM )rF rG rH rI )rJ rD r )rC r
� test_read_gbq_with_new_kwargsl s rN c sB i � � fdd�}| j d|� tjd� d� ks2t�d� ks>t�d S )Nc s � j |� tdgg�S )Ng �?)r@ r )rA rB )rC r r
rD } s
z7test_read_gbq_without_new_kwargs.<locals>.mock_read_gbqzpandas_gbq.read_gbqzSELECT 1rL rM )rF rG rH rI )rJ rD r )rC r
� test_read_gbq_without_new_kwargsz s
rO � progress_barZfooc s: i � � fdd�}| j d|� tjd|d� d� ks6t�d S )Nc s � j |� tdgg�S )Ng �?)r@ r )rA rB )rC r r
rD � s
z<test_read_gbq_progress_bar_type_kwarg.<locals>.mock_read_gbqzpandas_gbq.read_gbqzSELECT 1)�progress_bar_typerQ )rF rG rH rI )rJ rP rD r )rC r
�%test_read_gbq_progress_bar_type_kwarg� s
rR c @ s` e Zd Zej� dd� �Zdd� Zejjddde � fdd ej
e j j
�fd
d e � fg�d d
� �ZdS )�-TestToGBQIntegrationWithServiceAccountKeyPathc c sp t � t� dt� }t� | _| jj|�| _| jjtj| j�� t� }|� d|� �}|V | jj | jdd� d S )NZpydata_pandas_bq_testing_�.T)Zdelete_contents)
r r
r) r �clientZdatasetZcreate_datasetr ZDatasetZdelete_dataset)�selfZ
dataset_idZ
table_name�destination_tabler r r
� gbq_dataset� s
z9TestToGBQIntegrationWithServiceAccountKeyPath.gbq_datasetc C sZ |}d}t |�}|j|t� d t� d� tjd|� �t� t� dd�}|d d |ksVt�d S )Ni!N )� chunksizer z!SELECT COUNT(*) AS num_rows FROM �standard)r r �dialect�num_rowsr )r? �to_gbqr r rG rH rI )rV rX rW r> �df�resultr r r
�test_roundtrip� s
z<TestToGBQIntegrationWithServiceAccountKeyPath.test_roundtripz)if_exists, expected_num_rows, expectation�appendi, Zfail�� �replace�d c C s� |}d}t |�}|j|t� d t� d� |�( |jd d� j|t� |d t� d� W d Q R X tjd|� �t� t� dd�}|d d |ks�t�d S )
Nrb )rY r rd )� if_existsrY r z!SELECT COUNT(*) AS num_rows FROM rZ )r r r[ r\ r )r? r] r r ZilocrG rH rI ) rV re Zexpected_num_rowsZ expectationrX rW r> r^ r_ r r r
�test_gbq_if_exists� s*
z@TestToGBQIntegrationWithServiceAccountKeyPath.test_gbq_if_existsN)�__name__�
__module__� __qualname__r ZfixturerX r` �mark� parametrize�does_not_raiseZraisesr ZgbqZTableCreationErrorrf r r r r
rS � s
rS )r ),�
contextlibr rl r r �platformr% r' Znumpyr9 r r. ZpandasrG r Z importorskipZapi_exceptionsr r r r r ZPRIVATE_KEY_JSON_CONTENTS�python_version�VERSIONr r
r r r r r �intr< r) r? rK rN rO rj rk rR ZsinglerS r r r r
�<module> s@
|