3
UB(d L � @ s� d Z dZdZddlZddlZddlmZ ddlmZm Z m
Z
m Z m Z m
Z
ddlmZ dd lmZ d
d lmZ ee�Zd Zejd
ej�Zejdd d�Zd dd�Z dd� fdd�Z G dd� de �Z G dd� de �Z G dd � d e �Z!dS ) z'Cyril Jaquier and Fail2Ban Contributorsz Copyright (c) 2004 Cyril JaquierZGPL� N)�Lock� )�re� DateTemplate�DatePatternRegex�
DateTai64n� DateEpoch�RE_EPOCH_PATTERN)�validateTimeZone)�Utils� )� getLogger� z(?<!\\)\{DATE\}i� �< )ZmaxCountZmaxTimec C s� |d kr | }d| kr | j � }tj|�}|s�d|kr~tj| �rNt| d|kd�}n0|dkrft|dkd�}n|dkr~t|dkd
d �}|d kr�|dkr�t|d kr�dndd�}nt| �}tj||� |S )N�%�EPOCH�LEPOCH)�pattern�longFrm�{^LN-BEG}EPOCH�^EPOCH)�
lineBeginOnly�{^LN-BEG}LEPOCH�^LEPOCHT)r r �TAI64N�{^LN-BEG}TAI64N�^TAI64N�startF)Z wordBegin)r r r )r r r )r r r ) �upper�DD_patternCache�getr �searchr r r �set)r �key�template� r% �"/usr/lib/python3.6/datedetector.py�_getPatternTemplate, s&
r' c C s d| S )Nz {^LN-BEG}r% )�sr% r% r& �<lambda>D s r) c C sb || j �}tj|�}|s^|t| d| j��}t| d�r>tj|�}|s^t| d�sVt|�}nt|�}|S )Nr )�namer r �getattr�regex�hasattrr' )r$ �wrapr* � template2r, r% r% r& �_getAnchoredTemplateD s
r0 c @ sf e Zd ZdZdd� Zedd� �Zdd� Zdd d
d d d
dddddddddddgZedd� �Z dd � Z
d S ) �DateDetectorCachez7Implements the caching of the default templates list.
c C s t � | _t� | _d S )N)r �_DateDetectorCache__lock�list� _DateDetectorCache__templates)�selfr% r% r& �__init__\ s zDateDetectorCache.__init__c C s8 | j r | j S | j� | j r | j S | j� | j S Q R X dS )z6List of template instances managed by the detector.
N)r4 r2 �_addDefaultTemplate)r5 r% r% r& � templates` s zDateDetectorCache.templatesc C sd |j }|jd� rP|jd� rPt|d�rPt|�}|j |krPd|_| jd j|� | jd j|� dS )z&Cache Fail2Ban's default template.
z {^LN-BEG}�^r, g Y@r r N)r* �
startswithr- r0 �weight� _DateDetectorCache__tmpcache�append)r5 r$ r* r/ r% r% r& �_cacheTemplatel s "
z DateDetectorCache._cacheTemplatezF%ExY(?P<_sep>[-/.])%m(?P=_sep)%d(?:T| ?)%H:%M:%S(?:[.,]%f)?(?:\s*%z)?z)(?:%a )?%b %d %k:%M:%S(?:\.%f)?(?: %ExY)?z$(?:%a )?%b %d %ExY %k:%M:%S(?:\.%f)?z1%d(?P<_sep>[-/])%m(?P=_sep)(?:%ExY|%Exy) %k:%M:%Sz=%d(?P<_sep>[-/])%b(?P=_sep)%ExY[ :]?%H:%M:%S(?:\.%f)?(?: %z)?z%m/%d/%ExY:%H:%M:%Sz %m-%d-%ExY %k:%M:%S(?:\.%f)?r z{^LN-BEG}%H:%M:%Sz^<%m/%d/%Exy@%H:%M:%S>z%Exy%Exm%Exd ?%H:%M:%Sz%b %d, %ExY %I:%M:%S %pz^%b-%d-%Exy %k:%M:%Sz6%ExY%Exm%Exd(?:T| ?)%ExH%ExM%ExS(?:[.,]%f)?(?:\s*%z)?z1(?:%Z )?(?:%a )?%b %d %k:%M:%S(?:\.%f)?(?: %ExY)?z1(?:%z )?(?:%a )?%b %d %k:%M:%S(?:\.%f)?(?: %ExY)?r c C s@ t tjd t�r:x(ttj�D ]\}}t|�}|tj|< q W tjS )Nr )�
isinstancer1 �DEFAULT_TEMPLATES�str� enumerater' )r5 �i�dtr% r% r& �defaultTemplates� s
z"DateDetectorCache.defaultTemplatesc C sB g g f| _ x| jD ]}| j|� qW | j d | j d | _| ` dS )z<Add resp. cache Fail2Ban's default set of date templates.
r r N)r< rE r> r4 )r5 rD r% r% r& r7 � s
z%DateDetectorCache._addDefaultTemplateN) �__name__�
__module__� __qualname__�__doc__r6 �propertyr8 r> r@ rE r7 r% r% r% r& r1 Y s. r1 c @ s0 e Zd ZdZd
Zdd� Zedd � �Zd
d � Zd S )�DateDetectorTemplateztUsed for "shallow copy" of the template object.
Prevents collectively usage of hits/lastUsed in cached templates
r$ �hits�lastUsed�distancec C s || _ d| _d| _d| _d S )Nr i���)r$ rL rM rN )r5 r$ r% r% r& r6 � s z DateDetectorTemplate.__init__c C s | j | jj td| j� S )Nr )rL r$ r; �maxrN )r5 r% r% r& r; � s zDateDetectorTemplate.weightc C s t | j|�S )zF Returns attribute of template (called for parameters not in slots)
)r+ r$ )r5 r* r% r% r& � __getattr__� s z DateDetectorTemplate.__getattr__N)r$ rL rM rN ) rF rG rH rI � __slots__r6 rJ r; rP r% r% r% r& rK � s
rK c @ sz e Zd ZdZe� Zdd� Zddd�Zdd� Zdd d �Z e
d
d� �Z dd� Z e
dd� �Z
e
jdd� �Z
ddd�Zdd� Zd S )� DateDetectorzjManages one or more date templates to find a date within a log line.
Attributes
----------
templates
c C s> t � | _t� | _d| _d| _d| _d| _d| _d | _ d | _
d S )Ni, r ��r )r N)rS N) r3 �_DateDetector__templatesr" �_DateDetector__known_names�_DateDetector__unusedTime�_DateDetector__lastPos�_DateDetector__lastEndPos�_DateDetector__lastTemplIdx�_DateDetector__firstUnusedZ_DateDetector__preMatch�_DateDetector__default_tz)r5 r% r% r& r6 � s zDateDetector.__init__Fc C sD |j }|| jkr$|rd S td| ��| jj|� | jjt|�� d S )Nz(There is already a template with name %s)r* rU �
ValueError�addrT r= rK )r5 r$ � ignoreDupr* r% r% r& �_appendTemplate� s
z DateDetector._appendTemplatec s� t |t�r�| � }d|kr"|j� � tj� �}|s�� dkrR� fdd�}| j|� dS d� krl| j|dd � dS � d
kr�td � �}n
t|� �}tj� |� | j|� t j
d t |d
d�|j � t j
dt |d
d�|j� dS )a� Add a date template to manage and use in search of dates.
Parameters
----------
template : DateTemplate or str
Can be either a `DateTemplate` instance, or a string which will
be used as the pattern for the `DatePatternRegex` template. The
template will then be added to the detector.
Raises
------
ValueError
If a template already exists with the same name.
r � {^LN-BEG}� {DEFAULT}c s � dkr| j tj@ S d S )Nz {^LN-BEG})�flagsr �
LINE_BEGIN)r$ )r# r% r& r) s z-DateDetector.appendTemplate.<locals>.<lambda>Nz{DATE}F)�preMatch� allDefaultsz{NONE}z{UNB}^z date pattern `%r`: `%s`r � z date pattern regex for %r: %s)r` ra )r? rA r r r �addDefaultTemplater' r" r_ �logSys�infor+ r* �debugr, )r5 r$ r Zfltr% )r# r& �appendTemplate s,
zDateDetector.appendTemplateNTc sn t | j�dk}xZ|r tjjntjjD ]B}|dk r<||� r<q$� dk rXt|� fdd�d�}| j||d� q$W dS )z0Add Fail2Ban's default set of date templates.
r Nc s t j� fdd���S )Nc s
t j� �S )N)r ZunboundPattern)�m)r( r% r& r) 7 s zCDateDetector.addDefaultTemplate.<locals>.<lambda>.<locals>.<lambda>)�RE_DATE_PREMATCH�sub)r( )rd )r( r& r) 7 s z1DateDetector.addDefaultTemplate.<locals>.<lambda>)r. )r^ )�lenrT rR � _defCacher8 rE r0 r_ )r5 ZfilterTemplaterd re r^ r$ r% )rd r& rg * s z DateDetector.addDefaultTemplatec C s | j S )z6List of template instances managed by the detector.
)rT )r5 r% r% r& r8 ; s zDateDetector.templatesc
C s� t | j�s| j� tj� tkr$tjndd� }|td d|� d}d}d}| j}|t | j�k �rh| j| }|j}|j t
j t
j B @ r�|td d|� |j
|�}|}�n8| jd | jd } }
|td d || |
|| d | � | jd || |
� ||
|
d � | jd
�
|| d | � | jd k�sL|| | d � | jd
k�r�| jd
j� �r�||
|
d � | jd
k�s�||
d |
� | jd k�r�| jd j� �r�|td d || |
� � |j
|| |
�}n |td d � |j
|�}|}|�r^|j� } |j� }
t | j�dk�s4|j t
j t
j B @ �s4| | jd k�rB|
| jd k�rB|td
|� n|td� || |
|f}d}n
|td� |�s�|tdt | j�� d}�x0| jD �]$}||k�r�|d7 }�q�|td d||j� |j}|j
|�}|�r�|j� } |j� }
|td|| |j| jd |j� |d t | j�k�rP |j t
j t
j B @ �r.P | dk�rX|j�rX| j|d jj �rXP | |jk�st| | jd k�r�|td� | |d k �r�|| |
|f}d}|d7 }�q�P |d7 }�q�W | �r�|d �r�|\}} }
}|td|� | j| }|j}|�r�| jd7 _tj� |_| |_| j|k�r0| jd7 _| || d | � || f| _|
||
d ||
|
d � f| _|�r�|| jk�r�| j|�}|| _||fS |td� dS )a� Attempts to find date on a log line using templates.
This uses the templates' `matchDate` method in an attempt to find
a date. It also increments the match hit count for the winning
template.
Parameters
----------
line : str
Line which is searched by the date templates.
Returns
-------
re.MatchObject, DateTemplate
The regex match returned from the first successfully matched
template.
c W s d S )Nr% )�argsr% r% r& r) V s z(DateDetector.matchTime.<locals>.<lambda>r z"try to match time for line: %.120sN��z/ try to match last anchored template #%02i ...r zJ try to match last template #%02i (from %r to %r): ...%r==%r %s %r==%r...r z+ boundaries are correct, search in part %rz, boundaries show conflict, try whole searchz" matched last time template #%02izB ** last pattern collision - pattern change, reserve & search ...z8 ** last pattern not found - pattern change, search ...z search template (%i) ...z try template #%02i: %sz2 matched time template #%02i (at %r <= %r, %r) %sz1 ** distance collision - pattern change, reservez use best time template #%02iz
no template.���)Nrr rr rs )NN)ro rT rg rh �getEffectiveLevel�logLevel�logrY r$ rb r rc ZLINE_ENDZ matchDaterW rX �isalnumr �endr* rN rL �timerM rZ �_reorderTemplate) r5 �linerv �match�foundZignoreBySearchrC �ddtemplr$ rN �endposr% r% r& � matchTimeA s�
. .
(
zDateDetector.matchTimec C s | j S )N)r[ )r5 r% r% r& �
default_tz� s zDateDetector.default_tzc C s t |�| _d S )N)r
r[ )r5 �valuer% r% r& r� � s c C s� |dkr| j |�}|d }|dk r�yT|j||d | jd�}|dk rttj� tkrptjtd|d |d jd�|j� |S W n t k
r� Y nX dS )a Attempts to return the date on a log line using templates.
This uses the templates' `getDate` method in an attempt to find
a date.
For the faster usage, always specify a parameter timeMatch (the previous tuple result
of the matchTime), then this will work without locking and without cycle over templates.
Parameters
----------
line : str
Line which is searched by the date templates.
Returns
-------
float
The Unix timestamp returned from the first successfully matched
template or None if not found.
Nr r )r� z& got time %f for %r using template %s)
r� ZgetDater[ rh rt ru rv �groupr* r\ )r5 r{ Z timeMatchr$ Zdater% r% r& �getTime� s
zDateDetector.getTimec s� �r�| j ��� � tj� tkr0tjtd�� j� � j| j �� j�| j �k rR| j n�d �� �����fdd�}|� s���d kr��S �d �|� s��S ��= � g��d�< x,| j t
��k r؈| j jr�| j d7 _ q�W tj� tkr�tjtd��� �S �S )z�Reorder template (bubble up) in template list if hits grows enough.
Parameters
----------
num : int
Index of template should be moved.
z% -> reorder template #%02i, hits: %rr c
sL �� j } tj� tkr6tjtd���| � j�� j� �| kpJ��� jkS )NzE -> compare template #%02i & #%02i, weight %.3f > %.3f, hits %r > %r)r; rh rt ru rv rL rM )Zpweight)r~ �num�posr8 �untimer; r% r& � _moveable
s
z0DateDetector._reorderTemplate.<locals>._moveabler r z" -> moved template #%02i -> #%02i) rT rh rt ru rv rL rM rV r; rZ ro )r5 r� r� r% )r~ r� r� r8 r� r; r& rz � s. z DateDetector._reorderTemplate)F)NNT)N)rF rG rH rI r1 rp r6 r_ rk rg rJ r8 r� r� �setterr� rz r% r% r% r& rR � s
*
$rR i )N)"�
__author__Z
__copyright__Z __license__�copyry Z threadingr Z datetemplater r r r r r Zstrptimer
Zutilsr Zhelpersr
rF rh ru �compile�
IGNORECASErm ZCacher r' r0 �objectr1 rK rR r% r% r% r& �<module> s$
n |