utils package

Submodules

utils.util module

class credsweeper.utils.util.DiffRowData(line_type, line_numb, line)

Bases: object

Class for keeping data of diff row.

line: str

line_numb: int

line_type: str

class credsweeper.utils.util.Util

Bases: object

Class that contains different useful methods.

default_encodings: Tuple[str, ...] = ('utf8', 'utf16', 'latin_1')

classmethod get_extension(file_path)

Return type

str

classmethod get_keyword_pattern(keyword, separator='=|:=|:')

Return type

Pattern

classmethod get_regex_combine_or(regex_strs)

Return type

str

classmethod get_shannon_entropy(data, iterator)

Borrowed from http://blog.dkbza.org/2007/05/scanning-data-for-entropy-anomalies.html.

Return type

float

classmethod is_entropy_validate(data)

Return type

bool

classmethod patch2files_diff(raw_patch, change_type)

Generate files changes from patch for added or deleted filepaths.

Parameters

• raw_patch (List[str]) – git patch file content

• change_type (str) – change type to select, “added” or “deleted”

Return type

Dict[str, List[Dict]]

Returns

return dict with {file paths: list of file row changes}, where elements of list of file row changes represented as:

{
    "old": line number before diff,
    "new": line number after diff,
    "line": line text,
    "hunk": diff hunk number
}

classmethod preprocess_file_diff(changes)

Generate changed file rows from diff data with changed lines (e.g. marked + or - in diff).

Parameters

changes (List[Dict]) – git diff by file rows data

Return type

List[DiffRowData]

Returns

diff rows data with as list of row change type, line number, row content

classmethod read_file(path, encodings=('utf8', 'utf16', 'latin_1'))

Read the file content using different encodings.

Try to read the contents of the file according to the list of encodings “encodings” as soon as reading occurs without any exceptions, the data is returned in the current encoding

Parameters

• path (str) – path to file

• encodings (Tuple[str, ...]) – supported encodings

Return type

List[str]

Returns

list of file rows in a suitable encoding from “encodings”, if none of the encodings match, an empty list will be returned

Module contents