API¶

SIPS files¶

class sippers.file.PackedSipsFile(path, strict=False)¶

Packed SIPS file.

Process of content of zip file is processed with iterators to keep the minimal memory footprint.

Example:

with PackedSipsFile('/tmp/PACKED.SIPS.zip') as packed:
    for sips_file in packed:
        for line in sips_file:
            print sips_file.stats.progress
            print line
        print packed.stats.progress

close()¶: Close the file descriptor.

class sippers.file.PackedSipsFileStats(size, n_files)¶

Stats for a Packed SIPS file (with zip).

progress¶

Progress in % and the number of files read.

Format example: 80% (8/10)

class sippers.file.SipsFile(path, fd=None, parser=None, strict=False, resume=None)¶

SIPS file.

Process of content of file is processed with iterators to keep the minimal memory footprint.

Example:

with SipsFile('/tmp/SIPS.TXT') as sips_file:
    for line in sips_file:
        print sips_file.stats.progress
        print line

Parameters:	path – Path of SIPS file fd – File descriptor (use this if you have already opened the file) parser – Force to use a parser strict – Strict schema validation resume (SipsFileStats) – Resume an imported file passing a SipsFileStats

class sippers.file.SipsFileStats(size)¶

Stats for a SIPS file

elapsed_time¶: Elapsed time processing the file.

progress¶: Progress in % of read content.

size¶: Human readeable size of the SIPS file.

Adapters¶

class sippers.adapters.MeasuresAdapter(extra=None, only=None, exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)¶: Base Measures Adapter.

class sippers.adapters.SipsAdapter(extra=None, only=None, exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)¶: Base SIPS Adapter.

sippers.adapters.pre_insert(fn=None, raw=False)¶

Filter to use before inserting the document to the database.

Useful when you must to do some operation between the parsed data and the already inserted data.

Example for Hidrocantabrico when the measures file doesn’t have the CUPS code and have the internal contract code:

@pre_insert
def fix_name(self, data):
    backend = self.backend
    result = backend.get(self.backend.ps_collection, {
        'ref': data['name'], 'cod_distri': '0026'}
    )
    if result:
        data['name'] = result[0]['name']
    return data

Endesa¶

endesa.TARIFFS = {'20A': '2.0A', '20DHA': '2.0DHA', '20DHS': '2.0DHS', '21A': '2.1A', '21DHA': '2.1DHA', '21DHS': '2.1DHS', '30A': '3.0A', '31A': '3.1A', '61': '6.1', '61A': '6.1A', '61B': '6.1B', '62': '6.2', '63': '6.3', '64': '6.4', '65': '6.5'}¶

class sippers.adapters.endesa.EndesaBaseAdapter(extra=None, only=None, exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)¶

Endesa SIPS Adapter

fix_dates(data)¶

Fix the dates in the SIPS file from ENDESA.

In the endesa sips file dates are YYYYMMDD or 0 or 00000000. With this all fields.DateTime fields are caugth and parsed to a correct format YYYY-MM-DDT00:00:00.

fix_floats(data)¶

Fix floats numbers.

Replace , to .

class sippers.adapters.endesa.EndesaMeasuresAdapter(extra=None, only=None, exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)¶

class sippers.adapters.endesa.EndesaSipsAdapter(extra=None, only=None, exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)¶

Endesa SIPS Adapter.

adapt_tarifa(data)¶

Fix the ATR Tariff code

Using TARIFFS

sippers.adapters.endesa.TARIFFS = {'20A': '2.0A', '20DHA': '2.0DHA', '20DHS': '2.0DHS', '21A': '2.1A', '21DHA': '2.1DHA', '21DHS': '2.1DHS', '30A': '3.0A', '31A': '3.1A', '61': '6.1', '61A': '6.1A', '61B': '6.1B', '62': '6.2', '63': '6.3', '64': '6.4', '65': '6.5'}: Mapping of ENDESA tariffs codes.

Iberdrola¶

class sippers.adapters.iberdrola.IberdrolaMeasuresAdapter(extra=None, only=None, exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)¶

class sippers.adapters.iberdrola.IberdrolaSipsAdapter(extra=None, only=None, exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)¶

Hidrocantábrico¶

class sippers.adapters.hidrocantabrico.HidrocantabricoMeasuresAdapter(extra=None, only=None, exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)¶

class sippers.adapters.hidrocantabrico.HidrocantabricoSipsAdapter(extra=None, only=None, exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)¶

Backends¶

sippers.backends.get_backend(url)¶

Get the backend class by and URL.

Parameters:	url – URL for identify a backend.

sippers.backends.register(name, cls)¶

Register a backend

Use this function to register a Backend class for and schema. If you want to register your own backend you can do:

class YourAwesomeBackend(BaseBackend):
    pass

register('awesome', YourAwsomeBackend)

Then with the URL awesome://user:pass@host/db with the function get_backend()

Parameters:	name – Backend name cls (class) – Backend class

Mongodb¶

class sippers.backends.mongodb.MongoDBBackend(uri=None)¶

MongoDB Backend

insert_cnmc_measure(value)¶: cnmc measures come a measure per line, cannot replace the whole block as in insert_measures

Models¶

class sippers.models.Document(data, adapter)¶

Document object

Parameters:	data – Data parsed. adapter – Adapter used to parse this data.

This document is used to encapsulated a object in sippers.parsers.parser.Parser.parse_line()

backend_data¶: Get data after using the filter sippers.adapters.pre_insert()

class sippers.models.MeasuresSchema(extra=None, only=None, exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)¶: Base model for measures.

class sippers.models.SipsSchema(extra=None, only=None, exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)¶: Base model for SIPS

Endesa¶

Iberdrola¶

Hidrocantábrico¶

Parsers¶

class sippers.parsers.parser.Parser¶

Base parser interface.

parse_line(line)¶

Parse a line of a SIPS file.

Parameters:	line – line of the file

sippers.parsers.parser.register(cls)¶

Register a parser

Endesa¶

Iberdrola¶

Hidrocantábrico¶

Utils¶

Bits & Bytes related humanization.

sippers.utils.build_dict(headers, data)¶

Build a dict with headers and list of data.

Example:

build_dict(['foo', 'bar'], [1, 2])
{'foo': 1, 'bar': 2}

Parameters:	headers (list) – List of headers data (list) – List of data

sippers.utils.naturalsize(value, binary=False, gnu=False, format='%.1f')¶: Format a number of byteslike a human readable filesize (eg. 10 kB). By default, decimal suffixes (kB, MB) are used. Passing binary=true will use binary suffixes (KiB, MiB) are used and the base will be 2**10 instead of 10**3. If gnu is True, the binary argument is ignored and GNU-style (ls -sh style) prefixes are used (K, M) with the 2**10 definition. Non-gnu modes are compatible with jinja2’s filesizeformat filter.

Logging¶

sippers.logging¶

Implements the logging support for SIPPERS

You can use logging everywhere using:

from sippers import logger
logger.info('Info message')

sippers.logging.setup_logging(level=None, logfile=None)¶

Setups sippers logging system.

It will setup sentry logging if SENTRY_DSN environment is defined

Parameters:	level – logging.LEVEL to set to logger (defaults INFO) logfile – File to write the log
Returns:	logger

API¶

SIPS files¶

Adapters¶

Endesa¶

Iberdrola¶

Hidrocantábrico¶

Backends¶

Mongodb¶

Models¶

Endesa¶

Iberdrola¶

Hidrocantábrico¶

Parsers¶

Endesa¶

Iberdrola¶

Hidrocantábrico¶

Utils¶

Logging¶

sippers.logging¶

Table Of Contents

Related Topics

This Page