mep-jobcontrol-callback-python: PROBA-V MEP Python JobControl client

This Python client uses the REST service of the PROBA-V MEP JobControl framework, allowing Python jobs to push information about jobruns, jobrun actions and log files to the PROBA-V MEP JobControl framework.

Installation

The package is available in the public PROBA-V MEP PyPi repository and can be easily installed using pip:

$ pip install jobcontrol-callback

When you are using a PROBA-V MEP Virtual Machine (VM), the package is already pre-installed for you.

Usage

Example registration of jobrun:

>>> from jobcontrol import callback
>>> cbc = callback.CallbackClient()
>>> jobrun = cbc.register_jobrun('application_1492674955132_0214', 'daemsd')

Example registration of jobrun action:

>>> jobrun_action = cbc.register_jobrun_action(jobrun.id_, 'pre-processing')

When adding the JobControlHandler to the log handlers, logging will be pushed to the JobControl framework, making it available through the JobControl dashboard.

>>> import logging
>>> from jobcontrol import loghandler
>>> logger = logging.getLogger(__name__)
>>> logger.addHandler(loghandler.jobcontrolhandler.JobControlHandler(jobrun.id_, jobrun_action.id_))
>>> logger.log('Started processing tile X10Y20')

API

This module allows registration and management of jobruns, jobrun actions and log files

class jobcontrol.callback.CallbackClient(baseurl='http://jobcontrol.vgt.vito.be:8090')

Client allowing to register jobruns, jobrun actions and logs.

add_jobrun_action_log(jobrun_id, jobrun_action_id, message)

Add a log entry for a jobrun action

add_jobrun_action_product_log(jobrun_id, jobrun_action_id, input_product_ref, message)

Add a log entry for a jobrun action on a specific input product

jobrun_action_failed(jobrun_id, jobrun_action_id, time=None)

Change the state of a jobrun action to FAILED

jobrun_action_finished(jobrun_id, jobrun_action_id, time=None)

Change the state of a jobrun action to FINISHED

jobrun_action_product_failed(jobrun_id, jobrun_action_id, input_product)

Change the state of a jobrun action input product to RUNNING

jobrun_action_product_finished(jobrun_id, jobrun_action_id, input_product)

Change the state of a jobrun action input product to RUNNING

jobrun_action_product_running(jobrun_id, jobrun_action_id, input_product)

Change the state of a jobrun action input product to RUNNING

jobrun_failed(jobrun_id, time=None)

Change the state of a jobrun to FAILED

jobrun_finished(jobrun_id, time=None)

Change the state of a jobrun to FINISHED

register_jobrun(job_id, username='docs', start_time=None)

Register a jobrun

register_jobrun_action(jobrun_id, name, start_time=None, input_products=None)

Register a jobrun action

class jobcontrol.callback.InputProduct(reference, timestamp=None, geometry=None)

This class represents a jobrun action input product.

class jobcontrol.callback.InputProductEncoder(skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, encoding='utf-8', default=None)

This class JSON encodes an InputProduct.

class jobcontrol.callback.JobRun(id_, job_id, username, state, start_time, end_time=None)

This class represents a jobrun.

static build(json_input)

Build a JobRun object from JSON input.

class jobcontrol.callback.JobRunAction(id_, jobrun_id, name, state, start_time, end_time=None)

This class represents a jobrun action.

add_input_product(input_product)

Add input product to list of action input products.

static build(json_input)

Build a JobRunAction object from JSON input.

This module provides a custom log handler to push logs to ElasticSearch

class jobcontrol.loghandler.jobcontrolhandler.JobControlHandler(jobrun_id, jobrun_action_id, input_product=None, level=0, baseurl='http://jobcontrol.vgt.vito.be:8090')

Logging handler which pushes logs to the JobControl REST service, persisting them in ElasticSearch

Indices and tables