mep-jobcontrol-callback-python: PROBA-V MEP Python JobControl client¶
This Python client uses the REST service of the PROBA-V MEP JobControl framework, allowing Python jobs to push information about jobruns, jobrun actions and log files to the PROBA-V MEP JobControl framework.
Installation¶
The package is available in the public PROBA-V MEP PyPi repository and can be easily installed using pip:
$ pip install jobcontrol-callback
When you are using a PROBA-V MEP Virtual Machine (VM), the package is already pre-installed for you.
Usage¶
Example registration of jobrun:
>>> from jobcontrol import callback
>>> cbc = callback.CallbackClient()
>>> jobrun = cbc.register_jobrun('application_1492674955132_0214', 'daemsd')
Example registration of jobrun action:
>>> jobrun_action = cbc.register_jobrun_action(jobrun.id_, 'pre-processing')
When adding the JobControlHandler to the log handlers, logging will be pushed to the JobControl framework, making it available through the JobControl dashboard.
>>> import logging
>>> from jobcontrol import loghandler
>>> logger = logging.getLogger(__name__)
>>> logger.addHandler(loghandler.jobcontrolhandler.JobControlHandler(jobrun.id_, jobrun_action.id_))
>>> logger.log('Started processing tile X10Y20')
API¶
This module allows registration and management of jobruns, jobrun actions and log files
-
class
jobcontrol.callback.
CallbackClient
(baseurl='http://jobcontrol.vgt.vito.be:8090')¶ Client allowing to register jobruns, jobrun actions and logs.
-
add_jobrun_action_log
(jobrun_id, jobrun_action_id, message)¶ Add a log entry for a jobrun action
-
add_jobrun_action_product_log
(jobrun_id, jobrun_action_id, input_product_ref, message)¶ Add a log entry for a jobrun action on a specific input product
-
jobrun_action_failed
(jobrun_id, jobrun_action_id, time=None)¶ Change the state of a jobrun action to FAILED
-
jobrun_action_finished
(jobrun_id, jobrun_action_id, time=None)¶ Change the state of a jobrun action to FINISHED
-
jobrun_action_product_failed
(jobrun_id, jobrun_action_id, input_product)¶ Change the state of a jobrun action input product to RUNNING
-
jobrun_action_product_finished
(jobrun_id, jobrun_action_id, input_product)¶ Change the state of a jobrun action input product to RUNNING
-
jobrun_action_product_running
(jobrun_id, jobrun_action_id, input_product)¶ Change the state of a jobrun action input product to RUNNING
-
jobrun_failed
(jobrun_id, time=None)¶ Change the state of a jobrun to FAILED
-
jobrun_finished
(jobrun_id, time=None)¶ Change the state of a jobrun to FINISHED
-
register_jobrun
(job_id, username='docs', start_time=None)¶ Register a jobrun
-
register_jobrun_action
(jobrun_id, name, start_time=None, input_products=None)¶ Register a jobrun action
-
-
class
jobcontrol.callback.
InputProduct
(reference, timestamp=None, geometry=None)¶ This class represents a jobrun action input product.
-
class
jobcontrol.callback.
InputProductEncoder
(skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, encoding='utf-8', default=None)¶ This class JSON encodes an InputProduct.
-
class
jobcontrol.callback.
JobRun
(id_, job_id, username, state, start_time, end_time=None)¶ This class represents a jobrun.
-
static
build
(json_input)¶ Build a JobRun object from JSON input.
-
static
-
class
jobcontrol.callback.
JobRunAction
(id_, jobrun_id, name, state, start_time, end_time=None)¶ This class represents a jobrun action.
-
add_input_product
(input_product)¶ Add input product to list of action input products.
-
static
build
(json_input)¶ Build a JobRunAction object from JSON input.
-
This module provides a custom log handler to push logs to ElasticSearch
-
class
jobcontrol.loghandler.jobcontrolhandler.
JobControlHandler
(jobrun_id, jobrun_action_id, input_product=None, level=0, baseurl='http://jobcontrol.vgt.vito.be:8090')¶ Logging handler which pushes logs to the JobControl REST service, persisting them in ElasticSearch