cdapsutil package
Community Detection
- class cdapsutil.cd.CommunityDetection(runner=<cdapsutil.runner.ServiceRunner object>)[source]
Runs Community Detection Algorithms packaged as Docker containers built for CDAPS service via
Runner
- Parameters:
runner (
Runner
) – Object used to run CommunityDetection algorithm.- Raises:
CommunityDetectionError – If runner is
None
- run_community_detection(net_cx, algorithm=None, temp_dir=None, arguments=None, weight_col=None, default_weight=None)[source]
Generates hierarchy network by running community detection algorithm specified by algorithm parameter on net_cx network.
- Parameters:
net_cx (
ndex2.nice_cx_network.NiceCXNetwork
) – Network to run community detection onalgorithm (str) – Name of algorithm to run. Depending on the
Runner
used this can be an algorithm name, a file, or a Docker imagetemp_dir (str) – Path to temporary directory used by some of the
Runner
runnersarguments (dict) – Any custom parameters for algorithm. The parameters should all be of type
str
If custom parameter is just a flag set value toNone
Example:{'--flag': None, '--cutoff': '0.2'}
weight_col (str) – Name of column containing weights, Set to
None
for unweighteddefault_weight (float) – Default weight value for edges where no weight was found. Only applicable if weight_col parameter is set. WARNING This not yet supported and will raise a
CommunityDetectionError
- Raises:
CommunityDetectionError – If there was an error running the algorithm or if weight_col parameter is set which is not yet supported
- Returns:
Hierarchy network
- Return type:
ndex2.nice_cx_network.NiceCXNetwork
Functional Enrichment
Runners
Runners provide different ways to run Community Detection Algorithms packaged as Docker containers built for CDAPS service.
Currently built Runners:
ExternalResultsRunner
- Parses already run output file/streamDockerRunner
- Runs locally via DockerServiceRunner
- Runs remotely via CDAPS REST Service
- class cdapsutil.runner.DockerRunner(binary_path='docker', processwrapper=<cdapsutil.runner.ProcessWrapper object>)[source]
Runner
that runs CDAPS Service Docker containers locally via Docker- Parameters:
binary_path (str) – Full path to Docker command
processwrapper (
ProcessWrapper
) – Object to run external process
- run(net_cx=None, algorithm=None, arguments=None, temp_dir=None)[source]
Runs Docker command returning a tuple with error code, standard out and standard error
- Parameters:
net_cx (
ndex2.nice_cx_network.NiceCXNetwork
) – Network to use as inputalgorithm (str) – docker image to run
arguments (dict) – Any custom parameters for algorithm. The parameters should all be of type
str
If custom parameter is just a flag set value toNone
Example:{'--flag': None, '--cutoff': '0.2'}
temp_dir (str) – temporary directory where docker can be run this should be a directory that docker can access when -v X:X flag is added to docker command
- Raises:
CommunityDetectionError – If there is an error in running job outside of non-zero exit code from command
- Returns:
(return code, stdout from subprocess, stderr from subprocess)
- Return type:
tuple
- class cdapsutil.runner.ExternalResultsRunner[source]
Runner
returns an already generated result set via algorithm parameter inrun()
method. This allows results generated externally to be processed.- run(net_cx=None, algorithm=None, arguments=None, temp_dir=None)[source]
Assumes algorithm contains path to file with result of invocation of algorithm. This allows for externally run algorithms to be loaded into this library.
A tuple is returned with return code,result and standard error of result.
In this implementation the return code is always
0
unless the contents of the file is in format matching result from CDAPS REST Service in which case a1
returncode may be sent if the status was NOTcomplete
. Any data inmessage
field will be set in the standard error of result.- Parameters:
net_cx (
ndex2.nice_cx_network.NiceCXNetwork
) – Ignoredalgorithm (str) – Path to file with results
arguments (dict) – Ignored
temp_dir (str) – Ignored
- Raises:
CommunityDetectionError – If there is an error in running job outside of non-zero exit code from command
- Returns:
(return code, result, error message)
- Return type:
tuple
- class cdapsutil.runner.ServiceRunner(service_endpoint='http://cdservice.cytoscape.org/cd/communitydetection/v1', requests_timeout=30, max_retries=600, poll_interval=1)[source]
Runner
that runs CDAPS Service containers remotely via CDAPS Service- Parameters:
service_endpoint (str) – URL for CDAPS REST Service
requests_timeout (int or float) – Timeout in seconds to pass to
requests
module for all web requestsmax_retries (int) – Number of times to check for task completion
poll_interval (int) – Time to wait in seconds between checks for task completion
- REST_ENDPOINT = 'http://cdservice.cytoscape.org/cd/communitydetection/v1'
Default Rest endpoint
- USER_AGENT_KEY = 'UserAgent'
User Agent Header label
- get_algorithms()[source]
Queries CDAPS service for list of available algorithms
Example result (only showing one algorithm):
{ "algorithms": { "name": "hidef", "displayName": "HiDeF", "description": "...", "version": "0.2.2", "dockerImage": "coleslawndex/cdhidef:0.2.2", "inputDataFormat": "EDGELISTV2", "outputDataFormat": "COMMUNITYDETECTRESULTV2", "customParameters": [ { "name": "--maxres", "displayName": "Maximum resolution parameter", "description": "Maximum resolution parameter. Increase to get more smaller communities", "type": "value", "defaultValue": "50.0", "validationType": "number", "validationHelp": "Should be a number", "validationRegex": null, "minValue": null, "maxValue": null }, { "name": "--alg", "displayName": "Algorithm", "description": "Choose to use Louvain or newer Leiden algorithm", "type": "value", "defaultValue": "louvain", "validationType": "string", "validationHelp": "Must be set to louvain or leiden", "validationRegex": "louvain|leiden", "minValue": null, "maxValue": null } ] } }
- Raises:
CommunityDetectionError – If there is an error
- Returns:
Algorithms available from service in example format shown above
- Return type:
dict
- get_result(task_id)[source]
Gets result from CDAPS service
- Parameters:
task_id (str) – Id of task
- Returns:
Result from service
- Return type:
dict
- run(net_cx=None, algorithm=None, arguments=None, temp_dir=None)[source]
Runs ‘algorithm’ via CDAPS service with error code, standard out and standard error derived from the service call
- Parameters:
net_cx (
ndex2.nice_cx_network.NiceCXNetwork
) – Network to use as inputalgorithm (str) – Algorithm to run
arguments (dict) – Any custom parameters for algorithm. The parameters should all be of type
str
If custom parameter is just a flag set value toNone
Example:{'--flag': None, '--cutoff': '0.2'}
temp_dir (str) – Ignored
- Raises:
CommunityDetectionError – If there is an error in running job outside of non-zero exit code from command
- Returns:
(return code, stdout from subprocess, stderr from subprocess)
- Return type:
tuple
- submit(algorithm=None, data=None, arguments=None)[source]
Submits algorithm to CDAPS service with endpoint set in constructor
- Parameters:
algorithm (str) – Name of algorithm to call
data – The data to pass to the algorithm
arguments (dict) – Any custom parameters for algorithm. The parameters should all be of type
str
If custom parameter is just a flag set value toNone
Example:{'--flag': None, '--cutoff': '0.2'}
- Returns:
Task id in this format
{'id': '<TASK ID'}
- Return type:
dict
- wait_for_task_to_complete(task_id, poll_interval=1, consecutive_fail_retry=5, max_retries=None)[source]
Waits for task with task_id id to complete.
- Parameters:
task_id (str) – Id of task
poll_interval (int) – How long to wait in seconds before checking again if task is complete
consecutive_fail_retry (int) – If the number of consecutive failure calls to get status exceeds this value an exception is raised
max_retries (int) – Total number of checks to perform before raising an exception. For example, if max_retries is set to
600
and poll_interval is1
then this method will wait 10 minutes for task to complete checking 600 times, or once per second. NOTE: If set to``None`` this method will poll indefinitely
- Raises:
CommunityDetectionError – If task_id is
None
, if max_fail_retry is exceeded, if max_retries is exceeded- Returns:
status response of completed task
- Return type:
dict
- class cdapsutil.runner.Runner[source]
Base class for objects that run Community Detection Algorithms packaged as Docker containers built for CDAPS service via various means.
Currently built Runners:
ExternalResultsRunner
- Parses already run output file/streamDockerRunner
- Runs locally via DockerServiceRunner
- Runs remotely via CDAPS Service- get_algorithm_name()[source]
Gets the algorithm name
- Returns:
Name of algorithm or empty string if unknown
- Return type:
str
- get_algorithms()[source]
Will always raise since subclasses should implement this
CommunityDetectionError
- Raises:
CommunityDetectionError – Will always raise this
- get_docker_image()[source]
Gets the name of the Docker image
- Returns:
Name of Docker image or empty string if unknown
- Return type:
str
- run(net_cx=None, algorithm=None, arguments=None, temp_dir=None)[source]
Must be implemented by subclasses. Will always raise
cdapsutil.exceptions.CommunityDetectionError
- Parameters:
net_cx (
ndex2.nice_cx_network.NiceCXNetwork
) – Network to use as inputalgorithm –
arguments – Any custom parameters for algorithm. The parameters should all be of type
str
If custom parameter is just a flag set value toNone
Example:{'--flag': None, '--cutoff': '0.2'}
temp_dir –
- Raises:
CommunityDetectionError – Will always raise this
- Returns:
None