cdapsutil package

Community Detection

class cdapsutil.cd.CommunityDetection(runner=<cdapsutil.runner.ServiceRunner object>)[source]

Runs Community Detection Algorithms packaged as Docker containers built for CDAPS service via Runner

Parameters:: runner (Runner) – Object used to run CommunityDetection algorithm.
Raises:: CommunityDetectionError – If runner is None

run_community_detection(net_cx, algorithm=None, temp_dir=None, arguments=None, weight_col=None, default_weight=None, uuid=None)[source]

Generates hierarchy network by running community detection algorithm specified by algorithm parameter on net_cx network.

Parameters:

net_cx (ndex2.nice_cx_network.NiceCXNetwork or ndex2.cx2.CX2Network) – Network to run community detection on
algorithm (str) – Name of algorithm to run. Depending on the Runner used this can be an algorithm name, a file, or a Docker image
temp_dir (str) – Path to temporary directory used by some of the Runner runners
arguments (dict) – Any custom parameters for algorithm. The parameters should all be of type str If custom parameter is just a flag set value to None Example: {'--flag': None, '--cutoff': '0.2'}
weight_col (str) – Name of column containing weights, Set to None for unweighted
default_weight (float) – Default weight value for edges where no weight was found. Only applicable if weight_col parameter is set. WARNING This not yet supported and will raise a CommunityDetectionError
uuid (str) – UUID of the network (parent network of the hierarchy to be build) that is used to add HCX annotation. WARNING: The tool does not perform a check if the uuid is correct.

Raises:

CommunityDetectionError – If there was an error running the algorithm or if weight_col parameter is set which is not yet supported

Returns:

Hierarchy network

Return type:

ndex2.nice_cx_network.NiceCXNetwork

Functional Enrichment

Runners

Runners provide different ways to run Community Detection Algorithms packaged as Docker containers built for CDAPS service.

Currently built Runners:

ExternalResultsRunner - Parses already run output file/stream
DockerRunner - Runs locally via Docker
ServiceRunner - Runs remotely via CDAPS REST Service

class cdapsutil.runner.DockerRunner(binary_path='docker', processwrapper=<cdapsutil.runner.ProcessWrapper object>)[source]

Runner that runs CDAPS Service Docker containers locally via Docker

Parameters:

binary_path (str) – Full path to Docker command
processwrapper (ProcessWrapper) – Object to run external process

run(net_cx=None, algorithm=None, arguments=None, temp_dir=None)[source]

Runs Docker command returning a tuple with error code, standard out and standard error

Parameters:

net_cx (ndex2.nice_cx_network.NiceCXNetwork) – Network to use as input
algorithm (str) – docker image to run
arguments (dict) – Any custom parameters for algorithm. The parameters should all be of type str If custom parameter is just a flag set value to None Example: {'--flag': None, '--cutoff': '0.2'}
temp_dir (str) – temporary directory where docker can be run this should be a directory that docker can access when -v X:X flag is added to docker command

Raises:

CommunityDetectionError – If there is an error in running job outside of non-zero exit code from command

Returns:

(return code, stdout from subprocess, stderr from subprocess)

Return type:

tuple

class cdapsutil.runner.ExternalResultsRunner[source]

Runner returns an already generated result set via algorithm parameter in run() method. This allows results generated externally to be processed.

run(net_cx=None, algorithm=None, arguments=None, temp_dir=None)[source]

Assumes algorithm contains path to file with result of invocation of algorithm. This allows for externally run algorithms to be loaded into this library.

A tuple is returned with return code,result and standard error of result.

In this implementation the return code is always 0 unless the contents of the file is in format matching result from CDAPS REST Service in which case a 1 returncode may be sent if the status was NOT complete. Any data in message field will be set in the standard error of result.

Parameters:

net_cx (ndex2.nice_cx_network.NiceCXNetwork) – Ignored
algorithm (str) – Path to file with results
arguments (dict) – Ignored
temp_dir (str) – Ignored

Raises:

CommunityDetectionError – If there is an error in running job outside of non-zero exit code from command

Returns:

(return code, result, error message)

Return type:

tuple

class cdapsutil.runner.ServiceRunner(service_endpoint='http://cdservice.cytoscape.org/cd/communitydetection/v1', requests_timeout=30, max_retries=600, poll_interval=1)[source]

Runner that runs CDAPS Service containers remotely via CDAPS Service

Parameters:

service_endpoint (str) – URL for CDAPS REST Service
requests_timeout (int or float) – Timeout in seconds to pass to requests module for all web requests
max_retries (int) – Number of times to check for task completion
poll_interval (int) – Time to wait in seconds between checks for task completion

REST_ENDPOINT = 'http://cdservice.cytoscape.org/cd/communitydetection/v1': Default Rest endpoint

USER_AGENT_KEY = 'UserAgent': User Agent Header label

get_algorithms()[source]

Queries CDAPS service for list of available algorithms

Example result (only showing one algorithm):

{ "algorithms":
    {
      "name": "hidef",
      "displayName": "HiDeF",
      "description": "...",
      "version": "0.2.2",
      "dockerImage": "coleslawndex/cdhidef:0.2.2",
      "inputDataFormat": "EDGELISTV2",
      "outputDataFormat": "COMMUNITYDETECTRESULTV2",
      "customParameters": [
        {
          "name": "--maxres",
          "displayName": "Maximum resolution parameter",
          "description": "Maximum resolution parameter. Increase to get more smaller communities",
          "type": "value",
          "defaultValue": "50.0",
          "validationType": "number",
          "validationHelp": "Should be a number",
          "validationRegex": null,
          "minValue": null,
          "maxValue": null
        },
        {
          "name": "--alg",
          "displayName": "Algorithm",
          "description": "Choose to use Louvain or newer Leiden algorithm",
          "type": "value",
          "defaultValue": "louvain",
          "validationType": "string",
          "validationHelp": "Must be set to louvain or leiden",
          "validationRegex": "louvain|leiden",
          "minValue": null,
          "maxValue": null
        }
      ]
    }
}

Raises:: CommunityDetectionError – If there is an error
Returns:: Algorithms available from service in example format shown above
Return type:: dict

get_result(task_id)[source]

Gets result from CDAPS service

Parameters:: task_id (str) – Id of task
Returns:: Result from service
Return type:: dict

run(net_cx=None, algorithm=None, arguments=None, temp_dir=None)[source]

Runs ‘algorithm’ via CDAPS service with error code, standard out and standard error derived from the service call

Parameters:

net_cx (ndex2.nice_cx_network.NiceCXNetwork) – Network to use as input
algorithm (str) – Algorithm to run
arguments (dict) – Any custom parameters for algorithm. The parameters should all be of type str If custom parameter is just a flag set value to None Example: {'--flag': None, '--cutoff': '0.2'}
temp_dir (str) – Ignored

Raises:

CommunityDetectionError – If there is an error in running job outside of non-zero exit code from command

Returns:

(return code, stdout from subprocess, stderr from subprocess)

Return type:

tuple

submit(algorithm=None, data=None, arguments=None)[source]

Submits algorithm to CDAPS service with endpoint set in constructor

Parameters:

algorithm (str) – Name of algorithm to call
data – The data to pass to the algorithm
arguments (dict) – Any custom parameters for algorithm. The parameters should all be of type str If custom parameter is just a flag set value to None Example: {'--flag': None, '--cutoff': '0.2'}

Returns:

Task id in this format {'id': '<TASK ID'}

Return type:

dict

wait_for_task_to_complete(task_id, poll_interval=1, consecutive_fail_retry=5, max_retries=None)[source]

Waits for task with task_id id to complete.

Parameters:

task_id (str) – Id of task
poll_interval (int) – How long to wait in seconds before checking again if task is complete
consecutive_fail_retry (int) – If the number of consecutive failure calls to get status exceeds this value an exception is raised
max_retries (int) – Total number of checks to perform before raising an exception. For example, if max_retries is set to 600 and poll_interval is 1 then this method will wait 10 minutes for task to complete checking 600 times, or once per second. NOTE: If set to``None`` this method will poll indefinitely

Raises:

CommunityDetectionError – If task_id is None, if max_fail_retry is exceeded, if max_retries is exceeded

Returns:

status response of completed task

Return type:

dict

class cdapsutil.runner.Runner[source]

Base class for objects that run Community Detection Algorithms packaged as Docker containers built for CDAPS service via various means.

Currently built Runners:

ExternalResultsRunner - Parses already run output file/stream

DockerRunner - Runs locally via Docker

ServiceRunner - Runs remotely via CDAPS Service

get_algorithm_name()[source]

Gets the algorithm name

Returns:: Name of algorithm or empty string if unknown
Return type:: str

get_algorithms()[source]

Will always raise since subclasses should implement this CommunityDetectionError

Raises:: CommunityDetectionError – Will always raise this

get_docker_image()[source]

Gets the name of the Docker image

Returns:: Name of Docker image or empty string if unknown
Return type:: str

run(net_cx=None, algorithm=None, arguments=None, temp_dir=None)[source]

Must be implemented by subclasses. Will always raise cdapsutil.exceptions.CommunityDetectionError

Parameters:

net_cx (ndex2.nice_cx_network.NiceCXNetwork) – Network to use as input
algorithm
arguments – Any custom parameters for algorithm. The parameters should all be of type str If custom parameter is just a flag set value to None Example: {'--flag': None, '--cutoff': '0.2'}
temp_dir

Raises:

CommunityDetectionError – Will always raise this

Returns:

None

set_algorithm_name(algoname)[source]

Sets algorithm name

Parameters:: algoname (str) – Name of algorithm
Returns:

set_docker_image(docker_image)[source]: Sets Docker image :return:

class cdapsutil.runner.ProcessWrapper[source]

Runs command line process

run(cmd)[source]

Runs external process

Parameters:: cmd (list) – Command to run. Should be a list of arguments that include invoking command. For example to run ls -la pass in [‘ls’,’-la’]
Returns:: (return code, stdout from subprocess, stderr from subprocess)
Return type:: tuple

Exceptions

exception cdapsutil.exceptions.CommunityDetectionError[source]

Bases: Exception

Base class for CommunityDetection Errors