Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Can't run pyscenic grn #24

Closed
f6v opened this issue Sep 6, 2020 · 1 comment
Closed

Can't run pyscenic grn #24

f6v opened this issue Sep 6, 2020 · 1 comment

Comments

@f6v
Copy link

f6v commented Sep 6, 2020

First off, thanks for publishing the package!

I'm having trouble running the example notebook though. I've got the error when running

!pyscenic grn {f_loom_path_scenic} {f_tfs} -o adj.csv --num_workers 20
I'm on python 3.6.11 and macOS 10.15.6. I've installed the protocol using Conda.

Is this something you have seen?

/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/dask/config.py:161: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  data = yaml.load(f.read()) or {}
OMP: Info #270: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.

2020-09-06 22:29:53,586 - pyscenic.cli.pyscenic - INFO - Loading expression matrix.

2020-09-06 22:29:55,980 - pyscenic.cli.pyscenic - INFO - Inferring regulatory networks.
/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/dask/config.py:161: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  data = yaml.load(f.read()) or {}
OMP: Info #270: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
OMP: Info #270: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
OMP: Info #270: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
OMP: Info #270: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
OMP: Info #270: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
OMP: Info #270: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
OMP: Info #270: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
OMP: Info #270: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
OMP: Info #270: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
OMP: Info #270: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
OMP: Info #270: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
OMP: Info #270: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
OMP: Info #270: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
OMP: Info #270: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
OMP: Info #270: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
OMP: Info #270: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
OMP: Info #270: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
OMP: Info #270: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
OMP: Info #270: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
OMP: Info #270: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
preparing dask client
parsing input
/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/arboreto/algo.py:214: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead.
  expression_matrix = expression_data.as_matrix()
creating dask graph
distributed.protocol.core - CRITICAL - Failed to deserialize
Traceback (most recent call last):
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/distributed/protocol/core.py", line 108, in loads
    header = msgpack.loads(header, use_list=False, **msgpack_opts)
  File "msgpack/_unpacker.pyx", line 195, in msgpack._cmsgpack.unpackb
ValueError: tuple is not allowed for map key
distributed.core - ERROR - tuple is not allowed for map key
Traceback (most recent call last):
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/distributed/core.py", line 347, in handle_comm
    msg = yield comm.read()
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/tornado/gen.py", line 735, in run
    value = future.result()
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/tornado/gen.py", line 742, in run
    yielded = self.gen.throw(*exc_info)  # type: ignore
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/distributed/comm/tcp.py", line 218, in read
    frames, deserialize=self.deserialize, deserializers=deserializers
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/tornado/gen.py", line 735, in run
    value = future.result()
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/tornado/gen.py", line 209, in wrapper
    yielded = next(result)
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/distributed/comm/utils.py", line 85, in from_frames
    res = _from_frames()
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/distributed/comm/utils.py", line 71, in _from_frames
    frames, deserialize=deserialize, deserializers=deserializers
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/distributed/protocol/core.py", line 108, in loads
    header = msgpack.loads(header, use_list=False, **msgpack_opts)
  File "msgpack/_unpacker.pyx", line 195, in msgpack._cmsgpack.unpackb
ValueError: tuple is not allowed for map key
distributed.protocol.core - CRITICAL - Failed to deserialize
Traceback (most recent call last):
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/distributed/protocol/core.py", line 108, in loads
    header = msgpack.loads(header, use_list=False, **msgpack_opts)
  File "msgpack/_unpacker.pyx", line 195, in msgpack._cmsgpack.unpackb
ValueError: tuple is not allowed for map key
not shutting down client, client was created externally
finished
Traceback (most recent call last):
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/bin/pyscenic", line 8, in <module>
    sys.exit(main())
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/pyscenic/cli/pyscenic.py", line 421, in main
    args.func(args)
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/pyscenic/cli/pyscenic.py", line 74, in find_adjacencies_command
    network = method(expression_data=ex_mtx, tf_names=tf_names, verbose=True, client_or_address=client, seed=args.seed)
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/arboreto/algo.py", line 41, in grnboost2
    early_stop_window_length=early_stop_window_length, limit=limit, seed=seed, verbose=verbose)
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/arboreto/algo.py", line 128, in diy
    seed=seed)
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/arboreto/core.py", line 403, in create_graph
    future_tf_matrix = client.scatter(tf_matrix, broadcast=True)
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/distributed/client.py", line 2071, in scatter
    hash=hash,
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/distributed/client.py", line 753, in sync
    return sync(self.loop, func, *args, **kwargs)
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/distributed/utils.py", line 331, in sync
    six.reraise(*error[0])
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/six.py", line 703, in reraise
    raise value
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/distributed/utils.py", line 316, in f
    result[0] = yield future
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/tornado/gen.py", line 735, in run
    value = future.result()
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/tornado/gen.py", line 742, in run
    yielded = self.gen.throw(*exc_info)  # type: ignore
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/distributed/client.py", line 1916, in _scatter
    timeout=timeout,
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/tornado/gen.py", line 735, in run
    value = future.result()
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/tornado/gen.py", line 742, in run
    yielded = self.gen.throw(*exc_info)  # type: ignore
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/distributed/core.py", line 739, in send_recv_from_rpc
    result = yield send_recv(comm=comm, op=key, **kwargs)
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/tornado/gen.py", line 735, in run
    value = future.result()
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/tornado/gen.py", line 742, in run
    yielded = self.gen.throw(*exc_info)  # type: ignore
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/distributed/core.py", line 533, in send_recv
    response = yield comm.read(deserializers=deserializers)
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/tornado/gen.py", line 735, in run
    value = future.result()
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/tornado/gen.py", line 742, in run
    yielded = self.gen.throw(*exc_info)  # type: ignore
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/distributed/comm/tcp.py", line 218, in read
    frames, deserialize=self.deserialize, deserializers=deserializers
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/tornado/gen.py", line 735, in run
    value = future.result()
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/tornado/gen.py", line 209, in wrapper
    yielded = next(result)
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/distributed/comm/utils.py", line 85, in from_frames
    res = _from_frames()
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/distributed/comm/utils.py", line 71, in _from_frames
    frames, deserialize=deserialize, deserializers=deserializers
  File "/Users/f6v/opt/miniconda3/envs/scenic_protocol/lib/python3.6/site-packages/distributed/protocol/core.py", line 108, in loads
    header = msgpack.loads(header, use_list=False, **msgpack_opts)
  File "msgpack/_unpacker.pyx", line 195, in msgpack._cmsgpack.unpackb
ValueError: tuple is not allowed for map key
@cflerin
Copy link
Contributor

cflerin commented Sep 8, 2020

Hi @f6v ,

I haven't seen this OMP error before, but the "tuple is not allowed..." is a known issue. See aertslab/pySCENIC/issues/163 for details, but my first suggestion would be to use the multiprocessing script here. This should allow you to complete the grn step.

@cflerin cflerin closed this as completed Sep 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@cflerin @f6v