Summing up the sizes of input files

[katevoss]

@jsotobroad commented on Fri Sep 22 2017

It would nice to have a way of summing up a list of floats like if you have a scatter task that you then want to gather the outputs of, its not straightforward on how you dynamically size that gather task. This is what we currently do (and if it's stupid let us know!)

scatter (bam in list_of_bams) {
  call MakeBam{
    inputs:
      bam
  }

  Float mapped_bam_size = size(MakeBam.output_bam, "GB")
}

call SumFloats {
  input:
    sizes = mapped_bam_size,
}

.....
task SumFloats {
  Array[Float] sizes

  command <<<
  python -c "print ${sep="+" sizes}"
  >>>
  output {
    Float total_size = read_float(stdout())
  }
  runtime {
    docker: "python:2.7"
    preemptible: preemptible_tries
  }
}

---

@geoffjentry commented on Fri Sep 22 2017

@jsotobroad Please file this in the wdl repo, not the cromwell repo

[cjllanwarne]

@katevoss @jsotobroad I think this is resolved by #169 - can we close this issue or are there other cases to cover?

[katevoss]

Fine by me, @jsotobroad?

[dinvlad]

Could you elaborate how #169 solves the general case of summing up Array[Float]? Perhaps we should revisit this.

[vdauwera]

Closing since the feature requested here was delivered by #169.

@dinvlad Despite the somewhat misleading original issue title, the scope of the actual request was limited to summing up input file sizes. If there is a more general use case that you're interested in that has not yet been addressed by other work I think it should be a new issue.