Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace configuration-copying Alpaka ESProducers with MoveToDeviceCache #47047

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

makortel
Copy link
Contributor

@makortel makortel commented Jan 7, 2025

PR description:

Before #43969 a common pattern to copy data from configuration to all devices was to use the EventSetup system. While that works, it adds more pieces to be managed and disconnects the configuration parameters of a module (in addition to slight abuse of EventSetup system as "a job" is not a real IOV). This PR proposes to replace such Alpaka ESProducers with the MoveToDeviceCache introduced in #43969.

Resolves cms-sw/framework-team#1129

PR validation:

Code compiles, matrix workflows 12834.{402,412,422} and 145.604 run with and without an NVIDIA GPU.

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 7, 2025

cms-bot internal usage

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 7, 2025

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-47047/43186

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 7, 2025

A new Pull Request was created by @makortel for master.

It involves the following packages:

  • CondFormats/DataRecord (db, alca)
  • CondFormats/EcalObjects (db, alca)
  • CondFormats/HcalObjects (db, alca)
  • HLTrigger/Configuration (hlt)
  • RecoLocalCalo/EcalRecProducers (reconstruction)
  • RecoLocalCalo/HcalRecProducers (reconstruction)
  • RecoParticleFlow/PFClusterProducer (reconstruction)

@Martin-Grunewald, @atpathak, @cmsbuild, @consuegs, @francescobrivio, @jfernan2, @mandrenguyen, @mmusich, @perrotta can you please review it and eventually sign? Thanks.
@JanChyczynski, @Martin-Grunewald, @PonIlya, @ReyerBand, @abdoulline, @apsallid, @argiro, @bsunanda, @denizsun, @felicepantaleo, @hatakeyamak, @lgray, @mariadalfonso, @missirol, @mmarionncern, @mmusich, @rchatter, @rovere, @rsreds, @salimcerci, @sameasy, @seemasharmafnal, @silviodonato, @thomreis, @tocheng, @wang0jin, @youyingli, @yuanchao this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

@makortel makortel force-pushed the useCachesInsteadOfESProducers branch from be750b4 to bac91f6 Compare January 7, 2025 00:08
@makortel
Copy link
Contributor Author

makortel commented Jan 7, 2025

@cms-sw/heterogeneous-l2

@makortel
Copy link
Contributor Author

makortel commented Jan 7, 2025

enable gpu

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 7, 2025

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-47047/43187

@makortel
Copy link
Contributor Author

makortel commented Jan 7, 2025

@cmsbuild, please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 7, 2025

Pull request #47047 was updated. @Martin-Grunewald, @atpathak, @consuegs, @francescobrivio, @jfernan2, @mandrenguyen, @mmusich, @perrotta can you please check and sign again.

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 7, 2025

+1

Size: This PR adds an extra 140KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-767973/43645/summary.html
COMMIT: bac91f6
CMSSW: CMSSW_15_0_X_2025-01-06-1100/el8_amd64_gcc12
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/47047/43645/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

GPU Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 24 differences found in the comparisons
  • DQMHistoTests: Total files compared: 7
  • DQMHistoTests: Total histograms compared: 53071
  • DQMHistoTests: Total failures: 874
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 52197
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
  • Checked 24 log files, 30 edm output root files, 7 DQM output files
  • TriggerResults: no differences found

@jfernan2
Copy link
Contributor

jfernan2 commented Jan 7, 2025

+1

@makortel
Copy link
Contributor Author

unhold

@cmsbuild cmsbuild removed the hold label Jan 10, 2025
@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-47047/43262

@cmsbuild
Copy link
Contributor

Pull request #47047 was updated. @Martin-Grunewald, @atpathak, @cmsbuild, @consuegs, @francescobrivio, @fwyzard, @jfernan2, @makortel, @mandrenguyen, @mmusich, @perrotta can you please check and sign again.

@makortel
Copy link
Contributor Author

@cmsbuild, please test

@cmsbuild
Copy link
Contributor

+1

Size: This PR adds an extra 24KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-767973/43722/summary.html
COMMIT: 8fd33a8
CMSSW: CMSSW_15_0_X_2025-01-10-1100/el8_amd64_gcc12
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/47047/43722/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 49
  • DQMHistoTests: Total histograms compared: 3818780
  • DQMHistoTests: Total failures: 61
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3818699
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
  • Checked 214 log files, 184 edm output root files, 49 DQM output files
  • TriggerResults: no differences found

GPU Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 24 differences found in the comparisons
  • DQMHistoTests: Total files compared: 7
  • DQMHistoTests: Total histograms compared: 53071
  • DQMHistoTests: Total failures: 867
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 52204
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
  • Checked 24 log files, 30 edm output root files, 7 DQM output files
  • TriggerResults: no differences found

@fwyzard
Copy link
Contributor

fwyzard commented Jan 10, 2025 via email

@perrotta
Copy link
Contributor

+1

@mmusich
Copy link
Contributor

mmusich commented Jan 12, 2025

@makortel this needs a rebase now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Replace configuration-only ESProducers with MoveToDeviceCache
7 participants