Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add multiplayer mode for coupled runs #1335

Closed
wants to merge 5 commits into from

Conversation

orichters
Copy link
Contributor

@orichters orichters commented Jun 15, 2023

Purpose of this PR

  • REMIND-MAgPIE cascades with more than 4 scenarios cannot be run on priority slots by a single user, given the PIK cluster limitations. This helper scripts allows to run a single cascade with multiple users.
  • Simply set qos to multiplayer, start your runs, and you and one or more colleagues run Rscript scripts/utils/multiplayer.R
  • If the current user running a run has no priority free, the sbatch command that needs to be executed is pasted into scripts/utils/multiplayer.sh. The multiplayer.R scripts checks whether there are commands to be executed and how many priority slots are free, and then runs as many as possible. Afterwards, it starts a slurm job that will do the same 15 minutes later.
  • Use --mem=8000 option only if running on 1 CPU. I asked in the RSE channel a while ago whether adding it would harm, and it does:
slurmstepd: error: Job 26812886 exceeded memory limit (8344256 > 8192000), being killed
slurmstepd: error: Exceeded job memory limit
slurmstepd: error: *** JOB 26812886 ON cs-f14c02b10 CANCELLED AT 2023-06-16T00:37:19 ***

Type of change

  • New feature
  • Requires documentation update (not done yet)

Checklist:

  • My code follows the coding etiquette
  • I performed a self-review of my own code
  • I explained my changes within the PR, particularly in hard-to-understand areas
  • I checked that the in-code documentation is up-to-date
  • All automated model tests pass (FAIL 0 in the output of make test)

Copy link
Contributor

@dklein-pik dklein-pik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, Olli, for this cool feature! Found one thing, see comment.

start_coupled.R Outdated Show resolved Hide resolved
@orichters orichters force-pushed the startscripts branch 3 times, most recently from a8fbbd2 to 824fb3a Compare June 16, 2023 10:30
start_bundle_coupled.R Outdated Show resolved Hide resolved
@orichters orichters marked this pull request as ready for review June 19, 2023 13:27
@orichters orichters force-pushed the startscripts branch 2 times, most recently from 4ca7d26 to 4b17f78 Compare June 19, 2023 14:03
@orichters
Copy link
Contributor Author

Not desired as discussed with @dklein-pik and @LaviniaBaumstark.

@orichters orichters closed this Jul 10, 2023
@orichters orichters deleted the startscripts branch December 6, 2023 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants