[Moabusers] How to keep non-preemptor jobs from running on top of preempted jobs?

[Douglas Wightman]

The problem stems from the fact that suspended jobs have no
reservations.  This means that jobs looking to run after the preemptor
job don't factor in the suspended job.

You can currently get around this problem if you give the preemptee jobs
the preemptor flag as well.  That way they can preempt the idle job
reservations.  However, it would probably make sense for us to allow
suspended jobs to preempt idle job reservations without the need for
setting a flag.

- Douglas

On Wed, 2007-10-31 at 17:46 -0400, Troy Baer wrote:
> Hello all,
> 
> I'm having a quirky problem WRT preemption.  On one of my systems, I
> have a job class called "serial" in which jobs are allowed to be
> preempted under certain circumstances, one of which is a job from user
> "special".  So a bunch of "serial" jobs are running, user "special"
> submits a job, some of the "serial" jobs are suspended, and the job from
> user "special" runs.  However, when the job from user "special" ends,
> not all of the "serial" jobs that were suspended wake back up; instead,
> other jobs that *aren't* marked as preemptors are run on those nodes
> instead.  This makes the users with the suspended jobs grumpy, and
> justifiably so.
> 
> Is there a way to prevent this?  (What I'd like is a way to tell Moab
> that if there's a preempted job on a node, the *only* jobs allowed to be
> run on that node are jobs marked as preemptors.)
> 
> Relevant moab.cfg entries:
> 
> RMCFG[torque]           SUSPENDSIG=suspend
> PREEMPTPOLICY           SUSPEND
> JOBPREEMPTMINACTIVETIME 0:01:00
> SUSPENDRESOURCES[ALL]   SWAP,DISK
> QOSCFG[preemptee]       QFLAGS=PREEMPTEE JOBFLAGS=SUSPENDABLE
> QOSCFG[preemptor]       QFLAGS=PREEMPTOR
> CLASSCFG[serial]        QDEF=preemptee&
> USERCFG[special]        QDEF=preemptor
> 
> This is using TORQUE 2.1.8 and Moab 5.1.0p2, BTW; I haven't had a chance
> to upgrade to the latest and greatest Moab yet on this system.
> 
> 	--Troy