[Moabusers] Standing Reservation for GRES devices

[Gareth.Williams at csiro.au]

> -----Original Message-----
> From: moabusers-bounces at supercluster.org [mailto:moabusers-
> bounces at supercluster.org] On Behalf Of Derek Gottlieb
> Sent: Friday, 23 January 2009 6:45 AM
> To: moabusers at supercluster.org
> Subject: [Moabusers] Standing Reservation for GRES devices
> 
> I've got a few GPUs in one of our clusters and I'm trying to figure out
> how to set up a reservation that would give GPU jobs priority access to
> those GPUs and associated CPUs.  I currently have 2 GPUs per node
> attached to nodes with 8 CPU cores and 64GB of memory.  Is there a way
> in Moab to specify that I'd like to reserve 2 CPUs and 16GB of memory on
> each node that has GPUs and keep these reserved for jobs that explicitly
> request GPUs?
> 
> Here's what I've been starting with:
> 
> NODETOJOBATTRMAP gpu
> SRCFG[gpusr] HOSTLIST=^gpuhost1$,^gpuhost2$
> SRCFG[gpusr] PERIOD=INFINITY
> SRCFG[gpusr] JOBATTRLIST=gpu
> #SRCFG[gpusr] TASKCOUNT=2 RESOURCES=PROCS:1,MEM:8192
> #SRCFG[gpusr] TASKCOUNT=4 RESOURCES=PROCS:1,MEM:8192
> 
> NODECFG[gpuhost1] FEATURES+=gpu GRES=tesla:2
> NODECFG[gpuhost2] FEATURES+=gpu GRES=tesla:2
> 
> 
> Problem is I'm not real sure how to reserve 2 CPUs on each of these
> nodes without setting up multiple SRs.  I would think if I put in the
> TASKCOUNT=4 line that I'd end up with 4 CPUs on one node and not reserve
> anything on the other.  I'd like to only reserve 1 CPU/1 GPU as opposed
> to reserving all CPUS on my GPU nodes for GPU jobs and leave them
> largely idle.
> 
> Any input would be appreciated,
> Derek

Hi Derek,

Our group may have a similar requirement within the next year.

It would be nice if one could add a GRES:tesla requirement to the resources
Perhaps something like:
SRCFG[gpusr] TASKCOUNT=4 RESOURCES=PROCS:1,MEM:8192,GRES:tesla:1

However the documentation does not indicate that such a thing is possible.  You could make a feature request to cluster resources.

I guess you could set up SRs for each node separately:
SRCFG[gpusr1] HOSTLIST=^gpuhost1$
SRCFG[gpusr1] JOBATTRLIST=gpu
SRCFG[gpusr1] TASKCOUNT=2 RESOURCES=PROCS:1,MEM:8192
SRCFG[gpusr2] HOSTLIST=^gpuhost2$
SRCFG[gpusr2] JOBATTRLIST=gpu
SRCFG[gpusr2] TASKCOUNT=2 RESOURCES=PROCS:1,MEM:8192
.
.
.

Ugly but perhaps workable!

Cheers,

Gareth Williams
Outreach Manager
CSIRO IM&T - ASC
Ph +61 3 8601 3804 (CSIRO)
Ph +61 3 9669 8114 (BoM)
http://intra.hpsc.csiro.au
calendar: http://tinyurl.com/3mkl5d

> _______________________________________________
> moabusers mailing list
> moabusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/moabusers