?
This sound like it's just a tough process to model. No matter what you do, you can't have it both ways such that you could deploy to either a subset of DevX servers to an arbitrary subset of QAX servers. The options you'll have to decide on are either:
use different workflows and different environments to separate Dev1 from Dev2, which will force releases to use a specific set of servers after a build is created for it
use abstract environments (Dev, QA) with server variables that resolve to different server groups (e.g. DevSet1, DevSet2, or whatever), and select the result either at release, build, or promotion time so it provides historical context with the release/build
In BuildMaster, the idea of an Environment (i.e. a different stage of testing) is meant to be abstracted from a Server (i.e. a physical/virtual machine), so whatever follows most closely to this paradigm would be the way to go.