Thanks for the very detailed write-up @paul-reeves_6112!
Just some background -- back in the v5 days, we introduced Text Templates and intended it to just totally replace the Configuration Files feature. It seemed simpler/better at the time... but I was wrong, that was a mistake... and well, now we have two features that kind of overlap, and are quite confusing at times.
Aside from simplicity (in many use-cases), the main benefit to Configuration Files is restricting view/editing of different instances. There are other benefits as well, but that's what most users really like about them -- especially since it's a lot simpler than using secret storage for values.
Attaching versions to releases is also nice, as it provides better visibility in changes. The main benefit in this all is that developers can "see everything except the actual production values" (like they can see prod was changed, just not what was changed), and then help operations (who can see everything) debug/diagnose/etc.
As @stevedennis mentioned, this is definitely something I want to improve throughout v7, from content and enhancements. So i'm very open to your (seemingly) advanced use case :)
[1] Selecting Templates on Deploy
This is should be pretty easy to do, but let's just "pin" that to make sure I can makes sure it would be the right solution. I wonder if even using a Text Template or just a file on disk do the transform? That way, the template could be stored in source control, and the values in BuildMaster.
[2] $InstanceId per Server
The $InstanceId approach most definitely should work, and actually -- exactly how we envisioned the cascading variables to work, when integrated into configuration files :)
But but then I saw this: it seems I cannot manually deploy a configuration file however
This sounds like a bug to me... when you manually deploy a configuration file, you have to pick a Server... and that Server should be in context when doing variable resolution. I haven't tested it myself, but that might be what's missing here? That could be a regression, or an untested corner case, or something.
[3] Conditional $WebKey per environment
I should probably know this offhand... but are you saying the OtterScript blocks don't work inside of Configuration Files? In theory you should be able to do it, but that might not have been something we brought over. It should be easy to implement, and we'd want to make it opt-in.
But if that's the only difference between the two Templates (i.e. a conditional $WebKey), then the OtterScript block you cameup seems like a good idea. There's also $ListIndexOf(@ServersInRole(Web), $ServerName) >= 0 to test the server's roles, though a simpler function would be nicer now that I think about it...
[4] Variables assigned in several different places, global, environment, roles, and application
Generally speaking, "convention-driven" tends to help minimize configuration (like using role names, application names, etc., when possible), but otherwise... the variables are a good fit for this. All told, this is pretty common, and it can be a lot easier to maintain as well.
You can see all variables in all scopes on the ADmin > variables page, and it helps make it pretty clear how things are configured and to sort of audit.
Conclusion
Overall I think Configuration Files is a better fit, and whilst some improvements would be good I like the Key/Value pairs.
Agreed, they seem like a better fit for your use case - especially since I assume the $Database is a secret.
Versioning probably doesn't matter too much as any new settings won't be understood by the old software versions if a downgrade was required.
Agreed; the main benefit on top of using different/old versions is "seeing which version was deployed with which relesae" in a very easy way. But keep in mind, "rollbacks" are where old versions might be nice to have tied to the release.
Equally deploying the WebKey to all server roles also probably doesn't actually cause an impact.
Also agreed; but... philosophically we don't want users to have to change their release processes just to use BuildMaster, so I want to make sure at least we support this case ;)
Hopefully you can see a use case in being able to select a template within a configuration file; I actually have two different potential use cases
For sure -- and even more than that, transforming with Text Templates or files.
All told -- thanks for the dialog. I'd love to hear your feedback on these other ideas., and we'll try fix some of these edge case bugs ASAP .
Alex