Lets say I have 3 or more manager only nodes (which should not schedule any tasks) and we have 2 or more worker only nodes (which should schedule the tasks), what would the recommended Portainer deployment stack file look like or the topology look like ?
Here are the scenarios I have tried:
- Draining the manager nodes, does not work as the agents needs to be still running on those nodes.
- Deploying the agents and the ui service on the managers (constraining with labels) and then pausing the availability on the manager nodes post deployment. This works for a time being only as after a while the portainer agents on the manager nodes get unloaded by the swarm scheduler and as such it stops running.
Any ideas on how to better handle this, other than requiring each and every task to be placed via placement constraints just because portainer needs agents running on the manager nodes which kind of goes against the manager nodes should not be running any tasks?