Unable to retrieve volumes

Portainer host is connected to 3 endpoints via the agent.
Only 1 endpoint is giving an error “Unable to retrieve volumes”, the remaining 2 and local endpoint work. All other functions work.
Error from Portainer logs = “2020/09/03 15:39:38 http: proxy error: missing identifier in Docker resource list response”
I proceeded to upgrade the portainer-agent to the latest API 2.0 version, which did not have any affect.

I deployed a new portainer-ce container which does not have /data mapped so we can excluded any database errors. I added the endpoint and still received the same error.

I deployed a portainer 1.24.1 container again without /data mapped and added the endpoint and I can see the volumes.

Due to the above this appears to be specific to portainer-ce and I am happy to provide further information for troubleshooting. Docker version of problematic host is 1.13.1 where all my other hosts are running much newer versions 19.03. RHEL 3.10.0-1127.13.1.el7.x86_64

Hi there, i recommend making sure you update to the latest agent on all nodes, as the way volumes are handled in ce is changed. We now build an identifier for them based on name & creation date.

Also was this volume created outside of portainer and with the local docker volume driver?

Hi, All the agents are now the latest version. Yes the volumes on the problem node were created outside of portainer but this is true for all the volumes on all nodes.

I have just noticed that on the problem node which is running docker 1.13.1 there is no identifier “CreatedAt” listed when inspecting a volume which I guess is causing the problem with Portainer being unable to create an identifier?

Here is the output:
-sh-4.2$ docker inspect fd0be57178bcd73cd231fcd0d00eff32da67d404329fe97bbc7e18672f1ef50e
[
{
“Driver”: “local”,
“Labels”: null,
“Mountpoint”: “/var/lib/docker/volumes/fd0be57178bcd73cd231fcd0d00eff32da67d404329fe97bbc7e18672f1ef50e/_data”,
“Name”: “fd0be57178bcd73cd231fcd0d00eff32da67d404329fe97bbc7e18672f1ef50e”,
“Options”: {},
“Scope”: “local”
}
]

This looks like the cause, we support docker versions n-2 so the oldest version we would currently support would be 18.06 I believe. I would recommend to update your docker version, be aware that a jump from 1.13.1 to 18.06 or newer will probably break a lot of stuff. I’ll also pass this on to the dev team to see if we can better handle volumes with no creation date