I had this same issue and was able to work-around it by adding annotations to the helm ingress values:
community:
annotations:
nginx.ingress.kubernetes.io/use-regex: "true"
The nginx.ingress.kubernetes.io/use-regex: “true” was needed to have the ingress controller use regex location blocks for ‘/agent., /inventory.’ etc… without this config, such calls to these urls would not match the location blocks in the ingress controller and instead hit the default location / and its non-grpc upstream.