Cross namespace service

[Ngob]

NOTE: GitHub issues are reserved for bug reports only.
For anything else, please join the conversation
in Kong Nation https://discuss.konghq.com/c/kubernetes.

---

Summary

I am trying to add an Ingress in namespace A, associated to a service S1 (referenced as ExternalName) in namespace A, linked to service S2 in namespace B.

Kong Ingress controller version

0.2.2

Kong or Kong Enterprise version

0.14 CE

Kubernetes version

paste kubectl version output

Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.7", GitCommit:"0c38c362511b20a098d7cd855f1314dad92c2780", GitTreeState:"clean", BuildDate:"2018-08-20T10:09:03Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"10+", GitVersion:"v1.10.6-gke.11", GitCommit:"42df8ec7aef509caba40b6178616dcffca9d7355", GitTreeState:"clean", BuildDate:"2018-11-08T20:06:00Z", GoVersion:"go1.9.3b4", Compiler:"gc", Platform:"linux/amd64"}

Environment

• Cloud provider or hardware configuration: GKE
• OS (e.g. from /etc/os-release): COS
• Kernel (e.g. uname -a): Linux gke-prod-1-pool-8cpu-0cb67ce0-3x2f 4.14.56+ Status of controller? #1 SMP Sat Aug 11 23:13:59 PDT 2018 x86_64 Intel(R) Xeon(R) CPU @ 2.00GHz GenuineIntel GNU/Linux
• Install tools:
• Others:

What happened

I am trying to reference a service in an Ingress from another namespace. I know this is not directly possible, so I added a service as external name. Relevant configuration:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: identityapi
  namespace:  api
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  tls:
  - hosts:
    - api.my-host.com
    secretName: api-my-host-tls
  rules:
    - host: api.my-host.com
      http:
        paths:
          - path: "/identity"
            backend:
              serviceName: identityapi
              servicePort: http
---
apiVersion: configuration.konghq.com/v1
kind: KongIngress
metadata:
  name: identityapi
  namespace:  api
route:
  strip_path: false
proxy:
  protocol: http
---
apiVersion: v1
kind: Service
metadata:
  name: identityapi
  namespace:  api
spec:
  type: ExternalName
  externalName: identity-api.identity.svc.cluster.local
  ports:
  - name: http
    port: 80
    protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
  name: identity-api
  namespace:  identity
  labels:
    app: identity
spec:
  type: ClusterIP
  clusterIP: None
  ports:
  - name: http
    port: 80
    targetPort: 8080
  selector:
    app: identity

Now, when I run curl "https://api.my-host.com/identity", kong response is

An invalid response was received from the upstream server

Probably because a wrong port is used

For information, everything is OK if my ingress is directly in the correct namespace, EG:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: identityapi
  namespace:  identity
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  tls:
  - hosts:
    - api.my-host.com
    secretName: api-my-host-tls
  rules:
    - host: api.my-host.com
      http:
        paths:
          - path: "/identity"
            backend:
              serviceName: identity-api
              servicePort: http
---
apiVersion: configuration.konghq.com/v1
kind: KongIngress
metadata:
  name: identityapi
  namespace:  identity
route:
  strip_path: false
proxy:
  protocol: http
---
apiVersion: v1
kind: Service
metadata:
  name: identity-api
  namespace:  identity
  labels:
    app: identity
spec:
  type: ClusterIP
  clusterIP: None
  ports:
  - name: http
    port: 80
    targetPort: 8080
  selector:
    app: identity

And everything is working if I define my port explicitly:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: identityapi
  namespace:  api
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  tls:
  - hosts:
    - api.my-host.com
    secretName: api-my-host-tls
  rules:
    - host: api.my-host.com
      http:
        paths:
          - path: "/identity"
            backend:
              serviceName: identityapi
              servicePort: http
---
apiVersion: configuration.konghq.com/v1
kind: KongIngress
metadata:
  name: identityapi
  namespace:  api
route:
  strip_path: false
proxy:
  protocol: http
---
apiVersion: v1
kind: Service
metadata:
  name: identityapi
  namespace:  api
spec:
  type: ExternalName
  externalName: identity-api.identity.svc.cluster.local
  ports:
  - name: http
    port: 80
    targetPort: 8030
    protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
  name: identity-api
  namespace:  identity
  labels:
    app: identity
spec:
  type: ClusterIP
  clusterIP: None
  ports:
  - name: http
    port: 80
    targetPort: 8030
  selector:
    app: identity

Samples has been tested on my environment with the correct host.

I would like to be able to define my ingress in another namespace using a port defined in another service. This will allow, in my case, to only have 1 SSL certificate generated for all services served under "api.my-host.com".

For information, SSL is working when the ingress is in the correct namespace but my secret is in another namespace.

Expected behavior

Kong proxying to my pod on port 8030

Steps To Reproduce

1. Deploy an app associated to a service S1 in namespace A, with a different port and targetPort
2. Create a service S2 in namespace B, with a port equal to the port of service S1
3. Create an ingress rule to redirect to service S2 in namespace B
4. Run curl

[hbagdi]

Please also have a look at kubernetes/ingress-nginx#2371

Breaking the namespace isolation of sharing secrets across namespaces in a controller for Kubernetes will certainly surprise users at the very least.

The simplest solution would be to write a very trivial k8s controller which syncs your secret and then you don't need to worry about duplication.

[Ngob]

Hello,

From what I understand, that would allow me to use the secret cross-namespace so I could have one certificate renew per host. That work, I still need to be able to use my ingress as I stated: One ingress which redirect to an "ExternalName" SVC which redirect to a standard SVC. One of my other case is this one:
Doing a request to http://bo.whatever.com/api/foo redirect to the external SVC to kong-kong-proxy.kong.svc.cluster.local with a applied plugin modifying the Host header with api.whatever.com. This way, KONG should resolve bo.whatever.com/api/foo to api.whatever.com/foo

[hbagdi]

Please see kubernetes/kubernetes#17088