Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

execution_config.file.watch stops working after first reload #1460

Open
Juricakov opened this issue Dec 19, 2024 · 2 comments
Open

execution_config.file.watch stops working after first reload #1460

Juricakov opened this issue Dec 19, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@Juricakov
Copy link

Component(s)

router

Component version

ghcr.io/wundergraph/cosmo/router:0.150.0

wgc version

0.71.1

controlplane version

cloud

router version

0.150.0

What happened?

We are self hosting the router in K8S, and relying on the cloud shema registry and control plane.

Router is configured to watch schema file in the file system.

execution_config:
  file:
    path: /schema/schema.json
    watch: true

A cron periodically pulls the schema from the registry into a ConfigMap that is mounted as a volume, here are the relevant parts of the Deployment.

spec:
  template:
    spec:
      containers:
        - name: api
          image: ghcr.io/wundergraph/cosmo/router:0.150.0
          volumeMounts:
            - name: schema-volume
              mountPath: /schema
      volumes:
        - name: schema-volume
          configMap:
            name: schema
            items:
              - key: schema.json
                path: schema.json

Note that we explicitly avoid volumeMounts.subPath as that would break configmap update propagation.

When the schema is changed, the server properly detects it and restarts, however, any successive changes are not picked up until a new Pod is created.

It looks like the file watcher context is cancelled when the old server is shutdown after server swap.

Environment information

No response

Router configuration

No response

Router execution config

No response

Log output

12:43:10 PM INFO core/router.go:1043 Config file changed. Updating server with new config {"hostname": "api-54c9d64dc9-z6n6g", "pid": 1, "component": "@wundergraph/router", "service_version": "0.150.0", "path": "/schema/schema.json"}
12:43:10 PM DEBUG core/websocket.go:127 Net poller is available {"hostname": "api-54c9d64dc9-z6n6g", "pid": 1, "component": "@wundergraph/router", "service_version": "0.150.0"} 
12:43:10 PM DEBUG core/graph_server.go:1078 Shutdown of graph server initiated. Waiting for in-flight requests to finish. {"hostname": "api-54c9d64dc9-z6n6g", "pid": 1, "component": "@wundergraph/router", "service_version": "0.150.0", "config_version": "51f95dcc-9a51-40cc-a6f5-24d8f5bb7c36"} 
12:43:10 PM DEBUG core/graph_server.go:1091 Shutdown of graph server resources {"hostname": "api-54c9d64dc9-z6n6g", "pid": 1, "component": "@wundergraph/router", "service_version": "0.150.0", "grace_period": "30s", "config_version": "51f95dcc-9a51-40cc-a6f5-24d8f5bb7c36"} 
12:43:10 PM DEBUG core/graph_server.go:1113 Shutting down pubsub providers {"hostname": "api-54c9d64dc9-z6n6g", "pid": 1, "component": "@wundergraph/router", "service_version": "0.150.0"} 
12:43:10 PM DEBUG [email protected]/zap.go:61 subscriptionClient.close {"hostname": "api-54c9d64dc9-z6n6g", "pid": 1, "component": "@wundergraph/router", "service_version": "0.150.0", "reason": "netPoll closed by context"}

Additional context

No response

@Juricakov Juricakov added the bug Something isn't working label Dec 19, 2024
Copy link

WunderGraph commits fully to Open Source and we want to make sure that we can help you as fast as possible.
The roadmap is driven by our customers and we have to prioritize issues that are important to them.
You can influence the priority by becoming a customer. Please contact us here.

@StarpTech
Copy link
Contributor

Hi @Juricakov thanks for reporting. We'll take a look and come back.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants