Specifying control plane IP in kubeadm config template #6745

sharifelgamal · 2020-02-22T01:50:39Z

No description provided.

k8s-ci-robot · 2020-02-22T01:51:24Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sharifelgamal

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [sharifelgamal]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

codecov-io · 2020-02-22T06:13:26Z

Codecov Report

❗ No coverage uploaded for pull request base (master@32a5846). Click here to learn what that means.
The diff coverage is 61.29%.

@@            Coverage Diff            @@
##             master    #6745   +/-   ##
=========================================
  Coverage          ?   38.51%           
=========================================
  Files             ?      142           
  Lines             ?     8688           
  Branches          ?        0           
=========================================
  Hits              ?     3346           
  Misses            ?     4923           
  Partials          ?      419

Impacted Files	Coverage Δ
pkg/minikube/registry/registry.go	`100% <ø> (ø)`
pkg/minikube/bootstrapper/bootstrapper.go	`0% <ø> (ø)`
pkg/minikube/config/node.go	`0% <0%> (ø)`
pkg/minikube/machine/cache_images.go	`0% <0%> (ø)`
pkg/minikube/cluster/iso.go	`0% <0%> (ø)`
pkg/minikube/tunnel/cluster_inspector.go	`75% <100%> (ø)`
pkg/addons/addons.go	`45.67% <100%> (ø)`
pkg/minikube/machine/start.go	`55% <100%> (ø)`
pkg/minikube/machine/fix.go	`32.59% <100%> (ø)`
pkg/minikube/bootstrapper/bsutil/kubeadm.go	`83.56% <100%> (ø)`
... and 5 more

medyagh · 2020-02-22T16:49:35Z

/ok-to-test

minikube-pr-bot · 2020-02-22T16:57:23Z

Error: running mkcmp: exit status 1

medyagh · 2020-02-22T17:42:01Z

"TestDockerFlags" fails consistently on all 3 docker tests (ubuntu 16, ubuntu 18) and jenkins docker linux

on my mac also fails same way:
you can run the test with :

$ make integration -e TEST_ARGS="-test.run TestDockerFlags --profile=minikube --cleanup=false"

=== RUN   TestDockerFlags
=== PAUSE TestDockerFlags
=== CONT  TestDockerFlags
--- FAIL: TestDockerFlags (98.47s)
    docker_test.go:40: (dbg) Run:  out/minikube start -p minikube --cache-images=false --install-addons=false --wait=false --docker-env=FOO=BAR --docker-env=BAZ=BAT --docker-opt=debug --docker-opt=icc=true --alsologtostderr -v=5
    docker_test.go:40: (dbg) Done: out/minikube start -p minikube --cache-images=false --install-addons=false --wait=false --docker-env=FOO=BAR --docker-env=BAZ=BAT --docker-opt=debug --docker-opt=icc=true --alsologtostderr -v=5 : (1m27.977719038s)
    docker_test.go:45: (dbg) Run:  out/minikube -p minikube ssh "systemctl show docker --property=Environment --no-pager"
    docker_test.go:45: (dbg) Non-zero exit: out/minikube -p minikube ssh "systemctl show docker --property=Environment --no-pager": exit status 1 (283.495587ms)
        -- stdout --
        Failed to connect to bus: No such file or directory

        -- /stdout --
        ** stderr **
        ssh: Process exited with status 1

        ** /stderr **
    docker_test.go:47: [../../out/minikube -p minikube ssh systemctl show docker --property=Environment --no-pager] failed: exit status 1
    docker_test.go:52: env var FOO=BAR missing: Failed to connect to bus: No such file or directory
        .
    docker_test.go:52: env var BAZ=BAT missing: Failed to connect to bus: No such file or directory
        .
    docker_test.go:56: (dbg) Run:  out/minikube -p minikube ssh "systemctl show docker --property=ExecStart --no-pager"
    docker_test.go:56: (dbg) Non-zero exit: out/minikube -p minikube ssh "systemctl show docker --property=ExecStart --no-pager": exit status 1 (277.150529ms)
        -- stdout --
        Failed to connect to bus: No such file or directory

        -- /stdout --
        ** stderr **
        ssh: Process exited with status 1

        ** /stderr **
    docker_test.go:58: [../../out/minikube -p minikube ssh systemctl show docker --property=ExecStart --no-pager] failed: exit status 1
    docker_test.go:62: out/minikube -p minikube ssh "systemctl show docker --property=ExecStart --no-pager" = "Failed to connect to bus: No such file or directory\r\n", want *--debug*
    panic.go:563: *** TestDockerFlags FAILED at 2020-02-22 10:04:03.61446 -0800 PST m=+88.560601670
    helpers.go:188: (dbg) Run:  out/minikube status --format={{.Host}} -p minikube
    helpers.go:188: (dbg) Done: out/minikube status --format={{.Host}} -p minikube: (1.184685608s)
    helpers.go:194: <<< TestDockerFlags FAILED: start of post-mortem logs <<<
    helpers.go:195: (dbg) Run:  out/minikube -p minikube logs --problems
    helpers.go:195: (dbg) Done: out/minikube -p minikube logs --problems: (6.657142317s)
    helpers.go:200: TestDockerFlags logs: * Problems detected in kube-controller-manager [085c57054407]:
          - failed to create listener: failed to listen on 0.0.0.0:10252: listen tcp 0.0.0.0:10252: bind: address already in use
    helpers.go:202: (dbg) Run:  out/minikube status --format={{.APIServer}} -p minikube
    helpers.go:208: (dbg) Run:  kubectl --context minikube get po -A --show-labels
    helpers.go:208: (dbg) Done: kubectl --context minikube get po -A --show-labels: (1.003206053s)
    helpers.go:213: (dbg) kubectl --context minikube get po -A --show-labels:
        NAMESPACE     NAME                               READY   STATUS               RESTARTS   AGE   LABELS
        kube-system   coredns-6955765f44-52vvj           0/1     Running              4          13m   k8s-app=kube-dns,pod-template-hash=6955765f44
        kube-system   coredns-6955765f44-8kk8n           0/1     ContainerCannotRun   3          13m   k8s-app=kube-dns,pod-template-hash=6955765f44
        kube-system   etcd-minikube                      0/1     CrashLoopBackOff     4          14m   component=etcd,tier=control-plane
        kube-system   kindnet-skj7w                      0/1     Error                2          13m   app=kindnet,controller-revision-hash=6f48886b45,k8s-app=kindnet,pod-template-generation=1,tier=node
        kube-system   kube-apiserver-minikube            1/1     Running              3          14m   component=kube-apiserver,tier=control-plane
        kube-system   kube-controller-manager-minikube   0/1     CrashLoopBackOff     3          14m   component=kube-controller-manager,tier=control-plane
        kube-system   kube-proxy-gf8xh                   1/1     Running              3          13m   controller-revision-hash=5fc47fcbf5,k8s-app=kube-proxy,pod-template-generation=1
        kube-system   kube-scheduler-minikube            1/1     Running              3          14m   component=kube-scheduler,tier=control-plane
        kube-system   storage-provisioner                0/1     Error                2          14m   addonmanager.kubernetes.io/mode=Reconcile,integration-test=storage-provisioner
    helpers.go:215: (dbg) Run:  kubectl --context minikube describe node
    helpers.go:219: (dbg) kubectl --context minikube describe node:
        Name:               minikube
        Roles:              master
        Labels:             beta.kubernetes.io/arch=amd64
                            beta.kubernetes.io/os=linux
                            kubernetes.io/arch=amd64
                            kubernetes.io/hostname=minikube
                            kubernetes.io/os=linux
                            minikube.k8s.io/commit=c272862898d95b5bcd0181efa08149c1cb3afdf1
                            minikube.k8s.io/name=minikube
                            minikube.k8s.io/updated_at=2020_02_22T09_50_07_0700
                            minikube.k8s.io/version=v1.7.3
                            node-role.kubernetes.io/master=
        Annotations:        kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock
                            node.alpha.kubernetes.io/ttl: 0
                            volumes.kubernetes.io/controller-managed-attach-detach: true
        CreationTimestamp:  Sat, 22 Feb 2020 09:50:02 -0800
        Taints:             <none>
        Unschedulable:      false
        Conditions:
          Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
          ----             ------  -----------------                 ------------------                ------                       -------
          MemoryPressure   False   Sat, 22 Feb 2020 10:04:02 -0800   Sat, 22 Feb 2020 09:49:53 -0800   KubeletHasSufficientMemory   kubelet has sufficient memory available
          DiskPressure     False   Sat, 22 Feb 2020 10:04:02 -0800   Sat, 22 Feb 2020 09:49:53 -0800   KubeletHasNoDiskPressure     kubelet has no disk pressure
          PIDPressure      False   Sat, 22 Feb 2020 10:04:02 -0800   Sat, 22 Feb 2020 09:49:53 -0800   KubeletHasSufficientPID      kubelet has sufficient PID available
          Ready            True    Sat, 22 Feb 2020 10:04:02 -0800   Sat, 22 Feb 2020 09:50:17 -0800   KubeletReady                 kubelet is posting ready status
        Addresses:
          InternalIP:  172.17.0.2
          Hostname:    minikube
        Capacity:
         cpu:                4
         ephemeral-storage:  61255492Ki
         hugepages-2Mi:      0
         memory:             2037260Ki
         pods:               110
        Allocatable:
         cpu:                4
         ephemeral-storage:  56453061334
         hugepages-2Mi:      0
         memory:             1934860Ki
         pods:               110
        System Info:
         Machine ID:                 1ccf9dc149854bdd9a96d2157b0a85c0
         System UUID:                071eeff2-e057-45fb-9784-10a7253ca78d
         Boot ID:                    c2de8504-214d-4d47-a253-ad9df89b1576
         Kernel Version:             4.19.76-linuxkit
         OS Image:                   Ubuntu 19.10
         Operating System:           linux
         Architecture:               amd64
         Container Runtime Version:  docker://19.3.2
         Kubelet Version:            v1.17.3
         Kube-Proxy Version:         v1.17.3
        PodCIDR:                     10.244.0.0/24
        Non-terminated Pods:         (9 in total)
          Namespace                  Name                                CPU Requests  CPU Limits  Memory Requests  Memory Limits  AGE
          ---------                  ----                                ------------  ----------  ---------------  -------------  ---
          kube-system                coredns-6955765f44-52vvj            100m (2%)     0 (0%)      70Mi (3%)        170Mi (8%)     13m
          kube-system                coredns-6955765f44-8kk8n            100m (2%)     0 (0%)      70Mi (3%)        170Mi (8%)     13m
          kube-system                etcd-minikube                       0 (0%)        0 (0%)      0 (0%)           0 (0%)         14m
          kube-system                kindnet-skj7w                       100m (2%)     100m (2%)   50Mi (2%)        50Mi (2%)      13m
          kube-system                kube-apiserver-minikube             250m (6%)     0 (0%)      0 (0%)           0 (0%)         14m
          kube-system                kube-controller-manager-minikube    200m (5%)     0 (0%)      0 (0%)           0 (0%)         14m
          kube-system                kube-proxy-gf8xh                    0 (0%)        0 (0%)      0 (0%)           0 (0%)         13m
          kube-system                kube-scheduler-minikube             100m (2%)     0 (0%)      0 (0%)           0 (0%)         14m
          kube-system                storage-provisioner                 0 (0%)        0 (0%)      0 (0%)           0 (0%)         14m
        Allocated resources:
          (Total limits may be over 100 percent, i.e., overcommitted.)
          Resource           Requests     Limits
          --------           --------     ------
          cpu                850m (21%)   100m (2%)
          memory             190Mi (10%)  390Mi (20%)
          ephemeral-storage  0 (0%)       0 (0%)
        Events:
          Type     Reason                   Age                From                  Message
          ----     ------                   ----               ----                  -------
          Normal   NodeHasSufficientMemory  14m (x6 over 14m)  kubelet, minikube     Node minikube status is now: NodeHasSufficientMemory
          Normal   NodeHasNoDiskPressure    14m (x6 over 14m)  kubelet, minikube     Node minikube status is now: NodeHasNoDiskPressure
          Normal   NodeHasSufficientPID     14m (x7 over 14m)  kubelet, minikube     Node minikube status is now: NodeHasSufficientPID
          Normal   Starting                 14m                kubelet, minikube     Starting kubelet.
          Normal   NodeHasSufficientMemory  14m                kubelet, minikube     Node minikube status is now: NodeHasSufficientMemory
          Normal   NodeHasNoDiskPressure    14m                kubelet, minikube     Node minikube status is now: NodeHasNoDiskPressure
          Normal   NodeHasSufficientPID     14m                kubelet, minikube     Node minikube status is now: NodeHasSufficientPID
          Normal   NodeAllocatableEnforced  14m                kubelet, minikube     Updated Node Allocatable limit across pods
          Normal   NodeReady                13m                kubelet, minikube     Node minikube status is now: NodeReady
          Warning  readOnlySysFS            13m                kube-proxy, minikube  CRI error: /sys is read-only: cannot modify conntrack limits, problems may arise later (If running Docker, see docker issue #24000)
          Normal   Starting                 13m                kube-proxy, minikube  Starting kube-proxy.
          Normal   Starting                 19s                kubelet, minikube     Starting kubelet.
          Normal   NodeAllocatableEnforced  19s                kubelet, minikube     Updated Node Allocatable limit across pods
          Normal   NodeHasSufficientMemory  18s (x8 over 19s)  kubelet, minikube     Node minikube status is now: NodeHasSufficientMemory
          Normal   NodeHasNoDiskPressure    18s (x8 over 19s)  kubelet, minikube     Node minikube status is now: NodeHasNoDiskPressure
          Normal   NodeHasSufficientPID     18s (x7 over 19s)  kubelet, minikube     Node minikube status is now: NodeHasSufficientPID
          Warning  readOnlySysFS            7s                 kube-proxy, minikube  CRI error: /sys is read-only: cannot modify conntrack limits, problems may arise later (If running Docker, see docker issue #24000)
          Normal   Starting                 7s                 kube-proxy, minikube  Starting kube-proxy.
    helpers.go:221: <<< TestDockerFlags FAILED: end of post-mortem logs <<<
    helpers.go:165: skipping cleanup of minikube (--cleanup=false)
FAIL
FAIL	k8s.io/minikube/test/integration	98.517s
FAIL
make: *** [integration] Error 1

medyagh · 2020-02-22T18:13:07Z

and on a hyperkit driver, I can not do ssh

$ ./out/minikube ssh
ssh: dial tcp 192.168.64.48:22: connect: connection refused

medyagh · 2020-02-22T18:25:27Z

to replicate locally I tried this :

./out/minikube start --cache-images=false --install-addons=false --wait=false --docker-env=FOO=BAR --docker-env=BAZ=BAT --docker-opt=debug --docker-opt=icc=true --alsologtostderr -v=5

and you will see

I0222 10:23:23.908801   61319 main.go:110] libmachine: Error dialing TCP: dial tcp 192.168.64.51:22: connect: connection refused
I0222 10:23:27.926582   61319 main.go:110] libmachine: Error dialing TCP: dial tcp 192.168.64.51:22: connect: connection refused
I0222 10:23:31.930985   61319 main.go:110] libmachine: Error dialing TCP: dial tcp 192.168.64.51:22: connect: connection refused
I0222 10:23:35.936583   61319 main.go:110] libmachine: Error dialing TCP: dial tcp 192.168.64.51:22: connect: connection refused
I0222 10:23:39.941502   61319 main.go:110] libmachine: Error dialing TCP: dial tcp 192.168.64.51:22: connect: connection refused
I0222 10:23:43.949684   61319 main.go:110] libmachine: Error dialing TCP: dial tcp 192.168.64.51:22: connect: connection refused
I0222 10:23:47.957392   61319 main.go:110] libmachine: Error dialing TCP: dial tcp 192.168.64.51:22: connect: connection refused
I0222 10:23:51.963299   61319 main.go:110] libmachine: Error dialing TCP: dial tcp 192.168.64.51:22: connect: connection refused
I0222 10:23:55.965575   61319 main.go:110] libmachine: Error dialing TCP: dial tcp 192.168.64.51:22: connect: connection refused
I0222 10:23:59.975435   61319 main.go:110] libmachine: Error dialing TCP: dial tcp 192.168.64.51:22: connect: connection refused
I0222 10:24:03.984332   61319 main.go:110] libmachine: Error dialing TCP: dial tcp 192.168.64.51:22: connect: connection refused
I0222 10:24:07.998939   61319 main.go:110] libmachine: Error dialing TCP: dial tcp 192.168.64.51:22: connect: connection refused
I0222 10:24:12.002965   61319 main.go:110] libmachine: Error dialing TCP: dial tcp 192.168.64.51:22: connect: connection refused
I0222 10:24:16.008500   61319 main.go:110] libmachine: Error dialing TCP: dial tcp 192.168.64.51:22: connect: connection refused

sharifelgamal · 2020-02-22T18:39:39Z

All of that worked locally for me, so I'm not sure what's going on

medyagh · 2020-02-22T18:45:03Z

All of that worked locally for me, so I'm not sure what's going on

ah! I think I found the problem

I tried again I with docker driver, ( I think the ssh problem I mentioned was irrelaevent)

"this is where the test fails !

docker@minikube:~$ systemctl show docker --property=Environment --no-pager
Failed to connect to bus: No such file or directory

but it passes with sudo

docker@minikube:~$ sudo systemctl show docker --property=Environment --no-pager
Environment=FOO=BAR BAZ=BAT

@sharifelgamal I recommend adding sudo to the integration tests to see if that will pass the test
( while we are at it, I would do sudo to any test command that uses systemctl)

"ssh", "sudo" "systemctl show docker --property=Environment --no-pager"))

sharifelgamal · 2020-02-22T18:58:12Z

that also works fine for me:

out/minikube -p minikube ssh "systemctl show docker --property=ExecStart --no-pager" 
ExecStart={ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=hyperkit --insecure-registry 10.96.0.0/12 --debug --icc=true ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }

also I'm not sure what that has to do with my change.

medyagh · 2020-02-22T18:59:54Z

that also works fine for me:

out/minikube -p minikube ssh "systemctl show docker --property=ExecStart --no-pager" 
ExecStart={ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=hyperkit --insecure-registry 10.96.0.0/12 --debug --icc=true ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }

also I'm not sure what that has to do with my change.

you are right ...it seems to be a docker-driver only thing. we could do sudo in a different PR for docker driver. lets merge this

sharifelgamal added 2 commits February 21, 2020 17:48

Specifying control plane IP in kubeadm config template

32a5846

Merge branch 'master' of github.com:kubernetes/minikube into cpip

81e9bcb

sharifelgamal requested a review from medyagh February 22, 2020 01:50

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Feb 22, 2020

k8s-ci-robot requested review from josedonizetti and tstromberg February 22, 2020 01:51

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 22, 2020

fix build error

fca5101

k8s-ci-robot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label Feb 22, 2020

medyagh approved these changes Feb 22, 2020

View reviewed changes

medyagh merged commit 9634132 into kubernetes:master Feb 22, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Specifying control plane IP in kubeadm config template #6745

Specifying control plane IP in kubeadm config template #6745

sharifelgamal commented Feb 22, 2020

k8s-ci-robot commented Feb 22, 2020

codecov-io commented Feb 22, 2020

medyagh commented Feb 22, 2020

minikube-pr-bot commented Feb 22, 2020

medyagh commented Feb 22, 2020 •

edited

Loading

medyagh commented Feb 22, 2020

medyagh commented Feb 22, 2020 •

edited

Loading

sharifelgamal commented Feb 22, 2020

medyagh commented Feb 22, 2020 •

edited

Loading

sharifelgamal commented Feb 22, 2020 •

edited

Loading

medyagh commented Feb 22, 2020

Specifying control plane IP in kubeadm config template #6745

Specifying control plane IP in kubeadm config template #6745

Conversation

sharifelgamal commented Feb 22, 2020

k8s-ci-robot commented Feb 22, 2020

codecov-io commented Feb 22, 2020

Codecov Report

medyagh commented Feb 22, 2020

minikube-pr-bot commented Feb 22, 2020

medyagh commented Feb 22, 2020 • edited Loading

medyagh commented Feb 22, 2020

medyagh commented Feb 22, 2020 • edited Loading

sharifelgamal commented Feb 22, 2020

medyagh commented Feb 22, 2020 • edited Loading

sharifelgamal commented Feb 22, 2020 • edited Loading

medyagh commented Feb 22, 2020

medyagh commented Feb 22, 2020 •

edited

Loading

medyagh commented Feb 22, 2020 •

edited

Loading

medyagh commented Feb 22, 2020 •

edited

Loading

sharifelgamal commented Feb 22, 2020 •

edited

Loading