Question: } for _, test := range tests { <del> nodes := makeNodeList([]string{"1"}) </del> <ins> t.Run(test.name, func(t *testing.T) { nodes := makeNodeList([]string{"1"}) </ins> <del> plugin := st.FakeFilterPlugin{} registerFakeFilterFunc := st.RegisterFilterPlugin( "FakeFilter", func(_ runtime.Object, fh framework.Handle) (framework.Plugin, error) { return &plugin, nil }, ) registerPlugins := []st.RegisterPluginFunc{ st.RegisterQueueSortPlugin(queuesort.Name, queuesort.New), registerFakeFilterFunc, st.RegisterBindPlugin(defaultbinder.Name, defaultbinder.New), } fwk, err := st.NewFramework( registerPlugins, frameworkruntime.WithPodNominator(internalqueue.NewPodNominator()), ) if err != nil { t.Fatal(err) } </del> <ins> plugin := st.FakeFilterPlugin{} registerFakeFilterFunc := st.RegisterFilterPlugin( "FakeFilter", func(_ runtime.Object, fh framework.Handle) (framework.Plugin, error) { return &plugin, nil }, ) registerPlugins := []st.RegisterPluginFunc{ st.RegisterQueueSortPlugin(queuesort.Name, queuesort.New), registerFakeFilterFunc, st.RegisterBindPlugin(defaultbinder.Name, defaultbinder.New), } fwk, err := st.NewFramework( registerPlugins, frameworkruntime.WithPodNominator(internalqueue.NewPodNominator()), ) if err != nil { t.Fatal(err) } </ins> <del> scheduler := makeScheduler(nodes) if err := scheduler.cache.UpdateSnapshot(scheduler.nodeInfoSnapshot); err != nil { t.Fatal(err) } fwk.PreemptHandle().AddNominatedPod(&v1.Pod{ObjectMeta: metav1.ObjectMeta{UID: "nominated"}, Spec: v1.PodSpec{Priority: &midPriority}}, "1") </del> <ins> scheduler := makeScheduler(nodes) if err := scheduler.cache.UpdateSnapshot(scheduler.nodeInfoSnapshot); err != nil { t.Fatal(err) } fwk.PreemptHandle().AddNominatedPod(&v1.Pod{ObjectMeta: metav1.ObjectMeta{UID: "nominated"}, Spec: v1.PodSpec{Priority: &midPriority}}, "1") </ins> <del> _, _, err = scheduler.findNodesThatFitPod(context.Background(), fwk, framework.NewCycleState(), test.pod) </del> <ins> _, _, err = scheduler.findNodesThatFitPod(context.Background(), fwk, framework.NewCycleState(), test.pod) </ins> <del> if err != nil { t.Errorf("unexpected error: %v", err) } if test.expectedCount != plugin.NumFilterCalled { t.Errorf("predicate was called %d times, expected is %d", plugin.NumFilterCalled, test.expectedCount) } </del> <ins> if err != nil { t.Errorf("unexpected error: %v", err) } if test.expectedCount != plugin.NumFilterCalled { t.Errorf("predicate was called %d times, expected is %d", plugin.NumFilterCalled, test.expectedCount) } }) </ins> } }

<!-- Please only use this template for submitting enhancement requests --What would you like to be : Currently some unit tests in pkg/scheduler is not organized as Use subtests can make it easier to find out which case fails and leverage benefit of parallelism . Why is this needed: Follow Go test practice.
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/remove-kind feature /kind cleanup /sig scheduling /assign


Question: return testCase.nodeAddresses, nil } <del> // cloud provider is expected to be nil if external provider is set </del> <ins> // cloud provider is expected to be nil if external provider is set or there is no cloud provider </ins> var cloud cloudprovider.Interface <del> if testCase.externalCloudProvider { cloud = nil } else { </del> <ins> if testCase.cloudProviderType == cloudProviderLegacy { </ins> cloud = &fakecloud.Cloud{ Addresses: testCase.nodeAddresses, Err:       nil, } } // construct setter

I am reproducing this in OpenShift using the OpenStack cloud provider, but nothing here is specific to either OpenShift or OpenStack. When migrating a cluster which uses in-tree cloud provider to use external cloud-provider, we use the following process: cloud controllers in KCM CCM kubelets with --cloud-provider=external Between steps 1&2 the node-lifecycle, routes, and service controllers are not running anywhere, but this situation isn't unsafe for the cluster and we can keep it short. Between steps 2&3 is more interesting. The node controller is running in CCM, and the kubelets are running with --cloud-provider=openstack (or whatever, just not external). The main loop of node-controller will ignore a newly created node because . However, the loop does not have a similar exclusion. . Note that we shortcut if the kubelet is using an external cloud provider, but not if we don't. So between steps 2&3 both kubelet and CCM's node controller are managing Node.Addresses. If the legacy cloud provider and external CCM don't return the same list of NodeAddresses this causes flapping. Running OpenShift on OpenStack we have 2 problems. Firstly, our legacy cloud provider and external CCM produce different lists. We don't think this would be a problem in practise, except for the second problem. Kubelet adds the annotation to the node containing the node IP. This allows the CCM node controller to apply the same filtering based on node IP as kubelet does for the legacy cloud provider. This means that, although the lists are different, the node IP is still listed first which is the only thing that matters in practise. The problem is that kubelet only adds the annotation when --cloud-provider=external which, between steps 2&3, it is not. This means that Node.Addresses flaps between a correct and a broken configuration, as the NodeAddresses returned by the external CCM cannot be filtered by node IP without the annotation. The ideal solution here would be to ensure that only 1 controller is managing Node.Addresses during this period in the same way we do for Node creation with the external CCM taint. This would fix the issue entirely.

A fix with a much smaller scope would be to have kubelet add the node IP annotation unconditionally so, although Node.Address can potentially flap, it will at least be between 2 valid configurations. Given that this should only ever be a problem during the upgrade window, the smaller fix may be more expedient. Node.Addresses should remain valid at all times when kubelet is configured with --cloud-provider=openstack but the CCM is also running. Good luck No response 24 OpenStack (but as described, not OpenStack-specific) No response No response No response No response
/sig cloud-provider
we've an agenda item for next week's (11 May) SIG cloud provider meeting about this topic
/triage accepted
Would it work to swap steps 2 and 3?
Good question. I don't think so. That would leave us without the Node, Node lifecycle, and Service controllers until all kubelets were updated. This would mean: No cluster scale-up or scale-down No creation, deletion, or modification to LB Services I think we could handle this in OpenShift, although it's a bit fragile and I suspect updating all the kubelets can take a really long time on a big cluster. However, I think this would break cluster-api or any deployment tooling which rolls out updates by creating new machines. I think this would never complete: the new nodes would never come up because the node controller isn't running.
I am still trying to understand the issues here while reviewing your pr: , but I agreed with your comments above: That would leave us without the Node, Node lifecycle, and Service controllers until all kubelets were updated. This would mean:


Question: result := c.client.clientBase.Get(). AbsPath(path). <del> Namespace(c.namespace). </del> <ins> NamespaceIfScoped(c.namespace, c.namespace != ""). </ins> Resource(gvr.Resource). Name(name). SubResource("scale").

<!-- Please only use this template for submitting enhancement requests --What would you like to be : The scale client () currently works fine with namespaced resources. However, it doesn't work for non-namespaced resources e.g. cluster-scoped CRDs. The current implementation should be extended in order to support non-namespaced resources. The current interface: /area custom-resources /sig autoscaling
I think scale subresource support for CRDs falls more into sig-apimachinery domain. After DirectXMan12 moved on from sig-autoscaling I don't think there is anyone in our sig-autoscaling with knowledge of scale subresource internals. /sig api-machinery
/assign
The bug report sounds legit. Does the generic scale client need to be changed (I think so), or does solve your issue?
No, the dashboard PR doesn't solve this. I discovered this issue while working on that PR so when gets merged, it will only work for namespaced CRDs. I think the generic scale client needs to be updated, maybe by adding another getter to the ScalesGetter interface: /kind bug
PR:


Question: By(fmt.Sprintf("WARNING: possibly leaked static IP: %vn", ipErr)) } <del> // Fail if the controller didn't cleanup </del> <ins> // Logging that the GLBC failed to cleanup GCE resources on ingress deletion // See kubernetes/ingress#431 </ins> if pollErr != nil { <del> Failf("L7 controller failed to delete all cloud resources on time. %v", pollErr) </del> <ins> Logf("error: L7 controller failed to delete all cloud resources on time. %v", pollErr) </ins> } }

https://k8s- Failed: [] Loadbalancing: L7 [Feature:Ingress] [] GCE [Slow] [Feature: Ingress] shoud create ingress with given static-ip {Kubernetes e2e suite} Previous issues for this test:
https://k8s- Failed: [] Loadbalancing: L7 [Feature:Ingress] [] GCE [Slow] [Feature: Ingress] shoud create ingress with given static-ip {Kubernetes e2e suite}
We are seeing these a lot in etcd rollback tests. Two different error messages.
Oh, here we go again. Another case for . Let me adjust the quota on that project.
I doubled the quota to 10. Watch upcoming runs .
points out that we may be leaking resources across runs, but I don't see a persistent leak. I doubled the quota on static IP addresses.
For unknown reasons, the GLBC is failing to delete the ingress resources. Unfortunately, the e2e fallback cleanup mechanism is also failing. It's trying to list resources via exec.Command( gcloud ... list --regexp=XYZ ) but always coming back with 0 resources. Putting the regular expression in quotes returns valid results. Expect a PR this afternoon.
PR is successfully deleting the remaining resources. The etcd upgrade test is now failing roughly half the time rather than continuously. After glancing over the recent failures, I've noticed a few consistent behaviors: objects are created and the l7 address is successfully hit. ingress objects are not cleaned up by the GLBC. (The test framework has to manually delete them) file does not contain log messages saying the ingress objects are being created. It also doesn't say that a delete ingress obj notification has been received. Log of glbc. glbc. I'm puzzled to why the ingress resources exist without the noting their creation. The other question is why the delete notification is not received/logged. Will investigate further...
for why the logs look incomplete, please see kubernetes/test-infra.
explains why this test does not clean up resources - causing the flake. [tl;dr] The GLBC only deletes L7 resources for ingress objects that it's cached locally. Caching happens soon after startup, but this races with the test's ingress deletion.
Thanks, Nick.

Oh, and now I see a comment that is relevant to this. That case should probably be .
this is to be closed by he is going to remove when this trips
-- can we bump this from the release-1.6? It does not seem to be blocking for the release.
done


Question: &PartialObjectMetadataList{}, ) <ins> if err := scheme.AddConversionFuncs( Convert_Slice_string_To_v1alpha1_IncludeObjectPolicy, ); err != nil { panic(err) } </ins> // register manually. This usually goes through the SchemeBuilder, which we cannot use here. //scheme.AddGeneratedDeepCopyFuncs(GetGeneratedDeepCopyFuncs()...) }

Making an API request for table output and specifying the parameter causes a server error in parameter conversion, and an inaccurate error message in response: server log: E0804 09:41:14. ] apiserver received an error that is not an metav1.Status: converting (url.Values).[]string to (v1alpha1.TableOptions).v1alpha1.IncludeObjectPolicy: couldn't copy '[]string' into 'v1alpha1.IncludeObjectPolicy'; didn't understand types
Ugh, typed strings.


Question: "io" "os" "path/filepath" <del> goruntime "runtime" </del> "testing" "time"

The test is failing on Windows [0]: The idea of the test is to check that kubelet still follows and reads the Container Logs after a Log Rotation occured. However, because the followed file is open, it cannot be renamed, and thus, the container logs cannot be rotated. This issue could be prevented if the file would be opened with the [0] [1] Kubelet should not block container log rotation. Run the test ` is failing on Windows. /sig windows /kind bug /kind failing-test latest, master. This unit test has been failing for a while. N/A Windows nodes </detailsN/A N/A N/A
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/kind failing-test /sig windows


Question: container: .container-${OS}-$(ARCH) .container-linux-$(ARCH): bin/$(BIN)-$(OS)-$(ARCH) docker buildx build --pull --output=type=${OUTPUT_TYPE} --platform ${OS}/$(ARCH)  <del> -t $(IMAGE_WITH_OS_ARCH):$(TAG) --build-arg BASE=${BASE} --build-arg ARCH=$(ARCH) . </del> <ins> -t $(IMAGE):$(TAG)-${OS}-$(ARCH) --build-arg BASE=${BASE} --build-arg ARCH=$(ARCH) . </ins> touch $@ .container-windows-$(ARCH): $(foreach binary, ${BIN}, bin/${binary}-${OS}-${ARCH}) docker buildx build --pull --output=type=${OUTPUT_TYPE} --platform ${OS}/$(ARCH)  <del> -t $(IMAGE_WITH_OS_ARCH)-${OSVERSION}:$(TAG) --build-arg BASE=${BASE}:${OSVERSION} --build-arg ARCH=$(ARCH) -f Dockerfile_windows . </del> <ins> -t $(IMAGE):$(TAG)-${OS}-$(ARCH)-${OSVERSION} --build-arg BASE=${BASE}:${OSVERSION} --build-arg ARCH=$(ARCH) -f Dockerfile_windows . </ins> touch $@ # Useful for testing, not automatically included in container image

We could either: where appropriate - least amount of code change but cumbersome OS version in the tag instead of the repository /cc
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig windows
/kind bug


Question: Name(pod.Name). Namespace(pod.Namespace). SubResource("attach"). <del> Param("container", containerName) </del> <ins> Param("container", p.GetContainerName(pod)) </ins> return p.Attach.Attach(req, p.Config, stdin, p.Out, p.Err, tty) } <ins> // GetContainerName returns the name of the container to attach to, with a fallback. func (p *AttachOptions) GetContainerName(pod *api.Pod) string { if len(p.ContainerName) > 0 { return p.ContainerName } glog.V(4).Infof("defaulting container name to %s", pod.Spec.Containers[0].Name) return pod.Spec.Containers[0].Name } </ins>

runs foo and then waits for it to be ready or terminated. If the pod is terminated, it prints the log, and if it's not terminated, it attaches the output. However, it does not handle the case that the pod terminates between returning the ready status and attaching, as can be the case in the : Options to fix this: Fix kubectl: If attach fails, fall back to printing the logs If attach fails, check the status again and if it's terminated, print the logs Fix the flaky test: Prevent the container from exiting (e.g. `sh -c 'echo running in container && sleep 30') Prevent the container from entering ready state (e.g. )
Thanks for debugging. I think we should probably do both: If attach fails and the container is now terminated, fall back to printing logs Make the e2e more robust by not exiting for a while to ensure time to attach. I'm happy to review PRs


Question: return false default: // Normal equality suffices <del> if v1.CanInterface() && v2.CanInterface() { return v1.Interface() == v2.Interface() </del> <ins> if !v1.CanInterface() || !v2.CanInterface() { panic(unexportedTypePanic{}) </ins> } <del> return v1.CanInterface() == v2.CanInterface() </del> <ins> return v1.Interface() == v2.Interface() </ins> } }

This tree has a commit that demonstrates two obviously different objects that should not pass the semantic DeepEqual but do.
What shall we do with this issue for triage? Labels? Priority?
P2 for when he returns , Satnam Singh wrote:
OK I am now sufficiently worried about this. Fixes incoming.


Question: "//staging/src/k8s.io/apimachinery/pkg/api/errors:go_default_library", "//staging/src/k8s.io/apimachinery/pkg/apis/meta/v1:go_default_library", "//staging/src/k8s.io/apimachinery/pkg/labels:go_default_library", <ins> "//staging/src/k8s.io/apimachinery/pkg/util/rand:go_default_library", </ins> "//staging/src/k8s.io/apimachinery/pkg/util/wait:go_default_library", "//staging/src/k8s.io/client-go/kubernetes:go_default_library", "//test/e2e/framework:go_default_library",

Which jobs are flaking: pull-kubernetes-e2e-kind Which test(s) are flaking: Testgrid link: Reason for failure: Anything else we need to know: /sig apps
Failure at
Hi I looked into the failures and think it was because the test required the job succeeds 3 times with pseudorandom 50% failure chance within 15 minutes, while there is an exponential back-off delay (10s, 20s, 40s …) capped at 6 minutes before recreating failed pods. As 7 consecutive failures (1/128 chance) could exceed the timeout, the test failed intermittently because of "timed out waiting for the condition". For example, in the failure of , the job pods succeeded and failed in the following orders: After the last failure at , the delay has increased to 640s so no pod was created any more before the test failed at . I think it can be fixed by using a deterministic approach to make the job pods succeed and fail alternately, and its executing time can be reduced to tens of seconds stably. Assigning myself. /assign


Question: Expect(err).NotTo(HaveOccurred(), "while deleting pod") } <del> func testPodFailSubpath(f *framework.Framework, pod *v1.Pod) { testPodFailSubpathError(f, pod, "subPath") </del> <ins> func testPodFailSubpath(f *framework.Framework, pod *v1.Pod, allowContainerTerminationError bool) { testPodFailSubpathError(f, pod, "subPath", allowContainerTerminationError) </ins> } <del> func testPodFailSubpathError(f *framework.Framework, pod *v1.Pod, errorMsg string) { </del> <ins> func testPodFailSubpathError(f *framework.Framework, pod *v1.Pod, errorMsg string, allowContainerTerminationError bool) { </ins> By(fmt.Sprintf("Creating pod %s", pod.Name)) pod, err := f.ClientSet.CoreV1().Pods(f.Namespace.Name).Create(pod) Expect(err).ToNot(HaveOccurred(), "while creating pod")

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!--What happened: The read only sig-storage tests validate that Pods can come up when readOnly is set. But it doesn't actually validate that the container cannot write to the volume. What you expected to happen: Tests should validate that the container cannot write to readonly volumes. <!-- DO NOT EDIT BELOW THIS LINE --/kind bug
/help
This request has been marked as needing help from a contributor. Please ensure the request meets the requirements listed . If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : /help Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
can this issue be closed after your last PR or is there more to work on? If yes I'd like to know what's left because I'm looking for something to work on
I think we should actually reinstate the test case I removed and enhance it to handle the two scenarios I mentioned. It would also be good to see if we have other readOnly test cases that could be enhanced as well
Thanks ! I have a few questions: 1) how do I run the tests in locally? and 2) how can I assert writing to volume has failed? Thank you in advance!!


Question: filteredMinions := v.saltMinionsByRole(minions, "kubernetes-pool") var instances []string for _, instance := range filteredMinions { <del> instances = append(instances, instance.Host) </del> <ins> // With no dedicated DNS setup in cluster, IP address is used as hostname instances = append(instances, instance.IP) </ins> } return instances, nil

In the guestbook example, the redis slaves are instructed to connect to the redis-master using the proxy provided by the minion on which they reside. The minion is designated by its hostname "kubernetes-minion-N". The container cannot resolve this name to an IP address and so slave connections fail. 1) create vagrant cluster 2) create redis-master pod 3) create redis-master service 4) create redis-slave replicationController Verify redis-slave pod locations: list pods ID Image(s) Host Labels Status redis-master-2 dockerfile/redis kubernetes-minion-1/10.245.2.2 name=redis-master Running -3839-11e4-a922- brendanburns/redis-slave kubernetes-minion-1/10.245.2.2 name=redisslave,replicationController=redisSlaveController Running -3839-11e4-a922- brendanburns/redis-slave kubernetes-minion-3/10.245.2.4 name=redisslave,replicationController=redisSlaveController Running Log onto a minion containing a slave (note: vagrant hostnames are minion-N, not kubernetes-minion-N) Find the slave container docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES brendanburns/redis-slave:latest /bin/sh -c About an hour ago Up About an hour k8s--slave.---3839-11e4-a922- docker logs | tail [7] 09 Sep 17:27:08.428  Connecting to MASTER kubernetes-minion-1: [7] 09 Sep 17:27:08.430 # Unable to connect to MASTER: No such file or directory [7] 09 Sep 17:27:09.435  Connecting to MASTER kubernetes-minion-1: sudo nsenter -m -u -n -i -p -t $(docker inspect --format '{{ .State.Pid }}' ) /bin/bash [ root ]$ ping kubernetes-minion-1 ping: unknown host kubernetes-minion-1 [ root ]$ cat /etc/hosts 244.1.4 -3839-11e4-a922- 127.0.0.

1 localhost ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters [ root ]$ cat nameserver 10.0.2.3 cat /proc/1/environ HOME=/rootPATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/binHOSTNAME=-3839-11e4-a922-0800279696e1REDISMASTERSERVICEPORT=10000REDISMASTERPORT=tcp://kubernetes-minion-1:10000REDISMASTERPORT6379TCP=tcp://kubernetes-minion-1:10000REDISMASTERPORT6379TCPPROTO=tcpREDISMASTERPORT6379TCPPORT=10000REDISMASTERPORT6379TCPADDR=kubernetes-minion-1SERVICE_HOST=kubernetes-
Tim - are you the right guy to handle this? I think we need to resolve the name of the service host and only give IPs to the minion in env variables. We can't assume the containers have the same DNS set up as the host system.
To be clear -- this example is for the checked in Vagrant stuff? If that is the case would be the guy? I'm trying to catch up on IRC discussions from earlier today.
I can look at this, but there are two possible solutions. The first solution is in the vagrant cluster (or any other local vm setup) we could run a dns name server on the kubernetes-master that can resolve each minion by host name. We would then pass a -dns option to the docker daemon to specify that name server to use in each container. The second solution is that we pass IP address instead of host name and make no assumption or override on the containers specified name server. Is there any obvious negative to this solution? Has much thought been done on the expected Nameserver used by containers spawned by our service? - if you can weigh in, I would be happy to resolve using either approach. My preference is solution two.
So, the vagrant case is a little special as there is no sane DNS set up by default. Another suggestion (3) would be to revert the stuff and just devolve to using ip addresses directly. I would do (3) before (2) before (1).

But I'm cool with any of them. As for the nameserver of the containers that are spawned -- docker currently just takes the of the host and copies that into the container. It doesn't do that for . that is the core of the issue here. If/when we do offer some level of split horizon DNS (I'd love to) we can start exposing an extra layer of DNS on top of whatever is configured for the host. That is part of the "ip per service portal" stuff that is being talked about a lot right now.
If I do (3) there is a piece of code in pod storage that mangles my hostname where I return 10.245.x.x and it asks the cloud provider to resolve the IP address for "10". Since other providers appear to also be returning IP as host, I will look to change this so kubernetes makes no modifications to hostnames that it's given. Look for PR tomorrow.
you're right I'd missed that point. This really only applies to the Vagrant cluster and so a fix to the vagrant setup is sufficient. I also agree with your priorities. Injecting a line into /etc/hosts (if that's reasonable) is the simplest solution. Either reverting or allowing a switch to pass the minion public IP rather than hostname is next. Adding a DNS service to the vagrant master and forcing DNS resolution to the master host is most complicated and least preferable. Thanks both of you for helping me think this through better. Mark
I don't grok what the difference between 2 and 3 is? I sent a PR to docker a while back () to insert arbitrary lines to /etc/hosts. I still think it is a reasonable thing to want to do.
- difference between (2) and (3) is that in the past, the vagrant cloud provider was returning the IP address as the literal hostname for the minion. I had changed this after noticing that it had some ugly side-effects in the CLI, where a call to list pods would report pod ip addresses as IP / nil.

The issue (which I now believe is a bug that I want to change independent of what we do here) is that the code at this line mangles the hostname returned by the cloudprovider: This makes using the IP address as a hostname (10.245.2.2) to have the ugly side-effect of the code then asking the cloudprovider for the IP address of hostname "10", and then the cloudprovider having no idea how to proceed. Independent of what is done in this issue, I want to submit a PR tomorrow to stop that code from manipulating the reported hostname that was provided by the cloudprovider before asking that same cloudprovider to return the IP address using a potentially different input value. so with that backlog (3) means revert vagrant cloudprovider to use IPAddress as hostname, which means when we pass in what we believe is hostname in this environment, it will just work because it is in-fact the IPAddress. (2) means do not pass in hostname but pass in IPAddress by changing more core k8s code. all that being said, there is an option (4) that I am intrigued by, - after following your PR through its final conclusion, it looks like as of docker 1.2.0 you can change the /etc/hosts file of a running container. this introduces an option where the kubelet injects the /etc/hosts file of minion into the running container so it has the /etc/hosts values of the minion. I would need to play with that support a little more to know pros/cons more. I think longer term, we need a plan on what nameserver our containers are expecting to reference, and if they are inherently different in some way. I have run into a number of times the issue where doing things like running docker in docker causes the nested container to default to the nameserver 8.8.8.8, which happens to be blocked on the Red Hat network ;-) For now to unblock I will probably pursue (3) and fix the hostname bug referenced here at the same time. It does make the CLI uglier when working with this environment, but not a huge deal, at least it will function.
Thanks both for the complete explanation of what's happening and for the suggested fixes.

I think in the long run using hostnames in SERVICEHOST and ensuriing either through IP injection into container /etc/hosts or by providing DNS are the best options. I'm not sure if there might still be cases where resolution of SERVICEHOST would still be best done by presenting the IP address or not.


Question: go run ./hack/e2e.go -v -build # Push to GCS <del> yes | ./build/push-ci-build.sh </del> <ins> ./build/push-ci-build.sh </ins> sha256sum _output/release-tars/kubernetes*.tar.gz

It looks like a test configuration failure:
There are no sig labels on this issue. Please by:<br(1) mentioning a sig: <bre.g., for API Machinery<br(2) specifying the label manually: <bre.g., for sig/scalability<br<brNote: method (1) will trigger a notification to the team. You can find the team list and label list
/assign
last passing run: https://k8s- first failing run: https://k8s-
https://k8s- /close


Question: } if sumPriorities < minSumPriorities { minSumPriorities = sumPriorities <del> minSumPriorityNodes = nil </del> <ins> lenNodes1 = 0 </ins> } if sumPriorities == minSumPriorities { <del> minSumPriorityNodes = append(minSumPriorityNodes, node) </del> <ins> minNodes1[lenNodes1] = node lenNodes1++ </ins> } } <del> if len(minSumPriorityNodes) == 1 { return minSumPriorityNodes[0] </del> <ins> if lenNodes1 == 1 { return minNodes1[0] </ins> } // There are a few nodes with minimum highest priority victim and sum of priorities. // Find one with the minimum number of pods. minNumPods := math.MaxInt32 <del> var minNumPodNodes []*v1.Node for _, node := range minSumPriorityNodes { </del> <ins> lenNodes2 = 0 for i := 0; i < lenNodes1; i++ { node := minNodes1[i] </ins> numPods := len(nodesToVictims[node].pods) if numPods < minNumPods { minNumPods = numPods <del> minNumPodNodes = nil </del> <ins> lenNodes2 = 0 </ins> } if numPods == minNumPods { <del> minNumPodNodes = append(minNumPodNodes, node) </del> <ins> minNodes2[lenNodes2] = node lenNodes2++ </ins> } } // At this point, even if there are more than one node with the same score, // return the first one. <del> if len(minNumPodNodes) > 0 { return minNumPodNodes[0] </del> <ins> if lenNodes2 > 0 { return minNodes2[0] </ins> } glog.Errorf("Error in logic of node scoring for preemption. We should never reach here!") return nil

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: /kind feature What happened: Fix the following : The basic logic in of using the is continuing reducing the nodes slices through , , and . It needs 4 slices to do this. This is my thought to reuse the slices: Use 2 nodes slices, such as , The reducing process becomes: -Clear -Clear - please give some suggestion, thanks :smile:
/sig scheduling
The challenge is that clearing a slice and reusing it still needs GC clean up, so it may not be effective. The size of slices shrink as we proceed through the function. So, having two slices and over-writing the elements in them can help reduce GC time, but this approach needs some additional checks to ensure we are not using stale elements.
To avoid clearing the slices, this is my thought: Declare to indicate the actual length of the 2 node slices, instead of . Appending a node to a node slice will add 1 to . Clearing a node slice will set to 0. WDYT
Yes, I can't think of any better solution either, but remember that you should append only to the slice in the first for loop of the function. The second slice should be initialized with the same number of elements as the slice used in the first for loop. Setting elements of the slices should be done by indexing them (not by append) in the second, third, and forth loop.
OK, I will submit a PR to implement it.


Question: return []func(*v1.Node) error{ kl.setNodeAddress, withoutError(kl.setNodeStatusInfo), <ins> withoutError(kl.setNodeOODCondition), </ins> withoutError(kl.setNodeMemoryPressureCondition), withoutError(kl.setNodeDiskPressureCondition), withoutError(kl.setNodeReadyCondition),

Both https://k8s- and https://k8s- started failing recently. Their 1.7 counterparts are passing. The difference between the last passing and the first failing tests is: Probably something in broke the tests. /sig scheduling /cc /kind bug
It looks like there isn't enough cpu on the master to run the cos-nvidia-installer daemonset. Did we add any addons recently?
My previous comment is a red herring. It is an issue with . Ill post a fix soon. From the scheduler logs on the master:
that's a benign error. The daemonset is not expected to run on the master. But there is no easy way to express that request in our APIs.
Yes, the pod not scheduling on the master is benign. The actual error is that we do not report node condition , so it is left as . The fix is to report that is false.
The tests are still failing. /reopen
/assign
I will be updating to track progress.
tests are green again! /close


Question: "k8s.io/kubernetes/pkg/kubelet/apis/resourcemetrics/v1alpha1" kubecontainer "k8s.io/kubernetes/pkg/kubelet/container" "k8s.io/kubernetes/pkg/kubelet/prober" <ins> servermetrics "k8s.io/kubernetes/pkg/kubelet/server/metrics" </ins> "k8s.io/kubernetes/pkg/kubelet/server/portforward" remotecommandserver "k8s.io/kubernetes/pkg/kubelet/server/remotecommand" "k8s.io/kubernetes/pkg/kubelet/server/stats"

<!-- Please only use this template for submitting enhancement requests --What would you like to be : 1) add metrics into kubelet to monitor ; 2) add throttling to . Why is this needed: Kubelet provides lots of its functionalities through its http server, and currently there is no server-side monitoring and throttling mechanisms. Adding metrics into kubelet to monitor its http server will improve its observability and debuggability; adding throttling will prevent kubelet being overloaded from serving too many http requests.
/sig node


Question: ####################### # Make sure the UI can be proxied <del> start-proxy --api-prefix=/ </del> <ins> start-proxy </ins> check-curl-proxy-code /ui 301 check-curl-proxy-code /metrics 200 <ins> check-curl-proxy-code /api/ui 404 </ins> if [[ -n "${version}" ]]; then check-curl-proxy-code /api/${version}/namespaces 200 fi <ins> check-curl-proxy-code /static/ 200 </ins> stop-proxy <del> # Default proxy locks you into the /api path (legacy behavior) </del> <ins> # Make sure the in-development api is accessible by default </ins> start-proxy <del> check-curl-proxy-code /ui 404 check-curl-proxy-code /metrics 404 check-curl-proxy-code /api/ui 404 if [[ -n "${version}" ]]; then check-curl-proxy-code /api/${version}/namespaces 200 fi </del> <ins> check-curl-proxy-code /apis 200 check-curl-proxy-code /apis/extensions/ 200 </ins> stop-proxy # Custom paths let you see everything. <del> start-proxy --api-prefix=/custom </del> <ins> start-proxy /custom </ins> check-curl-proxy-code /custom/ui 301 check-curl-proxy-code /custom/metrics 200 if [[ -n "${version}" ]]; then

Now that we've moved the experimental api to /apis/experimental... it's inaccessible from by default. We should fix that. who stepped on this earlier
security questions for you: this prefix was apparently in response to a security review quite some time ago. I don't understand how this adds much security. Can we just remove it? If not, is it acceptable to change the default from "/api/" to "/api", which would match both api locations? Otherwise we'll have to take a comma-separated list or something...
Ref
Oh, so this is not related to the security requests! And the original use case was just to re-root the URL path so it didn't interfere with static content. Sorry, I got confused because the last time I monkeyed with that, it was in conjunction with the allow/deny list that actually was the security request. In that case, let's see if we can change the default to "/", if that will break anything.
That change got introduced as part of to protect against XSRF in kubectl proxy. But I think the more important part of that change was that it disallows POST/PUT/PATCH by default. So as long as there is no state changing action outside of /api/ that can be triggered with a GET it should be safe to remove.


Question: MASTER_EXTRA_METADATA=${KUBE_MASTER_EXTRA_METADATA:-${KUBE_EXTRA_METADATA:-}} # MASTER_EXTRA_METADATA is the extra instance metadata on node instance separated by commas. NODE_EXTRA_METADATA=${KUBE_NODE_EXTRA_METADATA:-${KUBE_EXTRA_METADATA:-}} <ins> # KUBELET_TEST_ARGS are extra arguments passed to kubelet. KUBELET_TEST_ARGS=${KUBE_KUBELET_EXTRA_ARGS:-} </ins> NETWORK=${KUBE_GCE_NETWORK:-default} # Enable network deletion by default (for kube-down), unless we're using 'default' network.

Sometimes users do need to specify some kubelet flags, e.g. configure kubelet container runtime cgroup based on their environment. Currently, user can only specify kubelet args with a hidden env for kube- We should formalize the name exposed to user.
/cc


Question: } if len(o.Output) > 0 { <del> return o.PrintObject(o.Cmd, o.Local, o.Mapper, obj, o.Out) </del> <ins> if err := o.PrintObject(o.Cmd, o.Local, o.Mapper, obj, o.Out); err != nil { return err } continue </ins> } cmdutil.PrintSuccess(o.Mapper, o.ShortOutput, o.Out, info.Mapping.Resource, info.Name, false, "env updated")

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: What happened: my yaml: exec kubectl set resources command,just update first object: What you expected to happen: How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration**: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others:
/sig cli
/kind bug
/assign


Question: testPod := &v1.Pod{ ObjectMeta: metav1.ObjectMeta{ <del> SelfLink:  "/api/v1/namespaces/baz/pods/foo", </del> Name:      "foo", Namespace: "baz", UID:       "bar",

Which jobs are flaking: periodic-kubernetes-bazel-test-master (xref ) Which test(s) are flaking: TestEventSeriesf Testgrid link: Reason for failure: Timeout: Anything else we need to know: To reproduce with stress: /cc
There are no sig labels on this issue. Please add an appropriate label by using one of the following commands: - Please see the for a listing of the SIGs, working groups, and committees available. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign
filed to address this issue.
contains the correct analysis of flakes (thanks a lot!), I commented on way of fixing it (which I'm not convinced about).


Question: func (s *sourceEtcd) run() { boundPods := api.BoundPods{} <del> err := s.helper.ExtractToList(s.key, &boundPods) </del> <ins> err := s.helper.ExtractObj(s.key, &boundPods, false) </ins> if err != nil { glog.Errorf("etcd failed to retrieve the value for the key %q. Error: %v", s.key, err) return

I believe this is a regression with HEAD. Everything runs happily until I upgrade it to HEAD: influx-grafana/ influxdb kubernetes/heapster_influxdb kubernetes-minion- name=influxdb Running Then, I ssh into minion-2 and stop the kubelet. After this, all pods' status in cluster become to Unknown. Then I restart kubelet, and wait for long time (more than 20 mins), but all pods' status stay at Unknown:
I will take a look.
Now I suspect it is not caused by an unhealthy minion. I noticed that another issue filed by today (), all pods' status are Unknown too.
True problem is ExtractList instead of ExtractObj. This will always read an empty boundpods object: But the watch afterwards works correctly. So the weird behavior is that it kills everything when the watch expires. sorry I failed to notice this when reviewing your change! BoundPods isn't technically a list even though it looks like it. (the bound pods are not stored as sub-keys in etcd)
It turns out the issue is more serious than what I initially expected. Any pods scheduled to a node by apiserver, and running for a while will be killed by kubelet because a bug caused by But any changes to etcd key of that boundpods under /registry/nodes/<XX/boundpods will make watch result correct, and restarting previous killed boundPods. To prove my theory, I delete one pod php1 from my test cluster. Before deletion, all pods are unknown: Now after I delete php1 by running: delete -f testing/php1 Running: cluster/../cluster/gce/../../_output/dockerized/bin/linux/amd64/kubectl delete -f testing/php1 php1 All pods on same node (kubernetes-minion-) are restarted I think I know the root cause is, and will sent a fix soon.
I think you hit the same issue earlier.
maybe you can write a little test for this? :)


Question: store Store // listerWatcher is used to perform lists and watches. listerWatcher ListerWatcher <del> // period controls timing between an unsuccessful watch ending and // the beginning of the next list. period time.Duration // The period at which ShouldResync is invoked </del> <ins> // backoff manages backoff of ListWatch backoffManager wait.BackoffManager </ins> resyncPeriod time.Duration // ShouldResync is invoked periodically and whenever it returns `true` the Store's Resync operation is invoked ShouldResync func() bool

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: We recently had a production issue where one of our large cluster (~3k nodes) has it's API server OOMKilled. This caused all kubelets' connection to master broken, and caused a global synchronization of kubelet's reconnecting to master. Due to the burst of API call (18K QPS, 7x than normal), API server went into even worse condition, and caused more replicas to get OOMKilled. Current reflector impl has fixed 1sec backoff before calling next , which shall be the root cause of such instability. What you expected to happen: reflectors to backoff with jitter How to reproduce it (as minimally and precisely as possible): a cluster with a few thousands of nodes api server and reboot it Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others: /assign /sig api-machinery scalability
/cc


Question: return allErrs } <del> // Validates that a Quantity is not negative </del> <ins> // ValidateNonnegativeQuantity checks that a Quantity is not negative. </ins> func ValidateNonnegativeQuantity(value resource.Quantity, fldPath *field.Path) field.ErrorList { allErrs := field.ErrorList{} if value.Cmp(resource.Quantity{}) < 0 {

Anyone itching to understand the code and try fixing things in Kubernetes, please help fixing the lint failures Step 1: Remove one or two packages (related packages) from the list in - (note: do not change generated files, the name of defaulting functions, or the name of conversion functions... packages failing linting due to issues in those files will need to be left alone) Step 2: Run and look at the logged errors Step 3: Fix the code to ensure that runs cleanly. Step 4: Run and to make sure we fix any problems with gofmt or bazel. Step 5: File a PR with the change Step 6: Go to step :) Please use the tracking spreadsheet For everyone who is going to contribute, please append the packages you picked at the end. For existing contributor, please make sure this list is update to date
/sig architecture /sig testing
Hey, I would like to resolve this issue. Could you assign it to me ?
there are a lot of packages so there will be many people working on this. Also we cannot assign issue to folks who are not in the github org for kubernetes. please feel free to pick up a few packages like i mentioned and file a PR. Make sure you link back to this issue, so folks can see which packages you filed PR for.
OK. I see. I will firstly pick up the package called to fix.
Hi dims. Should i submit my PR before i remove one or two packages (related packages) from the list in - Or I first remove the package from list, then I fix the issue and submit a PR?
in your PR, you will have both the fixes to packages and some lines removed from /.golint_failures
When I run , I found some errors as below: I found that there is a go file in But there is still a error.
Say your GOPATH looks like this Your kubernetes directory should be () cd to and then run from that directory.
Is it necessary to obey all rules in golint? Rules in golint are set in stone. Maybe using gometalinter to ignore some specific errors is better?
I revise some erros about golint in some packages. Please review it.
which errors would you like to ignore?

if you can open a new issue with the list of errors to ignore (and proposing gometalinter), we can get some opinions there from others in the community.
Let me take the kubectl cmd part...
Can someone please review and let me know if I am doing it right?
I ll work on all the packages related to
we get 800-ish pkgs left to fix. my concern is that if we fix them package by package, then such pulls will be flooding the repo. can't we possibly make it in larger pulls? the larger the better?
+1 to slightly larger pulls with related packages. if allow too large then the reviewers will not be able to do a thorough review. we should spread the load among multiple reviewers. (if we allow a really large pull, we will end up with top level reviewers who will face the burden and the PR(s) will just stagnate)
oh good point. we could put one pkg's change into one commit, and merge all changes under one area into one pull so that it wont spread the load. wdyt? for each pull, we can have hundred-ish lines of change. but it will still be 30~50 pulls i guess? good news is that we can take our time to deal with it, at least before 1.12
sounds great! please feel free to help make it happen :) (by suggesting groups of packages that kinda go together)
I'm looking at fixing pkg/proxy, and I think I see what was talking about ignoring some rules. proxy.ProxyProvider is being suggested because golint is saying that it stutters. In that example, I think it's okay renaming it to proxy.Provider, but I could see an example where you would want to keep the stutter if the package that it's declared in uses it heavily.
sure thing, feel free to propose a PR for switching to gometalinter independently of other patches. We can work through that there.
I sent the pull request to fix pkg/volume/aws_ebs
I've fixed a few of these, but get the following error when I try to run : Are there any instructions on what I need to setup in order to resolve this?

you can find this file in
I have raised a PR to fix golint errors in pkg/api
I have fixed some pkgs under /pkg/kubelet pkg/kubelet/images pkg/kubelet/kuberuntime pkg/kubelet/leaky pkg/kubelet/pleg
I submitted a pull request for golint errors in pkg/security/ edit: I can work on picking up pkg/controller/bootstrap through pkg/controller/namespace/... (114-129)
fix golint errors in package cmd/kubeadm/app, since the codes under cmd/kubeadm/app/apis/kubeadm/v1alpha2 and cmd/kubeadm/app/apis/kubeadm/v1alpha3 are auto-generated, i keep them untouched.
I am working on some packages under pkg/kubelet, anyone interested in this plz leave messages for me :)
fix golint for pkg/util/dbus pkg/util/labels pkg/util/node pkg/util/strings pkg/util/workqueue/prometheus
PR fixes golint errors in pkg/routes,pkg/securitycontext, pkg/serviceaccount
PR fixes golint in pkg/probe
should we have a checklist as part of the issue summary saying what is in progress and what is done? Since this issue is getting long, this will make new contributors to pick up issues easier.
Great idea!, want to throw a google doc with a check-list so it's easy for participants to organize themselves? (please!) If it is the issue summary then one of us with privileges will be a bottle neck.
Created a with all the package that have on going PRs. Maybe you can add this to the summary. For everyone who is going to contribute, please append the packages you picked at the end. For existing contributor, please make sure this list is update to date
Thanks updated the spread sheet
Raised PR to fix golint issues in: pkg/controller/clusterroleaggregation pkg/controller/cronjob pkg/controller/daemon pkg/controller/deployment pkg/controller/deployment/util pkg/controller/disruption pkg/controller/endpoint
This issue is close in accident by . could you reopen it?
/reopen
Reopening this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I found a problem trying to fix golint messages for .

Golint complaints on 's function because of the snake case. When I try to fix it, fails because it is being used in a code generated file. If I clean generated files, tests pass, but the generated file is not invoking it anymore. I failed to find the generator definition using this so I could change the hook to just . Also I'm not sure if this is wanted. I also found other packages in also probably affected by this. I read somewhere about replacing golint with gometalinter, which would allow to define directives in code to omit certain rules. I would consider to have a second exclusion file for golint with these cases, to be updated if move to gometalinter is finally done. This would make this issue completion more feasible. Meanwhile, I'll move onto fixing golint messages for other packages, until this discussion gets cleared.
I also found a problem fixing . A test fails because the directory is marked as readonly (see tests @ Shall we also ignore this directory for this issue?
Is this issue still relevant because hack/verify- is giving me no errors.
the failures are still present in there are details in the first message on how to claim + solve issues :)
Sorry I misunderstood the issue
I'm working on this package:
pkg/apis/admission/v1beta1 pkg/apis/authentication/v1beta1 pkg/apis/authorization/v1beta1 pkg/apis/coordination/v1 pkg/apis/coordination/v1beta1 pkg/apis/imagepolicy/v1alpha1 pkg/apis/scheduling/v1alpha1 pkg/apis/scheduling/v1beta1 pkg/apis/settings/v1alpha1 pkg/apis/storage/v1alpha1
pkg/cloudprovider/providers/vsphere/vclib pkg/kubelet/apis/deviceplugin/v1alpha pkg/kubelet/apis/pluginregistration/v1alpha1 pkg/kubelet/apis/pluginregistration/v1beta1
:
I'm looking at the pkg/registry/*, all those which are left. I expect to give an update on what all is fixed in the next 24-48 hours.
hi there is a google sheet that is a bit outdated, maybe you can update that?
hi I seem to only have view access for the sheet. Can't edit.
I fixed the permission.

Please try it again
pkg/apis/authentication pkg/apis/coordination pkg/apis/events pkg/apis/imagepolicy pkg/apis/settings
hi I think this issue was closed prematurely by (using fixes #issue in commit)
I'll put in a PR shortly for the package
Looking at next
hi thanks! there is a spreadsheet here that tracks the progress on fixing these issues, consider adding the packages you're work on there so we can keep everything organized :+1:
Starting on now
I did indeed see the Google Sheet link (thank you and to take it a little further
test/e2e
test/e2e_node/builder
pkg/registry/admissionregistration/mutatingwebhookconfiguration/storage pkg/registry/admissionregistration/rest pkg/registry/admissionregistration/validatingwebhookconfiguration/storage pkg/registry/apps/daemonset/storage
Hi. I opened a PR - Please review.
Working on Will open a PR soon.
Starting on and should have a PR soon
I keep getting this golint verify error on and was wondering if others saw it: But when I that package to the golint failures in my PR, my reviewer said it isn't necessary? What does that mean? What did I miss? Should I use a different kind of verify? cf:
looks like the disconnect is because that problem does not show up in the version of we use in the CI. please see
Still confused: is that not the golint version I ran? it installs from vendor and uses that I think?
i installed from and i still don't see the problem, can you please check what version of golint you have?
I'm confused why I wouldn't be using the vendored version? how would that get out of sync?
/me looking
Note: I am reserving for a KubeCon Contributor Workshop demonstration. It is noted in the spreadsheet.
I think may not be ideal cause there's a lot of auto generated files there. What do you think about reserving instead for the KubeCON Contributor Workshop?
pkg/apis/scheduling pkg/apis/storage/util pkg/apis/storage/v1beta1/util pkg/apis/storage/v1/util
/priority backlog
Hi I am a new contributor here, I'd like to contribute and make changes to the following file/s. I have marked it as WIP in the Google Sheet. Can someone let me know if I have to do anything else before I start working on this PR?

Umm so there's an issue where Golint tells that underscore shouldn't be allowed. Here's a log of the go-lint warnings:
Thanks The following should not be modified, as they are designed like this for code generation.
I'd like to pick a package and do my first PR, is there a list of packages to be done? Or should I correlate with this issue thread? Thanks in advance!
please see step 1 () please scan other PRs linked to this issue as well just in case.
Will do. Thank you. On Thu, Oct 3, 2019, 7:25 AM Davanum Srinivas <:
Hi there, I'm looking for a good first issue. I'll start taking a look at if that's ok? There doesn't seem to be anyone currently tracked as looking at it in the attached spreadsheet.
I've found a few others I'm actively looking at, you won't get any argument from me. On Sat, Oct 12, 2019, 10:38 AM Pamela Kelly <:
I am working on the iptables pkg. Updated in the google docs
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I have problem running in step 2 $ hack/verify- what should I do?
PTAL for my pr to this issue, thx
you can move kubernetes directory to and re-run this script.
I'd like to take Can we fix linter issues for generated files? Or we should skip packages with zz_generated?
There're 2 lint errors that affects 292/507 packages, meaning it's not possible to cleanup the entire file without solving it: default pattern of Kubectl is the usage of [cmd]Options for initial options structs and it raises: /apis/{custommetrics,externalmetrics} for example raises the underscore error, but It's probably not worth to break backward compatibility renaming these folder for lint error fix: Any advice on these?

/reopen hm, cannot reopen this issue, it was closed by mistake
You can't reopen an issue/PR unless you authored it or you are a collaborator. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I am sure this is all not fixed. Am I missing something? ./hack/verify- installing golint from vendor Congratulations! All Go source files have been linted.
you need to remove package from first.
/remove-sig testing SIG testing doesn't own code failing lints, we own the linter. :upsidedownface:
Hello, is this still a relevent issue ? I would like to work upon
I have fixed
Hi Folks, Some of the golint fixes, require us to change the exposed function's name, in simple words, "change API". I was not sure, if that is welcomed and what is the process.
Heres what Im getting after going over the following steps as mentioned by Say your GOPATH looks like this GOPATH=/Users/dims/go Your kubernetes directory should be ($) cd to and then run hack/verify- from that directory. I get the following output - installing golint Congratulations! All Go source files have been linted. Im not really sure what I am doing wrong. Any help is appreciated!
I think I understand. I misunderstood the steps around .golint_failures. The first step is to remove the pkg that we are targeting from this file and then run the My mistake was to assume that the packages in the file need to be removed after we are done cleaning the golint errors.
Can someone please review and let me know if I've done this right?
After a couple years of seeing changes spawned from this issue, I don't think this is a good use of contributors' or reviewers' time.

golint is very opinionated about: exported type and function names, leading to PRs attempting to change exported things people use downstream missing documentation on interface implementation functions, which leads to not-very-useful copy/paste of interface godoc the govet and staticcheck verify jobs are much more useful since they catch correctness issues, but I would recommend closing this issue and dropping the golint checks from CI golint doc itself discourages from using its output in an automated enforced way: from
i guess one can say gofmt is opinionated too, but something good about Golang is that it comes with an opinion and a set of standard tools. for API documentation at least we found inconsistencies on how most exported API items are documented: golint can help with that, but if there are no "pragmas" to avoid breaking changes and given we have codegen generating non-compliant names enabling golint on API packages is sort of blocked in Kubernetes. for non-API, non-library, non-public code, i think being inconsistent is mostly OK. no preference on my side. shrug
true, but code formatting has the benefits of mostly eliminating meaningless diffs in PRs due to code movement/whitespace/etc if there are specific packages (like API packages) we lack documentation on, adding checks specifically for that could make sense exactly, I don't think golint is the right tool for making sure we have API docs
golint itself doesn't have these and the go project is against this sort of comment AIUI, however golangci-lint does have them. I think there are plenty of other static analysis lints we should use instead, beyond just staticcheck and vet there is e.g.
btw, if we remove golint can we see e.g. all-caps constants or functions with underscores dodging review? perhaps we should add golangci-lint soon after:
Completely agree there are better places to spend review bandwidth. I was curious to see what the effort looked like. There have been 890 commits that touched over the past 3+ years !
/remove-help /remove-good-first-issue
Overall agree on the bandwidth. Also note that it was a good first issue with very low bar to get them in the repo. May be we need to find a similar low bar, but high value good first issue.

UT's come to mind, are there other things we curate for new contributors ?
Do we have an alternative way of enforcing that new exported code (not necessarily API fields) has at least some minimum documentation? As a reviewer, I ask contributors to add meaningful comments. But reality is that I sometimes stumble into old code that doesn't have documentation because we didn't have golint at some point. I agree that fix-golint PRs just add burden to reviewers. But, before removing the lint check, can we wait and see if it's enough to discourage people from sending more PRs like this? Additionally, we can encourage reviewers to just close PRs that only fix golint issues.
+1 for adding burden to reviewers. However, as a new contributor, I think this a very good opportunity for new contributors to study the code base and submit their first PR. In my opinion, I think this issue would be able to attract more new contributors, get familiar with the code base, and get familiar with the git workflow.
thanks everyone for getting us this far. We essentially gave up on the existing structure/system for golint: At some point we can look at adding golangci-lint, there's a WIP in progress in /close
Closing this issue. <detailsIn response to : thanks everyone for getting us this far. We essentially gave up on the existing structure/system for golint: Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: volName string // the underlying plugin plugin *flexVolumePlugin <ins> // the metric plugin volume.MetricsProvider </ins> } // volume.Volume interface

kind feature What happened: Currently there is no mechanism to report metrics as the flex plugins are not enabled to collect metrics. All flex volumes lack this feature as compared to in-tree/built-in drivers even though it is possible to collect such metrics for most of the plugins. What you expected to happen: Depending on the availability of the metrics from the mounted FS, flex plugins should allow to report metrics for the FS. How to reproduce it (as minimally and precisely as possible): The default behavior that can be proved easily by just running a command similar to the following command on a node that is running a flex volume. This will show that there are no metrics collected for flex volumes. kubectl proxy $curl 127.0.0.1:8001/api/v1/nodes/<IP/proxy/stats/summary Environment**: Kubernetes version (use ): 1.12 Cloud provider or hardware configuration: local x86 OS (e.g. from /etc/os-release): Ubuntu 16.04 Kernel (e.g. ): Linux KubeStorage 4.4.0-130-generic -Ubuntu SMP Thu Jun 14 08:53:28 UTC 2018 x8664 x8664 x86_64 GNU/Linux Install tools: Others: /sig storage
+1 Yes, it will be good, if flex volume can support atleast the default metrics( i.e ), which most of the volume plugins support now. Based upon flex plugin "capabilities" parameters, it can be decided, whether the plugin supports "default metrics" or "nil metrics".


Question: if pod == nil { return nil } <del> tolerations, err := getTolerationListFromPod(pod) if err != nil { return nil } </del> <ins> tolerationsPreferNoSchedule := getAllTolerationPreferNoSchedule(pod.Spec.Tolerations) </ins> return &priorityMetadata{ nonZeroRequest: getNonZeroRequests(pod), <del> podTolerations: tolerations, </del> <ins> podTolerations: tolerationsPreferNoSchedule, </ins> affinity:       schedulercache.ReconcileAffinity(pod), } }

Kubernetes version (use kubectl version): master What happened: Unnecessary always nil return value for function getTolerationListFromPod. What you expected to happen: We need to remove error return value for getTolerationListFromPod function.
/cc
cc


Question: "github.com/onsi/ginkgo/v2" "github.com/onsi/gomega" <ins> "github.com/onsi/gomega/gstruct" </ins> ) // Validate PV/PVC, create and verify writer pod, delete the PVC, and validate the PV's

[x] test approval issue : [x] test pr : [x] Flake [x] two weeks soak end date : 31 Oct 2023 [X] test promotion pr : [X] remove endpoint from According to following APIsnoop query, there are three PersistentVolume and three PersistentVolumeClaim endpoints that are untested. Using a number of existing e2e test practices a new has been created to provide future Conformance coverage for the 6 endpoints. The e2e logs for this test are listed below. This query shows the following PV and PVC endpoints are hit within a short period of running this e2e test. If a test with these calls gets merged, test coverage will go up by 6 points This test is also created with the goal of conformance promotion. /sig testing /sig architecture /area conformance
/triage accepted /cc
/reopen For tracking the rest of the steps to Promotion
Reopened this issue. <detailsIn response to : /reopen For tracking the rest of the steps to Promotion Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: _ "k8s.io/kubernetes/test/e2e/framework/providers/azure" _ "k8s.io/kubernetes/test/e2e/framework/providers/gce" _ "k8s.io/kubernetes/test/e2e/framework/providers/kubemark" <ins> _ "k8s.io/kubernetes/test/e2e/framework/providers/openstack" </ins> ) var (

What happened: When specifying "openstack" as cloud-provider of e2e test, the test is failed like What you expected to happen: We can run e2e test with openstack cloud provider. How to reproduce it (as minimally and precisely as possible): Just kick the following command: ` Anything else we need to know?: This issue doesn't happen on v1.13, so this issue seems to happen during this dev cycle for v1.14. Environment: Kubernetes version (use ): master for v1.14 Cloud provider or hardware configuration: OpenStack OS (e.g: ): Ubuntu 16.04.5 LTS Kernel (e.g. ): 4.4.0-142-generic Install tools: kubeadm Others:
/sig openstack /sig testing
The exception raises at
The reason of this issue comes from
Ideally it is nice to put a new directory for registering the cloud provider.
/assign
I was wondering we could remove "openstack" provider specific tests and conditions from e2e tests, but there are storage tests which are related to OpenStack in e2e tests. So it is nice to add registration for OpenStack at this time.


Question: // DeleteRoute from the cloud environment. func (g *Cloud) DeleteRoute(ctx context.Context, clusterName string, route *cloudprovider.Route) error { <del> timeoutCtx, cancel := cloud.ContextWithCallTimeout() </del> <ins> timeoutCtx, cancel := context.WithTimeout(ctx, 1*time.Hour) </ins> defer cancel() mc := newRoutesMetricContext("delete")

What happened: Ref , there are some GCE wrapper functions that don't properly respect the passed in context, e.g: We need to figure out a better way to handle this, but I'm hesitate to change it right away without enough thinking. File this issue for tracking purpose. cc
/sig cloud-provider /area provider/gcp
How about checking the deadline of the passed in ctx: If the timeout is longer than what cloud.ContextWithCallTimeout expects, create context with 1 hour timeout.
We should consider () as well as deadline. How about monitor user's timeout like this? cancel our context when timeout.
If the time period till ctx.Deadline() is shorter than 1 hour, there is no need to call cloud.ContextWithCallTimeout(). We can use ctx directly in that case.
Another option may be better:
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale


Question: return utilerrors.NewAggregate(errList) } <del> // Find process(es) with a specified name (exact match) </del> <ins> // Find process(es) with a specified name (regexp match) </ins> // and return their pid(s) func PidOf(name string) ([]int, error) { if len(name) == 0 {

The description of says: However the implementation will interpret the name as a regex. So for example "..." will match any three-letter string. This is not what I understand by "exact match".
Will open an PR fix this comment


Question: Expect(badEvents).NotTo(BeNumerically(">", int(math.Floor(0.01*float64(totalPods))))) }) } <del> type Scalability struct { skip          bool totalPods     int podsPerMinion int rcsPerThread  int } scalabilityTests := []Scalability{ {totalPods: 500, podsPerMinion: 10, rcsPerThread: 5, skip: true}, {totalPods: 500, podsPerMinion: 10, rcsPerThread: 25, skip: true}, } for _, testArg := range scalabilityTests { // # of threads calibrate to totalPods threads := (testArg.totalPods / (testArg.podsPerMinion * testArg.rcsPerThread)) name := fmt.Sprintf( "should be able to launch %v pods, %v per minion, in %v rcs/thread.", testArg.totalPods, testArg.podsPerMinion, testArg.rcsPerThread) if testArg.skip { name = "[Skipped] " + name } itArg := testArg It(name, func() { podsLaunched := 0 var wg sync.WaitGroup wg.Add(threads) // Create queue of pending requests on the api server. for i := 0; i < threads; i++ { go func() { defer wg.Done() for i := 0; i < itArg.rcsPerThread; i++ { name := "my-short-lived-pod" + string(util.NewUUID()) n := itArg.podsPerMinion * minionCount expectNoError(RunRC(c, name, ns, "gcr.io/google_containers/pause:go", n)) podsLaunched += n Logf("Launched %v pods so far...", podsLaunched) err := DeleteRC(c, ns, name) expectNoError(err) } }() } // Wait for all the pods from all the RC's to return. wg.Wait() Logf("%v pods out of %v launched", podsLaunched, itArg.totalPods) }) } </del> })

The issue is to follow up discussions in : running conformance test with the commonly used tool, sonobuoy, failed because the tool's test runner is lack of permission towards . The original issue was resolved by changing the e2e (conformance) test of to an integration test. Several problems remained: privilege requirement in is outdated. discovery is one of the least-privileged permissions, and is expected to be granted to every authenticated user but is not actually. the result returned by should be part of conformance? 1st and 2nd are straightforward to fix, not sure about 3rd, should we add a separate test or add a verification step in existing one like With the fix of 2nd, e2e for should no longer fail with any test runner. cc - should be reflect the accurate permission requirement discovery permission should be granted to all authenticated users. test should check the result returned by . Inapplicable No response <details</details<details</details<details</details<details</details<details</details<details</details
/sig api-machinery /sig testing
I agree with this. We have conformance tests around things like webhooks and custom resource definitions. I'd recommend rephrasing that as "limits itself to capabilities exposed via APIs (e.g., does not require root on nodes, access to raw network interfaces) and does not require write access to system namespaces (like kube-system)" I'm not sure I agree with this. No binaries Kubernetes provides (that I'm aware of) and nothing in client-go needs to make use of root discovery information. I'd be very slow to expand endpoints included in . Unless we have compelling reasons for people to use and rely on this data, I would lean towards no.
can we fix 1 , that seems that everybody agree and looks like the most clear one? the other sound more contentious
thanks for your comments. Created for 1 and closed the other one.
Thanks all. Agree on this outcome.
/triage accepted


Question: } func TestPluginsConfigurationCompatibility(t *testing.T) { <ins> defaultPlugins := map[string][]config.Plugin{ "QueueSortPlugin": { {Name: "PrioritySort"}, }, "PreFilterPlugin": { {Name: "NodeResourcesFit"}, {Name: "NodePorts"}, {Name: "InterPodAffinity"}, {Name: "PodTopologySpread"}, }, "FilterPlugin": { {Name: "NodeUnschedulable"}, {Name: "NodeResourcesFit"}, {Name: "NodeName"}, {Name: "NodePorts"}, {Name: "NodeAffinity"}, {Name: "VolumeRestrictions"}, {Name: "TaintToleration"}, {Name: "EBSLimits"}, {Name: "GCEPDLimits"}, {Name: "NodeVolumeLimits"}, {Name: "AzureDiskLimits"}, {Name: "VolumeBinding"}, {Name: "VolumeZone"}, {Name: "InterPodAffinity"}, {Name: "PodTopologySpread"}, }, "PreScorePlugin": { {Name: "InterPodAffinity"}, {Name: "DefaultPodTopologySpread"}, {Name: "TaintToleration"}, {Name: "PodTopologySpread"}, }, "ScorePlugin": { {Name: "NodeResourcesBalancedAllocation", Weight: 1}, {Name: "ImageLocality", Weight: 1}, {Name: "InterPodAffinity", Weight: 1}, {Name: "NodeResourcesLeastAllocated", Weight: 1}, {Name: "NodeAffinity", Weight: 1}, {Name: "NodePreferAvoidPods", Weight: 10000}, {Name: "DefaultPodTopologySpread", Weight: 1}, {Name: "TaintToleration", Weight: 1}, {Name: "PodTopologySpread", Weight: 1}, }, "BindPlugin": {{Name: "DefaultBinder"}}, } </ins> testcases := []struct { <del> name        string plugins     config.Plugins wantPlugins map[string][]config.Plugin </del> <ins> name         string plugins      config.Plugins wantPlugins  map[string][]config.Plugin pluginConfig []config.PluginConfig </ins> }{ { <del> name: "No plugins specified", wantPlugins: map[string][]config.Plugin{ "QueueSortPlugin": { {Name: "PrioritySort"}, </del> <ins> name:        "default plugins", wantPlugins: defaultPlugins, }, { name:        "default plugins with customized plugin config", wantPlugins: defaultPlugins, pluginConfig: []config.PluginConfig{ { Name: "NodeResourcesFit", Args: runtime.Unknown{ Raw: []byte(`{ "ignoredResources": [ "foo", "bar" ] }`), }, </ins> }, <del> "PreFilterPlugin": { {Name: "NodeResourcesFit"}, {Name: "NodePorts"}, {Name: "InterPodAffinity"}, {Name: "PodTopologySpread"}, </del> <ins> { Name: "PodTopologySpread", Args: runtime.Unknown{ Raw: []byte(`{ "defaultConstraints": [ { "maxSkew": 1, "topologyKey": "foo", "whenUnsatisfiable": "DoNotSchedule" }, { "maxSkew": 10, "topologyKey": "bar", "whenUnsatisfiable": "ScheduleAnyway" } ] }`), }, </ins> }, <del> "FilterPlugin": { {Name: "NodeUnschedulable"}, {Name: "NodeResourcesFit"}, {Name: "NodeName"}, {Name: "NodePorts"}, {Name: "NodeAffinity"}, {Name: "VolumeRestrictions"}, {Name: "TaintToleration"}, {Name: "EBSLimits"}, {Name: "GCEPDLimits"}, {Name: "NodeVolumeLimits"}, {Name: "AzureDiskLimits"}, {Name: "VolumeBinding"}, {Name: "VolumeZone"}, {Name: "InterPodAffinity"}, {Name: "PodTopologySpread"}, </del> <ins> { Name: "RequestedToCapacityRatio", Args: runtime.Unknown{ Raw: []byte(`{ "shape":[ "Utilization": 5, "Score": 5 ], "resources":[ "Name": "cpu", "Weight": 10 ] }`), }, </ins> }, <del> "PreScorePlugin": { {Name: "InterPodAffinity"}, {Name: "DefaultPodTopologySpread"}, {Name: "TaintToleration"}, {Name: "PodTopologySpread"}, </del> <ins> { Name: "InterPodAffinity", Args: runtime.Unknown{ Raw: []byte(`{ "HardPodAffinityWeight": 100 }`), }, </ins> }, <del> "ScorePlugin": { {Name: "NodeResourcesBalancedAllocation", Weight: 1}, {Name: "ImageLocality", Weight: 1}, {Name: "InterPodAffinity", Weight: 1}, {Name: "NodeResourcesLeastAllocated", Weight: 1}, {Name: "NodeAffinity", Weight: 1}, {Name: "NodePreferAvoidPods", Weight: 10000}, {Name: "DefaultPodTopologySpread", Weight: 1}, {Name: "TaintToleration", Weight: 1}, {Name: "PodTopologySpread", Weight: 1}, </del> <ins> { Name: "NodeLabel", Args: runtime.Unknown{ Raw: []byte(`{ "presentLabels": [ "foo", "bar" ], "absentLabels": [ "apple" ], "presentLabelsPreference": [ "dog" ], "absentLabelsPreference": [ "cat" ] }`), }, }, { Name: "ServiceAffinity", Args: runtime.Unknown{ Raw: []byte(`{ affinityLabels: [ "foo", "bar" ], antiAffinityLabelsPreference: [ "disk", "flash" ] }`), }, </ins> }, <del> "BindPlugin": {{Name: "DefaultBinder"}}, </del> }, }, { <del> name: "Disable some default plugins", </del> <ins> name: "disable some default plugins", </ins> plugins: config.Plugins{ PreFilter: &config.PluginSet{ Disabled: []config.Plugin{

has compatibility tests for Policy v1. We are lacking similar tests for other options of , such as: [x] : This can be right away to test that default plugins can be disabled, changed order, etc. [x] : This one is trickier, as it requires a new method in to expose all configurations and a new method in so that they can say which args they used. /sig scheduling /help /kind cleanup
/assign
you can work on the first one. I'll have a PR soon that should clarify how the second one should be done.
I have sent a pr for the first one. And I'm working on the second one, I think it's better to keep 2 PRs for this issue? If so, PTAL at the first one, thanks.
Yes, 2 PRs is ideal. I'll review later today.
I have one small question about do I need to mock some fake plugins for testing , or just testing the default plugins?
You should use the existing plugins, because these are compatibility tests.
Okay
Actually, I think adding a new method such as to interface is a little redundant. In fact, most plugins don't need any argument for building them, so we are talking about adding a new method that needs to be implemented by all of the existing and coming plugins. And users who write out-of-tree plugins also hardly have the need for using this method. IMO, it would be more meaningful to move up to the interface. Then by this method, we can simply implement for default plugins who actually need args. WDYT?
should remain unexported because it provides pointers. However, I agree that might be too much to ask for all plugins. Instead, we can add a new interface and then plugins can opt-int. Not sure if we should follow this extension pattern Any thoughts
On the other hand, simply returning doesn't seem like much.
Sure, then I will add like what we want before(add this method to interface), and if there is any better idea later, please let me know.
I think we should just add it to the plugins that actually have args, this will require custom code in the test, but I think it is better than having all plugins implement a function means something to only 1% of them.
Thanks, but now there is a problem that we can't get the registered plugins with current API.

So, I'm not sure if we can just implement the method in with some reflect code for checking if one plugin has implement method? This could help us with keeping the interface clean. cc
you don't need to check if it implements BuildArgs, since this is for testing, we can hardcode which plugins should have their args verified.
The test will not be pretty, but I think it is worth the tradeoff.
Do you mean that we don't test the default plugins, but define some custom plugins in the test file? And we don't need the ListPluginArgs method anymore, am I right?
and I discussed this offline and arrived to this conclusion: For the compatiblity test, we can mock ( in ) to capture the arguments the configuration that is being passed. This can also be used by with We want another test for that checks the build args for the in-tree plugins accessing the internal pointers. These 2 tests should cover the whole stack. Sorry for the back-and-forth. Sometimes it's useful to see a full PR to realize that the initial idea wasn't good enough. Thanks a lot for it.
Sure, that sounds much more clear, thanks. :D
One more question, the is under package , and the compatibility test is under package . So for mocking , we need to change to a function variable in , and add a method for it, right It will look like: Maybe we can add something like to , but that's the implementation details. I want to make sure if the above design is proper.
Or rather, you could add something like: . Then you can call the capturer from if there is one defined.
Got your message in slack, let me reply here. sounds good, and do I need to revert the change in ? I think is not necessary now.
I think they will still be useful for the other test:
Hello are you still working on this?
Ah, I'm so sorry that I'm so busy in the last few weeks, but I will send prs fot this issue in this weekend. Sorry for my delay again.


Question: SyncFrequency:      *syncFrequency, HTTPCheckFrequency: *httpCheckFrequency, } <del> my_kubelet.RunKubelet(*config, *manifestUrl, *etcdServers, *address, *port) </del> <ins> my_kubelet.RunKubelet(*config, *manifestUrl, *etcdServers, *address, *dockerEndpoint, *port) </ins> }

When using or with a key/value that has a multi-line string as value, that value will appear as quoted string in the log message, i.e. the log message will still be a single line. It would be nicer to print the value in text mode so that it goes across multiple lines, with clear indicators for start and end of the value. Indention could be used to avoid mistaking some embedded line as the start of a new log message. In JSON mode, the value should be formatted normally. /sig instrumentation /wg structured-logging Always printing as a single line can can make the log output unreadable ().
When adding something like this, can be reverted and implemented differently.
My original thinking was that only objects that explicitly ask for multi-line output get formatted that way. But that means that we need to review log calls very carefully. An alternative is to automatically switch from quoting strings to multi-line output based on some heuristic, for example when there's more than one line break.
Here's a potential implementation:
If structured logging is needed in any other case I can contribute, please inform if needed.
/triage accepted /assign /priority important-soon /area logging /cc /cc
We have the same problem as in kube-scheduler also in kubelet, it's just that no-one has seen and/or complained about the multi-line strings. With the benchmark + log analysis tool from I found the following multi-line string parameters in a kubelet log: <details</details


Question: Name: "example-no-match", }, Spec: v1.ServiceSpec{ <del> Selector: map[string]string{labelPod3: labelValue}, </del> <ins> Selector:                 map[string]string{labelPod3: labelValue}, PublishNotReadyAddresses: true, </ins> Ports: []v1.ServicePort{{ Name:       "example-no-match", Port:       80,

Which jobs are flaking: ci-kubernetes-kind-e2e-parallel pr:pull-kubernetes-e2e-gce pr:pull-kubernetes-e2e-gce-ubuntu-containerd pr:pull-kubernetes-e2e-gce-ubuntu-containerd ci-kubernetes-kind-ipv6-e2e-parallel Which test(s) are flaking: Second flakiest test in the e2e job according to Testgrid link: Reason for failure: Anything else we need to know: /sig network /assign
needs investigation since we're switching kube-proxy to endpointslice in this release
/triage unresolved Comment when the issue is assessed and confirmed. I am a bot run by vllry.
made me think of , but that's in already.
It's possible it's a test timeout that's just too close, but we need verification of that checking the controller and apiserver logs to make sure there isn't a latent bug
Thanks for catching this! It looks like the timeouts are actually associated with waiting for 2 pods to be ready, not actually anything related to EndpointSlice functionality - timeout is here: Given that, it seems like the obvious solution would be to update the Services to set to and update this block to wait for the Pods to have IPs, not for them to be ready. If that sounds reasonable, I'll work on getting a fix in tomorrow.
/remove-triage unresolved
If that is still exercises the same conditions (I vaguely remember publishNotReadyAddresses changing code paths in the endpoints controller... does it do the same in the endpointslice controller?), that seems reasonable to me.
I think the code path is very similar here, the only real difference should be how the EndpointSlice controller calculates readiness. I've gone ahead and filed a PR that uses this approach, hopefully it helps:
This flaked again this morning /reopen
Reopened this issue. <detailsIn response to : This flaked again this morning Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
This should be /priority critical-urgent given
The top level error message does not make this clear but the issue here is that too many EndpointSlices are being created - 3 instead of 2.
Interesting... that condition should not persist for O(minutes), right?

that seems like it might be a bug, not just a test issue
There's potential that it's not consistently merging 2 ports with 1 endpoint into a single slice and instead creating a slice per port. It also could represent a larger bug. Trying to find a way to consistently reproduce this.
Despite my best efforts I have been unable to reproduce this on my own e2e cluster. I've had 10 test runs going at the same time repeatedly without any luck, any ideas for how to reproduce flaky test failures local would be welcome. In the meantime, I've created a follow up PR that will provide more helpful information if/when this test fails again: My leading theory right now is that this could be a caching issue. The EndpointSlice controller runs a sync, sees that we don't have an EndpointSlice for the given port combination in cache and then goes ahead and creates one, even though a matching EndpointSlice already exists and has not been cached yet. That would result in an annoying situation where an essentially duplicate EndpointSlice was created due to a stale cache. I would think the EndpointSliceTracker would resolve this issue for us though, so not sure. Getting the above PR in will help validate this theory. It's worth noting that if this is actually a duplicate EndpointSlice, clients like kube-proxy will handle this seamlessly because they already have to handle potential duplication in EndpointSlices. That would potentially explain why a test failure like this does not seem to be causing other things to break.
It appears that my theory was correct as witnessed here: I create a .

The key points I've noticed so far: 1 EndpointSlice references pod2 (example-named-port-f7vq5) 2 EndpointSlices reference pod1 (example-named-port-fkgcc and example-named-port-vfp2q) 2 EndpointSlices were created at "2020-07-28T04:57:51Z": example-named-port-f7vq5 (references pod2) example-named-port-vfp2q (references pod1) 1 EndpointSlice was created 1 second later ("2020-07-28T04:57:52Z") example-named-port-fkgcc (references pod1) All 3 EndpointSlices are: On generation 1 - no updates have occurred With a state of ready: true The 2 EndpointSlices referencing pod1 are effectively identical  Of course this is something the EndpointSliceTracker should be handling for us, but even that approach seems less than ideal here. It wouldn't actually prevent the duplicate EndpointSlices from being created, it should just be cleaning up duplicates after they happen. This combined with the race conditions referenced in make me think we may want to find a way to slow down the sync cycle for each Service here. This also feels at least tangentially related to the need for better rate limiting in both this and the Endpoints controller discussed in . /cc
Hmm - actually I don't understand why this is happening. Maybe I'm missing something, but I don't think slowing down controller is needed to solve this problem. Even though the controller usually is not lagging my more than 1s, it is possible that it can be lagging by a minute or more in pathological cases. So we can't just rely on the fact that it's guaranteed to be synced. Can you explain it a bit deeper why this is happening? i.e. some more detailed timeline like: T1: state of cache is XYZ T2: controller computes the ES x1, x2 T3: state of cache in ABC T4: controller computes ES y1 [why?]
Agree, I'm not in favor of slowing down the controller globally to address this. Delaying resync of a particular service until we observe the existence of new endpointslices we know we created (with some max delay) could be reasonable.

That's exactly where I was hoping to end up here, although I'm not quite sure how to do that cleanly yet. To be clear, this is not just to fix this flaky test. This flaky test is a symptom of a much larger and more annoying issue with the current implementation. The EndpointSlice controller is consistently running into errors where it's trying to update EndpointSlices that are stale. This is caused by follow up calls to running before the informer cache is up to date, which happens all the time during rolling updates where rapid churn of EndpointSlices is expected. Although this eventually resolves itself, it's causing a lot more load on API Server than necessary and can potentially result in some bad states where an update fails but a delete succeeds. In this specific instance, it looks like a syncService run is occurring after the first 2 EndpointSlices have been created but before the one referencing pod1 is in the informer cache. That results in the controller creating a new EndpointSlice to reference pod1 and thus we end up with a duplicate. If anything in the future triggers another syncService call that would be resolved. I'd initially thought that EndpointSliceTracker would be thing to trigger that follow up syncService, but I was wrong. That tracker just tracks the latest resource version for each EndpointSlice the controller has created/updated and compares that with EndpointSlices received through create/update events. If the controller doesn't think that EndpointSlice should exist or the controller's resource version for that slice does not match it triggers another syncService. In this case neither of those conditions would be met so no additional syncService call would be triggered. So that's a long way of saying that I think our best possible solution here is to introduce some delay between when Services can be synced. I think the easiest solution that would solve 99% of our problems would be to add something like a 1s delay in between syncs simply by delaying when was called. This would not be ideal, but it would result in an immediate improvement and be easy to backport. It's worth noting that although duplicate EndpointSlices like this or even attempts to update a stale EndpointSlice are not ideal, they are not actually invalid and will eventually resolve themselves. Consumers of this API already have to do some form of deduping.

With that said, I'm hopeful that there's a clean way to implement a solution that actually waited for each EndpointSlice change the controller has made to be reflected in the cache because that would be a better solution. I'll spend some time looking into possible implementations here.
I've created a draft PR that contains my initial progress towards the approach I described above. I don't particularly love this approach so I'm very open to other ways we could solve this issue. It's also still very much a WIP and has significant work remaining. I opened the PR early just so I could get some initial feedback on the direction and potentially inspire other ideas around approaches we could take here.
is still the best approach I can come up with here. It is ready for review if anyone has time to take a look.
Quick update here, I've limited the scope here to the following changes as discussed in Update the flaky test to consider duplicate EndpointSlices valid. Update the docs to make it very clear that consumers of the EndpointSlice API will have to do deduplication (as kube-proxy already does). Update the controller error handling to return immediately if a create, update, or delete errors (in that order). Reviews would be appreciated on all of the above.
Reopening since is not yet merged.
Thanks for catching that Closing now that is merged. /close
Closing this issue. <detailsIn response to : Thanks for catching that Closing now that is merged. Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: func makeEvent(ref *api.ObjectReference, reason, message string) *api.Event { t := util.Now() <ins> namespace := ref.Namespace if namespace == "" { namespace = api.NamespaceDefault } </ins> return &api.Event{ ObjectMeta: api.ObjectMeta{ Name:      fmt.Sprintf("%v.%x", ref.Name, t.UnixNano()), <del> Namespace: ref.Namespace, </del> <ins> Namespace: namespace, </ins> }, InvolvedObject: *ref, Reason:         reason,

Any node object related events from the kubelet does not get recorded in the apiserver. I digged a bit deeper and it looks like kubelet does not sets namespace to be empty and this results in the kubelet trying to post to `/api/<version/events', which a resource that is not exported by the apiserver.
cc


Question: - ./hack/verify-description.sh - ./hack/travis/install-std-race.sh - ./hack/build-go.sh <ins> - GOPATH=$PWD/Godeps/_workspace:$GOPATH go install ./... </ins> script: - KUBE_RACE="-race" KUBE_COVER="y" KUBE_GOVERALLS_BIN="$HOME/gopath/bin/goveralls" KUBE_TIMEOUT='-timeout 300s' KUBE_COVERPROCS=8 ./hack/test-go.sh -- -p=2

Several times over the past few months I've run into HEAD throwing compilation errors when doing , which may not prevent proper functioning of kubernetes but makes some commands during development like very hard to do. It would be great if we could add to Travis CI to prevent this from happening.
See for discussion.
Fix is merged.
This should be re-opened. Travis still doesn't build all packages with or so HEAD still breaks. See for rationale.
Do you mind reopening for now?
Sure. the fix that was merged was for the specific problem, but this bug is about the general class of problems.


Question: # Use skopeo to find these values: https://github.com/containers/skopeo # # Example <del> # Manifest: skopeo inspect docker://gcr.io/k8s-staging-build-image/debian-base:buster-v1.2.0 # Arches: skopeo inspect --raw docker://gcr.io/k8s-staging-build-image/debian-base:buster-v1.2.0 </del> <ins> # Manifest: skopeo inspect docker://gcr.io/k8s-staging-build-image/debian-base:buster-v1.3.0 # Arches: skopeo inspect --raw docker://gcr.io/k8s-staging-build-image/debian-base:buster-v1.3.0 </ins> _DEBIAN_BASE_DIGEST = { <del> "manifest": "sha256:ea668d3febd312e0edfbbdab6bd7d86448ddc8fddb484e20ec76b36a7aeac04c", "amd64": "sha256:2f3e61995bcd4b3a1a0aef49e4a7a6817c978031734b09df2aaaa28181898b0e", "arm": "sha256:d1073dcf8f1d55fbbd297e5b280375b6f276ea83a08a25fd59dc4f3ca43c6d50", "arm64": "sha256:dadcff1ab81177de4914f6db0e7d78a52e525daf7a1efb246cb3545de5e818d1", "ppc64le": "sha256:bfb24dc0d1e71e1deb0f04a078fadf2c94070266746b1b5acc4e739aa57d5601", "s390x": "sha256:cfe6a3508b7ee198cb5a0b3a62e0981676b1dfa4b3049f36398d03e6bd35a801", </del> <ins> "manifest": "sha256:d66137c7c362d1026dca670d1ff4c25e5b0770e8ace87ac3d008d52e4b0db338", "amd64": "sha256:a5ab028d9a730b78af9abb15b5db9b2e6f82448ab269d6f3a07d1834c571ccc6", "arm": "sha256:94e611363760607366ca1fed9375105b6c5fc922ab1249869b708690ca13733c", "arm64": "sha256:83512c52d44587271cd0f355c0a9a7e6c2412ddc66b8a8eb98f994277297a72f", "ppc64le": "sha256:9c8284b2797b114ebe8f3f1b2b5817a9c7f07f3f82513c49a30e6191a1acc1fc", "s390x": "sha256:d617637dd4df0bc1cfa524fae3b4892cfe57f7fec9402ad8dfa28e38e82ec688", </ins> } # Use skopeo to find these values: https://github.com/containers/skopeo # # Example <del> # Manifest: skopeo inspect docker://gcr.io/k8s-staging-build-image/debian-iptables:buster-v1.3.0 # Arches: skopeo inspect --raw docker://gcr.io/k8s-staging-build-image/debian-iptables:buster-v1.3.0 </del> <ins> # Manifest: skopeo inspect docker://gcr.io/k8s-staging-build-image/debian-iptables:buster-v1.4.0 # Arches: skopeo inspect --raw docker://gcr.io/k8s-staging-build-image/debian-iptables:buster-v1.4.0 </ins> _DEBIAN_IPTABLES_DIGEST = { <del> "manifest": "sha256:4c9410a4ee555dcb0e8b7bd6fc77c65ac400f7c5bd4555df8187630efaea6ea4", "amd64": "sha256:e30919918299988b318f0208e7fd264dee21a6be9d74bbd9f7fc15e78eade9b4", "arm": "sha256:bf59578f532bfd3378c4a713eeb14cf0dbed224d5ad03f549165f8d853997ca4", "arm64": "sha256:3d7ede6013b0516f1ec3852590895d4a7b6ec8f5e15bebc1a55237bba4538da2", "ppc64le": "sha256:ebd3bb280f8da8fc6a7158b7c2fc59b4552487bffd95d0b5ac1b190aff7b0fd9", "s390x": "sha256:a73b94aace7a571f36149aa917d4c7ee13453ed24a31f26549dd13b386dae4c1", </del> <ins> "manifest": "sha256:87f97cf2b62eb107871ee810f204ccde41affb70b29883aa898e93df85dea0f0", "amd64": "sha256:da837f39cf3af78adb796c0caa9733449ae99e51cf624590c328e4c9951ace7a", "arm": "sha256:bb6677337a4dbc3e578a3e87642d99be740dea391dc5e8987f04211c5e23abcd", "arm64": "sha256:6ad4717d69db2cc47bc2efc91cebb96ba736be1de49e62e0deffdbaf0fa2318c", "ppc64le": "sha256:168ccfeb861239536826a26da24ab5f68bb5349d7439424b7008b01e8f6534fc", "s390x": "sha256:5a88d4f4c29bac5b5c93195059b928f7346be11d0f0f7f6da0e14c0bfdbd1362", </ins> } # Use skopeo to find these values: https://github.com/containers/skopeo

What happened: The latest release of the debian-base image, v2.1.3, contains a vulnerable version of the apt package. CVE-2020- impacts apt version 1.8.2.1. What you expected to happen: Image should be updated to use at least apt 1.8.2.2 to clear the vulnerability. How to reproduce it (as minimally and precisely as possible):
/sig release
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
kube-proxy is using debain as base image. /assign Let me check other components
checked the base image in and I seems that no new debian or ubuntu official image is built for this CVE ?
That is correct, the purpose of this issue is to get a new image built and released, one which contains a resolution for the CVE.
per comments of @ tianon in Will update it once debain releases it.