Question: succeed. One list MUST be found. It MUST succeed at deleting a collection of jobs via a label selector. */ <del> ginkgo.It("should manage the lifecycle of a job", func() { </del> <ins> framework.ConformanceIt("should manage the lifecycle of a job", func() { </ins> jobName := "e2e-" + utilrand.String(5) label := map[string]string{"e2e-job-label": jobName} labelSelector := labels.SelectorFromSet(label).String()

[x] Test approval issue : [x] Test PR : - Needed an update on the handling of anotations 27 March 2022 [x] Two weeks soak end date : 6 April 2022 [x] Test promotion PR : [x] Update Test PR: 15 April 2022 [x] Two weeks soak end date : 4 May 2022 with start of 1.25 [x] Update Test Promotion: According to these APIsnoop queries, there are still four remaining Job endpoints which are untested. Using a number of existing e2e test practices a new has been created for this job lifecycle test. The e2e logs for this test are listed below. This query shows the endpoints hit within a short period of running the e2e test If a test with these calls gets merged, test coverage will go up by 4 points This test is also created with the goal of conformance promotion. /sig testing /sig architecture /area conformance
/triage accepted
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen for tracking e2e testgrid and promotion PR
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: import ( "context" <del> "regexp" </del> "strings" v1 "k8s.io/api/core/v1"

What would you like to be : a check of output for the dnsPolicy tests Why is this needed: dnsPolicy tests right now are confusing to interpret, bc they grep a regex out of the output of . its great to print the output of but i think its more important to really measure wether DNS is plumbed properly into the container by directly attempting to resolve a hostname, since that is what the definition of implies .
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig windows
/assign
cc


Question: func logout(vs *VSphere) { for _, vsphereIns := range vs.vsphereInstanceMap { <del> if vsphereIns.conn.Client != nil { vsphereIns.conn.Logout(context.TODO()) } </del> <ins> vsphereIns.conn.Logout(context.TODO()) </ins> } } // Instances returns an implementation of Instances for vSphere.

What happened: Data race in tests in an unrelated PR Logs https://k8s- /kind bug /area platform/vsphere /area cloudprovider /sig cloud-provider
/kind flake
Another time it flaked in an unrelated PR https://k8s-
bringing this to your attention as a vsphere dev.
/assign
Looks like the race was in Feb 2017 here: But that code path wasn't covered by go test until vcsim support was here: I'll work on a fix.


Question: ```console $ kubectl create -f docs/admin/limitrange/namespace.yaml <del> namespaces/limit-example </del> <ins> namespace "limit-example" created </ins> $ kubectl get namespaces <del> NAME            LABELS             STATUS default         <none>             Active limit-example   <none>             Active </del> <ins> NAME            LABELS    STATUS    AGE default         <none>    Active    5m limit-example   <none>    Active    53s </ins> ``` Step 2: Apply a limit to the namespace

Since resource QoS is implemented, limitrange also supports request constraint, we can add an example for that.
cc


Question: Containers: []v1.Container{ { Image: busyboxImage, <del> Name:  checkContName, </del> <ins> Name:  podName, </ins> // If we find expected log file and contains right content, exit 0 // else, keep checking until test timeout <del> Command: []string{"sh", "-c", "while true; do if [ -e " + expectedlogFile + " ] && grep -q " + logString + " " + expectedlogFile + "; then exit 0; fi; sleep 1; done"}, </del> <ins> Command: []string{"sh", "-c", "while true; do if [ -e " + expectedLogPath + " ] && grep -q " + log + " " + expectedLogPath + "; then exit 0; fi; sleep 1; done"}, </ins> VolumeMounts: []v1.VolumeMount{ { <del> Name: logDirVolumeName, </del> <ins> Name: "logdir", </ins> // mount ContainerLogsDir to the same path in container <del> MountPath: expectedlogFile, </del> <ins> MountPath: expectedLogPath, </ins> ReadOnly:  true, }, },

The should be updated to check CRI log paths.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
May I try to help with this?
GitHub didn't allow me to assign the following users: fatsheep9146. Note that only and repo collaborators can be assigned. For more information please see <detailsIn response to : /assign Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: // Node1 // Image: gcr.io/40:latest 40MB, gcr.io/300:latest 300MB <del> // Score: 100 * ((40M + 300M)/2 - 23M)/(1000M - 23M) = 15 </del> <ins> // Score: 100 * ((40M + 300M)/2 - 23M)/(1000M * 2 - 23M) = 7 </ins> // Node2 // Image: not present // Score: 0 pod:          &v1.Pod{Spec: test40300}, nodes:        []*v1.Node{makeImageNode("machine1", node403002000), makeImageNode("machine2", node25010)}, <del> expectedList: []framework.NodeScore{{Name: "machine1", Score: 15}, {Name: "machine2", Score: 0}}, </del> <ins> expectedList: []framework.NodeScore{{Name: "machine1", Score: 7}, {Name: "machine2", Score: 0}}, </ins> name:         "two images on one node, prefer this node", }, { <del> // Pod: gcr.io/2000 gcr.io/10 </del> <ins> // Pod: gcr.io/4000 gcr.io/10 </ins> // Node1 <del> // Image: gcr.io/2000:latest 2000MB // Score: 100 (2000M/2 >= 1000M, max-threshold) </del> <ins> // Image: gcr.io/4000:latest 2000MB // Score: 100 (4000 * 1/2 >= 1000M * 2, max-threshold) </ins> // Node2 // Image: gcr.io/10:latest 10MB // Score: 0 (10M/2 < 23M, min-threshold) pod:          &v1.Pod{Spec: testMinMax}, <del> nodes:        []*v1.Node{makeImageNode("machine1", node403002000), makeImageNode("machine2", node25010)}, </del> <ins> nodes:        []*v1.Node{makeImageNode("machine1", node400030), makeImageNode("machine2", node25010)}, </ins> expectedList: []framework.NodeScore{{Name: "machine1", Score: framework.MaxNodeScore}, {Name: "machine2", Score: 0}}, name:         "if exceed limit, use limit", }, { <del> // Pod: gcr.io/2000 gcr.io/10 </del> <ins> // Pod: gcr.io/4000 gcr.io/10 </ins> // Node1 <del> // Image: gcr.io/2000:latest 2000MB // Score: 100 * (2000M/3 - 23M)/(1000M - 23M) = 65 </del> <ins> // Image: gcr.io/4000:latest 4000MB // Score: 100 * (4000M/3 - 23M)/(1000M * 2 - 23M) = 66 </ins> // Node2 // Image: gcr.io/10:latest 10MB <del> // Score: 0 (10M/2 < 23M, min-threshold) </del> <ins> // Score: 0 (10M*1/3 < 23M, min-threshold) </ins> // Node3 // Image: // Score: 0 pod:          &v1.Pod{Spec: testMinMax}, <del> nodes:        []*v1.Node{makeImageNode("machine1", node403002000), makeImageNode("machine2", node25010), makeImageNode("machine3", nodeWithNoImages)}, expectedList: []framework.NodeScore{{Name: "machine1", Score: 65}, {Name: "machine2", Score: 0}, {Name: "machine3", Score: 0}}, </del> <ins> nodes:        []*v1.Node{makeImageNode("machine1", node400030), makeImageNode("machine2", node25010), makeImageNode("machine3", nodeWithNoImages)}, expectedList: []framework.NodeScore{{Name: "machine1", Score: 66}, {Name: "machine2", Score: 0}, {Name: "machine3", Score: 0}}, </ins> name:         "if exceed limit, use limit (with node which has no images present)", }, <ins> { // Pod: gcr.io/300 gcr.io/600 gcr.io/900 // Node1 // Image: gcr.io/600:latest 600MB, gcr.io/900:latest 900MB // Score: 100 * (600M * 2/3 + 900M * 2/3 - 23M) / (1000M * 3 - 23M) = 32 // Node2 // Image: gcr.io/300:latest 300MB, gcr.io/600:latest 600MB, gcr.io/900:latest 900MB // Score: 100 * (300M * 1/3 + 600M * 2/3 + 900M * 2/3 - 23M) / (1000M *3 - 23M) = 36 // Node3 // Image: // Score: 0 pod:          &v1.Pod{Spec: test300600900}, nodes:        []*v1.Node{makeImageNode("machine1", node60040900), makeImageNode("machine2", node300600900), makeImageNode("machine3", nodeWithNoImages)}, expectedList: []framework.NodeScore{{Name: "machine1", Score: 32}, {Name: "machine2", Score: 36}, {Name: "machine3", Score: 0}}, name:         "pod with multiple large images, machine2 is preferred", }, { // Pod: gcr.io/30 gcr.io/40 // Node1 // Image: gcr.io/20:latest 20MB, gcr.io/30:latest 30MB gcr.io/40:latest 40MB // Score: 100 * (30M + 40M * 1/2 - 23M) / (1000M * 2 - 23M) = 1 // Node2 // Image: 100 * (30M - 23M) / (1000M * 2 - 23M) = 0 // Score: 0 pod:          &v1.Pod{Spec: test3040}, nodes:        []*v1.Node{makeImageNode("machine1", node203040), makeImageNode("machine2", node400030)}, expectedList: []framework.NodeScore{{Name: "machine1", Score: 1}, {Name: "machine2", Score: 0}}, name:         "pod with multiple small images", }, </ins> } for _, test := range tests {

<!-- Please only use this template for submitting enhancement requests --What would you like to be : redefine the formula, the score should be capped per images instead of pod. Why is this needed: For the pod that going to be scheduled, calculate the score based on the below steps, Scaled score base on the image size and it's spread over the cluster, image[i].nodes: number of nodes that already have image ! Capped score based on the assumption 90% image size on dockerhub falls into the range [23M ~ 1000M] lowerbound: 23M upperbound:1000M ! Final scores returned for scheduling, maxscore: 100 ! The second step is not well guarded, the assumption is based on the image size while the pod may hold multiple images, it is not surprise to see that total image size of the pods is larger than 1000M, while each of might be only around 300M. In this below case, following the above formula, NODE1 and NODE2 willl get the same scores(100), since both of them are capped by the max score (100), the ideal result is score of NODE2 should be larger than NODE1 (all requested images are ready on NODE2) and is preferred, the larger the sample1 here the more scores should be expected for NODE2. !
The label(s) cannot be applied, because the repository doesn't have them <detailsIn response to : /sig scheduler Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scheduling
cc
Thanks a lot for summarizing the problem. I agree that the formula needs improvement. ~I think doing the capping for each image solves the original problem and what said in There are two options: only the total from to for each container. We have to ask ourselves how much signal each formula gives and depending on what. Think about different scenarios: a pod with a lot of very small containers, a pod with a few big containers, etc.
/assign
Here is my thoughts, step1: calculate image level priority, for each image (should be same as exsting func ) step2: cap the each image's score with thresholds, minThreshold and maxThreshold.

(should be same as existing func except that it's per image instead of pod) step3: score normalization per image. step4: calculate pod level priority, and return the final score, range (0~100) step1 and step2 primarily update the exiting code to make it is image level priorities instead of pod level. Based on the above steps, consider the below two cases, a pod with four images, case1: All images are ready on node2 node1 (0, 80, 90, 60) - final score is: (80+90+60) / 4 = 57 node2 (90, 80, 90, 60) - final score is: (90+80+90+60) / 4 = 80 √ case2: each node has three images ready but image1 is larger than image2. node1 (0, 80, 70, 60) - final score is: (80+70+60) / 4 = 52 node2 (90, 0, 70, 60) - final score is: (90 +70+60) / 4 = 55 √ thoughts? cc
Thanks can you update the the examples with the score that we would have gotten with the current function?
For reference, see this analysis for a different plugin
Also it's worth to see the difference with just capping per pod, instead of per container.
can you also describe the image sizes you used? the example in the original post assumed three images, while your analysis is assuming four.
agree with and the explanation in looks good but just to be sure it would be nice to see the math worked out with image sizes, and compared to the current formula
Sorry for the confusion, let me switch back to the original three images, update the image size a little bit to make it easier to understand, and also fit to the case of and . Here is the pod, ! and three nodes in the cluster, ! Let's see the difference between two approaches, capping per pod, step1: ! NODE1: scores = 30 2/3 + 1500 2/3 = 1020 NODE2: scores = 30 2/3 + 600 2/3 + 1500 2/3 = 1420 step2: !

NODE1: 1020 1000 (maxThreshold), cappedscore = 1000 NODE2: 1420 1000 (maxThreshold), cappedscore = 1000 step3: ! upperbound = 100 / lowerbound = 23 / maxscore = 100 NOTE1: score = 100 (1000 - 23) / (1000 - 23) = 100 NODE2: score = 100 (1000 - 23) / (1000 - 23) = 100 potentially, NODE1 will be chose for scheduling. capping per images, step1: calculate image level priority, NODE1: image1: score = 30 2/3 = 20 image2: score = 0 image3: score = 1500 2/3 = 1000 NODE2: image1: score = 30 2/3 = 20 image2: score = 600 2/3 = 400 image3: score = 1500 2/3 = 1000 step2/3: cap each image's score with thresholds and calculate the image score, ! NODE1: image1: score = 100 (23 - 23) / ( 1000 - 23) = 0 image2: score = 0 image3: score = 100 (1000 - 23) / (1000 - 23) = 100 NODE2: image1: score = 100 (23 - 23) / ( 1000 - 23) = 0 image2: score = 100 (400 - 23) / (1000 - 23) = 35 image3: score = 100 * (1000 - 23) / (1000 - 23) = 100 step4: final score NODE1: finalscore = 100 / 3 = 33 NODE2: final_score = 135 / 3 = 45 NODE2 will be chosen for scheduling.
Computing the score per image and normalizing by the number of images make sense to me at first glance, but I would like to see more examples.

It would be much easier to digest the analysis if it was summarized in tabular form comparing three scoring functions: (1) current one, (2) one in and (3) multiple big images, multiple small images (total less than the threshold) and a mix. Also, compare cases where the spread is high vs low.
Yes, we don't need to see the individual steps, but a good set of scenarios with final scores. I didn't mean just the old function. We know it's very ineffective. What I meant is capping per pod using the strategy you used in
Did some comparison here, spread has been included based on the layout of each node, it's not easy for me to pull this factor out to focus on this factor only. I will design more testcases when the patch is ready for review which I hope could be more clear. column 1: pod with three images column 2: three nodes' images layout column 3 / 4 / 5: score of each node for different approaches. pod cap and normalizing per image
[10, 20, 30] & small [0,0,0] [800, 1200, 1500] & big [33, 59, 67] [30, 600, 1500] & mix [33, 45, 0]
Normalizing by the number of images gives each image the same "weight". Another variation of the third option is to try and multiply each image score with its size divided by the total size of all images, can you add that as another option?
I don't understand that suggestion. The score of an image is already proportional to its size. I would say the difference between the two suggestions is: Capping per pod cares about total downloaded size Capping per image gives roughly the same importance to each image. I think capping per pod makes more sense. However, there is some information we are losing in both suggestions, as seen in the first case. Even if a lot of small images need to be downloaded, we are giving a 0 score to all nodes. I think a cap from makes more sense, as suggested in the initial PR.
It helps giving more importance to larger images.
If the statistics are accurate with regards to image size distribution, then having many small images all scoring zero isn't common.

For the more common cases (sizes between 32-1000), it seems that both approaches will likely give similar results, so I am fine with both approaches.
That would make the score proportional to the square of the size. Just to play safe (ie, not make the score too strong in a way that could affect spreading or resource allocation) I suggest we go with capping per pod. And then we can re-evaluate next release.
Speaking of statistics. They might have changed since this plugin was first developed. Is there a reliable source?
True for uncapped images, but everything we are doing is just heuristics, and the suggestion is just for the sake of exploring options, not advocating a specific one. Playing it safe sounds good to me.
This approach needs the smallest change and the minimum impact.
+1, this is not common case we can see that a pod with lots of small images (all < 23M), but a corner case that might be improved. No perfect solution so far, but either one is better than the original one IMHO.
/cc , Do you have any comments so far?
btw, has been updated to the second approach (capping and normalization per image).
I would go with this approach as well, it preserves the behavior for multiple small images, and improve it for multiple big ones.
seems like a simple enough compromise for now, and running through some of the examples with that formula it does get more of the specificity you're looking for with large images without normalizing all smaller images right to 0 (so we keep some of the detail there too). I would like to see some updated statistics on the thresholds, and while 90% of the images might fall between them I'm wondering if that's a totally normal distribution. suggestion in got me (tangentially) thinking if we could use thresholds that are more relevant to the pod in question, rather than all of Dockerhub. Because as it is now hypothetical pods (and clusters) which have all small images don't get much benefit from ImageLocality. But I recognize that could be intentional since download resources for small images is ultimately negligible.
I also believe that scoring of small images by zero is intentional, but seems like making lots of small images in pod accountable for the scheduling is slightly better.
It looks like we have consensus. Can you update the PR?


Question: }, Subsets: []v1.EndpointSubset{{ Addresses: []v1.EndpointAddress{{IP: "1.2.3.4", NodeName: &emptyNodeName, TargetRef: &v1.ObjectReference{Kind: "Pod", Name: "pod0", Namespace: ns}}}, <del> Ports:     []v1.EndpointPort{{Port: 0, Protocol: "TCP"}}, </del> <ins> Ports:     []v1.EndpointPort{}, </ins> }}, }) endpointsHandler.ValidateRequestCount(t, 1)

What happened: When creating a headless service locally with set as , field has to be specified. Otherwise, endpoint controller will not be able to create an endpoint for the headless service successfully. This is because port number of to-be-created endpoint is defaulted to 0 when the field is not specified, which results in an error (0 is a reserved number not available for TCP and UDP usage). This explicitly suggests that the field is optional, which can be confusing to those unfamiliar with kubernetes networking. What you expected to happen: Port number of to-be-created endpoint should be defaulted to any value from 1 to , not 0. Apart from this, the tutorial should not suggest the field as optional until the defaulting is fixed. How to reproduce it (as minimally and precisely as possible): : an to refer, but remember to specify field into controller manager log and search for errors related to endpoint creation for the headless service Environment: Kubernetes version (use ):Cloud provider or hardware configuration: local cluster OS (e.g. from /etc/os-release): /cc
In endpoints controller we have explicit logic allowing headless service not to have ports: Though confirmed creating a headless service with no ports defined hit the exact same error: Will investigate more to see if this is a recent regression on endpoint API validation.
Ref , which reported the same bug.
Thanks for looking at the issue! Tell me if I can be of any help. :)
Sure thing :) I have a fix opened . /assign

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: When deploying a headless service without port definition, there is no DNS endpoint created. What you expected to happen: That there would be a valid DNS entry for the service that has just been created. How to reproduce it (as minimally and precisely as possible): Use the example configuration from , but leave out the following section: Anything else we need to know?: Searching trough the issues, it seems I expect the "unexpected" behavior from ticket , which might been fixed in and PR Environment: Kubernetes version (use ):Cloud provider or hardware configuration:
Reiterating the mentions to trigger a notification: <detailsIn response to : Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
allows a headless service without ports and updates service endpoints for the service. I note that you mentioned DNS endpoint. How did you check the DNS endpoint? Thanks!
If I copy the complete example found in the you can resolve the endpoints as follows: If we recreate the example leaving out the ports, we get the following result: Describing the also does not show any endpoints:
In I'm encountering this problem too. I create a headless service with an empty Ports list Create a statefulset referring to that service Pods created by the stateful set cannot resolve each others stable names. I see the following errors in the cluster logs
Also experiencing something similar though it may not be the exact same thing. I have GlusterFS configured in a 1.8 cluster. The gluster replicas were created via a stateful set and thus has a headless service to go with it. If I try to use the glusterfs volume in a pod, in the same namespace (default) then the pod comes online fine and is able to mount the volume. I tried using the same gluster volume in a pod in a different namespace but it fails to resolve the endpoints. I've tried , and but all result in the same issue saying endpoint cannot be found. I moved the deployment into the default namespace and it started working as expected (using , haven't tested the other names but I expect they now work too).

Hi I believe there's a bug in that is not allowing the endpoints for a headless service with no ports to be created. At and , the endpoints controller is adding a port with to the EndpointSubset that is eventually included in the newEndpoints which gets created/updated. In the API, this EndpointSubset is being validated at , and the port with is failing validation at This results in the server returning a 422, and the endpoints controller dropping the service out of its queue: Logs from API: Logs from endpoint controller: Steps to reproduce (logs above from this example): Hope this helps.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
Open for fixing.
/assign
I had been missing this issue for a long time. Just found it again a couple days ago but haven't found a solution yet. Thanks for the fixing!


Question: created *Simple // These are set when Watch is called <del> fakeWatch                *watch.FakeWatcher requestedLabelSelector   labels.Selector requestedFieldSelector   labels.Selector requestedResourceVersion string </del> <ins> fakeWatch                  *watch.FakeWatcher requestedLabelSelector     labels.Selector requestedFieldSelector     labels.Selector requestedResourceVersion   string requestedResourceNamespace string </ins> // The id requested, and location to return for ResourceLocation requestedResourceLocationID string

is always using the cross namespace context. This needs to be fixed to support scoped watches.


Question: if _, err := fmt.Fprintf(w, "%st%st%s", name, strings.Join(status, ","), translateTimestamp(node.CreationTimestamp)); err != nil { return err } <ins> if options.Wide { if _, err := fmt.Fprintf(w, "t%s", getNodeExternalIP(node)); err != nil { return err } } </ins> // Display caller specify column labels first. if _, err := fmt.Fprint(w, AppendLabels(node.Labels, options.ColumnLabels)); err != nil { return err

Nice to have for nodePort service debugging, creating manual firewalls, hitting hostNetwork or hostPort pods etc. Currently I need to either -o yaml, --template, or switch to gcloud.
This should be quite easy, should be good for contribution.
I could take a look at this. What information exactly that is available in yaml output should be shown? This is example yaml from my node: There should be an additional address with type , right? Should all types be supported or only external when is used?
-o wide should just dump what we get from the api-server right? What changes should be in kubectl?
-o yaml dumps apiserver outout, -o wide appends useful information at the cost of overflowing 80 chars. I'm asking that -o wide does for nodes what it already does for pods. I currently have such a function in my bash rc: I use it pretty often. That's a signal that more people are probably doing the same.
thanks!


Question: func waitForPodContainerRestart(f *framework.Framework, pod *v1.Pod, volumeMount string) { By("Failing liveness probe") <del> out, err := framework.RunKubectl("exec", fmt.Sprintf("--namespace=%s", pod.Namespace), pod.Name, "--container", pod.Spec.Containers[0].Name, "--", "/bin/sh", "-c", fmt.Sprintf("rm %v", volumeMount)) </del> <ins> stdout, stderr, err := f.ExecShellInPodWithFullOutput(pod.Name, fmt.Sprintf("rm %v", volumeMount)) </ins> <del> framework.Logf("Pod exec output: %v", out) </del> <ins> framework.Logf("Pod exec output: %v / %v", stdout, stderr) </ins> Expect(err).ToNot(HaveOccurred(), "while failing liveness probe") // Check that container has restarted

Which jobs are failing: Tests which exec into containers and alter lifecycles Which test(s) are failing: All new tests introduced for api change which exec into containers New alpha subpathExpr tests [Feature:VolumeSubpathEnvExpansion][NodeAlphaFeature:VolumeSubpathEnvExpansion] Since when has it been failing: 27/February/2019 Testgrid link: https://k8s- Reason for failure: is only set in e2e tests and the binary is not available in node e2e tests Need to restructure and call the API rather than through the framework convenience function Anything else we need to know:
/sig node /sig storage /assign /cc


Question: case v1.ResourceCPU, v1.ResourceMemory, v1.ResourceEphemeralStorage, pidlimit.PIDs: q, err := resource.ParseQuantity(v) if err != nil { <del> return nil, err </del> <ins> return nil, fmt.Errorf("failed to parse quantity %q for %q resource: %w", v, k, err) </ins> } if q.Sign() == -1 { return nil, fmt.Errorf("resource quantity for %q cannot be negative: %v", k, v)

During kubelet start up if the system reserved or kube reserved values failed to parse, the kubelet won't start up. The error logged looks like, However, it doesn't not inform the user which system reserved or kube reserved quantity has failed and what was the corresponding value of that quantity. The error log should also container quantity type and the value that failed to parse, so that user can fix them easily. Try to start kubelet with unparsable values of system reserved, something like, The error logged would look like, As you can see above, it doesn't say was not able to parse with the value No response <details</details<details</details<details</details<details</details<details</details<details</details
/kind feature /sig node
/assign
Thanks for Should I close in favour of yours?
Oh sorry, I didn't see your PR. Either way works for me. You can either incorporate ideas from my PR and I'll drop mine, or drop your PR. I don't feel strongly either way.
/triage accepted /priority backlog /assign
Let's go with yours!
Thanks /close (in favor of )
Closing this issue. <detailsIn response to : Thanks (in favor of ) Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: case opListChain: if table.Has(chainName) { return []byte{}, nil <del> } else { return []byte{}, fmt.Errorf("no such chain %q", chainName) </del> } <ins> return []byte{}, fmt.Errorf("no such chain %q", chainName) </ins> case opDeleteChain: table.Delete(chainName) return []byte{}, nil

Anyone itching to understand the code and try fixing things in Kubernetes, please help fixing the lint failures Step 1: Remove one or two packages (related packages) from the list in - (note: do not change generated files, the name of defaulting functions, or the name of conversion functions... packages failing linting due to issues in those files will need to be left alone) Step 2: Run and look at the logged errors Step 3: Fix the code to ensure that runs cleanly. Step 4: Run and to make sure we fix any problems with gofmt or bazel. Step 5: File a PR with the change Step 6: Go to step :) Please use the tracking spreadsheet For everyone who is going to contribute, please append the packages you picked at the end. For existing contributor, please make sure this list is update to date
/sig architecture /sig testing
Hey, I would like to resolve this issue. Could you assign it to me ?
there are a lot of packages so there will be many people working on this. Also we cannot assign issue to folks who are not in the github org for kubernetes. please feel free to pick up a few packages like i mentioned and file a PR. Make sure you link back to this issue, so folks can see which packages you filed PR for.
OK. I see. I will firstly pick up the package called to fix.
Hi dims. Should i submit my PR before i remove one or two packages (related packages) from the list in - Or I first remove the package from list, then I fix the issue and submit a PR?
in your PR, you will have both the fixes to packages and some lines removed from /.golint_failures
When I run , I found some errors as below: I found that there is a go file in But there is still a error.
Say your GOPATH looks like this Your kubernetes directory should be () cd to and then run from that directory.
Is it necessary to obey all rules in golint? Rules in golint are set in stone. Maybe using gometalinter to ignore some specific errors is better?
I revise some erros about golint in some packages. Please review it.
which errors would you like to ignore?

if you can open a new issue with the list of errors to ignore (and proposing gometalinter), we can get some opinions there from others in the community.
Let me take the kubectl cmd part...
Can someone please review and let me know if I am doing it right?
I ll work on all the packages related to
we get 800-ish pkgs left to fix. my concern is that if we fix them package by package, then such pulls will be flooding the repo. can't we possibly make it in larger pulls? the larger the better?
+1 to slightly larger pulls with related packages. if allow too large then the reviewers will not be able to do a thorough review. we should spread the load among multiple reviewers. (if we allow a really large pull, we will end up with top level reviewers who will face the burden and the PR(s) will just stagnate)
oh good point. we could put one pkg's change into one commit, and merge all changes under one area into one pull so that it wont spread the load. wdyt? for each pull, we can have hundred-ish lines of change. but it will still be 30~50 pulls i guess? good news is that we can take our time to deal with it, at least before 1.12
sounds great! please feel free to help make it happen :) (by suggesting groups of packages that kinda go together)
I'm looking at fixing pkg/proxy, and I think I see what was talking about ignoring some rules. proxy.ProxyProvider is being suggested because golint is saying that it stutters. In that example, I think it's okay renaming it to proxy.Provider, but I could see an example where you would want to keep the stutter if the package that it's declared in uses it heavily.
sure thing, feel free to propose a PR for switching to gometalinter independently of other patches. We can work through that there.
I sent the pull request to fix pkg/volume/aws_ebs
I've fixed a few of these, but get the following error when I try to run : Are there any instructions on what I need to setup in order to resolve this?

you can find this file in
I have raised a PR to fix golint errors in pkg/api
I have fixed some pkgs under /pkg/kubelet pkg/kubelet/images pkg/kubelet/kuberuntime pkg/kubelet/leaky pkg/kubelet/pleg
I submitted a pull request for golint errors in pkg/security/ edit: I can work on picking up pkg/controller/bootstrap through pkg/controller/namespace/... (114-129)
fix golint errors in package cmd/kubeadm/app, since the codes under cmd/kubeadm/app/apis/kubeadm/v1alpha2 and cmd/kubeadm/app/apis/kubeadm/v1alpha3 are auto-generated, i keep them untouched.
I am working on some packages under pkg/kubelet, anyone interested in this plz leave messages for me :)
fix golint for pkg/util/dbus pkg/util/labels pkg/util/node pkg/util/strings pkg/util/workqueue/prometheus
PR fixes golint errors in pkg/routes,pkg/securitycontext, pkg/serviceaccount
PR fixes golint in pkg/probe
should we have a checklist as part of the issue summary saying what is in progress and what is done? Since this issue is getting long, this will make new contributors to pick up issues easier.
Great idea!, want to throw a google doc with a check-list so it's easy for participants to organize themselves? (please!) If it is the issue summary then one of us with privileges will be a bottle neck.
Created a with all the package that have on going PRs. Maybe you can add this to the summary. For everyone who is going to contribute, please append the packages you picked at the end. For existing contributor, please make sure this list is update to date
Thanks updated the spread sheet
Raised PR to fix golint issues in: pkg/controller/clusterroleaggregation pkg/controller/cronjob pkg/controller/daemon pkg/controller/deployment pkg/controller/deployment/util pkg/controller/disruption pkg/controller/endpoint
This issue is close in accident by . could you reopen it?
/reopen
Reopening this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I found a problem trying to fix golint messages for .

Golint complaints on 's function because of the snake case. When I try to fix it, fails because it is being used in a code generated file. If I clean generated files, tests pass, but the generated file is not invoking it anymore. I failed to find the generator definition using this so I could change the hook to just . Also I'm not sure if this is wanted. I also found other packages in also probably affected by this. I read somewhere about replacing golint with gometalinter, which would allow to define directives in code to omit certain rules. I would consider to have a second exclusion file for golint with these cases, to be updated if move to gometalinter is finally done. This would make this issue completion more feasible. Meanwhile, I'll move onto fixing golint messages for other packages, until this discussion gets cleared.
I also found a problem fixing . A test fails because the directory is marked as readonly (see tests @ Shall we also ignore this directory for this issue?
Is this issue still relevant because hack/verify- is giving me no errors.
the failures are still present in there are details in the first message on how to claim + solve issues :)
Sorry I misunderstood the issue
I'm working on this package:
pkg/apis/admission/v1beta1 pkg/apis/authentication/v1beta1 pkg/apis/authorization/v1beta1 pkg/apis/coordination/v1 pkg/apis/coordination/v1beta1 pkg/apis/imagepolicy/v1alpha1 pkg/apis/scheduling/v1alpha1 pkg/apis/scheduling/v1beta1 pkg/apis/settings/v1alpha1 pkg/apis/storage/v1alpha1
pkg/cloudprovider/providers/vsphere/vclib pkg/kubelet/apis/deviceplugin/v1alpha pkg/kubelet/apis/pluginregistration/v1alpha1 pkg/kubelet/apis/pluginregistration/v1beta1
:
I'm looking at the pkg/registry/*, all those which are left. I expect to give an update on what all is fixed in the next 24-48 hours.
hi there is a google sheet that is a bit outdated, maybe you can update that?
hi I seem to only have view access for the sheet. Can't edit.
I fixed the permission.

Please try it again
pkg/apis/authentication pkg/apis/coordination pkg/apis/events pkg/apis/imagepolicy pkg/apis/settings
hi I think this issue was closed prematurely by (using fixes #issue in commit)
I'll put in a PR shortly for the package
Looking at next
hi thanks! there is a spreadsheet here that tracks the progress on fixing these issues, consider adding the packages you're work on there so we can keep everything organized :+1:
Starting on now
I did indeed see the Google Sheet link (thank you and to take it a little further
test/e2e
test/e2e_node/builder
pkg/registry/admissionregistration/mutatingwebhookconfiguration/storage pkg/registry/admissionregistration/rest pkg/registry/admissionregistration/validatingwebhookconfiguration/storage pkg/registry/apps/daemonset/storage
Hi. I opened a PR - Please review.
Working on Will open a PR soon.
Starting on and should have a PR soon
I keep getting this golint verify error on and was wondering if others saw it: But when I that package to the golint failures in my PR, my reviewer said it isn't necessary? What does that mean? What did I miss? Should I use a different kind of verify? cf:
looks like the disconnect is because that problem does not show up in the version of we use in the CI. please see
Still confused: is that not the golint version I ran? it installs from vendor and uses that I think?
i installed from and i still don't see the problem, can you please check what version of golint you have?
I'm confused why I wouldn't be using the vendored version? how would that get out of sync?
/me looking
Note: I am reserving for a KubeCon Contributor Workshop demonstration. It is noted in the spreadsheet.
I think may not be ideal cause there's a lot of auto generated files there. What do you think about reserving instead for the KubeCON Contributor Workshop?
pkg/apis/scheduling pkg/apis/storage/util pkg/apis/storage/v1beta1/util pkg/apis/storage/v1/util
/priority backlog
Hi I am a new contributor here, I'd like to contribute and make changes to the following file/s. I have marked it as WIP in the Google Sheet. Can someone let me know if I have to do anything else before I start working on this PR?

Umm so there's an issue where Golint tells that underscore shouldn't be allowed. Here's a log of the go-lint warnings:
Thanks The following should not be modified, as they are designed like this for code generation.
I'd like to pick a package and do my first PR, is there a list of packages to be done? Or should I correlate with this issue thread? Thanks in advance!
please see step 1 () please scan other PRs linked to this issue as well just in case.
Will do. Thank you. On Thu, Oct 3, 2019, 7:25 AM Davanum Srinivas <:
Hi there, I'm looking for a good first issue. I'll start taking a look at if that's ok? There doesn't seem to be anyone currently tracked as looking at it in the attached spreadsheet.
I've found a few others I'm actively looking at, you won't get any argument from me. On Sat, Oct 12, 2019, 10:38 AM Pamela Kelly <:
I am working on the iptables pkg. Updated in the google docs
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I have problem running in step 2 $ hack/verify- what should I do?
PTAL for my pr to this issue, thx
you can move kubernetes directory to and re-run this script.
I'd like to take Can we fix linter issues for generated files? Or we should skip packages with zz_generated?
There're 2 lint errors that affects 292/507 packages, meaning it's not possible to cleanup the entire file without solving it: default pattern of Kubectl is the usage of [cmd]Options for initial options structs and it raises: /apis/{custommetrics,externalmetrics} for example raises the underscore error, but It's probably not worth to break backward compatibility renaming these folder for lint error fix: Any advice on these?

/reopen hm, cannot reopen this issue, it was closed by mistake
You can't reopen an issue/PR unless you authored it or you are a collaborator. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I am sure this is all not fixed. Am I missing something? ./hack/verify- installing golint from vendor Congratulations! All Go source files have been linted.
you need to remove package from first.
/remove-sig testing SIG testing doesn't own code failing lints, we own the linter. :upsidedownface:
Hello, is this still a relevent issue ? I would like to work upon
I have fixed
Hi Folks, Some of the golint fixes, require us to change the exposed function's name, in simple words, "change API". I was not sure, if that is welcomed and what is the process.
Heres what Im getting after going over the following steps as mentioned by Say your GOPATH looks like this GOPATH=/Users/dims/go Your kubernetes directory should be ($) cd to and then run hack/verify- from that directory. I get the following output - installing golint Congratulations! All Go source files have been linted. Im not really sure what I am doing wrong. Any help is appreciated!
I think I understand. I misunderstood the steps around .golint_failures. The first step is to remove the pkg that we are targeting from this file and then run the My mistake was to assume that the packages in the file need to be removed after we are done cleaning the golint errors.
Can someone please review and let me know if I've done this right?
After a couple years of seeing changes spawned from this issue, I don't think this is a good use of contributors' or reviewers' time.

golint is very opinionated about: exported type and function names, leading to PRs attempting to change exported things people use downstream missing documentation on interface implementation functions, which leads to not-very-useful copy/paste of interface godoc the govet and staticcheck verify jobs are much more useful since they catch correctness issues, but I would recommend closing this issue and dropping the golint checks from CI golint doc itself discourages from using its output in an automated enforced way: from
i guess one can say gofmt is opinionated too, but something good about Golang is that it comes with an opinion and a set of standard tools. for API documentation at least we found inconsistencies on how most exported API items are documented: golint can help with that, but if there are no "pragmas" to avoid breaking changes and given we have codegen generating non-compliant names enabling golint on API packages is sort of blocked in Kubernetes. for non-API, non-library, non-public code, i think being inconsistent is mostly OK. no preference on my side. shrug
true, but code formatting has the benefits of mostly eliminating meaningless diffs in PRs due to code movement/whitespace/etc if there are specific packages (like API packages) we lack documentation on, adding checks specifically for that could make sense exactly, I don't think golint is the right tool for making sure we have API docs
golint itself doesn't have these and the go project is against this sort of comment AIUI, however golangci-lint does have them. I think there are plenty of other static analysis lints we should use instead, beyond just staticcheck and vet there is e.g.
btw, if we remove golint can we see e.g. all-caps constants or functions with underscores dodging review? perhaps we should add golangci-lint soon after:
Completely agree there are better places to spend review bandwidth. I was curious to see what the effort looked like. There have been 890 commits that touched over the past 3+ years !
/remove-help /remove-good-first-issue
Overall agree on the bandwidth. Also note that it was a good first issue with very low bar to get them in the repo. May be we need to find a similar low bar, but high value good first issue.

UT's come to mind, are there other things we curate for new contributors ?
Do we have an alternative way of enforcing that new exported code (not necessarily API fields) has at least some minimum documentation? As a reviewer, I ask contributors to add meaningful comments. But reality is that I sometimes stumble into old code that doesn't have documentation because we didn't have golint at some point. I agree that fix-golint PRs just add burden to reviewers. But, before removing the lint check, can we wait and see if it's enough to discourage people from sending more PRs like this? Additionally, we can encourage reviewers to just close PRs that only fix golint issues.
+1 for adding burden to reviewers. However, as a new contributor, I think this a very good opportunity for new contributors to study the code base and submit their first PR. In my opinion, I think this issue would be able to attract more new contributors, get familiar with the code base, and get familiar with the git workflow.
thanks everyone for getting us this far. We essentially gave up on the existing structure/system for golint: At some point we can look at adding golangci-lint, there's a WIP in progress in /close
Closing this issue. <detailsIn response to : thanks everyone for getting us this far. We essentially gave up on the existing structure/system for golint: Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: <ins> /* Copyright 2019 The Kubernetes Authors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ // Note: the example only works with the code within the same release/branch. package main import ( "bufio" "flag" "fmt" "os" "path/filepath" apiv1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" "k8s.io/apimachinery/pkg/runtime/schema" "k8s.io/client-go/dynamic" "k8s.io/client-go/tools/clientcmd" "k8s.io/client-go/util/homedir" "k8s.io/client-go/util/retry" // // Uncomment to load all auth plugins // _ "k8s.io/client-go/plugin/pkg/client/auth // // Or uncomment to load specific auth plugins // _ "k8s.io/client-go/plugin/pkg/client/auth/azure" // _ "k8s.io/client-go/plugin/pkg/client/auth/gcp" // _ "k8s.io/client-go/plugin/pkg/client/auth/oidc" // _ "k8s.io/client-go/plugin/pkg/client/auth/openstack" ) func main() { var kubeconfig *string if home := homedir.HomeDir(); home != "" { kubeconfig = flag.String("kubeconfig", filepath.Join(home, ".kube", "config"), "(optional) absolute path to the kubeconfig file") } else { kubeconfig = flag.String("kubeconfig", "", "absolute path to the kubeconfig file") } flag.Parse() namespace := "default" config, err := clientcmd.BuildConfigFromFlags("", *kubeconfig) if err != nil { panic(err) } client, err := dynamic.NewForConfig(config) if err != nil { panic(err) } deploymentRes := schema.GroupVersionResource{Group: "apps", Version: "v1", Resource: "deployments"} deployment := &unstructured.Unstructured{ Object: map[string]interface{}{ "apiVersion": "apps/v1", "kind":       "Deployment", "metadata": map[string]interface{}{ "name": "demo-deployment", }, "spec": map[string]interface{}{ "replicas": 2, "selector": map[string]interface{}{ "matchLabels": map[string]interface{}{ "app": "demo", }, }, "template": map[string]interface{}{ "metadata": map[string]interface{}{ "labels": map[string]interface{}{ "app": "demo", }, }, "spec": map[string]interface{}{ "containers": []map[string]interface{}{ { "name":  "web", "image": "nginx:1.12", "ports": []map[string]interface{}{ { "name":          "http", "protocol":      "TCP", "containerPort": 80, }, }, }, }, }, }, }, }, } // Create Deployment fmt.Println("Creating deployment...") result, err := client.Resource(deploymentRes).Namespace(namespace).Create(deployment, metav1.CreateOptions{}) if err != nil { panic(err) } fmt.Printf("Created deployment %q.n", result.GetName()) // Update Deployment prompt() fmt.Println("Updating deployment...") //    You have two options to Update() this Deployment: // //    1. Modify the "deployment" variable and call: Update(deployment). //       This works like the "kubectl replace" command and it overwrites/loses changes //       made by other clients between you Create() and Update() the object. //    2. Modify the "result" returned by Get() and retry Update(result) until //       you no longer get a conflict error. This way, you can preserve changes made //       by other clients between Create() and Update(). This is implemented below //			 using the retry utility package included with client-go. (RECOMMENDED) // // More Info: // https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency retryErr := retry.RetryOnConflict(retry.DefaultRetry, func() error { // Retrieve the latest version of Deployment before attempting update // RetryOnConflict uses exponential backoff to avoid exhausting the apiserver result, getErr := client.Resource(deploymentRes).Namespace(namespace).Get("demo-deployment", metav1.GetOptions{}) if getErr != nil { panic(fmt.Errorf("Failed to get latest version of Deployment: %v", getErr)) } // update replicas to 1 if err := unstructured.SetNestedField(result.Object, int64(1), "spec", "replicas"); err != nil { panic(fmt.Errorf("Failed to set replica value: %v", err)) } // extract spec containers containers, found, err := unstructured.NestedSlice(result.Object, "spec", "template", "spec", "containers") if err != nil || !found || containers == nil { panic(fmt.Errorf("Deployment containers not found or error in spec: %v", err)) } // update container[0] image if err := unstructured.SetNestedField(containers[0].(map[string]interface{}), "nginx:1.13", "image"); err != nil { panic(err) } if err := unstructured.SetNestedField(result.Object, containers, "spec", "template", "spec", "containers"); err != nil { panic(err) } _, updateErr := client.Resource(deploymentRes).Namespace(namespace).Update(result, metav1.UpdateOptions{}) return updateErr }) if retryErr != nil { panic(fmt.Errorf("Update failed: %v", retryErr)) } fmt.Println("Updated deployment...") // List Deployments prompt() fmt.Printf("Listing deployments in namespace %q:n", apiv1.NamespaceDefault) list, err := client.Resource(deploymentRes).Namespace(namespace).List(metav1.ListOptions{}) if err != nil { panic(err) } for _, d := range list.Items { replicas, found, err := unstructured.NestedInt64(d.Object, "spec", "replicas") if err != nil || !found { fmt.Printf("Replicas not found for deployment %s: error=%s", d.GetName(), err) continue } fmt.Printf(" * %s (%d replicas)n", d.GetName(), replicas) } // Delete Deployment prompt() fmt.Println("Deleting deployment...") deletePolicy := metav1.DeletePropagationForeground deleteOptions := &metav1.DeleteOptions{ PropagationPolicy: &deletePolicy, } if err := client.Resource(deploymentRes).Namespace(namespace).Delete("demo-deployment", deleteOptions); err != nil { panic(err) } fmt.Println("Deleted deployment.") } func prompt() { fmt.Printf("-> Press Return key to continue.") scanner := bufio.NewScanner(os.Stdin) for scanner.Scan() { break } if err := scanner.Err(); err != nil { panic(err) } fmt.Println() } </ins>

What would you like to be : Add documentation/example showing the usage of dynamic package in the client-go repo. /sig api-machinery


Question: totals = totalChanges(existingSlices[0], desiredSet) if totals.added == 0 && totals.updated == 0 && totals.removed == 0 && apiequality.Semantic.DeepEqual(endpoints.Labels, compareLabels) && <del> apiequality.Semantic.DeepEqual(compareAnnotations, existingSlices[0].Annotations) { </del> <ins> apiequality.Semantic.DeepEqual(compareAnnotations, existingSlices[0].Annotations) && !needRebuildExistingSlices(endpoints, existingSlices[0]) { if !r.endpointSliceTracker.Has(existingSlices[0]) { r.endpointSliceTracker.Update(existingSlices[0]) // Always ensure each EndpointSlice is being tracked. } </ins> return slices, totals } }

create a service without selector, and manual create endpoints for this service endpointslicemirroring controller will create an endpointslice for this service. after endpoint slice created, and kube-controller-manager restart. during the kube-controller-manager restart, the endpoint was deleted and recreated after kube-controller-manager restart, garbagecollector controller delete this endpointslice and endpointslice not recreate until endpoint/service update event or kube-controller-manager restart endpointslice will be recreated create a service without selector, and manual create endpoints for this service endpointslicemirroring controller will create an endpointslice for this service. after endpoint slice created, and kube-controller-manager restart. during the kube-controller-manager restart, the endpoint was deleted and recreated No response <details</details<details</details<details</details<details</details<details</details<details</details
/sig network
/assign
Trying to understand this, mostly just restating what you've said above to make sure I'm getting it: during the kube-controller-manager restart, the endpoint was deleted and recreated after kube-controller-manager restart, garbagecollector controller delete this endpointslice and endpointslice not recreate until endpoint/service update event or kube-controller-manager restart So an Endpoints resource is being deleted while kube-controller-manager is not running. The EndpointSlice mirroring controller to point to the Endpoints they're mirroring. When kube-controller-manager is restarted, the garbage collector will delete the mirrored EndpointSlice because it's owned by an Endpoints resource that was previously deleted. That deletion by garbage collection would then be , but if the EndpointSliceTracker doesn't think it should have that resource, it will ignore the deletion and not reconcile, resulting in the mirrored EndpointSlice not being recreated until something updates the Service or Endpoints resource.
/triage accepted


Question: } } <ins> func TestSubpath_PrepareSafeSubpath(t *testing.T) { //complete code defaultPerm := os.FileMode(0750) tests := []struct { name string // Function that prepares directory structure for the test under given // base. prepare      func(base string) ([]string, string, string, error) expectError  bool expectAction []mount.FakeAction mountExists  bool }{ { name: "subpath-mount-already-exists-with-mismatching-mount", prepare: func(base string) ([]string, string, string, error) { volpath, subpathMount := getTestPaths(base) mounts := []string{subpathMount} if err := os.MkdirAll(subpathMount, defaultPerm); err != nil { return nil, "", "", err } subpath := filepath.Join(volpath, "dir0") return mounts, volpath, subpath, os.MkdirAll(subpath, defaultPerm) }, expectError:  false, expectAction: []mount.FakeAction{{Action: "unmount"}}, mountExists:  false, }, { name: "subpath-mount-already-exists-with-samefile", prepare: func(base string) ([]string, string, string, error) { volpath, subpathMount := getTestPaths(base) mounts := []string{subpathMount} subpathMountRoot := filepath.Dir(subpathMount) if err := os.MkdirAll(subpathMountRoot, defaultPerm); err != nil { return nil, "", "", err } targetFile, err := os.Create(subpathMount) if err != nil { return nil, "", "", err } defer targetFile.Close() if err := os.MkdirAll(volpath, defaultPerm); err != nil { return nil, "", "", err } subpath := filepath.Join(volpath, "file0") // using hard link to simulate bind mounts err = os.Link(subpathMount, subpath) if err != nil { return nil, "", "", err } return mounts, volpath, subpath, nil }, expectError:  false, expectAction: []mount.FakeAction{}, mountExists:  true, }, } for _, test := range tests { klog.V(4).Infof("test %q", test.name) base, err := ioutil.TempDir("", "bind-subpath-"+test.name+"-") if err != nil { t.Fatalf(err.Error()) } defer os.RemoveAll(base) mounts, volPath, subPath, err := test.prepare(base) if err != nil { os.RemoveAll(base) t.Fatalf("failed to prepare test %q: %v", test.name, err.Error()) } fm := setupFakeMounter(mounts) subpath := Subpath{ VolumeMountIndex: testSubpath, Path:             subPath, VolumeName:       testVol, VolumePath:       volPath, PodDir:           filepath.Join(base, "pod0"), ContainerName:    testContainer, } _, subpathMount := getTestPaths(base) bindMountExists, bindPathTarget, err := prepareSubpathTarget(fm, subpath) if bindMountExists != test.mountExists { t.Errorf("test %q failed: expected bindMountExists %v, got %v", test.name, test.mountExists, bindMountExists) } logActions := fm.GetLog() if len(test.expectAction) == 0 && len(logActions) > 0 { t.Errorf("test %q failed: expected no actions, got %v", test.name, logActions) } if len(test.expectAction) > 0 { foundMatchingAction := false testAction := test.expectAction[0] for _, action := range logActions { if action.Action == testAction.Action { foundMatchingAction = true break } } if !foundMatchingAction { t.Errorf("test %q failed: expected action %q, got %v", test.name, testAction.Action, logActions) } } if test.expectError { if err == nil { t.Errorf("test %q failed: expected error, got success", test.name) } if bindPathTarget != "" { t.Errorf("test %q failed: expected empty bindPathTarget, got %v", test.name, bindPathTarget) } if err = validateDirNotExists(subpathMount); err != nil { t.Errorf("test %q failed: %v", test.name, err) } } if !test.expectError { if err != nil { t.Errorf("test %q failed: %v", test.name, err) } if bindPathTarget != subpathMount { t.Errorf("test %q failed: expected bindPathTarget %v, got %v", test.name, subpathMount, bindPathTarget) } if err = validateFileExists(subpathMount); err != nil { t.Errorf("test %q failed: %v", test.name, err) } } } } </ins> func TestSafeOpen(t *testing.T) { defaultPerm := os.FileMode(0750)

When we merged - we introduced comparison of subpath sources: But subpaths source comparison never works properly for all volume types, even if subpath and source remains unchaged. This is because we are comparing wrong things. For a given volume - is usually: But according - the root is - . This means that - if creation of a container times out, then another container can not start until old container exits, because subapth comparison above will be false and kubelet will try to umount the subpath bind mount and it will keep failing with "device is busy" errors. So subpath bind mount can not be reused at all. This is sort of expensive, for containers that use selinux. Because it means each container creation will potentially run recursive chcon again on the entire subpath etc.
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig node /sig storage
cc
/remove-sig node


Question: rc.stopCPU <- 0 rc.stopMem <- 0 rc.stopCustomMetric <- 0 <ins> rc.stopWaitGroup.Wait() </ins> } // Pause starts background goroutines responsible for consuming resources.

https://k8s-
There are no sig labels on this issue. Please by:<br(1) mentioning a sig: <br(2) specifying the label manually: <br<brNote: method (1) will trigger a notification to the team. You can find the team list and label list
might be a culprit, but maybe not
This could be related to how the test initializes the framework or cleans it up, maybe.
Another failure in the same run, not just this one test:
I'm not positive but the stack trace might be wrong due to inlining optimizations. rc.framework == nil was my first guess, but that would fail earlier in that method.
Framework AfterEach runs namespace deletion, which may mean that .Namespace is nil (if inlining is indeed present).
I am looking into this right now.
It's unrelated. I reverted from the current master HEAD locally (via ), then did and finally and it did not help. The test still fails with the same message "Expected <*v1.Initializers | 0x0: nil not to be nil" (location: )
Note that I have executed one of the initializers e2e test (not autoscaling), because it is one of the tests reported by https://k8s- as panicked.
Looking a bit further I found out that commit the test above as already failing.
I have just realized that I have discovered yesterday a race condition in ResourceConsumer implementation, and might actually make it worse. Let me quickly make a PR out of a fix.
Let me try to recap what I have found out so far: The test that is being referred to at the top of this issue () fails "out of the box" and probably deserves a separate issue to track it Other failing tests that look like they have a similar failure pattern (NPE) fail because of trying to send a "consume CPU" request to a non-existent pod (unrelated to above, as far as I can tell) PR might cause the effect above (and probably does) because it actually deletes the pods that should have been deleted However, the root cause is that there is a race condition inside ResourceConsumer that makes "consume CPU" requests being sometimes sent already after "CleanUp" got called (fix prepared in PR )
Initializer was fixed in a different PR, depending on what cluster you test on.

, Andrzej Wasylkowski < :
Yeah, confirmed initializers test is now passing on GCE and disabled on GKE (which doesn't enable initializers configuration group yet) , Clayton Coleman <: The test that is being referred to at the top of this issue () fails "out of the box" and probably deserves a separate issue to track it Other failing tests that look like they have a similar failure pattern (NPE) fail because of trying to send a "consume CPU" request to a non-existent pod (unrelated to above, as far as I can tell) PR cause the effect above (and probably does) because it actually deletes the pods that should have been deleted However, the root cause is that there is a race condition inside ResourceConsumer that makes "consume CPU" requests being sometimes sent already after "CleanUp" got called (fix prepared in PR ) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub , or mute the thread
Clayton, can you assign this to me? I cannot do this myself.