Question: return clientset.CoreV1().Services(namespace).Create(service) } <ins> // checkServicesCreation checks if the service have been created successfuly in all the clusters. // if the service is not present in at least one of the clusters, this function returns an error. func checkServicesCreation(namespace, serviceName string, clusters fedframework.ClusterSlice) error { framework.Logf("check if service %q have been created in %d clusters", serviceName, len(clusters)) for _, cluster := range clusters { name := cluster.Name err := wait.PollImmediate(framework.Poll, fedframework.FederatedDefaultTestTimeout, func() (bool, error) { var err error _, err = cluster.Clientset.CoreV1().Services(namespace).Get(serviceName, metav1.GetOptions{}) if err != nil && !errors.IsNotFound(err) { // Get failed with an error, try again. framework.Logf("Failed to find service %q in namespace %q, in cluster %q: %v. Trying again in %s", serviceName, namespace, name, err, framework.Poll) return false, err } else if errors.IsNotFound(err) { framework.Logf("Service %q in namespace %q in cluster %q not found. Trying again in %s", serviceName, namespace, name, framework.Poll) return false, nil } By(fmt.Sprintf("Service %q in namespace %q in cluster %q found", serviceName, namespace, name)) return true, nil }) if err != nil { return err } } return nil } </ins> func createServiceOrFail(clientset *fedclientset.Clientset, namespace, name string) *v1.Service { service, err := createService(clientset, namespace, name) framework.ExpectNoError(err, "Creating service %q in namespace %q", service.Name, namespace)

In order to support running tests in parallel, we randomized NodePort selection. There appears to be some evidence that this is not sufficient and is causing e2e test flakes: https://k8s-
Hi, As a need contributor, I'm interesting of solving this issue. I already check the current code, and propose to retry the service creation until success or all possible ports (between FederatedSvcNodePortFirst and FederatedSvcNodePortLast) have been tried. If this approach is fine for you, I can submit a PR soon.
How do you plan to check if the NodePorts are available? In our presubmit test environment, tests run across many instances of federation control planes in parallel, but share the same underlying clusters. So this approach might be viable depending on how and where you plan to check for these used NodePorts.
my approach for getting an available port is a "Dummy" approach: create a list of possible available port, get one of them randomly, then try to create the Service. If Service creation failed. try to use another possible available port, until service creation success or all possible available ports have been tested.
Sounds reasonable to me. Go for it. A couple of caveats: need to create a real list, you could just use the range boundaries. careful about service creation failure. You need to check if the service shard creation failed in the underlying clusters and rollback all those shards before retrying.
thanks for the caveats, I will do soon a PR.
Hi, what is the state of this issue? Is working in this issues with the others issues related to e2e tests?
has a PR to fix this issue -


Question: // CleanupLeftovers clean up all ipvs and iptables rules created by ipvs Proxier. func CleanupLeftovers(ipvs utilipvs.Interface, ipt utiliptables.Interface, ipset utilipset.Interface, cleanupIPVS bool) (encounteredError bool) { <ins> if canUse, _ := CanUseIPVSProxier(NewLinuxKernelHandler(), ipset); !canUse { return false } </ins> if cleanupIPVS { // Return immediately when ipvs interface is nil - Probably initialization failed in somewhere. if ipvs == nil {

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: What happened: kube proxy report following error even if ipvs not enabled. What you expected to happen: How to reproduce it (as minimally and precisely as possible): Install kubernetes and start proxy. Anything else we need to know?: Environment: Kubernetes version (use ): We are using 1.9.1 Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others: /cc comments?
/sig network
It's a noise, but it's thrown by libnetwork/ipvs util. Is that a big deal?
please take a look when you have a chance :)
Yes, it always bring noise for production when customer got warning messages. Why printing some ipvs message even it was not enabled?
/area ipvs


Question: <del> apiVersion: v1 clusters: - cluster: certificate-authority: /etc/secret-volume/kube-ca server:  https://146.148.35.28 name: kubernetes contexts: - context: cluster: kubernetes user: kubernetes-admin name: kubernetes current-context: kubernetes kind: Config preferences: {} users: - name: kubernetes-admin user: auth-path: /etc/secret-volume/kube-auth-path </del>

This PR was prematurely merged: Need to update the PR based on the actual steps that were needed to get what was demonstrated at the public Kubernetes meet-up last week.
Any chance you can get rid of the hostPort from the example?
- depending on the Kubernetes provider, yes. For Vagrant based provider, I think I still need the hostPort because I do not have routes. For the GCE provider, I should be able to get rid of it because I have routes. In both providers, there were a number of setup steps that did not make the merged PR. For those that are tracking this issue (which I know folks have pinged me on separately) needs to be configured to allow privelaged containers needs to be configured to allow pushing to an insecure registry (this is point in time) Ideally, I could use a ReplicationController in the example to load balance OpenShift, but until we can provide a pattern to run etcd as a pod, I am unable to do this and need to use the embedded etcd we end up running.
Note: I will resolve this issue in 2 phases. Vagrant (which was demonstrated) to document using GCE (which has some additional steps on firewall rules)
Turns out OpenShift Origin is too secure now ;-) This is blocked until OpenShift Origin allows you to mint certs on the client, and then supply the certs on startup as a secret to the pod. This feature is coming soon. /cc
I now have all the required pieces to get this to work. I hope to get a PR submitted to fix this issue by end of this week.


Question: # Install Salt # <del> # We specify -X to avoid a race condition that can cause minion failure to # install.  See https://github.com/saltstack/salt-bootstrap/issues/270 # </del> # -M installs the master curl -sS -L --connect-timeout 20 --retry 6 --retry-delay 10 https://bootstrap.saltstack.com | sh -s -- -M

Jenkins build Node/job/heapster-dockercanarybuild-ci/58/ failed since last Thursday due to compiling issue: looks like it is caused by
encountered this error when compiling heapster at local.
After updating the gce util library Heapster requires now go in version = 1.5. I'll fix the version in Jenkins config. ref , ,
The problem is with: heapster-dockercanarybuild-ci heapster-gce-e2e-ci could you please bump go version to 1.6 for those Jenkins jobs?
how are these builds getting the Go runtime? Just whatever's installed on the system?
/usr/local/go
Configured here:
Ah, okay, it is whatever is installed on the system. We should probably change this to use something version-controlled. (Ideally this would just be running inside Docker.) I can upgrade to go 1.6 in the interim. I think the only other use of the system Go runtime is in the e2e tests, which shouldn't care about the version of go.
Updated to go1.6 on post-commit Jenkins VMs. Haven't touched PR Jenkins (yet).
in case you see any breakages, see ^
Thanks for the heads up
this is still failing. And actually looking at the latest build log, I see why: You need to update
Thanks I'll fix it.
Well, that's for , at least. is using system Go, and it's failing due to changes in Go 1.6, I think.
Log from :
seems to work. This used to happen before.
Suite heapster-gce-e2e-ci is still failing, this time with the message:
sorry, I missed that you pasted the same failure. I'll take w look.
has not passed in over a month. I'm going to send out a PR to disable it
As I wrote in the job has passed only once during its first run and then it failed 1000+ times. Was it supposed to work anytime?
If the solution is to delete this job that is totally fine by me
Decreasing priority since the job is disabled. do you know who set up the job?
do you know who created the suite? What should it test? The suite passed only 1/1175 runs.

had setup these jobs originally as part of migrating over to k8s jenkins server from my own jenkins server. The PR builder and HEAD CI should both be sharing the same framework ideally.
Thanks for the explanation. What is HEAD CI? I'll try to resurrect the suite. did you finish all the required work on this? Especially the suite passed only once and then failed 1000+ times.
I believe the suite was passing before the failures you see, but when it was converted to use jenkins-job-builder, the job was renamed and the count reset. It is possible the failures are related to something with the jjb change. After the initial migration from vish's jenkins server, I remember that they were passing for a while.
CI head is the "heapster-gce-e2e-ci" job running on the jenkins continuous integration server (as opposed to the jenkins PR builder server)
Thanks for clarification.
Let me know if it would be helpful to do a knowledge transfer about how the jenkins jobs are setup.
Removing flake label as this is not a flake.
There are no sig labels on this issue. Please by:<br(1) mentioning a sig: <br(2) specifying the label manually: <br<brNote: method (1) will trigger a notification to the team. You can find the team list .
/sig instrumentation because heapster /assign closing due to inactivity
/close


Question: close(sched.config.StopEverything) } <ins> // Cache returns the cache in scheduler for test to check the data in scheduler. func (sched *Scheduler) Cache() schedulercache.Cache { return sched.config.SchedulerCache } </ins> // Configurator defines I/O, caching, and other functionality needed to // construct a new scheduler. An implementation of this can be seen in // factory.go.

While lookint into , I spent few minutes looking into this test: And there is a problem that is relatively easy to reproduce. The problem is that we are running its test cases one by one. But nothing in the meantime is ensuring that the current state that the test assumes is reflected in scheduler. In particular the problem that I observed couple time is that: we finished one test case we call cleanupNodes [but we don't ensure that scheduler actually cleaned up this node] we start the next test case we try to schedule a pod that has some toleration (but shouldn't fit on our node) by accident the toleration make it fit on to the previous node so scheduler schedules it on the node from previous test case
/assign


Question: "kuberuntime_container_test.go", "kuberuntime_gc_test.go", "kuberuntime_image_test.go", <del> "kuberuntime_logs_test.go", </del> "kuberuntime_manager_test.go", "kuberuntime_sandbox_test.go", "labels_test.go",

We should make the CRI log parsing code in kubelet to be a library. So that container runtimes and external tools don't need to copy the whole code ( vendor it instead ). Referring
/assign
/sig node
I can help on it.
/assign
GitHub didn't allow me to assign the following users: yanxuean. Note that only can be assigned. <detailsIn response to : /assign Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Thanks. I have a branch ready for this. Will open a PR later and ping you when it is out.
ok


Question: type RlimitStats struct { Time metav1.Time `json:"time"` <del> // The max PID of OS. </del> <ins> // The max number of extant process (threads, precisely on Linux) of OS. See RLIMIT_NPROC in getrlimit(2). // The operating system ceiling on the number of process IDs that can be assigned. // On Linux, tasks (either processes or threads) consume 1 PID each. </ins> MaxPID *int64 `json:"maxpid,omitempty"` <del> // The number of running process in the OS. </del> <ins> // The number of running process (threads, precisely on Linux) in the OS. </ins> NumOfRunningProcesses *int64 `json:"curproc,omitempty"` }

PIDPressure is not set to true when the system has insufficient pid resource. PIDPressure is set to true correctly when the system is insufficient in pid resource. See cause below The max pid is not solely controlled by which is referred by . But it is also controlled by (whichever is smaller). However, they have different semantics resulting in the same constraint to how many processes/threads can be running on the system. <details</details<details</details<details</details<details</details<details</details<details</details
/sig node
I asked the same issue before,
My opinion: the minimum of and is the right number to count, for Linux. That will provide better instrumentation. If you're reserving a number of task IDs (PIDs) for system use, that minimum is the number that you're reserving from.
/triage accepted /priority important-longterm
/retitle System maximum PID inconsiderate of sysctl 'threads-max'
BTW I made a thing that might help folks test a fix: Originally I used it to create PIDs, to test whether apps cope with long PIDs. This version keeps the threads around so that you see them in .


Question: {"", "v1", "Service", true}, {"batch", "v1", "Job", true}, {"batch", "v1beta1", "CronJob", true}, <del> {"extensions", "v1beta1", "Ingress", true}, </del> <ins> {"networking.k8s.io", "v1", "Ingress", true}, </ins> {"apps", "v1", "DaemonSet", true}, {"apps", "v1", "Deployment", true}, {"apps", "v1", "ReplicaSet", true},

What happened: Sweeping uses of extensions/v1beta1 in preparation for , the addon manager stood out as using extensions/v1beta1 Ingress by default: What you expected to happen: All components shipped in 1.21 must use Ingress in order to work properly against a 1.22 API server. /sig networking cc per cluster/addons/addon-manager/OWNERS cc I'm not sure what the addon manager does if given a prune list of Ingress objects while still using a manifest containing extensions/v1 Ingress objects... this needs to be tested. /milestone v1.21 /priority important-soon /lifecycle frozen
The label(s) cannot be applied, because the repository doesn't have them <detailsIn response to : What happened: cc /priority important-soon /lifecycle frozen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig network
/triage accepted
how "urgent" is this? Can wait until next tuesday? Otherwise I can work on the tests this weekend :)
important, not urgent. it should happen early in the 1.21 cycle, but does not require work out of business hours
Hello! I am from the Bug Triage team! This issue has not been updated for a few weeks, so I'd like to check on the status. Code freeze is starting March 9th, 2021 (about 1 week from now). We want to ensure that each PR has a chance to be merged before code freeze. As the Issue is tagged for 1.21, is it still planned for this release?
we're waiting to the PR to get merged :) I guess it's important for the v1.21 release, for the sake of extensions apiGroup deprecation :) Will ping reviewers on the PR


Question: <-called close(ch) <del> <-called </del> <ins> // wait for 'called' to be closed for { if _, ok := <-called; !ok { break } } </ins> //-- ch = make(chan struct{}) <del> called = make(chan struct{}) </del> <ins> called2 := make(chan struct{}) </ins> running := make(chan struct{}) After(func() { Until(func() { close(running) <del> called <- struct{}{} </del> <ins> called2 <- struct{}{} </ins> }, 2*time.Second, ch) <del> }).Then(func() { close(called) }) </del> <ins> }).Then(func() { close(called2) }) </ins> <-running close(ch) <del> <-called // unblock the goroutine </del> <ins> <-called2 // unblock the goroutine </ins> now := time.Now() <del> <-called </del> <ins> // wait for 'called2' to be closed for { if _, ok := <-called2; !ok { break } } </ins> if time.Since(now) > 1800*time.Millisecond { t.Fatalf("Until should not have waited the full timeout period since we closed the stop chan") }

It failed on cc - please fix or reassign.
ping I am tempted to just disable the test per our standard protocol for dealing with Mesos test problems that aren't fixed in a timely fashion, but since it's a flake I'm willing to wait until it flakes again (unless you fix it first).
FYI - I've opened a PR w/ a fix , David Oppenheimer


Question: echo "OPTIONS='-b=kbr0 --selinux-enabled ${DOCKER_OPTS}'" >/etc/sysconfig/docker systemctl daemon-reload systemctl start docker } EOF

It appears that the provision-network script that is run on the kubernetes-master can cause salt provisioning on the minions to fail or block in a forever retry timeout error loop. I think it's because of the service network.restart that happens after openvswitch is installed in provision-network. I only say this because when I disable installing docker on kubernetes-master in vagrant, I do not have an issue. - is there a different way we can structure this on the kubernetes-master? For example, could we run the script in or something?


Question: res.Body.Close() } <del> func captureStdErr() (func() string, func(), error) { </del> <ins> func TestErrConnKilled(t *testing.T) { </ins> var buf bytes.Buffer <del> reader, writer, err := os.Pipe() if err != nil { return nil, nil, err } stderr := os.Stderr readerClosedCh := make(chan struct{}) stopReadingStdErr := func() string { writer.Close() <-readerClosedCh return buf.String() } klog.LogToStderr(true) cleanUp := func() { os.Stderr = stderr klog.LogToStderr(false) stopReadingStdErr() } os.Stderr = writer go func() { io.Copy(&buf, reader) readerClosedCh <- struct{}{} close(readerClosedCh) }() klog.LogToStderr(true) </del> <ins> klog.SetOutput(&buf) klog.LogToStderr(false) defer klog.LogToStderr(true) </ins> <del> return stopReadingStdErr, cleanUp, nil } func TestErrConnKilled(t *testing.T) { readStdErr, cleanUp, err := captureStdErr() if err != nil { t.Fatalf("unable to setup the test, err %v", err) } defer cleanUp() </del> handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { // this error must be ignored by the WithPanicRecovery handler // it is thrown by WithTimeoutForNonLongRunningRequests handler when a response has been already sent to the client and the handler timed out

/sig api-machinery /kind bug /priority important-soon
It seems something is racing writhing the logs ? cc:
/cc /triage accepted
Failed with this: pull-kubernetes-unit
this looks like a test-only race where the test overwriting the global os.Stderr in order to capture error output races with an async attempt to log to it from another test in the same package
which still needs to be fixed
It seems to me that this bug can not be fixed with as it is, namely used without a logger object or even a . The smallest tweak that occurs to me would be to add an option to to add an output destination, and later remove that addition. Thus, a test could add logging to its own buffer for the duration of running the code under test.
This may be similar to . How can I reproduce this issue?


Question: } if len(longest_tag) > 0 { tags.Insert(longest_tag) <del> } else if len(tags) > 0 { return nil, fmt.Errorf("Some, but not all, instances have prefix tags (%s is missing)", instance.Name) </del> <ins> } else { return nil, fmt.Errorf("Could not find any tag that is a prefix of instance name for instance %s", instance.Name) </ins> } } }

In some custom setup cases the prefix-matching logic for GCE load-balancers can break and end up creating a firewall rule with no tags. We should instead refuse to create the rule. In fact, if we find it, we should probably delete the rule, but that's maybe too aggressive. Looking at the logic, if we return error in the tag-selection, we'll abort the whole load-balancer creation, which is unfortunate, but not TERRIBLE. We could just silently not create the firewall but not return error, which will just retry over and over.
/cc
Are you active on this? may have some starter-project bandwidth to help.
Not active. It would indeed be a good starter project :)


Question: GCE_HOSTS= GCE_IMAGE_CONFIG_PATH=test/e2e_node/jenkins/image-config.yaml GCE_ZONE=us-central1-f <del> GCE_PROJECT=kubernetes-jenkins </del> <ins> GCE_PROJECT=k8s-jkns-ci-node-e2e </ins> CLEANUP=true GINKGO_FLAGS='--skip="[Flaky]|[Serial]"' SETUP_NODE=false

I want to isolate these in their own project:
Right now just the CI jobs, but after we also need to do this for the PR jobs.


Question: }, }, } <del> // run failed cases for i, failedTest := range v1FailTestInputs { </del> <ins> // run asymmetric cases for _, tc := range asymmetricInputs { testInput := tc.in </ins> corePodStatus := core.PodStatus{} // convert.. <del> if err := corev1.Convert_v1_PodStatus_To_core_PodStatus(&failedTest, &corePodStatus, nil); err == nil { t.Errorf("%v: Convert v1.PodStatus to core.PodStatus should have failed for input %+v", i, failedTest) </del> <ins> if err := corev1.Convert_v1_PodStatus_To_core_PodStatus(&testInput, &corePodStatus, nil); err != nil { t.Errorf("%s: Convert v1.PodStatus to core.PodStatus failed with error:%v for input %+v", tc.name, err.Error(), testInput) } if !reflect.DeepEqual(corePodStatus, tc.out) { t.Errorf("%s: expected %#v, got %#v", tc.name, tc.out.PodIPs, corePodStatus.PodIPs) </ins> } }

What happened: The cluster master in a Kubernetes v1.14.10 cluster was upgraded to V1.16.7. The process of upgrading nodes is under way and has been spread over a week so far. The cluster owner reported seeing a high percentage of pod status PATCH failures. The apiserver logs show errors like: The errors are all from the v1.14 nodes and have all been for pods associated with DaemonSets. We don't yet know why pods' IPs are changing, but it appears that pod status patches that include a changed result in an object where contains the old IP, and has the new IP. We believe this is because was introduced in Kubernetes v1.15 and the patch request from a v1.14 node results in both and being present in the result. If you aren't monitoring for request failures it may not obvious that any error is occurring: The pods continue to show their last successfully reported state. Deleting such pods, however, results in the pod being stuck in a state. We are only seeing this for DaemonSet pods - I suspect because whatever resulted in pod IPs changing also causes non-DaemonSet pods to be deleted and recreated. We have a number of cluster still on v1.14 that we expect to upgrade to v1.15 or v1.16. What you expected to happen: Kubernetes v1.14 nodes should be able to update pod status during the transition from v1.14 to v1.16 even if pod IPs change during that time. How to reproduce it (as minimally and precisely as possible): We have recreated this in the IBM Cloud Kubernetes Service by "reloading" nodes. The reload process leaves the Kubernetes node object in place, and essentially reloads the image with the proper kubeconfig to rejoin the cluster. The cluster I was debugging does not appear to have had nodes "reloaded" - something else is triggering the pod IP changes. Create a v1.14 cluster Deploy a DaemonSet (anything with pod networking should suffice) Upgrade cluster master to v1.16 Reload a v1.14 node. One the node is back up, shows the pod's network IP has changed. Kubelet and apiserver logs show the errors describe earlier. Anything else we need to know?

: Environment: Kubernetes version (use ): v1.14.x (nodes) and v1.16.7 (master) Cloud provider or hardware configuration: IBM Cloud Kubernetes Service OS (e.g: ): 18.04.4 LTS Kernel (e.g. ): 4.15.0-76-generic Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig api-machinery
/sig network
cc
/priority critical-urgent if it's possible for podIPs to change and the server defaults to , then the API server has to treat the singular pod IP as authoritative if there's a mismatch or the patch case breaks
/triage unresolved Comment when the issue is assessed and confirmed. I am a bot run by vllry.
/remove-sig api-machinery


Question: "containers": [ { "name": "webserver", <del> "image": "gcr.io/google_containers/nettest:1.7", </del> <ins> "image": "gcr.io/google_containers/nettest:1.8", </ins> "args": [ "-service=nettest", "-delay-shutdown=10"

<!-- Please only use this template for submitting enhancement requests --What would you like to be : The documentation suggests that "The default pull policy is IfNotPresent" This is not the case when imagePullPolicy is defined under the spec section in a Pod definition without a specific value. Example: This will result in a Pod having imagePullPolicy as Always. Why is this needed: This is needed for clarification of the default behaviour. It is necessary for the appropriate handling of helm charts, in cases that the imagePullPolicy is empty.
/sig docs
The imagePullPolicy default value is .
In this paragraph you can see that In your reference it points out the default action for the kubelet, not the behaviour when imagePullPolicy is declared. Please do have in mind that the imagePullPolicy option for Never is mentioned only in this paragraph in the document.
If an image tag of "latest" is used, the imagePullPolicy defaults to to "Always": If no tag and no digest are specified on the image, an image tag of "latest" is used: The behavior when no image digest or tag is specified should be documented in the API docs for the and fields:
I am interested on helping this issue, but I am new to kubernete repo, it could take me sometime
I found the following which is well documented in "Best Practices for Configuration" It would be expected to include more information on the options in the main link under
I'd like to work on this one.
Very welcome, but yesterday also mentioned that he is following up on this issue, you can communicate with him before starting

cc
This has failed The two failures are different, and both seemingly totally legitimate. The first one is simply a pod never entering running state within the 5 minute timeout. Expected error: <*errors.errorString | 0xc208475150: { s: "gave up waiting for pod 'nettest-dtwk3' to be 'running' after 5m0s", } gave up waiting for pod 'nettest-dtwk3' to be 'running' after 5m0s not to have occurred The second is not being reachable: Dec 7 01:22:47.275: Unable to connect/talk to the internet: Get dial tcp 74.125.202.147:80: i/o timeout The latter could be a result of a missing MASQ rule on the node, or GCP egress problems. I don't think that this is worth trying to debug further in the context of this test for intra-pod communication. I'll check more generally whether we have a current intermittent problems with respect to pods entering running state, or reaching the internet.
I think this test may be causing builds to time out, too. See comments from me in .
For the future, I actually don't think we need the test at all in the networking test. Maybe even move it do a different test altogether.
This is clearly a bug. I'll send a PR shortly: Regarding the ping to , why did you put in in the first place?
hmmm, not sure what you mean by "in the first place" ? I did not write the test...
sorry, I thought that you'd written that part of the test. I will find out who wrote that, figure out why they put it there in the first place, and then either remove it, or rewrite it to work correctly. Clearly that part of the test needs at least some basic retry to work consistently.
ok great, no problem just wasnt sure what you meant. SGTM. Id be okay w/ deleting the ping entirely regardless of who wrote it, its a very old snippet which doesnt really add much to the overall test suite (and in fact is questionable wether it should even be an assertion at all )
somebody put it there for some reason. Better to understand why before just deleting it.

i looked, its actually rob who i work with :) i will ask him in the morning unless you find him first :) (rrati)
Regarding my alleged "clear bug" above, I was completely wrong. There's nothing wring with that code. And unfortunately, when this test goes into a presumed endless loop (see comment above) it does not produce any output (as the output is cached by Ginkgo until the test completes). So it's very difficult to debug. But in summary I agree with this test is probably never completing, and causing tests to time out. Someone will need to isolate this test in a environment where it produces output and can be debugged. I'll keep trying, but need to head home soon, so if anyone wants to pick it up overnight, go ahead.
Smoking gun:
hmm - I'm just wondering why only this particular test is hanging forever (if we agree that's the case). I don' think that some requests are hanging forever in this test, but not in any other test...
Closing in favor of which should fix this. Or at least some way to set client timeout, which and are still figuring out.
It's happening again.
Talked to offline. I am assigning this one to you for triage. Let me know if you need any help.
AFAICT the last failure was just the pod nettest-7h5jj not coming up
Looks like this should be resolved by your pr: too. :-)
Maybe. Though the webserver only deadlocks in the failure case, so the test is probably still flaky...
Looks like this test is now explicitly flaking by failing, as I expected.
Based on the , it took 15 minutes to pull the image. closing this one as dup of
New Failure:
This looks like a race between populating the endpoints list and the /status, /read methods of the service exposed through the master proxy. It looks like the webserver pod is just trying to confirm that it can reach all peers. We can do this synchronously after having observed all expected endpoints in the test (or re-write the test to use the same pattern we do in services e2e).
The test looks really racy to me. I think the following 15 retries prevents it to fail more often, but not completely.

I will submit a pr to enfore webservers to wait till alll endpoints are up before they start to contact each other.
New failure
I am making this P0
It took more than 4 minutes for a nettest pod to come up. The nettest container is waiting for all endpoints to be ready for 2 minutes plus 15 attempts (5 sec interval) to contact its peers. This happens in less than 4 minutes. Hence the first 2 nettest pods finished contacting peers and declared failure.
The node you want is: I see: that's too much silence ...
From the It looks like kubelet is slow to start the container. Due to the lower logging level in the gke, it's hard to guess what could've slowed down the goroutine...


Question: return nil }) out.Write([]byte("nTo further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.n")) <del> return nil </del> <ins> return err </ins> // TODO consider printing more information about cluster }

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: What happened: reports that the cluster is running and returns exit code 0. correctly identifies that the cluster is not running and exists with exit code 1. What you expected to happen: should fail when the cluster is not running. How to reproduce it (as minimally and precisely as possible): Start and stop a kubernetes cluster using minikube and then type kubectl cluster-info. Anything else we need to know?: Environment: Kubernetes version (use ): Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.0", GitCommit:"", GitTreeState:"clean", BuildDate:"2017-09-28T22:57:57Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.5", GitCommit:"", GitTreeState:"clean", BuildDate:"2017-10-06T20:53:14Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"} Cloud provider or hardware configuration**: Local OS (e.g. from /etc/os-release): Ubuntu 16.04 Kernel (e.g. ): Linux Ubuntu16-HP-800 4.10.0-35-generic ~16.04.1-Ubuntu SMP Wed Sep 13 09:02:42 UTC 2017 x8664 x8664 x86_64 GNU/Linux Install tools: snap for kubectl. minikube local for k8s. Others:
/assign /sig cli


Question: {"invalid number", "{+12.3.0}", "cannot parse number +12.3.0"}, {"unterminated array", "{[1}", "unterminated array"}, {"unterminated filter", "{[?(.price]}", "unterminated filter"}, <ins> {"invalid multiple recursive descent", "{........}", "invalid multiple recursive descent"}, </ins> } for _, test := range failParserTests { _, err := Parse(test.name, test.text)

What happened: This issue was originally reported via the Kubernetes HackerOne bug bounty by HackerOne user but we have decided that due to the high privileges required, it can be addressed publicly. A user able to create CRDs could create a malicious CRD such that listing CRs will cause enormous amounts of CPU usage on the API server. What you expected to happen: The function should not expand strings with more than two consecutive periods. Either it should collapse additional repeated periods or it should return an error. How to reproduce it (as minimally and precisely as possible): /kind bug /area apiserver /area security /committee product-security /priority important-longterm
How about we take it a step further by rejecting the jsonpath which contains when parsing it? as more than 2 periods seem to make no sense IMO
good idea. I have posted to do what you suggest.
/sig api-machinery
/cc


Question: for _, taint := range node.Spec.Taints { if taint.Key == algorithm.DeprecatedTaintNodeNotReady { <del> // delete old taint </del> tDel := taint taintsToDel = append(taintsToDel, &tDel) <del> // add right taint </del> tAdd := taint tAdd.Key = algorithm.TaintNodeNotReady taintsToAdd = append(taintsToAdd, &tAdd) <ins> } </ins> <del> glog.Warningf("Detected deprecated taint key: %v on node: %v, will substitute it with %v", algorithm.DeprecatedTaintNodeNotReady, node.GetName(), algorithm.TaintNodeNotReady) </del> <ins> if taint.Key == algorithm.DeprecatedTaintNodeUnreachable { tDel := taint taintsToDel = append(taintsToDel, &tDel) </ins> <del> break </del> <ins> tAdd := taint tAdd.Key = algorithm.TaintNodeUnreachable taintsToAdd = append(taintsToAdd, &tAdd) </ins> } } if len(taintsToAdd) == 0 && len(taintsToDel) == 0 { return nil } <ins> glog.Warningf("Detected deprecated taint keys: %v on node: %v, will substitute them with %v", taintsToDel, node.GetName(), taintsToAdd) </ins> if !util.SwapNodeControllerTaint(nc.kubeClient, taintsToAdd, taintsToDel, node) { return fmt.Errorf("failed to swap taints of node %+v", node) }

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: What happened: When discussing issue in , I noticed: although is using key , it has actually been changed to in for quite a while. I am thinking we may need to fix this inconsistency in 1.9, by moving out of alpha. Anything else we need to know?: Task tracker: [x] update [x] update kubernetes doc
/sig scheduling


Question: func dataSourceIsEnabled(pvcSpec *core.PersistentVolumeClaimSpec) bool { if pvcSpec.DataSource != nil { <del> if pvcSpec.DataSource.Kind == pvc && *pvcSpec.DataSource.APIGroup == "" && utilfeature.DefaultFeatureGate.Enabled(features.VolumePVCDataSource) { </del> <ins> apiGroup := "" if pvcSpec.DataSource.APIGroup != nil { apiGroup = *pvcSpec.DataSource.APIGroup } if utilfeature.DefaultFeatureGate.Enabled(features.VolumePVCDataSource) && pvcSpec.DataSource.Kind == pvc && apiGroup == "" { </ins> return true } <del> if pvcSpec.DataSource.Kind == volumeSnapshot && *pvcSpec.DataSource.APIGroup == "snapshot.storage.k8s.io" && utilfeature.DefaultFeatureGate.Enabled(features.VolumeSnapshotDataSource) { </del> <ins> if utilfeature.DefaultFeatureGate.Enabled(features.VolumeSnapshotDataSource) && pvcSpec.DataSource.Kind == volumeSnapshot && apiGroup == "snapshot.storage.k8s.io" { </ins> return true } }

What happened: While implementing the new VolumePVCDataSource feature I discovered that even though the we don't use APIGroup (the appropriate setting is ) if the APIGroup is omitted it results in a nil pointer exception on the API server. What you expected to happen: APIGroup should be able to be omitted from the spec How to reproduce it (as minimally and precisely as possible): Create a spec to create a clone of a PVC with out the APIGroup parameter in the DataSource spec: The result will be an nil pointer exception during the verification of the datasource field. Adding the APIGroup entry will address this as a work-around: Anything else we need to know?: Environment: Kubernetes version (use ): Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.3", GitCommit:"", GitTreeState:"clean", BuildDate:"2019-06-06T01:44:30Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"7+", GitVersion:"v1.7.0-alpha.1.+-dirty", GitCommit:"", GitTreeState:"dirty", BuildDate:"2019-06-12T22:12:02Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"} Install tools: local-up-
I have a simple fix, will post PR in a moment
/sig storage
/priority critical-urgent /milestone v1.15


Question: apiVersion: v1 kind: Service metadata: <del> name: coredns </del> <ins> name: kube-dns </ins> namespace: kube-system labels: <del> k8s-app: coredns </del> <ins> k8s-app: kube-dns </ins> kubernetes.io/cluster-service: "true" addonmanager.kubernetes.io/mode: Reconcile kubernetes.io/name: "CoreDNS" spec: selector: <del> k8s-app: coredns </del> <ins> k8s-app: kube-dns </ins> clusterIP: $DNS_SERVER_IP ports: - name: dns

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: e2e test "gci-gce" is running in 2 versions: with kube-dns as DNS Service : https://k8s- with CoreDNS as DNS Service : https://k8s- In CoreDNS use case the test is failing What you expected to happen: This e2e tests should not depend on what DNS service is running as default in K8s. CoreDNS is currently BETA, will move to GA soon (after v1.10 though) See : How to reproduce it (as minimally and precisely as possible): look at logs of dashboard : https://k8s- go run -v --test --test_args='--[sig-autoscaling]sDNSshorizontalsautoscalings[DisabledForLargeClusters]skube-dns-autoscalersshouldsscaleskube-dnsspodssinsbothsnonfaultysandsfaultysscenarios$' Anything else we need to know?: Environment: Kubernetes version (use ): branch master Cloud provider or hardware configuration: GCE Install tools: deployment intra-test Others: see https://k8s-
/sig autoscaling
/unsig cli
Looks like the test is searching specifically for a deployment labeled "kube-dns". There are several references to "kube-dns" scattered around it, actually.
Right now the tool "dns-horizontal-autoscaller" seems dedicated for kube-dns. This tools is deployed as "add-on" when enabled by this ENV variable : I guess we should have the definition of another dns-horizontal-autoscaller, dedicated to coredns (unless we generate the yaml file). and the choice of deployment is based on the ENV variable . see
Currently, the translation of the deployment of "dns-horizontal-autoscaller" for CoreDNS is in progress by It is expected that the corresponding e2e test will work as is (as the artifacts are selected by labels, which are the same for CroeDNS and kube-dns).


Question: if t.Status.Reason == "AppArmor" { return true, nil } <ins> // Loading a profile not available on disk should return a container creation error if len(t.Status.ContainerStatuses) > 0 && t.Status.ContainerStatuses[0].State.Waiting.Reason == kuberuntime.ErrCreateContainer.Error() { return true, nil } </ins> } return false, nil })

Also appears to be failing on: Failures as of 01/05. /kind failing-test <!-- If this is a flake, please add: /kind flake --/sig node
/triage accepted /priority important-soon Guessing since all tests with apparmor started failing - maybe app armor config was removed or some PR submitted recently.
/assign
ci-cri-containerd-node-e2e-features Last green run: First failed run:
This () is the only PR merged in kubernetes during the time between: Test started yesterday at 8:11 PM passed after 13m50s. (more info) Test started yesterday at 9:11 PM failed after 16m6s. (more info) And it happens to be related to AppArmor:
Working on a fix in


Question: } // Updated updates the lastUpdated timestamp. <del> func (hs *ProxierHealthServer) Updated() { </del> <ins> func (hs *proxierHealthServer) Updated() { </ins> hs.lastUpdated.Store(hs.clock.Now()) } // QueuedUpdate updates the lastQueued timestamp. <del> func (hs *ProxierHealthServer) QueuedUpdate() { </del> <ins> func (hs *proxierHealthServer) QueuedUpdate() { </ins> hs.lastQueued.Store(hs.clock.Now()) } // Run starts the healthz http server and returns. <del> func (hs *ProxierHealthServer) Run() { </del> <ins> func (hs *proxierHealthServer) Run() { </ins> serveMux := http.NewServeMux() serveMux.Handle("/healthz", healthzHandler{hs: hs}) server := hs.httpFactory.New(hs.addr, serveMux)

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: , from whatever combination of inputs we stuck into it, segfaulted. The segfault, which originates from a slightly modified fork of v1.17, looks as such: Translated to master, where the problem appears to be present, the segfault originates from this call: And that is called from here: This seems odd, because The only pointer involved in the segfaulting call is itself The caller nil-checks before calling a method on . I did some digging and found , which is a fix for this problem. However, this fix appears to be undone in . There doesn't seem to be a reason for this to have been changed in the latter PR. What you expected to happen: should not segfault from any bizarre combination of user input. If the situation is invalid, it should fail gracefully. How to reproduce it (as minimally and precisely as possible): Honestly, I'm not 100% sure what combination of inputs we crammed into , but I can locate them and post them if necessary. Anything else we need to know?: I don't feel confident opening a PR to fix this myself, but in addition to recreating the fix from , a comment should be explaining why there is this weird, seemingly pointless layer of indirection when assigning to the interface, so that this doesn't get un-fixed again.
Maybe decompose the above statement into multiple lines which should give us more idea which field was nil.
/sig network These SIGs are my best guesses for this issue. Please comment if I am incorrect about one. I am a bot run by vllry.
can't be nil, as it is of type , which is not an interface or pointer is an interface. However, when built from the exported constructor, , it is set to a literal , which is likewise not a pointer Therefore, the only remaining field is itself. Plus, I don't quite recall my go segfault stack trace syntax, but I'm relatively sure that indicates that the first argument to the method is , nil, and that the first argument to methods is the object being called on.

/triage unresolved Comment when the issue is assessed and confirmed. I am a bot run by vllry.
When config.HealthzBindAddress is empty, we would encounter the scenario depicted in the stack trace. I have proposed a fix.
/assign
That commit, , is part of the PR that I believe introduced the problem. I strongly believe the problem to be one of a non-nil interface with a nil receiver. This is the only way that could be nil in the snippet of code from . Before PR (the changes from which are present in the version I ran), there was this odd looking set of code: The variable healthzServer, which is a pointer of type , was assigned to the variable , which is an interface of type . Then, later on, the variable was passed to , as an argument of type . This looks unexpected, and it appears that the variable is extraneous. After the changes in PR , the variable was removed, and was passed directly to as the argument for interface type . The former set of changes, with the intermediate variable, are meant to avoid the situation of a non-nil interface with a nil value. If the variable is never set in the conditional statement, then it is a nil interface. If is passed directly, then the result is the argument to is an an interface who's value is and who's concrete type is . This means that is called with as the receiver. I strongly believe this to be the root cause of the issue, but I do not know enough about kubernetes to open a PR with the correct fix, which would be to reinstate the variable (after the renaming in , this would be a variable of type ) to use as a go-between, to once again avoid this situation of a non-nil interface with a nil receiver. I believe that the removal of the variable was out of a misunderstanding of its purpose. If my assessment of the problem is incorrect, please let me know so I can understand where my logic is wrong. It is very possible that I am misunderstanding the phenomenon of a non-nil interface with a nil receiver. I would open a PR myself, but I do not know enough about Kubernetes to be confident in my ability to correctly fix and test this problem.
/remove-triage unresolved


Question: <ins> /* Copyright 2023 The Kubernetes Authors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package volumepathhandler import ( "fmt" "testing" ) func pathWithSuffix(suffix string) string { return fmt.Sprintf("%s%s", "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/pvc-1d205234-06cd-4fe4-a7ea-0e8f3e2faf5f/dev/e196ebd3-2ab1-4185-bed4-b997ba38d1dc", suffix) } func TestCleanBackingFilePath(t *testing.T) { const defaultPath = "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/pvc-1d205234-06cd-4fe4-a7ea-0e8f3e2faf5f/dev/e196ebd3-2ab1-4185-bed4-b997ba38d1dc" testCases := []struct { name          string input         string expectedOuput string }{ { name:          "regular path", input:         defaultPath, expectedOuput: defaultPath, }, { name:          "path is suffixed with whitespaces", input:         fmt.Sprintf("%srtn ", defaultPath), expectedOuput: defaultPath, }, { name:          "path is suffixed with "(deleted)"", input:         pathWithSuffix("(deleted)"), expectedOuput: defaultPath, }, { name:          "path is suffixed with "(deleted)" and whitespaces", input:         pathWithSuffix(" (deleted)t"), expectedOuput: defaultPath, }, } for _, tc := range testCases { t.Run(tc.name, func(t *testing.T) { output := cleanBackingFilePath(tc.input) if output != tc.expectedOuput { t.Fatalf("expected %q, got %q", tc.expectedOuput, output) } }) } } </ins>

A pod uses a PVC with . The block volume is backed by a block device on the node (which is created by some CSI storage providers). If the underlying block device corresponding to the volume is accidentally disconnected from the node (due to network, hardware problem, etc...), the user cannot delete the pod. The pod would be stuck in terminating forever since Kubelet refuses to unmount the block volume. This prevents the user from cleaning up the pod and spinning up a new replacement pod thus leading to a long service degradation. For example, if the pod is part of a statefulset, the replacement pod cannot come up due to the old pod being stuck terminating. Kubelet should be able to finish terminating a pod when the corresponding block device is already removed from the node. a CSI storage provider that supports . For example, a statefulset that has volumeClaimTemplates with . For example: <details close<summaryclick to open yaml manifest</summary</detailsa network/hardware problem in which the backing block device is removed from the host unexpectedly. In our example, we delete a Longhorn engine instance manager which leads to the underlying block device being removed from the host. that the pod cannot access the block volume, try to delete the pod. pod is stuck in terminating state forever. Logs from kubelet: <details close<summaryClick to open logs</summary<br</detailsPlease pay attention to the log . Kubelet is refusing to terminate the pod because the underlying block device is removed from the host. If we run on the host, we can see that the backing file of the corresponding loop device is marked as : In our humble opinion, we think that Kubelet should process to remove the loop device corresponding to the deleted block device so that pod can finish terminating as detailed in this PR: We have tested that PR and can confirm that it fixes the issue. We have prepared a PR at : We have tested that PR and can confirm that it fixes the issue. <details</details<detailsRancher RKE2 </details<details</details<detailsRancher RKE2 </details<details</details<details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance.

The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig storage
Ref:
/cc
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle rotten [1]:
/remove-lifecycle rotten
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle rotten
/remove-lifecycle stale
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale
kubelet log 13:31:57 wanguo-72 kubelet[1131]: E0428 13:31:57. 1131 ] Operation for "{^-54fa-4068-bbad- podName:-9521-4aed-9bfe- nodeName:}" failed. No retries permitted until 2023-04-28 13:33:59. +0800 CST m=+8838. (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "pvc--a4e4-48a7-9057-" (UniqueName: "^-54fa-4068-bbad-") pod "-9521-4aed-9bfe-" (UID: "-9521-4aed-9bfe-") : blkUtil.DetachFileDevice failed. globalUnmapPath:, podUID: -9521-4aed-9bfe-, bindMount: true: failed to unmap device from map path. mapPath is empty node poweroff restart , pod use block mode volume, and pod stuck in terminating
so it is a new issue, and I encounter it as well with XSKY CSI after node restart. My kubernetes version is 1.25.8
which Kubernetes version is it?
Hi, I also see the issue reported by I'm using v1.25.10. I assume it happens when using containerized kubelet. In my environment, losetup reports weird backing file paths. This can be reproduced as follows. Until this moment, the correct path is visible from the host. But, after terminating the container, I get the wrong results. I consider this behavior causes failing unmap. Any thoughts?


Question: }, { "kind":"None", <del> "metadata":{"name":"127.0.0.2"}, </del> <ins> "metadata":{ "name":"127.0.0.2", "labels":{ "kubernetes.io/hostname":"127.0.0.2" } }, </ins> "status":{ "capacity":{"cpu":"8"}, "addresses":[

I love using but I cannot figure out how to print an annotation that contains a special character like or or . This does not work: nor this: nor this: nor this: kubectl get pods tmp--dhpx0 -o jsonpath="{$['metadata']['annotations']['']}" (the last one should work according to what I can find about jsonPath, but doesn't, and anyhow is hard to get shell escaping right for. Need a way to quote a
A use case for this is extracting "created-by" or "init-container" annotations. I've wanted to do both in the last month and fought jsonpath.
FYI, this is possible with go-template. We should still fix the jsonpath though. Similar issue:


Question: <ins> /* Copyright 2016 The Kubernetes Authors All rights reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package cache import "testing" func Test_AddVolume_Positive_NewVolume(t *testing.T) { // Arrange vc := NewAttachDetachVolumeCache() volumeName := "volume-name" // Act vc.AddVolume(volumeName) // Assert volumeExists := vc.VolumeExists(volumeName) if !volumeExists { t.Fatalf("Added volume %q does not exist, it should.", volumeName) } } func Test_AddVolume_Positive_ExistingVolume(t *testing.T) { // Arrange vc := NewAttachDetachVolumeCache() volumeName := "volume-name" vc.AddVolume(volumeName) // Act vc.AddVolume(volumeName) // Assert volumeExists := vc.VolumeExists(volumeName) if !volumeExists { t.Fatalf("Added volume %q does not exist, it should.", volumeName) } } func Test_AddNode_Positive_NewNodeVolumeExists(t *testing.T) { // Arrange vc := NewAttachDetachVolumeCache() volumeName := "volume-name" nodeName := "node-name" vc.AddVolume(volumeName) // Act nodeErr := vc.AddNode(nodeName, volumeName) // Assert if nodeErr != nil { t.Fatalf("AddNode failed. Expected: <no error> Actual: <%v>", nodeErr) } nodeExists, nodeExistsErr := vc.NodeExists(nodeName, volumeName) if nodeExistsErr != nil { t.Fatalf("NodeExists failed. Expected: <no error> Actual: <%v>", nodeExistsErr) } if !nodeExists { t.Fatalf("Added node %q does not exist, it should.", nodeName) } } func Test_AddNode_Positive_NodeExistsVolumeExists(t *testing.T) { // Arrange vc := NewAttachDetachVolumeCache() volumeName := "volume-name" nodeName := "node-name" vc.AddVolume(volumeName) nodeErr1 := vc.AddNode(nodeName, volumeName) if nodeErr1 != nil { t.Fatalf("First call to AddNode failed. Expected: <no error> Actual: <%v>", nodeErr1) } // Act nodeErr2 := vc.AddNode(nodeName, volumeName) // Assert if nodeErr2 != nil { t.Fatalf("Second call to AddNode failed. Expected: <no error> Actual: <%v>", nodeErr2) } nodeExists, nodeExistsErr := vc.NodeExists(nodeName, volumeName) if nodeExistsErr != nil { t.Fatalf("NodeExists failed. Expected: <no error> Actual: <%v>", nodeExistsErr) } if !nodeExists { t.Fatalf("Added node %q does not exist, it should.", nodeName) } } func Test_AddNode_Negative_NewNodeVolumeDoesntExists(t *testing.T) { // Arrange vc := NewAttachDetachVolumeCache() volumeName := "volume-name" nodeName := "node-name" // Act nodeErr := vc.AddNode(nodeName, volumeName) // Assert if nodeErr == nil { t.Fatalf("AddNode did not fail. Expected: <"failed to add node...no volume with that name exists in the list of managed volumes"> Actual: <no error>") } nodeExists, nodeExistsErr := vc.NodeExists(nodeName, volumeName) if nodeExistsErr == nil { t.Fatalf("NodeExists did not fail. Expected: <failed to check if node...no volume with that name exists in the list of managed volumes> Actual: <no error>") } if nodeExists { t.Fatalf("Added node %q exists, it should not.", nodeName) } } func Test_AddPod_Positive_NewPodNodeExistsVolumeExists(t *testing.T) { // Arrange vc := NewAttachDetachVolumeCache() volumeName := "volume-name" nodeName := "node-name" podName := "pod-name" vc.AddVolume(volumeName) nodeErr := vc.AddNode(nodeName, volumeName) if nodeErr != nil { t.Fatalf("AddNode failed. Expected: <no error> Actual: <%v>", nodeErr) } // Act podErr := vc.AddPod(podName, nodeName, volumeName) // Assert if podErr != nil { t.Fatalf("AddPod failed. Expected: <no error> Actual: <%v>", podErr) } podExists, podExistsErr := vc.PodExists(podName, nodeName, volumeName) if podExistsErr != nil { t.Fatalf("PodExists failed. Expected: <no error> Actual: <%v>", podExistsErr) } if !podExists { t.Fatalf("Added pod %q does not exist, it should.", podName) } } func Test_AddPod_Positive_PodExistsNodeExistsVolumeExists(t *testing.T) { // Arrange vc := NewAttachDetachVolumeCache() volumeName := "volume-name" nodeName := "node-name" podName := "pod-name" vc.AddVolume(volumeName) nodeErr := vc.AddNode(nodeName, volumeName) if nodeErr != nil { t.Fatalf("AddNode failed. Expected: <no error> Actual: <%v>", nodeErr) } podErr1 := vc.AddPod(podName, nodeName, volumeName) if podErr1 != nil { t.Fatalf("AddPod failed. Expected: <no error> Actual: <%v>", podErr1) } // Act podErr2 := vc.AddPod(podName, nodeName, volumeName) // Assert if podErr2 != nil { t.Fatalf("AddPod failed. Expected: <no error> Actual: <%v>", podErr2) } podExists, podExistsErr := vc.PodExists(podName, nodeName, volumeName) if podExistsErr != nil { t.Fatalf("PodExists failed. Expected: <no error> Actual: <%v>", podExistsErr) } if !podExists { t.Fatalf("Added pod %q does not exist, it should.", podName) } } func Test_AddPod_Negative_NewPodNodeDoesntExistsVolumeExists(t *testing.T) { // Arrange vc := NewAttachDetachVolumeCache() volumeName := "volume-name" nodeName := "node-name" podName := "pod-name" vc.AddVolume(volumeName) // Act podErr := vc.AddPod(podName, nodeName, volumeName) // Assert if podErr == nil { t.Fatalf("AddPod did not fail. Expected: <"failed to add pod...no node with that name exists in the list of attached nodes for that volume"> Actual: <no error>") } podExists, podExistsErr := vc.PodExists(podName, nodeName, volumeName) if podExistsErr == nil { t.Fatalf("PodExists did not fail. Expected: <"failed to check if pod exists...no node with that name exists in the list of attached nodes for that volume"> Actual: <no error>") } if podExists { t.Fatalf("Added pod %q exists, it should not.", podName) } } func Test_AddPod_Negative_NewPodNodeDoesntExistsVolumeDoesntExists(t *testing.T) { // Arrange vc := NewAttachDetachVolumeCache() volumeName := "volume-name" nodeName := "node-name" podName := "pod-name" // Act podErr := vc.AddPod(podName, nodeName, volumeName) // Assert if podErr == nil { t.Fatalf("AddPod did not fail. Expected: <"failed to add pod...no volume with that name exists in the list of managed volumes"> Actual: <no error>") } podExists, podExistsErr := vc.PodExists(podName, nodeName, volumeName) if podExistsErr == nil { t.Fatalf("PodExists did not fail. Expected: <"failed to check if node...no volume with that name exists in the list of managed volumes"> Actual: <no error>") } if podExists { t.Fatalf("Added pod %q exists, it should not.", podName) } } func Test_VolumeExists_Positive_NonExistantVolume(t *testing.T) { // Arrange vc := NewAttachDetachVolumeCache() notAddedVolumeName := "volume-not-added-name" // Act notAddedVolumeExists := vc.VolumeExists(notAddedVolumeName) // Assert if notAddedVolumeExists { t.Fatalf("Not added volume %q exists, it should not.", notAddedVolumeName) } } func Test_NodeExists_Positive_NonExistantNodeVolumeExists(t *testing.T) { // Arrange vc := NewAttachDetachVolumeCache() volumeName := "volume-name" notAddedNodeName := "node-not-added-name" vc.AddVolume(volumeName) // Act notAddedNodeExists, notAddedNodeExistsErr := vc.NodeExists(notAddedNodeName, volumeName) // Assert if notAddedNodeExistsErr != nil { t.Fatalf("NodeExists failed. Expected: <no error> Actual: <%v>", notAddedNodeExistsErr) } if notAddedNodeExists { t.Fatalf("Not added node %q exists, it should not.", notAddedNodeName) } } func Test_NodeExists_Negative_NonExistantNodeVolumeDoesntExist(t *testing.T) { // Arrange vc := NewAttachDetachVolumeCache() volumeName := "volume-name" notAddedNodeName := "node-not-added-name" // Act notAddedNodeExists, notAddedNodeExistsErr := vc.NodeExists(notAddedNodeName, volumeName) // Assert if notAddedNodeExistsErr == nil { t.Fatalf("NodeExists did not fail. Expected: <failed to check if node...no volume with that name exists in the list of managed volumes> Actual: <no error>") } if notAddedNodeExists { t.Fatalf("Added node %q exists, it should not.", notAddedNodeName) } } func Test_PodExists_Positive_NonExistantPodNodeExistsVolumeExists(t *testing.T) { // Arrange vc := NewAttachDetachVolumeCache() volumeName := "volume-name" nodeName := "node-name" notAddedPodName := "pod-not-added-name" vc.AddVolume(volumeName) addNodeErr := vc.AddNode(nodeName, volumeName) if addNodeErr != nil { t.Fatalf("AddNode for node %q failed. Expected: <no error> Actual: <%v>", nodeName, addNodeErr) } // Act notAddedPodExists, notAddedPodExistsErr := vc.PodExists(notAddedPodName, nodeName, volumeName) // Assert if notAddedPodExistsErr != nil { t.Fatalf("PodExists failed. Expected: <no error> Actual: <%v>", notAddedPodExistsErr) } if notAddedPodExists { t.Fatalf("Not added pod %q exists, it should not.", notAddedPodName) } } func Test_PodExists_Negative_NonExistantPodNodeDoesntExistVolumeExists(t *testing.T) { // Arrange vc := NewAttachDetachVolumeCache() volumeName := "volume-name" nodeName := "node-name" notAddedPodName := "pod-not-added-name" vc.AddVolume(volumeName) // Act notAddedPodExists, notAddedPodExistsErr := vc.PodExists(notAddedPodName, nodeName, volumeName) // Assert if notAddedPodExistsErr == nil { t.Fatalf("PodExists did not fail. Expected: <"failed to check if pod exists...no node with that name exists in the list of attached nodes for that volume"> Actual: <no error>") } if notAddedPodExists { t.Fatalf("Added pod %q exists, it should not.", notAddedPodName) } } func Test_PodExists_Negative_NonExistantPodNodeDoesntExistVolumeDoesntExist(t *testing.T) { // Arrange vc := NewAttachDetachVolumeCache() volumeName := "volume-name" nodeName := "node-name" notAddedPodName := "pod-not-added-name" // Act notAddedPodExists, notAddedPodExistsErr := vc.PodExists(notAddedPodName, nodeName, volumeName) // Assert if notAddedPodExistsErr == nil { t.Fatalf("PodExists did not fail. Expected: <"failed to check if node...no volume with that name exists in the list of managed volumes"> Actual: <no error>") } if notAddedPodExists { t.Fatalf("Added pod %q exists, it should not.", notAddedPodName) } } func Test_DeleteVolume_Positive_VolumeExists(t *testing.T) { // Arrange vc := NewAttachDetachVolumeCache() volumeName := "volume-name" vc.AddVolume(volumeName) // Act deleteVolumeErr := vc.DeleteVolume(volumeName) // Assert if deleteVolumeErr != nil { t.Fatalf("DeleteVolume failed. Expected: <no error> Actual: <%v>", deleteVolumeErr) } volumeExists := vc.VolumeExists(volumeName) if volumeExists { t.Fatalf("Deleted volume %q still exists, it should not.", volumeName) } } func Test_DeleteVolume_Negative_VolumeDoesntExists(t *testing.T) { // Arrange vc := NewAttachDetachVolumeCache() notAddedVolumeName := "volume-not-added-name" // Act deleteVolumeErr := vc.DeleteVolume(notAddedVolumeName) // Assert if deleteVolumeErr == nil { t.Fatalf("DeleteVolume did not fail. Expected: <"failed to delete volume...no volume with that name exists in the list of managed volumes"> Actual: <no error>") } notAddedVolumeExists := vc.VolumeExists(notAddedVolumeName) if notAddedVolumeExists { t.Fatalf("Not added volume %q exists, it should not.", notAddedVolumeName) } } func Test_DeleteNode_Positive_NodeExistsVolumeExists(t *testing.T) { // Arrange vc := NewAttachDetachVolumeCache() volumeName := "volume-name" nodeName := "node-name" vc.AddVolume(volumeName) nodeErr := vc.AddNode(nodeName, volumeName) if nodeErr != nil { t.Fatalf("AddNode failed. Expected: <no error> Actual: <%v>", nodeErr) } // Act deleteNodeErr := vc.DeleteNode(nodeName, volumeName) // Assert if deleteNodeErr != nil { t.Fatalf("DeleteNode failed. Expected: <no error> Actual: <%v>", deleteNodeErr) } nodeExists, nodeExistsErr := vc.NodeExists(nodeName, volumeName) if nodeExistsErr != nil { t.Fatalf("NodeExists failed. Expected: <no error> Actual: <%v>", nodeExistsErr) } if nodeExists { t.Fatalf("Deleted node %q still exists, it should not.", nodeName) } } func Test_DeleteNode_Negative_NodeDoesntExistVolumeExists(t *testing.T) { // Arrange vc := NewAttachDetachVolumeCache() volumeName := "volume-name" notAddedNodeName := "node-not-added-name" vc.AddVolume(volumeName) // Act deleteNodeErr := vc.DeleteNode(notAddedNodeName, volumeName) // Assert if deleteNodeErr == nil { t.Fatalf("DeleteNode did not fail. Expected: <"failed to delete node...no node with the that name exists in the list of attached nodes for that volume"> Actual: <no error>") } notAddedNodeExists, notAddedNodeExistsErr := vc.NodeExists(notAddedNodeName, volumeName) if notAddedNodeExistsErr != nil { t.Fatalf("NodeExists failed. Expected: <no error> Actual: <%v>", notAddedNodeExistsErr) } if notAddedNodeExists { t.Fatalf("Not added node %q exists, it should not.", notAddedNodeName) } } func Test_DeleteNode_Negative_NodeDoesntExistVolumeDoesntExist(t *testing.T) { // Arrange vc := NewAttachDetachVolumeCache() volumeName := "volume-name" notAddedNodeName := "node-not-added-name" // Act deleteNodeErr := vc.DeleteNode(notAddedNodeName, volumeName) // Assert if deleteNodeErr == nil { t.Fatalf("DeleteNode did not fail. Expected: <"failed to delete node...no volume with that name exists in the list of managed volumes"> Actual: <no error>") } notAddedNodeExists, notAddedNodeExistsErr := vc.NodeExists(notAddedNodeName, volumeName) if notAddedNodeExistsErr == nil { t.Fatalf("NodeExists did not fail. Expected: <failed to check if node...no volume with that name exists in the list of managed volumes"> Actual: <no error>") } if notAddedNodeExists { t.Fatalf("Not added node %q exists, it should not.", notAddedNodeName) } } func Test_DeletePod_Positive_PodExistsNodeExistsVolumeExists(t *testing.T) { // Arrange vc := NewAttachDetachVolumeCache() volumeName := "volume-name" nodeName := "node-name" podName := "pod-name" vc.AddVolume(volumeName) nodeErr := vc.AddNode(nodeName, volumeName) if nodeErr != nil { t.Fatalf("AddNode failed. Expected: <no error> Actual: <%v>", nodeErr) } podErr := vc.AddPod(podName, nodeName, volumeName) if podErr != nil { t.Fatalf("AddPod failed. Expected: <no error> Actual: <%v>", podErr) } // Act deletePodErr := vc.DeletePod(podName, nodeName, volumeName) // Assert if deletePodErr != nil { t.Fatalf("DeletePod failed. Expected: <no error> Actual: <%v>", podName) } podExists, podExistsErr := vc.PodExists(podName, nodeName, volumeName) if podExistsErr != nil { t.Fatalf("PodExists failed. Expected: <no error> Actual: <%v>", podExistsErr) } if podExists { t.Fatalf("Deleted pod %q still exists, it should not.", podName) } } func Test_DeletePod_Positive_PodDoesntExistNodeExistsVolumeExists(t *testing.T) { // Arrange vc := NewAttachDetachVolumeCache() volumeName := "volume-name" nodeName := "node-name" podName := "pod-name" vc.AddVolume(volumeName) nodeErr := vc.AddNode(nodeName, volumeName) if nodeErr != nil { t.Fatalf("AddNode failed. Expected: <no error> Actual: <%v>", nodeErr) } // Act deletePodErr := vc.DeletePod(podName, nodeName, volumeName) // Assert if deletePodErr == nil { t.Fatalf("DeletePod did not fail. Expected: <"failed to delete pod...no pod with that name exists in the list of scheduled pods under that node/volume"> Actual: <no error>") } podExists, podExistsErr := vc.PodExists(podName, nodeName, volumeName) if podExistsErr != nil { t.Fatalf("PodExists failed. Expected: <no error> Actual: <%v>", podExistsErr) } if podExists { t.Fatalf("Deleted pod %q still exists, it should not.", podName) } } func Test_DeletePod_Positive_PodDoesntExistNodeDoesntExistVolumeExists(t *testing.T) { // Arrange vc := NewAttachDetachVolumeCache() volumeName := "volume-name" nodeName := "node-name" podName := "pod-name" vc.AddVolume(volumeName) // Act deletePodErr := vc.DeletePod(podName, nodeName, volumeName) // Assert if deletePodErr == nil { t.Fatalf("DeletePod did not fail. Expected: <"failed to delete pod...no node with that name exists in the list of attached nodes for that volume"> Actual: <no error>") } podExists, podExistsErr := vc.PodExists(podName, nodeName, volumeName) if podExistsErr == nil { t.Fatalf("PodExists did not fail. Expected: <failed to check if pod...no node with that name exists in the list of attached nodes for that volume"> Actual: <no error>") } if podExists { t.Fatalf("Deleted pod %q still exists, it should not.", podName) } } func Test_DeletePod_Positive_PodDoesntExistNodeDoesntExistVolumeDoesntExist(t *testing.T) { // Arrange vc := NewAttachDetachVolumeCache() volumeName := "volume-name" nodeName := "node-name" podName := "pod-name" // Act deletePodErr := vc.DeletePod(podName, nodeName, volumeName) // Assert if deletePodErr == nil { t.Fatalf("DeletePod did not fail. Expected: <"failed to delete pod...no volume with that name exists in the list of managed volumes"> Actual: <no error>") } podExists, podExistsErr := vc.PodExists(podName, nodeName, volumeName) if podExistsErr == nil { t.Fatalf("PodExists did not fail. Expected: <failed to check if pod...no volume with that name exists in the list of managed volumes"> Actual: <no error>") } if podExists { t.Fatalf("Deleted pod %q still exists, it should not.", podName) } } /* t.Fatalf("%q", notAddedNodeExistsErr) */ </ins>

e2e-cos-alpha-features All I'm not sure it was ever running. No response No response /sig node /priority important-longterm since it's alpha features, setting priority lower
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
kubelet log shows error below:
We tried to fix it in before. I opened


Question: } } <ins> // Echo handler that returns the contents of request headers in the body headerEchoHandler := func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(200) output := "" for k, arr := range r.Header { for _, v := range arr { output += fmt.Sprintf("%s: %sn", k, v) } } w.Write([]byte(output)) } </ins> prober := New() testCases := []struct { handler    func(w http.ResponseWriter, r *http.Request) reqHeaders http.Header health     probe.Result accBody    string <ins> notBody    string </ins> }{ // The probe will be filled in below.  This is primarily testing that an HTTP GET happens. {

Grabbed tcpdump of a readiness probe - the src ip was the bridge address (10.244.1.1), but apart from this, there is no identifying information in the HTTP request that this is a Kubernetes daemon requesting readiness checks - this might help QoS/priority assignment by the container application.
IIUC, you mean we should add a better when kubernetes daemon request a readiness check to identify this request comes from kubernetes daemon?
Yes, we should override the default net/http user-agent with our own. Better usability for users and we have no future upgrade hazards if users start hardcoding and expecting this default User-Agent string in their analytic tools.
Any news about this one? It would be nice! I've this little hack in my nginx configuration (in order to not be spamed by health check in logs) :)
This can probably be closed now that probes have support.


Question: } } <ins> // Save volume info in pod dir // persist volume info data for teardown nodeName := string(c.plugin.host.GetNodeName()) volData := map[string]string{ volDataKey.specVolID:           c.spec.Name(), volDataKey.volHandle:           volumeHandle, volDataKey.driverName:          string(c.driverName), volDataKey.nodeName:            nodeName, volDataKey.volumeLifecycleMode: string(c.volumeLifecycleMode), volDataKey.attachmentID:        getAttachmentName(volumeHandle, string(c.driverName), nodeName), } err = saveVolumeData(parentDir, volDataFileName, volData) defer func() { // Only if there was an error and volume operation was considered // finished, we should remove the directory. if err != nil && volumetypes.IsOperationFinishedError(err) { // attempt to cleanup volume mount dir if removeerr := removeMountDir(c.plugin, dir); removeerr != nil { klog.Error(log("mounter.SetUpAt failed to remove mount dir after error [%s]: %v", dir, removeerr)) } } }() if err != nil { errorMsg := log("mounter.SetUpAt failed to save volume info data: %v", err) klog.Error(errorMsg) return volumetypes.NewTransientOperationFailure(errorMsg) } </ins> err = csi.NodePublishVolume( ctx, volumeHandle,

Instantiating the interface causes the creation of pod local volume directory and json data file and not - This causes a problem where is failed (and as a result never gets called) then those pod local volume directories will never be cleaned up. /sig storage cc
Looks like is the PR that introduced this change and moved data file creation calls to .
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
/remove-lifecycle stale
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-testing, kubernetes/test-infra and/or . /close
Closing this issue. <detailsIn response to : Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/repen /remove-lifecycle stale
/reopen
Reopened this issue. <detailsIn response to : /reopen </details
/remove-lifecycle stale
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-testing, kubernetes/test-infra and/or . /close
Closing this issue.

<detailsIn response to : Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen /remove-lifecycle rotten
Reopened this issue. <detailsIn response to : /reopen /remove-lifecycle rotten Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-contributor-experience at . /close
Closing this issue. <detailsIn response to : Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen /lifecycle frozen
Reopened this issue. <detailsIn response to : /reopen /lifecycle frozen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment.

<detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: func initReg() RegistryList { registry := RegistryList{ GcAuthenticatedRegistry: "gcr.io/authenticated-image-pulling", <del> DockerLibraryRegistry:   "docker.io/library", DockerGluster:           "docker.io/gluster", </del> E2eRegistry:             "gcr.io/kubernetes-e2e-test-images", E2eVolumeRegistry:       "gcr.io/kubernetes-e2e-test-images/volume", PromoterE2eRegistry:     "k8s.gcr.io/e2e-test-images",

Part of umbrella to migrate kubernetes e2e test images/registries to community-owned infrastructure: The following images are used by 1.20 e2e tests We should migrate them to community owned infrastructure (ref: for deciding how to implement a mirror registry) We should consider doing this for release branches as well, which may use different images/tags. So these wouldn't necessarily be cherry-picks Two possible approaches: change the registry in source (I would prefer eliminating the extra "docker library" registry, and consolidate to a single e2e test image registry) start running tests with a config that swaps out for some other registry /sig testing /sig release /area release-eng /wg k8s-infra /milestone v1.21
/kind cleanup
/triage accepted /help cc: Give a shout if any of you are interested in working on this!
I'm in! /assign
/remove-help
cc
/cc
I've had 2 PRs opened for this, before it was decited that the test jobs would use the registry mirror configuration. Has there been any consensus on the approach? If so, I can un-abandon these PRs if necessary:
ooh thanks for sharing I don't think there has been any consensus, though I personally think your work on kubernetes/kubernetes is relevant, especially that you've covered everything except gluster and redis.
We already have the redis image here [1], we'll just have to trigger a job for it afterwards. Indeed, I was missing the gluster image, I only considered the image for it.
so I've been thinking about this for a bit and I'm a bit concerned with the visibility of this setup to test authors -- I'm not sure if this is straightforward / intuitive enough for them to follow. alternatively, are there other lighter (in terms of maintenance) solutions that we can pursue?
Hm, what do you mean by the visibility of this setup of test authors? If I'm not mistaken, the current goal is to not rely so heavily on dockerhub for testing anymore. For this purpose we are mirroring these images into our own registries, and then we are going to update to use our own registry instead of the dockerhub one. Test authors shouldn't be impacted by this, IMO. From their point of view, it's the same image, from a different source.

And in terms of maintenance... hm, techically, if someone would want to use an newer image, say , so it would be consistent.
yes, I think we're on the same page on this yeah what I'm wondering is: are there ways we make it even less significant work on the first step of modifying and files? if that step could disappear altogether, especially in scenario of adding new images, that would be great.
I don't really see how. Other images are doing something similar too. For example, the pause image also has its BASEIMAGEs and VERSION in its Makefile. Not all images are built exactly the same, or have the same base image, or are built for the same arch types. Some of them are from scratch, others are from an alpine image, others are from a different alpine image, others are from a debian image... and then we also have windows base images. So, having those BASEIMAGE files addresses this issue. Plus, it also allows us to control what platforms we build for and what we test for. For example, the s390x support wasn't there from the start, it was afterwards. Another example, we want to add the base images at some point. Anyways, modifying the BASEIMAGE files is not very common, but when it is, there's a good reason behind it, so it can't really be optional. Plus, adding / modifying the BASEIMAGE and VERSION files is not significant work. For example, mirroring the 7 images in this PR [1] probably took me ~10 minutes or so. From what I can tell, you want some fast way to just say: "mirror me this image entirely onto our registry". I don't know if we have something like that, and if we do, it'll probably also mirror images with arch types we don't need, and most likely won't have images for platforms we actually need (for example, the ` images on dockerhub does not have any Windows images, but we need them). The test images building process for the test images was not designed for mirroring images, but it can easily do it and we can use it to do so. In addition to that, we can even extend those manifest lists with Windows support [2]. [1] [2]
oh right, apologies for the confusion.

I was referring to having a Dockerfile as a whole, but your explanation is very helpful for me to gain more context about image usage in our infrastructure -- thank you for elaborating I agree the what isn't significant, but the where worries me a bit. I guess in some ways I kind of wish that there is way for this work to simply exist in without touching k/k proper, though now I'm not too sure if that would be overall less complex than this approach. yeah, I was trying to find more solutions towards this direction though I haven't found anything significant which aligns with this topic.
/priority important-soon
In order to complete this, we are going to need these 2 trivial PRs: Retrigger image builder jobs for some images: Small fix for Windows httpd images:
seems like much of the work has been done by Claudiu or I do not have access to finish -- I'll unassign myself to keep this aligned with the state of things /unassign
/assign who has done a lot of the work for this


Question: "sort" "k8s.io/kubernetes/pkg/api/meta" <ins> "k8s.io/kubernetes/pkg/api/unversioned" </ins> "k8s.io/kubernetes/pkg/api/v1" "k8s.io/kubernetes/pkg/runtime" <ins> "k8s.io/kubernetes/pkg/util/integer" </ins> "k8s.io/kubernetes/pkg/util/jsonpath" "github.com/golang/glog"

I'd like to be able to sort by a timestamp, e.g. but that fails with an exception: I appreciate that the root of the problem is probably outside of kubernetes, yet I hope you'd agree that sorting by a timestamp is very useful functionality.
Agree with sorting by timestamp.. Just make it clear, is of type: while only these types are allowed to be sorted:


Question: metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/runtime/serializer" <ins> "k8s.io/apimachinery/pkg/runtime/serializer/json" </ins> utilruntime "k8s.io/apimachinery/pkg/util/runtime" "k8s.io/apimachinery/pkg/util/uuid" "k8s.io/apimachinery/pkg/util/wait"

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: We face But there is no way to generate default config file How to reproduce it (as minimally and precisely as possible): Environment: Kubernetes version 1.9.2
I'll take a look it /sig scheduling
/assign
Hi, same issue with controller-manager
Any updates here? , and don't have this flag. Is it even will be supported?
/cc
if not set, component should be use default value; why we'd like to dump all default config?
Because the documentation on the config file format is either nonexistent or difficult to find. If the program itself can generate a template, that fulfills (most of) the need for documentation of the format.
+1 for this, from my expirence, add an option to write the default config is really helpful.
/assign
also in 1.10
Deprecating all flags, prompting folks to use config files in advance is great, except when there's no documentation nor example anywhere. That means that we can't actually migrate to the new settings, and I can't seem to find clear alpha/beta/stable status for the new config file spec, nor timeline for the removal. It is also pretty confusing that the scheduler expects us to use a config file, but the controller-manager does not?
This issue indeed applies to kube-controller-manager. And there should already had deprecation policy.
According to , the flag will be available in . Since I can't find any reference online to an example config file, perhaps one could be provided or linked to here?
will you be able to pick my patch and generate a config file? I do not get my devbox handy so can only post here until Monday.


Question: Volumes       []Volume      `json:"volumes"` Containers    []Container   `json:"containers"` RestartPolicy RestartPolicy `json:"restartPolicy,omitempty"` <del> // Optional: Set DNS policy.  Defaults to "ClusterFirst" </del> <ins> // Required: Set DNS policy. </ins> DNSPolicy DNSPolicy `json:"dnsPolicy,omitempty"` // NodeSelector is a selector which must be true for the pod to fit on a node NodeSelector map[string]string `json:"nodeSelector,omitempty"`

Many comments in there say "if not specified, default is ..." - but we previously asserted that there are no defaults for the internal API. We should strip those comments out, and leave them in versioned structs only.
cc


Question: } } <del> // CreateServicePods creates a replication controller with the label same as service </del> <ins> // CreateServicePods creates a replication controller with the label same as service. Service listens to HTTP. </ins> func (j *TestJig) CreateServicePods(c clientset.Interface, ns string, replica int) { config := testutils.RCConfig{ Client:       c,

<!-- Please only use this template for submitting enhancement requests --What would you like to be : Following are two e2e testing service functioning over same port but different protocol, but do not test service reachability. be able to update NodePorts with two same port numbers but different protocols use same NodePort with same port but different protocols Note: Both the e2e are verifying the same behavior of "service using same port but different protocols" differs in and . Primary goal: Test service reachability for these E2Es. Secondary goal: As both the above e2e are verifying same E2E behavior, it's good to have single e2e in spite of 2 differing in "updating" and "create only" nodePort service. For "create only", suite has another specific e2e . Why is this needed: E2E Improvement and make e2e eligible for Conformance. Service related e2e are getting promoted in /area conformance /sig testing /assign


Question: writeLine(proxier.natRules, append(args, "-j", string(KubeMarkMasqChain))...) } <ins> // Accept all traffic with destination of ipvs virtual service, in case other iptables rules // block the traffic, that may result in ipvs rules invalid. // Those rules must be in the end of KUBE-SERVICE chain proxier.acceptIPVSTraffic() </ins> // If the masqueradeMark has been added then we want to forward that same // traffic, this allows NodePort traffic to be forwarded even if the default // FORWARD policy is not accept.

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: In a k8s cluster with IPVS proxy mode, when you create an ingress, the loadbalancer of this ingress can't be createautomatically, which caused the ipvs ci "gci-gce-ipvs" failed, link: http://k8s- The failed case is , with logs: Time out when waiting for for ingress to get an address. I'm not sure why, but I can see the pod keep restarting, And with container logs: AFAIK, ingress IP have nothing to do with kube-proxy, but when I change proxy mode to iptables, then ingress got it's IP in few seconds, though still in status. Is there any conflict between ipvs proxy rules and ingress controller? Maybe have this possibility. What you expected to happen: Ingress can got it's IP in ipvs proxy mode. How to reproduce it (as minimally and precisely as possible): a cluster in gce environment with proxy mode IPVS. . ingress with backends. for the ingress IP. TT Anything else we need to know?: Environment: Kubernetes version (use ): master Cloud provider or hardware configuration: gce OS (e.g. from /etc/os-release): Ubuntu16.04.3 Kernel (e.g. ): 4.13.0-1008-gcp -Ubuntu SMP Thu Jan 25 11:08:44 UTC 2018 x8664 x8664 x8664 GNU/Linux Install tools: cluster/kube-up Others:
/sig network /area ipvs
I found this maybe related with iptables rules , ingress-controller need to validate the default service by access . But the iptables like: That means the access of port will be directly DNAT to nginx-ingress-controller. In iptables proxy mode, DNAT will be done in chain, so the access of still functional. Thoughts?
I think the problem here is that the ingress controller is crash looping. You are correct that kube-proxy has nothing to do with the LB getting an ip address so I think we can rule out iptables / ipvs as the issue. I'm wondering why the nginx ingress controller in the test does not have permission to list objects in kube-system? Let me try reproducing and see what I find.

When you switched back to iptables mode, you say the container was still crashlooping. Was the error the same?
My mistake, actually the error log I pasted before is in iptables proxy mode, ipvs mode ingress-controller log are: I didn't notice the difference before. And now I'm sure the is created by kubenet, in host port map workaround. Problem is the access of is treated as , so match this rule:
So I'm confused. Why don't we see this problem in iptables mode then? What is the fix on the IPVS side?
I think that's because in iptables proxy mode, we don DNAT in , so traffic of service will never come to . Besides, iptables will match all virtual server IP in ipvs rules, make things even worse.
I think they are same issue.
/assign
Still get this issue with kubernetes 1.11.0. See below iptable rules in my host,