Question: if TestContext.DeleteNamespace && (TestContext.DeleteNamespaceOnFailure || !CurrentGinkgoTestDescription().Failed) { for _, ns := range f.namespacesToDelete { By(fmt.Sprintf("Destroying namespace %q for this suite.", ns.Name)) <del> timeout := 5 * time.Minute </del> <ins> timeout := DefaultNamespaceDeletionTimeout </ins> if f.NamespaceDeletionTimeout != 0 { timeout = f.NamespaceDeletionTimeout }

/assign The test flakes presubmit testing in PRs six times per week The site can focus on just this test: Which shows that 83 out of 85 errors match this pattern:
Previous attempt to address this flake:
CC
Took a quick look at a failed test: https://k8s- The indicates: . or and can you help investigate.
CC
The failure in the GKE multizone test suite is probably due to the GKE issue I fixed last week.
We'll dig into this and see what's up.
Expectations: This work should interrupt and take priority over other activity while it is assigned to you. Expectation for sig leads is to identify someone to fix the issue, them and yourself. Expectation for assignees are to investigate the issue and either a) resolve it or b) it to a more appropriate person and b) yourself.
I repeatedly ran this test on my e2e cluster since 7pm last night, and it has not failed yet. Seems like it fails more frequently in the PR env instead of the CI env, so may need to look into differences between the two. The test is failing during the deletion of the namespace. It seems like there are still some resources not getting deleted. I'm trying to dig deeper in the logs to see if I can find what resources are not getting cleaned up.
Here's one failed : From the API server initial namespace deletion request from the e2e test: I0427 23:23:29. 5 ] DELETE /api/v1/namespaces/e2e-tests-pv-cx3bh: (4.744775ms) 200 [[ (linux/amd64) kubernetes/] 104.197.101.155:] controller triggers deletion on all the objects in the namespace: I0427 23:24:37. 5 ] DELETE /apis/apps/v1beta1/namespaces/e2e-tests-pv-cx3bh/statefulsets: (91.176059ms) 200 [[kube-controller-manager/v1.7.0 (linux/amd64) kubernetes//system:serviceaccount:kube-system:namespace-controller] [::1]:] I0427 23:24:37.

5 ] DELETE /api/v1/namespaces/e2e-tests-pv-cx3bh/resourcequotas: (7.608267ms) 200 [[kube-controller-manager/v1.7.0 (linux/amd64) kubernetes//system:serviceaccount:kube-system:namespace-controller] [::1]:] etc ... gets finally deleted by controller I0427 23:26:30. 5 ] DELETE /apis/apps/v1beta1/namespaces/e2e-tests-pv-cx3bh/statefulsets: (87.184389ms) 200 [[kube-controller-manager/v1.7.0 (linux/amd64) kubernetes//system:serv iceaccount:kube-system:namespace-controller] [::1]:] I0427 23:26:30. 5 ] DELETE /api/v1/namespaces/e2e-tests-pv-cx3bh/resourcequotas: (21.517822ms) 200 [[kube-controller-manager/v1.7.0 (linux/amd64) kubernetes//system:serviceaccoun t:kube-system:namespace-controller] [::1]:] etc ... deletion finished I0427 23:26:31. 5 ] DELETE /api/v1/namespaces/e2e-tests-pv-cx3bh: (16.58598ms) 200 [[kube-controller-manager/v1.7.0 (linux/amd64) kubernetes//system:serviceaccount:kube-system:namespace-controller] [::1]:] 2E test cleanup tries to delete it again: I0427 23:26:31. 5 ] DELETE /api/v1/namespaces/e2e-tests-pv-cx3bh: (9.104808ms) 404 [[ (linux/amd64) kubernetes/] 104.197.101.155:] <= This is the e2e test failure cleanup Total time to delete 3 mins Here's a successful : : deletion request I0428 00:45:14. 5 ] DELETE /api/v1/namespaces/e2e-tests-pv-c4t8j: (11.208829ms) 200 [[ (linux/amd64) kubernetes/] 104.197.153.246:] starts object deletions I0428 00:45:46.

5 ] DELETE /apis/apps/v1beta1/namespaces/e2e-tests-pv-c4t8j/deployments: (33.104152ms) 200 [[kube-controller-manager/v1.7.0 (linux/amd64) kubernetes//system:servi ceaccount:kube-system:namespace-controller] [::1]:] I0428 00:45:46. 5 ] DELETE /api/v1/namespaces/e2e-tests-pv-c4t8j/podtemplates: (3.447937ms) 200 [[kube-controller-manager/v1.7.0 (linux/amd64) kubernetes//system:serviceaccount:k ube-system:namespace-controller] [::1]:] etc ... gets finally deleted by controller I0428 00:46:17. 5 ] DELETE /api/v1/namespaces/e2e-tests-pv-c4t8j/podtemplates: (10.679029ms) 200 [[kube-controller-manager/v1.7.0 (linux/amd64) kubernetes//system:serviceaccount: kube-system:namespace-controller] [::1]:] I0428 00:46:20. 5 ] DELETE /apis/extensions/v1beta1/namespaces/e2e-tests-pv-c4t8j/deployments: (6.130617ms) 200 [[kube-controller-manager/v1.7.0 (linux/amd64) kubernetes//system: serviceaccount:kube-system:namespace-controller] [::1]:] etc ... deletion is finished I0428 00:46:21. 5 ] DELETE /api/v1/namespaces/e2e-tests-pv-c4t8j: (21.10359ms) 200 [[kube-controller-manager/v1.7.0 (linux/amd64) kubernetes//system:serviceaccount:kube-system:na mespace-controller] [::1]:] 2E test cleanup I0428 00:46:32.

5 ] DELETE /api/v1/namespaces/e2e-tests-pv-c4t8j: (1.436939ms) 404 [[ (linux/amd64) kubernetes/] 104.197.153.246:] Total time to delete ~1 minute Comparing the bad and good runs, these were some supicious areas: the bad run, the period between 1) and 2) was 1 minute, whereas in the good run it was 30 seconds the bad run, the period between 2) and 3) was ~2 minutes, whereas in the good run it was ~20 seconds. The slowness in the bad run seems systematic, I did not find any one object that was taking a particularly long time to delete, all of them seemed to be taking longer compared to the good run. I'm not sure where to dig down further to find the cause of the general slowness. I don't know if this is expected, and we should just increase the timeout? The test framework seems to have a default timeout of 5 minutes, whereas this volume test timeout is only 3 minutes.
Great analysis! Since no singular event stands out as taking too much time how about, in the name of expediency, just changing the timeout to 5m and see how the tests behave?
The delay of ns deleteion might related to this PR and . is still open.
Those two issues do not seem to impact the deletion of the namespace itself, so I don't think fixing those will solve this issue.
But I think delaying the deletion of a pod will affect the time to delete the namespace.


Question: require ( github.com/gogo/protobuf v1.3.2 google.golang.org/grpc v1.56.3 <del> k8s.io/apimachinery v0.0.0 </del> ) require ( <del> github.com/go-logr/logr v1.2.4 // indirect </del> github.com/golang/protobuf v1.5.3 // indirect golang.org/x/net v0.17.0 // indirect golang.org/x/sys v0.13.0 // indirect golang.org/x/text v0.13.0 // indirect google.golang.org/genproto/googleapis/rpc v0.0.0-20230525234030-28d5490b6b19 // indirect google.golang.org/protobuf v1.31.0 // indirect <del> k8s.io/klog/v2 v2.100.1 // indirect k8s.io/utils v0.0.0-20230726121419-3b25d923346b // indirect </del> ) <del> replace ( k8s.io/apimachinery => ../apimachinery k8s.io/kms => ../kms ) </del> <ins> replace k8s.io/kms => ../kms </ins>

Drop the and dependencies from to keep the import lighter. /sig auth /triage accepted
/assign Hi, , I created a PR to fix this. I am not sure If I am correct due to the repo seems generated. Would you like to provides some hints for me?
This will likely require a new go module in to move the client-go dep and then mark it as internal, ex: . is the only API machinery dep so we can probably come up with some way to re-implement the wait logic locally? Then we drop these lines so that these imports do not come back again:
Whoa, yeah, I'd take a for loop and a time.Sleep in a test to drop those deps
What is kms using client-go for? Tests only?
These were all needed by the KMS repo back when we had a much more complicated reference implementation which required a key hierarchy. Now they are just used in some tests and mock implementations, so we should be able to simplify things.
I found a place for rate limit. #liggitt
that package just delegates to "", switch that mock provider to use that directly?


Question: }, Provisioner: "nondefault1", } <ins> classWithCreateTime1 := &storagev1.StorageClass{ TypeMeta: metav1.TypeMeta{ Kind: "StorageClass", }, ObjectMeta: metav1.ObjectMeta{ Name:              "default1", CreationTimestamp: metav1.NewTime(time.Date(2022, time.Month(1), 1, 0, 0, 0, 1, time.UTC)), Annotations: map[string]string{ storageutil.IsDefaultStorageClassAnnotation: "true", }, }, } classWithCreateTime2 := &storagev1.StorageClass{ TypeMeta: metav1.TypeMeta{ Kind: "StorageClass", }, ObjectMeta: metav1.ObjectMeta{ Name:              "default2", CreationTimestamp: metav1.NewTime(time.Date(2022, time.Month(1), 1, 0, 0, 0, 0, time.UTC)), Annotations: map[string]string{ storageutil.IsDefaultStorageClassAnnotation: "true", }, }, } </ins> claimWithClass := &api.PersistentVolumeClaim{ TypeMeta: metav1.TypeMeta{

Presumably this would occur when changing the default from one class to another. E.g. Class X exists and is marked default Class Y is created Class Y is marked as default Uh oh!! 2 defaults Class X is marked as not default At the case in point, when there are 2 defaults, we currently do not assign default values to new PVCs. We should change this semantic - while there are two, pick one randomly or heuristically. It's still correct. Example: Suppose that you have one user changing the default and another user creating a PVC at exactly the same time. What happens? Depending on network, CPU scheduling, memory allocations, (etc - totally non-deterministic) the API server might see either of those operations first. The PVC creator will either win the race and get the old default class or it will lose the race and get the new default class. The difference could literally be whose network has a faster path to the API server. My point is that whichever one you pick, as long as it is one of those two defaults, it is a correct choice! Any other FooClass things we do should probably change, too. I think that's IngressClass and RuntimeClass.
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign
/assign
GitHub didn't allow me to assign the following users: Octopusjust. Note that only , repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : /assign Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
picking either sounds reasonable but what if we allow only one default to be set and disallow/warn users if they try to create another default.

I can think of issues with this too, mainly when switching over to a new default by removing an existing class A and then setting B as a default but managing that orphaned period while the transition happens, maybe we can have the heuristic there? I'm sure I'm missing something here.
/assign
GitHub didn't allow me to assign the following users: Octopusjust. Note that only , repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : /assign Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
There is a problem when there is no default storage class for a while and another user creates a PVC. Such PVC won't get the new SC. We're trying to fix it in Until then, having two default storage classes for a brief moment is IMO slightly better, because at least users get an error message and can retry PVC creation. In addition, (in-tree) admission plugin in HA environment could be racy, as different API servers are admitting updates of different StorageClasses in parallel.
/assign
I wouldn't make 2 an error - just pick one.
Ok, I'd take a shot at this. Thanks for clearing things up.
+1 Still, having two default classes for a longer time is IMO bad, there should be a metric + alert if there are 2 or more default SCs for X minutes.
it's not ideal - a metric seems OK
hello there, im new to open source. i don't know how to contribute. can anyone hep me please?
I see this issue was labeled as "good first issue" and I would like to start working on it!
I see there is a pr related to this issue , are you still working on this?
Is this issue still open?
/assign
Yes, there's already a . It's linked to the issue itself.
The PR is specifically for StorageClass, but we also need to audit other FooClass things - they should probably change, too. I think that's IngressClass and RuntimeClass. Someone can open new PRs for these.
Oh, I see the second PR, too.

Will look at them ASAP
/remove-sig network
Do we emit a response header when someone makes a request that ends up setting two IngressClasses as default?
that seems a nice thing to do, I didn't know you can add warnings on admissions plugins but seems implemented it here
Is this issue still open??
StorageClass implementation is in I am not sure about those. I'd suggest opening separate issues. /close
Closing this issue. <detailsIn response to : whichever one you pick, as long as it is one of those two defaults, it is a correct choice! </details


Question: "extender_test.go", "framework_test.go", "main_test.go", <ins> "plugins_test.go", </ins> "predicates_test.go", "preemption_test.go", "priorities_test.go",

What would you like to be : An integration test for NodeResourceLimits plugin. Why is this needed: No end-to-end test coverage current exist for this alpha feature. /sig scheduling /help
/assign /remove-help


Question: subnetID:         aws.StringValue(instance.SubnetId), } <del> // We lazy-init deviceMappings self.deviceMappings = nil </del> <ins> self.attaching = make(map[mountDevice]string) </ins> return self }

Environment: CoreOS 815.0.0, k8s 1.1.0 () It happens to me quite often that a volume marked as "available" on AWS remains in a "detached" state according to the kubelet: Deleting the pod and recreating it eventually succeeds.
I have some fixes in progress that should fix this
/cc
I can reliably reproduce similar issue, I am not sure it's the same: create a AWS EBS, let's say it has ID Create a pod that uses this EBS: Create the pod. It succeeds eventually: Delete the pod Create the pod again. It will be forever IMO, it's caused by not removing the detached volume from internal map of attached volumes . Subsequent will find it there and it won't even try to attach it again. Patch is available in referenced PR.
can you please confirm that your bugs is the same as I can reproduce? Or do you see different bug, only with similar symptoms?
It is
Was this fixed by ?
I think you referenced the wrong issue / PR, didn't you? As far as I know this is not fixed yet.
Yes, I meant to link . mentioned there that it might fix this issue.
Oh right! It made it to v1.2.0-alpha.4
did not fix this issue (but fixed the DeviceMapping cache invalidation)
I think, what is happening is that you run into an error or timeout while waiting for the disk operation. That leaves the devicecache yet again in an inconsistent state. I have a fix here that just discards the cache on any error. Additionally, it is totally possible that there is concurrent operations which lead to an error and bork the cache. The fix: Unfortunately, this is in our tests still not 100% rock solid. Especially in timeout cases. I'm missing the capability to properly test this to engage in a bigger refactoring of this whole issue. A workaround to actually fixing it properly would be to add a TTL to the cache, so that it eventually starts working again.
Just for information for people who might use the GitHub search, as of Kubernetes 1.1.7 the errors logged by the look like: Deleting the pod or restarting doesn't help, the instance keeps failing attaching the EBS with the same errors if the pod get rescheduled on the same host.

thanks for the fix!
Oh... the same host. I bet I messed up that case... I'm going to move into 1.2 so I take another look at this!
What's the status on this?
I just don't think this is fixable with the volume cache we have. I proposed a PR to remove the cache, and replace it with a map of active attachment operations in progress. A little less efficient, but a lot more correct (presuming passes testing).
How to check what K8 version has this fix?
Is there any progress on this issue? I'm running K8s version v1.5.2, and I'm having similar issues. awsElasticBlockStore volumes are virtually unusable due to one of three errors I'm getting: My volume claim is ReadWriteOnce, so this next error confuses me: And finally: [SchedulerPredicates failed due to persistentvolumeclaims "mysql" not found, which is unexpected., SchedulerPredicates failed due to persistentvolumeclaims "mysql" not found, which is unexpected., SchedulerPredicates failed due to persistentvolumeclaims "mysql" not found, which is unexpected.] Unable to mount volumes for pod "staging-connect-mysql-<id(<id)": timeout expired waiting for volumes to attach/mount for pod "default"/"<id". list of unattached/unmounted volumes=[mysql] Error syncing pod, skipping: timeout expired waiting for volumes to attach/mount for pod "default"/"<id". list of unattached/unmounted volumes=[mysql]
Also having the same issue as ^ Should we be using PV+VPC instead of claiming the EBS volume directly from the deployment spec?


Question: Dump-DebugInfoToConsole Set-PrerequisiteOptions $kube_env = Fetch-KubeEnv <del> Disable-WindowsDefender </del> if (Test-IsTestCluster $kube_env) { Log-Output 'Test cluster detected, installing OpenSSH.'

Two weeks ago we were observing Windows nodes crashing and rebooting during our e2e test runs on GCE (). Analyzing the crash dumps showed a fault in the NTFS driver while the process was running. appears to be part of the Windows Defender anti-malware component. To address the crashes we did two things: 1) disabled Windows Defender on our Windows e2e test nodes (), and 2) included February's Windows updates on our Windows nodes. After these two things the crashes seem to have stopped. Two days ago we re-enabled Windows Defender on our Windows nodes: Unfortunately we started seeing the Windows node crashes again, starting with run 228 on (To determine if a node crash occurred we open the results link for the test run, open the artifacts for each Windows node and look for a file (e.g. )). We analyzed one of these crash dump files and it looked the same as what we saw a couple weeks ago: fault in NTFS while was running. For now we've merged we re-disables Windows Defender on our e2e Windows nodes. Since that PR merged early this morning (starting with run 246) none of our Windows nodes have crashed, strongly implicating Windows Defender as contributing to the crashes. Since the failure is happening in the NTFS driver we'll need help from Microsoft folks to debug it. The memory dump files are publicly available from the test artifacts (runs 228 - 245), and I've copied a WinDbg crash analysis below. Additionally, some Windows version information is available in the serial-1.log file for each Windows node, also copied below. I don't have any great ideas about why the combination of K8s e2e tests and Windows Defender seems to cause the crash; we don't have any other reports of this failure happening in Windows VMs on GCE. Version info: Analysis:
/assign pjh cc is there anyone on your side who can look at this? It's fairly easily reproducible in our e2e test environment. We're working around it for now by disabling Windows Defender; I think we're ok with this for now, but it would be good to find the cause.
/sig windows Not blocking for 1.14.

/cc /cc in case you've seen the same crash
did do some deeper analysis of one of the crash dumps early last week and found this which might be helpful:
cc as well
I fwd'ed the dmp internally
Thanks! also passed this along internally. I'll leave this issue assigned to me while awaiting updates from Microsoft.
The folks at Microsoft root-caused this issue and we've verified a fix for it. The fix will make it into Windows Server 2019 at some point, not sure about the timing for SAC releases.
/assign craiglpeters For tracking.
I believe this issue should be fixed via KB4501371, can you verify once it becomes available?
/assign pjh Yes, I'll verify the fix when that KB is available (end of June).
There's an optional fix out: It will be included in the July cumulative update too


Question: return "", err } if _, err := os.Stat(filepath.Join(*k8sBinDir, bin)); err != nil { <del> return "", fmt.Errorf("Could not find kube-apiserver under directory %s.", absPath) </del> <ins> return "", fmt.Errorf("Could not find %s under directory %s.", bin, absPath) </ins> } return filepath.Join(absPath, bin), nil }

All the LoadBalancers tests It seems that for a long time IPVS and iptable implementations differ, it may be possible the test do some wrong assumptions? At this point is up to the IPVS maintainers if they want to run those tests or not, is ok if they don't have time to investigate to disable the tests, but we should not have flaky test running or we'll have a lot of noise /sig network /area ipvs /help
/sig network /area ipvs /help
This request has been marked as needing help from a contributor. Please ensure that the issue body includes answers to the following questions: Why are we solving this issue? To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points? Does this issue have zero to low barrier of entry? How can the assignee reach out to you for help? For more details on the requirements of such an issue, please see and ensure that they are met. If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : /sig network /area ipvs /help Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: allErrs = append(allErrs, cErrs.PrefixIndex(i)...) } // Check for colliding ports across all containers. <del> // TODO(thockin): This really is dependent on the network config of the host (IP per pod?) // and the config of the new manifest.  But we have not specced that out yet, so we'll just // make some assumptions for now.  As of now, pods share a network namespace, which means that // every Port.HostPort across the whole pod must be unique. </del> allErrs = append(allErrs, checkHostPortConflicts(containers)...) return allErrs

Pods which use to bind both TCP and UDP sockets to the same port number are rejected with the error: An example pod follows.
Looks like we don't check protocol
The issue was discussed before in , which was closed due to it being low priority.
Hmm, that's 2 pods and the scheduler, this is 2 containers in the same pod and validation at create time. Maybe we should just be more clear about where we stand with hostPort.
Posting here what I posted in the other bug: This can't be worked around when running NFS from a pod. Linux does not allow the NFS port (2049) to be changed, and the Portmap port (111) also cannot be changed. Additionally, you can't separate the UDP from the TCP ports.
Why does NFS need host ports? , Russell Harmon wrote:
Fix is in , Tim Hockin wrote:


Question: } func getFinishTimeFromContainers(p *v1.Pod) *time.Time { <del> var finishTime *time.Time for _, containerState := range p.Status.ContainerStatuses { if containerState.State.Terminated == nil { return nil </del> <ins> finishTime := latestFinishTime(nil, p.Status.ContainerStatuses, nil) if utilfeature.DefaultFeatureGate.Enabled(features.SidecarContainers) { // We need to check InitContainerStatuses here also, // because with the sidecar (restartable init) containers, // sidecar containers will always finish later than regular containers. names := sets.New[string]() for _, c := range p.Spec.InitContainers { if c.RestartPolicy != nil && *c.RestartPolicy == v1.ContainerRestartPolicyAlways { names.Insert(c.Name) } } finishTime = latestFinishTime(finishTime, p.Status.InitContainerStatuses, func(status v1.ContainerStatus) bool { return names.Has(status.Name) }) } return finishTime } func latestFinishTime(prevFinishTime *time.Time, cs []v1.ContainerStatus, check func(status v1.ContainerStatus) bool) *time.Time { var finishTime = prevFinishTime for _, containerState := range cs { if check != nil && !check(containerState) { continue </ins> } <del> if containerState.State.Terminated.FinishedAt.Time.IsZero() { </del> <ins> if containerState.State.Terminated == nil || containerState.State.Terminated.FinishedAt.Time.IsZero() { </ins> return nil } if finishTime == nil || finishTime.Before(containerState.State.Terminated.FinishedAt.Time) {

Today the function () only account for the regular containers finish time, presumably assuming that init containers have completed before before. However, with the sidecar (restartable init) containers, sidecar containers will always finish later than regular containers. And those needs to be accounted for the calculation. /kind bug The sidecar finish time will be accounted when calculating the job's finish time. Sidecar container with prolonged termination logic on a job will result in incorrect finish time reporting. /sig apps /priority backlog KEP:
/assign
/triage accepted


Question: maxRetries: maxRetries, } if cl != nil && cl.Core().RESTClient().GetRateLimiter() != nil { <del> metrics.RegisterMetricAndTrackRateLimiterUsage("serviceaccount_controller", cl.Core().RESTClient().GetRateLimiter()) </del> <ins> metrics.RegisterMetricAndTrackRateLimiterUsage("serviceaccount_tokens_controller", cl.Core().RESTClient().GetRateLimiter()) </ins> } e.serviceAccounts = serviceAccounts.Lister()

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: After start kubernetes controller manager, I get the following error: /--------------------- E0726 11:55:39. 1 ] Metric for serviceaccount_controller already registered /--------------------- What you expected to happen: successed to start kube-controller-manager service without error. How to reproduce it (as minimally and precisely as possible): every time. Anything else we need to know?: Environment: Kubernetes version (use ): v1.6.4 Cloud provider or hardware configuration**: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others:
There are no sig labels on this issue. Please by: a sig: e.g., to notify the contributor experience sig, OR the label manually: e.g., to apply the label Note: Method 1 will trigger an email to the group. You can find the group list and label list . The in the method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals
/kind support
fix it, close it.


Question: "math" "math/rand" "path/filepath" <ins> "strconv" </ins> "strings" "time"

From here we see that the method takes 512 bytes sector size as a default option for verifying block read. This makes the test fail for the block devices that uses 4k sector size because the read is using . However, with modern kernels the area being operated on, must be aligned to and its size must be a multiple of the disk's block size. So using dd iflag=direct with size 512B on a 4k sector size device will fail. This is introduced in this PR: There should be proper handling of different block sector size for the test. Either a customizable parameter or something. Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others: /cc /sig storage
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
cc
Since we are using Debian image for this test. I am wondering does it make sense to use instead of for the dd read? I guess my question is if we use , while it read after the cache is dropped or before?
Or I am wondering can we just change the size to 4096. A read/write to the multiply of the sector size should be okay I guess?
I am not sure that option is available now, notes from September last year () indicate I tried that as well. I think you still need to write blocks of 512 or 4096 bytes, not sure if (or the kernel) is smart enough to split larger requests into smaller blocks. has a pointer to detect the blocksize dynamically, which would be a cleaner way and more future proof.
I am not sure when you mention the "option is not available" you mean for busybox or debian. I think for this specific test we are using Debian image which has this option.

Yes, this is what we use in the test and we found the test fail because in our case shows 4096 instead of 512
The option () used to be only applicable for (in Debian and CentOS) with , not with . I do not know if the Debian image was updated, and if it contains a new that supports the option.
+1 for using 4096 bytes. Or even more, 1MiB? Kernel/dd is smart and it will handle sequence of well.
I think that it's OK to just use large block size if there's no issue by doing so. However, I've made a PoC implementation of auto detect logic of sector size to compare alternative ideas. One issue is that current debian-iptables image that is used for this test doesn't contain blockdev command. In the PoC code, it is installed on execution time, however it should be included in the image, by either (1) adding it to debian-iptables image or (2) creating a new image that include it. If the issue can be resolved easily, this alternative approach will also be good.
I found that image contains both command and command and can be used for this test. I think that this additional detection logic is simple enough, so I made a PR above. Could you review it?


Question:
0cf8789f-00da-4da0-976a-b6b1dc831159 ``` <del> Add an index.html inside this directory and use `curl` to see this HTML file served up by nginx. </del> <ins> You can also see the mountpoint by inspecting the docker container on that host. </ins> ```sh  $ docker inspect -f "{{.Mounts}}" <container-id> | grep flocker ...{ /flocker/0cf8789f-00da-4da0-976a-b6b1dc831159 /usr/share/nginx/html true} ``` Add an index.html inside this directory and use `curl` to see this HTML file served up by nginx. ```sh $ echo "
Hello, World
" | tee /flocker/0cf8789f-00da-4da0-976a-b6b1dc831159/index.html  $ curl ip ```  ### More Info Read more about the [Flocker Cluster Architecture](https://docs.clusterhq.com/en/latest/concepts/architecture.html) and learn more about Flocker by visiting the [Flocker Documentation](https://docs.clusterhq.com/). #### Video Demo To see a demo example of using Kubernetes and Flocker, visit [Flocker's blog post on High Availability with Kubernetes and Flocker](https://clusterhq.com/2015/12/22/ha-demo-kubernetes-flocker/)   [![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/examples/flocker/README.md?pixel)]()

We found that some users coming to the document were getting a little confused. It should include more detail about where to install Flocker services, how to set environment variables, that you can create volumes now with Docker CLI and give more detail in example including how to use it with a spec that includes a replication controller.


Question: Phase: api.NamespaceActive, }, }, true}, <ins> // Cannot set deletionTimestamp via status update </ins> {api.Namespace{ ObjectMeta: api.ObjectMeta{ Name: "foo"}},

Honestly I'm not sure if this is a bug or a feature, but didn't expect it. Filing this to track.
I think it's a bug. Nothing in the code base depends on it that I can see (edit: namespace deletion does, but only via an in-server call which I'd like to see fixed), and we don't let other delete-related fields mutate during update (deleteGracePeriodSeconds)


Question: // Checks if a node has sufficient resources, such as cpu, memory, gpu, opaque int resources etc to run a pod. // It returns a list of insufficient resources, if empty, then the node has all the resources requested by the pod. func (f *Fit) Filter(ctx context.Context, cycleState *framework.CycleState, pod *v1.Pod, nodeInfo *framework.NodeInfo) *framework.Status { <del> if !f.enableSidecarContainers && hasRestartableInitContainer(pod) { // Scheduler will calculate resources usage for a Pod containing // restartable init containers that will be equal or more than kubelet will // require to run the Pod. So there will be no overbooking. However, to // avoid the inconsistency in resource calculation between the scheduler // and the older (before v1.28) kubelet, make the Pod unschedulable. return framework.NewStatus(framework.UnschedulableAndUnresolvable, "Pod has a restartable init container and the SidecarContainers feature is disabled") } </del> s, err := getPreFilterState(cycleState) if err != nil { return framework.AsStatus(err)

/kind feature /sig scheduling Filter will return for all nodes when is and is . This can be done is PreFilter. Executing this code in the filter will cause it to be repeated tens of times, with the worst-case scenario being repeated thousands of times. Maybe we should move this to PreFilter like:
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Sounds reasonable to me :) I think you can go ahead and file a pr for this.
/assign
Sure.
Hi, If you don't mind, I can try to do this.
/assign


Question: options := api.ListOptions{LabelSelector: selector} rss, err := c.Extensions().ReplicaSets(ns).List(options) Expect(err).NotTo(HaveOccurred()) <del> Expect(rss.Items).Should(HaveLen(0)) </del> <ins> // RSes may be created by overlapping deployments right after this deployment is deleted, ignore them if len(overlapWith) == 0 { Expect(rss.Items).Should(HaveLen(0)) } else { noOverlapRSes := []extensions.ReplicaSet{} for _, rs := range rss.Items { if !strings.HasPrefix(rs.Name, overlapWith) { noOverlapRSes = append(noOverlapRSes, rs) } } Expect(noOverlapRSes).Should(HaveLen(0)) } </ins> framework.Logf("Ensuring deployment %s's Pods were deleted", deploymentName) var pods *api.PodList if err := wait.PollImmediate(time.Second, timeout, func() (bool, error) {

https://k8s- Failed: [] Deployment overlapping deployment should not fight with each other {Kubernetes e2e suite}
https://k8s- Failed: [] Deployment overlapping deployment should not fight with each other {Kubernetes e2e suite}
https://k8s- Failed: [] Deployment overlapping deployment should not fight with each other {Kubernetes e2e suite} Happened on a presubmit run in .
[FLAKE-PING] This flaky-test issue would love to have more attention.
https://k8s- Failed: [] Deployment overlapping deployment should not fight with each other {Kubernetes e2e suite}
https://k8s- Failed: [] Deployment overlapping deployment should not fight with each other {Kubernetes e2e suite}
Looking
Flakes happened when stopping overlapping deployments at clean up step.
You probably need to move the check before the deletionTimestamp check. Just make sure that observedGeneration is updated for the first case.
Because the deployments are overlapping, when deleting one deployment and determining if its RSes are deleted, there might be a race when the other deployment creates an RS
[FLAKE-PING] This flaky-test issue would love to have more attention.


Question: // PodMetrics sets resource usage metrics of a pod. type PodMetrics struct { metav1.TypeMeta <ins> // Standard object's metadata. // More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata // +optional </ins> metav1.ObjectMeta // The following fields define time interval from which metrics were

no explanation for metadata filed of NodeMetrics/PodMetrics, code: should be like $ kubectl explain NodeMetrics KIND: NodeMetrics VERSION: DESCRIPTION: NodeMetrics sets resource usage metrics of a node. FIELDS: apiVersion <stringAPIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: kind <stringKind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: metadata <Objecttimestamp <string-required- The following fields define time interval from which metrics were collected from the interval [Timestamp-Window, Timestamp]. usage <map[string]string-required- The memory usage is the memory working set. window <string-required- and $ kubectl explain PodMetrics KIND: PodMetrics VERSION: DESCRIPTION: PodMetrics sets resource usage metrics of a pod. FIELDS: apiVersion <stringAPIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: containers <[]Object-required- Metrics for all containers are collected within the same time window. kind <stringKind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: metadata <Objecttimestamp <string-required- The following fields define time interval from which metrics were collected from the interval [Timestamp-Window, Timestamp]. window <string-required- able to see the explanation for metadata filed of NodeMetrics/PodMetrics Kubernetes Version: v1.22.0-rc.0+
/sig instrumentation
/kind documentation /assign /triage accepted


Question: framework.Logf("Waiting for %s to complete.", podClient.Name) err = e2epod.WaitForPodSuccessInNamespace(f.ClientSet, podClient.Name, ns.Name) if err != nil { <del> // Collect pod logs when we see a failure. logs, logErr := e2epod.GetPodLogs(f.ClientSet, f.Namespace.Name, podClient.Name, "client") if logErr != nil { framework.Failf("Error getting container logs: %s", logErr) } // Collect current NetworkPolicies applied in the test namespace. policies, err := f.ClientSet.NetworkingV1().NetworkPolicies(f.Namespace.Name).List(context.TODO(), metav1.ListOptions{}) if err != nil { framework.Logf("error getting current NetworkPolicies for %s namespace: %s", f.Namespace.Name, err) } // Collect the list of pods running in the test namespace. podsInNS, err := e2epod.GetPodsInNamespace(f.ClientSet, f.Namespace.Name, map[string]string{}) if err != nil { framework.Logf("error getting pods for %s namespace: %s", f.Namespace.Name, err) } pods := []string{} for _, p := range podsInNS { pods = append(pods, fmt.Sprintf("Pod: %s, Status: %sn", p.Name, p.Status.String())) } </del> <ins> pods, policies, logs := collectPodsAndNetworkPolicies(f, podClient) </ins> framework.Failf("Pod %s should be able to connect to service %s, but was not able to connect.nPod logs:n%snn Current NetworkPolicies:nt%vnn Pods:nt%vnn", podClient.Name, service.Name, logs, policies.Items, pods) // Dump debug information for the test namespace.

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: The NetworkPolicy e2e test fails regardless of which CNI plugin is enabled. The test assumes the client Pod can recheck connectivity after updating its label, but the client won't restart after the first failure, so the second check will always fail. What you expected to happen: The test should succeed if a CNI plugin implements NetworkPolicy correctly. How to reproduce it (as minimally and precisely as possible): a CNI that supports NetworkPolicy. the e2e test by focusing "should allow ingress access from updated pod". test will always fail. Anything else we need to know?: Environment: Kubernetes version (use ): 16.3 Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig testing /assign
/remove-sig testing /sig network
/triage unresolved Comment when the issue is assessed and confirmed. I am a bot run by vllry.
/remove-triage unresolved
I see this in Calico 3.10 as well confirming that its definitely an issue.


Question: kubectl expose rc my-nginx --port=80 --type=LoadBalancer ``` <del> This should print the service that has been created, and map an external IP address to the service. </del> <ins> This should print the service that has been created, and map an external IP address to the service. Where to find this external IP address will depend on the environment you run in.  For instance, for Google Compute Engine the external IP address is listed as part of the newly created service and can be retrieved by running ```bash kubectl get services ``` In order to access your nginx landing page, you also have to make sure that traffic from external IPs is allowed. Do this by opening a firewall to allow traffic on port 80. </ins> ### Next: Configuration files Most people will eventually want to use declarative configuration files for creating/modifying their applications.  A [simplified introduction](simple-yaml.md)

nginx example missing step to create firewall rule (tcp:80 targettag:kubernetes-minion) from these instructions:


Question: framework.ExpectEqual(retrievedDriver2.UID, createdDriver2.UID) ginkgo.By("listing") <del> driverList, err = client.List(context.TODO(), metav1.ListOptions{}) </del> <ins> driverList, err := client.List(context.TODO(), metav1.ListOptions{LabelSelector: "test=" + f.UniqueName}) </ins> framework.ExpectNoError(err) <del> framework.ExpectEqual(len(driverList.Items), numDriversAfter, "list should have %d items, got: %s", numDriversAfter, driverList) </del> <ins> framework.ExpectEqual(len(driverList.Items), 2, "filtered list should have 2 items, got: %s", driverList) </ins> ginkgo.By("deleting") <del> err = client.Delete(context.TODO(), createdDriver1.Name, metav1.DeleteOptions{}) framework.ExpectNoError(err) err = client.Delete(context.TODO(), createdDriver2.Name, metav1.DeleteOptions{}) framework.ExpectNoError(err) driverList, err = client.List(context.TODO(), metav1.ListOptions{}) framework.ExpectNoError(err) framework.ExpectEqual(len(driverList.Items), numDriversBefore, "list should have %d items, got: %s", numDriversBefore, driverList) </del> <ins> for _, driver := range driverList.Items { err := client.Delete(context.TODO(), driver.Name, metav1.DeleteOptions{}) framework.ExpectNoError(err) retrievedDriver, err := client.Get(context.TODO(), driver.Name, metav1.GetOptions{}) switch { case apierrors.IsNotFound(err): // Okay, normal case. case err != nil: framework.Failf("expected 404, got %#v", err) case retrievedDriver.DeletionTimestamp != nil: // Okay, normal case. default: framework.Failf("CSIDriver should have been deleted or have DeletionTimestamp, but instead got: %s", retrievedDriver) } } </ins> }) // TODO: promote to framework.ConformanceIt

https://k8s- CSIInlineVolumes should support ephemeral VolumeLifecycleMode in CSIDriver API 8/5 https://k8s- No response /sig storage
/triage accepted
/assign


Question: podVolume.Name, format.Pod(pod), err) <ins> allVolumesAdded = false </ins> continue }

Deployment/Service manifest: logs from events: In aws, the volume is attached, On the box: In only one is mounted: Where can I debug this further?
Take a look at the on your master for events related to your volume . That will show you when the volume was attached/detached. It appears that your volume attached without issue so there should be nothing unusual here. Then take a look at on the node that the volume is supposed to be attached to. Looks for the UID of your pod (). I suspect one of your mounts is failing. It looks like you are using hyperkube so it might be a mount namespace propagation issue.
I believe this happens when the RC/Pod is being created shortly after the PV or PVC. The latter two appear to need several seconds to be fully available (even if the status is already set to "Bound"). I ran into the same issue with a NFS PV. When I delete and create PV+PVC+SVC+RC all in one go, the associated Pod will be in ContainerCreating state for several seconds (10,<60), then log the warning event and afterwards be created without any problems or further delays. When I first create PV+PVC, wait for several seconds (30) and only then create SVC+RC, the pod will be created immediately and no warning event will be logged. One oddity I noticed is that the PVC will have capacity 0 and empty access modes (as shown by ), even though it was setup to request ROX for 1Gi and the PV provides 1Gi with ROX. This might be a different bug, though.
Thanks for digging into it The capacity 0 issue () was fixed by PR and is in v1.3.5. This is more or less expected behavior. The mount logic has exponential backoff so if it rapidly encounters PVC/PV does not exist, it'll back off the retries resulting in a longer mount time. We can consider optimizing this particular scenario. But I'll lower the priority on this for now and remove the milestone.
Maybe the "Bound" status could be delayed, so the RC would be held back for a while and be scheduled only when the PVC is actually ready?
I don't think the RC actually waits for PVC to be bound before scheduling.

I have also this problem using deployment with HELM, I deploy all my pv,pvc,deploy,svc in one Charts, and my pod who requiere a volume from the PVC get scheduled and stay in ContainerCreating mode. I need to wait minutes or delete pods for they correctly get rescheduled using the volume. Here is the log when the pod get automaticly deleted (after minutes) : It would be awesome if the deployment or RC wait for the PVC to be bound before schedule !
may slightly improve this situation as the maximum backoff time of 2 minutes will not fire anymore.
Note that the same error is thrown when node-size == C5 or M5 for kubectl and with kops . Solution is to set node as t2 machines. Due to it seems.
Reading from the new solution is:


Question: ### Starting a Kubemark cluster <del> To start a Kubemark cluster on GCE you need to create an external cluster (it can be GCE, GKE or any other cluster) by yourself, build a kubernetes release (e.g. by running `make quick-release`) and run `test/kubemark/start-kubemark.sh` script. This script will create a VM for master components, Pods for HollowNodes </del> <ins> To start a Kubemark cluster on GCE you need to create an external kubernetes cluster (it can be GCE, GKE or anything else) by yourself, make sure that kubeconfig points to it by default, build a kubernetes release (e.g. by running `make quick-release`) and run `test/kubemark/start-kubemark.sh` script. This script will create a VM for master components, Pods for HollowNodes </ins> and do all the setup necessary to let them talk to each other. It will use the configuration stored in `cluster/kubemark/config-default.sh` - you can tweak it however you want, but note that some features may not be implemented yet, as

A glitch I experienced when following the : Although it might be obvious, it doesn't hurt to expand "external cluster" to "external kubernetes cluster". Also, the guide doesn't mention I need to configure my kubectl to target the "external kubernetes cluster" while it's actually required.


Question: # KUBE_RACE="-race" KUBE_GOFLAGS="-tags 'integration no-docker' "  KUBE_RACE=""  <ins> KUBE_TIMEOUT="${KUBE_TIMEOUT}"  </ins> KUBE_TEST_API_VERSIONS="$1"  KUBE_API_VERSIONS="v1,experimental/v1alpha1"  "${KUBE_ROOT}/hack/test-go.sh" test/integration

The first line logged from the integration test is the last line from the service account token test is then the test panics with: I think the tests are just running too close the the timeout. On successful runs, the tests take ~115 seconds, which is ridiculously close to the threshold. For example: see
ok, do you want to send a PR to increase the timeout to 3 or 4 minutes?
Sure, I can do that tonight

cc
Has this test failed more than once, or did the two-minute timeout just happen to occur in the middle of it?
I think 2 minutes is not sufficient for the integration tests. See comment at


Question: <del> /* Copyright 2018 The Kubernetes Authors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package util import ( "time" ) // Clock provides an interface for getting the current time type Clock interface { Now() time.Time } // RealClock implements a clock using time type RealClock struct{} // Now returns the current time with time.Now func (RealClock) Now() time.Time { return time.Now() } </del>

We should use /kind cleanup
/remove-kind feature /good-first-issue
This request has been marked as suitable for new contributors. Please ensure that the issue body includes answers to the following questions: Why are we solving this issue? To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points? Does this issue have zero to low barrier of entry? How can the assignee reach out to you for help? For more details on the requirements of such an issue, please see and ensure that they are met. If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : /remove-kind feature /good-first-issue Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scheduling /triage accepted
/assign


Question: } for _, rs := range rss { if rsToDelete.RealServer.Equal(rs) { <del> // Delete RS with no connections // For UDP, ActiveConn is always 0 // For TCP, InactiveConn are connections not in ESTABLISHED state if rs.ActiveConn+rs.InactiveConn != 0 { </del> <ins> // For UDP traffic, no graceful termination, we immediately delete the RS //     (existing connections will be deleted on the next packet because sysctlExpireNoDestConn=1) // For other protocols, don't delete until all connections have expired) if rsToDelete.VirtualServer.Protocol != "udp" && rs.ActiveConn+rs.InactiveConn != 0 { </ins> klog.Infof("Not deleting, RS %v: %v ActiveConn, %v InactiveConn", rsToDelete.String(), rs.ActiveConn, rs.InactiveConn) return false, nil }

What happened: We're running pod behind and sending logs via UDP to it. It is k8s v1.13.3 with ipvs services. When pod gets restarted logs getting blackholed. Simplified example on a node: So, old pod had IP 192.168.129.116, new pod - 192.168.128.193. Weight correctly forwards all new traffic to new pod, but there are some which are still sending traffic to non-existing IP: Minimal value for UDP connection expiration is 1s: But logs are being sent at much faster rates, so these broken connections are never expire. Issue introduced by Looks like this supposed to keep old pod running, while there are still connections to it But is there a reason to still keep these realServer entries, while old pod already cleaned up and no such IP exist? What you expected to happen: would cleanup all non-existing realServers from ipvs entries. How to reproduce it (as minimally and precisely as possible): Repro steps are same as for host-port issue: Anything else we need to know?: sysctls are default: Environment: Kubernetes version (use ):Cloud provider or hardware configuration: on-premise OS (e.g: ): Debian 9 Kernel (e.g. ): 4.9.0-8-amd64 Install tools: Others:
/sig network
/triage unresolved Comment when the issue is assessed and confirmed. I am a bot run by
cc
We observe the same behavior in k8s v1.12.7 and it hits us hard. In our case Nginx in a pod uses coredns service IP as DNS resolver address and every time coredns pods get new addresses DNS queries are blackholed. Query rate is higher than ipvs UDP connection timeout (30s in our case), so the connection to a non-existent ip address stays forever. Kube-proxy keeps non-existent real server entry in ipvs service and conntrack entry is never getting removed either. Kube-proxy logs: ipvsadm -ln:
I've been putting some thoughts on the issue (we actually have the same one). Currently, we handle graceful termination the same way for udp as we do for tcp. However on heavily loaded UDP services, (DNS for instance) udp ports are reused quickly and old backends are not deleted fast enough.

There are two possible solutions: use more aggressive timeouts (30s instead of the default 300s). Not perfect but better immediately delete real servers and associated connections when a backend is not ready anymore (very good for DNS queries but this means no graceful termination for udp) / what do you think? I'm not sure what the exact behavior with iptables I can create a detailed issue with options to have a wider discussion. Or even directly send an email to the sig-network group
I like the idea. BTW, iptables also does not support graceful termination for UDP connections.
this is very easy to implement: change the test here: make sure is set (which we do : )
If we agree this is the way to go, I can implement it (if iptables works this way this is the best solution, I agree) We could still tweak timeouts for TCP (defaults are very long: 900s, which can lead to complete blackholing for 15mn on a crash of a target because the connection is never closed)
I agree.
/area ipvs
If you have a test environment I have a test image with the fix discussed just above: Commit here: This is not perfect, but I believe it is much better. What happens is the following: udp target pod becomes NotReady IP is removed from RealServers immediately Because of any new packet reusing the port will generate a write error and the connection will be removed from the ipvs connection table (if the application retries you are fine) I think in addition to that we should set much more aggressive UDP timeouts (5s? instead of the default: 300s) and make it configurable with a flag for specific use-case (I haven't seen a use case for long lived udp "connections" with very low amount of traffic)
Sorry, i was unable to quick test this on local dev cluster. Tried to replace image with but it seems it does not work with k8s 1.13. It then generates a lot of the same messages, and no any ipvs VIPs are created. I've also tried on fresh new cluster with the same result. Then tried but was unable to even start the cluster (with modified config from our 1.13) Can you please cherry-pick this to 1.13 or 1.14 or am I doing something wrong?
If this issue has been triaged, please comment .

If you aren't able to handle this issue, consider unassigning yourself and/or adding the label. I am a bot run by vllry.
We tested this patch to kubernetes v1.12.7 and it works well in production. BTW, I don't know much about SCTP. But question is do we need to threat it as UDP or TCP in this case? But it seems more close to UDP.
Thanks for testing! The PR is limiting graceful termination to TCP which seems the safest. I don't know SCTP but making it behave like TCP would be an easy change.
This feels like a fundamental problem w/ endpoints... ideally the proxier would remove the realserver when the pod is nonexistent rather than relying on arbitrary timeouts. Otherwise, there is no way to support graceful termination with UDP based protocols. Not a big deal for most UDP things, but I imagine e.g. QUIC users would be affected. Immediately terminating udp connections is in line with the iptables behavior, which seems to be okay for most people, so the change seems reasonable until there is a better way to determine pod state.
The PR that was just merged will improve things a lot for UDP but I completely agree with you on the fundamental problem with endpoints. I think we should have 3 different states: Ready (accepting traffic), in which case we should route traffic to the endpoint NotReady (terminating for instance), in which case we should stop routing new traffic to the endpoint but should keep existing connections "Gone" (no in the endpoint object anymore), in which case we should remove the endpoint altogether Today the endpoint object is too simple to handle these differences. I hope the new EndpointCondition status in the EndpointSlice API will help address this problem. More than happy to integrate this in IPVS when it is released (cc / )


Question: "github.com/GoogleCloudPlatform/kubernetes/pkg/api/errors" "github.com/GoogleCloudPlatform/kubernetes/pkg/client" "github.com/GoogleCloudPlatform/kubernetes/pkg/client/cache" <ins> "github.com/GoogleCloudPlatform/kubernetes/pkg/client/record" </ins> "github.com/GoogleCloudPlatform/kubernetes/pkg/cloudprovider" "github.com/GoogleCloudPlatform/kubernetes/pkg/fields" "github.com/GoogleCloudPlatform/kubernetes/pkg/labels"

If a LoadBalancer cannot be created (e.g. quota limit hit), there is currently no way to know why other than reviewing the logs (?). We could put an status message / object into LoadBalancerStatus.
We could make an event for this too Justin Santa Barbara < :
+1 for event
I think that is reasonable, but we need to be careful how we define it. Event also sounds good. , Justin Santa Barbara < :
I'm in favor of status as well as an event. The pattern I recommend is an array of conditions, similar to NodeCondition, where each contains at least a Status, Reason, and Message.
+1 for an event. I'd argue this is important enough for usability that we should get it into v1.
How often does this failure mode occur?
I unfortunately don't have stats, but it can happen for a handful of reasons. I think these are the most likely: Cloud provider unavailability/errors Ran out of quota from the cloud provider The cluster is using a cloud provider that doesn't support external load balancers I expect the third one is most common by far. We could perhaps move that into the API object validation rather than writing events to handle that particularly case, but using events would solve all the cases together.


Question: } if !bFoundDisk { <del> return nil, fmt.Errorf("detach azure disk failure, disk %s not found, diskURI: %s", diskName, diskURI) </del> <ins> // only log here, next action is to update VM status with original meta data klog.Errorf("detach azure disk: disk %s not found, diskURI: %s", diskName, diskURI) </ins> } newVM := compute.VirtualMachineScaleSetVM{

What happened: made sure that API calls to Azure that return a HTTP code 200 with an embedded error are retried. Nevertheless, we observe a series of errors during the Azure Disk detach backoff while calling , and the detach operation is immediately cancelled. Happy scenario: the disk is to be re-attached to the same VM. The reconciler's will update the VMSS instance and the previous detach error will be masked. It takes 15 min for the detach call to timeout but the disk will eventually get re-attached after several VMSS updates (). Not-so-happy scenario: the disk is to be re-attached to another VM. The same errors as in will be thrown indefinitely. controller-manager's logs: What you expected to happen: The backoff succeeds and the failed detach operation gets retried. How to reproduce it: A single Pod with a single Azure Disk. This seems quite hard to reproduce, but it is happening consistently in one of our customer's resource group (eastus). Anything else we need to know?: Environment: Kubernetes version: branch (kubernetes/kubernetes Cloud provider: Azure (via AKS-Engine) OS: Ubuntu 16.04.6 LTS Kernel: 4.15.0-1042-azure Install tools: Network plugin and version: Others: /sig azure paging
/assign


Question: ) load("@io_bazel_rules_go//go:def.bzl", "go_repositories") <del> load("@io_bazel_rules_docker//docker:docker.bzl", "docker_repositories") </del> <ins> load("@io_bazel_rules_docker//docker:docker.bzl", "docker_repositories", "docker_pull") </ins> go_repositories( go_version = "1.8.3",

BUG REPORT Kubernetes version 2 Environment: Install tools: What happened: The code in takes care to supply the flag so it will wait on a lock to avoid interference between multiple processes modifying the table. If you read the source you discover it does this by creating a file . However, when is running in a container, is in a local filesystem, so it is not able to lock against processes running on the host or in other containers. And before you decide that the answer is to mount from the host, look at all the issues around leaking netns, e.g. Credit for pointing this out to me. Credit for pointing out locking functionality was to a few months back
Ironically, this is caused by a bugfix that moved to file-based locking. Previously, iptables would lock by reserving an abstract unix socket, which is tied to the network namespace.
Some discussion also in and following comments.
I experimented with this: but it didn't work on CoreOS, as the host wasn't using a version of iptables that used the file lock yet. Just ended up creating a directory call
I guess I should file this as an issue against kubeadm and any of the other sources of kube-proxy container config.
/assign
Should we leave this open (waiting for ... ?)
Waiting for I guess... (which I'm reviewing now)
i think this should not be marked as complete until the kubeadm fixes are in.
/reopen
/close fix in


Question: err := e2epod.WaitForPodSuccessInNamespace(f.ClientSet, verifyPod.Name, f.Namespace.Name) framework.ExpectNoError(err) }) <ins> ginkgo.It("should add resources for new huge page sizes on kubelet restart", func() { ginkgo.By("Stopping kubelet") startKubelet := stopKubelet() ginkgo.By(`Patching away support for hugepage resource "hugepages-2Mi"`) patch := []byte(`[{"op": "remove", "path": "/status/capacity/hugepages-2Mi"}, {"op": "remove", "path": "/status/allocatable/hugepages-2Mi"}]`) result := f.ClientSet.CoreV1().RESTClient().Patch(types.JSONPatchType).Resource("nodes").Name(framework.TestContext.NodeName).SubResource("status").Body(patch).Do(context.TODO()) framework.ExpectNoError(result.Error(), "while patching") ginkgo.By("Starting kubelet again") startKubelet() ginkgo.By("verifying that the hugepages-2Mi resource is present") gomega.Eventually(func() bool { node, err := f.ClientSet.CoreV1().Nodes().Get(context.TODO(), framework.TestContext.NodeName, metav1.GetOptions{}) framework.ExpectNoError(err, "while getting node status") _, isPresent := node.Status.Capacity["hugepages-2Mi"] return isPresent }, 30*time.Second, framework.Poll).Should(gomega.Equal(true)) }) </ins> } // Serial because the test updates kubelet configuration. var _ = SIGDescribe("HugePages [Serial] [Feature:HugePages][NodeFeature:HugePages]", func() { f := framework.NewDefaultFramework("hugepages-test") <ins> ginkgo.It("should remove resources for huge page sizes no longer supported", func() { ginkgo.By("mimicking support for 9Mi of 3Mi huge page memory by patching the node status") patch := []byte(`[{"op": "add", "path": "/status/capacity/hugepages-3Mi", "value": "9Mi"}, {"op": "add", "path": "/status/allocatable/hugepages-3Mi", "value": "9Mi"}]`) result := f.ClientSet.CoreV1().RESTClient().Patch(types.JSONPatchType).Resource("nodes").Name(framework.TestContext.NodeName).SubResource("status").Body(patch).Do(context.TODO()) framework.ExpectNoError(result.Error(), "while patching") node, err := f.ClientSet.CoreV1().Nodes().Get(context.TODO(), framework.TestContext.NodeName, metav1.GetOptions{}) framework.ExpectNoError(err, "while getting node status") ginkgo.By("Verifying that the node now supports huge pages with size 3Mi") value, ok := node.Status.Capacity["hugepages-3Mi"] framework.ExpectEqual(ok, true, "capacity should contain resouce hugepages-3Mi") framework.ExpectEqual(value.String(), "9Mi", "huge pages with size 3Mi should be supported") ginkgo.By("restarting the node and verifying that huge pages with size 3Mi are not supported") restartKubelet() ginkgo.By("verifying that the hugepages-3Mi resource no longer is present") gomega.Eventually(func() bool { node, err = f.ClientSet.CoreV1().Nodes().Get(context.TODO(), framework.TestContext.NodeName, metav1.GetOptions{}) framework.ExpectNoError(err, "while getting node status") _, isPresent := node.Status.Capacity["hugepages-3Mi"] return isPresent }, 30*time.Second, framework.Poll).Should(gomega.Equal(false)) }) </ins> ginkgo.Context("With config updated with hugepages feature enabled", func() { ginkgo.BeforeEach(func() { ginkgo.By("verifying hugepages are supported")

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: When I change default hugepage size 2Mi to 1Gi and reboot, kubelet can't update node status What you expected to happen: kubelet can update node info and node status was ready How to reproduce it (as minimally and precisely as possible): change the node condition validate Anything else we need to know?: Environment: Kubernetes version (use ):Cloud provider or hardware configuration:OS (e.g: ):Kernel (e.g. ):Install tools:Network plugin and version (if this is a network-related bug):Others:
/sig node
Hi, Ref. the error message, it looks like you have allocated hugepages for multiple sizes, . Can you run: to verify? If more than one of the numbers is , that is the reason. This is currently being discussed in , and the reason for implementing the check is here: Imo. this should be supported, and it would only require removing an if statement in order to implement it. I see no reason to limit the node from allocating huge pages with more than one size. Because of how it is implemented, adding support for allowing a pod to request huge pages with multiple sizes isn't trivial, and will require some potentially breaking changes, but that is not relevant to this problem. /cc
Also, can you also provide the result of , that will list the supported huge page sizes (both when is set, and when it is not.).
Thanks ! Looks like your kernel only enables huge pages with the default size. My kernel 5.0+ enable all (atleast 2MiB and 1GiB on x8664) by default. When the node booted with defaulthugepagesz=1GB it "removed support" the 2MiB one, and becuse kubelet use patch on the node status when connecting to the api server, you ended up with both and , even tho 2Mi wast supported by your machine (the sw, not the hw). Even if the error is removed, this will result in a crash when kubelet tries to write to the 2MiB hugetlb cgroup files.

I think using should work, otherwise the only current solution is to remove the node from the cluster, and rejoin afterwords. Ref. it looks like you have sorted out the problem, but I still think this is a valid issue, so don't close it. The correct behaviour for kubelet is to remove unsupported sizes, and only allow use of the supported ones.
"Imo. this should be supported, and it would only require removing an if statement in order to implement it. I see no reason to limit the node from allocating huge pages with more than one size." Could you fix this one as well? We use 2M pages only for pods but we need 1G pages for system processes. So it was little surprising when k8s doesn't start because two different pages were configured no matter they are really used or not. Thanks!
I am hitting the same, I used tuned to change the default size from 2Mi to 1Gi, i hit the same problem, after the reboot node is NotReady and it cannot rejoin , giving the error: Node "test-worker-0" is invalid: [status.capacity.hugepages-2Mi: Invalid value: resource.Quantity{i:resource.int64Amount{value:, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"20Mi", Format:"BinarySI"}: may not have pre-allocated hugepages for multiple page sizes, status.allocatable.hugepages-2Mi: Invalid value: resource.Quantity{i:resource.int64Amount{value:, scale:0}, My system just reports information for 1Gi, not 2Mi anymore. Removing the node and rebooting fixed it, the node rejoined again reporting the right capacity. But this is not a solution for us, changing sizes should be supported.
Hitting same K8s installed with my ubuntu 18 enabled 2Mi hugepage by default. As I change my master node's hugepage to 1Gi, this happened. I have only one master, so I have to rebuild the k8s cluster.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with .

Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
I'm a little confused. when i used tuned to change the hugepagesz from "defaulthugepagesz=2M hugepagesz=2M hugepages=8192" to " defaulthugepagesz=1G hugepagesz=1G hugepages=16", i hit the same problem, after the reboot node is NotReadybut when i used tuned to change the hugepagesz from " defaulthugepagesz=1G hugepagesz=1G hugepages=16" to "default_hugepagesz=2M hugepagesz=2M hugepages=8192", after the rebootkubelet status can still keep Ready. why?
Hi This will be fixed in , but it is still not merged. Hoping to get it into next release. In order to solve the issue (and use 1GiB huge pages) you have to delete the node in kubernetes via , and join the node again.
Thanks
Thanks
I think this is already fixed by can you confirm?
Hi WIth this would not cause a crash, but is still required in order to fix this properly.
I am facing the same issue. After changing the hugepage setting to . I stopped the kubelet process on the worker node and removed the node from the cluster using command and rebooted it. But after it comes back online and kubelet starts, it struggles to join the cluster . Do I have to run the join command again ?


Question: <del> /* Copyright 2022 The Kubernetes Authors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ // The code below was copied from // https://github.com/container-orchestrated-devices/container-device-interface/blob/v0.5.3/pkg/cdi/annotations.go // https://github.com/container-orchestrated-devices/container-device-interface/blob/v0.5.3/pkg/cdi/qualified-device.go // to avoid a dependency on that package and the indirect dependencies that // this would have implied. // // Long term it would be good to avoid this duplication: // https://github.com/container-orchestrated-devices/container-device-interface/issues/97 package cdi import ( "errors" "fmt" "strings" "k8s.io/apimachinery/pkg/types" kubecontainer "k8s.io/kubernetes/pkg/kubelet/container" ) const ( // annotationPrefix is the prefix for CDI container annotation keys. annotationPrefix = "cdi.k8s.io/" ) // GenerateAnnotations generate container annotations using CDI UpdateAnnotations API. func GenerateAnnotations( claimUID types.UID, driverName string, cdiDevices []string, ) ([]kubecontainer.Annotation, error) { if len(cdiDevices) == 0 { return nil, nil } annotations, err := updateAnnotations(map[string]string{}, driverName, string(claimUID), cdiDevices) if err != nil { return nil, fmt.Errorf("can't generate CDI annotations: %+v", err) } kubeAnnotations := []kubecontainer.Annotation{} for key, value := range annotations { kubeAnnotations = append(kubeAnnotations, kubecontainer.Annotation{Name: key, Value: value}) } return kubeAnnotations, nil } // updateAnnotations updates annotations with a plugin-specific CDI device // injection request for the given devices. Upon any error a non-nil error // is returned and annotations are left intact. By convention plugin should // be in the format of "vendor.device-type". func updateAnnotations(annotations map[string]string, plugin string, deviceID string, devices []string) (map[string]string, error) { key, err := annotationKey(plugin, deviceID) if err != nil { return annotations, fmt.Errorf("CDI annotation failed: %v", err) } if _, ok := annotations[key]; ok { return annotations, fmt.Errorf("CDI annotation failed, key %q used", key) } value, err := annotationValue(devices) if err != nil { return annotations, fmt.Errorf("CDI annotation failed: %v", err) } if annotations == nil { annotations = make(map[string]string) } annotations[key] = value return annotations, nil } // annotationKey returns a unique annotation key for an device allocation // by a K8s device plugin. pluginName should be in the format of // "vendor.device-type". deviceID is the ID of the device the plugin is // allocating. It is used to make sure that the generated key is unique // even if multiple allocations by a single plugin needs to be annotated. func annotationKey(pluginName, deviceID string) (string, error) { const maxNameLen = 63 if pluginName == "" { return "", errors.New("invalid plugin name, empty") } if deviceID == "" { return "", errors.New("invalid deviceID, empty") } name := pluginName + "_" + strings.ReplaceAll(deviceID, "/", "_") if len(name) > maxNameLen { return "", fmt.Errorf("invalid plugin+deviceID %q, too long", name) } if c := rune(name[0]); !isAlphaNumeric(c) { return "", fmt.Errorf("invalid name %q, first '%c' should be alphanumeric", name, c) } if len(name) > 2 { for _, c := range name[1 : len(name)-1] { switch { case isAlphaNumeric(c): case c == '_' || c == '-' || c == '.': default: return "", fmt.Errorf("invalid name %q, invalid charcter '%c'", name, c) } } } if c := rune(name[len(name)-1]); !isAlphaNumeric(c) { return "", fmt.Errorf("invalid name %q, last '%c' should be alphanumeric", name, c) } return annotationPrefix + name, nil } // annotationValue returns an annotation value for the given devices. func annotationValue(devices []string) (string, error) { value, sep := "", "" for _, d := range devices { if _, _, _, err := parseQualifiedName(d); err != nil { return "", err } value += sep + d sep = "," } return value, nil } // parseQualifiedName splits a qualified name into device vendor, class, // and name. If the device fails to parse as a qualified name, or if any // of the split components fail to pass syntax validation, vendor and // class are returned as empty, together with the verbatim input as the // name and an error describing the reason for failure. func parseQualifiedName(device string) (string, string, string, error) { vendor, class, name := parseDevice(device) if vendor == "" { return "", "", device, fmt.Errorf("unqualified device %q, missing vendor", device) } if class == "" { return "", "", device, fmt.Errorf("unqualified device %q, missing class", device) } if name == "" { return "", "", device, fmt.Errorf("unqualified device %q, missing device name", device) } if err := validateVendorName(vendor); err != nil { return "", "", device, fmt.Errorf("invalid device %q: %v", device, err) } if err := validateClassName(class); err != nil { return "", "", device, fmt.Errorf("invalid device %q: %v", device, err) } if err := validateDeviceName(name); err != nil { return "", "", device, fmt.Errorf("invalid device %q: %v", device, err) } return vendor, class, name, nil } // parseDevice tries to split a device name into vendor, class, and name. // If this fails, for instance in the case of unqualified device names, // parseDevice returns an empty vendor and class together with name set // to the verbatim input. func parseDevice(device string) (string, string, string) { if device == "" || device[0] == '/' { return "", "", device } parts := strings.SplitN(device, "=", 2) if len(parts) != 2 || parts[0] == "" || parts[1] == "" { return "", "", device } name := parts[1] vendor, class := parseQualifier(parts[0]) if vendor == "" { return "", "", device } return vendor, class, name } // parseQualifier splits a device qualifier into vendor and class. // The syntax for a device qualifier is // //	"<vendor>/<class>" // // If parsing fails, an empty vendor and the class set to the // verbatim input is returned. func parseQualifier(kind string) (string, string) { parts := strings.SplitN(kind, "/", 2) if len(parts) != 2 || parts[0] == "" || parts[1] == "" { return "", kind } return parts[0], parts[1] } // validateVendorName checks the validity of a vendor name. // A vendor name may contain the following ASCII characters: //   - upper- and lowercase letters ('A'-'Z', 'a'-'z') //   - digits ('0'-'9') //   - underscore, dash, and dot ('_', '-', and '.') func validateVendorName(vendor string) error { if vendor == "" { return fmt.Errorf("invalid (empty) vendor name") } if !isLetter(rune(vendor[0])) { return fmt.Errorf("invalid vendor %q, should start with letter", vendor) } for _, c := range string(vendor[1 : len(vendor)-1]) { switch { case isAlphaNumeric(c): case c == '_' || c == '-' || c == '.': default: return fmt.Errorf("invalid character '%c' in vendor name %q", c, vendor) } } if !isAlphaNumeric(rune(vendor[len(vendor)-1])) { return fmt.Errorf("invalid vendor %q, should end with a letter or digit", vendor) } return nil } // validateClassName checks the validity of class name. // A class name may contain the following ASCII characters: //   - upper- and lowercase letters ('A'-'Z', 'a'-'z') //   - digits ('0'-'9') //   - underscore and dash ('_', '-') func validateClassName(class string) error { if class == "" { return fmt.Errorf("invalid (empty) device class") } if !isLetter(rune(class[0])) { return fmt.Errorf("invalid class %q, should start with letter", class) } for _, c := range string(class[1 : len(class)-1]) { switch { case isAlphaNumeric(c): case c == '_' || c == '-': default: return fmt.Errorf("invalid character '%c' in device class %q", c, class) } } if !isAlphaNumeric(rune(class[len(class)-1])) { return fmt.Errorf("invalid class %q, should end with a letter or digit", class) } return nil } // validateDeviceName checks the validity of a device name. // A device name may contain the following ASCII characters: //   - upper- and lowercase letters ('A'-'Z', 'a'-'z') //   - digits ('0'-'9') //   - underscore, dash, dot, colon ('_', '-', '.', ':') func validateDeviceName(name string) error { if name == "" { return fmt.Errorf("invalid (empty) device name") } if !isAlphaNumeric(rune(name[0])) { return fmt.Errorf("invalid class %q, should start with a letter or digit", name) } if len(name) == 1 { return nil } for _, c := range string(name[1 : len(name)-1]) { switch { case isAlphaNumeric(c): case c == '_' || c == '-' || c == '.' || c == ':': default: return fmt.Errorf("invalid character '%c' in device name %q", c, name) } } if !isAlphaNumeric(rune(name[len(name)-1])) { return fmt.Errorf("invalid name %q, should end with a letter or digit", name) } return nil } func isLetter(c rune) bool { return ('A' <= c && c <= 'Z') || ('a' <= c && c <= 'z') } func isDigit(c rune) bool { return '0' <= c && c <= '9' } func isAlphaNumeric(c rune) bool { return isLetter(c) || isDigit(c) } </del>

Kubelet should stop using annotations to pass CDI device IDs to CRI runtimes. CDIDevices CRI field should be used for this purpose. This should be done when Kubelet that doesn't support CRI field reaches EOL and two major CRI runtimes support CRI field. Here is a PR comment that gives more details about this: Kubelet started using CDIDevices CRI field in . Previous version 1.27 will reach EOL .CRI-O started using it in . Previous version 1.27 will reach EOL together with Kubernetes 1.27.Containerd started using it in . Version 1.7 will reach EOL in a 6 months after Containerd 2.0 release Previous version 1.6 doesn't support CDI injections and should not be considered. So, safer option would be to wait until Containerd 1.7 reaches EOL. However it's relatively safe to do it earlier when 1.7.0 and 1.7.1 have little chances to be in real use. Containerd v1.7.0 and v1.7.1 is not used by the Linux distributions from , so it's safe to remove annotations support from the Kubelet in 1.31 in my opinion.
/sig node
/triage accepted /priority important-longterm
/assign


Question: expectedID: "/subscriptions/subscription/resourceGroups/rg/providers/" + "Microsoft.Network/publicIPAddresses/pip1", }, <ins> { desc: "ensurePublicIPExists shall update existed PIP's dns label", existingPIPs: []network.PublicIPAddress{{ Name: to.StringPtr("pip1"), PublicIPAddressPropertiesFormat: &network.PublicIPAddressPropertiesFormat{ DNSSettings: &network.PublicIPAddressDNSSettings{ DomainNameLabel: to.StringPtr("previousdns"), }, }, }}, expectedPIP: &network.PublicIPAddress{ Name: to.StringPtr("pip1"), ID: to.StringPtr("/subscriptions/subscription/resourceGroups/rg" + "/providers/Microsoft.Network/publicIPAddresses/pip1"), PublicIPAddressPropertiesFormat: &network.PublicIPAddressPropertiesFormat{ DNSSettings: &network.PublicIPAddressDNSSettings{ DomainNameLabel: to.StringPtr("newdns"), }, }, }, expectedDNS: "newdns", }, </ins> } for i, test := range testCases {

Is this a BUG REPORT or FEATURE REQUEST?: What happened: Azure will not update the DNS label of the public IP, when update the service only changing the annotation of the DNS label from empty. annotations: -annotations: "to-update" What you expected to happen: It should update the dns label of the assigned public IP How to reproduce it (as minimally and precisely as possible): It probably caused when applying function 'getPublicIPAddress' in This function returns a bool 'existsPip' to judge the public ip existence. When only the annotations changed, the existsPip still returns true. Then, the reference in line 553 and line 446 step into this bug. pip, existsPip, err := az.getPublicIPAddress(pipResourceGroup, pipName) if err != nil { return false, err } if !existsPip { return true, nil } Firstly, in line 554, it will make function 'isFrontendIPChanged' return false, and then will not run 'ensurePublicIP' in line 691. Secondly, even if 'ensurePublicIP' in line 691 runs, it will detect the public ip exists and make no update to the DNS label in line 446_. Environment: Kubernetes version (use ): Cloud provider or hardware configuration: Azure
/sig azure
/assign
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
-- what's the status on this?
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-testing, kubernetes/test-infra and/or . /close
Closing this issue. <detailsIn response to : Rotten issues close after 30d of inactivity.

Reopen the issue with . Mark the issue as fresh with . /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/open I will take a look at this
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-testing, kubernetes/test-infra and/or . /close
Closing this issue. <detailsIn response to : Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Still not fixed. /reopen /kind bug /priority important-soon /milestone v1.16
Reopened this issue. <detailsIn response to : Still not fixed. /kind bug /priority important-soon /milestone v1.16 Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig cloud-provider
i'm working on it.
/assign


Question: }, { // failed syncVolume do not bind when pvc is prebound to pv with mismatching volumeModes <del> "14-8-1 - do not bind when pv is prebound to pvc with mismatching volumeModes", </del> <ins> "14-8-1 - do not bind when pvc is prebound to pv with mismatching volumeModes", </ins> withVolumeVolumeMode(&modeBlock, newVolumeArray("volume14-8-1", "10Gi", "", "claim14-8-1", v1.VolumePending, v1.PersistentVolumeReclaimRetain, classEmpty)), withVolumeVolumeMode(&modeBlock, newVolumeArray("volume14-8-1", "10Gi", "", "claim14-8-1", v1.VolumePending, v1.PersistentVolumeReclaimRetain, classEmpty)), withClaimVolumeMode(&modeFile, newClaimArray("claim14-8-1", "uid14-8-1", "10Gi", "", v1.ClaimPending, nil)),

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: If a user specifies volumeMode: Block in creating PVC and neither plug-in nor external provisioner doesn't support block, a PV is created, but is not bound to the PVC because the PV and PVC don't match. So, PVC will stay pending forever without any error messages. What you expected to happen: In above case, a proper error message is shown to users to make them understand what's wrong. How to reproduce it (as minimally and precisely as possible): up plug-in and external provisioner that don't support block. PVC using volumeMode: Block. Anything else we need to know?: Environment: Kubernetes version (use ):Cloud provider or hardware configuration: KVM OS (e.g. from /etc/os-release):Kernel (e.g. ):Install tools: hack/local-up- Others: /sig storage


Question: cloud.google.com/go/compute v1.19.1/go.mod h1:6ylj3a05WF8leseCdIf77NK0g1ey+nj5IKd5/kvShxE= cloud.google.com/go/compute/metadata v0.2.3/go.mod h1:VAV5nSsACxMJvgaAuX6Pk2AawlZn8kiOGuCv6gTkwuA= <del> github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs= </del> github.com/census-instrumentation/opencensus-proto v0.4.1/go.mod h1:4T9NM4+4Vw91VeyqjLS6ao50K5bOcLKN6Q42XnYaRYw= github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= github.com/cncf/udpa/go v0.0.0-20220112060539-c52dc94e7fbe/go.mod h1:6pvJx4me5XPnfI9Z40ddWsdw2W/uZgQLFXToKeRcDiI= github.com/cncf/xds/go v0.0.0-20230607035331-e9ce68804cb4/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= <del> github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= </del> github.com/envoyproxy/go-control-plane v0.11.1-0.20230524094728-9239064ad72f/go.mod h1:sfYdkwUW4BA3PbKjySwjJy+O4Pu0h62rlqCMHNk+K+Q= github.com/envoyproxy/protoc-gen-validate v0.10.1/go.mod h1:DRjgyB0I43LtJapqN6NiRwroiAU2PaFuvk/vjgh61ss= <del> github.com/evanphx/json-patch v4.12.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= github.com/go-logr/logr v1.2.0/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= github.com/go-logr/logr v1.2.4 h1:g01GSCwiDw2xSZfjJ2/T9M+S6pFdcNtFYsp+Y43HYDQ= github.com/go-logr/logr v1.2.4/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= github.com/go-openapi/jsonpointer v0.19.6/go.mod h1:osyAmYz/mB/C3I+WsTTSgw1ONzaLJoLCyoi6/zppojs= github.com/go-openapi/jsonreference v0.20.2/go.mod h1:Bl1zwGIM8/wsvqjsOQLJ/SH+En5Ap4rVB5KVcIDZG2k= github.com/go-openapi/swag v0.22.3/go.mod h1:UzaqsxGiab7freDnrUUra0MwWfN/q7tE4j+VcZ0yl14= github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572/go.mod h1:9Pwr4B2jHnOSGXyyzV8ROjYa2ojvAY6HCGYYfMoC3Ls= </del> github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q= github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q= github.com/golang/glog v1.1.0/go.mod h1:pfYeQZ3JWZoXTV5sFc986z3HTpwQs9At6P4ImfuP3NQ= github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk= github.com/golang/protobuf v1.5.3 h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg= github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY= <del> github.com/google/gnostic-models v0.6.8/go.mod h1:5n7qKqH0f5wFt+aWF8CW6pZLLNOfYuF5OpfBSENuI8U= </del> github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38= github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= <del> github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= </del> github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= <del> github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y= github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo= </del> github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= <del> github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc= github.com/moby/spdystream v0.2.0/go.mod h1:f7i0iNDQJ059oMTcWxx8MA/zKFIuD/lY+0GqbN2Wy8c= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk= github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f/go.mod h1:ZdcZmHo+o7JKHSa8/e818NopupXU1YMK5fe1lsApnBw= github.com/onsi/ginkgo/v2 v2.13.0/go.mod h1:TE309ZR8s5FsKKpuB1YAQYBzCaAfUgatB/xlT/ETL/o= github.com/onsi/gomega v1.28.0/go.mod h1:A1H2JE76sI14WIP57LMKj7FVfCHx3g3BcZVjJG8bjX8= github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog= github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= github.com/stretchr/testify v1.8.3/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= </del> github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=

Drop the and dependencies from to keep the import lighter. /sig auth /triage accepted
/assign Hi, , I created a PR to fix this. I am not sure If I am correct due to the repo seems generated. Would you like to provides some hints for me?
This will likely require a new go module in to move the client-go dep and then mark it as internal, ex: . is the only API machinery dep so we can probably come up with some way to re-implement the wait logic locally? Then we drop these lines so that these imports do not come back again:
Whoa, yeah, I'd take a for loop and a time.Sleep in a test to drop those deps
What is kms using client-go for? Tests only?
These were all needed by the KMS repo back when we had a much more complicated reference implementation which required a key hierarchy. Now they are just used in some tests and mock implementations, so we should be able to simplify things.
I found a place for rate limit. #liggitt
that package just delegates to "", switch that mock provider to use that directly?


Question: rbacv1helpers.NewRule(Write...).Groups(legacyGroup).Resources("replicationcontrollers", "replicationcontrollers/scale", "serviceaccounts", "services", "services/proxy", "endpoints", "persistentvolumeclaims", "configmaps", "secrets").RuleOrDie(), <ins> rbacv1helpers.NewRule(Write...).Groups(discoveryGroup).Resources("endpointslices").RuleOrDie(), </ins> rbacv1helpers.NewRule(Write...).Groups(appsGroup).Resources( "statefulsets", "statefulsets/scale", "daemonsets",

Users are not able to view by default as part of the or roles like they are with Users with or roles should be able to view like they are with Have a user with permissions on a namespace. Try to run the command I asked about this in slack and was told that it was an oversight that it wasn't to the default roles. Kubernetes version (use ): 1.21 Cloud provider or hardware configuration: n/a OS (e.g: ): n/a Kernel (e.g. ): n/a Install tools: n/a Network plugin and version (if this is a network-related bug): n/a Others: n/a
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig security
/sig auth


Question: // ConfigFile is the location of the scheduler server's configuration file. ConfigFile string <ins> // WriteConfigTo is the path where the default configuration will be written. WriteConfigTo string </ins> // config is the scheduler server's configuration object. config *componentconfig.KubeSchedulerConfiguration

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: We face But there is no way to generate default config file How to reproduce it (as minimally and precisely as possible): Environment: Kubernetes version 1.9.2
I'll take a look it /sig scheduling
/assign
Hi, same issue with controller-manager
Any updates here? , and don't have this flag. Is it even will be supported?
/cc
if not set, component should be use default value; why we'd like to dump all default config?
Because the documentation on the config file format is either nonexistent or difficult to find. If the program itself can generate a template, that fulfills (most of) the need for documentation of the format.
+1 for this, from my expirence, add an option to write the default config is really helpful.
/assign
also in 1.10
Deprecating all flags, prompting folks to use config files in advance is great, except when there's no documentation nor example anywhere. That means that we can't actually migrate to the new settings, and I can't seem to find clear alpha/beta/stable status for the new config file spec, nor timeline for the removal. It is also pretty confusing that the scheduler expects us to use a config file, but the controller-manager does not?
This issue indeed applies to kube-controller-manager. And there should already had deprecation policy.
According to , the flag will be available in . Since I can't find any reference online to an example config file, perhaps one could be provided or linked to here?
will you be able to pick my patch and generate a config file? I do not get my devbox handy so can only post here until Monday.


Question: <ins> apiVersion: v1 kind: Service metadata: name: test-service labels: name: test-service spec: ports: - port: 80 selector: name: test-rc </ins>

kubectl create, delete, etc. can already take a List object. kubectl apply can take the list object when it does creation, but it gives an (ugly) error when it's configuring the objects. Then... expected: Actual:
what's you ?
0-alpha.5
Sounds like a bug. Are you interested in taking a look
sure I will take a look during this weekend.
I think the root cause is , the actually is , in the origin , is not , but after the , this object is nil. Would you like to double check this?
I'll take a look.
if you have make sure about this, please let me know. Thanks :)
Is there any ways we could get this into 1.2? It's actually making our config generation quite painful to have to make a separate file for every object. It looks like got pretty far? would you like to make a PR that we can review?
I don't see how. It could have made 1.2 if a PR had been sent within a few days of the issue being filed.
I unassigned all issues assigned to you. Please only assign issues that are actively being worked on, so that others can know which issues are available to be picked up.
My apologies. If I'm not mistaken, this issue was filed after the code freeze for 1.2. I didn't realize that it was time critical or that it could make the release at that time. However, I should have asked. Sorry for the inconvenience. Thank you. I still plan to address those issues for the next release, as discussed, but will assign each one to myself as I work on it.
I created which fixes this. I'm really hoping we can get this into 1.2 since it's a pretty small, isolated fix but a pretty big bug that is complicating our config generation system. Let me know what else I can do to help.
Thanks for the PR. We're short on reviewer bandwidth at the moment. I assigned it to Clayton for review or delegation.