Question: // wait as required for all 3 pods to be running ginkgo.By("waiting for all 3 pods to be running") <del> err := e2epod.WaitForPodsRunningReady(f.ClientSet, f.Namespace.Name, 3, 0, framework.PodStartTimeout, nil) </del> <ins> err := e2epod.WaitForPodsRunningReady(f.ClientSet, f.Namespace.Name, 3, 0, f.Timeouts.PodStart, nil) </ins> framework.ExpectNoError(err, "3 pods not found running.") // delete Collection of pods with a label in the current namespace

pull-kubernetes-e2e-kind-ipv6 Pods should delete a collection of pods According to testgrid, March 18th Timeout in No response /sig node
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Looking at The test is here - It creates 3 pods which go into Failed phase and then expects them to within . Looking at failure in the build-log, there were three pods created, , , . did not get deleted from the API-server within the 1 minute timeout causing the test to fail. From the build-log, the test starts checking for all the pods to be deleted at: So the test expected all the pods to be deleted at 1 minute + 18:35:03 = 18:36:03 Looking at the logs of which landed on : DELETE was sent at landed on DELETE was sent at which is ~20 sec after when the test with the 1 minute timeout expected which caused the test to fail.
I suggest we bump the timeout a bit since everything seems to be working, it's just taking a bit longer than the timeout is currently set to. This also may be affected by recent which slightly increases how long it takes for pod to be marked as terminal by kubelet.


Question: func InjectHtml(client clientset.Interface, config VolumeTestConfig, volume v1.VolumeSource, content string) { By(fmt.Sprint("starting ", config.Prefix, " injector")) podClient := client.CoreV1().Pods(config.Namespace) <ins> podName := fmt.Sprintf("%s-injector-%s", config.Prefix, rand.String(4)) volMountName := fmt.Sprintf("%s-volume-%s", config.Prefix, rand.String(4)) </ins> injectPod := &v1.Pod{ TypeMeta: metav1.TypeMeta{

Failing Job: These tests started flaking recently (they were consistently green for a long time): Mounted volume expand[Slow] Should verify mounted devices can be resized Volume expand [Slow] Verify if editing PVC allows resize Subpath [Volume type: gcePDPVC] should support restarting containers using file as subpath [Slow] Subpath [Volume type: gcePDPVC] should fail for new directories when readOnly specified in the volumeSource Subpath [Volume type: gcePDPVC] should fail if subpath with backstepping is outside the volume [Slow] Volumes PD should be mountable with ext4 Affected runs: https://k8s- https://k8s- https://k8s- /kind bug /sig storage /priority important-soon /milestone v1.12 /cc - Could you PTAL or delegate? Thanks.
Volume expansion tests have been fixed by
/assign can you investigate the remaining failures?
[MILESTONENOTIFIER] Milestone Issue Needs Approval Action required: This issue must have the label applied by a SIG maintainer. If the label is not applied within 4 days, the issue will be moved out of the v1.12 milestone. <details<summaryIssue Labels</summary- : Issue will be escalated to these SIGs if needed. : Escalate to the issue owners and SIG owner; move out of milestone after several unsuccessful escalation attempts. : Fixes a bug discovered during the current release. </details<details<summaryHelp</summary<ul<li<a href=""Additional instructions</a</li<li<a href=""Commands for setting labels</a</li</ul</details
Reopening this issue, as a bunch of storage-related e2es started flaking in our large-scale jobs recently: https://k8s- https://k8s- /reopen
I think the latest failures are caused by gce API rate limiting: /assign
It seems to only be failing on scalability test suites. I wonder if it's due to some different cidr configuration
/sig scalability
/kind failing-test
/status approved-for-milestone Friendly ping. This is affecting release-blocking scalability job.
hey what needs to be done here? by whom?
Could you comment on what needs to be done here and by whom?
can you guys take a look at the API quota issue?

This is happening consistently in the scale test environment
as Code Freeze is today, this needs to be of priority . Please triage! cc
It looks like the latest run on 9/1 has passed.
Marking closed as the fixes have been merged and the test runs are now passing /close
Closing this issue. <detailsIn response to : Marking closed as the fixes have been merged and the test runs are now passing /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: func TestPidListerFailure(t *testing.T) { _, err := getPids("/does/not/exist") <del> assert.True(t, cgroups.IsNotFound(err), "expected getPids to return not exists error. Got %v", err) </del> <ins> assert.True(t, cgroups.IsNotFound(err) || os.IsNotExist(err), "expected getPids to return not exists error. Got %v", err) </ins> }

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: What you expected to happen: The test should pass How to reproduce it (as minimally and precisely as possible): I think the key here is being on a linux host with cgroupsv2 enabled. Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others: /sig node
/assign
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: RuntimeConfig              util.ConfigurationMap KubeletConfig              client.KubeletConfig ClusterName                string <ins> SyncPodStatus              bool </ins> } // NewAPIServer creates a new APIServer object with default parameters

Split off as a smaller more specific work item. Apiserver should have a flag that makes it not attempt to GET from kubelets. This would be used in conjunction with . After we have converted the top N Getting-started-guides to the new style, we can rip out the kubelet client from the apiserver altogether.
This is tightly related to so I'll take it.


Question: // A Set is used here since we end up calculating endpoint topology multiple times for the same Service // if it has multiple ports but each Service should only be counted once. serviceNoLocalEndpointsExternal sets.Set[string] <ins> // lbNoNodeAccessIPPortProtocolEntries represents the set of loadBalancers IP + Port + Protocol that should not be accessible from K8s nodes // We cannot directly restrict LB access from node using LoadBalancerSourceRanges, we need to install // additional iptables rules. // (ref: https://github.com/kubernetes/kubernetes/issues/119656) lbNoNodeAccessIPPortProtocolEntries []*utilipset.Entry </ins> } // Proxier implements proxy.Provider

When using loadBalancerSourceRanges to allow access only from CIDRs outside the cluster segment, access from the K8s nodes (or pods with hostNetwork:true) should be rejected, but for proxy-mode=ipvs it works. /sig network /area kube-proxy /area ipvs Access from K8s nodes should be rejected in the same way as for proxy-mode=iptables Define a service with loadBalancerSourceRanges that allows access from a cidr outside the cluster, e.g. The K8s nodes have addresses on 192.168.1.0/24 and fd00::c0a8:100/120. Make sure a loadBalancerIP is assigned (in this example 10.0.0.0/32) and try to access it from a K8s node: Same for both ipv4 and ipv6. This is related to since a fix requires that the passed to the is correct. Please see v1.27.4 and master None Linux 6.4.4 None N/A N/A
wanted to work on this
Great! But please coordinate with I hope that one get accepted, and then ~the s (both families) passed to the proxiers should be usable.~ No, they are not, please see
Discussed this one in the SIG Network meeting today: we agree this is something we'd like to implement, however we weren't sure exactly how to do it with ipvs during the meeting. Whoever takes this on needs to come up with and communicate a clear plan of how it will be accomplished. /triage accepted
My comment is not feasible. For proxy-mode=ipvs the src and dst is both loadBalancerIP. So the nodeIP will not help :disappointed:
I did a small experimented with kind + metallb. I created a loadbalancer service backed by . loadBalancerIP: 172.18.255.200 podIP: 10.244.1.3 conntrack picks the src and dst both as loadBalancerIP. Tried to create an iptables rules matching source and destination with loadbalancerIP to drop packets but it didn't work. After some exploration I found and with the following rules I was able to prevent access to lbIP from k8s nodes / host network.

The solution looks hacky though, we only need this rule if source ranges are defined for the service and node ip is not covered by any of them. Will try to program this in the ipvs proxier.
Good find! :+1: I had missed the ipvs extension. I started to check nodeIP and the (flawed) option to experiment with setting a nodeIP as "src" for some routes to dodge the lbip-both-as-src-and-dst problem, and an old "vision" I have to not assign clusterIPs to at all, but have one rule to replace them all, like: It can be done, and in a cluster with 1000's of clusterIP's it would simplify


Question: addProfilingFlags(flags) <del> kubeConfigFlags := genericclioptions.NewConfigFlags() </del> <ins> kubeConfigFlags := genericclioptions.NewConfigFlags(true) </ins> kubeConfigFlags.AddFlags(flags) matchVersionKubeConfigFlags := cmdutil.NewMatchVersionFlags(kubeConfigFlags) matchVersionKubeConfigFlags.AddFlags(cmds.PersistentFlags())

/kind bug What happened: takes ~11 seconds when there are thousands of CRDs and is around 28KiB. logs the following message once per resource: Removing unnecessary entries from reduces the runtime by around ten seconds. What you expected to happen: only parses once, and therefore doesn't run unnecessarily slowly. How to reproduce it (as minimally and precisely as possible): shows the config being loaded multiple times (assuming you have multiple pods in the current cluster). Alternatively, add 10-15 clusters to and create ~3000 of a cheap resource (eg custom resources or empty config maps). Run on this resource and observe that it is unnecessarily slow. Anything else we need to know?: I've created a CPU profile using the latest CI build of kubectl, which shows ~12 seconds being spent in . Environment: Kubernetes version (use ): v1.12.1 (also Cloud provider or hardware configuration: GKE or minikube --vm-driver=none OS (e.g. from /etc/os-release): Debian Buster-like Kernel (e.g. ): 4.17.0
/sig cli
/assign
Same problem here. I have less resources but more contexts in my kubeconfig file. With 90 pods and ~300 contexts in the kubeconfig it also takes ~10 seconds for a simple kubectl get pods Note: I tried it with kubectl: 1.11.0, 1.12. With kubectl 1.10.5 the kubeconfig was loaded only once.
I'm unable to reproduce this as of the latest master. My kubeconfig is only loaded 4 times, no matter how many resources are loaded.
When I try a kubectl v1.13.0-alpha.3 against unfortunately a 1.9.11 Kubernetes Cluster I still have the problem: `
Maybe the bug is Linux-only? Presumably the kubeconfig location is different on different platforms, so maybe it has a different code path. I can repro on latest master:
I filed to fixed this. You can try to help validate it. Thanks.
Works for me, thanks!


Question: } } <ins> func verifyUnorderedCalls(t *testing.T, fakeDocker *dockertools.FakeDockerClient, calls []string) { err := fakeDocker.AssertUnorderedCalls(calls) if err != nil { t.Error(err) } } </ins> func verifyStringArrayEquals(t *testing.T, actual, expected []string) { invalid := len(actual) != len(expected) if !invalid {

Seeing random failures.
Looks like lately refactory makes docker API requests are not deterministic. Will look into it.
Any update? It seems to be failing a lot.
I just looked into this and found that is the only test that there are two pods in the which is fed to I think podworker was syncing the two pods in parallel, this causes non-deterministic behaviour I have tried , no errors. , error happens continuously. I can try to fix this :)
Yes, that is what I believe here, but never have time working on it. Thanks for looking into this, and re-assign it to you.
I saw your old PR for this, I think we still need to keep the parallel sync here. Just need to create a for this case :)


Question: Start the server: ```sh <del> cluster/kubectl.sh proxy -www=$PWD/www </del> <ins> cluster/kubectl.sh proxy --www=$PWD/www </ins> ``` The UI should now be running on [localhost](http://localhost:8001/static/index.html#/groups//selector)

I have been following the instructions here: to try to explore the kube UI. by executing: I get the following which shows it successfully started: When I try to access the page I get a 404:
could you try running ? I believe is a typo of
Also, try accessing instead.
thanks


Question: {Min: 1, Max: 3}, {Min: 4, Max: 4}, }, <ins> expectedError: "group 5 must be in the ranges: [{1 3} {4 4}]", </ins> }, "in range 1": { groups: []int64{2}, ranges: []extensions.GroupIDRange{ {Min: 1, Max: 3}, }, <del> pass: true, </del> }, "in range boundary min": { groups: []int64{1}, ranges: []extensions.GroupIDRange{ {Min: 1, Max: 3}, }, <del> pass: true, </del> }, "in range boundary max": { groups: []int64{3}, ranges: []extensions.GroupIDRange{ {Min: 1, Max: 3}, }, <del> pass: true, </del> }, "singular range": { groups: []int64{4}, ranges: []extensions.GroupIDRange{ {Min: 4, Max: 4}, }, <del> pass: true, </del> }, }

Is this a BUG REPORT or FEATURE REQUEST?: This is request for improvements. What happened: When PodSecurityPolicy provider validates or fields of a pod, it shows an error like "999 is not an allowed group" if the group doesn't belong to the range(s). What you expected to happen: It should include to the error message also the allowed values -- the ranges against what we were validating this group. This will simplify user experience and debugging. How to reproduce it (as minimally and precisely as possible): run kubernetes cluster with PSP support create a PSP that allows fsGroup/SupplementalGroups in some ranges grant to a user access to this PSP try to create a pod that has fsGroup/SupplementalGroups outside of the allowed ranges Anything else we need to know?: In order to improve the current behavior, we need to modify function here: Reported-by: /kind help
This looks like a good first issue! My suggestion would be something like . I would also suggest to merge overlaying ranges in the log message, like as , to keep the log as concise as possible. I will look into it.
Maybe something like ? Thank you!
We can use the same approach that strategy uses:
Sure, that is probably the most consistent solution.
/remove-help


Question: } func TestDelNode(t *testing.T) { <ins> defer func() { now = time.Now }() var tick int64 now = func() time.Time { t := time.Unix(tick, 0) tick++ return t } </ins> evictor := NewRateLimitedTimedQueue(flowcontrol.NewFakeAlwaysRateLimiter()) evictor.Add("first") evictor.Add("second")

Running show failures at times: The expected queue is while got , see code here: I would give an idea that these cases have something similar, so referring and /cc
cc
I think so as well.
The debugging I to the test is obviously not correct - I'll fix that separately.
I wonder if on very short intervals we're not getting a goroutine schedule swap, which causes this race to manifest (based on comments in the other thread).
Does heap insertion guarantee order if the heap value is equivalent? I don't think so.
This seems like it is that now() is not monotonically increasing, so we violate the add ordering condition. Since we already have a lock in most cases, we may want to either wrap now() to make it monotonic inc, or have the queue check for equality in the heap and add one nanosecond until there is no matching heap item. On Apr 29, 2016, at 7:12 PM, Marek Grabowski wrote: cc — You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub
Thanks for clarification, so it looks like we need to give some time break between doing , I'll see whether that helps.
I believe this is a bug in rate limited queue, so we definitely need to fix it (esp. since it's happening in the real world)
you wrote Would you be a good person to pick an owner for this bug? Giving P1 since we think happens in real world and I don't understand severity.
and I are probably the two who have touched it, I think as well but may be misremembering. It should be a straightforward fix to make now monotonic, I don't know when I can get to it
Me neither. If it's P1 I most likely won't have time before v1.3.
I think override the function call in as you did would resolve this issue, what do you think, if you agree with this, I am gonna to try that.
It would fix it, although I think this is broken in the core code, not just in the test. We suspect real workloads have been impacted by this.
Opened to determine whether we actually need to fix the RLQ
Assigning to per


Question: allStatus := append(append([]v1.ContainerStatus{}, s.ContainerStatuses...), s.InitContainerStatuses...) s.Phase = getPhase(&pod.Spec, allStatus) klog.V(4).InfoS("Got phase for pod", "pod", klog.KObj(pod), "oldPhase", oldPodStatus.Phase, "phase", s.Phase) <ins> // Perform a three-way merge between the statuses from the status manager, // runtime, and generated status to ensure terminal status is correctly set. if s.Phase != v1.PodFailed && s.Phase != v1.PodSucceeded { switch { case oldPodStatus.Phase == v1.PodFailed || oldPodStatus.Phase == v1.PodSucceeded: klog.V(4).InfoS("Status manager phase was terminal, updating phase to match", "pod", klog.KObj(pod), "phase", oldPodStatus.Phase) s.Phase = oldPodStatus.Phase case pod.Status.Phase == v1.PodFailed || pod.Status.Phase == v1.PodSucceeded: klog.V(4).InfoS("API phase was terminal, updating phase to match", "pod", klog.KObj(pod), "phase", pod.Status.Phase) s.Phase = pod.Status.Phase } } </ins> if s.Phase == oldPodStatus.Phase { // preserve the reason and message which is associated with the phase s.Reason = oldPodStatus.Reason

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --When a pod is evicted under resource pressure, it goes into an error state. The pod terminates normally. Note that the 130GB file is based on 90% of total disk space - current utilization to hit the default eviction threshold. Full pod spec after going into error state: <details</detailsSee also Eviction tests () are currently broken (), there's a PR up to fix: Kubernetes version (use ): HEAD/1.23.x (kubeletVersion="v1.23.0-alpha.3.47+") Cloud provider or hardware configuration: local-up-cluster OS (e.g: ), Kernel (e.g. ):Install tools: local-up- Network plugin and version (if this is a network-related bug): N/A Others: Also reproducible on OpenShift 4.9 (1.23-based) /sig node /cc
mentions this looks very similar to
Confirmed that this is a regression from 1.21. When running the reproducer against the release-1.21 branch, the pod above correctly stays in Evicted status (since pods are mortal). On 1.22, it tries to restart, entering either an Error or ContainerCreating state. /kind regression
/assign
/triage accepted
From : is the first bad commit This was
Seems likely that wasn't actually fixed. The broken eviction tests made it much harder to verify.
Attaching logs from HEAD from reproducer: v=4 Let me also try to get a repro where it goes into Error
Logs from HEAD for reproducer when pod goes into Error: v=4
Roughly, kubelet syncTerminatingPod sends "Failed" to status manager (which looks like it generates the correct apiserver patch for phase running -failed), then somehow by the time Kubelet syncTerminatedPod is called it sends "Running but with errors" to status manager, and status manager allows that transition (internally) and then sends it to the apiserver. So the pod goes Running -Failed -Running which should never happen (we "lost" that the pod was stopped).

Bug 1: pod patch status probably shouldn't allow terminal -nonterminal phase transitions (didn't we fix that??) Bug 2: pod worker is supposed to be getting most up to date status from status manager before calling syncTerminatedPod, which should include "Failed", but that isn't happening? thus the status manager then sends a "non-terminal state" (shouldn't ever happen) Roughly, figure out why status passed to syncTerminatedPod is older than what was sent by syncTerminatingPod, then figure out why status manager allowed that transition back to running, then figure out why apiserver didn't reject.
Staring at logs, I've figured out where the illegal phase transition is happening: If the container failed to properly terminate and generate a status saying as much, generateAPIPodStatus won't have any hints available to it that the pod died because it was killed. (This is happening in the reproducer case because the container won't exit before the termination grace period, and needs to be force killed.) Thus, it will assume that since the container used to be running, its current status is running. This results in an illegal phase transition.
From what I can tell, the issue is that is being passed a pod that still has running containers. From kubelet logs: which gets set as part of . It looks like that will always run so long as succeeds. Thus, I feel like syncTerminatingPod must have the bug: it returns successfully, but the containers of the pod aren't all actually terminated.
/priority critical-urgent based on priority of and the impact of this regression (Kubelet-initiated evictions frequently result in error state pods)
The containers ARE terminated, we just haven't retrieved their status. This is because of a decision to not refresh the container status cache in syncTerminatingPods. So syncTerminatingPod is not a bug.
So then perhaps the right thing to do here would be to refresh the container status cache before syncing another status to the API server in ? Then we can avoid syncing the outdated cached status
Updating from an internal discussion last night - bug 2 isn't actually a bug.

Lantau convinced me we wouldn't refresh the status cache, so what happens is we call generatedAPIStatus with: API status (mostly irrelevant) status manager status to date container cache (but 2s) after The key thing is that phase from runtime status is irrelevant when 1 or 2 are terminal - because runtime has no idea about whether a pod is terminal. So roughly we should be ignoring phase from 3 except when determining when to go from pending to running. The bug here is that we're thinking that this is an "illegal transition" but still copying the latest reason and message (runtime status doesn't get to weigh in on overal terminal phases). We shouldn't need to refresh status at all.
Okay, so there is definitely a bug in what's being synced to the API server for evicted pod statuses right now, possibly for other terminated pods. In 1.21 and earlier, when a pod is evicted, it looks like this: <details</detailsNote that the status field is very minimal, and there are no container statuses. In 1.22+, we attempt to sync container statuses on termination, and sometimes they are wrong (e.g. changing phase from Failed -Running). Here's an example of what an evicted pod could look like on 1.22: <details</details
This appears to be broken even for runOnce pods, tested against latest HEAD (). With spec: After scheduling the pod... And the final pod status: <details</details
Elana identified the issue on status here - previously we NEVER reported container status after an eviction, so the refactor (which correctly ensured container exit codes showed up) was returning containers with state=waiting,lastState=terminated, which is incorrect. generateAPIStatus needs to know the difference between "current container status, waiting for more to show up" and "any status we have is the best we have". So once killPods succeeds in syncTerminatingPods and in syncTerminatedPods, we're no longer "waiting". So generating pod status should correctly summarize the container status as and not inject .


Question: } func NewResourceExpirationEvaluator(currentVersion apimachineryversion.Info) (ResourceExpirationEvaluator, error) { <del> ret := &resourceExpirationEvaluator{ // TODO https://github.com/kubernetes/kubernetes/issues/101951 set this back to false after beta is tagged. strictRemovedHandlingInAlpha: true, } </del> <ins> ret := &resourceExpirationEvaluator{} </ins> if len(currentVersion.Major) > 0 { currentMajor64, err := strconv.ParseInt(currentVersion.Major, 10, 32) if err != nil {

once the we hit a beta release, we can reset the default value for strict handling to false to prep for 1.23. /assign /kind bug /priority important-soon /milestone v1.22
/triage accepted
Hi Bug Triage from SIG Release here .. Checking to see if this issue is going to make it to v1.22 milestone bearing in mind code freeze is 07/08/2021. Feel free to reach out if there are any questions/concerns.
Hi, Bug Triage here, removing this issue from 1.22, as there hasn't been activity on that issue for 16 days. /milestone clear
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:


Question: setInitCommand(input.pod, fmt.Sprintf("ln -s ../ %s", input.subPathDir)) // Pod should fail <del> testPodFailSubpath(input.f, input.pod) </del> <ins> testPodFailSubpath(input.f, input.pod, false) </ins> }) It("should support creating multiple subpath from same volumes [Slow]", func() {

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!--What happened: The read only sig-storage tests validate that Pods can come up when readOnly is set. But it doesn't actually validate that the container cannot write to the volume. What you expected to happen: Tests should validate that the container cannot write to readonly volumes. <!-- DO NOT EDIT BELOW THIS LINE --/kind bug
/help
This request has been marked as needing help from a contributor. Please ensure the request meets the requirements listed . If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : /help Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
can this issue be closed after your last PR or is there more to work on? If yes I'd like to know what's left because I'm looking for something to work on
I think we should actually reinstate the test case I removed and enhance it to handle the two scenarios I mentioned. It would also be good to see if we have other readOnly test cases that could be enhanced as well
Thanks ! I have a few questions: 1) how do I run the tests in locally? and 2) how can I assert writing to volume has failed? Thank you in advance!!


Question: return false } <ins> // Since containers and initContainers cannot currently be added, removed, or updated, it is unnecessary // to mutate and validate limitrange on pod updates. Trying to mutate containers or initContainers on a pod // update request will always fail pod validation because those fields are immutable once the object is created. if a.GetKind().GroupKind() == api.Kind("Pod") && a.GetOperation() == admission.Update { return false } </ins> return a.GetKind().GroupKind() == api.Kind("Pod") || a.GetKind().GroupKind() == api.Kind("PersistentVolumeClaim") }

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: Pod stuck in terminating state. What you expected to happen: Pod successfully cleaned up. How to reproduce it (as minimally and precisely as possible): up a LimitRange. (Say "limits: defaultRequest: cpu: 100m type: Container") a mutating webhook on pod which creates an additional container on the pod (Eg. ) without the limit range. a Pod with foreground deletion finalizer. that you have a Pod with 2 containers. The 1 you created which now has a limit and the istio container which does not. to delete the Pod. garbage collector tries to update the pod to remove the finalizer, however this triggers the limit range admission control to add the limit to the istio container. This then causes the API server to get the error "invalid: spec: Forbidden: pod updates may not change fields other than , , or " the pod is now stuck in terminating. Anything else we need to know?: Environment: Kubernetes version (use ): 1.9.6 /sig api-machinery
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale PR open


Question: controller.classLister = p.ClassInformer.Lister() controller.classListerSynced = p.ClassInformer.Informer().HasSynced controller.podLister = p.PodInformer.Lister() <ins> controller.podIndexer = p.PodInformer.Informer().GetIndexer() </ins> controller.podListerSynced = p.PodInformer.Informer().HasSynced controller.NodeLister = p.NodeInformer.Lister() controller.NodeListerSynced = p.NodeInformer.Informer().HasSynced <ins> // This custom indexer will index pods by its PVC keys. Then we don't need // to iterate all pods every time to find pods which reference given PVC. if err := controller.podIndexer.AddIndexers(cache.Indexers{ pvcKeyIndex: indexByPVCKey, }); err != nil { return nil, fmt.Errorf("Could not initialize PersistentVolume Controller: %v", err) } </ins> csiTranslator := csitrans.New() controller.translator = csiTranslator controller.csiMigratedPluginManager = csimigration.NewPluginManager(csiTranslator)

What happened: With a StorageClass with , it's not clear why a PVC is not bound when there is no available PV. On Pod, there is a clear event what's wrong: On the PVC side we get only this event: Note that the message says "waiting for first consumer'. But the pod already exists. The reason why it's not bound yet is that the StorageClass has no dynamic provisioning configured () and there are no free PVs to satisfy the claim. What you expected to happen: An event "1 node(s) didn't find available persistent volumes to bind" on the PVC.
/sig scheduling /sig storage
I tried to fix this in VolumeBinding predicate, The predicate is called for each pod - node combination, so it would generate event for each node. Which is too much and it's even wrong when there is a node that has an available PV. We need to evaluate all nodes and then, if the pod is still unschedulable, distribute events to all unbound PVCs of that pod. Scheduler does not offer any way how to track unbound PVCs at that stage - whole volume logic is hidden in VolumeBinding predicate. One option would be to extend  struct to contain also list of objects where to send events (except the pod itself): Scheduler would then send all failed scheduling events it sends to a pod to all the , without any interpretation. is used to manage duplicates easily - each object (its UID) can be in the map only once. Simple array would work too, but then it needs extra work in removing duplicates. Not sure how easy it is to detect duplicate instances. is extensible to allow any object type. predicate would put all unbound PVCs there. does not need any special handling of each PVC. Message "2 node(s) didn't find available persistent volumes to bind" sounds just right in case a pod uses multiple PVCs, each of them might be bind-able on different nodes (node A may satisfy PVC1, but not PVC2, while node B may satisfy PVC2 only). what do you think about such design?
We used to have extension point, which I believe is the right moment for recording the event for unbounds PVCs. But recently it's been refactored so doesn't fit. Do you think this case can be a good example of using as an informational extension point?

Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale wdyt?
I have some other concerns. What if the pod is deleted? Should the scheduler send events that PVC is not in volume binding? The PV controller must be updated to stop emit event in this scenario. It might be simpler to solve on the PV controller side. In this function, we can check if the PVC is being used by a pod. If it is, we can send event like , then people can go to check the progress of pod "foo/bar". If the pod "foo/bar" is deleted, the PV controller can emit "waiting for first consumer ..." event again. What do you think? cc
Sounds better than it's now. My only issue may be performance - PV controller has to check if a pod exists, i.e. list all pods in the same namespace as PVC being reconciled. I am probably too worried.
current attach detach controller has a custom indexer to index pods by PVC keys to avoid iterating all pods every time to find pods which reference given PVC. And there are other places we need to check if a PVC is used by a pod or not from pod informer, e.g. pvcprotectioncontroller pvcontroller we can try to share this custom indexer among these volume controllers in the future. This will improve the overall performance of the IMO. What do you think
Yes, using the indexer looks like a good idea.
/remove-sig scheduling I also agree that solving in PVC side seems like a better idea. Thanks
/assign unbound claims are reconciled in the PV controller every 15 seconds (default), we may have bad worst-case performance (e.g. too many pods in the same namespace). we can add an indexer in the PV controller locally to solve this issue first (also optimize ), then find a way to in manager to improve the overall performance.


Question: return "", fmt.Errorf("no plugin name provided") } <ins> if strings.ContainsRune(plugin, os.PathSeparator) { return "", fmt.Errorf("invalid plugin name: %s", plugin) } </ins> if len(paths) == 0 { return "", fmt.Errorf("no paths provided") }

is identified with with severity rating as high. This is found using snyk scan while running against k/k master We're not using directly however, it gets tagged for vulnerability scans as we use . The fix is already present in and we can bump the dep. Bump dep to which has the mentioned CVE fix. and check Kubernetes version (use ): n/a Cloud provider or hardware configuration: any OS (e.g: ): n/a Kernel (e.g. ): n/a Install tools: n/a Network plugin and version (if this is a network-related bug): n/a Others: n/a /sig architecture /area code-organization for the dependency
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: {MountPath: "/dir/nested", Name: "vol2"}, {MountPath: "/ignore2", Name: "vol5"}, {MountPath: "/dir", Name: "vol1"}, <ins> {MountPath: "/dir/nested-vol", Name: "vol6"}, {MountPath: "/dir/nested.vol", Name: "vol7"}, {MountPath: "/dir/nested2/double", Name: "vol8"}, </ins> {MountPath: "/dir/nested2", Name: "vol3"}, }, },

to have all the mount points sorted and grouped. For example, the following strings are sorted in this exact order: With the current implementation, test, we would get: This is worse on Windows, where paths usually have backslashes instead (with no test changes): [1] ` should group mount points correctly. /kind bug /sig windows Kubernetes master, Kubernetes v1.25 and older. N/A N/A <details</details<details</details<details</details
/triage accepted /milestone v1.26


Question: // A null selector selects no pods. // An empty selector ({}) also selects no pods, which differs from standard behavior of selecting all pods. // In policy/v1, an empty selector will select all pods in the namespace. <del> // +patchStrategy=replace </del> // +optional optional k8s.io.apimachinery.pkg.apis.meta.v1.LabelSelector selector = 2;

PDB was upgraded to v1 in 1.21. Despite the changelog saying that the behavior of the v1beta1 implementation should have stayed the same, it appears that the for the field was set to . I believe this is the change in question: When running with a starting config that had a with both a key and a key, and a patch that had a with just a key, the behavior between 1.20 and 1.21 differs. What we get is the key gets deleted, rather than merged with the patch's key to make one unified selector. In 1.20, the behavior we'd get is thus, which is desirable for us because it lets us inject a base assumption about our PDBs into everything we do: This is my repro case: : Compare by using the command from earlier in this report: I also brought this up in Slack: I don't think we want to switch to using just for PDBs, because that wouldn't solve issues where our users want to use without having to re-specify the from our base. Putting aside the fact that changing the behavior of the v1beta1 implementation was probably not intentional, it also seems undesirable to change how selectors work in general. I don't know how widely scoped this change will be across various resource types, too, so we're starting down quite a bit of nasty discovery time. <detailsThis is the version I used to demonstrate my repro case, but I confirmed that this occurs even in kubectl v1.21.0. For my "happy" repro case, this is what the version output looks like: </details<detailsAWS (the hard way) </detailsNo response No response No response No response
There are no sig labels on this issue. Please add an appropriate label by using one of the following commands: - Please see the for a listing of the SIGs, working groups, and committees available. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment.

<detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
as requested, I am tagging you and on this one
We should narrowly revert the strategic merge patch selector behavior for v1beta1 to match pre-1.21 behavior for backwards compatibility. More broadly, it is unexpected for multiple actors to be writing to a single selector. The pre-1.21 strategic merge patch behavior for v1beta1 PDB was already inconsistent w.r.t. merging portions of the selector and replacing others (it would merge and replace ) For that reason, server-side apply treats selector writes as owning the entire selector, and the v1 PodDisruptionBudget API will do the same for strategic merge patch. edit: reworded the inconsistencies with the < 1.21 selector handling
More broadly, it is unexpected for multiple actors to be writing to a single selector. The pre-1.21 strategic merge patch behavior for v1beta1 PDB has problems with data getting left in the selector incorrectly if the same user submits a patch with two keys, and then submits a patch with only a single key. Would that be considered a bug in the implementation, or was there some inherent, design-caused race condition in patch application that I don't understand? Cuz if I were to sequentially apply patches with the old strategic merge, I would think the selector would just accumulate keys, which seems fine to me–but I can see how a race condition would turn it into chaos
edit: I was wrong about the previous behavior... the issue was that it already inconsistently merged the selector... it would merge and replace , which is still pretty incoherent If you really want to tweak field-level things, jsonpatch can still do surgical insertion/appending/removal of specific fields.
Should we expect to use jsonpatch for anything that can have a selector, going forward? If it changed for PDBs then it should change for everything else, I would assume edit: we discussed more in slack. The answer is no, the fields for other resource types' label selectors are not changing, so we only need to treat PDBs as a special case.


Question: echo "+++ Copying release artifacts to ${gcs_destination}" # First delete all objects at the destination <del> gsutil -q rm -f -R "${gcs_destination}" >/dev/null 2>&1 || true </del> <ins> if gsutil ls "${gcs_destination}" >/dev/null 2>&1; then echo "!!! ${gcs_destination} not empty." read -p "Delete everything under ${gcs_destination}? [y/n] " -r || { echo "EOF on prompt.  Skipping upload" return } [[ $REPLY =~ ^[yY]$ ]] || { echo "Skipping upload" return } gsutil -m rm -f -R "${gcs_destination}" fi </ins> # Now upload everything in release directory gsutil -m "${gcs_options[@]+${gcs_options[@]}}" cp -r "${RELEASE_DIR}"/* "${gcs_destination}" # Having the "template" scripts from the GCE cluster deploy hosted with the # release is useful for GKE.  Copy everything from that directory up also. <del> gsutil -m "${gcs_options[@]+${gcs_options[@]}}" cp "${KUBE_ROOT}/cluster/gce/templates/*.sh" "${gcs_destination}extra/gce-templates/" </del> <ins> gsutil -m "${gcs_options[@]+${gcs_options[@]}}" cp  "${RELEASE_STAGE}/full/kubernetes/cluster/gce/templates/*.sh"  "${gcs_destination}extra/gce-templates/" # Upload the "naked" binaries to GCS.  This is useful for install scripts that # download the binaries directly and don't need tars. local platform platforms platforms=($(cd "${RELEASE_STAGE}/client" ; echo *)) for platform in "${platforms[@]}"; do local src="${RELEASE_STAGE}/client/${platform}/kubernetes/client/bin/*" local dst="${gcs_destination}bin/${platform/-//}/" # We assume here the "server package" is a superset of the "client package" if [[ -d "${RELEASE_STAGE}/server/${platform}" ]]; then src="${RELEASE_STAGE}/server/${platform}/kubernetes/server/bin/*" fi gsutil -m "${gcs_options[@]+${gcs_options[@]}}" cp  "$src" "$dst" done </ins> # TODO(jbeda): Generate an HTML page with links for this release so it is easy # to see it.  For extra credit, generate a dynamic page that builds up the

Sometimes it is nice to have non-tar'd binaries for releases. We should upload those along with the our packaged tarballs. This will help fix


Question: # # -M installs the master set +x <del> wget -q -O - https://bootstrap.saltstack.com | sh -s -- -M -X </del> <ins> curl -L --connect-timeout 20 --retry 6 --retry-delay 10 https://bootstrap.saltstack.com | sh -s -- -M -X </ins> set -x <del> echo $MASTER_HTPASSWD > /srv/salt/nginx/htpasswd </del>

When wgetting to this address during the installation process, this just fails silently. Turns out the issue is that if you take a raw and curl that address you get a TLS error. I've fixed it by changing to , but obviously that's more risky.
Related (filed after an email exchange around wget) I don't have easy access to the vSphere so I can't verify how it relates to my debian install.
Yes, that is exactly the error I was experiencing.
Does curl work? Or is it busted too?
Curl is not installed on the image by default. I'm building a new image based on Debian 7.7 where I pull in curl by default, so we can switch to using that everywhere (instead of an awkward mix of wget and curl).
Thanks for reporting this I hadn't seen this yet. I uploaded a new VMDK that includes curl and updated the deployment scripts to use curl. I verified that deployment works again for me.
I can confirm that this works now with latest build from master on ESX. Probably still won't work with v0.4.3 (even w new image) since the scripts still used wget back then.
We are probably going to fork for v0.5 soon. But I can backport if this trips folks up. sent from tiny keyboard On Nov 12, 2014 9:09 AM, "Ben Smith" wrote:


Question: klog.V(5).InfoS("Updating claims objects to trigger volume provisioning", "pod", klog.KObj(pod), "PVC", klog.KObj(claim)) newClaim, err := b.kubeClient.CoreV1().PersistentVolumeClaims(claim.Namespace).Update(ctx, claim, metav1.UpdateOptions{}) if err != nil { <ins> klog.V(4).InfoS("Updating PersistentVolumeClaim: binding to volume failed", "PVC", klog.KObj(claim), "err", err) </ins> return err } // Save updated object from apiserver for later checking. claimsToProvision[i] = newClaim lastProcessedProvisioning++

When scheduling pod, we encountered the following event , then the scheduler will backoff to reschedule again, but this add more waiting time. From the event message, we can see it is the PVC update conflict leads to the , so need to add retry logic on PVC updating.### Need to add retry logic on PVC updating, to avoid the backoff, to improve the scheduling efficiency. a PVC and then create Pod; scheduing; the PVC annotation or label before the Pod scheduled success; will encounter this issue. No response <details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig storage note that this is not a supported client/server skew: Server Version: version.Info{Major:"1", Minor:"18+", GitVersion:"v1.18.4-tke.23", GitCommit:"", GitTreeState:"clean", BuildDate:"2022-03-15T09:25:31Z", GoVersion:"go1.15.10", Compiler:"gc", Platform:"linux/amd64"}
/sig scheduling
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle rotten [1]:
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Reopen this issue with Mark this issue as fresh with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /close not-planned [1]:
Closing this issue, marking it as "Not Planned". <detailsIn response to : The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closedReopen this issue with Mark this issue as fresh with Offer to help out with [Issue Triage][1] Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: var newHnsEndpoint *hcsshim.HNSEndpoint hnsNetworkName := proxier.network.name var err error <ins> // targetPort is zero if it is specified as a name in port.TargetPort, so the real port should be got from endpoints. // Note that hcsshim.AddLoadBalancer() doesn't support endpoints with different ports, so only port from first endpoint is used. // TODO(feiskyer): add support of different endpoint ports after hcsshim.AddLoadBalancer() add that. if svcInfo.targetPort == 0 { svcInfo.targetPort = int(ep.port) } </ins> if len(ep.hnsID) > 0 { newHnsEndpoint, err = hcsshim.GetHNSEndpointByID(ep.hnsID) }

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . If the matter is security related, please disclose it privately via --Is this a BUG REPORT or FEATURE REQUEST?: What happened: Tried to create a service using targetPort as a string referencing the name of the pod port. Traffic was not routed to pod. What you expected to happen: Service to route traffic to pod How to reproduce it (as minimally and precisely as possible): See line targetPort Anything else we need to know?: Link that describes how this feature is supposed to work Environment: Kubernetes version (use ): Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.2", GitCommit:"", GitTreeState:"clean", BuildDate:"2018-04-27T09:22:21Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"windows/amd64"} Server Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.0", GitCommit:"", GitTreeState:"clean", BuildDate:"2018-09-27T16:55:41Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"} Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): WindowsBuildLabEx : .1000.amd64fre.rs_prerelease.-1413 Kernel (e.g. ): Install tools: Others:
/sig windows
/assign


Question: up{8}, down{oldReady: 4, newReady: 8, to: 2}, up{10}, <del> down{oldReady: 10, newReady: 2, to: 0}, </del> <ins> down{oldReady: 1, newReady: 10, to: 0}, </ins> }, output: `Created foo-v2 Scaling up foo-v2 from 5 to 10, scaling down foo-v1 from 6 to 0 (keep 10 pods available, don't exceed 12 pods)

I intentionally cause the rolling update to fail by specifying a non-existent image in the spec, so that it times out. rollback then fails with a timeout as well, despite the original replication controller having 1 healthy replica: This is on 1.1.2. I expect the rollback to succeed as the original rc (copper) has 1 healthy replica still.
With 2 replicas the rollback seems to work as expected.
I found the bug inside of the function of . It erroneously assumes reducing replicas on scale down will always drop the minimum available, when this isn't the case if you are rolling back from a failed deployment (where the scale down will be on the NotReady/failed pods). It needs to consider not ready replicas as well in each RC when deciding if it can scale down. This test case illustrates the problem:


Question: os.MkdirAll(dir, 0750) var options []string <ins> options = append(options, "allow-usermapping-in-volumename") </ins> if mounter.readOnly { options = append(options, "ro") }

What happened: Containers cannot access Quobyte volumes provided via a Quobyte mount run by the Quobyte plugins mount command due to permission denied issues. Even if user mappings are provided these are ignored by the Quobyte client due to a missing mount flag. What you expected to happen: Access should work with the provided user mapping. How to reproduce it (as minimally and precisely as possible): Quobyte volumes without providing an externally managed plugin mount, thus allowing the Quobyte plugin to run the mount command. Quobyte based StorageClass, PVC and pod desc. the pod Anything else we need to know?: 'm preparing a pull request with a fix is to run an externally managed mount for the plugin, e.g. via a systemd unit or a k8s daemonset. Ensure that mount activates the allow-usermapping-in-volumename option. In this case the plugin uses the existing mount. Environment: not relevant. Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Others: /kind bug /sig storage
/assign
/milestone v1.14 We're entering and this looks relevant to the 1.14 release. Please if I am incorrect
Hello , I'm a bug triage shadow and I'd like to remind that code freeze is coming in 2 days (Friday, 8th March). :snowflake: Is it still planned for this issue to be fixed for 1.14?
I'm happy to merge this asap, ideas on how to bring this forward?


Question: framework.ExpectEqual(cur.Revision, int64(2)) checkDaemonSetPodsLabels(listDaemonPods(c, ns, label), hash) }) <ins> ginkgo.It("should list and delete a collection of DaemonSets", func() { label := map[string]string{daemonsetNameLabel: dsName} labelSelector := labels.SelectorFromSet(label).String() dsClient := f.ClientSet.AppsV1().DaemonSets(ns) cs := f.ClientSet one := int64(1) ginkgo.By(fmt.Sprintf("Creating simple DaemonSet %q", dsName)) testDaemonset, err := c.AppsV1().DaemonSets(ns).Create(context.TODO(), newDaemonSetWithLabel(dsName, image, label), metav1.CreateOptions{}) framework.ExpectNoError(err) ginkgo.By("Check that daemon pods launch on every node of the cluster.") err = wait.PollImmediate(dsRetryPeriod, dsRetryTimeout, checkRunningOnAllNodes(f, testDaemonset)) framework.ExpectNoError(err, "error waiting for daemon pod to start") err = checkDaemonStatus(f, dsName) framework.ExpectNoError(err) ginkgo.By("listing all DeamonSets") dsList, err := cs.AppsV1().DaemonSets("").List(context.TODO(), metav1.ListOptions{LabelSelector: labelSelector}) framework.ExpectNoError(err, "failed to list Daemon Sets") framework.ExpectEqual(len(dsList.Items), 1, "filtered list wasn't found") ginkgo.By("DeleteCollection of the DaemonSets") err = dsClient.DeleteCollection(context.TODO(), metav1.DeleteOptions{GracePeriodSeconds: &one}, metav1.ListOptions{LabelSelector: labelSelector}) framework.ExpectNoError(err, "failed to delete DaemonSets") ginkgo.By("Verify that ReplicaSets have been deleted") dsList, err = c.AppsV1().DaemonSets("").List(context.TODO(), metav1.ListOptions{LabelSelector: labelSelector}) framework.ExpectNoError(err, "failed to list DaemonSets") framework.ExpectEqual(len(dsList.Items), 0, "filtered list should have no daemonset") }) </ins> }) // randomPod selects a random pod within pods that causes fn to return true, or nil

[x] test approval issue : [X] test pr : [x] two weeks soak end date : 7 June 2021 [x] test promotion pr : kubernetes/kubernetes According to this APIsnoop query, there are still some remaining ReplicaSet endpoints which are untested. Create a Daemonset List all DaemonSets DeleteCollection of DaemonSetss Varify the DaemonSets have been deleted Using an existing conformance test as a template for a which validates that two new endpoints are hit. example endpoint | useragent -------------------------------------------|
createAppsV1NamespacedDaemonSet | (linux/amd64) kubernetes/$Format -- [sig-apps] Daemon set [Serial] should list and delete a collection of DaemonSets deleteAppsV1CollectionNamespacedDaemonSet | (linux/amd64) kubernetes/$Format -- [sig-apps] Daemon set [Serial] should list and delete a collection of DaemonSets listAppsV1DaemonSetForAllNamespaces | (linux/amd64) kubernetes/$Format -- [sig-apps] Daemon set [Serial] should list and delete a collection of DaemonSets listAppsV1NamespacedDaemonSet | (linux/amd64) kubernetes/$Format -- [sig-apps] Daemon set [Serial] should list and delete a collection of DaemonSets readAppsV1NamespacedDaemonSet | (linux/amd64) kubernetes/$Format -- [sig-apps] Daemon set [Serial] should list and delete a collection of DaemonSets (5 rows) If a test with these calls gets merged, test coverage will go up by 2 points This test is also created with the goal of conformance promotion. /sig apps /sig testing /sig architecture /area conformance
/triage accepted For Release 1.22
/open
/reopen
Reopened this issue. <detailsIn response to : /reopen </details


Question: podSubnet: "" serviceSubnet: 10.96.0.1/12 nodeName: "" <ins> privilegedPods: false </ins> token: "" unifiedControlPlaneImage: "" `),

Dear All: I'm not sure if I'm right. When i use whith .I found controller-manager logs Someone can help me? Thanks a lot.
There are no sig labels on this issue. Please by:<br(1) mentioning a sig: <br(2) specifying the label manually: <br<brNote: method (1) will trigger a notification to the team. You can find the team list and label list
Reiterating the mentions to trigger a notification: <detailsIn response to : Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Please give more information about your set up cc
Sorry for report later. My openstack didn't support metadata service,but support config-drive service. More info:
Thanks Based on this, I think it's an openstack cloudprovider problem more than a kubeadm problem...
Thank you for help me. I will change the label.
/area cloudprovider
/remove-kind support
Should be fixed with you can specify a search order for metadata and config drive /close
Specifying a search order doesn't really solve the problem, especially if the metadata api isn't available which is what ZhenyangZhao said . I'm getting the same issue and since the config drive is my only option at the moment.
Like for this to be /open
can you please provide logs/versions for your problem in your environment? Thanks, Dims
Let me know if you need anything else Kubernetes version: 1.8.4 Docker version: 17.09.0-ce Linux: CentOS 7.4 Looking at the docker container log for the api-server (I don't have problems with the controller-manager):
and share the following:
Hi! Thank you for filing this issue. We appreciate your interest in Kubernetes and kubeadm. In order to make the kubernetes/kubernetes issue tracker more usable, we're routing issues to different repositories with different focuses. I think this issue belongs in the kubeadm repository. Please move this issue to the in order for the kubeadm team to be able to track this efficiently. If you disagree with my assessment of moving this issue, please speak up and/or reopen the issue. Finally, please try to reproduce the bug with the latest Kubernetes version, it might as well already have been fixed :) Thank you and have a good day!


Question: t.Error("Unexpected error creating custom resource but metadata validation rule") } }) <ins> t.Run("CRD creation MUST pass for an CRD with empty field", func(t *testing.T) { structuralWithValidators := crdWithSchema(t, "WithEmptyObject", structuralSchemaWithEmptyObject) _, err := fixtures.CreateNewV1CustomResourceDefinition(structuralWithValidators, apiExtensionClient, dynamicClient) if err != nil { t.Errorf("unexpected error creating CRD with empty field: %v", err) } }) </ins> t.Run("CR creation MUST fail if a x-kubernetes-validations rule exceeds the runtime cost limit", func(t *testing.T) { structuralWithValidators := crdWithSchema(t, "RuntimeCostLimit", structuralSchemaWithCostLimit) crd, err := fixtures.CreateNewV1CustomResourceDefinition(structuralWithValidators, apiExtensionClient, dynamicClient)

While migrating webhooks to CEL for one of the structs in our CRD api, I noticed that the empty object in CEL is interpreted wrongly. Empty map object seems to be interpreted correctly. In the example below, both map and object are sent empty () in json output. returns true json (captured on the client side) sent to the apiserver CRD schema No response <details</details<details</details<details</details<details</details<details</details<details</details
/sig api-machinery
CEL only sees the JSON sent to the server, and is correctly evaluated as present. The issue is with the client sending the request populating a value in that field. How is the request being constructed? /remove-kind bug /kind support
/cc /triage accepted
Upon investigation, the problem lies there: We requires either or be set.
Thanks for raising this issue! I was debugging offline with and seems like CRD/CR validation didn't handle the situation of unsetting properties with field type specified . It is a corner case in CRD since we normally would ask for properties defined if the field type is (I could not think of use case which having properties unset). But it might ok to allow it through since it won't hurt from CRD perspective. We will fix in CEL values handling which treat this as empty map passed. /assign
GitHub didn't allow me to assign the following users: jiahui. Note that only with read permissions, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : Thanks for raising this issue! I was debugging offline with and seems like CRD/CR validation didn't handle the situation of unsetting properties with field type specified 'Object1object`(I could not think of use case which having properties unset). But it might ok to allow it through since it won't hurt from CRD perspective. We will fix in CEL values handling which treat this as empty map passed. /assign </details
also check if the CRD validation allows the or values, and if so, if the CEL code handles those properly as well
FYI: We do not allow in crd structured schema now:


Question: <del> linux/amd64=fedora:26 linux/arm64=arm64v8/fedora:26 linux/ppc64le=ppc64le/fedora:26 </del> <ins> linux/amd64=fedora:33 linux/arm64=arm64v8/fedora:33 linux/ppc64le=ppc64le/fedora:33 </ins>

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --When I run some e2e storage testcases locally, below testcases as an example, I consistently saw the below error message, ceph should be deployed successfully and testcase should pass. Run the testcases mentioned above. e.g. Kubernetes version (use ): ALL Cloud provider or hardware configuration: ARM64 OS (e.g: ): ubuntu18 Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig testing /sig storage
/assign
is this happening in CI? please provide links to occurences?
Hey, Ben, I update the issue description a little bit, this won't be an issue on X86 which is currently the default arch CI is running, but tons of testcases will fail on ARM64 due to this issue. Eventually, we hope the k8s will have a ARM CI along with the more adoption of arch beside the X86, and thus, those e2e testcases should workable on ARM64 firstly. I will explain why it failed on ARM64 in the PR desc, pls help to review, thanks!
/triage accepted


Question: local cmd for cmd in gcloud gsutil; do if ! which "${cmd}" >/dev/null; then <del> local resp </del> <ins> local resp="n" </ins> if [[ "${KUBE_PROMPT_FOR_UPDATE}" == "y" ]]; then echo "Can't find ${cmd} in PATH.  Do you wish to install the Google Cloud SDK? [Y/n]" read resp <del> else resp="y" </del> fi if [[ "${resp}" != "n" && "${resp}" != "N" ]]; then curl https://sdk.cloud.google.com | bash

<!-- Thanks for filing an issue! Before hitting the button, please answer these questions.--Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see ): What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.): Is this a BUG REPORT or FEATURE REQUEST? (choose one): FEATURE REQUEST <!-- If this is a BUG REPORT, please: Fill in as much of the template below as you can. If you leave out information, we can't help you as well. If this is a FEATURE REQUEST, please: Describe in detail the feature/behavior/change you'd like to see. In both cases, be ready for followup questions, and please respond in a timely manner. If we can't reproduce a bug or think a feature already exists, we might close your issue. If we're wrong, PLEASE feel free to reopen it and explain why. --Kubernetes version (use ): all :smile: Environment: Cloud provider or hardware configuration: GCE OS (e.g. from /etc/os-release): all Kernel (e.g. ): all Install tools: Others: What happened: uses the cli a lot and does a (in the commonly used function ) before doing things like . updates can introduce breaking changes (for example in the recent 132.0.0 release it removed a bunch of flags). This has caused cross-the-board test failures like . But this is not limited to just tests. Any kubernetes GCE users who rely on the and commands could have been broken. What you expected to happen: Stop updating gcloud automatically. We could either warn user to have the required dependencies installed, or pin to specific versions of dependencies like gcloud. How to reproduce it (as minimally and precisely as possible): download 1.4.4 release tarball run and Anything else do we need to know:
cc
/cc
cc I think it's reasonable to remove gcloud update calls from scripts, as I believe gcloud is updated out of band by k8s continuous testing. The scripts will still need to work with latest, so maybe they should check for a minimum version.
Good idea.
Yeah, we definitely want to continue using latest in CI so that we know if things break, but I think it's reasonable to remove the update itself from the kubernetes cluster util scripts.


Question: local -a nonstatics=() local -a tests=() for binary in "${binaries[@]}"; do <del> if [[ "${binary}" =~ ".test"$ ]]; then </del> <ins> # TODO(IBM): Enable hyperkube builds for ppc64le again # The current workaround creates a text file with help text instead of a binary # We're doing it this way so the build system isn't affected so much if [[ "${binary}" == *"hyperkube" && "${platform}" == "linux/ppc64le" ]]; then echo "hyperkube build for ppc64le is disabled. Creating dummy text file instead." local outfile=$(kube::golang::output_filename_for_binary "${binary}" "${platform}") mkdir -p $(dirname ${outfile}) echo "Not available at the moment. Please see: https://github.com/kubernetes/kubernetes/issues/25886 for more information." > ${outfile} elif [[ "${binary}" =~ ".test"$ ]]; then </ins> tests+=($binary) elif kube::golang::is_statically_linked_library "${binary}"; then statics+=($binary)

disabled PPC builds due to the following error: Additional comments here
cc
The issue on PowerPC is both for cross and native-builds. The issue appears to be with the toolchain (golang) and how the linking is being done. The object sections have grown too large to link. We are looking at it cc
Have you tried to upgrade to a newer version? It might or might not help... You might also try to look at what in the gcc linker differs from for example, where it doesn't fail
We are looking into this and it might take some more time. In the meanwhile is it possible to remove hyperkube from the ppc64le builds? All other binaries are building perfectly fine. We can resume the hyperkube for ppc64le build as soon as this issue resolves. cc:
Ah, this is only partially fixed. Reopening...
The error occurs because the hyperkube binary is very large and when building the binary, golang is generating a single go.o file with a single text section that is larger than 2^26 on ppc64le. When there is a call instruction within that section where the call target offset is beyond what can fit in the instruction the relocation truncation error will be generated by the GNU linker. When there are multiple, smaller text sections sent to the GNU linker for linking, it can generate appropriate call instructions even the call target is beyond that limit. I'm working on a fix to do that but hitting other issues as a result which I'm working through. I suspect that this same error will be hit at some point on arm64 unless their linker has another way to handle this situation. On ppc64le we are very close to the limit and from build to build it varies so that some days it will build and others it won't.
Interesting. How does amd64 handle it?
I don't know for sure how arm64 handles it because I don't have access to a machine that is set up to build for arm64. I suspect their text section size is not as large as it is on ppc64le so they haven't hit the limit yet.
Ok, you may easily see what's going on for arm64 by investigating what does in kube-cross.

But I asked about what amd64 (x86_64) does to prevent this :) (I'm assuming this will never happen to amd64)
Sorry I misread your question about amd64... The call instruction in amd64 has a 32 bit field instead of a 24 bit field in ppc64le for the call target address. That means they wouldn't hit this error when linking hyperkube until its text section was larger than 2^32. If you search on "relocation truncated to fit" you will see that this error has happened in the past on x86 with other programming languages.
Once a fix is ready, how does it need to be provided?
I saw https://go- probably won't make 1.7, and that's sad. We would just upgrade golang version I think, not really sure what you're asking for
I assume you only use official golang versions or those provided with distros for your builds? Are you doing all your builds as cross builds from a golang on an x86?
Official versions, yes. We're using the docker image. All official builds are built on amd64 hosts.
Where are you getting this docker image from? Will you be moving to golang:1.7 when it is released?
We're using the official images, and will pick up as soon as it is released for testing I think. Then if the testing and scalability teams think it's safe to use, we'll switch to 1.7 BTW, where is your patch for this? Can you link to it?
is also broken right now - see .
Hi, I'm sorry I missed your last question to me. Here is the CL for my patch for the PPC64 relocations truncation error. https://go- The error with kubemark is also size related but somewhat different because that one is being built as static. Just curious, why are some built static and some non-static?
just to tie everything together, disabled linux/ppc64le compilation entirely. hopefully by go1.8 we can reenable it.
Well, we've hit 1.8 I think with Want to link to any other relevant issues to kinda tie this one together, and close this if it seems appropriate?
yeah, ppc64le builds have been re-enabled for k8s 1.6.

The issue on PowerPC is both for cross and native builds. The issue appears to be with the toolchain (golang) and how the linking is being done. The object sections have grown too large to link. We are looking at it cc
this issue is quite old. Can you provide more context to your comment? Specifically, more detail to help me reproduce. I was unable to find anything related to ppc64le on .


Question: echo cat "${LICENSE_ROOT}/LICENSE" echo <del> echo "= LICENSE $(md5sum < "${LICENSE_ROOT}/LICENSE" | awk '{print $1}')" </del> <ins> echo "= LICENSE $(kube::util::md5 "${LICENSE_ROOT}/LICENSE")" </ins> echo "================================================================================" ) > ${TMP_LICENSE_FILE}

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: run on macOS, all md5 hashes will be stripped from the output and the script will not fail or warn. What you expected to happen: The script should work or detect this issue and fail. Currently it silently ignores not existing on the host. We have some warning logic, but solely based on bash version, if the user upgraded their bash installation but did not install md5sum we don't warn them. How to reproduce it (as minimally and precisely as possible): run the script Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others: /sig testing /priority important-longterm /assign
cc


Question: MUST cause pods created by that RC to be orphaned. release: v1.9 file: test/e2e/apimachinery/garbage_collector.go <ins> - testname: Namespace, apply finalizer to a namespace codename: '[sig-api-machinery] Namespaces [Serial] should apply a finalizer to a Namespace [Conformance]' description: Attempt to create a Namespace which MUST be succeed. Updating the namespace with a fake finalizer MUST succeed. The fake finalizer MUST be found. Removing the fake finalizer from the namespace MUST succeed and MUST NOT be found. release: v1.26 file: test/e2e/apimachinery/namespace.go </ins> - testname: Namespace, apply update to a namespace codename: '[sig-api-machinery] Namespaces [Serial] should apply an update to a Namespace [Conformance]'

[X] Test approval issue: [X] Test PR: 15 Oct 2022 [X] Two weeks soak end date: 28 Oct 2022 [X] Test promotion PR: According to this APIsnoop query, there are still one remaining Namespace endpoint which is untested. replaceCoreV1NamespaceFinalize Note: The test also removes the fake finalize from before calling the finalize action on the test namespace a second time. This will allow the test framework to clean up all namespaces when the e2e finishes, else the test namespace will be stuck in the state. Using a number of existing e2e test practices a new has been created for one Namespace endpoint. The e2e logs for this test are listed below. This query shows that the Finalize endpoint is hit within a short period of running this e2e test If a test with these calls gets merged, test coverage will go up by 1 point This test is also created with the goal of conformance promotion. /sig testing /sig architecture /area conformance
/triage accepted
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: driverName: driverName, } if shouldTimeout { <del> timeout := plugin.PluginClientTimeout + 10*time.Millisecond </del> <ins> timeout := plugin.PluginClientTimeout + 30*time.Millisecond </ins> fakeDRADriverGRPCServer.timeout = &timeout }

test Since Oct 25th No response stack trace: Test has been flaking with master golang as well. /sig node
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
seems like we need to increase the timeout.. /assign
I tried with different values of timeout in function viz and running test binary using stress tool. In most cases, test is getting passed.
Flake in master blocking board: /cc
Flake in as well.
is waiting for approve been a while, which try to fix the flake
We can still observe this flakiness with 30ms timeout in CI Wondering, if timeout has to be increased further. Please let us know if more information is required.
It flaked in today as well. /reopen
Reopened this issue. <detailsIn response to : It flaked in today as well. </details
We're seeing this flake frequently downstream as well, both in TestUnprepareResources and TestPrepareResources.
cc
is trying to fix it but still pending


Question: } topoKeys.Insert(key) <del> for _, msg := range validation.IsQualifiedName(key) { allErrs = append(allErrs, field.Invalid(fldPath, driver.TopologyKeys, msg)) } </del> <ins> allErrs = append(allErrs, apivalidation.ValidateQualifiedName(key, fldPath)...) </ins> } return allErrs

After got merged, we have new helper function here: It will be good for us to use this simpler helper function to replace previous function. /good-first-issue
This request has been marked as suitable for new contributors. Please ensure the request meets the requirements listed . If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : After got merged, we have new helper function here: </details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig usability
I would like to take this one /assign
I worked in on this as well, but kept your changed files untouched (as written in my PR description). same for you. I hope this is OK for you.
This should be reopened until and got merged (my PR excluded the changes from the other PRs).
/reopen per last comment
Reopened this issue. <detailsIn response to : This should be reopened until and got merged (my PR excluded the changes from the other PRs). per last comment Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
can you assign me
/assign
/assign
I'd love to work on this issue.

/assign
Hi, , I am a first commitor for kubernetes and I have committed a PR in , could you help me to review it, or give me some suggestions. Thanks a lot.
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle rotten [1]:
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Reopen this issue or PR with Mark this issue or PR as fresh with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /close [1]:
Closing this issue. <detailsIn response to : The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closedReopen this issue or PR with Mark this issue or PR as fresh with Offer to help out with [Issue Triage][1] Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: return nil, nil } <ins> // getIPFromLoopbackInterface gets the IPs on a loopback interface and returns a global unicast address, if any. // The loopback interface must be up, the IP must in the family requested, and the IP must be a global unicast address. func getIPFromLoopbackInterface(forFamily AddressFamily, nw networkInterfacer) (net.IP, error) { intfs, err := nw.Interfaces() if err != nil { return nil, err } for _, intf := range intfs { if !isInterfaceUp(&intf) { continue } if intf.Flags&(net.FlagLoopback) != 0 { addrs, err := nw.Addrs(&intf) if err != nil { return nil, err } klog.V(4).Infof("Interface %q has %d addresses :%v.", intf.Name, len(addrs), addrs) matchingIP, err := getMatchingGlobalIP(addrs, forFamily) if err != nil { return nil, err } if matchingIP != nil { klog.V(4).Infof("Found valid IPv%d address %v for interface %q.", int(forFamily), matchingIP, intf.Name) return matchingIP, nil } } } return nil, nil } </ins> // memberOf tells if the IP is of the desired family. Used for checking interface addresses. func memberOf(ip net.IP, family AddressFamily) bool { if ip.To4() != nil {

What happened: When attempting to kubeadm join a control plane node or worker on a machine with global IPv4 unicast addresses on a loopback address (router to the host w/ BGP Unnumbered), the join fails with the following output: See this issue for details : Also: What you expected to happen: The correct IPv4 address should be selected from the loopback adapter, or we should be able to specify an IPv4 address to use for egress. The IPv6 addresses on the network adapters themselves are only used to establish upstream peering. IPv4 nexthops are fake 169.254.x.x addresses artificially programmed via RFC5549 into the ARP table. How to reproduce it (as minimally and precisely as possible): On a node with the following conditions: Routing to the host with FRR IPv6 link locals on physical network interfaces No IPv4 addresses on physical network interfaces Unicast IPv4 on the loopback adapter IPv4 default route present via RFC5549 Attempt to kubeadm join a node, and it will fail. Anything else we need to know?: We've been patching kubeadm with , which works as expected. There was a fix applied to allow initing control plane nodes w/ bind address 127.0.0.1 which would then select an IPv4 from the loopback, but it doesn't appear to have fixed the joining issues. Pinging and by request on kubeadm issues. Environment: Kubernetes version (use ): 1.17.12 Cloud provider or hardware configuration: baremetal OS (e.g: ): Centos 7 Kernel (e.g. ): 3.10.0-1127.19.1.el7.x86_64 Install tools: kubeadm Network plugin and version (if this is a network-related bug): Calico Others: FRR for peering with routers on physical network interfaces
/sig api-machinery
is defined but we should note the same function is used for the --bind-address of the kube-apiserver. /sig network
/assign
/triage accepted


Question: config: "kind: KubeSchedulerConfiguration", expErr: "unexpected config type", }, <ins> { name:   "Missing quotes around :: bindAddress", config: "bindAddress: ::", expErr: "mapping values are not allowed in this context", }, </ins> } version := "apiVersion: componentconfig/v1alpha1" for _, tc := range testCases {

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: /kind feature What happened: Upon code inspection, found that kube-proxy server takes into consideration a bind address of 0.0.0.0 (IPv4), but does not check for a bind address of :: (all-zeros IPv6) when calculating its node IP. What you expected to happen: IPv6 address "::" should be taken into consideration when calculating node IP. How to reproduce it (as minimally and precisely as possible): Code inspection. Anything else we need to know?: Environment: Kubernetes version (use ): Latest master Cloud provider or hardware configuration**: VirtualBox cluster, Ubuntu 16.04.1 host, CentOS 7 guests OS (e.g. from /etc/os-release): CentOS 7 Kernel (e.g. ): 3.10.0-514.26.2.el7.x8664 SMP Tue Jul 4 15:04:05 UTC 2017 x8664 x8664 x8664 GNU/Linux Install tools: Others:
There are no sig labels on this issue. Please by: a sig: e.g., to notify the contributor experience sig, OR the label manually: e.g., to apply the label Note: Method 1 will trigger an email to the group. You can find the group list and label list . The in the method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals
/area ipv6 /sig network


Question: // ScaleApplyConfiguration constructs an declarative configuration of the Scale type for use with // apply. func Scale() *ScaleApplyConfiguration { <del> return &ScaleApplyConfiguration{} </del> <ins> b := &ScaleApplyConfiguration{} b.WithKind("Scale") b.WithAPIVersion("extensions/v1beta1") return b </ins> } // WithKind sets the Kind field in the declarative configuration to the given value

Most of the FooApplyConfiguration constructors pre-fill the Kind and APIVersion fields to make using them smooth. But () is missing this, making patterns like this fail: Currently this code fails with (server-side) error . The constructor should set the minimum required fields for the resulting object to be usable with calls, similar to how e.g. the `D Create a deployment (however) Create a clientset, and get the Deployments client from it Try to call deployments.ApplyScale using a ScaleApplyConfiguration created by the Scale() constructor I can see that the presence or absence of these metadata fields is driven by some code generation, but I wasn't able to follow through to what configures that code generation. I assume it's something in the schemas somewhere, but that wasn't clear to me. <details</details<detailsminikube v1.25.1 </detailsN/A N/A N/A N/A
/sig api-machinery
/assign /triage accepted
/assign


Question: } } <ins> // test GetMetrics with a volume that does not support stats func TestGetMetricsDriverNotSupportStats(t *testing.T) { tests := []struct { name          string volumeID      string targetPath    string expectSuccess bool }{ { name:          "volume created by simple driver", expectSuccess: true, volumeID:      "foobar", targetPath:    "/mnt/foo", }, } for _, tc := range tests { metricsGetter := &metricsCsi{volumeID: tc.volumeID, targetPath: tc.targetPath} metricsGetter.csiClient = &csiDriverClient{ driverName: "com.simple.SimpleDriver", nodeV1ClientCreator: func(addr csiAddr) (csipbv1.NodeClient, io.Closer, error) { nodeClient := fake.NewNodeClientWithVolumeStats(false /* VolumeStatsCapable */) fakeCloser := fake.NewCloser(t) nodeClient.SetNodeVolumeStatsResp(getRawVolumeInfo()) return nodeClient, fakeCloser, nil }, } metrics, err := metricsGetter.GetMetrics() if err == nil { t.Fatalf("for %s: expected error, but got nil error", tc.name) } if !volume.IsNotSupported(err) { t.Fatalf("for %s, expected not supported error but got: %v", tc.name, err) } if metrics != nil { t.Fatalf("for %s, expected nil metrics, but got: %v", tc.name, metrics) } } } </ins> func getRawVolumeInfo() *csipbv1.NodeGetVolumeStatsResponse { return &csipbv1.NodeGetVolumeStatsResponse{ Usage: []*csipbv1.VolumeUsage{

What happened: kubelet crashed due to nil pointer dereference inside calcAndStoreStats function in Jul 03 06:50:42 kubelet[4440]: E0703 06:50:42. 4440 ] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference) Jul 03 06:50:42 kubelet[4440]: goroutine 3224 [running]: Jul 03 06:50:42 kubelet[4440]: (0x3992120, 0x773bdd0) Jul 03 06:50:42 kubelet[4440]: +0x7b Jul 03 06:50:42 kubelet[4440]: (0x0, 0x0, 0x0) Jul 03 06:50:42 kubelet[4440]: +0x82 Jul 03 06:50:42 kubelet[4440]: panic(0x3992120, 0x773bdd0) Jul 03 06:50:42 kubelet[4440]: +0x1b5 Jul 03 06:50:42 kubelet[4440]: (volumeStatCalculator).parsePodVolumeStats(0xc00178ef00, 0xc000c96cc0, 0xe, 0xc0014b8880, 0x0, 0xc000c96cc0, 0xe, 0x0, 0x0, 0x0, .) Jul 03 06:50:42 kubelet[4440]: +0x73 Jul 03 06:50:42 kubelet[4440]: (volumeStatCalculator).calcAndStoreStats(0xc00178ef00) Jul 03 06:50:42 kubelet[4440]: +0x753 Jul 03 06:50:42 kubelet[4440]: (volumeStatCalculator).StartOnce.func1.

1() Jul 03 06:50:42 kubelet[4440]: +0x2a Jul 03 06:50:42 kubelet[4440]: (0xc001c02d10) Jul 03 06:50:42 kubelet[4440]: +0x54 Jul 03 06:50:42 kubelet[4440]: (0xc001c02d10, 0xdf8475800, 0x3ff0000000000000, 0xc000f1dd01, 0xc0012c37a0) Jul 03 06:50:42 kubelet[4440]: +0xf8 Jul 03 06:50:42 kubelet[4440]: created by (volumeStatCalculator).StartOnce.func1 Jul 03 06:50:42 kubelet[4440]: +0x9d I think this happens because GetMetrics function can return nil for both metric(), and error, but calcAndStoreStats expects error to be not nil when metric is nil() What you expected to happen: Skip calculating stats for volumes that do not support volume stats. How to reproduce it (as minimally and precisely as possible): Create a volume using a csi driver that does not support volume stats. Anything else we need to know?: Environment: Kubernetes version (use ): Server version 15.0 Cloud provider or hardware configuration: OS (e.g: ): Ubuntu 16.04 LTS Kernel (e.g. ): 4.4.0-141-generic Install tools:kubeadm Network plugin and version (if this is a network-related bug): Others: Tested using synology-csi driver()
Sounds like you have a PR ready. Mind letting us know the PR number ? Thanks
I do not have a PR yet, but happy to make one. I'll add the PR number once I created it.
I created a PR and linked to this issue.
Didnt this solve this issue ?
No, it does not fix the problem that metric itself can be nil. Also, my pull request fixes the problem that csi_metrics returns nil value for error and the caller expects non nill value when csi driver does not support metrics.


Question: "path/filepath" "strings" "sync" <ins> "time" </ins> "github.com/golang/glog" <ins> "k8s.io/kubernetes/pkg/util/wait" </ins> ) // DockerConfigJson represents ~/.docker/config.json file info

http://pr-
Something holding the port already?
The previous masterless case just barely passed with a 12.5 second timeout (25 tries at 0.5 second interval) I think we just need to increase the timeout.
I randomly sampled 5 green builds and they all have this test (starting kubelet in masterful/less mode) finished in 3s or less. Judging for the successful masterless mode startup before the failure, kubelet spent the most time trying to register credential providers (a 10s gap in the log). The same roughly applies to the failed "masterful mode", based on where the log message stopped. From , the registering process basically tries to access the metadata url to see if the credential provider is enabled. I think this may just be the slow response from the metadata server.
I am wondering even if this issue might be related to a slow response from the metadata server, why it does not show "Registering credential provider: .dockercfg" always in failed logs?
If there is a possibility that ReadUrl in may take longer than expected, would it be preferred to use a Timeout (0) with the http client to not get stuck longer than expected?
I am unsure why it didn't show that, but even in the passing round we see a 10 second gap, which almost maxed out the timeout. I think adding a timeout is reasonable, but I am not sure what value is reasonable given that the previous request took 10s. do you have any idea?
Even I am not sure exactly what should be a reasonable value. My guess would be that if in normal cases, average response time is x seconds (perhaps a guestimate could be computed based on "normal" successful tests), then x could be a starting point, or otherwise another option is choosing something randomly (like 5s). And I think that if at some point, the server is responding in 10s, and even though the test passes, its not normal and something is going wrong.
uses a 10 second timeout, but the request will be retried. Given that we don't retry this request, a short timeout is worrisome. Maybe we should add both retry and timeout in this case.
SGTM.
I reopen this one since pr was reverted.

Noticed a similar hang in: The logs are a bit different from those observed in
if kubelet couldn't start up properly, the test itself should've timed out and logged "Timed out waiting for kubelet to answer at http://127.0.0.1:/healthz; tried 25 waiting 0.5 between each". I am not sure if this failure is related.
failure is not relevant to this issue. The most recent report of such flake was over a week ago. I think the cause is the slow metadata url response in GCE, and that should've been fixed by the recent rollout. I am going to close this issue. Feel free to reopen if there is another occurrence.


Question: "strings" "k8s.io/klog" <ins> "k8s.io/utils/keymutex" </ins> utilpath "k8s.io/utils/path" )

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: I am using Windows containers (Server Core ltsc 2019). These all have volumes mounted to three different Azure file shares under a single storage account. Multiple instances of these containers run on each node. On deploying the containers I get the following errors. Sometimes this does manage to resolve itself. MountVolume.SetUp failed for volume "log-share" : Remove-SmbGlobalMapping failed: exit status 1, output: "Remove-SmbGlobalMapping : No MSFTSmbGlobalMapping objects found with property 'RemotePath' equal to rn'storageaccounturllogs'. Verify the value of the property and retry.rnAt line:1 char:1rn+ Remove-SmbGlobalMapping -RemotePath $Env:smbremotepath -Forcern+ ~rn + CategoryInfo : ObjectNotFound: (storageaccounturllogs:String) [Remove-SmbGlobalMapping], Ci rn mJobExceptionrn + FullyQualifiedErrorId : CmdletizationQueryNotFoundRemotePath,Remove-SmbGlobalMappingrn rn" MountVolume.SetUp failed for volume "config-share" : New-SmbGlobalMapping failed: exit status 1, output: "New-SmbGlobalMapping : Generic failure rnAt line:1 char:190rn+ ... ser, Env:smbremotepath -Cred ...rn+ ~~~rn + CategoryInfo : NotSpecified: (MSFTSmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlobalMa rn pping], CimExceptionrn + FullyQualifiedErrorId : HRESULT 0x80041001,New-SmbGlobalMappingrn rn" What you expected to happen: For the volumes to be mounted. This all worked ok in k8s 1.12.1 using Windows 1803. I have now upgraded to 1.13.4 and Windows ltsc2019.

How to reproduce it (as minimally and precisely as possible): Use aks-engine to create a 1.13.4 cluster with Windows nodes Create a storage account Create multiple file shares under the account. Add access credentials to the storage account as a secret on k8s Create a windows container with folders that will be used to mount the Azure storage account file shares. Deploy to k8s defining volume mounts and volumes in the container section of the deployment e.g. volumeMounts: name: folder1-share mountPath: "c:appfolder1" name: folder2-share mountPath: "c:appfolder2" volumes: name: folder1-share azureFile: secretName: azure-file-share-cred shareName: share1 readOnly: false name: folder2-share azureFile: secretName: azure-file-share-cred shareName: share2 readOnly: true Anything else we need to know?: This maybe related to the fix for Environment: Kubernetes version (use ): 1.13.4 Cloud provider or hardware configuration: Azure OS (e.g: ): Windows 2019 Kernel (e.g. ): 10.0..316 Install tools: aks-engine 0.13.2 Others:
/sig Azure
/sig windows
/assign
hi in your logs, the remotepath is , so is the share name? also, could you log on to that agent node, and provide , there are logs like
I think this is a race condition issue, let me file a PR to fix this, thanks for reporting