Question: "//vendor/k8s.io/api/networking/v1:go_default_library", "//vendor/k8s.io/api/policy/v1beta1:go_default_library", "//vendor/k8s.io/api/rbac/v1:go_default_library", <del> "//vendor/k8s.io/api/rbac/v1alpha1:go_default_library", </del> "//vendor/k8s.io/api/rbac/v1beta1:go_default_library", <del> "//vendor/k8s.io/api/scheduling/v1alpha1:go_default_library", "//vendor/k8s.io/api/settings/v1alpha1:go_default_library", </del> "//vendor/k8s.io/api/storage/v1:go_default_library", "//vendor/k8s.io/api/storage/v1beta1:go_default_library", "//vendor/k8s.io/apimachinery/pkg/api/errors:go_default_library",

Need to follow-up on and i assigned it to you because turning off settings/v1alpha1 requires some changes in the tests.
There are no sig labels on this issue. Please by:<br(1) mentioning a sig: <br(2) specifying the label manually: <br<brNote: method (1) will trigger a notification to the team. You can find the team list and label list
/sig api-machinery
/assign
did you mean I should add a priority label?
[MILESTONENOTIFIER] Milestone Labels Complete Issue label settings: sig/api-machinery: Issue will be escalated to these SIGs if needed. priority/important-soon: Escalate to the issue owners and SIG owner; move out of milestone after several unsuccessful escalation attempts. kind/bug: Fixes a bug discovered during the current release. <detailsAdditional instructions available <a href=""here</a</details


Question: testCannotConnect(f, f.Namespace, "client-b", service, 81) }) }) <ins> It("should allow egress access on one named port [Feature:NetworkPolicy]", func() { clientPodName := "client-a" protocolUDP := v1.ProtocolUDP policy := &networkingv1.NetworkPolicy{ ObjectMeta: metav1.ObjectMeta{ Name: "allow-client-a-via-named-port-egress-rule", }, Spec: networkingv1.NetworkPolicySpec{ // Apply this policy to client-a PodSelector: metav1.LabelSelector{ MatchLabels: map[string]string{ "pod-name": clientPodName, }, }, // Allow traffic to only one named port: "serve-80". Egress: []networkingv1.NetworkPolicyEgressRule{{ Ports: []networkingv1.NetworkPolicyPort{ { Port: &intstr.IntOrString{Type: intstr.String, StrVal: "serve-80"}, }, // Allow DNS look-ups { Protocol: &protocolUDP, Port:     &intstr.IntOrString{Type: intstr.Int, IntVal: 53}, }, }, }}, }, } policy, err := f.ClientSet.NetworkingV1().NetworkPolicies(f.Namespace.Name).Create(policy) Expect(err).NotTo(HaveOccurred()) defer cleanupNetworkPolicy(f, policy) By("Creating client-a which should be able to contact the server.", func() { testCanConnect(f, f.Namespace, clientPodName, service, 80) }) By("Creating client-a which should not be able to contact the server on port 81.", func() { testCannotConnect(f, f.Namespace, clientPodName, service, 81) }) }) </ins> }) })

Test Cases: [ ] Egress blocking based on cidr to a set of pods [ ] Egress blocking based on namespace to a set of pods [ ] Egress blocking based on a set of pods to another set of pods
Update: I'm working to try to enable these new features in a few CNI plug-ins so the tests can run locally.
Can we please move out of 1.8 milestone?
/label status/approved-for-milestone
/kind cleanup
/remove-kind bug
Those labels are not set on the issue: <detailsIn response to : /remove-kind bug Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/kind cleanup
/status approved-for-milestone /kind cleanup /remove-kind bug
[MILESTONENOTIFIER] Milestone Issue Current <details open<summaryIssue Labels</summary- : Issue will be escalated to these SIGs if needed. : Escalate to the issue owners and SIG owner; move out of milestone after several unsuccessful escalation attempts. : Adding tests, refactoring, fixing old bugs. </details<details<summaryHelp</summary<ul<li<a href=""Additional instructions</a</li<li<a href=""Commands for setting labels</a</li</ul</details
Can we please move out of 1.9 milestone? :)
this will be partially fixed with . I think it is safe to keep in the milestone.


Question: } func (obj InfoObject) Name() string { <del> return obj.Info.Name </del> <ins> group := "" if obj.Info.Mapping.GroupVersionKind.Group != "" { group = fmt.Sprintf("%v.", obj.Info.Mapping.GroupVersionKind.Group) } return group + fmt.Sprintf( "%v.%v:%v.%v", obj.Info.Mapping.GroupVersionKind.Version, obj.Info.Mapping.GroupVersionKind.Kind, obj.Info.Namespace, obj.Info.Name, ) </ins> } // Differ creates two DiffVersion and diffs them.

What happened: I get a diff, when I specify a file directly, but on recursive mode, no diff is shown anymore: What you expected to happen: Recursive mode should work How to reproduce it (as minimally and precisely as possible): Have a bunch of YAML manifest, but with name collisions, e.g. Service , Deployment , etc Anything else we need to know?: I think the problem is that all objects have the same name, which should not be a problem per se. Only heapster-binding has a different name, and looking at the generated temp files, only and are present: Environment: Kubernetes version (use ): $ kubectl version --short Client Version: v1.13.0 Server Version: v1.13.0 <!-- DO NOT EDIT BELOW THIS LINE --/kind bug
/sig cli
I think the files are overwriting each other?
Yeah, good catch, I'll try to fix that asap
Thanks! By the way, very nice work with dry-run and diff. I was waiting for this.
Feel free to look at the PR and give your opinion!


Question: return nil, err } if klet.enablePluginsWatcher { <del> klet.pluginWatcher = pluginwatcher.NewWatcher(klet.getPluginsDir()) </del> <ins> klet.pluginWatcher = pluginwatcher.NewWatcher(klet.getPluginsRegistrationDir()) </ins> } // If the experimentalMounterPathFlag is set, we do not want to

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug This is more of an enhancement. What happened: The Plugin Watcher is setup to trigger on file activities when they occur in the subdirectory that it scans. During startup, the Watcher is passed (from within the Kubelet) the Kubelet's root plugin directory as a location to scan for newly plugins - Because the kubelet's plugins subdirectory is also used by existing in-tree drivers (including csi) as a location to store plugins artifacts, the Plugin Watcher is getting triggered by directory/file activities occurring in that subdirectory that has nothing to do with plugin registration as shown in the output below: What you expected to happen: The Plugin Watcher should not be getting triggered by file/directory activities not related to plugin registration/unregistration. One possibility is to have the Watcher scan one directory below plugins like or something similar How to reproduce it (as minimally and precisely as possible): Run plugin watcher with a CSI driver. cc
/sig node
I think you should only place the plugin socket file in the plugin registration directory but not the plugin data.
agree with we can only place the registration file in the specified path instead of placing all data into it. On the other hand, you can place data into the /var/lib/kubelet/plugins/registration even if registration created, that's a no-end loop.
As I mentioned in issue, existing in-tree volume plugins were already using that directory as a location to store storage operation artifacts (not just csi). Here is a shortened list of driver locations that use /var/lib/kubelet/plugins in their code already: And several more locations. The fix would be to have plugin watcher scan a subdirectory of plugins (say ) instead of plugins directly as it is doing today. The watcher functions as intended. Except now it picks up all directory activities whether related or not to plugin registration.
Well given the fact that the pluginwatcher is beta this is a bit of an issue. My understanding of the "beta" guarantees is that we want to be backwards compatible for all plugins published with the beta API. Meaning that even if we move the registration directory to a new location, we'll need to keep scanning the old directory (to pick up v1beta plugins).

From a 10 000 feet point of view it seems easier to me if we move the in-tree volume plugin's data. Though it might be a lot of work. Another less painful option, but a bit grey is the following: Enforce plugins to end with this suffix is recommended for device plugins (though not enforced) and I believe is a convention on linux. This is a bit grey because we would still be breaking v1beta users but since we're moving a convention from "recommended" to "enforced" it seems "less" breaking to me (and I haven't seen device plugins not using that extension). What do you think ?
I like the suffix idea. It can be a recommendation (which csi is already doing).
Hmm, I've just deployed a new k8s cluster on 1.12.0 and noticed this repeating in the logs on a few of our pods. Just for my own sanity, is this something I should be worried about or can I ignore it until a fix is released?
you can ignore these errors. if it's a recommendation then we'll keep spamming the logs with all the data files. Another thing to keep in mind, it seems like CSI plugins puts a lot of data (and creates a lot of directories). It might be worth considering limiting the recursion to one level. I'm concerned we might exhaust the amount of fds by tracking the plugins data.
yeah CSI keeps track of volume operation metadata in the plugins dir. This probably will not change. I understand this is beta. However, besides CSI and Device plugin, are there any other implementers out there for this feature ? Would it be possible to propose a new location (i.e. <pluginsDir/reg/?
let me know if you open a PR for this as discussed.
thanks for open the issue with detail logs, i can see two problems by looking at logs and code. 1) plugin watcher trying to dial non-socket files. 2) too many logs generated (CREATE and DELETE). I a suggestion to fix of this issue we can fix those problems by: 1) check file mode when handle CREATE event, only attempt to dial file when file mode is socket.

2) increase log verbose level for both 'handleDeleteEvent()' and 'handleCreateEvent()' plugin watcher still has activity when any file create/delete under the directory, but they become light weighted (CREATE file will only trigger file mode check, DELETE file event will trigger plugin searching) i am not sure we want to go further to limit directory depth where plugin can live. it has two drawbacks IMO: 1) it is hard to enforce, not sure everyone will read code/readme. 2) it provides less flexibility to plugins: if want to organize many plugin socket files with directory.
These are definitely good suggestions! It seems like there might be some confusion as to why we are thinking of doing this: Most plugins use the plugin directory to store data (actually a lot of files / dirs) Watching all the directories and files created inside the plugin directory puts more burden on the Kubelet in terms of cpu / memory consumption Watching all the directories and files created inside the plugin directory quickly raises the number of fd consumed By limiting the search of the plugins to one directory, we would be severely limiting the ability of the module to unintentionally harm the overall Kubelet. It seems actually pretty easy to enforce, as it's a simple condition. As for not everyone reading code / readme, we can add the documentation to the gRPC interface if this is a concern. In terms of flexibility, is there a real need to create a plugin in a lower directory? Is there a plugin that has so many sockets that a single directory underneath the plugin directory isn't enough? Naively it seems to me that is enough for the different plugins we enable.
ping on the above :)
Naively it seems to me that $ is enough for the different plugins we enable. i can not speak from plugin developer, but both below looks reasonable to me , or as long as we can communicate to plugin developers clearly and effectively (gRPC interface documentation is needed), the directory depth restriction is reasonable to me.
I like . However, why not put plugintype one level up as . For instance
Is the suggestion to hard code the number of levels that the recursion algorithm will search to 3? Or just expect a socket only at a hard coded level (e.g. socket must be nested 3 deep)? Copying my comment from for consistency: Another option I can think of: recursively scanning but before scanning a directory check if it is mounted ().

This will be backwards compatible and will prevent scanning of volume data Thoughts?
i like this idea better if it is always the case that: volume data is mounted, we can modify the watcher code to not watch the mounted directory in the first place.
Skipping mounted dir along with restructuring search path to known (shallow) dir sounds like a good plan. Though with the new suggested paths, sockets will be placed at a location where mounts should not exist.
I think that would be a safe assumption. The plugin directory is only used as a target path for global mounts.
Ephemeral volume types may not need to mount, it could just be a symlink or plain directory. Can we assume any ephemeral volume CSI driver won't implement global mount?
Yes, in-tree ephemeral volumes today don't do a global mount so the mounts directory is which is not the plugin dir. So no issue there. For CSI the will be the global mount path (this plugin dir). Emphermal volumes shouldn't implement call -- but nothing prevents them from doing so. Maybe let's start with just checking for mount and ignoring mount directories. If it is still an issue we can consider adding a configurable black list of paths. Thoughts?
That sounds good. Also, putting the socket files in a specific location that does not clash with mounts will also help (i.e. )
/milestone v1.13 /priority important-soon
I just want to point out that one of the issue we have with recursively watching all directories is that it will significantly increase the amount of fd consumed. Given that plugins actually store data there and in a dynamic way my fear is that it will exhaust the available fds for no real good reasons. Limiting the depth sounds like a reasonable solution to that issue.
btw, because of this verbosity in my cluster writes 5G logs every day and filled up my rootfs (which is small). Is there any way to decrease this verbosity?
We can do both. Don't search mounted dirs, and max depth of search. Who is working on this?
That's pretty bad, can you open another issue to track that.
we have a PR in flight discussing this issue (recursive scanning) here: Thinking more about this, the fact that too much logs is generated is clearly linked to this specific issue and how we process plugins.

In any case I'll be fixing that issues too
how confident are we in getting this fixed by Code freeze tomorrow? 5pm PST
I expect things to wind up today, changes in the above PR aren't very contentious and don't involve significant changes


Question: } // if m.Current() != nil, then we should use the second client <del> server.expectUserAgent = "SecondClient" </del> <ins> server.SetExpectUserAgent("SecondClient") </ins> if ok, err := r.RotateCerts(); !ok || err != nil { t.Fatalf("unexpected rotation err: %t %v", ok, err) }

<!-- Please only use this template for submitting reports about failing tests in Kubernetes CI jobs --Which jobs are failing: pull-kubernetes-bazel-test Which test(s) are failing: Test_buildClientCertificateManager Testgrid link: Reason for failure: Anything else we need to know:
/remove-kind failing-test /kind flake /sig scheduling
/sig node


Question: EnableServer:            true, EnableDebuggingHandlers: true, SyncFrequency:           3 * time.Second, <ins> MinimumGCAge:            10 * time.Second, MaxContainerCount:       5, </ins> } RunKubelet(&kcfg) }

Things like SyncFrequency should have a lower-bound of acceptable, and that really should be 0ns.
The main point of this is now fixed, though we could be better about input checking all over


Question: claimSize:    "2Gi", expectedSize: "2Gi", } <ins> // gce or gke if getDefaultPluginName() == "kubernetes.io/gce-pd" { // using GB not GiB as e2e test unit since gce-pd returns GB, // or expectedSize may be greater than claimSize. test.claimSize = "2G" test.expectedSize = "2G" } </ins> claim := newClaim(test, ns, "default") testDynamicProvisioning(test, c, claim, nil) })

GCE PDs are allocated in chunks of GBs (not GiB) but function incorrectly creates volume in chunks of GiB - cc /sig storage
/kind bug
/assign
/priority important-soon
Hey I submit a pr for this issue. Your suggestions are very important. Thanks.


Question: expectError:  "name in URL does not match name in Eviction object: BadRequest", podName:      "t7", }, <ins> { name: "matching pdbs with no disruptions allowed, pod running, empty selector", pdbs: []runtime.Object{&policyv1.PodDisruptionBudget{ ObjectMeta: metav1.ObjectMeta{Name: "foo", Namespace: "default"}, Spec:       policyv1.PodDisruptionBudgetSpec{Selector: &metav1.LabelSelector{}}, Status:     policyv1.PodDisruptionBudgetStatus{DisruptionsAllowed: 0}, }}, eviction:    &policy.Eviction{ObjectMeta: metav1.ObjectMeta{Name: "t8", Namespace: "default"}, DeleteOptions: metav1.NewDeleteOptions(0)}, expectError: "Cannot evict pod as it would violate the pod's disruption budget.: TooManyRequests: The disruption budget foo needs 0 healthy pods and has 0 currently", podPhase:    api.PodRunning, podName:     "t8", policies:    []*policyv1.UnhealthyPodEvictionPolicyType{nil, unhealthyPolicyPtr(policyv1.IfHealthyBudget)}, // AlwaysAllow would terminate the pod since Running pods are not guarded by this policy }, </ins> } for _, unhealthyPodEvictionPolicy := range []*policyv1.UnhealthyPodEvictionPolicyType{nil, unhealthyPolicyPtr(policyv1.IfHealthyBudget), unhealthyPolicyPtr(policyv1.AlwaysAllow)} {

with an empty selector selects all the pods as expected. But, during the /, the pdb is not being honored correctly i.e. all the pods are getting evicted. This issue is only observed with the empty selector and works as expected if a valid label selector is provided that selects all the pods in a namespace. When an empty selector is provided in the 's spec, the created needs to be honored by not evicting all the pods during the api call Create a simple busybox deployment with 3 replicas on a local cluster (or any cluster)Create a pdb object with an selectorCheck the status of the created pdb object that has the observed to be 3 (which is expected)Call the api to see the pdb not being honored and all the pods being evictedDelete the above pdb object and create a new one with the label selectorObserve the expected status of the pdb object same as the old one with the set to 3Call the api to observe the pdb being honored as expected No response <details</details<detailslocal cluster - N/A </details<details</details<details</details<detailsCRI-O </details<details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig apps
/assign


Question: framework.ExpectNoError(err) for _, p := range updatedPodList.Items { if p.Name == nonCriticalBestEffort.Name { <del> framework.ExpectNotEqual(p.Status.Phase, v1.PodFailed, fmt.Sprintf("pod: %v should be preempted", p.Name)) </del> <ins> framework.ExpectEqual(p.Status.Phase, v1.PodRunning, fmt.Sprintf("pod: %v should not be preempted with status: %#v", p.Name, p.Status)) </ins> } else { <del> framework.ExpectEqual(p.Status.Phase, v1.PodFailed, fmt.Sprintf("pod: %v should not be preempted", p.Name)) </del> <ins> framework.ExpectEqual(p.Status.Phase, v1.PodFailed, fmt.Sprintf("pod: %v should be preempted with status: %#v", p.Name, p.Status)) </ins> } } })

<!-- Please only use this template for submitting reports about failing tests in Kubernetes CI jobs --Setting PodPriority directly when adminssion Priority enable,the Pod is rejected. To fix this, we need to create new priority classes for these tests. Which jobs are failing: Which test(s) are failing: I0822 19:01:41.165] should be able to create and delete a critical pod [It] Since when has it been failing: Testgrid link: Reason for failure: because adminssion Priority enableIf the priority class is not found, the Pod is rejected. Anything else we need to know:
/area e2e-test-framework /sig testing /kind bug /priority important-soon
This has now changed to this error: /reopen
Reopened this issue. <detailsIn response to : This has now changed to this error: /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
For reference: testgrid link /milestone v1.17
is this still in progress?
pr has updated, PTAL.
looks like has to merge for this to close
/milestone v1.18


Question: (Note: the syntax of volumeID is currently awkward; #10181 fixes it) <del> ### NFS </del> <ins> ### nfs An `nfs` volume allows an existing NFS (Network File System) share to be mounted into your pod. Unlike `emptyDir`, which is erased when a Pod is removed, the contents of an `nfs` volume are preserved and the volume is merely unmounted.  This means that an NFS volume can be pre-populated with data, and that data can be "handed off" between pods.  NFS can be mounted by multiple writers simultaneuosly. __Important: You must have your own NFS server running with the share exported before you can use it__ See the [NFS example](../examples/nfs/) for more details. For example, [this file](../examples/nfs/nfs-web-pod.yaml) demonstrates how to specify the usage of an NFS volume within a pod. </ins> <del> Kubernetes NFS volumes allow an existing NFS share to be made available to containers within a pod. </del> <ins> In this example one can see that a `volumeMount` called "nfs" is being mounted onto `/var/www/html` in the container "web".  The volume "nfs" is defined as type `nfs`, with the NFS server serving from `nfs-server.default.kube.local` and exporting directory `/` as the share.  The mount being created in this example is writeable. </ins> <del> See the [NFS Pod examples](https://github.com/GoogleCloudPlatform/kubernetes/blob/master/examples/nfs/) section for more details. For example, [nfs-web-pod.yaml](https://github.com/GoogleCloudPlatform/kubernetes/blob/master/examples/nfs/nfs-web-pod.yaml) demonstrates how to specify the usage of an NFS volume within a pod. In this example one can see that a `volumeMount` called "nfs" is being mounted onto `/var/www/html` in the container "web". The volume "nfs" is defined as type `nfs`, with the NFS server serving from `nfs-server.default.kube.local` and exporting directory `/` as the share. The mount being created in this example is not read only. </del> <ins> ### iscsi An `iscsi` volume allows an existing iSCSI (SCSI over IP) volume to be mounted into your pod.  Unlike `emptyDir`, which is erased when a Pod is removed, the contents of an `iscsi` volume are preserved and the volume is merely unmounted.  This means that an iscsi volume can be pre-populated with data, and that data can be "handed off" between pods. __Important: You must have your own iSCSI server running with the volume created before you can use it__ A feature of iSCSI is that it can be mounted as read-only by multiple consumers simultaneously.  This means that you can pre-populate a volume with your dataset and then serve it in parallel from as many pods as you need.  Unfortunately, iSCSI volumes can only be mounted by a single consumer in read-write mode - no simultaneous readers allowed. See the [iSCSI example](../examples/iscsi/) for more details. ### glusterfs A `glusterfs` volume allows an [Glusterfs](http://www.gluster.org) (an open source networked filesystem) volume to be mounted into your pod.  Unlike `emptyDir`, which is erased when a Pod is removed, the contents of a `glusterfs` volume are preserved and the volume is merely unmounted.  This means that a glusterfs volume can be pre-populated with data, and that data can be "handed off" between pods.  GlusterFS can be mounted by multiple writers simultaneuosly. __Important: You must have your own GlusterFS installation running before you can use it__ See the [GlusterFS example](../examples/glusterfs/) for more details. ### rbd An `rbd` volume allows a [Rados Block Device](http://ceph.com/docs/master/rbd/rbd/) volume to be mounted into your pod.  Unlike `emptyDir`, which is erased when a Pod is removed, the contents of an `rbd` volume are preserved and the volume is merely unmounted.  This means that a glusterfs volume can be pre-populated with data, and that data can be "handed off" between pods. __Important: You must have your own Ceph installation running before you can use RBD__ A feature of RBD is that it can be mounted as read-only by multiple consumers simultaneously.  This means that you can pre-populate a volume with your dataset and then serve it in parallel from as many pods as you need.  Unfortunately, RBD volumes can only be mounted by a single consumer in read-write mode - no simultaneous readers allowed. See the [RBD example](../examples/rbd/) for more details. ### gitRepo A `gitRepo` volume is an example of what can be done as a volume plugin.  It mounts an empty directory and clones a git repository into it for your pod to use.  In the future, such volumes may be moved to an even more decoupled model, rather than extending the Kubernetes API for every such use case. </ins> ### Secrets <del> Secret volumes are used to pass sensitive information, such as passwords, to pods that mount these volumes. Secrets are described [here](secrets.md). </del> <ins> A `secret` volume is used to pass sensitive information, such as passwords, to pods.  You can store secrets in the Kubernetes API and mount them as files for use by pods without coupling to Kubernetes directly.  `secret` volumes are backed by tmpfs (a RAM-backed filesystem) so they are never written to non-volatile storage. __Important: You must create a secret in the Kubernetes API before you can use it__ Secrets are described in more detail [here](secrets.md). ### persistentVolumeClaim A `persistentVolumeClaim` volume is used to mount a [PersistentVolume](persistent-volumes.md) into a pod.  PersistentVolumes are a way for users to "claim" durable storage (such as a GCE PersistentDisk or an iSCSI volume) without knowing the details of the particular cloud environment. See the [PersistentVolumes example](../examples/persistent-volumes/) for more details. </ins> ## Resources <del> The storage media (Disk, SSD, or memory) of an EmptyDir volume is determined by the media of the filesystem holding the kubelet root dir (typically `/var/lib/kubelet`). There is no limit on how much space an EmptyDir or HostPath volume can consume, and no isolation between containers or between pods. </del> <ins> The storage media (Disk, SSD, etc) of an `emptyDir` volume is determined by the medium of the filesystem holding the kubelet root dir (typically `/var/lib/kubelet`).  There is no limit on how much space an `emptyDir` or `hostPath` volume can consume, and no isolation between containers or between pods. In the future, we expect that `emptyDir` and `hostPath` volumes will be able to request a certain amount of space using a [resource](./compute_resources.md) specification, and to select the type of media to use, for clusters that have several media types. </ins> <del> In the future, we expect that EmptyDir and HostPath volumes will be able to request a certain amount of space using a [compute resource](./compute_resources.md) specification, and to select the type of media to use, for clusters that have several media types. </del> [![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/volumes.md?pixel)]()

only talks about these volumes: EmptyDir HostPath GCEPersistentDisk NFS However, we many more volume types. Documentation should be updated to reflect this.
cc
Is anyone working on this? If not, I would like to do it.
go for it :) Improving PV docs is on my TODO list, too.
Sure, thank you. I will start now, hope to send a PR ASAP.


Question: PrintFlags *genericclioptions.PrintFlags ToPrinter  func(string) (printers.ResourcePrinter, error) <del> DeleteOptions *delete.DeleteOptions </del> <ins> DeleteOptions *cmddelete.DeleteOptions </ins> ServerSideApply bool ForceConflicts  bool

As per , I'm trying to remove CPU limits. But the behaviour is different depending whether the ResourceQuota exists or not: I would expect to be idempotent, and for its behaviour to result in the same end configuration regardless of the initial configuration. a new namespace named this command: that the ResourceQuota is created and is set to it again that is removed each subsequent time it's run, as long as resourcequota/compute-resource exists resourcequota/compute-resource again that the ResourceQuota is created and is set to again, and so forth the exact same behaviour if is changed to No response <details</details<detailsOpenShift 4.13.12 </details<details</details<details</details<details</details<details</details
/sig node
Tried to replicate the issue and observed the same behaviour.
/remove-sig node /sig api-machinery /cc
What's the goal of applying here? What are you trying to achieve? I agree that this is not the right behavior though.
I'm trying to remove a previously set property. I cannot remove the ResourceQuota altogether because it contains other properties (which I left out of my example) that I do not wish to remove.
But 0 is the same as unset, isn't it?
No, if I set it to 0 then any deployment with set to higher than 0 will fail to create any pods because it will exceed the quota.
/triage accepted /cc
/assign
Hello, I'm having a similar issue. Here's my use case: We use Flux to deploy things on our cluster - in particular, we use the component and set chart values in . Some of the charts we deploy have default values we want to remove, so we want to set a value to Editing an HelmRelease with this kind of ajustments drops the null keys, so default values end up being setup again and generated manifests are invalid :( Let me know if you need anything else to keep this issue moving!
I think the correct behavior is to set the struct to the default value when or , because in the standard library does this as well, and I think we expect the result to be as similar as possible to . So our real issue here is that we didn't set the to its default value when patch.
Based on the above, I think this question is not related to the current issue.

Tracing it back to , maybe the current result is what we want. The explicit null values means that we want to remove it in patch. As mentioned , our behavior in create is also as expected. I'm not quite sure if we actually want to fix it, and how /cc I hope to get your advice as you are the implementer of
I think client-side on create should do what on update will accomplish, namely, explicit nulls in the manifest are treated as the removal of the attributes they are values of. Proof of concept opened at , I'd be happy for someone to pick that up, add test coverage, and work through any failing tests that change uncovers
Thank you, but I still have a question. If we solve the difference between create and patch behaviors in ApiServer on the client side, will third-party callers still have the same problem? Or should we not cover this scenario?
I would consider changing behavior of null in a create request server-side to be a compatibility break. values are defined for client-side apply to mean "remove the key associated with this value". I think fixing the client-side apply creation case to have the same semantic is the most narrowly scoped fix we can make while preserving compatible server-side behavior generally.
Okay, I got it, I'll follow up with


Question: _ = client.CoreV1().Namespaces().Delete(namespace, nil) _ = client.RbacV1beta1().ClusterRoles().Delete("wardler", nil) _ = client.RbacV1beta1().ClusterRoleBindings().Delete("wardler:sample-system:anonymous", nil) <del> aggrclient := f.AggregatorClient _ = aggrclient.ApiregistrationV1beta1().APIServices().Delete("v1alpha1.wardle.k8s.io", nil) </del> if block { _ = wait.Poll(100*time.Millisecond, 5*time.Second, func() (bool, error) { _, err := client.CoreV1().Namespaces().Get("sample-system", metav1.GetOptions{})

Seen in https://k8s- Aug 26 09:35:45.074: Couldn't delete ns: "e2e-tests-containers-9cscj": an error on the server ("missing route (service "sample-api" not found)") has prevented the request from succeeding (get ) (&errors.StatusError{ErrStatus:v1.Status{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ListMeta:v1.ListMeta{SelfLink:"", ResourceVersion:""}, Status:"Failure", Message:"an error on the server ("missing route (service "sample-api" not found)") has prevented the request from succeeding (get )", Reason:"InternalError", Details:(*v1.StatusDetails)(0xc4217a4900), Code:500}})
https://k8s-


Question: n.Spec.Taints = taints return n } <ins> // PersistentVolumeClaimWrapper wraps a PersistentVolumeClaim inside. type PersistentVolumeClaimWrapper struct{ v1.PersistentVolumeClaim } // MakePersistentVolumeClaim creates a PersistentVolumeClaim wrapper. func MakePersistentVolumeClaim() *PersistentVolumeClaimWrapper { return &PersistentVolumeClaimWrapper{} } // Obj returns the inner PersistentVolumeClaim. func (p *PersistentVolumeClaimWrapper) Obj() *v1.PersistentVolumeClaim { return &p.PersistentVolumeClaim } // Name sets `s` as the name of the inner PersistentVolumeClaim. func (p *PersistentVolumeClaimWrapper) Name(s string) *PersistentVolumeClaimWrapper { p.SetName(s) return p } // Namespace sets `s` as the namespace of the inner PersistentVolumeClaim. func (p *PersistentVolumeClaimWrapper) Namespace(s string) *PersistentVolumeClaimWrapper { p.SetNamespace(s) return p } // Annotation sets a {k,v} pair to the inner PersistentVolumeClaim. func (p *PersistentVolumeClaimWrapper) Annotation(key, value string) *PersistentVolumeClaimWrapper { metav1.SetMetaDataAnnotation(&p.ObjectMeta, key, value) return p } // VolumeName sets `name` as the volume name of the inner // PersistentVolumeClaim. func (p *PersistentVolumeClaimWrapper) VolumeName(name string) *PersistentVolumeClaimWrapper { p.PersistentVolumeClaim.Spec.VolumeName = name return p } // AccessModes sets `accessModes` as the access modes of the inner // PersistentVolumeClaim. func (p *PersistentVolumeClaimWrapper) AccessModes(accessModes []v1.PersistentVolumeAccessMode) *PersistentVolumeClaimWrapper { p.PersistentVolumeClaim.Spec.AccessModes = accessModes return p } // Resources sets `resources` as the resource requirements of the inner // PersistentVolumeClaim. func (p *PersistentVolumeClaimWrapper) Resources(resources v1.ResourceRequirements) *PersistentVolumeClaimWrapper { p.PersistentVolumeClaim.Spec.Resources = resources return p } // PersistentVolumeWrapper wraps a PersistentVolume inside. type PersistentVolumeWrapper struct{ v1.PersistentVolume } // MakePersistentVolume creates a PersistentVolume wrapper. func MakePersistentVolume() *PersistentVolumeWrapper { return &PersistentVolumeWrapper{} } // Obj returns the inner PersistentVolume. func (p *PersistentVolumeWrapper) Obj() *v1.PersistentVolume { return &p.PersistentVolume } // Name sets `s` as the name of the inner PersistentVolume. func (p *PersistentVolumeWrapper) Name(s string) *PersistentVolumeWrapper { p.SetName(s) return p } // AccessModes sets `accessModes` as the access modes of the inner // PersistentVolume. func (p *PersistentVolumeWrapper) AccessModes(accessModes []v1.PersistentVolumeAccessMode) *PersistentVolumeWrapper { p.PersistentVolume.Spec.AccessModes = accessModes return p } // Capacity sets `capacity` as the resource list of the inner PersistentVolume. func (p *PersistentVolumeWrapper) Capacity(capacity v1.ResourceList) *PersistentVolumeWrapper { p.PersistentVolume.Spec.Capacity = capacity return p } // HostPathVolumeSource sets `src` as the host path volume source of the inner // PersistentVolume. func (p *PersistentVolumeWrapper) HostPathVolumeSource(src *v1.HostPathVolumeSource) *PersistentVolumeWrapper { p.PersistentVolume.Spec.HostPath = src return p } </ins>

We need an integration test for the scheduler enforcement of the alpha feature. This is required for beta graduation; we need to collect signal on the current behavior before we can proceed. See this comment for context:
/sig storage /sig scheduler
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : /sig storage /sig scheduler Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scheduling


Question: }) /* <del> Testname: var-expansion-subpath Description: Make sure a container's subpath can be set using an expansion of environment variables. </del> <ins> Release : v1.19 Testname: VolumeSubpathEnvExpansion, subpath expansion Description: Make sure a container's subpath can be set using an expansion of environment variables. </ins> */ <del> ginkgo.It("should allow substituting values in a volume subpath [sig-storage]", func() { </del> <ins> framework.ConformanceIt("should allow substituting values in a volume subpath [sig-storage]", func() { </ins> podName := "var-expansion-" + string(uuid.NewUUID()) pod := &v1.Pod{ ObjectMeta: metav1.ObjectMeta{

Is this a BUG REPORT or FEATURE REQUEST?: /kind api-change /sig storage /sig node /sig testing Description: Promote e2e tests for VolumeSubpathEnvExpansion to Conformance as it is GA approved. Related Issues: GA Upgrade: Feature Request: KEP:
/assign
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale


Question: } klog.Infof("Endpoints = %s", addrs) <del> fmt.Printf("discovery.zen.ping.unicast.hosts: [%s]n", strings.Join(addrs, ", ")) </del> <ins> fmt.Printf("discovery.seed_hosts: [%s]n", strings.Join(addrs, ", ")) fmt.Printf("cluster.initial_master_nodes: [%s]n", strings.Join(addrs, ", ")) </ins> }

What happened: On latest fluentd-elasticsearch, the elasticsearch replicas seems failed to form a valid cluster with the service discovery helper written in golang. What you expected to happen: Successfully boot up an elasticsearch cluster I guess. How to reproduce it (as minimally and precisely as possible): Just use the yamls in and the elasticsearch-logging pods will keep spamming warning messages. I think thats because options are deprecated in elasticsearch 7.x.x Anything else we need to know?: IMHO I think thats all. Environment: Kubernetes version (use ): 1.12.4 Cloud provider or hardware configuration: AKS OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig instrumentation
Thanks


Question: }, } <del> _, err = f.ClientSet.CoreV1().ConfigMaps(ns2).Create(context.TODO(), hooks, metav1.CreateOptions{}) </del> <ins> _, err = f.ClientSet.CoreV1().ConfigMaps(driverns).Create(context.TODO(), hooks, metav1.CreateOptions{}) </ins> framework.ExpectNoError(err) if len(m.javascriptHooks) > 0 {

Tracking bug for /sig storage /important-longterm /assign
/sig storage /triage accepted
/priority important-longterm


Question: FROM BASEIMAGE <del> # If we're building for another architecture than amd64, the CROSS_BUILD_ placeholder is removed so e.g. CROSS_BUILD_COPY turns into COPY # If we're building normally, for amd64, CROSS_BUILD lines are removed CROSS_BUILD_COPY qemu-ARCH-static /usr/bin/ RUN DEBIAN_FRONTEND=noninteractive apt-get update -y  && DEBIAN_FRONTEND=noninteractive apt-get -yy -q install  iptables  ebtables  ethtool  ca-certificates  conntrack  util-linux  socat  git  jq  nfs-common  glusterfs-client  cifs-utils  ceph-common  && DEBIAN_FRONTEND=noninteractive apt-get upgrade -y  && DEBIAN_FRONTEND=noninteractive apt-get autoremove -y  && DEBIAN_FRONTEND=noninteractive apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* # CACHEBUST RUN cp /usr/bin/nsenter /nsenter # Copy the the cni-bin folder into /opt/cni/bin COPY cni-bin/bin /opt/cni/bin </del> # Create symlinks for each hyperkube server # Also create symlinks to /usr/local/bin/ where the server image binaries live, so the hyperkube image may be # used instead of gcr.io/google_containers/kube-* without any modifications.

I'm pretty sure that building the hyperkube images takes longer than building kubernetes itself now. At least a large part of this seems to be that we download and install ~172 extra debian packages several times (once for each arch). <details</detailsNot sure how to improve this - maybe better caching? Do we actually want to install these extra packages? Should we use a better base image than ? cc
cc (not really sure who "owns" hyperkube, if anyone)
The hyperkube binary is one thing. I think the image ends up being big as we drag in so much stuff. I wasn't involved in creating the docker image so I can't comment. Ideally this thing would be slim and would have minimal external dependencies. Fixing this starts with mapping why we have each of those deps imported.
I actually have written a such patch in my local env, but I never uploaded. It removed the docker and docker-multinode-specific stuff and all the unused deps. Also, we will come one step closer once is in. Should I submit the patch so you can take a look at it?
sure!
This needs to be triaged as a release-blocker or not for 1.5
punting to v1.6.
The main problem here is that we're installing huge packages for supporting some volume types: git nfs-common glusterfs-client cifs-utils With containerized volume mounters we'll get rid of that.
When it comes to local rebuilding, should help here. As I've already noted in the PR, maybe we should make CACHEBUST=0 the default for all hack/dev- scripts. This would at least ease development and testing.
can this be closed? or moved to 1.7?
should this be in v1.7? is there an appropriate sig to own this?
cries nope, not v1.7. I'm still not sure who owns hyperkube.
Everyone and no one I suppose. Yeah, agree it's sad.
/sig release guess we're saying sig-release owns hyperkube under the general guise of "cutting bits" until/unless there is a better home for it
Yeah, hyperkube definitely belongs to sig-release as it hasn't been owned by a sig so far...


Question: t.Errorf("cannot create state file: %s", err.Error()) } defer os.RemoveAll(sDir) <del> defer func() { if err := recover(); err != nil { if testCase.panicMsg != "" { if !strings.Contains(err.(string), testCase.panicMsg) { t.Errorf("Unexpected panic message. Have: %q wants %q", err, testCase.panicMsg) } } else { t.Errorf("Unexpected panic: %q", err) } } else if testCase.panicMsg != "" { t.Error("Expected panic hasn't been raised") } }() </del> mgr, err := NewManager(testCase.cpuPolicyName, 5*time.Second, machineInfo, testCase.nodeAllocatableReservation, sDir) if testCase.expectedError != nil {

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: When enable cpumanager by , but not set and , kubelet will get panic error in log What you expected to happen: Should graceful exit with error not panic. How to reproduce it (as minimally and precisely as possible): set and but not set and Anything else we need to know?:
/sig node
Great that it's not panicing anymore, but I'd also love to get the "requires + to be greater than zero" hint in this particular case. It looks like even with this doesn't seem to be the case (since it errors out with the non-actionable "unable to determine reserved CPU resources for static policy". Should I open another issue for this?


Question: attributes:           endpointCreateAttributes(), expression:           "variables.name == 'endpoints1'", expectErr:            true, <del> expectedErrorMessage: `composited variable "name" fails to compile:`, </del> <ins> expectedErrorMessage: `found no matching overload for '_==_' applied to '(int, string)'`, </ins> }, { name: "delayed eval error", variables: []NamedExpressionAccessor{ &testVariable{ <del> name:       "name", expression: "object.spec.subsets[114514].addresses.size()", // array index out of bound </del> <ins> name:       "count", expression: "object.subsets[114514].addresses.size()", // array index out of bound </ins> }, }, attributes:           endpointCreateAttributes(), <del> expression:           "variables.name == 'endpoints1'", </del> <ins> expression:           "variables.count == 810", </ins> expectErr:            true, <del> expectedErrorMessage: `composited variable "name" fails to evaluate:`, </del> <ins> expectedErrorMessage: `composited variable "count" fails to evaluate: index out of bounds: 114514`, </ins> }, { name: "out of budget during lazy evaluation",

Variables field is to ValidatingAdmissionPolicy in 1.28. Currently the usage of variables would expect a operator associated. e.g. (thanks to Mariam Fahmy for raising the issue) Above will fail with error: Meanwhile the will pass. The reason is when variables be evaluated to non dyn type(like bool or string), it will require special handling(like casting) since we use dyn type everywhere. It might confuse user.
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/triage accepted /sig api-machinery /cc
Tests I'd expect to see verify this behavior: PASS: A variable is of type bool, an expression returns the variable PASS: A variable is of type string, a messageExpression returns the variable PASS: A variable is of type dyn, at runtime the dyn is a bool, an expression returns the variable PASS: A variable is of typedyn, at runtime the dyn is a string, an messageExpression returns the variable PASS: A expression is of type dyn, at runtime the dyn is of type bool PASS: A messageExpression is of type dyn, at runtime the dyn is of type string COMPILE ERROR: A variable is of type bool, an expression tries to perform a non-bool (e.g. string replace) operation on the variable, this should resultsin a compile error COMPILE ERROR: A variable is of type string, an expression tries to perform a non-string (e.g. less than) operation on the variable, this should result in a compile error RUNTIME ERROR: A variable is of type dyn, at runtime the dyn is a bool, an expression tries to perform a non-bool (e.g. string replace) operation on the variable, this should result in a runtime error I'd also expect to see some cases covering variables of other types (maps, lists, duration, quantity, ...) to make sure we're over-fitting for bool and string.
/retitle ValidatingAdmissionPolicy: variable expression type not recognized as return type in expression that evaluates the variable
/assign


Question: assert.Nil(t, err) checkNumberOfEvents(t, 1, len(receivedEvents)) <del> ensureProperEventReturned(t, fakeEvent, receivedEvents[0]) </del> <ins> ensureProperEventReturned(t, fakeEvent2, receivedEvents[0]) </ins> } func TestGetEventsForNoTypeRequested(t *testing.T) {

I have some reports of a Node reporting a a resource capacity for cpu as a negative value. Any idea why cpu could report as a negative value? Is this an expected normal outcome? In its current state, it prevents any pods from being scheduled to that Node. Tips on how to debug further are appreciated. - any ideas?
Can you check machineInfo on Cadvisor or kubelet on the node?
a.k.a:
It is a bug in cAdvisor side. I am assigning it to you :-)
here is contents of /proc/cpuinfo Pertinent pieces of a log:
Machine is AWS instance if that matters....
We build the topology from system/cpu info. I think we can return partial topology information and ignore caches if its not present. This isn't a recent change though. Is it happening on just a single machine? I'll file a cAdvisor issue.
Can you link the referenced issue once created?
Re-assigning to since he's working on google/cadvisor
Could be this was caused by getting a Node on EC2 with old hypervisor, and I am the first victim...
That sounds likely. Most of the cache information is there in /proc/cpuinfo. So looks like the sysfs bits are disabled.
Can you try this cadvisor image on your machine and check /api/v1.3/machine output: sudo docker run  --volume=/:/rootfs:ro  --volume=/var/run:/var/run:rw  --volume=/sys:/sys:ro  --volume=/var/lib/docker/:/var/lib/docker:ro  --publish=8080:8080  --detach=true  --name=cadvisor  jnagal/cadvisor:canary
hmm
ah, selinux :( You'll need to run it with --privileged, I think. who's been dealing with it.
No luck with --privileged either on this box...
This has to do with SELinux, I think we had to disable it for this to work (some of our mounts are being denied). We're working on a better solution... This is not a problem in Kubernetes since we don't run Kubelet/cAdvisor in a Docker container.
ok, unfortunately, i have managed to kill docker on the box in question, and i cannot get the daemon to restart right now so I am not sure when that box will be available to debug against again.

I can provide an environment with a similar issue if required.
The fix is checked in to head. Can you build the binary and try?


Question: }) framework.ExpectNoError(err, "failed to marshal JSON for WatchEvent patch") ginkgo.By("patching the Endpoint") <del> _, err = f.ClientSet.CoreV1().Endpoints(ns).Patch(context.TODO(), testEndpointName, types.StrategicMergePatchType, []byte(endpointPatch), metav1.PatchOptions{}) </del> <ins> _, err = f.ClientSet.CoreV1().Endpoints(testNamespaceName).Patch(context.TODO(), testEndpointName, types.StrategicMergePatchType, []byte(endpointPatch), metav1.PatchOptions{}) </ins> framework.ExpectNoError(err, "failed to patch Endpoint") <ins> ctx, cancel = context.WithTimeout(context.Background(), 30*time.Second) defer cancel() _, err = watchtools.Until(ctx, endpoints.ResourceVersion, w, func(event watch.Event) (bool, error) { switch event.Type { case watch.Modified: if endpoints, ok := event.Object.(*v1.Endpoints); ok { found := endpoints.ObjectMeta.Name == endpoints.Name && endpoints.Labels["test-endpoint-static"] == "true" return found, nil } default: framework.Logf("observed event type %v", event.Type) } return false, nil }) framework.ExpectNoError(err, "failed to see %v event", watch.Modified) </ins> ginkgo.By("fetching the Endpoint") <del> endpoint, err := f.ClientSet.CoreV1().Endpoints(ns).Get(context.TODO(), testEndpointName, metav1.GetOptions{}) </del> <ins> endpoints, err = f.ClientSet.CoreV1().Endpoints(testNamespaceName).Get(context.TODO(), testEndpointName, metav1.GetOptions{}) </ins> framework.ExpectNoError(err, "failed to fetch Endpoint") <del> framework.ExpectEqual(endpoint.ObjectMeta.Labels["testservice"], "second-modification", "failed to patch Endpoint with Label") endpointSubsetOne := endpoint.Subsets[0] </del> <ins> framework.ExpectEqual(endpoints.ObjectMeta.Labels["test-service"], "patched", "failed to patch Endpoint with Label") endpointSubsetOne := endpoints.Subsets[0] </ins> endpointSubsetOneAddresses := endpointSubsetOne.Addresses[0] endpointSubsetOnePorts := endpointSubsetOne.Ports[0] framework.ExpectEqual(endpointSubsetOneAddresses.IP, "10.0.0.25", "failed to patch Endpoint")

[x] Original approval issue: [x] Two weeks soak end date: This issue is created to allow edit by /wip /hold This issue proposes the writing of a new test for the API endponts , , , with the intention of promoting it to Conformance. This issue contains a very basic mock test in order to start the conversation about what a good e2e test would look like for this endpoint. The code below is verified to be hitting the intended endpoint, show as per the queries in APIsnoop's live view of the cluster's audit logs. According to this APIsnoop query, there are still some remaining Endpoints endpoints which are untested. a Service in a namespace all Endpoints in all Namespaces find the created Service(1) /update the Namespaced Endpoint for the Service(1) the Namespaced Endpoint for the Service(1) the Namespaced Endpoint for the Service(1) Namespaced Endpoint(1) via a Collection with a LabelSelector Discover useragents: If a test with these calls gets merged, Conformance coverage will go up by 4 points* /sig testing /sig architecture /area conformance
/reopen Wait for Promotion PR to Merge.
Reopened this issue. <detailsIn response to : /reopen Wait for Promotion PR to Merge. Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: const ( // https://docs.docker.com/engine/reference/api/docker_remote_api/ <del> // docker version should be at least 1.11.x MinimumDockerAPIVersion = "1.23.0" </del> <ins> // docker version should be at least 1.13.1 MinimumDockerAPIVersion = "1.26.0" </ins> // Status of a container returned by ListContainers. StatusRunningPrefix = "Up"

The current node e2e tests include images using Docker 1.11.x to , with the pending PR to test 18.06 () We should update the images to drop older docker versions. I think we can start by getting rid of 1.11 and 1.12, so that we still have three minor versions tested: , , and 1.13.x /cc (for cos and ubuntu)
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/assign Yang, could you take a look? Thanks


Question: return err } <del> type sortableGroupResource []groupResource </del> <ins> type sortableResource struct { resources []groupResource sortBy    string } </ins> <del> func (s sortableGroupResource) Len() int      { return len(s) } func (s sortableGroupResource) Swap(i, j int) { s[i], s[j] = s[j], s[i] } func (s sortableGroupResource) Less(i, j int) bool { ret := strings.Compare(s[i].APIGroup, s[j].APIGroup) </del> <ins> func (s sortableResource) Len() int { return len(s.resources) } func (s sortableResource) Swap(i, j int) { s.resources[i], s.resources[j] = s.resources[j], s.resources[i] } func (s sortableResource) Less(i, j int) bool { ret := strings.Compare(s.compareValues(i, j)) </ins> if ret > 0 { return false } else if ret == 0 { <del> return strings.Compare(s[i].APIResource.Name, s[j].APIResource.Name) < 0 </del> <ins> return strings.Compare(s.resources[i].APIResource.Name, s.resources[j].APIResource.Name) < 0 </ins> } return true } <ins> func (s sortableResource) compareValues(i, j int) (string, string) { switch s.sortBy { case "name": return s.resources[i].APIResource.Name, s.resources[j].APIResource.Name case "kind": return s.resources[i].APIResource.Kind, s.resources[j].APIResource.Kind } return s.resources[i].APIGroup, s.resources[j].APIGroup } </ins>

<!-- Please only use this template for submitting enhancement requests --What would you like to be : Add a sort parameter (or similar concept) to the command. Why is this needed: With the numerous amount of API resources (not counting custom ones), it is sometimes difficult locating them in the output. By default, API resources are not sorted by the name column (the first on the left) and often this is the column holding the information one wishes to query. A sort parameter would help more easily identify the object to be understood.
/sig usability


Question: set -o nounset set -o pipefail <ins> USER_ID=$(id -u) GROUP_ID=$(id -g) </ins> DOCKER_OPTS=${DOCKER_OPTS:-""} DOCKER=(docker ${DOCKER_OPTS}) DOCKER_HOST=${DOCKER_HOST:-""}

Is this a BUG REPORT or FEATURE REQUEST? (choose one):BUG REPORT Kubernetes version (use ):Client Version: version.Info{Major:"1", Minor:"6+", GitVersion:"v1.6.0-alpha.0.2850+-dirty", GitCommit:"", GitTreeState:"dirty", BuildDate:"2017-01-06T09:06:04Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"6+", GitVersion:"v1.6.0-alpha.0.2753+-dirty", GitCommit:"", GitTreeState:"dirty", BuildDate:"2017-01-04T08:46:56Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"} What happened: How to reproduce it (as minimally and precisely as possible): the owner of file is not right
Do you need to be building as root?
I get an annoying message when building as normal user: So I run as root. There will return a error if the outpt build directory is not owned by root when running as root. I think we could use to fix it.
I hope to get some of your suggestions
did you previously build as ? you might need to before a build as a normal user works again.


Question: } func (a statusStrategy) PrepareForUpdate(ctx context.Context, obj, old runtime.Object) { <ins> // update is only allowed to set status </ins> newCustomResourceObject := obj.(*unstructured.Unstructured) <del> oldCustomResourceObject := old.(*unstructured.Unstructured) </del> newCustomResource := newCustomResourceObject.UnstructuredContent() <del> oldCustomResource := oldCustomResourceObject.UnstructuredContent() // update is not allowed to set spec and metadata _, ok1 := newCustomResource["spec"] _, ok2 := oldCustomResource["spec"] switch { case ok2: newCustomResource["spec"] = oldCustomResource["spec"] case ok1: delete(newCustomResource, "spec") } </del> <ins> status, ok := newCustomResource["status"] </ins> <del> newCustomResourceObject.SetAnnotations(oldCustomResourceObject.GetAnnotations()) newCustomResourceObject.SetFinalizers(oldCustomResourceObject.GetFinalizers()) newCustomResourceObject.SetGeneration(oldCustomResourceObject.GetGeneration()) newCustomResourceObject.SetLabels(oldCustomResourceObject.GetLabels()) newCustomResourceObject.SetOwnerReferences(oldCustomResourceObject.GetOwnerReferences()) newCustomResourceObject.SetSelfLink(oldCustomResourceObject.GetSelfLink()) </del> <ins> // copy old object into new object oldCustomResourceObject := old.(*unstructured.Unstructured) // overridding the resourceVersion in metadata is safe here, we have already checked that // new object and old object have the same resourceVersion. *newCustomResourceObject = *oldCustomResourceObject.DeepCopy() // set status newCustomResource = newCustomResourceObject.UnstructuredContent() if ok { newCustomResource["status"] = status } else { delete(newCustomResource, "status") } </ins> } // ValidateUpdate is the default update validation for an end user updating status.

the said: However the are different, it does not allow to set spec and metadata, but other changes are not ignored. And updates to other parts are not validated.
/cc for some suggestions.
/assign
Made a pull request:
Is the documentation wrong or the implementation? I think there was a longer discussion about the correct behaviour in the PR, but haven't found it yet.
It strange that other parts rather than metadata and spec can be modified and they are not validated. So I think the implementation is wrong, at least.
We should follow the design doc, I think.
/sig api-machinery
/priority important-soon
I agree the current behaviour is wrong. Added some comments to Especially, worries me whether we really have to right /status behaviour. Not really related to the given issue. But as we are touching the /status behaviour, we should clarify that as well.
[MILESTONENOTIFIER] Milestone Issue Needs Approval Action required: This issue must have the label applied by a SIG maintainer. If the label is not applied within 3 days, the issue will be moved out of the v1.11 milestone. <details<summaryIssue Labels</summary- : Issue will be escalated to these SIGs if needed. : Escalate to the issue owners and SIG owner; move out of milestone after several unsuccessful escalation attempts. : Fixes a bug discovered during the current release. </details<details<summaryHelp</summary<ul<li<a href=""Additional instructions</a</li<li<a href=""Commands for setting labels</a</li</ul</details


Question: func mergePluginSet(defaultPluginSet, customPluginSet v1beta2.PluginSet) v1beta2.PluginSet { disabledPlugins := sets.NewString() enabledCustomPlugins := make(map[string]pluginIndex) <ins> // replacedPluginIndex is a set of index of plugins, which have replaced the default plugins. replacedPluginIndex := sets.NewInt() </ins> for _, disabledPlugin := range customPluginSet.Disabled { disabledPlugins.Insert(disabledPlugin.Name) }

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --Recent commit: import an issue when multiple custom plugin configured in scheduler config file, and those plugins are default plugins too. It was manipulating slice with index, but the slice is updated while index is not, which will cause the index out of range, see the exceptions listed below, config file should be handled properly without such exception override multiple default plugin in scheduler's config file. Kubernetes version (use ): latest (1.22) Cloud provider or hardware configuration: OS (e.g: ): all Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others: /assign
/sig scheduling /triage accepted


Question: }}, } pleg.relist() <del> // event channel is full, discard events expected = []*PodLifecycleEvent{ </del> <ins> allEvents := []*PodLifecycleEvent{ </ins> {ID: "1234", Type: ContainerRemoved, Data: "c1"}, {ID: "1234", Type: ContainerDied, Data: "c2"}, {ID: "1234", Type: ContainerStarted, Data: "c3"}, {ID: "4567", Type: ContainerRemoved, Data: "c1"}, <ins> {ID: "4567", Type: ContainerStarted, Data: "c4"}, </ins> } <ins> // event channel is full, discard events </ins> actual = getEventsFromChannel(ch) <del> verifyEvents(t, expected, actual) </del> <ins> assert.True(t, len(actual) == 4, "channel length should be 4") assert.Subsetf(t, allEvents, actual, "actual events should in all events") </ins> } func TestDetectingContainerDeaths(t *testing.T) {

Repro:
/assign
GitHub didn't allow me to assign the following users: changyaowei. Note that only and repo collaborators can be assigned and that issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : /assign Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
this test case was introduced by . Could you verify it ? cc
I believe this is fixed by - which is waiting on a merge. I'll re-ping the assigned reviewer :)
I already fixed it in , it waiting to be merged.


Question: if err != nil { return false, err } <del> existingListeners = append(existingListeners, listenerList...) </del> <ins> for _, l := range listenerList { for _, lb := range l.Loadbalancers { if lb.ID == id { existingListeners = append(existingListeners, l) break } } } </ins> return true, nil })

Kubernetes version: $ kubectl version Client Version: version.Info{Major:"1", Minor:"3+", GitVersion:"v1.3.8-hpe.1", GitCommit:"", GitTreeState:"clean", BuildDate:"2016-09-22T23:27:29Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"3+", GitVersion:"v1.3.8-hpe.1.3+-dirty", GitCommit:"", GitTreeState:"dirty", BuildDate:"2016-09-29T14:43:35Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"} Environment: Cloud provider or hardware configuration: OpenStack Liberty, Neutron LBaaS V2 using Octavia OS (e.g. from /etc/os-release): N/A Kernel (e.g. ): N/A Install tools: N/A Others: N/A What happened: Deleting a LoadBalancer service results in the deletion of all LoadBalancer service's LBaaS Pools and Monitors. What you expected to happen: The Pools and Monitors for unrelated services should not be deleted How to reproduce it (as minimally and precisely as possible): 2 services with type=LoadBalancer for LBaaS resources to be created 1 of the 2 services created earlier output, all pools will have been deleted. Anything else do we need to know: Neutron's API ignores unknown paramaters. When listing pools etc, K8 attempts to filter on "LoadBalancerID", which is not a valid filter. As such, it is ignored by Neutron, and a list of all pools is returned. K8 then proceeds to delete each of the pools.
We're still experiencing this problem, or at least one with very similar symptoms -- having launched services of type LoadBalancer and then deleting one of them, it deletes all of the pools of all services. As the original report, we're also using OpenStack Liberty, Neutron LBaaS V2, and just updated to K8s 1.5.1 (from 1.4.5) hoping that this fixed our issue, but we are still experiencing the same problem after the upgrade.

Is there any other way in which we can help to debug the issue, without delving into the code? (On a side note, the commit which is supposed to fix this is included as part of the 1.5.0 release, so I am not sure if this issue has not been closed for some reason).
it is possible I missed a case in the original fix, the simplest test I found for this issue we had was to create 2x services with type=loadbalancer, wait for all the LBaaS resources to be provisioned, then delete one of the services and ensure the other's resources are still present. If this is failing, either these been a regression, or something else has gone horribly wrong :)
Hi thanks for coming back to this issue. Apparently not all cases deleting the LoadBalancers in OpenStack/liberty were doing the extra step of filtering. We were working on a fix, and submitted a pull request, which is being considered right now:
has merged. closing this issue /close


Question: var endpointColumns = []string{"NAME", "ENDPOINTS", "AGE"} var nodeColumns = []string{"NAME", "STATUS", "AGE"} var daemonSetColumns = []string{"NAME", "DESIRED", "CURRENT", "NODE-SELECTOR", "AGE"} <del> var eventColumns = []string{"FIRSTSEEN", "LASTSEEN", "COUNT", "NAME", "KIND", "SUBOBJECT", "TYPE", "REASON", "SOURCE", "MESSAGE"} </del> <ins> var eventColumns = []string{"LASTSEEN", "FIRSTSEEN", "COUNT", "NAME", "KIND", "SUBOBJECT", "TYPE", "REASON", "SOURCE", "MESSAGE"} </ins> var limitRangeColumns = []string{"NAME", "AGE"} var resourceQuotaColumns = []string{"NAME", "AGE"} var namespaceColumns = []string{"NAME", "STATUS", "AGE"}

Hello Kubernetes, The order of the columns when you get events confused me. I was thinking, Turns out it's just a visual problem where these events are sorted correctly (by ), but the first column is . I think the first column should be because that's the order the events are listed.
This also confused me when I first saw it - I think this ordering would be more intuitive and help users discover the aggregation feature logically, instead of first having to grapple with why events are "out of order"
Should the order be always LASTSEEN,FIRSTSEEN or just when using -w? I have an ugly PR for the latter, but perhaps we should just go for the former.


Question: return retval, nil }, WatchFunc: func(options metav1.ListOptions) (watch.Interface, error) { <del> w := watch.NewFake() </del> <ins> w := watch.NewRaceFreeFake() </ins> if options.ResourceVersion == "1" { go func() { // Close with a "Gone" error when trying to start a watch from the first list

In , when timeout or other error occurs, is called. This will make current routine terminated and miss the , which will lead to some leak routines. For example, this routine will block at waiting for No leak make any of these errors happen No response <details</details<details</details<details</details<details</details<details</details<details</details
sig Testing
I would like to take this up!
Actually, I have a fix in , but its fine if you have a better solution.
/sig api-machinery
/assign /triage accepted


Question: name: "no scale up stabilization", key:  "", recommendations: []timestampedRecommendation{ <del> {4, time.Now().Add(-2 * time.Minute)}, {5, time.Now().Add(-1 * time.Minute)}}, </del> <ins> {4, now.Add(-2 * time.Minute)}, {5, now.Add(-1 * time.Minute)}}, </ins> currentReplicas:              1, // to apply scaleDown delay we should have current > desired prenormalizedDesiredReplicas: 3, expectedStabilizedReplicas:   3, expectedRecommendations: []timestampedRecommendation{ <del> {4, time.Now()}, {5, time.Now()}, {3, time.Now()}, </del> <ins> {4, now}, {5, now}, {3, now}, </ins> }, scaleDownStabilizationWindowSeconds: 60 * 5, },

When configuring new behaviors section in HPA I two scenarios that I think are incorrect. First one: stabilizationWindowSeconds set up to 5 minutes for both scale up and scale down min replica count 2 Application consumes big amount of resources during warm-up 0 minute: current replica count: 2 1 minute: desired replica count: 3 - no scale up stabilization window worked 2 minute: desired replica count: 1 (!) - scale up to 3 (max number from previous recommendations during stabilization window period) ! Second one Stabilization window for scale up 0 Stabilization window for scale down 5 minutes min replica count 2 behavior set to scale up new pod in 2 minutes scaleUp: policies: type: Pods value: 1 periodSeconds: 120 What happens: 0 minute: 2 replicas 1 minute : performance spike scale up to 3 (desired value 4) 2 minute : no spike no scale up 3 minute: no spike scale up to 4(!) - supposedly desired value 1 and supposedly worked this way because of scale down window set to 5 minutes picking max value from that 5 minutes window that was 4 replicas Expected behaviors 1 Scenario no scale up for 5 minutes 2 Scenario correct measurement and no scale up when there is no need for it Environment: Kubernetes version (use ): client 1.19.3 Server 1.19.0 Cloud provider or hardware configuration: Azure aks
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig autoscaling
/assign josephburnett
The HPA controller keeps a flat history of recommendations for stabilization. ! When downscale stabilizing the maximum recommendation is selected. Purple is pre-stabilized recommendation. Green is final recommendation. Blue is max recommendation within window. ! When upscale stabilization the minimum recommendation is selected. ! However when both up and down scale stabilization is configured, the interpretation of the history changes depending on the direction of movement.

Red is the undesired, actuated recommendation. ! What we want is to keep the stabilized recommendation within the envelope of the minimum and maximum over configured stabilization windows. We should only move when the envelope forces a move. ! We first stabilize up and down, and check if the current value fits within it. If it does (or the range is "inverted"), we do nothing. If it does not, we emit the "nearest possible" value, which will be lower or upper bound, depending on where the current size is in relation to the range. (thanks and for the paragraph above)


Question: if sysruntime.GOOS == "linux" { // AppArmor is a Linux kernel security module and it does not support other operating systems. klet.appArmorValidator = apparmor.NewValidator() <del> klet.softAdmitHandlers.AddPodAdmitHandler(lifecycle.NewAppArmorAdmitHandler(klet.appArmorValidator)) </del> <ins> klet.admitHandlers.AddPodAdmitHandler(lifecycle.NewAppArmorAdmitHandler(klet.appArmorValidator)) </ins> } leaseDuration := time.Duration(kubeCfg.NodeLeaseDurationSeconds) * time.Second

Kubelet soft admission handlers were created as a way for the Kubelet to block pods that cannot be run for reasons that can be resolved. In practice, this mechanism was only ever used by AppArmor (). Problems with soft-admission rejection: is only used by AppArmor, and only for situations that cannot realistically be resolved (host or build does not support apparmor) held in the blocked state must be manually cleaned up. While there they hold reserved capacity from the scheduler and continually retry syncpod. other features work this way - usually Kubelet hard rejects pods that it knows it cannot run, or the runtime fails to start the pod/container. This was originally to prevent controllers from continuously recreating pods that cannot run, but that is a separate problem that should be approached globally, rather than with this 1-off AppArmor solution. I propose we remove Kubelet soft admission entirely, and move the AppArmor admission handler to the hard-admission list. /sig node /milestone v1.31 /kind bug
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
We have a different, alpha mechanism: . We could change AppArmor, seccomp and other mechanisms to work either with PodSchedulingContext or its eventual replacement. IMO we only need to sketch out an approach and see that it looks viable; we don't have to block graduation for AppArmor support on a fleshed-out design.
From my admittedly limited PoV I'm not against removing soft admission, and I got no opinions about AppArmor specifically. However this seems to connected to the recent conversation we're having in WG serving and SIG-node about retriable vs unretriable errors so I think we would need some coordination cc
+1
My understanding is that after this PR: soft admission just doesn't make sense any longer.
Hello ! Just a reminder, the code freeze is starting (about from now), and while there is still plenty of time, we want to ensure that each PR has a chance to be merged on time.

As the issue is tagged for , is it still planned for this release?
Discussed in sig-node, decided to move forward with this (move AppArmor admission to the hard admission set, remove soft-admission code)
/assign
- do you want me to incorporate this change into


Question: }), "Memory": ptrMatchAllFields(gstruct.Fields{ "Time":            recent(maxStatsAge), <del> "AvailableBytes":  bounded(100*framework.Mb, 100*framework.Gb), "UsageBytes":      bounded(10*framework.Mb, 10*framework.Gb), "WorkingSetBytes": bounded(10*framework.Mb, 10*framework.Gb), </del> <ins> "AvailableBytes":  bounded(100*framework.Mb, memoryLimit), "UsageBytes":      bounded(10*framework.Mb, memoryLimit), "WorkingSetBytes": bounded(10*framework.Mb, memoryLimit), </ins> // this now returns /sys/fs/cgroup/memory.stat total_rss <del> "RSSBytes":        bounded(1*framework.Kb, 1*framework.Gb), </del> <ins> "RSSBytes":        bounded(1*framework.Kb, memoryLimit), </ins> "PageFaults":      bounded(1000, 1E9), "MajorPageFaults": bounded(0, 100000), }),

/kind bug What happened: e2e node summary test fails on machines with more than 10Gb of ram. What you expected to happen: memory limit bounds to look at node capacity rather than hardcode an upper bound. How to reproduce it (as minimally and precisely as possible): have more ram on a box and run e2e-node FOCUS=Summary Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others:
/sig node


Question: return "", imagePullResult.err.Error(), ErrImagePull } <del> m.podPullingTimeRecorder.RecordImageFinishedPulling(pod.UID) m.logIt(ref, v1.EventTypeNormal, events.PulledImage, logPrefix, fmt.Sprintf("Successfully pulled image %q in %v", container.Image, time.Since(startTime)), klog.Info) </del> <ins> m.logIt(ref, v1.EventTypeNormal, events.PulledImage, logPrefix, fmt.Sprintf("Successfully pulled image %q in %v (%v including waiting)", container.Image, imagePullResult.pullDuration, time.Since(startTime)), klog.Info) </ins> m.backOff.GC() return imagePullResult.imageRef, "", nil }

While going over Kubernetes events I noticed that image pulls for some small images were taking several minutes according to the "Successfully pulled image" event. From investigating this is misleading as the actual pull time is much smaller but due to --serialize-image-pulls=true being the default in the observed cluster and other pods being initialized at the time, the reported pull time ends up accounting for time while waiting for other pods to finish pulling. I expected to see the time it takes to download the image itself from a registry. Customers seem to rely on this metric for this information and we have received some complaints as a result where customers noted slow pull times as a result of misunderstanding the current behavior of this metric. Start with a K8s cluster with --serialize-image-pulls=true (default) - For my check I used default minikube v1.26.1 pod-large where pod-large pulls any large image (the image cannot be locally cached) after step 1 (within seconds) create pod-small where pod-small pulls any small image Check the output for the event after running: Note that despite the small size, pull time for the image in pod-small is several minutes, we should instead see it take seconds (which is how long the pull actually takes). To note this is the case you can try pulling another small image like alpine that isnt cached locally and compare after the above are done. Here is the code that pulls an image in kubernetes and logs the event: As can be seen from it, it's not true pull time, just time from invoking the puller to receiving a result. The serial puller is using a Golang channel as a queue and will just wait until a previous image is done pulling as I noted before, thus it will end up recording the amount of time in the queue + pull time yielding a misleading metric. <details</details<detailsShould reproduce anywhere (tried it on minikube) but have personally observed this in Azure as well </details<details</details<details</details<details</details<details</details
/sig node
But means that kubelet can pull multi images at the same time. For kubelet, is by default true. is for promoting the use of parallel image pulls. The problem maybe with . I did some simple test with 3 ubuntu images as below. The time was collected from events.

-- --serialize-image-pulls=true -- -- image pulling serialize ubuntu:22.10 73.6 MiB 1m21.284220239s ubuntu:20.04 75.7 MiB 2m7.876426944s ubuntu:14.04 212.9 MiB 4m52.884660938s
You are right thanks for taking a look at this. It seems I got the two reversed in my head. The issue indeed happens when --serialize-image-pulls=true which is the default as you pointed out. I'll update the issue to reflect that
/triage accepted /priority important-longterm


Question: return jwt.Encode() } <del> func generateGoodToken(t *testing.T, op *oidctesting.OIDCProvider, iss, sub, aud string, usernameClaim, value, groupsClaim string, groups []string) string { </del> <ins> func generateGoodToken(t *testing.T, op *oidctesting.OIDCProvider, iss, sub, aud string, usernameClaim, value, groupsClaim string, groups interface{}) string { </ins> return generateToken(t, op, iss, sub, aud, usernameClaim, value, groupsClaim, groups, time.Now(), time.Now().Add(time.Hour)) } <del> func generateMalformedToken(t *testing.T, op *oidctesting.OIDCProvider, iss, sub, aud string, usernameClaim, value, groupsClaim string, groups []string) string { </del> <ins> func generateMalformedToken(t *testing.T, op *oidctesting.OIDCProvider, iss, sub, aud string, usernameClaim, value, groupsClaim string, groups interface{}) string { </ins> return generateToken(t, op, iss, sub, aud, usernameClaim, value, groupsClaim, groups, time.Now(), time.Now().Add(time.Hour)) + "randombits" } <del> func generateExpiredToken(t *testing.T, op *oidctesting.OIDCProvider, iss, sub, aud string, usernameClaim, value, groupsClaim string, groups []string) string { </del> <ins> func generateExpiredToken(t *testing.T, op *oidctesting.OIDCProvider, iss, sub, aud string, usernameClaim, value, groupsClaim string, groups interface{}) string { </ins> return generateToken(t, op, iss, sub, aud, usernameClaim, value, groupsClaim, groups, time.Now().Add(-2*time.Hour), time.Now().Add(-1*time.Hour)) }

<!-- Thanks for filing an issue! Before hitting the button, please answer these questions.--Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see ): No, already discussed on sig-auth What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.): confirmed issue on sig-auth channel Is this a BUG REPORT or FEATURE REQUEST? (choose one): Depends on interpretation <!-- If this is a BUG REPORT, please: Fill in as much of the template below as you can. If you leave out information, we can't help you as well. If this is a FEATURE REQUEST, please: Describe in detail the feature/behavior/change you'd like to see. In both cases, be ready for followup questions, and please respond in a timely manner. If we can't reproduce a bug or think a feature already exists, we might close your issue. If we're wrong, PLEASE feel free to reopen it and explain why. --Kubernetes version (use ): Client Version: version.Info{Major:"1", Minor:"3", GitVersion:"v1.3.0", GitCommit:"", GitTreeState:"clean", BuildDate:"2016-07-01T19:26:38Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"darwin/amd64"} Server Version: version.Info{Major:"1", Minor:"3", GitVersion:"v1.3.4+coreos.0", GitCommit:"", GitTreeState:"clean", BuildDate:"2016-08-02T00:54:53Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"} Environment: Cloud provider or hardware configuration: CoreOS Vagrant Image OS (e.g. from /etc/os-release): NAME=CoreOS ID=coreos VERSION=1122.0.0 VERSIONID=1122.0.0 BUILDID=2016-07-27-0739 PRETTYNAME="CoreOS 1122.0.0 (MoreOS)" ANSICOLOR="1;32" HOMEURL="" BUGREPORTURL="" Kernel (e.g.

): Linux localhost 4.7.0-coreos SMP Wed Jul 27 07:30:04 UTC 2016 x8664 Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz GenuineIntel GNU/Linux Install tools: Vagrant Others: What happened: The claim named in --oidc-groups-claim MUST be list of strings, even if there is only one value. Talking to this is by design since its assumed that users will have more then one group. What you expected to happen: The OIDC authenticator should accept either a list of strings or a single string since: There is no concept of schema that marks an attribute as single value or multi value in JSON or a JWT There is no standard for representing groups in a JWT There is no standard for representing groups in OIDC An OIDC should be flexible in how it accepts data. How to reproduce it (as minimally and precisely as possible): Deploy with RBAC enabled Create a role binding for the group "admin" Generate a JWT such as the one that does NOT work below User won't be able to login Generate a JWT where the role is multi-value User will be authorized Does NOT Work: DOES Work: Anything else do we need to know:
These should be called groups, not roles, right? Those are different. And, is it normal for a JWT to contain keys which are arrays? That seems weird.
where do you see role defined? you mean the user_role claim?
Claims can be structured, yes. The name of the groups claim is arbitrary. Could easily have been "group". In my mind, this is all about enabling mapping a "groupish" claim from whatever IDP issued the JWT, even if it is a single value instead of an array.
That's the way it works currently. Yeah I'm fine with this.
Okay.


Question: selector: matchLabels: k8s-app: elasticsearch-logging <del> version: v7.4.2 </del> <ins> version: v7.4.2-1 </ins> template: metadata: labels: k8s-app: elasticsearch-logging <del> version: v7.4.2 </del> <ins> version: v7.4.2-1 </ins> spec: serviceAccountName: elasticsearch-logging containers: <del> - image: quay.io/fluentd_elasticsearch/elasticsearch:v7.4.2 name: elasticsearch-logging imagePullPolicy: Always resources: # need more cpu upon initialization, therefore burstable class limits: cpu: 1000m memory: 3Gi requests: cpu: 100m memory: 3Gi ports: - containerPort: 9200 name: db protocol: TCP - containerPort: 9300 name: transport protocol: TCP livenessProbe: tcpSocket: port: transport initialDelaySeconds: 5 timeoutSeconds: 10 readinessProbe: tcpSocket: port: transport initialDelaySeconds: 5 timeoutSeconds: 10 volumeMounts: - name: elasticsearch-logging mountPath: /data env: - name: "NAMESPACE" valueFrom: fieldRef: fieldPath: metadata.namespace </del> <ins> - image: quay.io/fluentd_elasticsearch/elasticsearch:v7.4.2 name: elasticsearch-logging imagePullPolicy: Always resources: # need more cpu upon initialization, therefore burstable class limits: cpu: 1000m memory: 3Gi requests: cpu: 100m memory: 3Gi ports: - containerPort: 9200 name: db protocol: TCP - containerPort: 9300 name: transport protocol: TCP livenessProbe: tcpSocket: port: transport initialDelaySeconds: 5 timeoutSeconds: 10 readinessProbe: tcpSocket: port: transport initialDelaySeconds: 5 timeoutSeconds: 10 volumeMounts: - name: elasticsearch-logging mountPath: /data env: - name: "NAMESPACE" valueFrom: fieldRef: fieldPath: metadata.namespace - name: "MINIMUM_MASTER_NODES" value: "1" </ins> volumes: <del> - name: elasticsearch-logging emptyDir: {} </del> <ins> - name: elasticsearch-logging emptyDir: {} </ins> # Elasticsearch requires vm.max_map_count to be at least 262144. # If your OS already sets up this number to a higher value, feel free # to remove this init container. initContainers: <del> - image: alpine:3.6 command: ["/sbin/sysctl", "-w", "vm.max_map_count=262144"] name: elasticsearch-logging-init securityContext: privileged: true </del> <ins> - image: alpine:3.6 command: ["/sbin/sysctl", "-w", "vm.max_map_count=262144"] name: elasticsearch-logging-init securityContext: privileged: true </ins>

What happened: I download fluentd-elasticsearch in addons. The following error occurs then I use "kubectl apply -f es- " Then, i find the elasticsearch-logging pod restart lots of time, and keep status with CrashLoopBackOff. [root ~]# kubectl logs -f elasticsearch-logging-0 -n kube-system export NODENAME=elasticsearch-logging-0 NODENAME=elasticsearch-logging-0 export NODEMASTER=true NODEMASTER=true export NODEDATA=true NODEDATA=true export HTTPPORT=9200 HTTPPORT=9200 export TRANSPORTPORT=9300 TRANSPORTPORT=9300 export MINIMUMMASTERNODES=2 MINIMUMMASTERNODES=2 chown -R elasticsearch:elasticsearch /data ./bin/elasticsearchloggingdiscovery I0902 14:25:06. 9 ] Kubernetes Elasticsearch logging discovery I0902 14:25:06. 9 ] Found [] I0902 14:25:16. 9 ] Found [] I0902 14:25:26. 9 ] Found [] I0902 14:25:36. 9 ] Found [] I0902 14:25:46. 9 ] Found [] I0902 14:25:56. 9 ] Found []
unclear whether the addon is up to date and supported at this point. /sig instrumentation
cc
/assign I will take a look this week.
I have the same problem k8s version:v1.18.8
same
I've pushed a change to the stateful set that should unblock people as there was a 1.18 compatibility problem. however, please make sure you are also creating the service via , the discovery program finds the other elasticsearch nodes via service registration.
please feel free to re-open this issue if the latest version doesn't work for you.
Why was this PR to v1.20?
probably because that is the next release. most people who use this addon do so from master. Do you need this packaged in some other way?


Question: "//cmd/kubeadm/app/constants:go_default_library", "//cmd/kubeadm/app/util/config:go_default_library", "//staging/src/k8s.io/api/core/v1:go_default_library", <del> "//staging/src/k8s.io/apimachinery/pkg/api/errors:go_default_library", </del> "//staging/src/k8s.io/apimachinery/pkg/apis/meta/v1:go_default_library", "//staging/src/k8s.io/apimachinery/pkg/runtime:go_default_library", "//staging/src/k8s.io/client-go/kubernetes/fake:go_default_library", <del> "//staging/src/k8s.io/client-go/testing:go_default_library", </del> ], )

What happened: Running fails consistently with timeout errors What you expected to happen: Unit tests should complete without any errors. If this was a configuration problem (e.g. unsupported go version), then I should see a meaningful error rather than just a timeout error. How to reproduce it (as minimally and precisely as possible): Run Anything else we need to know?: This issue has been present in my system for months. Environment: Kubernetes version (use ): master Cloud provider or hardware configuration: OS (e.g: ): Ubuntu 20.04 LTS (Focal Fossa) Kernel (e.g. ): Linux desktop 5.4.0-37-generic -Ubuntu SMP Wed Jun 3 18:57:02 UTC 2020 x8664 x8664 x86_64 GNU/Linux Install tools: Network plugin and version (if this is a network-related bug): Others: go version go1.13.12 linux/amd64 (snap)
/sig testing
/assign
Issues targeted at the specific problematic packages and responsible SIGs will be more effective in getting this addressed
/remove-sig testing Unless we expect some broad testing change (like reconfiguring go test) this isn't SIG testing. This is SIG storage, node, and cluster lifecycle
/sig node /sig cluster-lifecycle /sig storage tagged the right sigs, though per-package issues are far more likely to drive getting this fixed
fix for cmd/kubeadm/app/phases/uploadconfig:
I'm going to open two more issues for the other two packages and work on them
sounds good, in that case please close this one.
New issues for reconciler and csi packages: ,
Closing. This problem has come up in the past, the solution is to just increase the timeout. I missed this the first time I read the doc.


Question: return &nodeCopy, nil } } <del> return nil, nil </del> <ins> return nil, apierrors.NewNotFound(schema.GroupResource{Resource: "nodes"}, name) </ins> } func (m *FakeNodeHandler) runAsyncCalls() {

pkg/controller in I transformed that testcase into a fuzz driver and tested it with ]() It crash with "invalid memory address or nil pointer dereference" when some argument related data are fuzzed. Always No response No response Simplified PoC code is here: and it crashed as: /sig
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/kind cleanup /sig node /priority backlog This nil pointer is never reported by end users which means that before we call AddOrUpdateTaintOnNode, the node existence is checked. To make the code more stable, fixing it is valid. This would avoid future bugs if somewhere calls AddOrUpdateTaintOnNode without checking whether the node exists. I opened for this. /remove-kind failing-test (which means falling test ci )
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale
The Kubernetes project currently lacks enough contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue as fresh with Close this issue with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue as fresh with Close this issue with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle rotten [1]:


Question: # Delete the master replica pd (possibly leaked by kube-up if master create failed). # TODO(jszczepkowski): remove also possibly leaked replicas' pds <del> local -r replica-pd="${REPLICA_NAME:-${MASTER_NAME}}-pd" if gcloud compute disks describe "${replica-pd}" --zone "${ZONE}" --project "${PROJECT}" &>/dev/null; then </del> <ins> local -r replica_pd="${REPLICA_NAME:-${MASTER_NAME}}-pd" if gcloud compute disks describe "${replica_pd}" --zone "${ZONE}" --project "${PROJECT}" &>/dev/null; then </ins> gcloud compute disks delete  --project "${PROJECT}"  --quiet  --zone "${ZONE}"  <del> "${replica-pd}" </del> <ins> "${replica_pd}" </ins> fi # Delete disk for cluster registry if enabled

./hack/e2e- line 1192: local: `replica-pd=jenkins-us-central1-b-master-pd': not a valid identifier FYI - this looks related to your recent work? I don't see any obvious PR to blame, but I seem to remember you guys touching this part of the code recently? BTW, I deleted a bunch of leaked resources in the relevant test project on Friday (9/16), assuming that the test scripts would re-create whatever they needed. It's possible (but unlikely) that this has anything to do with that.
Here are some example logs to aid debugging: https://k8s-
I believe this to be the source of dysfunction: Which is turn caused by what you found, which causes the disk to not be cleaned up ahead of time: You can try this out in your local shell: git blames leads us to this recent commit: cc the author The root problem is unfortunatley very annoying: this is how bash tells you that variable names (in this case, $replica-pd) cannot contain dashes!
fix is up in
Awesome many thanks. I'll make sure it gets merged ASAP.