Question: Follow:         true, Previous:       true, SinceSeconds:   &sinceSeconds, <ins> TailLines:      nil, </ins> SinceTime:      &sinceTime, // test a custom marshaller EmptyTime:      nil,        // test a nil custom marshaller without omitempty NonPointerTime: sinceTime,

How works: : prints all lines. : prints the last line. : prints no lines. (Usually used with to follow new logs) How works: : prints all lines. : prints the last line. : prints all lines. <-------------- Diffierent. The reason is that the is ignored when converting the parameters Confirmed with What is the expected behavior? /cc
I've also noticed that even if or some arbitrarily low number is specified, it tries to read the whole logfile into memory. If it's a big file it can crash the worker node.
Do you mean the docker implementation? If I read their code correctly, docker read the whole log before tail into memory once tail is specified, which may cause problem if tail is a very large number. So in our own implementation, I've avoided that. ;) However, where do you see the ?
Tail should be *int in the API, is it not? On Oct 21, 2016, at 6:01 PM, Lantao Liu wrote: How docker logs works: docker logs --tail=-1 CONTAINERID: prints all lines. docker logs --tail=1 CONTAINERID: prints the last line. docker logs --tail=0 CONTAINERID: prints no lines. (Usually used with -f to follow new logs) How kubectl logs works: kubectl logs --tail=-1 PODNAME: prints all lines. kubectl logs --tail=1 PODNAME: prints the last line. kubectl logs --tail=0 PODNAME: prints all lines. <-------------- Diffierent. The reason is that the tail=0 is ignored when converting the parameters . Confirmed with /cc — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub , or mute the thread .
Agree tail=0 no logs is intended behavior On Oct 21, 2016, at 6:01 PM, Lantao Liu wrote: How docker logs works: docker logs --tail=-1 CONTAINERID: prints all lines. docker logs --tail=1 CONTAINERID: prints the last line. docker logs --tail=0 CONTAINERID: prints no lines. (Usually used with -f to follow new logs) How kubectl logs works: kubectl logs --tail=-1 PODNAME: prints all lines.

kubectl logs --tail=1 PODNAME: prints the last line. kubectl logs --tail=0 PODNAME: prints all lines. <-------------- Diffierent. The reason is that the tail=0 is ignored when converting the parameters . Confirmed with /cc — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub , or mute the thread .
Yeah, tail is . But if , kubelet will see .
I think the linked to is the problem.
The obvious workaround is to instead use: kubectl logs --tail=1
/sig cli
The line pasted by chao implies omitempty should not be false for tail.
I hate to bump, but 'bump'. Still an issue here. Can confirm works but does not.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale This is still relevant.
Any good news?


Question: return newSizeQuant, nil } <ins> if result.DiskProperties.DiskState != compute.Unattached { return oldSize, fmt.Errorf("azureDisk - disk resize is only supported on Unattached disk, current disk state: %s, already attached to %s", result.DiskProperties.DiskState, to.String(result.ManagedBy)) } </ins> diskParameter := compute.DiskUpdate{ DiskUpdateProperties: &compute.DiskUpdateProperties{ DiskSizeGB: &requestGiB,

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: resize Azure disk issue when it's in attached state: What you expected to happen: How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others: /priority important-soon /sig cloud-provider /area provider/azure /triage accepted /kind bug /assign


Question: - [Server Binaries](#server-binaries) - [Node Binaries](#node-binaries) - [Kubernetes v1.14 Release Notes](#kubernetes-v114-release-notes) <del> - [1.14 What’s New](#114-what’s-new) </del> <ins> - [1.14 What’s New](#114-whats-new) </ins> - [Known Issues](#known-issues) - [Urgent Upgrade Notes](#urgent-upgrade-notes) <del> - [(No, really, you MUST do this before you upgrade)](#no-really-you-must-do-this-before-you-upgrade) </del> <ins> - [(No, really, you MUST do this before you upgrade)](#no-really-you-must-read-this-before-you-upgrade) </ins> - [Deprecations](#deprecations) - [Removed and deprecated metrics](#removed-and-deprecated-metrics) - [Removed metrics](#removed-metrics)

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: When I click the "1.14 What's New"and "(No, really, you MUST do this before you upgrade)" I found that it can not jump to the content. What you expected to happen: Click the "1.14 What's New" then I can jump to the "1.14 What's New"content. How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Others:


Question: }}, }, true) } <del> func TestForgetWatcher(t *testing.T) { backingStorage := &dummyStorage{} cacher, _, err := newTestCacher(backingStorage) require.NoError(t, err) defer cacher.Stop() require.Equal(t, 0, len(cacher.watchers.allWatchers)) require.Equal(t, 0, len(cacher.watchers.valueWatchers)) var forgetWatcherFn func(bool) var forgetCounter int forgetWatcherWrapped := func(drainWatcher bool) { forgetCounter++ forgetWatcherFn(drainWatcher) } w := newCacheWatcher( 0, func(_ string, _ labels.Set, _ fields.Set) bool { return true }, nil, storage.APIObjectVersioner{}, testingclock.NewFakeClock(time.Now()).Now().Add(2*time.Minute), true, schema.GroupResource{Resource: "pods"}, "1", ) forgetWatcherFn = forgetWatcher(cacher, w, 0, namespacedName{}, "", false) cacher.watchers.addWatcher(w, 0, namespacedName{}, "", false) require.Equal(t, 0, len(cacher.watchers.valueWatchers)) require.Equal(t, 1, len(cacher.watchers.allWatchers)) forgetWatcherWrapped(false) require.Equal(t, 0, len(cacher.watchers.allWatchers)) require.Equal(t, 0, len(cacher.watchers.valueWatchers)) require.Equal(t, 1, forgetCounter) forgetWatcherWrapped(false) require.Equal(t, 0, len(cacher.watchers.allWatchers)) require.Equal(t, 0, len(cacher.watchers.valueWatchers)) require.Equal(t, 2, forgetCounter) } </del>

A data race manifested in job link: NA NA No response No response No response /sig api-machinery
/cc
/assign
ack, fixing ...
/triage accepted


Question: fmtField("Parent timestamp", fmt.Sprintf("%s", t)) fmtField("Reserved", hexs(header.Reserved[:])) <del> parentName := utf16BytesToString(header.ParentUnicodeName[:], binary.BigEndian) </del> <ins> parentNameBytes, _, err := transform.Bytes( unicode.UTF16(unicode.BigEndian, unicode.IgnoreBOM).NewDecoder(), header.ParentUnicodeName[:], ) if err != nil { panic(err) } parentName := string(parentNameBytes) </ins> fmtField("Parent Name", parentName) // Parent locator entries ignored since it's a dynamic disk sum := 0

I ran an OSS license scan of a codebase that has this project as a dependency and came across an interesting hit. This go-vhd dependency specified here: has a snippet of code from StackOverflow which is license under creative commons sharealike. Here's the code in question: I'm not a license expert, so I don't know if there are any real issues/conflicts here or not, but wanted to share to see what you all think. Here's SO's license info for reference: My understanding is that the potential issue would be with this clause: which could potentially require go-vhd and projects that depend on it to use the Creative Commons license as well.
Thanks for the report! can you please take a look if this is concerning? If so, we'll check if we can avoid this dependency (looks unlikely unless we duplicate what it's ) or we'll follow up upstream to update the code. /cc OWNERS for this repo
CC-BY-SA can be problematic in Apache-2.0 projects like k8s because of the ShareAlike aspect as noted. I'd recommend working with upstream to rewrite just that one function particularly since it looks like it's fairly limited.
let's please move this issue to kubernetes/kubernetes?
/provider azure
/sig release /area code-organization
The label(s) cannot be applied, because the repository doesn't have them <detailsIn response to : /sig release /area code-organization Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
cc /assign
/area code-organization
/assign
/milestone v1.19 /priority important-soon
Not a huge difference, but this is likely actually licensed under , not 4.0. See
the k/k code actually does not reference , since that function is in , do you mean we should remove from k/k totally?
Yes, we would either need to remove from k/k or ask the maintainers of to update . This means that (and k/k if we end up using it) would need to be licensed until CC-BY-SA.
Would this ultimately be backported to older versions of k8s or would you just do it in 1.19?

thanks, Let me remove the code in v1.19, actually that part of functionality is deprecated in Azure cloud provider already. My initial thought is only do this in v1.19
/kind bug /priority important-soon /sig cloud-provider /area provider/azure
Since it isn't trivial to remove as a dependency entirely (), I've created upstream to track updating the code there.


Question: var externalKubeClient clientset.Interface clientConfig, err := createAPIServerClientConfig(s) <ins> if err != nil { return fmt.Errorf("invalid kubeconfig: %v", err) } </ins> var clientCertificateManager certificate.Manager <del> if err == nil { if s.RotateCertificates && utilfeature.DefaultFeatureGate.Enabled(features.RotateKubeletClientCertificate) { clientCertificateManager, err = kubeletcertificate.NewKubeletClientCertificateManager(s.CertDirectory, nodeName, clientConfig.CertData, clientConfig.KeyData, clientConfig.CertFile, clientConfig.KeyFile) if err != nil { return err } // we set exitIfExpired to true because we use this client configuration to request new certs - if we are unable // to request new certs, we will be unable to continue normal operation if err := kubeletcertificate.UpdateTransport(wait.NeverStop, clientConfig, clientCertificateManager, true); err != nil { return err } } kubeClient, err = clientset.NewForConfig(clientConfig) </del> <ins> if s.RotateCertificates && utilfeature.DefaultFeatureGate.Enabled(features.RotateKubeletClientCertificate) { clientCertificateManager, err = kubeletcertificate.NewKubeletClientCertificateManager(s.CertDirectory, nodeName, clientConfig.CertData, clientConfig.KeyData, clientConfig.CertFile, clientConfig.KeyFile) </ins> if err != nil { <del> glog.Warningf("New kubeClient from clientConfig error: %v", err) } else if kubeClient.CertificatesV1beta1() != nil && clientCertificateManager != nil { glog.V(2).Info("Starting client certificate rotation.") clientCertificateManager.SetCertificateSigningRequestClient(kubeClient.CertificatesV1beta1().CertificateSigningRequests()) clientCertificateManager.Start() </del> <ins> return err </ins> } <del> externalKubeClient, err = clientset.NewForConfig(clientConfig) if err != nil { glog.Warningf("New kubeClient from clientConfig error: %v", err) </del> <ins> // we set exitIfExpired to true because we use this client configuration to request new certs - if we are unable // to request new certs, we will be unable to continue normal operation if err := kubeletcertificate.UpdateTransport(wait.NeverStop, clientConfig, clientCertificateManager, true); err != nil { return err </ins> } <ins> } </ins> <del> // make a separate client for events eventClientConfig := *clientConfig eventClientConfig.QPS = float32(s.EventRecordQPS) eventClientConfig.Burst = int(s.EventBurst) eventClient, err = v1core.NewForConfig(&eventClientConfig) if err != nil { glog.Warningf("Failed to create API Server client for Events: %v", err) } </del> <ins> kubeClient, err = clientset.NewForConfig(clientConfig) if err != nil { glog.Warningf("New kubeClient from clientConfig error: %v", err) } else if kubeClient.CertificatesV1beta1() != nil && clientCertificateManager != nil { glog.V(2).Info("Starting client certificate rotation.") clientCertificateManager.SetCertificateSigningRequestClient(kubeClient.CertificatesV1beta1().CertificateSigningRequests()) clientCertificateManager.Start() } externalKubeClient, err = clientset.NewForConfig(clientConfig) if err != nil { glog.Warningf("New kubeClient from clientConfig error: %v", err) } </ins> <del> // make a separate client for heartbeat with throttling disabled and a timeout attached heartbeatClientConfig := *clientConfig heartbeatClientConfig.Timeout = s.KubeletConfiguration.NodeStatusUpdateFrequency.Duration heartbeatClientConfig.QPS = float32(-1) heartbeatClient, err = v1core.NewForConfig(&heartbeatClientConfig) if err != nil { glog.Warningf("Failed to create API Server client for heartbeat: %v", err) } } else { switch { case s.RequireKubeConfig: return fmt.Errorf("invalid kubeconfig: %v", err) case s.KubeConfig.Provided(): glog.Warningf("invalid kubeconfig: %v", err) } </del> <ins> // make a separate client for events eventClientConfig := *clientConfig eventClientConfig.QPS = float32(s.EventRecordQPS) eventClientConfig.Burst = int(s.EventBurst) eventClient, err = v1core.NewForConfig(&eventClientConfig) if err != nil { glog.Warningf("Failed to create API Server client for Events: %v", err) } // make a separate client for heartbeat with throttling disabled and a timeout attached heartbeatClientConfig := *clientConfig heartbeatClientConfig.Timeout = s.KubeletConfiguration.NodeStatusUpdateFrequency.Duration heartbeatClientConfig.QPS = float32(-1) heartbeatClient, err = v1core.NewForConfig(&heartbeatClientConfig) if err != nil { glog.Warningf("Failed to create API Server client for heartbeat: %v", err) </ins> } kubeDeps.KubeClient = kubeClient

This will be the end of the deprecation period for --require-kubeconfig.
There are no sig labels on this issue. Please by:<br(1) mentioning a sig: <br(2) specifying the label manually: <br<brNote: method (1) will trigger a notification to the team. You can find the team list .
ref
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. Prevent issues from auto-closing with an comment. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale


Question: targetIP := *ipConfig.PrivateIPAddress return targetIP, nil } <ins> // splitProviderID converts a providerID to a NodeName. func splitProviderID(providerID string) (types.NodeName, error) { matches := providerIDRE.FindStringSubmatch(providerID) if len(matches) != 2 { return "", errors.New("error splitting providerID") } return types.NodeName(matches[1]), nil } </ins>

This is required to make Cloud Controller Manager operational on the various clouds that Kubernetes runs on. We have already PRs for the following clouds: GCE - AWS - Azure - OpenStack - The following clouds still need to be addresses: CloudStack Mesos OVirt Photon Rackspace vSphere If anyone is interested in contributing to this effort, please ping me directly (on kubernetes slack or through email)
There are no sig labels on this issue. Please by:<br(1) mentioning a sig: <br(2) specifying the label manually: <br<brNote: method (1) will trigger a notification to the team. You can find the team list and label list
/sig cluster-lifecycle
cc:
I will take care of the vSphere and Photon.
I will take care of the Rackspace.
Merging for OpenStack
Reposting I am trying to use my custom out-of-tree cloud provider with Kubernetes. But I noticed that kubelet calls apis like with empty for cloud provider. Digging in the code for 1.7 branch, I see that is null for cloud-providers. As a result returns empty string. As a result, ByID() methods don't work for cloud providers since is not set.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. Prevent issues from auto-closing with an comment. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten /remove-lifecycle stale
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-testing, kubernetes/test-infra and/or . /close


Question: names[ref.Name] = ref } for i := range pod.Spec.InitContainers { <del> ref := &pod.Spec.Containers[i] </del> <ins> ref := &pod.Spec.InitContainers[i] </ins> names[ref.Name] = ref } for i := range pod.Spec.EphemeralContainers { <del> ref := &pod.Spec.Containers[i] </del> <ins> ref := (*corev1.Container)(&pod.Spec.EphemeralContainers[i].EphemeralContainerCommon) </ins> names[ref.Name] = ref } return names

What happened: When using to create a debug container for a pod that has multiple init containers or ephemeral containers, fails with . What you expected to happen: should create a debug container in pod successfully. How to reproduce it (as minimally and precisely as possible): The EphemeralContainers feature gate must be enabled by using flag in your cluster. Anything else we need to know?: Environment: Kubernetes version (use ):Cloud provider or hardware configuration: OS (e.g: ): NAME="CentOS Linux" VERSION="7 (Core)" Kernel (e.g. ): Linux node-1 3.10.0-957.el7.x8664 SMP Thu Nov 8 23:39:32 UTC 2018 x8664 x8664 x8664 GNU/Linux Install tools: Network plugin and version (if this is a network-related bug): Others: /sig cli
I have investigated this issue and it looks like this issue happens in the following cases. When using without option When using with option In those cases, the containerNameToRef function is called and each for-loop in the function refers to the same despite the range is , or .
cc If this issue needs to be fixed, I would like to work on this.
/assign
Thanks for the report and for fixing it!


Question: <del> /* Copyright 2016 The Kubernetes Authors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package kuberuntime import ( "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "github.com/stretchr/testify/assert" "testing" ) func TestVerifyRunAsNonRoot(t *testing.T) { pod := &v1.Pod{ ObjectMeta: metav1.ObjectMeta{ UID:       "12345678", Name:      "bar", Namespace: "new", }, Spec: v1.PodSpec{ Containers: []v1.Container{ { Name:            "foo", Image:           "busybox", ImagePullPolicy: v1.PullIfNotPresent, Command:         []string{"testCommand"}, WorkingDir:      "testWorkingDir", }, }, }, } rootUser := int64(0) anyUser := int64(1000) runAsNonRootTrue := true runAsNonRootFalse := false for _, test := range []struct { desc     string sc       *v1.SecurityContext uid      *int64 username string fail     bool }{ { desc: "Pass if SecurityContext is not set", sc:   nil, uid:  &rootUser, fail: false, }, { desc: "Pass if RunAsNonRoot is not set", sc: &v1.SecurityContext{ RunAsUser: &rootUser, }, uid:  &rootUser, fail: false, }, { desc: "Pass if RunAsNonRoot is false (image user is root)", sc: &v1.SecurityContext{ RunAsNonRoot: &runAsNonRootFalse, }, uid:  &rootUser, fail: false, }, { desc: "Pass if RunAsNonRoot is false (RunAsUser is root)", sc: &v1.SecurityContext{ RunAsNonRoot: &runAsNonRootFalse, RunAsUser:    &rootUser, }, uid:  &rootUser, fail: false, }, { desc: "Fail if container's RunAsUser is root and RunAsNonRoot is true", sc: &v1.SecurityContext{ RunAsNonRoot: &runAsNonRootTrue, RunAsUser:    &rootUser, }, uid:  &rootUser, fail: true, }, { desc: "Fail if image's user is root and RunAsNonRoot is true", sc: &v1.SecurityContext{ RunAsNonRoot: &runAsNonRootTrue, }, uid:  &rootUser, fail: true, }, { desc: "Fail if image's username is set and RunAsNonRoot is true", sc: &v1.SecurityContext{ RunAsNonRoot: &runAsNonRootTrue, }, username: "test", fail:     true, }, { desc: "Pass if image's user is non-root and RunAsNonRoot is true", sc: &v1.SecurityContext{ RunAsNonRoot: &runAsNonRootTrue, }, uid:  &anyUser, fail: false, }, { desc: "Pass if container's user and image's user aren't set and RunAsNonRoot is true", sc: &v1.SecurityContext{ RunAsNonRoot: &runAsNonRootTrue, }, fail: false, }, } { pod.Spec.Containers[0].SecurityContext = test.sc err := verifyRunAsNonRoot(pod, &pod.Spec.Containers[0], test.uid, test.username) if test.fail { assert.Error(t, err, test.desc) } else { assert.NoError(t, err, test.desc) } } } </del>

Hello, What happened: When we apply PodSecurityPolicy with rule: WIndows containers are still blocked instead we put: the pod is still blocked with logs: What you expected to happen: the Container should start How to reproduce it (as minimally and precisely as possible): Create Psp disableling Root Users: Create a pod Windows with runAsUserName: Anything else we need to know?: Environment: Kubernetes version (use ): v1.18.3 Cloud provider or hardware configuration: OS (e.g: ): CentOS Linux 7 (Core) / windows server 2019 version 1809 Kernel (e.g. ): 3.10.0-514.10.2.el7.x86_64 Others: /sig windows
/sig windows
/assign I will test it and then post the test results
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale


Question: "AllowedPrefixes": [ "k8s.io/utils/buffer", "k8s.io/utils/exec", <ins> "k8s.io/utils/inotify", </ins> "k8s.io/utils/integer", <ins> "k8s.io/utils/io", "k8s.io/utils/keymutex", "k8s.io/utils/mount", </ins> "k8s.io/utils/net", "k8s.io/utils/nsenter", "k8s.io/utils/path",

What happened: There is to manage import ways under but that doesn't work at our upstream CI. We can enable it by But if doing that, we will face the following error with What you expected to happen: I hope the check is always running to block invalid import ways at upstream CI system. How to reproduce it (as minimally and precisely as possible): As line into arguments.InputDirs of
/sig testing /area e2e-test-framework


Question: Replicas int `json:"replicas" description:"number of replicas desired"` // Selector is a label query over pods that should match the Replicas count. <del> Selector map[string]string `json:"selector,omitempty" description:"label keys and values that must match in order to be controlled by this replication controller"` </del> <ins> // If Selector is empty, it is defaulted to the labels present on the Pod template. Selector map[string]string `json:"selector,omitempty" description:"label keys and values that must match in order to be controlled by this replication controller, if empty defaulted to labels on Pod template"` </ins> // TemplateRef is a reference to an object that describes the pod that will be created if // insufficient replicas are detected.

It doesn't make sense to define pod template with a set of labels that is strict subset of replicaSelector (or at least I can't find an use-case). For that reason I think that the labels in pod template part of replication controller should be optional and user should care about them only if he wants to define a superset of what replicaSelector already defines.
I've heard the same comments from a client. I think that it'd be more intuitive if we always specify pod labels in the template (so that we can copy&paste pod definition), and replicaSelector is optional (by default pod labels).
It would be reasonable for the selector to be a subset. I'm fine with intelligent defaulting but we need to be careful not to conflate the 2 concepts, which I think would increase the amount of confusion.
Setting RC selector from pod template labels SGTM. Any concerns It wouldn't have to work with separate templates, though it would be desirable if it did. It would be nice if we could add preconditions based on other objects (resource version of pod template in this case) to etcd transactions.
I think we agreed in another issue that defaulting the selector from the pod template labels is reasonable. No argument from me. One problem with pod templates is that we lose the strict relationship between selector and template, so if you change the template the RC needs to behave intelligently (I refuse to create more pods because the template selector doesn't match mine, etc)
We should do this, and we should default replicas to 1. We could also consider defaulting replication controller labels from pod template labels. That way, essentially the only thing that would need to be specified for a singleton RC would be the pod template.
I don't think we can default replicas to 1, since we can't tell the difference between an explicit zero and a default zero, and I think there are lots of reasonable cases where you want to set replicas == 0 on create. (esp. in the context of socket activation)
Responded on the PR: We can distinguish default from 0 if we make it a pointer.


Question: LastHeartbeatTime:  metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC), LastTransitionTime: fakeNow, }, <ins> { Type:               v1.NodePIDPressure, Status:             v1.ConditionUnknown, Reason:             "NodeStatusNeverUpdated", Message:            "Kubelet never posted node status.", LastHeartbeatTime:  metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC), LastTransitionTime: fakeNow, }, </ins> }, }, },

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . If the matter is security related, please disclose it privately via --Is this a BUG REPORT or FEATURE REQUEST?: What happened: when the node lost connection with master , using 'kubectl describe node' find the NodePIDPressure is false , no switch to unknown. What you expected to happen: PIDPressure Unknown Fri, 14 Sep 2018 05:13:09 +0800 Fri, 14 Sep 2018 05:13:53 +0800 NodeStatusUnknown Kubelet stopped posting node status. How to reproduce it (as minimally and precisely as possible): stop kubelet, wait the node Notready check the node status 'kubectl describe node' Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others:
/sig node


Question: err = e2enetwork.WaitForService(f.ClientSet, ns, serviceName, true, 5*time.Second, e2eservice.TestTimeout) framework.ExpectNoError(err, fmt.Sprintf("error while waiting for service:%s err: %v", serviceName, err)) <del> ginkgo.By("dumping iptables rules on a node") </del> hostExec := utils.NewHostExec(f) defer hostExec.Cleanup() node, err := e2enode.GetRandomReadySchedulableNode(cs) framework.ExpectNoError(err) cmd := "sudo iptables-save" <del> framework.Logf("Executing cmd %q on node %v", cmd, node.Name) result, err := hostExec.IssueCommandWithResult(cmd, node) if err != nil { framework.Failf("Interrogation of iptables rules failed on node %v", node.Name) </del> <ins> if framework.TestContext.ClusterIsIPv6() { cmd = "sudo ip6tables-save" </ins> } <ins> err = wait.PollImmediate(framework.Poll, e2eservice.KubeProxyLagTimeout, func() (bool, error) { framework.Logf("Executing cmd %q on node %v", cmd, node.Name) result, err := hostExec.IssueCommandWithResult(cmd, node) if err != nil { framework.Logf("Interrogation of iptables rules failed on node %v", node.Name) return false, nil } </ins> <del> ginkgo.By("checking that iptables contains the necessary iptables rules") kubeService := false for _, line := range strings.Split(result, "n") { if strings.Contains(line, "-A KUBE-SERVICES") && strings.Contains(line, "-p sctp") { kubeService = true break </del> <ins> for _, line := range strings.Split(result, "n") { if strings.Contains(line, "-A KUBE-SERVICES") && strings.Contains(line, "-p sctp") { return true, nil } </ins> } <del> } if !kubeService { </del> <ins> framework.Logf("retrying ... no iptables rules found for service with sctp ports on node %v", node.Name) return false, nil }) if err != nil { </ins> framework.Failf("iptables rules are not set for a clusterip service with sctp ports") } ginkgo.By("validating sctp module is still not loaded")

<!-- Please only use this template for submitting reports about continuously failing tests or jobs in Kubernetes CI --Which jobs are failing: https://k8s- Which test(s) are failing: sig-network] SCTP [Feature:SCTP] [LinuxOnly] should create a Pod with SCTP HostPort Since when has it been failing: Test [sig-network] SCTP [Feature:SCTP] [LinuxOnly] should create a Pod with SCTP HostPort has failed 3 times since it last passed at commit/. It started failing between commit/ (Tue Jun 09 13:25 PDT) and commit/ (Wed Jun 10 01:26 PDT): Testgrid link: https://k8s- Reason for failure: Anything else we need to know: I'm not sure if we already did it but there was a proposal to change from dockershim to containerd for the GCE jobs
There are no sig labels on this issue. Please add an appropriate label by using one of the following commands: - Please see the for a listing of the SIGs, working groups, and committees available. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign /cc
GitHub didn't allow me to assign the following users: janosi. Note that only , repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : /assign /cc Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
yes, that would break us... the test checks "are we using kubenet", but it didn't occur to me that we would need to deal with kubenet with CRI instead of kubenet with dockershim. It should be easy to make it check for both the dockershim rule and the CNI portmap rule though...
yes, according to the "Changes" link on the dashboard, containerd replaced docker in GCE right before the failures.

Let's see if I can make it :)
So actually the test is pretty generic (it just looks for and ) and it seems like it ought to pass correctly with CNI portmap too. I'm not sure what's failing. You may need to add some debugging...
OK, so as I can see, containerd's CRI implementation just ignores all HostPort definitions that do not have TCP or UDP Protocol value. I.e. containerd will not send HostPort mapping definitions to the CNI if the protocol is not TCP or UDP.
seems this is the reason why sctp was skipped
Yes, exactly. Introduced right after SCTP support was merged in k8s. Just found this in the package: Though not sure it "knows" the right runtime at runtime, because there is no user of it in :)
just curious, why do we skip in containerd and allow it with dockershim?
Hmm, do you mean why do not we skip in case of containerd? The reason is that I did not remember that containerd PR about the SCTP support. Let's see if I can skip it.
Hm, that is not true. My search engine was wrong. After a restart I found usage of that in e2e_node
Has a bug been filed upstream in containerd? Just skipping the tests seems like the wrong approach here.
sorry, I meant why we are not allowing it in containerd, does this feature should work in all container runtimes? CRIO, docker and containerd? if affirmative, we can submit a PR to containerd/cri to allow SCTP, I can do it if you don't have time
Excuse me, I misunderstood your question. Yep, why not. I mean, it could work in all container runtimes. As I understand the containerd implementation is based on a misunderstanding: that the host port mapping loads the SCTP kernel module and that would be bad for those apps that want to use user space SCTP. But as long as there is no SCTP socket created on a node the SCTP kernel module is not loaded.
I can initiate a discussion with containerd about this.
seems this dates back to the original SCTP KEP, seems like it should be a pretty simple fix (?) thanks for filing the bug and for the details there!

CRI-O seems to have the same issue:
Just confirming you want this filter to be gone now. Or do you need the filter to be manageable in some way.. filtered by a config switch?
For TCP and UDP HostPorts, the runtime is expected to do two things: a listening socket on that port so the kernel will think it's in use and not allocate it to anyone else iptables rules (or whatever) to route traffic from the host port to the container port For SCTP HostPorts, the runtime should do the second part (iptables rules), but should not do the first part (listening socket), for two reasons: Some users want to use "userspace SCTP", in which case they will be referring to SCTP at the Kubernetes level but need to keep the kernel module from ever being loaded. (This is discussed more .) If is not loaded when Kubernetes starts, we want to avoid having Kubernetes itself cause it to be auto-loaded, for security reasons. (More discussion of this .) So, runtimes should be filtering out SCTP for the opening-listening-sockets part, but not for the making-iptables-rules part. (Some implementations, notably CNI , don't actually do the opening-listening-sockets part, so they wouldn't be filtering at all.)
Exactly. The current state in the runtimes: Docker (dockershim, actually): OK. The logic was implemented together with the basic SCTP feature, so listening on the host port is filtered for SCTP, but the iptables rules are created Containerd (CRI, actually): the patch which was applied 2 years ago filters out every SCTP host port handling, because it filters SCTP ports from the list of port mappings that are passed to CNI (portmap) later. Actually, containerd (CRI) does not perform the "listen" part at all, so the patch is not needed at all CRI-O: the patch applied 2 years ago filters out everything. CRI-O imports the host port manager implementation from kubelet/dockershim. For this reason it has the logic that can filter out SCTP ports from the "listen" part, but still applies the iptables rules. I.e. the patch is not needed in CRI-O either.
Issues go stale after 90d of inactivity. Mark the issue as fresh with .

Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale The test is flaky now :thinking: https://k8s-
I guess it is a timing issue? Do you think some extra polling rounds would help? (I did not disappear, but currently I am very loaded with my regular job...)
I've submitted a PR to fix it and another one to test the sctp connectivity, please review them, I've pinged you there :smile:


Question: expectedLBs = make([]network.LoadBalancer, 0) setMockLBs(az, ctrl, &expectedLBs, "service", 1, i+1, c.isInternalSvc) // finally, delete it. <ins> if c.wrongRGAtDelete { az.LoadBalancerResourceGroup = "nil" } </ins> err = az.EnsureLoadBalancerDeleted(context.TODO(), testClusterName, &c.service) expectedLBs = make([]network.LoadBalancer, 0) mockLBsClient := mockloadbalancerclient.NewMockInterface(ctrl)

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: controller-manager is stuck in deleting an incorrectly configured LB type service. it cannot be recovered until restarted. What you expected to happen: controller-manager should recover. At least it should allow us to create the service without restarting it. How to reproduce it (as minimally and precisely as possible): put a random IP in above yaml and make sure the resource group doen't exist. Anything else we need to know?: Environment: Kubernetes version (use ): 1.12.4 Cloud provider or hardware configuration: AKS OS (e.g: ): Kernel (e.g. ): Install tools: Others:
/sig azure
I also want to add that the above repro is stuck in 403. It's possible for it to be stuck in 404 as well when the SP has the right permission.
Can you share the log for 404?
Thanks for reporting the issue. /assign Do you mean other services couldn't be configured when this happens?
correct.
FYI, we had the same issue when the Idle Timeout annotation was set to "3" which isn't a supported case. Will this PR also cover this edge case ?
Unfortunately , this is still not fixed. The idle timeout is not 403 or 404. We need another fix for this (and possibly more for other annotations)


Question: <ins> /* Copyright 2014 Google Inc. All rights reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package e2e import ( "fmt" "net/http" "time" "github.com/GoogleCloudPlatform/kubernetes/pkg/api" "github.com/GoogleCloudPlatform/kubernetes/pkg/client" "github.com/GoogleCloudPlatform/kubernetes/pkg/util" . "github.com/onsi/ginkgo" . "github.com/onsi/gomega" ) var _ = Describe("Networking", func() { var c *client.Client BeforeEach(func() { c = loadClientOrDie() }) It("should function for pods", func() { if testContext.provider == "vagrant" { By("Skipping test which is broken for vagrant (See https://github.com/GoogleCloudPlatform/kubernetes/issues/3580)") return } // Test basic external connectivity. resp, err := http.Get("http://google.com/") if err != nil { Fail(fmt.Sprintf("unable to talk to the external internet: %v", err)) } if resp.StatusCode != http.StatusOK { Fail(fmt.Sprintf("unexpected error code. expected 200, got: %v (%v)", resp.StatusCode, resp)) } ns := api.NamespaceDefault // TODO(satnam6502): Replace call of randomSuffix with call to NewUUID when service //                   names have the same form as pod and replication controller names. name := "nettest-" + randomSuffix() svc, err := c.Services(ns).Create(&api.Service{ ObjectMeta: api.ObjectMeta{ Name: name, Labels: map[string]string{ "name": name, }, }, Spec: api.ServiceSpec{ Port:          8080, ContainerPort: util.NewIntOrStringFromInt(8080), Selector: map[string]string{ "name": name, }, }, }) By(fmt.Sprintf("Creating service with name %s", svc.Name)) if err != nil { Fail(fmt.Sprintf("unable to create test service %s: %v", svc.Name, err)) } // Clean up service defer func() { defer GinkgoRecover() By("Cleaning up the service") if err = c.Services(ns).Delete(svc.Name); err != nil { Fail(fmt.Sprintf("unable to delete svc %v: %v", svc.Name, err)) } }() By("Creating a replication controller") rc, err := c.ReplicationControllers(ns).Create(&api.ReplicationController{ ObjectMeta: api.ObjectMeta{ Name: name, Labels: map[string]string{ "name": name, }, }, Spec: api.ReplicationControllerSpec{ Replicas: 8, Selector: map[string]string{ "name": name, }, Template: &api.PodTemplateSpec{ ObjectMeta: api.ObjectMeta{ Labels: map[string]string{"name": name}, }, Spec: api.PodSpec{ Containers: []api.Container{ { Name:    "webserver", Image:   "kubernetes/nettest:latest", Command: []string{"-service=" + name}, Ports:   []api.Port{{ContainerPort: 8080}}, }, }, }, }, }, }) if err != nil { Fail(fmt.Sprintf("unable to create test rc: %v", err)) } // Clean up rc defer func() { defer GinkgoRecover() By("Cleaning up the replication controller") rc.Spec.Replicas = 0 rc, err = c.ReplicationControllers(ns).Update(rc) if err != nil { Fail(fmt.Sprintf("unable to modify replica count for rc %v: %v", rc.Name, err)) } if err = c.ReplicationControllers(ns).Delete(rc.Name); err != nil { Fail(fmt.Sprintf("unable to delete rc %v: %v", rc.Name, err)) } }() By("Waiting for connectivity to be verified") const maxAttempts = 60 passed := false var body []byte for i := 0; i < maxAttempts && !passed; i++ { time.Sleep(2 * time.Second) body, err = c.Get().Prefix("proxy").Resource("services").Name(svc.Name).Suffix("status").Do().Raw() if err != nil { fmt.Printf("Attempt %v/%v: service/pod still starting. (error: '%v')n", i, maxAttempts, err) continue } switch string(body) { case "pass": fmt.Printf("Passed on attempt %v. Cleaning up.n", i) passed = true break case "running": fmt.Printf("Attempt %v/%v: test still runningn", i, maxAttempts) break case "fail": if body, err = c.Get().Prefix("proxy").Resource("services").Name(svc.Name).Suffix("read").Do().Raw(); err != nil { Fail(fmt.Sprintf("Failed on attempt %v. Cleaning up. Error reading details: %v", i, err)) } else { Fail(fmt.Sprintf("Failed on attempt %v. Cleaning up. Details:n%v", i, string(body))) } break } } if !passed { if body, err = c.Get().Prefix("proxy").Resource("services").Name(svc.Name).Suffix("read").Do().Raw(); err != nil { Fail(fmt.Sprintf("Timed out. Cleaning up. Error reading details: %v", err)) } else { Fail(fmt.Sprintf("Timed out. Cleaning up. Details:n%v", string(body))) } } Expect(string(body)).To(Equal("pass")) }) It("should provide unchanging URLs", func() { tests := []struct { path string }{ {path: "/validate"}, {path: "/healthz"}, // TODO: test proxy links here } for _, test := range tests { By(fmt.Sprintf("testing: %s", test.path)) data, err := c.RESTClient.Get(). AbsPath(test.path). Do(). Raw() if err != nil { Fail(fmt.Sprintf("Failed: %vnBody: %s", err, string(data))) } } }) }) </ins>

Create a ginkgo networking test suite and add the following tests to it: TestNetwork TestImportantURLs
Part of the ginkgo-ization of the e2e tests


Question: return fmt.Errorf("unable to add because the healthz endpoint has already been created") } s.healthzChecks = append(s.healthzChecks, checks...) <del> return s.addLivezChecks(livezGracePeriod, checks...) </del> <ins> if err := s.AddLivezChecks(livezGracePeriod, checks...); err != nil { return err } return s.AddReadyzChecks(checks...) </ins> } <del> // addReadyzChecks allows you to add a HealthCheck to readyz. func (s *GenericAPIServer) addReadyzChecks(checks ...healthz.HealthChecker) error { </del> <ins> // AddReadyzChecks allows you to add a HealthCheck to readyz. func (s *GenericAPIServer) AddReadyzChecks(checks ...healthz.HealthChecker) error { </ins> s.readyzLock.Lock() defer s.readyzLock.Unlock() if s.readyzChecksInstalled {

<!-- Please only use this template for submitting enhancement requests --What would you like to be : When using GenericAPIServer you can only add new checks via function which adds them to all , , and endpoints. All other methods like are private. I would like to request those methods to be made public: Why is this needed: Developers of apiservers should be able to define their readiness probes add them to specific endpoint of their choosing. For example in case of Metrics Server we defined probes: Readiness - Metrics cache is full. As MS serves data only from cache, traffic will be only sent after cache is full. Liveness - Metric scraping loop is not deadlocked. It doesn't make sense to have those probes to both endpoints. Not ready metric cache should not cause MS being killed via liveness probe. Delayed scrape should not cause traffic not being served from cache due failing readiness probe. /assign
/sig apimachinery
The label(s) cannot be applied, because the repository doesn't have them <detailsIn response to : /sig apimachinery Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig api-machinery
/cc /triage accepted
I can see this. PR welcome.


Question: framework.ExpectNoError(err) scheduledPods, notScheduledPods := e2epod.GetPodsScheduled(masterNodes, allPods) <del> printed := false printOnce := func(msg string) string { if !printed { printed = true return msg } return "" } framework.ExpectEqual(len(notScheduledPods), expectedNotScheduled, printOnce(fmt.Sprintf("Not scheduled Pods: %#v", notScheduledPods))) framework.ExpectEqual(len(scheduledPods), expectedScheduled, printOnce(fmt.Sprintf("Scheduled Pods: %#v", scheduledPods))) </del> <ins> framework.ExpectEqual(len(notScheduledPods), expectedNotScheduled, fmt.Sprintf("Not scheduled Pods: %#v", notScheduledPods)) framework.ExpectEqual(len(scheduledPods), expectedScheduled, fmt.Sprintf("Scheduled Pods: %#v", scheduledPods)) </ins> } // verifyReplicasResult is wrapper of verifyResult for a group pods with same "name: labelName" label, which means they belong to same RC

What happened: When an issue happens at the following check in the conformance test "validates resource limits of pods that are allowed to run": the debugging message ("Scheduled Pods: %#v") doesn't appear in the log at all like the following: ` and that makes debugging difficult actually. What you expected to happen: The debugging message should appear in the log if an issue happens. Environment: Kubernetes version (use ): v1.15.1 Cloud provider or hardware configuration: OpenStack OS (e.g: ): Ubuntu 18.04.1 LTS Kernel (e.g. ): 4.15.0-55-generic Install tools: kubeadm Network plugin and version (if this is a network-related bug): Others:
/sig testing /sig scheduling


Question: return "", false } <del> // make a directory like /var/lib/kubelet/plugins/kubernetes.io/pod/rbd/pool-image-image </del> <ins> // make a directory like /var/lib/kubelet/plugins/kubernetes.io/rbd/mounts/pool-image-image </ins> func makePDNameInternal(host volume.VolumeHost, pool string, image string) string { <del> return path.Join(host.GetPluginDir(rbdPluginName), "rbd", pool+"-image-"+image) </del> <ins> // Backward compatibility for the deprecated format: /var/lib/kubelet/plugins/kubernetes.io/rbd/rbd/pool-image-image deprecatedDir := path.Join(host.GetPluginDir(rbdPluginName), "rbd", pool+"-image-"+image) info, err := os.Stat(deprecatedDir) if err == nil && info.IsDir() { // The device mount path has already been created with the deprecated format, return it. glog.V(5).Infof("Deprecated format path %s found", deprecatedDir) return deprecatedDir } // Return the canonical format path. return path.Join(host.GetPluginDir(rbdPluginName), mount.MountsInGlobalPDPath, pool+"-image-"+image) </ins> } // make a directory like /var/lib/kubelet/plugins/kubernetes.io/rbd/volumeDevices/pool-image-image

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: What happened: The method for ceph rbd volume plugin generates the path with code as follows: So the device mount path's format will be: , this implementation has two problems: with other volume plugins, as all the other plugins return , see as an example. method of ceph rbd plugin is not working properly, because rbd plugin uses method to find the device mount path(just like all the other plugins do), and method assumes the device mount path's format is (see ). What you expected to happen: The device mount path of rbd plugin should be . Anything else we need to know?: Do we have any reason for this design? If not, I will submit a PR to fix this, the adjusted method maybe look like this: /sig storage
/assign
nice catch There are two ways to fix this: Change mount path. But this will introduce compatibility issues during upgrade. Use , instead of to replace PR welcome
cc
Hi , there is a difference between and : returns the device path, such as , and returns the device mounted path, such as . What we really want is the second one, as we need to find the pool and image name. So we can't just simply replace with . Maybe we can find the mounted path specially for rbd plugin, use , the code looks like this:
thanks let make a forward moving plan: fix and append to the global path keep in . If no path is found, fall back to old path scheme using your code
Nice idea , I've submit a PR for this, PTAL.
I've checked and the reason is that I tested with the old release branch, the has been changed with in mount path, and causes the problem. thanks for your PR
i read again. There is another return path . So in rbd, the missing dir will not make into , but the second return should still give us the dir we need
You are right , indeed has two return path, but the second return still can't meet our needs. Because the parameter of is the mount path for a specific pod, such as , its format is , so will return the volume name. This volume name is returned by (see ), usually is the PV's name or the name specified in , both of these are not .


Question: t.Fatal(err) } if o.GenerateName != "" { <del> t.Errorf("expected null json value to be read as "" string, but got: %q", o.GenerateName) </del> <ins> t.Errorf("expected null json generateName value to be read as "" string, but got: %q", o.GenerateName) } if o.Generation != 0 { t.Errorf("expected null json generation value to be read as zero, but got: %q", o.Generation) </ins> } if got, expected := o.Labels, map[string]string{"foo": ""}; !reflect.DeepEqual(got, expected) { t.Errorf("unexpected labels, expected=%#v, got=%#v", expected, got)

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . If the matter is security related, please disclose it privately via --Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: CRDs created with old Kubernetes version are missing : Example: CRD created recently: $ curl -s '…' {creationTimestamp, generation}' { "creationTimestamp": "2018-07-23T17:24:58Z", "generation": 1 } CRD created with one of the old versions: $ curl -s 'https://…' {creationTimestamp, generation}' { "creationTimestamp": "2018-04-11T12:38:47Z", "generation": null } What you expected to happen: I'd expect to be always present so I can use it in controllers and monitoring without resorting to ugly hacks. How to reproduce it (as minimally and precisely as possible): Create a CRD with an old version of Kubernetes with CRD subresources disabled (not sure which one, 1.10 or maybe 1.9). Update the cluster to 1.11. Create another CRD. Observe that is always present in the new one, but doesn't show up in the old one. Environment: Kubernetes version (use ): Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.3", GitCommit:"", GitTreeState:"clean", BuildDate:"2018-09-09T18:02:47Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"darwin/amd64"} Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.3", GitCommit:"", GitTreeState:"clean", BuildDate:"2018-09-09T17:53:03Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"} Cloud provider or hardware configuration: AWS OS (e.g. from /etc/os-release): CoreOS 1800.7.0 Kernel (e.g. ): 4.14.63-coreos Install tools:
Reiterating the mentions to trigger a notification: <detailsIn response to : Instructions for interacting with me using PR comments are available .

If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Custom resources start setting as 1 during creation from v1.10 because that's when the subresources feature was introduced. The subresources feature says that I want to follow the spec/status conventions and hence, want generation to be initialized to 1 and be incremented whenever spec changes. In <=v1.9, even if we initialized it to 1, it does not hold much meaning because is not allowed to be modified during updates. It is also not incremented at all, since we have no way of knowing if the user wants to follow spec/status conventions. So it will always remain as 1 and won't be helpful in controllers. The condition is surprising. In the older version, if it is absent with , it should also be absent with . I am not able to reproduce this though. would you know on which version you tested the export against?
/area custom-resources /cc
I don't expect anything from old clusters, that's completely understandable. However, if I have a 1.11 cluster with subresources enabled, I expect all CRDs to behave in the same way, not just the ones that were created after 1.10. This doesn't seem to be the case. Version information is in the original issue. All our clusters are currently running v1.11.3. I think that some of the buggy CRDs were created with v1.10 (without CRD subresources enabled), but it might've been v1.9.
I think the older cluster version here is 1.9 because we always set generation to 1 during creation from 1.10: 10 did change the default to 1. However, I am not sure if we should be changing the default values for a 1.9 cluster now. Moreover, we maintain only 3 releases at a time - so I think we only cherry-pick to 1.12, 1.11 and 1.10 now (?). thoughts?
What I'm trying to say is that clusters running 1.11 should automatically handle resources with missing or unset by updating it to 1. It doesn't make sense to keep them in this broken state forever just because they were created in one of the previous versions.

Actually, after editing the resource the is updated to 1, so I can just default to in the controller. For some reason I thought that the resource stays broken forever, which is not the case. Sorry about that!
We can still do this, but I'm wondering if that's ok or not because that wouldn't represent the "true" generation value for that resource. Have the same questions for , which initializes and increments generation for all other resources. :thinking:
I've updated the issue since export is not relevant (and I think the logic resets generation to 1). However, the fact that for old CRDs is (internally) set to 0 is still problematic. Any custom controllers or monitoring code that checks will not be able to tell whether they've been processed successfully by a controller or not (because both and are missing).
/assign
Are you suggesting we set generation on read if it is missing?
That would be one option, or just patch them once after update.
it would probably make sense to make this part of metadata coercion on read, based on whether the CRDs has spec/status enabled
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
/remove-lifecycle rotten
Hello! I'm bug triage lead for the 1.16 release cycle and considering this issue is tagged for 1.16, but not updated for a long time, I'd like to check its status. The code freeze is starting on August 29th (about 1.5 weeks from now), which means that there should be a PR ready (and merged) until then. Do we still target this issue to be fixed for 1.16?
No idea, I'm not a maintainer.
Yes


Question: "strconv" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" <ins> utilnet "k8s.io/apimachinery/pkg/util/net" </ins> "k8s.io/kubernetes/pkg/api" "k8s.io/kubernetes/pkg/kubectl/cmd/templates" cmdutil "k8s.io/kubernetes/pkg/kubectl/cmd/util"

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: kubectl cluster-info displays proxy urls for cluster services If those services are using https, the referenced urls do not work (they try to proxy using http) What you expected to happen: Services with a port name of https or 443 should use the ‘https:<name:<port‘ form How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration**: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others: /assign /sig cli


Question: ExpiresIn:    json.Number(expiresIn), ExpiresOn:    json.Number(expiresOn), NotBefore:    json.Number(expiresOn), <del> Resource:     resourceID, </del> <ins> Resource:     tokenAudience, </ins> Type:         tokenType, }, environment: environment,

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: When using kubectl, authentication fails with the below error For some reason kubectl is adding a lot of s to the start of the SPN - this can be observed in What you expected to happen: Authentication works as in previous versions of Kubectl How to reproduce it (as minimally and precisely as possible): Create a cluster using Azure AD authentication. Authenticate from a client and wait until the token expires. Use Kubectl and have it automatically refresh the token; it will fail due to the extra 'spn:' entries. Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: on-premise OS (e.g: ): macOS 10.14.6 Kernel (e.g. ): Darwin Hostname 18.7.0 Darwin Kernel Version 18.7.0: Mon Feb 10 21:08:45 PST 2020; root:xnu-4903.278.28~1/RELEASEX8664 x86_64 Install tools: N/A Network plugin and version (if this is a network-related bug): N/A Others:
/sig cli
Kubectl version 1.18.0 Kubernetes version 1.16.6
can you confirm whether your original kubeconfig already has in apiserver-id or not? in the mean time, you may try it can convert your kubeconfig from azure module to exec module and do the token thing outside kubectl
At this point if I remove the tokens and try to reauth i get the error described above. I've tried it both ways with apiserver-id having an initially and not, both lead to the same behavior in the end.
i know what's going on now. will prepare a fix. the root cause is that token's target resource (adal.Token.Resource) needs to have prefix and this prefix gets unexpectedly to as well, which gets written back to kubeconfig. So every time kubectl is run, it adds to apiserver-id
/priority important-soon
I'm seeing this with kubectl 1.17.3 and kubernetes 1.16.7.

It's unclear to me what versions are impacted and in what version it is fixed, since only version 1.18 is mentioned.
This still happens on kubectl 1.16.12 and 1.17.8
This still happens on kubectl v1.18.5
same
can you guys share your kubectl version --client and kubeconfig with me? you can find me on kubernetes slack. the slack user name is same as my github handle.
Version 1.17.6 works fine.
Since I was mistaken thinking was still causing this for me, I figured I would just drop a note for others in case it's helpful. Turns out my problem was actually due to and looks to have been resolved as of the v3.3.0 release.
This issue has now reappeared for me - although not necessarily the same root cause. AKS 1.19.0 Authentication works, but it spams frequently.


Question: candidates = append(candidates, containerStatus) } } <del> if len(candidates) <= p.containersToKeep { </del> <ins> if len(candidates) <= containersToKeep { </ins> return containerStatusbyCreatedList{} } sort.Sort(candidates) <del> return candidates[p.containersToKeep:] </del> <ins> return candidates[containersToKeep:] </ins> } // deleteContainersInPod issues container deletion requests for containers selected by getContainersToDeleteInPod. <del> func (p *podContainerDeletor) deleteContainersInPod(exitedContainerID string, podStatus *kubecontainer.PodStatus) { for _, candidate := range p.getContainersToDeleteInPod(exitedContainerID, podStatus) { </del> <ins> func (p *podContainerDeletor) deleteContainersInPod(exitedContainerID string, podStatus *kubecontainer.PodStatus, removeAll bool) { containersToKeep := p.containersToKeep if removeAll { containersToKeep = 0 } for _, candidate := range getContainersToDeleteInPod(exitedContainerID, podStatus, containersToKeep) { </ins> select { case p.worker <- candidate.ID: default:

Kubelet evicts pods whenever there is memory or disk pressure. It marks the pods as to evict them. To help with reducing disk pressure, kubelet should delete all containers belonging to pods right away. This will result in loss of logs. In the future, we can copy out logs from dead containers before deleting them. Logs can be rotated separately from dead containers. IMO, this feature is required for disk usage based evictions to work reliably in kubelet.


Question: "k8s.io/apimachinery/pkg/util/wait" "k8s.io/apiserver/pkg/server/healthz" "k8s.io/apiserver/pkg/server/options/encryptionconfig" <ins> "k8s.io/apiserver/pkg/server/options/encryptionconfig/metrics" </ins> "k8s.io/client-go/util/workqueue" "k8s.io/klog/v2" )

ref - /sig auth
/triage accepted
/retitle encryption-at-rest: add success or failure metric when encryption configuration attempts to dynamically update
/assign


Question: framework.ExpectNotEqual(nodeList.Size(), 0) ginkgo.By("Getting memory details from node status and kubelet config") status := nodeList.Items[0].Status nodeName := nodeList.Items[0].ObjectMeta.Name <del> kubeletConfig, err := e2ekubelet.GetCurrentKubeletConfig(nodeName, f.Namespace.Name, true) </del> <ins> framework.Logf("Getting configuration details for node %s", nodeName) request := f.ClientSet.CoreV1().RESTClient().Get().Resource("nodes").Name(nodeName).SubResource("proxy").Suffix("configz") rawbytes, err := request.DoRaw(context.Background()) framework.ExpectNoError(err) kubeletConfig, err := decodeConfigz(rawbytes) </ins> framework.ExpectNoError(err) systemReserve, err := resource.ParseQuantity(kubeletConfig.SystemReserved["memory"])

ci-kubernetes-e2e-aks-engine-azure-master-windows-containerd-serial-slow Kubernetes e2e suite: [sig-windows] [Feature:Windows] Memory Limits [Serial] [Slow] Allocatable node memory should be equal to a calculated allocatable memory value expandmore this month No response_ /sig windows
/assign
/triage accepted /priority important-soon


Question: nodeInformer := informerFactory.Core().V1().Nodes() pvcInformer := informerFactory.Core().V1().PersistentVolumeClaims() pvInformer := informerFactory.Core().V1().PersistentVolumes() <ins> volumeAttachmentInformer := informerFactory.Storage().V1().VolumeAttachments() </ins> adc := &attachDetachController{ <del> kubeClient:  fakeKubeClient, pvcLister:   pvcInformer.Lister(), pvcsSynced:  pvcInformer.Informer().HasSynced, pvLister:    pvInformer.Lister(), pvsSynced:   pvInformer.Informer().HasSynced, podLister:   podInformer.Lister(), podsSynced:  podInformer.Informer().HasSynced, nodeLister:  nodeInformer.Lister(), nodesSynced: nodeInformer.Informer().HasSynced, cloud:       nil, </del> <ins> kubeClient:             fakeKubeClient, pvcLister:              pvcInformer.Lister(), pvcsSynced:             pvcInformer.Informer().HasSynced, pvLister:               pvInformer.Lister(), pvsSynced:              pvInformer.Informer().HasSynced, podLister:              podInformer.Lister(), podsSynced:             podInformer.Informer().HasSynced, nodeLister:             nodeInformer.Lister(), nodesSynced:            nodeInformer.Informer().HasSynced, volumeAttachmentLister: volumeAttachmentInformer.Lister(), volumeAttachmentSynced: volumeAttachmentInformer.Informer().HasSynced, cloud:                  nil, </ins> } // Act

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: Background: The attach/detach controller create objects for CSI Volumes to indicate that a volume needs to be attached to a node. It deletes the object to indicate that the volume should be detached. Problem: If the Attach/Detach controller crashes, and in the meantime the pod using that volume is deleted, when the AD controller restarts, it won't know that the volume was previously attached and should be cleaned up. What you expected to happen: Proposal: We should have the AD controller periodically list all objects and for any that are not accounted for, update its internal actual/desired state caches so that the volume is gracefully detached. How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: /sig storage Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
could I take it?
/assign
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-testing, kubernetes/test-infra and/or . /close
Closing this issue. <detailsIn response to : Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . /close Instructions for interacting with me using PR comments are available .

If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-testing, kubernetes/test-infra and/or . /close
Closing this issue. <detailsIn response to : Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/lifecycle frozen /reopen
Reopened this issue. <detailsIn response to : /lifecycle frozen /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign
should address this


Question: "k8s.io/apimachinery/pkg/util/mergepatch" "k8s.io/apimachinery/pkg/util/sets" "k8s.io/apimachinery/pkg/util/strategicpatch" <ins> "k8s.io/apimachinery/pkg/util/validation/field" </ins> "k8s.io/apiserver/pkg/admission" "k8s.io/apiserver/pkg/audit" "k8s.io/apiserver/pkg/authorization/authorizer"

/kind bug /sig api-machinery /cc What happened: patching (SMP) any object's annotation using invalid patch content will result in an (500) internal-server error. expecting a (400) bad-request error in this case. patch body: received error: the error is thrown from converter instead of the patcher, so it seems that patcher considers the body as valid then the converter complained. Anything else we need to know?: when my grandpa was 65 years old he started running one mile a day to keep fit. he's 70 now and we have no idea where he is.
I just tried the same patch with "MergePatchType", and it also fails with 500.
I can't make it fail with JSONPatchType, I get a 422
I suspect the error should read "the server rejected your request due to an error in our request"


Question: time.Sleep(rs.podPollPeriod) } } <del> return pod, nil </del> }

For example: struct field tag not compatible with reflect.StructTag.Get missing argument for Errorf("%v"): format reads arg 2, have only 1 args range variable pod enclosed by function struct field tag not compatible with reflect.StructTag.Get arg minion for printf verb %s of wrong type: unreachable code unreachable code
Those all look really important
I'm going to fix this unless I hear that someone else is working on this in the next couple minutes.


Question: VisitPodSecretNames(pod, func(name string) bool { extractedNames.Insert(name) return true <del> }) </del> <ins> }, AllContainers) </ins> // excludedSecretPaths holds struct paths to fields with "secret" in the name that are not actually references to secret API objects excludedSecretPaths := sets.NewString(

Fully described in , and will visit these fields for all containers, regardless of whether they're hidden behind a feature gate (e.g. ). This is usually correct, these methods are often used to drop fields and in this case should visit all containers that are persisted regardless of whether they're disabled, but the callers of these methods should signal that explicitly rather than relying on the implicit behavior. /cc /priority important-longterm /kind bug /sig node
The label(s) cannot be applied, because the repository doesn't have them <detailsIn response to : Fully described in , and will visit these fields for all containers, regardless of whether they're hidden behind a feature gate (e.g. ). /priority important/long-term /kind bug /sig node Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
are you interested in taking this one, too?
VisitPodSecretNames and VisitPodConfigmapNames already specify AllContainers for VisitContainers Can you let me know what else needs to be done?
Callers should specify whether they want to visit all containers or not. There are valid reasons for each, so we shouldn't assume the desired behavior, even if 100% of the current users want to visit all containers. Jordan's original comment in has some examples of current uses. I don't think this needs to be as configurable as , so my naive first attempt to implement this would probably just be a new to and


Question: // assume modifies `assumedPod` by setting NodeName=scheduleResult.SuggestedHost err = sched.assume(assumedPod, scheduleResult.SuggestedHost) if err != nil { <ins> metrics.PodScheduleError(prof.Name, metrics.SinceInSeconds(start)) </ins> // This is most probably result of a BUG in retrying logic. // We report an error here so that pod scheduling can be retried. // This relies on the fact that Error will check if the pod has been bound // to a node and if so will not add it back to the unscheduled pods queue // (otherwise this would cause an infinite loop). sched.recordSchedulingFailure(prof, assumedPodInfo, err, SchedulerError, "") <del> metrics.PodScheduleError(prof.Name, metrics.SinceInSeconds(start)) </del> return } // Run the Reserve method of reserve plugins. if sts := prof.RunReservePluginsReserve(schedulingCycleCtx, state, assumedPod, scheduleResult.SuggestedHost); !sts.IsSuccess() { <del> sched.recordSchedulingFailure(prof, assumedPodInfo, sts.AsError(), SchedulerError, "") </del> metrics.PodScheduleError(prof.Name, metrics.SinceInSeconds(start)) // trigger un-reserve to clean up state associated with the reserved Pod prof.RunReservePluginsUnreserve(schedulingCycleCtx, state, assumedPod, scheduleResult.SuggestedHost) <ins> if forgetErr := sched.Cache().ForgetPod(assumedPod); forgetErr != nil { klog.Errorf("scheduler cache ForgetPod failed: %v", forgetErr) } sched.recordSchedulingFailure(prof, assumedPodInfo, sts.AsError(), SchedulerError, "") </ins> return }

This is a bug found in The scheduler fails to schedule the pod on the first try due to PV is out of sync. This may happen because the PV/PVC may be modified before the scheduler binder assumes them. However, the scheduler skips the pod update event and never try to scheduler the pod again: This is because when the reserve plugins fails, we didn't try to forget the pod. After the is called, the pod is in the state.
/sig scheduling /sig storage /assign /priority critical-urgent
The label(s) cannot be applied, because the repository doesn't have them <detailsIn response to : /sig scheduling /sig storage /assign /priority critical-urgent Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: } delete(cache.podStates, key) default: <del> return fmt.Errorf("pod state wasn't added but get removed. Pod key: %v", key) </del> <ins> return fmt.Errorf("pod %v is not found in scheduler cache, so cannot be removed from it", key) </ins> } return nil }

/kind bug There are several error messages in the scheduler cache that are confusing to read, at least one of these surfaced up to end users via Events. I noticed these two in particular but there may be others: Whether or not these should actually be surfacing as events, the messages should be cleaned up.
cc
Seems that these are internal or implement details in scheduler, do our end users can fully understand what the events really mean? I think it is depend on our own implement. idea?
I believe, the problem is with in the message but I am not able to reproduce it, can you please share steps to reproduce this.
No unfortunately, one of our users reported seeing it when there system was in a bad state, not in a reproducible way. The point of the issue is just that the grammar in those messages is bad, which should be fixable even if you can't get them to fire.


Question: return clientcmdapi.Cluster{}, fmt.Errorf("cluster %q does not exist", clusterInfoName) } mergo.MergeWithOverwrite(mergedClusterInfo, config.overrides.ClusterInfo) <del> // An override of --insecure-skip-tls-verify=true and no accompanying CA/CA data should clear already-set CA/CA data // otherwise, a kubeconfig containing a CA reference would return an error that "CA and insecure-skip-tls-verify couldn't both be set" </del> <ins> // * An override of --insecure-skip-tls-verify=true and no accompanying CA/CA data should clear already-set CA/CA data // otherwise, a kubeconfig containing a CA reference would return an error that "CA and insecure-skip-tls-verify couldn't both be set". // * An override of --certificate-authority should also override TLS skip settings and CA data, otherwise existing CA data will take precedence. </ins> caLen := len(config.overrides.ClusterInfo.CertificateAuthority) caDataLen := len(config.overrides.ClusterInfo.CertificateAuthorityData) <del> if config.overrides.ClusterInfo.InsecureSkipTLSVerify && caLen == 0 && caDataLen == 0 { mergedClusterInfo.CertificateAuthority = "" mergedClusterInfo.CertificateAuthorityData = nil </del> <ins> if config.overrides.ClusterInfo.InsecureSkipTLSVerify || caLen > 0 || caDataLen > 0 { mergedClusterInfo.InsecureSkipTLSVerify = config.overrides.ClusterInfo.InsecureSkipTLSVerify mergedClusterInfo.CertificateAuthority = config.overrides.ClusterInfo.CertificateAuthority mergedClusterInfo.CertificateAuthorityData = config.overrides.ClusterInfo.CertificateAuthorityData </ins> } return *mergedClusterInfo, nil

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: Created cluster with , which generated its own self-signed certificate and put in . Put (API) nodes behind HTTPS ELB and for SSL termination. Run any command that has to hit the API (That's annoying because the CA that signed the cert is a public CA, and the root cert is in my OS keystore. Potentially another bug?) Download the appropriate key from (In my case, AWS ACM certificate = ) Use the certificate to attempt the same request What you expected to happen: Commandline-specified certificate overrides any other options How to reproduce it (as minimally and precisely as possible): Put any in for your host and attempt to override on cmdline. Anything else we need to know?: seems to be where the problem comes from, it's not differentiating "param came from cmdline" from "have param". If I remove the from then it works for both intended cases - either with specifying the CA cert on cmdline or via system certs. Environment: Kubernetes version (use ):Cloud provider or hardware configuration**: AWS OS (e.g. from /etc/os-release): OSX Kernel (e.g. ):Install tools:Others:
There are no sig labels on this issue. Please by:<br(1) mentioning a sig: <bre.g., for API Machinery<br(2) specifying the label manually: <bre.g., for sig/scalability<br<brNote: method (1) will trigger a notification to the team. You can find the team list and label list
Reiterating the mentions to trigger a notification: <detailsIn response to : Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. Prevent issues from auto-closing with an comment. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten /remove-lifecycle stale
/remove-lifecycle rotten
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Still a problem. /remove-lifecycle stale
Just ran into this as well with 1.9.0
also with 1.9.3 This is very annoying. It says "certificate-authority-data will override" but it will fail and do nothing, exiting with 0. It should at least do the override and try to execute
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale Any chance of someone looking at this? Or at least freezing status, since it's evergreen until actually fixed?
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale References from other tickets don't change this timer?
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with .

Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
/remove-lifecycle rotten
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
/assign
any chance this can be reopened? still seems to be a problem with 1.20 of kubectl


Question: }, }, }, <del> }, s.AllOf...) </del> <ins> }, s.ValueValidation.AllOf...) </ins> } return true

What happened: API server panics handling requests for a CRD with OpenAPI validation with What you expected to happen: CRD is applied successfully. How to reproduce it (as minimally and precisely as possible): kind create cluster --image kindest/node:v1.16.1 then kubectl apply: Partial logs from API server: Anything else we need to know?: Environment: Kind Kubernetes version (use ):OS (e.g: ):
/sig api-machinery
/cc
where It seems that s.AnyOf was referencing some invalid address.
yes, which is very strange, given s was dereferenced earlier in the same method: is investigating
reproduced and has a fix
I had the same issue with prometheus (kube 1.16.1). Deploying is fine but when trying to delete, it crashes the api server. For those who can't upgrade to 1.16.2+ (fixed version) there is a workaround : using etcd to delete the resources that crash the api server : here I grep on coreos because I know it is prometheus that crashes the api server Remove every item in the list that causes the crash :
Note this is also resolved in 1.15.5+ and 1.16.2+, upgrading to 1.17.x is not required to resolve this


Question: Description: Scheduling Pods MUST fail if the resource limits exceed Machine capacity. */ framework.ConformanceIt("validates resource limits of pods that are allowed to run ", func() { <del> framework.WaitForStableCluster(cs, masterNodes) </del> <ins> WaitForStableCluster(cs, masterNodes) </ins> nodeMaxAllocatable := int64(0) nodeToAllocatableMap := make(map[string]int64) for _, node := range nodeList.Items {

What would you like to be : is used at the beginning of some e2e scheduler tests to wait until the cluster is in a stable state. It currently waits until all existing pods are scheduled, but it should also wait until pods from previous tests are actually deleted. We can do this by assuming that pods should exist only in the kube-system namespace, and so WaitForStableCluster should wait until there are no pods in any other namespace. Why is this needed: /help /sig testing /sig scheduling
/milestone v1.17 /priority critical-urgent
/cc
I'll take this one, can't figure out how to get ci bot to assign to me...
/assign
/remove-help


Question: FailedPredicates: failedPredicateMap, } } <ins> metrics.SchedulingAlgorithmPredicateEvaluationDuration.Observe(metrics.SinceInMicroseconds(startPredicateEvalTime)) </ins> trace.Step("Prioritizing") <ins> startPriorityEvalTime := time.Now() </ins> // When only one node after predicate, just use it. if len(filteredNodes) == 1 { <ins> metrics.SchedulingAlgorithmPriorityEvaluationDuration.Observe(metrics.SinceInMicroseconds(startPriorityEvalTime)) </ins> return filteredNodes[0].Name, nil }

<!-- Thanks for filing an issue! Before hitting the button, please answer these questions.--Is this a BUG REPORT or FEATURE REQUEST? (choose one): FEATURE REQUEST <!-- If this is a BUG REPORT, please: Fill in as much of the template below as you can. If you leave out information, we can't help you as well. If this is a FEATURE REQUEST, please: Describe in detail the feature/behavior/change you'd like to see. In both cases, be ready for followup questions, and please respond in a timely manner. If we can't reproduce a bug or think a feature already exists, we might close your issue. If we're wrong, PLEASE feel free to reopen it and explain why. --As of now in , we have 3 metrics, for end2endLatency, algorithmLatency and BindingLatency. The problem with algorithmLatency is it is a combination of predicate and priority latencies. So breaking down this metric into predicate and priority latencies would be really helpful for performance analysis. For example, as part of the analysis after these metrics were , we were able to find latencies when we have taints & tolerations alone. In fact, ability to drill down on every predicate and priority would be ideal but want to know other's opinions on this.
/cc
+1 please send a PR.
We also have some demonstrations of how this helps with things like highlighting taint and toleration performance and so on. We should update readme info on how to use these along side the scheduler perf stuff as we've done internally .
You need to be careful about what latency means if we are computing multiple different predicates in parallel (if we decide to split it by predicate). Splitting predicate vs priority is definitely a good idea.
Please label this as part of our 1.7 scheduling forensics milestone .
Let's remove the idea of latency entirely, it is confusing - this is just an attempt at measuring predicate and priority timescales, and there is no response latency that we care about in those small (20ms) timescales , (with the exception of maybe the latency of an etcd hit that might happen in a client if the cache is missing information I guess)
Not really - there were cases of predicates which computation could take over a second.

Yes, and even longer on large clusters with things light affinity.... However here I'm just disputing use of the term latency since latency is usually used to refer to the time between the end of a request and the beggining of a response. What we endeavor to do here is really measure round trip execution time, for predicates, in isolation. If we want to measure some kind of latency then I believe we would need to define some notion of a request/response, which has diminished value outside of something like RPC. If we truly were measuring latency (and it might be interesting to do so with tims tracing work) then we might thus look at the time between writing a new unscheduled pod to the system, and the time it takes to start running the scheduler fit functions or something... Tl;dr there are a lot of varying usages and definitions for latency, but almost all of them, with the exception of measurements of virtual function calls, refer to some unavoidable minimum cost that occurs during distributed communication, and in this case, we are simply measuring "execution time of the predicate filter" which is called from inside the same process. (Alternatively I guess it's ok to call it "fitfunctionpredicate_latency", I guess, since then we are using the fit function as the context for a response measurement).
OK, IIUC you object usage of the word 'latency', right? We use it in this context for quite some time, and as I agree that it doesn't fit formal definition, there's no point in changing it now, as it would confuse people even more.
yeah, i really think they should all be refactored to not be named as such, that is my broader point . but if we want to keep those metrics id suggest we at least update the definition in the fields to clarify that latency is defined WRT rather then to a RPC or some other typical latency calc scenario. aplogies for the bike shed, ill move this to a PR :)
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. Prevent issues from auto-closing with an comment. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale


Question: concurrencyLimit:       6, evalDuration:           time.Second * 40, expectedFair:           []bool{true}, <del> expectedFairnessMargin: []float64{0.35}, </del> <ins> expectedFairnessMargin: []float64{0.38}, </ins> expectAllRequests:      true, evalInqueueMetrics:     true, evalExecutingMetrics:   true,

<!-- Please only use this template for submitting reports about flaky tests or jobs (pass or fail with no underlying change in code) in Kubernetes CI --pull-kubernetes-unit TestDifferentWidths links to appreciated links to specific failures in spyglass appreciated
/sig api-machinery
/assign /cc /triage accepted
Yes, a couple of the unit tests have too stringent evaluations. See


Question: fakeEtcd := util.MakeFakeEtcdClient(t) manager := MakeReplicationManager(fakeEtcd, nil) var testControllerSpec api.ReplicationController <del> receivedCount := 0 </del> <ins> received := make(chan bool) </ins> manager.syncHandler = func(controllerSpec api.ReplicationController) error { if !reflect.DeepEqual(controllerSpec, testControllerSpec) { t.Errorf("Expected %#v, but got %#v", testControllerSpec, controllerSpec) } <del> receivedCount++ </del> <ins> close(received) </ins> return nil } go manager.watchControllers() <del> time.Sleep(10 * time.Millisecond) </del> <ins> fakeEtcd.WaitForWatchCompletion() </ins> // Test normal case testControllerSpec.ID = "foo"

Detected by
Create a Pod as follows: Name: jenkins Using image: jenkins In a new Kubenetes namespace named website-frontend kubectl create ns website-frontend apiVersion: v1 kind: Pod metadata: name: jenkins namespace: website-frontend spec: containers: name: jenkins image: jenkins kubectl create -f


Question: }) It("should function for client IP based session affinity: udp", func() { <del> startTime := time.Now() </del> config := framework.NewNetworkingTestConfig(f) <del> By(fmt.Sprintf("dialing(udp) %v --> %v:%v (config.clusterIP)", config.TestContainerPod.Name, config.ClusterIP, framework.ClusterHttpPort)) </del> <ins> By(fmt.Sprintf("dialing(udp) %v --> %v:%v (config.clusterIP)", config.TestContainerPod.Name, config.ClusterIP, framework.ClusterUdpPort)) </ins> updateSessionAffinity := func(svc *v1.Service) { svc.Spec.SessionAffinity = v1.ServiceAffinityClientIP } <del> framework.UpdateService(f.ClientSet, config.NodePortService.Namespace, config.NodePortService.Name, updateSessionAffinity) firstEndpoints, err := config.GetEndpointsFromTestContainer("udp", config.ClusterIP, framework.ClusterUdpPort, config.MaxTries, 0) </del> <ins> _, err := framework.UpdateService(f.ClientSet, config.NodePortService.Namespace, config.NodePortService.Name, updateSessionAffinity) if err != nil { framework.Failf("Failed to update service session affinity, error: %v", err) } // Fetch first endpoints when visiting service firstEndpoints, err := config.GetEndpointsFromTestContainer("udp", config.ClusterIP, framework.ClusterUdpPort, config.MaxTries) </ins> if err != nil { framework.Failf("Unable to get endpoints from test containers: %v", err) } <ins> // Check if first endpoints are equal to endpoints which are fetched later </ins> for i := 0; i < framework.SessionAffinityChecks; i++ { <del> eps, err := config.GetEndpointsFromTestContainer("http", config.ClusterIP, framework.ClusterUdpPort, config.MaxTries, 0) </del> <ins> eps, err := config.GetEndpointsFromTestContainer("udp", config.ClusterIP, framework.ClusterUdpPort, config.MaxTries) </ins> if err != nil { framework.Failf("Unable to get endpoints from test containers: %v", err) }

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug /priority failing-test /sig network What happened: From https://k8s-, seems to be broken right after they were by Example error message: and Anything else we need to know?: Is it as expected?
Oct 11 16:37:49.106: test session affinity, cost time: 1m31.742990485s This is a mistake. Sorry for my debug lines sneak in. I was just wondering why this test case doesn't run in upstream CI, so I this debug line. Now, I see because this test case only runs in . Oct 11 16:39:42.406: Expect endpoints: map[], got: map[netserver-1:{}] This check failed unexpectedly, let me try to find out why. I am working on it and will let you know soon when I have some updates.
Thanks. These two tests don't run on pull jobs as they are marked as , though https://k8s-, https://k8s- (and other CIs that include ) would have them running.
Do you mean when I raise a PR, these test cases will not be triggered? My question is how can I make sure my follow-up PR can fix this bug? Where can I access the log message when I raise a new PR?
Can I see the updated test results via directly visiting https://k8s-, https://k8s-
Yep, that is the current behavior. A hacky way to make it run on PR --- add a temporary commit to remove the word from test name so some pull jobs ( I think?) will run it. And you could view logs by clicking the button.
Nope, this CI only includes changes that are merged. It is basically a cronjob that runs tests against head.


Question: "TestPlugin2": framework.NewStatus(framework.Unschedulable, "injected filter status").WithFailedPlugin("TestPlugin2"), }, }, <del> { name: "SuccessFilterWithRunAllFilters", plugins: []*TestPlugin{ { name: "TestPlugin", inj:  injectedResult{FilterStatus: int(framework.Success)}, }, }, runAllFilters: true, wantStatus:    nil, wantStatusMap: framework.PluginToStatus{}, }, { name: "ErrorAndErrorFilters", plugins: []*TestPlugin{ { name: "TestPlugin1", inj:  injectedResult{FilterStatus: int(framework.Error)}, }, { name: "TestPlugin2", inj:  injectedResult{FilterStatus: int(framework.Error)}, }, }, runAllFilters: true, wantStatus:    framework.AsStatus(fmt.Errorf(`running "TestPlugin1" filter plugin: %w`, errInjectedFilterStatus)).WithFailedPlugin("TestPlugin1"), wantStatusMap: framework.PluginToStatus{ "TestPlugin1": framework.AsStatus(fmt.Errorf(`running "TestPlugin1" filter plugin: %w`, errInjectedFilterStatus)).WithFailedPlugin("TestPlugin1"), }, }, { name: "ErrorAndErrorFilters", plugins: []*TestPlugin{ { name: "TestPlugin1", inj:  injectedResult{FilterStatus: int(framework.UnschedulableAndUnresolvable)}, }, { name: "TestPlugin2", inj:  injectedResult{FilterStatus: int(framework.Unschedulable)}, }, }, runAllFilters: true, wantStatus:    framework.NewStatus(framework.UnschedulableAndUnresolvable, "injected filter status", "injected filter status").WithFailedPlugin("TestPlugin1"), wantStatusMap: framework.PluginToStatus{ "TestPlugin1": framework.NewStatus(framework.UnschedulableAndUnresolvable, "injected filter status").WithFailedPlugin("TestPlugin1"), "TestPlugin2": framework.NewStatus(framework.Unschedulable, "injected filter status").WithFailedPlugin("TestPlugin2"), }, }, </del> } for _, tt := range tests {

Since scheduler Policy API was total removed, we don't support any more. Remove unsupported configuration.
/sig scheduling /assign
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scheduling /cc
/sig scheduling (trying to tell the bots to remove )
The label(s) cannot be applied. These labels are supported: <detailsIn response to : /remove-needs sig /remove-label needs-sig Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
It's true scheduler doesn't quite use this field anymore, but I'm wondering if some other project like ClusterAutoscaler or simulator still consumes this via .
There's no place we can config this field now by configure file(before we can apply the change by policy api, but now it's removed). So it seems like unless we support this feature with CC, or this field is useless. To other projects, it's the same. When they are using legacy policy api, they can configure this field, but once they synced with the latest kubernetes, with the policy api removed, they can not configure this anymore. In this situation, we should add support in CC. I don't quite understand the relationship between k/k with k/autoscaling or others, I think they sync with k/k periodic.
As I mentioned above, it's still accessible via a framework runtime programmatically, and thus may be still useful. For example, upon a scheduling failure, a ClusterAutoscaler's implementation may choose to dryrun through all filter plugins to ensure a new provisioned node can fit the pod, instead of resolving the failure while failing at another latter filter plugin.
Oh, I haven't thought of configuration via programmatically, that saved my concern. But a little strange.
That's exactly what I meant. Given it's a public option in framework, and there is no alternative if we disabled it, it'd be good to check with potential consumers.

For now I can think of CA ( and k-sigs/scheduler-simulator ( If they can confirm it's not needed in their usecases, I'm fine with removing .
To k/autoscaler, at least these codes are still there together with the legacy policy api. , looking forward to the response.
is not used in sigs/kube-scheduler-simulator.
this isn't used by cluster autoscaler, we can remove it.