Question: v1 "k8s.io/api/core/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" <ins> "k8s.io/apimachinery/pkg/api/resource" </ins> metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/util/wait" utilfeature "k8s.io/apiserver/pkg/util/feature" "k8s.io/client-go/kubernetes" featuregatetesting "k8s.io/component-base/featuregate/testing" <ins> "k8s.io/component-helpers/storage/volume" </ins> "k8s.io/kubernetes/pkg/features" st "k8s.io/kubernetes/pkg/scheduler/testing" testutils "k8s.io/kubernetes/test/integration/util"

We need an integration test for the scheduler enforcement of the alpha feature. This is required for beta graduation; we need to collect signal on the current behavior before we can proceed. See this comment for context:
/sig storage /sig scheduler
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : /sig storage /sig scheduler Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scheduling


Question: pp.allowPermit = false } <del> // NewPermitPlugin is the factory for permit plugin. func NewPermitPlugin(_ *runtime.Unknown, fh framework.FrameworkHandle) (framework.Plugin, error) { perPlugin.fh = fh return perPlugin, nil </del> <ins> // NewPermitPlugin returns a factory for permit plugin with specified PermitPlugin. func NewPermitPlugin(permitPlugin *PermitPlugin) framework.PluginFactory { return func(_ *runtime.Unknown, fh framework.FrameworkHandle) (framework.Plugin, error) { permitPlugin.fh = fh return permitPlugin, nil } </ins> } // TestPrefilterPlugin tests invocation of prefilter plugins.

<!-- Please only use this template for submitting reports about failing tests in Kubernetes CI jobs --Which jobs are failing: pull-kubernetes-integration Which test(s) are failing: TestPreemptWithPermitPlugin Since when has it been failing: 13:30 PST, Sep 9, 2019 Testgrid link: Reason for failure: Need investigations. Anything else we need to know: /sig scheduling
Another flaky It happens at 2019-08-06 05:43:47 +0000 UTC
The above repeated for 30 seconds.
I checked the code and log again. I think the reason is as following: When scheduler preempted , it failed to delete the pod(same as the case for , there might be something wrong with etcd), so it could not schedule because had not been deleted successfully.
/assign Find the root cause now, will send a PR soon.
Thanks for your work! Not sure whether you are still investigating it, the flaky test is impacting many PRs now, so I send a PR for it now, sorry for it.


Question: tlsConfig = nil } cfg := clientv3.Config{ <del> Endpoints: c.ServerList, TLS:       tlsConfig, </del> <ins> DialKeepAliveTime:    keepaliveTime, DialKeepAliveTimeout: keepaliveTimeout, Endpoints:            c.ServerList, TLS:                  tlsConfig, </ins> } client, err := clientv3.New(cfg) if err != nil {

Hi, running 1.6.4 on etcd 3.0.17 in the following setup: 3 etcd nodes (bootstrapped according to the ) 9 kubernetes cluster nodes all of them running kube-apiserver (--apiserver-count=9) with --bind-address to local kube-apiserver instance. All instances pointing to the same etcd cluster (--etcd-servers=http://IP1:4001,http://IP2:4001,http://IP3:4001) I tried to simulate 1 etcd node crash by simply turing off one of the machines. Result: timeouts running kubectl (get nodes, get pods) commands lots of these error messages in kube-apiserver logs: "] apiserver received an error that is not an metav1.Status: http: Handler timeout" ETCD correctly reports unhealthy node, but healthy cluster Inspecting netstat shows lots of ESTABLISHED connections to the dead etcd server, port 4001 - these connections are cleared only by restarting the kube-apiserver(s). Workarounds: Switching back to etcd 2.2.1 and using --storage-backend=etcd2 ... ESTABLISHED connections are cleared within several seconds Leaving etcd 3.0.17 and using --storage-backend=etcd2 ... ESTABLISHED connections are cleared within several seconds In both above cases kubectl shows expected results. The dead node is evicted (NotReady), other nodes & pods are in tact. I noticed that client contains TCP KeepAlive timeout, while client does not. Could this be related to the described scenario? <!-- If this is a BUG REPORT, please: Fill in as much of the template below as you can. If you leave out information, we can't help you as well. If this is a FEATURE REQUEST, please: Describe in detail the feature/behavior/change you'd like to see. In both cases, be ready for followup questions, and please respond in a timely manner. If we can't reproduce a bug or think a feature already exists, we might close your issue. If we're wrong, PLEASE feel free to reopen it and explain why.

--Kubernetes version Client Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.4", GitCommit:"", GitTreeState:"clean", BuildDate:"2017-05-19T18:44:27Z", GoVersion:"go1.7.5", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.4", GitCommit:"", GitTreeState:"clean", BuildDate:"2017-05-19T18:33:17Z", GoVersion:"go1.7.5", Compiler:"gc", Platform:"linux/amd64"} Environment: OS: NAME="Oracle Linux Server" VERSION="6.7" ID="ol" VERSIONID="6.7" PRETTYNAME="Oracle Linux Server 6.7" ANSICOLOR="0;31" CPENAME="cpe:/o:oracle:linux:6:7" HOMEURL="" BUGREPORTURL="" ORACLEBUGZILLAPRODUCT="Oracle Linux 6" ORACLEBUGZILLAPRODUCTVERSION=6.7 ORACLESUPPORTPRODUCT="Oracle Linux" ORACLESUPPORTPRODUCTVERSION=6.7 Kernel
There are no sig labels on this issue. Please by:<br(1) mentioning a sig: <br(2) specifying the label manually: <br<brNote: method (1) will trigger a notification to the team. You can find the team list .
/area etcd
Reiterating the mentions to trigger a notification: <detailsIn response to : Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
This seems to depend on an issue with etcd, there is no option for a KeepAlive I believe. I've sent a PR, can fix this once that gets merged.
The mentioned PR has been merged. We may need to wait for an etcd release / update the dependency on etcd for us to be able to fix this in Kubernetes.
Had a look at the current CHANGELOG on etcd. The fix for this issue is in master, and will be released in etcd clientv3 3.3.0.

Here is the which shows the fix has been merged into the . Kubernetes is currently on clientv3 3.2.11 it appears (according to ).


Question: } glog.V(3).Infof("Removing %s/%s from '%s' with CNI '%s' plugin and runtime: %+v", namespace, name, config.Network.Name, config.Network.Type, rt) <del> if err := plugin.cniConfig.DelNetwork(config, rt); err != nil { </del> <ins> err = plugin.cniConfig.DelNetwork(config, rt) // The pod may not get deleted successfully at the first time. // Ignore "no such file or directory" error in case the network has already been deleted in previous attempts. if err != nil && !strings.Contains(err.Error(), "no such file or directory") { </ins> return fmt.Errorf("Error removing container from network: %v", err) } return nil

/kind bug What happened: During pod teardown, we sometimes see the following error-level kubelet logs: What you expected to happen: Clean pod shutdown, no error-level logs. How to reproduce it (as minimally and precisely as possible): We've got two kube crons that run every minute, across 12 regions in production -- so related pods daily -- and we see this about 3-12 times per day. Anything else we need to know?: Relevant kubelet log context of a cron pod that manifested this error: Controller manager for the pod: Scheduler for the pod: Cron yaml: Environment: Kubernetes version (use ):Cloud provider or hardware configuration: Bare metal OS (e.g. from /etc/os-release): 16.04.1 LTS (Xenial Xerus) Kernel (e.g. ): 4.4.0-89-generic -Ubuntu Install tools: N/A Others: N/A
/sig network
Same problem here. Every time I try to of my deploy, the old pod cannot be terminated successfully. below is output of
Have you checked the ns net files of those failed-to-deleted pods? /cc Could we just ignore these errors?
Sorry, yeah, that seems like a fair point to note -- the directories are in fact being deleted. Seems like it's either functionally succeeding yet generating error-level logs, or perhaps generating error-level logs and then succeeding on internal retry.
I prefer the later one. Perhaps kubelet does not kill the pod completely at the first time. Such error may occur during a retry.
So this is just an overly scary log message?
Yes, I think so. In the first attempt to delete the pod, the ns network file has already been deleted. But the pod does not get deleted successfully. Then "no such file or directory" will be raised in the second attempt.


Question: "k8s.io/apimachinery/pkg/util/runtime" "k8s.io/apimachinery/pkg/util/wait" "k8s.io/client-go/tools/record" <ins> runtimeapi "k8s.io/cri-api/pkg/apis/runtime/v1alpha2" </ins> "k8s.io/klog/v2" kubecontainer "k8s.io/kubernetes/pkg/kubelet/container" "k8s.io/kubernetes/pkg/kubelet/events"

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --Restart: Always pods went into OutOfCpu error state after kubelet restart. Pods should remain running as node had sufficient capacity for them. Completed, RestartNever pod should not revert from Completed to OutOfCpu. Launch kubelet with: Run the following script against the local cluster: <details</detailsObserve pod status: Restart the kubelet: Observe pods: xref /sig node Kubernetes version (use ): v1.23.0-alpha.3.183+ - Cloud provider or hardware configuration: Local machine OS (e.g: ) & Kernel (e.g. ):Install tools: local-up- Network plugin and version (if this is a network-related bug): N/A Others:
/triage accepted /priority critical-urgent Reproducible on 1.22
From the reproducer: yields However, in the kubelet logs, it sees 7100m cpu allocated?
Looks like the topology admit handler thinks that the completed pods are using CPU which is why when it gets to the 8th it fails it with OutOfCpu. Must be a bug in there somewhere.
/assign
Unclear if could be related but linking for posterity
Confirmed this only affects 1.22, repro on 1.21 has expected behaviour: /sig scheduling I'll start a bisect.
is
Added some logging With this we see I think that the terminated pods are getting filtered, but for whatever reason they are still getting their resource requests to NodeInfo?
points out that my logging language was a bit backwards and that the completed pods should not have been returned, because they already successfully completed and exited. This code is suspect: The logs show that the completed pods are never filtered, so they are failing one of these checks. I am going to try to write a test case that hits this issue so we can test fixes against it.

The completed pods definitely show up as complete, but for some reason they do not get filtered (note I have changed my patch above for the log lines to make sense now): So I guess isPodTerminationRequested=true on already exited pods, but I would only expect that to be true if the pod was actively in the process of termination and not already exited. cc Log patch: <details</details
I tried changing to but apparently it thinks that's true :( Edit: doesn't work either.
The bug is that the only component in the kubelet that is allowed to authoritatively state a container is terminated (pod worker) takes non-zero time to do so during startup (because we have to call kill container to guarantee it's stopped). This races with the pod config loop and admission, which sends all the containers at once. In order for the race to be eliminated, some equivalent of a sync has to be completed during startup on terminal but potentially non-terminated pod workers BEFORE other adds are processed. That could be running HandlePodCleanups once all sources are synced but before HandlePod* methods are called, but HandlePodAdditions() cannot be invoked until after the pod worker has had a chance to verify all terminal pods are actually terminal. EDIT: Another potential approach is for the pod worker to hold starting pods until SyncKnownPods observes all terminal pods are accounted for at least once, then allowing starts.
Also, we appear to be missing a kubelet restart test that verifies admission with completed pods (which I thought was on our list of "missing e2e tests that would have caught these regressions" already). Such a test would have to: pods that consume resources for admission until they are running some of them with failed and` successful states the kubelet some of the running pods the kubelet none of the terminal pods are restarted, all running pods are restarted, and that only non-terminal pods are accounted to admission


Question: } } } else { <del> if err := apiclient.CreateOrUpdateConfigMap(client, coreDNSConfigMap); err != nil { </del> <ins> // If the Corefile is modified and doesn't require any migration, it'll be retained for the benefit of the user if err := apiclient.CreateOrRetainConfigMap(client, coreDNSConfigMap, kubeadmconstants.CoreDNSConfigMap); err != nil { </ins> return err } }

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: does not retaining coredns configmap using v1.19.4 makes no difference What you expected to happen: should follow documentation and retain coredns configmap How to reproduce it (as minimally and precisely as possible): upgrading from v1.18.6 to v1.19.3 Anything else we need to know?: works like a charm Environment: Kubernetes version (use ): v1.18.6 Cloud provider or hardware configuration: OS (e.g: ): CentOS 7 Kernel (e.g. ): 4.20.13- Install tools: Network plugin and version (if this is a network-related bug): Others:
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig cluster-lifecycle
seems to be a regression ?
/assign
Would you check you dns configmap for ? As , federation is removed from coredns. If you have federation enabled in your coredns configmap, the check may fail in
We don't have any parameter in our coredns configmap, or any kube-dns parameters in kubeadm config
Looking closely at the dry run messages above, this may not be a regression. The apparent reason it works in 1.18.12 is that support for CoreDNS 1.7.0 was not until 1.19. i.e. 1.18 sees an unsupported version of CoreDNS and therefore skips the corefile migration. Whereas in 1.19, CoreDNS 1.7.0 is supported, and the migration takes place. What does your complete Corefile look like before the migration?
If it's possible, can you also attach the entire dry run logs?
Sure.
Thanks.

I think the issue may be in the detection of the "default" configuration. That is, in the upgrade process, if it detects that you are using the "default" coredns corefile, it simply installs the new default over top of it, rather than migrating the existing one. I think since the default config stanza is in there in its original form, the whole Corefile is incorrectly being detected as a default. If i'm right, the fix could either go in the package (should at least eventually be fixed there), and/or kubeadm could just always migrate the existing config instead of deploying the default over top of the existing one.
I just tested against the package manually - and it correctly identifies the config as "non-default", so that's not it... And I also tested a migration on the Corefile, and the result included the extra stanzas as expected. So, the issue doesn't appear to lie in the migration package.
After testing with the code, I find the issue in , I fixed it in . A workaround for you is to create an empty kube-dns configmap. ———————————————————— edited new comments here I find still fix a problem , when kube-dns configmap is using federations.
Thanks, I'll try it
empty kube-dns configmap doesn't helped.
sorry for my mistake. According to my test, the create or update will trigger is no migrating. Even your corefile will meet some dep that need to be migrated. max_concurrent is new default. However, there is a typo in . Below is a workaround(not good one), add upstream to your corefile in kubernetes. ! It works for me.
if your current coredns is 1.6.7, upstream is deprecated but not removed. In new 1.7.0 coredns, upstream is removed.
thanks for the discussion and PRs this one is mergable for 1.21 as non critical: what do you think about this one and should we backport it as critical (also, to which k8s versions)?
/area kubeadm
It's not obvious to me how this relates to . The kubeadm dry run log appears to be replacing the existing config with the default. It should only be doing this if the existing config is semantically the same as a default Corefile, i.e.

if is true.
just explains that the migration is not triggered at all. if is not a fix for this issue, please remove this line from the PR description.
I think issue here is in/around , and how the result is interpreted... The code correctly assumes that if a Corefile is a default Corefile, then no migration is required. However later it incorrectly assumes the reciprocal: If migration is not required, then the corefile must be a default. Hence, it ends up replacing any Corefile that does not require migration with a default… even if it wasn't a default to begin with.
yes. The need be fixed as well, I will learn to fix it later if no one is assigned. I changed releaseNote to None.


Question: (disk.ManagedDisk != nil && diskURI != "" && strings.EqualFold(*disk.ManagedDisk.ID, diskURI)) { // found the disk klog.V(2).Infof("azureDisk - detach disk: name %q uri %q", diskName, diskURI) <del> if strings.EqualFold(as.cloud.Environment.Name, "AZURESTACKCLOUD") { </del> <ins> if strings.EqualFold(as.cloud.Environment.Name, AzureStackCloudName) { </ins> disks = append(disks[:i], disks[i+1:]...) } else { disks[i].ToBeDetached = to.BoolPtr(true)

<!-- Please only use this template for submitting enhancement requests --What would you like to be : PR() already proves that old api-version on Azure stack works on global Azure too, we could support azure stack in azure cloud provider lib, and then make sure following components works in Azure stack: CSI drivers: azure disk, azure file CCM Since it's mainly api-version change, we could also add e2e tests against global Azure too. Why is this needed: /assign /priority important-soon /sig cloud-provider /area provider/azure
there is blocking issue here(also need azure stack supports latest data model struct): just making the change in interfaces and not go-sdk but facing this error, since we don't have a switch inside go-sdk, we only do switch in azure cloud provider lib,
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale


Question: # This file creates a standard build environment for building cross # platform go binary for the architecture kubernetes cares about. <del> FROM golang:1.11.2 </del> <ins> FROM golang:1.11.3 </ins> ENV GOARM 7 ENV KUBE_DYNAMIC_CROSSPLATFORMS

The go team has released 1.11.3 with three security fixes. From their e-mail announcement: We should kick off upgrades of both the kubernetes build system, and our CI, as soon as possible. Do you have an ETA for the rules_go bump that will contain 1.11.3 toolchain support? cc:
I'm preparing releases that use Go 1.11.3 by default. Those will be on the rules_go 0.14, 0.15, and 0.16 branches. I expect to tag them tonight as soon as they get through CI.
Fantastic! Thank you
rules_go 0.14.7, 0.15.9, and 0.16.4 are now tagged and ready to go.
related commits and issue:
Reopening to track other branches, as well as the CI bump
go1.11.3 and go1.10.6 introduced a regression, and so go1.11.4 and go1.10.7 were released Friday afternoon with a fix. We'll need to bump everything again on master, and should hold off on cherry-picks for now.
This looks fixed by Is there anything else that needs to be done as well?
Still need to merge for k8s 1.13.
Closing as all branches are now up to date.


Question: // Tell the cache to assume that a pod now is running on a given node, even though it hasn't been bound yet. // This allows us to keep scheduling without waiting on binding to occur. <del> err = sched.assume(pod, suggestedHost) </del> <ins> assumedPod := *pod // assume modifies `assumedPod` by setting NodeName=suggestedHost err = sched.assume(&assumedPod, suggestedHost) </ins> if err != nil { return } // bind the pod to its host asynchronously (we can do this b/c of the assumption step above). go func() { <del> err := sched.bind(pod, &v1.Binding{ ObjectMeta: metav1.ObjectMeta{Namespace: pod.Namespace, Name: pod.Name, UID: pod.UID}, </del> <ins> err := sched.bind(&assumedPod, &v1.Binding{ ObjectMeta: metav1.ObjectMeta{Namespace: assumedPod.Namespace, Name: assumedPod.Name, UID: assumedPod.UID}, </ins> Target: v1.ObjectReference{ Kind: "Node", Name: suggestedHost,

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: Pod got stuck in Pending state forever when failed to bind to host. What you expected to happen: For it to be rescheduled on another host How to reproduce it (as minimally and precisely as possible): Not always reproducible. But simply created an RC replicas=2 on a 3 host setup, and one of the Pods got stuck in pending state for a while. Following error messages were found in the scheduler logs: Describe pod output: RC yml: Anything else we need to know?: Environment: Kubernetes version (use ): 1.7.1 Cloud provider or hardware configuration**: OS (e.g. from /etc/os-release): RHEL7, Docker native 1.12.6 Kernel (e.g. ): Install tools: Others:
There are no sig labels on this issue. Please by: a sig: e.g., to notify the contributor experience sig, OR the label manually: e.g., to apply the label Note: Method 1 will trigger an email to the group. You can find the group list and label list . The in the method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals
/sig scheduling
Can you provide the details steps for this issue? And the output of ?
The log just showed the node was notReady or unreachable. Seems the kubelet node is not quite stable that the connections with the master always get lost.
Another pod of the same RC (replicas=2) was started successfully on one of the nodes. I'd expect the failed one to at least be attempted to start again, may be on the node where its peer is running (as there are no anti affinity rules defined on it). But it was stuck in pending state forever (and there was no other reference for the failed pod besides the original log I've attached). I was able to create new RCs of the same kind on the same set of nodes successfully after the Pod scheduling failure, so it looks like some nodes were available for allocation. Perhaps there are some other logs I can look at /provide? no definite steps.

We just extensive validation tests for kubernetes were a bunch of RCs/Deployments/Services/Ingress controllers get created. And starting k8s 1.7.x started observing this failure. Once we see it again, I'll fetch the node status and update the bug.
From the log, it seemed that failed to bind your pod to the node and also failed to forget the pod, which led to unable to reschedule the pod to other nodes. Would you please append when you start and reproduce this issue. Also the kubelet log of that erroneous node.
^^ could you start the schedule with the above param next time for validation test run?
/cc
I ran our tests again, and hit this issue with several pods: There were a few more. Since none of them is assigned to a node. I included logs for all kubelet's. I also included scheduler log with : :
Seeing this as well. hypothesis: broke the error handling when fails. Before that commit, when failed it would log an error and pass the pod to the error handler (), which is in charge of retrying scheduling the pod. After that commit, when fails it skips the error handling and scheduling is never retried. I'm doing some experiments with patching that and I think there's more to this bug than just that (for example why is failing in the first place? is the scheduler cache corrupted? why?), but that's what I've got so far.
/cc
Can you post the description of per namespace?() Are you enabled ? And what's the config of component?
there are no specific parameters passed on the scheduler start besides address and cloudconfig
I posted a patch that i believe fixes this issue at -- for us it's resolved the issue so far, but I'm not very reliably able to reproduce so it's a bit hard to check. could you apply and see if your validation tests pass?
thanks!! We will be able to validate it either today or early next week
Are you able to validate the fix provided by through ? Just saw this issue, and I think it is a serious regression in 1.7 release and we should patch it. cc/ the patch manager for 1.7 patch release.
our QA have been busy with the current release, and didn't have a chance to validate the fix yet. Hopefully this week.

Patching sounds like a great idea. It is a regression indeed as we haven't hit this issue in the previous versions of k8s with the same set of validation tests.
Thanks everyone who helped investigate this!
just ran validation test against the patched branch - no pods stuck in Pending any more, so the fix did the job! thank you
I have this problem with k8s v1.7.0 . How can I fix it, upgrading to v1.7.X? Thank you!
I am seeing this in 1.9. What steps I need to take to solve this problem?
I am seeing this periodically on Azure VMs
any chance you could open a new issue linked to this one with the requisite troubleshooting info (versions, etc.) and label it with ` Thanks!


Question: if transformer == nil { value.RecordCacheMiss() <ins> requestInfo := getRequestInfoFromContext(ctx) </ins> uid := string(uuid.NewUUID()) <del> klog.V(6).InfoS("Decrypting content using envelope service", "uid", uid, "key", string(dataCtx.AuthenticatedData())) </del> <ins> klog.V(6).InfoS("decrypting content using envelope service", "uid", uid, "key", string(dataCtx.AuthenticatedData()), "group", requestInfo.APIGroup, "version", requestInfo.APIVersion, "resource", requestInfo.Resource, "subresource", requestInfo.Subresource, "verb", requestInfo.Verb, "namespace", requestInfo.Namespace, "name", requestInfo.Name) </ins> key, err := t.envelopeService.Decrypt(ctx, uid, &kmsservice.DecryptRequest{ Ciphertext:  encryptedObject.EncryptedDEK, KeyID:       encryptedObject.KeyID,

Add additional logging at level that associates UID with the metadata from the request. xref: /sig auth
/triage accepted
/milestone v1.26
Mind if I take a look at this? /assign
Hello I am wondering how's this coming along. Is there anything I can help with?
Sorry. I got a bit occupied with a different project.. Will have a PR opened for this for review tomorrow.. I am trying to find a way to inject the Verbs into log. But haven't found the right way to include that into the context.. Any suggestion on what can be done?
any suggestion on what is the best way to get the additional metadata for verbs into this log? Move that info across using ?
Hello , bug triage team here, just wanted to touch base with you for updates on this ticket. Thanks!
Thanks! This is being worked on and is still on track for .
There is a Pull request opened on this already as well. But Waiting on some clarification to see if that needs additional changes.
/reopen /assign /unassign
Reopened this issue. <detailsIn response to : /reopen /assign /unassign Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/milestone v1.27
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale
Hi Bug Triage Team for v1.27 here o/ Just wanted to touch base with you for updates on this ticket. Thanks!
I am not working on this anymore Carlos. This was re-assigned
This is tracked for . I've linked the PR.


Question: gvr("storage.k8s.io", "v1alpha1", "csistoragecapacities"): { Stub:             `{"metadata": {"name": "csc-12345-1"}, "storageClassName": "sc1"}`, ExpectedEtcdPath: "/registry/csistoragecapacities/" + namespace + "/csc-12345-1", <del> ExpectedGVK:      gvkP("storage.k8s.io", "v1beta1", "CSIStorageCapacity"), </del> <ins> ExpectedGVK:      gvkP("storage.k8s.io", "v1", "CSIStorageCapacity"), </ins> }, // --

The override in can be removed in Kubernetes v1.25 because then the oldest supported kube-apiserver for downgrades will be v1.24 which will understand the v1 format. Also need to adapt See and Code cleanup...
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign
/sig storage
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale /lifecycle frozen


Question: } wantModules := sets.NewString() loadModules := sets.NewString() <ins> linuxKernelHandler := NewLinuxKernelHandler() _, ipvsModules, _ := utilipvs.GetKernelVersionAndIPVSMods(linuxKernelHandler.executor) </ins> wantModules.Insert(ipvsModules...) loadModules.Insert(mods...) modules := wantModules.Difference(loadModules).UnsortedList()

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!--kube-proxy cannot use ipvs mode on Linux 4.19 due to a check for the module, which was changed to (). uses to check for : It should instead accept on Linux 4.19 or above. What happened: kube-proxy fell back from ipvs mode to iptables mode. What you expected to happen: kube-proxy using ipvs. How to reproduce it (as minimally and precisely as possible): Launch a cluster on an image using Linux 4.19 or above with kube-proxy using ipvs mode (see ). Environment: Kubernetes version (use ): All Kernel (e.g. ): 4.19 <!-- DO NOT EDIT BELOW THIS LINE --/kind bug
/sig network
/area ipvs /assign /cc
Okay, so should check {nfconntrackipv4 | nf_conntrack} and expect load at least one module?
I think so
Please just go ahead :)


Question: // volumeConfigName is the configmap passed to bootstrapper and provisioner volumeConfigName = "local-volume-config" // provisioner image used for e2e tests <del> provisionerImageName = "quay.io/external_storage/local-volume-provisioner:v2.0.0" </del> <ins> provisionerImageName = "quay.io/external_storage/local-volume-provisioner:v2.1.0" </ins> // provisioner daemonSetName name daemonSetName = "local-volume-provisioner" // provisioner default mount point folder

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: We need to update local volume provisioner used in local PV e2e tests for k8s 1.10, by leveraging a new provisioner image (v2.1.0). Environment: Kubernetes version: v1.10
/sig storage /assign
should this block the 1.10 release in your opinion?
No not a blocker


Question: // Config contains the configuration for a given Cache. type Config struct { // Maximum size of the history cached in memory. <del> // // DEPRECATED: Cache capacity is dynamic and this field is no longer used. </del> CacheCapacity int // An underlying storage.Interface.

<!-- Please only use this template for submitting reports about continuously failing tests or jobs in Kubernetes CI --Which jobs are failing: Which test(s) are failing: Since when has it been failing: Testgrid link: Reason for failure: Anything else we need to know: /cc /priority critical-urgent /milestone v1.19
/sig api-machinery
Suspect it has to do with cc /assign (for CI signal ownership)
Seems unlikely since multiple kind e2e tests passed on the PR itself
I was thinking that as well, but I don't see any other relevant changes
Actually, I take that back. Looks like the same failure was encountered as a flake on some of the PR runs. Looking at the test, it appears to assume the resource version of the kubernetes service will always be in the watch window, which is a bad assumption. I'd expect us to fix that assumption in the test, but also make sure we didn't unintentionally shrink the watch window for resources.
/assign
Note: this also appears to be breaking
Failure seems to be timeout related as well shows the runtime ~tripled after merged
revert open in can investigate test failure and timeouts and bring the change back in
I take it back, the test tolerates an expired error explicitly.
showed 17 of the 22 runs since merged took ~1 hour. The first run after the revert at returned to less than 30 minutes for the run. Will keep an eye on that to see if subsequent runs match.
- thanks a lot for looking into it. I don't think my PR changed anything that anyone should rely on. I will take a deeper look into it and try to fix that (hopefully later this week).
contains the explanation and describes the bug (in fact this bug still exists in the codebase, that PR was only increasing the probability of this particular test hitting it)


Question: else clean: clean_meta build/make-clean.sh <del> git clean -Xdf </del> <ins> hack/make-rules/clean.sh </ins> endif define CLEAN_META_HELP_INFO

Is this a BUG REPORT or FEATURE REQUEST?: /kind feature What happened: After setting up an IntelliJ project files for the Kubernetes repo, I had IntelliJ related project files in the directory. Wanting to clean up some generated files, I ran . I was surprised to see that my IntelliJ project files were deleted as well. What you expected to happen: I would have expected to remove only files generated by kubernetes development commands. I would not have expected it to not delete files that I created and intended to be long-lasting, but were only usable by me, and thus gitignored. How to reproduce it (as minimally and precisely as possible): Create a IntelliJ project for Kubernetes that places or files in the repo. Run . The IntelliJ project will be deleted. Anything else we need to know?: I'm not entirely ensure what the correct behavior is here. I see the change to use was made recently by in this . The reasons stated in the pr make a lot of sense to me. However, I think this doesn't cover the edge case in which a file is gitignored because it should be long-lasting and it will be unique to each development environment, and thus should not be removed on every . I'd love to work on figuring out the proper behavior here, and submitting a pr to fix - please let me know how I can help :)
There are no sig labels on this issue. Please by: a sig: e.g., to notify the contributor experience sig, OR the label manually: e.g., to apply the label Note: Method 1 will trigger an email to the group. You can find the group list and label list . The in the method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals
/sig contributor-experience
I'd be interested if you have any suggestions of what to do here. There's a few file system workarounds (such as having your dir outside of the project and symlinking it in.. then you'd just have to restore the symlink after a . I think it's more the assumption that it would only remove build artifacts, rather than removing all temp/.gitignored files.
I'm not super unfamiliar with idiomatic files, so part of this may be my misunderstanding.

My assumption was would remove artifacts that could've been created by some make command (i.e. ). I wouldn't have assumed it would remove all files. What about having a which only removes files generated by and then a (or or ) which removes any gitignored files? Thanks for your help with this :)
Agree on should only remove those things it created by itself (can be any make sub-commands in Makefile). It'd better not remove other files (even if they're untracked) which may be created by users or user tools. Maybe we can create a blacklist file which includes those things created by and should be removed by .
that sounds like a good plan to me - what would you think of that is a fix? My understanding is that that would be similar to what was in place before, except before the list was missing artifacts we wanted to remove.
I think it's just a difference in assumption around what the command does. When I run "make clean" I want to ensure that that my working copy doesn't have any files that may change or interfere with a build. To me, this is any files we don't check into the repo. Maintaining a list of files that is safe to clean leaves us open to the list falling out of date (which it previously did, which is why the functionality was changed in the first place).
Yeah, I think you make a great point that it seems like there are two different interpretations of what does. I'm not super familiar with - do you know which one is the more "traditional" interpretation? One thing I think about is that it seems slightly more dangerous to delete too many, instead of too few files - one operation is very easy to address, whereas the other isn't. I think that leads me to prefer a slightly less aggressive . Thoughts on if it would be helpful for me to write a pr so we have some concrete code to base this chat on?
My opinion is that deletes only things that were generated by something did. If I want to delete other files I created, I call instead.
oh, but we're using , which only deletes ignored files. that's better, though still potentially unexpected.
I created a with an optional different implementation so we could have something to compare the current behavior too.

Let me know your thoughts :)
IMO, The point of moving to using was that anything in your dir that isn't a part of the checked in repo has the potential to impact the build. Temp files, generated files, _output directories, etc. The previous functionality maintained a smaller, manual list that did become out of date (not just in theory), and caused problems. I'm open to solutions, but I don't feel great about returning to a manual list as that's what broke things in the first place.
You make a great point about the manual white list becoming out of date - and I think its particularly convincing that that's happened in practice. I have two ideas: you think that having two options - and - like in the diff - helps address this? The typical developer can use on their local system, while automated processes creating builds can use ? , what if instead of having a whitelist of files to delete, we had a blacklist of files not to delete? This solves the issue which caused me to report this "bug" and the potential drift of the blacklist seems much less concerning. Do either of those sound preferable? If so, I'll update my pr. Thanks for your thought on this :)
I believe this is a bug introduced by , therefore mark this as a bug instead of feature. etcd will also be removed by , which isn't right. cc
where is the local etcd being stored? do you have details on the bug?
Are you talking about builds that take place outside of a docker container (or bazel sandbox)? I don't think we should offer any guarantees about getting a good build outside of docker or a bazel sandbox. If our containerized build is pulling in the wrong files, though, it seems we could fix that by writing code that is more specific about what it includes.
Yeah this is a terrible thing to do to people, I have all sorts of stuff in some of my kubernetes directories that I would not be happy if make clean obliterated.
I have dozens of utility shell scripts and example yaml files that are ignored via . I lost all of them after T_T
etcd installed by hack/install- is in .gitignore too, and therefore gets deleted by :
Got it. Yeah, that's a problem. I opened to move to a back to a static list of files to clean.

It is more correct than a revert as it adds missing patterns, and it's safer than an .


Question: for i in "$@"; do # fsid=0: needed for NFSv4 echo "$i *(rw,fsid=0,insecure,no_root_squash)" >> /etc/exports <ins> # move index.html to here /bin/cp /tmp/index.html $i/ chmod 644 $i/index.html </ins> echo "Serving $i" done <ins> # start rpcbind if it is not started yet /usr/sbin/rpcinfo 127.0.0.1 > /dev/null; s=$? if [ $s -ne 0 ]; then echo "Starting rpcbind" /usr/sbin/rpcbind -w fi </ins> mount -t nfsd nfds /proc/fs/nfsd <del> # -N 2 -N 3: disable NFSv2+3 # -V 4.x: enable NFSv4 /usr/sbin/rpc.mountd -N 2 -N 3 -V 4 -V 4.1 </del> <ins> # -N 4.x: disable NFSv4 # -V 3: enable NFSv3 /usr/sbin/rpc.mountd -N 2 -V 3 -N 4 -N 4.1 </ins> /usr/sbin/exportfs -r # -G 10 to reduce grace time to 10 seconds (the lowest allowed) <del> /usr/sbin/rpc.nfsd -G 10 -N 2 -N 3 -V 4 -V 4.1 2 </del> <ins> /usr/sbin/rpc.nfsd -G 10 -N 2 -V 3 -N 4 -N 4.1 2 /usr/sbin/rpc.statd --no-notify </ins> echo "NFS started" }

GCE minion VM OS version is The nfs server image and hits with the following dump stack
Image version now exists!
thanks!


Question: }, Spec: api.ReplicationControllerSpec{ Replicas: 3, <del> Selector: podLabels, </del> <ins> Selector: rcPodLabels, </ins> Template: &api.PodTemplateSpec{ ObjectMeta: api.ObjectMeta{ <del> Labels: podLabels, </del> <ins> Labels: rcPodLabels, </ins> }, Spec: api.PodSpec{ Containers: []api.Container{

cc
Thanks for flagging it! Its not flaky, it is broken as we did not enable deployments for that test suite. Sent to fix that.


Question: // Defaults to 7080 if not set. ReadOnlyPort int // The port on PublicAddress where a read-write server will be installed. <del> // Defaults to 443 if not set. </del> <ins> // Defaults to 6443 if not set. </ins> ReadWritePort int // If nil, the first result from net.InterfaceAddrs will be used.

Trying to hit the read-write kubernetes service from inside a cluster does not work. tailing the kube-proxy logs shows: And sure enough, kube-apiserver is not listening on the NIC on that port. 8080 is only for the localhost HTTP communications between master components (and nginx redirect after HTTPS termination). The "kubernetes" master service needs to get created with Port set to 443.
+1
This needs to be port 6443 the auth_port On Feb 6, 2015 10:14 AM, "Paulo Pires" wrote:
The code says 8443, but then a flag changes to 6443. Will fix. , Eric Tune wrote:
Holy macaroni this is a mess. , Tim Hockin wrote:


Question: }) } } <ins> func TestHistogramVec_GetAggregatedSampleCount(t *testing.T) { tests := []struct { name string vec  HistogramVec want uint64 }{ { name: "nil case", want: 0, }, { name: "zero case", vec: HistogramVec{ &Histogram{&dto.Histogram{SampleCount: uint64Ptr(0), SampleSum: pointer.Float64Ptr(0.0)}}, }, want: 0, }, { name: "standard case", vec: HistogramVec{ &Histogram{&dto.Histogram{SampleCount: uint64Ptr(1), SampleSum: pointer.Float64Ptr(2.0)}}, &Histogram{&dto.Histogram{SampleCount: uint64Ptr(2), SampleSum: pointer.Float64Ptr(4.0)}}, &Histogram{&dto.Histogram{SampleCount: uint64Ptr(4), SampleSum: pointer.Float64Ptr(8.0)}}, }, want: 7, }, { name: "mixed case", vec: HistogramVec{ &Histogram{&dto.Histogram{SampleCount: uint64Ptr(1), SampleSum: pointer.Float64Ptr(2.0)}}, &Histogram{&dto.Histogram{SampleCount: uint64Ptr(0), SampleSum: pointer.Float64Ptr(0.0)}}, &Histogram{&dto.Histogram{SampleCount: uint64Ptr(2), SampleSum: pointer.Float64Ptr(4.0)}}, }, want: 3, }, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { if got := tt.vec.GetAggregatedSampleCount(); got != tt.want { t.Errorf("GetAggregatedSampleCount() = %v, want %v", got, tt.want) } }) } } func TestHistogramVec_GetAggregatedSampleSum(t *testing.T) { tests := []struct { name string vec  HistogramVec want float64 }{ { name: "nil case", want: 0.0, }, { name: "zero case", vec: HistogramVec{ &Histogram{&dto.Histogram{SampleCount: uint64Ptr(0), SampleSum: pointer.Float64Ptr(0.0)}}, }, want: 0.0, }, { name: "standard case", vec: HistogramVec{ &Histogram{&dto.Histogram{SampleCount: uint64Ptr(1), SampleSum: pointer.Float64Ptr(2.0)}}, &Histogram{&dto.Histogram{SampleCount: uint64Ptr(2), SampleSum: pointer.Float64Ptr(4.0)}}, &Histogram{&dto.Histogram{SampleCount: uint64Ptr(4), SampleSum: pointer.Float64Ptr(8.0)}}, }, want: 14.0, }, { name: "mixed case", vec: HistogramVec{ &Histogram{&dto.Histogram{SampleCount: uint64Ptr(1), SampleSum: pointer.Float64Ptr(2.0)}}, &Histogram{&dto.Histogram{SampleCount: uint64Ptr(0), SampleSum: pointer.Float64Ptr(0.0)}}, &Histogram{&dto.Histogram{SampleCount: uint64Ptr(2), SampleSum: pointer.Float64Ptr(4.0)}}, }, want: 6.0, }, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { if got := tt.vec.GetAggregatedSampleSum(); got != tt.want { t.Errorf("GetAggregatedSampleSum() = %v, want %v", got, tt.want) } }) } } func TestHistogramVec_Quantile(t *testing.T) { tests := []struct { name     string samples  [][]float64 bounds   []float64 quantile float64 want     []float64 }{ { name: "duplicated histograms", samples: [][]float64{ {0.5, 0.5, 0.5, 0.5, 1.5, 1.5, 1.5, 1.5, 3, 3, 3, 3, 6, 6, 6, 6}, {0.5, 0.5, 0.5, 0.5, 1.5, 1.5, 1.5, 1.5, 3, 3, 3, 3, 6, 6, 6, 6}, {0.5, 0.5, 0.5, 0.5, 1.5, 1.5, 1.5, 1.5, 3, 3, 3, 3, 6, 6, 6, 6}, }, bounds: []float64{1, 2, 4, 8}, want:   []float64{2, 6.4, 7.2, 7.84}, }, { name: "random numbers", samples: [][]float64{ {8, 35, 47, 61, 56, 69, 66, 74, 35, 69, 5, 38, 58, 40, 36, 12}, {79, 44, 57, 46, 11, 8, 53, 77, 13, 35, 38, 47, 73, 16, 26, 29}, {51, 76, 22, 55, 20, 63, 59, 66, 34, 58, 64, 16, 79, 7, 58, 28}, }, bounds: []float64{10, 20, 40, 80}, want:   []float64{44.44, 72.89, 76.44, 79.29}, }, { name: "single histogram", samples: [][]float64{ {6, 34, 30, 10, 20, 18, 26, 31, 4, 2, 33, 17, 30, 1, 18, 29}, }, bounds: []float64{10, 20, 40, 80}, want:   []float64{20, 36, 38, 39.6}, }, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { var vec HistogramVec for _, sample := range tt.samples { histogram := samples2Histogram(sample, tt.bounds) vec = append(vec, &histogram) } var got []float64 for _, q := range []float64{0.5, 0.9, 0.95, 0.99} { got = append(got, math.Round(vec.Quantile(q)*100)/100) } if !reflect.DeepEqual(got, tt.want) { t.Errorf("Quantile() = %v, want %v", got, tt.want) } }) } } func TestHistogramVec_Validate(t *testing.T) { tests := []struct { name string vec  HistogramVec want error }{ { name: "nil SampleCount", vec: HistogramVec{ &Histogram{&dto.Histogram{SampleCount: uint64Ptr(1), SampleSum: pointer.Float64Ptr(1.0)}}, &Histogram{&dto.Histogram{SampleSum: pointer.Float64Ptr(2.0)}}, }, want: fmt.Errorf("nil or empty histogram SampleCount"), }, { name: "valid HistogramVec", vec: HistogramVec{ &Histogram{&dto.Histogram{SampleCount: uint64Ptr(1), SampleSum: pointer.Float64Ptr(1.0)}}, &Histogram{&dto.Histogram{SampleCount: uint64Ptr(2), SampleSum: pointer.Float64Ptr(2.0)}}, }, }, { name: "different bucket size", vec: HistogramVec{ &Histogram{&dto.Histogram{ SampleCount: uint64Ptr(4), SampleSum:   pointer.Float64Ptr(10.0), Bucket: []*dto.Bucket{ {CumulativeCount: uint64Ptr(1), UpperBound: pointer.Float64Ptr(1)}, {CumulativeCount: uint64Ptr(2), UpperBound: pointer.Float64Ptr(2)}, {CumulativeCount: uint64Ptr(5), UpperBound: pointer.Float64Ptr(4)}, }, }}, &Histogram{&dto.Histogram{ SampleCount: uint64Ptr(3), SampleSum:   pointer.Float64Ptr(8.0), Bucket: []*dto.Bucket{ {CumulativeCount: uint64Ptr(1), UpperBound: pointer.Float64Ptr(2)}, {CumulativeCount: uint64Ptr(3), UpperBound: pointer.Float64Ptr(4)}, }, }}, }, want: fmt.Errorf("found different bucket size: expect 3, but got 2 at index 1"), }, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { if got := tt.vec.Validate(); fmt.Sprintf("%v", got) != fmt.Sprintf("%v", tt.want) { t.Errorf("Validate() = %v, want %v", got, tt.want) } }) } } </ins>

The new scheduler performance test was introduced back in 1.18, back then, the metric it supported are all . However, along with , some metrics are updated to be , and hence it's no longer appropriate to return the first Histogram: However, it's very common (for instance Preemption) that the metrics may hold multiple histograms: Additionally, metrics and have been removed, we should update them to : /sig scheduling
/triage accepted /assign
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-contributor-experience at . /lifecycle stale
/remove-lifecycle stale


Question: pluralize("type", "types", len(badTypes)), joinQuote(badTypes.List()), )) <del> if setUser { badData = append(badData, "user may not be set") } if setRole { badData = append(badData, "role may not be set") } </del> <ins> } if setUser { badData = append(badData, "user may not be set") } if setRole { badData = append(badData, "role may not be set") </ins> } return CheckResult{

If I configure the namespace to enforce PSS checks, the return message ends abruptly for : The resource manifest that can be used to reproduce:
/transfer kubernetes
/sig auth /cc
Thanks for the report, fix in
Thanks for the quick fix !


Question: } }) <del> ginkgo.It("should prevent Ingress creation if more than 1 IngressClass marked as default", func() { </del> <ins> ginkgo.It("should prevent Ingress creation if more than 1 IngressClass marked as default [Serial]", func() { </ins> ingressClass1, err := createIngressClass(cs, "ingressclass1", true, f.UniqueName) framework.ExpectNoError(err) defer deleteIngressClass(cs, ingressClass1.Name)

<!-- Please only use this template for submitting reports about flaky tests or jobs (pass or fail with no underlying change in code) in Kubernetes CI --Which jobs are flaking: gce-master-scale-correctness Which test(s) are flaking: [sig-network] Ingress API should support creating Ingress API operations [Conformance] Testgrid link: https://k8s- Reason for failure: Anything else we need to know: /kind flake /sig scalability /sig network /cc
/triage unresolved Comment when the issue is assessed and confirmed. I am a bot run by vllry.
this is caused by the issue described in a different test is creating a default ingressclass, which breaks creation of ingress objects if there was already a default ingressclass set up in the cluster /assign
this should resolve the issue:
I take it back, looks like the conformance test is also creating default ingressclass objects... it should not fix in


Question: expectedLB:          expectedSLb5, expectedError:       nil, }, <ins> { desc:            "reconcileLoadBalancer shall reconcile UDP services", loadBalancerSku: "basic", service:         service6, existingLB:      lb6, wantLb:          true, expectedLB:      expectedLB6, expectedError:   nil, }, { desc:            "reconcileLoadBalancer shall reconcile probes for local traffic policy UDP services", loadBalancerSku: "basic", service:         service7, existingLB:      lb7, wantLb:          true, expectedLB:      expectedLB7, expectedError:   nil, }, </ins> } for i, test := range testCases {

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: Refer , health probes are not associated for UDP services on Azure, hence source IPs are not preserved for UDP services on Azure. What you expected to happen: When is set to for UDP services, source IPs should be preserved and the health probe should be associated with UDP load balancer rule. Per the , the health probe only supports TCP, HTTP and HTTPS. UDP is not supported as the probing protocol, hence we can’t enable the health probes for UDP services. However, when the externalTrafficPolicy is Local, Kubernetes would open an HTTP endpoint for UDP services, hence it’s actually supported on Azure. So, should associate the probes on externalTrafficPolicy=Local cases. How to reproduce it (as minimally and precisely as possible): Create a UDP service with : Then check the source IPs. Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others: /kind bug /sig cloud-provider /area provider/azure


Question: } } <del> func Reset() { requestCounter.Reset() requestLatencies.Reset() requestLatenciesSummary.Reset() responseSizes.Reset() } </del> // InstrumentRouteFunc works like Prometheus' InstrumentHandlerFunc but wraps // the go-restful RouteFunction instead of a HandlerFunc plus some Kubernetes endpoint specific information. func InstrumentRouteFunc(verb, resource, subresource, scope string, routeFunc restful.RouteFunction) restful.RouteFunction {

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: endpoint of different k8s components expose Prometheus metrics that are not for them. For example, exposes metrics: or : What you expected to happen: Prometheus metrics only for to appear on kubelet's endpoint. How to reproduce it (as minimally and precisely as possible): Access the endpoint of all Kubernetes components Anything else we need to know?: This problem comes from usage of functions in different packages that automatically register Prometheus metrics. Just importing that package or some package that depends on it can cause those metrics to appear in unexpected places. In the list bellow you can find the different packages that have init functions and affected binaries: [ ] () cloud-controller-manager hyperkube kube-apiserver kube-controller-manager kube-proxy kube-scheduler [ ] () cloud-controller-manager hyperkube kube-apiserver kube-controller-manager kube-proxy kube-scheduler [ ] cloud-controller-manager hyperkube kube-apiserver kube-controller-manager [ ] cloud-controller-manager hyperkube kube-apiserver kube-controller-manager kubelet [ ] cloud-controller-manager hyperkube kube-apiserver kube-controller-manager kube-proxy kube-scheduler [ ] cloud-controller-manager hyperkube kube-apiserver kube-controller-manager kube-proxy kube-scheduler kubectl kubelet [ ] cloud-controller-manager hyperkube kube-apiserver kube-controller-manager kube-proxy kube-scheduler kubelet [ ] hyperkube kube-apiserver kube-controller-manager [ ] hyperkube kube-apiserver [ ] cloud-controller-manager hyperkube kube-apiserver kube-controller-manager kubelet [ ] cloud-controller-manager hyperkube kube-apiserver kube-controller-manager kube-proxy kube-scheduler kubelet Environment: Kubernetes version (use ):Cloud provider or hardware configuration: N/A OS (e.g. from /etc/os-release): N/A Kernel (e.g. ): N/A Install tools: N/A Others: N/A /sig instrumentation
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-testing, kubernetes/test-infra and/or . /close
Closing this issue. <detailsIn response to : Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: // 	useHyperKubeImage: false // 	clusterName: "example-cluster" // 	--- <del> // 	apiVersion: kubelet.config.k8s.io/v1beta2 </del> <ins> // 	apiVersion: kubelet.config.k8s.io/v1beta1 </ins> // 	kind: KubeletConfiguration // 	# kubelet specific options here // 	---

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: references and a broken link to this (because it doesn't exist!) I'm guessing this was a bad find and replace when copying kubeadm's own v1beta1 to v1beta2 What you expected to happen: accurate documentation How to reproduce it (as minimally and precisely as possible): read the docs Anything else we need to know?: this template has a lot of fields :^) Environment: anything Kubernetes version (use ): n/a Cloud provider or hardware configuration: n/a OS (e.g: ): n/a Kernel (e.g. ): n/a Install tools: kubeadm Network plugin and version (if this is a network-related bug): n/a Others: n/a /assign /sig cluster-lifecycle


Question: var map_ObjectMeta = map[string]string{ "":                           "ObjectMeta is metadata that all persisted resources must have, which includes all objects users must create.", "name":                       "Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/identifiers#names", <del> "generateName":               "GenerateName is an optional prefix, used by the server, to generate a unique name ONLY IF the Name field has not been provided. If this field is used, the name returned to the client will be different than the name passed. This value will also be combined with a unique suffix. The provided value has the same validation rules as the Name field, and may be truncated by the length of the suffix required to make the value unique on the server.nnIf this field is specified and the generated name exists, the server will NOT return a 409 - instead, it will either return 201 Created or 500 with Reason ServerTimeout indicating a unique name could not be found in the time allotted, and the client should retry (optionally after the time indicated in the Retry-After header).nnApplied only if Name is not specified. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#idempotency", </del> <ins> "generateName":               "GenerateName is an optional prefix, used by the server, to generate a unique name ONLY IF the Name field has not been provided. If this field is used, the name returned to the client will be different than the name passed. This value will also be combined with a unique suffix. The provided value has the same validation rules as the Name field, and may be truncated by the length of the suffix required to make the value unique on the server.nnIf this field is specified and the generated name exists, the server will return a 409.nnApplied only if Name is not specified. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#idempotency", </ins> "namespace":                  "Namespace defines the space within which each name must be unique. An empty namespace is equivalent to the "default" namespace, but "default" is the canonical representation. Not all objects are required to be scoped to a namespace - the value of this field for those objects will be empty.nnMust be a DNS_LABEL. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/namespaces", "selfLink":                   "Deprecated: selfLink is a legacy read-only field that is no longer populated by the system.", "uid":                        "UID is the unique in time and space value for this object. It is typically generated by the server on successful creation of a resource and is not allowed to change on PUT operations.nnPopulated by the system. Read-only. More info: http://kubernetes.io/docs/user-guide/identifiers#uids",

The error code of is 409. Is this an expected behavior? The doc seems to say no: NOT return a 409 - instead, it will either return 201 Created or 500 with Reason ServerTimeout indicating a unique name could not be found in the time allotted, and the client should retry (optionally after the time indicated in the Retry-After header). seems to create which error code is 409. The error code of isn't 409. Or, the doc doesn't say "If this field is specified and the generated name exists, the server will NOT return a 409". Create a large number of Pods with generateName, and cause name conflicts. Just question: if the API is correct in returning 409 for a GenerateName-conflict, how do we determine that error? We want to retry when GenerateName conflict returns in most cases because the retry may solve this conflict. latest master (maybe after ) <details</details<details</details<details</details<details</details<details</details
Sorry if not an appropriate SIG. /sig api-machinery
/assign
It seems related to . If it's expected behavior, the document isn't correct..?
is a mistake. /unassign
Hi do you think this is a good first issue to tackle? If so, I would like to try contributing here.
/assign /triage accepted
Forgive me to ping other people who might know. /cc /cc /cc
I suspect we just failed to update the documentation.
Thanks Okay. I'll create the PR for that. /kind documentation /assign


Question: continue } <ins> nodeAffinityValid := true </ins> if node != nil { // Scheduler path, check that the PV NodeAffinity // is satisfied by the node err := volumeutil.CheckNodeAffinity(volume, node.Labels) if err != nil { <del> continue </del> <ins> nodeAffinityValid = false </ins> } }

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: /assign What happened: Volume got bound to PV on node "kubernetes-minion-group-82c8" When the pod went back through scheduling, the predicate returned success for other nodes even though we should have updated the PV cache to point to the PVC: I think in , instead of continuing if the node affinity doesn't match, it should fail if the PV is prebound.
This causes two issues: On the second pass, the scheduler may choose a different node. Then in Assume/Bind volumes, we'll also prebind a second set of PVs. If the scheduler chose a different node on the second pass, and the PVCs became fully bound in between the predicates and assume volumes, then assume volumes will return success and the pod will get scheduled to the wrong node.


Question: return nil } <del> func NewTableSorter(table *metav1beta1.Table, field string) *TableSorter { </del> <ins> func NewTableSorter(table *metav1beta1.Table, field string) (*TableSorter, error) { </ins> var parsedRows [][][]reflect.Value parser := jsonpath.New("sorting").AllowMissingKeys(true) err := parser.Parse(field) if err != nil { <del> klog.Fatalf("sorting error: %vn", err) </del> <ins> return nil, fmt.Errorf("sorting error: %v", err) </ins> } <ins> fieldFoundOnce := false </ins> for i := range table.Rows { parsedRow, err := findJSONPathResults(parser, table.Rows[i].Object.Object) if err != nil { <del> klog.Fatalf("Failed to get values for %#v using %s (%#v)", parsedRow, field, err) </del> <ins> return nil, fmt.Errorf("Failed to get values for %#v using %s (%#v)", parsedRow, field, err) </ins> } parsedRows = append(parsedRows, parsedRow) <ins> if len(parsedRow) > 0 && len(parsedRow[0]) > 0 { fieldFoundOnce = true } } if len(table.Rows) > 0 && !fieldFoundOnce { return nil, fmt.Errorf("couldn't find any field with path %q in the list of objects", field) </ins> } return &TableSorter{ obj:        table, field:      field, parsedRows: parsedRows, <del> } </del> <ins> }, nil </ins> } func findJSONPathResults(parser *jsonpath.JSONPath, from runtime.Object) ([][]reflect.Value, error) { if unstructuredObj, ok := from.(*unstructured.Unstructured); ok {

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!--What happened: seems to have regressed in v1.13.0 and no longer sorts by creation timestamp. What you expected to happen: should sort by creation timestamp. How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: N/A Environment: Kubernetes version (use ): <!-- DO NOT EDIT BELOW THIS LINE --/kind bug
/sig cli
/assign
bisected to () /assign
opened to fix
/assign


Question: roundedVal: int32(1000), }, { <del> name:        "round overflowed quantity to int32", resource:    resource.MustParse("73786976299133170k"), </del> <ins> name:       "round decimal to GiB", resource:   resource.MustParse("1.2Gi"), roundedVal: int32(2), }, { name:       "round big value to GiB", resource:   resource.MustParse("2047Pi"), roundedVal: int32(2146435072), }, { name:        "round quantity to GiB that would lead to an int32 overflow", resource:    resource.MustParse("2048Pi"), </ins> roundedVal:  int32(0), expectError: true, },

After upgrading our clusters to Kubernetes 1.19, the provisioning of PersistentVolumes started failing with the following Events: The value matches the actual storage request of 3.2Gi. It doesn't seem special in any way, not too big, not too small, and it always worked until 1.19. We look into the issue with and we believe the PR introduced a change that broke the PersistentVolume provisioning even when it doesn’t overflow int64. Amounts of resource in Kubernetes are represented with the . In the PersistentVolume drivers, there are some functions that manipulate storage request in order to allocate a disk with the right size. Before the mentioned PR, those quantities were retrieved by calling which relies on to return the value as an int64. This worked probably fine in most of the cases, but it could happen that the represented value would overflow an int64, and in this case the result of would be wrong. The function’s description states that it’s the caller’s responsibility to check this: This is likely what the PR wanted to handle. The change replaced all calls with which has a different signature. From where it’s used now, you would think that the second return value signals an overflow: : This is probably where the error we’re seeing is generated. I believe that the signature of was misinterpreted. From my understanding of its description, returning doesn’t mean we’re overflowing . It only means that internally, the quantity is represented as a decimal amount: Internally, depending on their values, quantities are stored/represented in : for DecimalExponent (e.g. ) or DecimalSI (e.g. ) for BinarySI that don’t have a decimal part but a unit (simplified) (e.g. ) for all the rest With the PR, Kubernetes now fails to provision volumes that have a storage request represented as . Many of the storage requests are represented this way and still perfectly valid in our opinion. I would have expected the provisioning to continue to work as those requests seemed valid and were always accepted by our underlying cloud-provider in earlier version of Kubernetes.

Create a StatefulSet with a PersistentVolumeClaim that uses vSphere storage: Kubernetes version (use ): 1.19.7 Cloud provider or hardware configuration: vSphere, but I think it impacts other cloud-provider that were changed in the PR: /sig storage /kind bug /cc Possibly related to
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I think the only reason we do not use the quantity library for scaling and why we tried to roll our own rounding code is because when scaling stuff in AsScaledInt64(3) or ScaledValue(3), the library assumes decimal SI (M, 1000), not binary SI units (Mi, 1024): You are right, our self-rolled rounding code doesn't handle values with millibyte or higher precision like 3.2Gi. Our code uses AsInt64 and then performs its own scaling. If AsInt64 returns false, the int64 returned is not in bytes (it must be in millibytes or higher precision), so our rounding code won't work, so it returns that " is too great, overflows int64". In 1.18, we used Value() to round to bytes but we didn't check it against ScaleValue as the signature suggested. So the PR in question fixed cases where Value() overflowed but inadvertently broke your case. The correct way to round, which the resource library's rounding does, is to drop the useless precision and round up. , but as I said it doesn't seem to handle binary SI quantities correctly. For decimal SI quantities, we should rely on the resource library's AsScaledInt64 and be done with it. (We could also use Value() and ScaledValue as the signature suggests but calling AsScaledInt64 directly seems simpler). Like this: For binary SI quantities, we need to do something fancier involving CanonicalizeBytes I think, I'm not sure yet.
I guess for binary SI we can go back to using Value but actually check the overflow this time.

Hi Thanks for the quick feedback on the problem's description already. Indeed, the Scaled() functions don't handle BinarySI and that's why I quickly discarded the idea of using them. Since this is about storage, I suppose most people are using this format. I just finished wrapping up a PR that I hope address both the issue we are facing, but also what was meant to be fixed in the PR your reviewed back then. It doesn't make any difference between the 3 formats however.
Yeah I commented within seconds of your PR. I glanced at it , will review more soon, but it makes sense to me to just use VAlue and check ofr overflow for both decimal SI and binary SI rather than using Scaled() for one and not the other :+1:
I think it means both. i.e - when incoming value is decimal or if value is so big that type can't store it as it switches to decimal. So while my PR fixed later case, it broke the former. :/


Question: // Initialize passes a Kubernetes clientBuilder interface to the cloud provider func (c *Cloud) Initialize(clientBuilder controller.ControllerClientBuilder) { c.clientBuilder = clientBuilder <del> c.kubeClient = clientBuilder.ClientOrDie("cloud-provider") </del> <ins> c.kubeClient = clientBuilder.ClientOrDie("aws-cloud-provider") </ins> c.eventBroadcaster = record.NewBroadcaster() c.eventBroadcaster.StartLogging(glog.Infof) c.eventBroadcaster.StartRecordingToSink(&v1core.EventSinkImpl{Interface: v1core.New(c.kubeClient.CoreV1().RESTClient()).Events("")}) <del> c.eventRecorder = c.eventBroadcaster.NewRecorder(scheme.Scheme, v1.EventSource{Component: "aws-cloudprovider"}) </del> <ins> c.eventRecorder = c.eventBroadcaster.NewRecorder(scheme.Scheme, v1.EventSource{Component: "aws-cloud-provider"}) </ins> } // Clusters returns the list of clusters.

Recently introduced fix of tainting the node if EBS volume on a node is stuck for too long in "attaching" state does not work on RBAC enabled clusters because of missing service account permissions. /sig storage /sig aws


Question: <ins> /* Copyright 2015 The Kubernetes Authors All rights reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package benchmark import ( "net/http" "net/http/httptest" "github.com/golang/glog" "k8s.io/kubernetes/pkg/api" "k8s.io/kubernetes/pkg/api/resource" "k8s.io/kubernetes/pkg/api/testapi" "k8s.io/kubernetes/pkg/client/record" client "k8s.io/kubernetes/pkg/client/unversioned" "k8s.io/kubernetes/pkg/master" "k8s.io/kubernetes/plugin/pkg/scheduler" _ "k8s.io/kubernetes/plugin/pkg/scheduler/algorithmprovider" "k8s.io/kubernetes/plugin/pkg/scheduler/factory" "k8s.io/kubernetes/test/integration/framework" ) // mustSetupScheduler starts the following components: // - k8s api server (a.k.a. master) // - scheduler // It returns scheduler config factory and destroyFunc which should be used to // remove resources after finished. // Notes on rate limiter: //   - The BindPodsRateLimiter is nil, meaning no rate limits. //   - client rate limit is set to 5000. func mustSetupScheduler() (schedulerConfigFactory *factory.ConfigFactory, destroyFunc func()) { framework.DeleteAllEtcdKeys() var m *master.Master masterConfig := framework.NewIntegrationTestMasterConfig() m = master.New(masterConfig) s := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) { m.Handler.ServeHTTP(w, req) })) c := client.NewOrDie(&client.Config{ Host:         s.URL, GroupVersion: testapi.Default.GroupVersion(), QPS:          5000.0, Burst:        5000, }) schedulerConfigFactory = factory.NewConfigFactory(c, nil) schedulerConfig, err := schedulerConfigFactory.Create() if err != nil { panic("Couldn't create scheduler config") } eventBroadcaster := record.NewBroadcaster() schedulerConfig.Recorder = eventBroadcaster.NewRecorder(api.EventSource{Component: "scheduler"}) eventBroadcaster.StartRecordingToSink(c.Events("")) scheduler.New(schedulerConfig).Run() destroyFunc = func() { glog.Infof("destroying") close(schedulerConfig.StopEverything) s.Close() glog.Infof("destroyed") } return } func makeNodes(c client.Interface, nodeCount int) { glog.Infof("making %d nodes", nodeCount) baseNode := &api.Node{ ObjectMeta: api.ObjectMeta{ GenerateName: "scheduler-test-node-", }, Spec: api.NodeSpec{ ExternalID: "foobar", }, Status: api.NodeStatus{ Capacity: api.ResourceList{ api.ResourcePods:   *resource.NewQuantity(32, resource.DecimalSI), api.ResourceCPU:    resource.MustParse("4"), api.ResourceMemory: resource.MustParse("32Gi"), }, Phase: api.NodeRunning, Conditions: []api.NodeCondition{ {Type: api.NodeReady, Status: api.ConditionTrue}, }, }, } for i := 0; i < nodeCount; i++ { if _, err := c.Nodes().Create(baseNode); err != nil { panic("error creating node: " + err.Error()) } } } // makePods will setup specified number of scheduled pods. // Currently it goes through scheduling path and it's very slow to setup large number of pods. // TODO: Setup pods evenly on all nodes and quickly/non-linearly. func makePods(c client.Interface, podCount int) { glog.Infof("making %d pods", podCount) basePod := &api.Pod{ ObjectMeta: api.ObjectMeta{ GenerateName: "scheduler-test-pod-", }, Spec: api.PodSpec{ Containers: []api.Container{{ Name:  "pause", Image: "gcr.io/google_containers/pause:1.0", Resources: api.ResourceRequirements{ Limits: api.ResourceList{ api.ResourceCPU:    resource.MustParse("100m"), api.ResourceMemory: resource.MustParse("500Mi"), }, Requests: api.ResourceList{ api.ResourceCPU:    resource.MustParse("100m"), api.ResourceMemory: resource.MustParse("500Mi"), }, }, }}, }, } threads := 30 remaining := make(chan int, 1000) go func() { for i := 0; i < podCount; i++ { remaining <- i } close(remaining) }() for i := 0; i < threads; i++ { go func() { for { _, ok := <-remaining if !ok { return } for { _, err := c.Pods("default").Create(basePod) if err == nil { break } } } }() } } </ins>

We have found that on 1k nodes cluster scheduling takes 60-200 ms and becomes the bottleneck. We want to have a standard way to reproduce scheduling latency metrics result and benchmark scheduler as simple and fast as possible. Only with that can we: 1. figure out underlying behaviors and potential bottlenecks quickly; 2. publish results in a standard way and prevent future performance degradation. We have the following goals: Goal 1: Save time on testing. More accurate scheduling latency. We mock up kubelets and only measure scheduling latency and do not care about the latency to start the pod. This is different from Kubemark -- scheduled pods won’t be run by mocked kubelets. We only need to set up the master components. Goal 2: Profiling runtime metrics and find out performance bottleneck. Write scheduler integration test. Take advantage of go test and profiling tools, like cpu-profiling, memory-profiling and block-profiling. This is very fine-grained metrics. Goal 3: Reproduce test result easily. We want to have a known place to do the performance related test for scheduler. Developers should just run one script to collect all the information they need. We have the following plans for writing scheduler tests: density test (by adding a new Go test) schedule 30k pods on 1000 (fake) nodes and 3k pods on 100 (fake) nodes print out scheduling rate every second let you learn the rate changes vs number of scheduled pods benchmark (by modifying the existing benchmark in integration or simply add a new one) make use of and report nanosecond/op. schedule b.N pods when the cluster has N nodes and P scheduled pods. Since it takes relatively long time to finish one round, b.N is small -- level of 10. We are new to k8s codebase. We also want to know where is the best place (directory) to commit in these tests? Any suggestions are welcomed!
/cc
This extends
So density already does this... the one option it looks like you may want is custom control , real or fake is irrelevant as the e2e's just need to point to an api-server. re: benchmarks, seems like we could simply add to the unit-tests.
After chatting with lets add this as an agenda item for the SIG meeting tomorrow.

+1 for adding benchmark (howeveer this should be integration test, not unit-test in my opinion)
cc The next SIG on scheduler meeting is on 12.07, isn't it? If there is one tomorrow, can I have the information please?
retract, +1 to . I was adding this to the list given the context, but it crosses over into both.
Slides from today's meeting:
Here are are the action item notes from the SIG meeting: Everyone agrees this is a good idea and long overdue, but the question is where would this live: New category of tests? (isolated/sandbox’d/component) The current testing structure allows to easily append. create new directory (component) + hack/script to execute Future "we could" gate on known good data, and fail performance regressions
/cc
/cc


Question: } } } <del> return nil </del> } func (c *threadSafeMap) Resync() error {

Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG Kubernetes version (use ): k8s1.3.5. still exist in 1.6 What happened: when exec threadSafeMap.Index func, the returned items contain nil object. That is to say, index is in threadSafeMap but the corresponding has already been deleted. What you expected to happen: when object is deleted from threadSafeMap, all its index must be deleted. How to reproduce it (as minimally and precisely as possible): Suppose we init threadSafeMap with two indexers: indexFunc1 and indexFunc2. Obviously, indexFunc may return error if the object has something wrong or some other reasons. when we add an object, will first add the object to , then compute the index and add the index into threadSafe.indices. when we delete the object, will first delete the index from indices, then delete it from Suppose IndexFunc1(object) return error and indexFunc2(object) works well. I think this circumstance is absolutely possible to happen. Also we know that the order to exec indexFunc is not decided. Feature of iterating map. When we add object, suppose first exec indexFunc2, and then exec indexFunc1. So the index of indexFunc2 is to indices. But the index of indexFunc1 is not , because indexFunc1(obj) return err. Then we delete the object, suppose first exec indexFunc1. It will return error and IndexFunc2 has no chance to run. So the corresponding index indexFunc2(obj) can't be removed. But then the object is removed from items. This problem may result in subtle problems. For example, when we exec threadSafeMap.Index( ) func the returned list may include nil object. Anything else we need to know The reason is that index is still is threadSafeMap but the corresponding object has already been removed. We must clear all corresponding indexs when delete object. And the reason why index is not deleted is that when for-range map, the order is not decided.
Who can help me? Do i miss something important or misunderstand it ?
, can you share a unit test to show it? I'd like to help on it.
Thank you for your help. I think i have decribed the condition resulting in the bug indetail.

Be simple, because the order of exec indexFunc is random when update indices, and some function don't care the error returned, the obj is deleted but corresponding indices is still in it. NOTE: the obj is deleted but corresponding indices is still in it So when we run index func, the returned list may contain nil object. Because when get obj from threadsafemap with indice, not care whether the obj exist in this map. Then nil object .
. I am sure this problem exists. And i think we can find a better way to deal with it.
/sig api-machinery threadSafeMap is in client-go
/kind bug
Opened a PR with a fix.
In my understanding, should be deterministic, could you elaborate more how your IndexFunc implemented? If an object can not be called on this , then the question is, how could it exist in at the first beginning?
threadSafeStore should tolerate some bad conditions and deal with it gracefully, instead generating strange and confusing phenomenon. ThreadSafeStore can contain mutiple index funcs. If one of them is not implemented very well, or one of objects we pass to it has something wrong, STRANGE AND CONFUSING phenomenon happens, see what I said above.
That's the point I would like to know more, the call on an object should succeed or fail unconditionally(deterministic).
wechat
I also think index function must be a pure function. See my comment on the pr


Question: {Name: "net.ipv4.ip_unprivileged_port_start", Value: "1024"}, {Name: "net.ipv4.ip_local_reserved_ports", Value: "1024-4999"}, {Name: "net.ipv4.tcp_keepalive_time", Value: "7200"}, <ins> {Name: "net.ipv4.tcp_fin_timeout", Value: "60"}, {Name: "net.ipv4.tcp_keepalive_intvl", Value: "75"}, {Name: "net.ipv4.tcp_keepalive_probes", Value: "9"}, </ins> } }), }

Add the following to be safe sysctls: net.ipv4.tcpfintimeout net.ipv4.tcpkeepaliveintvl net.ipv4.tcpkeepaliveprobes These are related settings to net.ipv4.tcpkeepalivetime that was enabled in These are related ipv4 TCP settings for tuning network connections between a client and a pod.
/sig network
May you help provide some tests to ensure that setting this sysctl can't affect any other pods or the node.
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : May you help provide some tests to ensure that setting this sysctl can't affect any other pods or the node. /sig nodes Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig node
I adjusted it with the plugin.
Would this change apply at the node level or the pod level? I assumed it would be at the pod level, and should not affect any other pods--however Kubernetes does not natively have any limits on network connections that I am aware of--but only the pod with the changed net settings would be affected.
To verify that a sysctl has an effect on other pods, we need to verify that it's a namespace sysctl.
Here's a simple test Use net.ipv4.tcpfintimeout sysctl to create a container check if it set on the host =It is the default value on host run another container attach to the network namespace of the one where the sysctl is set =results in sysctl being set Verify in another container under a different network namespace that it is not set =It is the default value According to the test results, net.ipv4.tcpfintimeout is a namespaced sysctl
I likewise verified and , which are also namespaced sysctl, and I believe we can add them to the safe sysctl /assign
Probably need to wait for to finish before start working on it
/triage accepted


Question: ... ``` <del> If you don't specify master and minion sizes, the scripts will attempt to guess the correct size of the master and worker nodes based on `${NUM_NODES}`. In particular, for clusters less than 50 nodes it will use a `t2.micro`, for clusters between 50 and 150 nodes it will use a `t2.small` and for clusters with greater than 150 nodes it will use a `t2.medium`. </del> <ins> If you don't specify master and minion sizes, the scripts will attempt to guess the correct size of the master and worker nodes based on `${NUM_NODES}`. In version 1.2 these default are: * For the master, for clusters of less than 150 nodes it will use an `m3.medium`, for clusters of greater than 150 nodes it will use an `m3.large`. * For worker nodes, for clusters less than 50 nodes it will use a `t2.micro`, for clusters between 50 and 150 nodes it will use a `t2.small` and for clusters with greater than 150 nodes it will use a `t2.medium`. </ins> WARNING: beware that `t2` instances receive a limited number of CPU credits per hour and might not be suitable for clusters where the CPU is used consistently. As a rough estimation, consider 15 pods/node the absolute limit a `t2.large` instance can handle before it starts exhausting its CPU credits steadily, although this number depends heavily on the usage. <ins> In prior versions of kubernetes, we defaulted the master node to a t2-class instance, but found that this sometimes gave hard-to-diagnose problems when the master ran out of memory or CPU credits.  If you are running a test cluster and want to save money, you can specify `export MASTER_SIZE=t2.micro` but if your master pauses do check the CPU credits in the AWS console. For production usage, we recommend at least `export MASTER_SIZE=m3.medium` and `export NODE_SIZE=m3.medium`.  And once you get above a handful of nodes, be aware that one m3.large instance has more storage than two m3.medium instances, for the same price. We generally recommend the m3 instances over the m4 instances, because the m3 instances include local instance storage.  Historically local instance storage has been more reliable than AWS EBS, and performance should be more consistent. The ephemeral nature of this storage is a match for ephemeral container workloads also! If you use an m4 instance, or another instance type which does not have local instance storage, you may want to increase the `NODE_ROOT_DISK_SIZE` value, although the default value of 32 is probably sufficient for the smaller instance types in the m4 family. </ins> The script will also try to create or reuse a keypair called "kubernetes", and IAM profiles called "kubernetes-master" and "kubernetes-minion". If these already exist, make sure you want them to be used here.

I propose that we default to for the master node (and maybe for 150 nodes?). We've seen problems with running out of memory on the master with the , and we've seen pauses that look like CPU exhaustion on the t2 class in general. Also the master is the hardest instance to resize. It is doable to reconfigure the ASG for the nodes (or add a new one, but the master is tricky). what do you think? cc


Question: ginkgo.By("creating") eps1 := epsTemplate.DeepCopy() eps1.Endpoints = []discoveryv1.Endpoint{ <del> {Addresses: []string{pod1.Status.PodIP}}, </del> <ins> { Addresses:  []string{pod1.Status.PodIP}, Conditions: discoveryv1.EndpointConditions{Ready: &readyCondTrue}, }, </ins> } eps1.Ports = []discoveryv1.EndpointPort{{ Name:     pointer.String("port80"),

The test create custome EndpointSlices to validate some behaviors sending traffic, so it is clear that it expects the EndpointSlices to be ready, The API doc is vague about this situation and leaves room to interpret the "unknown behavior", quoting Since the test expects the EndpointSlices to be Ready we should set them, if we decide to change the API behavior, then we should add an additional e2e test to validate this behavior, but not use the same test to validate two different things /kind bug /kind failing-testing /sig network Slack thread
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : The test create custome EndpointSlices to validate some behaviors sending traffic, so it is clear that it expects the EndpointSlices to be ready, /kind failing-testing /sig network Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign
GitHub didn't allow me to assign the following users: aauren. Note that only with read permissions, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : /assign Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Thanks I'll take this one and draft up a PR this weekend. (this comment should hopefully allow the k8s ci robot to assign this to me as well)
if you are planning to contribute more often you should apply as org member


Question: ctx, cancel = getContextWithCancel() defer cancel() <del> if _, err := c.common.cloud.DisksClient.CreateOrUpdate(ctx, c.common.resourceGroup, diskName, result); err != nil { </del> <ins> if _, err := c.common.cloud.DisksClient.CreateOrUpdate(ctx, resourceGroup, diskName, result); err != nil { </ins> return oldSize, err }

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . If the matter is security related, please disclose it privately via --Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: there will be azure disk creation issue when specifying an external resource group, after azure disk creation succeeded, it fails to get azure disk state What you expected to happen: How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): v1.11 Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others: /assign /azure
/sig azure
/kind bug


Question: cfg:      &kubeadmapi.NodeConfiguration{}, expected: false, }, <ins> { cfg: &kubeadmapi.NodeConfiguration{ DiscoveryTokenAPIServers: []string{"192.168.1.15"}, }, expected: false, }, { cfg: &kubeadmapi.NodeConfiguration{ DiscoveryTokenAPIServers: []string{"2001:1234::1:15"}, }, expected: false, }, </ins> } for _, rt := range tests {

BUG REPORT /kind bug What happened: Kubeadm checks that bridge-nf-call-iptables (IPv4) is set, but does not check for bridge-nf-call-ip6tables (IPv6) What you expected to happen: If IPv6 is being used, then Kubeadm should check for bridge-nf-call-ip6tables How to reproduce it (as minimally and precisely as possible): Just running Kubeadm, however it's only a problem when running IPv6 Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration**: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others:
/area ipv6 /area kubeadm
/sig network


Question: labels: app: cockroachdb annotations: <del> scheduler.alpha.kubernetes.io/affinity: > { "podAntiAffinity": { "preferredDuringSchedulingIgnoredDuringExecution": [{ "weight": 100, "labelSelector": { "matchExpressions": [{ "key": "app", "operator": "In", "values": ["cockroachdb"] }] }, "topologyKey": "kubernetes.io/hostname" }] } } </del> # Init containers are run only once in the lifetime of a pod, before # it's started up for the first time. It has to exit successfully # before the pod's main containers are allowed to start.

<!-- Thanks for filing an issue! Before hitting the button, please answer these questions.--Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see ): What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.): Is this a BUG REPORT or FEATURE REQUEST? (choose one): <!-- If this is a BUG REPORT, please: Fill in as much of the template below as you can. If you leave out information, we can't help you as well. If this is a FEATURE REQUEST, please: Describe in detail the feature/behavior/change you'd like to see. In both cases, be ready for followup questions, and please respond in a timely manner. If we can't reproduce a bug or think a feature already exists, we might close your issue. If we're wrong, PLEASE feel free to reopen it and explain why. --Kubernetes version (use ): master Environment: Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others: What happened: Check , it is still using , we should move it to . What you expected to happen: How to reproduce it (as minimally and precisely as possible): Anything else we need to know:
/assign
probably will want to move the init container also to the PodSpec
I think the init container still depend on annotations , can I specify in ? If so, I want to file another issue to track it
FYI Init containers have graduated to GA and now appear as a field. The beta annotation value will still be respected and overrides the field value. (,
Seems like that the doc should be updated. EDIT: NOTE: This feature is beta in Kubernetes 1.5. Once the feature exits beta, Init Containers will be specified in the PodSpec alongside the app containers array.
Thanks I filed an issue here
What about merge this first and then handle init-container in a separate PR?
Sounds good to me, thanks.


Question: // add old missing cache data with nil entries to prevent aggressive // ARM calls during cache invalidation for name, vmEntry := range oldCache { <del> // if the nil cache entry has existed for 15 minutes in the cache </del> <ins> // if the nil cache entry has existed for vmssVirtualMachinesCacheTTL in the cache </ins> // then it should not be added back to the cache <del> if vmEntry.virtualMachine == nil && time.Since(vmEntry.lastUpdate) > 15*time.Minute { </del> <ins> if vmEntry.virtualMachine == nil && time.Since(vmEntry.lastUpdate) > vmssVirtualMachinesCacheTTL { </ins> klog.V(5).Infof("ignoring expired entries from old cache for %s", name) continue }

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: When a node is deleted and then recreated in short time, the "instance not found" error would be reported from Azure cloud provider and hence the node object is deleted by kube-controller-manager. It is because the deleted VMSS VM would be cached for 15 minutes (this is used to avoid throttling issues when a large number of nodes are deleted): The cache above is too long and should be reduced to a shorter value, e.g. 3 minutes. What you expected to happen: The newly created Node should not be deleted. How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others: /kind bug /priority critical-urgent /area provider/azure /sig cloud-provider
is anyone already working on this?
is working on a fix for this. Would you like to have a look?
will do
Thank you for working on this, and /assign