Question: totalMemory += container.Memory } } <del> percentageCPU := (totalCPU * 100) / resources.GetIntegerResource(node.NodeResources.Capacity, resources.CPU, 0) percentageMemory := (totalMemory * 100) / resources.GetIntegerResource(node.NodeResources.Capacity, resources.Memory, 0) </del> <ins> percentageCPU := calculatePercentage(totalCPU, resources.GetIntegerResource(node.NodeResources.Capacity, resources.CPU, 0)) percentageMemory := calculatePercentage(totalMemory, resources.GetIntegerResource(node.NodeResources.Capacity, resources.Memory, 0)) </ins> glog.V(4).Infof("Least Requested Priority, AbsoluteRequested: (%d, %d) Percentage:(%d%m, %d%)", totalCPU, totalMemory, percentageCPU, percentageMemory) return HostPriority{

HPA tests are currently failing in gke-large cluster suite: I did some debugging and the problem seems to be related to pod-nanny. Looking into logs from the last run: was first scheduled on the large node dedicated to it pod-nany updated heapster deployment then heapster was scheduled on the normal node (when it actually shouldn't fit) that point it was never updated by the pod nanny. that point, heapster started crashlooping (OOMing) [which in fact was expected in too small machine] So it's pretty clear that that pod-nanny changed the amount of resources to something smaller, which in fact it shouldn't.


Question: type listenPortOpener struct{} // OpenLocalPort holds the given local port open. <del> func (l *listenPortOpener) OpenLocalPort(lp *utilproxy.LocalPort) (utilproxy.Closeable, error) { return openLocalPort(lp) </del> <ins> func (l *listenPortOpener) OpenLocalPort(lp *utilproxy.LocalPort, isIPv6 bool) (utilproxy.Closeable, error) { return openLocalPort(lp, isIPv6) </ins> } <del> func openLocalPort(lp *utilproxy.LocalPort) (utilproxy.Closeable, error) { </del> <ins> func openLocalPort(lp *utilproxy.LocalPort, isIPv6 bool) (utilproxy.Closeable, error) { </ins> // For ports on node IPs, open the actual port and hold it, even though we // use ipvs to redirect traffic. // This ensures a) that it's safe to use that port and b) that (a) stays

/kind bug What happened: When creating a node port application (in this case the example guestbook) on a host server that is dual stacked and has AAAA and A DNS entries, the node port on the v6 address is open to kube-proxy and hangs. What you expected to happen: The v6 address would refuse the connection and the client would fall back onto the v4 address which would connect correctly. How to reproduce it (as minimally and precisely as possible): kubeadm init with the above, and join in some workers. Then install the guestbook application Obtain the node port with and telnet to one of the servers on that port from a dual stacked client. The client will connect and refuse to respond to 'GET /' telnet directly to the IPv4 address of the server on the port and all works as expected. Anything else we need to know?: kube-proxy is noted as listening on the port. Environment: Kubernetes version (use ): ubuntu kubectl version Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.1", GitCommit:"", GitTreeState:"clean", BuildDate:"2018-07-17T18:53:20Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.1", GitCommit:"", GitTreeState:"clean", BuildDate:"2018-07-17T18:43:26Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"} Cloud provider or hardware configuration: Brightbox OS (e.g. from /etc/os-release): Ubuntu 18.04.1 LTS Kernel (e.g. ): Linux srv-jidtj 4.15.0-29-generic -Ubuntu SMP Tue Jul 17 15:39:52 UTC 2018 x8664 x8664 x86_64 GNU/Linux Install tools: kubeadm 1.11 Others:
/sig network
Looks like kube-proxy in ipvs mode listens on the nodeport ports to ensure they're not in use by something else. It just binds and listens, it doesn't serve requests.

It binds on both ipv4 and ipv6 but never expects to receive connections because the iptables/ipvs rules intercept them, but on ipv4 clusters there is nothing stopping ipv6 connections coming in. The comment there does admit that only a bind is necessary to do this, but doing a full listen just made e2e tests easier. /cc
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-testing, kubernetes/test-infra and/or . /close
Closing this issue. <detailsIn response to : Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen This now causes conformance failure on e2e tests due to the promotion of the Nodeport test in
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Still an issue on 1.16 when running in standard IPv4 mode
As I understand this thread, I have the same problem. I thought that it is a node configuration or some kubernetes flag that I didn't set but it seems it ain't.

I ask already on SO: Environment Kubernetes version (use kubectl version): Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"", GitTreeState:"clean", BuildDate:"2019-09-18T14:36:53Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.1", GitCommit:"", GitTreeState:"clean", BuildDate:"2019-10-02T16:51:36Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"} Cloud provider or hardware configuration: On premise, on virtual machines using hyper-v OS (e.g. from /etc/os-release): NAME="Ubuntu" VERSION="18.04.3 LTS (Bionic Beaver)" Kernel (e.g. uname -a): Linux kubernetes 5.0.0-29-generic # 31~18.04.1-Ubuntu SMP Thu Sep 12 18:29:21 UTC 2019 x8664 x8664 x86_64 GNU/Linux Install tools: kubeadm version: &version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"", GitTreeState:"clean", BuildDate:"2019-09-18T14:34:01Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
/lifecycle frozen
/assign bowei
If this issue has been triaged, please comment . If you aren't able to handle this issue, consider unassigning yourself and/or adding the label. I am a bot run by vllry.
Looking through the comments, this looks like a bug that could have a targeted fix. /remove-triage unresolved /good-first-issue /help-wanted
This request has been marked as suitable for new contributors. Please ensure the request meets the requirements listed . If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : Looking through the comments, this looks like a bug that could have a targeted fix.

/good-first-issue /help-wanted </details
I will attempt to solve this issue.
any success or progress? If you left this issue I would like to give it a go as a first time contributor. Thanks, Yuval
I'm still working on this as my first issue, and planning to produce a solution by 20th.
I wasn't able to tackle the issue and will not continue trying, feel free to take it.
/assign yuvalsteuer
Still, open -- anyone actively working on this? I would like to attempt if not. :) /assign rkc007
OK, again still open, doesn't look like anyone is working on this. I'd like to try it. The fix that occurs to me is simply to only listen to IPv6 addresses if the cluster is IPv6. If the e2e tests fail in this case (from looking at I don't think they will) then maybe they need to be updated. Does that make sense? What I'm not sure about is if we should do nothing with IPv6 ports or bind (to claim the port) but not listen.
/assign michaelbeaumont


Question: incoming <- &copied } <del> var ( watchChannelHWM HighWaterMark ) </del> // translate pulls stuff from etcd, converts, and pushes out the outgoing channel. Meant to be // called as a goroutine. func (w *etcdWatcher) translate() {

https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite} Previous issues for this test:
This is pretty much the same failure, that are happening in "gce-enormous" suite and we currently debugging them with The main reason of those failures is that we are terminating all watchers for a particular resource type from time to time, and some of them are actually causing a snowball effect. After more looking into it, my hypothesis is that our machinery for aggregating watchers in apiserver sometimes is just to slow. In particular, I some log in my local branch and what I'm observing is that in etcd_watcher, we are: accumulating objects in incoming channel but also accumulating objects in outgoing channel, and the accumulation there is usually higher [I will send a PR to put those logs into master - they seem to be pretty useful for debugging such issues.] So my current hypothesis is that dispatching watch events to watchers is what is too slow. We will continue debugging.
The logs that I seem to confirm my hypothesis - outgoing channel is generally more full then the incoming channel in etcd_watcher: vs I'm working on the PR that should at least improve the situation.
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
[FLAKE-PING] This flaky-test issue would love to have more attention.

https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
Please apply the label to this issue if it does not need to be fixed before we launch 1.4. Otherwise please advise on how to proceed with respect to 1.4.
This was a regression - I fixed it by reverting an offending PR: This seems to be fixed now.
I have 33 pods in my container (1 is substantive, the other 32 just run on log files). I get this all the time now. I'm running 1.5.4.
- to be honest, I don't undertsand what your wrote. what is "container" in "33 pods in my container"? Also, if you believe there is a bug somewhere, please open a new issue instead of adding comment to more than half-year-old one.


Question: kubectl config delete-context minikube`) ) <del> func NewCmdConfigDeleteContext(out io.Writer, configAccess clientcmd.ConfigAccess) *cobra.Command { </del> <ins> func NewCmdConfigDeleteContext(out, errOut io.Writer, configAccess clientcmd.ConfigAccess) *cobra.Command { </ins> cmd := &cobra.Command{ Use:     "delete-context NAME", Short:   i18n.T("Delete the specified context from the kubeconfig"), Long:    "Delete the specified context from the kubeconfig", Example: delete_context_example, Run: func(cmd *cobra.Command, args []string) { <del> err := runDeleteContext(out, configAccess, cmd) </del> <ins> err := runDeleteContext(out, errOut, configAccess, cmd) </ins> cmdutil.CheckErr(err) }, }

<!-- Thanks for filing an issue! Before hitting the button, please answer these questions.--Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see ): No, this is a bug report. What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.): Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG REPORT <!-- If this is a BUG REPORT, please: Fill in as much of the template below as you can. If you leave out information, we can't help you as well. If this is a FEATURE REQUEST, please: Describe in detail the feature/behavior/change you'd like to see. In both cases, be ready for followup questions, and please respond in a timely manner. If we can't reproduce a bug or think a feature already exists, we might close your issue. If we're wrong, PLEASE feel free to reopen it and explain why. --Kubernetes version (use ): Environment: Cloud provider or hardware configuration: MacBook Pro Mid-2015 OS (e.g. from /etc/os-release): macOS Kernel (e.g. ): Install tools: Others: N/A What happened: When deleting the context that was my current context, I entered a state where I am seemingly unable to set my context. The shows none as the current: Also, when trying to take any actions that require a context the following error is presented: When viewing the full config () I can see that my selected context is still the one I deleted. What you expected to happen: I believe there are two expectations I had: should allow me to delete my current context, but not enter an inconsistent state in the process -or- should either provide a warning or disallow removing your current context How to reproduce it (as minimally and precisely as possible): I believe that you just need a configuration with multiple contexts. I had three before deleting the context that has me unable to set a new context.

list all contexts to find your current context () delete your current context list your current contexts again, and you'll see no contexts are created try to set your context with one of the ones that exist () listing your contexts again will show none being set Anything else we need to know: 3 contexts were configured before starting to delete one 2 of them were GKE clusters, automatically set up by the binary 1 is a context set up by the context I deleted was one of the contexts The issue was resolved by manually setting my context by editing .
I think this is the confusion of doesn't switch contexts, it lets you set fields on a context. is what switches contexts.
It's technically valid to have no current context, but I agree a warning message like "warning: this removed your active context, use to select a different one" would go a long way.
SGTM.


Question: # Make sure docker is installed gcutil ssh ${MINION_NAMES[$i]} which docker > /dev/null if [ "$?" != "0" ]; then <del> echo "Docker failed to install on ${MINION_NAMES[$i]}. Your cluster is unlikely to work correctly." echo "Please run ./cluster/kube-down.sh and re-create the cluster. (sorry!)" </del> <ins> echo "Docker failed to install on ${MINION_NAMES[$i]}. Your cluster is unlikely to work correctly." 1>&2 echo "Please run ./cluster/kube-down.sh and re-create the cluster. (sorry!)" 1>&2 </ins> exit 1 fi done

I'm following the documentation on this doc: Everything works until this command: I get this error consistently from different VMs (a Debian and a CoreOS). Maybe I have to edit /etc/hosts to add the "external-ip" with the public IP of some server? Docs don't seem clear. Thanks, Riccardo
Its failing to look up your master correctly. You can try setting KUBERNETES_MASTER=http://<your IP hearBrendan On Sep 21, 2014 5:45 AM, "Riccardo Carlesso" wrote:


Question: } else { e2epod.ImagePrePullList.Insert(gpuDevicePluginImage) } <del> if samplePluginImage, err := getContainerImageFromE2ETestDaemonset(SampleDevicePluginDSYAML); err != nil { </del> <ins> if samplePluginImage, err := getSampleDevicePluginImage(); err != nil { </ins> klog.Errorln(err) } else { e2epod.ImagePrePullList.Insert(samplePluginImage) } <del> if samplePluginImageCtrlReg, err := getContainerImageFromE2ETestDaemonset(SampleDevicePluginControlRegistrationDSYAML); err != nil { klog.Errorln(err) } else { e2epod.ImagePrePullList.Insert(samplePluginImageCtrlReg) } </del> } func getNodeProblemDetectorImage() string {

Failing: ` Failing since . Last passing: First failing:
The diff () shows that went in which looks related
/sig node
/kind regression /kind bug /triage accepted


Question: } } <del> func TestWaitingForGatesCondition(t *testing.T) { </del> <ins> func TestSchedulingGatedCondition(t *testing.T) { </ins> tests := []struct { name           string pod            *api.Pod

<img width="844" alt="image" src="" <img width="942" alt="image" src="" Here are some incorrect comments and function name about SchedulingGated Comments and function names need to be changed to SchedulingGated nothing No response <details</details<details</details<details</details<details</details<details</details<details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign
/sig scheduling
/remove-kind bug /kind cleanup


Question: batchv1 "k8s.io/api/batch/v1" corev1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" <ins> "k8s.io/apimachinery/pkg/util/runtime" </ins> "k8s.io/apimachinery/pkg/util/wait" "k8s.io/client-go/informers" clientset "k8s.io/client-go/kubernetes"

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --This started happening when we upgraded to Kubernetes 1.21. The following message occurs: The Cronjob is pretty simple: No error. But in fact, it does appear jobs ARE being removed properly regardless of the error message. Use the above. We tried deleting and recreating the cronjob both with and and neither seems to work. Note the API version was at batch/v1beta1 before we upgraded, and we did change it to batch/v1 to squelch the deprecation notice. Kubernetes version (use ):Cloud provider or hardware configuration: AWS OS (e.g: ): From nodes (master is identical):Kernel (e.g. ): Install tools: Kops Network plugin and version (if this is a network-related bug): Others:
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig apps
i think it's due to the upgrade of your cluster. The error means resource version can not match? what's your cluster version before? your new created cronjob is working well, right? like below:
The previous cluster version was 1.18, and yes I know we did a bad thing by skipping two minor releases. The newly created cronjob is working fine. And strangely, there are no old jobs that spawned - except for the three most recently run, all of them obviously created in the new cluster (we use the defaults here). So jobs ARE getting cleaned up. That's what makes the error messages baffling.
is set to default 3, is set to default 1. see for details.
It's a bug of controllerv2 for cronjob. It passes a cronjob with old resourceVersion to update its status when cleaning up jobs.

The CronJobControllerV2 feature flag graduates to beta and set to be enabled by default in v1.21. So , this started happening when you upgraded to Kubernetes 1.21. I will try to submit a PR to fix it. /assign
We are also seeing this error message in our cloudwatch logs. Some of our cronjobs say that there's an active job running, yet the job referenced has already been completed. Could that be a result of this bug? For example: As you can see it successfully completed. Yet the cronjob lists it as active: Relevant Cloudwatch logs:
seeing the same thing on GKE 1.21.5-gke.1302
hi! any news on that PR? it seems there is a need for rebase
The whole thing with JobAlreadyActive is that it prevents next schedules of a Job, since we have as our main mechanism to prevent overlapping.
Thanks for reminding me. I've updated the PR.
can you please confirm, that this bug is what causing CronJobs to stuck on JobAlreadyActive reason?
I‘m not sure whether it is also a result of this bug since I didn't reproduce that as you described. But according to the code and logs, they seem to be related. Maybe you can retest it when this fixed.
OK, I'll try to take a closer look at it recently.
we have the same thing with Forbid we use helm templates to deploy using Spinnaker so everytime we recreate cronjobs they have the old . I think they automatically get modified when cronjob objects are created to upgrade to I didn't know that objects were automatically modified when they have the old apiVersion. I noticed Ingress objects were also modified. so I'm updating our helm charts to fix this
/cc this is related to cronjob v2 controller
I wrote a bash script to clean up the jobs affected by this bug it looks for active cronjobs then their active jobs checks the actual status of the jobs deletes the jobs that are complete
the fix is merged and backported to 1.21; are you able to use the latest version to test the fix? It is available in 1.21.9, 1.22.6 and 1.23.3
thanks for confirming the patch versions, I wasn't sure what patch version it would come in.

we're on the latest version of EKS (1.21.5). we'll need to wait for it to come to EKS


Question: updateSessionAffinity := func(svc *v1.Service) { svc.Spec.SessionAffinity = v1.ServiceAffinityClientIP } <del> framework.UpdateService(f.ClientSet, config.NodePortService.Namespace, config.NodePortService.Name, updateSessionAffinity) firstEndpoints, err := config.GetEndpointsFromTestContainer("http", config.ClusterIP, framework.ClusterHttpPort, config.MaxTries, 0) </del> <ins> _, err := framework.UpdateService(f.ClientSet, config.NodePortService.Namespace, config.NodePortService.Name, updateSessionAffinity) </ins> if err != nil { <del> framework.Failf("Unable to get endpoints from test containers: %v", err) </del> <ins> framework.Failf("Failed to update service session affinity, error: %v", err) </ins> } <ins> // Fetch first endpoints when visiting service firstEndpoints, err := config.GetEndpointsFromTestContainer("http", config.ClusterIP, framework.ClusterHttpPort, config.MaxTries) if err != nil { framework.Failf("Unable to get endpoints from test container: %v", err) } // Check if first endpoints are equal to endpoints which are fetched later </ins> for i := 0; i < framework.SessionAffinityChecks; i++ { <del> eps, err := config.GetEndpointsFromTestContainer("http", config.ClusterIP, framework.ClusterHttpPort, config.MaxTries, 0) </del> <ins> eps, err := config.GetEndpointsFromTestContainer("http", config.ClusterIP, framework.ClusterHttpPort, config.MaxTries) </ins> if err != nil { <del> framework.Failf("Unable to get endpoints from test containers: %v", err) </del> <ins> framework.Failf("Unable to get endpoints from test container: %v", err) </ins> } if !eps.Equal(firstEndpoints) { framework.Failf("Expect endpoints: %v, got: %v", firstEndpoints, eps)

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug /priority failing-test /sig network What happened: From https://k8s-, seems to be broken right after they were by Example error message: and Anything else we need to know?: Is it as expected?
Oct 11 16:37:49.106: test session affinity, cost time: 1m31.742990485s This is a mistake. Sorry for my debug lines sneak in. I was just wondering why this test case doesn't run in upstream CI, so I this debug line. Now, I see because this test case only runs in . Oct 11 16:39:42.406: Expect endpoints: map[], got: map[netserver-1:{}] This check failed unexpectedly, let me try to find out why. I am working on it and will let you know soon when I have some updates.
Thanks. These two tests don't run on pull jobs as they are marked as , though https://k8s-, https://k8s- (and other CIs that include ) would have them running.
Do you mean when I raise a PR, these test cases will not be triggered? My question is how can I make sure my follow-up PR can fix this bug? Where can I access the log message when I raise a new PR?
Can I see the updated test results via directly visiting https://k8s-, https://k8s-
Yep, that is the current behavior. A hacky way to make it run on PR --- add a temporary commit to remove the word from test name so some pull jobs ( I think?) will run it. And you could view logs by clicking the button.
Nope, this CI only includes changes that are merged. It is basically a cronjob that runs tests against head.


Question: func startKubelet(k kubelet.Bootstrap, podCfg *config.PodConfig, kubeCfg *kubeletconfiginternal.KubeletConfiguration, kubeDeps *kubelet.Dependencies, enableCAdvisorJSONEndpoints, enableServer bool) { // start the kubelet <del> go wait.Until(func() { k.Run(podCfg.Updates()) }, 0, wait.NeverStop) </del> <ins> go k.Run(podCfg.Updates()) </ins> // start the kubelet server if enableServer {

If the kubelet Run method panics (e.g. ) Run gets called multiple times: That would start multiple goroutines calling relist() concurrently From a PR that doesn't touch the PLEG (but might stress it?), panic in kubelet from pleg:
/sig node /priority important-soon
Could be related to but not seeing how yet
I don't see how this map could have escaped the PLEG unless it was started twice. I see no signs of that. This appears to be code that should never fail this way if the go runtime is correct (I don't see the pod record object escaping this type, and I see no public interface that touches a field outside of an atomic accessor).
Note nothing in this package has changed since last year in July (all the metrics stuff should already be thread safe)
So far appears to be able to consistently reproduce it (because we improve the efficiency of some failure modes and change some loops), but I don't see any other kind jobs failing on it. So this PR exacerbates (but does not cause?) the issue, potentially.
now no longer causes this after a rebase. I am suspicious that this may have been something related to how the code was built in go (environment, optimizations, strange merge) but I don't have any evidence to support that. Will keep watching.
I think different relist period which happens every second may trigger race of . What do you think ? see
Looking at the logs there are 3 fatal errors that occur: , (2x), and . All are caused by concurrent calls.
these concurrent operations happens in different place, I wonder two goroutines use podRecords at the same time (after all 1 second relist period is too short).
I think the kubelet is panicing somewhere else (e.g. ) and Run is getting called multiple times That would start multiple goroutines calling relist() concurrently
it looks like the kubelet does not exit on panics (ReallyCrashForTesting defaults false):
The first panic seen in that kubelet log is:
ah I see. For the record, it does look like the kubelet crashes on the s. kubelet pid changes after each one. But it makes sense if the kubelet isn't fully crashing on the first panic, leading to concurrent runs of the , thus leading to the eventual fatal error.

Yeah.... I think that is the real issue
the panic makes another Run loop start, which sets up a concurrent map read/write, which actually causes the process exit :-)
retitled and bumped priority
We were just talking about this in sig-node yesterday; changing the kubelet to crash on panic. This is definitely evidence that we should do that. I didn't realize this was possible with the panic recovery. /cc
So there are two issues here: was a bug in the PR under test, fixed before merge the kubelet to exit on panic, not live on in zombie state
looks like the particular panic was an actual bug in that PR (ContainerStatuses indexed in a range of InitContainerStatuses): the bug was
Should we change the default for ReallyCrashForTesting to true ? In that case, the flag doesn't need to exist.
Options: . Run() exiting is never expected, and calling it again is never correct. ReallyCrashForTesting completely and leave the panic handler at its default behavior of exiting. This would affect all uses of HandleCrash, which at first glance, looks like these for the kubelet:
Working on a patch for option now.
Yes we talked about making kubelet actually crash in the sig meeting yesterday. I think we had this consensus back in , but kubelet wasn't changed because of the . Yesterday I think we agreed it's good to make kubelet crash for debugging purpose. One concern was if there was a panic caused by plugin / container runtime, kubelet could end up in a crashlooping state. For that concern, we talked about every cluster setup should have some monitoring around how many times / how frequent kubelet restarted. systemd service has a restart counter. I haven't dig into whether it has the restart interval / frequency information I'm working on a PR to remove the ReallyCrashForTesting flag. I think we should do that in 1.19 and have release notes about the behavior change and monitoring suggestions.
We should never wait.Until things that don't tolerate it. So removing the wait.Until is appropriate.
I would recommend removing the wait.Until around the Run() invocation in 1.15-1.18, since that is known to be wrong, and consider the removal of ReallyCrashForTesting in 1.19
Agreed, that is the safest approach.


Question: } return highestSupportedVersion, nil } <ins> // waitForAPIServerForever waits forever to get a CSINode instance as a proxy // for a healthy APIServer func waitForAPIServerForever(client clientset.Interface, nodeName types.NodeName) error { var lastErr error err := wait.PollImmediateInfinite(time.Second, func() (bool, error) { // Get a CSINode from API server to make sure 1) kubelet can reach API server // and 2) it has enough permissions. Kubelet may have restricted permissions // when it's bootstrapping TLS. // https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/ _, lastErr = client.StorageV1().CSINodes().Get(context.TODO(), string(nodeName), meta.GetOptions{}) if lastErr == nil || apierrors.IsNotFound(lastErr) { // API server contacted return true, nil } klog.V(2).Infof("Failed to contact API server when waiting for CSINode publishing: %s", lastErr) return false, nil }) if err != nil { // In theory this is unreachable, but just in case: return fmt.Errorf("%v: %v", err, lastErr) } return nil } </ins>

What happened: Upgraded from 1.16.6 to 1.17.2 and will no longer start on control-plane hosts. It fails with: What you expected to happen: to start, static pods to launch, to connect to in the static pod How to reproduce it (as minimally and precisely as possible): Have the , , , etc in the static pod manifest folder. Try to start the with the container images not preloaded on the host. Just in case it is needed: is used as the backend is run using Anything else we need to know?: This looks to be two things interacting poorly: 1- It takes a long time for static pods to be created for the first time when the can't talk to yet. 2- The new CSI migration code force terminates causing these log lines to appear in : It looks like introduced this new fatal error. Environment: Kubernetes version (use ): 1.17.2 Cloud provider or hardware configuration: Bare-metal OS (e.g: ): Fedora 31..1 (CoreOS preview)" Kernel (e.g. ): 5.3.12-300.fc31.x86_64 Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig cluster-lifecycle
Actually, probably: /sig node
The referenced PR was in 1.14, so that interaction is not new. What is likely new is the size increase of hyperkube in 1.17 (xref ). When not pre-pulled, this can increase the time it takes to start hyperkube for the first time. Does the image pull make progress as the kubelet restarts/retries?
It might make sense for the initializeCSINode() method to: first wait to successfully talk to the kube apiserver at all, perhaps indefinitely after successfully talking to the kube apiserver, then try to register the CSINode object with backoff, and fail if it cannot do that successfully
/sig storage /remove-sig cluster-lifecycle
It doesn't seem to. Once the connection to containerd is closed, it cancels the context. Otherwise I would expect this to succeed eventually if the pull happened in the background. That sounds reasonable to me assuming it isn't blocking the static pod creation code. Do you happen to know why it takes a very long time for the static pod creation code to be called?

I would expect the pods to be created very soon after the kubelet is started.
Pulling the image and starting the processes for those pods happens immediately (from the containerd logs, you can see the image pull happening... it just doesn't complete in time before the independent kubelet logic to register the CSI node info times out).
Ah, yes. I just upgraded a couple of days ago so I didn't check whether I still saw the 2 minute delay in creating the "pod sandboxes" (this is before the pull even happens). It now is only a 23 second delay: containerd: kubelet:
ack, I like the idea Will work on a fix today and cherry-pick it as necessary


Question: // getListener creates a listener on the interface targeted by the given hostname on the given port with // the given protocol. protocol is in net.Listen style which basically admits values like tcp, tcp4, tcp6 func (pf *PortForwarder) getListener(protocol string, hostname string, port *ForwardedPort) (net.Listener, error) { <del> listener, err := net.Listen(protocol, fmt.Sprintf("%s:%d", hostname, port.Local)) </del> <ins> listener, err := net.Listen(protocol, net.JoinHostPort(hostname, strconv.Itoa(int(port.Local)))) </ins> if err != nil { return nil, fmt.Errorf("Unable to create listener: Error %s", err) }

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: BUG REPORT What happened: When using IPv6 addresses, the kubelet passes the address to client-go, which does not correctly format the address for use in API requests. For example: Failed to list v1.Pod: Get https://2001:129::136:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dnode136&resourceVersi on=0: dial tcp: too many colons in address 2001:129::136:6443 What you expected to happen: The URI should have https://[2001:129::136]:6443, so that SplitHostPort() will correctly be able to separate the host and port components. How to reproduce it (as minimally and precisely as possible): Create cluster using IPv6 address (e.g. kubeadm init --apiserver-advertise-address=2001:129::136). When the kubelet starts up, it'll log this error. Anything else we need to know?: This is being handled as part of effort to support IPv6 in Kubernetes. Was found by a co-worker, I'll work on a commit to fix. Environment: Kubernetes version (use ): Using latest, via KubeAdm Cloud provider or hardware configuration*: bare-metal OS (e.g. from /etc/os-release): Ubuntu 16.04 Kernel (e.g. ): 4.4.0-81-generic Install tools: Using KubeAdm Others:
/area client-libraries /area kubelet /area ipv6
/assign
The point of failure is here: SplitHostPort will accept IPv4 and IPv6 addresses, but the latter must have square brackets around the IPv6 address.
There are no sig labels on this issue. Please by:<br(1) mentioning a sig: <bre.g., for API Machinery<br(2) specifying the label manually: <bre.g., for sig/scalability<br<brNote: method (1) will trigger a notification to the team. You can find the team list and label list
/sig api-machinery because client-go
Should we open this on the repository?

I see in the client-go repo read me that it states: Does that confirm that I should do a PR here?
One thing that I was wonder, was whether or not the hostname/address provided in the call to getListener, has been validated. If so, and it can only be a hostname, IPv4, or IPv6 address as a string, we could just check if there is a ":" in the name to determine IPv6. Otherwise, I could use net.ParseIP(), and net.To4() to determine if IPv6. I'll start working with the latter assumption, but please let me know, if the former could be used.
Also, I'm making the assumption that we'd want to make client-go be able to handle IPv6 addresses, rather than having each user of client-go (e.g. kubelet) from ensuring that the IP address is of the right form. IOW, have client-go add the needed [] around IPv6 addresses, rather than have kubelet do that. My thought is that there may be cases where we want just the IP address and not the modified IP. Does that sound OK?


Question: latenciesSummary *metrics.SummaryVec } <del> func newMetricSet(name string, labels []string, helpTemplate string, hasSummary bool) *metricSet { var summary *metrics.SummaryVec if hasSummary { summary = metrics.NewSummaryVec( &metrics.SummaryOpts{ Namespace:      namespace, Subsystem:      subsystem, Name:           fmt.Sprintf("%s_admission_duration_seconds_summary", name), Help:           fmt.Sprintf(helpTemplate, "latency summary in seconds"), MaxAge:         latencySummaryMaxAge, StabilityLevel: metrics.ALPHA, }, labels, ) } return &metricSet{ latencies: metrics.NewHistogramVec( &metrics.HistogramOpts{ Namespace:      namespace, Subsystem:      subsystem, Name:           fmt.Sprintf("%s_admission_duration_seconds", name), Help:           fmt.Sprintf(helpTemplate, "latency histogram in seconds"), Buckets:        latencyBuckets, StabilityLevel: metrics.ALPHA, }, labels, ), latenciesSummary: summary, } } </del> // MustRegister registers all the prometheus metrics in the metricSet. func (m *metricSet) mustRegister() { legacyregistry.MustRegister(m.latencies)

I would like to propose the graduation of the following apiserver metrics from alpha to stable: With the exception of the metric these metrics have been stable for 2 years. The metric has been stable for 12 months.
/sig api-machinery /sig instrumentation
/assign /triage accepted
I'm okay with promoting all of these except I am hesitant about since it is relatively new. Can we omit that one?
I agree with the metric was patched a year ago as part of , so it would be best to wait a bit more until the majority of Kubernetes users catch up to the patched version and start using the new version of the metric in production.
A couple of renaming suggestions, since it is important to get it right before graduation: apiserverresponsesizes =apiserverresponsesizebytes aggregatorunavailableapiservice =aggregatorunavailableapiservices One concern regarding . It was very recently changed but the change was later reverted by In the end, the metric wasn't really changed, but it might raise the concern that some users may still need additional information from this metric so I am on the fence with this one. Side implementation note that we will need to keep in mind if we graduate is that the graduation should only affect the webhook admission metric and not the other, so we will need to pass the stability level to
Thank you and for the feedback. To clarify then, are the decisions to not graduate and final?
I'm actually okay with , the namespace addition was recent but it was a bad idea and was reverted. It has otherwise been in use for quite some time, so I'd be comfortable with promoting it.
I'm willing to make a PR for this one
+1 it wasn't a hard no from my side, just wanted to raise some concerns in case anyone had an objection.
/assign /unassign
Thank you for the feedback, and To summarize, we are in agreement to graduate all but the metric. Please correct me if I'm wrong.
/cc
since all the three metrics that are created with this function are graduating to stable, do you think I should still pass the to the function or should I just set the metrics created by this functions as stable by default? update: Seems like is designed to parse whatever's in front of . So putting a variable there (that is passed to the function) will cause the script to fail. Please correct me if I'm wrong.

update: Even setting the metrics to be stable by default will cause the script to fail and you'll get an error saying for the following line: It appears that the name of the stable metrics should be given to that object as a string and using the output of a function ( in this case) will cause the script to fail.
The PR is ready to be reviewed:


Question: }, errorType: "FieldValueRequired", }, <ins> "Address is link-local": { endpoints: api.Endpoints{ ObjectMeta: api.ObjectMeta{Name: "mysvc", Namespace: "namespace"}, Subsets: []api.EndpointSubset{ { Addresses: []api.EndpointAddress{{IP: "169.254.169.254"}}, Ports:     []api.EndpointPort{{Name: "p", Port: 93, Protocol: "TCP"}}, }, }, }, errorType:   "FieldValueInvalid", errorDetail: "link-local", }, </ins> } for k, v := range errorCases { <del> if errs := ValidateEndpoints(&v.endpoints); len(errs) == 0 || errs[0].(*errors.ValidationError).Type != v.errorType || errs[0].(*errors.ValidationError).Detail != v.errorDetail { </del> <ins> if errs := ValidateEndpoints(&v.endpoints); len(errs) == 0 || errs[0].(*errors.ValidationError).Type != v.errorType || !strings.Contains(errs[0].(*errors.ValidationError).Detail, v.errorDetail) { </ins> t.Errorf("Expected error type %s with detail %s for %s, got %v", v.errorType, v.errorDetail, k, errs) } }

Reported by It's possible to turn the master into a proxy to localhost or the GCE metadata server by adding a new endpoint that points to the service. To reproduce, add a new endpoint by posting to http://apiserver/api/v1beta3/endpoints with a description like: If you then go to http://apiserver/api/v1beta3/proxy/namespaces/default/services/myendpoint/ you can interact with the master's metadata server. We can't validate that Endpoints are within the cluster CIDR because we deliberately allow importing of external addresses -- that was the original purpose of allowing users to post Endpoints. I wouldn't disallow link-local and localhost addresses in Endpoints, either, since that might be useful in the future. I'd prefer to just disallow proxying to such addresses. Obviously we'd need to resolve DNS names before checking. See also cc
I think it would be simpler to disallow 169.254.0.0/16 as endpoints, document why and move on. The alternatives are more complicated and difficult to internalize and debug. , Robert Bailey wrote:
The user would either get an error when trying to post the Endpoints, or when trying to access the proxy. The latter is unlikely, IMO, except in malicious circumstances. The former would prevent using services to do something like publish a local metadata service (say, similar to the ) on a link-local address. But I'm ok with that approach for now, since it seems easy to change later.
Yeah, we can open it up as real requirements emerge, was my feeling. , Brian Grant wrote:
I agree w/ fix. Also we should block any similar range that AWS uses, etc.
Or resolve "metadata" and block just that.


Question: } var printer kubecfg.ResourcePrinter <del> if *json { </del> <ins> switch { case *json: </ins> printer = &kubecfg.IdentityPrinter{} <del> } else if *yaml { </del> <ins> case *yaml: </ins> printer = &kubecfg.YAMLPrinter{} <del> } else { </del> <ins> case len(*templateFile) > 0 || len(*templateStr) > 0: var data []byte if len(*templateFile) > 0 { var err error data, err = ioutil.ReadFile(*templateFile) if err != nil { glog.Fatalf("Error reading template %s, %vn", *templateFile, err) return false } } else { data = []byte(*templateStr) } tmpl, err := template.New("output").Parse(string(data)) if err != nil { glog.Fatalf("Error parsing template %s, %vn", string(data), err) return false } printer = &kubecfg.TemplatePrinter{ Template: tmpl, } default: </ins> printer = &kubecfg.HumanReadablePrinter{} }

To remove dependence on jq and generally make scripting much easier, we should make another output flag which takes a formatter. We can use go's template package, which is very flexible.


Question: .:53 { errors health <ins> ready </ins> kubernetes {{ pillar['dns_domain'] }} in-addr.arpa ip6.arpa { pods insecure <del> upstream </del> fallthrough in-addr.arpa ip6.arpa ttl 30 }

Which jobs are failing: Multiple jobs on . Which test(s) are failing: on the following jobs: Since when has it been failing: Since 5/17. Testgrid links: See above. Reason for failure: All the failures mentioned above seem to have to have the same root cause, namely failure while waiting for CoreDNS to be ready. Example: /sig testing /priority critical-urgent /kind failing test /milestone v1.15 /cc
Some of this (possible all of it) is due to , which is dependent on being merged.
/sig cluster-lifecycle
Not the gce jobs thoght. Only the kinder job. So the gce jobs are failing with a problem outside of kubeadm/kinder but possibly still related to the corefile plugin change
/assign (At kubecon this week)
The gce failures appear to show the same behavior, but with kube-up instead of kubeadm. I.e. it appears that kube-up may also be retaining old configs when upgrading/downgrading. If thats true, then we may need to integrate the migration tools into kube-up as well.
I have opened which aims to fix all the failing tests except , which will be fixed via
Great job so far and Would it be possible to ask for the open PRs to be merged and/or a solution found soon? We have these tests failing for quite some time and code freeze coming up in the next days. Thank you in advance for your efforts!
thanks for the heads up. let me know if you don't have the time for the refactor that i requested and i can try to take over there. the alternative is to rollback the coredns version which i assume is not ideal, due to a variety of fixes in the new version.
is there any more work planned for this issue? I see that some of the kinder jobs are still failing after merged:
merging and will resolve and close this issue.
Thank you for the update !
updates: kinder tests were fixed via
/unassign i was here for: it's a bit flaky (which is due to the tooling), but green.
Since and we're moved to 1.16, and we're waiting for to merge, can we move this issue to 1.16?
? is that the right PR?

Good catch, I meant
The GCE test failures were fixed by and /close
Closing this issue. <detailsIn response to : The GCE test failures were fixed by and /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: f.eventRecorder.Eventf(p, nil, v1.EventTypeWarning, podWaitingReason, "", "") }() <del> return framework.NewStatus(framework.Wait), 100 * time.Second </del> <ins> return framework.NewStatus(framework.Wait), permitTimeout </ins> } var _ framework.PermitPlugin = &fakePermitPlugin{}

/kind failing-test /kind flake First seen in an unrelated PR, but also occurs in periodic jobs. /sig scheduling
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign This feature is reverted once, and I fixed the flake, I'm cherry-pick that back again.


Question: return nil } <del> if lbaas.opts.FloatingNetworkId != "" && loadbalancer != nil { </del> <ins> if loadbalancer != nil && loadbalancer.VipPortID != "" { </ins> portID := loadbalancer.VipPortID floatingIP, err := getFloatingIPByPortID(lbaas.network, portID) if err != nil && err != ErrNotFound {

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: After deleting loadbalancer service, openstack cloud provider don't delete floatingip. What you expected to happen: After deleting loadbalancer service, openstack cloud provider delete floatingip. Anything else we need to know?: If ServiceAnnotationLoadBalancerFloatingNetworkId is specified to 'x.x.x.x' and the 'floating-network-id' is empty, openstack cloud provider don't delete floatingip. Environment: Kubernetes version (use ):latest Cloud provider or hardware configuration**: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others:
/sig openstack
There are no sig labels on this issue. Please by: a sig: e.g., to notify the contributor experience sig, OR the label manually: e.g., to apply the label Note: Method 1 will trigger an email to the group. You can find the group list and label list . The in the method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals


Question: // Close stopCh to force checking if informers are synced now. close(stopCh) <del> var informersByStarted map[bool][]string for informerType, started := range i.sharedInformerFactory.WaitForCacheSync(stopCh) { </del> <ins> informersByStarted := make(map[bool][]string) for informerType, started := range i.cacheSyncWaiter.WaitForCacheSync(stopCh) { </ins> informersByStarted[started] = append(informersByStarted[started], informerType.String()) }

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: for extension API server panics with: Apparently introduces this panic with . What you expected to happen: to do not panic. How to reproduce it (as minimally and precisely as possible): extension API server with - for example the endpoint that the apiserver panics with Anything else we need to know?: Environment: Kubernetes version (use ): / Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/kind regression
/sig api-machinery
/assign
cc
Oops - sorry for that. It seems it is basically this comment: that I blindly applied without e2e retesting (the problem seem to be lack of tests, which is a bit surprising because bugs in my previous implementations were caught by tests). ^^ BTW - this has been cherrypicked to previous releases: 16: 17: 18: so it's actually even more critical...
I can file the automated cherry-pick PRs once the PR against the master branch is merged. :)
Reopening - since we need to cherrypick it still.
I now opened the cherry-picks: 16 - 17 - 18 -
Fix and picks are merged


Question: }, } _, _, w, done := NewIndexerInformerWatcher(lw, &corev1.Secret{}) <ins> defer w.Stop() </ins> // Expect secret add select {

In , when timeout or other error occurs, is called. This will make current routine terminated and miss the , which will lead to some leak routines. For example, this routine will block at waiting for No leak make any of these errors happen No response <details</details<details</details<details</details<details</details<details</details<details</details
sig Testing
I would like to take this up!
Actually, I have a fix in , but its fine if you have a better solution.
/sig api-machinery
/assign /triage accepted


Question: }, }, { <del> name: "ordinals.start positive value " + enableStatefulSetStartOrdinal, </del> <ins> name: "ordinals.start positive value", </ins> set: apps.StatefulSet{ ObjectMeta: metav1.ObjectMeta{Name: "abc-123", Namespace: metav1.NamespaceDefault}, Spec: apps.StatefulSetSpec{

This applies to StatefulSetStartOrdinal feature gate. Field validation should be done regardless of the feature gate state. From
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign
/sig apps


Question: // When these values are updated, also update test/e2e/framework/util.go defaultPodInfraContainerImageName    = "gcr.io/google_containers/pause" defaultPodInfraContainerImageVersion = "3.0" <ins> // Auto detect cloud provider. AutoDetectCloudProvider = "auto-detect" </ins> ) // Returns the arch-specific pause image that kubelet should use as the default

It appears that the kubelet will detect and use the cloud provider as detected by cadvisor thus overriding the cloud provider passed in at runtime: This will break our private cloud configuration where we do not use our cloud provider's load balancing and instead use our own load balancing tier. In my opinion, the cloud provider specified at kubelet runtime should be respected for the node. Am I reading this correctly? Any thoughts on how to handle this properly? Thank you!
If you specify any cloud provider through flags, it will be used by kubelet (see ). It's only when you don't specify any cloud provider that kubelet will detect the cloud provider from the machine info, as described by the comment.
that's exactly my situation, I want to run in our private cloud environment without any of the cloud integration. If I specify cloud-provider="" at runtime it will get overridden.
Ah I see. I guess specifying an empty string is the same as not specifying any cloud provider. This auto-detection feature was introduced by /cc the author and reviewer to see if this is an intended behavior:
Legitimate concern. I'm posting a fix. Do you consider this to be a regression or are you just getting started with k8s.
I am sorry, This was not the original intention. We can keep the auto detection with a flag, or make a flag to ask kubelet not to do it. Unless there is a way to see if the variable was assigned to that I missed. I can post a fix tomorrow if it won't exit by then.
Thank you. We're running on 1.2.4 and won't be running on 1.3 for a few months yet anyways and was just testing to look for future issues when I found this.
I assigned this issue to you since , and I can't assign this to anyway :-)
Thanks for testing our 1.3 release candidate, and reporting this regression to us. We are including the fix to 1.3.


Question: // Executes the rule check without using the "-C" flag, instead parsing iptables-save. // Present for compatibility with <1.4.11 versions of iptables. func (runner *runner) checkRuleWithoutCheck(table Table, chain Chain, args ...string) (bool, error) { <ins> glog.V(1).Infof("running iptables-save -t %s", string(table)) </ins> out, err := runner.exec.Command("iptables-save", "-t", string(table)).CombinedOutput() if err != nil { return false, fmt.Errorf("error checking rule: %v", err)

I am running kubernetes 0.5.2 + and I've noticed that the number of entries in iptables increases every few seconds. I started with a clean install, no pods or replicationControllers, there is now two services by default (kubernetes-ro and kubernetes). I ran and got . Next, I turned up 7 services and 3 replicationControllers which resulted in 3 pods also being turned up. I started a watch... I saw "38", "43, "50, "64", "66", "68", "70", ..., "115" and on until I stopped . It appears the proxy continues to add REDIRECT's for the services, for example the lines that keep being look like: A similar line for each service gets every second or so, it appears to all be in the table. I'm thinking the newly function might be the culprit, since calls and in the log I see every few seconds (looks like does a sync every 5 seconds).
I noticed that the 0.5.3 and 0.5.4 releases don't include the , though it's in master. Perhaps the issue I've filed is already known and I missed it when searching the issues.
I have not heard this issue. Can you paste a listing of iptables-save -t nat when you are seeing this problem? Alternatively, I can cut you a branch with some extra debug instrumentation if you're willing to test it out today...
I'm not able to test it out today. As far as the iptables output, I'd have to type it in (can't copy/paste -- different physical network). However, the output has lines, like the one given in the issue, which are every 5 seconds to the "nat" table. It seems like the check to see if the rule already exists is perhaps not functioning properly. I am working on a fix for (because it uses the iptables -C flag which isn't supported in older iptables), in that fix I created a function to check if an existing rule already exists: Example call: Using may not be ideal, perhaps breaking up the output of the iptables command into a slice and checking each line would be better(?) sure yet.
Fix is pending - would be great if someone could verify it against an actually old iptables (I just faked the version string).

should be able to try it out friday, maybe monday
Feel free to rip off our checks with out echeck function :) On Nov 26, 2014 7:36 PM, "Craig Wickesser" wrote:
that it actually works. On Nov 26, 2014 7:40 PM, "Tim Hockin" wrote:
I fired up kuberenetes 0.5.2 (plus + your changes) and things looked good initially (when starting up kubernetes, no pods/services, etc). The number of entries iptables-save remained constant. I then turned up some services and watched the length of iptables-save increase. I turned down the services and the number of entries in iptables-save continues to increase. At the moment, every 5 seconds it's back to adding two entries (one for the kubernetes service and one for the kubernetes-ro service). With log level set to 0, I see "Opened iptables portal for service.." messages in the proxy log every 5 seconds ().
Are you sure you have the right proxy version running? I could repro the initial problem, but not this - it works just fine. To be clear when you say it increases after creating a service - do you mean it increases by 1 rule or that it increases continually? , Craig Wickesser wrote:
On Monday/Tuesday I'll try the latest from master, perhaps I messed something up. To clarify, the increase was continuous. When I initially setup kubernetes (with a fresh/clean etcd), things looked normal with regards to iptables. I then deployed some pods and services and kept watching iptables-save. The number of entries increased every 5 seconds. I deleted pods, replication controllers and services and the number of entries continued to increase, but only by two (for the kubernetes and kubernetes-ro services) every 5 seconds.
Did you paste the output of iptables-save? Its possible that it is slightly different from my version. This is why parsing text is no fun :( On Nov 28, 2014 3:42 PM, "Craig Wickesser" wrote:
I did not include the output of iptables-save. I'll see if I can get additional output besides the line included in the initial issue.
Just a couple lines so I can see the format.

On Nov 28, 2014 7:32 PM, "Craig Wickesser" wrote:
here's a few lines from : All of the lines follow the exact same format. Looks like the proxier (to fit with the output I provided) would need to look like this:
FWIW - I'm using CentOS 6.5 with iptables version 1.4.7.
I recompiled with changes to proxier as mentioned earlier (changed the order of args) and did not resolve the issue. I continue seeing (in the proxy log):
I've some additional logging to log the and the . I am logging them using these two lines: When I reduce the output between the and the (when it should return "true", that fields is a superset), the result looks like: Notice has quotes around it in the set, and the does not? I think perhaps that's the issue and that's why the check is false.
I just tried changing to: and that's working. Basically, strip out quotes from the line. There might be a better alternative.
Uggh, seriously? Your iptables add quotes around the comment, mine omits the quotes for single tokens. I guess I can force it to not be a single word, something like "service: " service name. What a hack. Any better ideas? , Craig Wickesser wrote:
Hmm, it seems worse than that. If I feed in a comment with quotes, I get ""foo"", so I can't do that to be compatible with your implementation. Can you try something? iptables -t nat -I PREROUTING -d 1.2.3.4 -m comment --comment "foo1" iptables -t nat -I PREROUTING -d 1.2.3.4 -m comment --comment '"foo2"' iptables-save | grep foo -A PREROUTING -d 1.2.3.4/32 -m comment --comment ""foo2"" -A PREROUTING -d 1.2.3.4/32 -m comment --comment foo1 , Tim Hockin wrote:
i'll try that in about 8 hours :)
here's what I got:
ugh, that is SO BROKEN. I'm going to have to do something really unpleasant.

, Craig Wickesser wrote:
I have a potential fix, but airport wifi prevents me from pushing git <shrug. will publish it later.
sounds good, I appreciate the effort.
PR posted - please test before we commit.
I've tried the PR and a comment to it.
got the same problem here using Kubernetes v0.6.0-12-gd927031aecdeb6 on redhat 6.5
It looks like 0.6 was cut before - - can we patch that in, or should we wait of 0.7? , polariss0i wrote:
got the same problem on kubernetes v1.0.3 and iptables-v1.4.7. Should I file a new issue?


Question: configs[PrometheusDummyExporter] = Config{gcRegistry, "prometheus-dummy-exporter", "v0.1.0"} configs[PrometheusToSd] = Config{gcRegistry, "prometheus-to-sd", "v0.5.0"} configs[Redis] = Config{dockerLibraryRegistry, "redis", "5.0.5-alpine"} <del> configs[RegressionIssue74839] = Config{e2eRegistry, "regression-issue-74839-amd64", "1.0"} </del> <ins> configs[RegressionIssue74839] = Config{promoterE2eRegistry, "regression-issue-74839", "1.2"} </ins> configs[ResourceConsumer] = Config{e2eRegistry, "resource-consumer", "1.5"} configs[SdDummyExporter] = Config{gcRegistry, "sd-dummy-exporter", "v0.2.0"} configs[VolumeNFSServer] = Config{e2eVolumeRegistry, "nfs", "1.0"}

<!-- Please only use this template for submitting reports about continuously failing tests or jobs in Kubernetes CI --Which jobs are failing: n/a Which test(s) are failing: "should resolve connection reset issue [Slow]" Since when has it been failing: it never worked Testgrid link: n/a Reason for failure: The image used for the test is using only IPv4 Anything else we need to know: We should: [x] refactor the code to support IPv6 [ ] ~consolidate the image in agnhost~ [x] configure an IPv6 job to test network https://k8s-,%20IPv6,%20master
/assign /sig-network
GitHub didn't allow me to assign the following users: Billy99. Note that only , repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : /assign /sig-network Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig network
/triage unresolved Comment when the issue is assessed and confirmed. I am a bot run by vllry.
/remove-triage unresolved
Those labels are not set on the issue: <detailsIn response to : /remove-triage unresolved Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/help wanted
/help wanted /good-first-issue
/remove-good-first-issue the agnhost promotion process is not something easy for new contributors
Hi, I would like to contribute to this.
good, thanks
sorry , I needed to fix it and I took over, thanks for stepping up /assign /remove help-wanted
OK.Yesterday I have refactor the code to support IPv6, but I understand wrong about consolidate the image in agnhost. I'm sorry for not being able to help.
no worries, it was trickier than I thought, thanks again for your help
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale


Question: "apiVersion": "group/version", "kind":       "TheKindList", "metadata": map[string]interface{}{ <ins> "continue":        "", </ins> "resourceVersion": "", }, },

The fake dynamic client does not support mock pagination or the self link. Add ability to propagate the metadata fields from a mock unstructured list to the served list by the fake lister. Repro linked in the MR x-ref. No response <details</detailsN/A N/A <detailsN/A </details<detailsN/A </details<detailsN/A </details
/sig api-machinery
/cc /triage accepted
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale


Question: // The IP protocol for this port. Supports "TCP", "UDP", and "SCTP". // Default is TCP. <ins> // +default="TCP" </ins> // +optional Protocol Protocol `json:"protocol,omitempty" protobuf:"bytes,2,opt,name=protocol,casttype=Protocol"`

It looks like the openapi is not routed properly to set-up the apiservice so apply requests will fail because the fieldmanager is not set-up properly. /cc /sig api-machinery /priority important-soon /kind bug
/wg apply /cc
OpenAPIModels is set to nil for the kube-aggregator server config (which serves the APIServices API): This is done because the kube-aggregator has a special openapi handler that aggregates all the other server's openapi docs. The field controls (at least) two things: /openapi endpoint serving (which we want kube-aggregator to continue to customize) creation of the field manager (which we want kube-aggregator to behave consistently on) We need to decouple presence of OpenAPIConfig and construction of StaticOpenAPISpec (which is needed by the field manager) from construction/installation of OpenAPIVersionedService. That might be a little tricky to untangle, since construction of StaticOpenAPISpec includes paths/operations. The field manager only needs the gvk/schemas.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/lifecycle frozen This is not something we want closed by accident.
/assign
For reference:
Additional lines for reference: This is roughly how SSA consumes the openapi spec. openapi spec -openapi proto model -typeconverter -SSA field manager For built in types, openapi spec -openapi model openapi model -typeconverter A similar thing happens for CRDs As liggitt mentioned, SSA only requires a subset of the information in the openapi generation, and the openapi generation in the aggregator slightly different than the process for builtins and CRDs.
/unassign
/assign


Question: pkg/kubectl/cmd/top pkg/kubectl/cmd/util pkg/kubectl/cmd/util/openapi <del> pkg/kubectl/cmd/version </del> pkg/kubectl/cmd/wait pkg/kubectl/describe/versioned pkg/kubectl/generate

Anyone itching to understand the code and try fixing things in Kubernetes, please help fixing the lint failures Step 1: Remove one or two packages (related packages) from the list in - (note: do not change generated files, the name of defaulting functions, or the name of conversion functions... packages failing linting due to issues in those files will need to be left alone) Step 2: Run and look at the logged errors Step 3: Fix the code to ensure that runs cleanly. Step 4: Run and to make sure we fix any problems with gofmt or bazel. Step 5: File a PR with the change Step 6: Go to step :) Please use the tracking spreadsheet For everyone who is going to contribute, please append the packages you picked at the end. For existing contributor, please make sure this list is update to date
/sig architecture /sig testing
Hey, I would like to resolve this issue. Could you assign it to me ?
there are a lot of packages so there will be many people working on this. Also we cannot assign issue to folks who are not in the github org for kubernetes. please feel free to pick up a few packages like i mentioned and file a PR. Make sure you link back to this issue, so folks can see which packages you filed PR for.
OK. I see. I will firstly pick up the package called to fix.
Hi dims. Should i submit my PR before i remove one or two packages (related packages) from the list in - Or I first remove the package from list, then I fix the issue and submit a PR?
in your PR, you will have both the fixes to packages and some lines removed from /.golint_failures
When I run , I found some errors as below: I found that there is a go file in But there is still a error.
Say your GOPATH looks like this Your kubernetes directory should be () cd to and then run from that directory.
Is it necessary to obey all rules in golint? Rules in golint are set in stone. Maybe using gometalinter to ignore some specific errors is better?
I revise some erros about golint in some packages. Please review it.
which errors would you like to ignore?

if you can open a new issue with the list of errors to ignore (and proposing gometalinter), we can get some opinions there from others in the community.
Let me take the kubectl cmd part...
Can someone please review and let me know if I am doing it right?
I ll work on all the packages related to
we get 800-ish pkgs left to fix. my concern is that if we fix them package by package, then such pulls will be flooding the repo. can't we possibly make it in larger pulls? the larger the better?
+1 to slightly larger pulls with related packages. if allow too large then the reviewers will not be able to do a thorough review. we should spread the load among multiple reviewers. (if we allow a really large pull, we will end up with top level reviewers who will face the burden and the PR(s) will just stagnate)
oh good point. we could put one pkg's change into one commit, and merge all changes under one area into one pull so that it wont spread the load. wdyt? for each pull, we can have hundred-ish lines of change. but it will still be 30~50 pulls i guess? good news is that we can take our time to deal with it, at least before 1.12
sounds great! please feel free to help make it happen :) (by suggesting groups of packages that kinda go together)
I'm looking at fixing pkg/proxy, and I think I see what was talking about ignoring some rules. proxy.ProxyProvider is being suggested because golint is saying that it stutters. In that example, I think it's okay renaming it to proxy.Provider, but I could see an example where you would want to keep the stutter if the package that it's declared in uses it heavily.
sure thing, feel free to propose a PR for switching to gometalinter independently of other patches. We can work through that there.
I sent the pull request to fix pkg/volume/aws_ebs
I've fixed a few of these, but get the following error when I try to run : Are there any instructions on what I need to setup in order to resolve this?

you can find this file in
I have raised a PR to fix golint errors in pkg/api
I have fixed some pkgs under /pkg/kubelet pkg/kubelet/images pkg/kubelet/kuberuntime pkg/kubelet/leaky pkg/kubelet/pleg
I submitted a pull request for golint errors in pkg/security/ edit: I can work on picking up pkg/controller/bootstrap through pkg/controller/namespace/... (114-129)
fix golint errors in package cmd/kubeadm/app, since the codes under cmd/kubeadm/app/apis/kubeadm/v1alpha2 and cmd/kubeadm/app/apis/kubeadm/v1alpha3 are auto-generated, i keep them untouched.
I am working on some packages under pkg/kubelet, anyone interested in this plz leave messages for me :)
fix golint for pkg/util/dbus pkg/util/labels pkg/util/node pkg/util/strings pkg/util/workqueue/prometheus
PR fixes golint errors in pkg/routes,pkg/securitycontext, pkg/serviceaccount
PR fixes golint in pkg/probe
should we have a checklist as part of the issue summary saying what is in progress and what is done? Since this issue is getting long, this will make new contributors to pick up issues easier.
Great idea!, want to throw a google doc with a check-list so it's easy for participants to organize themselves? (please!) If it is the issue summary then one of us with privileges will be a bottle neck.
Created a with all the package that have on going PRs. Maybe you can add this to the summary. For everyone who is going to contribute, please append the packages you picked at the end. For existing contributor, please make sure this list is update to date
Thanks updated the spread sheet
Raised PR to fix golint issues in: pkg/controller/clusterroleaggregation pkg/controller/cronjob pkg/controller/daemon pkg/controller/deployment pkg/controller/deployment/util pkg/controller/disruption pkg/controller/endpoint
This issue is close in accident by . could you reopen it?
/reopen
Reopening this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I found a problem trying to fix golint messages for .

Golint complaints on 's function because of the snake case. When I try to fix it, fails because it is being used in a code generated file. If I clean generated files, tests pass, but the generated file is not invoking it anymore. I failed to find the generator definition using this so I could change the hook to just . Also I'm not sure if this is wanted. I also found other packages in also probably affected by this. I read somewhere about replacing golint with gometalinter, which would allow to define directives in code to omit certain rules. I would consider to have a second exclusion file for golint with these cases, to be updated if move to gometalinter is finally done. This would make this issue completion more feasible. Meanwhile, I'll move onto fixing golint messages for other packages, until this discussion gets cleared.
I also found a problem fixing . A test fails because the directory is marked as readonly (see tests @ Shall we also ignore this directory for this issue?
Is this issue still relevant because hack/verify- is giving me no errors.
the failures are still present in there are details in the first message on how to claim + solve issues :)
Sorry I misunderstood the issue
I'm working on this package:
pkg/apis/admission/v1beta1 pkg/apis/authentication/v1beta1 pkg/apis/authorization/v1beta1 pkg/apis/coordination/v1 pkg/apis/coordination/v1beta1 pkg/apis/imagepolicy/v1alpha1 pkg/apis/scheduling/v1alpha1 pkg/apis/scheduling/v1beta1 pkg/apis/settings/v1alpha1 pkg/apis/storage/v1alpha1
pkg/cloudprovider/providers/vsphere/vclib pkg/kubelet/apis/deviceplugin/v1alpha pkg/kubelet/apis/pluginregistration/v1alpha1 pkg/kubelet/apis/pluginregistration/v1beta1
:
I'm looking at the pkg/registry/*, all those which are left. I expect to give an update on what all is fixed in the next 24-48 hours.
hi there is a google sheet that is a bit outdated, maybe you can update that?
hi I seem to only have view access for the sheet. Can't edit.
I fixed the permission.

Please try it again
pkg/apis/authentication pkg/apis/coordination pkg/apis/events pkg/apis/imagepolicy pkg/apis/settings
hi I think this issue was closed prematurely by (using fixes #issue in commit)
I'll put in a PR shortly for the package
Looking at next
hi thanks! there is a spreadsheet here that tracks the progress on fixing these issues, consider adding the packages you're work on there so we can keep everything organized :+1:
Starting on now
I did indeed see the Google Sheet link (thank you and to take it a little further
test/e2e
test/e2e_node/builder
pkg/registry/admissionregistration/mutatingwebhookconfiguration/storage pkg/registry/admissionregistration/rest pkg/registry/admissionregistration/validatingwebhookconfiguration/storage pkg/registry/apps/daemonset/storage
Hi. I opened a PR - Please review.
Working on Will open a PR soon.
Starting on and should have a PR soon
I keep getting this golint verify error on and was wondering if others saw it: But when I that package to the golint failures in my PR, my reviewer said it isn't necessary? What does that mean? What did I miss? Should I use a different kind of verify? cf:
looks like the disconnect is because that problem does not show up in the version of we use in the CI. please see
Still confused: is that not the golint version I ran? it installs from vendor and uses that I think?
i installed from and i still don't see the problem, can you please check what version of golint you have?
I'm confused why I wouldn't be using the vendored version? how would that get out of sync?
/me looking
Note: I am reserving for a KubeCon Contributor Workshop demonstration. It is noted in the spreadsheet.
I think may not be ideal cause there's a lot of auto generated files there. What do you think about reserving instead for the KubeCON Contributor Workshop?
pkg/apis/scheduling pkg/apis/storage/util pkg/apis/storage/v1beta1/util pkg/apis/storage/v1/util
/priority backlog
Hi I am a new contributor here, I'd like to contribute and make changes to the following file/s. I have marked it as WIP in the Google Sheet. Can someone let me know if I have to do anything else before I start working on this PR?

Umm so there's an issue where Golint tells that underscore shouldn't be allowed. Here's a log of the go-lint warnings:
Thanks The following should not be modified, as they are designed like this for code generation.
I'd like to pick a package and do my first PR, is there a list of packages to be done? Or should I correlate with this issue thread? Thanks in advance!
please see step 1 () please scan other PRs linked to this issue as well just in case.
Will do. Thank you. On Thu, Oct 3, 2019, 7:25 AM Davanum Srinivas <:
Hi there, I'm looking for a good first issue. I'll start taking a look at if that's ok? There doesn't seem to be anyone currently tracked as looking at it in the attached spreadsheet.
I've found a few others I'm actively looking at, you won't get any argument from me. On Sat, Oct 12, 2019, 10:38 AM Pamela Kelly <:
I am working on the iptables pkg. Updated in the google docs
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I have problem running in step 2 $ hack/verify- what should I do?
PTAL for my pr to this issue, thx
you can move kubernetes directory to and re-run this script.
I'd like to take Can we fix linter issues for generated files? Or we should skip packages with zz_generated?
There're 2 lint errors that affects 292/507 packages, meaning it's not possible to cleanup the entire file without solving it: default pattern of Kubectl is the usage of [cmd]Options for initial options structs and it raises: /apis/{custommetrics,externalmetrics} for example raises the underscore error, but It's probably not worth to break backward compatibility renaming these folder for lint error fix: Any advice on these?

/reopen hm, cannot reopen this issue, it was closed by mistake
You can't reopen an issue/PR unless you authored it or you are a collaborator. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I am sure this is all not fixed. Am I missing something? ./hack/verify- installing golint from vendor Congratulations! All Go source files have been linted.
you need to remove package from first.
/remove-sig testing SIG testing doesn't own code failing lints, we own the linter. :upsidedownface:
Hello, is this still a relevent issue ? I would like to work upon
I have fixed
Hi Folks, Some of the golint fixes, require us to change the exposed function's name, in simple words, "change API". I was not sure, if that is welcomed and what is the process.
Heres what Im getting after going over the following steps as mentioned by Say your GOPATH looks like this GOPATH=/Users/dims/go Your kubernetes directory should be ($) cd to and then run hack/verify- from that directory. I get the following output - installing golint Congratulations! All Go source files have been linted. Im not really sure what I am doing wrong. Any help is appreciated!
I think I understand. I misunderstood the steps around .golint_failures. The first step is to remove the pkg that we are targeting from this file and then run the My mistake was to assume that the packages in the file need to be removed after we are done cleaning the golint errors.
Can someone please review and let me know if I've done this right?
After a couple years of seeing changes spawned from this issue, I don't think this is a good use of contributors' or reviewers' time.

golint is very opinionated about: exported type and function names, leading to PRs attempting to change exported things people use downstream missing documentation on interface implementation functions, which leads to not-very-useful copy/paste of interface godoc the govet and staticcheck verify jobs are much more useful since they catch correctness issues, but I would recommend closing this issue and dropping the golint checks from CI golint doc itself discourages from using its output in an automated enforced way: from
i guess one can say gofmt is opinionated too, but something good about Golang is that it comes with an opinion and a set of standard tools. for API documentation at least we found inconsistencies on how most exported API items are documented: golint can help with that, but if there are no "pragmas" to avoid breaking changes and given we have codegen generating non-compliant names enabling golint on API packages is sort of blocked in Kubernetes. for non-API, non-library, non-public code, i think being inconsistent is mostly OK. no preference on my side. shrug
true, but code formatting has the benefits of mostly eliminating meaningless diffs in PRs due to code movement/whitespace/etc if there are specific packages (like API packages) we lack documentation on, adding checks specifically for that could make sense exactly, I don't think golint is the right tool for making sure we have API docs
golint itself doesn't have these and the go project is against this sort of comment AIUI, however golangci-lint does have them. I think there are plenty of other static analysis lints we should use instead, beyond just staticcheck and vet there is e.g.
btw, if we remove golint can we see e.g. all-caps constants or functions with underscores dodging review? perhaps we should add golangci-lint soon after:
Completely agree there are better places to spend review bandwidth. I was curious to see what the effort looked like. There have been 890 commits that touched over the past 3+ years !
/remove-help /remove-good-first-issue
Overall agree on the bandwidth. Also note that it was a good first issue with very low bar to get them in the repo. May be we need to find a similar low bar, but high value good first issue.

UT's come to mind, are there other things we curate for new contributors ?
Do we have an alternative way of enforcing that new exported code (not necessarily API fields) has at least some minimum documentation? As a reviewer, I ask contributors to add meaningful comments. But reality is that I sometimes stumble into old code that doesn't have documentation because we didn't have golint at some point. I agree that fix-golint PRs just add burden to reviewers. But, before removing the lint check, can we wait and see if it's enough to discourage people from sending more PRs like this? Additionally, we can encourage reviewers to just close PRs that only fix golint issues.
+1 for adding burden to reviewers. However, as a new contributor, I think this a very good opportunity for new contributors to study the code base and submit their first PR. In my opinion, I think this issue would be able to attract more new contributors, get familiar with the code base, and get familiar with the git workflow.
thanks everyone for getting us this far. We essentially gave up on the existing structure/system for golint: At some point we can look at adding golangci-lint, there's a WIP in progress in /close
Closing this issue. <detailsIn response to : thanks everyone for getting us this far. We essentially gave up on the existing structure/system for golint: Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: if spec.Volume != nil && spec.Volume.GCEPersistentDisk != nil { volumeSource = spec.Volume.GCEPersistentDisk readOnly = volumeSource.ReadOnly <ins> glog.V(4).Infof("volume source %v spec %v, readonly flag retrieved from source: %v", volumeSource.PDName, spec.Name(), readOnly) </ins> } else { volumeSource = spec.PersistentVolume.Spec.GCEPersistentDisk readOnly = spec.ReadOnly <ins> glog.V(4).Infof("volume source %v spec %v, readonly flag retrieved from spec: %v", volumeSource.PDName, spec.Name(), readOnly) </ins> } return volumeSource, readOnly }

Eg: Suspect it's because of:
If this hasn't been seen in 3 months, okay to close?
Yes we fixed it with


Question: <ins> /* Copyright 2015 Google Inc. All rights reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package conversion import ( "reflect" ) var deepCopier = NewConverter() // DeepCopy makes a deep copy of source. Won't work for any private fields! // For nil slices, will return 0-length slices. These are equivilent in // basically every way except for the way that reflect.DeepEqual checks. func DeepCopy(source interface{}) (interface{}, error) { src := reflect.ValueOf(source) v := reflect.New(src.Type()).Elem() s := &scope{ converter: deepCopier, } if err := deepCopier.convert(src, v, s); err != nil { return nil, err } return v.Interface(), nil } </ins>

E.g., it calls Convert() for every byte of a slice of uint8. E0330 22:15:23. 5352 ] TIM: converting bool E0330 22:15:23. 5352 ] TIM: converting []v1.NamedExtension E0330 22:15:23. 5352 ] TIM: converting []v1.NamedCluster E0330 22:15:23. 5352 ] TIM: converting []v1.NamedCluster E0330 22:15:23. 5352 ] TIM: converting v1.Cluster E0330 22:15:23. 5352 ] TIM: converting v1.Cluster E0330 22:15:23. 5352 ] TIM: converting string E0330 22:15:23. 5352 ] TIM: converting string E0330 22:15:23. 5352 ] TIM: converting bool E0330 22:15:23. 5352 ] TIM: converting string E0330 22:15:23. 5352 ] TIM: converting []uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23.

5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 ... thousands of these.
There's an easy fix for this :/


Question: err = fakeRuntimeSvc.applyPlatformSpecificContainerConfig(containerConfig, &pod.Spec.Containers[0], pod, new(int64), "foo", nil) require.NoError(t, err) <del> expectedCpuMax := ((10000 * 3000) / int64(runtime.NumCPU()) / 1000) </del> <ins> limit := int64(3000) expectedCpuMax := 10 * limit / int64(winstats.ProcessorCount()) </ins> expectedWindowsConfig := &runtimeapi.WindowsContainerConfig{ Resources: &runtimeapi.WindowsContainerResources{ CpuMaximum:         expectedCpuMax,

When submitting a windows pod using resource limits on a node with more than one processor group, the specified count is incorrect due to the usage of which does not take processor groups into account. For example: node with 96 cores cpu limit 94 returns This results in a CPU maximum of since the range is from [1, ]: Our initial goal for this was to limit a process to not use all available CPU and therefore render the node uncontrollable. A CPU maximum of instead of for that specific case. For the calculation, it should use the CPU count for all available processor groups instead of just the current one. Launch a node with a lot of CPUs, for example and schedule the following pod on it: Then SSH into the node and inspect the running container using . The value will be set to . (which just uses ) is able to correctly find out the number of CPUs available to a system. Using for calculation or incorporating that code might solve this issue. <details</details<detailsAzure in-tree </details<details</details<details</details<detailscontainerd://1.6.10 </details<details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig windows
There is already some code which uses GetActiveProcessorCount:
/assign
/triage accepted


Question: // NewWithDelegate returns a new instance of APIAggregator from the given config. func (c completedConfig) NewWithDelegate(delegationTarget genericapiserver.DelegationTarget) (*APIAggregator, error) { <del> // Prevent generic API server to install OpenAPI handler. Aggregator server // has its own customized OpenAPI handler. openAPIConfig := c.GenericConfig.OpenAPIConfig c.GenericConfig.OpenAPIConfig = nil </del> genericServer, err := c.GenericConfig.New("kube-aggregator", delegationTarget) if err != nil { return nil, err

It looks like the openapi is not routed properly to set-up the apiservice so apply requests will fail because the fieldmanager is not set-up properly. /cc /sig api-machinery /priority important-soon /kind bug
/wg apply /cc
OpenAPIModels is set to nil for the kube-aggregator server config (which serves the APIServices API): This is done because the kube-aggregator has a special openapi handler that aggregates all the other server's openapi docs. The field controls (at least) two things: /openapi endpoint serving (which we want kube-aggregator to continue to customize) creation of the field manager (which we want kube-aggregator to behave consistently on) We need to decouple presence of OpenAPIConfig and construction of StaticOpenAPISpec (which is needed by the field manager) from construction/installation of OpenAPIVersionedService. That might be a little tricky to untangle, since construction of StaticOpenAPISpec includes paths/operations. The field manager only needs the gvk/schemas.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/lifecycle frozen This is not something we want closed by accident.
/assign
For reference:
Additional lines for reference: This is roughly how SSA consumes the openapi spec. openapi spec -openapi proto model -typeconverter -SSA field manager For built in types, openapi spec -openapi model openapi model -typeconverter A similar thing happens for CRDs As liggitt mentioned, SSA only requires a subset of the information in the openapi generation, and the openapi generation in the aggregator slightly different than the process for builtins and CRDs.
/unassign
/assign


Question: dInfo := driver.GetDriverInfo() var isSupported bool <del> // 1. Check if Whether SnapshotType is supported by driver from its interface // if isSupported, we still execute the driver and suite tests if len(pattern.SnapshotType) > 0 { switch pattern.SnapshotType { case testpatterns.DynamicCreatedSnapshot: _, isSupported = driver.(SnapshottableTestDriver) default: isSupported = false } if !isSupported { e2eskipper.Skipf("Driver %s doesn't support snapshot type %v -- skipping", dInfo.Name, pattern.SnapshotType) } } else { // 2. Check if Whether volType is supported by driver from its interface switch pattern.VolType { case testpatterns.InlineVolume: _, isSupported = driver.(InlineVolumeTestDriver) case testpatterns.PreprovisionedPV: _, isSupported = driver.(PreprovisionedPVTestDriver) case testpatterns.DynamicPV, testpatterns.GenericEphemeralVolume: _, isSupported = driver.(DynamicPVTestDriver) case testpatterns.CSIInlineVolume: _, isSupported = driver.(EphemeralTestDriver) default: isSupported = false } </del> <ins> // 1. Check if Whether volType is supported by driver from its interface switch pattern.VolType { case testpatterns.InlineVolume: _, isSupported = driver.(InlineVolumeTestDriver) case testpatterns.PreprovisionedPV: _, isSupported = driver.(PreprovisionedPVTestDriver) case testpatterns.DynamicPV, testpatterns.GenericEphemeralVolume: _, isSupported = driver.(DynamicPVTestDriver) case testpatterns.CSIInlineVolume: _, isSupported = driver.(EphemeralTestDriver) default: isSupported = false } </ins> <del> if !isSupported { e2eskipper.Skipf("Driver %s doesn't support %v -- skipping", dInfo.Name, pattern.VolType) } </del> <ins> if !isSupported { e2eskipper.Skipf("Driver %s doesn't support %v -- skipping", dInfo.Name, pattern.VolType) } </ins> <del> // 3. Check if fsType is supported if !dInfo.SupportedFsType.Has(pattern.FsType) { e2eskipper.Skipf("Driver %s doesn't support %v -- skipping", dInfo.Name, pattern.FsType) } if pattern.FsType == "xfs" && framework.NodeOSDistroIs("gci", "cos", "windows") { e2eskipper.Skipf("Distro doesn't support xfs -- skipping") } if pattern.FsType == "ntfs" && !framework.NodeOSDistroIs("windows") { e2eskipper.Skipf("Distro %s doesn't support ntfs -- skipping", framework.TestContext.NodeOSDistro) } </del> <ins> // 2. Check if fsType is supported if !dInfo.SupportedFsType.Has(pattern.FsType) { e2eskipper.Skipf("Driver %s doesn't support %v -- skipping", dInfo.Name, pattern.FsType) } if pattern.FsType == "xfs" && framework.NodeOSDistroIs("gci", "cos", "windows") { e2eskipper.Skipf("Distro doesn't support xfs -- skipping") } if pattern.FsType == "ntfs" && !framework.NodeOSDistroIs("windows") { e2eskipper.Skipf("Distro %s doesn't support ntfs -- skipping", framework.TestContext.NodeOSDistro) </ins> } <del> // 4. Check with driver specific logic </del> <ins> // 3. Check with driver specific logic </ins> driver.SkipUnsupportedTest(pattern) }

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: In 1.19 we a "dynamic snapshot" option to the default TestPattern. But it is causing tests to skip for the drivers that don't support snapshots. Even though the test is not actually testing snapshot functionality. Like So it seems we should not add snapshots to default options. And instead we should have an explicit snapshots testpattern for snapshot tests. How to reproduce it (as minimally and precisely as possible): /sig storage /kind bug /cc
/assign could you help take a look at this?