Question: ginkgo.It("should verify ResourceQuota's priority class scope (quota set to pod count: 1) against 2 pods with different priority class.", func() { _, err := f.ClientSet.SchedulingV1().PriorityClasses().Create(context.TODO(), &schedulingv1.PriorityClass{ObjectMeta: metav1.ObjectMeta{Name: "pclass3"}, Value: int32(1000)}, metav1.CreateOptions{}) <del> framework.ExpectEqual(err == nil || apierrors.IsAlreadyExists(err), true) </del> <ins> if err != nil && !apierrors.IsAlreadyExists(err) { framework.Failf("unexpected error while creating priority class: %v", err) } </ins> hard := v1.ResourceList{} hard[v1.ResourcePods] = resource.MustParse("1")

The tests are full of assertions which compare a boolean against true or false, often without any additional explanation. When those assertions fail, the error message is useless for understanding what went wrong, often just saying: Including an explanation is better, but then we still have the distracting noise about the comparison: All of those assertions should be replace with . Such locations can be found by grepping: UPDATE (2024-08-26): in cases where such an if check is not possible (typically in combination with or ), should be used instead of . They emit only the explanation given to them, without including the noise above. The Ginkgo linter (enabled in golangci-lint) points that out. Some progress has been made in the meantime: v1.23.0: 250 v1.24.0-beta.0: 177 However, we were down to 175 at some point, so the number is going back up. Example: Some other places compare values. Those should be replaced with a comparison assertion. Example: It makes sense to split this work into individual PRs, one per OWNERS file under test/e2e. Here's a list of directories: [ ] test/e2e/apimachinery [x] test/e2e/apps: [x] test/e2e/auth [x] test/e2e/autoscaling: [x] test/e2e/cloud/gcp [x] test/e2e/cloud [x] test/e2e/common/network [x] test/e2e/common/node: [x] test/e2e/common/storage: [x] test/e2e/framework: [x] test/e2e/framework/providers/vsphere [x] test/e2e/framework/volume [x] test/e2e/instrumentation/logging [x] test/e2e/instrumentation: [x] test/e2e/kubectl [x] test/e2e/lifecycle [x] test/e2e/node: [x] test/e2e/scheduling: [x] test/e2e/storage: (prior PR: ) [x] test/e2e/testing-manifests/storage-csi [x] test/e2e/upgrades/apps: [x] test/e2e/windows: , When creating a PR, use as title. Use one commit per PR, squash when addressing feedback.

Unless noted otherwise, will work on PRs. Help may be welcome. /sig testing /help Better output for developers when tests fail.
This request has been marked as needing help from a contributor. Please ensure that the issue body includes answers to the following questions: Why are we solving this issue? To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points? Does this issue have zero to low barrier of entry? How can the assignee reach out to you for help? For more details on the requirements of such an issue, please see and ensure that they are met. If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : ### What would you like to be ? Example: Some other places compare values. Those should be replaced with a comparison assertion. Example: /sig testing /help </details
/good-first-issue
This request has been marked as suitable for new contributors. Please ensure that the issue body includes answers to the following questions: Why are we solving this issue? To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points? Does this issue have zero to low barrier of entry? How can the assignee reach out to you for help? For more details on the requirements of such an issue, please see and ensure that they are met. If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : /good-first-issue </details
/triage accepted
I would like to take this up as my first issue
/assign
I would like to take up this issue as my first issue
Hello I am already working on this issue , can you please take some other issue ?
Okay sure ,how do I un assign myself from this issue?
I don't think you are assigned to this issue.
Oh okay
Hey as discussed in pr I am assuming that you are only working for tests under area.

also finished the work on , see
and only one PR is to be made for the correction under test/e2e/storage/, so how can we distribute the task and also make only single PR. Or can we make multiple PR's
I have a TODO list to the issue description. I'll try to keep that up-to-date with what has been done and who is working on it.
you can continue with in
what is to be changed in this case !
can you inform on which directory you are working.
Currently working on apimachinery
Nothing, just leave it as-is.
But that isn't under - before starting somewhere else, please coordinate with
test/e2e/instrumentation/logging don't have any cases like "Expect.Equal([^,], *(true|false)", so I think it should be removed from tasks to do.
I am thinking that the output should be if !ok { framework.Failf(uexec.ExitError) } in this case, am I thinking reasonable here.
isn't valid Go. The first parameter must be a format string. The code checks whether err is a : Therefore the change should be:
Why don't we just add extra explanation into calls like for easy investigation? I might miss some point.
You have to consider the complete output when that case fails. Gomega will write print the actual value and the expected value, then in addition you get the same information in the custom message. Printing just the custom failure message is shorter and more readable, in particular for a comparison of booleans. For a comparison of integers I see two solutions: ExpectEqual with a fixed message that just explains what is being compared. An if check with Failf where the message includes the actual value. The later seems more useful when the expected value is zero because that value doesn't need to be printed when its obvious.
I am getting difficulty in figuring out what should be output here, what should the appropriate output here. Also what steps you took in knowing so, it would help to improve my understanding.
The "ok" value is the result of checking for a specific key in a map. What a developer therefore wants to know when that check fails is: what was the key what's in the map Both can be included in:
can I work for the section :- test/e2e/scheduling ?

And there was only one changes needed for test/e2e/node which I did but my PR was closed as it only contains trivial edits. That was my first PR, so can you please guide me further?
/assign I'll try to coordinate the work, but won't create PRs myself. I ticked of line items which had no remaining work. feel free to start working on .
I replied there and also suggested an improvement. For those following this issue: try to include more information than before in the error message when the assertion fails. Then it is more obvious to the reviewer that the change is useful without having to read this issue. Examples:
Another tip: don't just reference this issue as explanation for the PR. Reviewers are busy, they won't necessarily invest the time and follow the link to understand what the PR is about. Instead, take the first sentence from this issue and adapt it as needed to the specific changes in the code.
can I start working on test/e2e/common/storage ?
Okay.
for test/e2e/network/netpol , there are already message associated with all of the cases, for example :- So should I just change all of them to ? And same for all the cases in test/e2e/network except these two cases : framework.ExpectEqual(netutils.I sIPv4String(internalIPs[0]) != netutils.IsIPv4String(internalIPs[1]), true) framework.ExpectEqual(netutils.IsIPv4String(p.Status.PodIPs[0].IP) != netutils.IsIPv4String(p.Status.PodIPs[1].IP), true )
Yes.
Okay then I will be working on and
It might make sense to combine them in a single PR, depending on how many changes there are - just use common sense.
IMO : this will never fail because whether err will be nil or it will exist. So, we don't need to change this code. What I had done in this case was :
Note that that is redundant (= not wrong, just not needed). I later clarified that in
/reopen
Reopened this issue. <detailsIn response to : /reopen </details
Hi! I've created a PR to address the tests in e2e/apps It is my first PR so hopefully, everything is ok. Thanks a lot!
Hi can I take test/e2e/common/node?
Yes, please do.

Have we considered / helpers that can skip the comparison output? - is noisy and makes it harder to skim tests, especially given that it's a pretty commonly required pattern.
No, we haven't. Do you find it noisy because it's two lines instead of one? I don't have a problem with that. It seems to be a common approach in unit tests and I find it easy to read, partly because it matches how production code would be written.
It becomes three, plus extra indention - but more importantly stands out in a series of expectations. This draws extra scrutiny to those lines. many unit tests using testify would use to express this because of that.
Isn't that a good thing? is bad without any or no useful extra explanation. The if check makes it obvious that a good failure message is needed. Does print less useless boilerplate code than Gomega? We could use that with Ginkgo (), but then we still need to ensure that the optional explanation is useful.
xref
Hi can I take test/e2e/auth?
for test/e2e/auth, for example : Should I just change it to ?
feel free to work on test/e2e/auth. This one particular example is a bit different: here we are comparing string values, and for those it makes sense to print the actual value because the failure message then adds information. I would leave that one as it is.
OK. test/e2e/auth doesn't need replacing. I : <img width="731" alt="Screen Shot 2022-05-30 at 9 12 59 PM" src="https://user-"There are a lot of positions that need to be modified in . I'll take test/e2e/apps.
That looks like a good candidate. Please go for it. When you need to make up new failure messages, try to think about what information may be useful to have and add that. Then the resulting PR is clearly better than the status quo.
Hi can you please assign me a directory. I want to give it a try. Thanks,
Hi I already posted a PR for e2e/apps some days ago in this issue: If it is fine I'd love to proceed with that. Just to avoid us repeating work.
sorry for not noticing the overlap sooner. Let's go with the older PR.

you can take over test/e2e/storage from There is an incomplete PR for it in You can check out that branch, fix it and submit a new PR, or start from scratch.
it looks like is not active anymore. I've remove him from the todo list and you instead for test/e2e/common/storage.
OK. Thank you. Is there an incomplete PR for .
I in and it returns 0. What should I do?
needs modifying. <img width="1440" alt="Screen Shot 2022-05-31 at 2 40 29 PM" src="https://user-"Can I work on it?
Indeed, did test/e2e/common/storage in You can work on test/e2e/windows.
OK. I'm working on it currently.
Sure, I will look into it. Thanks,
: do you want to continue with test/e2e/apimachinery?
Yes. There's an older PR:
Hi is there anything I could take over?
Hey I am not sure how updated the list is, but I am available to help. Is there any part I could start or continue?
sorry for the late reply. I had (and still have) some big changes for the E2E tests going and had put this on the backseat for a while. If you are interested, how about doing test/e2e/apimachinery and test/e2e/autoscaling? Some work was done on the latter, but there are still several calls (again?).
Sure, I can take that!
Hey I'd also like to contribute on this issue. Is there anything I could take over?
feel free to take my part:
Perfect, then I will take part
Hi :smile: , I would like to contribute to this issue, there is anything that I could do?
I was reluctant to start new work on this because I had one big PR pending that touched a lot of test code (PR ), but that has been merged. you are taking test/e2e/autoscaling, right? test/e2e/apimachinery? you can do test/e2e/apps. That leave test/e2e_node in case someone wants to continue.
yes e2e/autoscaling. I had some trouble to get the tests going initially, but seems fine now!
Hi, Can I do something to contribute to this issue?

you could work on test/e2e_node.
Ok, I will start work on test/e2e_node thanks!
Hi , I would like to contribute. Is there anything I could take over?
thanks for your interest, but right now everything seems to be covered.
Hi Thanks. I am searching for "good-first-issue", feel free to assign one to me. I am exploring api-machinery, but happy to contribute in other sig as well.
maybe you can take a look at if you have some time :)
Hi , There is anything that I could do to contribute with this issue?
Hi , I was looking and found the test/e2e/apimachinery without doing it and only then did I see that it was already assigned if this is a problem just refuse my pr and if you have something else to do let me know.
The remaining work will probably finished in , so there isn't anything else to do. If someone wants to investigate writing a linter for this, then this would be a cool science project - not easy, though. I would start with looking at for this. It has the advantage that it is supported indirectly in golangci-lint ().
Hello I would like to try to do a liter for it. It will be a cool project whit a lot of opportunities to learn. There will be a separate issue for the linter or we continue the discussion here?
Let's continue here.
I ended up creating a separate issue for linter ideas: let's continue there.
Ok!
New tests have been again with this anti-pattern since the last time that I checked. It's clear that without a linter, we'll not get rid of it. So let's focus on that first.
is this issue still open
Yes, this is still open. Current status as of v1.28.0-rc.0: Two calls are in , the rest are in . We have such a linter now, with ginkgolinter in golangci-lint and (soon) the ability to prevent usage of , so it makes sense to continue with removing those calls. do you want to prepare one PR for all remaining calls? I think one PR is enough, no need to split it up.
also has it, which was intentional. I'm undecided whether that should be replaced.

Several of those calls are being touched in , let's wait for that PR to be merged before doing any further work.
no longer finds anything. We might be done... :tada: But not quite. What we have left now is checking a boolean via or , which has the same problem. In some cases, there is an extra message (which is still noisy, see ), in other cases there isn't even that (which makes the failure message fairly useless). We can and should replace some of those calls with the pattern.
is almost resolved. My work-in-progress branch forbids the usage of in favor of the new or plain Go. In some cases, moving the Go code that returns true/false into a matcher might be the best solution because that allows dumping the full actual value that was being checked. Here are the remaining issues: <details</detailsDoes anybody want to work on those?
I would love to!
: please go ahead. Assign PRs to me when you create them. Let's start with a few of those calls initially, do a round of reviews, then continue. /assign
You will have to start with my branch, current master doesn't have gomega with thew new API yet.
I'm starting new to kubernetes. Is there anything easy that I could do ?
Hey Is there anything left I can help you with in this issue ?
Thanks for your interest. There's not much to be done at the moment, though.
Should this be closed?
Oh wow! Looks like we are indeed done. Thanks everyone. /close
Closing this issue. <detailsIn response to : Oh wow! Looks like we are indeed done. Thanks everyone. </details


Question: assert.Equal(t, test.expectedErrMsg, err, test.name) assert.Equal(t, test.expected, exist, test.name) } <ins> vmssTestCases := []struct { name       string providerID string scaleSet   string vmList     []string expected   bool rerr       *retry.Error }{ { name:       "InstanceExistsByProviderID should return true if VMSS and VM exist", providerID: "azure:///subscriptions/script/resourceGroups/rg/providers/Microsoft.Compute/virtualMachineScaleSets/vmssee6c2/virtualMachines/0", scaleSet:   "vmssee6c2", vmList:     []string{"vmssee6c2000000"}, expected:   true, }, { name:       "InstanceExistsByProviderID should return false if VMSS exist but VM doesn't", providerID: "azure:///subscriptions/script/resourceGroups/rg/providers/Microsoft.Compute/virtualMachineScaleSets/vmssee6c2/virtualMachines/0", scaleSet:   "vmssee6c2", expected:   false, }, { name:       "InstanceExistsByProviderID should return false if VMSS doesn't exist", providerID: "azure:///subscriptions/script/resourceGroups/rg/providers/Microsoft.Compute/virtualMachineScaleSets/missing-vmss/virtualMachines/0", rerr:       &retry.Error{HTTPStatusCode: 404}, expected:   false, }, } for _, test := range vmssTestCases { ss, err := newTestScaleSet(ctrl) assert.NoError(t, err, test.name) cloud.VMSet = ss mockVMSSClient := mockvmssclient.NewMockInterface(ctrl) mockVMSSVMClient := mockvmssvmclient.NewMockInterface(ctrl) ss.cloud.VirtualMachineScaleSetsClient = mockVMSSClient ss.cloud.VirtualMachineScaleSetVMsClient = mockVMSSVMClient expectedScaleSet := buildTestVMSS(test.scaleSet, test.scaleSet) mockVMSSClient.EXPECT().List(gomock.Any(), gomock.Any()).Return([]compute.VirtualMachineScaleSet{expectedScaleSet}, test.rerr).AnyTimes() expectedVMs, _, _ := buildTestVirtualMachineEnv(ss.cloud, test.scaleSet, "", 0, test.vmList, "succeeded", false) mockVMSSVMClient.EXPECT().List(gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any()).Return(expectedVMs, test.rerr).AnyTimes() mockVMsClient := ss.cloud.VirtualMachinesClient.(*mockvmclient.MockInterface) mockVMsClient.EXPECT().List(gomock.Any(), gomock.Any()).Return([]compute.VirtualMachine{}, nil).AnyTimes() exist, _ := cloud.InstanceExistsByProviderID(context.Background(), test.providerID) assert.Equal(t, test.expected, exist, test.name) } </ins> } func TestNodeAddressesByProviderID(t *testing.T) {

What happened: We saw several occurrences of kube-controller-manager failing to check a k8s node existence after we deleted that node's parent VMSS (and by extension, cloud instance). The leaked k8s nodes remained present in the cluster until manually deleted. Cloud provider retries caused a VMSS VMs API call surge, degenerating in hard throttling for the whole subscription. In those situations, kube-controller-manager would continuously log errors like the following: Looking at Azure cloud-provider implementation shows that all the VMSS API calls errors (including a 404/not found on the VMSS object) are bubbled up as generic errors, and never used as an indication of a missing instance. What you expected to happen: k8s nodes being removed as their VMSS went missing. How to reproduce it (as minimally and precisely as possible): Create and delete many VMSS providing instances registering to a cluster. Environment: Kubernetes version (use ): 1.19.0 Cloud provider or hardware configuration: Azure (self managed cluster, non AKS, ) OS (e.g: ): Linux, Ubuntu Focal 20.04.1 LTS Kernel (e.g. ): 5.4.0-1022-azure /sig cloud-provider /area provider/azure
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: return &rbd{ podUID:   podUID, volName:  spec.Name, <del> mon:      source.CephMonitors, image:    source.RBDImage, pool:     pool, id:       id, keyring:  keyring, secret:   secret, </del> <ins> Mon:      source.CephMonitors, Image:    source.RBDImage, Pool:     pool, Id:       id, Keyring:  keyring, Secret:   secret, </ins> fsType:   source.FSType, <del> readOnly: source.ReadOnly, </del> <ins> ReadOnly: source.ReadOnly, </ins> manager:  manager, mounter:  mounter, plugin:   plugin,

etcd cluster backing apiserver died spontaneously, after restart apiserver pulled up the pods on different machines. Node storage was backed by ext4 over rbd, which became mounted on several nodes at once. That could have possibly led to data corruption. I guess that the proper way to mount rbd volume is to take a lock on it first, which will at least fail if it's locked somewhere else, requiring manual intervention, but not destroying data. The lock should be optional, in case a proper cluster fs is mounted over the rbd. In this case there was clearly no need for accessing the data from several pods so ext4 is deemed a proper fs.
, Vladimir Pouzanov < :
want to get some detail. How is your pod-to-rbd mapping like? Since rbd image is a block device, it can be used by one pod at a time. I am not sure if locking is a proper way to go - if the pod holds a lock but dies in the middle of mounting, the rbd image (or any other volume type) is then excluded from access. In such case, a high level coordination at apiserver is a better place to resolve this problem.
Here's my rc: I guess it's better to have a lock to be cleaned up manually rather than to have non-cluster fs to be mounted in several locations at once?
thanks, this pod looks good to me. what would cause multiple nodes run the same pod?
I don't know the exact reason of why kubelet didn't unmount the old fs, but after apiserver and scheduler were restarted pod got allocated on different node.
It is a scary thing to see etcd down, it is meant to be reliable...
Unfortunately I don't have long-term logging in test cluster so I have no idea why it actually died. but using kubedns as primary node resolver was clearly a bad idea, once it died, nodes stuck with no resolver at all.
your thought on HA?
Wrong Tim St. Clair?
- you HAVE to be ready to handle network partitions - the old node might still be alive and even using the volume, but the apiserver can't see the node, so it thinks it deletes the pod and starts it elsewhere. If RBD corrupts itslef in this case, it is not useful and we should strip it from the codebase.

, Huamin Chen wrote:
that makes sense. rbd doesn't corrupt itself. This is inconsistency happens when you have two pods mounting the same rbd image. How does GCE PD handle/prevent this case?
It doesn't allow more than one read-write attachment. Multiple read-only attachments is OK. Better to fail than to allow corruption. , Huamin Chen wrote:
That's neat. I have a lock fencing for rbd, testing now.


Question: "k8s.io/apiserver/pkg/storage/value" ) <ins> // The short keepalive timeout and interval have been chosen to aggressively // detect a failed etcd server without introducing much overhead. var ( keepaliveTime    = 30 * time.Second keepaliveTimeout = 10 * time.Second ) </ins> func newETCD3Storage(c storagebackend.Config) (storage.Interface, DestroyFunc, error) { tlsInfo := transport.TLSInfo{ CertFile: c.CertFile,

Hi, running 1.6.4 on etcd 3.0.17 in the following setup: 3 etcd nodes (bootstrapped according to the ) 9 kubernetes cluster nodes all of them running kube-apiserver (--apiserver-count=9) with --bind-address to local kube-apiserver instance. All instances pointing to the same etcd cluster (--etcd-servers=http://IP1:4001,http://IP2:4001,http://IP3:4001) I tried to simulate 1 etcd node crash by simply turing off one of the machines. Result: timeouts running kubectl (get nodes, get pods) commands lots of these error messages in kube-apiserver logs: "] apiserver received an error that is not an metav1.Status: http: Handler timeout" ETCD correctly reports unhealthy node, but healthy cluster Inspecting netstat shows lots of ESTABLISHED connections to the dead etcd server, port 4001 - these connections are cleared only by restarting the kube-apiserver(s). Workarounds: Switching back to etcd 2.2.1 and using --storage-backend=etcd2 ... ESTABLISHED connections are cleared within several seconds Leaving etcd 3.0.17 and using --storage-backend=etcd2 ... ESTABLISHED connections are cleared within several seconds In both above cases kubectl shows expected results. The dead node is evicted (NotReady), other nodes & pods are in tact. I noticed that client contains TCP KeepAlive timeout, while client does not. Could this be related to the described scenario? <!-- If this is a BUG REPORT, please: Fill in as much of the template below as you can. If you leave out information, we can't help you as well. If this is a FEATURE REQUEST, please: Describe in detail the feature/behavior/change you'd like to see. In both cases, be ready for followup questions, and please respond in a timely manner. If we can't reproduce a bug or think a feature already exists, we might close your issue. If we're wrong, PLEASE feel free to reopen it and explain why.

--Kubernetes version Client Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.4", GitCommit:"", GitTreeState:"clean", BuildDate:"2017-05-19T18:44:27Z", GoVersion:"go1.7.5", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.4", GitCommit:"", GitTreeState:"clean", BuildDate:"2017-05-19T18:33:17Z", GoVersion:"go1.7.5", Compiler:"gc", Platform:"linux/amd64"} Environment: OS: NAME="Oracle Linux Server" VERSION="6.7" ID="ol" VERSIONID="6.7" PRETTYNAME="Oracle Linux Server 6.7" ANSICOLOR="0;31" CPENAME="cpe:/o:oracle:linux:6:7" HOMEURL="" BUGREPORTURL="" ORACLEBUGZILLAPRODUCT="Oracle Linux 6" ORACLEBUGZILLAPRODUCTVERSION=6.7 ORACLESUPPORTPRODUCT="Oracle Linux" ORACLESUPPORTPRODUCTVERSION=6.7 Kernel
There are no sig labels on this issue. Please by:<br(1) mentioning a sig: <br(2) specifying the label manually: <br<brNote: method (1) will trigger a notification to the team. You can find the team list .
/area etcd
Reiterating the mentions to trigger a notification: <detailsIn response to : Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
This seems to depend on an issue with etcd, there is no option for a KeepAlive I believe. I've sent a PR, can fix this once that gets merged.
The mentioned PR has been merged. We may need to wait for an etcd release / update the dependency on etcd for us to be able to fix this in Kubernetes.
Had a look at the current CHANGELOG on etcd. The fix for this issue is in master, and will be released in etcd clientv3 3.3.0.

Here is the which shows the fix has been merged into the . Kubernetes is currently on clientv3 3.2.11 it appears (according to ).


Question: metadata: name: external-provisioner-runner rules: <del> - apiGroups: [""] resources: ["secrets"] verbs: ["get", "list"] </del> <ins> # The following rule should be uncommented for plugins that require secrets # for provisioning. # - apiGroups: [""] #   resources: ["secrets"] #   verbs: ["get", "list"] </ins> - apiGroups: [""] resources: ["persistentvolumes"] <del> verbs: ["get", "list", "watch", "create", "delete"] </del> <ins> verbs: ["get", "list", "watch", "update", "create", "delete"] </ins> - apiGroups: [""] resources: ["persistentvolumeclaims"] verbs: ["get", "list", "watch", "update"]

<!-- Please only use this template for submitting reports about failing tests in Kubernetes CI jobs --Which jobs are failing: pull-kubernetes-e2e-gce Which test(s) are failing: Since when has it been failing: unknown Testgrid link: Reason for failure: unknown Anything else we need to know: PR that introduces the new test: /assign /sig storage /kind flake /priority critical-urgent the job is PR blocking and is flaking mutliple times in a row.
/remove-kind bug
from SIG Testing mentioned that probably the whole job is timing out, so this might not be a SIG Storage problem per-se. we need the owners to triage the problem.
Here's one example run:
i've sent an email to k-dev to see who reacts. increasing the timeout is the preferred option for now.
To clarify I think this is a real failure that really happened at least twice, but it is likely not the primary cause of pull-kubernetes-e2e-gce currently being broken, and almost all failed pull-kubernetes-e2e-gce runs that I see had this test pass.
Let's use this bug to triage this specific test case
The first provision succeeded: 0404 18:32:48. 1 ] provision "multivolume-4799/pvc-mvjvq" class "multivolume-4799-csi-hostpath-multivolume-4799-scjmgx4": started The second one failed because it couldn't find the storage class: I0404 18:32:50. 1 ] provision "multivolume-4799/pvc-985xh" class "multivolume-4799-csi-hostpath-multivolume-4799-scm9tnq": started E0404 18:32:50. 1 ] provision "multivolume-4799/pvc-985xh" class "multivolume-4799-csi-hostpath-multivolume-4799-scm9tnq": error getting claim's StorageClass's fields: storageClass "multivolume-4799-csi-hostpath-multivolume-4799-scm9tnq" not found
The provisioner doesn't retry/requeue if the storageclass is not found.

I guess it is flaky because the pvc is created immediately after the sc and even though the provisioner has watches on both, it's possible it sees pvc before sc? It's a bug in the external library, not an issue with in-tree because it retries everything every 15s. So I'll need to update the library, update hostpath plugin, then update our manifests here. btw, how did you find that log
There is probably something else wrong too* because every other dynamic provisioning test creates pvc immediately after the sc. Why does only this one flake?
We store all the CSI driver logs for each test case under artifacts
Should we just add wait before and ? Or are there any ways to ensure that storageClass is ready? (like checking that storageClass can be got after creation.)
I don't think wait is the solution. Even if the test does a Get after Create, there is no guarantee that the provisioner pod's informer has seen the StorageClass too. I've opened for now I am still unsure why only this test is flaking frequently, if the issue is race between controller seeing PVC first or SC first, one would expect a lot of other external provisioner tests to fail too...
I guess given the frequency of failures, waiting for SC to exist after the Create call could be a good stopgap solution. Fixing the lib-hostpath plugin-test will take a lot more time.
Can we wait until 's actual fix for this issue or do we need workaround to reduce this flake?
Let's go with 's fix. I don't think it should take too long


Question: "io" "strings" "text/tabwriter" <ins> "text/template" </ins> "github.com/GoogleCloudPlatform/kubernetes/pkg/api" "github.com/GoogleCloudPlatform/kubernetes/pkg/labels"

To remove dependence on jq and generally make scripting much easier, we should make another output flag which takes a formatter. We can use go's template package, which is very flexible.


Question: rm -rf "${docker_build_path}" mkdir -p "${docker_build_path}" ln "${binary_dir}/${binary_name}" "${docker_build_path}/${binary_name}" <del> printf " FROM ${base_image} n ADD ${binary_name} /usr/local/bin/${binary_name}n" > "${docker_file_path}" </del> <ins> ln "${KUBE_ROOT}/build/nsswitch.conf" "${docker_build_path}/nsswitch.conf" chmod 0644 "${docker_build_path}/nsswitch.conf" cat <<EOF > "${docker_file_path}" FROM ${base_image} COPY ${binary_name} /usr/local/bin/${binary_name} EOF # ensure /etc/nsswitch.conf exists so go's resolver respects /etc/hosts if [[ "${base_image}" =~ busybox ]]; then echo "COPY nsswitch.conf /etc/" >> "${docker_file_path}" fi </ins> "${DOCKER[@]}" build --pull -q -t "${docker_image_tag}" "${docker_build_path}" >/dev/null "${DOCKER[@]}" tag "${docker_image_tag}" "${deprecated_image_tag}" >/dev/null "${DOCKER[@]}" save "${docker_image_tag}" "${deprecated_image_tag}" > "${binary_dir}/${binary_name}.tar"

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . If the matter is security related, please disclose it privately via --Is this a BUG REPORT or FEATURE REQUEST?: What happened: My broke after , after much debugging we discovered that the API server could not reach itself on , further debugging showed that DNS lookups for localhost were occuring. Long story short, while we have entries for localhost, the api server was not respecting them. What you expected to happen: our standard control plane deployments should not lookup localhost via dns. they should respect /etc/hosts How to reproduce it (as minimally and precisely as possible): create a cluster and monitor network traffic. then try again after adding a minimal to the api server pod. Anything else we need to know?: Prior art: This happens because we build with go's build in DNS resolution, which respects glibc's , but we build images based on busybox for some components (like the apiserver) which does not ship with one. Environment: kubernetes using Kubernetes version (use ): /sig release
/assign cc Thanks a million to and for looking into this with me! this was tricky to resolve.
We should be able to fix this by just adding a minimal to the images make go's resolver happy. alternatively we can standardize on eg which already ships one
oh wow. tough one to debug :)
/milestone v1.13 /priority important-soon
I have a PR under way, will file after lunch!
For now adding the , but discussing with and it sounds like switching to debian-base everywhere is the best long term answer. It will actually save a mb or so on each node :sweat_smile: ... and give a more consistent base for component images to build on.
I just got bit by this.
This issue will bite anyone deploying a statically linked go binary, not just the api server. Can we get kubelet to slide in a default that is sane so it wont keep biting people? Similar to
We'd need a feature request to sig-node and probably sig-network. I do like this idea, but, is used for things other than networking so injecting one might be slightly problematic. In this case it was fine because we control the images and the binaries.

People deploying go images need to make sure they have a sane for now, or not deploy staticly linked binaries, unfortunately.
For 1.13, we should just document the problem
would it be difficult to add the file only if it doesn't exist in the image?
workaround for 1.12: echo hosts: files dns myhostname and add to the ClusterConfig:
Yeah that work around for now, I'm not sure how difficult / expensive it would be for the kubelet to conditionally add this, and it might be a surprising behavior change for arbitrary containers / pods.
it may be surprising, but its very surprising now too, that dns takes precident to hosts file unexpectedly. :( I'd rather it be secure by default and surprise them that its safe. I didn't even know glibc had this backwards behavior when was missing. I haven't run up against a glibc based system that didn't have an nsswitch file. All distro's add one.
hmm... etcd is crashing too.... I hope there is an equiv of etcdExtraVolumes....
never mind on the etcd thing that was something else. The above yaml tweaks did solve the issue.
We have just wasted a TON of time on this when a misconfigured Samsung TV on the other side of the world, plus lack of rigor or our DDNS policy broke our Kube 1.12 clusters. I really don't like that these control plane images are hitting DNS for every lookup of localhost. I am very keen to see implemented.
The control plane images in future releases already contain , which could be cherry picked back. Doing this in an automated fashion like that issue would require bringing a feature to sig-network and sig-node for discussion. I suspect supplying a file only if it doesn't exist in the image would be somewhat expensive compared to what we do currently, and it's definitely a breaking change. In the meantime, if you're containerizing go and not using CGO for networking, you need to add this to your own images. On Tue, Nov 20, 2018, 10:00 nickperry wrote:
I would argue, that 1.12 was a broken release. 1.11 and below did not have the issue.

As K8S is maturing, some of us, especially in more enterprise like environments, are not upgrading as fast, so the issues are being found slower. I think backporting the fix for the control plane images is appropriate to resolve the regression until a more universal fix can be put in place.
that sounds like a great WHODUNIT war story :)
generally I'm not upgrading production workloads particularly quickly either :-) We can file cherry picks back to 1.12, but lack of an in the control plane images is not new: EG does not contain an . IMO, setting on control plane pods was certainly broken, which I questioned, but not just for this reason, and that change was eventually reverted IIRC. /cc Regarding -- because Kubernetes wants to be stable there are processes for proposing and vetting features prior to adding them, particularly for user facing changes! If you'd like to see this, please discuss it with sig-network and sig-node. I don't personally have the bandwidth for this right now and I'm not even sure that it is even the right solution.
yes the change was reverted. it also spawned a little bit of a discussion here for a rather complicated workaround: given this affects images that are already deployed at GCR, it's unlikely that it can be cherry picked. the images do support revisions, but this would also mean porting changes to all the bundled tools that use them as hardcoded tags. also this means revisions for all patch release images. overall i see this as something that will not be approved by sig-network and the GCR maintainers.
I guess to prevent a future regression, we probably need a test that inserts a <some other addressrecord somewhere during the test.


Question: "fmt" "io/ioutil" "net/http" <ins> "os" </ins> "reflect" "testing"

<!-- Please only use this template for submitting reports about failing tests in Kubernetes CI jobs --Which jobs are failing: ci-kubernetes-e2e-gci-gce-alpha-enabled-default Which test(s) are failing: CSI hostpath-v0 tests Since when has it been failing: 8/8 Testgrid link: https://k8s- Reason for failure: Anything else we need to know: We are removing CSI 0.3 support in 1.18, which is before selflink is going to be removed in 1.20. So we should just disable hostpath-v0 from alpha tests. /assign
/milestone v1.16


Question: } // DEPRECATED 1.9 - This group version of IPBlock is deprecated by networking/v1/IPBlock. <del> // IPBlock describes a particular CIDR (Ex. "192.168.1.1/24") that is allowed to the pods // matched by a NetworkPolicySpec's podSelector. The except entry describes CIDRs that should // not be included within this rule. </del> <ins> // IPBlock describes a particular CIDR (Ex. "192.168.1.1/24","2001:db9::/64") that is allowed // to the pods matched by a NetworkPolicySpec's podSelector. The except entry describes CIDRs // that should not be included within this rule. </ins> message IPBlock { // CIDR is a string representing the IP Block <del> // Valid examples are "192.168.1.1/24" </del> <ins> // Valid examples are "192.168.1.1/24" or "2001:db9::/64" </ins> optional string cidr = 1; // Except is a slice of CIDRs that should not be included within an IP Block <del> // Valid examples are "192.168.1.1/24" </del> <ins> // Valid examples are "192.168.1.1/24" or "2001:db9::/64" </ins> // Except values will be rejected if they are outside the CIDR range // +optional repeated string except = 2;

<!-- Please only use this template for submitting enhancement requests --What would you like to be : mentions an IPBlock with only IPv4 examples. Add a comment or example to clarify that IPv6 network blocks are (or possibly aren't) valid here. Why is this needed: Cluster operators who use IPv6 networking want to understand how to isolate different workloads inside their cluster network (and without reading source code). Logged against kubernetes/kubernetes rather than the website repo because this is generated documentation. Kubernetes already has single-stack support for IPv6 clusters KEP for dual-stack support is /sig docs /sig network /priority backlog
/triage unresolved Comment when the issue is assessed and confirmed. I am a bot run by vllry.
/assign /remove-triage unresolved can you guide me on what steps are needed to publish the examples?
I via the pull request. (Although I want these examples , it's not a task I'm familiar with).


Question: }, { "ImportPath": "github.com/pelletier/go-buffruneio", <del> "Comment": "v0.1.0", </del> "Rev": "df1e16fde7fc330a0ca68167c23bf7ed6ac31d6d" }, {

Is this a BUG REPORT or FEATURE REQUEST? (choose one): bug report Kubernetes version (use ): Environment: Cloud provider or hardware configuration: vsphere OS (e.g. from /etc/os-release):Kernel (e.g. ): 4.9.24-coreos Install tools: Others: What happened: After bringing a cluster up, there are no container metrics in the kubelet at /metrics What you expected to happen: I expect to see the metrics, but there are none How to reproduce it (as minimally and precisely as possible): Anything else we need to know: If I try and access the metrics in cAdvisor directly on port 4194 instead of on port then I see the following type of error for all of the metrics:
There are no sig labels on this issue. Please by:<br(1) mentioning a sig: <br(2) specifying the label manually: <br<brNote: method (1) will trigger a notification to the team. You can find the team list and label list
/sig node
This appears to be a v1.7 issue. If I run 1.6.6 I get these metrics. Seems like something has happened in 1.7. I'm running the 1.7 beta because I need nodePort for cni, so now I'm stuck.
Can you triage the issue since?
cc/
Can you help verifying is this is a new regression introduced in 1.7 cadvisor through node-e2e? Thanks!
I've tried this on both cos and cvm in 1.7 cluster, and no container metrics show up. However, I created a 1.6.4 GKE cluster, and container metrics are there. This is a regression in 1.7. Will figure out what is wrong here.
Here is a corresponding cadvisor issue
Even the user still can get the container metrics / stats from kubelet /summary api (recommended way), but since this is a regression introduced in 1.7 release, and we aren't dropping prometheus support, let's fix this for 1.7. Thanks!
Fixed in Will close this issue after building new cadvisor release and cherrypick into 1.7
Thanks for the quick find everyone! I'll be happy to give this a go on the next release!

I'm still facing this same issue. My kubernetes cluster is deployed with kubeadm on Centos 7. I run a k8S v1.8.1 cluster. When I try to fetch containers metrics this way: http://localhost:8001/api/v1/proxy/nodes/node1:0/metrics http://localhost:8001/api/v1/proxy/nodes/node1:/metrics http://localhost:8001/api/v1/proxy/nodes/node1:/metrics I find no metrics. kubelet is started with . If I start kubelet with . metrics are available through .


Question: For management convenience and consistency, `services` and `replicationControllers` may themselves have labels and would generally carry the labels their corresponding pods have in common. <del> Sets identified by labels and label selectors could be overlapping (think Venn diagrams). For instance, a service might point to all pods with `tier in (frontend), environment in (prod)`.  Now say you have 10 replicated pods that make up this tier.  But you want to be able to 'canary' a new version of this component.  You could set up a `replicationController` (with `replicas` set to 9) for the bulk of the replicas with labels `tier=frontend, environment=prod, track=stable` and another `replicationController` (with `replicas` set to 1) for the canary with labels `tier=frontend, environment=prod, track=canary`.  Now the service is covering both the canary and non-canary pods.  But you can mess with the `replicationControllers` separately to test things out, monitor the results, etc. </del> <ins> Sets identified by labels and label selectors could be overlapping (think Venn diagrams). For instance, a service might target all pods with `tier in (frontend), environment in (prod)`.  Now say you have 10 replicated pods that make up this tier.  But you want to be able to 'canary' a new version of this component.  You could set up a `replicationController` (with `replicas` set to 9) for the bulk of the replicas with labels `tier=frontend, environment=prod, track=stable` and another `replicationController` (with `replicas` set to 1) for the canary with labels `tier=frontend, environment=prod, track=canary`.  Now the service is covering both the canary and non-canary pods.  But you can mess with the `replicationControllers` separately to test things out, monitor the results, etc. </ins> Note that the superset described in the previous example is also heterogeneous. In long-lived, highly available, horizontally scaled, distributed, continuously evolving service applications, heterogeneity is inevitable, due to canaries, incremental rollouts, live reconfiguration, simultaneous updates and auto-scaling, hardware upgrades, and so on.

Currently replication controllers define an exact value. Are there disadvantages to allowing the replication controller to define more of a policy - at least X, at most X, exactly X? Since replication controllers aren't really aware of overlapping sets, I guess you can't really use that to guide allocation between two pools because they would just end up fighting unproductively
What is the use case? This seems sort of like auto-scaling policy. I'd prefer to put those objects in the auto-scaler.
It sounds like you want a canary-controller that scales the old version down while scaling the new one up. Combine with a service that spreads traffic across both versions and this should Just Work (tm).
I'm thinking about this from the mechanics of doing a deployment by composing replication controllers. With controllers as they are today: existing replication controller with old template, name=foo,deployment=1 x5 create new replication controller with new template, name=foo,deployment=2 x0 alter controller 2 to be x1 (or up by the batch size) alter controller 1 to be x4 (or down by the batch size) ... repeat until done delete controller 1 The thought around atLeast was about explicitly representing the invariant you want during an available deployment: existing replication controller with old template, name=foo,deployment=1 x5 create new replication controller with new template, name=foo, atLeast=5 set controller to be atMost=5 delete a pod from the name=foo,deployment=1 query ... continue until 0 pods in that query delete controller 1 update controller 2 to change the label query to name=foo,deployment=2 I'm not sure that's better.... just the for loop for deletes seems more reasonable. Not a huge deal, just trying to work through how we'd model common deploy cases for end users.
Overlapping replicationController sets: I think we should validate that this isn't possible upon creation of a replicationController. We can do this by determining whether the label selector may overlap with any others, though to do this analytically without looking at the combinations of labels attached to pods it requires that all replicationController label selectors have at least one key in common, with a different value than all other replicationControllers. Policy: As mentioned, we intended the policy to go in a separate auto-scaling policy entity, in a higher-level layer / separate component.

Deployment: As pointed out, it is possible to create a new replicationController and gradually scale it up while scaling the original down. Or, one could do the Asgard trick of creating a new replicationController and then shift traffic to it. It's possible to add/remove instances from a service by tweaking their labels. For instance one could toggle the value of an in_service=true/false label. The update demo changes the template of a single replication controller and then kills instances one by one until they all get replaced. I don't like that due to the risk of unpredictable replacements and complexity involved with rollbacks. Splitting out the template ( ) would address the rollback issue, but not the unpredictability. It seems to me that the atMost approach suffers from the same risk. That said, I see what you mean that atLeast/atMost are not auto-scaling policies, but are invariants to maintain. I could imagine that flexibility could be useful. For example, a worker pool manager could use that to opportunistically request resources, where some pod requests may fail. Or, perhaps in the case that we need to temporarily create a new pod for some reason (CRIU-based migration?) but we don't want an old one to be killed. So, I like the idea, just not the use case. :-) FWIW, exactly X could be implemented by combining atLeast and atMost.
Agree that without a kill policy on the replication controller the uses cases described don't work. Is a kill policy something you've discussed for the replication controller (oldest, ones that don't match my template first, ones on most overloaded systems)?
I agree that kill policy would be useful and all of the policies you mention are reasonable in particular scenarios. However, I see this as a rich, unbounded policy space, much like scheduling. IIRC, Joe and/or Brendan proposed kill policies early on and I asked for them to be removed. Instead, I propose that we leave the default policy entirely up to the system, much like scheduling, so we can take as many factors into account as we like, and otherwise I'd like to enable the user to implement an arbitrary policy using event hooks . For instance, I could imagine POSTing to a URL whenever a pod needed to be killed.

The user's hook would have N seconds to respond (perhaps configurable), after which the replication controller would kill its default choice.
Thanks for the clarification - I think I'll close this for now until the API stabilizes, and then add some of this context to the doc.
I suggest re-opening this issue, but change the title to "add XXX context to doc YYY", so that we can reference in the PR for posterity.


Question: <ins> /* Copyright 2019 The Kubernetes Authors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package volumemanager import ( "k8s.io/api/core/v1" "k8s.io/kubernetes/pkg/kubelet/config" "k8s.io/kubernetes/pkg/kubelet/container" "k8s.io/kubernetes/pkg/volume/util/types" ) // FakeVolumeManager is a test implementation that just tracks calls type FakeVolumeManager struct { volumes       map[v1.UniqueVolumeName]bool reportedInUse map[v1.UniqueVolumeName]bool } // NewFakeVolumeManager creates a new VolumeManager test instance func NewFakeVolumeManager(initialVolumes []v1.UniqueVolumeName) *FakeVolumeManager { volumes := map[v1.UniqueVolumeName]bool{} for _, v := range initialVolumes { volumes[v] = true } return &FakeVolumeManager{ volumes:       volumes, reportedInUse: map[v1.UniqueVolumeName]bool{}, } } // Run is not implemented func (f *FakeVolumeManager) Run(sourcesReady config.SourcesReady, stopCh <-chan struct{}) { } // WaitForAttachAndMount is not implemented func (f *FakeVolumeManager) WaitForAttachAndMount(pod *v1.Pod) error { return nil } // GetMountedVolumesForPod is not implemented func (f *FakeVolumeManager) GetMountedVolumesForPod(podName types.UniquePodName) container.VolumeMap { return nil } // GetExtraSupplementalGroupsForPod is not implemented func (f *FakeVolumeManager) GetExtraSupplementalGroupsForPod(pod *v1.Pod) []int64 { return nil } // GetVolumesInUse returns a list of the initial volumes func (f *FakeVolumeManager) GetVolumesInUse() []v1.UniqueVolumeName { inuse := []v1.UniqueVolumeName{} for v := range f.volumes { inuse = append(inuse, v) } return inuse } // ReconcilerStatesHasBeenSynced is not implemented func (f *FakeVolumeManager) ReconcilerStatesHasBeenSynced() bool { return true } // VolumeIsAttached is not implemented func (f *FakeVolumeManager) VolumeIsAttached(volumeName v1.UniqueVolumeName) bool { return false } // MarkVolumesAsReportedInUse adds the given volumes to the reportedInUse map func (f *FakeVolumeManager) MarkVolumesAsReportedInUse(volumesReportedAsInUse []v1.UniqueVolumeName) { for _, reportedVolume := range volumesReportedAsInUse { if _, ok := f.volumes[reportedVolume]; ok { f.reportedInUse[reportedVolume] = true } } } // GetVolumesReportedInUse is a test function only that returns a list of volumes // from the reportedInUse map func (f *FakeVolumeManager) GetVolumesReportedInUse() []v1.UniqueVolumeName { inuse := []v1.UniqueVolumeName{} for reportedVolume := range f.reportedInUse { inuse = append(inuse, reportedVolume) } return inuse } </ins>

<!-- Please only use this template for submitting reports about failing tests in Kubernetes CI jobs --Which jobs are failing: Which test(s) are failing: [sig-storage] CSI Volumes [Driver: csi-hostpath-v0] [Testpattern: Dynamic PV (default fs)] subPath should unmount if pod is gracefully deleted while kubelet is down [Disruptive][Slow] [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (default fs)] subPath should unmount if pod is gracefully deleted while kubelet is down [Disruptive][Slow] [sig-storage] CSI Volumes [Driver: ][Serial] [Testpattern: Dynamic PV (default fs)] subPath should unmount if pod is gracefully deleted while kubelet is down [Disruptive][Slow] Since when has it been failing: 2019-01-15 (when they were ) Testgrid link: Reason for failure: They all seem to be failing with: The failures seem to start with .
/sig storage
Summoning and This seems to have been failing since it was in master-blocking.
This is the same as
Thanks closing this one. /close
Closing this issue. <detailsIn response to : Thanks closing this one. Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen I took a look at the latest test failures and I think it is a different issue than
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Looking at one test failure here: Sequence of events: stopped, pod deleted: started again: volume to dsw started and failed because it's not in the node's volumesinuse list. This repeats for a while. also started, but we skipped cleaning up because the volume is in dsw so we want to let volume manager gracefully cleanup don't remove volume from dsw because actual state has not been updated? So it seems like there's a problem with populating the node's volumesinuse after a kubelet restart.
cc
/assign will continue investigating.

i will try repro-ing at without different commits to try to figure out which pr introduced the regression
/priority important-soon
I ran these tests against a cluster with the NodeLease feature disabled, and the tests pass. So I think the issue is related to the new NodeLease feature enabled a month ago: cc
I discussed with VolumesInUse is updated periodically in the Node.Status. So we need to make sure this flow is working well with the NodeLease feature.
and I walked through the code. The NodeLease feature introduced an early exit to the node status update function, so our theory is we need to add a call to . I will make the change and test it out


Question: rc.clientSet.AutoscalingV2().HorizontalPodAutoscalers(rc.nsName).Delete(context.TODO(), autoscalerName, metav1.DeleteOptions{}) } <del> func HPABehaviorWithDownscaleStabilizationSeconds(downscaleStabilizationSeconds int32) *autoscalingv2.HorizontalPodAutoscalerBehavior { return &autoscalingv2.HorizontalPodAutoscalerBehavior{ ScaleDown: &autoscalingv2.HPAScalingRules{ StabilizationWindowSeconds: &downscaleStabilizationSeconds, }, } } func HPABehaviorWithUpscaleStabilizationSeconds(upscaleStabilizationSeconds int32) *autoscalingv2.HorizontalPodAutoscalerBehavior { return &autoscalingv2.HorizontalPodAutoscalerBehavior{ ScaleUp: &autoscalingv2.HPAScalingRules{ StabilizationWindowSeconds: &upscaleStabilizationSeconds, }, } } </del> //SidecarStatusType type for sidecar status type SidecarStatusType bool

gce-cos-autoscaling-hpa-cpu Kubernetes e2e suite.[sig-autoscaling] [Feature:HPA] [Serial] [Slow] Horizontal pod autoscaling (non-default behavior) with long upscale stabilization window should scale up only after the stabilization period Since it was in (first run at 2022-06-14 13:13 CEST failed, all subsequent runs failed too). https://k8s- Times out when waiting for scale down (), logs from (): I'm not sure why exactly, I have 2 ideas: It looks like the consumer briefly consumes 440 mCPU, which could result in a higher recommendation being generated HPA stabilises initial size of the consumer Increasing timeout should fix the issue No response_ /sig autoscaling
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
FYI
This was already reverted (and later the test got fixed and merged). /close
Closing this issue. <detailsIn response to : This was already reverted (and later the test got fixed and merged). Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: } } <del> func TestExemptClusterScopedResource(t *testing.T) { </del> <ins> func TestNotExemptClusterScopedResource(t *testing.T) { </ins> hook := &registrationv1beta1.Webhook{ NamespaceSelector: &metav1.LabelSelector{}, }

If I am not mistaken, means that cluster-scoped resources other than namespaces are broken with admission webhooks. There is even a unit test checking that:
To clear any ambiguity around the word "broken", the current code behavior is that most cluster-scoped resources are not subjected to any webhooks. I wrote them like this because I was concerned with the potential bootstrapping deadlocks: a webhook that enacts policies for cluster-scoped resources might cause bootstrapping deadlocks for the cluster or for its own deployment. Thinking out loud, perhaps for cluster-scoped resources, the apiserver should only call the webhooks if the resources' labels match the , so that the critical bootstrapping resources can be excluded from the webhooks, thus breaking the bootstrapping deadlocks. This aligns with what we do for (see ). The downside is that the field name "namespaceSelector" becomes inappropriate.
I think we still need namespaceSelectors. We don't want to label pods. But I agree we actually want another objectLabelSelector. For namespaces these two coincide, i.e. we only have to define how they are applied if both are set or we forbid both at the same time. I tend to the later.
raised up a security issue regarding . A user can change labels to skip webhooks. doesn't have this issue because a user usually doesn't have permission to change the namespace's labels.
Users normally cannot create cluster-scoped objects either. But techincally it's a valid point. On the other hand, my main use-case are CRDs. They are not critical to the cluster health (maybe self-hosted clusters are different with etcd operator), i.e. you normally cannot shoot yourself into the foot with a webhook on them. For CRDs we do not need a selector at all. In other words: IMO we should enable webhooks for cluster-scoped objects, even if we have no in place.
Sounds reasonable. I'll create a PR ()
We would like to have the ability to apply the admission controller on specific namespaces only, and not on cluster level resources. This is useful if the list of possible resource Kinds that we want to apply the controller for is large, and we want to limit when it runs as much as possible to avoid affecting the whole cluster in case there's an issue.

I think this can be done by adding a SkipClusterResources flag (or something similar) in addition to the NamespaceSelector, to the Webhook struct. What do you think?
I think adding a flag is reasonable. It doesn't have the security issue as the idea does (). The expressiveness of is limited compared to , e..g., admins cannot use to run webhooks on some cluster-level resources while skipping others. However it's not a big drawback, because admins can specify more detailed GVK to limit the scope of webhook in that case.
Zeroth thought: s/SkipClusterResources/scope=[clusterboth]/g First thought: can't you just exclude cluster scoped resources, or include only namespace scoped resources in the webhook definition? Second thought: can a namespace selector be crafted that excludes the empty string?
0: Are you sure that's cleaner? The namespace selector picks which namespaces to use, and skipCluster decides whether to include cluster resources or not. Not setting it maintains current behavior (skipCluster=false) and whether cluster level resources are included is very clear. scope=cluster/namespace/both will require us to default to "both" which I find slightly more confusing. Also what to do if I specify cluster AND include namespace selector - is that an invalid spec or do we ignore the selector? 1: Yes. This is the current best thing we can do. If the list of resources is large though (for example, if I am implementing an alternative resource quota controller) then this solution requires us to maintain and update a long list of resources. This may be ok, since as long as our controller can't support certain types of resources it doesn't make sense to watch for them... 2: Doesn't look like it - the current code specifically checks if the attribute is a cluster level resource and skips the entire label matching process.
Re: 0: fair point, but let's at least name it in the positive: matchClusterScopedResources=true|false or something like that.
I proposed it in the negative to make it backwards compatible, so that "false" keeps current behavior. I agree that negatively named booleans are not not bad. I'll treat it as true if unspecified and use matchClusterScopedResources.
In today's semantic, such a namespace selector will exclude namespaced objects whose namespace has no label.


Question: err: fmt.Errorf("Unable to create gce instance with running docker daemon for image %s.  %v", image, err), } } <del> return testHost(host, archive) </del> <ins> return testHost(host, archive, false) </ins> } // Provision a gce instance using image

Looks like it maybe the ssh connect is getting killed
The relevant log snippet:
And another one, not just coreos.


Question: id: fluentd-to-gcp containers: - name: fluentd-gcp-container <del> image: kubernetes/fluentd-gcp:1.0 </del> <ins> image: kubernetes/fluentd-gcp:1.1 </ins> volumeMounts: - name: containers mountPath: /var/lib/docker/containers

As Satnam mentioned in PR , our Fluentd pod tends to not collect the first chunk of logs from each container. I've noticed that it seems to miss different amounts of logs depending on how quickly the container generates them, so it may be a timing issue. cc
See


Question: expectErrorPod:   podWithID("foo", testNode.Name), expectForgetPod:  podWithID("foo", testNode.Name), eventReason:      "FailedScheduling", <del> }, { </del> <ins> }, { </ins> name:        "deleting pod", sendPod:     deletingPod("foo"), algo:        mockScheduler{core.ScheduleResult{}, nil},

This is a bug found in The scheduler fails to schedule the pod on the first try due to PV is out of sync. This may happen because the PV/PVC may be modified before the scheduler binder assumes them. However, the scheduler skips the pod update event and never try to scheduler the pod again: This is because when the reserve plugins fails, we didn't try to forget the pod. After the is called, the pod is in the state.
/sig scheduling /sig storage /assign /priority critical-urgent
The label(s) cannot be applied, because the repository doesn't have them <detailsIn response to : /sig scheduling /sig storage /assign /priority critical-urgent Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: # # storage.backlog.mem_limit 5M [INPUT] Name         winlog Interval_Sec 2 # Channels Setup,Windows PowerShell Channels     application,system,security Tag          winevent.raw <del> DB           winlog.sqlite   # </del> <ins> DB           /var/run/google-fluentbit/pos-files/winlog.db </ins> # Json Log Example: # {"log":"[info:2016-02-16T16:04:05.930-08:00] Some log text heren","stream":"stdout","time":"2016-02-17T00:04:05.931087621Z"}

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --The default fluent-bit configuration for GCE Windows nodes produces unparsed JSON logs and does not forward container logs. Container log output is produced. fluent-bit on a given windows image. a GCE Windows e2e cluster and view the stackdriver output for pods. Kubernetes version (use ): 1.20+ Cloud provider or hardware configuration: GKE/GCE OS (e.g: ): Windows Kernel (e.g. ): Any version Install tools: fluent-bit Network plugin and version (if this is a network-related bug): Others:
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig windows


Question: "//staging:all-srcs", "//test:all-srcs", "//third_party:all-srcs", <del> "//vendor/bitbucket.org/bertimus9/systemstat:all-srcs", "//vendor/bitbucket.org/ww/goautoneg:all-srcs", "//vendor/cloud.google.com/go/compute/metadata:all-srcs", "//vendor/cloud.google.com/go/internal:all-srcs", "//vendor/github.com/Azure/azure-sdk-for-go/arm/compute:all-srcs", "//vendor/github.com/Azure/azure-sdk-for-go/arm/containerregistry:all-srcs", "//vendor/github.com/Azure/azure-sdk-for-go/arm/disk:all-srcs", "//vendor/github.com/Azure/azure-sdk-for-go/arm/network:all-srcs", "//vendor/github.com/Azure/azure-sdk-for-go/arm/storage:all-srcs", "//vendor/github.com/Azure/azure-sdk-for-go/storage:all-srcs", "//vendor/github.com/Azure/go-ansiterm:all-srcs", "//vendor/github.com/Azure/go-autorest/autorest:all-srcs", "//vendor/github.com/MakeNowJust/heredoc:all-srcs", "//vendor/github.com/Microsoft/go-winio:all-srcs", "//vendor/github.com/NYTimes/gziphandler:all-srcs", "//vendor/github.com/PuerkitoBio/purell:all-srcs", "//vendor/github.com/PuerkitoBio/urlesc:all-srcs", "//vendor/github.com/Sirupsen/logrus:all-srcs", "//vendor/github.com/abbot/go-http-auth:all-srcs", "//vendor/github.com/appc/spec/schema:all-srcs", "//vendor/github.com/armon/circbuf:all-srcs", "//vendor/github.com/asaskevich/govalidator:all-srcs", "//vendor/github.com/aws/aws-sdk-go/aws:all-srcs", "//vendor/github.com/aws/aws-sdk-go/private/protocol:all-srcs", "//vendor/github.com/aws/aws-sdk-go/private/waiter:all-srcs", "//vendor/github.com/aws/aws-sdk-go/service/autoscaling:all-srcs", "//vendor/github.com/aws/aws-sdk-go/service/ec2:all-srcs", "//vendor/github.com/aws/aws-sdk-go/service/ecr:all-srcs", "//vendor/github.com/aws/aws-sdk-go/service/elb:all-srcs", "//vendor/github.com/aws/aws-sdk-go/service/route53:all-srcs", "//vendor/github.com/aws/aws-sdk-go/service/sts:all-srcs", "//vendor/github.com/beorn7/perks/quantile:all-srcs", "//vendor/github.com/blang/semver:all-srcs", "//vendor/github.com/boltdb/bolt:all-srcs", "//vendor/github.com/chai2010/gettext-go/gettext:all-srcs", "//vendor/github.com/cloudflare/cfssl/auth:all-srcs", "//vendor/github.com/cloudflare/cfssl/certdb:all-srcs", "//vendor/github.com/cloudflare/cfssl/config:all-srcs", "//vendor/github.com/cloudflare/cfssl/crypto/pkcs7:all-srcs", "//vendor/github.com/cloudflare/cfssl/csr:all-srcs", "//vendor/github.com/cloudflare/cfssl/errors:all-srcs", "//vendor/github.com/cloudflare/cfssl/helpers:all-srcs", "//vendor/github.com/cloudflare/cfssl/info:all-srcs", "//vendor/github.com/cloudflare/cfssl/log:all-srcs", "//vendor/github.com/cloudflare/cfssl/ocsp/config:all-srcs", "//vendor/github.com/cloudflare/cfssl/signer:all-srcs", "//vendor/github.com/clusterhq/flocker-go:all-srcs", "//vendor/github.com/codedellemc/goscaleio:all-srcs", "//vendor/github.com/codegangsta/negroni:all-srcs", "//vendor/github.com/containernetworking/cni/libcni:all-srcs", "//vendor/github.com/containernetworking/cni/pkg/invoke:all-srcs", "//vendor/github.com/containernetworking/cni/pkg/types:all-srcs", "//vendor/github.com/containernetworking/cni/pkg/version:all-srcs", "//vendor/github.com/coreos/etcd/alarm:all-srcs", "//vendor/github.com/coreos/etcd/auth:all-srcs", "//vendor/github.com/coreos/etcd/client:all-srcs", "//vendor/github.com/coreos/etcd/clientv3:all-srcs", "//vendor/github.com/coreos/etcd/compactor:all-srcs", "//vendor/github.com/coreos/etcd/discovery:all-srcs", "//vendor/github.com/coreos/etcd/error:all-srcs", "//vendor/github.com/coreos/etcd/etcdserver:all-srcs", "//vendor/github.com/coreos/etcd/integration:all-srcs", "//vendor/github.com/coreos/etcd/lease:all-srcs", "//vendor/github.com/coreos/etcd/mvcc:all-srcs", "//vendor/github.com/coreos/etcd/pkg/adt:all-srcs", "//vendor/github.com/coreos/etcd/pkg/contention:all-srcs", "//vendor/github.com/coreos/etcd/pkg/cpuutil:all-srcs", "//vendor/github.com/coreos/etcd/pkg/crc:all-srcs", "//vendor/github.com/coreos/etcd/pkg/fileutil:all-srcs", "//vendor/github.com/coreos/etcd/pkg/httputil:all-srcs", "//vendor/github.com/coreos/etcd/pkg/idutil:all-srcs", "//vendor/github.com/coreos/etcd/pkg/ioutil:all-srcs", "//vendor/github.com/coreos/etcd/pkg/logutil:all-srcs", "//vendor/github.com/coreos/etcd/pkg/monotime:all-srcs", "//vendor/github.com/coreos/etcd/pkg/netutil:all-srcs", "//vendor/github.com/coreos/etcd/pkg/pathutil:all-srcs", "//vendor/github.com/coreos/etcd/pkg/pbutil:all-srcs", "//vendor/github.com/coreos/etcd/pkg/runtime:all-srcs", "//vendor/github.com/coreos/etcd/pkg/schedule:all-srcs", "//vendor/github.com/coreos/etcd/pkg/testutil:all-srcs", "//vendor/github.com/coreos/etcd/pkg/tlsutil:all-srcs", "//vendor/github.com/coreos/etcd/pkg/transport:all-srcs", "//vendor/github.com/coreos/etcd/pkg/types:all-srcs", "//vendor/github.com/coreos/etcd/pkg/wait:all-srcs", "//vendor/github.com/coreos/etcd/proxy/grpcproxy:all-srcs", "//vendor/github.com/coreos/etcd/raft:all-srcs", "//vendor/github.com/coreos/etcd/rafthttp:all-srcs", "//vendor/github.com/coreos/etcd/snap:all-srcs", "//vendor/github.com/coreos/etcd/store:all-srcs", "//vendor/github.com/coreos/etcd/version:all-srcs", "//vendor/github.com/coreos/etcd/wal:all-srcs", "//vendor/github.com/coreos/go-oidc/http:all-srcs", "//vendor/github.com/coreos/go-oidc/jose:all-srcs", "//vendor/github.com/coreos/go-oidc/key:all-srcs", "//vendor/github.com/coreos/go-oidc/oauth2:all-srcs", "//vendor/github.com/coreos/go-oidc/oidc:all-srcs", "//vendor/github.com/coreos/go-semver/semver:all-srcs", "//vendor/github.com/coreos/go-systemd/daemon:all-srcs", "//vendor/github.com/coreos/go-systemd/dbus:all-srcs", "//vendor/github.com/coreos/go-systemd/journal:all-srcs", "//vendor/github.com/coreos/go-systemd/unit:all-srcs", "//vendor/github.com/coreos/go-systemd/util:all-srcs", "//vendor/github.com/coreos/pkg/capnslog:all-srcs", "//vendor/github.com/coreos/pkg/dlopen:all-srcs", "//vendor/github.com/coreos/pkg/health:all-srcs", "//vendor/github.com/coreos/pkg/httputil:all-srcs", "//vendor/github.com/coreos/pkg/timeutil:all-srcs", "//vendor/github.com/coreos/rkt/api/v1alpha:all-srcs", "//vendor/github.com/cpuguy83/go-md2man/md2man:all-srcs", "//vendor/github.com/davecgh/go-spew/spew:all-srcs", "//vendor/github.com/daviddengcn/go-colortext:all-srcs", "//vendor/github.com/dgrijalva/jwt-go:all-srcs", "//vendor/github.com/docker/distribution/digest:all-srcs", "//vendor/github.com/docker/distribution/reference:all-srcs", "//vendor/github.com/docker/docker/api/types:all-srcs", "//vendor/github.com/docker/docker/client:all-srcs", "//vendor/github.com/docker/docker/pkg/jsonlog:all-srcs", "//vendor/github.com/docker/docker/pkg/jsonmessage:all-srcs", "//vendor/github.com/docker/docker/pkg/longpath:all-srcs", "//vendor/github.com/docker/docker/pkg/mount:all-srcs", "//vendor/github.com/docker/docker/pkg/stdcopy:all-srcs", "//vendor/github.com/docker/docker/pkg/symlink:all-srcs", "//vendor/github.com/docker/docker/pkg/system:all-srcs", "//vendor/github.com/docker/docker/pkg/term:all-srcs", "//vendor/github.com/docker/docker/pkg/tlsconfig:all-srcs", "//vendor/github.com/docker/engine-api/client:all-srcs", "//vendor/github.com/docker/engine-api/types:all-srcs", "//vendor/github.com/docker/go-connections/nat:all-srcs", "//vendor/github.com/docker/go-connections/sockets:all-srcs", "//vendor/github.com/docker/go-connections/tlsconfig:all-srcs", "//vendor/github.com/docker/go-units:all-srcs", "//vendor/github.com/docker/spdystream:all-srcs", "//vendor/github.com/elazarl/go-bindata-assetfs:all-srcs", "//vendor/github.com/elazarl/goproxy:all-srcs", "//vendor/github.com/emicklei/go-restful:all-srcs", "//vendor/github.com/emicklei/go-restful-swagger12:all-srcs", "//vendor/github.com/evanphx/json-patch:all-srcs", "//vendor/github.com/exponent-io/jsonpath:all-srcs", "//vendor/github.com/fatih/camelcase:all-srcs", "//vendor/github.com/fsnotify/fsnotify:all-srcs", "//vendor/github.com/garyburd/redigo/internal:all-srcs", "//vendor/github.com/garyburd/redigo/redis:all-srcs", "//vendor/github.com/ghodss/yaml:all-srcs", "//vendor/github.com/go-ini/ini:all-srcs", "//vendor/github.com/go-openapi/analysis:all-srcs", "//vendor/github.com/go-openapi/errors:all-srcs", "//vendor/github.com/go-openapi/jsonpointer:all-srcs", "//vendor/github.com/go-openapi/jsonreference:all-srcs", "//vendor/github.com/go-openapi/loads:all-srcs", "//vendor/github.com/go-openapi/runtime:all-srcs", "//vendor/github.com/go-openapi/spec:all-srcs", "//vendor/github.com/go-openapi/strfmt:all-srcs", "//vendor/github.com/go-openapi/swag:all-srcs", "//vendor/github.com/go-openapi/validate:all-srcs", "//vendor/github.com/godbus/dbus:all-srcs", "//vendor/github.com/gogo/protobuf/gogoproto:all-srcs", "//vendor/github.com/gogo/protobuf/plugin/compare:all-srcs", "//vendor/github.com/gogo/protobuf/plugin/defaultcheck:all-srcs", "//vendor/github.com/gogo/protobuf/plugin/description:all-srcs", "//vendor/github.com/gogo/protobuf/plugin/embedcheck:all-srcs", "//vendor/github.com/gogo/protobuf/plugin/enumstringer:all-srcs", "//vendor/github.com/gogo/protobuf/plugin/equal:all-srcs", "//vendor/github.com/gogo/protobuf/plugin/face:all-srcs", "//vendor/github.com/gogo/protobuf/plugin/gostring:all-srcs", "//vendor/github.com/gogo/protobuf/plugin/marshalto:all-srcs", "//vendor/github.com/gogo/protobuf/plugin/oneofcheck:all-srcs", "//vendor/github.com/gogo/protobuf/plugin/populate:all-srcs", "//vendor/github.com/gogo/protobuf/plugin/size:all-srcs", "//vendor/github.com/gogo/protobuf/plugin/stringer:all-srcs", "//vendor/github.com/gogo/protobuf/plugin/testgen:all-srcs", "//vendor/github.com/gogo/protobuf/plugin/union:all-srcs", "//vendor/github.com/gogo/protobuf/plugin/unmarshal:all-srcs", "//vendor/github.com/gogo/protobuf/proto:all-srcs", "//vendor/github.com/gogo/protobuf/protoc-gen-gogo/descriptor:all-srcs", "//vendor/github.com/gogo/protobuf/protoc-gen-gogo/generator:all-srcs", "//vendor/github.com/gogo/protobuf/protoc-gen-gogo/grpc:all-srcs", "//vendor/github.com/gogo/protobuf/protoc-gen-gogo/plugin:all-srcs", "//vendor/github.com/gogo/protobuf/sortkeys:all-srcs", "//vendor/github.com/gogo/protobuf/vanity:all-srcs", "//vendor/github.com/golang/glog:all-srcs", "//vendor/github.com/golang/groupcache/lru:all-srcs", "//vendor/github.com/golang/mock/gomock:all-srcs", "//vendor/github.com/golang/protobuf/jsonpb:all-srcs", "//vendor/github.com/golang/protobuf/proto:all-srcs", "//vendor/github.com/golang/protobuf/ptypes:all-srcs", "//vendor/github.com/google/btree:all-srcs", "//vendor/github.com/google/cadvisor/api:all-srcs", "//vendor/github.com/google/cadvisor/cache/memory:all-srcs", "//vendor/github.com/google/cadvisor/client/v2:all-srcs", "//vendor/github.com/google/cadvisor/collector:all-srcs", "//vendor/github.com/google/cadvisor/container:all-srcs", "//vendor/github.com/google/cadvisor/devicemapper:all-srcs", "//vendor/github.com/google/cadvisor/events:all-srcs", "//vendor/github.com/google/cadvisor/fs:all-srcs", "//vendor/github.com/google/cadvisor/healthz:all-srcs", "//vendor/github.com/google/cadvisor/http:all-srcs", "//vendor/github.com/google/cadvisor/info/v1:all-srcs", "//vendor/github.com/google/cadvisor/info/v2:all-srcs", "//vendor/github.com/google/cadvisor/machine:all-srcs", "//vendor/github.com/google/cadvisor/manager:all-srcs", "//vendor/github.com/google/cadvisor/metrics:all-srcs", "//vendor/github.com/google/cadvisor/pages:all-srcs", "//vendor/github.com/google/cadvisor/storage:all-srcs", "//vendor/github.com/google/cadvisor/summary:all-srcs", "//vendor/github.com/google/cadvisor/utils:all-srcs", "//vendor/github.com/google/cadvisor/validate:all-srcs", "//vendor/github.com/google/cadvisor/version:all-srcs", "//vendor/github.com/google/cadvisor/zfs:all-srcs", "//vendor/github.com/google/certificate-transparency/go:all-srcs", "//vendor/github.com/google/gofuzz:all-srcs", "//vendor/github.com/googleapis/gnostic/OpenAPIv2:all-srcs", "//vendor/github.com/googleapis/gnostic/compiler:all-srcs", "//vendor/github.com/googleapis/gnostic/extensions:all-srcs", "//vendor/github.com/gophercloud/gophercloud:all-srcs", "//vendor/github.com/gorilla/context:all-srcs", "//vendor/github.com/gorilla/mux:all-srcs", "//vendor/github.com/gorilla/websocket:all-srcs", "//vendor/github.com/grpc-ecosystem/go-grpc-prometheus:all-srcs", "//vendor/github.com/grpc-ecosystem/grpc-gateway/runtime:all-srcs", "//vendor/github.com/grpc-ecosystem/grpc-gateway/utilities:all-srcs", "//vendor/github.com/hashicorp/golang-lru:all-srcs", "//vendor/github.com/hashicorp/hcl:all-srcs", "//vendor/github.com/hawkular/hawkular-client-go/metrics:all-srcs", "//vendor/github.com/heketi/heketi/client/api/go-client:all-srcs", "//vendor/github.com/heketi/heketi/pkg/glusterfs/api:all-srcs", "//vendor/github.com/heketi/heketi/pkg/utils:all-srcs", "//vendor/github.com/howeyc/gopass:all-srcs", "//vendor/github.com/imdario/mergo:all-srcs", "//vendor/github.com/inconshreveable/mousetrap:all-srcs", "//vendor/github.com/influxdata/influxdb/client:all-srcs", "//vendor/github.com/influxdata/influxdb/models:all-srcs", "//vendor/github.com/influxdata/influxdb/pkg/escape:all-srcs", "//vendor/github.com/jmespath/go-jmespath:all-srcs", "//vendor/github.com/jonboulle/clockwork:all-srcs", "//vendor/github.com/jteeuwen/go-bindata:all-srcs", "//vendor/github.com/juju/ratelimit:all-srcs", "//vendor/github.com/kardianos/osext:all-srcs", "//vendor/github.com/karlseguin/ccache:all-srcs", "//vendor/github.com/kr/fs:all-srcs", "//vendor/github.com/kr/pty:all-srcs", "//vendor/github.com/libopenstorage/openstorage/api:all-srcs", "//vendor/github.com/libopenstorage/openstorage/pkg/units:all-srcs", "//vendor/github.com/libopenstorage/openstorage/volume:all-srcs", "//vendor/github.com/lpabon/godbc:all-srcs", "//vendor/github.com/magiconair/properties:all-srcs", "//vendor/github.com/mailru/easyjson/buffer:all-srcs", "//vendor/github.com/mailru/easyjson/jlexer:all-srcs", "//vendor/github.com/mailru/easyjson/jwriter:all-srcs", "//vendor/github.com/matttproud/golang_protobuf_extensions/pbutil:all-srcs", "//vendor/github.com/miekg/coredns/middleware/etcd/msg:all-srcs", "//vendor/github.com/miekg/dns:all-srcs", "//vendor/github.com/mistifyio/go-zfs:all-srcs", "//vendor/github.com/mitchellh/go-wordwrap:all-srcs", "//vendor/github.com/mitchellh/mapstructure:all-srcs", "//vendor/github.com/mreiferson/go-httpclient:all-srcs", "//vendor/github.com/mvdan/xurls:all-srcs", "//vendor/github.com/mxk/go-flowrate/flowrate:all-srcs", "//vendor/github.com/onsi/ginkgo:all-srcs", "//vendor/github.com/onsi/gomega:all-srcs", "//vendor/github.com/opencontainers/runc/libcontainer:all-srcs", "//vendor/github.com/pborman/uuid:all-srcs", "//vendor/github.com/pelletier/go-buffruneio:all-srcs", "//vendor/github.com/pelletier/go-toml:all-srcs", "//vendor/github.com/pkg/errors:all-srcs", "//vendor/github.com/pkg/sftp:all-srcs", "//vendor/github.com/pmezard/go-difflib/difflib:all-srcs", "//vendor/github.com/prometheus/client_golang/prometheus:all-srcs", "//vendor/github.com/prometheus/client_model/go:all-srcs", "//vendor/github.com/prometheus/common/expfmt:all-srcs", "//vendor/github.com/prometheus/common/internal/bitbucket.org/ww/goautoneg:all-srcs", "//vendor/github.com/prometheus/common/model:all-srcs", "//vendor/github.com/prometheus/procfs:all-srcs", "//vendor/github.com/quobyte/api:all-srcs", "//vendor/github.com/rackspace/gophercloud:all-srcs", "//vendor/github.com/rancher/go-rancher/client:all-srcs", "//vendor/github.com/renstrom/dedent:all-srcs", "//vendor/github.com/robfig/cron:all-srcs", "//vendor/github.com/rubiojr/go-vhd/vhd:all-srcs", "//vendor/github.com/russross/blackfriday:all-srcs", "//vendor/github.com/satori/uuid:all-srcs", "//vendor/github.com/seccomp/libseccomp-golang:all-srcs", "//vendor/github.com/shurcooL/sanitized_anchor_name:all-srcs", "//vendor/github.com/spf13/afero:all-srcs", "//vendor/github.com/spf13/cast:all-srcs", "//vendor/github.com/spf13/cobra:all-srcs", "//vendor/github.com/spf13/jwalterweatherman:all-srcs", "//vendor/github.com/spf13/pflag:all-srcs", "//vendor/github.com/spf13/viper:all-srcs", "//vendor/github.com/square/go-jose:all-srcs", "//vendor/github.com/storageos/go-api:all-srcs", "//vendor/github.com/stretchr/objx:all-srcs", "//vendor/github.com/stretchr/testify/assert:all-srcs", "//vendor/github.com/stretchr/testify/mock:all-srcs", "//vendor/github.com/stretchr/testify/require:all-srcs", "//vendor/github.com/syndtr/gocapability/capability:all-srcs", "//vendor/github.com/ugorji/go/codec:all-srcs", "//vendor/github.com/vishvananda/netlink:all-srcs", "//vendor/github.com/vmware/govmomi:all-srcs", "//vendor/github.com/vmware/photon-controller-go-sdk/SSPI:all-srcs", "//vendor/github.com/vmware/photon-controller-go-sdk/photon:all-srcs", "//vendor/github.com/xanzy/go-cloudstack/cloudstack:all-srcs", "//vendor/github.com/xiang90/probing:all-srcs", "//vendor/github.com/xyproto/simpleredis:all-srcs", "//vendor/go.pedge.io/pb/go/google/protobuf:all-srcs", "//vendor/go4.org/errorutil:all-srcs", "//vendor/golang.org/x/crypto/bcrypt:all-srcs", "//vendor/golang.org/x/crypto/blowfish:all-srcs", "//vendor/golang.org/x/crypto/curve25519:all-srcs", "//vendor/golang.org/x/crypto/ed25519:all-srcs", "//vendor/golang.org/x/crypto/nacl/secretbox:all-srcs", "//vendor/golang.org/x/crypto/pkcs12:all-srcs", "//vendor/golang.org/x/crypto/poly1305:all-srcs", "//vendor/golang.org/x/crypto/salsa20/salsa:all-srcs", "//vendor/golang.org/x/crypto/ssh:all-srcs", "//vendor/golang.org/x/exp/inotify:all-srcs", "//vendor/golang.org/x/net/context:all-srcs", "//vendor/golang.org/x/net/html:all-srcs", "//vendor/golang.org/x/net/http2:all-srcs", "//vendor/golang.org/x/net/idna:all-srcs", "//vendor/golang.org/x/net/internal/timeseries:all-srcs", "//vendor/golang.org/x/net/lex/httplex:all-srcs", "//vendor/golang.org/x/net/proxy:all-srcs", "//vendor/golang.org/x/net/trace:all-srcs", "//vendor/golang.org/x/net/websocket:all-srcs", "//vendor/golang.org/x/oauth2:all-srcs", "//vendor/golang.org/x/sys/unix:all-srcs", "//vendor/golang.org/x/sys/windows:all-srcs", "//vendor/golang.org/x/text/cases:all-srcs", "//vendor/golang.org/x/text/encoding:all-srcs", "//vendor/golang.org/x/text/internal:all-srcs", "//vendor/golang.org/x/text/language:all-srcs", "//vendor/golang.org/x/text/runes:all-srcs", "//vendor/golang.org/x/text/secure/bidirule:all-srcs", "//vendor/golang.org/x/text/secure/precis:all-srcs", "//vendor/golang.org/x/text/transform:all-srcs", "//vendor/golang.org/x/text/unicode/bidi:all-srcs", "//vendor/golang.org/x/text/unicode/norm:all-srcs", "//vendor/golang.org/x/text/width:all-srcs", "//vendor/golang.org/x/time/rate:all-srcs", "//vendor/golang.org/x/tools/container/intsets:all-srcs", "//vendor/google.golang.org/api/cloudkms/v1:all-srcs", "//vendor/google.golang.org/api/cloudmonitoring/v2beta2:all-srcs", "//vendor/google.golang.org/api/compute/v0.alpha:all-srcs", "//vendor/google.golang.org/api/compute/v0.beta:all-srcs", "//vendor/google.golang.org/api/compute/v1:all-srcs", "//vendor/google.golang.org/api/container/v1:all-srcs", "//vendor/google.golang.org/api/dns/v1:all-srcs", "//vendor/google.golang.org/api/gensupport:all-srcs", "//vendor/google.golang.org/api/googleapi:all-srcs", "//vendor/google.golang.org/api/logging/v2beta1:all-srcs", "//vendor/google.golang.org/api/monitoring/v3:all-srcs", "//vendor/google.golang.org/api/pubsub/v1:all-srcs", "//vendor/google.golang.org/genproto/googleapis/rpc/status:all-srcs", "//vendor/google.golang.org/grpc:all-srcs", "//vendor/gopkg.in/gcfg.v1:all-srcs", "//vendor/gopkg.in/inf.v0:all-srcs", "//vendor/gopkg.in/natefinch/lumberjack.v2:all-srcs", "//vendor/gopkg.in/warnings.v0:all-srcs", "//vendor/gopkg.in/yaml.v2:all-srcs", "//vendor/k8s.io/gengo/args:all-srcs", "//vendor/k8s.io/gengo/examples/deepcopy-gen/generators:all-srcs", "//vendor/k8s.io/gengo/examples/defaulter-gen/generators:all-srcs", "//vendor/k8s.io/gengo/examples/import-boss/generators:all-srcs", "//vendor/k8s.io/gengo/examples/set-gen/generators:all-srcs", "//vendor/k8s.io/gengo/examples/set-gen/sets:all-srcs", "//vendor/k8s.io/gengo/generator:all-srcs", "//vendor/k8s.io/gengo/namer:all-srcs", "//vendor/k8s.io/gengo/parser:all-srcs", "//vendor/k8s.io/gengo/types:all-srcs", "//vendor/k8s.io/heapster/metrics/api/v1/types:all-srcs", "//vendor/k8s.io/kube-openapi/pkg/aggregator:all-srcs", "//vendor/k8s.io/kube-openapi/pkg/builder:all-srcs", "//vendor/k8s.io/kube-openapi/pkg/common:all-srcs", "//vendor/k8s.io/kube-openapi/pkg/generators:all-srcs", "//vendor/k8s.io/kube-openapi/pkg/handler:all-srcs", "//vendor/k8s.io/kube-openapi/pkg/util:all-srcs", "//vendor/k8s.io/utils/exec:all-srcs", "//vendor/vbom.ml/util/sortorder:all-srcs", </del> <ins> "//vendor:all-srcs", </ins> ], tags = ["automanaged"], )

At this point I don't know if it is a valid file or not?
There are no sig labels on this issue. Please by: a sig: e.g., to notify the contributor experience sig, OR the label manually: e.g., to apply the label Note: Method 1 will trigger an email to the group. You can find the group list and label list . The in the method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals
I reproduce it on my env. After removing , will not create and update ; Manually creating an empty file , will successfully rewrite and update ; /cc Seems file will not be created automatically if missing.
This is working as intended, or at least as designed. We only create new files if there are go sources in that directory, and doesn't have any go sources, so won't be auto-created. We have a solely for the and rules, but those are only managed in existing files. Without , we'd have a lot of lines in , which would be painful. I learned last week that the say to remove all of , which mucks with this process a little bit. I a step in the instructions there to recreate the file, but maybe should just be doing that? I'm not super excited about adding special cases there, but is kinda weird, so maybe it's unavoidable.
the more automatic the better
Specifically, this would cause many merge conflicts.
I'm a little confused, since this has been the way things have worked since merged in April, but 4+ people asking me about this in the last week have convinced me that this needs to be more automatic. Sent .


Question: t.Errorf("Expected publicIP resource exists, when it is not an internal service") } <del> if publicIP.Tags == nil || publicIP.Tags["service"] == nil { t.Errorf("Expected publicIP resource has tags[service]") </del> <ins> if publicIP.Tags == nil || publicIP.Tags[serviceTagKey] == nil { t.Errorf("Expected publicIP resource has tags[%s]", serviceTagKey) </ins> } serviceName := getServiceName(service) <del> if serviceName != *(publicIP.Tags["service"]) { t.Errorf("Expected publicIP resource has matching tags[service]") </del> <ins> if serviceName != *(publicIP.Tags[serviceTagKey]) { t.Errorf("Expected publicIP resource has matching tags[%s]", serviceTagKey) </ins> } <ins> if publicIP.Tags[clusterNameKey] == nil { t.Errorf("Expected publicIP resource has tags[%s]", clusterNameKey) } if *(publicIP.Tags[clusterNameKey]) != testClusterName { t.Errorf("Expected publicIP resource has matching tags[%s]", clusterNameKey) } </ins> // We cannot use service.Spec.LoadBalancerIP to compare with // Public IP's IPAddress // Because service properties are updated outside of cloudprovider code

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: We deployed two Kubernetes clusters to the same Azure Resource Group. Created services of load balancer type on both clusters. One of the services never received a public IP. Upon investigation, we noticed that the first Kubernetes cluster deleted the Azure Public IP that was created for the service by the other cluster. Apparently, all public IPs in the resource group are retrieved: then public IPs which don't match this criteria: are deleted. This logic besides the name of the kubernetes service should also consider the K8s cluster where the public IP originated from to avoid stepping on each other's toes when multiple clusters share the same resource group. What you expected to happen: In case multiple clusters run in the same resource group a cluster should delete only resources created by itself. How to reproduce it (as minimally and precisely as possible): Create two Kubernetes clusters (1.14) with different cluster names to the same resource group than create kubernetes services of load balancer type in each cluster. Anything else we need to know?: Environment: Kubernetes version (use ): 1.4 Cloud provider or hardware configuration: Azure OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
Reiterating the mentions to trigger a notification: <detailsIn response to : Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
thanks for reporting the issue, this is the bug we should fix for sharing resource group. /sig azure /kind bug /assign


Question: return svc.Networks.Get(networkProjectID, networkID).Do() } <ins> // listSubnetworksOfNetwork returns a list of subnetworks for a particular region of a network. func listSubnetworksOfNetwork(svc *compute.Service, networkProjectID, networkID, region string) ([]*compute.Subnetwork, error) { var subnets []*compute.Subnetwork err := svc.Subnetworks.List(networkProjectID, region).Filter(fmt.Sprintf("network eq .*/%v$", networkID)).Pages(context.Background(), func(res *compute.SubnetworkList) error { subnets = append(subnets, res.Items...) return nil }) return subnets, err } </ins> // getProjectID returns the project's string ID given a project number or string func getProjectID(svc *compute.Service, projectNumberOrID string) (string, error) { proj, err := svc.Projects.Get(projectNumberOrID).Do()

/kind bug What happened: Continuation of . The fix to that bug only takes the first subnetwork in the correct region. However, if the user manual subnets in the region that the cluster exists, it's possible the cloudprovider could choose the wrong subnet. What you expected to happen: If no subnet is specified for a non-legacy network, always take the subnet which was automatically created. /assign /sig network /area platform/gce


Question: initialConfig.ShutdownGracePeriodCriticalPods = metav1.Duration{Duration: nodeShutdownGracePeriodCriticalPods} }) <ins> ginkgo.BeforeEach(func() { ginkgo.By("Wait for the node to be ready") waitForNodeReady() }) </ins> ginkgo.AfterEach(func() { ginkgo.By("Emitting Shutdown false signal; cancelling the shutdown") err := emitSignalPrepareForShutdown(false)

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --Testgrid link: Reason for failure:
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign /sig node


Question: if refreshToken == "" { return nil, fmt.Errorf("no refresh token in cfg: %s", cfgRefreshToken) } <ins> environment := ts.cfg[cfgEnvironment] if environment == "" { return nil, fmt.Errorf("no environment in cfg: %s", cfgEnvironment) } </ins> clientID := ts.cfg[cfgClientID] if clientID == "" { return nil, fmt.Errorf("no client ID in cfg: %s", cfgClientID)

What happened: AAD support in kubectl does not work for sovereign cloud The 'environment' parameter was introduced to support sovereign cloud, but it works in first round. The 'environment' value is not persisted, so follow-up requests will fail. What you expected to happen: 'environment' parameter persisted in kubectl config file. How to reproduce it (as minimally and precisely as possible): Use kubectl to connect to Azure sovereign cloud, make the token expire, and do auth. Anything else we need to know?: Environment: Kubernetes version (use ): 1.11 Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others: <!-- DO NOT EDIT BELOW THIS LINE --/kind bug
There are no sig labels on this issue. Please add a sig label by either: a sig: e.g., to notify the contributor experience sig, OR the label manually: e.g., to apply the label Note: Method 1 will trigger an email to the group. See the . The in method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: srcs = [ "attacher.go", "vsphere_volume.go", <ins> "vsphere_volume_block.go", </ins> "vsphere_volume_util.go", ], importpath = "k8s.io/kubernetes/pkg/volume/vsphere_volume", deps = [ "//pkg/cloudprovider/providers/vsphere:go_default_library", "//pkg/cloudprovider/providers/vsphere/vclib:go_default_library", <ins> "//pkg/features:go_default_library", </ins> "//pkg/util/keymutex:go_default_library", "//pkg/util/mount:go_default_library", "//pkg/util/strings:go_default_library", "//pkg/volume:go_default_library", "//pkg/volume/util:go_default_library", <ins> "//pkg/volume/util/volumepathhandler:go_default_library", </ins> "//staging/src/k8s.io/api/core/v1:go_default_library", "//staging/src/k8s.io/apimachinery/pkg/api/resource:go_default_library", "//staging/src/k8s.io/apimachinery/pkg/apis/meta/v1:go_default_library", "//staging/src/k8s.io/apimachinery/pkg/types:go_default_library", <ins> "//staging/src/k8s.io/apiserver/pkg/util/feature:go_default_library", </ins> "//staging/src/k8s.io/cloud-provider:go_default_library", "//vendor/github.com/golang/glog:go_default_library", ],

FEATURE REQUEST: vSphere volume plugin does not support volume device yet. Need to add volumeMode in PV spec as other plugin did. release 1.12
/sig storage


Question: service := obj.(*v1.Service) prevService := prevObj.(*v1.Service) if service == nil || prevService == nil { <del> utilruntime.HandleError(fmt.Errorf("onServiceUpdate() expected type v1.Endpoints, got %T, %T", prevObj, obj)) </del> <ins> utilruntime.HandleError(fmt.Errorf("onServiceUpdate() expected type v1.Service, got %T, %T", prevObj, obj)) </ins> return } if (service.Spec.Selector == nil) != (prevService.Spec.Selector == nil) {

with Selector is created is removed Endpoints and EndpointSlice controllers do not clean up Endpoint(Slice)s mirroring controller starts mirroring the Endpoints resource originally created by the Endpoints controller is back to the Service backing Service change created by mirroring controller still references original IP from step 4 At least one of the following: Endpoints and EndpointSlices to be cleaned up when the Service has the selector removed () mirrored EndpointSlice to be removed when the Service has a selector back See "What Happened" No response Most noticeable in Kubernetes 1.19+ since that's when kube-proxy starts reading from EndpointSlices by default. Any No response No response No response No response
/sig network /triage accepted /priority important-soon /cc
/assign