Question: // _bindata is a table, holding each asset generator, mapped to its name. var _bindata = map[string]func() (*asset, error){ <ins> "translations/OWNERS":                             translationsOwners, </ins> "translations/extract.py":                         translationsExtractPy, "translations/kubectl/OWNERS":                     translationsKubectlOwners, "translations/kubectl/de_DE/LC_MESSAGES/k8s.mo":   translationsKubectlDe_deLc_messagesK8sMo,

/assign /cc Please ensure that every sub directory of / contains an owners file. The following sub directories do not define them: The result of this is that any PR which changes anything in api, Godeps, staging, vendor, etc will always get assigned to one of you for both the review and approval.
I recall someone also mentioning that certain files or dirs may not need approvers files. If that would help, we can add that functionality (i've already started the work)
staging and api should be easy to find owners for. may have ideas for translations. The big one is Godeps, third_party, vendor. Those are just grunt work. And figuring out criteria for approving is hard as they are a lot of code and it is unclear that things are tested well or automatically. (I'm thinking things like libraries for accessing cloud APIs). Having a dedicated team/process there might help.
I'll open some proposals for staging and API.
Godep in particular needs to be watched for things like licenses, which I have mostly been doing.
Thanks Clayton for the staging and api additions! How about if we pick two of you at random and shove them in as owners for each of the remaining directories?
What's left: translations: the kubectl specific translations is being handled by sig-cli, but the rest of the files like README and test have been handled and approved mostly by thirdparty: this is a tricky mix of stuff like forked code.. mostly handled by the likes of or for approvals. Godep/vendor: These could probably be solved by having a process to follow, and then delegating to a team in OWNERSALIASES for both folders.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. Prevent issues from auto-closing with an comment. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Still relevant. /remove-lifecycle stale


Question: was configured with a subpath. release: v1.12 file: test/e2e/storage/subpath.go <ins> - testname: VolumeAttachment, lifecycle codename: '[sig-storage] VolumeAttachment Conformance should run through the lifecycle of a VolumeAttachment [Conformance]' description: Creating an initial VolumeAttachment MUST succeed. Reading the VolumeAttachment MUST succeed with with required name retrieved. Patching a VolumeAttachment MUST succeed with its new label found. Listing VolumeAttachment with a labelSelector MUST succeed with a single item retrieved. Deleting a VolumeAttachment MUST succeed and it MUST be confirmed. Creating a second VolumeAttachment MUST succeed. Updating the second VolumentAttachment with a new label MUST succeed with its new label found. Creating a third VolumeAttachment MUST succeed. Updating the third VolumentAttachment with a new label MUST succeed with its new label found. Deleting both VolumeAttachments via deleteCollection MUST succeed and it MUST be confirmed. release: v1.30 file: test/e2e/storage/volume_attachment.go </ins>

[x] two weeks soak end date : 14 Mar 2024 : According to following APIsnoop query, there are six VolumeAttachment endpoints that are untested. Using a number of existing e2e test practices a new has been created to provide future Conformance coverage for the 7 endpoints. The e2e logs for this test are listed below. This query shows the following VolumeAttachment endpoints are hit within a short period of running this e2e test. If a test with these calls gets merged, test coverage will go up by 7 points This test is also created with the goal of conformance promotion. /sig testing /sig architecture /area conformance
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
The Kubernetes project currently lacks enough contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue as fresh with Close this issue with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: if err != nil { return nil, err } <del> if existsPip { return &pip, nil } </del> serviceName := getServiceName(service) <del> if shouldPIPExisted { return nil, fmt.Errorf("PublicIP from annotation azure-pip-name=%s for service %s doesn't exist", pipName, serviceName) } pip.Name = to.StringPtr(pipName) pip.Location = to.StringPtr(az.Location) pip.PublicIPAddressPropertiesFormat = &network.PublicIPAddressPropertiesFormat{ PublicIPAllocationMethod: network.Static, </del> <ins> if existsPip { // return if pip exist and dns label is the same if getDomainNameLabel(&pip) == domainNameLabel { return &pip, nil } klog.V(2).Infof("ensurePublicIPExists for service(%s): pip(%s) - updating", serviceName, *pip.Name) if pip.PublicIPAddressPropertiesFormat == nil { pip.PublicIPAddressPropertiesFormat = &network.PublicIPAddressPropertiesFormat{ PublicIPAllocationMethod: network.Static, } } } else { if shouldPIPExisted { return nil, fmt.Errorf("PublicIP from annotation azure-pip-name=%s for service %s doesn't exist", pipName, serviceName) } pip.Name = to.StringPtr(pipName) pip.Location = to.StringPtr(az.Location) pip.PublicIPAddressPropertiesFormat = &network.PublicIPAddressPropertiesFormat{ PublicIPAllocationMethod: network.Static, } pip.Tags = map[string]*string{ serviceTagKey:  &serviceName, clusterNameKey: &clusterName, } if az.useStandardLoadBalancer() { pip.Sku = &network.PublicIPAddressSku{ Name: network.PublicIPAddressSkuNameStandard, } } klog.V(2).Infof("ensurePublicIPExists for service(%s): pip(%s) - creating", serviceName, *pip.Name) </ins> } <del> if len(domainNameLabel) > 0 { </del> <ins> if len(domainNameLabel) == 0 { pip.PublicIPAddressPropertiesFormat.DNSSettings = nil } else { </ins> pip.PublicIPAddressPropertiesFormat.DNSSettings = &network.PublicIPAddressDNSSettings{ DomainNameLabel: &domainNameLabel, } } <del> pip.Tags = map[string]*string{ serviceTagKey:  &serviceName, clusterNameKey: &clusterName, } if az.useStandardLoadBalancer() { pip.Sku = &network.PublicIPAddressSku{ Name: network.PublicIPAddressSkuNameStandard, } } klog.V(2).Infof("ensurePublicIPExists for service(%s): pip(%s) - creating", serviceName, *pip.Name) </del> klog.V(10).Infof("CreateOrUpdatePIP(%s, %q): start", pipResourceGroup, *pip.Name) err = az.CreateOrUpdatePIP(service, pipResourceGroup, pip) if err != nil { <del> klog.V(2).Infof("ensure(%s) abort backoff: pip(%s) - creating", serviceName, *pip.Name) </del> <ins> klog.V(2).Infof("ensure(%s) abort backoff: pip(%s)", serviceName, *pip.Name) </ins> return nil, err } klog.V(10).Infof("CreateOrUpdatePIP(%s, %q): end", pipResourceGroup, *pip.Name)

Is this a BUG REPORT or FEATURE REQUEST?: What happened: Azure will not update the DNS label of the public IP, when update the service only changing the annotation of the DNS label from empty. annotations: -annotations: "to-update" What you expected to happen: It should update the dns label of the assigned public IP How to reproduce it (as minimally and precisely as possible): It probably caused when applying function 'getPublicIPAddress' in This function returns a bool 'existsPip' to judge the public ip existence. When only the annotations changed, the existsPip still returns true. Then, the reference in line 553 and line 446 step into this bug. pip, existsPip, err := az.getPublicIPAddress(pipResourceGroup, pipName) if err != nil { return false, err } if !existsPip { return true, nil } Firstly, in line 554, it will make function 'isFrontendIPChanged' return false, and then will not run 'ensurePublicIP' in line 691. Secondly, even if 'ensurePublicIP' in line 691 runs, it will detect the public ip exists and make no update to the DNS label in line 446_. Environment: Kubernetes version (use ): Cloud provider or hardware configuration: Azure
/sig azure
/assign
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
-- what's the status on this?
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-testing, kubernetes/test-infra and/or . /close
Closing this issue. <detailsIn response to : Rotten issues close after 30d of inactivity.

Reopen the issue with . Mark the issue as fresh with . /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/open I will take a look at this
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-testing, kubernetes/test-infra and/or . /close
Closing this issue. <detailsIn response to : Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Still not fixed. /reopen /kind bug /priority important-soon /milestone v1.16
Reopened this issue. <detailsIn response to : Still not fixed. /kind bug /priority important-soon /milestone v1.16 Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig cloud-provider
i'm working on it.
/assign


Question: }) err := wait.Poll(5*time.Second, 3*time.Minute, func() (bool, error) { <del> if !podClient.PodIsReady(pod1.Name) { framework.Logf("Pod 1 not ready yet") return false, nil } if !podClient.PodIsReady(pod2.Name) { framework.Logf("Pod 2 not ready yet") return false, nil } </del> var err error pod1, err = podClient.Get(context.TODO(), pod1.Name, metav1.GetOptions{}) if err != nil { return false, err } <ins> if len(pod1.Status.PodIPs) == 0 { return false, nil } </ins> pod2, err = podClient.Get(context.TODO(), pod2.Name, metav1.GetOptions{}) if err != nil { return false, err } <ins> if len(pod2.Status.PodIPs) == 0 { return false, nil } </ins> return true, nil }) <del> framework.ExpectNoError(err) </del> <ins> framework.ExpectNoError(err, "timed out waiting for Pods to have IPs assigned") </ins> ginkgo.By("referencing a single matching pod") expectEndpointsAndSlices(cs, f.Namespace.Name, svc1, []*v1.Pod{pod1}, 1, 1, false)

Which jobs are flaking: ci-kubernetes-kind-e2e-parallel pr:pull-kubernetes-e2e-gce pr:pull-kubernetes-e2e-gce-ubuntu-containerd pr:pull-kubernetes-e2e-gce-ubuntu-containerd ci-kubernetes-kind-ipv6-e2e-parallel Which test(s) are flaking: Second flakiest test in the e2e job according to Testgrid link: Reason for failure: Anything else we need to know: /sig network /assign
needs investigation since we're switching kube-proxy to endpointslice in this release
/triage unresolved Comment when the issue is assessed and confirmed. I am a bot run by vllry.
made me think of , but that's in already.
It's possible it's a test timeout that's just too close, but we need verification of that checking the controller and apiserver logs to make sure there isn't a latent bug
Thanks for catching this! It looks like the timeouts are actually associated with waiting for 2 pods to be ready, not actually anything related to EndpointSlice functionality - timeout is here: Given that, it seems like the obvious solution would be to update the Services to set to and update this block to wait for the Pods to have IPs, not for them to be ready. If that sounds reasonable, I'll work on getting a fix in tomorrow.
/remove-triage unresolved
If that is still exercises the same conditions (I vaguely remember publishNotReadyAddresses changing code paths in the endpoints controller... does it do the same in the endpointslice controller?), that seems reasonable to me.
I think the code path is very similar here, the only real difference should be how the EndpointSlice controller calculates readiness. I've gone ahead and filed a PR that uses this approach, hopefully it helps:
This flaked again this morning /reopen
Reopened this issue. <detailsIn response to : This flaked again this morning Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
This should be /priority critical-urgent given
The top level error message does not make this clear but the issue here is that too many EndpointSlices are being created - 3 instead of 2.
Interesting... that condition should not persist for O(minutes), right?

that seems like it might be a bug, not just a test issue
There's potential that it's not consistently merging 2 ports with 1 endpoint into a single slice and instead creating a slice per port. It also could represent a larger bug. Trying to find a way to consistently reproduce this.
Despite my best efforts I have been unable to reproduce this on my own e2e cluster. I've had 10 test runs going at the same time repeatedly without any luck, any ideas for how to reproduce flaky test failures local would be welcome. In the meantime, I've created a follow up PR that will provide more helpful information if/when this test fails again: My leading theory right now is that this could be a caching issue. The EndpointSlice controller runs a sync, sees that we don't have an EndpointSlice for the given port combination in cache and then goes ahead and creates one, even though a matching EndpointSlice already exists and has not been cached yet. That would result in an annoying situation where an essentially duplicate EndpointSlice was created due to a stale cache. I would think the EndpointSliceTracker would resolve this issue for us though, so not sure. Getting the above PR in will help validate this theory. It's worth noting that if this is actually a duplicate EndpointSlice, clients like kube-proxy will handle this seamlessly because they already have to handle potential duplication in EndpointSlices. That would potentially explain why a test failure like this does not seem to be causing other things to break.
It appears that my theory was correct as witnessed here: I create a .

The key points I've noticed so far: 1 EndpointSlice references pod2 (example-named-port-f7vq5) 2 EndpointSlices reference pod1 (example-named-port-fkgcc and example-named-port-vfp2q) 2 EndpointSlices were created at "2020-07-28T04:57:51Z": example-named-port-f7vq5 (references pod2) example-named-port-vfp2q (references pod1) 1 EndpointSlice was created 1 second later ("2020-07-28T04:57:52Z") example-named-port-fkgcc (references pod1) All 3 EndpointSlices are: On generation 1 - no updates have occurred With a state of ready: true The 2 EndpointSlices referencing pod1 are effectively identical  Of course this is something the EndpointSliceTracker should be handling for us, but even that approach seems less than ideal here. It wouldn't actually prevent the duplicate EndpointSlices from being created, it should just be cleaning up duplicates after they happen. This combined with the race conditions referenced in make me think we may want to find a way to slow down the sync cycle for each Service here. This also feels at least tangentially related to the need for better rate limiting in both this and the Endpoints controller discussed in . /cc
Hmm - actually I don't understand why this is happening. Maybe I'm missing something, but I don't think slowing down controller is needed to solve this problem. Even though the controller usually is not lagging my more than 1s, it is possible that it can be lagging by a minute or more in pathological cases. So we can't just rely on the fact that it's guaranteed to be synced. Can you explain it a bit deeper why this is happening? i.e. some more detailed timeline like: T1: state of cache is XYZ T2: controller computes the ES x1, x2 T3: state of cache in ABC T4: controller computes ES y1 [why?]
Agree, I'm not in favor of slowing down the controller globally to address this. Delaying resync of a particular service until we observe the existence of new endpointslices we know we created (with some max delay) could be reasonable.

That's exactly where I was hoping to end up here, although I'm not quite sure how to do that cleanly yet. To be clear, this is not just to fix this flaky test. This flaky test is a symptom of a much larger and more annoying issue with the current implementation. The EndpointSlice controller is consistently running into errors where it's trying to update EndpointSlices that are stale. This is caused by follow up calls to running before the informer cache is up to date, which happens all the time during rolling updates where rapid churn of EndpointSlices is expected. Although this eventually resolves itself, it's causing a lot more load on API Server than necessary and can potentially result in some bad states where an update fails but a delete succeeds. In this specific instance, it looks like a syncService run is occurring after the first 2 EndpointSlices have been created but before the one referencing pod1 is in the informer cache. That results in the controller creating a new EndpointSlice to reference pod1 and thus we end up with a duplicate. If anything in the future triggers another syncService call that would be resolved. I'd initially thought that EndpointSliceTracker would be thing to trigger that follow up syncService, but I was wrong. That tracker just tracks the latest resource version for each EndpointSlice the controller has created/updated and compares that with EndpointSlices received through create/update events. If the controller doesn't think that EndpointSlice should exist or the controller's resource version for that slice does not match it triggers another syncService. In this case neither of those conditions would be met so no additional syncService call would be triggered. So that's a long way of saying that I think our best possible solution here is to introduce some delay between when Services can be synced. I think the easiest solution that would solve 99% of our problems would be to add something like a 1s delay in between syncs simply by delaying when was called. This would not be ideal, but it would result in an immediate improvement and be easy to backport. It's worth noting that although duplicate EndpointSlices like this or even attempts to update a stale EndpointSlice are not ideal, they are not actually invalid and will eventually resolve themselves. Consumers of this API already have to do some form of deduping.

With that said, I'm hopeful that there's a clean way to implement a solution that actually waited for each EndpointSlice change the controller has made to be reflected in the cache because that would be a better solution. I'll spend some time looking into possible implementations here.
I've created a draft PR that contains my initial progress towards the approach I described above. I don't particularly love this approach so I'm very open to other ways we could solve this issue. It's also still very much a WIP and has significant work remaining. I opened the PR early just so I could get some initial feedback on the direction and potentially inspire other ideas around approaches we could take here.
is still the best approach I can come up with here. It is ready for review if anyone has time to take a look.
Quick update here, I've limited the scope here to the following changes as discussed in Update the flaky test to consider duplicate EndpointSlices valid. Update the docs to make it very clear that consumers of the EndpointSlice API will have to do deduplication (as kube-proxy already does). Update the controller error handling to return immediately if a create, update, or delete errors (in that order). Reviews would be appreciated on all of the above.
Reopening since is not yet merged.
Thanks for catching that Closing now that is merged. /close
Closing this issue. <detailsIn response to : Thanks for catching that Closing now that is merged. Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: defer watcher.Stop() done := make(chan struct{}) <del> defer close(done) </del> <ins> var wg sync.WaitGroup wg.Add(1) defer wg.Wait()   // We must wait for the waitgroup to exit before we terminate the cache or the server in prior defers defer close(done) // call close first, so the goroutine knows to exit </ins> go func() { <ins> defer wg.Done() </ins> for i := 0; i < 100; i++ { select { case <-done:

<!-- Please only use this template for submitting reports about flaky tests or jobs (pass or fail with no underlying change in code) in Kubernetes CI --Which jobs are flaking: bazel-test-master Which test(s) are flaking: - TestWatchBookmarksWithCorrectResourceVersion Testgrid link: Reason for failure: Anything else we need to know: links to appreciated Somehow nothing seen on triage - wonder why? links to specific failures in spyglass appreciated <!-- Please see the deflaking doc () for more guidance! --/sig testing
is still only reporting up to Sept 21 Other instances of non-deterministic result (NDR)
/sig api-machinery /sig storage Not 100% sure which SIG should triage the data race on which is producing non-deterministic results (NDRs) during CI can you both take a look and let me know? Many thanks, CI Signal Team Lead for 1.20 (release team)
The label(s) cannot be applied, because the repository doesn't have them <detailsIn response to : Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Looks like the bot didn't pick up the SIG tags - /sig api-machinery /sig storage
/sig scalability /remove-sig storage
I'm sending a fix
t.Fatal was getting called long after the test had exited, because the test was shutting down the framework before the goroutine had exited.


Question: HEAPSTER_GCP_BASE_CPU="${HEAPSTER_GCP_BASE_CPU:-80m}" HEAPSTER_GCP_CPU_PER_NODE="${HEAPSTER_GCP_CPU_PER_NODE:-0.5}" <ins> # Optional: custom system banner for dashboard addon CUSTOM_KUBE_DASHBOARD_BANNER="${CUSTOM_KUBE_DASHBOARD_BANNER:-}" </ins> # Default Stackdriver resources version exported by Fluentd-gcp addon LOGGING_STACKDRIVER_RESOURCE_TYPES="${LOGGING_STACKDRIVER_RESOURCE_TYPES:-old}"

you this variable - PTAL
/sig gcp
Ping? I see that in all failure log messages for . Not sure if thats the cause or just an error message I can ignore?
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale /assign since I bumped into this


Question: ErrCacheNotFound = errors.New("not found") ) <ins> // Server returning empty ResourceList for Group/Version. type emptyResponseError struct { gv string } func (e *emptyResponseError) Error() string { return fmt.Sprintf("received empty response for: %s", e.gv) } </ins> var _ discovery.CachedDiscoveryInterface = &memCacheClient{} // isTransientConnectionError checks whether given error is "Connection refused" or

I think this got stale and closed because it was posted on but is still very relevant: The created by memory.NewMemCacheClient(...) returns an error when calling ServerResourcesForGroupVersion if the APIResources list returned is empty, as seen here: Our particular case came up due to the API group being registered with the API server, but having no resources. If using like to discover with zero current resources there should not be an error returned. Do a discovery with a against a Kubernetes cluster that has at least one with zero resources. The condition on the server that triggers the described issue can be expressed as e.g.: I would be good to get to know the reasoning behind the emptiness check in the , maybe there is good reason for it. <detailsClient Version: v1.20.15 Server Version: v1.22.8-gke.202 </details<detailsGoogle Cloud (GKE) </detailsNo response No response No response No response
/sig api-machinery
/cc /triage accepted
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at .

/lifecycle stale [1]:
Reviving this issue based on This behavior has become quite annoying for the custom/external metrics aggregated APIs since after , client-go is now producing a very spammy error log when an aggregated API is found without resource. This is a very common scenario with the metric APIs since metrics are dynamically registered as Resources to the aggregated APIs so on startup the aggregated apiserver may not expose any resource and that's completely normal. It is even more common for implementations of these APIs that register metric Resources from CRDs as CRs tend to be created later than APIServices and the aggregated apiserver. I can see a few path forward to improve the situation for us: the error if we consider that having no resources exposed by an aggregated API is acceptable the verbosity of the log to just show it in debugging scenarios somehow update to defer registration until we have at least one Metric resource. could you perhaps share your insight on that one please? /remove-lifecycle stale
Hello folks! Any update on this topic? The message is really spammy
Removing triage accepted manually since there was no update on this for more than 6 months. /remove-triage accepted
For context, I traced it back and check has been considered an error since the first iteration of the in-memory discovery client in : It is unexpected for a built-in or CRD groupversion to have no resources, but for aggregated-apiservice I can see it being possible that the object is created and set up for discovery before the aggregated-apiservice is ready.
I think this is which introduces (but maybe I'm wrong). There is also in that repo about the same issue after upgrading the kube-client pkg. in custom-metrics repo be correct from Kubernetes pov? IMO we need to find a solution soon or the amount of noise will increase once people bump their deps and tooling (, kubectl is affected, and any tool which uses kube-client is affected)
/triage accepted


Question: "os" "k8s.io/kubernetes/pkg/healthz" <del> "k8s.io/kubernetes/pkg/util" </del> "k8s.io/kubernetes/pkg/util/flag" "k8s.io/kubernetes/pkg/util/logs" "k8s.io/kubernetes/pkg/version/verflag"

When I tried to use the following command to build kubernetes: I encountered the following errors: I'm using Ubuntu 14.04 and the code from master branch. will submit a patch then to fix this problem.
whoa, I just landed in the exact same place, I am trying to build k8s on a CentOS7 box. Tried with removing the unused utils and works like a charm !


Question: if len(n.podsWithAffinity) > 0 { clone.podsWithAffinity = append([]*v1.Pod(nil), n.podsWithAffinity...) } <ins> if len(n.taints) > 0 { clone.taints = append([]v1.Taint(nil), n.taints...) } </ins> return clone }

Looking into profiles of scheduler, there shows a bunch of things that we are doing unnecessarily. In particular, instead of computing the same information for a given node multiple times, we should cache it as part of NodeInfo: The information that should be cached and reuse later include: [x] node taints: [x] node conditions: disk/memory/inode pressure
Another quick optimization should be optimizing "selectHost" method: In fact it doesn't need to sort the whole list (which is O(nlgn). All we need to do is to choose the elements that have highest score, and sort only those by node name (to leave the logic that we currently have with round-robic logic of choosing from those).
And we should also limit the amount of places where we call "Parallelize" (because those are relatively expensive). The whole MR-like priorities is a step in this direction too.
OK - I've just sent out a PR fixing the first two things.
can we close this, or are there still work items here?
Some of those are not yet done.
- it's not a high priority for us (for now), so if you want it to be done soon, you probably need to find someone to do it.
- assigned.
Why "Parallelize" is considered as relatively expensive ? (comparing to MR style?)
You do MR when there are certain points that need some synchronizations. Or fully parallelized computation will always be faster.
To clarify, I was referring to the Parallelize function: Using this for many very small functions is pretty expensive - you can take a look into cpu-profiles. It happens that 20% or sth of cpu usage is spent on the operations on the synchronization channel.
Is all of the cacheing part completed? I assume "yes". What is remaining here.... Just an audit of parallelization ? Also, I assume "yes". If so I can audit where the parelellization is getting us hung up and then we can close this one out.
It's hard to tell if caching is done - when adding new priority functions/moving existing ones to MR-framework one pretty often discovers that more things need to be cached. There's my PR , that moves basic spreading priority function to MR-framework, which for some reason makes it worse. I didn't have time to look into that though.

This needs to be done, which may require modifying MR-framework slightly (allowing different type as a Map phase result).
Im not sure what the MR Framework stuff is referring to. Can you provide me a pointer ? Should we close this issue since the original patches for caching were done? Sounds like the work we're referring to now is actually more like a second iteration. if not, what work remains to be done before we can close this ticket? maybe we can make a checklist.
"MR-style" parallelization, that was mentioned before. Yeah, I guess we can close this - this is more like an ongoing process thing, when everyone who adds anything to the scheduler needs to keep it in mind.


Question: // AddFlags adds flags for a specific SchedulerServer to the specified FlagSet func (o *Options) AddFlags(fs *pflag.FlagSet) { fs.StringVar(&o.ConfigFile, "config", o.ConfigFile, "The path to the configuration file.") <ins> fs.StringVar(&o.WriteConfigTo, "write-config-to", o.WriteConfigTo, "If set, write the configuration values to this file and exit.") </ins> // All flags below here are deprecated and will eventually be removed.

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: We face But there is no way to generate default config file How to reproduce it (as minimally and precisely as possible): Environment: Kubernetes version 1.9.2
I'll take a look it /sig scheduling
/assign
Hi, same issue with controller-manager
Any updates here? , and don't have this flag. Is it even will be supported?
/cc
if not set, component should be use default value; why we'd like to dump all default config?
Because the documentation on the config file format is either nonexistent or difficult to find. If the program itself can generate a template, that fulfills (most of) the need for documentation of the format.
+1 for this, from my expirence, add an option to write the default config is really helpful.
/assign
also in 1.10
Deprecating all flags, prompting folks to use config files in advance is great, except when there's no documentation nor example anywhere. That means that we can't actually migrate to the new settings, and I can't seem to find clear alpha/beta/stable status for the new config file spec, nor timeline for the removal. It is also pretty confusing that the scheduler expects us to use a config file, but the controller-manager does not?
This issue indeed applies to kube-controller-manager. And there should already had deprecation policy.
According to , the flag will be available in . Since I can't find any reference online to an example config file, perhaps one could be provided or linked to here?
will you be able to pick my patch and generate a config file? I do not get my devbox handy so can only post here until Monday.


Question: // In GCI cluster, if gci mounter is used for mounting, the container started by mounter // script will cause additional mounts created in the container. Since these mounts are // irrelevant to the original mounts, they should be not considered when checking the <del> // mount references. Current solution is to filter out those mount paths that contain // the string of original mount path. // Plan to work on better approach to solve this issue. </del> <ins> // mount references. The current solution is to filter out those mount paths that contain // the k8s plugin suffix of original mount path. </ins> func HasMountRefs(mountPath string, mountRefs []string) bool { <ins> // A mountPath typically is like //   /var/lib/kubelet/plugins/kubernetes.io/some-plugin/mounts/volume-XXXX // Mount refs can look like //   /home/somewhere/var/lib/kubelet/plugins/kubernetes.io/some-plugin/... // but if /var/lib/kubelet is mounted to a different device a ref might be like //   /mnt/some-other-place/kubelet/plugins/kubernetes.io/some-plugin/... // Neither of the above should be counted as a mount ref as those are handled // by the kubelet. What we're concerned about is a path like //   /data/local/some/manual/mount // As unmonting could interrupt usage from that mountpoint. // // So instead of looking for the entire /var/lib/... path, the plugins/kuberentes.io/ // suffix is trimmed off and searched for. // // If there isn't a /plugins/... path, the whole mountPath is used instead. pathToFind := mountPath if i := strings.Index(mountPath, kubernetesPluginPathPrefix); i > -1 { pathToFind = mountPath[i:] } </ins> for _, ref := range mountRefs { <del> if !strings.Contains(ref, mountPath) { </del> <ins> if !strings.Contains(ref, pathToFind) { </ins> return true } }

See This happens on ubuntu images on GKE when the node ephemeral storage is on local SSD. /sig storage /assign
/triage accepted


Question: Type:                 []string{"object"}, AdditionalProperties: &spec.SchemaOrBool{Schema: stringSchema}, }} <ins> emptyObjectSchema = &spec.Schema{ SchemaProps: spec.SchemaProps{Type: []string{"object"}}, } </ins> ) func TestEquality(t *testing.T) {

While migrating webhooks to CEL for one of the structs in our CRD api, I noticed that the empty object in CEL is interpreted wrongly. Empty map object seems to be interpreted correctly. In the example below, both map and object are sent empty () in json output. returns true json (captured on the client side) sent to the apiserver CRD schema No response <details</details<details</details<details</details<details</details<details</details<details</details
/sig api-machinery
CEL only sees the JSON sent to the server, and is correctly evaluated as present. The issue is with the client sending the request populating a value in that field. How is the request being constructed? /remove-kind bug /kind support
/cc /triage accepted
Upon investigation, the problem lies there: We requires either or be set.
Thanks for raising this issue! I was debugging offline with and seems like CRD/CR validation didn't handle the situation of unsetting properties with field type specified . It is a corner case in CRD since we normally would ask for properties defined if the field type is (I could not think of use case which having properties unset). But it might ok to allow it through since it won't hurt from CRD perspective. We will fix in CEL values handling which treat this as empty map passed. /assign
GitHub didn't allow me to assign the following users: jiahui. Note that only with read permissions, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : Thanks for raising this issue! I was debugging offline with and seems like CRD/CR validation didn't handle the situation of unsetting properties with field type specified 'Object1object`(I could not think of use case which having properties unset). But it might ok to allow it through since it won't hurt from CRD perspective. We will fix in CEL values handling which treat this as empty map passed. /assign </details
also check if the CRD validation allows the or values, and if so, if the CEL code handles those properly as well
FYI: We do not allow in crd structured schema now:


Question: require.NoError(t, err) return out } <del> makeReplicaSet := func(name string) *example.ReplicaSet { return &example.ReplicaSet{ </del> <ins> makeReplicaSet := func(name string) *example2v1.ReplicaSet { return &example2v1.ReplicaSet{ </ins> ObjectMeta: metav1.ObjectMeta{Namespace: "ns", Name: name}, } } <del> createReplicaSet := func(obj *example.ReplicaSet) *example.ReplicaSet { </del> <ins> createReplicaSet := func(obj *example2v1.ReplicaSet) *example2v1.ReplicaSet { </ins> key := "replicasets/" + obj.Namespace + "/" + obj.Name <del> out := &example.ReplicaSet{} </del> <ins> out := &example2v1.ReplicaSet{} </ins> err := etcdStorage.Create(context.TODO(), key, obj, out, 0) require.NoError(t, err) return out

The test is flaking: /assign /sig api-machinery /priority important-soon
Ok - it's fairly easy to reproduce by just adding into the test I will validate the fix.
/triage accepted


Question: break } // Deserialize the TypeMeta information of this byte slice <del> if err := yaml.Unmarshal(b, &typeMetaInfo); err != nil { </del> <ins> gvk, err := yamlserializer.DefaultMetaFactory.Interpret(b) if err != nil { </ins> return nil, err } <del> // Require TypeMeta information to be present if len(typeMetaInfo.APIVersion) == 0 || len(typeMetaInfo.Kind) == 0 { errs = append(errs, errors.New("invalid configuration: kind and apiVersion is mandatory information that needs to be specified in all YAML documents")) continue </del> <ins> if len(gvk.Group) == 0 || len(gvk.Version) == 0 || len(gvk.Kind) == 0 { return nil, errors.New("invalid configuration: kind and apiVersion is mandatory information that needs to be specified") </ins> } <del> // Check whether the kind has been registered before. If it has, throw an error if known := knownKinds[typeMetaInfo.Kind]; known { errs = append(errs, errors.Errorf("invalid configuration: kind %q is specified twice in YAML file", typeMetaInfo.Kind)) continue } knownKinds[typeMetaInfo.Kind] = true // Build a GroupVersionKind object from the deserialized TypeMeta object gv, err := schema.ParseGroupVersion(typeMetaInfo.APIVersion) if err != nil { errs = append(errs, errors.Wrap(err, "unable to parse apiVersion")) </del> <ins> // Check whether the kind has been registered before. If it has, throw an error if known := knownKinds[gvk.Kind]; known { errs = append(errs, errors.Errorf("invalid configuration: kind %q is specified twice in YAML file", gvk.Kind)) </ins> continue } <del> gvk := gv.WithKind(typeMetaInfo.Kind) </del> <ins> knownKinds[gvk.Kind] = true </ins> // Save the mapping between the gvk and the bytes that object consists of <del> gvkmap[gvk] = b </del> <ins> gvkmap[*gvk] = b </ins> } if err := errorsutil.NewAggregate(errs); err != nil { return nil, err

<!-- Please only use this template for submitting enhancement requests --What would you like to be : Add an external MetaFactory that allows for serializing multiple YAML (or JSON) files in kubeadm. The logic of marshalling YAML should be factored out from to . Why is this needed: This will allow processing component configs containing multiple documents per file. Additionally, an out-of-tree library would allow sharing this functionality with other components.
I will try work on this as carry-on from /sig cluster-lifecycle /cc
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale


Question: actualCfg, err := clientBuilder.ClientConfig() if err != nil { <del> t.Errorf("Unexpected error: %v", err) </del> <ins> t.Fatalf("Unexpected error: %v", err) </ins> } matchBoolArg(true, actualCfg.Insecure, t)

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: Created cluster with , which generated its own self-signed certificate and put in . Put (API) nodes behind HTTPS ELB and for SSL termination. Run any command that has to hit the API (That's annoying because the CA that signed the cert is a public CA, and the root cert is in my OS keystore. Potentially another bug?) Download the appropriate key from (In my case, AWS ACM certificate = ) Use the certificate to attempt the same request What you expected to happen: Commandline-specified certificate overrides any other options How to reproduce it (as minimally and precisely as possible): Put any in for your host and attempt to override on cmdline. Anything else we need to know?: seems to be where the problem comes from, it's not differentiating "param came from cmdline" from "have param". If I remove the from then it works for both intended cases - either with specifying the CA cert on cmdline or via system certs. Environment: Kubernetes version (use ):Cloud provider or hardware configuration**: AWS OS (e.g. from /etc/os-release): OSX Kernel (e.g. ):Install tools:Others:
There are no sig labels on this issue. Please by:<br(1) mentioning a sig: <bre.g., for API Machinery<br(2) specifying the label manually: <bre.g., for sig/scalability<br<brNote: method (1) will trigger a notification to the team. You can find the team list and label list
Reiterating the mentions to trigger a notification: <detailsIn response to : Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. Prevent issues from auto-closing with an comment. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten /remove-lifecycle stale
/remove-lifecycle rotten
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Still a problem. /remove-lifecycle stale
Just ran into this as well with 1.9.0
also with 1.9.3 This is very annoying. It says "certificate-authority-data will override" but it will fail and do nothing, exiting with 0. It should at least do the override and try to execute
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale Any chance of someone looking at this? Or at least freezing status, since it's evergreen until actually fixed?
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale References from other tickets don't change this timer?
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with .

Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
/remove-lifecycle rotten
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
/assign
any chance this can be reopened? still seems to be a problem with 1.20 of kubectl


Question: } // actual maps FrontendPort to an elbv2.Listener <del> actual := map[int64]*elbv2.Listener{} </del> <ins> actual := map[int64]map[string]*elbv2.Listener{} </ins> for _, listener := range listenerDescriptions.Listeners { <del> actual[*listener.Port] = listener </del> <ins> if actual[*listener.Port] == nil { actual[*listener.Port] = map[string]*elbv2.Listener{} } actual[*listener.Port][*listener.Protocol] = listener </ins> } actualTargetGroups, err := c.elbv2.DescribeTargetGroups(

What happened: What you expected to happen: kubernetes creating an AWS NLB for UDP How to reproduce it (as minimally and precisely as possible): Create service with annotation and port with Anything else we need to know?: AWS recently announced that UDP is available within their NLB Environment: Kubernetes version (use ): 1.11.9 Cloud provider or hardware configuration: AWS
/sig aws
/kind feature
/remove-kind bug
/milestone v1.16
You must be a member of the GitHub team to set the milestone. If you believe you should be able to issue the /milestone command, please contact your and have them propose you as an additional delegate for this responsibility. <detailsIn response to : /milestone v1.16 Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/milestone v1.16
I am taking up this Feature Development. How can this be assigned to me?
/assign
+1
+1 I am experiencing the same issue. NLB also supports UDP_TCP type, any service using "nlb" should be able to mix TCP and UDP port configs.
Any progress? I can collaborate on this, if you need a hand.
+1
+1 On Thu, Jul 18, 2019 at 12:58 Adam Murphy <:
+1
I just ran into this. Any updates?
Attempting to fix it with
Thanks, The Changes looks good.
This should be closed I think.
This feature only for 1.16?
Could feature back ported into 1.15 1.14 1.13
Could someone review please?
/sig cloud-provider
Any update on this?
As we are in the code freeze and this issue is still not fixed, I'll move it to the 1.17 milestone. I'm also going to add the PR that fixes this issue to the milestone. If there are any concerns, please ping me or ping SIG-Release on Slack. /milestone v1.17
Do you have a timeline when this will be released?
This is a breaking issue in multiple versions but the is not prioritized. Does anyone have a workaround at the moment?
For AWS you can associate elastic IP with pod. Need make aws cli call.

If you need static private IP then you can create DaemonSet attach NIC with static IP as second NIC to EC2 instance and node some label (like ). Later you start pod with network, get node label, bound to required host IP. If you need static public and private IPs, then you can customize node-group init script and attach NIC with permanent private IP and Elastic IP at node startup. Later simple start pod with network
Bug triage for release 1.17 here. Since this issue has been tagged for the milestone, we would like to check it's status. Code freeze is on Nov. 14. Will this issue be resolved before then? If not, we should move it out of the milestone.
Hi bug triage team for 1.17, will this issue be resolved before code freeze on Nov. 14 (Next Thursday)?
/milestone v1.18 (v1.17 code freeze)
looking for a first issue to work, can I take this
gave some good feedback to the original PR. Does this need a reset with a fresh PR? I also have a need for this. I don't have a strong urge to write, or reuse, an operator for NLBs in AWS for UDP. This will eventually get done in k8s anyways, so I'd rather put the effort there.
BTW, you can always setup an NLB manually to forward UDP to a nodePort service on your k8s instances. The issue with that is that you have to pick a NodePort service for UDP before hand, and manually handle the NLB and the target groups. One cluster upgrade and your target group behind the NLB disappears since all of the instances get replaced. It's not ideal but in a pinch...
I don't mind updating a the PR, seems like a good first issue for me.
/assign
+1
+1
+1
+1
+1
Hello ! Bug Triage team here for the 1.18 release. This is a friendly reminder that code freeze is scheduled for 5 March, which is a few weeks from now. Is this issue still intended for milestone 1.18? Thanks in advance!

C/O Code freeze for release 1.18 is starting in 5 March (less than 1 week from now) and we want to ensure that each issue has a chance to be finished on time. Is this still on track for this deadline?
/remove-kind bug
Is there any work remaining to be done on this? Willing to contribute if so, would really love to see this make it into 1.18!
Thanks for the reply This has been a long running issue, and given that there has not been any recent activity, would it make more sense to once again move the milestone? We are 2 days away from code freeze (5 March).
I wonder what "activity" you are expecting.
We are now in code freeze for 1.18, so I am moving this to the next milestone. /milestone v1.19
Hello, as of now, I see is 65% completed. May we please have an update on estimated release date of UDP support for AWS NLB?
People have been trying to be constructive here. But the team doesn't even try to tell us what is pending. Sad.
Would also love to see this feature for EKS - and am willing to help out. Even if we could get it in 1.19 it would still take a long time to become available in EKS (which is currently at 1.15). So the sooner we can start work on this - the better :-)
+1 Also hope we could get in 1.19 and happy to help out!
any update on this pretty please ?
whats the update here?
From what I can see, the main issues that remain unresolved since the first PR last year: is still pending if you need a service that supports both TCP and UDP, e.g., DNS. DNS is also a more-complicated example because it supports TCP and UDP on the same port number, and that'll need more refactoring here, as AWS treats that as a single multi-protocol target. Edit: To be clear, not really a blocker for UDP support, but it's been raised in the discussion earlier. AWS NLB can only do TCP-based health checks (including HTTP and HTTPS), so your service needs to have a health-check TCP port listening. Current PRs silently assume it's on the same port number as the UDP service you're advertising.

If you're using , then it'll use the health check from  instead anyway. So this is probably only an issue for LoadBalancer Services with set. See "Things to know" on the .


Question: # # Due to GCS caching of public objects, it may take time for this to be widely # distributed. <del> curl -L -O https://storage.googleapis.com/kubernetes-release/easy-rsa/easy-rsa.tar.gz > /dev/null 2>&1 </del> <ins> # # Use ~/kube/easy-rsa.tar.gz if it exists, so that it can be # pre-pushed in cases where an outgoing connection is not allowed. if [ -f ~/kube/easy-rsa.tar.gz ]; then ln -s ~/kube/easy-rsa.tar.gz . else curl -L -O https://storage.googleapis.com/kubernetes-release/easy-rsa/easy-rsa.tar.gz > /dev/null 2>&1 fi </ins> tar xzf easy-rsa.tar.gz > /dev/null 2>&1 cd easy-rsa-master/easyrsa3

cluster/saltbase/salt/generate-cert/make-ca- sets but does not echo what it is doing, so in the case of an error there is no evidence of which command produced the error.


Question: return claimed, utilerrors.NewAggregate(errlist) } <del> // AdoptReplicaSet sends a patch to take control of the ReplicaSet and also // sets the finalizers to foregroundDeletion. It returns the error if // the patching fails. </del> <ins> // AdoptReplicaSet sends a patch to take control of the ReplicaSet. It returns // the error if the patching fails. </ins> func (m *ReplicaSetControllerRefManager) AdoptReplicaSet(rs *extensions.ReplicaSet) error { if err := m.canAdopt(); err != nil { return fmt.Errorf("can't adopt ReplicaSet %v/%v (%v): %v", rs.Namespace, rs.Name, rs.UID, err)

Seeing this in v1.6.1. a Deployment and a Service the Deployment add an ownerRef that points back to the Service delete the Service The Service and the Deployment are deleted, but the ReplicaSet and pods remain. This is neither Orphan nor Foreground/Background policy. This behavior should never happen. Note: if I make a delete call to the API with propagationPolicy set explicitly then things behave as expected, so this only occurs when the resource's default propagation policy is being used.
There are no sig labels on this issue. Please by:<br(1) mentioning a sig: <br(2) specifying the label manually: <br<brNote: method (1) will trigger a notification to the team. You can find the team list and label list
cc
This is expected. The default garbage collection policy for old controller types (rc, rs, deployment) etc. is orphaning, for backward compatibility. And when doing BackgroundGC, the GC controller doesn't set the deleteOptions.propagationPolicy. We met a similar problem for deleting deployment -RS -pods. Because that is a very common use case, we made a special fix in As you noticed, if you set deleteOptions.propagationPolicy to Orphan or ForegroundGC, it works as you expected.
that seems wrong. if I'm using the garbage collection feature to propagate (whether a user specified it or it was a default of a parent resource), I expect GC to set propagation policy.
rather than a one-off fix in the RS controller, why not address the break in the propagation chain in the GC controller?
In the backgroundGC, the owner is deleted immediately, so unlike orphaning or the foregroundGC, there is no finalizer left for the GC controller to determine the intention of the user. In the absence of the user's intention, the GC controller doesn't set deleteOptions.PropagationPolicy when deleting dependents.
If it's not orphaning, and it's not foregroundGC, the only logical option that remains is backgroundGC
There is another case, user might intentionally choose not to set DeleteOptions.Propagation policy and wants to respect the existing finalizers of the dependents or the default GCstrategy defined by the REST storage of the dependents.

except that it isn't... if I delete resource X and specify no propagation policy, that resource strategy's default propagation policy should take effect just as if the user had specified it.
Add CronJobs to the list of affected resources, since you have CronJob-Job-Pod chain. :+1: to what Jordan said just above.

BUG REPORT: <!-- If this is a BUG REPORT, please: Fill in as much of the template below as you can. If you leave out information, we can't help you as well. If this is a FEATURE REQUEST, please: Describe in detail the feature/behavior/change you'd like to see. In both cases, be ready for followup questions, and please respond in a timely manner. If we can't reproduce a bug or think a feature already exists, we might close your issue. If we're wrong, PLEASE feel free to reopen it and explain why. --Kubernetes version: 1.6.1 Environment: Hardware configuration: 2 identical machines, with i7-2600K CPUs and 4GB RAM. OS: CentOS 7 Kernel: 3.10.0-514.10.2.el7.x86_64 Install tools: KubeADM What happened: Deleting a deployment via the API with the DeleteOptions settings or deletes the Deployment and the replica set but NOT the pods. Deleting it with the DeleteOptions setting works as intended. What you expected to happen: The replica set deletion should propagate to the pods. I've read that the garbage collection should work for Deployments in 1.6. How to reproduce it: Anything else we need to know: I am using the and at the moment it only supports the orphanDependants option, since it's still at api version 1.5.
cc:
We discussed the problem in I'll change the GC to make sure it's cascading.
Sent
Any progress on this?
Still trying to figure out a sane fix:
I've encountered the same issue. My pods are still running even though I deleted the deployment.
lets come up with a fix for 1.7. This should work from the first time we introduced cascading deletion in Deployments.
Yes, we should. Do you have any comments on
replied. Let's move this forward. If the fix is small, I am going to backport it to 1.6.
Offline discussion with we punt this to v1.8.
Discussed with if we can get the PR fixed soon, it can still go in 1.7. It's a real bug fix.
Yes, we need to fix this for 1.7 and also backport to 1.6.

Should this be targeting 1.7?
yes, the fix that went in in is going to cause us data issues we don't want
Are you working on this? What's the timeline to resolution?
Based on above comment, looks like actually introduced a new regression. Am I right?
In , I reverted what's introduced in .
- is working on it
I approved this for 1.7 based on comment at We should revert , but are we ok with since it also introducing a new behavior. I will leave to decide.
replaces with a fix that addresses this issue systemically. While this bug did exist in 1.6.x, more things rely on GC in 1.7 than in 1.6, and without that fix, GC leaves more resources vulnerable to unintentional orphaning. Since the issue is actually with general GC behavior, should weigh in.


Question: <ins> /* Copyright 2015 The Kubernetes Authors All rights reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ // Channel Manager keeps track of multiple channels package operationmanager import ( "testing" ) func TestStart(t *testing.T) { // Arrange cm := NewOperationManager() chanId := "testChanId" testMsg := "test message" // Act ch, startErr := cm.Start(chanId, 1 /* bufferSize */) sigErr := cm.Send(chanId, testMsg) // Assert if startErr != nil { t.Fatalf("Unexpected error on Start. Expected: <no error> Actual: <%v>", startErr) } if sigErr != nil { t.Fatalf("Unexpected error on Send. Expected: <no error> Actual: <%v>", sigErr) } if actual := <-ch; actual != testMsg { t.Fatalf("Unexpected testMsg value. Expected: <%v> Actual: <%v>", testMsg, actual) } } func TestStartIdExists(t *testing.T) { // Arrange cm := NewOperationManager() chanId := "testChanId" // Act _, startErr1 := cm.Start(chanId, 1 /* bufferSize */) _, startErr2 := cm.Start(chanId, 1 /* bufferSize */) // Assert if startErr1 != nil { t.Fatalf("Unexpected error on Start1. Expected: <no error> Actual: <%v>", startErr1) } if startErr2 == nil { t.Fatalf("Expected error on Start2. Expected: <id already exists error> Actual: <no error>") } } func TestStartAndAdd2Chans(t *testing.T) { // Arrange cm := NewOperationManager() chanId1 := "testChanId1" chanId2 := "testChanId2" testMsg1 := "test message 1" testMsg2 := "test message 2" // Act ch1, startErr1 := cm.Start(chanId1, 1 /* bufferSize */) ch2, startErr2 := cm.Start(chanId2, 1 /* bufferSize */) sigErr1 := cm.Send(chanId1, testMsg1) sigErr2 := cm.Send(chanId2, testMsg2) // Assert if startErr1 != nil { t.Fatalf("Unexpected error on Start1. Expected: <no error> Actual: <%v>", startErr1) } if startErr2 != nil { t.Fatalf("Unexpected error on Start2. Expected: <no error> Actual: <%v>", startErr2) } if sigErr1 != nil { t.Fatalf("Unexpected error on Send1. Expected: <no error> Actual: <%v>", sigErr1) } if sigErr2 != nil { t.Fatalf("Unexpected error on Send2. Expected: <no error> Actual: <%v>", sigErr2) } if actual := <-ch1; actual != testMsg1 { t.Fatalf("Unexpected testMsg value. Expected: <%v> Actual: <%v>", testMsg1, actual) } if actual := <-ch2; actual != testMsg2 { t.Fatalf("Unexpected testMsg value. Expected: <%v> Actual: <%v>", testMsg2, actual) } } func TestStartAndAdd2ChansAndClose(t *testing.T) { // Arrange cm := NewOperationManager() chanId1 := "testChanId1" chanId2 := "testChanId2" testMsg1 := "test message 1" testMsg2 := "test message 2" // Act ch1, startErr1 := cm.Start(chanId1, 1 /* bufferSize */) ch2, startErr2 := cm.Start(chanId2, 1 /* bufferSize */) sigErr1 := cm.Send(chanId1, testMsg1) sigErr2 := cm.Send(chanId2, testMsg2) cm.Close(chanId1) sigErr3 := cm.Send(chanId1, testMsg1) // Assert if startErr1 != nil { t.Fatalf("Unexpected error on Start1. Expected: <no error> Actual: <%v>", startErr1) } if startErr2 != nil { t.Fatalf("Unexpected error on Start2. Expected: <no error> Actual: <%v>", startErr2) } if sigErr1 != nil { t.Fatalf("Unexpected error on Send1. Expected: <no error> Actual: <%v>", sigErr1) } if sigErr2 != nil { t.Fatalf("Unexpected error on Send2. Expected: <no error> Actual: <%v>", sigErr2) } if sigErr3 == nil { t.Fatalf("Expected error on Send3. Expected: <error> Actual: <no error>", sigErr2) } if actual := <-ch1; actual != testMsg1 { t.Fatalf("Unexpected testMsg value. Expected: <%v> Actual: <%v>", testMsg1, actual) } if actual := <-ch2; actual != testMsg2 { t.Fatalf("Unexpected testMsg value. Expected: <%v> Actual: <%v>", testMsg2, actual) } } </ins>

The "" E2E test will sometimes fail because it fails to delete the PD used in the test after the test is finished. The error: Detailed test log: The disk that fails to delete still shows up in : My attempts to manually try to delete the PD fail with: (the work around for ) doesn't change anything here:
is the disk still mounted ? It looks like the detach call did not succeed for some reason
I'll have to get another repro and verify.
okay cool. If you get a repro check to see if the disk is still mounted. If so then the cleanup code is not being called. If not then the clean up started, detach was called but failed for some reason; look in the logs for something like "googleapi: some error about detaching". It can also just be a race condition. That is, detach was called but is taking too long; if the disk does not detach then you cannot delete it.


Question: // Verify call goes through all the way expectedSet := 0 <del> if !tc.stageUnstageSet { </del> <ins> if !tc.stageUnstageSet || tc.volID == "" { </ins> expectedSet = 1 } staged := cdc.nodeClient.GetNodeStagedVolumes()

Hello, I've recently been testing the reliability of storage for k8s v1.25.0. A strange phenomenon occurs occasionally during the test: the volume cannot be detached after the PVC and pod are deleted(the reclaimPolicy of pv is delete. So I first went to check the corresponding and objects and found that they both existed. Then I checked the kube-controllerManager and kubelet logs and found that kubelet logs kept printing: But I found that at the beginning, the volume was unmountDevice succeed, as shown in the following kubelet log: it is so wired ! Then I some log variables to the unmountDetachDevices method: The results show that And I've drawn up a time-series chart of the contents of the log. ! Let's go back to the unmountDetachDevices method. According to the above-mentioned log content, the first operation successfully changes of the corresponding attachedVolume to , but the function in the subsequent operation still uses stale DeviceMountState. that the unmountDevice operation is not performed after the first unmountDevice operation succeeds. multiple unmountDevice operations are allowed, ensure that the correct pvname can be obtained in the method. the volume is detached after the pod and PVC objects are deleted This problem can be reproduced by repeatedly deleting and recreating pods that require PVC in k8s 1.25.0 cluster No response <details</details<details</details<details</details<details</details<details</details<details</details
/sig storage
we can't safely assume that the json is always there.
ping
Thanks for the detailed analysis . I'm still sorting through this, do you think that is involved with this?
Yes. look at this log content: The global mount path is changed in the As a result, the method obtains incorrect pvName(e6104b10fea107e004e3307b0444fd7b775fedd804ab47aa997118b7927c32e0). The unmountDevice operation may be performed multiple times in [1.19 - 1.25] kubernetes versions. This problem does not occur before because can still obtain the correct pvName from the global mount path. However, this problem occurs after is merged. https://user- <br</brAfter the first unmountDevice operation is performed successfully, the file is deleted.

So the subsequent unmountDevice operation will perform method because file doesn't exist.
Awesome, thanks. Let me find someone to work on this.
I ll take a look at the issue, since its related to the change I made Note: this change needs the node to drained before upgrading to a version with this change . Please check the release notes
Thanks a lot !
yep, I always reproduce this problem in a newly created k8s 1.25.0 cluster.
Hihas there been any progress on this issue?
/priority important-soon
/triage accepted
Hihas there been any progress?
I also hit this error after pvc and pod deleted on k8s 1.25: and then these two volumes cannot be unmounted on the node forever: I tried to delete pv, it's stuck in terminating state forever
restart the kubelet on the node in problem could mitigate this issue, but we really need to fix this issue ASAP, I could repro this issue easily when running 1000 disk attach/detach test
I think this is again a TOCTOU bug in unmountDetachDevices.
The Unmount operation is , thus the state may not be set as before next reconcile, so it is possible that unmountDevice operation performed multiple times. So the root cause of this issue is that got wrong PV name, since changed global mount path.
I have found a similar issue in our environment, which is based on Kubernetes 1.25.3. If I install a Pod with a PVC, and I force move the Pod with NodeAffinity around the cluster, around 10-20 retries the Pod does not come up: the VolumeAttachment is bound to the old node and is not moved by the system anymore. If this is the same issue as reported here I do not know, but the logs are similar: From the Worker's kubelet where the volumeattachment is bound to: Also I see lots of these: But, if this is the same, I would like to know that the referenced PR was merged into exactly which Kubernetes version, as I have found in the RELEASE NOTES, that it was in 1.24.0-alpha: But we could not reproduce the issue on Kubernetes 1.24.2. How come?
As per the logs, it's the same issue. Kubelet operates in a reconcile loop, and is an async operation.

This bug only happened when is operated for multiple times, so it cannot be 100% reproduced. However, if the possibility of reproduction in 1.25 is much higher than the cluster lower than 1.25, we should figure out whether there are some other changes that makes this bug happened more frequently.
I have 1K disk attach/detach to 40 nodes test, if it runs multiple times, it could 100% repro, we could use this patch for testing, if 100 test runs pass, then it could prove the fix really works.
Sure. I mean not every single Mount/Unmount operation will lead to this error, but if we do it enough times, then it could be 100% to reproduce.
May I ask what could be a workaround in production systems or even CI systems on this issue? Our CI is struggling with this and other random issues, to the point we would like to stabilize it by kicking the system in a way to fix things... I was trying a kubelet restart on the Worker where the Mount is stuck... but many cases the kubelet restart itself hangs or does not produce a correct restart. So what could be a workaround if we find out that there is a hung volumeattachment?
restart kubelet, and if restart kubelet hangs, reboot the node.
Volumeattachment failed to detach from old node When statefulset application with pvc scheduled on worker1 during deployment its volumeattachment bounds to worker1 when same application upgraded scheduled on another worker node say worker2 and struck in ContainerCreating state and volumeattachment failed to detach from old node .

when pod described: Events: Warning FailedMount 23m (x820 over 6d2h) kubelet Unable to attach or mount volumes: unmounted volumes=[datadir], unattached volumes=[ datadir]: timed out waiting for the condition Warning FailedMount 5m22s (x3069 over 6d3h) kubelet Unable to attach or mount volumes: unmounted volumes=[datadir], unattached volumes=[datadir ]: timed out waiting for the condition Environment: NAME="SLES" VERSION="15-SP4" VERSIONID="15.4" PRETTYNAME="SUSE Linux Enterprise Server 15 SP4" ID="sles" IDLIKE="suse" ANSICOLOR="0;32" CPENAME="cpe:/o:suse:sles:15:sp4" DOCUMENTATIONURL=""' Kubernetes version: v1.25.3 kernel: 5.14.21-.24.41-default Storage: Rook: v1.9.12 ceph: 16.2.10 logs of kubelet: Mar 21 11:35:55 kubelet[]: E0321 11:35:55. ] "Unable to attach or mount volumes for pod; skipping pod" err="unmounted volumes=[datadir], unattached volumes=[datadir ]: timed out waiting for the condition" pod="test/ -0" Mar 21 11:35:55 kubelet[]: E0321 11:35:55. ] "Error syncing pod, skipping" err="unmounted volumes=[datadir], unattached volumes=[datadir ]: timed out waiting for the condition" pod="test/ -0" podUID=-1432-473d-af85- Mar 21 11:38:13 kubelet[]: E0321 11:38:13. ] "Unable to attach or mount volumes for pod; skipping pod" err="unmounted volumes=[datadir], unattached volumes=[datadir ]: timed out waiting for the condition" pod="test/ -0" Mar 21 11:38:13 kubelet[]: E0321 11:38:13. ]

"Error syncing pod, skipping" err="unmounted volumes=[datadir], unattached volumes=[datadir ]: timed out waiting for the condition" pod="test/ -0" podUID=-1432-473d-af85- Mar 21 11:40:29 kubelet[]: E0321 11:40:29. ] "Unable to attach or mount volumes for pod; skipping pod" err="unmounted volumes=[datadir], unattached volumes=[datadir ]: timed out waiting for the condition" pod="test/ -0" Mar 21 11:40:29 kubelet[]: E0321 11:40:29. ] "Error syncing pod, skipping" err="unmounted volumes=[datadir], unattached volumes=[datadir ]: timed out waiting for the condition" pod="test/ -0" podUID=-1432-473d-af85-
FYI. this issue is fixed in 1.24.14, 1.25.10, 1.26.5, and has been fixed in v1.27.0
Hello , can you have a look at this reconciler run's operationExecutor fails to umount volume due to missing vol- It appears to be Kube v1.22 not .25 to boot is this the same issue?
I think they are the same issue and I have left some comments in
Issue reoccurs in kubelet version


Question: if err := nim.RemoveNodeInfo(driverName); err != nil { glog.Errorf("Error unregistering CSI driver: %v", err) <ins> return err </ins> } <ins> return nil </ins> }

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: When the Kubelet plugin watcher invokes method, right now the CSI code does nothing (method is not implemented). What you expected to happen: When the Kubelet plugin watcher no longer detects the socket file for a given driver, it invokes the method on the CSI registration handler, the CSI code should then clean up plugin information and remove the corresponding CSIDriver object. How to reproduce it (as minimally and precisely as possible): Delete the socket file of a CSI plugin exposed for registration, nothing happens. /sig storage
/assign


Question: klog.InfoS("Volume is in pending operation, skip cleaning up mounts") } klog.V(2).InfoS("Reconciler sync states: could not find pod information in desired state, update it in actual state", "reconstructedVolume", reconstructedVolume) <ins> if cachedInfo, ok := volumesNeedUpdate[reconstructedVolume.volumeName]; ok { gvl = cachedInfo } gvl.addPodVolume(reconstructedVolume) </ins> volumesNeedUpdate[reconstructedVolume.volumeName] = gvl }

This issue is very similar to , they have the same symptom: remote volumes of the pods are becoming local mount point after rebooting the Node. And this issue is triggered by The difference is that in , kubelet didn't call , so the global mount point is local, but it will still call for the pod to bind mount the pod volume to the global mount point. Whereas in this issue, kubelet neither call globally nor call for the pod. This is more harmful: After writing some data to the pod volume, pod can't be deleted because kubelet encounter error while unmounting the pod volume: ! Reproduce steps: a cluster with one Node a deployment with two replicas(say and ) using Non-attachable volume (for example Azurefile) the Node with option (Use force deletion to simulate the situation that mount point was left over before rebooting the Node.) back the Node is always in Running state, however, after rebooting, it won't call and the volume it uses is local volume. should call . As mentioned above. No response <details</details<detailsN/A </details<details</details<details</details<details</details<details</details
Clusters with enabled is not impacted.
/sig node storage (for triage)
/remove-sig node sig-storage please triage this first. Thanks
/triage accepted


Question: } if err := vec.Validate(); err != nil { <del> klog.Error(err) </del> <ins> klog.ErrorS(err, "the validation for HistogramVec is failed. The data for this metric won't be stored in a benchmark result file", "metric", metric, "labels", labels) return nil } if vec.GetAggregatedSampleCount() == 0 { klog.InfoS("It is expected that this metric wasn't recorded. The data for this metric won't be stored in a benchmark result file", "metric", metric, "labels", labels) </ins> return nil }

Schedulerperf sometimes causes "json: unsupported value: NaN" panic . Not panic. in my PC, when running schedulerperf with the below config, there is a high probability of the panic. No response_ latest master <details</details<details</details<details</details<details</details<details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scheduling /assign
/kind failing-test
No. This problem is expected to exist for quite some time. Also, this test is just for measuring performance and does not block merging. Correct labeling requires background knowledge of the issue. For example, if, hypothetically, this test is built into ci and blocks the merge when it fails, then you are right, it would be a high priority.


Question: return nil, nil, false, err } if status == nil { <del> // service is not om this load balancer </del> <ins> // service is not on this load balancer </ins> continue }

/kind bug What happened: ILB Frontend IP configuration does not change while updating service What you expected to happen: LB Frontend IP configuration should get updated How to reproduce it (as minimally and precisely as possible): Source issue : AKS service is created using .yaml manifest. ILB is described as it is shown in the documentation: annotations: "true" spec: ports: port: 80 type: LoadBalancer loadBalancerIP: 10.240.200.2 ILB instance with the "kubernetes-internal" name is created with a correct private IP address pointing to the service described. If I change the "loadBalancerIP" to another internal IP address and re-deploy the service, the last is deployed with the correct new IP address, but "kubernetes-internal" ILB still has the old Frontend IP configuration. Environment: Cloud provider or hardware configuration: Azure
/sig azure
This also not works for public LB. /cc Could you fix the bug and cherry pick to old versions?
I got this bug today, I changed the external LoadBalancerIP on a LoadBalancer service. The service event log showed that it immediately recognised the change ("oldIP =newIP") but I had to delete/recreate the service to see the change in the Azure loadbalancer.


Question: <del> /* Copyright 2015 The Kubernetes Authors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package chaosclient import ( "net/http" "net/http/httptest" "net/url" "testing" ) type TestLogChaos struct { *testing.T } func (t TestLogChaos) OnChaos(req *http.Request, c Chaos) { t.Logf("CHAOS: chaotic behavior for %s %s: %v", req.Method, req.URL.String(), c) } func unwrapURLError(err error) error { if urlErr, ok := err.(*url.Error); ok && urlErr != nil { return urlErr.Err } return err } func TestChaos(t *testing.T) { server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { w.WriteHeader(http.StatusOK) })) defer server.Close() client := http.Client{ Transport: NewChaosRoundTripper(http.DefaultTransport, TestLogChaos{t}, ErrSimulatedConnectionResetByPeer), } resp, err := client.Get(server.URL) if unwrapURLError(err) != ErrSimulatedConnectionResetByPeer.error { t.Fatalf("expected reset by peer: %v", err) } if resp != nil { t.Fatalf("expected no response object: %#v", resp) } } func TestPartialChaos(t *testing.T) { server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { w.WriteHeader(http.StatusOK) })) defer server.Close() seed := NewSeed(1) client := http.Client{ Transport: NewChaosRoundTripper( http.DefaultTransport, TestLogChaos{t}, seed.P(0.5, ErrSimulatedConnectionResetByPeer), ), } success, fail := 0, 0 for { _, err := client.Get(server.URL) if err != nil { fail++ } else { success++ } if success > 1 && fail > 1 { break } } } </del>

/kind bug /sig release Such as:
Cool issue. /assign
I see the code in was pushed by you. I don't understand what Intercept() does. Do you think there is an Infinite recursive call?
Chaos client isn’t used anywhere, and we need to remove it soon.
Ok, I can help remove it.
But there are also one place using it. How to deal with it? Delete it directly? Or something we need to do?
Has never been anything other than experimental and was not used in practice. We can remove it and it’s flag from kubelet as it was experimental. are we deprecating experimental flags for one release or can we just remove?


Question: // This flag enables or disables reconcile.  Is false by default, and thus enabled. DisableAttachDetachReconcilerSync bool // ReconcilerSyncLoopPeriod is the amount of time the reconciler sync states loop <del> // wait between successive executions. Is set to 5 sec by default. </del> <ins> // wait between successive executions. Is set to 60 sec by default. </ins> ReconcilerSyncLoopPeriod metav1.Duration }

// RecommendedDefaultAttachDetachControllerConfiguration defaults a pointer to a // AttachDetachControllerConfiguration struct. This will set the recommended default // values, but they may be subject to change between API versions. This function // is intentionally not registered in the scheme as a "normal" // function to allow consumers of this type to set whatever defaults for their // embedded configs. Forcing consumers to use these defaults would be problematic // as defaulting in the scheme is done as part of the conversion, and there would // be no easy way to opt-out. Instead, if you want to use this defaulting method // run it in your wrapper struct of this type in its method. func RecommendedDefaultAttachDetachControllerConfiguration(obj kubectrlmgrconfigv1alpha1.AttachDetachControllerConfiguration) { zero := metav1.Duration{} if obj.ReconcilerSyncLoopPeriod == zero { obj.ReconcilerSyncLoopPeriod = metav1.Duration{Duration: 60 time.Second} } } Default is set to 60 second. but, the comment said is 5 seconds. // AttachDetachControllerConfiguration contains elements describing AttachDetachController. type AttachDetachControllerConfiguration struct { // Reconciler runs a periodic loop to reconcile the desired state of the with // the actual state of the world by triggering attach detach operations. // This flag enables or disables reconcile. Is false by default, and thus enabled. DisableAttachDetachReconcilerSync bool // ReconcilerSyncLoopPeriod is the amount of time the reconciler sync states loop // wait between successive executions. Is set to 5 sec by default. ReconcilerSyncLoopPeriod metav1.Duration } default should change to 60 seconds. comment errror None <detailsall version until now </details<detailsnot relevent </details<details</details<details</details<details</details<details</details
/sig storage
/remove-kind api-change /kind cleanup
Those labels are not set on the issue: <detailsIn response to : /remove-kind api-change /kind cleanup Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/triage accepted


Question: event.InvolvedObject.FieldPath, event.Type, event.Reason, <del> event.Source, </del> <ins> formatEventSource(event.Source), </ins> event.Message, ); err != nil { return err

Greetings, currently, when doing then in column, the ui result is maybe can do something like this instead? remove then join it with
How about something like ?
Greetings, thanks for the interest, I thought I may throw hacking on it. btw, how about for ?
For we can just print since it's master's component.
Hi looks like there is the same issue reported on the OpenShift Origin github: It would make more sense to solve that upstream. Should we do something together to solve it?
There is a pull request for that:


Question: cmd.Flags().DurationVar(&options.Timeout, "timeout", 0, "Only relevant during a force apply. The length of time to wait before giving up on a delete of the old resource, zero means determine a timeout from the size of the object. Any other values should contain a corresponding time unit (e.g. 1s, 2m, 3h).") cmdutil.AddValidateFlags(cmd) cmd.Flags().StringVarP(&options.Selector, "selector", "l", "", "Selector (label query) to filter on, supports '=', '==', and '!='.") <del> cmd.Flags().Bool("all", false, "[-all] to select all the specified resources.") </del> <ins> cmd.Flags().Bool("all", false, "select all resources in the namespace of the specified resource types.") </ins> cmd.Flags().StringArray("prune-whitelist", []string{}, "Overwrite the default whitelist with <group/version/kind> for --prune") cmdutil.AddDryRunFlag(cmd) cmdutil.AddPrinterFlags(cmd)

It actually doesn't support --all.
/assign
ohh, thanks, will make a pr to address that right now..


Question: fmt.Fprintf(out, "Labels:t%sn", formatLabels(pod.Labels)) fmt.Fprintf(out, "Status:t%sn", string(pod.Status.Phase)) fmt.Fprintf(out, "Replication Controllers:t%sn", printReplicationControllersByLabels(rcs)) <ins> fmt.Fprintf(out, "Containers:n") describeContainers(pod.Status.ContainerStatuses, out) </ins> if len(pod.Status.Conditions) > 0 { fmt.Fprint(out, "Conditions:n  TypetStatusn") for _, c := range pod.Status.Conditions {

Kubectl describe pods id only has top level pod information, without any information related to containers. Taking kube-dns pod as an example, which has 3 containers specified: If a pod with RestartAlways policy, but all contianers are in crashloop, the user cannot interpret the true state based on above information. Proposed to include ContainerStatus here.
Why isn't this redundant with ?
is for kubectl get pods id.


Question: return nil, []error{err} } if proxy != nil { <del> return []error{fmt.Errorf("Connection to %q uses proxy %q. If that is not intended, adjust your proxy settings", url, proxy)}, nil </del> <ins> return []error{fmt.Errorf("Connection to %q uses proxy %q. If that is not intended, adjust your proxy settings", u, proxy)}, nil </ins> } return nil, nil }

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: When setting an HTTPPROXY with an IPv6 address, the is no HTTPProxy warning. What you expected to happen: There should be a warning that a proxy is being used. How to reproduce it (as minimally and precisely as possible): Use Kubeadm with IPv6 addresses, and IPv6 proxy in the environment. Such as: NOPROXY=2001:420:2c50:2021::/64 Anything else we need to know?: Environment: Kubernetes version (use ): 9 Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others:
/assign rpothier
/sig cluster-lifecycle
/area ipv6


Question: "container/list" ) <del> // removeFromFIFOFunc removes a designated element from the list. // The complexity of the runtime cost is O(1) // It returns the request that has been removed from the list, // it returns nil if the request has already been removed. </del> <ins> // removeFromFIFOFunc removes a designated element from the list // if that element is in the list. // The complexity of the runtime cost is O(1). // The returned value is the element removed, if indeed one was removed, // otherwise `nil`. </ins> type removeFromFIFOFunc func() *request // walkFunc is called for each request in the list in the // oldest -> newest order. // ok: if walkFunc returns false then the iteration stops immediately. <ins> // walkFunc may remove the given request from the fifo, // but may not mutate the fifo in any othe way. </ins> type walkFunc func(*request) (ok bool) // Internal interface to abstract out the implementation details

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via -- Kubernetes version (use ):Cloud provider or hardware configuration: OS (e.g: ):Kernel (e.g. ):Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig api-machinery
/triage accepted
adds a defensive check so we can avoid a panic. I was looking into why can be . This is the call chain, from the stack trace: makes sure there is at least one request waiting - acquires a lock on the queueset in the beginning. So should have at least one request waiting in a queue and that queue should be selected. could this be an issue with the locking somewhere, any thoughts
/cc
is the client version, not the server. In that commit, line 725 of is a comment.
check the first link at the top of the issue description -
Something is violating locking or counting discipline. Perhaps some cheap sanity checks on counts (like, never negative) would at least confirm that this is where the problem lies. With locking or counting being done wrong, a defense in one place will just make symptoms appear elsewhere. We need to find the problem. The needed discipline is very mechanical. A careful read of all the code should find any violations.
Also, we could make queueSet::UpdateObservations check that the queueSet has the right sums over its queues (under the queueSet lock, of course). I think it is called infrequently enough for that to be tolerable. The point would be to reveal something to us developers. We could even make it do repair if an error is found; this would not really solve the problem, of course, but might make a helpful reduction in the severity of the symptoms.
Also, some comments are outdated, incomplete, or missing. Some comments on the fields in reference a now-non-existent field . .. reads a bit odd, there is no "array" in sight. has no comment. The presence of "without ..." suggests that it is adding information, but as far as I can tell it is not. I would feel more comfortable with an explicit statement about concurrency and that returned .

Perhaps something like updating the comment on to say something like the following. .. should also describe the significance of the parameters. Since it is the same as for , a reference to that would suffice. Checking that locking discipline is followed would be easier if each non-Locked method of queueSet had a comment that explicitly says whether the method is lock-oblivious or an entry method. This would make it a local check to avoid the case of an entry method calling another entry method (which would cause a deadlock, not the symptom in this Issue). The function comment on uses the wrong capitalization for the function name. s/modify/modifying/
.. continued issues with comments. s/returns/returned/ The fields of should be re-ordered to start with immutable ones then have the mutable ones, and have a comment about locking. would be better named . The values of type should have comments defining their meaning. Note that both and can involve a timeout, and the comments/log-messages in say simply "time out", but they are different timeouts --- one is the timeout defined in APF and the other is the timeout/cancel from the request's context. should be named . Why are the and methods of public? The method of should be named .
and do not take into account the possibility that the decision was already set to . This obliviousness can lead to confusion about what is in and mis-counting. I.e., cause the symptom in this Issue.
I will propose a fix.
Thanks for your investigation. I still have some confusions need your clarification. the above issue occur under what conditions? As far as I know, will be set to just at the stage of for timeout, and this occurs at the , but will set decision prior to this and decision can only be set once. the can only be set once, is this parameter a bit redundant? two identical requests arrive at the same apiserver almost at the same time, what will happen and how to deal with it?
given particular request X might not get dispatched during its ; request X might have to wait a while if there are not enough seats available at the moment. The goroutine that calls for X then proceeds to call , which calls --- and that calls the method of X's . If X's context times out or is cancelled before X is dispatched or rejected by APF logic then X's gets set to .

hope my answer to 1. also answers this. am not sure I understand what potential problem you are thinking about here. Each request is a distinct , even if the client and request message are identical. In turn, each request gets a distinct .
Thank you for your clarification. Aout question 2, I mean the has been set to , the setting of will no longer work. maybe nil is better.
The attempt to set a request's decision to is made at dispatch time. This might happen after the request's context has been canceled or timed out and the method has thus already set the decision to .