Question: return false, nil } <ins> // hasReferencedToken returns true if the serviceAccount references a service account token secret func (e *TokensController) hasReferencedToken(serviceAccount *v1.ServiceAccount) (bool, error) { if len(serviceAccount.Secrets) == 0 { return false, nil } allSecrets, err := e.listTokenSecrets(serviceAccount) if err != nil { return false, err } referencedSecrets := getSecretReferences(serviceAccount) for _, secret := range allSecrets { if referencedSecrets.Has(secret.Name) { return true, nil } } return false, nil } </ins> func (e *TokensController) secretUpdateNeeded(secret *v1.Secret) (bool, bool, bool) { caData := secret.Data[v1.ServiceAccountRootCAKey] needsCA := len(e.rootCA) > 0 && bytes.Compare(caData, e.rootCA) != 0

At least twice now, we've created a namespace and seen that the corresponding service account default secret token is never created. (The service account itself is, but not the default secret.) Namespace creation + apiserver logs logstash-ingester: namespace creation triggered service account creation but no secret logstash-broker: namespace creation triggered service account creation and secret creation Resulting service accounts Resulting secrets Controller-manager logs for broken case Controller-manager logs for working case Version Info:
Dupe of , fixed in
Actually, not a dupe. This may be a consequence of other refactors to controller logic - possibly we are missing / dropping events out of the queue. I would expect it to get fixed on the sync interval though.
We are also seeing this happening from time to time. In our case, it wasn't fixed in the next sync interval but it took actually a little bit more of 24 hours.
can we resolve this and remove it from the 1.6 milestone?
fixed in in 1.4
We just hit this exact issue running 1.5.2. Maybe the fix didn't get the root cause or there was a regression since 1.4? This should be reopened. I waited about an hour to see if the sync interval would resolve this but it did not. I fixed by deleting and recreating the namespace. I can share logs, but they are identical to what the OP noted. Version info:
Working with this seems like it's not fixed in v1.5.2 -- what should we do here? We don't have permission to re-open the ticket, but filing the same ticket with the same logs (just fresher dates) seems passive-aggressive. :-) What's your preference?
no problem, just getting back to this. can reopen this issue.
can you recreate with the controller manager running at and include controller manager log lines from ?
Fresh repo, offending namespace "foo96", with "--v=5" since we already have that logging level in place on my dev stack. This took 96 repros of a "create namespace / monitor for token creation / delete namespace" loop.

Our script's commands, starting just after 16:36:43: Controller manager log lines from The first reference to the offending namespace (foo96) and the one other entry afterwards: All controller manager log lines citing the offending namespace: For comparison, the previous successfully-created namespace: Status as of 20 minutes later:
Also, here's the apiserver logs + controller manager logs, for more action/reaction context: Namespace: foo96 (problem) Namespace: foo95 (success)
that's helpful, thanks. it looks like while the token controller is processing the creation event for the service account, it does a live lookup against etcd and fails to find the service account. what version of etcd are you running with? are you running HA etcd? do you have quorum read enabled?
We run etcd version 2.3.7 (previously saw this on 2.2.x as well). We run etcd HA, with a 3 node cluster. We do not run the apiserver with the --etcd-quorum-read flag enabled.
(Note: My specific logs above from this morning are on etcd 2.2.5, but yeah we've seen repro on 2.3.7 as well.)
yeah, the non-quorum HA etcd config is the issue. I can look into updating the token controller to tolerate that, but I'm positive there are similar issues in many controllers that assume live lookups are reliable
Ahh excellent, glad to have an explanation! So -- now that we know what to google for, we found -- where you and clayton addressed what appears to be this same problem for service accounts, and specifically noted that you wanted to avoid recommending quorum reads. Why is the answer different here? Is it because the downstream impact of non-quorum reads is too diverse to reliably retry every relevant action? (Totally fine if that's the answer, just trying to make sure we understand the reason for what appears to be an entirely opposite conclusion to the same type of problem.)

I still think controllers should be built to tolerate weak reads, and I'm willing to fix the token controller bug here, but three factors make me think this position is a losing battle: quorum read got a lot cheaper with etcd3 there are strong opinions that etcd quorum read should be on () which means that very few people are writing controllers designed to tolerate weak reads all the CI tests I'm aware of run with a single etcd or with quorum reads on
Perfect, works for us. As for fixing this particular token controller bug, I no longer have skin in that game -- your call entirely. :-)


Question: o.PrintFlags.AddFlags(cmd) cmd.Flags().BoolVar(&o.overwrite, "overwrite", o.overwrite, "If true, allow annotations to be overwritten, otherwise reject annotation updates that overwrite existing annotations.") <ins> cmd.Flags().BoolVar(&o.list, "list", o.list, "If true, display the annotations for a given resource.") </ins> cmd.Flags().BoolVar(&o.local, "local", o.local, "If true, annotation will NOT contact api-server but run locally.") cmd.Flags().StringVarP(&o.selector, "selector", "l", o.selector, "Selector (label query) to filter on, not including uninitialized ones, supports '=', '==', and '!='.(e.g. -l key1=value1,key2=value2).") cmd.Flags().StringVar(&o.fieldSelector, "field-selector", o.fieldSelector, "Selector (field query) to filter on, supports '=', '==', and '!='.(e.g. --field-selector key1=value1,key2=value2). The server only supports a limited number of field queries per type.")

<!-- Please only use this template for submitting enhancement requests --What would you like to be : add --list option to kubectl annotate command Why is this needed: like kubectl label commandshow annotations quickly
/sig cli These SIGs are my best guesses for this issue. Please comment if I am incorrect about one. I am a bot run by vllry.
/assign


Question: if err != nil { return err } <ins> if len(externalIPs) > 0 { for _, externalIP := range externalIPs { err = testReachabilityOverExternalIP(externalIP, servicePort, pod) if err != nil { return err } } } </ins> } return nil }

What would you like to be : e2e test for service ExternalIPs Why is this needed: There is no test covering Service ExternalIPs Since we can not assume that the e2e binary will have connectivity against the nodes and that we can add routes to the environment, we are going to test it from a pod. I've verified that this works locally with the following commands: The test should leverage current framework, maybe something as A good example is here, maybe adding a ?
/sig network /help wanted /triage accepted
/help wanted /good-first-issue
/assign
I'd like to pick this up!
Ok,I'm not started :) /unassign /assign
There are two different externalIP codepaths, one for the case where the claimed external IP exists as an IP on the default interface, and one for where it does not. I suppose we could just temporarily add an IP to the default interface...
good idea Dan Please remember, it has to work for IPv6 and for IPv4 ... anyway, we can address these things during the review
This has been partially implemented with . Covering the code path for when an IP is does not exist on a node. I'll assign this back to you so you can split this issue or close it in favor of another issue. Thanks! /unassign /assign
what is the different codepath for ExternalIPs that already exist on the host? ~My reading is that the only difference is that it opens a local socket on that port, can you expand on this?~ Found it
so, it seems that this work in some environments because they are not using the bridge plugin why is not allowed that pods can reach ExternalIPs?
Wow, this is ancient and needs some fixing. This is not about preventing pod traffic from reaching those IPs. I had to really stare at it, and I am not 100% in my interpretation. First, we recognize such traffic from "off machine" (!physdev-is-in && !src=LOCAL, which we should probably switch to localDetector) and steer it to the Service. Second, we recognize ANY such traffic that is destined for a LOCAL address and shortcut that to the Service, Implicitly, traffic destined for this IP where the IP is not local goes to the system's default routing logic.

The reliance on the bridge plugin is awful and archaic and I will buy a beverage of your choice for whomever gets rid of that. I think localDetector can cover this, but not 100% sure
but the ExternalIP belongs to a service and we(kube-proxy) "knows" where to send it (to the service) , that is the part I'm not sure to understand, because whatever the user is doing with that ExternalIP it has to send it back to the cluster to reach the service. However, there is always context that is important and maybe I'm missing something. Assuming we want to handle ALL the ExternalIP traffic with kube-proxy, the logic will be much simpler: traffic to an externalIP will be ALWAYS sent to the Service (DNAT) if traffic is NOT LOCAL, we also SNAT
/remove-good-first-issue /remove-help-wanted


Question: v1.NodeOutOfDisk, v1.NodeMemoryPressure, v1.NodeDiskPressure, <ins> v1.NodePIDPressure, </ins> // We don't change 'NodeNetworkUnavailable' condition, as it's managed on a control plane level. // v1.NodeNetworkUnavailable, }

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . If the matter is security related, please disclose it privately via --Is this a BUG REPORT or FEATURE REQUEST?: What happened: when the node lost connection with master , using 'kubectl describe node' find the NodePIDPressure is false , no switch to unknown. What you expected to happen: PIDPressure Unknown Fri, 14 Sep 2018 05:13:09 +0800 Fri, 14 Sep 2018 05:13:53 +0800 NodeStatusUnknown Kubelet stopped posting node status. How to reproduce it (as minimally and precisely as possible): stop kubelet, wait the node Notready check the node status 'kubectl describe node' Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others:
/sig node


Question: } func (s *ServiceController) loadBalancerName(service *api.Service) string { <del> return cloudprovider.GetLoadBalancerName(s.clusterName, service.Namespace, service.Name) </del> <ins> return cloudprovider.GetLoadBalancerName(service) </ins> } func getTCPPorts(service *api.Service) ([]int, error) {

Since this PR got merged, the ELB e2e test, which was consistently succeeding, is now consistently failing: Revert now that is in, and fix the tests that were relying on it. (details) Commit by arob Revert now that is in, and fix the tests that were relying on it. The file was modified The file was modified The file was modified I'm going to roll it back to confirm.


Question: inputFieldMappingFuncs: map[reflect.Type]FieldMappingFunc{}, inputDefaultFlags:      map[reflect.Type]FieldMatchingFlags{}, } <ins> c.RegisterConversionFunc(byteSliceCopy) return c } // Prevent recursing into every byte... func byteSliceCopy(in *[]byte, out *[]byte, s Scope) error { *out = make([]byte, len(*in)) copy(*out, *in) return nil </ins> } // Scope is passed to conversion funcs to allow them to continue an ongoing conversion.

E.g., it calls Convert() for every byte of a slice of uint8. E0330 22:15:23. 5352 ] TIM: converting bool E0330 22:15:23. 5352 ] TIM: converting []v1.NamedExtension E0330 22:15:23. 5352 ] TIM: converting []v1.NamedCluster E0330 22:15:23. 5352 ] TIM: converting []v1.NamedCluster E0330 22:15:23. 5352 ] TIM: converting v1.Cluster E0330 22:15:23. 5352 ] TIM: converting v1.Cluster E0330 22:15:23. 5352 ] TIM: converting string E0330 22:15:23. 5352 ] TIM: converting string E0330 22:15:23. 5352 ] TIM: converting bool E0330 22:15:23. 5352 ] TIM: converting string E0330 22:15:23. 5352 ] TIM: converting []uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 E0330 22:15:23.

5352 ] TIM: converting uint8 E0330 22:15:23. 5352 ] TIM: converting uint8 ... thousands of these.
There's an easy fix for this :/


Question: }, expectedElements: []string{"Terminating (lasts 10y)"}, }, <ins> { name: "pvc-datasource", pvc: &corev1.PersistentVolumeClaim{ ObjectMeta: metav1.ObjectMeta{ Namespace: "foo", Name:      "bar", }, Spec: corev1.PersistentVolumeClaimSpec{ VolumeName:       "volume10", StorageClassName: &goldClassName, DataSource: &corev1.TypedLocalObjectReference{ Name: "srcpvc", Kind: "PersistentVolumeClaim", }, }, Status: corev1.PersistentVolumeClaimStatus{}, }, expectedElements: []string{"nDataSource:n  Name:      srcpvcn  Kind:      PersistentVolumeClaim"}, }, { name: "snapshot-datasource", pvc: &corev1.PersistentVolumeClaim{ ObjectMeta: metav1.ObjectMeta{ Namespace: "foo", Name:      "bar", }, Spec: corev1.PersistentVolumeClaimSpec{ VolumeName:       "volume10", StorageClassName: &goldClassName, DataSource: &corev1.TypedLocalObjectReference{ Name:     "src-snapshot", Kind:     "VolumeSnapshot", APIGroup: &snapshotAPIGroup, }, }, Status: corev1.PersistentVolumeClaimStatus{}, }, expectedElements: []string{"DataSource:n  Name:      src-snapshotn  Kind:      VolumeSnapshotn"}, }, </ins> } for _, test := range testCases {

Currently there's now way for a user to check what they entered in the DataSource field of a PVC create request. In the past, this wasn't as confusing because if one were to submit invalid info (unsupported Kind) the verification checks would fail and the request would be rejected with a helpful message indicating why. As of 1.14 that's no longer the case, a user is able to put anything they wish in these fields and the create is performed and they get a PVC, and there's no way for them to see what is or isn't set. Adding output of DataSource to would be useful, not only for invalid cases where the input might just get eaten, but also for users to have the ability to check for originating sources of their PVCs.
/sig storage
/assign
GitHub didn't allow me to assign the following users: me. Note that only and repo collaborators can be assigned and that issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : /assign me Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten


Question: } func TestActiveReplicaSetsFiltering(t *testing.T) { <del> var replicaSets []*apps.ReplicaSet replicaSets = append(replicaSets, newReplicaSet("zero", 0)) replicaSets = append(replicaSets, nil) replicaSets = append(replicaSets, newReplicaSet("foo", 1)) replicaSets = append(replicaSets, newReplicaSet("bar", 2)) expectedNames := sets.NewString() for _, rs := range replicaSets[2:] { expectedNames.Insert(rs.Name) } got := FilterActiveReplicaSets(replicaSets) gotNames := sets.NewString() for _, rs := range got { gotNames.Insert(rs.Name) </del> <ins> rsUuid := uuid.NewUUID() tests := []struct { name            string replicaSets     []*apps.ReplicaSet wantReplicaSets []*apps.ReplicaSet }{ { name: "Filters active replica sets", replicaSets: []*apps.ReplicaSet{ newReplicaSet("zero", 0, rsUuid), nil, newReplicaSet("foo", 1, rsUuid), newReplicaSet("bar", 2, rsUuid), }, wantReplicaSets: []*apps.ReplicaSet{ newReplicaSet("foo", 1, rsUuid), newReplicaSet("bar", 2, rsUuid), }, }, </ins> } <del> if diff := cmp.Diff(expectedNames.List(), gotNames.List()); diff != "" { t.Errorf("Active replica set names (-want,+got):n%s", diff) </del> <ins> for _, test := range tests { t.Run(test.name, func(t *testing.T) { gotReplicaSets := FilterActiveReplicaSets(test.replicaSets) if diff := cmp.Diff(test.wantReplicaSets, gotReplicaSets); diff != "" { t.Errorf("Active replica set names (-want,+got):n%s", diff) } }) </ins> } }

During implementation of KEP 3393 (PodReplacementPolicy for Jobs) we found that the unit tests of do not follow common patterns established in k/k. 1) The tests are not written using table driven design (). 2) The tests use rather than . This probably should be done as two separate PRs but wanted to highlight some areas that could use some cleanup. Not using means that we don't see diffs in the unit tests that makes this difficult. And not using table driven tests makes it difficult to add new test cases as it usually is brought up during review that we should use a table-driven test but the entire file does not obey this format.
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig apps /kind cleanup /help /good-first-issue
This request has been marked as suitable for new contributors. Please ensure that the issue body includes answers to the following questions: Why are we solving this issue? To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points? Does this issue have zero to low barrier of entry? How can the assignee reach out to you for help? For more details on the requirements of such an issue, please see and ensure that they are met. If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : /sig apps /kind cleanup /help /good-first-issue Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I'd like to work on it /assign
I'd also like to make some contributions. /assign
Looks like this need two PRs. tests are not written using table driven design (). tests use Differences rather than Cmp.Diff. I think I can work on the 2nd one and leave the 1st one to you.

Sounds good?
By looking through the , I found the main function to comapre the results have already used cmp.Diff. Can you elaborate more on which part you expect to use cmp.Diff? func verifyDeletedAndPatchedPods(t testing.T, client fake.Clientset, wantDeletedPodNames, wantPatchedPodNames sets.String) { t.Helper() deletedPodNames := getDeletedPodNames(client) if diff := cmp.Diff(wantDeletedPodNames, deletedPodNames); diff != "" { t.Errorf("Deleted pod names (-want,+got):n%s", diff) } patchedPodNames := getPatchedPodNames(client) if diff := cmp.Diff(wantPatchedPodNames, patchedPodNames); diff != "" { t.Errorf("Patched pod names (-want,+got):n%s", diff) } }
I am really sorry about this. Too much multitasking!! I meant not
Hey ! Sorry, I didn't see this before. No problem at all, but consider that one change depends on the other, and since my task implies a big refactor, there might be conflicts when merging your PR... That's why I see two possible scenarios happening: Your PR gets merged before I raise mine: Then fine, I'll rebase your changes and adapt my code Mine is ready before yours is merged: In this case, I propose you commit your changes to my PR. Both of us will appear as coauthors of the change Getting a PR merged in the Kubernetes project takes quite some time (trust me, I talk by experience and can confirm ) WDYT?
Let's see how it goes. We can always rebase the code change to adapt new.
btw, did you already begin to work on task 1? If not, I may just go ahead and work on it right away since I have already dig into it.
yes, I already started And If you ask me, better to keep it this way so we both have an opportunity to have a contribution
cool. Sounds good.
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Hi if this issue needs to be worked on any further, please assign me for the same.
has a PR up that will solve the second part of this issue.


Question: increasedSize += increasedSizes[key] } setMigSizes(increasedSizes) <del> ExpectNoError(WaitForClusterSizeFunc(c, func(size int) bool { return size == increasedSize }, DNSscaleUpTimeout)) </del> <ins> Expect(WaitForClusterSizeFunc(c, func(size int) bool { return size == increasedSize }, scaleUpTimeout)).NotTo(HaveOccurred()) By("Wait for kube-dns scaled to expected number") getExpectReplicasLinear := getExpectReplicasFuncLinear(c, &DNSParams_1) Expect(waitForDNSReplicasSatisfied(c, getExpectReplicasLinear, DNSdefaultTimeout)).NotTo(HaveOccurred()) By("Replace the dns autoscaling parameters with another testing parameters") Expect(updateDNSScalingConfigMap(c, packDNSScalingConfigMap(DNSParams_3.data))).NotTo(HaveOccurred()) </ins> By("Wait for kube-dns scaled to expected number") <del> ExpectNoError(waitForDNSReplicasSatisfied(c, increasedSize, DNSdefaultTimeout)) </del> <ins> getExpectReplicasLinear = getExpectReplicasFuncLinear(c, &DNSParams_3) Expect(waitForDNSReplicasSatisfied(c, getExpectReplicasLinear, DNSdefaultTimeout)).NotTo(HaveOccurred()) </ins> By("Restoring cluster size") setMigSizes(originalSizes) <del> framework.ExpectNoError(framework.WaitForClusterSize(c, nodeCount, DNSscaleDownTimeout)) </del> <ins> Expect(framework.WaitForClusterSize(c, sum, scaleDownTimeout)).NotTo(HaveOccurred()) </ins> By("Wait for kube-dns scaled to expected number") <del> ExpectNoError(waitForDNSReplicasSatisfied(c, nodeCount, DNSdefaultTimeout)) </del> <ins> Expect(waitForDNSReplicasSatisfied(c, getExpectReplicasLinear, DNSdefaultTimeout)).NotTo(HaveOccurred()) </ins> }) It("kube-dns-autoscaler should scale kube-dns pods in both nonfaulty and faulty scenarios", func() { By("--- Scenario: should scale kube-dns based on changed parameters ---") <del> By("Replace the dns autoscaling parameters with the second testing parameters") ExpectNoError(updateDNSScalingConfigMap(c, packDNSScalingConfigMap(DNSParams_2))) </del> <ins> By("Replace the dns autoscaling parameters with another testing parameters") Expect(updateDNSScalingConfigMap(c, packDNSScalingConfigMap(DNSParams_3.data))).NotTo(HaveOccurred()) </ins> By("Wait for kube-dns scaled to expected number") <del> ExpectNoError(waitForDNSReplicasSatisfied(c, int(math.Ceil(float64(nodeCount)/2.0)), DNSdefaultTimeout)) </del> <ins> getExpectReplicasLinear := getExpectReplicasFuncLinear(c, &DNSParams_3) Expect(waitForDNSReplicasSatisfied(c, getExpectReplicasLinear, DNSdefaultTimeout)).NotTo(HaveOccurred()) </ins> By("--- Scenario: should re-create scaling parameters with default value when parameters got deleted ---") By("Delete the ConfigMap for autoscaler") err := deleteDNSScalingConfigMap(c) <del> ExpectNoError(err) </del> <ins> Expect(err).NotTo(HaveOccurred()) </ins> By("Wait for the ConfigMap got re-created") configMap, err := waitForDNSConfigMapCreated(c, DNSdefaultTimeout) <del> ExpectNoError(err) </del> <ins> Expect(err).NotTo(HaveOccurred()) </ins> By("Check the new created ConfigMap got the same data as we have") Expect(reflect.DeepEqual(previousParams, configMap.Data)).To(Equal(true)) <del> By("Replace the dns autoscaling parameters with the second testing parameters") ExpectNoError(updateDNSScalingConfigMap(c, packDNSScalingConfigMap(DNSParams_2))) </del> <ins> By("Replace the dns autoscaling parameters with another testing parameters") Expect(updateDNSScalingConfigMap(c, packDNSScalingConfigMap(DNSParams_2.data))).NotTo(HaveOccurred()) </ins> By("Wait for kube-dns scaled to expected number") <del> ExpectNoError(waitForDNSReplicasSatisfied(c, int(math.Ceil(float64(nodeCount)/2.0)), DNSdefaultTimeout)) </del> <ins> getExpectReplicasLinear = getExpectReplicasFuncLinear(c, &DNSParams_2) Expect(waitForDNSReplicasSatisfied(c, getExpectReplicasLinear, DNSdefaultTimeout)).NotTo(HaveOccurred()) </ins> By("--- Scenario: should recover after autoscaler pod got deleted ---") By("Delete the autoscaler pod for kube-dns") <del> ExpectNoError(deleteDNSAutoscalerPod(c)) </del> <ins> Expect(deleteDNSAutoscalerPod(c)).NotTo(HaveOccurred()) </ins> <del> By("Replace the dns autoscaling parameters with the third testing parameters") ExpectNoError(updateDNSScalingConfigMap(c, packDNSScalingConfigMap(DNSParams_3))) </del> <ins> By("Replace the dns autoscaling parameters with another testing parameters") Expect(updateDNSScalingConfigMap(c, packDNSScalingConfigMap(DNSParams_1.data))).NotTo(HaveOccurred()) </ins> By("Wait for kube-dns scaled to expected number") <del> ExpectNoError(waitForDNSReplicasSatisfied(c, int(math.Ceil(float64(nodeCount)/3.0)), DNSdefaultTimeout)) </del> <ins> getExpectReplicasLinear = getExpectReplicasFuncLinear(c, &DNSParams_1) Expect(waitForDNSReplicasSatisfied(c, getExpectReplicasLinear, DNSdefaultTimeout)).NotTo(HaveOccurred()) </ins> }) }) <ins> type DNSParamsLinear struct { data            map[string]string nodesPerReplica float64 coresPerReplica float64 } type getExpectReplicasFunc func(c clientset.Interface) int func getExpectReplicasFuncLinear(c clientset.Interface, params *DNSParamsLinear) getExpectReplicasFunc { return func(c clientset.Interface) int { var replicasFromNodes float64 var replicasFromCores float64 nodes := framework.GetReadySchedulableNodesOrDie(c).Items if params.nodesPerReplica > 0 { replicasFromNodes = math.Ceil(float64(len(nodes)) / params.nodesPerReplica) } if params.coresPerReplica > 0 { replicasFromCores = math.Ceil(float64(getScheduableCores(nodes)) / params.coresPerReplica) } return int(math.Max(1.0, math.Max(replicasFromNodes, replicasFromCores))) } } func getScheduableCores(nodes []v1.Node) int64 { var sc resource.Quantity for _, node := range nodes { if !node.Spec.Unschedulable { sc.Add(node.Status.Capacity[v1.ResourceCPU]) } } scInt64, scOk := sc.AsInt64() if !scOk { framework.Logf("unable to compute integer values of schedulable cores in the cluster") return 0 } return scInt64 } </ins> func fetchDNSScalingConfigMap(c clientset.Interface) (*v1.ConfigMap, error) { cm, err := c.Core().ConfigMaps(DNSNamespace).Get(DNSAutoscalerLabelName) if err != nil {

https://k8s- Failed: [] DNS horizontal autoscaling kube-dns-autoscaler should scale kube-dns pods in both nonfaulty and faulty scenarios {Kubernetes e2e suite}
https://k8s- Failed: [] DNS horizontal autoscaling kube-dns-autoscaler should scale kube-dns pods in both nonfaulty and faulty scenarios {Kubernetes e2e suite}
https://k8s- Failed: [] DNS horizontal autoscaling kube-dns-autoscaler should scale kube-dns pods in both nonfaulty and faulty scenarios {Kubernetes e2e suite}
https://k8s- Failed: [] DNS horizontal autoscaling kube-dns-autoscaler should scale kube-dns pods in both nonfaulty and faulty scenarios {Kubernetes e2e suite}
https://k8s- Failed: [] DNS horizontal autoscaling kube-dns-autoscaler should scale kube-dns pods in both nonfaulty and faulty scenarios {Kubernetes e2e suite}
https://k8s- Failed: [] DNS horizontal autoscaling kube-dns-autoscaler should scale kube-dns pods in both nonfaulty and faulty scenarios {Kubernetes e2e suite}
https://k8s- Failed: [] DNS horizontal autoscaling kube-dns-autoscaler should scale kube-dns pods in both nonfaulty and faulty scenarios {Kubernetes e2e suite}
https://k8s- Failed: [] DNS horizontal autoscaling kube-dns-autoscaler should scale kube-dns pods in both nonfaulty and faulty scenarios {Kubernetes e2e suite}
https://k8s- Failed: [] DNS horizontal autoscaling kube-dns-autoscaler should scale kube-dns pods in both nonfaulty and faulty scenarios {Kubernetes e2e suite}
https://k8s- Failed: [] DNS horizontal autoscaling kube-dns-autoscaler should scale kube-dns pods in both nonfaulty and faulty scenarios {Kubernetes e2e suite}
https://k8s- Failed: [] DNS horizontal autoscaling kube-dns-autoscaler should scale kube-dns pods in both nonfaulty and faulty scenarios {Kubernetes e2e suite}
https://k8s- Failed: [] DNS horizontal autoscaling kube-dns-autoscaler should scale kube-dns pods in both nonfaulty and faulty scenarios {Kubernetes e2e suite}
https://k8s- Failed: [] DNS horizontal

autoscaling kube-dns-autoscaler should scale kube-dns pods in both nonfaulty and faulty scenarios {Kubernetes e2e suite}
[FLAKE-PING] This flaky-test issue would love to have more attention.
https://k8s- Failed: [] DNS horizontal autoscaling kube-dns-autoscaler should scale kube-dns pods in both nonfaulty and faulty scenarios {Kubernetes e2e suite}
should this issue be moved into the 1.5 milestone and is it a release blocker? The test seems to flake fairly infrequently but we need to get the issue triaged as to whether it's a 1.5 release blocker. Please update this issue by 1PM PST 30 November 2016. Thanks! cc:
Since the feature it is testing is introduced for 1.5, so it should have the 1.5 milestone? This test is flaking because it is not well written, hence I don't think it is a release blocker. And I already sent out a which got LGTM before but recently rebased.
https://k8s- Failed: [] DNS horizontal autoscaling kube-dns-autoscaler should scale kube-dns pods in both nonfaulty and faulty scenarios {Kubernetes e2e suite}
Thanks for triaging
https://k8s- Failed: [] DNS horizontal autoscaling kube-dns-autoscaler should scale kube-dns pods in both nonfaulty and faulty scenarios {Kubernetes e2e suite}
https://k8s- Failed: [] DNS horizontal autoscaling kube-dns-autoscaler should scale kube-dns pods in both nonfaulty and faulty scenarios {Kubernetes e2e suite}
https://k8s- Failed: [] DNS horizontal autoscaling kube-dns-autoscaler should scale kube-dns pods in both nonfaulty and faulty scenarios {Kubernetes e2e suite}


Question: setupMetrics() namespace := "test" endpoints := corev1.Endpoints{ <del> ObjectMeta: metav1.ObjectMeta{Name: "test-ep", Namespace: namespace, Labels: tc.epLabels}, </del> <ins> ObjectMeta: metav1.ObjectMeta{Name: "test-ep", Namespace: namespace, Labels: tc.epLabels, Annotations: tc.epAnnotations}, </ins> Subsets:    tc.subsets, }

If I modify a custom endpoint, the kubectl annotation is mirrored to the generated endpoint slice Since the kubect last-applied-configuration tracks the status of the object, it shouldn't be mirrored since it doesn't apply to the generated object a custom endpoint an update on any of the fields the generated endpoint slice annotations /sig network
ping
if we agree this is a bug, the fix is just and adding some tests of course :)
Thanks for catching this! Agree this is a bug that we should fix.
/triage accepted
/good-first-issue
This request has been marked as suitable for new contributors. Please ensure the request meets the requirements listed . If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : /good-first-issue Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Hey, I'm on it!
Hi While I have made the update mentioned in the above comment, I feel a bit lost on where to add/update tests. I understand that I probably shouldn't raise a PR without trying to add a few tests for this issue. So could you possibly guide me to the correct files(s) I should be looking into where the annotations for updated EndpointSlices are being tested?
Hey I think the easiest place to add tests might be here: You could add another test case where the Endpoints object had this label and then ensure that label was not copied over to an EndpointSlice.
Alright I'll check out these tests. Thank you!
/assign


Question: storage.requestedLabelSelector = label storage.requestedFieldSelector = field storage.requestedResourceVersion = resourceVersion <ins> storage.requestedResourceNamespace = api.Namespace(ctx) </ins> if err := storage.errors["watch"]; err != nil { return nil, err }

is always using the cross namespace context. This needs to be fixed to support scoped watches.


Question: } defer resp.Body.Close() if resp.StatusCode != http.StatusInternalServerError { <del> t.Errorf("expected status code %d, got %d", http.StatusOK, resp.StatusCode) </del> <ins> t.Errorf("expected status code %d, got %d", http.StatusInternalServerError, resp.StatusCode) </ins> } }

We've run into issues like where the Kubelet was up but not healthy. The monit monitoring for Kubelet should ensure it is "healthy" rather than just alive. We need to define what healthy means in this context and monitor it based on that. An alternate option is for the Kubelet to exit when it gets into a situation it can't serve from. This may lead to crashloops so we'd need to be careful going this route. The answer is probably a bit of both.
Courtesy of healthchecking with monit:
Health check is under this umbrella issue:
Will send a PR shortly.


Question: glog.V(4).Infof("EnsureLoadBalancer using floatingPool: %v", floatingPool) var internalAnnotation bool <del> internal := getStringFromServiceAnnotation(apiService, ServiceAnnotationLoadBalancerInternal, "true") </del> <ins> internal := getStringFromServiceAnnotation(apiService, ServiceAnnotationLoadBalancerInternal, "false") </ins> switch internal { case "true": glog.V(4).Infof("Ensure an internal loadbalancer service.")

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: Creating a service with created a load balancer without a floating IP. What you expected to happen: Load balancer should have a floating IP assigned by default. How to reproduce it (as minimally and precisely as possible): Deploy this manifest on a cluster and check the resulting load balancer. No floating IP will be assigned. Anything else we need to know?: The default behavior changed from k8s 1.7.x. It appears that an annotation that will add a floating IP, but the default should not have changed without warning. This also breaks the workflow. For example: Environment: Kubernetes version (use ): v1.8.0-rc.1+coreos.0 Cloud provider or hardware configuration**: openstack OS (e.g. from /etc/os-release): Container Linux by CoreOS 1520.4.0 (Ladybug) Kernel (e.g. ): 4.13.3-coreos Install tools: Others:
There are no sig labels on this issue. Please by: a sig: e.g., to notify the contributor experience sig, OR the label manually: e.g., to apply the label Note: Method 1 will trigger an email to the group. You can find the group list and label list . The in the method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals
/sig openstack
Reiterating the mentions to trigger a notification: <detailsIn response to : Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Oops, default should be external. /assign


Question: 1. First we need a CA to later sign the client certificate: <del> ``` shell openssl req -nodes -new -x509 -keyout ca.key -out ca.crt ``` </del> <ins> ``` shell openssl req -nodes -new -x509 -keyout ca.key -out ca.crt ``` </ins> 2. Then we create a client cert signed by this CA for the user `development` in the superuser group `system:masters`: <del> ``` shell openssl req -out client.csr -new -newkey rsa:4096 -nodes -keyout client.key -subj "/CN=development/O=system:masters" openssl x509 -req -days 365 -in client.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out client.crt ``` </del> <ins> ``` shell openssl req -out client.csr -new -newkey rsa:4096 -nodes -keyout client.key -subj "/CN=development/O=system:masters" openssl x509 -req -days 365 -in client.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out client.crt ``` </ins> 3. As curl requires client certificates in p12 format with password, do the conversion: <del> ``` shell openssl pkcs12 -export -in ./client.crt -inkey ./client.key -out client.p12 -passout pass:password ``` </del> <ins> ``` shell openssl pkcs12 -export -in ./client.crt -inkey ./client.key -out client.p12 -passout pass:password ``` </ins> 4. With these keys and certs in-place, we start the server: <del> ``` shell etcd & sample-apiserver --secure-port 8443 --etcd-servers http://127.0.0.1:2379 --v=7  --client-ca-file ca.crt  --kubeconfig ~/.kube/config  --authentication-kubeconfig ~/.kube/config  --authorization-kubeconfig ~/.kube/config ``` The first kubeconfig is used for the shared informers to access Kubernetes resources. The second kubeconfig passed to `--authentication-kubeconfig` is used to satisfy the delegated authenticator. The third kubeconfig passed to `--authorized-kubeconfig` is used to satisfy the delegated authorizer. Neither the authenticator, nor the authorizer will actually be used: due to `--client-ca-file`, our development X.509 certificate is accepted and authenticates us as `system:masters` member. `system:masters` is the superuser group such that delegated authorization is skipped. </del> <ins> ``` shell etcd & sample-apiserver --secure-port 8443 --etcd-servers http://127.0.0.1:2379 --v=7  --client-ca-file ca.crt  --kubeconfig ~/.kube/config  --authentication-kubeconfig ~/.kube/config  --authorization-kubeconfig ~/.kube/config ``` The first kubeconfig is used for the shared informers to access Kubernetes resources. The second kubeconfig passed to `--authentication-kubeconfig` is used to satisfy the delegated authenticator. The third kubeconfig passed to `--authorized-kubeconfig` is used to satisfy the delegated authorizer. Neither the authenticator, nor the authorizer will actually be used: due to `--client-ca-file`, our development X.509 certificate is accepted and authenticates us as `system:masters` member. `system:masters` is the superuser group such that delegated authorization is skipped. </ins> 5. Use curl to access the server using the client certificate in p12 format for authentication: <del> ``` shell curl -fv -k --cert client.p12:password  https://localhost:8443/apis/wardle.k8s.io/v1alpha1/namespaces/default/flunders ``` </del> <ins> ``` shell curl -fv -k --cert client.p12:password  https://localhost:8443/apis/wardle.k8s.io/v1alpha1/namespaces/default/flunders ``` Or use wget: ``` shell wget -O- --no-check-certificate  --certificate client.crt --private-key client.key  https://localhost:8443/apis/wardle.k8s.io/v1alpha1/namespaces/default/flunders ``` </ins> Note: Recent OSX versions broke client certs with curl. On Mac try `brew install httpie` and then: <del> ``` shell http --verify=no --cert client.crt --cert-key client.key  https://localhost:8443/apis/wardle.k8s.io/v1alpha1/namespaces/default/flunders ``` </del> <ins> ``` shell http --verify=no --cert client.crt --cert-key client.key  https://localhost:8443/apis/wardle.k8s.io/v1alpha1/namespaces/default/flunders ``` </ins>

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!--What happened: I wanted to test the latest version of the sample-apiserver. I still did not find instructions or a script for building it. I found the script, but that relies on the directory produced when building k/k. What you expected to happen: I expected this sample to have standalone build instructions or scripting. How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others:
I created a script of my own to build it:
Also: since the file has a fixed image reference, only someone who pushes to that image can use that file unmodified. The README should say something about this.
Also, the existing is not designed to be invoked by a developer who has done but rather is designed to work only in the context of a full build of k/k.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale Being addressed by


Question: gcloud compute routes delete  --project "${PROJECT}"  --quiet  <del> "${routes[@]::10}" || true </del> <ins> "${routes[@]::10}" </ins> routes=( "${routes[@]:10}" ) done

'kubernetes-soak-weekly-deploy' sometimes fails to deploy because the clean-up cluster step fails to delete the template: Then the subsequent cluster step fails to create the template, because it already exists: It keeps retrying and failing, because the template already exists, until it times out and the deployment fails. Suggested fixes: Cluster down should try harder to delete the template (maybe wait/retry). Cluster up should recognize that the template exists, and try to delete/recreate, or just re-use the existing.
Looks like ran in to this on Monday, May 11, 2015 around 1:00 PM PST. I'll assign this to my self and take a crack at fixing it.
This is causing most of our e2e tests to fail in continuous integration as well as users private e2e test clusters. Bumping to priority/p0 and v1.0 milestone. Saad thinks he's figured it out, and is working on a fix as we speak.
In essence... Saad: "I think what's happening is the previous clean up step to delete the managed instance group happens asynchronously, therefore there is a race condition to delete the template. Working on a fix".


Question: return future.Response(), err } <ins> func (az *azVirtualMachinesClient) Update(ctx context.Context, resourceGroupName string, VMName string, parameters compute.VirtualMachineUpdate, source string) (resp *http.Response, err error) { // /* Write rate limiting */ if !az.rateLimiterWriter.TryAccept() { err = createRateLimitErr(true, "VMUpdate") return } klog.V(10).Infof("azVirtualMachinesClient.Update(%q, %q): start", resourceGroupName, VMName) defer func() { klog.V(10).Infof("azVirtualMachinesClient.Update(%q, %q): end", resourceGroupName, VMName) }() mc := newMetricContext("vm", "update", resourceGroupName, az.client.SubscriptionID, source) future, err := az.client.Update(ctx, resourceGroupName, VMName, parameters) if err != nil { return future.Response(), err } err = future.WaitForCompletionRef(ctx, az.client.Client) mc.Observe(err) return future.Response(), err } </ins> func (az *azVirtualMachinesClient) Get(ctx context.Context, resourceGroupName string, VMName string, expand compute.InstanceViewTypes) (result compute.VirtualMachine, err error) { if !az.rateLimiterReader.TryAccept() { err = createRateLimitErr(false, "VMGet")

<!-- Please only use this template for submitting enhancement requests --What would you like to be : Instead of using the VM CreateOrUpdate () function of the Azure Go SDK, use the VM Update function () in the azureclient () Why is this needed: Currently, the Azure Cloud Provider is using the VM CreateOrUpdate call when e.g. detach/attaching disks to a VM. In some scenarios, this will re-create a VM that was deleted by an external application. This can lead to orphaned VMs or blocked resources. The Update function will only update an existing VM and will fail if the VM does not exist.
/sig azure
/assign
/sig cloud-provider
Could you describe more about the scenarios? Have you noticed such issues? I wonder adding etags to the API call could help to prevent this?
If you have an application that maintains your K8s clusters, e.g. takes care of OS update/patching. If the application wants to patch the OS of a worker node, it would drain the node first. If draining times out, it will go ahead and delete the VM anyway. But K8s might still be detaching disks. These detach operations, because they use CreateOrUpdate, will recreate the VM. Is there a reason why you would want to create a VM i.e. why CreateOrUpdate is required and Update would not work? AFAIK, Azure Compute RP does not support etags.
that sounds reasonable, thanks change to use function would be safer since it's only detach/attach disk operation. I will do that.
thanks for the information. Reasonable enough. /assign
/milestone v1.16
BTW, the reason why we only use func in the beginning is that Azure Compute go sdk did not provide a func at that time...
Great. Thanks!


Question: .:53 { errors health <ins> ready </ins> kubernetes $DNS_DOMAIN in-addr.arpa ip6.arpa { pods insecure <del> upstream </del> fallthrough in-addr.arpa ip6.arpa ttl 30 }

Which jobs are failing: Multiple jobs on . Which test(s) are failing: on the following jobs: Since when has it been failing: Since 5/17. Testgrid links: See above. Reason for failure: All the failures mentioned above seem to have to have the same root cause, namely failure while waiting for CoreDNS to be ready. Example: /sig testing /priority critical-urgent /kind failing test /milestone v1.15 /cc
Some of this (possible all of it) is due to , which is dependent on being merged.
/sig cluster-lifecycle
Not the gce jobs thoght. Only the kinder job. So the gce jobs are failing with a problem outside of kubeadm/kinder but possibly still related to the corefile plugin change
/assign (At kubecon this week)
The gce failures appear to show the same behavior, but with kube-up instead of kubeadm. I.e. it appears that kube-up may also be retaining old configs when upgrading/downgrading. If thats true, then we may need to integrate the migration tools into kube-up as well.
I have opened which aims to fix all the failing tests except , which will be fixed via
Great job so far and Would it be possible to ask for the open PRs to be merged and/or a solution found soon? We have these tests failing for quite some time and code freeze coming up in the next days. Thank you in advance for your efforts!
thanks for the heads up. let me know if you don't have the time for the refactor that i requested and i can try to take over there. the alternative is to rollback the coredns version which i assume is not ideal, due to a variety of fixes in the new version.
is there any more work planned for this issue? I see that some of the kinder jobs are still failing after merged:
merging and will resolve and close this issue.
Thank you for the update !
updates: kinder tests were fixed via
/unassign i was here for: it's a bit flaky (which is due to the tooling), but green.
Since and we're moved to 1.16, and we're waiting for to merge, can we move this issue to 1.16?
? is that the right PR?

Good catch, I meant
The GCE test failures were fixed by and /close
Closing this issue. <detailsIn response to : The GCE test failures were fixed by and /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: }, } <ins> By("setting up watch") pods, err := podClient.List(labels.SelectorFromSet(labels.Set(map[string]string{"time": value}))) if err != nil { Fail(fmt.Sprintf("Failed to query for pods: %v", err)) } Expect(len(pods.Items)).To(Equal(0)) w, err := podClient.Watch( labels.SelectorFromSet(labels.Set(map[string]string{"time": value})), labels.Everything(), pods.ListMeta.ResourceVersion) if err != nil { Fail(fmt.Sprintf("Failed to set up watch: %v", err)) } </ins> By("submitting the pod to kubernetes") // We call defer here in case there is a problem with // the test so we can ensure that we clean up after // ourselves defer podClient.Delete(pod.Name) <del> _, err := podClient.Create(pod) </del> <ins> _, err = podClient.Create(pod) </ins> if err != nil { Fail(fmt.Sprintf("Failed to create pod: %v", err)) } By("verifying the pod is in kubernetes") <del> pods, err := podClient.List(labels.SelectorFromSet(labels.Set(map[string]string{"time": value}))) </del> <ins> pods, err = podClient.List(labels.SelectorFromSet(labels.Set(map[string]string{"time": value}))) </ins> if err != nil { Fail(fmt.Sprintf("Failed to query for pods: %v", err)) } Expect(len(pods.Items)).To(Equal(1)) <ins> By("veryfying pod creation was observed") select { case event, _ := <-w.ResultChan(): if event.Type != watch.Added { Fail(fmt.Sprintf("Failed to observe pod creation: %v", event)) } case <-time.After(podStartTimeout): Fail("Timeout while waiting for pod creation") } </ins> By("deleting the pod") podClient.Delete(pod.Name) pods, err = podClient.List(labels.SelectorFromSet(labels.Set(map[string]string{"time": value}))) <ins> if err != nil { Fail(fmt.Sprintf("Failed to delete pod: %v", err)) } </ins> Expect(len(pods.Items)).To(Equal(0)) <ins> By("veryfying pod deletion was observed") deleted := false timeout := false timer := time.After(podStartTimeout) for !deleted && !timeout { select { case event, _ := <-w.ResultChan(): if event.Type == watch.Deleted { deleted = true } case <-timer: timeout = true } } if !deleted { Fail("Failed to observe pod deletion") } </ins> }) It("should be updated", func() {

PodInterface should have Watch method behaving in the same way as ServiceInterface.Watch does. Use case: a monitoring system (namely Prometheus) expects to get the list of monitored servers performing a DNS-SD query. Special service (similar to kube2sky shipped with Kubernetes) mirrors status of running Pods into etcd, which is in turn monitored by SkyDNS server and exported as DNS-SD. Schematically the data flow looks like this: Kubernetes -kube2sky -etcd -SkyDNS -Prometheus. The biggest challenge is to deliver list of all running pods from Kubernetes to kube2sky. Currently kube2sky can only poll Kubernetes with PodInterface.List() each N seconds, which is not efficient. Having a Watch method would be much better.
cc Do you know why we do not support watch?
Every object should support Watch.
Watching status doesn't currently work -- that's .
I agree with OP. This is just an accident of history; some boilerplate in the client isn't completely filled out. As Brian mentions, not every field is currently watchable but that's being addressed elsewhere.
This is relevant to your interests potentially w/ your work on
This is a pretty simple change if someone wants to get their feet wet. :)
I'll work on it.


Question: load("//staging/src/k8s.io/component-base/version:def.bzl", "version_x_defs") go_binary( <del> name = "kubectl", </del> <ins> name = "kubectl-convert", </ins> embed = [":go_default_library"], pure = "on", visibility = ["//visibility:public"],

was removed in 1.20 in favor of a plugin. However, this is not included in release artifacts. The binary is included in release artifacts, and ideally easily installable via kubectl plugin managers like . /sig cli /assign
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
This question came up in a docs review of a page referencing () Once this is resolved, please ensure the website is updated to document installation/use of this tool.
/sig release


Question: if err != nil { return err } <del> errlist := []error{} </del> <ins> orphanRemovalErrors := []error{} orphanVolumeErrors := []error{} </ins> for _, uid := range found { if allPods.Has(string(uid)) { continue

What we see in if any mismatches for volumes are around: The Problem: Obviously this can get pretty large. Since theres not much variability, it seems like rolling up all the orphanXYZ messages would be more useful and readable. Note this runs in the sync loop, so it happens every few (2) seconds for me. Obviously we see this worse on larger nodes (100 * n) / before but , I've only reproduced this on a local cluster for this PR.


Question: "revisionHistoryLimit": { "type": "integer", "format": "int32", <del> "description": "The number of old ReplicaSets to retain to allow rollback. This is a pointer to distinguish between explicit zero and not specified. Defaults to 2." </del> <ins> "description": "The number of old ReplicaSets to retain to allow rollback. This is a pointer to distinguish between explicit zero and not specified. Defaults to 10." </ins> }, "paused": { "type": "boolean",

All controllers that use the field have a default value of 10 for the field, except for , which has a default of 2. We should increase it to 10 for consistency on its default value across controllers. /sig apps


Question: p:                 &ExecOptions{StreamOptions: StreamOptions{ContainerName: "bar"}}, args:              []string{"foo", "cmd"}, argsLenAtDash:     -1, <del> expectedPod:       "foo", </del> <ins> expectedPod:       execPod(), </ins> expectedContainer: "bar", expectedArgs:      []string{"cmd"}, name:              "cmd, container in flag", }, } for _, test := range tests { <del> f, tf, _, ns := NewAPIFactory() </del> <ins> f, tf, codec, ns := NewAPIFactory() </ins> tf.Client = &fake.RESTClient{ NegotiatedSerializer: ns, <del> Client:               fake.CreateHTTPClient(func(req *http.Request) (*http.Response, error) { return nil, nil }), </del> <ins> Client: fake.CreateHTTPClient(func(req *http.Request) (*http.Response, error) { switch m := req.Method; { case m == "GET": body := objBody(codec, test.expectedPod) return &http.Response{StatusCode: 200, Header: defaultHeader(), Body: body}, nil default: //Ensures no GET is performed when deleting by name t.Errorf("%s: unexpected request: %s %#vn%#v", test.name, req.Method, req.URL, req) return nil, fmt.Errorf("unexpected request") } }), </ins> } tf.Namespace = "test" tf.ClientConfig = defaultClientConfig()

I feel like I should be able to do: but I get
or we have to do something like this: sounds reasonable to me. needs more comments.
All commands that accept resources should accept resource/NAME syntax. In this case for back compat we must only accept a single argument, is familiar with this.
Exactly. ResourceNames in the resource builder is what is needed here to handle the default resource (pod in this case) and provide the ability to specify more.
Related
If nobody is working on it, I'd like to pick the issue, pls.
I don't think anyone is. , Illia Khudoshyn wrote:
So could you please assign me?
I can't assign to just anyone, unfortunately, but by responding here, anyone else who might work on it will see that you are already in progress. , Illia Khudoshyn wrote:
please go ahead, you did own this now. :)
Has this been reverted again? I cannot use “pod/podname” for v1.11:
^ Seconding the above. I'm getting the same 'may not contain /' error. Naturally it's trivial to use unix commands to filter this, but this is such a basic feature and matches the way the rest of kubectl works, so it's quite a shocking omission.
as a unix novice who's also experiencing this problem, I'd love to know your workaround
If you use selectors, you can do something like this: Otherwise you'll need to manually grep, something like this: is the syntax in bash to run the command and use its output as part of another command. You can also use backticks to achieve the same effect but that's considered bad form by many (myself included). For example: If going the grep route, take care to make sure your regex will only return one result, otherwise you'll have to pipe it to xargs or something similar.
Just ran into this on a brand new default installation. Local Mac Docker for Desktop Enabled k8 in preferences error: invalid resource name "svc/broken-gorilla-postgresql": [may not contain '/']
FWIW you can use combined with the flag, e.g.
Had the same problem. Updating kubectl resolved the issue for me.


Question: flags.StringVar(&framework.TestContext.SystemSpecName, "system-spec-name", "", "The name of the system spec (e.g., gke) that's used in the node e2e test. The system specs are in test/e2e_node/system/specs/. This is used by the test framework to determine which tests to run for validating the system requirements.") flags.Var(cliflag.NewMapStringString(&framework.TestContext.ExtraEnvs), "extra-envs", "The extra environment variables needed for node e2e tests. Format: a list of key=value pairs, e.g., env1=val1,env2=val2") flags.StringVar(&framework.TestContext.SriovdpConfigMapFile, "sriovdp-configmap-file", "", "The name of the SRIOV device plugin Config Map to load.") <del> flag.StringVar(&framework.TestContext.ClusterDNSDomain, "dns-domain", "cluster.local", "The DNS Domain of the cluster.") </del> <ins> flag.StringVar(&framework.TestContext.ClusterDNSDomain, "dns-domain", "", "The DNS Domain of the cluster.") </ins> } func init() {

<!-- Please only use this template for submitting reports about continuously failing tests or jobs in Kubernetes CI --Which jobs are failing: Which test(s) are failing: Since when has it been failing: From about 9 hours ago. Testgrid link: Reason for failure: Unknown. Anything else we need to know: Along with this job, even for started failing with similar tests. While is failing, is not failing indicating that something went in master in last 9 hours that's breaking the tests.
/sig node
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
One interesting note on CRIO job is, it's only failing when run by test-infra. If I trigger those jobs locally it completes successfully. Maybe something broke in ? not sure. That's the way to trigger node e2e from your local system (make sure gcloud is configured because just like test-infra, the actual tests are run on gcloud)
/cc
I believe I have identified the issue stemming from Working on fix now :+1: /assign
Fix in


Question: } } <ins> func withVolumeDeletionTimestamp(pvs []*v1.PersistentVolume) []*v1.PersistentVolume { result := []*v1.PersistentVolume{} for _, pv := range pvs { // Using time.Now() here will cause mismatching deletion timestamps in tests deleteTime := metav1.Date(2020, time.February, 18, 10, 30, 30, 10, time.UTC) pv.SetDeletionTimestamp(&deleteTime) result = append(result, pv) } return result } </ins> // newClaim returns a new claim with given attributes func newClaim(name, claimUID, capacity, boundToVolume string, phase v1.PersistentVolumeClaimPhase, class *string, annotations ...string) *v1.PersistentVolumeClaim { fs := v1.PersistentVolumeFilesystem

PV controller appears to be processing same object multiple times for same PVC delete event. The reason is mainly because when a PVC is deleted and PV is synced(either because of an event or because of resync), then each update to PV object causes another event which gets processed until PV object is removed from informer/store and really marked as deleted. This results in multiple delete requests being sent to storage provider for same volume. For example - In above gist you can see that volume deletion is being done more than once for pv - /sig storage cc
It looks like this is to some extent by design because events are pulled from workqueue by one goroutine and then do some proccessing and then put into another goroutine via nestedpendingoperations thingy. The problem is, this is inherently "racy" because first goroutine might be pulling events that were triggered by second goroutine and hence causing processing of events that are still in-progress. I take we can't use workqueues with a fixed number of workers because that would kinda limit number of PVs we can provision concurrently, but can we put the item pulled from queue directly into nested_pendingoperations so as we don't have this weird race condition?


Question: schedulerlisters "k8s.io/kubernetes/pkg/scheduler/listers" ) <ins> const ( // ErrReasonConstraintsNotMatch is used for PodTopologySpread filter error. ErrReasonConstraintsNotMatch = "node(s) didn't match pod topology spread constraints" ) </ins> // PodTopologySpread is a plugin that ensures pod's topologySpreadConstraints is satisfied. type PodTopologySpread struct { sharedLister schedulerlisters.SharedLister

Part of /priority important-soon See example /sig scheduling /kind cleanup
/help
/assign
/remove-help


Question: { "SelectorRegexp": "k8s[.]io/kubernetes/third_party/", "AllowedPrefixes": [ <del> "k8s.io/kubernetes/third_party/forked/golang/expansion" </del> <ins> "k8s.io/kubernetes/third_party/forked/golang/expansion" </ins> ], "ForbiddenPrefixes": [] },

What happened: There is to manage import ways under but that doesn't work at our upstream CI. We can enable it by But if doing that, we will face the following error with What you expected to happen: I hope the check is always running to block invalid import ways at upstream CI system. How to reproduce it (as minimally and precisely as possible): As line into arguments.InputDirs of
/sig testing /area e2e-test-framework


Question: "k8s.io/apiserver/pkg/endpoints/handlers/responsewriters" endpointmetrics "k8s.io/apiserver/pkg/endpoints/metrics" genericapirequest "k8s.io/apiserver/pkg/endpoints/request" <ins> genericfeatures "k8s.io/apiserver/pkg/features" utilfeature "k8s.io/apiserver/pkg/util/feature" </ins> utilflowcontrol "k8s.io/apiserver/pkg/util/flowcontrol" apiserverproxyutil "k8s.io/apiserver/pkg/util/proxy" "k8s.io/apiserver/pkg/util/x509metrics" "k8s.io/client-go/transport" <ins> "k8s.io/component-base/tracing" </ins> "k8s.io/klog/v2" apiregistrationv1api "k8s.io/kube-aggregator/pkg/apis/apiregistration/v1" apiregistrationv1apihelper "k8s.io/kube-aggregator/pkg/apis/apiregistration/v1/helper"

Extension API servers do not currently receive the header when requests are proxied through kube-aggregator, so spans are not linked properly. Traces from the extension API sever should be linked to the parent in kube-aggregator. tracing for kube-apiserver tracing for sample-apiserver and aggregate it to kube-apiserver a request to kube-apiserver for one of the resources from sample-apiserver sample-apiserver traces will not be linked to the kube-apiserver traces No response <detailsThis problem currently exists on . </details<details</details<details</details<details</details<details</details<details</details
/sig api-machinery
/cc /triage accepted


Question: "node-b": framework.Unschedulable, }, }, <ins> { // In this unit test, NodeAffinity plugin is not involved, so node-b still fits name: "incoming pod has nodeAffinity, pods spread as 0/~2~/0/1, hence node-a fits", pod: st.MakePod().Name("p").Label("foo", ""). NodeAffinityNotIn("node", []string{"node-b"}). SpreadConstraint(1, "zone", v1.DoNotSchedule, st.MakeLabelSelector().Exists("foo").Obj()). Obj(), nodes: []*v1.Node{ st.MakeNode().Name("node-a").Label("zone", "zone1").Label("node", "node-a").Obj(), st.MakeNode().Name("node-b").Label("zone", "zone1").Label("node", "node-b").Obj(), st.MakeNode().Name("node-x").Label("zone", "zone2").Label("node", "node-x").Obj(), st.MakeNode().Name("node-y").Label("zone", "zone2").Label("node", "node-y").Obj(), }, existingPods: []*v1.Pod{ st.MakePod().Name("p-b1").Node("node-b").Label("foo", "").Obj(), st.MakePod().Name("p-b2").Node("node-b").Label("foo", "").Obj(), st.MakePod().Name("p-y1").Node("node-y").Label("foo", "").Obj(), }, wantStatusCode: map[string]framework.Code{ "node-a": framework.Success, "node-b": framework.Success, // in real case, it's Unschedulable "node-x": framework.Unschedulable, "node-y": framework.Unschedulable, }, }, </ins> } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) {

We know that nodeAffinity/nodeSelector is honored during the calculating of PodTopolgySpread, however, when some existing pods match the incoming pod's topologySpreadConstratints, while also excluded by the nodeAfffinity, things get a bit tricky. Raising this issue to discuss if we should read it as a bug. Here are the repro steps: you have a 3-node cluster, worker1 and worker2 belong to zoneA, and worker3 belongs to zoneB: make a deployment with 3 replicas. Suppose 2 land on worker2, and 1 land on worker. a new deployment with topologySpreadConstraints as well as nodeAffinity (not schedule to worker2): test pod will land on worker3. This behavior concerns me: it looks like the 2 pods on worker2 are taken into consideration, even though worker2 is excluded by nodeAffinity, so the pods on zoneA/zoneB distribute as 2:2. However, shouldn't we disregard the matching pods that are excluded by nodeAffinity? In that case, the test pod should land on worker1 to reach a pods distribution as 1:1. Please share your thoughts. /sig scheduling /kind bug
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
cc/
That is not the issue, the 2 pods on work2 aren't actually being taken into consideration; the problem is similar to what describes. What happened is that zoneA is now not "visible" at all, and so it is as if we have a single zone. The solution should apply to both cases.
I think we should consider it as a bug. Imagine in this situation, all of these pods are labeled with , then we want to apply a like this: is excluded in scheduling. But now when we try to calculate the skew, we will still calculate the pods in . Since : is , then we will schedule the new pod to or , but since they are resource exhausted, then it will be in pending state.

But if we filtered out the , the pod will be scheduled to successfully. I think it's the right algorithm.
it's not the same as , even if worker2 becomes invisible, there are still 2 zones:
ah, I didn't notice existed. What part of the code is causing this behavior? I would have assumed that the spreading would happen between and . The following code shows that we should be skipping counting the pods on worker2
it's the that is passed to processNode(): We should pass on a filtered list of nodes to processNode(). included the fix in , and that bits should be extracted out to be a standalone PR, if we get to a consensus on this issue is a bug.
I still don't get where the bug is. I think iterating over all node is the right way and the line I pointed to above should filter out the nodes that don't match the affinity. Passing a filteredList (as in exclude nodes that were filtered by previous filters) is certainly not the right solution because we need to count pods on all nodes. Let alone the filteredlist is different based on how we order the filters, which should never be a factor in the semantics of the PodTopologySpread.
The bug lies in the preFilterState's calculation. For the above case, it's expected to be , but it ended up with . The code I posted above is in PodTopologySpread's PreFitler, not Filter.
ok, this is indeed a bug, we need to check the node affinity constraints again inside the second loop.
actually, doesn't seem efficient that we have two loops in the first place
It seems we are doing it right in the score function, excluding those nodes:
Feel the same. Let's fix the correctness issue first, and then improve the efficiency.
Do we know of a use case where this would be useful? It's kind of odd that there are matching pods in a node where we don't intend to schedule. Maybe during some sort of migration?
It's about clarifying/correcting the semantics of constraints (nodeAffinity, and the to-be- ones like Unschedulable taint) embedded in PodTopologySpread. So IMO we should fix the correctness issue, no matter there is a use case or not.
One thing to be aware of is whether a potential fix could render some pods unschedulable, thus be a breaking change.


Question: } } <del> func getFakeCSIPVInfo(volumeName, driverName string) FakePersistentVolumeInfo { return FakePersistentVolumeInfo{ { ObjectMeta: metav1.ObjectMeta{Name: volumeName}, Spec: v1.PersistentVolumeSpec{ PersistentVolumeSource: v1.PersistentVolumeSource{ CSI: &v1.CSIPersistentVolumeSource{ Driver:       driverName, VolumeHandle: volumeName, }, }, }, }, { ObjectMeta: metav1.ObjectMeta{Name: volumeName + "-2"}, Spec: v1.PersistentVolumeSpec{ PersistentVolumeSource: v1.PersistentVolumeSource{ CSI: &v1.CSIPersistentVolumeSource{ Driver:       driverName, VolumeHandle: volumeName + "-2", }, }, }, }, { ObjectMeta: metav1.ObjectMeta{Name: volumeName + "-3"}, Spec: v1.PersistentVolumeSpec{ PersistentVolumeSource: v1.PersistentVolumeSource{ CSI: &v1.CSIPersistentVolumeSource{ Driver:       driverName, VolumeHandle: volumeName + "-3", </del> <ins> func getFakeCSIPVInfo(volumeName string, driverNames ...string) FakePersistentVolumeInfo { pvInfos := FakePersistentVolumeInfo{} for _, driver := range driverNames { for j := 0; j < 4; j++ { volumeHandle := fmt.Sprintf("%s-%s-%d", volumeName, driver, j) pv := v1.PersistentVolume{ ObjectMeta: metav1.ObjectMeta{Name: volumeHandle}, Spec: v1.PersistentVolumeSpec{ PersistentVolumeSource: v1.PersistentVolumeSource{ CSI: &v1.CSIPersistentVolumeSource{ Driver:       driver, VolumeHandle: volumeHandle, }, </ins> }, }, <del> }, }, </del> <ins> } pvInfos = append(pvInfos, pv) } </ins> } <ins> return pvInfos </ins> } <del> func getFakeCSIPVCInfo(volumeName string) FakePersistentVolumeClaimInfo { return FakePersistentVolumeClaimInfo{ { ObjectMeta: metav1.ObjectMeta{Name: volumeName}, Spec:       v1.PersistentVolumeClaimSpec{VolumeName: volumeName}, }, { ObjectMeta: metav1.ObjectMeta{Name: volumeName + "-2"}, Spec:       v1.PersistentVolumeClaimSpec{VolumeName: volumeName + "-2"}, }, </del> <ins> func getFakeCSIPVCInfo(volumeName, scName string, driverNames ...string) FakePersistentVolumeClaimInfo { pvcInfos := FakePersistentVolumeClaimInfo{} for _, driver := range driverNames { for j := 0; j < 4; j++ { v := fmt.Sprintf("%s-%s-%d", volumeName, driver, j) pvc := v1.PersistentVolumeClaim{ ObjectMeta: metav1.ObjectMeta{Name: v}, Spec:       v1.PersistentVolumeClaimSpec{VolumeName: v}, } pvcInfos = append(pvcInfos, pvc) } } pvcInfos = append(pvcInfos, v1.PersistentVolumeClaim{ ObjectMeta: metav1.ObjectMeta{Name: volumeName + "-4"}, Spec:       v1.PersistentVolumeClaimSpec{StorageClassName: &scName}, }) pvcInfos = append(pvcInfos, v1.PersistentVolumeClaim{ ObjectMeta: metav1.ObjectMeta{Name: volumeName + "-5"}, Spec:       v1.PersistentVolumeClaimSpec{}, }) // a pvc with missing PV but available storageclass. pvcInfos = append(pvcInfos, v1.PersistentVolumeClaim{ ObjectMeta: metav1.ObjectMeta{Name: volumeName + "-6"}, Spec:       v1.PersistentVolumeClaimSpec{StorageClassName: &scName, VolumeName: "missing-in-action"}, }) return pvcInfos } func getFakeCSIStorageClassInfo(scName, provisionerName string) FakeStorageClassInfo { return FakeStorageClassInfo{ </ins> { <del> ObjectMeta: metav1.ObjectMeta{Name: volumeName + "-3"}, Spec:       v1.PersistentVolumeClaimSpec{VolumeName: volumeName + "-3"}, </del> <ins> ObjectMeta:  metav1.ObjectMeta{Name: scName}, Provisioner: provisionerName, </ins> }, } }

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!--What happened: Volume binding mode doesn't consider the volume attach limit per node. What you expected to happen: volume binding mode should have respected the attach limit, similar to binding mode. How to reproduce it (as minimally and precisely as possible): In CSI driver, configured the volume-attach limit to 1 per node on a 3 node cluster. In volume binding mode, only 3 pods with 3 PV/PVCs runs. In binding mode, all 5 pods runs with 5PV/PVCs on 3 node cluster. (Actually 4 pods scheduled on 1 node and 1 on other node, 3rd didn't have any pod scheduled) Anything else we need to know?: I was able to reproduced it with minimal node and configuration, and also hit this for stress case with 100 Volume limit and 500 pods running on the 5 mode cluster. Environment: Kubernetes version (use ): ~ kubectl version Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.2", GitCommit:"", GitTreeState:"clean", BuildDate:"2018-10-30T21:39:38Z", GoVersion:"go1.11.1", Compiler:"gc", Platform:"darwin/amd64"} Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.0", GitCommit:"", GitTreeState:"clean", BuildDate:"2018-12-03T20:56:12Z", GoVersion:"go1.11.2", Compiler:"gc", Platform:"linux/amd64"} Cloud provider or hardware configuration: On-prem linux server OS (e.g. from /etc/os-release): Centos 7Kernel (e.g. ): 3.10.0 Install tools: Others:
cc
/sig storage
/assign Currently the predicate PVCs that are not bound. It should lookup the storageclass to get the provisioner/driver name. This solution will require a StorageClass object is even if you don't use dynamic provisioning.

I had a thought that it wouldn't be so simple to count unbound PVCs of other pods since binding is still in progress, but I think it's fine actually. We only count other pods that have been assigned to the same node. Those pods are assigned to the node AND still unbound because of one of these reasons: It's actually been scheduled to the node and someone manually came in and unbound the PVC: unlikely and an unsupported sequence since we implemented StorageProtection The Pod was initially created with nodeName already set, bypassing the scheduler The Pod is going to be scheduled (it's been assumed), and it's currently going through binding. This is normal with delayed binding. Either way, the fact that the Pod's nodename is set to this node means that the pod should be counting towards the node's limit.
Hmm, so after opening the PR that fixes this, I realize that - from storageclass we don't really know if the volume we are evaluating is CSI volume or not. I think a similar problem exists for existing predicate though. Like AzureDisk predicate does not know if PVC it is over counting is of type AzureVolume or not. So, I suspect that those 4 hardcoded predicates are counting all delayed binding volumes.
if the storageclass.provisioner name is not found, can we just ignore it?
I think we can ignore it. The PR currently does that.


Question: } func MakeFakeEtcdClient(t *testing.T) *FakeEtcdClient { <del> return &FakeEtcdClient{ </del> <ins> ret := &FakeEtcdClient{ </ins> t:    t, Data: map[string]EtcdResponseWithError{}, } <ins> // There are three publicly accessible channels in FakeEtcdClient: //  - WatchResponse //  - WatchInjectError //  - WatchStop // They are only available when Watch() is called.  If users of // FakeEtcdClient want to use any of these channels, they have to call // WaitForWatchCompletion before any operation on these channels. // Internally, FakeEtcdClient use condWatchCompleted to indicate if the // Watch() method has been called. WaitForWatchCompletion() will wait // on condWatchCompleted. By the end of the Watch() method, it will // call Broadcast() on condWatchCompleted, which will awake any // goroutine waiting on this condition. ret.condWatchCompleted = sync.NewCond(&ret.condLock) return ret </ins> } func (f *FakeEtcdClient) AddChild(key, data string, ttl uint64) (*etcd.Response, error) {

Detected by
Create a Pod as follows: Name: jenkins Using image: jenkins In a new Kubenetes namespace named website-frontend kubectl create ns website-frontend apiVersion: v1 kind: Pod metadata: name: jenkins namespace: website-frontend spec: containers: name: jenkins image: jenkins kubectl create -f


Question: } } <ins> // TODO(phase1+) this we are missing args from the help text, there should be a way to tell cobra about it if len(args) == 0 && len(cfg.MasterAddresses) == 0 { return nil, fmt.Errorf("must specify master IP address (see --help)") } cfg.MasterAddresses = append(cfg.MasterAddresses, args...) </ins> if !skipPreFlight { fmt.Println("Running pre-flight checks") <del> err := preflight.RunJoinNodeChecks() </del> <ins> err := preflight.RunJoinNodeChecks(cfg) </ins> if err != nil { return nil, &preflight.PreFlightError{Msg: err.Error()} }

Environment: VMware CentOS Linux release 7.2.1511 (Core) kernel: 3.10.0-327.36.1.el7.x8664 Installed packages: kubeadm.x8664 1.5.0-0.alpha.0.1534.gcf7301f kubectl.x8664 1.4.0-0 kubelet.x8664 1.4.0-0 kubernetes-cni.x8664 0.3.0.1-0. docker-engine.x8664 1.11.2- docker-engine- 1.12.1- What happened: kubeadm init hangs indefintely after: kubeadm init <master/tokensgenerated token: "." <master/pkicreated keys and certificates in "/etc/kubernetes/pki" <util/kubeconfigcreated "" <util/kubeconfigcreated "" <master/apiclientcreated API client configuration <master/apiclientcreated API client, waiting for the control plane to become readykubeadm init What you expected to happen: Master node initialize Anything else do we need to know: Only the kubeadm and kublet processes are running. When I do docker ps no containers are running. If i type docker images the only image that has been downloaded are: I've also tried: kubeadm init --api-advertise-addresses but with the same outcome (followed the clean-up at ) Im behind a firewall so I'm using proxy settings for docker. I have confirmed I can reach images on docker hub and google cloud registry
will download the required binaries viz etcd. When you are behind firewall kubeadm will hang/wait forever to download the binaries. I dont know whether it is addressed correctly in kubeadm implementation. I am also facing the same issue and waiting for the resolution.
Thx, I suspected it to be some firewall issue. Do you know if there's a way to add httpsproxy & httpproxy settings to kubeadm?
Kubeadm doesn't seem to pick proxy environment variables. I am looking at recompiling with proxy support may work. But I dont know how to do that. Waiting for updates about proxy
This issue same as Lets join there. The solution mentioned there may work.

You can try kubeadm with patch applied from PR : It works at least in my environment behind restrictive proxy.
thanks, I will look into it


Question: if eps.Len() >= *peerCount { break } <del> state.Logf("%v/%v has %v endpoints, which is less than %v as expected. Waiting for all endpoints to come up.", *namespace, *service, len(eps), *peerCount) </del> <ins> state.Logf("%v/%v has %v endpoints (%v), which is less than %v as expected. Waiting for all endpoints to come up.", *namespace, *service, len(eps), eps.List(), *peerCount) </ins> } // Do this repeatedly, in case there's some propagation delay with getting

cc
This has failed The two failures are different, and both seemingly totally legitimate. The first one is simply a pod never entering running state within the 5 minute timeout. Expected error: <*errors.errorString | 0xc208475150: { s: "gave up waiting for pod 'nettest-dtwk3' to be 'running' after 5m0s", } gave up waiting for pod 'nettest-dtwk3' to be 'running' after 5m0s not to have occurred The second is not being reachable: Dec 7 01:22:47.275: Unable to connect/talk to the internet: Get dial tcp 74.125.202.147:80: i/o timeout The latter could be a result of a missing MASQ rule on the node, or GCP egress problems. I don't think that this is worth trying to debug further in the context of this test for intra-pod communication. I'll check more generally whether we have a current intermittent problems with respect to pods entering running state, or reaching the internet.
I think this test may be causing builds to time out, too. See comments from me in .
For the future, I actually don't think we need the test at all in the networking test. Maybe even move it do a different test altogether.
This is clearly a bug. I'll send a PR shortly: Regarding the ping to , why did you put in in the first place?
hmmm, not sure what you mean by "in the first place" ? I did not write the test...
sorry, I thought that you'd written that part of the test. I will find out who wrote that, figure out why they put it there in the first place, and then either remove it, or rewrite it to work correctly. Clearly that part of the test needs at least some basic retry to work consistently.
ok great, no problem just wasnt sure what you meant. SGTM. Id be okay w/ deleting the ping entirely regardless of who wrote it, its a very old snippet which doesnt really add much to the overall test suite (and in fact is questionable wether it should even be an assertion at all )
somebody put it there for some reason. Better to understand why before just deleting it.

i looked, its actually rob who i work with :) i will ask him in the morning unless you find him first :) (rrati)
Regarding my alleged "clear bug" above, I was completely wrong. There's nothing wring with that code. And unfortunately, when this test goes into a presumed endless loop (see comment above) it does not produce any output (as the output is cached by Ginkgo until the test completes). So it's very difficult to debug. But in summary I agree with this test is probably never completing, and causing tests to time out. Someone will need to isolate this test in a environment where it produces output and can be debugged. I'll keep trying, but need to head home soon, so if anyone wants to pick it up overnight, go ahead.
Smoking gun:
hmm - I'm just wondering why only this particular test is hanging forever (if we agree that's the case). I don' think that some requests are hanging forever in this test, but not in any other test...
Closing in favor of which should fix this. Or at least some way to set client timeout, which and are still figuring out.
It's happening again.
Talked to offline. I am assigning this one to you for triage. Let me know if you need any help.
AFAICT the last failure was just the pod nettest-7h5jj not coming up
Looks like this should be resolved by your pr: too. :-)
Maybe. Though the webserver only deadlocks in the failure case, so the test is probably still flaky...
Looks like this test is now explicitly flaking by failing, as I expected.
Based on the , it took 15 minutes to pull the image. closing this one as dup of
New Failure:
This looks like a race between populating the endpoints list and the /status, /read methods of the service exposed through the master proxy. It looks like the webserver pod is just trying to confirm that it can reach all peers. We can do this synchronously after having observed all expected endpoints in the test (or re-write the test to use the same pattern we do in services e2e).
The test looks really racy to me. I think the following 15 retries prevents it to fail more often, but not completely.

I will submit a pr to enfore webservers to wait till alll endpoints are up before they start to contact each other.
New failure
I am making this P0
It took more than 4 minutes for a nettest pod to come up. The nettest container is waiting for all endpoints to be ready for 2 minutes plus 15 attempts (5 sec interval) to contact its peers. This happens in less than 4 minutes. Hence the first 2 nettest pods finished contacting peers and declared failure.
The node you want is: I see: that's too much silence ...
From the It looks like kubelet is slow to start the container. Due to the lower logging level in the gke, it's hard to guess what could've slowed down the goroutine...


Question: <ins> /* Copyright 2020 The Kubernetes Authors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package config import ( "testing" "github.com/google/go-cmp/cmp" ) func TestPluginsApply(t *testing.T) { tests := []struct { name            string customPlugins   *Plugins defaultPlugins  *Plugins expectedPlugins *Plugins }{ { name: "AppendCustomPlugin", customPlugins: &Plugins{ Filter: &PluginSet{ Enabled: []Plugin{ {Name: "CustomPlugin"}, }, }, }, defaultPlugins: &Plugins{ Filter: &PluginSet{ Enabled: []Plugin{ {Name: "DefaultPlugin1"}, {Name: "DefaultPlugin2"}, }, }, }, expectedPlugins: &Plugins{ QueueSort: &PluginSet{Enabled: []Plugin{}}, PreFilter: &PluginSet{Enabled: []Plugin{}}, Filter: &PluginSet{ Enabled: []Plugin{ {Name: "DefaultPlugin1"}, {Name: "DefaultPlugin2"}, {Name: "CustomPlugin"}, }, }, PostFilter: &PluginSet{Enabled: []Plugin{}}, Score:      &PluginSet{Enabled: []Plugin{}}, Reserve:    &PluginSet{Enabled: []Plugin{}}, Permit:     &PluginSet{Enabled: []Plugin{}}, PreBind:    &PluginSet{Enabled: []Plugin{}}, Bind:       &PluginSet{Enabled: []Plugin{}}, PostBind:   &PluginSet{Enabled: []Plugin{}}, Unreserve:  &PluginSet{Enabled: []Plugin{}}, }, }, { name: "InsertAfterDefaultPlugins2", customPlugins: &Plugins{ Filter: &PluginSet{ Enabled: []Plugin{ {Name: "CustomPlugin"}, {Name: "DefaultPlugin2"}, }, Disabled: []Plugin{ {Name: "DefaultPlugin2"}, }, }, }, defaultPlugins: &Plugins{ Filter: &PluginSet{ Enabled: []Plugin{ {Name: "DefaultPlugin1"}, {Name: "DefaultPlugin2"}, }, }, }, expectedPlugins: &Plugins{ QueueSort: &PluginSet{Enabled: []Plugin{}}, PreFilter: &PluginSet{Enabled: []Plugin{}}, Filter: &PluginSet{ Enabled: []Plugin{ {Name: "DefaultPlugin1"}, {Name: "CustomPlugin"}, {Name: "DefaultPlugin2"}, }, }, PostFilter: &PluginSet{Enabled: []Plugin{}}, Score:      &PluginSet{Enabled: []Plugin{}}, Reserve:    &PluginSet{Enabled: []Plugin{}}, Permit:     &PluginSet{Enabled: []Plugin{}}, PreBind:    &PluginSet{Enabled: []Plugin{}}, Bind:       &PluginSet{Enabled: []Plugin{}}, PostBind:   &PluginSet{Enabled: []Plugin{}}, Unreserve:  &PluginSet{Enabled: []Plugin{}}, }, }, { name: "InsertBeforeAllPlugins", customPlugins: &Plugins{ Filter: &PluginSet{ Enabled: []Plugin{ {Name: "CustomPlugin"}, {Name: "DefaultPlugin1"}, {Name: "DefaultPlugin2"}, }, Disabled: []Plugin{ {Name: "*"}, }, }, }, defaultPlugins: &Plugins{ Filter: &PluginSet{ Enabled: []Plugin{ {Name: "DefaultPlugin1"}, {Name: "DefaultPlugin2"}, }, }, }, expectedPlugins: &Plugins{ QueueSort: &PluginSet{Enabled: []Plugin{}}, PreFilter: &PluginSet{Enabled: []Plugin{}}, Filter: &PluginSet{ Enabled: []Plugin{ {Name: "CustomPlugin"}, {Name: "DefaultPlugin1"}, {Name: "DefaultPlugin2"}, }, }, PostFilter: &PluginSet{Enabled: []Plugin{}}, Score:      &PluginSet{Enabled: []Plugin{}}, Reserve:    &PluginSet{Enabled: []Plugin{}}, Permit:     &PluginSet{Enabled: []Plugin{}}, PreBind:    &PluginSet{Enabled: []Plugin{}}, Bind:       &PluginSet{Enabled: []Plugin{}}, PostBind:   &PluginSet{Enabled: []Plugin{}}, Unreserve:  &PluginSet{Enabled: []Plugin{}}, }, }, { name: "ReorderDefaultPlugins", customPlugins: &Plugins{ Filter: &PluginSet{ Enabled: []Plugin{ {Name: "DefaultPlugin2"}, {Name: "DefaultPlugin1"}, }, Disabled: []Plugin{ {Name: "*"}, }, }, }, defaultPlugins: &Plugins{ Filter: &PluginSet{ Enabled: []Plugin{ {Name: "DefaultPlugin1"}, {Name: "DefaultPlugin2"}, }, }, }, expectedPlugins: &Plugins{ QueueSort: &PluginSet{Enabled: []Plugin{}}, PreFilter: &PluginSet{Enabled: []Plugin{}}, Filter: &PluginSet{ Enabled: []Plugin{ {Name: "DefaultPlugin2"}, {Name: "DefaultPlugin1"}, }, }, PostFilter: &PluginSet{Enabled: []Plugin{}}, Score:      &PluginSet{Enabled: []Plugin{}}, Reserve:    &PluginSet{Enabled: []Plugin{}}, Permit:     &PluginSet{Enabled: []Plugin{}}, PreBind:    &PluginSet{Enabled: []Plugin{}}, Bind:       &PluginSet{Enabled: []Plugin{}}, PostBind:   &PluginSet{Enabled: []Plugin{}}, Unreserve:  &PluginSet{Enabled: []Plugin{}}, }, }, } for _, test := range tests { t.Run(test.name, func(t *testing.T) { test.defaultPlugins.Apply(test.customPlugins) if d := cmp.Diff(test.expectedPlugins, test.defaultPlugins); d != "" { t.Fatalf("plugins mismatch (-want +got):n%s", d) } }) } } </ins>

What would you like to be : Actually disable scheduler plugins according to the disabled list in the scheduler's component config. The semantics are defined here: Part of /sig scheduling /priority important-soon /assign
The following semantics seems vague, and I think we should reach consensus on some cases when disabled plugins are involved. Suppose we have two default plugins, and , the runs before , here are the cases we could meet with the scheduling framework, plugins which run after all the default plugins plugins which should run ahead of the plugins plugin which should run ahead of all the plugins -enable default or custom plugin should return errors
Do you have any other cases not mentioned here?
Those cases sound right, I don't have others.
Is this actually what we want? I don't think so. The comment needs rewording.
I plan to revisit the comments after


Question: "k8s.io/kubernetes/pkg/volume/util/volumepathhandler" ) <ins> const ( unknownVolumePlugin string = "UnknownVolumePlugin" ) </ins> var _ OperationGenerator = &operationGenerator{} type operationGenerator struct {

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!--What happened: In this test, volume mount failed because the driver-registrar took a minute to come up. However, it seems like volume mount operation is retrying without any sort of backoff. This message appears in 400+ times in less than a minute. What you expected to happen: Operation executor should not be retrying operations so frequently. <!-- DO NOT EDIT BELOW THIS LINE --/kind bug
I'll take a look at this. /assign
Problem is that reconciler depends on operation executor to do backoff. In the normal case operation executor returns a method that does the actual work. Any errors by that method result in exponential backoff. But in this case, the error is coming from the code creating the mount method which is not handled by the back off logic, and is called again and again by the reconciler resulting in log and event spam. This didn't use to be a problem because initialization errors were uncommon (plugins were built in), but with CSI the driver may not exist and these initialization errors are expected. I'll look in to cleaning this up.
Is this something that will affect users in 1.13 and needs CPing?
This will result in kubelet log spam and possibly kubernetes event spam. It happens when someone creates a pod referencing a CSI volume before the CSI driver for that volume is installed. Which happens frequently in E2E tests because they install/remove driver with every test. But should be rare in real world usage since most users are likely to install a driver first before using it.
If release team is worried about this, we could create a quick PR to suppress logging/event generation of this these errors. Otherwise we plan to address this more holistically for 1.14 and 1.13.1.


Question: { name:     "hpa-priority", arg:      "hpa", <del> expected: schema.GroupVersionResource{Resource: "superhorizontalpodautoscalers"}, </del> <ins> expected: schema.GroupVersionResource{Resource: "superhorizontalpodautoscalers", Group: "autoscaling"}, </ins> srvRes: []*metav1.APIResourceList{ { GroupVersion: "autoscaling/v1",

Copying the issue from Here is my CRD CRD Data Outputs Expected to see the resource list when kubectl get dpl Versions
/sig api-machinery /area custom-resources /kind bug
Can you check if the title is right for this case?
/cc
If you collide with another short name, you have to qualify with a group. It should follows normal resolution rules for priority
this isn't CRD-specific. isn't a colliding shortname. it expanded dpl -deployments, then resolved deployments against preferred group order, instead of limiting resolution to the group(s) that had the shortname
updated description to narrow the report to the case which is a bug
Did a little research, turn out kubectl will take as it's short name: I can give a shot to solve this problem, unless it's by design... : )
FYI. PR .
/assign


Question: t.Errorf("syncPetSet should return actual amount of pods") } } <ins> type fakeClient struct { fake_internal.Clientset petSetClient *fakePetSetClient } func (c *fakeClient) Apps() unversioned.AppsInterface { return &fakeApps{c, &fake.FakeApps{}} } type fakeApps struct { *fakeClient *fake.FakeApps } func (c *fakeApps) PetSets(namespace string) unversioned.PetSetInterface { c.petSetClient.Namespace = namespace return c.petSetClient } type fakePetSetClient struct { *fake.FakePetSets Namespace string replicas  int } func (f *fakePetSetClient) UpdateStatus(petSet *apps.PetSet) (*apps.PetSet, error) { f.replicas = petSet.Status.Replicas return petSet, nil } func TestPetSetReplicaCount(t *testing.T) { fpsc := &fakePetSetClient{} psc, _ := newFakePetSetController() psc.kubeClient = &fakeClient{ petSetClient: fpsc, } ps := newPetSet(3) psKey := fmt.Sprintf("%v/%v", ps.Namespace, ps.Name) psc.psStore.Store.Add(ps) if err := psc.Sync(psKey); err != nil { t.Errorf("Error during sync of deleted petset %v", err) } if fpsc.replicas != 1 { t.Errorf("Replicas count sent as status update for PetSet should be 1, is %d instead", fpsc.replicas) } } </ins>

Is this a request for help? Nope What keywords did you search in Kubernetes issues before filing this one? petset replicas current desired Is this a BUG REPORT or FEATURE REQUEST? BUG Kubernetes version Client Version: version.Info{Major:"", Minor:"", GitVersion:"v0.0.0-master+", GitCommit:"", GitTreeState:"not a git tree", BuildDate:"2016-09-01T10:49:36Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"", Minor:"", GitVersion:"v0.0.0-master+", GitCommit:"", GitTreeState:"not a git tree", BuildDate:"2016-09-01T10:49:36Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"} Environment: Cloud provider or hardware configuration: 24 cores, 64 GB RAM OS Arch Linux Kernel (e.g. ): Linux hostname 4.4.13-1-lts SMP Wed Jun 8 16:44:31 CEST 2016 x86_64 GNU/Linux Install tools: local-cluster- Others: What happened: While creating PetSet based on this tutorial: first PetSet pod was stuck in pending state due to lack of PersistentVolumes in cluster. When getting the petset info via I expected to see "CURRENT" set to 1, while the kubectl output was as follows: While clearly shows that only one pod is created: In json and yaml output, status.replicas value was also 2. What you expected to happen: I expected that CURRENT value shown by kubectl would be due to only one pod being created. How to reproduce it (as minimally and precisely as possible): Create via on empty cluster and run and . Anything else do we need to know: Nope
CURRENT is the number of pods that have been created, not that they are READY. See for example the latest api change for replica sets where we exactly for this reason. All controllers (petsets, daemonsets, deployments, etc.) should have ReadyReplicas cc:
See also
I guess it doesn't work anyway, since only one pod was created for this PetSet, but the CURRENT says that 2 were created. I will change bug description then.

Hmm, the idea is to sync every pet periodically, but only create them one at a time. The logic that counts status.replicas is wrong. This counter shouldn't be incremented in the case where we didn't update/sync or create a pet: Today we do increment it all pet creation is blocked on the first pet becoming unhealthy. Should be an easy fix if anyone wants to jump in ( fyi) otherwise I'll get to it once 1.4 is out.
I can try to fix it if that's OK. Thanks for the pointer to the source code!
Please do, co-assigned