Question: # $4: The kube-env metadata. function create-node-template { detect-project <ins> # First, ensure the template doesn't exist. # TODO(mbforbes): To make this really robust, we need to parse the output and #                 add retries. Just relying on a non-zero exit code doesn't #                 distinguish an ephemeral failed call from a "not-exists". if gcloud compute instance-templates describe "$1"; then echo "Instance template ${1} already exists; continuing." >&2 return fi </ins> local attempt=0 while true; do if ! gcloud compute instance-templates create "$1"

See issue as the roll-up for node upgrades. tl;dr: Update MIG template, then call a MIG rollingupdate. More details: When upgrading nodes, an in-place upgrade must worry about leftover state on the boot disk, VM configuration like IP tables, and the idempotency of salt. Getting a fresh VM instead would remove these concerns and provide a clean state for a new node. Furthermore, a versioned node spec () includes the tuple of (kubelet, kube-proxy, docker, kernel/OS). In the general case, changing docker or the kernel/OS means we have to stop the running pods. Once know we have to stop the pods, the simplest upgrade mechanism is to totally down the node and provision a new one. Provisioning a new node is also a great backup to have for node repairs. To be clear, though, this issue is entirely separate from node repairs; once we have the capability to grow the cluster by adding new nodes, this mechanism, and other repair mechanisms, will build on top of that capability. This was blocked by: nodes joining a running cluster (template-only metadata) References: The most relevant discussion regarding the worries of complications with state while doing in-place upgrades is .
FWIW, I think we need this as a fallback to repair completely borked nodes? That is, it may not be necessary for upgrades per se, but could end up being independently useful.
Thanks for the comment, I updated the description to be clear about what this issue is addressing. Having the ability to add new nodes to a cluster will both enable this upgrade path as well as give us a great "hammer" to use for repairs.
Working on this now.


Question: generic.ObjectCountQuotaResourceNameFor(schema.GroupResource{Resource: "persistentvolumeclaims"}): resource.MustParse("1"), }, }, <ins> "pvc-usage-rounded": { pvc: validClaimWithNonIntegerStorage, usage: corev1.ResourceList{ corev1.ResourceRequestsStorage:        resource.MustParse("2"), // 1001m -> 2 corev1.ResourcePersistentVolumeClaims: resource.MustParse("1"), generic.ObjectCountQuotaResourceNameFor(schema.GroupResource{Resource: "persistentvolumeclaims"}): resource.MustParse("1"), }, }, "pvc-usage-by-class-rounded": { pvc: validClaimByStorageClassWithNonIntegerStorage, usage: corev1.ResourceList{ corev1.ResourceRequestsStorage:                                                                    resource.MustParse("2"), // 1001m -> 2 corev1.ResourcePersistentVolumeClaims:                                                             resource.MustParse("1"), V1ResourceByStorageClass(classGold, corev1.ResourceRequestsStorage):                               resource.MustParse("2"), // 1001m -> 2 V1ResourceByStorageClass(classGold, corev1.ResourcePersistentVolumeClaims):                        resource.MustParse("1"), generic.ObjectCountQuotaResourceNameFor(schema.GroupResource{Resource: "persistentvolumeclaims"}): resource.MustParse("1"), }, }, </ins> } for testName, testCase := range testCases { actual, err := evaluator.Usage(testCase.pvc)

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: When a pvc was specified with a quantity including the suffix : attempts to include that quantity in the resourcequota's storage resulted in the following error, which blocked all other updates to the resourcequota: What you expected to happen: Updates to all other quantities in the resourcequota should not have been blocked by the error above In general, as a suffix (milli) doesn't really make sense when used in the context of memory, and causes issues like the above. Would it be possible to deny the usage of this suffix at admission time for things like pvcs? (Our mitigation right now is to add a rule that does this in Open Policy Agent). How to reproduce it (as minimally and precisely as possible): Create a resourcequota object with limits on pvcs and the a storage type, e.g. Then create a PVC with the following spec: The resource quota controller should start producing the error log above. Anything else we need to know?: Environment: Kubernetes version (use ): v1.17.3 Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig api-machinery
/remove-sig api-machinery /sig storage
i guess we should leave api-machinery! /sig api-machinery
/assign I think we should validate the value of field.
I've filed to address this issue.
looks like it is not easy to tighten the validation and that requires careful consideration xref:
/assign
"Updates to all other quantities in the resourcequota should not have been blocked by the error above" should be fixed separately
Opened to round storage requests up to the nearest byte in the quota evaluator.


Question: // according to whatever system is managing the endpoint. A nil value // indicates an unknown state. In most cases consumers should interpret this // unknown state as ready. For compatibility reasons, ready should never be <del> // "true" for terminating endpoints. </del> <ins> // "true" for terminating endpoints, except when the normal readiness // behavior is being explicitly overridden, for example when the associated // Service has set the publishNotReadyAddresses flag. </ins> // +optional optional bool ready = 1;

The EndpointSlice API says and it works as expected in normal circumstances I delete the pod and the endpoints conditions are the expected However, if during this time I set we run into an incoherence with the API doc, but it seems that we already documented this edge case in the code I think that at this point is just to amend this information to the API, no? /kind bug /kind documentation
/sig network /assign
I would like to work on this. I just want a bit more clarification on what needs to be done!
I proposed a solution, that is clarify it in the doc ... once we reach an agreement in this issue we implement it ...
+1 to docs. I'd make it a little more vague, like: """ when the normal readiness behavior is being explicitly overridden, for example when the associated Service has set the publishNotReadyAddresses flag. """
so, the thing is to update the docs here with Tim's suggestion and here and do to update the generated code and submit a PR
/assign
I am not able to run command in windows, I am checking it. This error message is coming: *
/triage accepted
Is there a PR for this? Happy to review
I will raise the PR if my above problem gets solved. I am trying to make an update but the error mentioned above is coming.
What platform are you on? If the Makefile were really broken EVERYTHING would be failing.
he's on windows ... you will need a linux/macos. see
Let me know if you want someone to take it over.
I think in that case, you can assign this to someone else.


Question: // The password of the client certificate for an AAD application with RBAC access to talk to Azure RM APIs AADClientCertPassword string `json:"aadClientCertPassword" yaml:"aadClientCertPassword"` // Use managed service identity for the virtual machine to access Azure ARM APIs <del> UseManagedIdentityExtension bool `json:"useManagedIdentityExtension"` </del> <ins> UseManagedIdentityExtension bool `json:"useManagedIdentityExtension" yaml:"useManagedIdentityExtension"` </ins> // The ID of the Azure Subscription that the cluster is deployed in SubscriptionID string `json:"subscriptionId" yaml:"subscriptionId"` }


Question: "os" "runtime" <del> pkg_runtime "k8s.io/kubernetes/pkg/runtime" </del> <ins> kruntime "k8s.io/kubernetes/pkg/runtime" </ins> "github.com/golang/glog" flag "github.com/spf13/pflag"

Forked from pointed out that all package names should be lower case, no under_scores or mixedCaps. The same rule applies to renaming imported packages. () generated a list of mixedCaps package names in our codebase: I will upload a PR later to rename kubeletTypes to kubetypes, and the rest is up for grabs.
sorry i have not saw this issue as i open a similar one(), do you mind i participate in this issue? i would happy to do some work for this. best regards.
of course, fixes are welcome. Thanks for picking up on this! We'll try to expedite the review and merge process of such PRs to avoid a lot of churns, but chances are that some rebases are needed :)
ok, i will send some fix sonner, and tag you for review. thanks again.
Not done:


Question: In order to access your nginx landing page, you also have to make sure that traffic from external IPs is allowed. Do this by opening a [firewall to allow traffic on port 80](services-firewalls.md). <ins> If you're running on AWS, Kubernetes creates an ELB for you.  ELBs use host names, not IPs, so you will have to do `kubectl describe svc my-nginx` and look for the `LoadBalancer Ingress` host name.  Traffic from external IPs is allowed automatically. </ins> ## Killing the application To kill the application and delete its containers and public IP address, do:

GCP uses IP addresses, which is shown on an extra line. AWS does not give an IP address. We should show it. GCP: AWS: I'd like to try to do this work.
cc
AWS uses hostnames for load balancers which is why they aren't showing up. The hostname is also pretty long, and I know we've had problems with the width of the kubectl output before. That said, I seem to recall a "--wide" option, and I think you can maybe choose which columns appear? I think it is a useful suggestion, and I think your biggest battle will be to keep the number of columns such that it doesn't normally wrap. Just so you know, I normally do to see the AWS ELB hostname.
Duping this into , we also have an lgtm'd pr out


Question: "context" "fmt" "sort" <ins> "strings" </ins> "sync/atomic" "time"

[It] [sig-storage] CSI Mock selinux on mount metrics SELinuxMount metrics [LinuxOnly][Feature:SELinux][Feature:SELinuxMountReadWriteOncePod][Serial] warning is not bumped on two Pods with the same context on RWO volume [sig-storage] [It] [sig-storage] CSI Mock selinux on mount metrics SELinuxMount metrics [LinuxOnly][Feature:SELinux][Feature:SELinuxMountReadWriteOncePod][Serial] warning is bumped on two Pods with a different context on RWO volume [sig-storage] Sample failure: October 25, 2023 It fails since The PR a label to some SELinux metric and the tests do not count with that. No response /sig storage
/assign /triage accepted


Question: return supported } <ins> func omitDuplicates(kl *Kubelet, pod *v1.Pod, combinedSearch []string) []string { uniqueDomains := map[string]bool{} for _, dnsDomain := range combinedSearch { if _, exists := uniqueDomains[dnsDomain]; !exists { combinedSearch[len(uniqueDomains)] = dnsDomain uniqueDomains[dnsDomain] = true } else { log := fmt.Sprintf("Found and omitted duplicated dns domain in host search line: '%s' during merging with cluster dns domains", dnsDomain) kl.recorder.Event(pod, v1.EventTypeWarning, "DNSSearchForming", log) glog.Error(log) } } return combinedSearch[:len(uniqueDomains)] } func formDNSSearchFitsLimits(kl *Kubelet, pod *v1.Pod, composedSearch []string) []string { // resolver file Search line current limitations resolvSearchLineDNSDomainsLimit := 6 resolvSearchLineLenLimit := 255 limitsExceeded := false if len(composedSearch) > resolvSearchLineDNSDomainsLimit { composedSearch = composedSearch[:resolvSearchLineDNSDomainsLimit] limitsExceeded = true } if resolvSearchhLineStrLen := len(strings.Join(composedSearch, " ")); resolvSearchhLineStrLen > resolvSearchLineLenLimit { cutDomainsNum := 0 cutDoaminsLen := 0 for i := len(composedSearch) - 1; i >= 0; i-- { cutDoaminsLen += len(composedSearch[i]) + 1 cutDomainsNum++ if (resolvSearchhLineStrLen - cutDoaminsLen) <= resolvSearchLineLenLimit { break } } composedSearch = composedSearch[:(len(composedSearch) - cutDomainsNum)] limitsExceeded = true } if limitsExceeded { log := fmt.Sprintf("Search Line limits were exceeded, some dns names have been omitted, the applied search line is: %s", strings.Join(composedSearch, " ")) kl.recorder.Event(pod, v1.EventTypeWarning, "DNSSearchForming", log) glog.Error(log) } return composedSearch } func (kl *Kubelet) formDNSSearchForDNSDefault(hostSearch []string, pod *v1.Pod) []string { return formDNSSearchFitsLimits(kl, pod, hostSearch) } func (kl *Kubelet) formDNSSearch(hostSearch []string, pod *v1.Pod) []string { if kl.clusterDomain == "" { formDNSSearchFitsLimits(kl, pod, hostSearch) return hostSearch } nsSvcDomain := fmt.Sprintf("%s.svc.%s", pod.Namespace, kl.clusterDomain) svcDomain := fmt.Sprintf("svc.%s", kl.clusterDomain) dnsSearch := []string{nsSvcDomain, svcDomain, kl.clusterDomain} combinedSearch := append(dnsSearch, hostSearch...) combinedSearch = omitDuplicates(kl, pod, combinedSearch) return formDNSSearchFitsLimits(kl, pod, combinedSearch) } func (kl *Kubelet) checkLimitsForResolvConf() { // resolver file Search line current limitations resolvSearchLineDNSDomainsLimit := 6 resolvSearchLineLenLimit := 255 f, err := os.Open(kl.resolverConfig) if err != nil { kl.recorder.Event(kl.nodeRef, v1.EventTypeWarning, "checkLimitsForResolvConf", err.Error()) glog.Error("checkLimitsForResolvConf: " + err.Error()) return } defer f.Close() _, hostSearch, err := kl.parseResolvConf(f) if err != nil { kl.recorder.Event(kl.nodeRef, v1.EventTypeWarning, "checkLimitsForResolvConf", err.Error()) glog.Error("checkLimitsForResolvConf: " + err.Error()) return } domainCntLimit := resolvSearchLineDNSDomainsLimit if kl.clusterDomain != "" { domainCntLimit -= 3 } if len(hostSearch) > domainCntLimit { log := fmt.Sprintf("Resolv.conf file '%s' contains search line consisting of more than %d domains!", kl.resolverConfig, domainCntLimit) kl.recorder.Event(kl.nodeRef, v1.EventTypeWarning, "checkLimitsForResolvConf", log) glog.Error("checkLimitsForResolvConf: " + log) return } if len(strings.Join(hostSearch, " ")) > resolvSearchLineLenLimit { log := fmt.Sprintf("Resolv.conf file '%s' contains search line which length is more than allowed %d chars!", kl.resolverConfig, resolvSearchLineLenLimit) kl.recorder.Event(kl.nodeRef, v1.EventTypeWarning, "checkLimitsForResolvConf", log) glog.Error("checkLimitsForResolvConf: " + log) return } return } </ins> // parseResolveConf reads a resolv.conf file from the given reader, and parses // it into nameservers and searches, possibly returning an error. // TODO: move to utility package

Kubernetes version v1.3.0 Kubelet concatenates host and cluster domain entries for my pod, resulting in an invalid config (too many entries with duplicates). The max is only 6 search domains. Any more and DNS resolution breaks when trying to leverage search domains. The example below has only cluster DNS entries, but illustrates the issue: Host search nameserver 10.233.0.2 nameserver 10.90.0.1 options timeout:2 options attempts:2 Container kubectl run -i --tty busybox --image=busybox -- sh (skip waiting text) / # cat search nameserver 10.233.0.2 options attempts:2 options ndots:5 I should be enable cluster DNS config in my host DNS settings and still have functional pod DNS. Kubelet should be intelligent enough to filter out duplicate search domain entries (or drop search domains beyond the 6th entry)
I think some sane checks needs to be too. libc limits search field to 6 records and 255(?) characters. This limitation should be checked before creation.
Hey guys, I'm working on it.
Dup of


Question: metricsBindAddress: "2.3.4.5:23456", }, { <ins> // Test for 'bindAddress: "::"' (IPv6 all-zeros) in kube-proxy // config file. The user will need to put quotes around '::' since // 'bindAddress: ::' is invalid yaml syntax. name:               "iptables mode, IPv6 "::" bind address", mode:               "iptables", bindAddress:        ""::"", clusterCIDR:        "fd00:1::0/64", healthzBindAddress: "[fd00:1::5]:12345", metricsBindAddress: "[fd00:2::5]:23456", }, { // Test for 'bindAddress: "[::]"' (IPv6 all-zeros in brackets) // in kube-proxy config file. The user will need to use // surrounding quotes here since 'bindAddress: [::]' is invalid // yaml syntax. name:               "iptables mode, IPv6 "[::]" bind address", mode:               "iptables", bindAddress:        ""[::]"", clusterCIDR:        "fd00:1::0/64", healthzBindAddress: "[fd00:1::5]:12345", metricsBindAddress: "[fd00:2::5]:23456", }, { // Test for 'bindAddress: ::0' (another form of IPv6 all-zeros). // No surrounding quotes are required around '::0'. name:               "iptables mode, IPv6 ::0 bind address", mode:               "iptables", bindAddress:        "::0", clusterCIDR:        "fd00:1::0/64", healthzBindAddress: "[fd00:1::5]:12345", metricsBindAddress: "[fd00:2::5]:23456", }, { </ins> name:               "ipvs mode, IPv6 config", mode:               "ipvs", bindAddress:        "2001:db8::1",

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: /kind feature What happened: Upon code inspection, found that kube-proxy server takes into consideration a bind address of 0.0.0.0 (IPv4), but does not check for a bind address of :: (all-zeros IPv6) when calculating its node IP. What you expected to happen: IPv6 address "::" should be taken into consideration when calculating node IP. How to reproduce it (as minimally and precisely as possible): Code inspection. Anything else we need to know?: Environment: Kubernetes version (use ): Latest master Cloud provider or hardware configuration**: VirtualBox cluster, Ubuntu 16.04.1 host, CentOS 7 guests OS (e.g. from /etc/os-release): CentOS 7 Kernel (e.g. ): 3.10.0-514.26.2.el7.x8664 SMP Tue Jul 4 15:04:05 UTC 2017 x8664 x8664 x8664 GNU/Linux Install tools: Others:
There are no sig labels on this issue. Please by: a sig: e.g., to notify the contributor experience sig, OR the label manually: e.g., to apply the label Note: Method 1 will trigger an email to the group. You can find the group list and label list . The in the method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals
/area ipv6 /sig network


Question: <ins> ### Warning: Only use this plugin  in Read Only mode. There is currently [a critical bug](https://github.com/GoogleCloudPlatform/kubernetes/issues/7317) that deletes your data when used in Read/Write mode. </ins> ## Glusterfs [Glusterfs](http://www.gluster.org) is an open source scale-out filesystem. These examples provide information about how to allow containers use Glusterfs volumes.

a set of endpoints to your GlusterFS cluster. Example a pod that uses the GlusterFS Volume Plugin in ReadWrite Mode (readonly=false) to mount a particular GlusterFS volume. Example the pod is running the contents of that GlusterFS Volume and see that your files exist. the pod - "kubectl delete pod glusterfs" the contents of your GlusterFS Volume and see that your files no longer exist. Environment: GlusterFS - verified with GlusterFS running on either Fedora 21 or RHEL 6.5. Kubernetes - running on Fedora 21 Note: This issue does not occur if the GlusterFS Volume Plugin is in ReadOnly Mode (readonly=true)
heh, this is why I wanted to refactor the mount/unmount/cleanup code - get it right once :)
cc
I went back to commit (the original merge for glusterfs volume) and couldn't replicate the bug. after updating to HEAD i could replicate. so something changed between first merge and HEAD causing issue.
It's caused by commit : This is called before unmounting of volumes, which are mounted somewhere into . why RemoveAll()? It was not there before your commit.
Fixed in PR
Sorry for injecting the bug. I was thinking of moving volume & pod dir cleanup into the function, but it didn't pan out, and the code was left unintentionally. Thanks for fixing it! The logic still seems strange in some cases. looks at the containers we failed to kill, and skip cleaning up volumes associated with them. But doesn't look at the those containers and would just go ahead and remove the directories for all the pods that we supposedly killed. I will file and issue later.
Is this fixed now? If so, please close. , Yu-Ju Hong wrote:


Question: err := kl.syncPod(pod, dockerContainers) if err != nil { glog.Errorf("Error syncing pod, skipping: %v", err) <ins> record.Eventf(pod, "", "failedSync", "Error syncing pod, skipping: %v", err) </ins> } }) }

During a PD testing, I created a pod, and found it has a trouble to be started for a half hour, but events show it properly started: $ get pods Running: cluster/../cluster/gce/../../output/dockerized/bin/linux/amd64/kubectl get pods POD CONTAINER(S) IMAGE(S) HOST LABELS STATUS testpd testpd kubernetes/pause kubernetes-minion- <nonePending $ get events | grep testpd Running: cluster/../cluster/gce/../../output/dockerized/bin/linux/amd64/kubectl get events Mon, 29 Dec 2014 20:58:38 +0000 testpd BoundPod waiting created Created with docker id eb3f53865a8825a3db3b64f6f319b64bfecbcbabc7115a73c2358c8c73793004 Mon, 29 Dec 2014 20:58:38 +0000 testpd Pod Pending scheduled Successfully assigned testpd to kubernetes-minion- Mon, 29 Dec 2014 20:58:38 +0000 testpd BoundPod waiting pulled Successfully pulled image kubernetes/pause:latest Mon, 29 Dec 2014 20:58:39 +0000 testpd BoundPod running started Started with docker id eb3f53865a8825a3db3b64f6f319b64bfecbcbabc7115a73c2358c8c73793004 The issue might be missing some key events or events are misleading. Looking into it.


Question: // If a revision annotation already existed and this replica set was updated with a new revision // then that means we are rolling back to this replica set. We need to preserve the old revisions // for historical information. <del> if ok && annotationChanged { </del> <ins> if ok && oldRevisionInt < newRevisionInt { </ins> revisionHistoryAnnotation := newRS.Annotations[RevisionHistoryAnnotation] oldRevisions := strings.Split(revisionHistoryAnnotation, ",") if len(oldRevisions[0]) == 0 { newRS.Annotations[RevisionHistoryAnnotation] = oldRevision } else { <del> oldRevisions = append(oldRevisions, oldRevision) newRS.Annotations[RevisionHistoryAnnotation] = strings.Join(oldRevisions, ",") </del> <ins> totalLen := len(revisionHistoryAnnotation) + len(oldRevision) + 1 // index for the starting position in oldRevisions start := 0 for totalLen > revHistoryLimitInChars && start < len(oldRevisions) { totalLen = totalLen - len(oldRevisions[start]) - 1 start++ } if totalLen <= revHistoryLimitInChars { oldRevisions = append(oldRevisions[start:], oldRevision) newRS.Annotations[RevisionHistoryAnnotation] = strings.Join(oldRevisions, ",") } else { klog.Warningf("Not appending revision due to length limit of %v reached", revHistoryLimitInChars) } </ins> } } // If the new replica set is about to be created, we need to add replica annotations to it.

What happened: We created a Deployment and continuously updated an annotation on it periodically for a week. Eventually, we noticed errors in the controller-manager logs due to the annotation growing too large. Example annotations: controller-manager error: What you expected to happen: that do not change the revision number should not be to the revision history. revision history should be capped at a fixed number of elements, it should not be possible to have unbounded growth for Deployments that are frequently updated. How to reproduce it (as minimally and precisely as possible): a Deployment modify an annotation on it and note that the annotation gets the same number appended every time. Environment: Kubernetes version (use ):
/sig scalability Not 100% sure where this one fits, I couldn't determine which group owns the Deployment/ReplicaSet controllers, but this is a scalability problem
/sig apps
Agree with these, and if the limit is hit, the beginning of the history can be removed.
/assign


Question: // the request entity is too large. // It supports wrapped errors. func IsRequestEntityTooLargeError(err error) bool { <del> if ReasonForError(err) == metav1.StatusReasonRequestEntityTooLarge { </del> <ins> reason, code := reasonAndCodeForError(err) if reason == metav1.StatusReasonRequestEntityTooLarge { </ins> return true } <del> if status := APIStatus(nil); errors.As(err, &status) { return status.Status().Code == http.StatusRequestEntityTooLarge </del> <ins> // IsRequestEntityTooLargeError's checking of code predates the checking of // the code in the other Is* functions. In order to maintain backward // compatibility, this does not check that the reason is unknown. if code == http.StatusRequestEntityTooLarge { return true </ins> } return false }

Checked a method error using when the error was caused by a validating webhook that returns a 403 response. In this particular case, the error was generated from a validating webhook implemented with controller-runtime that returns controller-runtime's response. to return true. returned false Many (but not all) of the other family of functions in suffer the same kind of problem. I have changes ready to submit in a PR if we can get agreement that this is a problem and that it should be fixed in the functions that ought to check the of an erorr value that is or wraps an Kubernetes version (use ): 19.3 for the server
/sig api-machinery
are you going to send a PR for this? The issue description looks reasonable. /triage accepted
Thanks, PR is submitted.
/assign


Question: cmdutil.AddDryRunFlag(cmd) cmd.Flags().StringVar(&o.ClusterRole, "clusterrole", "", i18n.T("ClusterRole this ClusterRoleBinding should reference")) cmd.MarkFlagRequired("clusterrole") <del> cmd.Flags().StringArrayVar(&o.Users, "user", o.Users, "Usernames to bind to the clusterrole") cmd.Flags().StringArrayVar(&o.Groups, "group", o.Groups, "Groups to bind to the clusterrole") cmd.Flags().StringArrayVar(&o.ServiceAccounts, "serviceaccount", o.ServiceAccounts, "Service accounts to bind to the clusterrole, in the format <namespace>:<name>") </del> <ins> cmd.Flags().StringArrayVar(&o.Users, "user", o.Users, "Usernames to bind to the clusterrole. The flag can be repeated to add multiple users.") cmd.Flags().StringArrayVar(&o.Groups, "group", o.Groups, "Groups to bind to the clusterrole. The flag can be repeated to add multiple groups.") cmd.Flags().StringArrayVar(&o.ServiceAccounts, "serviceaccount", o.ServiceAccounts, "Service accounts to bind to the clusterrole, in the format <namespace>:<name>. The flag can be repeated to add multiple service accounts.") </ins> cmdutil.AddFieldManagerFlagVar(cmd, &o.FieldManager, "kubectl-create") // Completion for relevant flags

I'm not sure whether this is the documentation being unclear, or a bug with kubectl itself but I suspect the former. Given this help page I expected to be able to use Instead I got the following error To get the same output as defining the flag twice aka and should give the same output, or the help page shouldn't list as the parameter type for --serviceaccount No response <details</details<detailsN/A </details<details</details<details</details<detailsarchitecture: amd64 bootID: -c6dc-40bb-9485- containerRuntimeVersion: containerd://1.5.5 kernelVersion: 5.4.0-88-generic kubeProxyVersion: v1.23.1 kubeletVersion: v1.23.1 </details<details</details
/sig cli
the flag can be specified multiple times, or (I think) as a comma-separated list. agree the notation is confusing... I think that's how cobra prints the default value (which is an empty list)
Hi I have tried and confirmed that it works with multiple flags: But for the comma-separated list, doesn't work. Kubectl version: v1.23.3 and k8s version v1.23.3
The same update for rolebinding: Should we write like this(same for clusterrolebinding):
Thanks for the investigation, another approach that might help is adding an example with the comma separated service accounts in the examples section. I do wonder if there's a clearer representation that could be output by cobra, to make it clear what the behaviour should be...?
Hi, I think adding the example is good so that users who reference this document get the proper information. but there is already mention of how to this flag with create command under the usage section: What do you suggest
in the specific flag help, we can mention the flag can be repeated to add multiple subjects
+1 I will update this. /assign
This sounds good to me, might be worth doing the same for other places that use the same behaviour?
Sure I will do that.
I have fixed it here: PTAL!
/triage accepted


Question: // ExternalID is a test-spy implementation of Instances.ExternalID. // It adds an entry "external-id" into the internal method call record. <ins> // It returns an external id to the mapped instance name, if not found, it will return "ext-{instance}" </ins> func (f *FakeCloud) ExternalID(instance string) (string, error) { f.addCall("external-id") <del> return f.ExtID, f.Err </del> <ins> return f.ExtID[instance], f.Err </ins> } // List is a test-spy implementation of Instances.List.

I see the following in the kube-controller-manager log: - This is happening to me on HEAD and is the reason why pod status is always showing unknown even when they are scheduled and running.
This is with the vagrant cloud provider. It looks like the delta between the new and old minion is the Node.Spec.ExternalID is being set on the incoming minion but is not set in the originally created minion.
I am not sure how this is not affecting other cloud providers. It looks to me that NodeController populated the node ExternalID here: It looks like the Kubelet provided value is missing the ExternalID
Actually, I am now confused on what the NodeController may or may not be doing here that is causing the status to not get updated.
It looks like the client is passing an ExternalID on the incoming request, but the original value that is in etcd does not have an ExternalID set. Not sure what changed that would have caused this issue to appear.
I think this causes problems: The memory is re-used in a for-loop by Go so referring to a pointer value in a map here will cause problems I think.
Ok, the referenced line is the problem. Sending a PR.
Well, I go back on that now. A simple way to reproduce this is to delete all the minions in your cluster, and verify the node controller is able to add them back. For me, this is causing the node controller to add them back in an unknown status, and then all updates to the node fail. The update fails because the node controller is not setting an ExternalID on the original object.
I am looking into this now. cc/ too
Related issue opened while debugging this:
Debugged through here: I can via manual fmt see that the object has an ExternalID set prior to the call Create. When I look at etcd, I end up with this persisted: Note: it is missing an ExternalID somehow ;-)
Ok, I think I see the issue Nodes returned from here: Do not have an ExternalID Which can make Nodes created from here: End up without an External ID. Having two different nodeClient.Create() paths in this code is confusing. Also we are doing bad things in maps holding pointers to memory allocated in for-loop. We need to fix that ;-)
Yeah, this is bug we should fix. Thanks for looking into this.

If you already have a potential fix, please feel free to grab the issue from me.
I think I have a PR that should fix. Testing that now. Will cc you on the PR.
Thanks for fixing - it did end up affecting AWS as well (AWS isn't in a fully working state just yet, particularly with node discovery).


Question: Ports: []api.ServicePort{{NodePort: 111}}, }, }, <ins> &api.Service{ ObjectMeta: metav1.ObjectMeta{Namespace: "six", Name: "six"}, Spec: api.ServiceSpec{ HealthCheckNodePort: 144, }, }, </ins> ) registry := &mockRangeRegistry{

I came across a cluster where it has 2 services with port collision. One service looks like this: The other looks like: I checked the logic that allocates nodeport and healthcheck nodeport. They went thru the same node port allocator. Port allocation mainly happens in , and . The file in question is
cc
/cc
I'm not clear on your context. Which case were you facing? config and create the two services as you mentioned manifests, then get errors due to the collided ports? create the two services without specifying the two ports, then get the services and find they have collided ports? Also, what is the expected behavior you were thinking of?
When I checked the cluster, there were 2 services with colliding ports. I am not sure what brought the cluster into this state.
It's strange... I will try to dig into it later.
Seems like there is a race condition. PR sent. PTAL.
I first thought there was a race condition too. But I checked the API log. The operation on the 2 services are quite separated. Then I am not sure.
Oh... So strange. I will take another look at the code to try to find some clues. Btw, does the lock to make any sense? It doesn't fix this issue though.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten /remove-lifecycle stale
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-testing, kubernetes/test-infra and/or . /close
you can't re-open an issue/PR unless you authored it or you are assigned to it.

<detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
you can't re-open an issue/PR unless you authored it or you are assigned to it. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
you can't re-open an issue/PR unless you authored it or you are assigned to it. <detailsIn response to : /assign /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen
/assign
Found the issue. It's due to the nodeport repair controller ignoring the healthCheckNodePort.
Will send a PR and cherrypicks.


Question: NS="${8:-$NS}"          # namespace NODE_PORT="${9:-$NODE_PORT}" #nodePort, see fe-s.json echo "Running w/ args: kubectl $kubectl version $VERSION sec $_SECONDS fe $FE lg $LG slave $SLAVE test = $TEST, NAMESPACE = $NS, NODE_PORT = $NODE_PORT" <ins> function create_ns { case "$NS" in "default" ) ;; "kube-system" ) ;; * ) cat << EOF > ns.json { "apiVersion": "v1", "kind": "Namespace", "metadata": { "name": "$NS" } } EOF $kubectl create -f ns.json esac } </ins> function create { cat << EOF > fe-rc.json

During the review of , it's better to also update and to use non-default namespace. cc .
cc
/ , I can continue to work on this one :).
/ , can you help to assign this issue to me? So I can trace the tasks in my hand :).
I'd be happy to assign you once you accept the invitation to the org. Do you plan to work on this now? If you're not actively working on it, I'd prefer to leave it unassigned.
, would you help to send the invitation ? Yes, I'd like to work on this now, that's a good start for me to learn example & e2e test suite :).
Already sent. Visit to accept. See
, I'm kuber now , thanks. I'm working on this now :). Checked and , openshift-origin is not using default namespace now; and we can update 's script to update namespace. But it seems and other example are still using default namespace, should we go through all example and update them accordingly?
, should I update all ns in one commit or one commit per example?
In which examples do you plan to make this change? cc
Making this change in some examples would be useful, but only if we explain why we're doing it. Additionally, linking to existing docs and blogs about namespaces would be helpful: I don't want to do this in all of the examples, because it does add an extra step, and labels also provide a means of managing a group of related resources within a namespace. Perhaps could recommend examples where this would make sense.


Question: // can't reliably simulate periodic sync of volumes/claims - it would be // either very timing-sensitive or slow to wait for real periodic sync. func TestControllerSync(t *testing.T) { <ins> expectedChanges := []int{1, 4, 1, 1} </ins> tests := []controllerTest{ // [Unit test set 5] - controller tests. // We test the controller as if

Not many details in the logs: FAIL 2.721s
I ran tests a few times against recent master but couldn't see this, could it be an edgy case?
i'll look at it
should help a lot, still there will be a short window where the test could get wrong. Patch will follow when is merged.
Today I saw a very similar error to the one reported in , but seeing as that was closed as a dupe of this, adding details here:
this looks like something new. Let's keep this issue open for this unit test flake: I filled for integration test flake you reported.


Question: runtime_config="--runtime-config=${RUNTIME_CONFIG}" fi <ins> # Let the API server pick a default address when API_HOST # is set to 127.0.0.1 advertise_address="" if [[ "${API_HOST}" != "127.0.0.1" ]]; then advertise_address="--advertise_address=${API_HOST}" fi </ins> APISERVER_LOG=/tmp/kube-apiserver.log sudo -E "${GO_OUT}/hyperkube" apiserver ${priv_arg} ${runtime_config} <ins> ${advertise_address}  </ins> --v=${LOG_LEVEL}  --cert-dir="${CERT_DIR}"  --service-account-key-file="${SERVICE_ACCOUNT_KEY}"

seems to have broken the kubernetes service when using : With this patch: Without it: Tested with v 1.3.2.
Looks like we don't like explicitly don't want to allow 127.0.0.1 loopback address here -


Question: return false } <del> func findRule(rules []network.LoadBalancingRule, rule network.LoadBalancingRule) bool { </del> <ins> func findRule(rules []network.LoadBalancingRule, rule network.LoadBalancingRule, wantLB bool) bool { </ins> for _, existingRule := range rules { if strings.EqualFold(to.String(existingRule.Name), to.String(rule.Name)) && <del> equalLoadBalancingRulePropertiesFormat(existingRule.LoadBalancingRulePropertiesFormat, rule.LoadBalancingRulePropertiesFormat) { </del> <ins> equalLoadBalancingRulePropertiesFormat(existingRule.LoadBalancingRulePropertiesFormat, rule.LoadBalancingRulePropertiesFormat, wantLB) { </ins> return true } }

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: When the Idle Timeout annotation was set to "3" which isn't a supported case, kube-controller-manager is stuck in deleting Azure LB. New service with the same name couldn't be created again even without the annotation. What you expected to happen: New service should be created successfully. How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Others: /sig azure /kind bug
/assign Since I'm seeing this on some of our clusters
/priority important-soon
Thanks
- The problem here seems that when we want to call (when processing a delete event) it reconciles the loadbalancer which in turn tries to get the idleTimeout, resulting in an error. one of the strategies is to still construct our diff in case of a delete and exercise the code path at the end. This basically translate into ignoring the error when getting idleTimeout and excluding it from finding if the rule is dirty or not when we are . toughts ? Generally speaking, whenever we update a loadbalancer with unsupported values on annotations, ensuring load balancer will fail resulting in an un-cleaned LB, this is a cross-provider issue. /sig cloud-provider cc
cc
lgtm. the delete operation should continue to do cleanup job even some wrong options are provided from users.


Question: port := int(svc.Spec.Ports[0].Port) if !isTransitionTest { <del> Expect(framework.CheckAffinity(jig, nil, ingressIP, port, true, false)).To(BeTrue()) </del> <ins> Expect(framework.CheckAffinity(jig, nil, ingressIP, port, true)).To(BeTrue()) </ins> } if isTransitionTest { svc = jig.UpdateServiceOrFail(svc.Namespace, svc.Name, func(svc *v1.Service) { svc.Spec.SessionAffinity = v1.ServiceAffinityNone }) <del> Expect(framework.CheckAffinity(jig, nil, ingressIP, port, false, true)).To(BeTrue()) </del> <ins> Expect(framework.CheckAffinity(jig, nil, ingressIP, port, false)).To(BeTrue()) </ins> svc = jig.UpdateServiceOrFail(svc.Namespace, svc.Name, func(svc *v1.Service) { svc.Spec.SessionAffinity = v1.ServiceAffinityClientIP }) <del> Expect(framework.CheckAffinity(jig, nil, ingressIP, port, true, true)).To(BeTrue()) </del> <ins> Expect(framework.CheckAffinity(jig, nil, ingressIP, port, true)).To(BeTrue()) </ins> } }

<!-- Please only use this template for submitting reports about failing tests in Kubernetes CI jobs --Which jobs are failing: gce-cos-master-slow Which test(s) are failing: [sig-network] Services should have session affinity work for LoadBalancer service with ESIPP on [Slow] [DisabledForLargeClusters] Since when has it been failing: This test started to be flaky on 11/18 Testgrid link: https://k8s- Reason for failure: Looks to be a pretty straightfowards test not passing: If we look at triage, this test has always been flaky, but it got significantly more flaky (2X failures) on 10/27: Anything else we need to know: This job is non-blocking for 1.13, except if this represents an actual bug in Affinity the 1.13 team needs to know ASAP. This failure may or may not be related to . It looks like this test was disabled for large clusters, but maybe the problem wasn't the cluster size. <!-- DO NOT EDIT BELOW THIS LINE --/kind flake /sig network /priority important-soon attn:
FWIW this seems to be a long standing issue, atleast as far bck as 1.12 timeframe
Sure. But the failures have become 3X as frequent, so I still want a sign-off by sig-network.
cc
Ack. Will work on this.
we plan to lift code freeze for 1.13 tomorrow,11/28 5pm PST. If there is a new regression in 1.13 thats increased the flakes and needs addressing we need to get the fix in asap. Can you please let us know if this is a blocker for 1.13 by tomorrow, 11/28, morning 10am PST ?
Great observation. I checked on the commits around that date but wasn't able to find any change that may cause regression. Hasn't been able to reproduce the same failure locally as well.
I just noticed that the same test is flaking a lot more on 1.12 branches as well https://k8s-
Good catch on that, so the recent increase of flakiness likely is not caused by 1.13 changes so we shouldn't block release on this either. cc
But we will be looking deeper into the root cause.
Several observations: It only happens when is false.

When it's true, the test does more stuff before actually check service session affinity. It doesn't happen when executing in pod, probably due to the same reason above, the test start test pod before actually testing service. I wrote a script which does the same thing in my cluster of version 1.11.3-gke.18 in us-central1-a. cannot reproduce it. The hypothesis is that the service session affinity spec takes some time to reconcile after the service is created. It could happen either on the node or on the GCLB. Since I haven't been able to reproduce it's hard to tell. But it's more likely on GCLB side, even it could be regional issue. Looks like a sleep of seconds will deflake it. but it's just hiding issues. I would rather open another issue to track the real issue.
The most recent failure is here: Basically what happened is GCLB doesn't honor session affinity and still sends the request to different VMs. The log shows that 3 requests are routed to 3 different VMs. Dec 19 07:40:58.094: INFO: [service-fsn65 service-bf8nc] Dec 19 07:40:58.094: INFO: Affinity should hold but didn't. Dec 19 07:40:58.196: INFO: [pod,node] pairs: [{Pod:service-bf8nc Node:bootstrap-e2e-minion-group-f3nb} {Pod:service-fsn65 Node:bootstrap-e2e-minion-group-9sp7} {Pod:service-rnrcw Node:bootstrap-e2e-minion-group-m5rj}]; err: <nil
More recent failures: Logs:
Hello, could this issue be prioritized to help with stabilizing the release blocking upgrade tests? They're currently flaking ~23% of the time in () ~30% of the time in ()
/assign can you take a look
GitHub didn't allow me to assign the following users: take, a, look, can, you. Note that only and repo collaborators can be assigned and that issues/PRs can only have 10 assignees at the same time.

For more information please see <detailsIn response to : /assign can you take a look Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I am preparing a PR, basically allow Affinity some time to be configured, instead of failing out instantly.
Thanks! Will keep an eye out for it.
The test is no longer flaky: https://k8s- Thanks everyone!
Thanks! Will keep an eye on it for a few days, on the following dashboards:
It's stabler now, but still flaking once/twice a day, e.g. in . - Is this the same issue as you looked at? Is there more to be done here? /reopen
Reopened this issue. <detailsIn response to : It's stabler now, but still flaking once/twice a day, e.g. in . Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
we should probably cherrypick into 1.13 in case of deflaking upgrade/downgrade job, because those jobs would run 1.13 tests against 1.14 (upgraded) cluster or 1.13 (downgraded) cluster. FYI we've hit similar situation for DNS test (ref ).
Good point. Cherrypick PR prepared.
14: <img width="1110" alt="Screen Shot 2019-03-16 at 12 11 07 PM" src="https://user-"13: <img width="1091" alt="Screen Shot 2019-03-16 at 12 11 22 PM" src="https://user-"Do we want to cherry-pick this back further?
Thanks for the nice graph. Look like the fix works. I'll create cherrypick PRs for 1.11 and 1.12 right away.
AFAICT this has been significantly improved: ! Closing this and thanks to /close
Closing this issue. <detailsIn response to : AFAICT this has been significantly improved: ! /close Instructions for interacting with me using PR comments are available .


Question: config := e2enetwork.NewNetworkingTestConfig(f, false, false) ginkgo.By(fmt.Sprintf("dialing(http) %v --> %v:%v (config.clusterIP)", config.TestContainerPod.Name, config.ClusterIP, e2enetwork.ClusterHTTPPort)) message := strings.Repeat("42", 1000) <del> config.DialEchoFromTestContainer("http", config.ClusterIP, e2enetwork.ClusterHTTPPort, config.MaxTries, 0, message) </del> <ins> err := config.DialEchoFromTestContainer("http", config.ClusterIP, e2enetwork.ClusterHTTPPort, config.MaxTries, 0, message) if err != nil { framework.Failf("failed dialing endpoint, %v", err) } </ins> }) ginkgo.It("should be able to handle large requests: udp", func() { config := e2enetwork.NewNetworkingTestConfig(f, false, false) ginkgo.By(fmt.Sprintf("dialing(udp) %v --> %v:%v (config.clusterIP)", config.TestContainerPod.Name, config.ClusterIP, e2enetwork.ClusterUDPPort)) message := "n" + strings.Repeat("o", 1999) <del> config.DialEchoFromTestContainer("udp", config.ClusterIP, e2enetwork.ClusterUDPPort, config.MaxTries, 0, message) </del> <ins> err := config.DialEchoFromTestContainer("udp", config.ClusterIP, e2enetwork.ClusterUDPPort, config.MaxTries, 0, message) if err != nil { framework.Failf("failed dialing endpoint, %v", err) } </ins> }) })

What would you like to be : Right now we poll one node at a time for intra-pod tests. We ultimately fail a test if any one node is unreachable. If we instead polled all nodes, multiple times, and collected the number of times a node was unreachable, this would tell us what % of nodes on the cluster were broken, rather then just telling us the first node that was broken. not break the testing contract (were still verifying intra-pod connectivity, just doing so in a way that gives us holistic cluster info) Why is this needed: I described why this would make triage alot easier, and in general make it easier to see why tests are failing, here ... i'll collect a little more data on this to justify the decision .... as this is a conformance test so the bar for modification is higher (even if the test itself isnt changing) then regular tests.
/sig network
/assign
/triage unresolved Comment when the issue is assessed and confirmed. I am a bot run by vllry.
/remove-triage unresolved


Question: package api import ( <del> "github.com/fsouza/go-dockerclient" </del> "github.com/GoogleCloudPlatform/kubernetes/pkg/util" <ins> "github.com/fsouza/go-dockerclient" </ins> ) // Common string formats

What would you like to be : We'd like to have the Prometheus stack () running in our scalability tests. Why is this needed: There are couple of reasons: will simplify writing SLI assertions/tests that are based on metrics not necessarily exported on master. Some examples here are the new In-Cluster Network Programming Latency SLI (exported in kube-proxy) or soon to be implemented RTT SLI where the metric will be exported in regular pods (probes). With Prometheus stack, collecting and aggregating such metrics is very simple. -prometheus provides many useful metrics about the state of the cluster, such metrics might be invaluable when debugging scalability test failures. mimics what customers do in their cluster, making the test setup more realistic.
/sig scalability
/assign
The GCE 5K Node Performance test with enabled Prometheus completed successfully. We can enable Prometheus in all tests.
Looks like kubemark-500 has some issue, test fails often with Kubemark-5000 also failed once with Will disable Prometheus in kubemark, will work on fixing it when I'm back from my vacation.
For the first kind of the failure that kubemark-500 suffered from, i.e. the root cause is Looks like the token for the "prometheus-operator" service account was not ready, as we can see in the logs it was created after the POST to create the servicemonitor was called.
It looks like the second failure (kubemark-5000) was caused by the prometheus-k8s pods not scheduled by the kube-scheduler. This is strange but may be related to other issues we had. Hard to investigate what really happened there, will leave it for now. Will re-enable prometheus tomorrow and will see if it happens again.
Closing, as this is basically done in a minimal version. Other improvements will be tracked in separate issues. /close
Closing this issue. <detailsIn response to : Closing, as this is basically done in a minimal version. Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: return tabbedString(func(out io.Writer) error { w := NewPrefixWriter(out) w.Write(LEVEL_0, "Name:t%sn", pdb.Name) <ins> w.Write(LEVEL_0, "Namespace:t%sn", pdb.Namespace) </ins> if pdb.Spec.MinAvailable != nil { w.Write(LEVEL_0, "Min available:t%sn", pdb.Spec.MinAvailable.String())

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: FEATURE REQUEST What happened: ! ! What you expected to happen: "kubectl describe pdb" may contain namespaces. How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): Client Version: version.Info{Major:"1", Minor:"8+", GitVersion:"v1.8.0-alpha.1.687+", GitCommit:"", GitTreeState:"clean", BuildDate:"2017-07-03T02:17:22Z", GoVersion:"go1.8", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"8+", GitVersion:"v1.8.0-alpha.1.687+", GitCommit:"", GitTreeState:"clean", BuildDate:"2017-07-03T02:17:22Z", GoVersion:"go1.8", Compiler:"gc", Platform:"linux/amd64"} Cloud provider or hardware configuration**: OS (e.g. from /etc/os-release): NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" IDLIKE="rhel fedora" VERSIONID="7" PRETTYNAME="CentOS Linux 7 (Core)" ANSICOLOR="0;31" CPENAME="cpe:/o:centos:centos:7" HOMEURL="" BUGREPORTURL="" CENTOSMANTISBTPROJECT="CentOS-7" CENTOSMANTISBTPROJECTVERSION="7" REDHATSUPPORTPRODUCT="centos" REDHATSUPPORTPRODUCTVERSION="7" Kernel (e.g. ): Linux k8s 3.10.0-514.26.1.el7.x8664 SMP Thu Jun 29 16:05:25 UTC 2017 x8664 x8664 x8664 GNU/LinuxInstall tools: rpm Others:
There are no sig labels on this issue. Please by: a sig: e.g., to notify the contributor experience sig, OR the label manually: e.g., to apply the label Note: Method 1 will trigger an email to the group. You can find the group list and label list .

The in the method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals
/sig cli
/sig cli
/kind bug


Question: apierrors "k8s.io/apimachinery/pkg/api/errors" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/labels" <ins> apitypes "k8s.io/apimachinery/pkg/types" </ins> utilerrors "k8s.io/apimachinery/pkg/util/errors" utilruntime "k8s.io/apimachinery/pkg/util/runtime" "k8s.io/apimachinery/pkg/util/sets"

A lot of different routines failing just when setting up an apiserver. Example: <details</detailsNo errors The test still passes, but those errors seem very worrisome <detailsmaster </details<details</details<details</details<details</details<details</details<details</details
/sig api-machinery /sig testing cc
yeah, the framework has some long standing tech debt /cc
all the panics I see are:
looks like again, due to
Looks like some integration tests must not be enabling SSA.
yeah, a lot don't, because it requires openapi, which also isn't universally enabled in integration. it is really expensive at API server startup, and when we tried enabling it universally in integration it blew up runtimes and integration timed out super frequently
that's what is still open for
/triage accepted


Question: // UpdateWithEventNamespace is the same as a Update // except that it sends the request to the event.Namespace. UpdateWithEventNamespace(event *v1beta1.Event) (*v1beta1.Event, error) <del> // PatchWithEventNamespace is the same as an Update </del> <ins> // PatchWithEventNamespace is the same as a Patch </ins> // except that it sends the request to the event.Namespace. PatchWithEventNamespace(event *v1beta1.Event, data []byte) (*v1beta1.Event, error) }

Follow up from We should cover events expansion - both for core v1: events v1beta: with unit tests - can you please take care of it?
There are no sig labels on this issue. Please add an appropriate label by using one of the following commands: - Please see the for a listing of the SIGs, working groups, and committees available. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign
/assign
GitHub didn't allow me to assign the following users: try, I, want, to, have, a. Note that only , repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : /assign I want to have a try Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: }) } } <ins> type fakeAuthorizer struct { accept bool } func (f *fakeAuthorizer) Authorize(a authorizer.Attributes) (bool, string, error) { if f.accept { return true, "", nil } return false, "denied", nil } func TestAdmitUpdate(t *testing.T) { tests := []struct { name             string oldInitializers  *metav1.Initializers newInitializers  *metav1.Initializers verifyUpdatedObj func(runtime.Object) (pass bool, reason string) err              string }{ { name:            "updates on initialized resources are allowed", oldInitializers: nil, newInitializers: nil, err:             "", }, { name:            "updates on initialized resources are allowed", oldInitializers: &metav1.Initializers{Pending: []metav1.Initializer{{Name: "init.k8s.io"}}}, newInitializers: &metav1.Initializers{}, verifyUpdatedObj: func(obj runtime.Object) (bool, string) { accessor, err := meta.Accessor(obj) if err != nil { return false, "cannot get accessor" } if accessor.GetInitializers() != nil { return false, "expect nil initializers" } return true, "" }, err: "", }, { name:            "initializers may not be set once initialized", oldInitializers: nil, newInitializers: &metav1.Initializers{Pending: []metav1.Initializer{{Name: "init.k8s.io"}}}, err:             "field is immutable once initialization has completed", }, } plugin := initializer{ config:     nil, authorizer: &fakeAuthorizer{true}, } for _, tc := range tests { oldObj := &v1.Pod{} oldObj.Initializers = tc.oldInitializers newObj := &v1.Pod{} newObj.Initializers = tc.newInitializers a := admission.NewAttributesRecord(newObj, oldObj, schema.GroupVersionKind{}, "", "foo", schema.GroupVersionResource{}, "", admission.Update, nil) err := plugin.Admit(a) switch { case tc.err == "" && err != nil: t.Errorf("%q: unexpected error: %v", tc.name, err) case tc.err != "" && err == nil: t.Errorf("%q: unexpected no error, expected %s", tc.name, tc.err) case tc.err != "" && err != nil && !strings.Contains(err.Error(), tc.err): t.Errorf("%q: expected %s, got %v", tc.name, tc.err, err) } } } </ins>

There are two issues: doesn't have merge key The validation code regards it as an error if both Initializers.Result and Initializers.Pending are nil. So if multiple initializer controllers sending patches removing themselves from Initializers.Pending at the same time, the last patch that gets applied will get a validation error.
/sig api-machinery


Question: var _ = SIGDescribe("server version", func() { f := framework.NewDefaultFramework("server-version") <del> ginkgo.It("should find the server version", func() { </del> <ins> /* Release : v1.19 Testname: Confirm a server version Description: Ensure that an API server version can be retrieved. Both the major and minor versions MUST only be an integer. */ framework.ConformanceIt("should find the server version", func() { </ins> ginkgo.By("Request ServerVersion")

3 July 2020 [x] Two weeks soak end date: 20 July 2020 According to this APIsnoop query, there are still an endpoint which is untested. Get the server version (hits api path /version) Check that the major version is current Discover useragents: List endpoints hit by the test: Display endpoint coverage change: If a test with these calls gets merged, test coverage will go up by 1 points This test is also created with the goal of conformance promotion. /sig testing /sig architecture /area conformance
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen was Merge but the merge was reversed by Issue and Test grid restarted.
Reopened this issue. <detailsIn response to : /reopen was Merge but the merge was reversed by Issue and Test grid restarted. </details
Automatic retest and Merge again


Question: etcd-servers: http://localhost:4001 metadata: "role=master" flannel: <del> interface: eth1 </del> units: - name: setup-network-environment.service command: start

Tried to run through this guide: On VMware, there is no eth1, so the flannel configuration dies. Once you get past that, there is also an issue where etcdctl doesn't properly update the flannel network settings because the etcd server isn't set to the master.


Question: } func expectTrackedResourceVersion(t *testing.T, tracker *endpointSliceTracker, slice *discovery.EndpointSlice, expectedRV string) { <del> rrv := tracker.relatedResourceVersions(slice) </del> <ins> rrv, _ := tracker.relatedResourceVersions(slice) </ins> rv, tracked := rrv[slice.Name] if !tracked { t.Fatalf("Expected EndpointSlice %s to be tracked", slice.Name)

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: The endpointSliceTracker creates a set of resource versions for each service, the resource versions in the set could be deleted when endpointslices are deleted, but the set and its key in the map is never deleted, leading to memory leak. What you expected to happen: The set of resource versions should be delete from endpointSliceTracker when the service is deleted. Environment: Kubernetes version (use ): 1.18.5
There are no sig labels on this issue. Please add an appropriate label by using one of the following commands: - Please see the for a listing of the SIGs, working groups, and committees available. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/cc


Question: out, err := rs.services.Create(ctx, service, createValidation, options) if err != nil { <del> err = rest.CheckGeneratedNameError(rs.strategy, err, service) </del> <ins> err = rest.CheckGeneratedNameError(ctx, rs.strategy, err, service) </ins> } if err == nil {

Spawned from is converting an AlreadyExists error into a server timeout error if it has a generateName That means that this returns really confusing errors:
/cc
- a simple if for the objectMeta.Name seems to do the trick... is this enough? or were you looking for better way instead of NewServerTimeoutForKind ?
if no name was provided originally, the alreadyexists error occurs after the name field is populated with a generated name, so returning in that case makes this entire function a no-op in every case I can find. I just think inferring a ServerTimeout error from an AlreadyExists error and the presence of GenerateName is really odd
This needs to be triaged as a release-blocker or not for 1.5
Not a blocker
thanks
Is it appropriate to move this to the next milestone or clear the 1.5 milestone? (and remove the non-release-blocker tag as well)
Issues go stale after 30d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. Prevent issues from auto-closing with an comment. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten /remove-lifecycle stale
/remove-lifecycle rotten
/help
This request has been marked as needing help from a contributor. Please ensure the request meets the requirements listed . If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : /help Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig cli
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with .

Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
Should we return another instead of to include the generate name information ?
seems like the correct error in other cases involving an object that already exists with that name.
Looks like is not active due to age. If anyone is interested, please feel free to work on this! Like mentioned in , a unit test along with the required changes would be good to add.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
Would be really helpful if solved :) /remove-lifecycle rotten
I am new to Kubernetes community and was going through role board recommendations on older threads - If everyone is ok, can I pick this issue and work on it.
It seems already created PR for this issue - It doesn't seem to have merged. - Let me know if you are still working, else I can take a stab (Though most of the work is already done by you) to further drive this.
I'm not working on it. Please take it forward. Thanks
Thanks for the confirmation. I will work on this.
I have taken a look at the complete flow of the issue. From what I understand, there are a couple of choices Option - (1) Validate in function if an object has both name and generateName and fail the request if that is the case. Option - (2) in , set generateName to empty if it also has a name. This will make sure that the error in CheckGeneratedNameError is (when the generated name is conflicting with the already existing name of some other object). This is essentially the approach took in this , which no longer is actively worked on. /cc
Issues go stale after 90d of inactivity.

Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
/lifecycle frozen
/needs-triage It's kind of embarrassing that this is still open.
(I think the fix may be to delete the mentioned function entirely-- why would we describe this situation as a timeout error?)
do you know someone who's interested working on this? We will come back on this issue in the next triage meeting.
/help
/triage accepted
The origin of :


Question: } } <ins> func TestDedupeAggregate(t *testing.T) { var slice []error = []error{fmt.Errorf("abc"), fmt.Errorf("abc")} var agg Aggregate agg = NewAggregate(slice) if agg == nil { t.Errorf("expected non-nil") } if s := agg.Error(); s != "abc" { t.Errorf("expected 'abc', got %q", s) } if s := agg.Errors(); len(s) != 2 { t.Errorf("expected two-elements slice, got %#v", s) } } func TestDedupePluralAggregate(t *testing.T) { var slice []error = []error{fmt.Errorf("abc"), fmt.Errorf("abc"), fmt.Errorf("123")} var agg Aggregate agg = NewAggregate(slice) if agg == nil { t.Errorf("expected non-nil") } if s := agg.Error(); s != "[abc, 123]" { t.Errorf("expected '[abc, 123]', got %q", s) } if s := agg.Errors(); len(s) != 3 { t.Errorf("expected three-elements slice, got %#v", s) } } func TestFlattenAndDedupeAggregate(t *testing.T) { var slice []error = []error{fmt.Errorf("abc"), fmt.Errorf("abc"), NewAggregate([]error{fmt.Errorf("abc")})} var agg Aggregate agg = NewAggregate(slice) if agg == nil { t.Errorf("expected non-nil") } if s := agg.Error(); s != "abc" { t.Errorf("expected 'abc', got %q", s) } if s := agg.Errors(); len(s) != 3 { t.Errorf("expected three-elements slice, got %#v", s) } } func TestFlattenAggregate(t *testing.T) { var slice []error = []error{fmt.Errorf("abc"), fmt.Errorf("abc"), NewAggregate([]error{fmt.Errorf("abc"), fmt.Errorf("def"), NewAggregate([]error{fmt.Errorf("def"), fmt.Errorf("ghi")})})} var agg Aggregate agg = NewAggregate(slice) if agg == nil { t.Errorf("expected non-nil") } if s := agg.Error(); s != "[abc, def, ghi]" { t.Errorf("expected '[abc, def, ghi]', got %q", s) } if s := agg.Errors(); len(s) != 3 { t.Errorf("expected three-elements slice, got %#v", s) } } </ins> func TestFilterOut(t *testing.T) { testCases := []struct { err      error

What happened: I configured my API server with a Webhook token authentication server by way of the "authentication-token-webhook-config-file" command-line flag, and confirmed that the API server does pass bearer tokens through, issuing HTTP POST requests with a JSON-encoded body. When my Webhook server indicates authentication failure, it responds with HTTP status code 200 and a JSON-encoded in the body, this time with the object's "Error" field populated. Here's an example: When the API server receives this response, it refuses the original motivating request with an HTTP status code 401 response, but the API server writes the following into its log file: This use of this default error message is due to an omission in file , where , and . What you expected to happen: The API server should instead log a message that includes the value from . Per a brief discussion with , he agreed that the API server log message should include such an error message from the Webhook server, provided it's present and nonempty. How to reproduce it (as minimally and precisely as possible): Configure an API server to delegate to a Webhook token authentication server. Force that Webhook server to respond "successfully" with an HTTP 200 response, but with the "Authenticated" field set to false (or omitted) and with a particular error message in the "Error" field. Make an HTTP request of the API server with an HTTP "Authentication" header with a value starting with the string "Bearer " followed by at least one more character. Inspect the logs from the API server, looking for a message that begins with "Unable to authenticate the request due to an error." Anything else we need to know?: Environment: Kubernetes version (use ):Cloud provider or hardware configuration: AWS EC2 OS (e.g. from /etc/os-release):Kernel (e.g. ):Install tools: kubeadm_
/sig auth
/assign


Question: // Check PV properties By("checking the PV") <del> Expect(pv.Spec.PersistentVolumeReclaimPolicy).To(Equal(*class.ReclaimPolicy)) </del> expectedAccessModes := []v1.PersistentVolumeAccessMode{v1.ReadWriteOnce} Expect(pv.Spec.AccessModes).To(Equal(expectedAccessModes)) Expect(pv.Spec.ClaimRef.Name).To(Equal(claim.ObjectMeta.Name)) Expect(pv.Spec.ClaimRef.Namespace).To(Equal(claim.ObjectMeta.Namespace)) <del> Expect(pv.Spec.MountOptions).To(Equal(class.MountOptions)) </del> <ins> if class == nil { Expect(pv.Spec.PersistentVolumeReclaimPolicy).To(Equal(v1.PersistentVolumeReclaimDelete)) } else { Expect(pv.Spec.PersistentVolumeReclaimPolicy).To(Equal(*class.ReclaimPolicy)) Expect(pv.Spec.MountOptions).To(Equal(class.MountOptions)) } </ins> // Run the checker if t.pvCheck != nil {

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: A couple of tests the the GKE and GCE suites have been consistently failing since yesterday. See What you expected to happen: Tests should pass. How to reproduce it (as minimally and precisely as possible): Look at failing tests. Anything else we need to know?: The failing tests correspond to the merge of CC
[MILESTONENOTIFIER] Milestone Labels Complete Issue label settings: sig/storage: Issue will be escalated to these SIGs if needed. priority/critical-urgent: Never automatically move out of a release milestone; continually escalate to contributor and SIG through all available channels. kind/bug: Fixes a bug discovered during the current release. <detailsAdditional instructions available <a href=""here</aThe commands available for adding these labels are documented <a href=""here</a</details
From one of the sample failures it looks like a nil reference: Introduced .
Unable to get in touch with so from my team will take a look. If we can fix the tests in the next couple of hours we'll send out a PR to do so, otherwise we will roll back the offending PR.
Looks like it might be caused by deferencing when it could be nil: and Verifying the fix.
Fix from is awaiting merge


Question: return p } <ins> // Volumes set the volumes and inject into the inner pod. func (p *PodWrapper) Volumes(volumes []v1.Volume) *PodWrapper { p.Spec.Volumes = volumes return p } </ins> // SchedulingGates sets `gates` as additional SchedulerGates of the inner pod. func (p *PodWrapper) SchedulingGates(gates []string) *PodWrapper { for _, gate := range gates {

We have a pod fake function in , but this is not a base util anymore, we have a lot of arguments, we can replace this with here /sig scheduling /kind cleanup /help This is an easy one to get start. Cleaner code for maintenance.
This request has been marked as needing help from a contributor. Please ensure that the issue body includes answers to the following questions: Why are we solving this issue? To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points? Does this issue have zero to low barrier of entry? How can the assignee reach out to you for help? For more details on the requirements of such an issue, please see and ensure that they are met. If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : ### What would you like to be ? /kind cleanup /help Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/remove-kind feature
I want to try this one, the problem we solve here is to replace the *v1.Pod with type PodWrapper {}? /assign
Replace with in
you mean replace ` in I check the code and didn't found out that we have makeBasePod in
I have tried it. But there will be error about cycle import, because package import package, where locate.
yes, I've tried it as well. guess I need to wait for kerthcet's response <img width="805" alt="image" src=""
Sorry, forgot to reply on this, I introduced another PR to help solve this import cycle, but waits for other maintainers' response. So generally, if that pr accepted, we can work on this one then. Thanks for the volunteer.
the import cycle is resolved, you can start your work now, thanks!
thanks for the ping!


Question: "net/url" "reflect" "sync" <del> "testing" </del> "time" v1 "k8s.io/api/core/v1"

We have many "test only" packages through out the project, containing fixtures, helpers, utilities, etc... These test libraries are not intended for use in production binaries. They often lack rigorous review and sufficient testing for use in production binaries. We should make sure they don't accidentally get pulled in. Strawman proposal, test only packages must import "testing". Blank import is sufficient: Then write a presubmit that fails if we find symbol in the release binary. This approach is trivial and fairly effective although a more sophisticated approach could print a dep chain.
/assign
I hope I will be able to contribute once this gets marked as .
/triage accepted
I think that will be enough a new script or a check in the building scripts /cc since he has experience from the providerless building effort
I got what we want to achieve here but I'm little confused about: If I execute: after adding the import then only it is giving - what do we want to achieve by executing this command exactly? I got what it is doing. I've opened a thread in slack as well:
Hello! Is there anyone working on this issue at the moment? If not and the issue it's still relevant, I would like to work on it if possible. Thank you.
I opened a PR in the meantime, if it's not needed anymore, I can close it.


Question: annotations[StarvedResourceKey] = string(resourceToReclaim) return } <ins> // getThresholdMetInfo get the threshold quantity and available for the resource resourceToReclaim func getThresholdMetInfo(resourceToReclaim v1.ResourceName, thresholds []evictionapi.Threshold, observations signalObservations) (quantity *resource.Quantity, available *resource.Quantity) { for i := range thresholds { threshold := thresholds[i] if signalToResource[threshold.Signal] == resourceToReclaim { observed, found := observations[threshold.Signal] if found { quantity := evictionapi.GetThresholdQuantity(threshold.Value, observed.capacity) return quantity, observed.available } } } return nil, nil } </ins>

when the pod is evicted due to low imagefs.available, the prompt is difficult to understand. it showed: But the container was evicted not by exceeding its request of 0 ephemeral-storage, but due to reaching the evictionHard of the node. The correct prompt message points out the reason for the pod eviction. don't print "which exceeds its request of 0.", because the container was evicted not by exceeding its request of 0. % oc label mcp worker custom-kubelet-eviction=small-pods evictionHard by creating a kubeletconfig % oc create -f custom-kubelet- custom-kubelet- several minutes, the mcp worker finish rolling out to the latest mc a pod % oc create -f hello-pod- hello-pod- a large file in container, make imagefs.available < 90Gi (the current available disk size is 111G ) command terminated with exit code 137 the node status the pod status % oc get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES hello-pod-2 0/1 ContainerStatusUnknown 1 7m58s 10.131.0.7 ip-10-0-213- <none<none% oc get pod hello-pod-2 -o yaml No response <details</details<detailsaws </details<details</details<details</details<details</details<details</details
/sig node
/assign
/assign
/triage accepted


Question: package drain import ( <ins> "errors" </ins> "io/ioutil" "net/http" "net/url"

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: kubectl drain crashed with panic when apiserver connection disappeared while there were pods that couldn't be evicted/deleted (sorry, the problem is a bit hard to reproduce and the log was lost) What you expected to happen: kubectl drain should exit with a proper error message How to reproduce it (as minimally and precisely as possible): Try kubectl drain multiple times, interrupting the connection during eviction/deletion Anything else we need to know?: The here can be in case if returns an error, and will panic upon accessing : This causes the c Environment: Kubernetes version (use ): The problem is present as of but is also present in earlier versions (I have a fix for this bug)
/priority important-longterm /sig cli


Question: exit 1 fi fi <del> # Services subnetwork. local subnet=$(gcloud beta compute networks subnets describe  --project "${PROJECT}"  --region ${REGION}  ${SERVICE_CLUSTER_IP_SUBNETWORK} 2>/dev/null) if [[ -z ${subnet} ]]; then if [[ ${SERVICE_CLUSTER_IP_SUBNETWORK} != ${INSTANCE_PREFIX}-subnet-services ]]; then echo "${color_red}Subnetwork ${NETWORK}:${SERVICE_CLUSTER_IP_SUBNETWORK} does not exist${color_norm}" exit 1 fi echo "Creating subnet for reserving service cluster IPs ${NETWORK}:${SERVICE_CLUSTER_IP_SUBNETWORK}" gcloud beta compute networks subnets create  ${SERVICE_CLUSTER_IP_SUBNETWORK}  --description "Automatically generated subnet for ${INSTANCE_PREFIX} cluster. This will be removed on cluster teardown."  --project "${PROJECT}"  --network ${NETWORK}  --region ${REGION}  --range ${SERVICE_CLUSTER_IP_RANGE} echo "Created subnetwork ${SERVICE_CLUSTER_IP_SUBNETWORK}" else echo "Subnet ${SERVICE_CLUSTER_IP_SUBNETWORK} already exists" fi </del> } function delete-firewall-rules() {

When using IP aliases on GCE, use a secondary range rather than subnetwork to reserve cluster IPs.
There are no sig labels on this issue. Please by: a sig: e.g., to notify the contributor experience sig, OR the label manually: e.g., to apply the label Note: Method 1 will trigger an email to the group. You can find the group list and label list . The in the method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals


Question: if err := options.Validate(args); err != nil { cmdutil.CheckErr(cmdutil.UsageError(cmd, err.Error())) } <del> if err := options.RunAnnotate(); err != nil { </del> <ins> if err := options.RunAnnotate(f); err != nil { </ins> cmdutil.CheckErr(err) } },

and work in this pattern: first issuing a "GET" to the API server to retrieve the latest object, then adding the label/annotation to the object, and finally issuing an "PUT" with the modified object. The request will fail if the object stored in the server has been updated between the "GET" and "PUT", even if the update has no conflict with the label/annotate request. In such cases, instead of returning an error, should retry without disturbing the end user. There are two ways to implement this: in kubectl: simply retry the label/annotate process several times if the server rejected the request because the object has been updated. The only exception is when the is set, then kubectl shouldn't retry. the strategicMergePatch instead of calling Update at client-side. PATCH is more robust because PATCH does the "get, modify, update" in the server-side, so there is less chance to have a conflict, and PATCH will retry at the server-side (see ). We need to make some adaptions though: kubectl annotate/label has the , when it's false, kubectl should retrieve the latest object to confirm there is no existing annotation/label of the same key, and then proceed with issuing the patch. if the is set, [UPDATE:] we just need to specify the resourVersion in the patch, then it will be treated as a precondition by the UpdateResource() handler in the API server. I plan to take the first approach as an immediate fix.
Immediate need is to fix the flaky e2e test AIUI.
e2e? do we have an issue out there? is an immediate fix to the flake in test-
If this is for 1.1, it needs to be P0 at this point. Should it be?
Fwiw I use annotations both from cmd line as an admin and from scripts, to experiment with loadbalancer settings. In the case of the former, I've never really had a collision, and in the case of the latter my scripts know how to retry like we retry all other updates.
Thank you When used in scripts, the annotate and label commands have collisions frequently, e.g. in the test-, see . IMO, we shouldn't require users to retry in their scripts if we know there is a simple way to avoid the collisions. Anyway, and are sent to fix this.


Question: return nil } <ins> func pruneNullsFromMap(data map[string]interface{}) { for k, v := range data { if v == nil { delete(data, k) } else { pruneNulls(v) } } } func pruneNullsFromSlice(data []interface{}) { for _, v := range data { pruneNulls(v) } } func pruneNulls(v interface{}) { switch v := v.(type) { case map[string]interface{}: pruneNullsFromMap(v) case []interface{}: pruneNullsFromSlice(v) } } </ins> // Saves the last-applied-configuration annotation in a separate SSA field manager // to prevent it from being dropped by users who have transitioned to SSA. //

As per , I'm trying to remove CPU limits. But the behaviour is different depending whether the ResourceQuota exists or not: I would expect to be idempotent, and for its behaviour to result in the same end configuration regardless of the initial configuration. a new namespace named this command: that the ResourceQuota is created and is set to it again that is removed each subsequent time it's run, as long as resourcequota/compute-resource exists resourcequota/compute-resource again that the ResourceQuota is created and is set to again, and so forth the exact same behaviour if is changed to No response <details</details<detailsOpenShift 4.13.12 </details<details</details<details</details<details</details<details</details
/sig node
Tried to replicate the issue and observed the same behaviour.
/remove-sig node /sig api-machinery /cc
What's the goal of applying here? What are you trying to achieve? I agree that this is not the right behavior though.
I'm trying to remove a previously set property. I cannot remove the ResourceQuota altogether because it contains other properties (which I left out of my example) that I do not wish to remove.
But 0 is the same as unset, isn't it?
No, if I set it to 0 then any deployment with set to higher than 0 will fail to create any pods because it will exceed the quota.
/triage accepted /cc
/assign
Hello, I'm having a similar issue. Here's my use case: We use Flux to deploy things on our cluster - in particular, we use the component and set chart values in . Some of the charts we deploy have default values we want to remove, so we want to set a value to Editing an HelmRelease with this kind of ajustments drops the null keys, so default values end up being setup again and generated manifests are invalid :( Let me know if you need anything else to keep this issue moving!
I think the correct behavior is to set the struct to the default value when or , because in the standard library does this as well, and I think we expect the result to be as similar as possible to . So our real issue here is that we didn't set the to its default value when patch.
Based on the above, I think this question is not related to the current issue.

Tracing it back to , maybe the current result is what we want. The explicit null values means that we want to remove it in patch. As mentioned , our behavior in create is also as expected. I'm not quite sure if we actually want to fix it, and how /cc I hope to get your advice as you are the implementer of
I think client-side on create should do what on update will accomplish, namely, explicit nulls in the manifest are treated as the removal of the attributes they are values of. Proof of concept opened at , I'd be happy for someone to pick that up, add test coverage, and work through any failing tests that change uncovers
Thank you, but I still have a question. If we solve the difference between create and patch behaviors in ApiServer on the client side, will third-party callers still have the same problem? Or should we not cover this scenario?
I would consider changing behavior of null in a create request server-side to be a compatibility break. values are defined for client-side apply to mean "remove the key associated with this value". I think fixing the client-side apply creation case to have the same semantic is the most narrowly scoped fix we can make while preserving compatible server-side behavior generally.
Okay, I got it, I'll follow up with


Question: // Caller is allocated a random IP if they do not specify an ipAddress. If an // ipAddress is specified, it must belong to the current project, eg: an // ephemeral IP associated with a global forwarding rule. <del> func (gce *GCECloud) ReserveGlobalAddress(addr *compute.Address) (*compute.Address, error) { </del> <ins> func (gce *GCECloud) ReserveGlobalAddress(addr *compute.Address) error { </ins> mc := newAddressMetricContext("reserve", "") op, err := gce.service.GlobalAddresses.Insert(gce.projectID, addr).Do() if err != nil { <del> return nil, mc.Observe(err) } if err := gce.waitForGlobalOp(op, mc); err != nil { return nil, err </del> <ins> return mc.Observe(err) </ins> } <del> return gce.GetGlobalAddress(addr.Name) </del> <ins> return gce.waitForGlobalOp(op, mc) </ins> } // DeleteGlobalAddress deletes a global address by name.

/kind bug What happened: Services of type LoadBalancer (GCE) with IP addresses promoted to static (by user or by orphan) cause the service controller to panic. The constant panicking will cause a reboot loop of the pod. How to reproduce it (as minimally and precisely as possible): a service on GCP it the GCP Console, promote the IP address of the LB to static and set the name to the loadBalancerName which is the same name as the forwarding rule. Environment: Kubernetes version: 1.7.0 AND 1.7.1 Cloud provider or hardware configuration: GCE Symptoms -controller-manager crash loops on the master. Mitigation** Option 1: Build your own version of branch for the master. Only the kube-controller-manager image needs to be updated. Option 2: For all affected services, set the field to the known external IP address. Steps: list of reserved IPs gcloud compute addresses list NAME REGION ADDRESS STATUS us-central1 35.35.35.35 INUSE list of all Services of type LoadBalancer kubectl get svc --all-namespaces | grep "LoadBalancer" default my-service LoadBalancer 10.0.99.201 35.35.35.35 80:/TCP,443:/TCP 58m each service of type LoadBalancer with an IP that's reserved (in the list above), continue to next step. the service's field equal to the known IP. Example_ apiVersion: v1 kind: Service metadata: creationTimestamp: 2017-07-13T00:09:31Z labels: app: echo name: my-service namespace: default resourceVersion: "" selfLink: /api/v1/namespaces/default/services/my-service uid: -675f-11e7-bee2- spec: clusterIP: 10.0.99.201 externalTrafficPolicy: Cluster loadBalancerIP: 35.35.35.35 <------- Field set to below. ports: name: port-1 nodePort: port: 80 protocol: TCP targetPort: 80 name: port-2 nodePort: port: 443 protocol: TCP targetPort: 443 selector: app: echo sessionAffinity: ClientIP type: LoadBalancer status: loadBalancer: ingress: ip: 35.35.35.35
/sig network /assign /area platform/gce
cc


Question: } e2elog.Logf("Got an error creating template %d: %v", i, err) } <del> ginkgo.Fail("Unable to create template %d, exiting", i) </del> <ins> e2elog.Fail("Unable to create template %d, exiting", i) </ins> }) pagedTable := &metav1beta1.Table{}

/help /kind cleanup /area e2e-testing-framework mini guide: make sure you have signed the CNCF clone kubernetes;/kubernetes search for ginkgo.Fail calls under test/e2e/framework e.g. replace them with this that was in this PR: send a PR for kubernetes/kubernetes
/remove-kind feature /area e2e-test-framework
/sig testing
I will take this.
thank you! Please feel free to ping #testing-commons in the Kubernetes slack if you have any questions :smiley: /cc
/remove-help
/assign


Question: } } <ins> // RequestDelegated informs the priority and fairness dispatcher that // a given request has been delegated to an aggregated API // server. No-op when priority and fairness is disabled. func RequestDelegated(ctx context.Context) { // The watch initialization signal doesn't traverse request // boundaries, so we generously fire it as soon as we know // that the request won't be serviced locally. Safe to call // for non-watch requests. WatchInitialized(ctx) } </ins> // InitializationSignal is an interface that allows sending and handling // initialization signals. type InitializationSignal interface {

On a cluster with many APIs provided by aggregated apiservers, with controllers managing those APIs, the number of occupied seats effectively has a floor of 1 per watch. If the combined max-inflight limit is reduced low enough, priority levels tend to become saturated by these long-running watch requests and other requests are starved. Watch requests consume a seat only during initialization. a PriorityLevelConfiguration and a corresponding FlowSchema to match watches against a resource served by an aggregated apiserver. some watch requests. the metric climb on the priority level per watch until all seats are exhausted. Kubernetes version (use ): (OpenShift 4.9) /sig api-machinery
cc
D'oh! I can see two obvious possible ways to cope: one apiserver not manage watch requests the are delegated to another apiserver; or a marker into the notification stream so that the non-origin apiservers can detect when initialization is done.
I would even say that for regular (non-long-running) requests, it's not obvious that they should be managed by P&F even. [I agree that for watch initialization it's particularly bad though.] - do we have a simple way to detect if the request will be served from our kube-apiserver or from some other aggregated apiserver?
There is not currently a way to know that ahead of time in the filters today. It require a list/watch, but it ought not be excessively expensive to make a 99% correct prediction, but the actual decision is made by the aggregated apiserver later in the request.
/triage accepted
: is there a way that we can plumb that knowledge forward to the filters?


Question: }, mutableSchedulingDirectivesEnabled: false, }, <ins> "invalid label selector": { job: &batch.Job{ ObjectMeta: metav1.ObjectMeta{ Name:            "myjob", Namespace:       metav1.NamespaceDefault, ResourceVersion: "0", Annotations:     map[string]string{"foo": "bar"}, }, Spec: batch.JobSpec{ Selector: &metav1.LabelSelector{ MatchLabels:      map[string]string{"a": "b"}, MatchExpressions: []metav1.LabelSelectorRequirement{{Key: "key", Operator: metav1.LabelSelectorOpNotIn, Values: []string{"bad value"}}}, }, ManualSelector: pointer.BoolPtr(true), Template:       validPodTemplateSpec, }, }, update: func(job *batch.Job) { job.Annotations["hello"] = "world" }, }, </ins> } for name, tc := range cases { t.Run(name, func(t *testing.T) {

The gets very long testing mulitple functions in such as , and . These checks should be performed in dedicated Test functions using table for different scenarios. Target functions and are being as part of The remaining checks should be migrated to these functions. Also, a test function should be to migrate the remaining tests.
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig apps
/assign
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale
Any progress on this? I'm working on and I noticed that has relatively low coverage so I was going to work on getting the coverage up.
I created that is somewhat related to this.
no progess. go ahead please
/assign