Question: return SampleAndWaterMarkObserverGenerator{ &sampleAndWaterMarkObserverGenerator{ clock:        clock, <ins> t0:           clock.Now(), </ins> samplePeriod: samplePeriod, samples:      compbasemetrics.NewHistogramVec(sampleOpts, labelNames), waterMarks:   compbasemetrics.NewHistogramVec(waterMarkOpts, append([]string{labelNameMark}, labelNames...)),

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: The apiserver is always restarting and I can't access it with any commands looks like . When I check the kubectl's status, the tells me that: And then I checked the container's status and log of the apiserver: status: log: I see that there is a line: And then I get the source code: What you expected to happen: How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): v1.19.3. But I can't use the now because of the problem, so I can't give you a formal result about the version. Cloud provider or hardware configuration: Ryzen 5 4500U 16GB memory 30GB SSD In the VMware Workstation 15.5.6 OS (e.g: ):Kernel (e.g. ):Install tools: Network plugin and version (if this is a network-related bug): Others:
The complete version for the log of apiserver:
/cc
: did you tweak that machine's clock backwards, or run any software that does this?
Time calculations should be written to use go's , not count on wall clock time always moving foward.
: The proximal problem is that uses and gets the answer that time went backwards.
sure, but should be immune to wall clock resets per
Exactly. It looks to me like this test for ordering is using the monotonic clock and somehow that is saying time went backwards.
does the quantization () factor into well-ordered detection? I'm not sure how the monotonic clock interacts with Unix / UnixNano
No, I see the quantization is buggy, it uses wall clock. But the ordering test does not depend on the quantization.
/triage accepted /priority urgent-critical
The label(s) cannot be applied, because the repository doesn't have them <detailsIn response to : /triage accepted /priority urgent-critical </details
very odd that this is repeatable. /priority critical-urgent
: it is very odd that this is repeatable. Do you have anything unusual in your environment ?

I see it is vmware workstation. Does that mess with the clock in any unusual way? I also wonder about the golang runtime; did you do your own build (if so, what version of golang) or fetch binaries from somewhere (where?); full output would be helpful.
I don't have anything unusual in my environment. I run it in the ubuntu driven by vmware workstation, because my real machine's operation system is Windows10 , and I need a virtual machine to run k8s. My computer is a laptop computer. I got the golang runtime by apt-get. I also got the distribution of k8s by apt-get too. I did not do my own build. ! Okay, I got my golang and k8s version now: !
I have exactly same problem in my local lab. Virt infra on Virtual Box 6.1.16 Guest OS
you try and, if so, does it fix your problem and does it log complaints about the monotic clock going backwards? you tell us about your host and the settings on your VM(s)?
exactly matches same error as posted by author of issue, I have NOT tried VM settings: 4 vCPUs 4GB RAM swap is OFF on OS level. Host HW: Core i9 9900K 32GB Ram BTW: I have managed to install 1.18.12 on same VM eventually - cause for me it is just lab infra to study, so 1.19.X is NOT that important for me to have on this infra.
: host OS? Is your VB "Version 6.1.16 r140961 (Qt5.6.3)"? VM settings match the following? <img width="616" alt="Screen Shot 2020-11-16 at 4 56 10 PM" src="https://user-"<img width="742" alt="Screen Shot 2020-11-16 at 4 55 58 PM" src="https://user-"<img width="720" alt="Screen Shot 2020-11-16 at 4 55 45 PM" src="https://user-"
OS Name: Microsoft Windows 10 Enterprise OS Version: 10.0.

N/A Build OS Manufacturer: Microsoft Corporation OS Configuration: Standalone Workstation OS Build Type: Multiprocessor Free ! ! ! !
I tried to reproduce the problem using Ubuntu 18.0.4.5 and Kubernetes 1.19.4 (installed by kubeadm, using Calico as the network plugin), but the problem did not strike. The guest was created from the Ubuntu live Server ISO with a very plain install, picking only OpenSSH from the things to add during the install. No explicit tweaks regarding timekeeping. In the guest: I am also running the exact same version of VBox as reported above. I have the ExtensionPack of the same version installed. My host is Windows 10 Pro version 2004. Intel Core i9-9900 (8 hyperthreaded cores, thus 16 logical processors), 32 GB RAM. Configured for automatic time sync. Last sync 34 minutes ago, with server
Same issue in my env: VM (CentOS 7.9.2009 4 CPU+12G Memory) on virtualbox(6.1) Host is Windows 10 (16G Memory, CPU Intel(R) Core(TM) i3- CPU @ 3.60GHz 3.60 GHz) docker version: docker-ce-19.03.9-3.el7.x8664 K8S version: 1.19.4 Part error log output from docker logs -f <kube-apiserver-containerid:
Hi, same here vitualbox 6.1.18 latest Linux master-1 5.4.0-72-generic latest /etc/os-release NAME="Ubuntu" VERSION="20.04.2 LTS (Focal Fossa)" ID=ubuntu IDLIKE=debian PRETTYNAME="Ubuntu 20.04.2 LTS" VERSIONID="20.04" HOMEURL="" SUPPORTURL="" BUGREPORTURL="" PRIVACYPOLICYURL="" VERSIONCODENAME=focal UBUNTU_CODENAME=focal tried with K8s 1st with 1.21.0-00 then 1.19.10-00 same Time went backwards message in the logs. Api server uses large amount of CPU at doing nothing. etcd cluster seems also to be totally lost. 3 masters setup.

thanks
Hm ... this must have something to do with the fact we're running in a VM. Funny enough ... you guys are reporting apiserver crashing. What led me to this issue is that my metrics-server crashes with this same log message instead :). I originally had ntpd installed on the VM, but removing it did not help. So ... if time goes backward, it must be something that VirtualBox does. I'll try to see if I can repro with HyperV. EDIT: I was able to get rid of this error with VirtualBox by using to disable some kind of automated clock sync that VirtualBox does by default. Don't forget to use NTPd, as you will need some time synchronization :). EDIT 2: Spoke too soon. That did not really help. It seems to have slightly delayed the issue, though, as it only presented itself after a couple of minutes... EDIT 3: So, the reason why it's still syncing time, is VBoxGuestAdditions. When they are running, the guest synces time with the host, even with the above time-sync setting disabled. One either has to stop the guest additions or pass them (but ... it's not really a configuration change: ). EDIT 4: The above doesn't seem to have an effect at all. Still, according to the metrics server, time goes backwards. Well ... let's try to completely disable the VirtualBox guest additions: (for my use case, this is totally fine, as I don't need any of the guest additions features such as shared folders or time sync). EDIT 5: Nothing helps. I reckon, either the check for "time went backwards" is somewhat buggy, or some odd time sync still takes place. For now, because I don't really need it, I am just disabling the metrics server. EDIT 6: Huh. Got it, finally. Never surrender, I guess :). Turns out, you gotta attack it from all sides. (see above) Add the argument to (see above) (this is new; turns out, even if one tells guest additions not to sync time, and disables the time syncing on the VM, something can still sync the time -- this setting makes the guest OS completely blind to the fact it is running in a VM).

I suggest to keep ntp and guest additions (with the disabled time sync) on at all times. EDIT 7: Also ... do no let your host sleep. When it sleeps and then wakes up, a lot of k8s cannot cope with it and begins rolling. It should recover, but if you can avoid sleeping, do. For me, this actually meant installing and using "Don't sleep" (alternative to "Caffeine" or similar) to keep it awake. For some reason, when you lock your computer, Windows will try to sleep. It should be possible to disable this mechanic (), but even though I did, it would still sleep. That's why I ended up with "Don't sleep". With all of this in place, it finally works as expected. None of the k8s' pods roll anymore.
I had exactly the same issue. Followed the steps listed above and it solved my issue. Thanks , much appreciated!


Question: // If a non-success status is returned, then the scheduling cycle is aborted. func (f *frameworkImpl) RunPreFilterPlugins(ctx context.Context, state *framework.CycleState, pod *v1.Pod) (_ *framework.PreFilterResult, status *framework.Status) { startTime := time.Now() <ins> skipPlugins := sets.New[string]() </ins> defer func() { <ins> state.SkipFilterPlugins = skipPlugins </ins> metrics.FrameworkExtensionPointDuration.WithLabelValues(metrics.PreFilter, status.Code().String(), f.profileName).Observe(metrics.SinceInSeconds(startTime)) }() var result *framework.PreFilterResult var pluginsWithNodes []string <del> skipPlugins := sets.New[string]() </del> logger := klog.FromContext(ctx) logger = klog.LoggerWithName(logger, "PreFilter") // TODO(knelasevero): Remove duplicated keys from log entry calls

In k8s 1.27, scheduler introduced a new "short circuit" semantics in PreFilter to bypass a Pod quickly. A change introduced here is it relies on that plugin to be stored in , which makes sense as what it was explained . However, the info, although is populated upon each plugin's PreFilter invocation, is not injected into when some PreFilter returns unschedulable: So think about this workflow: pluginA returns Skip in its PreFilter pluginB return Unschedulable in its PreFilter note: pluginA is not included in due to aforementioned bug a Pod is claimed as unschedulable (failed by pluginB) during PostFilter, it's supposed to bypass pluginA's PreEnqueueExtension's AddPod/RemovePod, but it doesn't calling a "skipped" Plugin's AddPod/RemovePod during preemption, sometimes will abort the whole preemption as the logic tries to read info from cycleState, and some plugin's "skip" version doesn't fill cycleState properly, e.g., InterPodAffinity (skip immediately w/o write preFilterStateKey). Short circuiting using Skip is fine, but should injected properly at all cases. Described in the workflow above. It's firstly captured in an out-of-tree PostFilter plugin: , but can also happen in in-tree plugins. No response v1.27 <details</details<details</details<details</details<details</details<details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/kind regression /sig scheduling
cc
Among in-tree plugins, 3 plugins employ non-nil and may return Skip in their PreFilter() implementation w/o writing to cycleState, while return error immediately if is missing in their Add/RemovePod() functions.

So technically, as long as a scheduling flow hit the following criteria, it would hit the bug (preemption doesn't function): plugin 1/2/3 return Skip in PreFilter arbitrary PreFilter plugin after the "skipped" plugin returns Unschedulable in its PreFilter
/cc
/cc
Cherry-pick PRs: 1.28: 1.27:


Question: Name:            name, Description:     fmt.Sprintf(`{"kubernetes.io/service-name":"%s"}`, serviceName), Instances:       instances, <del> SessionAffinity: translateAffinityType(affinityType), </del> <ins> SessionAffinity: translateAffinityType(svc.Spec.SessionAffinity), </ins> HealthChecks:    hcLinks, }

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: /kind bug /sig network /area cloudprovider What happened: The cloud provider interface that should . The GCE provider does this for but not for . Every other in-tree provider implements by fully updating the load balancer including updated backend nodes: , , , And the following out-of-tree providers do as well and Oracle (not oss yet). What you expected to happen: I expect that all cloud providers implement by creating or fully updating the load balancer if it already exists. Related discussion: /cc
/assign Make sense to make implementation in GCE consistent with others for the ease of up-coming service controller enhancement. This sounds like a 1.10 task.
is this task blocking you on another attempt at refactoring the service controller sync logic?
yup


Question: } } <del> // getUnschedulablePodsWithMatchingAffinityTerm returns unschedulable pods which have // any affinity term that matches "pod". </del> <ins> // getUnschedulablePodsWithCrossTopologyTerm returns unschedulable pods which either of following conditions is met: // - have any affinity term that matches "pod". // - rejected by PodTopologySpread plugin. </ins> // NOTE: this function assumes lock has been acquired in caller. <del> func (p *PriorityQueue) getUnschedulablePodsWithMatchingAffinityTerm(logger klog.Logger, pod *v1.Pod) []*framework.QueuedPodInfo { </del> <ins> func (p *PriorityQueue) getUnschedulablePodsWithCrossTopologyTerm(logger klog.Logger, pod *v1.Pod) []*framework.QueuedPodInfo { </ins> nsLabels := interpodaffinity.GetNamespaceLabelsSnapshot(logger, pod.Namespace, p.nsLister) var podsToMove []*framework.QueuedPodInfo for _, pInfo := range p.unschedulablePods.podInfoMap { <ins> if pInfo.UnschedulablePlugins.Has(podtopologyspread.Name) && pod.Namespace == pInfo.Pod.Namespace { // This Pod may be schedulable now by this Pod event. podsToMove = append(podsToMove, pInfo) continue } </ins> for _, term := range pInfo.RequiredAffinityTerms { if term.Matches(pod, nsLabels) { podsToMove = append(podsToMove, pInfo) break } } } return podsToMove }

/sig scheduling /kind cleanup Ref: A test helper function creates a goroutine to pop a Pod from the scheduling queue. This goroutine never stops until the scheduling queue is ed or is somehow stopped for waiting for a new Pod. Ideally, when reaching timeout, we should somehow stop from waiting for Pod to be enqueued to activeQ.
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
We can replace this function with and it should be enough? I think we no longer need to maintain this function.
Not enough, the core of this problem is that we cannot cancel once we start . Even if we change function to whatever, we will hit this problem. So, we need:
Or, alternatively and much more easily, maybe we can just replace all with and remove . kills the entire test if reaching timeout and we could care less about leaked goroutine then.
I don't think we can do this. From the test case: The expected behavior of this case is that we receive a timeout to make sure no pods are to . So, reaching timeout is acutally a success. So, I think: option 1: using sth like this in : option2: do sth in the test code This part is where it confused me. The current code is fine as we closed schdulerQueue after called , and after I think there is no test case using this this function anymore(seeems that you refactored the test), so are we trying to solve a potential problem?
Oh, then we can just remove NextPod in the PR.
/assign
The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue as fresh with Close this issue with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/close
Closing this issue. <detailsIn response to : /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details

/sig scheduling /kind bug AssignedPodAdded(): AssignedPodUpdated(): When assigned Pods are created/updated, cluster events (Pod/Add or Pod/Update) should be delivered for all plugins that subscribe those events. But, actually pre-filters Pods to receive those events based on Pods' affinity and . - meaning only PodAffinity/NodeResourceFit receive them actually. The problem here is similar to - in in-tree plugins, PodAffinity, PodTopologySpread and NodeResourceFit subscribe Pod add and/or update events. And, Pods rejected by PodTopologySpread will never be requeued to activeQ by Pod-related events. The same happens for custom plugins registering pod add and/or update events. All Pods will get chances to get back to activeQ by Pod-related events. Run ↓ in No response <detailsmaster </details<details</details<details</details<details</details<details</details<details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign
The Kubernetes project currently lacks enough contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue as fresh with Close this issue with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale
The Kubernetes project currently lacks enough contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue as fresh with Close this issue with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale


Question: } // We have unmamanged pods, instead of erroring and hotlooping in disruption controller, log and continue. if len(unmanagedPods) > 0 { <del> klog.V(4).Infof("found unmanaged pods associated with this PDB: %v", unmanagedPods) </del> <ins> klog.Warningf("found unmanaged pods associated with this PDB: %v", unmanagedPods) </ins> dc.recorder.Eventf(pdb, v1.EventTypeWarning, "UnmanagedPods", "Pods selected by this PodDisruptionBudget (selector: %v) were found "+ <del> "to be unmanaged or managed by an unsupported controller (missing a /scale subresource). As a result, the status of the PDB "+ "cannot be calculated correctly, which may result in undefined behavior. To account for these pods please set ".spec.minAvailable" "+ </del> <ins> "to be unmanaged. As a result, the status of the PDB cannot be calculated correctly, which may result in undefined behavior. "+ "To account for these pods please set ".spec.minAvailable" "+ </ins> "field of the PDB to an integer value.", pdb.Spec.Selector) }

When using PDBs with or (percentage), it is not supported to reference pods via a PDB selector, that do not have a supported controller owner reference: Deployment, ReplicationController, ReplicaSet, and StatefulSet controllers or a resource implementing subresource This is also documented Nevertheless, there are still people who create PDBs a pods without controllers that ignore this. For example PDBs that set , want to block eviction, and node drain (it works for them but the PDB status is invalid). Unfortunately there is no feedback for the user to receive when this happens. The KCM logs is the only place that informs the user when unmanaged pods are found. Which I think is insufficient. There should be some way to inform users about this. Since it cannot be know on admission we probably have to do it when reconciling the PDB and unmanaged pods are found. Options: an event when this is detected - probably needs to be throttled in some way a condition in a PDB status that would be set and admins could monitor these to detect misbehaving PDBs create a PDB create unmanaged pod observe incorrect status because this configuration is unsupported No response <detailsNA </details<detailsNA </details<detailsNA </details<details</details<details</details<details</details
/sig apps
/triage accepted This was discussed during today's sig-apps, we've agreed to add a warning event right next to the log statement from the original description
I'll take this on.
/assign
/reopen I did some more testing for the unsupported controllers and found out that these are covered by the event, so I posted a followup to reflect that The new event should cover only the naked pods
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: apiVersion: v1 kind: ReplicationController metadata: <del> name: kube-dns-v18 </del> <ins> name: kube-dns-v19 </ins> namespace: kube-system labels: k8s-app: kube-dns <del> version: v18 </del> <ins> version: v19 </ins> kubernetes.io/cluster-service: "true" spec: replicas: {{ pillar['dns_replicas'] }} selector: k8s-app: kube-dns <del> version: v18 </del> <ins> version: v19 </ins> template: metadata: labels: k8s-app: kube-dns <del> version: v18 </del> <ins> version: v19 </ins> kubernetes.io/cluster-service: "true" spec: containers:

Observed weird memory spikes for both the dns healthz and kube-dns pods in production. Kube-dns didn't oom because there's enough headroom, but healthz did. I can't explain the spikes yet. I tried a bunch of different repros: no , bad nslookup command, sharing the node with cpu hogs etc, but was unable to trigger it. The memory grows, but stays under 10Mi. As a short term mitigation, we stole some ram from kube-dns, because it was always comfortably within limits. Related b/
can be closed -- healthz has been removed
-- close issue?


Question: ctx, name, rest.DefaultUpdatedObjectInfo(obj, transformers...), <del> withAuthorization(rest.AdmissionToValidateObjectFunc(admit, staticAdmissionAttributes), scope.Authorizer, createAuthorizerAttributes), rest.AdmissionToValidateObjectUpdateFunc(admit, staticAdmissionAttributes), </del> <ins> withAuthorization(rest.AdmissionToValidateObjectFunc( admit, admission.NewAttributesRecord(nil, nil, scope.Kind, namespace, name, scope.Resource, scope.Subresource, admission.Create, userInfo)), scope.Authorizer, createAuthorizerAttributes), rest.AdmissionToValidateObjectUpdateFunc( admit, admission.NewAttributesRecord(nil, nil, scope.Kind, namespace, name, scope.Resource, scope.Subresource, admission.Update, userInfo)), </ins> false, ) wasCreated = created

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug /sig api-machinery What happened: The that we use to make a create validation function is given admission.Update as its operation field. In the create validation we then use the provided operation (admission.Update) , when we call the admission controllers, even though a new object is being created. A similar thing is happening with the mutating admission. This means that when an objects is created on update, the admission controllers will see this as an update request. Meaning there could be an objects in storage which never passed create admission validation and were never modified by create admission mutation. I think this is a bug. What you expected to happen: Admission controllers would handle a create-on-update exactly the same as a normal create
I think the solution to this would be to separate the static admission attributes into one for the create validation and one for the update validation. Also, the function which performs the mutating admission needs to determine whether or not it is an actual update or if the old object doesn't exist (probably by checking uid) and then call the mutating admission correctly based on which operation is being performed.
/assign I might be interested to give it a try.
How to reproduce this bug?
Instead of a reproduceable BUG, I think it's like a mismatch of conceptual design and actual implementation. If a resource allows create-on-update in its strategy e.g.: Then admission plugins would receive operations (CREATE/UPDATE) on UPDATE requests. Some problems might happen the admission plugin is written without considering the circumstances that admitting a UPDATE operation might result a object creation into the storage. Risky and confusing, WDYT?
With the current build in admission controllers, I'm not sure if there are any bugs. Maybe something with resource quota? The real problem is that there is no way to know which webhook authors will have a dependency on seeing a "create" operation for every object put into storage. If a webhook relies on creates to implement its custom logic (which should be allowed), then objects created with an update would currently be missed by the webhook.


Question: testcases := []struct { name        string zone        string <ins> location    string </ins> faultDomain string expected    string }{ { name:     "GetZone should get real zone if only node's zone is set", zone:     "1", <ins> location: "eastus", </ins> expected: "eastus-1", }, { name:        "GetZone should get real zone if both node's zone and FD are set", zone:        "1", <ins> location:    "eastus", </ins> faultDomain: "99", expected:    "eastus-1", }, { name:        "GetZone should get faultDomain if node's zone isn't set", <ins> location:    "eastus", </ins> faultDomain: "99", expected:    "99", }, <ins> { name:     "GetZone should get availability zone in lower cases", location: "EastUS", zone:     "1", expected: "eastus-1", }, </ins> } for _, test := range testcases {

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: Azure cloud provider gets the availability zone from IMDS directly and the zone is usually in lower cases in most regions. But there're chances this is not true in some regions, hence customer's workloads depend on availability zone may not work. What you expected to happen: The availability zone is always in lower cases. How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others: /kind bug /sig cloud-provider /area provider/azure
/assign
Fixed by .


Question: // created in. ServiceAnnotationILBAllowGlobalAccess = "networking.gke.io/internal-load-balancer-allow-global-access" <ins> // ServiceAnnotationILBSubnet is annotated on a service with the name of the subnetwork // the ILB IP Address should be assigned from. By default, this is the subnetwork that the // cluster is created in. ServiceAnnotationILBSubnet = "networking.gke.io/internal-load-balancer-subnet" </ins> // NetworkTierAnnotationKey is annotated on a Service object to indicate which // network tier a GCP LB should use. The valid values are "Standard" and // "Premium" (default).

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: ILB ip addresses come from the subnet that the K8s cluster uses. There is no ability to specify a different subnet. What you expected to happen: ILB ip addresses should be allocated from a different subnet if specified. How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: GCE OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/area platform/gce /assign
/sig network


Question: "containers": [ { "name": "webserver", <del> "image": "gcr.io/google_containers/nettest:1.1", "command": [ "-service=nettest" </del> <ins> "image": "gcr.io/google_containers/nettest:1.4", "imagePullPolicy": "Always", "args": [ "-service=nettest", "-port=8080", "-namespace=default", "-peers=2" </ins> ], "ports": [ {

test panicked: (root)/Kubernetes%%20suite/Monitoringverifymonitoringpodsandallclusternodesareavailableoninfluxdbusingheapster/ cc
Vish please take a look if you have cycles or reassign if there's someone with better context.
I looked at the panic and it was due to the last test (networking) timing out but still running. I think the fix is more in better cleanup between tests.
I just saw this... it's totally due to ginkgo's cleanup when a test times out. I can fix.
I will get this one.
Ugh, the thing ginkgo does when a test times out is totally lame...
Instead of working around this, I've filed I'm going to go back to a manual timeout on this test.


Question: "k8s.io/apimachinery/pkg/runtime/schema" serverstore "k8s.io/apiserver/pkg/server/storage" utilflag "k8s.io/apiserver/pkg/util/flag" <ins> "k8s.io/klog" </ins> ) // GroupVersionRegistry provides access to registered group versions.

https://k8s- sample failure: (also affecting other CI jobs) cc /sig gcp
/assign We're interested in disabling the job as blocking entirely. We're of the opinion this shouldn't have bee allowed to block the community in the first place. I swear there's a historical issue around tightening the regex somewhere.
... wrong button sorry about that
possibly caused by erroring on invalid values that were previously silently skipped opened to downgrade that error to a warning for a release
Well, I mentioned the idea of making the job optional in
PR out to drop the runifchanged regex from pull-kubernetes-e2e-gke
Confirmed turns the test green
Can call this closed when merges
/pony hooray
Couldn't find a pony matching that query. <detailsIn response to : /pony hooray Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: ARCH?=amd64 HYPERKUBE_BIN?=_output/dockerized/bin/linux/$(ARCH)/hyperkube <del> BASEIMAGE=gcr.io/google-containers/debian-hyperkube-base-$(ARCH):0.5 </del> <ins> BASEIMAGE=gcr.io/google-containers/debian-hyperkube-base-$(ARCH):0.6 </ins> TEMP_DIR:=$(shell mktemp -d -t hyperkubeXXXXXX) all: build

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: Pods with volumeMount gitRepo volumes in 1.8.0 see the following error in kubelet logs (also in ) It is important to not that the git repo clone is done via ssh. I think what is happening is that hyperkube image now used to have command in it and now it does not have it. So the git clone command that kubelet runs fails with the above stacktrace since it cannot find . What you expected to happen: gitRepo is volume mounted. How to reproduce it (as minimally and precisely as possible): Reproducible in kubernetes 1.8.0+ hyperkube images. Anything else we need to know?: This probably is due to missing ssh binary in the new 1.8.0 hyperkube images. The commit might be an issue. Will comment further if I find otherwise. Environment: Kubernetes version (use ): =1.8.0 Cloud provider or hardware configuration**: Not Application. OS (e.g. from /etc/os-release): hyperkube base image Kernel (e.g. ): Install tools: Others:
How did you installed Kubernetes may I ask?
how does this help?
Normally, it is suggest to install kubelet as a daemon on host instead of using hyperkube container. cc you may be interested in this issue.
Need your help in identifying the fix here. I can send something for review. Suggestions are welcome.
The problem is that ssh isn't installed inside of the container. I don't know if it should be either, as the hyperkube image grows so large. If you install the openssh-client, it should work
Yes I did identify that as the problem if you see the description I mentioned. I pointed to a git commit that might have caused this. I am not sure what package installation has changed in the hyperkube container. "ssh" used to be available and now it is not. For the gitrepo feature to work it should be available.
Ah okay. If it was there before, we might want to add it back. SGTM.
Yeah, I switched the hyperkube image base from to our with some additional dependencies back. Apparently due to lack of volume + hyperkube test coverage, I've missed a number of dependencies needed.

cc
we could use dropbear as a replacement for just the ssh client?
I would prefer to use the openssh client over the dropbear client. It has wider usage and a stronger security track record IMO. The dropbear client also has somewhat different usage and configuration, so arguably using it is still a regression over the previous state. Saving a couple MB isn't worth using worse utilities, however much fun "dockerfile-golfing" is.
to fix
cherrypick to release-1.8:
This should be fixed in v1.8.3.


Question: return nil } <ins> // getMatchingContainerFromList returns the first terminated container status in the list that matches the error code requirement, or nil if none match. // If the returned status is non-nil, it has a non-nil Terminated field </ins> func getMatchingContainerFromList(containerStatuses []v1.ContainerStatus, requirement *batch.PodFailurePolicyOnExitCodesRequirement) *v1.ContainerStatus { for _, containerStatus := range containerStatuses { <ins> if containerStatus.State.Terminated == nil { // This container is still be terminating. There is no exit code to match. // TODO(#113855): Remove this check when it's guaranteed that the // container is terminated. continue } </ins> if requirement.ContainerName == nil || *requirement.ContainerName == containerStatus.Name { if containerStatus.State.Terminated.ExitCode != 0 { if isOnExitCodesOperatorMatching(containerStatus.State.Terminated.ExitCode, requirement) {

Since the last days there is an unusual rate of flakiness on kind jobs, but couldn't find a pattern ! While looking at one test failure and checking at the controller manager logs, there is a panic on the controller manager that makes the controller manager to restart and impacts on all the test that are running at that time. ` Lessons learned, we should not increase timeouts forever or these problems will go unnoticed, still, they are really hard to find and is not the first time that this happens, maybe Kubernetes is too good reconciling XD It will be great to have a tool that detect panics on the components /sig scheduling
/release blocker /priority important-soon /kind bug /cc
looks like the exit code matching doesn't handle a mix of terminated and unterminated containers, needs something like this:
big +1, a restart or panic detector on control plane components would be excellent
cc
this sounds like a nice project ^^^ but is better to check first with and , I see there are some scripts that parse and dump the logs here maybe we can add some thing to report them somehow?
/assign
oops, it's holiday in Poland /unassign /assign
/remove-sig scheduling
I can add the fix proposed by but the underlying problem is . I'll prepare a PR for each and see if we are ok merging both or just the hotfix.
commented on we'll merge the panic fix right away in either case
Fix and unit test is up


Question: container_memory_working_set_bytes{container="container_b",namespace="namespace_a",pod="pod_a"} 1000 1624396278302 # HELP container_start_time_seconds [ALPHA] Start time of the container since unix epoch in seconds # TYPE container_start_time_seconds gauge <del> container_start_time_seconds{container="container_a",namespace="namespace_a",pod="pod_a"} 1.6243962483020916e+09 container_start_time_seconds{container="container_a",namespace="namespace_b",pod="pod_b"} 1.6243956783020916e+09 container_start_time_seconds{container="container_b",namespace="namespace_a",pod="pod_a"} 1.6243961583020916e+09 </del> <ins> container_start_time_seconds{container="container_a",namespace="namespace_a",pod="pod_a"} 1.6243962483020916e+09 1624396248302 container_start_time_seconds{container="container_a",namespace="namespace_b",pod="pod_b"} 1.6243956783020916e+09 1624395678302 container_start_time_seconds{container="container_b",namespace="namespace_a",pod="pod_a"} 1.6243961583020916e+09 1624396158302 </ins> `, }, {

<!-- Please only use this template for submitting reports about continuously failing tests or jobs in Kubernetes CI -- (containerd) (crio) [sig-node] ResourceMetricsAPI [NodeFeature:ResourceMetrics] when querying /resource/metrics should report resource usage through the resource metrics api Ever since this commit was merged (containerd) (crio) We need to see what the commit is breaking this particular NodeFeatures test on both crio and containerd.
/sig node
/cc
/cc
/triage accepted
ah, let's use the RT's issue /close
Closing this issue. <detailsIn response to : ah, let's use the RT's issue Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: // service to specify, in seconds, the interval between health checks. const ServiceAnnotationLoadBalancerHCInterval = "service.beta.kubernetes.io/aws-load-balancer-healthcheck-interval" <ins> // ServiceAnnotationLoadBalancerEIPAllocations is the annotation used on the // service to specify a comma separated list of EIP allocations to use as // static IP addresses for the NLB. Only supported on elbv2 (NLB) const ServiceAnnotationLoadBalancerEIPAllocations = "service.beta.kubernetes.io/aws-load-balancer-eip-allocations" </ins> // Event key when a volume is stuck on attaching state when being attached to a volume const volumeAttachmentStuck = "VolumeAttachmentStuck"

Is this a BUG REPORT or FEATURE REQUEST?: /kind feature /sig aws Anything else we need to know?: AWS NLBs support static IP addresses by associating an EIP per AZ. An already allocated EIP can be supplied with the . A user can then supply EIP allocation IDs in the service definition of kubernetes. By that the service will be available via those static IPs.
eip's for white listing are really important in the enterprise, we need this feature now, as a result we have to manually create nlb's, would really like to see this
This is also important for hosting providers, who might have huge numbers of apex domain DNS records pointed at EIPs. Associating an already-allocated EIP with a new Kubernetes-generated NLB would be huge!
This feature would save us weeks of work reimplementing the cloud controller just so we can preserve a special ELB that has static IP addresses... I really wish it was available
I'd like to contribute time to working on this. Is anyone else actively working on adding support?
That's awesome news. Thanks for working on this. Do you need assistance in testing this in an AWS environment? I might help out with that ...
Sure, the PR has been approved for testing so the binaries are available. I'm moving the code into our soak environment to get it some more exposure this week.
I've been trying with: But I only go a NLB without the EIPs deployed. Didn't find any errors in the logs. Do I have to enable a feature flag for it or kind of?
The reviewer found a typo where I may not be collecting the correct annotation. I'm waiting for the new build to complete and then rolling to our soak test. I'm passing through any AWS error and also error if the number of EIP and Subnets don't match. Let me get it online with the changes from code review and I'll dig in deeper. Also, are you using the binaries directly from the build system or just Kubernetes 1.12?
I checked with the fixes you provided yesterday. Works :) Created a service which spawns over 3AZs and supplied 3 EIP allocations. NLBs gets created correctly.
Can you also comment on the PR? I'm sure having multiple successful tests would be helpful to the reviewer.
Issues go stale after 90d of inactivity.

Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
Hey, thanks for doing this, I was just looking for a way to associate one eip to a nlb, one question though why do you need one eip for each AZ? now the nlb creates a 1 public IP that balance to 3 AZ using traefik in my case when I deploy the service, thanks
Due to the fact that EIPs are represented by network hardware in AWS's datacenter you'd like to have redundant instances. If you run with only one EIP and the AZ of this EIP goes down, you won't be able to reach the (still up and running) traefik deployment in the other two AZs.
Hi, I've seen the PR () has been waiting for an update for three months. do you need a hand? do you want me to pick it up? I am really interested on this to be merged :smile:
Can you help write the tests, ? I'm completely lost there to be honest.
Hi I've found out that test-coverage on the aws provider is low and there is no tests on the so I can't find tests on NLB. I am going to open a new issue and work on improving coverage for that and hopefully I'll find out how to add tests for the new annotation. As you can see below current coverage for and is low and can't find tests for NLB:
Thanks for the help. I thought I was just missing being able to locate them.
+1 need this ASAP We are hosted on EKS so only official version supported. Any idea when this will be merged?
.... As a workaround, I'm just manually creating a second NLB (with EIPs) and changing the listeners to forward the traffic to the automatically created target groups (from the original svc-linked NLB).
Waiting for this as well on EKS. Thanks!
would be great to have that in place. thanks! I can confirm that the manual workaround by works fine, but it is hard to automate that flow.
+1 Waiting for this as well on EKS.

+1
I have created a node project that runs as a cronjob and monitors a SOURCE target group and clones the targets to a DESTINATION target group. So when you create a LoadBalancer of type 'nlb' in kubernetes, all you need to do is manually create a new 'nlb' in AWS Console, assign your EIP to your NLB for a specific 'subnet'(availability zone), then create a target group with no targets; use the ARN from this target group as the destination and use the ARN of the kubernetes target group as the SOURCE. More information here: Public docker image:
I have hard time getting this working with NLB using ingress controller : Even subnets are not taking effect here , its not passing my configurations in the API that creates the NLB kind: Service apiVersion: v1 metadata: name: ingress-nginx namespace: ingress-nginx labels: ingress-nginx ingress-nginx annotations: nlb "eipalloc-,eipalloc-,eipalloc-" "subnet-,subnet-" spec: type: LoadBalancer selector: ingress-nginx ingress-nginx ports: name: http port: 80 targetPort: http name: https port: 443 targetPort: https =======================================================``
I am also getting the same error can anyone help me with this ??
what version of k8s you are using ? this is not supported before 1.15 as far as I know so my issue was that I tried to implement on EKS
Hi i am using the latest version of k8 1.15 .5
I am not using EKS with my solution but the problem is the same as you mentioned
New to EKS. Could you elaborate how the connectivity from NLB that we have created manually (as you mentioned ) to the ingress-nginx pod?
Has it worked for you in k8s 1.15 EKS?
How to get Allocation IDs?
my elastic eip ids cannot be found at service creation, so it just sits in pending forever. if I resort to listing the subnets on their own annotation, (not using subnet discovery) same thing. no matter what I do it seems to not find me elastic ip addresses. Are you sure nlb type is supported to work with Elastic IPs?

From the AWS docs it seem they are pretty heavily trying to discourage use of the nlb type (Cloud Provider Load Balancer) in favor of pushing the new AWS Load balancer.
See available annotations: https://kubernetes- If you have done that: what do your aws-lb-controller logs tell you? I guess there’s an access problem with your iam role. Is your aws-lb-controller iam autorized to list and modify eip’s? Are the eip annotations exactly correct, can they be found by the controller? The controller’s log stream should have some kind of an error there. I have successfully assigned eips using service annotations to provision an aws nlb, but pay attention, you need to install the aws-lb-controller yourself. The kubernetes built-in aws library does not support eips.
Here is the annotation needed for assigning a static Elastic IP to your NLB using the AWS Load Balancer (Ingress) Controller https://kubernetes- New Controller link https://kubernetes-


Question: framework.Failf("Expect endpoints: %v, got: %v", firstEndpoints, eps) } } <del> framework.Failf("test session affinity, cost time: %v", time.Now().Sub(startTime)) </del> }) }) })

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug /priority failing-test /sig network What happened: From https://k8s-, seems to be broken right after they were by Example error message: and Anything else we need to know?: Is it as expected?
Oct 11 16:37:49.106: test session affinity, cost time: 1m31.742990485s This is a mistake. Sorry for my debug lines sneak in. I was just wondering why this test case doesn't run in upstream CI, so I this debug line. Now, I see because this test case only runs in . Oct 11 16:39:42.406: Expect endpoints: map[], got: map[netserver-1:{}] This check failed unexpectedly, let me try to find out why. I am working on it and will let you know soon when I have some updates.
Thanks. These two tests don't run on pull jobs as they are marked as , though https://k8s-, https://k8s- (and other CIs that include ) would have them running.
Do you mean when I raise a PR, these test cases will not be triggered? My question is how can I make sure my follow-up PR can fix this bug? Where can I access the log message when I raise a new PR?
Can I see the updated test results via directly visiting https://k8s-, https://k8s-
Yep, that is the current behavior. A hacky way to make it run on PR --- add a temporary commit to remove the word from test name so some pull jobs ( I think?) will run it. And you could view logs by clicking the button.
Nope, this CI only includes changes that are merged. It is basically a cronjob that runs tests against head.


Question: readonly ENABLE_LEGACY_ABAC=false readonly ETC_MANIFESTS=${KUBE_HOME}/etc/kubernetes/manifests readonly KUBE_API_SERVER_DOCKER_TAG=v1.11.0-alpha.0.1808_3c7452dc11645d-dirty <del> readonly LOG_OWNER_USER=$(whoami) </del> <ins> readonly LOG_OWNER_USER=$(id -un) </ins> readonly LOG_OWNER_GROUP=$(id -gn) ENCRYPTION_PROVIDER_CONFIG={{.EncryptionProviderConfig}} ENCRYPTION_PROVIDER_CONFIG_PATH={{.EncryptionProviderConfigPath}}

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . If the matter is security related, please disclose it privately via --Is this a BUG REPORT or FEATURE REQUEST?: What happened: unit tests failed when run under What you expected to happen: unit tests pass How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: I've seen this on both docker for mac and linux, with varying uids (501 and 1000). This patch makes the package's tests pass: If this is acceptable, I will open a PR. Please let me know!
LGTM
PR:
/sig cluster-lifecycle
/assign
/assign


Question: "k8s.io/apimachinery/pkg/util/sets" "k8s.io/client-go/kubernetes" resourcev1alpha2listers "k8s.io/client-go/listers/resource/v1alpha2" <del> corev1helpers "k8s.io/component-helpers/scheduling/corev1" </del> "k8s.io/component-helpers/scheduling/corev1/nodeaffinity" "k8s.io/dynamic-resource-allocation/resourceclaim" "k8s.io/klog/v2"

The node selectors gets parsed by in for each node. That work can be done once in . Better performance.
/sig node /sig scheduling /triage accepted /priority backlog
I can try this one. /assign


Question: } type priorityQueueOptions struct { <del> clock                             util.Clock </del> <ins> clock                             clock.Clock </ins> podInitialBackoffDuration         time.Duration podMaxBackoffDuration             time.Duration podMaxInUnschedulablePodsDuration time.Duration

We should use /kind cleanup
/remove-kind feature /good-first-issue
This request has been marked as suitable for new contributors. Please ensure that the issue body includes answers to the following questions: Why are we solving this issue? To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points? Does this issue have zero to low barrier of entry? How can the assignee reach out to you for help? For more details on the requirements of such an issue, please see and ensure that they are met. If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : /remove-kind feature /good-first-issue Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scheduling /triage accepted
/assign


Question: CONVERSION_FILES := $(addsuffix /$(CONVERSION_FILENAME), $(CONVERSION_DIRS)) <ins> # Shell function for reuse in rules. RUN_GEN_CONVERSION =                                                           function run_gen_conversion() {                                            if [[ -f $(META_DIR)/$(CONVERSION_GEN).todo ]]; then                   ./hack/run-in-gopath.sh $(CONVERSION_GEN)                          --v $(KUBE_VERBOSE)                                            --logtostderr                                                  -i $$(cat $(META_DIR)/$(CONVERSION_GEN).todo | paste -sd, -)   -O $(CONVERSION_BASENAME)                                      "$$@";                                                         fi                                                                     };                                                                         run_gen_conversion </ins> # This rule aggregates the set of files to generate and then generates them all # in a single run of the tool. .PHONY: gen_conversion <del> gen_conversion: $(CONVERSION_FILES) if [[ -f $(META_DIR)/$(CONVERSION_GEN).todo ]]; then                   ./hack/run-in-gopath.sh $(CONVERSION_GEN)                          --v $(KUBE_VERBOSE)                                            --logtostderr                                                  -i $$(cat $(META_DIR)/$(CONVERSION_GEN).todo | paste -sd, -)   -O $(CONVERSION_BASENAME);                                     fi </del> <ins> gen_conversion: $(CONVERSION_FILES) $(CONVERSION_GEN) $(RUN_GEN_CONVERSION) .PHONY: verify_gen_conversion verify_gen_conversion: $(CONVERSION_GEN) $(RUN_GEN_CONVERSION) --verify-only </ins> # Establish a dependency between the deps file and the dir.  Whenever a dir # changes (files added or removed) the deps file will be considered stale.

Right now on head () if you run it will end up having a modified file. This happened because some type has been changed and this file should have been generated before commit, and verification should have failed but they didn't.
This is not only a problem with openapi-gen. deepcopy-gen also has the same problem. I've a field to a type (autoscaling/v1) and verify-all passed for the commit (without adding generated code to it). those files will be generated on verify-codegen and it will not fail and the commit can be merged without generated code.
Yeah. So when the Makefile stuff originally went in those files were not checked in. Then we un-did that part, but I didn't re-add them to verify...


Question: t.Errorf("Expected true for builder.IsReadOnly") } } <ins> func TestPortalBuilder(t *testing.T) { if portal := portalBuilder("127.0.0.1"); portal != "127.0.0.1:3260" { t.Errorf("wrong portal: %s", portal) } if portal := portalBuilder("127.0.0.1:3260"); portal != "127.0.0.1:3260" { t.Errorf("wrong portal: %s", portal) } } </ins>

Documentation says that port number for is optional, but I've found that not to be true. It's due to the way it detects whether the disk is attached or not. also must be an IP. Refer to:
/cc
I see what you meant. Good catch. According to , the disk path contains both ip and port. So port is indeed required even for default port 3260. If you don't have a pr yet, I'll provide a fix to the doc.
I don't have a CLA in file yet, so I'll leave it up to you. Thanks.
check this out


Question: }, }) Expect(err).NotTo(HaveOccurred()) <del> framework.Logf("Running containers:n") </del> <ins> framework.Logf("Running containers:") </ins> for _, c := range containers { <del> framework.Logf("%+vn", c) </del> <ins> framework.Logf("%+v", c) </ins> } } })

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: is always appending a new line to its message: In some places of code, the message is already has a newline character and it leads to doubled new lines in the logs. What you expected to happen: I'm not sure what is the correct behavior here. One of the options could be to append a new line conditionally.
/sig testing
I'm new to Kubernetes and would like to start contributing. Is it ok if I assign myself this issue?
/assign MorrisLaw
GitHub didn't allow me to assign the following users: MorrisLaw. Note that only can be assigned. <detailsIn response to : /assign MorrisLaw Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I can't assign this to myself, as I am not a member yet. But I am still going to take this issue.
I'll help him on reviews so assigning.


Question: }), }, }, <ins> "invalid node selector requirement in node affinity, key is invalid": { expectedError: "spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[0].matchExpressions[0].key", spec: core.Pod{ ObjectMeta: metav1.ObjectMeta{ Name:      "123", Namespace: "ns", }, Spec: validPodSpec(&core.Affinity{ NodeAffinity: &core.NodeAffinity{ RequiredDuringSchedulingIgnoredDuringExecution: &core.NodeSelector{ NodeSelectorTerms: []core.NodeSelectorTerm{ { MatchExpressions: []core.NodeSelectorRequirement{ { Key:      "invalid key ___@#", Operator: core.NodeSelectorOpExists, }, }, }, }, }, }, }), }, }, </ins> "invalid node field selector requirement in node affinity, more values for field selector": { expectedError: "spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[0].matchFields[0].values", spec: core.Pod{

SchedulerPredicates [Serial] validates that a pod with an invalid NodeAffinity is rejected Has been failing consistently since 4/17. Here are the /kind bug /priority failing-test /priority important-soon /sig scheduling /milestone 1.11 cc /assign for triage.
The provided milestone is not valid for this repository. Milestones in this repository: [, , , , , , , , , , ] Use to clear the milestone. <detailsIn response to : ## Failing Job SchedulerPredicates [Serial] validates that a pod with an invalid NodeAffinity is rejected /priority failing-test /priority important-soon /sig scheduling /milestone 1.11 Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/milestone v1.11
/status approved for milestone
Also fails as part of upgrade job as seen below
Also fails as part of following jobs https://k8s- https://k8s- https://k8s- https://k8s-
xref
/assign
[MILESTONENOTIFIER] Milestone Issue Needs Approval Action required: This issue must have the label applied by a SIG maintainer. If the label is not applied within 6 days, the issue will be moved out of the v1.11 milestone. <details<summaryIssue Labels</summary- : Issue will be escalated to these SIGs if needed. : Escalate to the issue owners and SIG owner; move out of milestone after several unsuccessful escalation attempts. : Fixes a bug discovered during the current release. </details<details<summaryHelp</summary<ul<li<a href=""Additional instructions</a</li<li<a href=""Commands for setting labels</a</li</ul</details
Seems like this is failing in (just with slightly different versions)


Question: verbs:     "get,watch,list", resources: "pods,pods", expectedRole: &rbac.Role{ <ins> TypeMeta: v1.TypeMeta{APIVersion: "rbac.authorization.k8s.io/v1", Kind: "Role"}, </ins> ObjectMeta: v1.ObjectMeta{ Name: roleName, },


Question: func IndexHandler(container *restful.Container, muxHelper *MuxHelper) func(http.ResponseWriter, *http.Request) { return func(w http.ResponseWriter, r *http.Request) { <ins> status := http.StatusOK if r.URL.Path != "/" && r.URL.Path != "/index.html" { // Since "/" matches all paths, handleIndex is called for all paths for which there is no handler registered. // We want to to return a 404 status with a list of all valid paths, incase of an invalid URL request. status = http.StatusNotFound } </ins> var handledPaths []string // Extract the paths handled using restful.WebService for _, ws := range container.RegisteredWebServices() {

Authenticated requests to master with an invalid URL path should return a non-status 200 response code. Preferably a 404 (not found). Sample output: { "paths": [ "/api", "/api/v1beta1", "/api/v1beta2", "/api/v1beta3", "/healthz", "/healthz/ping", "/logs/", "/metrics", "/static/", "/swagger-ui/", "/swaggerapi/", "/validate", "/version" ] }


Question: annotations = map[string]string{} } annotations[corev1.LastAppliedConfigAnnotation] = value <del> if isAnnotationsValid(annotations) != nil { </del> <ins> if err := apimachineryvalidation.ValidateAnnotationsSize(annotations); err != nil { </ins> delete(annotations, corev1.LastAppliedConfigAnnotation) } accessor.SetAnnotations(annotations)

Expose totalAnnotationSizeLimitB in the validation package: This value is useful for validating annotations outside of the validation package, such as in SSA: Otherwise, size validation on annotations outside of the package would diverge and cause issues later. /kind cleanup /wg api-expression cc
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign


Question: // JobStatus represents the current state of a Job. type JobStatus struct { // The latest available observations of an object's current state. <ins> // When a job fails, one of the conditions will have type == "Failed". </ins> // More info: https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/ // +optional // +patchMergeKey=type

The API docs for jobs are unclear as to what fields a caller would use to determine job success or failure in 1.11 (and we haven't updated them in a while) A casual reader would infer that completionTime is set when the job fails (but it isn't). Linking out to the docs is ok, but I shouldn't have to click through the links to understand how to use the Job object. The Job description and the job status fields should clearly indicate what fields are set in completion states on the job, because that's 100% of the point of docs. Taking a summary of the jobs-run-to-completion link and putting it in , , and job would improve the actual utility of our API doc. We should also do a pass on other objects and make the actual in server docs more useful.
as a consumer of the API, I had to read way too much to know how to use jobs correctly and how to write an API client that properly handled the conditions. Also, we should consider either setting completionTime on failures, or adding failedTime (the former may not be forwards compatible), because reading the conditions is too complex for a fundamental use case like this. This matches our decision w.r.t. conditions vs. fields - conditions are useful for extension and extra data, but status should have fields that allow an API consumer to easily understand state.
Also... our docs don't actually explain how to determine the job is complete via the API, so there's arguably no place this is captured.
I have the same problem right now, of correctly identifying a finished job, when it does not complete successfully and after reading the documentation and the reference and all the examples I could find, it's still not clear how I can check the conditions and make sure the job is finished when I do so: -what does each mean? can I rely on , is that set only when the job finishes, whether via or ? what other reasons are possible? It would definitely help to have specified on failure as well, and the differentiation to be made via the presence of .
/assign soltysh
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with .

Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
I have to say this issue is still very relevant. As of now, it is very hard to design something around Job Statuses. Anyway to move this forward?
Completely agree with previous comments! Even I found the way how to detect completed/failed jobs, still not sure am I covered all use cases. This part definitely needs clarifications/improvements/better documentation.
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
/remove-lifecycle rotten
With regards to: If you actually follow this link and then search on that page for "conditions" you will get exactly zero hits. This is extremely unhelpful.
This documentation really, really needs work. (So does the API, for that matter.)
So what is the correct way to determine if a job is finished and whether it completed successfully?
When a Job completes (finished successfully), its will be set, and Job conditions will have one with . When a Job fails, Job conditions will have one with .
Hi, if nobody has been working on it, can I take this one? I have gone through the issue which is mentioned here and I would like to work on this. Can you please help me to get started as I am beginner to the project.
I totally agree with this- currently the only way to get the full picture of a Job is to read its conditions along with status fields which is error prone and a lot of extra work. I think a good start would be to add a new field as suggested by so as to not break compatibility for users already depending on
This sound reasonable. I'm still unclear when can contain more than one element. Also what does a or mean?
Still, open -- anyone actively working on this? I would like to attempt if not. :)
Are you still working on this?
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
bump
This does not match what I just witnessed in my cluster… digging deeper to see if I need to file a bug but I have a Job with a single Pod that was executed. The Pod failed but the Job was marked as complete. This is the one (and only) pod that executed as my job: And this is the job that owns it (from kubectl describe): This is the job status object: on v1.16.7 but not finding any similar reports. just found this page as my only relevant search result.
I think it not only needs documentation, but also a change: It would be good to have 2 fields (probably needs better names), this way it's easy and clear when a job is finished and if it failed or not and when (someone can look at failed, succeeded and conditions for details): finishedtime: only set when the job finished (either completely failed or completely finished) finishtype: 'failed or completed'


Question: // Assumes that lock is already acquired. func (cache *schedulerCache) updatePod(oldPod, newPod *v1.Pod) error { <ins> if _, ok := cache.nodes[newPod.Spec.NodeName]; !ok { // The node might have been deleted already. // This is not a problem in the case where a pod update arrives before the // node creation, because we will always have a create pod event before // that, which will create the placeholder node item. return nil } </ins> if err := cache.removePod(oldPod); err != nil { return err }

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: If we delete node first, then remove pods on this node, the scheduler will crash in function What you expected to happen: Not crashed How to reproduce it (as minimally and precisely as possible): some pods on node A delete node A delete pods on node A Anything else we need to know?: Environment: Kubernetes version (use ): 1.16-1.18, even master Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig scheduling
can you show the core dump please of this crash?
/assign
I patched the unit test shown in into the , and branches. None crashed. I'll await for the crash dump.
Here is the fresh dump.
and which version was this?
This is master branch. And I've tested it from 1.16-1.18, it can be reproduced easily.
Thanks a lot. I was able to finally reproduce in unit tests. The scenario is: a node with at least 2 pods. node. pod1. In unit test, this is first an update to set the termination timestamp. No issues up to this point. pod2. Again, first an update to set the termination timestamp. Here it crashes. I'm working on a fix.
It looks like this is not an issue in 1.17 and below, because we preserved a placeholder node where the operations occur. do you have stacktraces for 1.17 and 1.16?
Yes, it seems only be found at 1.18. :smile:


Question: impersonationUser: "system:serviceaccount:foo:default", expectedUser: &user.DefaultInfo{ Name:   "system:serviceaccount:foo:default", <del> Groups: []string{"system:serviceaccounts", "system:serviceaccounts:foo"}, </del> <ins> Groups: []string{"system:serviceaccounts", "system:serviceaccounts:foo", "system:authenticated"}, Extra:  map[string][]string{}, }, expectedCode: http.StatusOK, }, { name: "anonymous-username-prevents-adding-authenticated-group", user: &user.DefaultInfo{ Name: "system:admin", }, impersonationUser: "system:anonymous", expectedUser: &user.DefaultInfo{ Name:   "system:anonymous", Groups: []string{}, Extra:  map[string][]string{}, }, expectedCode: http.StatusOK, }, { name: "unauthenticated-group-prevents-adding-authenticated-group", user: &user.DefaultInfo{ Name: "system:admin", }, impersonationUser:   "unknown", impersonationGroups: []string{"system:unauthenticated"}, expectedUser: &user.DefaultInfo{ Name:   "unknown", Groups: []string{"system:unauthenticated"}, Extra:  map[string][]string{}, }, expectedCode: http.StatusOK, }, { name: "unauthenticated-group-prevents-double-adding-authenticated-group", user: &user.DefaultInfo{ Name: "system:admin", }, impersonationUser:   "unknown", impersonationGroups: []string{"system:authenticated"}, expectedUser: &user.DefaultInfo{ Name:   "unknown", Groups: []string{"system:authenticated"}, </ins> Extra:  map[string][]string{}, }, expectedCode: http.StatusOK,

<!-- Thanks for filing an issue! Before hitting the button, please answer these questions.--Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see ): No What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.): impersonate system:authenticated Is this a BUG REPORT or FEATURE REQUEST? (choose one): FEATURE REQUEST <!-- If this is a BUG REPORT, please: Fill in as much of the template below as you can. If you leave out information, we can't help you as well. If this is a FEATURE REQUEST, please: Describe in detail the feature/behavior/change you'd like to see. In both cases, be ready for followup questions, and please respond in a timely manner. If we can't reproduce a bug or think a feature already exists, we might close your issue. If we're wrong, PLEASE feel free to reopen it and explain why. --Kubernetes version (use ): Environment: Cloud provider or hardware configuration: baremetal OS (e.g. from /etc/os-release): Centos7 Kernel (e.g. ): Install tools: custom Others: Using RBAC What happened: Impersonating a user is not setting group which causes impersonate to fail What you expected to happen: If a user is in the tokens file they and they are impersonated the group should be for authz. How to reproduce it (as minimally and precisely as possible): sure group has cluster role a test user via token file a user with impersonate access submit a command via kubectl That request will fail with 403 response even though the user is in the group. Anything else we need to know: cc
We can probably be smart and put the authenticated group adder around the result of the impersonation filter. you see any issues with that?
We provide the group field to provide this kind of information. I'm not sure I would expect my request to the server about a particular user/group tuple to be evaluated for a different user/group tuple.
for impersonation, we already add groups when known (service accounts). doing so for the system:authenticated/system:unauthenticated groups makes sense to me
Only when the groups aren't specified, right? If memory serves, that's how we did it downstream where we can make group determination.


Question: continue } <del> if containerIP := ds.getIP(c.ID, r); containerIP != "" { return containerIP </del> <ins> // Versions and feature support // ============================ // Windows version == Windows Server, Version 1709,, Supports both sandbox and non-sandbox case // Windows version == Windows Server 2016   Support only non-sandbox case // Windows version < Windows Server 2016 is Not Supported // Sandbox support in Windows mandates CNI Plugin. // Presence of CONTAINER_NETWORK flag is considered as non-Sandbox cases here // Todo: Add a kernel version check for more validation if networkMode := os.Getenv("CONTAINER_NETWORK"); networkMode == "" { // On Windows, every container that is created in a Sandbox, needs to invoke CNI plugin again for adding the Network, // with the shared container name as NetNS info, // This is passed down to the platform to replicate some necessary information to the new container // // This place is chosen as a hack for now, since ds.getIP would end up calling CNI's addToNetwork // That is why addToNetwork is required to be idempotent // Instead of relying on this call, an explicit call to addToNetwork should be // done immediately after ContainerCreation, in case of Windows only. TBD Issue # to handle this if r.HostConfig.Isolation == kubeletapis.HypervIsolationValue { // Hyper-V only supports one container per Pod yet and the container will have a different // IP address from sandbox. Return the first non-sandbox container IP as POD IP. // TODO(feiskyer): remove this workaround after Hyper-V supports multiple containers per Pod. if containerIP := ds.getIP(c.ID, r); containerIP != "" { return containerIP } } else { // Do not return any IP, so that we would continue and get the IP of the Sandbox. // Windows 1709 and 1803 doesn't have the Namespace support, so getIP() is called // to replicate the DNS registry key to the Workload container (IP/Gateway/MAC is // set separately than DNS). // TODO(feiskyer): remove this workaround after Namespace is supported in Windows RS5. ds.getIP(sandboxID, r) } } else { // ds.getIP will call the CNI plugin to fetch the IP if containerIP := ds.getIP(c.ID, r); containerIP != "" { return containerIP } </ins> } }

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . If the matter is security related, please disclose it privately via --Is this a BUG REPORT or FEATURE REQUEST?: What happened: PR tries to reduce CNI calls for sandboxed containers: case is for Windows Server 2016 and for Windows Server version greater than 1709 which use Hyper-V containers. Currently, the CNI on Windows fetches the IP from the containers within the pods regardless of the mode. This should be done only in the non-sandbox mode where the IP of the actual container will be different than the IP of the sandbox container. from the same pod will share the network details of the sandbox container. when this mode is supported. However, for Windows 1709 and 1803 we didnt have the Namespace support in Windows which we are bringing in RS5. So we need this second call to replicate the DNS registry key to the Workload container. This is a platform requirement and removing this will break DNS for the workload containers. IP/Gateway/MAC is set separately than DNS. (Refer ) In Rs5 once we move the namespace model, we will update CRI and CNI to add endpoints and add it to the namespace. So the PR breaks dns for windows containers. We should revert it. /sig windows /kind bug
Since is headed for 1.11, this issue should be similarly milestone 1.11 labeled and approved. /milestone v1.11 /status approved-for-milestone
You must be a member of the github team to set the milestone. <detailsIn response to : Since is headed for 1.11, this issue should be similarly milestone 1.11 labeled and approved. /status approved-for-milestone Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
You must be a member of the github team to add status labels.
/milestone v1.11 /status approved-for-milestone
if you can drop an update here on status of the PR revert of PR , and if there's anything additional that might be happening like a re-fix of what was being attempted in the earlier PR, the 1.11 release team would appreciate the additional context.

PR has been reviewed by but still waiting for sig-node's approval
/priority critical-urgent /status in-progress
[MILESTONENOTIFIER] Milestone Issue: Up-to-date for process Note: This issue is marked as , and must be updated every 1 day during code freeze. Example update: <details open<summaryIssue Labels</summary- : Issue will be escalated to these SIGs if needed. : Never automatically move issue out of a release milestone; continually escalate to contributor and SIG through all available channels. : Fixes a bug discovered during the current release. </details<details<summaryHelp</summary<ul<li<a href=""Additional instructions</a</li<li<a href=""Commands for setting labels</a</li</ul</details


Question: "time" <del> "k8s.io/api/core/v1" </del> <ins> v1 "k8s.io/api/core/v1" </ins> metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/util/wait" clientset "k8s.io/client-go/kubernetes"

created a spread sheet of the slowest running tests for the pull-kubernetes-gce job. cc discussion on slack: spreadsheet: pie chart of test durations from SIGs: ! it seems appropriate to evaluate the state of the tests and reduce these runtimes. possibly move tests to periodic by marking them as [Slow]. /assign /kind design /area test (?)
/remove-kind bug
Are pull jobs the right place to be calculating the time? Or should be be counting in CI instead?
pull is the one timing out, so probably?
Looking at some of the top slowness:: subPath should fail for new directories when readOnly specified in the volumeSource: This test case is no longer defined in master HostPathType: This test case has not been checked in yet and is trying to be in a PR PersistentVolumes-local StatefulSet with pod affinity should use volumes spread across nodes when pod has anti-affinity: This test is already marked as slow in master
Created This should address most of the top slow storage tests


Question: obj := event.Object.(*unstructured.Unstructured) return w.checkCondition(obj) } <ins> func extendErrWaitTimeout(err error, info *resource.Info) error { return fmt.Errorf("%s on %s/%s", err.Error(), info.Mapping.Resource.Resource, info.Name) } </ins>

<!-- Please only use this template for submitting enhancement requests --What would you like to be : let's say I run If the deployment is comes up in time it tells me which deployment is up, but if it times out it doesn't tell me on which deployment/resource. Could it be Why is this needed: That would make it easier to use, and the information provided is the same irregardless on whether the condition is met or not
/sig cli
Sounds reasonable for me, i can take care of this one if someone from owners will approve it as a feature.
/assign Zyqsempai
GitHub didn't allow me to assign the following users: Zyqsempai. Note that only and repo collaborators can be assigned and that issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : /assign Zyqsempai Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: # Detect instance group name generated by gke. # <del> # Note that this will only select instance groups in the same zone as the # cluster, meaning that it won't include all groups in a multi-zone cluster. </del> <ins> # Note that the NODE_INSTANCE_GROUPS var will only have instance groups in the # same zone as the cluster, meaning that it won't include all groups in a # multi-zone cluster. The ALL_INSTANCE_GROUP_URLS will contain all the # instance group URLs, which include multi-zone groups. </ins> # # Assumed vars: #   GCLOUD

The test job was passing consistently until midday on 6/18, e.g. Then it started failing consistently, e.g: ... with this error: 12:31:41 ... in gke:detect-node-instance-groups() 12:31:42 ERROR: (gcloud.compute.instance-) Some requests did not succeed: 12:31:42 - The resource 'projects/k8s-jkns-e2e-gke-multizone/zones/us-central1-f/instanceGroupManagers/gke-jenkins-e2e-default-pool--grp' was not found 12:31:42 Assigning to and to triage and assign someone to fix.
cc:
Almost certainly caused by . I'm not going to bother rushing in a rollback today since it's not a critical build, but could you fix this up tomorrow
Yeah, that's me. The change to support multiple MIGs in a single zone doesn't quite work. Didn't know of this test variation. Just to confirm, to run this test variation, the additional env vars I should set are (aside from the standard ones): export KUBERNETESPROVIDER=gke export ADDITIONALZONES=us-central1-a,us-central1-b Any other?
-- yet another instance of a PR passing all presubmit tests and breaking other jenkins jobs after being merged.
Can you tell me a bit more how this test is setup? I tried to do a "kube-" with the additional zones and it worked for me. My settings: export KUBERNETESPROVIDER=gke export ZONE=us-central1-f export CLOUDSDKBUCKET=gs://cloud-sdk-testing/ci/staging export CLOUDSDKAPIENDPOINTOVERRIDESCONTAINER=https://test- export ADDITIONAL_ZONES=us-central1-a,us-central1-b
Have you read ?
Oh, sorry, wrong link.
Take a look at and let me know if you need any more info.
The above PR was the one for GCE. The following is the one did for GKE. There are some relatively minor differences:
Hum, I'll have to dig a bit deeper. I tried the exact combination of env vars and my "kube-up" was successful (with the correct number of nodes across the zones).

Curious...
Any ETA, I'm getting tired of all the test failure emails :p
I think I found the problem. I have a workaround that'll make it work for now. I'm hoping sometime this afternoon.
The PR merged and gke-multizone Jenkins has gone green. Sorry for the trouble. There's still a larger issue of multi-zone handling in the test runner itself, but I think that's for another day.


Question: wantPodStatus: v1.PodStatus{ Phase:   v1.PodFailed, Reason:  "Evicted", <del> Message: "The node was low on resource: ephemeral-storage. ", </del> <ins> Message: "The node was low on resource: ephemeral-storage. Threshold quantity: 2Gi, available: 1536Mi. ", </ins> }, }, }

when the pod is evicted due to low imagefs.available, the prompt is difficult to understand. it showed: But the container was evicted not by exceeding its request of 0 ephemeral-storage, but due to reaching the evictionHard of the node. The correct prompt message points out the reason for the pod eviction. don't print "which exceeds its request of 0.", because the container was evicted not by exceeding its request of 0. % oc label mcp worker custom-kubelet-eviction=small-pods evictionHard by creating a kubeletconfig % oc create -f custom-kubelet- custom-kubelet- several minutes, the mcp worker finish rolling out to the latest mc a pod % oc create -f hello-pod- hello-pod- a large file in container, make imagefs.available < 90Gi (the current available disk size is 111G ) command terminated with exit code 137 the node status the pod status % oc get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES hello-pod-2 0/1 ContainerStatusUnknown 1 7m58s 10.131.0.7 ip-10-0-213- <none<none% oc get pod hello-pod-2 -o yaml No response <details</details<detailsaws </details<details</details<details</details<details</details<details</details
/sig node
/assign
/assign
/triage accepted


Question: By("deleting host1ROPod") framework.ExpectNoError(podClient.Delete(host1ROPod.Name, &api.DeleteOptions{}), "Failed to delete host1ROPod") <ins> By("Test completed successfully, waiting for PD to safely detach") waitForPDDetach(diskName, host0Name) waitForPDDetach(diskName, host1Name) </ins> }) It("should schedule a pod w/ a RW PD shared between multiple containers, write to PD, delete pod, verify contents, and repeat in rapid succession [Slow]", func() {

PD E2E tests have very aggressive clean up routine that does not wait for volumes to safely detached. After the test completes, the test cleanup routine unilaterally detaches the PD (instead of waiting for k8s to do it cleanly). Although this does not cause the test to fail, it results in errors in the kernel logs: We should modify these tests to wait for k8s to safely detach the disk.


Question: apimachineryvalidation "k8s.io/apimachinery/pkg/api/validation" metav1validation "k8s.io/apimachinery/pkg/apis/meta/v1/validation" "k8s.io/apimachinery/pkg/util/sets" <del> "k8s.io/apimachinery/pkg/util/validation" </del> "k8s.io/apimachinery/pkg/util/validation/field" api "k8s.io/kubernetes/pkg/apis/core" "k8s.io/kubernetes/pkg/apis/core/helper"

After got merged, we have new helper function here: It will be good for us to use this simpler helper function to replace previous function. /good-first-issue
This request has been marked as suitable for new contributors. Please ensure the request meets the requirements listed . If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : After got merged, we have new helper function here: </details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig usability
I would like to take this one /assign
I worked in on this as well, but kept your changed files untouched (as written in my PR description). same for you. I hope this is OK for you.
This should be reopened until and got merged (my PR excluded the changes from the other PRs).
/reopen per last comment
Reopened this issue. <detailsIn response to : This should be reopened until and got merged (my PR excluded the changes from the other PRs). per last comment Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
can you assign me
/assign
/assign
I'd love to work on this issue.

/assign
Hi, , I am a first commitor for kubernetes and I have committed a PR in , could you help me to review it, or give me some suggestions. Thanks a lot.
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle rotten [1]:
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Reopen this issue or PR with Mark this issue or PR as fresh with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /close [1]:
Closing this issue. <detailsIn response to : The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closedReopen this issue or PR with Mark this issue or PR as fresh with Offer to help out with [Issue Triage][1] Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: // wait a tick and then mark that we're finished (otherwise, we have no // way to indicate that we're finished, because the function decides not to do anything) time.Sleep(1 * time.Second) <ins> tc.Lock() </ins> tc.statusUpdated = true <ins> tc.Unlock() </ins> tc.processed <- "test-hpa" }()

What happened: The unit test for Horizontal Pod Autoscaler frequently deadlocked on the OpenShift CI system. What you expected to happen: The unit test should be robust and succeed without having to win a race condition to avoid deadlock. How to reproduce it (as minimally and precisely as possible): I'm not sure. Anything else we need to know?: Here are some logs of the problem happening: https://openshift-gce- The deadlocks seem to happen when the chan fills up while waiting for events. If that happens, there's a race where immediately after an event is read but before it can acquire the lock to verify the test, 2 event reactors are able to acquire the lock and write to the chan. The second will block forever since the chan is full and there's nobody to read from it. I've opened a PR with the fix. See
/sig autoscaling
/assign


Question: // See https://kubernetes.io/docs/reference/command-line-tools-reference/kube-proxy/ or https://godoc.org/k8s.io/kube-proxy/config/v1alpha1#KubeProxyConfiguration // for kube proxy official documentation. // <del> //    apiVersion: kubelet.config.k8s.io/v1beta2 </del> <ins> //    apiVersion: kubelet.config.k8s.io/v1beta1 </ins> //    kind: KubeletConfiguration //       ... // // The KubeletConfiguration type should be used to change the configurations that will be passed to all kubelet instances // deployed in the cluster. If this object is not provided or provided only partially, kubeadm applies defaults. // <del> // See https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/ or https://godoc.org/k8s.io/kubelet/config/v1beta2#KubeletConfiguration </del> <ins> // See https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/ or https://godoc.org/k8s.io/kubelet/config/v1beta1#KubeletConfiguration </ins> // for kubelet official documentation. // // Here is a fully populated example of a single YAML file containing multiple

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: references and a broken link to this (because it doesn't exist!) I'm guessing this was a bad find and replace when copying kubeadm's own v1beta1 to v1beta2 What you expected to happen: accurate documentation How to reproduce it (as minimally and precisely as possible): read the docs Anything else we need to know?: this template has a lot of fields :^) Environment: anything Kubernetes version (use ): n/a Cloud provider or hardware configuration: n/a OS (e.g: ): n/a Kernel (e.g. ): n/a Install tools: kubeadm Network plugin and version (if this is a network-related bug): n/a Others: n/a /assign /sig cluster-lifecycle


Question: old    networking.NetworkPolicy update networking.NetworkPolicy } <del> successCases := []npUpdateTest{ { </del> <ins> successCases := map[string]npUpdateTest{ "no change": { </ins> old: networking.NetworkPolicy{ ObjectMeta: metav1.ObjectMeta{Name: "foo", Namespace: "bar"}, Spec: networking.NetworkPolicySpec{

I tried to update the NetworkPolicy Spec field with more ingress rules but got It seems NetworkPolicy does not allow update on Spec This means if I want to have more ingress rules, I will have to add those in a new networkpolicy in the same namespace. Any reason why I should not add those in the existing one(update the existing networkpolicy)? Thanks!
This is as designed for now. The expectation is that updates to the spec will be allowed when the NetworkPolicy API moves from v1beta1 to GA.
(And that is planned for Kubernetes 1.6 release.)
good to know, thanks!
Note that the PR to move NetworkPolicy to v1 did not make it into v1.6, but is now planned for 1.7:
Oh, I didn't know about this bug and that didn't get changed in but we can do it as another PR after. (I'm trying to not make any more changes there so we can get it merged.)


Question: type TypeMeta struct { Kind              string    `json:"kind,omitempty" description:"kind of object, in CamelCase"` ID                string    `json:"id,omitempty" description:"name of the object; must be a DNS_SUBDOMAIN and unique among all objects of the same kind within the same namespace; used in resource URLs"` <del> UID               types.UID `json:"uid,omitempty" description:"UUID assigned by the system upon creation, unique across space and time"` CreationTimestamp util.Time `json:"creationTimestamp,omitempty" description:"RFC 3339 date and time at which the object was created; recorded by the system; null for lists"` SelfLink          string    `json:"selfLink,omitempty" description:"URL for the object"` ResourceVersion   uint64    `json:"resourceVersion,omitempty" description:"string that identifies the internal version of this object that can be used by clients to determine when objects have changed; value must be treated as opaque by clients and passed unmodified back to the server"` </del> <ins> UID               types.UID `json:"uid,omitempty" description:"unique UUID across space and time; populated by the system, read-only"` CreationTimestamp util.Time `json:"creationTimestamp,omitempty" description:"RFC 3339 date and time at which the object was created; populated by the system, read-only; null for lists"` SelfLink          string    `json:"selfLink,omitempty" description:"URL for the object; populated by the system, read-only"` ResourceVersion   uint64    `json:"resourceVersion,omitempty" description:"string that identifies the internal version of this object that can be used by clients to determine when objects have changed; populated by the system, read-only; value must be treated as opaque by clients and passed unmodified back to the server"` </ins> APIVersion        string    `json:"apiVersion,omitempty" description:"version of the schema the object should have"` Namespace         string    `json:"namespace,omitempty" description:"namespace to which the object belongs; must be a DNS_SUBDOMAIN; 'default' by default"`

Right now we have haphazard comments regarding which fields are read-only/output-only, and many, such as aren't documented at all. Swagger 2.0 actually supports a property tag, but we're using Swagger 1.2, which does not, officially. The official meaning doesn't match what we want, though. fields must not be included in a PUT, which would be painful for clients. We could simply document our convention, however. We should tag fields as readonly. Maybe we could have go-restful automatically add a comment to the description in that case.
/cc
Are there any known obstacles with upgrading to 2.0?
go-restful (the framework that we are using to generate swagger-spec) does not support 2.0 yet:
Disclaimer: for the doc-fix it day, I will be adding the read-only information into the description. This partially resolves this issue, but not entirely. I will unassign myself after description is done.
Note: users can set the node status when creating a node. This is a bug and should be fixed. After fixing that bug, the corresponding description should be updated.