Question: // ValidateAnnotations validates that a set of annotations are correctly defined. func ValidateAnnotations(annotations map[string]string, fldPath *field.Path) field.ErrorList { allErrs := field.ErrorList{} <del> var totalSize int64 for k, v := range annotations { </del> <ins> for k := range annotations { </ins> for _, msg := range validation.IsQualifiedName(strings.ToLower(k)) { allErrs = append(allErrs, field.Invalid(fldPath, k, msg)) } <del> totalSize += (int64)(len(k)) + (int64)(len(v)) </del> } <del> if totalSize > (int64)(totalAnnotationSizeLimitB) { allErrs = append(allErrs, field.TooLong(fldPath, "", totalAnnotationSizeLimitB)) </del> <ins> if err := ValidateAnnotationsSize(annotations); err != nil { allErrs = append(allErrs, field.TooLong(fldPath, "", TotalAnnotationSizeLimitB)) </ins> } return allErrs } <ins> func ValidateAnnotationsSize(annotations map[string]string) error { var totalSize int64 for k, v := range annotations { totalSize += (int64)(len(k)) + (int64)(len(v)) } if totalSize > (int64)(TotalAnnotationSizeLimitB) { return fmt.Errorf("annotations size %d is larger than limit %d", totalSize, TotalAnnotationSizeLimitB) } return nil } </ins> func validateOwnerReference(ownerReference metav1.OwnerReference, fldPath *field.Path) field.ErrorList { allErrs := field.ErrorList{} gvk := schema.FromAPIVersionAndKind(ownerReference.APIVersion, ownerReference.Kind)

Expose totalAnnotationSizeLimitB in the validation package: This value is useful for validating annotations outside of the validation package, such as in SSA: Otherwise, size validation on annotations outside of the package would diverge and cause issues later. /kind cleanup /wg api-expression cc
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign


Question: "k8s.io/kubernetes/pkg/kubelet/cm" containertest "k8s.io/kubernetes/pkg/kubelet/container/testing" "k8s.io/kubernetes/pkg/kubelet/dockertools" <del> kubetypes "k8s.io/kubernetes/pkg/kubelet/types" </del> "k8s.io/kubernetes/pkg/labels" "k8s.io/kubernetes/pkg/master" "k8s.io/kubernetes/pkg/runtime"

The integration test is hard to debug -- it runs tests concurrently with interspersed logs. I doubt that it's catching any bug that neither unit test nor a e2e test can catch, and both unit and e2e tests are easier to debug. I'd vote for deprecating this file and remove/move all the tests. Any objections? Here is a list of tests that it curretnly runs:
Yeah the rc and services tests are redundant, I think we exercise them enough in e2es. Don't know about: It quicky creates/deletes pods with the same hostport that get scheduled onto a single node. Don't see a reason why we can't move it to an e2e, but I don't think it's been done yet.
since you are the original author of , do you have any suggestions/preferences about this test?
That test is important, it has caught regressions two or three times. It should be movable to the other integration test section. , Yu-Ju Hong wrote:
looks like you're working on this already. Feel free to unassign if that's incorrect.
I uploaded a PR to remove rc and service tests based on I'll let each team decide whether they want to migrate or delete the .
maybe someone with spare time can rewrite that scheduler test as an e2e, and we can kill integration? I continue to see integration flake that's hard to debug, current SQ closing failure: Why did that fail? Ctrl+f "fail" gives me 700 matches, scrolling through the logs i see all sorts of nonsense. I'm going to guess that scheduler test wasn't able to put pods on nodes because of the kubeletNotReady condition, but that's just a guess.
Those failures even existed in passing integration test as well (I can feel your pain). This is due to . It would be great if original author of those tests can clarify what each test does and maintain it. Otw those tests are just legacy obstacles that we should get rid of.
the rc and services tests have been removed. The only ones remaining are put/patch/apiserver tests and the scheduler phantom pod tests IIRC. The former is relatively straightforward and only needs the apiserver (segmented failure mode), the latter would be much easier to maintain as an e2e. Bandwidth issue.
I see.

I can definitely help with it. Let me spend some time figuring them out. Meanwhile, anyone familiar with the scheduler tests should feel free to open another issue and track them.
What's your opinion on this?
I agree that integration either needs to be killed or massively overhauled
This issue describes the circumstances leading up to that scheduler integration test, for anyone attempting to port it: Though IIRC we got rid of the modeler, so things might be different now.
I think we actually have a spike in integration test failures:
unfortunately the failure you linked was not from this file () Those tests were in , and were considered easier to debug (at least from doc last time). Personally I don't have much experiences debugging this new category of integration tests, so I can't say much. They do seem to flake a lot recently in per-PR tests as well.
I concur on this regarding the "fail" message seen on scheduler test too. Re. "SchedulerNoPhantomPodsTest", it assumes two kubelets with predefined resources, which is easier to maintain in simulated environment. Right? If testing in is to hard to debug, we could make it a lightweight unit test. IIRC, it's testing one specific race condition that existed before in scheduler.
oh, hmm, different bug then Re shceduler test: If you can manage with a unittest that would be great. If not I'm fine running the integration tests serially. I don't think it'll take that much longer, and flakes that only show up in parallel runs are hopefully covered in gce-e2e.


Question: <del> 2.30 </del> <ins> 2.31 </ins>

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --e2e test fails with ECDSA Service Account key e2e test to succeed with ECDSA Service Account key Use ECDSA key for the service account key. Logs from the started test container: library defaults to by default as the only supported method while should be explicitly: Relevant code: On the API server side: Kubernetes version (use ): 1.21.0-rc.0 Cloud provider or hardware configuration: OS (e.g: ): Talos Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig testing
Changing the service account key type to RSA fixes the test:
this is a problem for us, as this test is part of test suite, and it fails with 1.21-rc.0 e2e tests
SIG Auth own their test cases (it's tagged in the test name), SIG Architecture own conformance. Testing owns the frameworks, CI, tooling, etc. EDIT: just clarifying and making sure the right people see it. I raised this to the conformance work group.
/assign /priority critical-urgent /milestone v1.21 because this is in the conformance suite, it must not restrict valid+supported cluster configurations
/reopen the image needs to be promoted and bumped in the e2e test
Reopened this issue. <detailsIn response to : /reopen the image needs to be promoted and bumped in the e2e test Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
promote image PR:
I believe with merged this can be closed as all the fixes were done and backported to 1.21
Thanks!


Question: } func (swg *sampleAndWaterMarkObserverGenerator) quantize(when time.Time) int64 { <del> return when.UnixNano() / int64(swg.samplePeriod) </del> <ins> return int64(when.Sub(swg.t0) / swg.samplePeriod) </ins> } // Generate makes a new TimedObserver

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: The apiserver is always restarting and I can't access it with any commands looks like . When I check the kubectl's status, the tells me that: And then I checked the container's status and log of the apiserver: status: log: I see that there is a line: And then I get the source code: What you expected to happen: How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): v1.19.3. But I can't use the now because of the problem, so I can't give you a formal result about the version. Cloud provider or hardware configuration: Ryzen 5 4500U 16GB memory 30GB SSD In the VMware Workstation 15.5.6 OS (e.g: ):Kernel (e.g. ):Install tools: Network plugin and version (if this is a network-related bug): Others:
The complete version for the log of apiserver:
/cc
: did you tweak that machine's clock backwards, or run any software that does this?
Time calculations should be written to use go's , not count on wall clock time always moving foward.
: The proximal problem is that uses and gets the answer that time went backwards.
sure, but should be immune to wall clock resets per
Exactly. It looks to me like this test for ordering is using the monotonic clock and somehow that is saying time went backwards.
does the quantization () factor into well-ordered detection? I'm not sure how the monotonic clock interacts with Unix / UnixNano
No, I see the quantization is buggy, it uses wall clock. But the ordering test does not depend on the quantization.
/triage accepted /priority urgent-critical
The label(s) cannot be applied, because the repository doesn't have them <detailsIn response to : /triage accepted /priority urgent-critical </details
very odd that this is repeatable. /priority critical-urgent
: it is very odd that this is repeatable. Do you have anything unusual in your environment ?

I see it is vmware workstation. Does that mess with the clock in any unusual way? I also wonder about the golang runtime; did you do your own build (if so, what version of golang) or fetch binaries from somewhere (where?); full output would be helpful.
I don't have anything unusual in my environment. I run it in the ubuntu driven by vmware workstation, because my real machine's operation system is Windows10 , and I need a virtual machine to run k8s. My computer is a laptop computer. I got the golang runtime by apt-get. I also got the distribution of k8s by apt-get too. I did not do my own build. ! Okay, I got my golang and k8s version now: !
I have exactly same problem in my local lab. Virt infra on Virtual Box 6.1.16 Guest OS
you try and, if so, does it fix your problem and does it log complaints about the monotic clock going backwards? you tell us about your host and the settings on your VM(s)?
exactly matches same error as posted by author of issue, I have NOT tried VM settings: 4 vCPUs 4GB RAM swap is OFF on OS level. Host HW: Core i9 9900K 32GB Ram BTW: I have managed to install 1.18.12 on same VM eventually - cause for me it is just lab infra to study, so 1.19.X is NOT that important for me to have on this infra.
: host OS? Is your VB "Version 6.1.16 r140961 (Qt5.6.3)"? VM settings match the following? <img width="616" alt="Screen Shot 2020-11-16 at 4 56 10 PM" src="https://user-"<img width="742" alt="Screen Shot 2020-11-16 at 4 55 58 PM" src="https://user-"<img width="720" alt="Screen Shot 2020-11-16 at 4 55 45 PM" src="https://user-"
OS Name: Microsoft Windows 10 Enterprise OS Version: 10.0.

N/A Build OS Manufacturer: Microsoft Corporation OS Configuration: Standalone Workstation OS Build Type: Multiprocessor Free ! ! ! !
I tried to reproduce the problem using Ubuntu 18.0.4.5 and Kubernetes 1.19.4 (installed by kubeadm, using Calico as the network plugin), but the problem did not strike. The guest was created from the Ubuntu live Server ISO with a very plain install, picking only OpenSSH from the things to add during the install. No explicit tweaks regarding timekeeping. In the guest: I am also running the exact same version of VBox as reported above. I have the ExtensionPack of the same version installed. My host is Windows 10 Pro version 2004. Intel Core i9-9900 (8 hyperthreaded cores, thus 16 logical processors), 32 GB RAM. Configured for automatic time sync. Last sync 34 minutes ago, with server
Same issue in my env: VM (CentOS 7.9.2009 4 CPU+12G Memory) on virtualbox(6.1) Host is Windows 10 (16G Memory, CPU Intel(R) Core(TM) i3- CPU @ 3.60GHz 3.60 GHz) docker version: docker-ce-19.03.9-3.el7.x8664 K8S version: 1.19.4 Part error log output from docker logs -f <kube-apiserver-containerid:
Hi, same here vitualbox 6.1.18 latest Linux master-1 5.4.0-72-generic latest /etc/os-release NAME="Ubuntu" VERSION="20.04.2 LTS (Focal Fossa)" ID=ubuntu IDLIKE=debian PRETTYNAME="Ubuntu 20.04.2 LTS" VERSIONID="20.04" HOMEURL="" SUPPORTURL="" BUGREPORTURL="" PRIVACYPOLICYURL="" VERSIONCODENAME=focal UBUNTU_CODENAME=focal tried with K8s 1st with 1.21.0-00 then 1.19.10-00 same Time went backwards message in the logs. Api server uses large amount of CPU at doing nothing. etcd cluster seems also to be totally lost. 3 masters setup.

thanks
Hm ... this must have something to do with the fact we're running in a VM. Funny enough ... you guys are reporting apiserver crashing. What led me to this issue is that my metrics-server crashes with this same log message instead :). I originally had ntpd installed on the VM, but removing it did not help. So ... if time goes backward, it must be something that VirtualBox does. I'll try to see if I can repro with HyperV. EDIT: I was able to get rid of this error with VirtualBox by using to disable some kind of automated clock sync that VirtualBox does by default. Don't forget to use NTPd, as you will need some time synchronization :). EDIT 2: Spoke too soon. That did not really help. It seems to have slightly delayed the issue, though, as it only presented itself after a couple of minutes... EDIT 3: So, the reason why it's still syncing time, is VBoxGuestAdditions. When they are running, the guest synces time with the host, even with the above time-sync setting disabled. One either has to stop the guest additions or pass them (but ... it's not really a configuration change: ). EDIT 4: The above doesn't seem to have an effect at all. Still, according to the metrics server, time goes backwards. Well ... let's try to completely disable the VirtualBox guest additions: (for my use case, this is totally fine, as I don't need any of the guest additions features such as shared folders or time sync). EDIT 5: Nothing helps. I reckon, either the check for "time went backwards" is somewhat buggy, or some odd time sync still takes place. For now, because I don't really need it, I am just disabling the metrics server. EDIT 6: Huh. Got it, finally. Never surrender, I guess :). Turns out, you gotta attack it from all sides. (see above) Add the argument to (see above) (this is new; turns out, even if one tells guest additions not to sync time, and disables the time syncing on the VM, something can still sync the time -- this setting makes the guest OS completely blind to the fact it is running in a VM).

I suggest to keep ntp and guest additions (with the disabled time sync) on at all times. EDIT 7: Also ... do no let your host sleep. When it sleeps and then wakes up, a lot of k8s cannot cope with it and begins rolling. It should recover, but if you can avoid sleeping, do. For me, this actually meant installing and using "Don't sleep" (alternative to "Caffeine" or similar) to keep it awake. For some reason, when you lock your computer, Windows will try to sleep. It should be possible to disable this mechanic (), but even though I did, it would still sleep. That's why I ended up with "Don't sleep". With all of this in place, it finally works as expected. None of the k8s' pods roll anymore.
I had exactly the same issue. Followed the steps listed above and it solved my issue. Thanks , much appreciated!


Question: } var rtStatus *metav1.Status var pdbName string <del> err = retry.RetryOnConflict(EvictionsRetry, func() error { </del> <ins> err = func() error { </ins> pdbs, err := r.getPodDisruptionBudgets(ctx, pod) if err != nil { return err

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: We have about 8000 pdbs in our cluster. When we call several eviction requests at the same time, etcd will meet a performance spike. What you expected to happen: eviction request read pdbs from apiserver cache How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig api-machinery
/sig scalability /cc throttling evictions at server-side may be a better fix i presume
We use eviction to auto migrate pods. If throttling eviction requests, it will make migration latency too long. I am not sure if list without rv by designed (get pdb consistently) or not (just forget to add rv).
What does eviction do (I can't remember from the top of my head)? Would you be able to quickly point me to the code?
Create create a eviction will list all pdbs, make sure eviction the pod won't break any pdb. If check passed, it will create the eviction, otherwise it will return 429 to reject.
Thanks - so this is what is problematic: So what you're saying is that you have 8000 PDBs in a single namespace? [because we only list pdbs from pod`s namespace]. I'm we should list from etcd, because otherwise we risk using old version of pdb ( are allowed to evict only 1 pod, and we just did it, but that wasn't yet propagated to apiserver cache and we would evict another pod). My personal guess would be that the problem may not really be number of PDB, but amount of concurrent requests touching the same pdb - because that would be resulting in conflicts and retrying them in: I'm not sure we can do much about it (at least I don't see any immediate low-hanging fruit).
Thanks a lot for your reply. How about retrying conflict update pdb, instead of re-listing all pdbs?

How about retrying conflict update pdb, instead of re-listing all pdbs? Hmm - I'm not sure I understand. Ohh, but I know what else we can do (maybe that's what you had on your mind). In case of conflict, instead of relisting all PDBs, we should simply GET the current version of that PDB (we already know that there is exactly one which we are supposed to use). That would reduce the cost of retries significantly. - would you be able to take a look into it and prepare a PR? I'm happy to review it.
- That's what I mean. I will take a look and send a PR in a few days.
- great! Please ping me on slack if I won't be responding to it.
alternatively, will pagination probably help ease the pressure? it'll skip the cache tho but relaxes the "spike"..
Does pagination mean range key prefix with limit?


Question: "context" "fmt" "sync/atomic" <ins> "time" </ins> v1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/runtime"

This is a bug found in The scheduler fails to schedule the pod on the first try due to PV is out of sync. This may happen because the PV/PVC may be modified before the scheduler binder assumes them. However, the scheduler skips the pod update event and never try to scheduler the pod again: This is because when the reserve plugins fails, we didn't try to forget the pod. After the is called, the pod is in the state.
/sig scheduling /sig storage /assign /priority critical-urgent
The label(s) cannot be applied, because the repository doesn't have them <detailsIn response to : /sig scheduling /sig storage /assign /priority critical-urgent Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: ginkgo.By("Create config map for the node problem detector") _, err = c.CoreV1().ConfigMaps(ns).Create(context.TODO(), &v1.ConfigMap{ ObjectMeta: metav1.ObjectMeta{Name: configName}, <del> Data:       map[string]string{path.Base(configFile): config}, }, metav1.CreateOptions{}) framework.ExpectNoError(err) ginkgo.By("Create the service account for node problem detector") _, err = f.ClientSet.CoreV1().ServiceAccounts(f.Namespace.Name).Create(context.TODO(), &v1.ServiceAccount{ ObjectMeta: metav1.ObjectMeta{ Name:      name, Namespace: ns, }, }, metav1.CreateOptions{}) framework.ExpectNoError(err) ginkgo.By("Bind the cluster role system:node-problem-detector with the service account for node problem detector") _, err = f.ClientSet.RbacV1().ClusterRoleBindings().Create(context.TODO(), &rbacv1.ClusterRoleBinding{ ObjectMeta: metav1.ObjectMeta{ Name: name, }, RoleRef: rbacv1.RoleRef{ APIGroup: rbacv1.GroupName, Kind:     "ClusterRole", Name:     "system:node-problem-detector", }, Subjects: []rbacv1.Subject{ { Kind:      rbacv1.ServiceAccountKind, Name:      name, Namespace: ns, }, </del> <ins> Data: map[string]string{ path.Base(configFile):     config, path.Base(kubeConfigFile): kubeConfig, </ins> }, }, metav1.CreateOptions{}) framework.ExpectNoError(err)

Which jobs are failing: Which test(s) are failing: Since when has it been failing: Last successful run (23 Oct 2020, 10:07 PDT): First unsuccessful run (23 Oct 2020, 12:08 PDT): Testgrid link: Reason for failure: Anything else we need to know: presubmit job in repository also started to fail in the same fashion. Attempted to debug it myself, found out the following stuff: last successful run was accessing the apiserver via HTTP: all unsuccessful runs attempted to access the apiserver via HTTPS but to no avail: perused the history of to find out that the PR is almost certainly the reason of the failures (or rather uncovered some problem with the NPD or its tests): first, it coincides with the start of the failures of , second, we changed the default hostname of the apiserver in node tests in that PR. Judging by the code and the changes in the aforementioned PR I think that a quick mitigation would be to add a flag to the of these jobs. /assign /sig node /cc
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Opened for a mitigation that would make those jobs stop failing.
This won't work, unfortunately. Copying response:
I have filed to address this issue, but it seems I could not run the test in advance to verify my patch, did you know how to manually trigger the presubmit job?
Unfortunately I don't :/. My PR was also a wild guess. could you take a look at the proposed fix? Thanks once again for your help - really appreciated :).
the submission of hasn't fixed the problems, unfortunately :/. Sample failing run: We still have some peculiar problem related to health checking: and now NPD panics, too:
Thanks for your report, I believe the reason is admission controller has been disabled in node e2e tests: We need to figure out a new approach now. /reopen
Reopened this issue.

<detailsIn response to : Thanks for your report, I believe the reason is admission controller has been disabled in node e2e tests: We need to figure out a new approach now. Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
filed . I hope we could completely address this issue this time.
/milestone v1.20


Question: obj.HairpinMode = PromiscuousBridge } if obj.EvictionHard == nil { <del> temp := "memory.available<100Mi,nodefs.available<10%,nodefs.inodesFree<5%" </del> <ins> temp := "memory.available<100Mi,nodefs.available<10%,nodefs.inodesFree<5%,imagefs.available<15%" </ins> obj.EvictionHard = &temp } if obj.EvictionPressureTransitionPeriod == zeroDuration {

What happened: if a node is configured with separate imagefs for container storage, eviction thresholds should align with image gc high threshold. image-gc high threshold is 85% and imagefs.available eviction should be < 15% to match.


Question: ConsistentListFromCache: {Default: false, PreRelease: featuregate.Alpha}, <del> ZeroLimitedNominalConcurrencyShares: {Default: false, PreRelease: featuregate.Beta}, </del> <ins> ZeroLimitedNominalConcurrencyShares: {Default: true, PreRelease: featuregate.GA, LockToDefault: true}, // remove in 1.32 </ins> }

In 1.30, Priority and Fairness should allow a zero value, either via v1 or v1beta3 (with the roundtrip annotation) in create and update requests See:
/sig api-machinery /cc /assign
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : /triage accpted Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/triage accepted


Question: } func TestEnsureLoadBalancerCreatesExternalLb(t *testing.T) { <del> t.Parallel() </del> vals := DefaultTestClusterValues() gce, err := fakeGCECloud(vals) require.NoError(t, err)

Since merged on Friday night, my PR () has failed several retests of pull-kubernetes-bazel-test. My PR is unrelated to the load balancer so I suspect that introduced a flake. I see several runs of that test against master that aren't failing, so clearly the test works sometimes (and the original PR passed it several times to before it merged). Perhaps the test harness sometimes runs problem tests in parallel and other times it doesn't. Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: My PR had one good run of pull-kubernetes-bazel-test before merged, but has failed several times since with no code changes to my PR. What you expected to happen: The tests should consistently pass unless related problem code is introduced.
/cc /cc /sig gcp /kind flake
My PR started failing recently too. See https://k8s-
Hey, thanks for identifying the issue. contains tests for - but the race condition was caused when tests with modified a singleton variable in a mock. The short-term fix is removing the calls, but the long-term fix is to not use a singleton. Fix is .


Question: PREFIX = quay.io/fluentd_elasticsearch IMAGE = elasticsearch <del> TAG = v7.1.1 </del> <ins> TAG = v7.2.0 </ins> build: <del> docker build  --tag ${PREFIX}/${IMAGE}:${TAG} . </del> <ins> docker build --tag ${PREFIX}/${IMAGE}:${TAG} . docker build --tag ${PREFIX}/${IMAGE}:latest . </ins> push: <del> docker push ${PREFIX}/${IMAGE}:${TAG} </del> <ins> docker push ${PREFIX}/${IMAGE}:${TAG} </ins> docker push ${PREFIX}/${IMAGE}:latest

What happened: On latest fluentd-elasticsearch, the elasticsearch replicas seems failed to form a valid cluster with the service discovery helper written in golang. What you expected to happen: Successfully boot up an elasticsearch cluster I guess. How to reproduce it (as minimally and precisely as possible): Just use the yamls in and the elasticsearch-logging pods will keep spamming warning messages. I think thats because options are deprecated in elasticsearch 7.x.x Anything else we need to know?: IMHO I think thats all. Environment: Kubernetes version (use ): 1.12.4 Cloud provider or hardware configuration: AKS OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig instrumentation
Thanks


Question: return &nfsMounter{ nfs: &nfs{ <del> volName: spec.Name(), mounter: mounter, pod:     pod, plugin:  plugin, </del> <ins> volName:         spec.Name(), mounter:         mounter, pod:             pod, plugin:          plugin, MetricsProvider: volume.NewMetricsStatFS(getPath(pod.UID, spec.Name(), plugin.host)), </ins> }, server:       source.Server, exportPath:   source.Path,

/kind feature What happened: on a cluster with an nfs persistent volume present: What you expected to happen: I expected metrics concerning the nfs persistent volume to be returned e.g: (from a separate aws cluster) Anything else we need to know?: I'm assuming that this is just not implemented currently? I couldn't find a referencing issue Environment: Kubernetes version (use ):Cloud provider or hardware configuration: on-prem OS (e.g. from /etc/os-release):Kernel (e.g. ):
/sig instrumentation
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-testing, kubernetes/test-infra and/or . /close
Closing this issue. <detailsIn response to : Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/open Metrics is missing for NFS driver.
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/remove-lifecycle rotten
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/sig storage


/kind bug https://k8s- This is likely caused by a missing REMOUNTVOLUMEPLUGIN_DIR flag in kube-env construction. /assign /sig storage
xref /priority failing-test /priority critical-urgent any update on this one?
/status approved-for-milestone
I'm aware of the solution, but I'm trying to understand how to limit the scope of the change to just COS images, instead of all VM distros.
xref
Fix submitted, awaiting review:
PR approved; awaiting 1.10 milestone marker
/status in-progress
You must be a member of the github team to add status labels.
/status in-progress
[MILESTONENOTIFIER] Milestone Issue Needs Attention Action Required: This issue has not been updated since Mar 11. Please provide an update. Note: This issue is marked as , and must be updated every 1 day during code freeze. Example update: <details<summaryIssue Labels</summary- : Issue will be escalated to these SIGs if needed. : Never automatically move issue out of a release milestone; continually escalate to contributor and SIG through all available channels. : Fixes a bug discovered during the current release. </details<details<summaryHelp</summary<ul<li<a href=""Additional instructions</a</li<li<a href=""Commands for setting labels</a</li</ul</details


Question: return resp, nil } <del> return t.fixLinks(req, resp) </del> <ins> return t.rewriteResponse(req, resp) </ins> } // rewriteURL rewrites a single URL to go through the proxy, if the URL refers

The ApiServer proxy wraps partial HTML responses to serve a complete HTML document in the proxy response. This is problematic for applications which expect HTML fragments, such as with AngularJS templates. Example: This is the response for a from the unproxied server: But the proxied response is: This issue is blocking


Question: Type:         "fake", } } <ins> // copied from go-autorest/adal type fakeTokenRefreshError struct { message string resp    *http.Response } // Error implements the error interface which is part of the TokenRefreshError interface. func (tre fakeTokenRefreshError) Error() string { return tre.message } // Response implements the TokenRefreshError interface, it returns the raw HTTP response from the refresh operation. func (tre fakeTokenRefreshError) Response() *http.Response { return tre.resp } </ins>

<!-- Please only use this template for submitting enhancement requests --What would you like to be : Currently, when AAD login is configured, refresh token timeout would make kubectl print an error, not able to re-login without manually remove the tokens. Expect kubectl to promote user to login again. Why is this needed: Configure refresh-token timeout from Azure is a way to force re-login after inactivity. kubectl doesn't promote re-login makes users harder to apply timeout policy.
/sig azure
The label(s) cannot be appled. These labels are supported: <detailsIn response to : /sig azure Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
The label(s) cannot be appled. These labels are supported: <detailsIn response to : /sig platform/azure Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/area provider/azure
the root cause of this issue is in this first gets the cached token (result of device code flow) at it's trying to refresh an expired token at and failed. An error is returned. Since the expired token is still in the cache, it will always hit this path until user removes the token from kubeconfig. the fix should be: the token from cache a retry to re-initiate the device code flow in
The label(s) cannot be appled. These labels are supported: <detailsIn response to : /sig platform/azure Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig cli These SIGs are my best guesses for this issue. Please comment if I am incorrect about one. I am a bot run by vllry.
I'm working on it.
are you working on it? I will grab it if I dont hear it from you :)
have the code ready will try PR this weekend Best, Ping
/assign
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
Closing as merged


Question: package options import ( <ins> "fmt" </ins> "github.com/spf13/pflag" jobconfig "k8s.io/kubernetes/pkg/controller/job/config"

What would you like to be : The ability to set for the kube-controller-manager at runtime. It looks like the option exists in the code, but a mechanism is not available to set it via kops. If I understand the code correctly, a flag needs to be in order to be able to set the value using kops. Why is this needed: We want to shift a lot of our batch processing jobs from running directly on our CI servers to running as jobs on Kubernetes. Unfortunately if too many jobs are launched on kubernetes, they begin to back up. We've noticed up to a 10 minute delay between when the logs indicate that a job has finished and when the kubernetes API actually reflects the same on our small ~30 node cluster.
/sig scalability
I think scalability is the best fit since this is impacting our ability to fully utilize and scale how we use our kubernetes cluster.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
/sig api-machinery
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-testing, kubernetes/test-infra and/or . /close
Closing this issue. <detailsIn response to : Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with .

/close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Heyo -- we're running into a similar issue as this with our Kubernetes cluster. Is there anything I can do to help get landed?
You can't reopen an issue/PR unless you authored it or you are a collaborator. <detailsIn response to : Heyo -- we're running into a similar issue as this with our Kubernetes cluster. Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign /remove-sig api-machinery /sig apps
/triage accepted /remove-lifecycle rotten
can you please describe your use case in a more detail? how How this affects you, and how this flag would help to achieve your goals?
-- I left a comment describing my use case in the PR:


Question: } } <ins> func TestLegacyFallbackNoCache(t *testing.T) { aggregatedResourceManager := discoveryendpoint.NewResourceManager() rootAPIsHandler := discovery.NewRootAPIsHandler(discovery.DefaultAddresses{DefaultAddress: "192.168.1.1"}, scheme.Codecs) legacyGroupHandler := discovery.NewAPIGroupHandler(scheme.Codecs, metav1.APIGroup{ Name: "stable.example.com", PreferredVersion: metav1.GroupVersionForDiscovery{ GroupVersion: "stable.example.com/v1", Version:      "v1", }, Versions: []metav1.GroupVersionForDiscovery{ { GroupVersion: "stable.example.com/v1", Version:      "v1", }, { GroupVersion: "stable.example.com/v1beta1", Version:      "v1beta1", }, { GroupVersion: "stable.example.com/v1alpha1", Version:      "v1alpha1", }, }, }) generateVersionResource := func(version string) metav1.APIResource { return metav1.APIResource{ Name:         "foos", SingularName: "foo", Group:        "stable.example.com", Version:      version, Namespaced:   false, Kind:         "Foo", Verbs:        []string{"get", "list", "watch", "create", "update", "patch", "delete", "deletecollection"}, Categories:   []string{"all"}, } } resources := map[string]metav1.APIResource{ "v1":       generateVersionResource("v1"), "v1beta1":  generateVersionResource("v1beta1"), "v1alpha1": generateVersionResource("v1alpha1"), } legacyResourceHandlerV1 := discovery.NewAPIVersionHandler(scheme.Codecs, schema.GroupVersion{ Group:   "stable.example.com", Version: "v1", }, discovery.APIResourceListerFunc(func() []metav1.APIResource { return []metav1.APIResource{ resources["v1"], } })) legacyResourceHandlerV1Beta1 := discovery.NewAPIVersionHandler(scheme.Codecs, schema.GroupVersion{ Group:   "stable.example.com", Version: "v1beta1", }, discovery.APIResourceListerFunc(func() []metav1.APIResource { return []metav1.APIResource{ resources["v1beta1"], } })) legacyResourceHandlerV1Alpha1 := discovery.NewAPIVersionHandler(scheme.Codecs, schema.GroupVersion{ Group:   "stable.example.com", Version: "v1alpha1", }, discovery.APIResourceListerFunc(func() []metav1.APIResource { return []metav1.APIResource{ resources["v1alpha1"], } })) handlerFunc := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { if r.URL.Path == "/apis/stable.example.com" { legacyGroupHandler.ServeHTTP(w, r) } else if r.URL.Path == "/apis/stable.example.com/v1" { // defer to legacy discovery legacyResourceHandlerV1.ServeHTTP(w, r) } else if r.URL.Path == "/apis/stable.example.com/v1beta1" { // defer to legacy discovery legacyResourceHandlerV1Beta1.ServeHTTP(w, r) } else if r.URL.Path == "/apis/stable.example.com/v1alpha1" { legacyResourceHandlerV1Alpha1.ServeHTTP(w, r) } else if r.URL.Path == "/apis" { rootAPIsHandler.ServeHTTP(w, r) } else { // Unknown url t.Fatalf("unexpected request sent to %v", r.URL.Path) } }) aggregatedManager := newDiscoveryManager(aggregatedResourceManager) aggregatedManager.AddAPIService(&apiregistrationv1.APIService{ ObjectMeta: metav1.ObjectMeta{ Name: "v1.stable.example.com", }, Spec: apiregistrationv1.APIServiceSpec{ Group:   "stable.example.com", Version: "v1", Service: &apiregistrationv1.ServiceReference{ Name: "test-service", }, }, }, handlerFunc) aggregatedManager.AddAPIService(&apiregistrationv1.APIService{ ObjectMeta: metav1.ObjectMeta{ Name: "v1beta1.stable.example.com", }, Spec: apiregistrationv1.APIServiceSpec{ Group:   "stable.example.com", Version: "v1beta1", Service: &apiregistrationv1.ServiceReference{ Name: "test-service", }, }, }, handlerFunc) aggregatedManager.AddAPIService(&apiregistrationv1.APIService{ ObjectMeta: metav1.ObjectMeta{ Name: "v1alpha1.stable.example.com", }, Spec: apiregistrationv1.APIServiceSpec{ Group:   "stable.example.com", Version: "v1alpha1", Service: &apiregistrationv1.ServiceReference{ Name: "test-service", }, }, }, handlerFunc) testCtx, cancel := context.WithCancel(context.Background()) defer cancel() go aggregatedManager.Run(testCtx.Done()) require.True(t, waitForEmptyQueue(testCtx.Done(), aggregatedManager)) // At this point external services have synced. Check if discovery document // includes the legacy resources _, _, doc := fetchPath(aggregatedResourceManager, "") aggregatedVersions := []apidiscoveryv2beta1.APIVersionDiscovery{} for _, resource := range resources { converted, err := endpoints.ConvertGroupVersionIntoToDiscovery([]metav1.APIResource{resource}) require.NoError(t, err) aggregatedVersions = append(aggregatedVersions, apidiscoveryv2beta1.APIVersionDiscovery{ Version:   resource.Version, Resources: converted, Freshness: apidiscoveryv2beta1.DiscoveryFreshnessCurrent, }) } aggregatedDiscovery := []apidiscoveryv2beta1.APIGroupDiscovery{{ ObjectMeta: metav1.ObjectMeta{ Name: resources["v1"].Group, }, Versions: aggregatedVersions, }} require.Equal(t, aggregatedDiscovery, doc.Items) } </ins> func TestLegacyFallback(t *testing.T) { aggregatedResourceManager := discoveryendpoint.NewResourceManager() rootAPIsHandler := discovery.NewRootAPIsHandler(discovery.DefaultAddresses{DefaultAddress: "192.168.1.1"}, scheme.Codecs)

Aggregated discovery caches when fetching from legacy discovery (). This causes apiservices who share the same service key to fetch from the cache and returns an empty document when their group-version differs from the initial apiservice that loaded the cache. See above See above No response_ 26 <detailslocal </details<details</details<details</details<details</details<details</details
/triage accepted
/sig api-machinery


Question: MASTER_NAME="${INSTANCE_PREFIX}-master" MASTER_TAG="${INSTANCE_PREFIX}-master" MINION_TAG="${INSTANCE_PREFIX}-minion" <del> MINION_NAMES=($(eval echo ${INSTANCE_PREFIX}-minion-{1..${NUM_MINIONS}})) </del> <ins> # Unable to use hostnames yet because DNS is not in cluster, so we revert external look-up name to use the minion IP #MINION_NAMES=($(eval echo ${INSTANCE_PREFIX}-minion-{1..${NUM_MINIONS}})) </ins> # IP LOCATIONS FOR INTERACTING WITH THE MINIONS MINION_IP_BASE="10.245.2." for (( i=0; i <${NUM_MINIONS}; i++)) do KUBE_MINION_IP_ADDRESSES[$i]="${MINION_IP_BASE}$[$i+2]" MINION_IP[$i]="${MINION_IP_BASE}$[$i+2]" <ins> MINION_NAMES[$i]="${MINION_IP[$i]}" </ins> VAGRANT_MINION_NAMES[$i]="minion-$[$i+1]" done

In the guestbook example, the redis slaves are instructed to connect to the redis-master using the proxy provided by the minion on which they reside. The minion is designated by its hostname "kubernetes-minion-N". The container cannot resolve this name to an IP address and so slave connections fail. 1) create vagrant cluster 2) create redis-master pod 3) create redis-master service 4) create redis-slave replicationController Verify redis-slave pod locations: list pods ID Image(s) Host Labels Status redis-master-2 dockerfile/redis kubernetes-minion-1/10.245.2.2 name=redis-master Running -3839-11e4-a922- brendanburns/redis-slave kubernetes-minion-1/10.245.2.2 name=redisslave,replicationController=redisSlaveController Running -3839-11e4-a922- brendanburns/redis-slave kubernetes-minion-3/10.245.2.4 name=redisslave,replicationController=redisSlaveController Running Log onto a minion containing a slave (note: vagrant hostnames are minion-N, not kubernetes-minion-N) Find the slave container docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES brendanburns/redis-slave:latest /bin/sh -c About an hour ago Up About an hour k8s--slave.---3839-11e4-a922- docker logs | tail [7] 09 Sep 17:27:08.428  Connecting to MASTER kubernetes-minion-1: [7] 09 Sep 17:27:08.430 # Unable to connect to MASTER: No such file or directory [7] 09 Sep 17:27:09.435  Connecting to MASTER kubernetes-minion-1: sudo nsenter -m -u -n -i -p -t $(docker inspect --format '{{ .State.Pid }}' ) /bin/bash [ root ]$ ping kubernetes-minion-1 ping: unknown host kubernetes-minion-1 [ root ]$ cat /etc/hosts 244.1.4 -3839-11e4-a922- 127.0.0.

1 localhost ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters [ root ]$ cat nameserver 10.0.2.3 cat /proc/1/environ HOME=/rootPATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/binHOSTNAME=-3839-11e4-a922-0800279696e1REDISMASTERSERVICEPORT=10000REDISMASTERPORT=tcp://kubernetes-minion-1:10000REDISMASTERPORT6379TCP=tcp://kubernetes-minion-1:10000REDISMASTERPORT6379TCPPROTO=tcpREDISMASTERPORT6379TCPPORT=10000REDISMASTERPORT6379TCPADDR=kubernetes-minion-1SERVICE_HOST=kubernetes-
Tim - are you the right guy to handle this? I think we need to resolve the name of the service host and only give IPs to the minion in env variables. We can't assume the containers have the same DNS set up as the host system.
To be clear -- this example is for the checked in Vagrant stuff? If that is the case would be the guy? I'm trying to catch up on IRC discussions from earlier today.
I can look at this, but there are two possible solutions. The first solution is in the vagrant cluster (or any other local vm setup) we could run a dns name server on the kubernetes-master that can resolve each minion by host name. We would then pass a -dns option to the docker daemon to specify that name server to use in each container. The second solution is that we pass IP address instead of host name and make no assumption or override on the containers specified name server. Is there any obvious negative to this solution? Has much thought been done on the expected Nameserver used by containers spawned by our service? - if you can weigh in, I would be happy to resolve using either approach. My preference is solution two.
So, the vagrant case is a little special as there is no sane DNS set up by default. Another suggestion (3) would be to revert the stuff and just devolve to using ip addresses directly. I would do (3) before (2) before (1).

But I'm cool with any of them. As for the nameserver of the containers that are spawned -- docker currently just takes the of the host and copies that into the container. It doesn't do that for . that is the core of the issue here. If/when we do offer some level of split horizon DNS (I'd love to) we can start exposing an extra layer of DNS on top of whatever is configured for the host. That is part of the "ip per service portal" stuff that is being talked about a lot right now.
If I do (3) there is a piece of code in pod storage that mangles my hostname where I return 10.245.x.x and it asks the cloud provider to resolve the IP address for "10". Since other providers appear to also be returning IP as host, I will look to change this so kubernetes makes no modifications to hostnames that it's given. Look for PR tomorrow.
you're right I'd missed that point. This really only applies to the Vagrant cluster and so a fix to the vagrant setup is sufficient. I also agree with your priorities. Injecting a line into /etc/hosts (if that's reasonable) is the simplest solution. Either reverting or allowing a switch to pass the minion public IP rather than hostname is next. Adding a DNS service to the vagrant master and forcing DNS resolution to the master host is most complicated and least preferable. Thanks both of you for helping me think this through better. Mark
I don't grok what the difference between 2 and 3 is? I sent a PR to docker a while back () to insert arbitrary lines to /etc/hosts. I still think it is a reasonable thing to want to do.
- difference between (2) and (3) is that in the past, the vagrant cloud provider was returning the IP address as the literal hostname for the minion. I had changed this after noticing that it had some ugly side-effects in the CLI, where a call to list pods would report pod ip addresses as IP / nil.

The issue (which I now believe is a bug that I want to change independent of what we do here) is that the code at this line mangles the hostname returned by the cloudprovider: This makes using the IP address as a hostname (10.245.2.2) to have the ugly side-effect of the code then asking the cloudprovider for the IP address of hostname "10", and then the cloudprovider having no idea how to proceed. Independent of what is done in this issue, I want to submit a PR tomorrow to stop that code from manipulating the reported hostname that was provided by the cloudprovider before asking that same cloudprovider to return the IP address using a potentially different input value. so with that backlog (3) means revert vagrant cloudprovider to use IPAddress as hostname, which means when we pass in what we believe is hostname in this environment, it will just work because it is in-fact the IPAddress. (2) means do not pass in hostname but pass in IPAddress by changing more core k8s code. all that being said, there is an option (4) that I am intrigued by, - after following your PR through its final conclusion, it looks like as of docker 1.2.0 you can change the /etc/hosts file of a running container. this introduces an option where the kubelet injects the /etc/hosts file of minion into the running container so it has the /etc/hosts values of the minion. I would need to play with that support a little more to know pros/cons more. I think longer term, we need a plan on what nameserver our containers are expecting to reference, and if they are inherently different in some way. I have run into a number of times the issue where doing things like running docker in docker causes the nested container to default to the nameserver 8.8.8.8, which happens to be blocked on the Red Hat network ;-) For now to unblock I will probably pursue (3) and fix the hostname bug referenced here at the same time. It does make the CLI uglier when working with this environment, but not a huge deal, at least it will function.
Thanks both for the complete explanation of what's happening and for the suggested fixes.

I think in the long run using hostnames in SERVICEHOST and ensuriing either through IP injection into container /etc/hosts or by providing DNS are the best options. I'm not sure if there might still be cases where resolution of SERVICEHOST would still be best done by presenting the IP address or not.


Question: { description: "sandbox with containers should not be garbage collected.", sandboxes: []sandboxTemplate{ <del> makeGCSandbox(pods[0], 0, runtimeapi.PodSandboxState_SANDBOX_NOTREADY, false), </del> <ins> makeGCSandbox(pods[0], 0, runtimeapi.PodSandboxState_SANDBOX_NOTREADY, false, 0), </ins> }, containers: []containerTemplate{ {pod: pods[0], container: &pods[0].Spec.Containers[0], state: runtimeapi.ContainerState_CONTAINER_EXITED},

kubernetes pre-submit unit tests for pass some times and fails some times causing the flakiness. Here is one such failure https://k8s- Passing: Failing: cc
Yep, I'm checking with this
Sorry for introducing a flakiness. Sent out the fix in .
/assign
Dup of . /close

https://k8s- Failed: TestSandboxGC {kuberuntime}
https://k8s- Failed: TestSandboxGC {kuberuntime}
/assign
Should've been fixed by


Question: metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" "k8s.io/apimachinery/pkg/util/intstr" <ins> "k8s.io/apimachinery/pkg/util/strategicpatch" </ins> "k8s.io/apimachinery/pkg/util/wait" "k8s.io/kubernetes/cmd/kube-apiserver/app/options" "k8s.io/kubernetes/pkg/controlplane"

Why PatchService method is used for deleting finalizers in cloud-controller-manager, This method updates with service/status, It seems to go unnoticed by the API server, resulting in the API server being unaware when the finalizer field on the service is deleted, leading to a delay in the release of service-related NodePorts. service nodeport be released promptly when service is deleted. create a loadblancer service with nodeport seted,then delete it and recreate. cloud-controller-manager add a finalizer on service. delete this service and then cloud-controller remove finalizer. but nodeport is not released when service deleted. ref i change like this, it seems like releasing nodeport when service is deleted : No response <details</details<details</details<details</details<details</details<details</details<details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig cloud-provider
Are you sure? that method updates the status and remove the finalizer ... otherwise , who removes the finalizer?
I apologize for any confusion. What I meant was that the update of the status and the removal of the finalizer are performed through updating service/status. Therefore, removing the finalizer does not release resources.It seems like removal of the finalizer by service/status, store.AfterDelete is nil. i think remove the finalizer should by patch servcie and updates the status should by patch service/status.Now we patch both by servcie/status.
no need to apologize, let me see if I can reproduce this locally first, Services api allocation is complex , it overrides the standard AfterDelete and release all allocated resources there /assign
/sig network ack the problem
will look for a possibly-related past cloud provider issue.
this will be fixed by , it is a problem in the apiserver logic
Cool! I forgot that you can patch finalizers thru /status. I repro'ed it on both ClusterIP and NodePort.

Saw this in the logs: So the repair loop, yet again, saves our butt but also hides a bug.
we need metrics here about to have visibility of the repairs
I also found this problem in versions 1.23 and 1.27
yep, I started the backports , but we can only backport to the supported versions, 1.23 and 1.24 are EOL


Question: package events <ins> // volume relevant event reasons </ins> const ( <del> // volume relevant event reasons </del> FailedBinding             = "FailedBinding" VolumeMismatch            = "VolumeMismatch" VolumeFailedRecycle       = "VolumeFailedRecycle"

Anyone itching to understand the code and try fixing things in Kubernetes, please help fixing the lint failures Step 1: Remove one or two packages (related packages) from the list in - (note: do not change generated files, the name of defaulting functions, or the name of conversion functions... packages failing linting due to issues in those files will need to be left alone) Step 2: Run and look at the logged errors Step 3: Fix the code to ensure that runs cleanly. Step 4: Run and to make sure we fix any problems with gofmt or bazel. Step 5: File a PR with the change Step 6: Go to step :) Please use the tracking spreadsheet For everyone who is going to contribute, please append the packages you picked at the end. For existing contributor, please make sure this list is update to date
/sig architecture /sig testing
Hey, I would like to resolve this issue. Could you assign it to me ?
there are a lot of packages so there will be many people working on this. Also we cannot assign issue to folks who are not in the github org for kubernetes. please feel free to pick up a few packages like i mentioned and file a PR. Make sure you link back to this issue, so folks can see which packages you filed PR for.
OK. I see. I will firstly pick up the package called to fix.
Hi dims. Should i submit my PR before i remove one or two packages (related packages) from the list in - Or I first remove the package from list, then I fix the issue and submit a PR?
in your PR, you will have both the fixes to packages and some lines removed from /.golint_failures
When I run , I found some errors as below: I found that there is a go file in But there is still a error.
Say your GOPATH looks like this Your kubernetes directory should be () cd to and then run from that directory.
Is it necessary to obey all rules in golint? Rules in golint are set in stone. Maybe using gometalinter to ignore some specific errors is better?
I revise some erros about golint in some packages. Please review it.
which errors would you like to ignore?

if you can open a new issue with the list of errors to ignore (and proposing gometalinter), we can get some opinions there from others in the community.
Let me take the kubectl cmd part...
Can someone please review and let me know if I am doing it right?
I ll work on all the packages related to
we get 800-ish pkgs left to fix. my concern is that if we fix them package by package, then such pulls will be flooding the repo. can't we possibly make it in larger pulls? the larger the better?
+1 to slightly larger pulls with related packages. if allow too large then the reviewers will not be able to do a thorough review. we should spread the load among multiple reviewers. (if we allow a really large pull, we will end up with top level reviewers who will face the burden and the PR(s) will just stagnate)
oh good point. we could put one pkg's change into one commit, and merge all changes under one area into one pull so that it wont spread the load. wdyt? for each pull, we can have hundred-ish lines of change. but it will still be 30~50 pulls i guess? good news is that we can take our time to deal with it, at least before 1.12
sounds great! please feel free to help make it happen :) (by suggesting groups of packages that kinda go together)
I'm looking at fixing pkg/proxy, and I think I see what was talking about ignoring some rules. proxy.ProxyProvider is being suggested because golint is saying that it stutters. In that example, I think it's okay renaming it to proxy.Provider, but I could see an example where you would want to keep the stutter if the package that it's declared in uses it heavily.
sure thing, feel free to propose a PR for switching to gometalinter independently of other patches. We can work through that there.
I sent the pull request to fix pkg/volume/aws_ebs
I've fixed a few of these, but get the following error when I try to run : Are there any instructions on what I need to setup in order to resolve this?

you can find this file in
I have raised a PR to fix golint errors in pkg/api
I have fixed some pkgs under /pkg/kubelet pkg/kubelet/images pkg/kubelet/kuberuntime pkg/kubelet/leaky pkg/kubelet/pleg
I submitted a pull request for golint errors in pkg/security/ edit: I can work on picking up pkg/controller/bootstrap through pkg/controller/namespace/... (114-129)
fix golint errors in package cmd/kubeadm/app, since the codes under cmd/kubeadm/app/apis/kubeadm/v1alpha2 and cmd/kubeadm/app/apis/kubeadm/v1alpha3 are auto-generated, i keep them untouched.
I am working on some packages under pkg/kubelet, anyone interested in this plz leave messages for me :)
fix golint for pkg/util/dbus pkg/util/labels pkg/util/node pkg/util/strings pkg/util/workqueue/prometheus
PR fixes golint errors in pkg/routes,pkg/securitycontext, pkg/serviceaccount
PR fixes golint in pkg/probe
should we have a checklist as part of the issue summary saying what is in progress and what is done? Since this issue is getting long, this will make new contributors to pick up issues easier.
Great idea!, want to throw a google doc with a check-list so it's easy for participants to organize themselves? (please!) If it is the issue summary then one of us with privileges will be a bottle neck.
Created a with all the package that have on going PRs. Maybe you can add this to the summary. For everyone who is going to contribute, please append the packages you picked at the end. For existing contributor, please make sure this list is update to date
Thanks updated the spread sheet
Raised PR to fix golint issues in: pkg/controller/clusterroleaggregation pkg/controller/cronjob pkg/controller/daemon pkg/controller/deployment pkg/controller/deployment/util pkg/controller/disruption pkg/controller/endpoint
This issue is close in accident by . could you reopen it?
/reopen
Reopening this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I found a problem trying to fix golint messages for .

Golint complaints on 's function because of the snake case. When I try to fix it, fails because it is being used in a code generated file. If I clean generated files, tests pass, but the generated file is not invoking it anymore. I failed to find the generator definition using this so I could change the hook to just . Also I'm not sure if this is wanted. I also found other packages in also probably affected by this. I read somewhere about replacing golint with gometalinter, which would allow to define directives in code to omit certain rules. I would consider to have a second exclusion file for golint with these cases, to be updated if move to gometalinter is finally done. This would make this issue completion more feasible. Meanwhile, I'll move onto fixing golint messages for other packages, until this discussion gets cleared.
I also found a problem fixing . A test fails because the directory is marked as readonly (see tests @ Shall we also ignore this directory for this issue?
Is this issue still relevant because hack/verify- is giving me no errors.
the failures are still present in there are details in the first message on how to claim + solve issues :)
Sorry I misunderstood the issue
I'm working on this package:
pkg/apis/admission/v1beta1 pkg/apis/authentication/v1beta1 pkg/apis/authorization/v1beta1 pkg/apis/coordination/v1 pkg/apis/coordination/v1beta1 pkg/apis/imagepolicy/v1alpha1 pkg/apis/scheduling/v1alpha1 pkg/apis/scheduling/v1beta1 pkg/apis/settings/v1alpha1 pkg/apis/storage/v1alpha1
pkg/cloudprovider/providers/vsphere/vclib pkg/kubelet/apis/deviceplugin/v1alpha pkg/kubelet/apis/pluginregistration/v1alpha1 pkg/kubelet/apis/pluginregistration/v1beta1
:
I'm looking at the pkg/registry/*, all those which are left. I expect to give an update on what all is fixed in the next 24-48 hours.
hi there is a google sheet that is a bit outdated, maybe you can update that?
hi I seem to only have view access for the sheet. Can't edit.
I fixed the permission.

Please try it again
pkg/apis/authentication pkg/apis/coordination pkg/apis/events pkg/apis/imagepolicy pkg/apis/settings
hi I think this issue was closed prematurely by (using fixes #issue in commit)
I'll put in a PR shortly for the package
Looking at next
hi thanks! there is a spreadsheet here that tracks the progress on fixing these issues, consider adding the packages you're work on there so we can keep everything organized :+1:
Starting on now
I did indeed see the Google Sheet link (thank you and to take it a little further
test/e2e
test/e2e_node/builder
pkg/registry/admissionregistration/mutatingwebhookconfiguration/storage pkg/registry/admissionregistration/rest pkg/registry/admissionregistration/validatingwebhookconfiguration/storage pkg/registry/apps/daemonset/storage
Hi. I opened a PR - Please review.
Working on Will open a PR soon.
Starting on and should have a PR soon
I keep getting this golint verify error on and was wondering if others saw it: But when I that package to the golint failures in my PR, my reviewer said it isn't necessary? What does that mean? What did I miss? Should I use a different kind of verify? cf:
looks like the disconnect is because that problem does not show up in the version of we use in the CI. please see
Still confused: is that not the golint version I ran? it installs from vendor and uses that I think?
i installed from and i still don't see the problem, can you please check what version of golint you have?
I'm confused why I wouldn't be using the vendored version? how would that get out of sync?
/me looking
Note: I am reserving for a KubeCon Contributor Workshop demonstration. It is noted in the spreadsheet.
I think may not be ideal cause there's a lot of auto generated files there. What do you think about reserving instead for the KubeCON Contributor Workshop?
pkg/apis/scheduling pkg/apis/storage/util pkg/apis/storage/v1beta1/util pkg/apis/storage/v1/util
/priority backlog
Hi I am a new contributor here, I'd like to contribute and make changes to the following file/s. I have marked it as WIP in the Google Sheet. Can someone let me know if I have to do anything else before I start working on this PR?

Umm so there's an issue where Golint tells that underscore shouldn't be allowed. Here's a log of the go-lint warnings:
Thanks The following should not be modified, as they are designed like this for code generation.
I'd like to pick a package and do my first PR, is there a list of packages to be done? Or should I correlate with this issue thread? Thanks in advance!
please see step 1 () please scan other PRs linked to this issue as well just in case.
Will do. Thank you. On Thu, Oct 3, 2019, 7:25 AM Davanum Srinivas <:
Hi there, I'm looking for a good first issue. I'll start taking a look at if that's ok? There doesn't seem to be anyone currently tracked as looking at it in the attached spreadsheet.
I've found a few others I'm actively looking at, you won't get any argument from me. On Sat, Oct 12, 2019, 10:38 AM Pamela Kelly <:
I am working on the iptables pkg. Updated in the google docs
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I have problem running in step 2 $ hack/verify- what should I do?
PTAL for my pr to this issue, thx
you can move kubernetes directory to and re-run this script.
I'd like to take Can we fix linter issues for generated files? Or we should skip packages with zz_generated?
There're 2 lint errors that affects 292/507 packages, meaning it's not possible to cleanup the entire file without solving it: default pattern of Kubectl is the usage of [cmd]Options for initial options structs and it raises: /apis/{custommetrics,externalmetrics} for example raises the underscore error, but It's probably not worth to break backward compatibility renaming these folder for lint error fix: Any advice on these?

/reopen hm, cannot reopen this issue, it was closed by mistake
You can't reopen an issue/PR unless you authored it or you are a collaborator. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I am sure this is all not fixed. Am I missing something? ./hack/verify- installing golint from vendor Congratulations! All Go source files have been linted.
you need to remove package from first.
/remove-sig testing SIG testing doesn't own code failing lints, we own the linter. :upsidedownface:
Hello, is this still a relevent issue ? I would like to work upon
I have fixed
Hi Folks, Some of the golint fixes, require us to change the exposed function's name, in simple words, "change API". I was not sure, if that is welcomed and what is the process.
Heres what Im getting after going over the following steps as mentioned by Say your GOPATH looks like this GOPATH=/Users/dims/go Your kubernetes directory should be ($) cd to and then run hack/verify- from that directory. I get the following output - installing golint Congratulations! All Go source files have been linted. Im not really sure what I am doing wrong. Any help is appreciated!
I think I understand. I misunderstood the steps around .golint_failures. The first step is to remove the pkg that we are targeting from this file and then run the My mistake was to assume that the packages in the file need to be removed after we are done cleaning the golint errors.
Can someone please review and let me know if I've done this right?
After a couple years of seeing changes spawned from this issue, I don't think this is a good use of contributors' or reviewers' time.

golint is very opinionated about: exported type and function names, leading to PRs attempting to change exported things people use downstream missing documentation on interface implementation functions, which leads to not-very-useful copy/paste of interface godoc the govet and staticcheck verify jobs are much more useful since they catch correctness issues, but I would recommend closing this issue and dropping the golint checks from CI golint doc itself discourages from using its output in an automated enforced way: from
i guess one can say gofmt is opinionated too, but something good about Golang is that it comes with an opinion and a set of standard tools. for API documentation at least we found inconsistencies on how most exported API items are documented: golint can help with that, but if there are no "pragmas" to avoid breaking changes and given we have codegen generating non-compliant names enabling golint on API packages is sort of blocked in Kubernetes. for non-API, non-library, non-public code, i think being inconsistent is mostly OK. no preference on my side. shrug
true, but code formatting has the benefits of mostly eliminating meaningless diffs in PRs due to code movement/whitespace/etc if there are specific packages (like API packages) we lack documentation on, adding checks specifically for that could make sense exactly, I don't think golint is the right tool for making sure we have API docs
golint itself doesn't have these and the go project is against this sort of comment AIUI, however golangci-lint does have them. I think there are plenty of other static analysis lints we should use instead, beyond just staticcheck and vet there is e.g.
btw, if we remove golint can we see e.g. all-caps constants or functions with underscores dodging review? perhaps we should add golangci-lint soon after:
Completely agree there are better places to spend review bandwidth. I was curious to see what the effort looked like. There have been 890 commits that touched over the past 3+ years !
/remove-help /remove-good-first-issue
Overall agree on the bandwidth. Also note that it was a good first issue with very low bar to get them in the repo. May be we need to find a similar low bar, but high value good first issue.

UT's come to mind, are there other things we curate for new contributors ?
Do we have an alternative way of enforcing that new exported code (not necessarily API fields) has at least some minimum documentation? As a reviewer, I ask contributors to add meaningful comments. But reality is that I sometimes stumble into old code that doesn't have documentation because we didn't have golint at some point. I agree that fix-golint PRs just add burden to reviewers. But, before removing the lint check, can we wait and see if it's enough to discourage people from sending more PRs like this? Additionally, we can encourage reviewers to just close PRs that only fix golint issues.
+1 for adding burden to reviewers. However, as a new contributor, I think this a very good opportunity for new contributors to study the code base and submit their first PR. In my opinion, I think this issue would be able to attract more new contributors, get familiar with the code base, and get familiar with the git workflow.
thanks everyone for getting us this far. We essentially gave up on the existing structure/system for golint: At some point we can look at adding golangci-lint, there's a WIP in progress in /close
Closing this issue. <detailsIn response to : thanks everyone for getting us this far. We essentially gave up on the existing structure/system for golint: Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: } return err } <ins> // buildOpenAPIModelsForApply constructs openapi models from any validation schemas specified in the custom resource, // and merges it with the models defined in the static OpenAPI spec. // Returns nil models if the ServerSideApply feature is disabled, or the static spec is nil, or an error is encountered. func buildOpenAPIModelsForApply(staticOpenAPISpec *spec.Swagger, crd *apiextensions.CustomResourceDefinition) (proto.Models, error) { if !utilfeature.DefaultFeatureGate.Enabled(features.ServerSideApply) { return nil, nil } if staticOpenAPISpec == nil { return nil, nil } specs := []*spec.Swagger{} for _, v := range crd.Spec.Versions { s, err := builder.BuildSwagger(crd, v.Name, builder.Options{V2: false, StripDefaults: true, StripValueValidation: true}) if err != nil { return nil, err } specs = append(specs, s) } mergedOpenAPI, err := builder.MergeSpecs(staticOpenAPISpec, specs...) if err != nil { return nil, err } models, err := utilopenapi.ToProtoModels(mergedOpenAPI) if err != nil { return nil, err } return models, nil } </ins>

What happened: While upgrading from v1.15 to try out kubernetes v1.16.0-beta.2, kube-apiserver is printing error messages: This also brakes the API server since the user is unable to even delete the "faulty" CR What you expected to happen: Not to print any error message and accept a CR that was already valid in 1.15 How to reproduce it (as minimally and precisely as possible): Install k8s 1.15.x (optional) Create the following CR Upgrade to k8s v1.16.0-beta.2check kube-apiserver for errors:Try delete the faulty CR (Won't be possible) Environment: Kubernetes version (use ): Anything else we need to know?: A quick search sent me to this code path that is being used to compile k8s It seems the problem it known but not yet fixed since this is in the latest commit of : /kind bug /king regression cc
/sig api-machinery
/assign /priority critical-urgent /milestone v1.16
This is caused by compiling the openapi spec for use by server-side apply here: SSA is using the method, which has significant gaps in handling valid openapi specs (see all the TODO comments in ) and until now was only used for client-side kubectl validation (as far as I'm aware). Using it in the server-side path is concerning. /assign
running with , this works as expected and is compatible with 1.15
/assign
fix and covering test is in
this is resolved in , available now


Question: klog.V(2).Infof("azureDisk - update(%s): vm(%s) - detach disk(%s)", nodeResourceGroup, vmName, diskName) ctx, cancel := getContextWithCancel() defer cancel() <ins> // Invalidate the cache right after updating defer as.cloud.vmCache.Delete(vmName) </ins> _, err = as.VirtualMachinesClient.CreateOrUpdate(ctx, nodeResourceGroup, vmName, newVM) if err != nil { klog.Errorf("azureDisk - detach disk(%s) failed, err: %v", diskName, err) } else { klog.V(2).Infof("azureDisk - detach disk(%s) succeeded", diskName) <del> // Invalidate the cache right after updating as.cloud.vmCache.Delete(vmName) </del> } return err }

What happened: Azure Disks get randomly mounted in a way that makes them unusable, returning for every disk I/O operation. Affected Pods remain in until someone manually recreates them. What you expected to happen: Azure Disks are consistently usable by Pods inside which they are mounted. How to reproduce it: Create the following StatefulSet. Adjust the replicas and delete the associated Pods () until the problem occurs (i.e. some Pod ends up in ). <details</summary</details</brDefault StorageClass (as created by ACS-Engine): <details</summary</details</brAnything else we need to know?: Nodes were updated from 1.10.4 to 1.10.10, then from 1.10.10 to without a reboot. According to written by the issue is fixed, but I can still observe it quite often. Ref Azure/acs-engine Environment: Kubernetes version: v1.10.10 + Cloud provider or hardware configuration: Azure (westeurope) OS: Ubuntu 16.04.4 LTS Kernel: 4.13.0-1018-azure Install tools: ACS-Engine Others: /kind bug /sig storage /sig azure
In my current test environment: (pod uid is -f23b-11e8-bf38-) on the node Notice the mismatch between the disk referenced by lun0 () and the actual mount (). on the node
After deleting the failing Pod and waiting for its successful recreation: (pod uid is now -f247-11e8-bf38-) on the node This time disks are matching (). on the node
/assign


Question: "k8s.io/component-base/metrics/legacyregistry" apiregistration "k8s.io/kube-aggregator/pkg/apis/apiregistration/v1" "k8s.io/utils/pointer" <ins> "k8s.io/utils/ptr" </ins> ) type targetHTTPHandler struct {

Extension API servers do not currently receive the header when requests are proxied through kube-aggregator, so spans are not linked properly. Traces from the extension API sever should be linked to the parent in kube-aggregator. tracing for kube-apiserver tracing for sample-apiserver and aggregate it to kube-apiserver a request to kube-apiserver for one of the resources from sample-apiserver sample-apiserver traces will not be linked to the kube-apiserver traces No response <detailsThis problem currently exists on . </details<details</details<details</details<details</details<details</details<details</details
/sig api-machinery
/cc /triage accepted


Question: return CreatePausePod(cs, InitPausePod(&conf)) } <ins> // CreatePVC creates a PersistentVolumeClaim with the given config and returns // its pointer and error status. func CreatePVC(cs clientset.Interface, pvc *v1.PersistentVolumeClaim) (*v1.PersistentVolumeClaim, error) { return cs.CoreV1().PersistentVolumeClaims(pvc.Namespace).Create(context.TODO(), pvc, metav1.CreateOptions{}) } // CreatePV creates a PersistentVolume with the given config and returns its // pointer and error status. func CreatePV(cs clientset.Interface, pv *v1.PersistentVolume) (*v1.PersistentVolume, error) { return cs.CoreV1().PersistentVolumes().Create(context.TODO(), pv, metav1.CreateOptions{}) } </ins> // RunPausePod creates a pod with "Pause" image and the given config and waits // until it is scheduled. It returns its pointer and error status. func RunPausePod(cs clientset.Interface, pod *v1.Pod) (*v1.Pod, error) {

We need an integration test for the scheduler enforcement of the alpha feature. This is required for beta graduation; we need to collect signal on the current behavior before we can proceed. See this comment for context:
/sig storage /sig scheduler
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : /sig storage /sig scheduler Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scheduling


Question: StabilityLevel: metrics.ALPHA, }, []string{"queue", "event"}) <ins> PermitWaitDuration = metrics.NewHistogramVec( &metrics.HistogramOpts{ Subsystem:      SchedulerSubsystem, Name:           "permit_wait_duration_seconds", Help:           "Duration of waiting in RunPermitPlugins.", Buckets:        metrics.ExponentialBuckets(0.001, 2, 15), StabilityLevel: metrics.ALPHA, }, []string{"result"}) </ins> metricsList = []metrics.Registerable{ scheduleAttempts, SchedulingLatency,

<!-- Please only use this template for submitting enhancement requests --What would you like to be : metric name Description -- -- permitwaitduration_seconds Duration of permit extension point finishes waiting. Why is this needed: In permit extension point, if the status is "Wait", we block until the pod is scheduled or timeout: Blocking is an important behavior that we should monitor. /sig scheduling
/assign


Question: Namespace string } <ins> // NetexecDialResponse represents the response returned by the `netexec` subcommand of `agnhost` type NetexecDialResponse struct { Responses []string `json:"responses"` Errors    []string `json:"errors"` } </ins> // DialFromEndpointContainer executes a curl via kubectl exec in an endpoint container. func (config *NetworkingTestConfig) DialFromEndpointContainer(protocol, targetIP string, targetPort, maxTries, minTries int, expectedEps sets.String) { config.DialFromContainer(protocol, echoHostname, config.EndpointPods[0].Status.PodIP, targetIP, EndpointHTTPPort, targetPort, maxTries, minTries, expectedEps)

Example failure: Stacktrace: The problem seem to be related to the fact that we're running the test on private cluster (nodes doesn't have external IP address).
/assign
/triage unresolved Comment when the issue is assessed and confirmed. I am a bot run by vllry.
Given that there are more issues with the NetLB tests than just this particular one, I think we should revert my PR that re-enabled them. ! Wojtek just sent a PR to do that -
But let's leave this issue open - we want to fix and re-enable it again.
The revert merged, but we didn't managed to get it in before the yesterday's run started. Today's run should be green.
Yeah - but that doesn't mean we should close it later - we should fix and reenable this test. The issue should be opened until that test is reenabled.
That's clear, I didn't say anywhere that we should close it
Ok, the tests have been disabled and they don't fail the correctness tests anymore. When it comes to fixing this particular test. I took a cursory glance and I think it is fixable. We should change the code to not expect public IPs on the nodes. And then when poking over http we should do it not from the test but proxy via some container. There is already a code written for that, see /help wanted
If this issue has been triaged, please comment . If you aren't able to handle this issue, consider unassigning yourself and/or adding the label. I am a bot run by vllry.
/good-first-issue
This request has been marked as suitable for new contributors. Please ensure the request meets the requirements listed . If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : /good-first-issue Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
If this issue has been triaged, please comment . If you aren't able to handle this issue, consider unassigning yourself and/or adding the label. I am a bot run by vllry.
/remove-triage unresolved
I would like to take this /assign
- thanks!

contains brief description on how to fix that
so are we going to access all the nodes via a test container?
Not sure if I fully understand your question, so if you meant something different let me know: to make the test work in large clusters (5k nodes), we can't access all - we may want to do tricks we do in other tests like access min(cluster size, some constant (e.g. 10 or 100) if you meant "via a test container" - yes, I think it's fine to just migrate the test to always use it
Reopening - since we need to reenable this test.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
Does this issue need an owner? I got a little lost wading through all of the issues/PRs attached to this issue. If I wanted to take up the reins, how do I trigger the 5k node test on a PR or is it automatically triggered?
The test is disabled. I think the crucial part is done now - which is: I can't remember what else left to be done. Unfortunately you can't do that from oss. We only have post-submits exposes as these are too expensive. We can try running it internally to check if it now works...
We reenabled this test and it seem to work. So closing this one. Thanks for all the help on it!


Question: diskName, err := createPDWithRetry() framework.ExpectNoError(err, "Error creating PD") numContainers := 4 <del> host0Pod := testPDPod([]string{diskName}, host0Name, false /* readOnly */, numContainers) </del> <ins> var host0Pod *api.Pod </ins> defer func() { By("cleaning up PD-RW test environment") // Teardown pods, PD. Ignore errors. // Teardown should do nothing unless test failed. <del> podClient.Delete(host0Pod.Name, api.NewDeleteOptions(0)) </del> <ins> if host0Pod != nil { podClient.Delete(host0Pod.Name, api.NewDeleteOptions(0)) } </ins> detachAndDeletePDs(diskName, []string{host0Name}) }()

PR moved some PD tests to flaky to resolve issue (and duplicate ). These tests started failing due to PR . is working on a patch to de-flake the tests. This issue is to track the fix and should be closed when: PD tests have been deflaked (verify by running the "" e2e tests back to back for a couple of hours to verify the issue is fixed). flaky label has been removed from the "" tests that were marked flaky in CC
how to watch the submit queue and find if the flake is gone?
I don't know. is there any way can verify whether test is no longer flaky?
I'm not sure what the publicly accessible link for test history is. I took a look for you: Pod Disks should schedule a pod w/ a RW PD shared between multiple containers, write to PD, delete pod, verify contents, and repeat in rapid succession Failed 3 times in the last 30 runs. Stability: 90 % Pod Disks should schedule a pod w/two RW PDs both mounted to one container, write to PD, verify contents, delete pod, recreate pod, verify contents, and repeat in rapid succession Failed 1 times in the last 30 runs. Stability: 96 % I have a PR pending that enables the new attach/detach controller and I was able to run the tests back to back overnight without any failures. So once that gets merged, we can take a look at moving these tests back.
Something in PD attach/detach affects also 'Dynamic provisioning' e2e test, a PD is still attached to a node ~15 minutes after appropriate pod was deleted and thus deleter can't delete it. From controller log:
please take a look at https://k8s-
thanks so at least one of the flake has a different error than "container not found" In one, the error is
^^ the above seems like a possible bug in exec
^
The error to me indicates that the original container is no longer around as exec can't find the ipc namespace fd. Are we sure the container that we are trying to join is still around?
my memory is rusty but did you have an idea to randomize the container id in this pd test?
see the info here:
Close we will track this with


Question: } podCounts[label]++ } numServicePods := len(nsServicePods) result := []schedulerapi.HostPriority{} //score int - scale of 0-maxPriority

<!-- Thanks for filing an issue! Before hitting the button, please answer these questions.--Kubernetes version (use ): master What happened: The functionality is hidden in a large function, and not explicit, testable. What you expected to happen: The logic of should read more clearly, with tests for individual filters.
has a PR for this .
Few more things include removing limits in entries like
yeah, im not sure why the thing is there, after all they are just appending to the slice and the limit wont have an effect , from what i can tell, will it? probably good idea to remove unless im missing something.
Its part of part of PR too.


Question: claims, err := tc.client.ResourceV1alpha2().ResourceClaims("").List(tc.ctx, metav1.ListOptions{}) require.NoError(t, err, "list claims") for _, claim := range claims.Items { <ins> claim := claim </ins> objects = append(objects, &claim) } schedulings, err := tc.client.ResourceV1alpha2().PodSchedulingContexts("").List(tc.ctx, metav1.ListOptions{}) require.NoError(t, err, "list pod scheduling") for _, scheduling := range schedulings.Items { <ins> scheduling := scheduling </ins> objects = append(objects, &scheduling) }

Apply this patch: Then run: This fails with: Test should pass. See above. If there are N objects in claims.Items, then this loop for _, claim := range claims.Items { objects = append(objects, &claim) } is appending the same pointer N times, pointing at the final item in the list. Adding fixes that mistake, but then it makes the test fail. Something else about the test is buggy, but the bug was being masked by this bug. In the repro case I to the loop a few lines down, just for good measure, but that one is not causing a problem. The line by itself causes the test failure. HEAD N/A N/A N/A N/A N/A
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scheduling /priority important-soon /cc Thanks for finding this and the detailed report
/assign
Can you help to fix this?
If we don't here from soonish, then I'll fix it myself.
Sorry, I've been busy with an office accident this week, if you want fix it, please take it
/assign
/unassign


Question: ] }, { <del> "path": "/api/v1/namespaces/{namespace}/componentstatuses", </del> <ins> "path": "/api/v1/componentstatuses", </ins> "description": "API at /api/v1", "operations": [ {

The selflink provided in the api response for component statuses (/api/v1/componentstatuses) is broken. For example: /api/v1/namespaces/componentstatuses/controller-manager (as specified in the response of /api/v1/componentstatuses), will return: { "kind": "Status", "apiVersion": "v1", "metadata": {}, "status": "Failure", "message": "the server could not find the requested resource", "reason": "NotFound", "details": {}, "code": 404 } while: /api/v1/namespaces/default/componentstatuses/controller-manager, will return the data. In addition, and this is asked for better understanding, are the component statuses namespace scoped? as far as I understood etcd, the scheduler and the controller manager were shared across the kubernetes cluster. If they aren't namespace scoped, what is the logic behind specifing a namespace in these api requests (as shown in ): /api/v1/namespaces/{namespace}/componentstatuses /api/v1/namespaces/{namespace}/componentstatuses/{name} If they are in fact namespace scoped, would you be kind and share an explanation/link to a documentation page?
I agree that it doesn't make a lot of sense for this to be namespaced given what's in it at the moment. Given what I'd like it to contain, the namespace might make more sense in the future. The self link is super broken right now, though, probably due to lack of actual namespace. there's probably a bug in the go-restful setup code for this. Can you take a look at some point? Also it might be worth mentioning that this will probably change in the near future, is working on a replacement.
The whole /componentstatus API is just broken. It needs rewriting.


Question: }, } <del> for _, test := range tests { imageMap := createImageExistenceMap(test.nodes) if !reflect.DeepEqual(test.expected, imageMap) { t.Errorf("expected: %#v, got: %#v", test.expected, imageMap) } </del> <ins> for i, test := range tests { t.Run(fmt.Sprintf("case_%d", i), func(t *testing.T) { imageMap := createImageExistenceMap(test.nodes) if !reflect.DeepEqual(test.expected, imageMap) { t.Errorf("expected: %#v, got: %#v", test.expected, imageMap) } }) </ins> } }

<!-- Please only use this template for submitting enhancement requests --What would you like to be : Currently some unit tests in pkg/scheduler is not organized as Use subtests can make it easier to find out which case fails and leverage benefit of parallelism . Why is this needed: Follow Go test practice.
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/remove-kind feature /kind cleanup /sig scheduling /assign