Question: podARecByUDPFileName := fmt.Sprintf("%s_udp@PodARecord", fileNamePrefix) podARecByTCPFileName := fmt.Sprintf("%s_tcp@PodARecord", fileNamePrefix) probeCmd += fmt.Sprintf(`podARec=$$(hostname -i| awk -F. '{print $$1"-"$$2"-"$$3"-"$$4".%s.pod.cluster.local"}');`, namespace) <del> probeCmd += fmt.Sprintf(`test -n "$$(dig +notcp +noall +answer +search $${podARec} A)" && echo OK > /results/%s;`, podARecByUDPFileName) probeCmd += fmt.Sprintf(`test -n "$$(dig +tcp +noall +answer +search $${podARec} A)" && echo OK > /results/%s;`, podARecByTCPFileName) </del> <ins> probeCmd += fmt.Sprintf(`check="$$(dig +notcp +noall +answer +search $${podARec} A)" && test -n "$$check" && echo OK > /results/%s;`, podARecByUDPFileName) probeCmd += fmt.Sprintf(`check="$$(dig +tcp +noall +answer +search $${podARec} A)" && test -n "$$check" && echo OK > /results/%s;`, podARecByTCPFileName) </ins> fileNames = append(fileNames, podARecByUDPFileName) fileNames = append(fileNames, podARecByTCPFileName)

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . If this may be security issue, please disclose it privately via --Is this a BUG REPORT or FEATURE REQUEST?: /kind bug /sig testing What happened: Tests checking DNS responses can have false positives, such as "should provide DNS for services". The tests rely on dig-ing certain domain names [1]: test -n "$(dig +notcp +noall +answer +search kubernetes.default A)" && echo OK /results/ The problems could be that, if a host / container doesn't have any entries in their : ubuntu dig +notcp +noall +answer +search kubernetes.default A ;; connection timed out; no servers could be reached , causing the test to pass and to write "OK" into the results file. [1] What you expected to happen: The tests shouldn't be able to pass if dig failed. How to reproduce it (as minimally and precisely as possible): Comment all nameservers or configure a bogus DNS server in, and run: test -n "$(dig +notcp +noall +answer +search kubernetes.default A)" && echo OK Anything else we need to know?: Environment: Kubernetes version (use ): 1.9.6 Cloud provider or hardware configuration: N/A OS (e.g. from /etc/os-release): "Ubuntu 16.04.3 LTS" Kernel (e.g. ): Linux ubuntu 4.4.0-87-generic
/assign
: But one of the purpose of the test is to ensure that the DNS of the cluster is replying. In other words, if it does not reply then we EXPECT this test to fail. If you are not using DNS then isn't it better to SKIP the test ?
I believe means to say "false negatives" ... meaning that in some situations, the test will errantly pass, when it should fail.
oops . Looks like I misread the description, and agree that the dig should work.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale


Question: // CronJob as last known start time. earliestTime = sj.ObjectMeta.CreationTimestamp.Time } <ins> if sj.Spec.StartingDeadlineSeconds != nil { // Controller is not going to schedule anything below this point schedulingDeadline := now.Add(-time.Second * time.Duration(*sj.Spec.StartingDeadlineSeconds)) if schedulingDeadline.After(earliestTime) { earliestTime = schedulingDeadline } } </ins> if earliestTime.After(now) { return []time.Time{}, nil }

<!-- Thanks for filing an issue! Before hitting the button, please answer these questions.--Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see ): No What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.): , , Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG REPORT <!-- If this is a BUG REPORT, please: Fill in as much of the template below as you can. If you leave out information, we can't help you as well. If this is a FEATURE REQUEST, please: Describe in detail the feature/behavior/change you'd like to see. In both cases, be ready for followup questions, and please respond in a timely manner. If we can't reproduce a bug or think a feature already exists, we might close your issue. If we're wrong, PLEASE feel free to reopen it and explain why. --Kubernetes version (use ): Environment: Cloud provider or hardware configuration: bare metal OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: , deployed with Others: What happened: I am using a scheduledjob with . An instance of my job has taken a long time to finish, thus skipping many start times. Ever since, the scheduledjob will not schedule anything new. Controller manager log has: What you expected to happen: I expect the scheduledjob not to be stuck not scheduling anything anymore. With , I expect that it should just re-schedule something as soon as the previously running job has completed. The "Too many missed start times to list" safeguard appears to apply mostly to cases where job concurrency is allowed to avoid creating a lot of jobs in case of timestamp issues. ` How to reproduce it (as minimally and precisely as possible): The issue shows up after about 100 minutes of this running. Anything else do we need to know:
cc
sorry for the long delay in response, there were a few issues solved in later versions of ScheduledJob, can you verify this against latest 1.4.x? I've tried cherry-picking all of the crucial ones, like: and I'll try to verify this on my own as well.

I am still getting the same issue with
thanks for the info, will dig deeper into this one.
Thanks a lot any chance a potential fix could make it in 1.5?
if possible - yes, I'll try to backport it to 1.5
did you look into this? If not, let me know if you want me to pick it up and work on a PR
yeah, go ahead and ping me in the PR.
I just thought of something which might make sense. We've modified the error message to make it more "actionable", and the user can set a deadline to avoid running into this. However, the error message is still only controller level AFAIK and thus not exposed to the user. From the user's perspective, there is not way to tell why the CronJob is not scheduling the next instance. For these reasons, maybe we should also emit an event in the corresponding CronJob? I can send a small PR for this modification.


Question: virsh pool-create-as $POOL dir --target "$POOL_PATH" fi <del> wget -N -P "$ROOT" http://${COREOS_CHANNEL:-alpha}.release.core-os.net/amd64-usr/current/coreos_production_qemu_image.img.bz2 </del> <ins> wget -N -P "$ROOT" https://${COREOS_CHANNEL:-alpha}.release.core-os.net/amd64-usr/current/coreos_production_qemu_image.img.bz2 </ins> if [[ "$ROOT/coreos_production_qemu_image.img.bz2" -nt "$POOL_PATH/coreos_base.img" ]]; then bunzip2 -f -k "$ROOT/coreos_production_qemu_image.img.bz2" virsh vol-delete coreos_base.img --pool $POOL 2> /dev/null || true

doesn't have a valid SSL certificate (it looks like it's a GCS bucket since says that I should be using a domain name). We should find a source for the image that can be fetch securely.
Split off of . /cc since I can't assign this to you directly.
you made a (meaningful) change in libvirt_coreos a few months ago. Does still work? Any idea on a better source?
works (at least for my needs) . I use it everyday for dev and I'd like to keep it working. I was not aware of this issue. Having a look and let you know.
I had a look at but I found no reference to maybe something changed since this issue is quite old. Could you check again and eventually share more details on the issue? Thanks in advance
Back in mid-2015 I was looking for places where we were fetching resources dynamically during cluster creation and trying to make sure that the fetches were done over SSL (to prevent a malicious actor from being able to inject code into new clusters) -- see The idea was to try and switch from to . If I run the wget command directly, it appears to work fine over SSL (and is actually way faster for me -- the http endpoint said it would take 3 hrs to download and the https endpoint took 12s).
I've created to switch it to https


Question: if err != nil { return err } <ins> if !found && metav1.HasAnnotation(volume.ObjectMeta, annBoundByController) { // If PV is bound by external PV binder (e.g. kube-scheduler), it's // possible on heavy load that corresponding PVC is not synced to // controller local cache yet. So we need to double-check PVC in //   1) informer cache //   2) apiserver if not found in informer cache // to make sure we will not reclaim a PV wrongly. // Note that only non-released and non-failed volumes will be // updated to Released state when PVC does not eixst. if volume.Status.Phase != v1.VolumeReleased && volume.Status.Phase != v1.VolumeFailed { obj, err = ctrl.claimLister.PersistentVolumeClaims(volume.Spec.ClaimRef.Namespace).Get(volume.Spec.ClaimRef.Name) if err != nil && !apierrs.IsNotFound(err) { return err } found = !apierrs.IsNotFound(err) if !found { obj, err = ctrl.kubeClient.CoreV1().PersistentVolumeClaims(volume.Spec.ClaimRef.Namespace).Get(volume.Spec.ClaimRef.Name, metav1.GetOptions{}) if err != nil && !apierrs.IsNotFound(err) { return err } found = !apierrs.IsNotFound(err) } } } </ins> if !found { glog.V(4).Infof("synchronizing PersistentVolume[%s]: claim %s not found", volume.Name, claimrefToClaimKey(volume.Spec.ClaimRef)) // Fall through with claim = nil

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: create a pod with local volume scheduler start to do bind, it will make pv = controller will call syncVolume to make pv bound , then syncClaim will make pvc bound. But from the log, I find that after step 2, pv controller start to reclaim the pv because pvc not in cache. logs: We can see after pv reclaimed, then the logs related with pvc start to print out. I know kube-scheduler will wait for the pvc to be bound, then it will be scheduled again in this situation. But is there a better way to handle this situation at pv_controller? What you expected to happen: PV not reclaimed. How to reproduce it (as minimally and precisely as possible): Create pod and local volume pvc in large scale Anything else we need to know?: Environment: Kubernetes version (use ): k8s 1.9
/sig storage Maybe syncVolume could wait to make PV bound until PVC exists. wdyt?
/assign
/assign
Perhaps we cannot wait to make PV bound until PVC exists, because syncVolume needs to release PV if PVC does not exist. Even if scheduler pre-binds a PV, when its PVC is deleted, PV controller should release PV. The problem is that after binding a PV scheduler waits for PV controller to bind PVC, but PV controller may not have corresponding PVC (not synced or deleted), so it will release PV. PV controller pre-binds a PV the same way as scheduler, but PV/PVC binding processes share same cache, so it will not have problem. To fix this, we can check PVC existence in apiserver instead of local cache in at the cost of performance. Possible optimizing is we can add a specific annotation for scheduler, then can check PVC existence in apiserver only when volume is bound by scheduler.
Another optimizing is that we can get PVC from apiserver only if PVC does not exist in cache. Double-check PVC from apiserver to make sure PV is not reclaimed on API sync delay.
We are doing the same way at our prod env.
Fix with the second optimizing is much simpler. I created a PR:
Next time, you can submit a PR if you have a fix for issues you encountered ;-) By the way, how many nodes in your large scale cluster?

For this issue, I think there should be solution which no need to touch api-server to fix this issue. That's why I didn't submit the PR. Our cluster has more than 1k BM nodes, and we have more than 10K local volume PV. This issue happens when I create 1000 pods(which use local volume) with a for loop script. It is easy to reproduce.
This issue happens when cache is out of sync (perhaps heavy load on apiserver), that would be better if we can fix it without touching apiserver.
I am not sure I fully understand the issue. How is it possible that PV controller does not see the PVC in its cache? It must have been created before pod was created and scheduler bound it to a PV. That's IMO long enough for a pod to be in the cache, especially when PV got there quick enough.
The issue is syncing PVC/PV caches across scheduler and pv controller. The PVs have been created long ago. Now a bunch of PVCs and Pods are created with delayed binding. Now, the scheduler will process the pod -pvc, and then bind PV by setting PV.ClaimRef. However, the PV controller could be slow to get API updates, and does not know about the new PVC object yet, but it got the PV update from the scheduler. So it sees that PV is bound and pointing to a ClaimRef, but PVC doesn't exist, so it goes into Release state.


Question: "fmt" "net" "net/url" <ins> "os" </ins> "path/filepath" "strings" "syscall"

No response No response /sig windows /kind failing-test
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign
/sig windows
i think this problem can be fixed when it imports os/net in Am i right?
I also found some other unit test bugs in CI, such as running linux-related tests on windows. I will try to fix them. /assign
/cc
Thanks for opening the issue! I didn't notice the failure through the testgrid dashboard. I've sent a PR here: . While fixing this issue, I noticed there's one unit test that was failing, so that PR also addresses that as well.
Yep, the imports you mentioned are needed. Keep in mind that imports are in alphabetical order. But in addition to that, there was another failing unit test which was masked by this import errors.


Question: } if len(subpathVolumePaths) > 0 { for _, subpathVolumePath := range subpathVolumePaths { <del> if err := syscall.Rmdir(subpathVolumePath); err != nil { </del> <ins> // Remove both files and empty directories here, as the subpath may have been a bind-mount of a file or a directory. if err := os.Remove(subpathVolumePath); err != nil { </ins> orphanVolumeErrors = append(orphanVolumeErrors, fmt.Errorf("orphaned pod %q found, but failed to rmdir() subpath at path %v: %v", uid, subpathVolumePath, err)) } else { klog.InfoS("Cleaned up orphaned volume subpath from pod", "podUID", uid, "path", subpathVolumePath)

when kubelet cleanupOrphanedPodDirsit may fail when the path is a file the log is ` when can see ,this path is a file ,not a directory kubelet can delete the file and path when deleting orphaned pod kubernetes 1.20.2+docker+nvidia-dockerand the pod has GPU allocated No response <details20.5 </details<detailsnone </details<details</details<details</details<details</details<details</details
kubelet only delete dirwhen the path is file ,it will fail my pod volume is like this
/sig node storage
/assign
Mounting a file using a subpath is fine, however I think mounting the same using twice isn't. Why do you need both? Can you try deleting and use in both places?
thanks, is this the reason for the problem?
It might be... can you do the test and confirm?
/triage needs-information let's wait for information on how things behave when not mounted twice
oki have a try
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
i delete cudagpu,but it still has same problem
Thanks let me come back to this soon...
Are you still using 1.20? Would it be possible to retry with a supported version of k8s, best would be 1.23?
using 1.20.
can you reproduce with 1.23? otherwise it'd be difficult to propose fixes to a k8s out of support.
ok. i will test in 1.23
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle rotten [1]:
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Reopen this issue or PR with Mark this issue or PR as fresh with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /close [1]:
Closing this issue. <detailsIn response to : The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closedReopen this issue or PR with Mark this issue or PR as fresh with Offer to help out with [Issue Triage][1] Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Hi, we got the same issue, could someone confirm whether this issue has been fixed? we're using 1.20.7.
/reopen I am able to reproduce it with v1.29, but only with dirty hacks. Something bad must happen during node reboot. a Pod that uses a CSI volume + file on the volume as a subpath. kubelet service on the node where the pod runs. -delete the Pod. remove from the pod local volume directory, i.e. . I don't know how to reproduce the issue without this step. How else can it happen that the file gets missing or its reconstruction fails?! reboot the machine.

After reboot, newly started kubelet will not see the Pod in the API server, and it will try to reconstruct its CSI volume mount from That file is missing, and therefore it will declare the volume as unmounted and cleaned, without cleaning its subpaths. The orphan cleanup routine will then try to remove subpaths of the volume, but it can handle only directories as subpath, not files -will fail.
Reopened this issue. <detailsIn response to : /reopen Stop kubelet service on the node. Force-delete the Pod. Manually remove from the pod local volume directory, i.e. . I don't know how to reproduce the issue without this step. How it can happen that the file gets missing or its reconstruction fails?! Hard reboot the machine. Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/remove-lifecycle rotten
/triage accepted
/assign


Question: } return -1 } <ins> func (s Status) String() string { if s == Healthy { return "healthy" } else if s == Unhealthy { return "unhealthy" } return "unknown" } </ins>

If the scheduler is not healthy, pods won't schedule and things will get stuck in pending. Right now, there is no user visible feedback (in logs or events or pod status) that indicates that this is true. We should really have at least some logging in the apiserver, and possibly more in-depth information. I'm thinking that we should add a /validatez handler to the apiserver that does a quick sanity check. Added bonus: we could rip out all of the scripted health checking in the startup scripts.
A simple status HTTP page would be useful, showing the health status of each system component. We should seriously think about self-hosting, too. It doesn't seem like it would be hard to run a couple scheduler replicas using master election, managed by a replication controller.
Yeah, I think system components should register themselves with apiserver.


Question: "io/ioutil" "net" "os" <ins> "reflect" </ins> "testing" certutil "k8s.io/client-go/util/cert"

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: Duplicate altnames in certs which generated by kubeadm ! What you expected to happen: No Duplicate altnames in certs which generated by kubeadm How to reproduce it (as minimally and precisely as possible): Anytime Anything else we need to know?: ! Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig cluster-lifecycle
/area kubeadm
Can you describe your detailed usage scenarios and reproduction steps, and explain the kubernetes version so that we can quickly locate
Thank for your response! I have found why and will fix it soon.
i have commented on the PR. /priority awaiting-more-evidence
/remove-priority awaiting-more-evidence /priority backlog


Question: }, "ReconcilerSyncLoopPeriod": { SchemaProps: spec.SchemaProps{ <del> Description: "ReconcilerSyncLoopPeriod is the amount of time the reconciler sync states loop wait between successive executions. Is set to 5 sec by default.", </del> <ins> Description: "ReconcilerSyncLoopPeriod is the amount of time the reconciler sync states loop wait between successive executions. Is set to 60 sec by default.", </ins> Default:     0, Ref:         ref("k8s.io/apimachinery/pkg/apis/meta/v1.Duration"), },

// RecommendedDefaultAttachDetachControllerConfiguration defaults a pointer to a // AttachDetachControllerConfiguration struct. This will set the recommended default // values, but they may be subject to change between API versions. This function // is intentionally not registered in the scheme as a "normal" // function to allow consumers of this type to set whatever defaults for their // embedded configs. Forcing consumers to use these defaults would be problematic // as defaulting in the scheme is done as part of the conversion, and there would // be no easy way to opt-out. Instead, if you want to use this defaulting method // run it in your wrapper struct of this type in its method. func RecommendedDefaultAttachDetachControllerConfiguration(obj kubectrlmgrconfigv1alpha1.AttachDetachControllerConfiguration) { zero := metav1.Duration{} if obj.ReconcilerSyncLoopPeriod == zero { obj.ReconcilerSyncLoopPeriod = metav1.Duration{Duration: 60 time.Second} } } Default is set to 60 second. but, the comment said is 5 seconds. // AttachDetachControllerConfiguration contains elements describing AttachDetachController. type AttachDetachControllerConfiguration struct { // Reconciler runs a periodic loop to reconcile the desired state of the with // the actual state of the world by triggering attach detach operations. // This flag enables or disables reconcile. Is false by default, and thus enabled. DisableAttachDetachReconcilerSync bool // ReconcilerSyncLoopPeriod is the amount of time the reconciler sync states loop // wait between successive executions. Is set to 5 sec by default. ReconcilerSyncLoopPeriod metav1.Duration } default should change to 60 seconds. comment errror None <detailsall version until now </details<detailsnot relevent </details<details</details<details</details<details</details<details</details
/sig storage
/remove-kind api-change /kind cleanup
Those labels are not set on the issue: <detailsIn response to : /remove-kind api-change /kind cleanup Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/triage accepted


Question: allErrs = append(allErrs, ValidateResourceRequirements(&ctr.Resources, idxPath.Child("resources"))...) allErrs = append(allErrs, ValidateSecurityContext(ctr.SecurityContext, idxPath.Child("securityContext"))...) } <del> // Check for colliding ports across all containers. allErrs = append(allErrs, checkHostPortConflicts(containers, fldPath)...) </del> <ins> if isInitContainers { // check initContainers one by one since they are running in sequential order. for _, initContainer := range containers { allErrs = append(allErrs, checkHostPortConflicts([]core.Container{initContainer}, fldPath)...) } } else { // Check for colliding ports across all containers. allErrs = append(allErrs, checkHostPortConflicts(containers, fldPath)...) } </ins> return allErrs }

I found that kubelet will not allocate hostports specified from initContainers. And PodFitsHostPorts in kube-scheduler does not take initContainer into consideration. Also, kube-apiserver think such pod is illegal, but I think it OK. We should make the behavior of spec.HostPorts in initContainer clear
/sig node
Exactly. I think this makes sense. Since initContainers are expected to be completed, we should not expose extra port mappings during creating pod sandbox, which may bring in security risks.
Sounds good to me.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale


Question: // to JSON allowed. type Unstructured interface { Object <del> // UnstructuredContent returns a non-nil, mutable map of the contents of this object. Values may be </del> <ins> // UnstructuredContent returns a non-nil map with this object's contents. Values may be </ins> // []interface{}, map[string]interface{}, or any primitive type. Contents are typically serialized to <del> // and from JSON. </del> <ins> // and from JSON. SetUnstructuredContent should be used to mutate the contents. </ins> UnstructuredContent() map[string]interface{} // SetUnstructuredContent updates the object content to match the provided map. SetUnstructuredContent(map[string]interface{})

What happened: In I introduced an optimization - do a deep copy instead of to and from JSON roundtrip to convert anything that implements . I then discovered that the method that is used there in both and may mutate the original object. This is problematic because previously (before ) there was no mutation and because this is unexpected and may lead to data races - it is bad behaviour to mutate original object when you just want a copy of it. This issue was fixed by . However the underlying issue is not fixed here - is brittle and dangerous. Method name does not imply that it mutates data when you call it. And godoc does not mention that either: Something needs to be done about it IMO. This method is misused in these places (mutation side effect is not intended): Also implementation in does not implement the behaviour required by godoc in . What you expected to happen: There should be a method that returns the current unstructured contents without mutation and perhaps not providing the write-through effect. /kind bug /sig api-machinery
/cc
Is this still required and could I work on it? I'm looking for something to work on as a new contributor :)
Yes and yes! :)
Perfect, I'll get started on it!
I've got out. I'd appreciate a review :)
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale


Question: for _, tc := range cases { t.Run(tc.name, func(t *testing.T) { <del> var mu sync.Mutex </del> // localAddr indicates how many TCP connection set up localAddr := make([]string, 0) <del> client, err := newGOAWAYClient(func(network, addr string, cfg *tls.Config) (conn net.Conn, err error) { </del> <ins> // create the http client dialFn := func(network, addr string, cfg *tls.Config) (conn net.Conn, err error) { </ins> conn, err = tls.Dial(network, addr, cfg) if err != nil { t.Fatalf("unexpect connection err: %v", err) } <del> mu.Lock() </del> localAddr = append(localAddr, conn.LocalAddr().String()) <del> mu.Unlock() </del> return <del> }) if err != nil { t.Fatalf("failed to set-up client, err: %v", err) </del> <ins> } tlsConfig := &tls.Config{ InsecureSkipVerify: true, NextProtos:         []string{http2.NextProtoTLS}, } tr := &http.Transport{ TLSHandshakeTimeout: 10 * time.Second, TLSClientConfig:     tlsConfig, // Disable connection pooling to avoid additional connections // that cause the test to flake MaxIdleConnsPerHost: -1, DialTLSContext: func(ctx context.Context, network, addr string) (net.Conn, error) { return dialFn(network, addr, tlsConfig) }, } if err := http2.ConfigureTransport(tr); err != nil { t.Fatalf("failed to configure http transport, err: %v", err) } client := &http.Client{ Transport: tr, </ins> } watchChs := make([]<-chan watchResponse, 0)

Before merged, TestClientReceivedGOAWAY failed with data race on . However, as far as I know, it's not the root cause of TestClientReceivedGOAWAY flake, because dial a new connection only when client send a new request, and in this case, requests are sent one by one. In other words, if data race on , then the additional unexpected connection set up, which means the test case or the configuration has problem. After merged, TestClientReceivedGOAWAY still failed in many jobs, such as , , which may be a good proof of my suspect. And I got a failed job with data race issue that read at (main testing goroutine) and write at , which told us the http client try to set-up a new connection even when we had already finished all requests(may caused by goroutine leak/block in http client). I'm still debug the root cause of TestClientReceivedGOAWAY flake, for now, I doubt that I used an incorrect http2 server/client configuration( or SETTINGSMAXCONCURRENT_STREAMS limited), or the dial function ignored context.Context which will cause more than one goroutine dial the connection when the first canceled(not found any where will cancel the requests). I still can not reproduce this failed case and it's hard to find the root cause. Couple of days may be needed to find the root cause, and I will post a PR to skip TestClientReceivedGOAWAY before I solved the problem. Sorry for any inconvenient.
/assign /sig testing
cc
Thank you for working on this!! How often does it flake? I'm just wondering if we should disable it in the mean time to avoid angry contributors.
On my PC, about 1/1000. I will post a PR to disable it for now.
Thank you so much! That's not a bad rate at all, but we run a lot of tests. This is super helpful!
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or .

/lifecycle stale
/lifecycle stale I'm still working on it, a new golang version(1.16 or 1.15.7+) release required
/sig api-machinery /remove-sig testing AFAICT this is specific to these tests, not the test framework or CI infrastructure.
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
/remove-lifecycle rotten
the tests re-enabled in started flaking again immediately Example failure: /milestone v1.21 the tests should be deflaked or disabled for 1.21
/cc
/triage accepted
+1 in recent PR tests
Sorry for my slow response. I'm working on it, looks like I need to debug golang HTTP client again :-P
Shall we revert in the meantime?
No, has already fixed by golang 1.15.8 I think. This one is caused by another problem.
I will disable TestClientReceivedGOAWAY case again if I can not find the root cause this night.
Can not reproduce on my PC, disable this test case before I figure out the root cause:
I run this test case with this command and can not reproduce. Does any thing different with command ?
moving out of the milestone and lowering priority becuase merged, leaving open because the fundamental issue is unresolved /milestone clear
will do something comparable under make. There's a bunch of stuff in there, generally it's not relevant for a unit test bug like this but the answer to what exactly it does is not easily concise. Note that CI may have more load than your local machine running it once, and also currently runs as root.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-contributor-experience at . /lifecycle stale
/lifecycle active /assign ref:


Question: 19: "3.4.13-0", 20: "3.4.13-0", 21: "3.4.13-0", <del> 22: "3.5.3-0", 23: "3.5.3-0", 24: "3.5.3-0", </del> <ins> 22: "3.5.4-0", 23: "3.5.4-0", 24: "3.5.4-0", 25: "3.5.4-0", </ins> } // KubeadmCertsClusterRoleName sets the name for the ClusterRole that allows

Please publish 3.5.4-0 image and update to to benefit from fix for , while still keeping v3.5.3 critical correctness fixes (, ). For recent PRs for update to etcd v3.5.3 see and
/sig release /sig api-machinery /kind bug
/cc /triage accepted
Hey, Can I work on this ? I'm a beginner, but all the good first issues are mostly assigned. Can you guide me on doing this ?
I am working on this. /assign
Hi I would like to work on Make sense to backport in v1.22, v1.23 & v1.24? Thanks!!
hi, it makes sense to me, but requires approval from dependency maintainers. i can only approve the kubeadm changes.
was this backported ?


Question: cmd.Flags().Var(&options.server, clientcmd.FlagAPIServer, clientcmd.FlagAPIServer+" for the cluster entry in kubeconfig") cmd.Flags().Var(&options.apiVersion, clientcmd.FlagAPIVersion, clientcmd.FlagAPIVersion+" for the cluster entry in kubeconfig") <del> cmd.Flags().Var(&options.insecureSkipTLSVerify, clientcmd.FlagInsecure, clientcmd.FlagInsecure+" for the cluster entry in kubeconfig") </del> <ins> f := cmd.Flags().VarPF(&options.insecureSkipTLSVerify, clientcmd.FlagInsecure, "", clientcmd.FlagInsecure+" for the cluster entry in kubeconfig") f.NoOptDefVal = "true" </ins> cmd.Flags().Var(&options.certificateAuthority, clientcmd.FlagCAFile, "path to "+clientcmd.FlagCAFile+" for the cluster entry in kubeconfig") <del> cmd.Flags().Var(&options.embedCAData, clientcmd.FlagEmbedCerts, clientcmd.FlagEmbedCerts+" for the cluster entry in kubeconfig") </del> <ins> f = cmd.Flags().VarPF(&options.embedCAData, clientcmd.FlagEmbedCerts, "", clientcmd.FlagEmbedCerts+" for the cluster entry in kubeconfig") f.NoOptDefVal = "true" </ins> return cmd }

It acts like a boolean flag but requires an argument, unlike boolean flag is pretty much every environment. We need to find a better way to express the things that this is expressing. Maybe a add a tristate to the pflag lib that behaves the same way but doesn't demand an argument?
cc
The handles "true", "false", "unset" and has a default value. I seem to recall you were looking at changing it, but we needed to preserve the value. The complain here is that you have to do instead of just . /cc
fixing is really easy....


Question: type fakeRoutesClient struct { mutex     *sync.Mutex FakeStore map[string]map[string]network.Route <ins> Calls     []string </ins> } func newFakeRoutesClient() *fakeRoutesClient {

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: Kubernetes shouldn't put route table when it's already been configured. Azure cloud provider should check whether the route has already been created before invoking . What you expected to happen: How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others: /sig cloud-provider /priority important-soon
Why this is a bug? A PUT operation should be idempotent and multiple calls should get the same result?
Right. But cloud provider shouldn't make unnecessary API calls.
To check whether it is configured, we need a GET call, so we're going to replace PUT call with GET call in this case?
it's not a replace, we need to LIST routes and check whether it has already been there, then PUT if it's not.
I got your point. But if we directly send a PUT request, seems route RP will also do this comparation? Is there any issue making us have to do the comparation by ourselves instead of sending PUT request to route RP?
It would help to reduce the chances of ARM PUT throttling.
I'd like to work on this
/assign


Question: // update the kubelet config with old values updateKubeletConfig(f, oldCfg) <ins> // wait until the kubelet health check will pass and will continue to pass for specified period of time gomega.Eventually(func() bool { return kubeletHealthCheck(kubeletHealthCheckURL) }, time.Minute, 10*time.Second).Should(gomega.BeTrue()) gomega.Consistently(func() bool { return kubeletHealthCheck(kubeletHealthCheckURL) }, time.Minute, 10*time.Second).Should(gomega.BeTrue()) </ins> }) ginkgo.Context("with static policy", func() {

<!-- Please only use this template for submitting reports about flaky tests or jobs (pass or fail with no underlying change in code) in Kubernetes CI --kubelet-serial-gce-e2e-memory-manager no specific test, because the failure happens during waiting for the kubelet to come back after the restart the memory manager under the teardown, updates the kubelet configuration via dynamic configuration - because of the update, the kubelet health check is not responding for some short time period - e2e node service restart the kubelet - next memory manager test started and it stops the kubelet - the e2enode service fails on the initial health check - to appreciated links to specific failures in spyglass appreciated <!-- Please see the deflaking doc () for more guidance! --
/sig node
/triage accepted


Question: o.config.HealthzBindAddress = fmt.Sprintf("%s:%d", o.config.HealthzBindAddress, o.healthzPort) } <ins> func (o *Options) applyDeprecatedMetricsPortToConfig() { if o.metricsPort == 0 { o.config.MetricsBindAddress = "" return } index := strings.Index(o.config.MetricsBindAddress, ":") if index != -1 { o.config.MetricsBindAddress = o.config.MetricsBindAddress[0:index] } o.config.MetricsBindAddress = fmt.Sprintf("%s:%d", o.config.MetricsBindAddress, o.metricsPort) } </ins> // loadConfigFromFile loads the contents of file and decodes it as a // KubeProxyConfiguration object. func (o *Options) loadConfigFromFile(file string) (*kubeproxyconfig.KubeProxyConfiguration, error) {

What happened: From We used to allow using or in and . Now the validation on kubeproxyconfig disallow that (it only allows ): This seems to break backward compatibility for those two flags. Please comment if I'm missing something. cc How to reproduce it (as minimally and precisely as possible): Try with kube-proxy. Environment: Kubernetes version (use ):
/sig network /kind bug
Tried , the error is: and , the error is: So maybe we need to loosen and revert ?
Loosen the validation sounds good. was meant for loosen the validation as well (allowing not just ip, but also ip:port), but for some reason it doesn't work as intended. Would be great to fix that instead of revert :)
find the root cause of not working as expected. introduced the regression: I send a PR to fix the two issues we mentioned in this thread.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-testing, kubernetes/test-infra and/or . /close
Closing this issue. <detailsIn response to : Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with .

/close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I believe this hasn't been fixed yet. /remove-lifecycle rotten /reopen
Reopened this issue. <detailsIn response to : I believe this hasn't been fixed yet. /remove-lifecycle rotten /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I agree There's also the issue of IPv6-only: Results in:
Thanks for your report. Do you mind opening a new issue regarding IPv6 since the original issue was fixed? Thanks.
Sure. Thanks
Please see:


Question: "k8s.io/klog" ) <del> const ( // period to relist statefulsets and verify pets statefulSetResyncPeriod = 30 * time.Second ) </del> // controllerKind contains the schema.GroupVersionKind for this controller type. var controllerKind = apps.SchemeGroupVersion.WithKind("StatefulSet")

<!-- Please only use this template for submitting enhancement requests --/area controller-manager What would you like to be : Don't resync statefulset in statefulset controller. Why is this needed: There are 2000+ sts in my k8s cluster, thus when i create a new sts, it takes a lot of time to create it's pod, i found that every sts will cost 40ms to sync it, then 2000 sts will cost 80s, and there is also a statefulSetResyncPeriod to resync all sts, I don't know why there need to have statefulSetResyncPeriod to resync all sts? Can we not resync sts and just watch it's changes?
/sig api-machinery
/remove-sig api-machinery /sig apps /sig scalability
the resync pattern is followed by all controllers to account for bugs in watch mechanism or missed notification due to any other reason. I dont think the individual controllers resync period is configurable or we could increase it . Also i am not sure if the current scale limits of 2000 StatefulSets is supported FYI
I have three cluster, A has 20 sts, B has 1300 sts and C has 2300 sts. I set kube-controller-manager loglevel to 4 and find that the cost of processing every sts is 2ms,30ms,45ms. Can we record the processed object into a processed cache and every time we get a new object to precess, we can compare it with the item in the cache, if they are same, then we don't need to sync it.
The "resync pattern" doesn't help with watch bugs (resync is done purely from internal cache, so in hypothetical situation that we missed watch event, it won't help at all). Resync-period should basically be used only in controllers that also sync some external things (like load-balancers), and watching changes of those is impossible. Unless I'm not missing anything, resync period can be safely dropped in this case.
I think all the controllers have the same problem because of the sharedinformerfactory has a default resync period if we don't set it to 0 in the config file. Also if there are more sts, the consumption for every sts will be larger.
interesting, but the resync is only happening for k8s objects.

Why is watching changes on those impossible ? And when you say external things you mean loadbalancers in gcloud ? resync is only happening for k8s services corresponding to those loadbalancers , right ? Or may be you are saying that if a k8s services is linked to a gcloud loadbalancer, the resync enables the controller code to go and check gcloud if the loadbalancing is provisioned or has anything changed there ? We need to document this in somwhere much like the apiconventions doc
- I mean loadbalancer in GCP as an example (but can also be other load balancer, persistent volumes in cloud or things like that). Yes - we resync on k8s objects, but that that let's you sync those. As an example - imagine service of type LoadBalancer - if you're periodically syncing the k8s service object, this triggers the operation of syncing the load-balancer (and we have an ability to check if it exists, is healthy etc.). - hmm, I would need to double check it, but I thought the default resync period is pretty long...
The default resync period is 12~24 hours. It is long but the bad experience will still happen every day.


Question: import ( "fmt" "path" <del> "strings" </del> "time" "k8s.io/api/core/v1"

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . If the matter is security related, please disclose it privately via --Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: Table-based e2e tests are not getting updated to leading to the following error- ( mentioned in ) framework.ConformanceIt() requires the first parameter to be and are associated with at runtime based on different cases/test-scenarios. What you expected to happen: Either disapprove the use of table-based e2e tests or suggestion on an alternative approach to add such tests to Conformance. How to reproduce it (as minimally and precisely as possible): framework.ConformanceIt() to . /area conformance cc
/sig testing
/cc
Approaches to work around this: wrap the whole table based test in a ConformanceIt block refactor to a more boilerplate approach that has a ConformanceIt block for each table entry update the conformance test walker to detect table based tests that use a well known field?
Thanks I am evaluating these approaches especially on "conformance test walker" to retrieve well-known fields like and variables used inside .
Does this close if closes or are there other tests that need to transition?
This issue would be No other e2e test needs the transition from table-based test structure. Though "only one" remaining test should be restructured as an individual test.
/close Closing as per other fruitful approach and efforts would be taken to refactor table-based tests into individual tests wherever needed.
Closing this issue. <detailsIn response to : /close Closing as per other fruitful approach and efforts would be taken to refactor table-based tests into individual tests wherever needed. Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: return pod } <del> // TestCleanup cleans both server and client pods. func TestCleanup(f *framework.Framework, config TestConfig) { </del> <ins> // TestServerCleanup cleans server pod. func TestServerCleanup(f *framework.Framework, config TestConfig) { </ins> ginkgo.By(fmt.Sprint("cleaning the environment after ", config.Prefix)) defer ginkgo.GinkgoRecover() <del> cs := f.ClientSet err := e2epod.DeletePodWithWaitByName(cs, config.Prefix+"-client", config.Namespace) gomega.Expect(err).To(gomega.BeNil(), "Failed to delete pod %v in namespace %v", config.Prefix+"-client", config.Namespace) if config.ServerImage != "" { err := e2epod.DeletePodWithWaitByName(cs, config.Prefix+"-server", config.Namespace) gomega.Expect(err).To(gomega.BeNil(), "Failed to delete pod %v in namespace %v", config.Prefix+"-server", config.Namespace) </del> <ins> if config.ServerImage == "" { return </ins> } <ins> err := e2epod.DeletePodWithWaitByName(f.ClientSet, config.Prefix+"-server", config.Namespace) gomega.Expect(err).To(gomega.BeNil(), "Failed to delete pod %v in namespace %v", config.Prefix+"-server", config.Namespace) </ins> } func runVolumeTesterPod(client clientset.Interface, config TestConfig, podSuffix string, privileged bool, fsGroup *int64, tests []Test) (*v1.Pod, error) {

<!-- Please only use this template for submitting enhancement requests --What would you like to be : In block volume tests, wait for the Pod to be deleted instead of letting it be handled by the framework's namespace deletion Why is this needed: I think the regression caused by could have been caught if the tests check for Pod deletion because the Pod object shouldn't be deleted if TearDown fails.
/assign
This issue is not specific to block tests. We should in ideally wait for pod deletion at the end of all types of tests to confirm cleanup logic is functioning
I checked the tests under test/e2e/storage/testsuites/ where DeletePodWithWait is used to wait for the deletion.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
/remove-lifecycle rotten is this still an issue? One way to check is to make a breaking change in cleanup and see if the pull job fails.
I went through all block volume usage in e2e tests: All of them but one wait for their pods to be deleted. I'm fixing the remaining test in


Question: return err } <del> if err := clientcmd.ModifyConfig(o.configAccess, *config, false); err != nil { return err } return nil </del> <ins> return clientcmd.ModifyConfig(o.configAccess, *config, false) </ins> } func (o *unsetOptions) complete(cmd *cobra.Command) error { endingArgs := cmd.Flags().Args() if len(endingArgs) != 1 { <del> cmd.Help() return fmt.Errorf("Unexpected args: %v", endingArgs) </del> <ins> return helpErrorf(cmd, "Unexpected args: %v", endingArgs) </ins> } o.propertyName = endingArgs[0]

The '-context' does not seem to add any value
add priority and team please.
Blocked on
/sig cli


Question: // We must avoid syncing service until the pod store has synced. If it hasn't synced, to // avoid a hot loop, we'll wait this long between checks. PodStoreSyncedPollPeriod = 100 * time.Millisecond <ins> // An annotation on the Service denoting if the endpoints controller should // go ahead and create endpoints for unready pods. This annotation is // currently only used by PetSets, where we need the pet to be DNS // resolvable during initialization. In this situation we create a headless // service just for the PetSet, and clients shouldn't be using this Service // for anything so unready endpoints don't matter. TolerateUnreadyEndpointsAnnotation = "service.alpha.kubernetes.io/tolerate-unready-endpoints" </ins> ) var (

This is mostly motivated by petset. Currently we create a governing service that is responsible for the network identity of each pet. The requirements of this governing service differ from a normal service in the following ways: have internal protocols to notice peer health, and a list of different checks (eg: ). In kube, peers use DNS to communicate. So we don't want a readiness check failure to remove the pet from DNS and abruptly cut it off from the set. "governing service" is headless so kube-proxy doesn't need to NAT the service ip. will not be using this Service, so it doesn't matter that unready pets show up in DNS under the svcName sub domain. In addition to these, there's one important reason some pets need to have DNS before readiness. To provide the guarantees required to safely bootstrap a pet, we need: 0 starts before pet1.. 't start pet1 till pet0 is running/ready pet1 starts, it sees pet0 in DNS The easiest way to guarantee 3 today is to do one of: poll in the entrypoint of pet0, till it sees itself poll in the init container of pet0, till it sees itself poll in the init containers of pet {1..N-1} till it sees all previous pets Third option is not a great idea because we don't know who else is supposed to exist, and for the first 2 to work with a readiness probe, we need a DNS record before the Service is ready.
I have a wip that does this through an annotation:
+1. this is useful even in traditional n rc's and n+1 services setup for clustered systems like n nodes zookeeper/etcd. Currently, there isn't a easy way to specify a readiness check in such setups, because the check in these cases would look for whether the pod member has joined the cluster successfully as follower, which wouldn't work as they discover each other through them appearing in service endpoints. this prevents one from being able to do a rolling upgrade of cluster members safely, because there is no readiness feedback after a member pod is upgraded.
Suggestion to hold this for alpha in 1.5, and in 1.6 spend a bit of time talking about how services are best intermixed with stateful sets (there's a lot of correlation between them).

That avoids us breaking people in 1.5 and gets a bit more think time.
SGTM
See also
Can we claim that if a pod is requesting pet DNS (i.e DNS name via the hostname/subdomain annotation) it never wants kube-style readiness, because it has its own internal protocol for peer health? If that's true we can get rid of the annotation and teach kubedns to always serve records for pet DNS ignoring readiness. My intuition tells me the answer to that question is no (i.e there exist pets that use DNS for discovery, and don't have an internal protocol for readiness, so they must use kube readiness to remove themselves from the peer group), and so the ability to toggle it on/off is useful, but I also want others to consider it.
I'd love to nix an annotation but I have not the expertise to answer this one.
I don't have a strong feel either way - I'd say readiness controls on a service feel orthogonal to independent names though. Since we don't have per port readiness, we need to consider that at some point we might need more sophistication on services or pods to control membership (unless we make label flipping easier). I would like to see us tease apart the "leader / follower" service member concept a bit more because that might inform how we handle readiness (seems like a separate bit, but we may want atomic changes).
They are somewhat orthogonal, but you presumably only want to discover healthy peers, unless you have a way to probe every peer you discover for health. Applying this policy based on endpoint info also feels a little weird, because it's harder to create overlapping Servics with different behavior. I think doing the common thing always and by default is fine, as long as we can annotate our way back to what we have today, but what form would the opposite of today's Service level annotation take? i.e something that says "even though my endpoints need pet names, apply the regular rediness behavior to their DNS records"
How bad would it be to start without an annotation, assume that pet-names == tolerate-unready, and see if anyone screams? Who should we consult?
SGTM, that's also what I was prposing in the previous comment
I guess the question is why do we even want to do that? Why make a change that couples logic vs leaving as is with decoupled logic?

We have people who want unready endpoints for deployments to form gossip pools - so I guess that counts as a use case where pet names is not coupled to unready. Is leaving the annotation on for another release really a problem? , Prashanth B <:
That's exactly the kind of use case I was looking for, something that doesn't require pet names. I get the feeling there will be other apps but I couldn't produce an example.
JBoss EAP uses it to form initial quorum for jgroups, which isn't a strong consensus system. So they use deployments and a minimum set to form an initial quorum and deal with split brain like jgroups normally does (infinispan has multiple consistent modes, some of which might prefer loose pools). Arguably smaller sets and some infinispan quorums might prefer statefulsets, but those use cases are also more tolerant of partitions in general. , Prashanth B <:
we are using tolerate-unready-endpoints annotation in our zookeeper deployments (not using pet/stateful set). Without unready endpoints, it is impossible to define a meaningful pod readiness check for zk, and without a readiness check, it is hard to do rolling updates (one ensemble member at a time).
The question is whether there's a need to have StatefulSets without tolerate-unready-endpoints. It sounds like maybe yes, which makes me grumpy, but I'll acquiesce if Clayton and Prashanth say so. , ravilr <:
isn't it enough to establish a need to tolerate-unready without statefulsets?
I have two services pointing to my etcd stateful set - one that is endpoints (peers) and tolerates unready. I have one that is for naive clients that uses readiness. On Dec 1, 2016, at 2:59 AM, Prashanth B <: isn't it enough to establish a need to tolerate-unready without statefulsets? — You are receiving this because you commented. Reply to this email directly, view it on GitHub , or mute the thread
fine.
/sig network
Right. This is exactly why etcd operator uses the alpha feature. Can we make it beta in 1.8 release?
/cc
We had discussed promoting since we need it for StatefulSet and no one has come up with anything better.

We should make a decision and include it in our GA roadmap for workload controllers.
Be sure to include sig-network On Jul 6, 2017, at 2:40 PM, Anthony Yeh <: had discussed promoting tolerate-unready-endpoints since we need it for StatefulSet and no one has come up with anything better. We should make a decision and include it in our GA roadmap for workload controllers. — You are receiving this because you commented. Reply to this email directly, view it on GitHub , or mute the thread
This is also useful for ingresses that need to retain the source IP using OnlyLocal node ports, which is important for firewalling with things like Calico. The problem that we see now is when a deployment occurs, the pod is removed from the node port before the PreStop completes, which doesn't give the upstream AWS load balancer time for the pod to fail out using normal health checks.
/close
Headless Services are a thing now


Question: } ipvsModules := utilipvs.GetRequiredIPVSModules(kernelVersion) <del> builtinModsFilePath := fmt.Sprintf("/lib/modules/%s/modules.builtin", kernelVersionStr) b, err := ioutil.ReadFile(builtinModsFilePath) if err != nil { klog.Warningf("Failed to read file %s with error %v. You can ignore this message when kube-proxy is running inside container without mounting /lib/modules", builtinModsFilePath, err) } </del> var bmods []string <del> for _, module := range ipvsModules { if match, _ := regexp.Match(module+".ko", b); match { bmods = append(bmods, module) } } // Try to load IPVS required kernel modules using modprobe first for _, kmod := range ipvsModules { err := handle.executor.Command("modprobe", "--", kmod).Run() if err != nil { klog.Warningf("Failed to load kernel module %v with modprobe. "+ "You can ignore this message when kube-proxy is running inside container without mounting /lib/modules", kmod) } } // Find out loaded kernel modules </del> <ins> // Find out loaded kernel modules. If this is a full static kernel it will thrown an error </ins> modulesFile, err := os.Open("/proc/modules") if err != nil { <ins> klog.Warningf("Failed to read file /proc/modules with error %v. Kube-proxy requires loadable modules support enabled in the kernel", err) </ins> return nil, err }

<Reopened this valid and in-demand bug after the auto-bot expired it/kind bug /sig network /area kube-proxy What happened: Building a fully static Linux kernel with kubeproxy's desired IPVS (Virtual Server) modules linked statically: CONFIGIPVS=y CONFIGIPVSRR=y CONFIGIPVSWRR=y CONFIGIPVSSH=y CONFIGNFCONNTRACKIPV4=y Causes kubeproxy to believe that these features don't exist because it can't find modules for them: What you expected to happen: If Kubeproxy's desired kernel features are linked statically in the kernel, kubeproxy should carry on and not attempt and throw error messages about modules not being found. Under the hood, kubeproxy should simply check for the existence of the desired kernel features in the /proc tree (ie: , and others) before attempting to look for and load modules. If, after module loads fails, then throw the error messages and switch to iptables. How to reproduce it (as minimally and precisely as possible): and extract the current stable kernel source version (today it's 4.18.9) from your existing running into the source tree as and answer any questions to switch to a fully static kernel and ensure the following exist, and are not : and deselect (box is empty) your boot loader's config () or () depending on your distribution. and select your new kernel with the kubeproxy modules linked statically. kubeproxy and watch it throw the above errors about not finding the modules. Anything else we need to know?: Fully static kernels will not generate so inspecting that file will fail - as well in the kubeproxy source. The gold-standard is to simply check for the features in the active tree first, and consider those as "built-in". The next best options is checking for statements, which tells you if they're built-in. Note that files are almost universally installed by OS maintainers. And finally, you could try checking , which is a compressed version of the externally held .config text file, in a similar manner noted above, however it's up to the maintainer to build that feature into the kernel (so it might not exist).

For reference, here's the list of modules kubeproxy wants: Here we see kube_proxy's errant module-centric code that needs to be made compatible w/ static kernels: Environment: Kubernetes version (use ): 1.11.3 Cloud provider or hardware configuration: On premise OS (e.g. from /etc/os-release): Leap 15.0 Kernel (e.g. ): 4.18.9 statically linked Install tools: gcc 8.1 Others: <reopened as it appears still unfixed and in-demandSee how far the team got on it before it was auto-closed here:
/triage unresolved Comment when the issue is assessed and confirmed. I am a bot run by vllry.
Hi is the issue just the log messages, or do the kube-proxy features actually fail?
If this issue has been triaged, please comment . If you aren't able to handle this issue, consider unassigning yourself and/or adding the label. I am a bot run by vllry.
/unassign
/remove-triage unresolved
To check kernel-modules instead of functions is not limited to in k8s. I build my own kernel which was static from the start (with ipvs support). It was a very tiresome trial-and-error time to find out which functions k8s is checking by it's modules. But those who build the Linux kernel probably can probably accept less "help", so a flag to the apropriate k8s programs may be a solution.
I've checked the code, this is only a warning. Maybe while loading the modules, this function could be reduced to something like "check if this exists exists in the kernel builtin modules and append it into the mod list, otherwise try to modprobe it". I've made a PR to it, let's see if this solves the "problem"
/assign
Those are not options, as compilling the kernel doesn' t imply that one will copy the config file into /boot after recompiling it. As an example, the step ` is optional. So the only way right now I've found to be plausible is to verify the existence of the related but what happens if this path changes in the future?

take a look at to see if this is some way to follow Don't think adding a new flag is a good idea (but I might be wrong either, and TBH I've tried ;) )
rikatz, that looks like a reasonable approach. Although the kernel API isn't standardized, the developers do treat /proc as close to stable as possible, and only change existing files and structures as a last resort (they realize that change here ripple across critial userspace, including stuff like systemd). So I think checking proc for the desired features is the first and most reliable approach and then falling back to those in the order that you've suggested in your write up. The error message can even indicate the name of the /boot/config-* file, to help those building their own kernels who might have forgot to provide a correspondingly-named config.
/reopen The patch was a partial solution, We still need to verify if the module loader will support kernels with all builtin
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale Em ter, 14 de jan de 2020 às 07:32, fejta-bot <:


Question: sed -i -e "s@{{ *pillar['dns_domain'] *}}@${DNS_DOMAIN}@g" "${coredns_file}" sed -i -e "s@{{ *pillar['dns_server'] *}}@${DNS_SERVER_IP}@g" "${coredns_file}" sed -i -e "s@{{ *pillar['service_cluster_ip_range'] *}}@${SERVICE_CLUSTER_IP_RANGE}@g" "${coredns_file}" <ins> if [[ "${ENABLE_DNS_HORIZONTAL_AUTOSCALER:-}" == "true" ]]; then setup-addon-manifests "addons" "dns-horizontal-autoscaler" "gce" local -r dns_autoscaler_file="${dst_dir}/dns-horizontal-autoscaler/dns-horizontal-autoscaler.yaml" sed -i'' -e "s@{{.Target}}@${COREDNS_AUTOSCALER}@g" "${dns_autoscaler_file}" fi </ins> } # Sets up the manifests of Fluentd configmap and yamls for k8s addons.

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: e2e test "gci-gce" is running in 2 versions: with kube-dns as DNS Service : https://k8s- with CoreDNS as DNS Service : https://k8s- In CoreDNS use case the test is failing What you expected to happen: This e2e tests should not depend on what DNS service is running as default in K8s. CoreDNS is currently BETA, will move to GA soon (after v1.10 though) See : How to reproduce it (as minimally and precisely as possible): look at logs of dashboard : https://k8s- go run -v --test --test_args='--[sig-autoscaling]sDNSshorizontalsautoscalings[DisabledForLargeClusters]skube-dns-autoscalersshouldsscaleskube-dnsspodssinsbothsnonfaultysandsfaultysscenarios$' Anything else we need to know?: Environment: Kubernetes version (use ): branch master Cloud provider or hardware configuration: GCE Install tools: deployment intra-test Others: see https://k8s-
/sig autoscaling
/unsig cli
Looks like the test is searching specifically for a deployment labeled "kube-dns". There are several references to "kube-dns" scattered around it, actually.
Right now the tool "dns-horizontal-autoscaller" seems dedicated for kube-dns. This tools is deployed as "add-on" when enabled by this ENV variable : I guess we should have the definition of another dns-horizontal-autoscaller, dedicated to coredns (unless we generate the yaml file). and the choice of deployment is based on the ENV variable . see
Currently, the translation of the deployment of "dns-horizontal-autoscaller" for CoreDNS is in progress by It is expected that the corresponding e2e test will work as is (as the artifacts are selected by labels, which are the same for CroeDNS and kube-dns).


Question: NumStats:               3, NumSamples:             2, CpuUsagePercentiles:    []int{10, 50, 90}, <del> MemoryUsagePercentages: []int{10, 80, 90}, </del> <ins> MemoryUsagePercentiles: []int{10, 80, 90}, </ins> } containerName := "/some/container" cinfo := itest.GenerateRandomContainerInfo(containerName, 4, query, 1*time.Second)

release-master-informing capg-conformance-main-ci-artifacts Kubernetes e2e suite.BeforeSuite 05-25 06:37 IST Failure Error waiting for all pods to be running and ready: 3 / 13 pods in namespace kube-system are NOT in RUNNING and READY state in 10m0s No response /sig network /sig node cc
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/remove sig-network /remove sig-node /unassign This seems a deployment problem ... on calico to be precise first point of contact for initial triage should be the job/sig owners ... this job also seems to be failing for a long time /sig cluster-lifecycle sig-cluster-lifecycle-cluster-api-alerts
/remove-sig network /remove-sig node
cc
will take a look /assign
Check the Calico version, there was a similar report that was fixed bumping the version. As a side not, I'm seeing a lot of problems related to Lots of projects/test/code were assuming the secret was created and is causing a lot of noise in different CIs
Hey thanks for the PR! The test still shows in the - will it take some time for the changes you made in the PR to reflect?
hello! those are not failures, they are flakys :/ afaik
Hey folks, this test has now started failing again on the dashboard. Should we open a new issue for it or is it okay to track it here itself? !
checking that
the capg conformance failed but now have flakys only
Thanks a ton


Question: "integration", ], deps = [ <ins> "//vendor/k8s.io/apimachinery/pkg/api/errors:go_default_library", </ins> "//vendor/k8s.io/apimachinery/pkg/api/meta:go_default_library", "//vendor/k8s.io/apimachinery/pkg/apis/meta/v1:go_default_library", "//vendor/k8s.io/apimachinery/pkg/apis/meta/v1/unstructured:go_default_library",

Kubernetes version (use ): Kubernetes HEAD Environment: Cloud provider or hardware configuration: linux OS (e.g. from /etc/os-release): Kernel (e.g. ):NAME=Fedora25 What happened: apiserver panic What you expected to happen: No panic How to reproduce it (as minimally and precisely as possible): Testcase is going to be sent as a PR
Fixed by


Question: // resizeFileSystem will resize the file system if user has requested a resize of // underlying persistent volume and is allowed to do so. <del> resizeSimpleError, resizeDetailedError := og.resizeFileSystem(volumeToMount, devicePath, deviceMountPath, volumePlugin.GetPluginName()) </del> <ins> resizeSimpleError, resizeDetailedError := og.resizeFileSystem(volumeToMount, devicePath, deviceMountPath, volumePluginName) </ins> if resizeSimpleError != nil || resizeDetailedError != nil { return resizeSimpleError, resizeDetailedError

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!--What happened: In this test, volume mount failed because the driver-registrar took a minute to come up. However, it seems like volume mount operation is retrying without any sort of backoff. This message appears in 400+ times in less than a minute. What you expected to happen: Operation executor should not be retrying operations so frequently. <!-- DO NOT EDIT BELOW THIS LINE --/kind bug
I'll take a look at this. /assign
Problem is that reconciler depends on operation executor to do backoff. In the normal case operation executor returns a method that does the actual work. Any errors by that method result in exponential backoff. But in this case, the error is coming from the code creating the mount method which is not handled by the back off logic, and is called again and again by the reconciler resulting in log and event spam. This didn't use to be a problem because initialization errors were uncommon (plugins were built in), but with CSI the driver may not exist and these initialization errors are expected. I'll look in to cleaning this up.
Is this something that will affect users in 1.13 and needs CPing?
This will result in kubelet log spam and possibly kubernetes event spam. It happens when someone creates a pod referencing a CSI volume before the CSI driver for that volume is installed. Which happens frequently in E2E tests because they install/remove driver with every test. But should be rare in real world usage since most users are likely to install a driver first before using it.
If release team is worried about this, we could create a quick PR to suppress logging/event generation of this these errors. Otherwise we plan to address this more holistically for 1.14 and 1.13.1.


Question: "k8s.io/kubernetes/plugin/pkg/scheduler/factory" "github.com/golang/glog" <ins> "github.com/prometheus/client_golang/prometheus" </ins> "github.com/spf13/cobra" "github.com/spf13/pflag" <del> utilflag "k8s.io/apiserver/pkg/util/flag" "github.com/prometheus/client_golang/prometheus" </del> ) // SchedulerServer has all the context and params needed to run a Scheduler

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: What happened: I pull the latest code, and run with below cmd. But the cluster is not up. And I notice the kube-, What you expected to happen: How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): GitCommit:"" Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others:
/sig cluster-lifecycle
/assign
Can you please try the fix above?
/assign
/unassign
your pr does not fix it yet? If not, I can help.
The fix from works for me! o/
ok, thanks
reopen , make less modifications.
sorry, i miss hyper...


Question: type Backoff struct { // The initial duration. Duration time.Duration <del> // Duration is multiplied by factor each iteration. Must be greater // than or equal to zero. </del> <ins> // Duration is multiplied by factor each iteration, if factor is not zero // and the limits imposed by Steps and Cap have not been reached. // Should not be negative. // The jitter does not contribute to the updates to the duration parameter. </ins> Factor float64 <del> // The amount of jitter applied each iteration. Jitter is applied after // cap. </del> <ins> // The sleep at each iteration is the duration plus an additional // amount chosen uniformly at random from the interval between // zero and `jitter*duration`. </ins> Jitter float64 <del> // The number of steps before duration stops changing. If zero, initial // duration is always used. Used for exponential backoff in combination // with Factor. </del> <ins> // The remaining number of iterations in which the duration // parameter may change (but progress can be stopped earlier by // hitting the cap). If not positive, the duration is not // changed. Used for exponential backoff in combination with // Factor and Cap. </ins> Steps int <del> // The returned duration will never be greater than cap *before* jitter // is applied. The actual maximum cap is `cap * (1.0 + jitter)`. </del> <ins> // A limit on revised values of the duration parameter. If a // multiplication by the factor parameter would make the duration // exceed the cap then the duration is set to the cap and the // steps parameter is set to zero. </ins> Cap time.Duration } <del> // Step returns the next interval in the exponential backoff. This method // will mutate the provided backoff. </del> <ins> // Step (1) returns an amount of time to sleep determined by the // original Duration and Jitter and (2) mutates the provided Backoff // to update its Steps and Duration. </ins> func (b *Backoff) Step() time.Duration { if b.Steps < 1 { if b.Jitter > 0 {

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: While revising clients of I noticed that some comments there are misleading or confusing or incomplete. I am looking , which is in the latest commit in at the moment. The comment on the field named says it must be greater than or equal to zero. The comment does not call out any special meaning for zero. The method tolerates any value in the field, but treats zero irregularly (taking it to mean roughly the same thing as 1). The comments about all say that it is applied after , and give no other information. The method reveals that there is a broader and clearer thing to say. There is logic that determines the progression of values for , and at each step the jitter is to the returned value after the update (if any) to . The comment on says flatly that it is "The number of steps before duration stops changing". But this is not true; also stops changing once it hits . The comments on and should more clearly indicate that they are peers in determining when the increases in stop. The comment on says "The returned duration will never be greater than cap before jitter is applied", but this is not true in the method. When , is ignored. The comment on only says it is used to determine when to stop changing . The function interprets as a limit on the number of iterations (making its effect of limiting increases moot). The comment on the function says "It checks the condition up to Steps times" but this is not true. If the hits the earlier than that, the iteration is cut short.
/remove-sig architecture


Question: package algorithmprovider import ( <del> "sort" "strings" </del> <ins> "fmt" </ins> utilfeature "k8s.io/apiserver/pkg/util/feature" "k8s.io/klog/v2"

Using scheduler like or , it always get unwanted output like: It simply because when adding the flags in scheduler, it will try to look up the And this will invoke: Since flag already has to prevent running other code when having : We should not print the klog info when using or right?
/sig scheduling
that's annoying. Can you help to fix it?
Ok I'll take care of this. /assign


Question: Manager: m, } <del> // Export the HTTP endpoint if a port was specified. if port > 0 { err = cadvisorClient.exportHTTP(port) if err != nil { return nil, err } </del> <ins> err = cadvisorClient.exportHTTP(port) if err != nil { return nil, err </ins> } return cadvisorClient, nil }

If is passed to the kubelet, container metrics are not included in the Prometheus metrics endpoint.


Question: t.Run(test.name, func(t *testing.T) { nodes := makeNodeList(test.nodes) snapshot := cache.NewSnapshot(test.pods, nodes) <del> ctx := context.Background() </del> <ins> ctx, cancel := context.WithCancel(context.Background()) defer cancel() </ins> informerFactory, err := populateAndStartInformers(ctx, test.rcs, test.rss, test.services, test.sss) if err != nil { t.Errorf("error creating informerFactory: %+v", err)

The ctx object () is context.Background(). Thus, its Done() method returns a nil channel. Later, the nil channel is used to call and as their argument. The goroutines created in the two functions are leaked. No goroutine leaks in TestSelectorSpreadScore(). Run TestSelectorSpreadScore() No response all versions after <details</details<details</details<details</details<details</details<details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scheduling (It sounds like leas in unit tests, which may be negligible. Does it cause some test failures?)
the test can pass.
/assign
/unassign /assign


Question: "k8s.io/apimachinery/pkg/watch" "k8s.io/apiserver/pkg/features" "k8s.io/apiserver/pkg/storage" <ins> "k8s.io/apiserver/pkg/storage/cacher/metrics" </ins> utilfeature "k8s.io/apiserver/pkg/util/feature" "k8s.io/client-go/tools/cache" featuregatetesting "k8s.io/component-base/featuregate/testing" <ins> k8smetrics "k8s.io/component-base/metrics" "k8s.io/component-base/metrics/testutil" </ins> "k8s.io/utils/clock" testingclock "k8s.io/utils/clock/testing" )

Part of From For now posted as a "help wanted" issue, however if we cannot hit the timeline for 1.30 code freeze () I would take it over so we don't miss. Still issue is simple enough that it can be quickly implemented. Implementation tips: Add a metric around waitUnitlFreshAndBlock method Metric should be a histogram with latency buckets similar to api latency Metric should have label for resource version. A way to monitor watch cache impact on request latency.
/sig api-machinery /triage accepted /help
This request has been marked as needing help from a contributor. Please ensure that the issue body includes answers to the following questions: Why are we solving this issue? To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points? Does this issue have zero to low barrier of entry? How can the assignee reach out to you for help? For more details on the requirements of such an issue, please see and ensure that they are met. If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : /sig api-machinery /triage accepted /help Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: `monit` is a lightweight process babysitting system for keeping kubelet and docker running. <ins> ### fluentd `fluentd` is a daemon which helps provide [cluster-level logging](#cluster-level-logging). </ins> <!-- BEGIN MUNGE: GENERATED_ANALYTICS --> [![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/admin/cluster-components.md?pixel)]()

has TODOs for how to bring up cluster addons like monitoring and logging. We should fill these in. requested by user:


Question: return rs.Address.Equal(other.Address) && rs.Port == other.Port } <ins> // GetKernelVersionAndIPVSMods returns the linux kernel version and the required ipvs modules func GetKernelVersionAndIPVSMods(Executor exec.Interface) (kernelVersion string, ipvsModules []string, err error) { kernelVersionFile := "/proc/sys/kernel/osrelease" out, err := Executor.Command("cut", "-f1", "-d", " ", kernelVersionFile).CombinedOutput() if err != nil { return "", nil, fmt.Errorf("error getting os release kernel version: %v(%s)", err, out) } kernelVersion = strings.TrimSpace(string(out)) // parse kernel version ver1, err := version.ParseGeneric(kernelVersion) if err != nil { return kernelVersion, nil, fmt.Errorf("error parsing kernel version: %v(%s)", err, kernelVersion) } // "nf_conntrack_ipv4" has been removed since v4.19 // see https://github.com/torvalds/linux/commit/a0ae2562c6c4b2721d9fddba63b7286c13517d9f ver2, _ := version.ParseGeneric("4.19") // get required ipvs modules if ver1.LessThan(ver2) { ipvsModules = append(ipvsModules, ModIPVS, ModIPVSRR, ModIPVSWRR, ModIPVSSH, ModNfConntrackIPV4) } else { ipvsModules = append(ipvsModules, ModIPVS, ModIPVSRR, ModIPVSWRR, ModIPVSSH, ModNfConntrack) } return kernelVersion, ipvsModules, nil } </ins>

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!--kube-proxy cannot use ipvs mode on Linux 4.19 due to a check for the module, which was changed to (). uses to check for : It should instead accept on Linux 4.19 or above. What happened: kube-proxy fell back from ipvs mode to iptables mode. What you expected to happen: kube-proxy using ipvs. How to reproduce it (as minimally and precisely as possible): Launch a cluster on an image using Linux 4.19 or above with kube-proxy using ipvs mode (see ). Environment: Kubernetes version (use ): All Kernel (e.g. ): 4.19 <!-- DO NOT EDIT BELOW THIS LINE --/kind bug
/sig network
/area ipvs /assign /cc
Okay, so should check {nfconntrackipv4 | nf_conntrack} and expect load at least one module?
I think so
Please just go ahead :)


Question: } } <ins> // Set the expire_nodest_conn sysctl we need for if val, _ := sysctl.GetSysctl(sysctlExpireNoDestConn); val != 1 { if err := sysctl.SetSysctl(sysctlExpireNoDestConn, 1); err != nil { return nil, fmt.Errorf("can't set sysctl %s: %v", sysctlExpireNoDestConn, err) } } // Set the expire_quiescent_template sysctl we need for if val, _ := sysctl.GetSysctl(sysctlExpireQuiescentTemplate); val != 1 { if err := sysctl.SetSysctl(sysctlExpireQuiescentTemplate, 1); err != nil { return nil, fmt.Errorf("can't set sysctl %s: %v", sysctlExpireQuiescentTemplate, err) } } </ins> // Set the ip_forward sysctl we need for if val, _ := sysctl.GetSysctl(sysctlForward); val != 1 { if err := sysctl.SetSysctl(sysctlForward, 1); err != nil {

What happened: After updating the pods backing a service with sessionAffinity set, all communications to the service stopped working. What you expected to happen: Communications should continue to work How to reproduce it (as minimally and precisely as possible): 1- Create a service with backed by a deployment of a single pod 2- Connect to the service from a pod 3- Delete the pod and wait for it to recreate When trying to reconnect to the service all packets will be lost until the real server is removed because the persistency remains regardless of . In addition, if the client keeps retrying, the number of connections to the real server will never reach 0 and it will not be deleted (until the persistency timeout expires, which by default takes 3h) Anything else we need to know?: I did some testing and setting fixes the issue. I'll create a PR tomorrow (including some changes discussed in which are related) Environment: Kubernetes version (use ): kube-proxy OS (e.g. from /etc/os-release): Kernel (e.g. ): /area ipvs /sig network /kind bug /assign


Question: # Configure kube-proxy to run as a windows service. flags+=" --windows-service=true" <del> # TODO(mtaufen): Configure logging for kube-proxy running as a service. # I haven't been able to figure out how to direct stdout/stderr into log # files when configuring it to run via sc.exe, so we just manually # override logging config here. flags+=" --log-file=${WINDOWS_LOGS_DIR}kube-proxy.log" # klog sets this to true internally, so need to override to false # so we actually log to the file flags+=" --logtostderr=false" </del> # Enabling Windows DSR mode unlocks newer network features and reduces # port usage for services. # https://techcommunity.microsoft.com/t5/networking-blog/direct-server-return-dsr-in-a-nutshell/ba-p/693710

/kind feature Hey, in K8s 1.23 we are planning to deprecate multiple klog flags. Deprecated flags: , , , , , , , , , , , One of the alpha milestones is requires us to remove usage of deprecated flags within K8s repo. Examples found for windows [ ] [ ] /sig windows /wg structured-logging
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign I've started looking into this. Part of the problem is that go-runner is not part of kubernetes-node-windows- I suspect that is where the scripts get and kube- from.