Question: } } <ins> if filterTags[TagNameKubernetesCluster] == "" { glog.Errorf("Tag %q not found; Kuberentes may behave unexpectedly.", TagNameKubernetesCluster) } </ins> awsCloud.filterTags = filterTags if len(filterTags) > 0 { glog.Infof("AWS cloud filtering on tags: %v", filterTags)

If there are two security groups on an instance, we currently make an arbitrary choice. We should choose based on tags, as we normally do. For back-compat, we need to cope if there is one security group and it is not tagged. We should make sure kube-up tags our security groups, though it isn't really required for kube-up, but it is just good behaviour. cf


Question: // Install func registers the logs handler. func (l Logs) Install(c *restful.Container) { // use restful: ws.Route(ws.GET("/logs/{logpath:*}").To(fileHandler)) <del> // See github.com/emicklei/go-restful/blob/master/examples/restful-serve-static.go </del> <ins> // See github.com/emicklei/go-restful/blob/master/examples/static/restful-serve-static.go </ins> ws := new(restful.WebService) ws.Path("/logs") ws.Doc("get log files")

<!-- Please only use this template for submitting enhancement requests --What would you like to be : // See right address of restful-serve- in the comment of Why is this needed: the address of restful-serve- in the comment of is wrong now
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign
/sig Docs
/kind cleanup /remove-kind feature


Question: # github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7 => github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7 ## explicit github.com/gregjones/httpcache <del> github.com/gregjones/httpcache/diskcache </del> # github.com/grpc-ecosystem/go-grpc-middleware v1.3.0 => github.com/grpc-ecosystem/go-grpc-middleware v1.3.0 ## explicit; go 1.14 github.com/grpc-ecosystem/go-grpc-middleware

We use a lot of CRDs (frequently more than 1,000) in the project. This has historically caused a bunch of problems with client-side API discovery, among other things. At the time of writing the latest releases of kubectl use a discovery client rate limiter with a generous QPS and burst (50 qps, 300 qps burst), so client-side rate limiting doesn't become a problem until the API server is serving more than 300 API groups. Despite rate-limiting no longer coming into effect, we noticed that discovery was very slow - taking 10-20 seconds to complete. After having a few folks at run a little test (see and following comments) we noticed that discovery specifically seems slow on MacOS - it's dramatically faster on Linux. Here's discovery happening on Linux with 1,577 CRDs across 343 API groups in around a second: ` Here's the same test run from MacOS against the exact same EKS cluster, taking 12+ seconds. I expected kubectl discovery on MacOS to be as fast as it was on Linux. an EKS cluster. ~1,500 CRDs across ~350 API groups. the discovery cache in how long the first command you run against the EKS cluster takes to complete Refer to and subsequent comments for more examples of folks reproducing this. <detailsNote that the version of kubectl was the same on Mac and Linux for my tests. </details<detailsAWS - EKS </details<detailsMacOS 12.4, M1 Max Linux as below </details<detailsN/A </details<detailsN/A </details<detailsN/A </details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig cli
will show you how many milliseconds was the GET request taken. There are 37 GET requests after was removed. Is there any difference?
I'll add more details ASAP, but we're confident the slow-down is not in the network (i.e. the many GET requests).

Initial profiling indicates it's something to do with writing what is fetched to the discovery cache on disk.
Watch out for anything where writes trigger : there was an issue on macOS where the only way for an app to be confident that a write had completed was to trigger a whole-filesystem sync. Libraries that like data integrity opted for the slower-but-effective approach. I've stopped using macOS so this hint is from a while back, but it might relevant.
Thanks for the tip The issue definitely seems to be in the . When I apply the diff below to switch over to a the slowness disappears and MacOS performance matches that of Linux - about a second for to perform discovery:
It seems to be a problem that would occur only once when establishing the . Should we add it to known issues? Some explain about fsync in MacOs in
Hi, I have no strong opinion here.
You're right, and this happens every 6 hours in the latest versions of kubectl when the cache is invalidated. It's still not an ideal user experience though - you can imagine kubectl 'hanging' for 10-20 seconds the first time you used it each day.
Also note that this should fix a lot of that problem:
has anyone been able to profile kubectl to see what it's doing?
Also note that your CPU is mostly doing nothing, so it's most likely stuck on IO
I think is correct - my bet is we're blocking on , which per uses on MacOS. Per : A mutex profile shows a lot of time waiting for inside , which you can see sets the argument to . I believe this means we're doing an for every HTTP response body we encounter while doing discovery per .
I've confirmed the slowdown is coming from the . Using the following diff: I see MacOS is down to around one second, similar to Linux:
How do folks feel about not explicitly calling for each file in the HTTP cache? Given that it is indeed only a cache I lean toward optimizing for write performance rather than data safety. If we did want to stop calling it seems like the best option would be to update the package of that we use today to make whether it calls with configurable. I'm skeptical we could get this change merged upstream though - the latest change to was three years ago. Another option might be to switch the the package.

I just took a brief shot at this, but it seems like it would require some refactoring to the to handle calling on the underlying database.
I don't care a ton if we drop a cache file coherently. I care a lot if a write fails in a way that leaves a partially written file in a way that pollutes the cache and makes the user have to delete the file from the cache manually. I don't know which of those we're opening ourselves up to by writing without sync. Can we find out?
Presuming we do find that not fsyncing can result in leaving the cache in a broken state (as opposed to a partial state) would falling back to invaliding the cache and recreating it be acceptable?
Little more insight at and Notably the second link mentions:
I dug into this a bit - dumping state here before I sign off for the day. First, note that the discovery client uses two layers of cache - a "discovery cache" and an "HTTP cache". The former appears to be backed by the latter. My -fu is not strong enough to figure out why we do this. The best I could find is , which refers to "restoring the old cached client behavior" and appears to (re?)introduce the caching HTTP transport. Perhaps or know why this is? Only the HTTP cache is explicitly calling . The discovery cache just writes the files and calls , so it's already subject to the risks of not using . Per that risk is that despite the write having been flushed from the OS to the disk controller (e.g. by successfully calling ): My understanding of the and code that powers the HTTP cache is: If a cache file read returns an error we just act as if the value isn't in cache. If a cache file read returns garbled data that can't parse as an http response would fall back to its underlying (i.e. make a call to the API server) and will happily cache over the top of an existing (potentially garbled) file. The caches will be invalidated after six hours. So the failure mode that wouldn't be covered if we stopped calling is that if power were cut to the drive before its controller could sync down to persistent storage and_ this resulted in a file that was garbled just right such that it appeared to be a valid HTTP response but contained an invalid body (i.e.

not real discovery data) someone would have to either manually remove their cache, or wait six hours for the cache to be automatically invalidated. It seems like we could probably make the discovery client force an uncached request through the caching HTTP transport if it encountered what appeared to be cached, invalid discovery data, but I do wonder how far we really need to go to ensure that this cache is never corrupted.
For either of these caches, there are techniques (checksums and the like) where we can allow the app to see whether or not it is looking at a coherent cache document or something that might not be correct. Whenever a cache entry doesn't look valid, the code can asset that the date is current and the entry checksum is OK. If the entry turns out to be invalid or stale, remove it and then act as if we got a read error. That should cover any kind of unsynchronised write failure, I think. For locking the cache we also don't need to ensure sync-to-disk, so long as all running processes see a consistent view of locking.
Agreed. My question is whether we need to add such techniques to this particular cache (or caches) before we'll feel comfortable removing the fsync. Removing the fsync from the HTTP cache would presumably bring it back down to the same level of guarantee as the discovery cache. My guess would be that the fact we were ever explicitly fsyncing each cache value in the HTTP cache was a happy accident rather than something that was explicitly a desirable quality for the HTTP cache.
It seemed like writing our own httpcache cache backend was the best way to go here, so I went ahead and one that uses checksums rather than syncs per the above PR.


Question: fi append_or_replace_prefixed_line "${basic_auth_csv}" "${KUBE_PASSWORD},${KUBE_USER},"      "admin,system:masters" fi local -r known_tokens_csv="${auth_dir}/known_tokens.csv" <ins> if [[ -e "${known_tokens_csv}" && "${METADATA_CLOBBERS_CONFIG:-false}" == "true" ]]; then rm "${known_tokens_csv}" fi </ins> if [[ -n "${KUBE_BEARER_TOKEN:-}" ]]; then append_or_replace_prefixed_line "${known_tokens_csv}" "${KUBE_BEARER_TOKEN},"             "admin,admin,system:masters" fi

/sig cluster-lifecycle
Very rarely we are seeing known tokens files filled with the NULL character, with ASCII value 000. Restarting the master doesn't fix the issue because the is saved between restarts.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. Prevent issues from auto-closing with an comment. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale


Question: fi append_or_replace_prefixed_line "${basic_auth_csv}" "${KUBE_PASSWORD},${KUBE_USER},"      "admin,system:masters" fi local -r known_tokens_csv="${auth_dir}/known_tokens.csv" <ins> if [[ -e "${known_tokens_csv}" && "${METADATA_CLOBBERS_CONFIG:-false}" == "true" ]]; then rm "${known_tokens_csv}" fi </ins> if [[ -n "${KUBE_BEARER_TOKEN:-}" ]]; then append_or_replace_prefixed_line "${known_tokens_csv}" "${KUBE_BEARER_TOKEN},"             "admin,admin,system:masters" fi

/sig cluster-lifecycle
Very rarely we are seeing known tokens files filled with the NULL character, with ASCII value 000. Restarting the master doesn't fix the issue because the is saved between restarts.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. Prevent issues from auto-closing with an comment. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale


Question: } } } <ins> // TestLimitRanger_GetLimitRangesFixed22422 Fixed Admission controllers can cause unnecessary significant load on apiserver #22422 func TestLimitRanger_GetLimitRangesFixed22422(t *testing.T) { limitRange := validLimitRangeNoDefaults() limitRanges := []corev1.LimitRange{limitRange} var count int64 mockClient := &fake.Clientset{} unhold := make(chan struct{}) mockClient.AddReactor("list", "limitranges", func(action core.Action) (bool, runtime.Object, error) { atomic.AddInt64(&count, 1) limitRangeList := &corev1.LimitRangeList{ ListMeta: metav1.ListMeta{ ResourceVersion: fmt.Sprintf("%d", len(limitRanges)), }, } for index, value := range limitRanges { value.ResourceVersion = fmt.Sprintf("%d", index) value.Namespace = action.GetNamespace() limitRangeList.Items = append(limitRangeList.Items, value) } // he always blocking before sending the signal <-unhold return true, limitRangeList, nil }) handler, _, err := newHandlerForTest(mockClient) if err != nil { t.Errorf("unexpected error initializing handler: %v", err) } attributes := admission.NewAttributesRecord(nil, nil, api.Kind("kind").WithVersion("version"), "test", "name", api.Resource("resource").WithVersion("version"), "subresource", admission.Create, &metav1.CreateOptions{}, false, nil) attributesTest1 := admission.NewAttributesRecord(nil, nil, api.Kind("kind").WithVersion("version"), "test1", "name", api.Resource("resource").WithVersion("version"), "subresource", admission.Create, &metav1.CreateOptions{}, false, nil) wg := sync.WaitGroup{} for i := 0; i < 10; i++ { wg.Add(2) // simulating concurrent calls after a cache failure go func() { defer wg.Done() ret, err := handler.GetLimitRanges(attributes) if err != nil { t.Errorf("unexpected error: %v", err) } for _, c := range ret { if c.Namespace != attributes.GetNamespace() { t.Errorf("Expected %s namespace, got %s", attributes.GetNamespace(), c.Namespace) } } }() // simulation of different namespaces is not a call go func() { defer wg.Done() ret, err := handler.GetLimitRanges(attributesTest1) if err != nil { t.Errorf("unexpected error: %v", err) } for _, c := range ret { if c.Namespace != attributesTest1.GetNamespace() { t.Errorf("Expected %s namespace, got %s", attributesTest1.GetNamespace(), c.Namespace) } } }() } // unhold all the calls with the same namespace handler.GetLimitRanges(attributes) calls, that have to be aggregated unhold <- struct{}{} go func() { unhold <- struct{}{} }() // and here we wait for all the goroutines wg.Wait() // since all the calls with the same namespace will be holded, they must be catched on the singleflight group, // There are two different sets of namespace calls // hence only 2 if count != 2 { t.Errorf("Expected 1 limit range, got %d", count) } // invalidate the cache handler.liveLookupCache.Remove(attributes.GetNamespace()) go func() { // unhold it is blocking until GetLimitRanges is executed unhold <- struct{}{} }() _, err = handler.GetLimitRanges(attributes) if err != nil { t.Errorf("unexpected error: %v", err) } close(unhold) if count != 3 { t.Errorf("Expected 2 limit range, got %d", count) } } </ins>

Recently we observed high latencies on POST pods on our scalability tests. For more context see: It appeared to be throttling related, e.g.: As an example look at resource quota admission controller. In there was some fallback to use lru cache in case of on objects. However, not what happens if we are sending multiple POST pods at the same time to apiserver. What happens in Admit() if there are no results in Indexer, we list ResourceQuotas. But if there are multiple calls at the same time, there can be multiple LISTs before the cache is updated: But this can cause throttling of these requests. What we should do is: if there is List() in flight, we should wait until it is finished and lookupCache is updated.
- thanks for the analysis. - fyi, this is an interesting side-effect of the LRU cache pattern, not sure if we have adopted it in other places as well, but good to keep in mind.
To clarify, it was adopted in and admission plug-ins in separate PRs.
- it took me some time, but was useful experience. Thanks for assigning yourself.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. Prevent issues from auto-closing with an comment. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten /remove-lifecycle stale
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-testing, kubernetes/test-infra and/or . /close
/reopen
You can't reopen an issue/PR unless you authored it or you are a collaborator. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: // jitter factor for resyncInterval workerResyncIntervalJitterFactor = 0.5 <del> // jitter factor for backOffPeriod </del> <ins> // jitter factor for backOffPeriod and backOffOnTransientErrorPeriod </ins> workerBackOffPeriodJitterFactor = 0.5 <ins> // backoff period when transient error occurred. backOffOnTransientErrorPeriod = time.Second </ins> ) type podWorkers struct {

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: Pods are synchronized by kubelet before network is ready. Synchronization in such case fails and pod is synchronized after 10 seconds. What you expected to happen: Kubelet should wait until network is ready before starting synchronizing pods. How to reproduce it (as minimally and precisely as possible): Configure cluster with Cluster Autoscaler enabled. Start pod so cluster is scaled up. In logs you might find following error: E0907 13:18:49. 972 ] Error syncing pod xxxxxx ("xxxxxxxxx(xxxxxxxxxxx)"), skipping: network is not ready: [runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: Kubenet does not have netConfig. This is most likely due to lack of PodCIDR] Environment: Build from HEAD Cloud provider or hardware configuration: GKE
/kind bug /sig node


Question: // and should be discarded "topology.kubernetes.io/region": "us-other-west", "topology.k8s.io/region":        "us-other-west", <ins> "kubernetes.io/region":          "us-other-west", "k8s.io/region":                 "us-other-west", </ins> // Should discard labels that already exist "my.custom.label/foo": "bar", "my.custom.label/bar": "foo", <ins> // Should add labels that not match regex "app.k8snio/bar": "foo", "k8snio/bar":     "foo", </ins> }, }, existingNode: &v1.Node{

For cloud provider nodes, there are some k8s label names are reserved. Like and The current regex is: This will match the dot . as any single character. This block adding a label name like: app.k8snio/name The regex format is incorrect, we should escape the dot in regex so that it can match the dot as a string. Can add additional label with name like: app.k8snio/name Configure AdditionalLabels in the instanceMeta to add additional label name with app.k8snio/name. No response <detailsClient Version: v1.30.1 Kustomize Version: v5.0.4-0.- Server Version: v1.30.0 </details<details</details<details</details<details</details<details</details<details</details
/sig cloud-provider
If it make sense to fix it, i would like to take it and raise a PR. /assign
Which Kubernetes component has the bug ? (eg: kubelet) Can you explain how to reproduce the problem?
It's the node controller in the shared library Though I really don't think cloud providers should be setting node labels like "", this is technically incorrect. This restriction is only applied to labels the cloud provider controller binary running this controller would attempt to add itself.
just noticed this after our SIG meeting today, cc
/triage accepted
please go ahead and raise a PR, thanks for reporting this =)
When fixing this, consider if there are any labels that cloud provider integration might legitimately set. For example: (documented as being set by cloud provider integration code). Some labels are OK to set; we could allowlist and explicitly. When rejecting some others, maybe setting a Condition on the Node ( perhaps) would help surface what's not right. Not sure. Making the simple fix is also fine by me.
OK. Thanks I have raised a PR for this: Not sure if this change looks good.


Question: Filter: v1beta2.PluginSet{ Enabled: []v1beta2.Plugin{ {Name: "Plugin1", Weight: pointer.Int32Ptr(2)}, <ins> {Name: "Plugin3", Weight: pointer.Int32Ptr(3)}, </ins> }, }, },

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --Recent commit: import an issue when multiple custom plugin configured in scheduler config file, and those plugins are default plugins too. It was manipulating slice with index, but the slice is updated while index is not, which will cause the index out of range, see the exceptions listed below, config file should be handled properly without such exception override multiple default plugin in scheduler's config file. Kubernetes version (use ): latest (1.22) Cloud provider or hardware configuration: OS (e.g: ): all Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others: /assign
/sig scheduling /triage accepted


Question: } } <ins> if podStatus.PodIP == "" { return nil, ErrNoPodInfraContainerInPod } </ins> if len(statuses) == 0 && podStatus.PodIP == "" { return nil, ErrNoContainersInPod }

While fixing the issue on describe events for pod, I found kubelet keep restarting monitoring-heapster-controller container: I retrieved the container log, and found it exit due to database creation failure: I didn't follow up on why influxdb issue. cc/
This happens when InfluxDB service is not up. Heapster retries a few times and then eventually fails. The plan is to avoid failing completely and keep retrying DB creation until it succeeds and bubble up appropriate errors. , Dawn Chen wrote:
Influxdb is running happily without issue on the same node:
Related kube-proxy log:
It looks like a kube-proxy issue.
It looks like there are no endpoints for 'monitoring-influxdb' service. The service itself is active and there is a pod that matches the selector of the service. Here is the 'monitoring-influxdb' service: Influxdb pod that matches the selector for 'monitoring-influxdb' service is running. Here are the endpoints from the apiserver: Wonder why there are no endpoints. cc
I ran into a similar issue where endpoints for monitoring-influxdb was not configured. In my case, the pod state took a long time to move to running (even though the containers on the machine were all running fine). But the endpoints were created as soon as the pod changed to state running.
Probed a bit further and it looks like the kubelet is not populating PodIP.
cc/ Can you take a quick look at this one?
FYI: I am still working on this issue.
Here is a little bit more updates: 1) PodIP is missing on all pods running on that particular node; but pods on other nodes with exact same kubelet do have PodIP associated with. 2) docker inspect shows that every pods on bad node have proper "NetworkMode": 3) docker inspect those pod infra containers, they all have proper IP assigned by docker.
The issue is that the kubelet does not ignore old network containers while computing pod status. Working on a fix.
Nice find :) On Mar 26, 2015 5:50 PM, "Vish Kannan" wrote:


Question: pkg/util/config pkg/util/ebtables pkg/util/goroutinemap/exponentialbackoff <del> pkg/util/iptables </del> pkg/util/iptables/testing pkg/util/labels # See previous effort in PR #80685 pkg/util/oom

Anyone itching to understand the code and try fixing things in Kubernetes, please help fixing the lint failures Step 1: Remove one or two packages (related packages) from the list in - (note: do not change generated files, the name of defaulting functions, or the name of conversion functions... packages failing linting due to issues in those files will need to be left alone) Step 2: Run and look at the logged errors Step 3: Fix the code to ensure that runs cleanly. Step 4: Run and to make sure we fix any problems with gofmt or bazel. Step 5: File a PR with the change Step 6: Go to step :) Please use the tracking spreadsheet For everyone who is going to contribute, please append the packages you picked at the end. For existing contributor, please make sure this list is update to date
/sig architecture /sig testing
Hey, I would like to resolve this issue. Could you assign it to me ?
there are a lot of packages so there will be many people working on this. Also we cannot assign issue to folks who are not in the github org for kubernetes. please feel free to pick up a few packages like i mentioned and file a PR. Make sure you link back to this issue, so folks can see which packages you filed PR for.
OK. I see. I will firstly pick up the package called to fix.
Hi dims. Should i submit my PR before i remove one or two packages (related packages) from the list in - Or I first remove the package from list, then I fix the issue and submit a PR?
in your PR, you will have both the fixes to packages and some lines removed from /.golint_failures
When I run , I found some errors as below: I found that there is a go file in But there is still a error.
Say your GOPATH looks like this Your kubernetes directory should be () cd to and then run from that directory.
Is it necessary to obey all rules in golint? Rules in golint are set in stone. Maybe using gometalinter to ignore some specific errors is better?
I revise some erros about golint in some packages. Please review it.
which errors would you like to ignore?

if you can open a new issue with the list of errors to ignore (and proposing gometalinter), we can get some opinions there from others in the community.
Let me take the kubectl cmd part...
Can someone please review and let me know if I am doing it right?
I ll work on all the packages related to
we get 800-ish pkgs left to fix. my concern is that if we fix them package by package, then such pulls will be flooding the repo. can't we possibly make it in larger pulls? the larger the better?
+1 to slightly larger pulls with related packages. if allow too large then the reviewers will not be able to do a thorough review. we should spread the load among multiple reviewers. (if we allow a really large pull, we will end up with top level reviewers who will face the burden and the PR(s) will just stagnate)
oh good point. we could put one pkg's change into one commit, and merge all changes under one area into one pull so that it wont spread the load. wdyt? for each pull, we can have hundred-ish lines of change. but it will still be 30~50 pulls i guess? good news is that we can take our time to deal with it, at least before 1.12
sounds great! please feel free to help make it happen :) (by suggesting groups of packages that kinda go together)
I'm looking at fixing pkg/proxy, and I think I see what was talking about ignoring some rules. proxy.ProxyProvider is being suggested because golint is saying that it stutters. In that example, I think it's okay renaming it to proxy.Provider, but I could see an example where you would want to keep the stutter if the package that it's declared in uses it heavily.
sure thing, feel free to propose a PR for switching to gometalinter independently of other patches. We can work through that there.
I sent the pull request to fix pkg/volume/aws_ebs
I've fixed a few of these, but get the following error when I try to run : Are there any instructions on what I need to setup in order to resolve this?

you can find this file in
I have raised a PR to fix golint errors in pkg/api
I have fixed some pkgs under /pkg/kubelet pkg/kubelet/images pkg/kubelet/kuberuntime pkg/kubelet/leaky pkg/kubelet/pleg
I submitted a pull request for golint errors in pkg/security/ edit: I can work on picking up pkg/controller/bootstrap through pkg/controller/namespace/... (114-129)
fix golint errors in package cmd/kubeadm/app, since the codes under cmd/kubeadm/app/apis/kubeadm/v1alpha2 and cmd/kubeadm/app/apis/kubeadm/v1alpha3 are auto-generated, i keep them untouched.
I am working on some packages under pkg/kubelet, anyone interested in this plz leave messages for me :)
fix golint for pkg/util/dbus pkg/util/labels pkg/util/node pkg/util/strings pkg/util/workqueue/prometheus
PR fixes golint errors in pkg/routes,pkg/securitycontext, pkg/serviceaccount
PR fixes golint in pkg/probe
should we have a checklist as part of the issue summary saying what is in progress and what is done? Since this issue is getting long, this will make new contributors to pick up issues easier.
Great idea!, want to throw a google doc with a check-list so it's easy for participants to organize themselves? (please!) If it is the issue summary then one of us with privileges will be a bottle neck.
Created a with all the package that have on going PRs. Maybe you can add this to the summary. For everyone who is going to contribute, please append the packages you picked at the end. For existing contributor, please make sure this list is update to date
Thanks updated the spread sheet
Raised PR to fix golint issues in: pkg/controller/clusterroleaggregation pkg/controller/cronjob pkg/controller/daemon pkg/controller/deployment pkg/controller/deployment/util pkg/controller/disruption pkg/controller/endpoint
This issue is close in accident by . could you reopen it?
/reopen
Reopening this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I found a problem trying to fix golint messages for .

Golint complaints on 's function because of the snake case. When I try to fix it, fails because it is being used in a code generated file. If I clean generated files, tests pass, but the generated file is not invoking it anymore. I failed to find the generator definition using this so I could change the hook to just . Also I'm not sure if this is wanted. I also found other packages in also probably affected by this. I read somewhere about replacing golint with gometalinter, which would allow to define directives in code to omit certain rules. I would consider to have a second exclusion file for golint with these cases, to be updated if move to gometalinter is finally done. This would make this issue completion more feasible. Meanwhile, I'll move onto fixing golint messages for other packages, until this discussion gets cleared.
I also found a problem fixing . A test fails because the directory is marked as readonly (see tests @ Shall we also ignore this directory for this issue?
Is this issue still relevant because hack/verify- is giving me no errors.
the failures are still present in there are details in the first message on how to claim + solve issues :)
Sorry I misunderstood the issue
I'm working on this package:
pkg/apis/admission/v1beta1 pkg/apis/authentication/v1beta1 pkg/apis/authorization/v1beta1 pkg/apis/coordination/v1 pkg/apis/coordination/v1beta1 pkg/apis/imagepolicy/v1alpha1 pkg/apis/scheduling/v1alpha1 pkg/apis/scheduling/v1beta1 pkg/apis/settings/v1alpha1 pkg/apis/storage/v1alpha1
pkg/cloudprovider/providers/vsphere/vclib pkg/kubelet/apis/deviceplugin/v1alpha pkg/kubelet/apis/pluginregistration/v1alpha1 pkg/kubelet/apis/pluginregistration/v1beta1
:
I'm looking at the pkg/registry/*, all those which are left. I expect to give an update on what all is fixed in the next 24-48 hours.
hi there is a google sheet that is a bit outdated, maybe you can update that?
hi I seem to only have view access for the sheet. Can't edit.
I fixed the permission.

Please try it again
pkg/apis/authentication pkg/apis/coordination pkg/apis/events pkg/apis/imagepolicy pkg/apis/settings
hi I think this issue was closed prematurely by (using fixes #issue in commit)
I'll put in a PR shortly for the package
Looking at next
hi thanks! there is a spreadsheet here that tracks the progress on fixing these issues, consider adding the packages you're work on there so we can keep everything organized :+1:
Starting on now
I did indeed see the Google Sheet link (thank you and to take it a little further
test/e2e
test/e2e_node/builder
pkg/registry/admissionregistration/mutatingwebhookconfiguration/storage pkg/registry/admissionregistration/rest pkg/registry/admissionregistration/validatingwebhookconfiguration/storage pkg/registry/apps/daemonset/storage
Hi. I opened a PR - Please review.
Working on Will open a PR soon.
Starting on and should have a PR soon
I keep getting this golint verify error on and was wondering if others saw it: But when I that package to the golint failures in my PR, my reviewer said it isn't necessary? What does that mean? What did I miss? Should I use a different kind of verify? cf:
looks like the disconnect is because that problem does not show up in the version of we use in the CI. please see
Still confused: is that not the golint version I ran? it installs from vendor and uses that I think?
i installed from and i still don't see the problem, can you please check what version of golint you have?
I'm confused why I wouldn't be using the vendored version? how would that get out of sync?
/me looking
Note: I am reserving for a KubeCon Contributor Workshop demonstration. It is noted in the spreadsheet.
I think may not be ideal cause there's a lot of auto generated files there. What do you think about reserving instead for the KubeCON Contributor Workshop?
pkg/apis/scheduling pkg/apis/storage/util pkg/apis/storage/v1beta1/util pkg/apis/storage/v1/util
/priority backlog
Hi I am a new contributor here, I'd like to contribute and make changes to the following file/s. I have marked it as WIP in the Google Sheet. Can someone let me know if I have to do anything else before I start working on this PR?

Umm so there's an issue where Golint tells that underscore shouldn't be allowed. Here's a log of the go-lint warnings:
Thanks The following should not be modified, as they are designed like this for code generation.
I'd like to pick a package and do my first PR, is there a list of packages to be done? Or should I correlate with this issue thread? Thanks in advance!
please see step 1 () please scan other PRs linked to this issue as well just in case.
Will do. Thank you. On Thu, Oct 3, 2019, 7:25 AM Davanum Srinivas <:
Hi there, I'm looking for a good first issue. I'll start taking a look at if that's ok? There doesn't seem to be anyone currently tracked as looking at it in the attached spreadsheet.
I've found a few others I'm actively looking at, you won't get any argument from me. On Sat, Oct 12, 2019, 10:38 AM Pamela Kelly <:
I am working on the iptables pkg. Updated in the google docs
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I have problem running in step 2 $ hack/verify- what should I do?
PTAL for my pr to this issue, thx
you can move kubernetes directory to and re-run this script.
I'd like to take Can we fix linter issues for generated files? Or we should skip packages with zz_generated?
There're 2 lint errors that affects 292/507 packages, meaning it's not possible to cleanup the entire file without solving it: default pattern of Kubectl is the usage of [cmd]Options for initial options structs and it raises: /apis/{custommetrics,externalmetrics} for example raises the underscore error, but It's probably not worth to break backward compatibility renaming these folder for lint error fix: Any advice on these?

/reopen hm, cannot reopen this issue, it was closed by mistake
You can't reopen an issue/PR unless you authored it or you are a collaborator. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I am sure this is all not fixed. Am I missing something? ./hack/verify- installing golint from vendor Congratulations! All Go source files have been linted.
you need to remove package from first.
/remove-sig testing SIG testing doesn't own code failing lints, we own the linter. :upsidedownface:
Hello, is this still a relevent issue ? I would like to work upon
I have fixed
Hi Folks, Some of the golint fixes, require us to change the exposed function's name, in simple words, "change API". I was not sure, if that is welcomed and what is the process.
Heres what Im getting after going over the following steps as mentioned by Say your GOPATH looks like this GOPATH=/Users/dims/go Your kubernetes directory should be ($) cd to and then run hack/verify- from that directory. I get the following output - installing golint Congratulations! All Go source files have been linted. Im not really sure what I am doing wrong. Any help is appreciated!
I think I understand. I misunderstood the steps around .golint_failures. The first step is to remove the pkg that we are targeting from this file and then run the My mistake was to assume that the packages in the file need to be removed after we are done cleaning the golint errors.
Can someone please review and let me know if I've done this right?
After a couple years of seeing changes spawned from this issue, I don't think this is a good use of contributors' or reviewers' time.

golint is very opinionated about: exported type and function names, leading to PRs attempting to change exported things people use downstream missing documentation on interface implementation functions, which leads to not-very-useful copy/paste of interface godoc the govet and staticcheck verify jobs are much more useful since they catch correctness issues, but I would recommend closing this issue and dropping the golint checks from CI golint doc itself discourages from using its output in an automated enforced way: from
i guess one can say gofmt is opinionated too, but something good about Golang is that it comes with an opinion and a set of standard tools. for API documentation at least we found inconsistencies on how most exported API items are documented: golint can help with that, but if there are no "pragmas" to avoid breaking changes and given we have codegen generating non-compliant names enabling golint on API packages is sort of blocked in Kubernetes. for non-API, non-library, non-public code, i think being inconsistent is mostly OK. no preference on my side. shrug
true, but code formatting has the benefits of mostly eliminating meaningless diffs in PRs due to code movement/whitespace/etc if there are specific packages (like API packages) we lack documentation on, adding checks specifically for that could make sense exactly, I don't think golint is the right tool for making sure we have API docs
golint itself doesn't have these and the go project is against this sort of comment AIUI, however golangci-lint does have them. I think there are plenty of other static analysis lints we should use instead, beyond just staticcheck and vet there is e.g.
btw, if we remove golint can we see e.g. all-caps constants or functions with underscores dodging review? perhaps we should add golangci-lint soon after:
Completely agree there are better places to spend review bandwidth. I was curious to see what the effort looked like. There have been 890 commits that touched over the past 3+ years !
/remove-help /remove-good-first-issue
Overall agree on the bandwidth. Also note that it was a good first issue with very low bar to get them in the repo. May be we need to find a similar low bar, but high value good first issue.

UT's come to mind, are there other things we curate for new contributors ?
Do we have an alternative way of enforcing that new exported code (not necessarily API fields) has at least some minimum documentation? As a reviewer, I ask contributors to add meaningful comments. But reality is that I sometimes stumble into old code that doesn't have documentation because we didn't have golint at some point. I agree that fix-golint PRs just add burden to reviewers. But, before removing the lint check, can we wait and see if it's enough to discourage people from sending more PRs like this? Additionally, we can encourage reviewers to just close PRs that only fix golint issues.
+1 for adding burden to reviewers. However, as a new contributor, I think this a very good opportunity for new contributors to study the code base and submit their first PR. In my opinion, I think this issue would be able to attract more new contributors, get familiar with the code base, and get familiar with the git workflow.
thanks everyone for getting us this far. We essentially gave up on the existing structure/system for golint: At some point we can look at adding golangci-lint, there's a WIP in progress in /close
Closing this issue. <detailsIn response to : thanks everyone for getting us this far. We essentially gave up on the existing structure/system for golint: Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: oomWatcher, err := oomwatcher.NewWatcher(kubeDeps.Recorder) if err != nil { <del> if libcontaineruserns.RunningInUserNS() { </del> <ins> if inuserns.RunningInUserNS() { </ins> if utilfeature.DefaultFeatureGate.Enabled(features.KubeletInUserNamespace) { // oomwatcher.NewWatcher returns "open /dev/kmsg: operation not permitted" error, // when running in a user namespace with sysctl value `kernel.dmesg_restrict=1`.

is no longer compilable on macOS in the master branch. It was compilable with Kubernetes v1.29.0 and v1.30.0-alpha.0. It should be compilable. No response <detailsmaster () (This is just the latest commit, not the commit that introduced the regression) </details<detailsN/A </details<detailsmacOS 14.2.1 (Intel) </details<detailsN/A </details<detailsN/A </details<detailsN/A </details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig testing
Regression in:


Question: // https://github.com/kubernetes/kubernetes/issues/72236 kubeIPVSFilterChain utiliptables.Chain = "KUBE-IPVS-FILTER" <ins> // kubeIPVSOutFilterChain filters access to load balancer services from node. // https://github.com/kubernetes/kubernetes/issues/119656 kubeIPVSOutFilterChain utiliptables.Chain = "KUBE-IPVS-OUT-FILTER" </ins> // defaultScheduler is the default ipvs scheduler algorithm - round robin. defaultScheduler = "rr"

When using loadBalancerSourceRanges to allow access only from CIDRs outside the cluster segment, access from the K8s nodes (or pods with hostNetwork:true) should be rejected, but for proxy-mode=ipvs it works. /sig network /area kube-proxy /area ipvs Access from K8s nodes should be rejected in the same way as for proxy-mode=iptables Define a service with loadBalancerSourceRanges that allows access from a cidr outside the cluster, e.g. The K8s nodes have addresses on 192.168.1.0/24 and fd00::c0a8:100/120. Make sure a loadBalancerIP is assigned (in this example 10.0.0.0/32) and try to access it from a K8s node: Same for both ipv4 and ipv6. This is related to since a fix requires that the passed to the is correct. Please see v1.27.4 and master None Linux 6.4.4 None N/A N/A
wanted to work on this
Great! But please coordinate with I hope that one get accepted, and then ~the s (both families) passed to the proxiers should be usable.~ No, they are not, please see
Discussed this one in the SIG Network meeting today: we agree this is something we'd like to implement, however we weren't sure exactly how to do it with ipvs during the meeting. Whoever takes this on needs to come up with and communicate a clear plan of how it will be accomplished. /triage accepted
My comment is not feasible. For proxy-mode=ipvs the src and dst is both loadBalancerIP. So the nodeIP will not help :disappointed:
I did a small experimented with kind + metallb. I created a loadbalancer service backed by . loadBalancerIP: 172.18.255.200 podIP: 10.244.1.3 conntrack picks the src and dst both as loadBalancerIP. Tried to create an iptables rules matching source and destination with loadbalancerIP to drop packets but it didn't work. After some exploration I found and with the following rules I was able to prevent access to lbIP from k8s nodes / host network.

The solution looks hacky though, we only need this rule if source ranges are defined for the service and node ip is not covered by any of them. Will try to program this in the ipvs proxier.
Good find! :+1: I had missed the ipvs extension. I started to check nodeIP and the (flawed) option to experiment with setting a nodeIP as "src" for some routes to dodge the lbip-both-as-src-and-dst problem, and an old "vision" I have to not assign clusterIPs to at all, but have one rule to replace them all, like: It can be done, and in a cluster with 1000's of clusterIP's it would simplify


Question: Rule: admissionregistrationv1.Rule{ APIGroups:   []string{""}, APIVersions: []string{"v1"}, <del> Resources:   []string{"*"}, </del> <ins> Resources:   []string{"configmaps"}, </ins> }, }}, ClientConfig: admissionregistrationv1.WebhookClientConfig{

/kind flake <!-- If this is a flake, please add: /kind flake --/sig storage
The last time I see the flake is in
/assign /sig apimechinery It's broken by a validation webhook with match conditions which is created by
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : /assign /sig apimechinery Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig api-machinery
I have raised a pr to fix it. can you help to review it?
/close duplicate of and thanks for the fix PR too.
Closing this issue. <detailsIn response to : /close duplicate of and thanks for the fix PR too. Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/triage accepted


Question: github.com/valyala/tcplisten => github.com/valyala/tcplisten v0.0.0-20161114210144-ceec8f93295a github.com/vishvananda/netlink => github.com/vishvananda/netlink v0.0.0-20171020171820-b2de5d10e38e github.com/vishvananda/netns => github.com/vishvananda/netns v0.0.0-20171111001504-be1fbeda1936 <del> github.com/vmware/govmomi => github.com/vmware/govmomi v0.20.1 </del> <ins> github.com/vmware/govmomi => github.com/vmware/govmomi v0.20.3 </ins> github.com/xiang90/probing => github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2 github.com/xlab/handysort => github.com/xlab/handysort v0.0.0-20150421192137-fb3537ed64a1 github.com/xordataexchange/crypt => github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77

Expectation- We should be able to create PVCs via a storageclass with storagePolicyName defined while using a singed ca cert for secure communications with vCenter. Bug- The PVC created with a storageclass with storagePolicyName defined fails with a error when PVCs created with a storageclass without the storagPolicyName work against the same vCenter and datastore. Scenario: Utilizing the vSphere in-tree plugin on kubernetes 1.12.7. Utilizing secure communication with vSphere (signed ca cert imported into the cluster) We are able to create PVCs with the following storage class: However, if we include a storagePolicyName we receive a failure: Here is the yaml for our sc1 storage class: /sig vmware
I think we are having issue with creating PBM client using certificate set on the vCenter Client. We need to create a PBM client to get list of compatible datastore for given storage policy. I think call is failing at - Can you please provide controller-manager logs for further debugging. Make sure log level is set to 4.
Sorry for the delayed response. I'm not confident I got the log level correct, however this is what I find:
I believe I have log level 4 now set:
For comparison, here is a PVC created w/o storagePolicyName on the same datastore which succeeds:
thanks for posting logs. Now this is confirmed, issue is happening while creating PBM client when we use certificate based authentication. One more thing I want to confirm regarding the certificate, are you using solution user as mentioned here -
Can you also provide value of in the file? Is it set to true or false / 1 or 0 ? Also confirm version of vCenter you are using.
is unset, as for the solution user I'd have to leave that to to expand on.
I don't believe we used any solution user because we simply used "" as the default user during installation. please correct me if I'm wrong.
How is the certificate and private key retrieved for the user . I am successfully able to connect to the PBM using vpxd certificte and key. I ssh into the vCenter and executed following commands to retrieve certificate and private key for vpxd. with vpxd cert and private key, I do not see this issue. In addition I also set while making connection.

Can you try setting Insecure to false? That's the flag setting that triggers the cert error.
with , I see the exact same error - , while creating VC client.
Isn't that because it's self-signed? In the case of the issue in this ticket the cert is signed by a trusted CA /cc to confirm.
The cert is self-signed and its CA isn't trusted.
I used vpxd certificate and issuer is I require to set .
Is there any way to make it work if insecure=false?
- then I’d say that’s expected? If the root CA isn’t trusted, then it is insecure?
well, while that's expected, I don't know if there's a method to add the root CA to a trusted list.
Correct me if I’m wrong but isn’t the trusted list the root CA store of the K8S node’s OS in this case? I’d imagine whatever CA Root signed the cert would need to be to the OS of choice’s CA store.
you are correct, we need to download trusted root certificate and add them in the store on node vm. For now on my mac, I have downloaded trusted root certificate from the vCenter -and them in the keychain and flagged as "Always Trust". ! After this, I am able to connect to VC client with or without specifying this flag at all using VPXD certificate and private key. So certificate needs to be to the CA trust store on the master node VM. after adding trusted root certificate from vCenter to the trust store, you should be able to make a connection using .
After importing the root cert by placing it in /etc/ssl/certs on the master node, I'm able to provision a PVC while specifying a storage policy. Thanks for all the help.
Can you close this issue and work with KK to get this automated in the product?
It turns out that importing the root cert into the host was just a workaround, and the actual bug was fixed in That fix is included in govmomi v0.21.0.


Question: By(test.description) if test.messageNum > 0 { By(fmt.Sprintf("Inject %d logs: %q", test.messageNum, test.message)) <del> err := injectLog(tmpLogFile, test.timestamp, test.message, test.messageNum) </del> <ins> err := injectLog(hostLogFile, test.timestamp, test.message, test.messageNum) </ins> Expect(err).NotTo(HaveOccurred()) }

Leads to this avc: In general, HostPath volumes in tests are bad news for selinux.


Question: accessModes: - ReadWriteOnce hostPath: <del> path: /tmp/pv-1 </del> <ins> path: /tmp/data/pv-1 </ins> --- apiVersion: v1 kind: PersistentVolume

Version: Running this example on RHEL 7.2 (assume same for centos), and using hostPath, I get a permissions error when the mysql container does a . If I set selinux to permissive on the host it works. If I manually run the container with it works. I was not able to get it to work when adding to the deployment. My fix was to locate the PVs in by changing the local- file, and doing . I this comment to local- If this issue is valid I will create a quick PR to doc this selinux solution.
See pr


Question: if err == nil { // Volume is already attached to node. <del> glog.V(4).Infof("Attach operation is successful. volume %q is already attached to node %q at lun %d.", volumeSource.DiskName, instanceid, lun) </del> <ins> glog.V(2).Infof("Attach operation is successful. volume %q is already attached to node %q at lun %d.", volumeSource.DiskName, instanceid, lun) </ins> } else { <del> glog.V(4).Infof("GetDiskLun returned: %v. Initiating attaching volume %q to node %q.", err, volumeSource.DataDiskURI, nodeName) </del> <ins> glog.V(2).Infof("GetDiskLun returned: %v. Initiating attaching volume %q to node %q.", err, volumeSource.DataDiskURI, nodeName) </ins> getLunMutex.LockKey(instanceid) defer getLunMutex.UnlockKey(instanceid)

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . If the matter is security related, please disclose it privately via --Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: Current logging is not enough for azure disk diagnostics, change some log level from 4 to 2 What you expected to happen: How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others: /sig azure /assign


Question: } // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. <ins> func (in *IPPortVar) DeepCopyInto(out *IPPortVar) { *out = *in if in.Val != nil { in, out := &in.Val, &out.Val if *in == nil { *out = nil } else { *out = new(string) **out = **in } } return } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new IPPortVar. func (in *IPPortVar) DeepCopy() *IPPortVar { if in == nil { return nil } out := new(IPPortVar) in.DeepCopyInto(out) return out } // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. </ins> func (in *IPVar) DeepCopyInto(out *IPVar) { *out = *in if in.Val != nil {

Whenever I try to change kube proxy metrics bind using flag , it gives me: Then, I the port , the error changes. I expect the metrics server to listen to 0.0.0.0:. Do I miss something? Environment: Kubernetes version: v1.7.8 Cloud provider or hardware configuration**: on-premise OS (e.g. from /etc/os-release): debian/stretch Kernel (e.g. ): Linux R11U2-7U20N-1 4.9.0-3-amd64 SMP Debian 4.9.30-2+deb9u3 (2017-08-06) x86_64 GNU/Linux Install tools: Others:
/sig network /area kube-proxy
The is being and does NOT allow specifying a port. Have you tried using a config file? The field allows you to specify a port (an example ).
/assign
Turned out we use a wrong type when validating and . Sent for fixing this. Will try to cherrypick the fix as well.
New finding, is broken on HEAD in a different way: Seems to be introduced by
Humm..We have some default logic for : Seems like for some reason it does not auto append the default port number OR the validation happens before defaulting.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. Prevent issues from auto-closing with an comment. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten /remove-lifecycle stale
/remove-lifecycle rotten
I got the same issue 0.0.0.0 0.0.0.0: both not work with kube-proxy 1.9.3 amd64
It is a BUG. Conflict between componentconfig.IPVar.Set(s string) and validation.validateHostPort.

Could you elaborate the issue a bit more? Which k8s version are you on?
I also meet the same issue with kubernetes 1.8. It's impossibe for Prometheus to fetch metrics from 127.0.0.1: in my case, any workaround? I think we'd better reopen this issue.
metrics binds to by default, are you trying to modify port as well? Thanks for reporting, I confirm the issue is remaining. (or any ip:port) does not work as intended even after the fix. Reopening this and filed a new issue for conflict kubeproxyconfig validation. /reopen
Thanks for reopening the issue. I want to bind metrics to 0.0.0.0: as well. With address 127.0.0.1:, Prometheus cannot pull metrics from this address unless they are on the same node.
is fixing this.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale (I just got bitten by this on kube-proxy v1.9.10)
should fix this.
Closing this as is merged. /close
Closing this issue. <detailsIn response to : Closing this as is merged. /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: } }) <ins> It("should use same NodePort with same port but different protocols", func() { serviceName := "nodeports" ns := f.Namespace.Name t := NewServerTest(c, ns, serviceName) defer func() { defer GinkgoRecover() errs := t.Cleanup() if len(errs) != 0 { framework.Failf("errors in cleanup: %v", errs) } }() By("creating service " + serviceName + " with same NodePort but different protocols in namespace " + ns) service := &api.Service{ ObjectMeta: api.ObjectMeta{ Name:      t.ServiceName, Namespace: t.Namespace, }, Spec: api.ServiceSpec{ Selector: t.Labels, Type:     api.ServiceTypeNodePort, Ports: []api.ServicePort{ { Name:     "tcp-port", Port:     53, Protocol: api.ProtocolTCP, }, { Name:     "udp-port", Port:     53, Protocol: api.ProtocolUDP, }, }, }, } result, err := t.CreateService(service) Expect(err).NotTo(HaveOccurred()) if len(result.Spec.Ports) != 2 { framework.Failf("got unexpected len(Spec.Ports) for new service: %v", result) } if result.Spec.Ports[0].NodePort != result.Spec.Ports[1].NodePort { framework.Failf("should use same NodePort for new service: %v", result) } }) </ins> It("should prevent NodePort collisions", func() { // TODO: use the ServiceTestJig here baseName := "nodeport-collision-"

Consider a service like The API refuses to accept this service, stating It works fine without . It works when using different ports. The iptables rules installed (in mode) are TCP/UDP specific. I'm not sure if this is deliberate but not well documented, or a bug? I don't see a reason not to allow a UDP and TCP node port to have the same number.
For context, I am trying to make the cluster DNS available to legacy systems with a bit of forwarding, and DNS really really wants to have UDP and TCP on the same port number.
what is your kubernetes version?
This is on 1.1.4. From a quick reading of the code this hasn't changed though.
This was intentionally omitted due to laziness and complexity. We hoped nobody would need it, since NodePorts are generally allocated automatically. It's mostly going to be a matter of adding a switch to look at 2 different saved allocation maps and figuring out how to transition. , Matthias Rampke wrote:
I've also hit this issue (see: ) when trying to expose DNS to queries from outside the cluster. Note that in my setup, the "NodePort" is different for UDP and TCP but only UDP appears to work (it's the first to be defined?).
I also run into this issue. I've got a service that wants to expose TCP and UDP port with the same number. Any plans to include this issue in any roadmap?
Help wanted. We're buried in high-priority bugs and features. I ACK that this behaves imperfectly, but I and my team will not have time to look at it in the immediate future.
FYI
I think the root cause is , when serviceport allocator attempts to reserve the provided port, it just use the port number not consider the port protocol. This is a bug of service port allocator I believe.
"bug", but yes. We'd need to store parallel bitmaps per protocol. Even then, it's not obvious when to use the same port number in different protocols, and whether that is REQUIRED or not.

If you have a Service with port 53 in TCP and UDP, and I can't find any port number that is available in both protocols, should I: a) fail and reject the REST operation b) choose a different port number c) retry forever and hang the REST operation d) accept the REST operation but log an event saying I failed to do my job NodePort was really made for load-balancers, where the port number doesn't matter. Humans using it is "unexpected". , TonyAdo wrote:
this is what I'd expect. Port exhaustion is not really a temporary Matthias Rampke Engineer SoundCloud Ltd. +49 173 Managing Director: Alexander Ljung | Incorporated in England & Wales with Company No. HRB 110657B
I am afraid PR will not fix this issue. We have to store parallel bitmaps per protocol. I will open a new PR to fix this issue.
This alone is insufficient, but we might get away without storing two maps by simply saying the port map is unified across protocols if a service has 2 ports that use the same port number but different protocols, allocate the first nodePort and use the same value for the 2nd port This will prevent the case of ports being used in one protocol but not the other and failing to allocate. Something like: I haven;t tried too hard to break it, but something like that MIGHT work.
Additionally, if you have to store multiple bitmaps, you have a serious hassle on your hands for upgrading cluster in-place. Better to avoid it if you can.
sure, I will think about this more.

Hi, I have created and deployed a service similar to below in minikube environment.. however I am not to connect to the ports sometimes TCP/UDP ` apiVersion: v1 kind: Service metadata: name: dns spec: selector: … clusterIP: … type: NodePort ports: name: grpc protocol: TCP port: targetPort: grpc-port name: port-udp port: targetPort: 5062 protocol: UDP nodePort: name: port-tcp port: targetPort: 5062 protocol: TCP nodePort: Example: $kubectl get all service/-service NodePort 10.106.159.94 <none50051:/TCP,30:/UDP,:/TCP $ telnet 192.168.99.100 telnet: can't connect to remote host (192.168.99.100): Connection refused $ telnet localhost telnet: can't connect to remote host (127.0.0.1): Connection refused $nc -z -v -u 192.168.99.100 Connection to 192.168.99.100 port [udp/] succeeded! $ nc -z -v 192.168.99.100 nc: connect to 192.168.99.100 port (tcp) failed: Connection refused $ kubectl version Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.1", GitCommit:"", GitTreeState:"clean", BuildDate:"2019-10-02T17:01:15Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"", GitTreeState:"clean", BuildDate:"2019-09-18T14:27:17Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"} $netstat -an tcp 0 0 ::: ::: LISTEN udp 0 0 ::: :::*
!


Question: fs.MarkHidden("experimental-dockershim") fs.StringVar(&s.DockershimRootDirectory, "experimental-dockershim-root-directory", s.DockershimRootDirectory, "Path to the dockershim root directory.") fs.MarkHidden("experimental-dockershim-root-directory") <del> fs.BoolVar(&s.DockerDisableSharedPID, "docker-disable-shared-pid", s.DockerDisableSharedPID, "Setting this to false causes Kubernetes to create pods using a shared process namespace for containers in a pod when running with Docker 1.13.1 or higher. A future Kubernetes release will make this configurable instead in the API.") </del> <ins> fs.BoolVar(&s.DockerDisableSharedPID, "docker-disable-shared-pid", s.DockerDisableSharedPID, fmt.Sprintf("Setting this to false causes Kubernetes to create pods using a shared process namespace for containers in a pod when running with Docker 1.13.1 or higher. A future Kubernetes release will make this configurable instead in the API. %s", dockerOnlyWarning)) </ins> fs.MarkDeprecated("docker-disable-shared-pid", "will be removed in a future release. This option will be replaced by PID namespace sharing that is configurable per-pod using the API. See https://features.k8s.io/495") <del> fs.StringVar(&s.PodSandboxImage, "pod-infra-container-image", s.PodSandboxImage, "The image whose network/ipc namespaces containers in each pod will use.") fs.StringVar(&s.DockerEndpoint, "docker-endpoint", s.DockerEndpoint, "Use this for the docker endpoint to communicate with") fs.DurationVar(&s.ImagePullProgressDeadline.Duration, "image-pull-progress-deadline", s.ImagePullProgressDeadline.Duration, "If no pulling progress is made before this deadline, the image pulling will be cancelled.") </del> <ins> fs.StringVar(&s.PodSandboxImage, "pod-infra-container-image", s.PodSandboxImage, fmt.Sprintf("The image whose network/ipc namespaces containers in each pod will use. %s", dockerOnlyWarning)) fs.StringVar(&s.DockerEndpoint, "docker-endpoint", s.DockerEndpoint, fmt.Sprintf("Use this for the docker endpoint to communicate with %s", dockerOnlyWarning)) fs.DurationVar(&s.ImagePullProgressDeadline.Duration, "image-pull-progress-deadline", s.ImagePullProgressDeadline.Duration, fmt.Sprintf("If no pulling progress is made before this deadline, the image pulling will be cancelled. %s", dockerOnlyWarning)) </ins> // Network plugin settings for Docker. <del> fs.StringVar(&s.NetworkPluginName, "network-plugin", s.NetworkPluginName, "<Warning: Alpha feature> The name of the network plugin to be invoked for various events in kubelet/pod lifecycle") fs.StringVar(&s.CNIConfDir, "cni-conf-dir", s.CNIConfDir, "<Warning: Alpha feature> The full path of the directory in which to search for CNI config files. Default: /etc/cni/net.d") fs.StringVar(&s.CNIBinDir, "cni-bin-dir", s.CNIBinDir, "<Warning: Alpha feature> A comma-separated list of full paths of directories in which to search for CNI plugin binaries. Default: /opt/cni/bin") fs.Int32Var(&s.NetworkPluginMTU, "network-plugin-mtu", s.NetworkPluginMTU, "<Warning: Alpha feature> The MTU to be passed to the network plugin, to override the default. Set to 0 to use the default 1460 MTU.") </del> <ins> fs.StringVar(&s.NetworkPluginName, "network-plugin", s.NetworkPluginName, fmt.Sprintf("<Warning: Alpha feature> The name of the network plugin to be invoked for various events in kubelet/pod lifecycle. %s", dockerOnlyWarning)) fs.StringVar(&s.CNIConfDir, "cni-conf-dir", s.CNIConfDir, fmt.Sprintf("<Warning: Alpha feature> The full path of the directory in which to search for CNI config files. Default: /etc/cni/net.d. %s", dockerOnlyWarning)) fs.StringVar(&s.CNIBinDir, "cni-bin-dir", s.CNIBinDir, fmt.Sprintf("<Warning: Alpha feature> A comma-separated list of full paths of directories in which to search for CNI plugin binaries. Default: /opt/cni/bin. %s", dockerOnlyWarning)) fs.Int32Var(&s.NetworkPluginMTU, "network-plugin-mtu", s.NetworkPluginMTU, fmt.Sprintf("<Warning: Alpha feature> The MTU to be passed to the network plugin, to override the default. Set to 0 to use the default 1460 MTU. %s", dockerOnlyWarning)) </ins> }

Is this a BUG REPORT or FEATURE REQUEST?: /kind feature What happened: When use remote container runtime, the remote can't get any PodSandboxImage config set by kubelet What you expected to happen: We want the all container runtime should know what's right PodSandboxImage How to reproduce it (as minimally and precisely as possible): in new node with into k8s cluster kubelet with options expect frakti can create sandbox container with , but get default (k8s version: v1.8.2) Anything else we need to know?: Environment: Kubernetes version (use ):Cloud provider or hardware configuration: bootstrap by kubeadm v1.8.2 OS (e.g. from /etc/os-release): Debian GNU/Linux 9 (stretch) Kernel (e.g. ): SMP Debian 4.9.30-2+deb9u2 (2017-06-26) x8664 GNU/Linux Install tools: kubeadm Others:
/sig node
good news! thanks!
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
Actually we do a no-op when the container runtime is . This is why config can not be set by flag for . In your case, the correct place to change the default sandbox image is in .


Question: if lastGraceful > 0 { return out, nil } <del> // fall through and delete immediately </del> <ins> // If we are here, the registry supports grace period mechanism and // we are intentionally delete gracelessly. In this case, we may // enter a race with other k8s components. If other component wins // the race, the object will not be found, and we should tolerate // the NotFound error. See // https://github.com/kubernetes/kubernetes/issues/19403 for // details. ignoreNotFound = true // exit the switch and delete immediately </ins> case errDeleteNow: // we've updated the object to have a zero grace period, or it's already at 0, so // we should fall through and truly delete the object.

cc
PTAL
The root cause is that the node controller and kubelet delete the pods using their names (they should use UID instead): (this is not an exhaustive list) In test-, we create and delete the same pod multiple times, so it's possible that the node controller or the kubelet sends a deletion request to the api server, with the intention of deleting the Nth occurrence of a pod, while it's actually deleting the N+1th occurrence of the pod. To reproduce the flake, I introduced a random delay between 500~1500 ms in the node controller's execution path (), then using a script to create/delete a pod repeatedly, and the flake will occur after several rounds.
Why is it not okay to delete by name?
namespace+name pair is not an unique identifier across time.
the client library and the API server don't support deletion by UID, I think we should support.
Thanks!! My thoughts: does node controller care about these pods? I'm pretty sure the original author wasn't expecting that. a temporary fix, is it hard to just use new names for every incarnation of these pods?
Deleting by UID is probably a bigger can of worms than we want to open right now, although it will eventually be important. Thinking out loud here, but maybe it'd be possible to add a resource version to the delete command. I.e., "delete only if it hasn't changed since rv=xxxxx". But I still think it's probably ideal to turn off the node controller for this test.
I'm also thinking if we can centralize all the pod deletion logic into kubelet, but it seems some of the logic need to stay in the node controller, e.g, , because only the node controller knows if the node exists. deletion probably can be done by the kubelet.
Looks like node controller is getting an update of the pod where DeleteionTimestamp is set, which seems like a bug if --grace-period is set to 0. My understanding is that if grace period is zero, the pod should get immediately deleted. That pod will never make it to a kubelet because it hasn't been scheduled yet. Thus which kubelet would delete it? I'm pretty sure none of those deletions can be handled in the kubelet.

Btw, when running hack/test- (), the node controller generates many messages about the failed deletions: or , these are because kubelet wins the race and has deleted the pods. These don't cause test- to fail.
per offline discussion with Mike, I think he's right. I'll check the api server's code.
That's my expectation, too (that grace period being zero should bypass all graceful deletion code and just immediately delete the pod).
When the grace period is zero, api server update the storage(), and then delete the pod from the storage (). The check of if grace period is 0 is after the update(). It seems it should not do the update if grace periods is zero.
Do you agree that the above is a bug?
Hm, is the problem a missing or incorrect BeforeDelete handler?
I think we are dealing with two problems here: grace-period=0, the registry shouldn't do an update. client library should provide a Delete() method that takes an unique identifier, the "name+namespace" pair is ambiguous across time.
Thinking about this more, I wonder if the update is intentional. If so, we need some coordination so that both apiserver and kublet don't try to delete.
Grace period 0 is intentional, but may not be required for correct behavior. If the user signals delete immediately, we record that intent so it shows up in watch and can be a signal kubelet to terminate immediately (although we don't actually do that today, so we could walk back the behavior). We definitely need a precondition delete on uid.
There will never be a single owner of pod lifecycle because pods are bound to nodes, and so all components have to deal with races to delete.
I chose to err on the side of being explicit about user intent with the update to zero out termination. But technically explicit deletion can be the same signal.
A delete operation will also trigger a watch event, so kubelet will be signaled even if we skip the update right? [edit] Oh I see you answered right before I asked.
It should be, yes. In the interest of fixing this test in the short term, I think the easiest thing to do might be to turn off the node controller for this section of the test. (Is it needed by anything in the test?)

But we should file at least two issues: "kubectl delete can race with graceful deletion"-- Things have to handle races, but there shouldn't be a race where you delete something and your delete fails because your own delete caused another component to issue a delete for the same thing... As far as I can see from the code, this could happen if apiserver context-switched at the wrong times (even if the more common thing of an old delete getting a new object is the issue at hand). controllers should delete via UID (or namespace/name + RV) -- I wouldn't be surprised if there's already an issue for this.
+1 to resourceVersion and uid preconditions on delete. Yes, maybe we shouldn't update the grace period before deletion in the case that the grace period is 0. Why can't we use unique pod names in the test? Turning off node controller seems like a bad idea. Index by uid is .
I don't see any flag option to turn of node controller. Pod's name is specified in the config file, is there any clean way to change the pod's name each time is called? Anyway, I'll try to make a PR to skip update if grace period is 0. That should alleviate this flake.
cc/ in case you are running into this issue.
As for how to change the name: we could use sed to replace the name. Or we could change the namespace.
This should be fixed by . I filed for deletion via UID or namespace/name+resourceVersion.
This happened again:
This error "] pods "redis-proxy" not found" here seems related to the error in .
I think the issue in is the same. The last few lines of logs: The symptom seems to align with my hypothesis in suggests the nodecontroller has successfully deleted a pod (probably "redis-master"), so kubectl fails with . I'll add more logs in nodecontroller to make sure this is the problem. In fact, we knew this race condition when investigating , but we discovered other race condition that are more likely to happen so we didn't solve this one.
Sent to show what pod nodecontroller has deleted.
Anywhere where we delete lists of pods (and other resources), including those filtered by label selector, we should ignore not-found errors. There are such loops in the controller reapers for cascading deletion. cc
makes safely ignore the NotFound error.

forces to issue a "GET" before submitting "DELETE". The "GET" will effectively check if the user is trying to delete a resource that is never created, so if later the API server sends a NotFound error in response to the "DELETE" request, kubectl can safely ignore it.


Question: import ( "context" <del> "encoding/json" </del> "fmt" "strings" "time" <del> v1 "k8s.io/api/core/v1" </del> metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/util/wait" clientset "k8s.io/client-go/kubernetes" <del> "k8s.io/kubernetes/test/e2e/framework" "github.com/onsi/ginkgo" "k8s.io/apimachinery/pkg/types" </del> ) // Action is a function to be performed by the system. type Action func() error <del> var _ = ginkgo.Describe("[sig-api-machinery] Events", func() { f := framework.NewDefaultFramework("events") /* Release : v1.19 Testname: Event resource lifecycle Description: Create an event, the event MUST exist. The event is patched with a new message, the check MUST have the update message. The event is deleted and MUST NOT show up when listing all events. */ framework.ConformanceIt("should ensure that an event can be fetched, patched, deleted, and listed", func() { eventTestName := "event-test" ginkgo.By("creating a test event") // create a test event in test namespace _, err := f.ClientSet.CoreV1().Events(f.Namespace.Name).Create(context.TODO(), &v1.Event{ ObjectMeta: metav1.ObjectMeta{ Name: eventTestName, Labels: map[string]string{ "testevent-constant": "true", }, }, Message: "This is a test event", Reason:  "Test", Type:    "Normal", Count:   1, InvolvedObject: v1.ObjectReference{ Namespace: f.Namespace.Name, }, }, metav1.CreateOptions{}) framework.ExpectNoError(err, "failed to create test event") ginkgo.By("listing all events in all namespaces") // get a list of Events in all namespaces to ensure endpoint coverage eventsList, err := f.ClientSet.CoreV1().Events("").List(context.TODO(), metav1.ListOptions{ LabelSelector: "testevent-constant=true", }) framework.ExpectNoError(err, "failed list all events") foundCreatedEvent := false var eventCreatedName string for _, val := range eventsList.Items { if val.ObjectMeta.Name == eventTestName && val.ObjectMeta.Namespace == f.Namespace.Name { foundCreatedEvent = true eventCreatedName = val.ObjectMeta.Name break } } framework.ExpectEqual(foundCreatedEvent, true, "unable to find the test event") ginkgo.By("patching the test event") // patch the event's message eventPatchMessage := "This is a test event - patched" eventPatch, err := json.Marshal(map[string]interface{}{ "message": eventPatchMessage, }) framework.ExpectNoError(err, "failed to marshal the patch JSON payload") _, err = f.ClientSet.CoreV1().Events(f.Namespace.Name).Patch(context.TODO(), eventTestName, types.StrategicMergePatchType, []byte(eventPatch), metav1.PatchOptions{}) framework.ExpectNoError(err, "failed to patch the test event") ginkgo.By("fetching the test event") // get event by name event, err := f.ClientSet.CoreV1().Events(f.Namespace.Name).Get(context.TODO(), eventCreatedName, metav1.GetOptions{}) framework.ExpectNoError(err, "failed to fetch the test event") framework.ExpectEqual(event.Message, eventPatchMessage, "test event message does not match patch message") ginkgo.By("deleting the test event") // delete original event err = f.ClientSet.CoreV1().Events(f.Namespace.Name).Delete(context.TODO(), eventCreatedName, metav1.DeleteOptions{}) framework.ExpectNoError(err, "failed to delete the test event") ginkgo.By("listing all events in all namespaces") // get a list of Events list namespace eventsList, err = f.ClientSet.CoreV1().Events("").List(context.TODO(), metav1.ListOptions{ LabelSelector: "testevent-constant=true", }) framework.ExpectNoError(err, "fail to list all events") foundCreatedEvent = false for _, val := range eventsList.Items { if val.ObjectMeta.Name == eventTestName && val.ObjectMeta.Namespace == f.Namespace.Name { foundCreatedEvent = true break } } framework.ExpectEqual(foundCreatedEvent, false, "should not have found test event after deletion") }) }) </del> // WaitTimeoutForEvent waits the given timeout duration for an event to occur. func WaitTimeoutForEvent(c clientset.Interface, namespace, eventSelector, msg string, timeout time.Duration) error { interval := 2 * time.Second

[x] Original approval issue: [x] Two weeks soak end date: 6 July 2020 This issue is created to allow edit by /wip /hold According to this APIsnoop query, there are still an endpoint which is untested. a set of 3 Events with a static label within a Namespace. that all 3 Events with the label in the Namespaces are created. the set of Namespaced Events with a label via DeleteCollection. that all Events with the label have been deleted. Discover useragents: List endpoints hit by the test: Display endpoint coverage change: Two other endpoints tested here have tests in progress to conformance already. If a test with these calls gets merged, test coverage will go up by 1 points This test is also created with the goal of conformance promotion.
/sig testing /sig architecture /area conformance
During Conformance Meeting: Is there a way to write this test so it is resource agnostic? If we are just changing the resource... can be parameterize the resource-type
/assign
GitHub didn't allow me to assign the following users: heyste. Note that only , repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : /assign Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen PR Merge and soaking
Reopened this issue. <detailsIn response to : /reopen PR Merge and soaking Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: type are suitable for your use!) #### AWS EBS Example configuration: ```yaml apiVersion: v1 kind: Pod metadata: <del> name: testpd </del> <ins> name: test-ebs </ins> spec: containers: <del> - image: kubernetes/pause name: testcontainer </del> <ins> - image: gcr.io/google_containers/test-webserver name: test-container </ins> volumeMounts: <del> - mountPath: /testpd name: testvolume </del> <ins> - mountPath: /test-ebs name: test-volume </ins> volumes: <del> - name: testvolume </del> <ins> - name: test-volume </ins> # This AWS EBS volume must already exist. awsElasticBlockStore: volumeID: aws://<availability-zone>/<volume-id>

only talks about these volumes: EmptyDir HostPath GCEPersistentDisk NFS However, we many more volume types. Documentation should be updated to reflect this.
cc
Is anyone working on this? If not, I would like to do it.
go for it :) Improving PV docs is on my TODO list, too.
Sure, thank you. I will start now, hope to send a PR ASAP.


Question: .PP JSON and YAML formats are accepted. <ins> .PP Please refer to the models in [la]https://htmlpreview.github.io/?https://github.com/GoogleCloudPlatform/kubernetes/HEAD/docs/api-reference/definitions.html[ra] to find if a field is mutable. </ins> .SH OPTIONS .PP

The current documentation is very vague and doesn't explain what can be updated in this way. Can I rename or stop the pod, change number of replicas in RC etc. For example it seems to be impossible to change pod name:
cc
After an in person discussion with we think the docs should point people to the swagger-ui to find if a field is mutable. I'll add this point into the docs.


Question: v1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/resource" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" <ins> "k8s.io/apimachinery/pkg/types" </ins> "k8s.io/apimachinery/pkg/util/uuid" "k8s.io/kubernetes/pkg/kubelet/cm" "k8s.io/kubernetes/test/e2e/framework"

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: When I change default hugepage size 2Mi to 1Gi and reboot, kubelet can't update node status What you expected to happen: kubelet can update node info and node status was ready How to reproduce it (as minimally and precisely as possible): change the node condition validate Anything else we need to know?: Environment: Kubernetes version (use ):Cloud provider or hardware configuration:OS (e.g: ):Kernel (e.g. ):Install tools:Network plugin and version (if this is a network-related bug):Others:
/sig node
Hi, Ref. the error message, it looks like you have allocated hugepages for multiple sizes, . Can you run: to verify? If more than one of the numbers is , that is the reason. This is currently being discussed in , and the reason for implementing the check is here: Imo. this should be supported, and it would only require removing an if statement in order to implement it. I see no reason to limit the node from allocating huge pages with more than one size. Because of how it is implemented, adding support for allowing a pod to request huge pages with multiple sizes isn't trivial, and will require some potentially breaking changes, but that is not relevant to this problem. /cc
Also, can you also provide the result of , that will list the supported huge page sizes (both when is set, and when it is not.).
Thanks ! Looks like your kernel only enables huge pages with the default size. My kernel 5.0+ enable all (atleast 2MiB and 1GiB on x8664) by default. When the node booted with defaulthugepagesz=1GB it "removed support" the 2MiB one, and becuse kubelet use patch on the node status when connecting to the api server, you ended up with both and , even tho 2Mi wast supported by your machine (the sw, not the hw). Even if the error is removed, this will result in a crash when kubelet tries to write to the 2MiB hugetlb cgroup files.

I think using should work, otherwise the only current solution is to remove the node from the cluster, and rejoin afterwords. Ref. it looks like you have sorted out the problem, but I still think this is a valid issue, so don't close it. The correct behaviour for kubelet is to remove unsupported sizes, and only allow use of the supported ones.
"Imo. this should be supported, and it would only require removing an if statement in order to implement it. I see no reason to limit the node from allocating huge pages with more than one size." Could you fix this one as well? We use 2M pages only for pods but we need 1G pages for system processes. So it was little surprising when k8s doesn't start because two different pages were configured no matter they are really used or not. Thanks!
I am hitting the same, I used tuned to change the default size from 2Mi to 1Gi, i hit the same problem, after the reboot node is NotReady and it cannot rejoin , giving the error: Node "test-worker-0" is invalid: [status.capacity.hugepages-2Mi: Invalid value: resource.Quantity{i:resource.int64Amount{value:, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"20Mi", Format:"BinarySI"}: may not have pre-allocated hugepages for multiple page sizes, status.allocatable.hugepages-2Mi: Invalid value: resource.Quantity{i:resource.int64Amount{value:, scale:0}, My system just reports information for 1Gi, not 2Mi anymore. Removing the node and rebooting fixed it, the node rejoined again reporting the right capacity. But this is not a solution for us, changing sizes should be supported.
Hitting same K8s installed with my ubuntu 18 enabled 2Mi hugepage by default. As I change my master node's hugepage to 1Gi, this happened. I have only one master, so I have to rebuild the k8s cluster.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with .

Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
I'm a little confused. when i used tuned to change the hugepagesz from "defaulthugepagesz=2M hugepagesz=2M hugepages=8192" to " defaulthugepagesz=1G hugepagesz=1G hugepages=16", i hit the same problem, after the reboot node is NotReadybut when i used tuned to change the hugepagesz from " defaulthugepagesz=1G hugepagesz=1G hugepages=16" to "default_hugepagesz=2M hugepagesz=2M hugepages=8192", after the rebootkubelet status can still keep Ready. why?
Hi This will be fixed in , but it is still not merged. Hoping to get it into next release. In order to solve the issue (and use 1GiB huge pages) you have to delete the node in kubernetes via , and join the node again.
Thanks
Thanks
I think this is already fixed by can you confirm?
Hi WIth this would not cause a crash, but is still required in order to fix this properly.
I am facing the same issue. After changing the hugepage setting to . I stopped the kubelet process on the worker node and removed the node from the cluster using command and rebooted it. But after it comes back online and kubelet starts, it struggles to join the cluster . Do I have to run the join command again ?


Question: to.terminated[c.Name] = channel mainContainerChannels = append(mainContainerChannels, channel) <del> // if its not a running container, pre-close the channel so nothing waits on it </del> <ins> // if it's not a running container, pre-close the channel so nothing // waits on it </ins> if _, isRunning := runningContainers[c.Name]; !isRunning { close(channel) }

In a situation where during the init phase of a , a native sidecar never becomes ready (because its is failing), then the is terminated, if there are multiple startup containers, a termination signal is never received by the init container and the Pod is stuck (until is hit). All running initContainer to receive the termination signal correctly. Create a with this bash script as an entrypoint: Create a deployment with two init containers (one normal one sidecar). Add an probe which means the startupProbe never passes Create the deployment, and it'll be stuck in Init, as you'd expect: the pod and you'll see the exit signal has been sent to : Delete the deployment / clean everything up. Now, add a third : As before, the process will be stuck on because of the bad probe: However when you delete the pod this time, no exit signal is sent to , the pod is now stuck until is reached. No response GKE v1.29.4-gke. Core OS on GCP <details</details<details</details<details</details
/sig node
/assign
/triage accepted confirmed that the scenario is reproduced. The problem is caused by the fact that the native sidecars should terminate in reverse order. I'll fix the code to ignore not-running init containers. /cc
Awesome, thank you :)
sig-node meeting notes: /triage accepted /priority important-soon
FYI: confirmed that the SITERM is sent to the containers after terminationGracePeriodSeconds (and wait for the application for 2 seconds before sending SIGKILL) will fix it anyway with


Question: if err != nil { return err } <ins> // Before we overwrite the config which holds the parsed // command line parameters, we need to copy all modified // logging settings over to the loaded config (i.e.  logging // command line flags have priority). Otherwise `--config // ... -v=5` doesn't work (config resets verbosity even // when it contains no logging settings). copyLogsFromFlags(fs, &c.Logging) </ins> o.config = c if err := o.initWatcher(); err != nil {

introduced configuration of additional logging aspects (e.g. format) through command line flags and config file. This was merged for Kubernetes 1.28. Now the following invocation doesn't work as expected when the config file doesn't configure logging: It should run with verbosity 4. Instead it runs with verbosity 0, from the config. Run with this config file: No response 28 rc n/a No response No response No response No response
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
The problem is caused by this line: At that point, is 4 (from the command line flag), and then it gets overwritten with == 0. The normal approach would be: set up config with defaults modify config through flags modify config further by parsing a config file into it. Can kube-proxy do this? It has the downside that a config file can only overwrite fields if they are in the file. A config field doesn't get reset to defaults just by parsing the file. /cc
it looks like kube-proxy special-cases the to let a flag override config... can/should it do the same thing for verbosity? are there any other flags like this?
I don't think that's the normal approach... it's more normal to make flags and config file mutually exclusive. That's difficult for things which started as flags and moved into config
/sig network /sig instrumentation /kind bug /kind regression /milestone v1.28
let's go with and warn it and comment as much as possible, I was going with something like but Patrick is working on it so I assign to him the issue
/assign
Hey folks, v1.28 Bug Triage Lead here Just wanted to know if this is already tracked by SIG leads and someone already working actively fixing it and set a priority?
Perhaps I should have said "this is how component-base/logs was meant to be used". Somehow it worked for kubelet.

Here's my preliminary solution: It needs more testing, but I wanted to share it early because I am running out of time today.
it is , I can take over Patrick's work in case is needed
It would be nice if the logs API could distinguish between and , how about make to a pointer? But this change is supposed to be a breaking change, I'm not sure of the scope of the impact. For side, if set and at the same time, should the command flag override the config file or warn it and be discarded? I think it should be overridden.
xref:


Question: "//vendor:k8s.io/apimachinery/pkg/util/validation/field", "//vendor:k8s.io/apimachinery/pkg/util/wait", "//vendor:k8s.io/apimachinery/pkg/util/yaml", <ins> "//vendor:k8s.io/apimachinery/pkg/version", </ins> "//vendor:k8s.io/apimachinery/pkg/watch", "//vendor:k8s.io/apiserver/pkg/util/flag", "//vendor:k8s.io/client-go/discovery",

<!-- Thanks for filing an issue! Before hitting the button, please answer these questions.--Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see ): no What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.): "kubectl version", "kubectl version" json Is this a BUG REPORT or FEATURE REQUEST? (choose one): Feature request Kubernetes version (use ): $ kubectl version --client Client Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.6", GitCommit:"", GitTreeState:"clean", BuildDate:"2016-11-12T05:22:15Z", GoVersion:"go1.7.1", Compiler:"gc", Platform:"darwin/amd64"} Environment: Cloud provider or hardware configuration: GKE OS (e.g. from /etc/os-release): MacOS 10.12.2 I find it difficult to parse the output of programmatically. I have to use a YAML parser to do it. I'd prefer to use JSON, or to be given just the string and asked to parse it with a regex. This is obviously a relatively low-priority request.
I wouldnt mind helping out on this, if there are no objections?
are you working on this? If not, I would like to work on it.
I plan to, but if i'll keep you posted if i get too delayed.
How can I direct message you?
didnt see this till now, felt bad i had not kept up on this so i decided to just push through it. If my email is not available in my profile, please let me know, otherwise AT gmail would be the best way to contact me.
Thanks!
Included below are some examples. Let me know what you think, my only concern is the the original output of included the key and then the object type which is a little strange. Note the output flag is optional so we don't break previous behavior and contract, which is now baked and part of e2e testing. here are the couple of samples: