Question: // paths in the case that a user specifies the same volume/volume mount name. func (c *controlPlaneHostPathMounts) AddExtraHostPathMounts(component string, extraVols []kubeadmapi.HostPathMount) { for _, extraVol := range extraVols { <del> fmt.Printf("[control-plane] Adding extra host path mount %q to %qn", extraVol.Name, component) </del> hostPathType := extraVol.PathType c.NewHostPathMount(component, extraVol.Name, extraVol.HostPath, extraVol.MountPath, extraVol.ReadOnly, &hostPathType) }

<!-- Please only use this template for submitting enhancement requests --What would you like to be : when using kubeadm init include Why is this needed: the log only show extra-volumes for that control-plane? like this
The label(s) cannot be applied, because the repository doesn't have them <detailsIn response to : /sig kubeadm Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
The label(s) cannot be applied, because the repository doesn't have them <detailsIn response to : /sig kubeadm Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/area kubeadm /kind cleanup /sig cluster-lifecycle
what version of kubeadm are you using?
this minor issue is in master too. the fix is in this PR:


Question: if len(meta.UID) == 0 { meta.UID = old.UID } <del> if meta.CreationTimestamp.IsZero() { </del> <ins> // ignore changes to timestamp if old.CreationTimestamp.IsZero() { old.CreationTimestamp = meta.CreationTimestamp } else { </ins> meta.CreationTimestamp = old.CreationTimestamp }

After the DST change: We should be ignoring creationTimestamp on update.
Could you please explain the cause of the failure in more detail? Do we not normalize the timezone in the representation used in the API?
When we serialize to JSON we are using a value that has a timezone embedded in it (time.Time). As a result the representation changed. We should normalize it as well during serialization, but in general creationTimestamp should not prevent an update since it can safely be ignored.
I think we should allow creationTimestamp to be empty and also convert time to UTC() before marshaling.
More specifically: We'd also want to change: to and to
Makes sense to me - clients should always control their timezone and with distributed masters you might see different values. ----- Original Message -----
I'll own the follow up on this. ----- Original Message -----
+1 for UTC serialization
Opened to marshal to utc when serializing


Question: return []string{devicePath}, nil } <ins> func fakeFilepathGlob2(devicePath string) (globs []string, err error) { return []string{ "/dev/disk/by-path/pci-0000:00:00.0-ip-127.0.0.1:3260-iqn.2014-12.com.example:test.tgt00-lun-0", }, nil } </ins> func TestExtractTransportname(t *testing.T) { fakeIscsiadmOutput := []string{ "# BEGIN RECORD 2.0-873n" +

Kubernetes version (use ): Client Version: version.Info{Major:"1", Minor:"7+", GitVersion:"v1.7.0-beta.1", GitCommit:"", GitTreeState:"not a git tree", BuildDate:"2017-06-08T13:08:32Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.3", GitCommit:"", GitTreeState:"clean", BuildDate:"2017-05-10T15:38:08Z", GoVersion:"go1.7.5", Compiler:"gc", Platform:"linux/amd64"} Environment: Cloud provider or hardware configuration: Bare-Metal, HP SE316M, 2x X5675, 144GB, iSCSI/RDMA/iSer OS: CentOS 7 Kernel: Linux at-host-18 4.9.31 SMP Wed Jun 7 22:40:40 UTC 2017 x8664 x8664 x8664 GNU/Linux Install tools: Others: What happened: I want to mount a iSCSI volume via iSER (RDMA over Infiniband) in this case the iSCSI iface is "iser" and not tcp, therefore the glob path for device mount is selected (), which then is passed to the fsck and mount commands without globbing the path (e.g. calling filepath.Glob) Therefore the iSCSI volume is not mounted and the pod cannot be started. Logs: What you expected to happen: To successfully mount the volume. How to reproduce it (as minimally and precisely as possible): Setup an iSCSI via RDMA (iSER) environment without multipath and try to mount a volume for a pod. Anything else we need to know: I believe it's just needed to add the globbing at line 259 like so: devicePaths = append(devicePaths, filepath.Glob(devicePath)) As I'm not a go programmer and have no experience with it, probably somebody could create a commit which I would gladly test!
There are no sig labels on this issue. Please by:<br(1) mentioning a sig: <br(2) specifying the label manually: <br<brNote: method (1) will trigger a notification to the team.

You can find the team list and label list
/sig storage
I'll take a look.
/sig storage
on a local build that fixed the issue: (e.g. it compiles)
I don't want to add filepath.Glob to that general path because that line is also executed "tcp" case. I'm considering the fix below. Pretty similar.
does that work from a go specific point? is that devicePath passed by reference? (from general c/c++/php/node/c# knowledge I would say that this would have no effect on the devicePath variable .. )
Can we suppose that filepath.Glob(devicePath) always only return one device path if transport is "tcp"?
does that work from a go specific point? is that devicePath passed by reference? (from general c/c++/php/node/c# knowledge I would say that this would have no effect on the devicePath variable .. ) Thank you for above comment. I updated the logic. Could you confirm the fix on the PR?
Pretty sure that tcp transport will always only have one path - and it will exist (there's no glob in the check path)


Question: -o "${tmp_kube_env}"  http://metadata.google.internal/computeMetadata/v1/instance/attributes/kube-env # Convert the yaml format file into a shell-style file. <del> eval "$(python3 -c ''' </del> <ins> eval "$(${PYTHON} -c ''' </ins> import pipes,sys,yaml <del> items = yaml.load(sys.stdin, Loader=yaml.BaseLoader).items() </del> <ins> # check version of python and call methods appropriate for that version if sys.version_info[0] < 3: items = yaml.load(sys.stdin).iteritems() else: items = yaml.load(sys.stdin, Loader=yaml.BaseLoader).items() </ins> for k, v in items: print("readonly {var}={value}".format(var=k, value=pipes.quote(str(v)))) ''' < "${tmp_kube_env}" > "${KUBE_HOME}/kube-env")"

<!-- Please only use this template for submitting reports about continuously failing tests or jobs in Kubernetes CI --ci-kubernetes-e2e-gce-device-plugin-gpu Cluster up Apr 23 8:30PM PDT Taken from: From a , there are supposed to be three worker/minion nodes set up as well. Started failing after -- so maybe related to ? /cc /sig testing More examples of failing runs: /priority critical-urgent (this is a job on the release/master-blocking board)
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
It uses image gke-1134-gke-rc5-cos-69--138-0-v190320-pre-nvda-gpu as node image kubekins-e2e image is built from Base image is bootstrap image is built from with debian:buster and install python3.
The node image gke-1134-gke-rc5-cos-69--138-0-v190320-pre-nvda-gpu has no python3. We need to upgrade it.
Thanks for catching this and sending revert. Separately, we should definitely upgrade that image, cos-69 is super old and not supported anymore: /cc regarding GPU image


Question: } } <ins> func normalizeName(name string) string { // If it's short enough, just leave it. if len(name) < LOAD_BALANCER_NAME_MAX_LENGTH-6 { return name } // Hash and truncate hash := md5.Sum([]byte(name)) truncated := name[0 : LOAD_BALANCER_NAME_MAX_LENGTH-6] shortHash := hash[0:6] return fmt.Sprintf("%s%s", truncated, string(shortHash)) } </ins> // CreateTCPLoadBalancer is an implementation of TCPLoadBalancer.CreateTCPLoadBalancer. // TODO(a-robinson): Don't just ignore specified IP addresses. Check if they're // owned by the project and available to be used, and use them if they are. <del> func (gce *GCECloud) CreateTCPLoadBalancer(name, region string, externalIP net.IP, ports []int, hosts []string, affinityType api.AffinityType) (string, error) { </del> <ins> func (gce *GCECloud) CreateTCPLoadBalancer(origName, region string, externalIP net.IP, ports []int, hosts []string, affinityType api.AffinityType) (string, error) { name := normalizeName(origName) </ins> err := gce.makeTargetPool(name, region, hosts, translateAffinityType(affinityType)) if err != nil { if !isHTTPErrorCode(err, http.StatusConflict) {

GCE has a length limit of 63 characters for its load balancer components (target pools and forwarding rules), and AWS has a length limit of 32 characters. Our current load balancer name construction of clustername-namespace-servicename can exceed both limits, especially AWS's. We should use a shorter name construction method less likely to hit these limits.
Thanks for filing this Alex. My suggestion would be to use a simple UID, as per utils.NewUID(). We'll need to store the mapping from {clustername, namespace, servicename} -UID somewhere stable like etcd. Using anything based directly on {cluster, namespace, name}, for example a stable hash of each, has the problem that the aforementioned tuple is not unique over time. This raises the possibility of e.g. the following scenario: creates mycluster/mynamespace/myservicename, which creates external load balancer named stablehash(mycluster,mynamespace,myservicename) deletes mycluster/mynamespace/myservicename, but at the time MyCloudProvider is down, so kubernetes can't delete ELB named stablehash(mycluster,mynamespace,myservicename). Kubernetes keeps trying in the background to delete the ELB. the mean time user creates a new service named the same as her old one mycluster/mynamespace/myservicename. Kubernetes tries to create an associated ELB named stablehash(mycluster,mynamespace,myservicename), but this fails, because the ELB exists already due to 2 above. ... etc. Badness ensues.
Assigning to Chao Xu, as this makes for a good starter bug.
Hmmmm.... can't seem to assign to caesarxuchao yet. Not sure why.
He has to have write privileges on the repo before issues/PRs can be assigned to him.
But we have many contributors with issues assigned to them that are not in the kubernetes-write group (that only has 48 members). I must be missing something. How do I grant him write privileges on the repo?
We do? That's not how I understand it to work: If you want to add him, go .
has been because the load balancer name changes if the service's UID changes (e.g. if it's deleted and recreated).

Think we can change this? A hash of cluster name, namespace, and service name should get around it, but there may be better ideas as well.
Sure. I'm not confident that I can fix this quickly. If it's urgent, maybe you can assign some one more experienced to fix it? One question: I guess the ELB's name should stay unchanged as long as the "zone+namespace+service name" doesn't change, am I right?
I think I'm the guilty party here. I was unaware that when we create a new service with the same name as a previously existing service, that it needed to keep the same load balancer name and IP. This seems broken to me, but I may be missing the larger context. We can quite safely roll back the change while we decide on the best alternative solution. Q , Chao Xu wrote:
I think the real breakage is that we don't have any way to claim an LB-IP at a larger lifespan than a single service. Alex, can you go back to OP and get a sense of what's happening there to trigger this? , Quinton Hoole wrote:
Will do. Even if the name is kept consistent as it was before, with the way everything currently works the IP won't be consistent.
yeah, I'm not positive that the name has to be consistent. What we need is a way to claim the same public IP address, and we already have that (at least for GCE) Assigning to me, and I'll validate the length requirements, make sure public ip assignment works for AWS and close out this issue.
Note there are two separate things here: (1) a purely-name-based construction can exceed length limits given a single service and (2) some form of across-service consistency is apparently needed. (1) is trivially solvable by either always using a hash or alternatively using a two-stage process where you generate a desired name and only hashify if it's longer than the length limits by truncating it and appending a hash of said name.
AWS's load balancer hasn't been merged yet, so there's not much to check out there. I'm going to be reducing the churn of load balancers later today, which should help prevent IP address changes for services that are modified.

I think we need to be able to figure out which Service an LB is attached to from the Service, so that LBs can't get orphaned. regarding being able to claim an LB IP. I don't think that is covered in revamp... , Brendan Burns wrote:
I think that we need to roll back and think this issue through a bit better. See comment on that PR.


Question: ``` --api-version="": api-version for the cluster entry in kubeconfig --certificate-authority="": path to certificate-authority for the cluster entry in kubeconfig <del> --embed-certs=false: embed-certs for the cluster entry in kubeconfig --insecure-skip-tls-verify=false: insecure-skip-tls-verify for the cluster entry in kubeconfig </del> <ins> --embed-certs[=false]: embed-certs for the cluster entry in kubeconfig --insecure-skip-tls-verify[=false]: insecure-skip-tls-verify for the cluster entry in kubeconfig </ins> --server="": server for the cluster entry in kubeconfig ```

It acts like a boolean flag but requires an argument, unlike boolean flag is pretty much every environment. We need to find a better way to express the things that this is expressing. Maybe a add a tristate to the pflag lib that behaves the same way but doesn't demand an argument?
cc
The handles "true", "false", "unset" and has a default value. I seem to recall you were looking at changing it, but we needed to preserve the value. The complain here is that you have to do instead of just . /cc
fixing is really easy....


Question: "ports":                    "The list of ports that are exposed by this service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies", "selector":                 "Route service traffic to pods with label keys and values matching this selector. If empty or not present, the service is assumed to have an external process managing its endpoints, which Kubernetes will not modify. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/", "clusterIP":                "clusterIP is the IP address of the service and is usually assigned randomly by the master. If an address is specified manually and is not in use by others, it will be allocated to the service; otherwise, creation of the service will fail. This field can not be changed through updates. Valid values are "None", empty string (""), or a valid IP address. "None" can be specified for headless services when proxying is not required. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies", <del> "type":                     "type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. "ExternalName" maps to the specified externalName. "ClusterIP" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is "None", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. "NodePort" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. "LoadBalancer" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services ", </del> <ins> "type":                     "type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. "ExternalName" maps to the specified externalName. "ClusterIP" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is "None", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. "NodePort" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. "LoadBalancer" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types", </ins> "externalIPs":              "externalIPs is a list of IP addresses for which nodes in the cluster will also accept traffic for this service.  These IPs are not managed by Kubernetes.  The user is responsible for ensuring that traffic arrives at a node with this IP.  A common example is external load-balancers that are not part of the Kubernetes system.", "sessionAffinity":          "Supports "ClientIP" and "None". Used to maintain session affinity. Enable client IP based session affinity. Must be ClientIP or None. Defaults to None. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies", "loadBalancerIP":           "Only applies to Service Type: LoadBalancer LoadBalancer will get created with the IP specified in this field. This feature depends on whether the underlying cloud-provider supports specifying the loadBalancerIP when a load balancer is created. This field will be ignored if the cloud-provider does not support the feature.",

<!-- Please only use this template for submitting reports about failing tests in Kubernetes CI jobs --Which jobs are failing: Which test(s) are failing: Since when has it been failing: 2019-02-19, , Anything else we need to know: Related: I wonder if there's an opportunity for automation/testing before end to end tests there /kind failing-test /priority important-soon /sig api-machinery cc
/assign
Thanks!
/reopen We're running into this again, slightly different error message
Reopened this issue. <detailsIn response to : /reopen We're running into this again, slightly different error message Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Last good: First failure: Leads me to
Paging (EDIT: ) any idea how this could be related to ? That's the only PR showing up in that commit range
/assign
This looks like another occurrence of the same failure, and is making me question whether we're actually kicking a can down the road vs. fixing whatever is causing these to happen
/remove-priority important-soon /priority critical-urgent this is on release-blocking
/close OK, it was poor form for me to re-open and re-purpose this issue, I opened ok I opened instead
Closing this issue. <detailsIn response to : /close OK, it was poor form for me to re-open and re-purpose this issue, I opened ok I opened instead Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: }, } f.SetFakeContainers(testContainers) <del> go c.processQueueItems(ctx, 1) </del> <ins> // Push the items into the queue for before starting the worker to avoid issue with the queue being empty. </ins> require.NoError(t, c.rotateLogs(ctx)) <del> pollTimeoutCtx, cancel := context.WithTimeout(ctx, 10*time.Second) defer cancel() err = wait.PollUntilContextCancel(pollTimeoutCtx, 20*time.Millisecond, false, func(ctx context.Context) (done bool, err error) { return c.queue.Len() == 0, nil }) require.NoError(t, err) </del> <ins> // Start a routine that can monitor the queue and shutdown the queue to trigger the retrun from the processQueueItems // Keeping the monitor duration smaller in order to keep the unwanted delay in the test to a minimal. go func() { pollTimeoutCtx, cancel := context.WithTimeout(ctx, 10*time.Second) defer cancel() err = wait.PollUntilContextCancel(pollTimeoutCtx, 5*time.Millisecond, false, func(ctx context.Context) (done bool, err error) { return c.queue.Len() == 0, nil }) require.NoError(t, err) c.queue.ShutDown() }() // This is a blocking call. But the above routine takes care of ensuring that this is terminated once the queue is shutdown c.processQueueItems(ctx, 1) </ins> timestamp := now.Format(timestampFormat) logs, err := os.ReadDir(dir)

TestRotateLogs from kubelet/logs/containerlogmanagertest Probably since the merge of change No response_ /sig node
/triage accepted /cc
/cc
/priority important-longterm Seems to impact on this arch only?
cc
/sig windows
ack.. Checking
what is the easiest way for me to test this one locally ? using based vm ?
poke and on slack please
unfortunately, I couldn't reproduce this flake locally. Will update here once I am able to.
I am now able to see the flake, when I run the test within a Pod with a resource limit as our CI job does. Steps to reproduce: clone kubernetes test -c -race - TestRotateLogs root stress - TestRotateLogs Please find the attached trace file
This is with or on a native x86_64 as well ?
I could reproduce this even on x8664. Also, the failures do happen even when run on the VM itself and NOT ONLY within the pod as I mentioned above. The flakiness % is higher when run from a pod rather than a VM.
I was able to reproduce this on regular machines as well with while running tests. Let me check and see what needs to be done to fix this.
This seem to take care of the issue. Do you think the poll interval of is too much ? cc
bash stress - TestRotateLogs -test.v 5s: 0 runs so far, 0 failures 10s: 48 runs so far, 0 failures 15s: 96 runs so far, 0 failures 20s: 146 runs so far, 0 failures 25s: 196 runs so far, 0 failures 30s: 244 runs so far, 0 failures 35s: 292 runs so far, 0 failures 40s: 339 runs so far, 0 failures 45s: 387 runs so far, 0 failures
Spoke too early.

bash 1m30s: 812 runs so far, 4 failures (0.49%) /var/folders/l3/tf0xj3gj67bfmk13h462668w0000gn/T/go-stress-20240223T144024- === RUN TestRotateLogs Error Trace: Error: "[- test-log-1 - test-log-2.- - test-log-3 - test-log-3.- - test-log-3.- - test-log-4]" should have 5 item(s), but has 6 Test: TestRotateLogs Error Trace: Error: Not equal: expected: "test-log-3.-" actual : "test-log-3" Diff: --- Expected +++ Actual -1 +1 -test-log-3.- +test-log-3 Test: TestRotateLogs Error Trace: Error: Not equal: expected: "test-log-3.-" actual : "test-log-3.-" Diff: --- Expected +++ Actual -1 +1 -test-log-3.- +test-log-3.- Test: TestRotateLogs Error Trace: Error: Not equal: expected: "test-log-4" actual : "test-log-3.-" Diff: --- Expected … 1m35s: 851 runs so far, 5 failures (0.59%)
/assign
/triage accepted
Had to add a few additional logs to identify the issue, but has a fix now. PTAL when possible.


Question: return err.HTTPStatusCode == http.StatusTooManyRequests || err.RetryAfter.After(now()) } <ins> // IsNotFound returns true the if the requested object wasn't found func (err *Error) IsNotFound() bool { if err == nil { return false } return err.HTTPStatusCode == http.StatusNotFound } </ins> // NewError creates a new Error. func NewError(retriable bool, err error) *Error { return &Error{

What happened: We saw several occurrences of kube-controller-manager failing to check a k8s node existence after we deleted that node's parent VMSS (and by extension, cloud instance). The leaked k8s nodes remained present in the cluster until manually deleted. Cloud provider retries caused a VMSS VMs API call surge, degenerating in hard throttling for the whole subscription. In those situations, kube-controller-manager would continuously log errors like the following: Looking at Azure cloud-provider implementation shows that all the VMSS API calls errors (including a 404/not found on the VMSS object) are bubbled up as generic errors, and never used as an indication of a missing instance. What you expected to happen: k8s nodes being removed as their VMSS went missing. How to reproduce it (as minimally and precisely as possible): Create and delete many VMSS providing instances registering to a cluster. Environment: Kubernetes version (use ): 1.19.0 Cloud provider or hardware configuration: Azure (self managed cluster, non AKS, ) OS (e.g: ): Linux, Ubuntu Focal 20.04.1 LTS Kernel (e.g. ): 5.4.0-1022-azure /sig cloud-provider /area provider/azure
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: ADD bin/pause-windows-${ARCH}.exe /pause.exe ADD bin/wincat-windows-amd64 /Windows/System32/wincat.exe <del> # NOTE(claudiub): We're replacing the diagtrack.dll as a means to disable the # DiagTrack service (it cannot run without this DLL). We do not need this # service in the pause image and there's no reason for it to have any CPU usage. ADD windows/pause.c /Windows/System32/diagtrack.dll </del> # NOTE(claudiub): docker buildx sets the PATH env variable to a Linux-like PATH, # which is not desirable. See: https://github.com/moby/buildkit/issues/1560 # TODO(claudiub): remove this once the issue has been resolved.

The Windows pause images is built using buildkit on Linux hosts and this configuration does not allow for the Windows registry to me modified during the container build process. When Windows containers are built on Windows host machines part of the build process creates 'delta' registry files when get stored in the layer files. These 'delta' registry hive files are applied to the containers active registry when the containers are created. Chaning the registry files in the container image layers without having the 'delta' files present will result in the changes NOT being present in the container image. We have an increasing demand for the ability to modify registry in the pause image on Windows such as Disable some services like DiagTrack that negatively impact performance Change behavior of RPCSS so it doesn't hijack RPC port (135) for all containers in the pod Things are working as expected given the current build process. There are linux utilities like that can modify Windows registry files that I have used to test this out. After running the container produced from these files you can see the registry changes made by persisted in the container image but these changes are not present in the container's active registry. vs One possible solution is to create an intermediate image based on (the current base image for the Windows pause image) that performs all the desired registry operations that is built on a Windows machine and published someplace (MCR maybe?) and the use that as the container base image for the Windows pause image. <detailsN/A </details<detailsN/A </details<detailsN/A </details<details</details<details</details<details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/cc
cc
my first concern was. will a community member be able to build this image themselves (will the sources be available and up-to-date and the image built from it). We could also do a github hosted windows runner to build / publish this new image as well. let's please revisit this once we wrap 1.24.

/milestone v1.25
We created which has the source/docker files used to build . AKS has made a pause image based on image available to some customers out of necessity to unblock some scenarios. We'd like to upstream the changes - and do so with as little destabilization to Kubernetes project as possible.
Seems like the intermediate base image step would be reasonable. I don't think community members can build the existing base windows images either for the same reason of depending on windows?
The windows pause image is currently based on 'nanoserver' which is one of several container base images provided by Microsoft (and only Microsoft can produce container base images).
yep. i'd expect community downstream teams (for example) would like to start with this nanoserver, have a set of dockerfiles to build this new image ... then build their own pause images based on this new image
or in other words if MSFT wants to roll all the changes into nanoserver itself (since we all already know that only MSFT can produce this), then there is no debate after that
We can't roll these changes into nanoserver because they may break other scenarios. Because of that - having an intermediate layer built somewhere was the best solution we came up with without forking the build process for the pause image. Microsoft is willing to host build/host the intermediate layers/images from open-sourced docker files. If we want to build the intermediate layers somewhere else too (like github hosted Windows runners) that would be fine too.
/sig windows
If this intermediate image used only by k8s then it is a good idea to build the image via GH actions in the kubernetes org so that k8s maintainers have better control on this or if this image is going to be generic for anyone to use then MSFT can build one and host it somewhere.
Picking this back up since 1.24 release is behind everyone. - Do you think a separate github repo for the dockerfile and github actions staging/prod container repos would be appropriate here? If so I can get started on creating these. Also - Is it possible to push to the staging repos from a github action job instead of PROW jobs? We'd need to use github actions to build the Windows image on a Windows machine.

I supposed we can store the .tar files as release artifacts if needed and then use a PROW to job push them to the staging repo if needed but being able to push for the action would be simpler.
/cc as FYI
I see two possible paths forward Continue to host the source for the intermediate layer at , publish the image to , and then update the Windows pause build docker files in k/k to consume this image instead of the or Create a new kubernetes-sigs repo to host the source and build steps for the image, publish the image to , and then update the Windows pause build docker files in k/k to consume this image. I'm fine with either approach but am not sure who the right folks to discuss this with are.
This looks more promising because this image you are creating is not a specific change for the pause container, IMO if we can change that image name to something generic and use the same wherever required.
I think those both sound reasonable personally. I think it's technically feasible to push from GitHub actions to staging GCR with workload identity but nobody has done this yet in Kubernetes.


Question: } // allocateHealthCheckNodePort allocates health check node port to service. <del> func (rs *REST) allocateHealthCheckNodePort(service *api.Service) error { </del> <ins> func (rs *REST) allocateHealthCheckNodePort(service *api.Service, nodePortOp *portallocator.PortAllocationOperation) error { </ins> healthCheckNodePort := service.Spec.HealthCheckNodePort if healthCheckNodePort != 0 { // If the request has a health check nodePort in mind, attempt to reserve it. <del> err := rs.serviceNodePorts.Allocate(int(healthCheckNodePort)) </del> <ins> err := nodePortOp.Allocate(int(healthCheckNodePort)) </ins> if err != nil { return fmt.Errorf("failed to allocate requested HealthCheck NodePort %v: %v", healthCheckNodePort, err) } <del> glog.Infof("Reserved user requested nodePort: %d", healthCheckNodePort) </del> <ins> glog.V(4).Infof("Reserved user requested healthCheckNodePort: %d", healthCheckNodePort) </ins> } else { // If the request has no health check nodePort specified, allocate any. <del> healthCheckNodePort, err := rs.serviceNodePorts.AllocateNext() </del> <ins> healthCheckNodePort, err := nodePortOp.AllocateNext() </ins> if err != nil { return fmt.Errorf("failed to allocate a HealthCheck NodePort %v: %v", healthCheckNodePort, err) } service.Spec.HealthCheckNodePort = int32(healthCheckNodePort) <del> glog.Infof("Reserved allocated nodePort: %d", healthCheckNodePort) </del> <ins> glog.V(4).Infof("Reserved allocated healthCheckNodePort: %d", healthCheckNodePort) </ins> } return nil }

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: healthcheck nodeport allocation doesn't have roll-back, this may lead to nodeport leak. What you expected to happen: How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: /sig network /assign Environment: Kubernetes version (use ): Cloud provider or hardware configuration**: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others:
/sig network /assign
Could you clarity "roll-back"? It seems that we basically in the same way.
I didn't mean the HealthCheck NodePort release process in . I meant the allocation process in and . Take for example, please see: There is a defer for common NodePorts in : Once fails, the above defer will finally exec. Then the allocated NodePorts will roll-back, i.e. these NodePorts will be released. But the HealthCheck NodePort allocation process doesn't use this roll-back. This will lead to leak. I run a simple test and see the HealthCheck NodePort leak. Will paste the result later.
Testing service manifest, . some simple testing code. See: . Note that I add this error intentionally: kube-apiserver with flag . . We will get below output. This is expected. again. We will get below output. Note the error . That means that when creating service fails at step 3, the NodePort and HealthCheck NodePort should be both released, but the was NOT.
Fortunately, several minute later, the leak was repaired and cleaned up. But anyway, we should release the HealthCheck NodePort when fails. We shouldn't depend on the later repair.
Thanks for tracing down, sent for fixing.
We're having this issue on service deletion. It looks like healthcheck nodeport is not deallocated immediately when deleting the service. This is causing issues for backup and restore of services with heathcheck node port.


Question: "k8s.io/apimachinery/pkg/util/uuid" netutils "k8s.io/utils/net" <ins> utilfeature "k8s.io/apiserver/pkg/util/feature" </ins> kubefeatures "k8s.io/kubernetes/pkg/features" kubeletconfig "k8s.io/kubernetes/pkg/kubelet/apis/config" "k8s.io/kubernetes/test/e2e/framework"

4 other Test Failures in ci-crio-cgroupv1-node-e2e-features. Test has failed 61 times (no passing result found). Earliest failure found at commit/ (Thu Oct 19 21:41 PDT). /cc { failed [FAILED] Error creating Pod: pods "pod-dualstack-host-ips9ddb9810-134c-4d35-a0c6-" is forbidden: violates PodSecurity "restricted:latest": host namespaces (hostNetwork=true) In [It] at: @ 10/23/23 01:38:06.024 } No response_ /sig network node
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign
is also failing on these tests
the test has a bug, it checks the contrary
, let's start over, 1 commit for the e2e test, other for the promotion, and let's validate everything is alright before merging


Question: verbs: - create - get <ins> - list - watch </ins> - apiGroups: - "" - events.k8s.io

When OwnerReferencesPermissionEnforcement admission plugin is used the PVCs are not deleted and KCM logs show PVCs should have ownerReferences set and the PVCs should be garbage collected on scale down 3 of 6 of the following tests fail when OwnerReferencesPermissionEnforcement admission plugin is turned on No response <details</details<detailsNA </details<details</details<details</details<details</details<details</details
/sig apps /triage accepted


Question: match: k8s.gcr.io/pause:d+.d+ - path: test/utils/image/manifest.go match: configs[Pause] = Config{gcRegistry, "pause", "d+.d+"} <ins> # metadata-concealment: bump this one first - name: "metadata-concealment" version: "1.6" refPaths: - path: test/images/metadata-concealment/VERSION match: d.d # then after merge and successful postsubmit image push / promotion, bump this - name: "metadata-concealment: dependents" version: "1.6" refPaths: - path: test/utils/image/manifest.go match: configs[CheckMetadataConcealment] = Config{promoterE2eRegistry, "metadata-concealment", "d+.d+"} </ins>

Warning: The v0.1 and v1beta1 metadata server endpoints are deprecated and scheduled for shutdown on September 30, 2020. This test (which runs in most e2e jobs that run on GCE): Schedules a container that runs this: Resulting in a number of legacy metadata endpoints getting hit. To see if this test is going to start failing across all release branches on Sep 30th, we could try disabling the endpoints ahead of time via: In fact and support for expecting different results based on whether is set. My concern is the test expects 200 for some root endpoints, and 403 for disabled endpoints. However the e-mail I received as a GCP customer tells me any legacy endpoints may return 404. I'm inclined to suggest we remove the legacy endpoints from the check, bump the image version, and cherry-pick use of the new image version back to all release-branches. WDYT since you were involved with the earlier PRs /sig auth /sig cloud-provider /provider gcp /sig testing /priority important-soon (I'm tempted to critical-urgent this but it's not drop-everything yet) /milestone v1.20
/kind cleanup
Heyho I wanted to check how it is going? The code freeze is starting 12.11.2020 (about 3-4 weeks from now) and while there is still plenty of time, we want to ensure that each relevant issue has a chance to be considered. As the issue is tagged for 1.20, is it still planned for this release?
Bug Triage here, we're pretty late into code freeze and it doesn't look like there has been much activity recently. I am going to clear the milestone on this issue. /milestone clear
any update here?
Try to cleanup the legacy endpoints in
/milestone v1.21
/reopen This needs to stay open until the new image has been verified to work and has been promoted (ref: )
Reopened this issue. <detailsIn response to : /reopen This needs to stay open until the new image has been verified to work and has been promoted (ref: ) Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/triage accepted /assign


Question: "net/url" "os" "path" <ins> sysruntime "runtime" </ins> "sort" "strings" "sync"

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: If I have the a psp with apparmor annotations: and I deploy a windows pod it gets blocked: What you expected to happen: Apparmor check is ignored for windows pods How to reproduce it (as minimally and precisely as possible): see above Anything else we need to know?: Should be fixed/documented with work around as per Environment: Kubernetes version (use ): kubectl version Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.5", GitCommit:"", GitTreeState:"clean", BuildDate:"2020-06-26T03:47:41Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.5", GitCommit:"", GitTreeState:"clean", BuildDate:"2020-06-26T19:51:26Z", GoVersion:"go1.13.6", Compiler:"gc", Platform:"linux/amd64"} Cloud provider or hardware configuration: OS (e.g: ): windows/unbuntu Kernel (e.g. ): Install tools: aks-engine Network plugin and version (if this is a network-related bug): Others:
/sig windows
/assign I can fix it, it may take some time
I don't actually see anything in that suggests we should ignore windows nodes for apparmor. Can you link to it?
That issue doesn't have guidance explicitly, was more about the need for someone to look into it. I did some investigation with outcomes on the comment and the spread sheet has each PSP detailed with what applies to windows as of now:


Question: <ins> /* Copyright 2024 The Kubernetes Authors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package node import ( "context" "fmt" "time" v1 "k8s.io/api/core/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/labels" "k8s.io/apimachinery/pkg/types" utilrand "k8s.io/apimachinery/pkg/util/rand" "k8s.io/client-go/util/retry" "k8s.io/kubernetes/test/e2e/framework" admissionapi "k8s.io/pod-security-admission/api" "github.com/onsi/ginkgo/v2" "github.com/onsi/gomega" ) var _ = SIGDescribe("Node Lifecycle", func() { f := framework.NewDefaultFramework("fake-node") f.NamespacePodSecurityLevel = admissionapi.LevelPrivileged ginkgo.It("should run through the lifecycle of a node", func(ctx context.Context) { // the scope of this test only covers the api-server nodeClient := f.ClientSet.CoreV1().Nodes() fakeNode := v1.Node{ ObjectMeta: metav1.ObjectMeta{ Name: "e2e-fake-node-" + utilrand.String(5), }, Status: v1.NodeStatus{ Phase: v1.NodeRunning, Conditions: []v1.NodeCondition{ { Status:  v1.ConditionTrue, Message: "Set from e2e test", Reason:  "E2E", Type:    v1.NodeReady, }, }, }, } ginkgo.By(fmt.Sprintf("Create %q", fakeNode.Name)) createdNode, err := nodeClient.Create(ctx, &fakeNode, metav1.CreateOptions{}) framework.ExpectNoError(err, "failed to create node %q", fakeNode.Name) gomega.Expect(createdNode.Name).To(gomega.Equal(fakeNode.Name), "Checking that the node has been created") ginkgo.By(fmt.Sprintf("Getting %q", fakeNode.Name)) retrievedNode, err := nodeClient.Get(ctx, fakeNode.Name, metav1.GetOptions{}) framework.ExpectNoError(err, "Failed to retrieve Node %q", fakeNode.Name) gomega.Expect(retrievedNode.Name).To(gomega.Equal(fakeNode.Name), "Checking that the retrieved name has been found") ginkgo.By(fmt.Sprintf("Patching %q", fakeNode.Name)) payload := "{"metadata":{"labels":{"" + fakeNode.Name + "":"patched"}}}" patchedNode, err := nodeClient.Patch(ctx, fakeNode.Name, types.StrategicMergePatchType, []byte(payload), metav1.PatchOptions{}) framework.ExpectNoError(err, "Failed to patch %q", fakeNode.Name) gomega.Expect(patchedNode.Labels).To(gomega.HaveKeyWithValue(fakeNode.Name, "patched"), "Checking that patched label has been applied") patchedSelector := labels.Set{fakeNode.Name: "patched"}.AsSelector().String() ginkgo.By(fmt.Sprintf("Listing nodes with LabelSelector %q", patchedSelector)) nodes, err := nodeClient.List(ctx, metav1.ListOptions{LabelSelector: patchedSelector}) framework.ExpectNoError(err, "failed to list nodes") gomega.Expect(nodes.Items).To(gomega.HaveLen(1), "confirm that the patched node has been found") ginkgo.By(fmt.Sprintf("Updating %q", fakeNode.Name)) var updatedNode *v1.Node err = retry.RetryOnConflict(retry.DefaultRetry, func() error { tmpNode, err := nodeClient.Get(ctx, fakeNode.Name, metav1.GetOptions{}) framework.ExpectNoError(err, "Unable to get %q", fakeNode.Name) tmpNode.Labels[fakeNode.Name] = "updated" updatedNode, err = nodeClient.Update(ctx, tmpNode, metav1.UpdateOptions{}) return err }) framework.ExpectNoError(err, "failed to update %q", fakeNode.Name) gomega.Expect(updatedNode.Labels).To(gomega.HaveKeyWithValue(fakeNode.Name, "updated"), "Checking that updated label has been applied") ginkgo.By(fmt.Sprintf("Delete %q", fakeNode.Name)) err = nodeClient.Delete(ctx, fakeNode.Name, metav1.DeleteOptions{}) framework.ExpectNoError(err, "failed to delete node") ginkgo.By(fmt.Sprintf("Confirm deletion of %q", fakeNode.Name)) gomega.Eventually(ctx, func(ctx context.Context) error { _, err := nodeClient.Get(ctx, fakeNode.Name, metav1.GetOptions{}) if apierrors.IsNotFound(err) { return nil } if err != nil { return fmt.Errorf("nodeClient.Get returned an unexpected error: %w", err) } return fmt.Errorf("node still exists: %s", fakeNode.Name) }, 3*time.Minute, 5*time.Second).Should(gomega.Succeed(), "Timeout while waiting to confirm Node deletion") }) }) </ins>

[x] two weeks soak end date : 2024-09-08 According to following APIsnoop query, there are two Node endpoints that are untested. Using a number of existing e2e test practices a new has been created to provide future Conformance coverage for the 2 endpoints. The e2e logs for this test are listed below. This query shows the following Node endpoints are hit within a short period of running this e2e test. If a test with these calls gets merged, test coverage will go up by 2 points This test is also created with the goal of conformance promotion. /sig testing /sig architecture /area conformance
/triage accepted
/sig node
/reopen Tracking the next steps to conformance
Reopened this issue. <detailsIn response to : /reopen Tracking the next steps to conformance Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen Tracking the final step
Reopened this issue. <detailsIn response to : /reopen Tracking the final step Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: // Introduce some small jittering to ensure that over time the requests won't start // accumulating at approximately the same time from the set of nodes due to priority and // fairness effect. <del> go wait.JitterUntil(kl.syncNodeStatus, kl.nodeStatusUpdateFrequency, 0.04, true, wait.NeverStop) </del> <ins> go func() { // Call updateRuntimeUp once before syncNodeStatus to make sure kubelet had already checked runtime state // otherwise when restart kubelet, syncNodeStatus will report node notReady in first report period kl.updateRuntimeUp() wait.JitterUntil(kl.syncNodeStatus, kl.nodeStatusUpdateFrequency, 0.04, true, wait.NeverStop) }() </ins> go kl.fastStatusUpdateOnce() // start syncing lease

After restart Kubelet, node will become notReady in first kubelet update period. node condition is : in kubelet's logs: when go in to next period (default 10s), node status resume to Ready. Restart Kubelet should not make node status change to notReady. Restart Kubelet, and watch node status for second The root case maybe here: When kubelet start, it will start two go routine, one detect runtime status, one check node status and update to APIServer; detect runtime status: in it will update : in it will check runtimeStatus include : is one of the notReady setter func: So if nodestatus execute before , kubelet will update a node status with notReady condition: In kubelet starting, both go routine and will be called before , so the first node status is notReady. In our cluster is v1.22.17, but I belive it can reproduce in latest version. <details</details<details</details<details</details<details</details<details</details<details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig node
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : /sig node Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I can reproduce this issue in v1.30 When the container runtime is healthy, the kubelet should not report with the reason We called in the function to update the status of the container runtime, but we did not do this in , We only need to execute function once before and , just like we did in the function, than this problem can be avoided. But in , seems to have some different opinions...
Yeah, in our env, we modify kubelet the same way as you said, to let it execute once before , it worked. I can submit a PR later.

I may be missing something but what happens if something has changed that caused the kubelet to not be ready ... imagine, the Network plugin is not working ... during the initial period until the check changes the state pods will be scheduled on the node and will fail .... I think that we are operating the assumption that this is a restart and during that time nothing changed, but is this a safe assumption or can we guarantee somehow that nothing has changed that could impact the node readiness state?
In this issue, what I found is that nothing has changed, every time I restart kubelet, node will become notReady only in first sync period, in next period it will be ready; It's not an assumption. or other problem of course will cause node notReady, but they are not in this issue's scope, in this scope, notReady reason is message, have not relationship with other problem;
ok, I misread this issue sorry, so it is not to blindly set the node to ready, is just to perform the runtime check before the other checks ... I think you both are right ... actually it seems that if wins the race then there is no problem, righ ?
Yes, but and run in different goroutines, so we cannot guarantee that will complete faster. So it seems like a good choice to execute before running , or do you have better suggestions?
We need to put the goroutines in the right order to make the startup more predictable
Yes, if execute very fast, may there is no problem, because in it will execute . I have test in v1.28 (which code in fastStatusUpdateOnce almost same as 1.30 ) , it will not always reproduce, but still occurr some times. In 1.22, will sleep for about 100ms before everything, it can be reproduced everytimes.
Arranging the goroutines in order can solve our problems in most cases, but in the method, we need to call the containerRuntime api to get the status. I am worried that its response will not be timely enough in special scenarios, there may be a risk that will complete faster.
moving will execute first
I think I must not have expressed clearly. What I'm worried about is that under special circumstances, calls the API of the container runtime and cannot return immediately (not sure if this situation really exists).

Even if the method is executed first, it will still complete later than :)
Add some detail: in it call runtime api to check runtime status, then set variable . in , it will check variable, if is nil, this issue's problem will occurr. Even if goroutine call firstly, call container runtime check will still need some time, it can't guarantee that when firstly call, is setted.
are you suggesting to lock syncNodeStatus on updateRuntimeUp? I'm afraid of some corner cases we can hit ... if the container runtime does not return do we block forever?
Container runtime API call have timeout.
Can you try this on a newer version? 1.22 is out of support and we have made changes to node readiness since. /triage needs-information
I have tried in 1.28, it can reproduce, can reproduce in 1.30.
/remove-triage needs-information


Question: // based on cached predicate results func (ec *EquivalenceCache) PredicateWithECache( podName, nodeName, predicateKey string, <del> equivalenceHash uint64, </del> <ins> equivalenceHash uint64, needLock bool, </ins> ) (bool, []algorithm.PredicateFailureReason, bool) { <del> ec.RLock() defer ec.RUnlock() </del> <ins> if needLock { ec.RLock() defer ec.RUnlock() } </ins> glog.V(5).Infof("Begin to calculate predicate: %v for pod: %s on node: %s based on equivalence cache", predicateKey, podName, nodeName) if algorithmCache, exist := ec.algorithmCache[nodeName]; exist {

Fix this TODO (raised by : Part of: One thoughts to fix this is, we can redesign the equivalence cache data structure to: bool instead of deleting the entry. certain version tag in the entry, update the version per WRITE op (Update and Invalidate). the version after eCache predicate, and check it before update eCache.
Another possible approach is: eCache before running a predicate. the predicate. the eCache for the predicate. eCache. Basically, in the existing code all predicates are run first and their results are collected and then at the end of the function the eCache is updated. Instead of that, we can move the eCache update for each predicate inside the for loop and next to each predicate execution and synchronize before start of the execution of each predicate. The downside to my suggested approach is that performance may suffer due to frequent locking and unlocking.
Prefer to bsalamat@ 's option which is simpler :). Comparing to performance improvement by eCache, the lock is acceptable to me.
yup same here, going with the locking is easier and is acceptable. - also using a flag for invalidation might just move the problem from eCache to the flag ?
Oh I missed this notification somehow, busy on cleaning scheduler integration test. Yep so if locking is acceptable, it should be cleanest way to fix this, optimistic control would be a more complicated story.
- are you handling this one ?
Already been taken by volunteer days ago :)


Question: HasRandomFully() bool } <ins> // Protocol defines the ip protocol either ipv4 or ipv6 </ins> type Protocol byte const ( <ins> // ProtocolIpv4 represents ipv4 protocol in iptables </ins> ProtocolIpv4 Protocol = iota + 1 <ins> // ProtocolIpv6 represents ipv6 protocol in iptables </ins> ProtocolIpv6 ) <ins> // Table represents different iptable like filter,nat, mangle and raw </ins> type Table string const ( <del> TableNAT    Table = "nat" </del> <ins> // TableNAT represents the built-in nat table TableNAT Table = "nat" // TableFilter represents the built-in filter table </ins> TableFilter Table = "filter" <ins> // TableMangle represents the built-in mangle table </ins> TableMangle Table = "mangle" ) <ins> // Chain represents the different rules </ins> type Chain string const ( <ins> // ChainPostrouting used for source NAT in nat table </ins> ChainPostrouting Chain = "POSTROUTING" <del> ChainPrerouting  Chain = "PREROUTING" ChainOutput      Chain = "OUTPUT" ChainInput       Chain = "INPUT" ChainForward     Chain = "FORWARD" </del> <ins> // ChainPrerouting used for DNAT (destination NAT) in nat table ChainPrerouting Chain = "PREROUTING" // ChainOutput used for the packets going out from local ChainOutput Chain = "OUTPUT" // ChainInput used for incoming packets ChainInput Chain = "INPUT" // ChainForward used for the packets for another NIC ChainForward Chain = "FORWARD" </ins> ) const (

Anyone itching to understand the code and try fixing things in Kubernetes, please help fixing the lint failures Step 1: Remove one or two packages (related packages) from the list in - (note: do not change generated files, the name of defaulting functions, or the name of conversion functions... packages failing linting due to issues in those files will need to be left alone) Step 2: Run and look at the logged errors Step 3: Fix the code to ensure that runs cleanly. Step 4: Run and to make sure we fix any problems with gofmt or bazel. Step 5: File a PR with the change Step 6: Go to step :) Please use the tracking spreadsheet For everyone who is going to contribute, please append the packages you picked at the end. For existing contributor, please make sure this list is update to date
/sig architecture /sig testing
Hey, I would like to resolve this issue. Could you assign it to me ?
there are a lot of packages so there will be many people working on this. Also we cannot assign issue to folks who are not in the github org for kubernetes. please feel free to pick up a few packages like i mentioned and file a PR. Make sure you link back to this issue, so folks can see which packages you filed PR for.
OK. I see. I will firstly pick up the package called to fix.
Hi dims. Should i submit my PR before i remove one or two packages (related packages) from the list in - Or I first remove the package from list, then I fix the issue and submit a PR?
in your PR, you will have both the fixes to packages and some lines removed from /.golint_failures
When I run , I found some errors as below: I found that there is a go file in But there is still a error.
Say your GOPATH looks like this Your kubernetes directory should be () cd to and then run from that directory.
Is it necessary to obey all rules in golint? Rules in golint are set in stone. Maybe using gometalinter to ignore some specific errors is better?
I revise some erros about golint in some packages. Please review it.
which errors would you like to ignore?

if you can open a new issue with the list of errors to ignore (and proposing gometalinter), we can get some opinions there from others in the community.
Let me take the kubectl cmd part...
Can someone please review and let me know if I am doing it right?
I ll work on all the packages related to
we get 800-ish pkgs left to fix. my concern is that if we fix them package by package, then such pulls will be flooding the repo. can't we possibly make it in larger pulls? the larger the better?
+1 to slightly larger pulls with related packages. if allow too large then the reviewers will not be able to do a thorough review. we should spread the load among multiple reviewers. (if we allow a really large pull, we will end up with top level reviewers who will face the burden and the PR(s) will just stagnate)
oh good point. we could put one pkg's change into one commit, and merge all changes under one area into one pull so that it wont spread the load. wdyt? for each pull, we can have hundred-ish lines of change. but it will still be 30~50 pulls i guess? good news is that we can take our time to deal with it, at least before 1.12
sounds great! please feel free to help make it happen :) (by suggesting groups of packages that kinda go together)
I'm looking at fixing pkg/proxy, and I think I see what was talking about ignoring some rules. proxy.ProxyProvider is being suggested because golint is saying that it stutters. In that example, I think it's okay renaming it to proxy.Provider, but I could see an example where you would want to keep the stutter if the package that it's declared in uses it heavily.
sure thing, feel free to propose a PR for switching to gometalinter independently of other patches. We can work through that there.
I sent the pull request to fix pkg/volume/aws_ebs
I've fixed a few of these, but get the following error when I try to run : Are there any instructions on what I need to setup in order to resolve this?

you can find this file in
I have raised a PR to fix golint errors in pkg/api
I have fixed some pkgs under /pkg/kubelet pkg/kubelet/images pkg/kubelet/kuberuntime pkg/kubelet/leaky pkg/kubelet/pleg
I submitted a pull request for golint errors in pkg/security/ edit: I can work on picking up pkg/controller/bootstrap through pkg/controller/namespace/... (114-129)
fix golint errors in package cmd/kubeadm/app, since the codes under cmd/kubeadm/app/apis/kubeadm/v1alpha2 and cmd/kubeadm/app/apis/kubeadm/v1alpha3 are auto-generated, i keep them untouched.
I am working on some packages under pkg/kubelet, anyone interested in this plz leave messages for me :)
fix golint for pkg/util/dbus pkg/util/labels pkg/util/node pkg/util/strings pkg/util/workqueue/prometheus
PR fixes golint errors in pkg/routes,pkg/securitycontext, pkg/serviceaccount
PR fixes golint in pkg/probe
should we have a checklist as part of the issue summary saying what is in progress and what is done? Since this issue is getting long, this will make new contributors to pick up issues easier.
Great idea!, want to throw a google doc with a check-list so it's easy for participants to organize themselves? (please!) If it is the issue summary then one of us with privileges will be a bottle neck.
Created a with all the package that have on going PRs. Maybe you can add this to the summary. For everyone who is going to contribute, please append the packages you picked at the end. For existing contributor, please make sure this list is update to date
Thanks updated the spread sheet
Raised PR to fix golint issues in: pkg/controller/clusterroleaggregation pkg/controller/cronjob pkg/controller/daemon pkg/controller/deployment pkg/controller/deployment/util pkg/controller/disruption pkg/controller/endpoint
This issue is close in accident by . could you reopen it?
/reopen
Reopening this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I found a problem trying to fix golint messages for .

Golint complaints on 's function because of the snake case. When I try to fix it, fails because it is being used in a code generated file. If I clean generated files, tests pass, but the generated file is not invoking it anymore. I failed to find the generator definition using this so I could change the hook to just . Also I'm not sure if this is wanted. I also found other packages in also probably affected by this. I read somewhere about replacing golint with gometalinter, which would allow to define directives in code to omit certain rules. I would consider to have a second exclusion file for golint with these cases, to be updated if move to gometalinter is finally done. This would make this issue completion more feasible. Meanwhile, I'll move onto fixing golint messages for other packages, until this discussion gets cleared.
I also found a problem fixing . A test fails because the directory is marked as readonly (see tests @ Shall we also ignore this directory for this issue?
Is this issue still relevant because hack/verify- is giving me no errors.
the failures are still present in there are details in the first message on how to claim + solve issues :)
Sorry I misunderstood the issue
I'm working on this package:
pkg/apis/admission/v1beta1 pkg/apis/authentication/v1beta1 pkg/apis/authorization/v1beta1 pkg/apis/coordination/v1 pkg/apis/coordination/v1beta1 pkg/apis/imagepolicy/v1alpha1 pkg/apis/scheduling/v1alpha1 pkg/apis/scheduling/v1beta1 pkg/apis/settings/v1alpha1 pkg/apis/storage/v1alpha1
pkg/cloudprovider/providers/vsphere/vclib pkg/kubelet/apis/deviceplugin/v1alpha pkg/kubelet/apis/pluginregistration/v1alpha1 pkg/kubelet/apis/pluginregistration/v1beta1
:
I'm looking at the pkg/registry/*, all those which are left. I expect to give an update on what all is fixed in the next 24-48 hours.
hi there is a google sheet that is a bit outdated, maybe you can update that?
hi I seem to only have view access for the sheet. Can't edit.
I fixed the permission.

Please try it again
pkg/apis/authentication pkg/apis/coordination pkg/apis/events pkg/apis/imagepolicy pkg/apis/settings
hi I think this issue was closed prematurely by (using fixes #issue in commit)
I'll put in a PR shortly for the package
Looking at next
hi thanks! there is a spreadsheet here that tracks the progress on fixing these issues, consider adding the packages you're work on there so we can keep everything organized :+1:
Starting on now
I did indeed see the Google Sheet link (thank you and to take it a little further
test/e2e
test/e2e_node/builder
pkg/registry/admissionregistration/mutatingwebhookconfiguration/storage pkg/registry/admissionregistration/rest pkg/registry/admissionregistration/validatingwebhookconfiguration/storage pkg/registry/apps/daemonset/storage
Hi. I opened a PR - Please review.
Working on Will open a PR soon.
Starting on and should have a PR soon
I keep getting this golint verify error on and was wondering if others saw it: But when I that package to the golint failures in my PR, my reviewer said it isn't necessary? What does that mean? What did I miss? Should I use a different kind of verify? cf:
looks like the disconnect is because that problem does not show up in the version of we use in the CI. please see
Still confused: is that not the golint version I ran? it installs from vendor and uses that I think?
i installed from and i still don't see the problem, can you please check what version of golint you have?
I'm confused why I wouldn't be using the vendored version? how would that get out of sync?
/me looking
Note: I am reserving for a KubeCon Contributor Workshop demonstration. It is noted in the spreadsheet.
I think may not be ideal cause there's a lot of auto generated files there. What do you think about reserving instead for the KubeCON Contributor Workshop?
pkg/apis/scheduling pkg/apis/storage/util pkg/apis/storage/v1beta1/util pkg/apis/storage/v1/util
/priority backlog
Hi I am a new contributor here, I'd like to contribute and make changes to the following file/s. I have marked it as WIP in the Google Sheet. Can someone let me know if I have to do anything else before I start working on this PR?

Umm so there's an issue where Golint tells that underscore shouldn't be allowed. Here's a log of the go-lint warnings:
Thanks The following should not be modified, as they are designed like this for code generation.
I'd like to pick a package and do my first PR, is there a list of packages to be done? Or should I correlate with this issue thread? Thanks in advance!
please see step 1 () please scan other PRs linked to this issue as well just in case.
Will do. Thank you. On Thu, Oct 3, 2019, 7:25 AM Davanum Srinivas <:
Hi there, I'm looking for a good first issue. I'll start taking a look at if that's ok? There doesn't seem to be anyone currently tracked as looking at it in the attached spreadsheet.
I've found a few others I'm actively looking at, you won't get any argument from me. On Sat, Oct 12, 2019, 10:38 AM Pamela Kelly <:
I am working on the iptables pkg. Updated in the google docs
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I have problem running in step 2 $ hack/verify- what should I do?
PTAL for my pr to this issue, thx
you can move kubernetes directory to and re-run this script.
I'd like to take Can we fix linter issues for generated files? Or we should skip packages with zz_generated?
There're 2 lint errors that affects 292/507 packages, meaning it's not possible to cleanup the entire file without solving it: default pattern of Kubectl is the usage of [cmd]Options for initial options structs and it raises: /apis/{custommetrics,externalmetrics} for example raises the underscore error, but It's probably not worth to break backward compatibility renaming these folder for lint error fix: Any advice on these?

/reopen hm, cannot reopen this issue, it was closed by mistake
You can't reopen an issue/PR unless you authored it or you are a collaborator. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I am sure this is all not fixed. Am I missing something? ./hack/verify- installing golint from vendor Congratulations! All Go source files have been linted.
you need to remove package from first.
/remove-sig testing SIG testing doesn't own code failing lints, we own the linter. :upsidedownface:
Hello, is this still a relevent issue ? I would like to work upon
I have fixed
Hi Folks, Some of the golint fixes, require us to change the exposed function's name, in simple words, "change API". I was not sure, if that is welcomed and what is the process.
Heres what Im getting after going over the following steps as mentioned by Say your GOPATH looks like this GOPATH=/Users/dims/go Your kubernetes directory should be ($) cd to and then run hack/verify- from that directory. I get the following output - installing golint Congratulations! All Go source files have been linted. Im not really sure what I am doing wrong. Any help is appreciated!
I think I understand. I misunderstood the steps around .golint_failures. The first step is to remove the pkg that we are targeting from this file and then run the My mistake was to assume that the packages in the file need to be removed after we are done cleaning the golint errors.
Can someone please review and let me know if I've done this right?
After a couple years of seeing changes spawned from this issue, I don't think this is a good use of contributors' or reviewers' time.

golint is very opinionated about: exported type and function names, leading to PRs attempting to change exported things people use downstream missing documentation on interface implementation functions, which leads to not-very-useful copy/paste of interface godoc the govet and staticcheck verify jobs are much more useful since they catch correctness issues, but I would recommend closing this issue and dropping the golint checks from CI golint doc itself discourages from using its output in an automated enforced way: from
i guess one can say gofmt is opinionated too, but something good about Golang is that it comes with an opinion and a set of standard tools. for API documentation at least we found inconsistencies on how most exported API items are documented: golint can help with that, but if there are no "pragmas" to avoid breaking changes and given we have codegen generating non-compliant names enabling golint on API packages is sort of blocked in Kubernetes. for non-API, non-library, non-public code, i think being inconsistent is mostly OK. no preference on my side. shrug
true, but code formatting has the benefits of mostly eliminating meaningless diffs in PRs due to code movement/whitespace/etc if there are specific packages (like API packages) we lack documentation on, adding checks specifically for that could make sense exactly, I don't think golint is the right tool for making sure we have API docs
golint itself doesn't have these and the go project is against this sort of comment AIUI, however golangci-lint does have them. I think there are plenty of other static analysis lints we should use instead, beyond just staticcheck and vet there is e.g.
btw, if we remove golint can we see e.g. all-caps constants or functions with underscores dodging review? perhaps we should add golangci-lint soon after:
Completely agree there are better places to spend review bandwidth. I was curious to see what the effort looked like. There have been 890 commits that touched over the past 3+ years !
/remove-help /remove-good-first-issue
Overall agree on the bandwidth. Also note that it was a good first issue with very low bar to get them in the repo. May be we need to find a similar low bar, but high value good first issue.

UT's come to mind, are there other things we curate for new contributors ?
Do we have an alternative way of enforcing that new exported code (not necessarily API fields) has at least some minimum documentation? As a reviewer, I ask contributors to add meaningful comments. But reality is that I sometimes stumble into old code that doesn't have documentation because we didn't have golint at some point. I agree that fix-golint PRs just add burden to reviewers. But, before removing the lint check, can we wait and see if it's enough to discourage people from sending more PRs like this? Additionally, we can encourage reviewers to just close PRs that only fix golint issues.
+1 for adding burden to reviewers. However, as a new contributor, I think this a very good opportunity for new contributors to study the code base and submit their first PR. In my opinion, I think this issue would be able to attract more new contributors, get familiar with the code base, and get familiar with the git workflow.
thanks everyone for getting us this far. We essentially gave up on the existing structure/system for golint: At some point we can look at adding golangci-lint, there's a WIP in progress in /close
Closing this issue. <detailsIn response to : thanks everyone for getting us this far. We essentially gave up on the existing structure/system for golint: Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: return } } klog.V(4).Infof("updatePod %s/%s for node %s", pod.Namespace, pod.Name, pod.Spec.NodeName) <ins> startTime := time.Now() </ins> g.graph.AddPod(pod) <ins> klog.V(5).Infof("updatePod %s/%s for node %s completed in %v", pod.Namespace, pod.Name, pod.Spec.NodeName, time.Since(startTime)) </ins> } func (g *graphPopulator) deletePod(obj interface{}) {

Log statements and metrics indicating when the node authorizer graph has be updated for a pod. There is already a log statement for when the graph starts an update, but this is before the graph lock is obtained and doesn't tell when the graph update is completed. We are seeing node authorizer failures which prevent pods from starting for extended periods of time. It is believed that this is caused by a race between: the node authorizer graph being updated the node requesting the secret It is important to have logging that indicates when the graph finished updating in order to reflect lock contention on the graph.
/sig auth
/triage accepted Seems like a reasonable feature request.
Hi, folks I would like to work on this. /assign
Hi Fixed this issue here PTAL!


Question: if err := kubeoptions.DefaultAdvertiseAddress(s.GenericServerRunOptions, s.InsecureServing); err != nil { return err } <del> _, apiServerServiceIP, err := master.DefaultServiceIPRange(s.ServiceClusterIPRange) </del> <ins> serviceIPRange, apiServerServiceIP, err := master.DefaultServiceIPRange(s.ServiceClusterIPRange) </ins> if err != nil { return fmt.Errorf("error determining service IP ranges: %v", err) } <ins> s.ServiceClusterIPRange = serviceIPRange </ins> if err := s.SecureServing.MaybeDefaultWithSelfSignedCerts(s.GenericServerRunOptions.AdvertiseAddress.String(), []string{"kubernetes.default.svc", "kubernetes.default", "kubernetes"}, []net.IP{apiServerServiceIP}); err != nil { return fmt.Errorf("error creating self-signed certificates: %v", err) }

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: What happened: Running kube-apiserver without specifing "--service-cluster-ip-range" will print “no --service-cluster-ip-range specified” and exit. But it provide a default value "10.0.0.0/24". Do we need remove validation of --service-cluster-ip-range specified? What you expected to happen: Since we have provided the default value '10.0.0.0/24', user need not to specify --service-cluster-ip-range in command line. How to reproduce it (as minimally and precisely as possible): Run kube-apiserver without "--service-cluster-ip-range". kube-apiserver --etcd-servers="http://localhost:2379" --storage-backend="etcd2" Anything else we need to know?: Environment: Kubernetes version (use ): Client Version: version.Info{Major:"1", Minor:"9+", GitVersion:"v1.9.0-alpha.0.572+", GitCommit:"", GitTreeState:"clean", BuildDate:"2017-09-13T05:45:41Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"9+", GitVersion:"v1.9.0-alpha.0.584+", GitCommit:"", GitTreeState:"clean", BuildDate:"2017-09-13T08:56:26Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"} Cloud provider or hardware configuration**: OS (e.g. from /etc/os-release): Kernel (e.g. ):4.10.0-33-generic Install tools: Others:
/sig api-machinery
I think the problem is that we do not set the default value for the ServiceClusterIPRange, we can see the code: and so the validation will fail.
No, it had set the default value in
We can see the execution sequence In the function we do not set the value and then we go to the function ,so there is something wrong. And the Line you point out is for construct . Maybe there is a little confusion.

So, I think why we must validate service-cluster-ip-range specificaiton which have been provided a default value after validation: )
I first think there may other use for Validate and ServiceClusterIPRange, So we can directly set the value for ServiceClusterIPRange and should not use DefaultServiceIPRange everywhere. I think you PR may be better
cc


Question: v1.LabelTopologyRegion, } <ins> func translateToGALabel(label string) string { if label == v1.LabelFailureDomainBetaRegion { return v1.LabelTopologyRegion } if label == v1.LabelFailureDomainBetaZone { return v1.LabelTopologyZone } return label } </ins> // Name returns name of the plugin. It is used in logs, etc. func (pl *VolumeZone) Name() string { return Name

In , we discovered that our plan for eventually removing the scheduling plugin when CSI migration is GA is not possible because the plugin also applies to non-migratable volume types. However, if we leave the plugin as is, then even migratable PVs will be broken if we remove beta topology labels from Nodes in the future. This is because the plugin does not have any handling for migrated volumes. I suggest we add CSI migration handling to the VolumeZone plugin. I think we just need to detect if the volume is migrated, and if so, skip the VolumeZone processing. For migrated volumes, we translate into so they end up getting processed by the plugin. In a cluster with csi migration for a specific intree volume type enabled, an intree PV with beta topology labels the beta topology label from a node to schedule a pod using that pv No response <details</details<detailsAny that supports csi migration </details<details</details<details</details<details</details<details</details
/sig storage /triage accepted /priority important-soon For e2e testing, this needs an environment that is configured with csi migration
Hi,I'd like to work on this issue. I previously wrote for plugin, so I'm familiar with it's logic. However I'm not familiar with the whole thing, and I may ask you some questions during the work :( Can I try this one? Here are some questions ahead: I think what you want to do is to schedule a pv with old labels to a node with only new labels. And you think we can translate into in volumezone during (maybe), and update this pv so that in stage, can , so we don't have compare Topology labels in . My question is: Can't we just translate this for all pvs? Why do we need to detect if the volume is migrated? What's the difference between a migrated pv and a non-migratable pv in this scenario?
I think this is also another alternative solution that could also work, but from , it may not scalable to do an automatic migration for all the different APIs and controllers that use the node labels. The main difference with CSI migration, is we have a good idea of all the different components that would be impacted and also have reasonable control over those components.
A few more questions Will the volumes be migrated before we take a pod off the scheduler queue?

If so, how can I detect this. Currently I find a function in , but nothing like . Is this translation done in plugin or before scheduling? I think currently in , we consider topology label and beta label as different labels. That is ,if we have a pv with and a node with , even if they have same values, will let the pod pass filter. If we're going to remove beta labels on node,maybe we should try to consider two labels as the same in ? Here's my understanding: For migrated volumes, we translate Topology labels to pv.NodeAffinity and update the pv, then we skip volume_zone(that is,don't compare labels with nodes). What about non-migratable volumes, I think we should continue to compare with node labels for non-migratable volumes?
For detecting if a PV is migrated, you can use the same method that's used in the VolumeBinding plugin: The only difference is you don't actually need to translate the PV in the VolumeZone plugin since it's already handled by the VolumeBinding plugin. For non-migratable volumes, I think we can continue with the same VolumeZone logic. For now, we'll rely on users to update their PVs with the GA labels.
Thanks ,for your detailed explanation. This is the implementation plan I prepared: In , we check if a pv is migrtable by using ,if so we skip volumezone since will translate this pv and handle it later, otherwise we save the label of this pv in state,and compare it with node's label in .Is that feasible to you?
Yup that sounds good to me!
Thanks! I'll try to work on this one. I'll file a pr a few days later and we can talk about the details in there. In the meantime, I think I previously wrote a bug in plugin, I wonder if you have time to confirm it /assign
This issue is labeled with but has not been updated in over 90 days, and should be re-triaged. Important-soon issues must be staffed and worked on either currently, or very soon, ideally in time for the next release. You can: Confirm that this issue is still relevant with (org members only) Deprioritize it with or Close this issue with For more details on the triage process, see /remove-triage accepted
/triage accepted


Question: if err != nil { t.Fatal(err) } <del> if !after.Has(111) || !after.Has(122) || !after.Has(133) { </del> <ins> if !after.Has(111) || !after.Has(122) || !after.Has(133) || !after.Has(144) { </ins> t.Errorf("unexpected portallocator state: %#v", after) } <del> if free := after.Free(); free != 98 { </del> <ins> if free := after.Free(); free != 97 { </ins> t.Errorf("unexpected portallocator state: %d free", free) } }

I came across a cluster where it has 2 services with port collision. One service looks like this: The other looks like: I checked the logic that allocates nodeport and healthcheck nodeport. They went thru the same node port allocator. Port allocation mainly happens in , and . The file in question is
cc
/cc
I'm not clear on your context. Which case were you facing? config and create the two services as you mentioned manifests, then get errors due to the collided ports? create the two services without specifying the two ports, then get the services and find they have collided ports? Also, what is the expected behavior you were thinking of?
When I checked the cluster, there were 2 services with colliding ports. I am not sure what brought the cluster into this state.
It's strange... I will try to dig into it later.
Seems like there is a race condition. PR sent. PTAL.
I first thought there was a race condition too. But I checked the API log. The operation on the 2 services are quite separated. Then I am not sure.
Oh... So strange. I will take another look at the code to try to find some clues. Btw, does the lock to make any sense? It doesn't fix this issue though.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten /remove-lifecycle stale
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-testing, kubernetes/test-infra and/or . /close
you can't re-open an issue/PR unless you authored it or you are assigned to it.

<detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
you can't re-open an issue/PR unless you authored it or you are assigned to it. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
you can't re-open an issue/PR unless you authored it or you are assigned to it. <detailsIn response to : /assign /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen
/assign
Found the issue. It's due to the nodeport repair controller ignoring the healthCheckNodePort.
Will send a PR and cherrypicks.


Question: ${BRIDGE_BASE}.0.0/16 dev ${DOCKER_BRIDGE} scope link src ${BRIDGE_ADDRESS} EOF # generate the post-configure script to be called by salt as cmd.wait cat <<EOF > ${POST_NETWORK_SCRIPT} #!/bin/bash set -e <del> # NAT interface fails to revive on network restart, so OR-gate to true systemctl restart network.service || true # set docker bridge up, and set stp on the ovs bridge ip link set dev ${DOCKER_BRIDGE} up ovs-vsctl set Bridge ${OVS_SWITCH} stp_enable=true </del> <ins> # Only do this operation once, otherwise, we get docker.servicee files output on disk, and the command line arguments get applied multiple times grep -q kbr0 /etc/sysconfig/docker || { # Stop docker before making these updates systemctl stop docker # NAT interface fails to revive on network restart, so OR-gate to true systemctl restart network.service || true # set docker bridge up, and set stp on the ovs bridge ip link set dev ${DOCKER_BRIDGE} up ovs-vsctl set Bridge ${OVS_SWITCH} stp_enable=true </ins> <del> # modify the docker service file such that it uses the kube docker bridge and not its own sed -ie "s/ExecStart=/usr/bin/docker -d/ExecStart=/usr/bin/docker -d -b=${DOCKER_BRIDGE} --iptables=false/g" /usr/lib/systemd/system/docker.service systemctl daemon-reload systemctl restart docker.service </del> <ins> # modify the docker service file such that it uses the kube docker bridge and not its own #echo "OPTIONS=-b=kbr0 --iptables=false --selinux-enabled" > /etc/sysconfig/docker echo "OPTIONS='-b=kbr0 --iptables=false --selinux-enabled'" >/etc/sysconfig/docker systemctl daemon-reload systemctl restart docker.service </ins> <del> # setup iptables masquerade rules so the pods can reach the internet iptables -t nat -A POSTROUTING -s ${BRIDGE_BASE}.0.0/16 ! -d ${BRIDGE_BASE}.0.0/16 -j MASQUERADE </del> <ins> # setup iptables masquerade rules so the pods can reach the internet iptables -t nat -A POSTROUTING -s ${BRIDGE_BASE}.0.0/16 ! -d ${BRIDGE_BASE}.0.0/16 -j MASQUERADE </ins> <del> # persist please iptables-save >& /etc/sysconfig/iptables </del> <ins> # persist please iptables-save >& /etc/sysconfig/iptables } </ins> EOF chmod +x ${POST_NETWORK_SCRIPT}

Which jobs are failing: capa-conformance-v1alpha3-k8s-master (ci-cluster-api-provider-aws-make-conformance-v1alpha3-k8s-ci-artifacts) Which test(s) are failing: [] Pods should get a host IP [NodeConformance] [Conformance] Since when has it been failing: 06-06 17:23 PDT Testgrid link: Reason for failure: <*errors.errorString | 0xc0000daf10: { s: "pod ran to completion", } pod ran to completion occurred skipped 89 lines unfold_more Jun 6 10:07:34.676: INFO: Latency metrics for node ip-10-0-0- Jun 6 10:07:34.676: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "pods-7990" for this suite. Anything else we need to know: /sig testing /cc /priority critical-urgent /milestone v1.19
/sig scheduling observing a timeout Jun 10 05:42:34.202: INFO: found a healthy node: test1-md-0-4qcf5 [It] runs ReplicaSets to verify preemption running path [Conformance] Jun 10 05:42:42.675: INFO: pods created so far: [1 1 1] Jun 10 05:42:42.675: INFO: length of pods created so far: 3 Jun 10 05:44:42.765: FAIL: pod sched-preemption-path-3834/pod4 failed to preempt other pods Unexpected error: <*errors.errorString | 0xc00011bdd0: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred Full Stack Trace (0xc000e48f20, 0xc001dbb800, 0x1bf08eb000) +0x16b skipped 318 lines unfold_more Jun 10 05:44:44.847: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "sched-preemption-4796" for this suite.

[AfterEach] [sig-scheduling] SchedulerPreemption [Serial]
/remove-sig scheduling I observing failure since yesterday as the cluster failed to get created using the CloudFormation template Attempting to create CloudFormation stack cluster-api-provider-aws-sigs-k8s-io Error: failed to create AWS CloudFormation stack: ResourceNotReady: failed waiting for successful resource stateError: failed to create AWS CloudFormation stack: ResourceNotReady: failed waiting for successful resource state
The label(s) cannot be applied, because the repository doesn't have them <detailsIn response to : /sig remove scheduling Error: failed to create AWS CloudFormation stack: ResourceNotReady: failed waiting for successful resource stateError: failed to create AWS CloudFormation stack: ResourceNotReady: failed waiting for successful resource state` Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/remove-sig scheduling
/assign
/remove-sig testing SIG testing doesn't own CAPA bringup failures
/sig cluster-lifecycle
It is expected that will provide a fix /cc
/assign
/lifecycle active


Question: # Starts cluster autoscaler. function start-cluster-autoscaler { <del> if [[ "${ENABLE_NODE_AUTOSCALER:-}" == "true" || "${ENABLE_CLUSTER_AUTOSCALER:-}" == "true" ]]; then </del> <ins> if [[ "${ENABLE_CLUSTER_AUTOSCALER:-}" == "true" ]]; then </ins> echo "Start kubernetes cluster autoscaler" prepare-log-file /var/log/cluster-autoscaler.log

In kube-up we use var while in GKE, documentation, issues etc I think we call it cluster autoscaler. We should make it consistent.
I assume this is what you were already thinking, but: I think we should change it to cluster autoscaler (everywhere). You scale the cluster by adding/removing nodes. You scale a node by adding/removing resources to/from the node, so node autoscaler isn't the right term IMO.


Question: func main() { rootCmd := &cobra.Command{ Use:     "app", <del> Version: "2.25", </del> <ins> Version: "2.26", </ins> } rootCmd.AddCommand(auditproxy.CmdAuditProxy)

<!-- STOP -- PLEASE READ! GitHub is not the right place for support requests. If you're looking for help, check the or our . You can also post your question on the or the forum. If the matter is security related, please disclose it privately via --s390x support for nonewprivs image was in and the commit from was cherry picked in to fix agnhost broken on s390x. However, the post merge job failed for both( ) as image builder was broken. Now that image builder has been fixed in , these jobs need to be restarted to push s390x tags of agnhost and nonewprivs in staging bucket. cc
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig testing
as you are on call this week, can you please run the CI job mentioned by hand?
, when I try to run the job with , it failed with . So I went over and looked at the history and found that this job was deleted by on 11/11. I don't think prow allows running a job that doesn't exist in its configmap, and searching the prow history with got: This is the list of jobs that have ever ran since prow keeps all history in GCS, which indicates that not all of the new jobs in has ran yet, can you take a look at and let us know which job is that you want?
thanks for looking. let's ask what jobs need to be triggered.
Hello, The post-kubernetes-push-e2e-test-images has been split into individual jobs for individual images and are now triggered by changes made in that image's folder. The jobs haven't been run yet since the split. You can trigger a job by sending a PR (e.g.: bumping the VERSION) and getting it merged.
/reopen Still failing according should fix it.
Reopened this issue. <detailsIn response to : /reopen Still failing according should fix it. Instructions for interacting with me using PR comments are available .

If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/close I am closing this issue as the agnhost job passed and the multiarch image is now available, it works fine for s390x as conformance job passed with the same
Closing this issue. <detailsIn response to : /close I am closing this issue as the agnhost job passed and the multiarch image is now available, it works fine for s390x as conformance job passed with the same Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details