Question: flag.StringVar(&TestContext.UpgradeImage, "upgrade-image", "", "Image to upgrade to (e.g. 'container_vm' or 'gci') if doing an upgrade test.") flag.StringVar(&TestContext.GCEUpgradeScript, "gce-upgrade-script", "", "Script to use to upgrade a GCE cluster.") flag.StringVar(&TestContext.FederationUpgradeTarget, "federation-upgrade-target", "ci/latest", "Version to upgrade to (e.g. 'release/stable', 'release/latest', 'ci/latest', '0.19.1', '0.19.1-669-gabac8c8') if doing an federation upgrade test.") <del> flag.StringVar(&TestContext.PrometheusPushGateway, "prom-push-gateway", "", "The URL to prometheus gateway, so that metrics can be pushed during e2es and scraped by prometheus. Typically something like 127.0.0.1:9091.") </del> flag.BoolVar(&TestContext.CleanStart, "clean-start", false, "If true, purge all namespaces except default and system before running tests. This serves to Cleanup test namespaces from failed/interrupted e2e runs in a long-lived cluster.") flag.BoolVar(&TestContext.GarbageCollectorEnabled, "garbage-collector-enabled", true, "Set to true if the garbage collector is enabled in the kube-apiserver and kube-controller-manager, then some tests will rely on the garbage collector to delete dependent resources.") }

TLDR: removed a feature but left documentation and the flag intact. Kubernetes version (use ): Client Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.0", GitCommit:"", GitTreeState:"clean", BuildDate:"2017-03-28T16:36:33Z", GoVersion:"go1.7.5", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.2", GitCommit:"", GitTreeState:"clean", BuildDate:"2017-01-12T04:52:34Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"} Environment: Cloud provider or hardware configuration: irrelevant OS (e.g. from /etc/os-release): irrelevant Kernel (e.g. ): irrelevant Install tools: kops Others: What happened: I am trying to set up periodic conformance tests on our cluster to make sure it is working as expected. Prometheus will be used for monitoring. After ing kubernetes I executed a command along the lines of: GINKGOPARALLEL=y KUBERNETESPROVIDER=local KUBEMASTER=local --host="" --test.v=true --"[(SlowSerial)]" --"[Conformance]" --prom-push-gateway=prometheus-push- The push gateway does never receive the metrics, because the feature was removed. What you expected to happen: The tests send the metrics to the push gateway. How to reproduce it (as minimally and precisely as possible): Search for the "PrometheusPushGateway" flag in this repo. It is used nowhere. Anything else we need to know: The feature was removed in but the flags were left in place and also in the documentation. It is frustrating to see that this flag is just a black hole. A solution to this would be to either reinstate the feature or remove the flag from docs and flags.
Blerg. pointed this out back in the PR that remove the functionality, let's follow through and remove the flags. Easy PR if someone wants it


Question: ARTIFACTS=${WORKSPACE}/_artifacts mkdir -p ${ARTIFACTS} <ins> if [[ -f "${KUBEKINS_SERVICE_ACCOUNT_FILE:-}" ]]; then echo 'Activating service account...'  # No harm in doing this multiple times. gcloud auth activate-service-account --key-file="${KUBEKINS_SERVICE_ACCOUNT_FILE}" # https://developers.google.com/identity/protocols/application-default-credentials export GOOGLE_APPLICATION_CREDENTIALS="${KUBEKINS_SERVICE_ACCOUNT_FILE}" unset KUBEKINS_SERVICE_ACCOUNT_FILE elif [[ -n "${KUBEKINS_SERVICE_ACCOUNT_FILE:-}" ]]; then echo "ERROR: cannot access service account file at: ${KUBEKINS_SERVICE_ACCOUNT_FILE}" fi </ins> go run test/e2e_node/runner/run_e2e.go  --logtostderr --vmodule=*=4 --ssh-env="gce"  --zone="$GCE_ZONE" --project="$GCE_PROJECT" --hosts="$GCE_HOSTS"  --images="$GCE_IMAGES" --image-project="$GCE_IMAGE_PROJECT"

I want to isolate these in their own project:
Right now just the CI jobs, but after we also need to do this for the PR jobs.


Question: // SpecSummaryOutput is the file to write ginkgo.SpecSummary objects to as tests complete. Useful for debugging and test introspection. SpecSummaryOutput string <ins> // DockerConfigFile is a file that contains credentials which can be used to pull images from certain private registries, needed for a test. DockerConfigFile string </ins> } // NodeKillerConfig describes configuration of NodeKiller -- a utility to

"[] Container Runtime blackbox test when running a container with a new image should be able to pull from private registry with secret [NodeConformance]" is failing consistently for Windows 1909 on GCE, but it's passing for 2019. Likely one of the containers it depends on hasn't been rebuilt as multi-arch? Failure message from
/sig windows cc
More useful output from build- Seems the container doesn't work on 1909. cc
/sig windows
This test is also failing on for Windows 1903.
see
Ah, thanks, I searched k/k for an existing issue but didn't think to search windows-testing. I posted a follow-up there.
Hi. It is a known issue, I've also reported it a while back:
, do you have push right to the ` manifest list to include the newer Windows OS versions. Basically, the commands are: docker manifest create docker manifest push --purge
/kind failing-test
I tried running this but got: I looked into the project and noticed and as owners. Are you able to assist?
/assign
docker manifest push Pushed ref with digest: sha256:ec76696da5a660a02bbfd304549d6e09d8c9fe48f2d4efe642f26cb1e46a6bde Pushed ref with digest: sha256:7c4ee446a647fdf8988672ae5e685cb66d53d4581e057b02e3a2bad728177f48 Pushed ref with digest: sha256:297cce9134ddaa2c09d91dea7ab7af0e58e06e59858c0311a0770143d144ddf1 sha256:fb9b25770487567c02bf90dd3edea7917323556d1b7ba81ec042ffd5f9effeae
this is fixed. the image manifest is available in
/close
Closing this issue. <detailsIn response to : /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details

<!-- Please only use this template for submitting reports about failing tests in Kubernetes CI jobs --Which jobs are failing: containerd-l2bridge-windows-master containerd-overlay-windows-master Which test(s) are failing: [] Container Runtime blackbox test when running a container with a new image should be able to pull from private registry with secret [NodeConformance] Since when has it been failing: Unknown. Testgrid link: Reason for failure: When creating the container, the following error occurs: Failed: Error: failed to create containerd task: hcsshim::CreateComputeSystem image-pull-test: The container operating system does not match the host operating system. Windows Server 2019 is used in the testing environment, and the image image, and the test is passing in all other jobs that are using docker instead of containerd. But the image has been built at the beggining of 2019, so it's most likely outdated (the Microsoft official images are updated regularly), which could lead to the OS version mismatch. Note that most of the other tests are passing for the containerd jobs, which are using the test images hosted in the image (LTS 2019 and 1809 have the same OS version), and they're always up-to-date. One solution could be to republish the image based on a freshly pulled nanoserver image. Anything else we need to know: /sig windows /sig testing
- is the login/password public so this can be debugged outside of the prow cluster?
- what containerd branch / hash was it built from?
- Some part of my memory thinks this was fixed in upstream containerd/cri. Do you recall any items around this recently?
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale


Question: var _ = SIGDescribe("Hostname of Pod [Feature:SetHostnameAsFQDN][NodeFeature:SetHostnameAsFQDN]", func() { f := framework.NewDefaultFramework("hostfqdn") <del> dnsDomain := "cluster.local" if cdn := framework.TestContext.ClusterDNSDomain; cdn != "" { dnsDomain = cdn } </del> /* Release: v1.19 Testname: Create Pod without fully qualified domain name (FQDN)

<!-- Please only use this template for submitting reports about continuously failing tests or jobs in Kubernetes CI --Which jobs are failing: / Which test(s) are failing: - Since when has it been failing: Since was merged Testgrid link: Reason for failure: Example: Looks like the at L122 and L144 doesn't actually get printed? Anything else we need to know: /sig node /cc /priority important-soon
Hi sorry about that, I will work on it asap. Are you aware of any difference between the CI that executes NodeFeature and NodeAlphaFeature? FYI this other PR merged at the same time
/assign javidiaz
Somehow, the /etc/hosts of the pod does not have subdomain # Kubernetes-managed hosts file. 127.0.0.1 localhost ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet fe00::0 ip6-mcastprefix fe00::1 ip6-allnodes fe00::2 ip6-allrouters 172.17.0.2 hostfqdn-.t.hostfqdn- hostfqdn- I think the issue is that framework.TestContext.ClusterDNSDomain is empty String.
Hi any idea of why cluster-subdomain is empty String in e2e node tests? These tests are failing because I made the wrong assumption that cluster-subdomain should be by default. So I set as the expected subdomain when framework.TestContext.ClusterDNSDomain is empty.
/triage accepted Thank you for looking into this, and for opening the issue :pray: Given i think you found the issue. node e2e tests have theirwn entrypoint so i guess we forgot to completely account for all the settings the e2e framework sets and to use those to configure the kubelet. I think in general we need to refactor the node test suite to prevent this sort of issue. Small note: full diff for the timeframe around which these tests began to fail (betwen 11:53 EST and 12:54 EST on 11-05).


Question: // Build ipvs kernel routes for each node ip address nodeIPs := make([]net.IP, 0) <del> addresses, err := utilproxy.GetNodeAddresses(proxier.nodePortAddresses, proxier.networkInterfacer) if err != nil { glog.Errorf("Failed to get node ip address matching nodeport cidr") continue } </del> for address := range addresses { if !utilproxy.IsZeroCIDR(address) { nodeIPs = append(nodeIPs, net.ParseIP(address))

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: kube-proxy was still binding to ::{port} for a node port service after it was started with the --nodeport-addresses argument What you expected to happen: kube-proxy to only bind to interfaces in the CIDR ranges specified by --nodeport-addresses. How to reproduce it (as minimally and precisely as possible): Start kube-proxy with --nodeport-addresses "127.0.0.1,::1" Create a service of type NodePort Observe kube-proxy bind to all listening interfaces Anything else we need to know?: In the proposal for the --nodeport-addresses feature, it specifies being able to have other services answering or listening on a given port. Since kube-proxy binds to all interfaces, this is still not possible so it appears one of the initial intents of the proposal was net met with the existing solution. [] Doesn't set an interface. Given that nodeport-addresses is specified as CIDR ranges extra work must be done to determine which IP's on a host exist that could be bound to. Alternatively, a flag such as --nodeport-addresses-bind could allow an operator to set an explicit value. To test functionality I was able to hardcode an address here and observe the desired behavior could be achieved. Environment: Kubernetes version:Cloud provider or hardware configuration: AWS OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Custom fabricated Others: As expected, this behaves the same way regardless of iptables or ipvs being used as the forwarding mechanism.
/sig network
/assign
Nice catch! I will fix it ASAP~
Please check


Question: ctx, cancel := context.WithCancel(context.Background()) defer cancel() err = PatchPodStatus(ctx, client, &tc.pod, &tc.statusToUpdate) <del> if err != nil { </del> <ins> if err != nil && tc.validateErr == nil { // shouldn't be error </ins> t.Fatal(err) } <ins> if tc.validateErr != nil { if !tc.validateErr(err) { t.Fatalf("Returned unexpected error: %v", err) } return } </ins> retrievedPod, err := client.CoreV1().Pods(tc.pod.Namespace).Get(ctx, tc.pod.Name, metav1.GetOptions{}) if err != nil {

If there is a connection failure when updating the Pod status, we don't retry. Furthermore, due to , we won't have another chance until much later. The problem is that if we don't mark the Pod as Unschedulable, other controllers (such as cluster-autoscaler) wouldn't react to these pods. We should have stronger guarantees to mark a Pod as Unschedulable. You need an unschedulable Pod (for example, a Pod with a non matching node affinity) and a flaky connection to apiserver. No response master This is particularly problematic in 1.24, due to . Any <details</details<details</details<details</details<details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scheduling
That is bad, retrying in place on transient errors should probably be ok.
I'll address it. /assign
Thanks!


Question: const nodeWithImpairedVolumes = "NodeWithImpairedVolumes" const ( <ins> // These constants help to identify if a node is a master or a minion labelKeyNodeRole    = "kubernetes.io/role" nodeMasterRole      = "master" nodeMinionRole      = "node" labelKeyNodeMaster  = "node-role.kubernetes.io/master" labelKeyNodeCompute = "node-role.kubernetes.io/compute" labelKeyNodeMinion  = "node-role.kubernetes.io/node" ) const ( </ins> // volumeAttachmentConsecutiveErrorLimit is the number of consecutive errors we will ignore when waiting for a volume to attach/detach volumeAttachmentStatusConsecutiveErrorLimit = 10 // most attach/detach operations on AWS finish within 1-4 seconds

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: We have pretty large clusters in aws and in production, we found an inefficiency in provisioning dynamic EBS volumes, where aws-ebs volume provisioner tries to look for available zone from aws directly. With scale, this can trigger massive from ec2, and affect cluster stability. This is also related to What you expected to happen: To simply get zones with available instances, we can get information from API server for efficiency. There might be a delay between node gets shutting down and such change is reflected in api server, but in case node is not ready or gets terminated, there will be higher level reconciliation triggered so we are safe. How to reproduce it (as minimally and precisely as possible): We only expect this to be hit when the cluster is huge (or aws account is huge) so not easy to reproduce. /assign /sig aws /sig cloud-provider
/assign


Question: boilerplate="$2" shift 2 ;; <ins> "--extra-peer-dir") extra_peers+=("$2") shift 2 ;; </ins> *) echo "unknown argument: $1" >&2 return 1

Earlier it was possible to pass flag to while calling script. Now with the new script, is called under which doesnot support any other flags apart from , and . As it can be seen here there is no option to pass flag to conversion-gen Earlier in the script, it was considering all the inputs passed flag is supported by under the hood, but the function in script is not allowing the flag.
/cc
/sig api-machinery
/assign for code generation /triage accepted
/assign
Can you explain why you need this flag? What I found in k/k was that we don't actually need it at all.
You know what, my memory was faulty. We do use it (until I fix that, which is in a development branch, not merged). But we don't use it in any of the "submodule" configs.
A lot of the "magic" behind the conversion logic is very brittle. Can you help me understand where / how this impacts you?
I whipped up a quick patch but have not tested much:
We are using the flag several places in our script On removal of the flag, the autoConvert functions in. files has the error : Althoug I can see the function is already present at . But there is also a comment like Hence, I think there is a need for the in order to use this pkg.
I see. The component-base is something I didn't account for. If you apply the above patch, does the new argument to work for you?
Above patch throws an error for the cases when flag is not passed to function. Instead I followed this approach :
I'd rather not accumulate into a string - can you tell me what error it throws? Note, mine wants Also - is the argument here a dir (e.g. /home/me/src/foo/bar) or a pkg (e.g. ) ? I think it's actually a pkg, right?
You implementation throws error for the cases when we don't specify the option while calling , as shown below: For such cases I am getting following error
It seems to be an old-bash problem. Try this one?
Yes this is working fine.
OK. I'll prep a PR soon.
Thank you


Question: len_sec=$((end_sec-start_sec)) set -o errexit kube::test::if_has_string "${output_message}" 'timed out waiting for the condition ' <del> kube::test::if_has_string "${len_sec}" '1' </del> <ins> test $len_sec -ge 1 && test $len_sec -le 2 </ins> # Clean deployment kubectl delete deployment dtest set +o nounset set +o errexit <del> } </del>  No newline at end of file <ins> } </ins>

test-cmd: runwaittests Dec 20 2022, 14:53:53 UTC+0100 No response No response /sig cli
/triage accepted /assign


Question: }) } <del> // Ignore changes that only affect managed fields // timestamps. FieldManager can't know about changes // like normalized fields, defaulted fields and other // mutations. // Only makes sense when SSA field manager is being used if scope.FieldManager != nil { transformers = append(transformers, fieldmanager.IgnoreManagedFieldsTimestampsTransformer) } </del> createAuthorizerAttributes := authorizer.AttributesRecord{ User:            userInfo, ResourceRequest: true,

Sending a patch built from a typed client to SSA will force a write to etcd (and bump resourceVersion) since creationTimestamp is serialized to null. This is a follow-up of which doesn't solve the issue properly. Sending a no-op SSA request with creationTimestamp to null shouldn't do anything Kubectl apply an object with creationTimestamp null twice and see that it updates the object. Probably something like this would cause it:
/sig api-machinery /wg api-expression /triage accepted /assign /priority important-soon


Question: import ( "bytes" <ins> "encoding/json" </ins> "fmt" "io" "strings"

and work in this pattern: first issuing a "GET" to the API server to retrieve the latest object, then adding the label/annotation to the object, and finally issuing an "PUT" with the modified object. The request will fail if the object stored in the server has been updated between the "GET" and "PUT", even if the update has no conflict with the label/annotate request. In such cases, instead of returning an error, should retry without disturbing the end user. There are two ways to implement this: in kubectl: simply retry the label/annotate process several times if the server rejected the request because the object has been updated. The only exception is when the is set, then kubectl shouldn't retry. the strategicMergePatch instead of calling Update at client-side. PATCH is more robust because PATCH does the "get, modify, update" in the server-side, so there is less chance to have a conflict, and PATCH will retry at the server-side (see ). We need to make some adaptions though: kubectl annotate/label has the , when it's false, kubectl should retrieve the latest object to confirm there is no existing annotation/label of the same key, and then proceed with issuing the patch. if the is set, [UPDATE:] we just need to specify the resourVersion in the patch, then it will be treated as a precondition by the UpdateResource() handler in the API server. I plan to take the first approach as an immediate fix.
Immediate need is to fix the flaky e2e test AIUI.
e2e? do we have an issue out there? is an immediate fix to the flake in test-
If this is for 1.1, it needs to be P0 at this point. Should it be?
Fwiw I use annotations both from cmd line as an admin and from scripts, to experiment with loadbalancer settings. In the case of the former, I've never really had a collision, and in the case of the latter my scripts know how to retry like we retry all other updates.
Thank you When used in scripts, the annotate and label commands have collisions frequently, e.g. in the test-, see . IMO, we shouldn't require users to retry in their scripts if we know there is a simple way to avoid the collisions. Anyway, and are sent to fix this.


Question: ginkgo.It("should reject an unloaded profile", func() { status := runAppArmorTest(f, false, v1.AppArmorBetaProfileNamePrefix+"non-existent-profile") <del> expectSoftRejection(status) </del> <ins> gomega.Expect(status.ContainerStatuses[0].State.Waiting.Message).To(gomega.ContainSubstring("apparmor")) </ins> }) ginkgo.It("should enforce a profile blocking writes", func() { status := runAppArmorTest(f, true, v1.AppArmorBetaProfileNamePrefix+apparmorProfilePrefix+"deny-write")

Also appears to be failing on: Failures as of 01/05. /kind failing-test <!-- If this is a flake, please add: /kind flake --/sig node
/triage accepted /priority important-soon Guessing since all tests with apparmor started failing - maybe app armor config was removed or some PR submitted recently.
/assign
ci-cri-containerd-node-e2e-features Last green run: First failed run:
This () is the only PR merged in kubernetes during the time between: Test started yesterday at 8:11 PM passed after 13m50s. (more info) Test started yesterday at 9:11 PM failed after 16m6s. (more info) And it happens to be related to AppArmor:
Working on a fix in


Question: var apiPodStatus api.PodStatus var podStatus *kubecontainer.PodStatus <ins> // Always generate the kubecontainer.PodStatus to know whether there are // running containers associated with the pod. podStatusPtr, apiPodStatusPtr, err := kl.containerRuntime.GetPodStatusAndAPIPodStatus(pod) if err != nil { glog.Errorf("Unable to get status for pod %q: %v", format.Pod(pod), err) return err } apiPodStatus = *apiPodStatusPtr podStatus = podStatusPtr </ins> if updateType == kubetypes.SyncPodCreate { // This is the first time we are syncing the pod. Record the latency // since kubelet first saw the pod if firstSeenTime is set. if !firstSeenTime.IsZero() { metrics.PodWorkerStartLatency.Observe(metrics.SinceInMicroseconds(firstSeenTime)) } <ins> // kubelet may have just been restarted. Re-use the last known // apiPodStatus. </ins> apiPodStatus = pod.Status apiPodStatus.StartTime = &unversioned.Time{Time: start} kl.statusManager.SetPodStatus(pod, apiPodStatus) <del> podStatus = &kubecontainer.PodStatus{ ID:        pod.UID, Name:      pod.Name, Namespace: pod.Namespace, } glog.V(3).Infof("Not generating pod status for new pod %q", format.Pod(pod)) } else { podStatusPtr, apiPodStatusPtr, err := kl.containerRuntime.GetPodStatusAndAPIPodStatus(pod) if err != nil { glog.Errorf("Unable to get status for pod %q: %v", format.Pod(pod), err) return err } apiPodStatus = *apiPodStatusPtr podStatus = podStatusPtr </del> <ins> glog.V(3).Infof("Reusing api pod status for new pod %q", format.Pod(pod)) </ins> } pullSecrets, err := kl.getPullSecretsForPod(pod)

As part of promoting TaintBasedEvictions to GA (), the docs around it will need to be updated to reflect the new status. /kind cleanup /sig scheduling
/assign
are you still planning to work on updating these docs?
sry, . assign by mistake. let me unassign.
What's the ETA? Till Friday? /assign
Hi Team - Seth on the 1.18 Docs team here, we have until 2/28 to get placeholder PRs in for docs for this release, please let me know if I can help!
Hi Jan just opened a PR here to reflect this change in the docs: could you please provide review to make sure we've properly listed this as GA?


Question: } // SplitAndParseResourceRequest separates the users input into a model and fields <del> func SplitAndParseResourceRequest(inResource string, mapper meta.RESTMapper) (string, []string, error) { </del> <ins> func SplitAndParseResourceRequest(inResource string, mapper meta.RESTMapper) (string, string, []string, error) { </ins> inResource, fieldsPath := splitDotNotation(inResource) <ins> group, inResource := splitGroupFromResource(inResource) </ins> inResource, _ = mapper.ResourceSingularizer(expandResourceShortcut(inResource)) <del> return inResource, fieldsPath, nil </del> <ins> return group, inResource, fieldsPath, nil </ins> } // PrintModelDescription prints the description of a specific model or dot path func PrintModelDescription(inModel string, fieldsPath []string, w io.Writer, swaggerSchema *swagger.ApiDeclaration, r bool) error { recursive = r // this is global for convenience <del> apiVer := swaggerSchema.ApiVersion + "." </del> <ins> apiVer := apiutil.GetVersion(swaggerSchema.ApiVersion) + "." </ins> var pointedModel *swagger.NamedModel for i := range swaggerSchema.Models.List {

It would be nice if let me explain experimental resources to aid users ability to experiment.
It's possible this works today, but it was non-obvious how to use. Attempts:
cc
cc
No, this isn't supported, since current implementation only use v1 Client but not experimental Client.
do we need to fix this for 1.1?
How difficult do you think it would be to fix?
It has dependency on ServerAPIVersions(), for which we plan to make a clean-up PR. I guess it needs 3-4 days to get two PRs merged.
With , this can be fixed now. What's the expected behavior of ? I assume in case of , it should search hpa in both v1 and extensions/v1bata1, and determines hpa as a resource in the extensions group? BTW, the use of "api-version" flag is problematic It adds extra meaning to the generic "api-version" flag.


Question: return allErrs } <ins> // ValidatePodLogOptions checks if options that are set are at the correct // value. Any incorrect value will be returned to the ErrorList. </ins> func ValidatePodLogOptions(opts *v1.PodLogOptions) field.ErrorList { allErrs := field.ErrorList{} if opts.TailLines != nil && *opts.TailLines < 0 {

Anyone itching to understand the code and try fixing things in Kubernetes, please help fixing the lint failures Step 1: Remove one or two packages (related packages) from the list in - (note: do not change generated files, the name of defaulting functions, or the name of conversion functions... packages failing linting due to issues in those files will need to be left alone) Step 2: Run and look at the logged errors Step 3: Fix the code to ensure that runs cleanly. Step 4: Run and to make sure we fix any problems with gofmt or bazel. Step 5: File a PR with the change Step 6: Go to step :) Please use the tracking spreadsheet For everyone who is going to contribute, please append the packages you picked at the end. For existing contributor, please make sure this list is update to date
/sig architecture /sig testing
Hey, I would like to resolve this issue. Could you assign it to me ?
there are a lot of packages so there will be many people working on this. Also we cannot assign issue to folks who are not in the github org for kubernetes. please feel free to pick up a few packages like i mentioned and file a PR. Make sure you link back to this issue, so folks can see which packages you filed PR for.
OK. I see. I will firstly pick up the package called to fix.
Hi dims. Should i submit my PR before i remove one or two packages (related packages) from the list in - Or I first remove the package from list, then I fix the issue and submit a PR?
in your PR, you will have both the fixes to packages and some lines removed from /.golint_failures
When I run , I found some errors as below: I found that there is a go file in But there is still a error.
Say your GOPATH looks like this Your kubernetes directory should be () cd to and then run from that directory.
Is it necessary to obey all rules in golint? Rules in golint are set in stone. Maybe using gometalinter to ignore some specific errors is better?
I revise some erros about golint in some packages. Please review it.
which errors would you like to ignore?

if you can open a new issue with the list of errors to ignore (and proposing gometalinter), we can get some opinions there from others in the community.
Let me take the kubectl cmd part...
Can someone please review and let me know if I am doing it right?
I ll work on all the packages related to
we get 800-ish pkgs left to fix. my concern is that if we fix them package by package, then such pulls will be flooding the repo. can't we possibly make it in larger pulls? the larger the better?
+1 to slightly larger pulls with related packages. if allow too large then the reviewers will not be able to do a thorough review. we should spread the load among multiple reviewers. (if we allow a really large pull, we will end up with top level reviewers who will face the burden and the PR(s) will just stagnate)
oh good point. we could put one pkg's change into one commit, and merge all changes under one area into one pull so that it wont spread the load. wdyt? for each pull, we can have hundred-ish lines of change. but it will still be 30~50 pulls i guess? good news is that we can take our time to deal with it, at least before 1.12
sounds great! please feel free to help make it happen :) (by suggesting groups of packages that kinda go together)
I'm looking at fixing pkg/proxy, and I think I see what was talking about ignoring some rules. proxy.ProxyProvider is being suggested because golint is saying that it stutters. In that example, I think it's okay renaming it to proxy.Provider, but I could see an example where you would want to keep the stutter if the package that it's declared in uses it heavily.
sure thing, feel free to propose a PR for switching to gometalinter independently of other patches. We can work through that there.
I sent the pull request to fix pkg/volume/aws_ebs
I've fixed a few of these, but get the following error when I try to run : Are there any instructions on what I need to setup in order to resolve this?

you can find this file in
I have raised a PR to fix golint errors in pkg/api
I have fixed some pkgs under /pkg/kubelet pkg/kubelet/images pkg/kubelet/kuberuntime pkg/kubelet/leaky pkg/kubelet/pleg
I submitted a pull request for golint errors in pkg/security/ edit: I can work on picking up pkg/controller/bootstrap through pkg/controller/namespace/... (114-129)
fix golint errors in package cmd/kubeadm/app, since the codes under cmd/kubeadm/app/apis/kubeadm/v1alpha2 and cmd/kubeadm/app/apis/kubeadm/v1alpha3 are auto-generated, i keep them untouched.
I am working on some packages under pkg/kubelet, anyone interested in this plz leave messages for me :)
fix golint for pkg/util/dbus pkg/util/labels pkg/util/node pkg/util/strings pkg/util/workqueue/prometheus
PR fixes golint errors in pkg/routes,pkg/securitycontext, pkg/serviceaccount
PR fixes golint in pkg/probe
should we have a checklist as part of the issue summary saying what is in progress and what is done? Since this issue is getting long, this will make new contributors to pick up issues easier.
Great idea!, want to throw a google doc with a check-list so it's easy for participants to organize themselves? (please!) If it is the issue summary then one of us with privileges will be a bottle neck.
Created a with all the package that have on going PRs. Maybe you can add this to the summary. For everyone who is going to contribute, please append the packages you picked at the end. For existing contributor, please make sure this list is update to date
Thanks updated the spread sheet
Raised PR to fix golint issues in: pkg/controller/clusterroleaggregation pkg/controller/cronjob pkg/controller/daemon pkg/controller/deployment pkg/controller/deployment/util pkg/controller/disruption pkg/controller/endpoint
This issue is close in accident by . could you reopen it?
/reopen
Reopening this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I found a problem trying to fix golint messages for .

Golint complaints on 's function because of the snake case. When I try to fix it, fails because it is being used in a code generated file. If I clean generated files, tests pass, but the generated file is not invoking it anymore. I failed to find the generator definition using this so I could change the hook to just . Also I'm not sure if this is wanted. I also found other packages in also probably affected by this. I read somewhere about replacing golint with gometalinter, which would allow to define directives in code to omit certain rules. I would consider to have a second exclusion file for golint with these cases, to be updated if move to gometalinter is finally done. This would make this issue completion more feasible. Meanwhile, I'll move onto fixing golint messages for other packages, until this discussion gets cleared.
I also found a problem fixing . A test fails because the directory is marked as readonly (see tests @ Shall we also ignore this directory for this issue?
Is this issue still relevant because hack/verify- is giving me no errors.
the failures are still present in there are details in the first message on how to claim + solve issues :)
Sorry I misunderstood the issue
I'm working on this package:
pkg/apis/admission/v1beta1 pkg/apis/authentication/v1beta1 pkg/apis/authorization/v1beta1 pkg/apis/coordination/v1 pkg/apis/coordination/v1beta1 pkg/apis/imagepolicy/v1alpha1 pkg/apis/scheduling/v1alpha1 pkg/apis/scheduling/v1beta1 pkg/apis/settings/v1alpha1 pkg/apis/storage/v1alpha1
pkg/cloudprovider/providers/vsphere/vclib pkg/kubelet/apis/deviceplugin/v1alpha pkg/kubelet/apis/pluginregistration/v1alpha1 pkg/kubelet/apis/pluginregistration/v1beta1
:
I'm looking at the pkg/registry/*, all those which are left. I expect to give an update on what all is fixed in the next 24-48 hours.
hi there is a google sheet that is a bit outdated, maybe you can update that?
hi I seem to only have view access for the sheet. Can't edit.
I fixed the permission.

Please try it again
pkg/apis/authentication pkg/apis/coordination pkg/apis/events pkg/apis/imagepolicy pkg/apis/settings
hi I think this issue was closed prematurely by (using fixes #issue in commit)
I'll put in a PR shortly for the package
Looking at next
hi thanks! there is a spreadsheet here that tracks the progress on fixing these issues, consider adding the packages you're work on there so we can keep everything organized :+1:
Starting on now
I did indeed see the Google Sheet link (thank you and to take it a little further
test/e2e
test/e2e_node/builder
pkg/registry/admissionregistration/mutatingwebhookconfiguration/storage pkg/registry/admissionregistration/rest pkg/registry/admissionregistration/validatingwebhookconfiguration/storage pkg/registry/apps/daemonset/storage
Hi. I opened a PR - Please review.
Working on Will open a PR soon.
Starting on and should have a PR soon
I keep getting this golint verify error on and was wondering if others saw it: But when I that package to the golint failures in my PR, my reviewer said it isn't necessary? What does that mean? What did I miss? Should I use a different kind of verify? cf:
looks like the disconnect is because that problem does not show up in the version of we use in the CI. please see
Still confused: is that not the golint version I ran? it installs from vendor and uses that I think?
i installed from and i still don't see the problem, can you please check what version of golint you have?
I'm confused why I wouldn't be using the vendored version? how would that get out of sync?
/me looking
Note: I am reserving for a KubeCon Contributor Workshop demonstration. It is noted in the spreadsheet.
I think may not be ideal cause there's a lot of auto generated files there. What do you think about reserving instead for the KubeCON Contributor Workshop?
pkg/apis/scheduling pkg/apis/storage/util pkg/apis/storage/v1beta1/util pkg/apis/storage/v1/util
/priority backlog
Hi I am a new contributor here, I'd like to contribute and make changes to the following file/s. I have marked it as WIP in the Google Sheet. Can someone let me know if I have to do anything else before I start working on this PR?

Umm so there's an issue where Golint tells that underscore shouldn't be allowed. Here's a log of the go-lint warnings:
Thanks The following should not be modified, as they are designed like this for code generation.
I'd like to pick a package and do my first PR, is there a list of packages to be done? Or should I correlate with this issue thread? Thanks in advance!
please see step 1 () please scan other PRs linked to this issue as well just in case.
Will do. Thank you. On Thu, Oct 3, 2019, 7:25 AM Davanum Srinivas <:
Hi there, I'm looking for a good first issue. I'll start taking a look at if that's ok? There doesn't seem to be anyone currently tracked as looking at it in the attached spreadsheet.
I've found a few others I'm actively looking at, you won't get any argument from me. On Sat, Oct 12, 2019, 10:38 AM Pamela Kelly <:
I am working on the iptables pkg. Updated in the google docs
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I have problem running in step 2 $ hack/verify- what should I do?
PTAL for my pr to this issue, thx
you can move kubernetes directory to and re-run this script.
I'd like to take Can we fix linter issues for generated files? Or we should skip packages with zz_generated?
There're 2 lint errors that affects 292/507 packages, meaning it's not possible to cleanup the entire file without solving it: default pattern of Kubectl is the usage of [cmd]Options for initial options structs and it raises: /apis/{custommetrics,externalmetrics} for example raises the underscore error, but It's probably not worth to break backward compatibility renaming these folder for lint error fix: Any advice on these?

/reopen hm, cannot reopen this issue, it was closed by mistake
You can't reopen an issue/PR unless you authored it or you are a collaborator. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I am sure this is all not fixed. Am I missing something? ./hack/verify- installing golint from vendor Congratulations! All Go source files have been linted.
you need to remove package from first.
/remove-sig testing SIG testing doesn't own code failing lints, we own the linter. :upsidedownface:
Hello, is this still a relevent issue ? I would like to work upon
I have fixed
Hi Folks, Some of the golint fixes, require us to change the exposed function's name, in simple words, "change API". I was not sure, if that is welcomed and what is the process.
Heres what Im getting after going over the following steps as mentioned by Say your GOPATH looks like this GOPATH=/Users/dims/go Your kubernetes directory should be ($) cd to and then run hack/verify- from that directory. I get the following output - installing golint Congratulations! All Go source files have been linted. Im not really sure what I am doing wrong. Any help is appreciated!
I think I understand. I misunderstood the steps around .golint_failures. The first step is to remove the pkg that we are targeting from this file and then run the My mistake was to assume that the packages in the file need to be removed after we are done cleaning the golint errors.
Can someone please review and let me know if I've done this right?
After a couple years of seeing changes spawned from this issue, I don't think this is a good use of contributors' or reviewers' time.

golint is very opinionated about: exported type and function names, leading to PRs attempting to change exported things people use downstream missing documentation on interface implementation functions, which leads to not-very-useful copy/paste of interface godoc the govet and staticcheck verify jobs are much more useful since they catch correctness issues, but I would recommend closing this issue and dropping the golint checks from CI golint doc itself discourages from using its output in an automated enforced way: from
i guess one can say gofmt is opinionated too, but something good about Golang is that it comes with an opinion and a set of standard tools. for API documentation at least we found inconsistencies on how most exported API items are documented: golint can help with that, but if there are no "pragmas" to avoid breaking changes and given we have codegen generating non-compliant names enabling golint on API packages is sort of blocked in Kubernetes. for non-API, non-library, non-public code, i think being inconsistent is mostly OK. no preference on my side. shrug
true, but code formatting has the benefits of mostly eliminating meaningless diffs in PRs due to code movement/whitespace/etc if there are specific packages (like API packages) we lack documentation on, adding checks specifically for that could make sense exactly, I don't think golint is the right tool for making sure we have API docs
golint itself doesn't have these and the go project is against this sort of comment AIUI, however golangci-lint does have them. I think there are plenty of other static analysis lints we should use instead, beyond just staticcheck and vet there is e.g.
btw, if we remove golint can we see e.g. all-caps constants or functions with underscores dodging review? perhaps we should add golangci-lint soon after:
Completely agree there are better places to spend review bandwidth. I was curious to see what the effort looked like. There have been 890 commits that touched over the past 3+ years !
/remove-help /remove-good-first-issue
Overall agree on the bandwidth. Also note that it was a good first issue with very low bar to get them in the repo. May be we need to find a similar low bar, but high value good first issue.

UT's come to mind, are there other things we curate for new contributors ?
Do we have an alternative way of enforcing that new exported code (not necessarily API fields) has at least some minimum documentation? As a reviewer, I ask contributors to add meaningful comments. But reality is that I sometimes stumble into old code that doesn't have documentation because we didn't have golint at some point. I agree that fix-golint PRs just add burden to reviewers. But, before removing the lint check, can we wait and see if it's enough to discourage people from sending more PRs like this? Additionally, we can encourage reviewers to just close PRs that only fix golint issues.
+1 for adding burden to reviewers. However, as a new contributor, I think this a very good opportunity for new contributors to study the code base and submit their first PR. In my opinion, I think this issue would be able to attract more new contributors, get familiar with the code base, and get familiar with the git workflow.
thanks everyone for getting us this far. We essentially gave up on the existing structure/system for golint: At some point we can look at adding golangci-lint, there's a WIP in progress in /close
Closing this issue. <detailsIn response to : thanks everyone for getting us this far. We essentially gave up on the existing structure/system for golint: Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: default: // This is a problem, report it and retry. logger.Error(err, "Creating gRPC stream for node resources failed") <del> // TODO (https://github.com/kubernetes/kubernetes/issues/123689): expontential backoff? </del> select { <del> case <-time.After(5 * time.Second): </del> <ins> case <-time.After(backOff.Get(backOffID)): backOff.Next(backOffID, time.Now()) </ins> case <-ctx.Done(): } }

has to retry after failures. This currently uses a fixed 5 second delay to avoid busy-looping. Exponential backoff might be better. /sig node /triage-accepted /priority important-longterm /lifecycle frozen /assign
/triage accepted
/assign


Question: "github.com/onsi/ginkgo/v2" "github.com/onsi/gomega" <ins> "github.com/onsi/gomega/gstruct" "github.com/onsi/gomega/types" </ins> "k8s.io/kubernetes/test/e2e/framework" <ins> e2emetrics "k8s.io/kubernetes/test/e2e/framework/metrics" </ins> e2enode "k8s.io/kubernetes/test/e2e/framework/node" e2epod "k8s.io/kubernetes/test/e2e/framework/pod" e2eskipper "k8s.io/kubernetes/test/e2e/framework/skipper"

In commit we podresources-related metrics, to make sure the feature was monitoring-friendly. Because of a implementation bug, the metrics have been , they are updated correctly, but they are never registered, so they don't show up when doing like the podresources related metrics should be reported in the endpoint in the kubelet the kubelet, even the simplistic environment of the tests is sufficient sure to enable the and feature gates (on by default in kube = 1.25 at least) the podresources endpoint issuing and calls. The actual output is not relevant the metrics: The bug was introduced in the original commit <details</details<detailsNot relevant. </details<detailsNot relevant </details<detailsNot relevant </details<detailsNot relevant </details<detailsNot relevant </details
/sig node /assign
/cc /triage-accepted
/triage accepted /cc


Question: "//vendor/golang.org/x/net/websocket:go_default_library", "//vendor/k8s.io/klog:go_default_library", "//vendor/k8s.io/utils/exec:go_default_library", <ins> "//vendor/k8s.io/utils/net:go_default_library", </ins> ], )

<!-- Please only use this template for submitting reports about failing tests in Kubernetes CI jobs --Which jobs are failing: https://k8s-(IPv6),%20master%20(dev) Which test(s) are failing: [sig-scheduling] SchedulerPredicates [Serial] validates that there is no conflict between pods with same hostPort but different hostIP and protocol [Conformance] Since when has it been failing: Testgrid link: https://k8s-(IPv6),%20master%20(dev) Reason for failure: The tests checks for 0.0.0.0 that is IPv4 only Anything else we need to know: Probably I should check with SIG-Testing how can we add an ipv6 job and update the KEP accordingly EDIT Probably I should check with SIG-Testing how can we add an ipv6 job as part of the k/k presubmits, and update the KEP accordingly
/assign /cc
/sig scheduling

<!-- Please only use this template for submitting enhancement requests --What would you like to be : Introduce a mechanism that allows specifying the IP protocol version used in the e2e testing. Why is this needed: This is a followup on , my understanding is that in order to discriminate between IPv4 or IPv6 we have to use the test labels and this adds complexity to the testing automation and doesn't allow to send this information to the context of the tests. Having a mechanism to specify the IP protocol will simplify the testing for IPv4 and IPv6 and will allow reusing most of the code.
/sig testing /cc /assign Does this make sense? If affirmative, what should be the best way to add this functionality? Checking the code I think that or can provide this, don't know if checking some of the cluster config parameters, ...
the testing-commons group (sig-testing) maintains the framework, but ideally the decision on how to proceed in terms of adding ipv6 detection should come from sig-network. possibly having this as agenda for a sig-network meeting is a good start, then discussing it in a testing-commons meeting. /sig network the linked issue is a big one, so i don't have all the context here. but IMO existing tests should just work for ipv6 without the requirement for a new "IPV6" tag in test names. dual-stack or protocol specific tests should have their descriptions contain the details properly (i.e. what is this test doing) we might want to add an utility function under: (there is a PR to move it BTW ) then such an utility might be useful only to tests that need to detect the protocol. one of the goals is to refactor the code in testcontext / framework, so adding this logic there is less preferred.
/area e2e-test-framework
I think that we can add an option to test_context to specify the cluster IP family the same way that exists one for the cloud provider this is simpler because when setting up the CI jobs you will know beforehand what IP family you want to test, it also will allow reusing all the e2e code for IPv6 and Dual Stack (cc: and by default will be IPv4, so it´s backward compatible.
Being able to override and specify exactly what you want the test to beleive about your cluster is nice, but ideally it should generally work without specifying more flags.

Surely we can detect this?
I was thinking during some time about this, detecting the IP family is very simple having IPv4 or IPv6 only clusters, but with the introduction of dual stack I´m afraid about the supportability and reliability of the autodetecting mechanisms in the long term. If you see the dual stack PRs there are still a lot of unknowns about the cluster behavior. My point is, let´s have something first stable that works and is simple, and once dual stack is implemented, we can replace the flag with a function in the framework . I guess that migrating from a flag variable to a function is straight forward, is just replacing by , what do you think?
in is as part of the context. do you mean adding as a function which will auto-detecting the ip family. less flags for the framework binary is preferred.
once we have this I will open a new issue to start cleaning up and , we no longer need to discriminate based on tags


Question: "//pkg/security/apparmor:go_default_library", "//pkg/security/podsecuritypolicy/seccomp:go_default_library", "//pkg/security/podsecuritypolicy/util:go_default_library", <ins> "//vendor:k8s.io/apimachinery/pkg/api/equality", </ins> "//vendor:k8s.io/apimachinery/pkg/apis/meta/v1", "//vendor:k8s.io/apimachinery/pkg/apis/meta/v1/validation", "//vendor:k8s.io/apimachinery/pkg/labels",

Kubernetes version (use ): Environment: Cloud provider or hardware configuration: DigitalOcean OS (e.g. from /etc/os-release):Kernel (e.g. ):Install tools:Others: What happened: when I do: it will run rolling update ONLY if there is a change. In case I do: Each time I run it - it will execute rolling update, which is misleading. What you expected to happen: I would like to have consistent behavior for Deployment and DaemonSet How to reproduce it (as minimally and precisely as possible): Create any DaemonSet with following:
Hmm, it shouldn't happen. TemplateGeneration is updated only when something changed. I will debug it.
I can reproduce this, and the DaemonSet is updated after every , even when nothing is changed. In , old/new DaemonSet template are seen as different with . I tried to change this part to use instead (needed to update code in validation as well), and the DaemonSet templates were then seen as the same, and thus the issue was gone. We used everywhere in the controllers. Is the right way to compare stuffs?
It's very strange. I tried to use pretty.Diff to print the differences but it returns empty slice. apiequality.Semantic.DeepEqual uses forked version of reflect.DeepEqual and it's little different from version in 1.7. will you prepare the patch?
The key of the reported DaemonSet config is its . Note that in the config file, (empty list); however, in the live DaemonSet object (created from the config file), (null list). That's why thinks they're different. If we use in the config file instead, won't change. From the field description, empty list and null list are different. Feel it's api machinery related
protobuf storage isn't able to store , so the object has that field set to when read back from storage. that is causing problems for that field in particular (where and have different meanings) see
cc for behavior of namespaces field
so it's not a DaemonSet bug
Well, still not sure whether Semantic.DeepEqual should be used or not. I'm still concerned with the behavior of the controller for other fields that can be populated with non-null empty lists by users, but serialize back out to null (or set omitempty)
deployments are using Maybe we should also switch?

If empty lists should be treated differently from null list, we should replace with in controller code.
that distinction is going away in all the fields I know of... I'm not super happy with Semantic.DeepEqual masking the difference between those two things, but that's probably the right thing for you to be using
Special meaning of [] was removed in I think a switch to Semantic.DeepEqual is still needed here
Filed


Question: informers.Extensions().V1beta1().DaemonSets(), nil, // CloudProvider cs, <del> time.Second, // Node monitor grace period </del> <ins> time.Hour,   // Node monitor grace period </ins> time.Second, // Node startup grace period time.Second, // Node monitor period time.Second, // Pod eviction timeout

While lookint into , I spent few minutes looking into this test: And there is a problem that is relatively easy to reproduce. The problem is that we are running its test cases one by one. But nothing in the meantime is ensuring that the current state that the test assumes is reflected in scheduler. In particular the problem that I observed couple time is that: we finished one test case we call cleanupNodes [but we don't ensure that scheduler actually cleaned up this node] we start the next test case we try to schedule a pod that has some toleration (but shouldn't fit on our node) by accident the toleration make it fit on to the previous node so scheduler schedules it on the node from previous test case
/assign


Question: *(newRS.Spec.Replicas) = newReplicasCount // Set new replica set's annotation <del> deploymentutil.SetNewReplicaSetAnnotations(d, &newRS, newRevision, false) </del> <ins> deploymentutil.SetNewReplicaSetAnnotations(d, &newRS, newRevision, false, maxRevHistoryLengthInChars) </ins> // Create the new ReplicaSet. If it already exists, then we need to check for possible // hash collisions. If there is any other error, we need to report it in the status of // the Deployment.

What happened: We created a Deployment and continuously updated an annotation on it periodically for a week. Eventually, we noticed errors in the controller-manager logs due to the annotation growing too large. Example annotations: controller-manager error: What you expected to happen: that do not change the revision number should not be to the revision history. revision history should be capped at a fixed number of elements, it should not be possible to have unbounded growth for Deployments that are frequently updated. How to reproduce it (as minimally and precisely as possible): a Deployment modify an annotation on it and note that the annotation gets the same number appended every time. Environment: Kubernetes version (use ):
/sig scalability Not 100% sure where this one fits, I couldn't determine which group owns the Deployment/ReplicaSet controllers, but this is a scalability problem
/sig apps
Agree with these, and if the limit is hit, the beginning of the history can be removed.
/assign


Question: } func containerLabels(c *cadvisorapi.ContainerInfo) map[string]string { <del> set := map[string]string{metrics.LabelID: c.Name} </del> <ins> // Prometheus requires that all metrics in the same family have the same labels, // so we arrange to supply blank strings for missing labels var name, image, podName, namespace, containerName string </ins> if len(c.Aliases) > 0 { <del> set[metrics.LabelName] = c.Aliases[0] } if image := c.Spec.Image; len(image) > 0 { set[metrics.LabelImage] = image </del> <ins> name = c.Aliases[0] </ins> } <ins> image = c.Spec.Image </ins> if v, ok := c.Spec.Labels[types.KubernetesPodNameLabel]; ok { <del> set["pod_name"] = v </del> <ins> podName = v </ins> } if v, ok := c.Spec.Labels[types.KubernetesPodNamespaceLabel]; ok { <del> set["namespace"] = v </del> <ins> namespace = v </ins> } if v, ok := c.Spec.Labels[types.KubernetesContainerNameLabel]; ok { <del> set["container_name"] = v </del> <ins> containerName = v } set := map[string]string{ metrics.LabelID:    c.Name, metrics.LabelName:  name, metrics.LabelImage: image, "pod_name":         podName, "namespace":        namespace, "container_name":   containerName, </ins> } return set }

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: Prometheus graphs with metrics coming from cAdvisor are full of holes. Graphs with metrics coming from Prometheus node exporter are fine. ! What you expected to happen: cAdvisor should return all metrics, all the time. How to reproduce it (as minimally and precisely as possible): Here the diff between the time it returned 81 containermemoryusage metrics and 14: Anything else we need to know?: Environment: Kubernetes version (use ): 1.7.3 Cloud provider or hardware configuration**: Azure OS (e.g. from /etc/os-release): 16.04.3 LTS (Xenial Xerus) Kernel (e.g. ): Linux k8s-master--0 4.4.0-89-generic -Ubuntu SMP Mon Jul 31 19:38:41 UTC 2017 x8664 x8664 x86_64 GNU/Linux Install tools: acs-engine Others:
There are no sig labels on this issue. Please by: a sig: e.g., to notify the contributor experience sig, OR the label manually: e.g., to apply the label Note: Method 1 will trigger an email to the group. You can find the group list and label list . The in the method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals
Reiterating the mentions to trigger a notification: <detailsIn response to : Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
could you please take a look? cc
Related:
We run a kubespray installation, currently at k8s 1.7.2, and we noticed the exact behavior from cAdvisor metrics. Kubespray runs kubelet in a container on each host, exposing port 4194. We found that direct repeated queries to this port will return an inconsistent number of metrics. Sometimes, a metric will be present. Sometimes, not. It appears to be completely random.

This causes prometheus to miss scrape values which, in turn, causes grafana do plot graphs with holes in them. ! For reference, I myself do run another unrelated instance of cAdvisor-v0.26.1 in a different Docker cluster, and it seems to work fine. This seems related to the metrics exposed by Kubernetes.
cc
When should we expect a release containing this fix?
8 Will definitely have it. It will land in 1.7.5 or 1.7.6, depending on how quickly the cherrypick makes it in. IIRC, 1.7.5 should be cut any day now.
Thank you for the quick reply . Would be great to see it in 1.7.5, monitoring is completely broken for us due to this bug
I didn't find anywhere the cherry-pick of this issue. Is it in progress? If not, what can we do to move it forward?


Question: if err := o.Validate(); err != nil { t.Errorf("Unexpected error: %v", err) } <del> // FIXME soltysh: // since we have defaulting to localhost:8080 in staging/src/k8s.io/client-go/tools/clientcmd/client_config.go#getDefaultServer // we need to ignore the localhost:8080 server, when above gets removed this should be dropped too if err := o.Run(); err != nil && !strings.Contains(err.Error(), "localhost:8080") { </del> <ins> if err := o.Run(); err != nil { </ins> t.Errorf("Cannot execute version command: %v", err) } if !strings.Contains(buf.String(), "Client Version") {

/sig cli run (or just ). observe failure: xref:
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Hi, I have filed to address this issue.
Thank you!


Question: } } <del> func waitForVolumeAttachedToNode( </del> <ins> func waitForVolumeAttachStateToNode( </ins> t *testing.T, volumeName v1.UniqueVolumeName, nodeName k8stypes.NodeName, <ins> expectedAttachState cache.AttachState, </ins> asw cache.ActualStateOfWorld) { err := retryWithExponentialBackOff( time.Duration(500*time.Millisecond), func() (bool, error) { attachState := asw.GetAttachState(volumeName, nodeName) <del> if attachState == cache.AttachStateAttached { </del> <ins> if attachState == expectedAttachState { </ins> return true, nil } <del> t.Logf( "Warning: Volume <%v> is not attached to node  <%v> yet. Will retry.", volumeName, nodeName) </del> <ins> t.Logf("Warning: expected attach state: %v, actual attach state: %v. Will retry.", expectedAttachState, attachState) </ins> return false, nil }, ) attachState := asw.GetAttachState(volumeName, nodeName) <del> if err != nil && attachState != cache.AttachStateAttached { t.Fatalf( "Volume <%v> is not attached to node  <%v>.", volumeName, nodeName) </del> <ins> if err != nil && attachState != expectedAttachState { t.Fatalf("Volume <%v> is not in expected attach state: %v, actual attach state: %v", volumeName, expectedAttachState, attachState) </ins> } <ins> t.Logf("Volume <%v> is attached to node <%v>: %v", volumeName, nodeName, expectedAttachState) </ins> } func waitForVolumeAddedToNode(

pull-kubernetes-unit reconciler TestRunOneVolumeDetachFailNodeWithReadWriteOnce unknown, maybe recently? No response /sig storage
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
The test is still flaking but it is almost rare. Please see the link below for the test history. The root cause is that the detach operation is not triggered or completed within 100ms. 1) not triggered: 2) triggered but not completed: It seems no better way to fix it unless we increase sleep time in the test. i.e. (100ms -200ms) I'm not sure if it is valuable to fix it. cc what do you think?
any flake should be fixed even if it's rare.
/assign


Question: return p, err } <ins> // WaitForRCToStabilize waits till the RC has a matching generation/replica count between spec and status. func WaitForRCToStabilize(c *client.Client, ns, name string, timeout time.Duration) error { options := api.ListOptions{FieldSelector: fields.Set{ "metadata.name":      name, "metadata.namespace": ns, }.AsSelector()} w, err := c.ReplicationControllers(ns).Watch(options) if err != nil { return err } _, err = watch.Until(timeout, w, func(event watch.Event) (bool, error) { switch event.Type { case watch.Deleted: return false, apierrs.NewNotFound(unversioned.GroupResource{Resource: "replicationcontrollers"}, "") } switch rc := event.Object.(type) { case *api.ReplicationController: if rc.Name == name && rc.Namespace == ns && rc.Generation <= rc.Status.ObservedGeneration && rc.Spec.Replicas == rc.Status.Replicas { return true, nil } Logf("Waiting for rc %s to stabilize, generation %v observed generation %v spec.replicas %d status.replicas %d", name, rc.Generation, rc.Status.ObservedGeneration, rc.Spec.Replicas, rc.Status.Replicas) } return false, nil }) return err } </ins> func WaitForPodToDisappear(c *client.Client, ns, podName string, label labels.Selector, interval, timeout time.Duration) error { return wait.PollImmediate(interval, timeout, func() (bool, error) { Logf("Waiting for pod %s to disappear", podName)

The desired behavior is: Create spec.Replica pods As pods keep coming down watch, update status.Replicas, so users have pseudo real-time feedback If during this time someone submits a new update to spec.Replicas that bumps up the generation, dont update status.Generation to match Update Generation when pending creates/deletes are observed Without the third check, we could confuse a client who quicky scales up/down into thinking the current is for their spec (because spec will match status, but we still have pending creates/deletes). Observed in: Where we see: Because kubectl rolling-update deleted the rc soon after creation.
See discussion in Un-assigning myself since immediate issue of test flake is solved, and we need to think through how we're going to expose generation.
Issues go stale after 30d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. Prevent issues from auto-closing with an comment. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten /remove-lifecycle stale
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-testing, kubernetes/test-infra and/or . /close


Question: nsKubeconfig.QPS *= 20 nsKubeconfig.Burst *= 100 namespaceKubeClient := clientset.NewForConfigOrDie(nsKubeconfig) <ins> return startModifiedNamespaceController(ctx, namespaceKubeClient, nsKubeconfig) } func startModifiedNamespaceController(ctx ControllerContext, namespaceKubeClient clientset.Interface, nsKubeconfig *restclient.Config) (http.Handler, bool, error) { </ins> dynamicClient, err := dynamic.NewForConfig(nsKubeconfig) if err != nil {

In we observed a crashlooping controller cause a cluster cold boot failure, as controller-manager wasn't able to stay up long enough to run the deployment that would have fixed the problem. We should audit everything that runs in kube-controller-manager and ensure that they all fail gracefully enough to prevent crashloops, or make them individually restart in that case, instead of the process. Longer term, we should consider classifying the controllers by run level or bootstrap order (), and splitting them into separate binaries or changing the start-up order in kcm to make sure this sort of thing doesn't happen in the future.
Same issue with metrics-server, using minikube. Also reproduced on our test cluster. Steps to reproduce: Set up localkube Install metrics-server Stop minikube: sudo minikube stop Start minikube: sudo -E minikube start --vm-driver=none Expected: Kubernetes is running Actual: Kubernetes is dead Logs: Workaround: To fix the system, we must delete the metrics-server: Now kubernetes will start normally. After that we can reinstall the metrics server, and it will be ok.
Related:
This will be a hard problem to fix permanently. In all likelihood, we should stop allowing Fatalf() in kubernetes/kubernetes, but as you can see we do this to ourselves right here...
/kind bug
I also think this bug would benefit from an e2e test where we misconfigure things in a variety of ways and verify that KCM still comes up and stays up: [ ] misconfigured apiservice [ ] broken service attached to that apiservice [ ] RBAC rules that disallow things. maybe everything! [ ] webhooks that disallow everything [ ] yr example here
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale


Question: ### Step Zero: Prerequisites <del> This example assumes that you have a basic understanding of kubernetes services and that you have forked the repository and [turned up a Kubernetes cluster](https://github.com/GoogleCloudPlatform/kubernetes#contents): This example requires a kubernetes cluster. </del> <ins> This example requires a kubernetes cluster.  See the [Getting Started guides](../../docs/getting-started-guides) for how to get started. </ins> <del> See the companion [Setup Kubernetes](https://github.com/GoogleCloudPlatform/kubernetes/blob/master/examples/guestbook/SETUP.md) for some quick notes on how to get started. *If* you are running from source, replace commands such as "kubectl" below with calls to cluster/kubectl.sh. </del> <ins> If you are running from source, replace commands such as `kubectl` below with calls to `cluster/kubectl.sh`. </ins> ### Step One: Fire up the redis master

Working through the , I was confused about how to get setup with an initial binary and cluster. I think the section should point to the , rather than the document. relies on , which is being deprecated, and just points to the getting-started guides anyway.
LGTM.


Question: <ins> /* Copyright 2021 The Kubernetes Authors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package validation import "testing" func TestMaskTrailingDash(t *testing.T) { testCases := []struct { beforeMasking        string expectedAfterMasking string description          string }{ { beforeMasking:        "", expectedAfterMasking: "", description:          "empty string", }, { beforeMasking:        "-", expectedAfterMasking: "-", description:          "only a single dash", }, { beforeMasking:        "-foo", expectedAfterMasking: "-foo", description:          "has leading dash", }, { beforeMasking:        "-foo-", expectedAfterMasking: "-foa", description:          "has both leading and trailing dashes", }, { beforeMasking:        "b-", expectedAfterMasking: "a", description:          "has trailing dash", }, { beforeMasking:        "ab", expectedAfterMasking: "ab", description:          "has neither leading nor trailing dashes", }, } for _, tc := range testCases { afterMasking := maskTrailingDash(tc.beforeMasking) if afterMasking != tc.expectedAfterMasking { t.Errorf("error in test case: %s. expected: %s, actual: %s", tc.description, tc.expectedAfterMasking, afterMasking) } } } </ins>

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --My controller had a bug where it was setting , and the client I was using () returned an internal server error when I tried to create it. In the API server logs, there was a panic at this line: apimachinery validation logic should handle this case gracefully, and the API server should return a helpful error to the client. Create a resource with the GenerateName field set to "-". This happened on Kind running locally & AKS, both v1.20.2 Kubernetes version (use ): v1.20.2 Cloud provider or hardware configuration: Kind/laptop, AKS OS (e.g: ): Darwin/LinuxKit (i.e. docker for macOS), AKS Install tools: Kind
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : /sig apimachinery Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig api-machinery
/assign
/assign /triage accepted


Question: return c.PersistentVolumeClaims(namespace).Get(name) } <del> // CachedNodeInfo implements NodeInfo type CachedNodeInfo struct { corelisters.NodeLister } // GetNodeInfo returns cached data for the node 'id'. func (c *CachedNodeInfo) GetNodeInfo(id string) (*v1.Node, error) { node, err := c.Get(id) if apierrors.IsNotFound(err) { return nil, err } if err != nil { return nil, fmt.Errorf("error retrieving node '%v' from cache: %v", id, err) } return node, nil } </del> // StorageClassInfo interface represents anything that can get a storage class object by class name. type StorageClassInfo interface { GetStorageClassInfo(className string) (*storagev1.StorageClass, error)

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: I recently saw an issue in which the scheduler hit this log line many times: After some digging, it turns out that PodAffinityChecker has a and . The points at the scheduler's cache instance, while the points at a light wrapper around a . In the scheduler, we use the cache to provide a globally consistent view of the cluster, even if that view is slightly out of date. Data read from the core may not have been processed by the scheduler cache yet. Letting data "skip ahead" like this can cause stale info to get persisted in the scheduler cache, and other subtle bugs. While the original problem is not likely caused by this issue, I think it needs to be cleaned up to avoid potential bugs down the line. What you expected to happen: Predicates should only read state from the scheduler's cache in order to maintain consistency. How to reproduce it (as minimally and precisely as possible): This was observed in a cluster with high rate of pod creation and node creation/deletion. Anything else we need to know?: Environment: Kubernetes version (use ): 1.12.5 Cloud provider or hardware configuration: GKE with preemptible nodes OS (e.g: ): Kernel (e.g. ): Install tools: Others:
/sig scheduling
We do have this kind of issues, and here is another one:
I'm not sure the urgency of this fix, but if its not super urgent, I would love to pick it up, if there's an eng more experienced with the scheduler who could give me some guidance :)
Hi I dug into the source code, and find out is used by the scheduler and kubelet. Maybe we could change the internal struct of , but I do not know why it appears in kubelet which seems irrelevant to the scheduler.
is just a wrapper of corelisters.NodeLister. kubelet can get node info by it.
I think the point is that the kubernetes scheduler was designed to support single instance at the beginning. Even with the addition of multiple scheduler support, the cache between each scheduler is isolated. So at this stage, we can accept this without a shared state solution.

After all, the node information is not actively modified by the kubernetes scheduler.
So is it necessary to ensure the consistency by using node lister instead of cache in the scheduler, or we do not need to fix this issue which seems like to caused by inconsistent cache.
We've recently hit this issue with the scheduler. It appears that our scheduler somehow gets out of sync with the nodes, and start scheduling pods on tainted nodes. The node's kubelet then (rightfully) refuse to admit the pod. Looking at the scheduler logs, they are flooded with the following : Similar scenario, this is a high churn cluster with thousands of pods scheduled per minute - and nodes being autoscaled. Can we try to get a fix ?
I'm glad to pick this up if we decide to fix this issue.
It would be awesome if you could fix this issue for us, we are currently seeing 100's of pods incorrectly being scheduled onto nodes with taints.
The code cited for logging "Node not found, %v" is now in :
/assign
Thanks for the interest on fixing this bug. Sorry I haven't had time to engage on this issue. Feel free to send PRs my way. I'm not 100% certain that this will fix the problem of pods getting scheduled to tainted nodes. How is the taint being applied to the node? Even if the scheduler is internally consistent, there could be a time between node creation and the taint being applied where we send pods to that node.


Question: accBody: "ok body", }, { <ins> handler:    headerCounterHandler, reqHeaders: http.Header{}, health:     probe.Success, accBody:    "4", }, { handler:    headerKeysNamesHandler, reqHeaders: http.Header{}, health:     probe.Success, accBody:    "AcceptnAccept-EncodingnConnectionnUser-Agent", }, { </ins> handler: headerEchoHandler, reqHeaders: http.Header{ "X-Muffins-Or-Cupcakes": {"muffins"},

<!-- Please only use this template for submitting enhancement requests --What would you like to be : I would like that besides User-agent we the Accept header as well with the "acept all" default. Just like curl does. Why is this needed: Because it was my expectation that the probe would work similarly to a curl but is not the case. I think more folks can hit this situation and waste some time to finally sort it out If this is not something you would like to add I would then like the docs to mention this specifically. _ PS: I would love to propose the improvement my self
/sig node network
/assign
Is the goal simply to avoid tripping over servers that look for that header? I don't think this presents any hazards - is that right? If so, it seems like a simple PR if you want to tackle it. /triage accepted /kind feature
cool! will look into it! thanks :)
Hello sorry but be bugging you, but I have opened the PR but I had 0 response since then. I think I have met all the requirements, so I was wondering (as this is my first PR in k8s) if you could help me a bit. Thanks anyway.


Question: if err != nil { return err } <del> if pod.Status.Phase != api.PodRunning { return fmt.Errorf("pod %s is not running and cannot be attached to; current phase is %s", p.PodName, pod.Status.Phase) </del> <ins> if pod.Status.Phase == api.PodSucceeded || pod.Status.Phase == api.PodFailed { return fmt.Errorf("cannot attach a container in a completed pod; current phase is %s", pod.Status.Phase) </ins> } p.Pod = pod // TODO: convert this to a clean "wait" behavior }

Related issue
is there a simple example that you used to recreate the problem?
I used exactly the example below: So Pod sflask-w-init gets into pending state "Init:0/1" forever. Then you can try to run and you can see that is not working while invoking corresponding command with docker works - it displays current date every second.
thanks for the detailed test, please see the PR :)
Duh! already has a PR


Question: return fakeRuntime, endpoint } <del> func createRemoteRuntimeService(endpoint string, t *testing.T) internalapi.RuntimeService { runtimeService, err := remote.NewRemoteRuntimeService(endpoint, 15*time.Second, oteltrace.NewNoopTracerProvider()) </del> <ins> func createRemoteRuntimeService(endpoint string, t *testing.T, tp oteltrace.TracerProvider) internalapi.RuntimeService { runtimeService, err := remote.NewRemoteRuntimeService(endpoint, 15*time.Second, tp) </ins> require.NoError(t, err) return runtimeService }

spans are not being parented to other spans - propagation of context seems broken. More context: spans should be parented to other spans enable kubelet tracing at v1.28 and hook up a tracing backend. No response <detailsv1.28 at least </details<detailsNot sure this matters. </details<details</details<details</details<details</details<details</details
/sig instrumentation /assign
/triage accepted
!
repro steps:
/assign Thanks for helping to look into this! Feel free to post the findings here you shared with me on slack
GitHub didn't allow me to assign the following users: fatsheep9146. Note that only with read permissions, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : /assign Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Thanks! I will post this ASAP
The broken issue is due to the pr , which try to propagate context brought by pod worker to syncPod function in order to let the syncPod can be interrupted in time by other asynchronous routines, for example eviction. But in the line we did not pass the context which contain the parent span and trace into function, so the trace is broken. So I want to fix this by passing the into , which is also desired in comment But passing the ctx into syncPod may have something other to do as described in comments, so I tried to fix as comments advised.
cc


Question: } func (s *CustomColumnsPrinter) PrintObj(obj runtime.Object, out io.Writer) error { <del> w := tabwriter.NewWriter(out, columnwidth, tabwidth, padding, padding_character, flags) </del> <ins> if w, found := out.(*tabwriter.Writer); !found { w = GetNewTabWriter(out) out = w defer w.Flush() } </ins> t := reflect.TypeOf(obj) if !s.NoHeaders && t != s.lastType {

Compare the following two outputs (note the bad alignment in the last line): The custom-columns string is exactly the same in both cases, but something messes up the column widths when using openapi print columns. Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: Columns printed when using are misaligned. What you expected to happen: Columns in all rows should be aligned. How to reproduce it (as minimally and precisely as possible): Deploy Service Catalog, patched with Run the above two commands Environment: Kubernetes version (use ): Client Version: version.Info{Major:"1", Minor:"9+", GitVersion:"v1.9.0-alpha.3", GitCommit:"", GitTreeState:"clean", BuildDate:"2017-11-15T23:06:17Z", GoVersion:"go1.9.1", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.0", GitCommit:"", GitTreeState:"dirty", BuildDate:"2017-10-17T15:09:55Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"} (same behavior with kubectl built from current master branch)
/sig api-machinery
/remove-sig api-machinery /sig cli PR is ready:
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale


Question: // kubelet.PortForwarder. type fakePortForwarder struct { lock sync.Mutex <ins> // stores data expected from the stream per port expected map[uint16]string </ins> // stores data received from the stream per port received map[uint16]string // data to be sent to the stream per port

I wanted to drop a note that I haven't forgotten about this issue. I have spent some time investigating and have yet to determine why some test runs result in the data not being forwarded correctly. I hope to resume digging into this soon, but I've got some planning tasks I need to complete first.
I have determined the root cause of this. In the test case, it: a port forwarding session, listening on a local port a TCP connection to the local port a goroutine copies data from the local port to the remote a goroutine copies data from the remote to the local port data to the TCP connection data from the TCP connection port forwarding, which results in the client streams being closed The problem is that the data written in step 3 is only to the local TCP port. There's no guarantee that the goroutine in 2.i copies the data to the remote before step 5 happens. In fact, if you reverse steps 3 () and 4 () so the write happens immediately before you close the stopChan, this test fails very quickly. I am thinking about how to fix this...
Not done yet:
Saw this again today:
- this was a different failure than this one. I've just opened for it and closing this one


Question: ) func GetDefaultServerImage() string { <del> return fmt.Sprintf("%s:%s", hyperkubeImageName, version.Get()) </del> <ins> return fmt.Sprintf("%s:%s", hyperkubeImageName, strings.Replace(version.Get().String(), "+", "_", 1)) </ins> } func Run() error {

/kind bug What happened: Federation set up using kubefed uses invalid reference format, breaking the Federation control plane. What you expected to happen: That it works. How to reproduce it (as minimally and precisely as possible): Try to set up a control plane using kubefed. Anything else we need to know?: Environment: Kubernetes version (use ): Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.0+", GitCommit:"", GitTreeState:"dirty", BuildDate:"2017-10-02T09:48:25Z", GoVersion:"go1.9", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"7+", GitVersion:"v1.7.6-gke.1", GitCommit:"", GitTreeState:"clean", BuildDate:"2017-09-27T21:21:34Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"} Cloud provider or hardware configuration**: GCP OS (e.g. from /etc/os-release): Ubuntu Xenial Kernel (e.g. ): Irrelevant Install tools: GKE
There are no sig labels on this issue. Please by: a sig: e.g., to notify the contributor experience sig, OR the label manually: e.g., to apply the label Note: Method 1 will trigger an email to the group. You can find the group list and label list . The in the method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals
Reiterating the mentions to trigger a notification: <detailsIn response to : /sig federation /cc Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
cc might be able to help.
/assign
Assigning to you from the build cop rotation.
the fix for this issue is identified and fixed in . But it will take a while to be available on stable releases. Meanwhile you can use flag to to pass specific hyperkube version.


Question: # source "${KUBE_ROOT}/hack/lib/test.sh" source "${KUBE_ROOT}/test/cmd/apply.sh" source "${KUBE_ROOT}/test/cmd/apps.sh" <ins> source "${KUBE_ROOT}/test/cmd/authentication.sh" </ins> source "${KUBE_ROOT}/test/cmd/authorization.sh" source "${KUBE_ROOT}/test/cmd/batch.sh" source "${KUBE_ROOT}/test/cmd/certificate.sh"

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: With the exec credentials plugin configured in my kubeconf, and a stale/expired token present in the same file issuing e.g. triggers the authentication flow as handled by the plugin - once the flow is returned to kubectl (by printing the ExecCredential object) the new token is stored to the kubeconfig but not immediately used to resolve the request, leading to: Immediately issuing again works as kubectl now seem to use the credentials (token) stored following the first request. What you expected to happen: That kubectl would read the new token as fetched by the exec credential plugin and use it already for the first command issued. How to reproduce it (as minimally and precisely as possible): the kubernetes API server with the OIDC authenticator enabled. kubectl to use an exec credential plugin printing the token received out of band. that the token was indeed written to the kubeconf file. that the first kubectl command failed. that the next identical command succeeds. Anything else we need to know?: This works as expected when there is no token previously stored in the kubeconf - the behavior is only triggered when there is a token stored but it expired, or for whatever other reason not accepted by the API server. Environment: Kubernetes version (use ): 1.14, also tried with 1.17 Cloud provider or hardware configuration: OS (e.g: ): Mac OS for kubectl client Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig auth /sig cli
could you provide more details around how is configured. For example, it is unclear to me if you are using the built in oidc plugin or a custom exec plugin. I think you are using a custom exec plugin but its also trying to write tokens to the file (which it should not do)? /triage needs-information
Thanks for looking into it . You are indeed correct in your assumption that I'm using a which stores the token in the after retrieval. I'd love to learn why this should not be done, but the problem remains also without persisting the token after retrieval.

Basically the steps to reproduce could be reduced to these: containing an expired token. same configuring an exec plugin which retrieves a valid token out of band and echoes it. Expected behavior: The valid token retrieved is used for the subsequent authentication. Possibly expected behavior: The exec plugin flow not started (or exited with a warning) due to an expired token being present in the . Current behavior seems to acknowledge the invalid token and hence tries to retrieve a new, valid one - which it then discards as soon as it arrives.
Seems we can reduce it even further by removing the token altogether from the file and instead provide it directly. So the only step needed to reproduce is essentially this: configuring an exec plugin which retrieves a valid token out of band and echoes it as part of an . Now any will trigger the exec plugin to retrieve a valid token, but will still not use it for subsequent authentication.
This one is caused by config merging with clientcmd overrides: and exec.Auth not being in this switch:
Interesting, thanks Will add it to the switch along with some tests and if everything works out well I'll submit a PR next week.
Having spent some time on this in the debugger, it seems somewhat more intricate than I first thought. Turns out the exec plugin is executed whether the presented is invalid or valid - the exec action is actually called before the first call is even dispatched. This due to the exec credential plugin possibly being needed to setup the client certificate requested for the transport. Once the exec plugin's is called it assign the authenticators function to : ..which is then called when setting up the transport, thus triggering the exec plugin regardless of a token being present or not: Since the exec plugin has no way of knowing whether a token or a client certificate will be returned by the exec action, this kind of makes sense. It however defeats letting any present to override the exec plugin, which clearly seems to be the intent: Possible options here that I can think of: (and fix) behavior of letting override the exec plugin. This looks doable by adding a check for token auth early in the 's method body: But might be there are consequences to this I'm not aware of. the behavior of letting override the exec plugin being called.

third, more complex option, would be to keep the token override but dispatch to the exec credential plugin should the response from the first call with the token come back with a 401. This was what I originally believed was the intent as my first tests always included an invalid token. While this behavior seems nice I'm more and more thinking it may violate the principle of least surprise. Either you present a token and use that for auth or you let the exec plugin fetch the credentials for you. What do you guys think?
As a colleague just made me aware of, certain tools like kubetail seem to have kubectl call the exec plugin so many times that eventually the threshold of 1000 is reached and warn logs (exec:201) start spewing - even though no certificate (and hence no rotation) should be involved. This has me more convinced that the first option is the right one, skipping the client TLS ceremonies should a token be present for authentication.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale


Question: ) const ( <del> // MaxUint defines the max unsigned int value. MaxUint = ^uint(0) // MaxInt defines the max signed int value. MaxInt = int(MaxUint >> 1) </del> // MaxTotalPriority defines the max total priority value. MaxTotalPriority = int64(math.MaxInt64) // MaxPriority defines the max priority value. MaxPriority = 10 // MaxWeight defines the max weight value. <del> MaxWeight = int64(math.MaxInt64 / MaxPriority) </del> <ins> MaxWeight = MaxTotalPriority / MaxPriority </ins> // DefaultPercentageOfNodesToScore defines the percentage of nodes of all nodes // that once found feasible, the scheduler stops looking for more nodes. DefaultPercentageOfNodesToScore = 50

MaxInt const is exported in scheduler api although it is not needed to be exported. It is only used by . discussed that the common practice is to have API constants in /sig scheduling
cc
The problem is that it is part of a v1 API :(
Ok, it turns out they are not actually considered part of the API, and so we can remove them:
Should I open a PR to address removing those const values from ?
Yes, we can remove those in a followup PR to the one handling sized values.
Yes, this was the plan. /assign
remember to remove as well :)