Question: // Individual tests follow: // <del> // Create an nfs PV, a claim that matches the PV, a pod that contains the // claim. Verify that the PV and PVC bind correctly and that the pod can // write to the nfs volume. It("should create a PersistentVolume, Claim, and Pod that will test write access of the volume [Flaky]", func() { </del> <ins> // Create an nfs PV, then a claim that matches the PV, and a pod that // contains the claim. Verify that the PV and PVC bind correctly, and // that the pod can write to the nfs volume. It("should create a non-pre-bound PV and PVC: test write access [Flaky]", func() { </ins> <del> pv, pvc, err = createPVandPVC(c, serverIP, ns, true /*pv first*/, false) </del> <ins> pv, pvc, err = createPVPVC(c, serverIP, ns, false) </ins> if err != nil { framework.Failf("%v", err) } <del> pv, pvc, err = waitAndValidatePVandPVC(c, ns, pv, pvc) </del> <ins> // validate PV-PVC, create and verify writer pod, delete PVC // and PV pv, pvc, err = completeTest(f, c, ns, pv, pvc) </ins> if err != nil { framework.Failf("%v", err) } <ins> }) </ins> <del> By("Checking pod has write access to PersistentVolume") if err = createWaitAndDeletePod(f, c, ns, pvc.Name); err != nil { framework.Failf("%v", err) } </del> <ins> // Create a claim first, then a nfs PV that matches the claim, and a // pod that contains the claim. Verify that the PV and PVC bind // correctly, and that the pod can write to the nfs volume. It("create a PVC and non-pre-bound PV: test write access [Flaky]", func() { </ins> <del> // Delete the PVC before deleting PV, wait for PV to be Available pv, pvc, err = deletePVCandValidatePV(c, ns, pvc, pv) </del> <ins> pv, pvc, err = createPVCPV(c, serverIP, ns, false) </ins> if err != nil { framework.Failf("%v", err) } <del> // Last cleanup step is to delete the pv pv, err = deletePersistentVolume(c, pv) </del> <ins> // validate PV-PVC, create and verify writer pod, delete PVC // and PV pv, pvc, err = completeTest(f, c, ns, pv, pvc) </ins> if err != nil { framework.Failf("%v", err) } }) <del> // Create an nfs PV that is *pre-bound* to a claim. Create a pod that // contains the claim. Verify that the PV and PVC bind correctly and that // the pod can write to the nfs volume. It("should create a pre-bound PersistentVolume, Claim, and Pod that will test write access of the volume [Flaky]", func() { </del> <ins> // Create a claim first, then a pre-bound nfs PV that matches the claim, // and a pod that contains the claim. Verify that the PV and PVC bind // correctly, and that the pod can write to the nfs volume. It("create a PVC and a pre-bound PV: test write access [Flaky]", func() { </ins> <del> pv, pvc, err = createPVandPVC(c, serverIP, ns, false /*pvc first*/, true /*prebind*/) </del> <ins> pv, pvc, err = createPVCPV(c, serverIP, ns, true) </ins> if err != nil { framework.Failf("%v", err) } <del> pv, pvc, err = waitAndValidatePVandPVC(c, ns, pv, pvc) </del> <ins> // validate PV-PVC, create and verify writer pod, delete PVC // and PV pv, pvc, err = completeTest(f, c, ns, pv, pvc) </ins> if err != nil { framework.Failf("%v", err) } <ins> }) </ins> <del> // checkPod writes to the nfs volume By("Checking pod has write access to pre-bound PersistentVolume") // Instantiate pod, wait for it to exit, then delete it if err = createWaitAndDeletePod(f, c, ns, pvc.Name); err != nil { framework.Failf("%v", err) } </del> <ins> // Create a nfs PV first, then a pre-bound PVC that matches the PV, // and a pod that contains the claim. Verify that the PV and PVC bind // correctly, and that the pod can write to the nfs volume. It("create a PV and a pre-bound PVC: test write access [Flaky]", func() { </ins> <del> // Delete the PVC before deleting PV, wait for PV to be Available pv, pvc, err = deletePVCandValidatePV(c, ns, pvc, pv) </del> <ins> pv, pvc, err = createPVPVC(c, serverIP, ns, true) </ins> if err != nil { framework.Failf("%v", err) } <del> // Last cleanup step is to delete the pv pv, err = deletePersistentVolume(c, pv) </del> <ins> // validate PV-PVC, create and verify writer pod, delete PVC // and PV pv, pvc, err = completeTest(f, c, ns, pv, pvc) </ins> if err != nil { framework.Failf("%v", err) } }) }) <del> // Returns a PV definition based on the nfs server IP. If the PVC is not nil then the // PV is defined with a ClaimRef which includes the PVC's namespace. If the PVC is // nil then the PV is not defined with a ClaimRef. // **Note: the passed-in claim does not have a name until it is created (instantiated) // and thus the PV's ClaimRef cannot be completely filled-in in this func. Therefore, // the ClaimRef's name is added later in createPVandPVC(). </del> <ins> // Returns a PV definition based on the nfs server IP. If the PVC is not nil // then the PV is defined with a ClaimRef which includes the PVC's namespace. // If the PVC is nil then the PV is not defined with a ClaimRef. // Note: the passed-in claim does not have a name until it is created //   (instantiated) and thus the PV's ClaimRef cannot be completely filled-in in //   this func. Therefore, the ClaimRef's name is added later in //   createPVCPV. </ins> func makePersistentVolume(serverIP string, pvc *api.PersistentVolumeClaim) *api.PersistentVolume { // Specs are expected to match this test's PersistentVolumeClaim var claimRef *api.ObjectReference <del> claimRef = nil </del> if pvc != nil { claimRef = &api.ObjectReference{ Name:      pvc.Name,

Context: NodeSwap feature gate was recently . Currently, as per , swap is only supported with cgroup v2 - cgroup v1 is no longer supported. For this reason, we decided to fail kubelet if NodeSwap feature gate is enabled and the node runs with cgroup v1. This would help prevent any surprises for the user by failing as soon as possible. This effort is being implemented in this PR: The problem: Currently, swap jobs are running with cgroup v1. Swap tests were in a way that they first check on which environment they're being run (e.g. which cgroup version? is NodeSwap FG enabled? Which swapBehavior is defined? etc) and then expect the right behavior to take place. With v1, the tests expect the containers to be defined without swap access. While the tests pass, they are much less valuable, since all they check is that swap is disabled for v1 workloads. The tests would be much more valuable if they would run with v2, therefore actually ensure that proper swap limitations are configured. In addition, is dependent on these changes, since as of now the jobs immediately fail with this PR since kubelet fails on an environment with cgroup v1 and swap enabled. What needs to change: [x] CRI-O swap related jobs should be defined to run with cgroup v2 (e.g. ) [x] containerd swap related jobs should be defined to run with cgroup v2 (e.g. ). Future work: In the future, we might consider introducing more jobs to check swap. Ideally, both and would be tested, perhaps by periodic jobs.
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig node
/sig testing
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: "//staging/src/k8s.io/apiserver/pkg/admission:go_default_library", "//staging/src/k8s.io/apiserver/pkg/admission/initializer:go_default_library", "//staging/src/k8s.io/client-go/kubernetes:go_default_library", <ins> "//vendor/k8s.io/klog:go_default_library", </ins> ], )

Look, here is the thing, this is only useful IF you are only exposing the kube api to the end user. AND even then if you do not prevent mounting the host into containers via a PodSecurityPolicy then you could just exec in by combining the file descriptors.... Like this window dressing of a feature is kinda un-useful and people are enabling it under circumstances where it is just plain stupid.... there is literally no point to this feature if you do not also have a pod security policy in place preventing people from mounting shit in to literally exec. Or if users have access to the nodes in the first place it makes literally no sense.
/sig auth
/sig node cc
Agreed. I would much prefer to see this done with a combination of RBAC, namespaces, and PodSecurityPolicy. For example: Allow privileged pods in the "privileged" namespace (PodSecurityPolicy), do not allow exec in that namespace (RBAC) Forbid privileged pods in the "unprivileged" namespace (PSP), allow exec in that namespace PSP actually covers enough ground to properly prevent escalating exec (privileged, capabilities, host paths, etc.).
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
/remove-lifecycle rotten
opened to mark as deprecated
Thanks


Question: func (j *TestJig) checkClusterIPServiceReachability(svc *v1.Service, pod *v1.Pod) error { clusterIP := svc.Spec.ClusterIP servicePorts := svc.Spec.Ports <del> externalIPs := svc.Spec.ExternalIPs </del> err := j.waitForAvailableEndpoint(ServiceEndpointsTimeout) if err != nil {

Which jobs are failing: Which test(s) are failing: - Since when has it been failing: Testgrid link: Reason for failure: Nov 4 11:59:41.385: Unexpected error: <*errors.errorString | 0xc00298c3d0: { s: "service is not reachable within 2m0s timeout on endpoint 203.0.113.250:80 over TCP protocol", } service is not reachable within 2m0s timeout on endpoint 203.0.113.250:80 over TCP protocol occurred Anything else we need to know: /priority important-soon /sig network /cc
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Apparently this is related to /cc
if you're busy and want me to take a look tomorrow you can assign that to me and I can deep dive into what's missing/happening :) Happy to help and to understand better the flakiness life
so, the test adds a "fake" external IP that should be nated to the service endpoint It went through the CI, you can check the test here passing more details in the issue I opened explaining the test
ok, this is interesting the test is failing when using container runtime DOCKER https://k8s- the same periodic job that uses container runtime CONTAINERD (same as the PR) is working https://k8s- We need to understand what is different regarding the network in GCE jbos, when using docker as runtime, kubenet??? cc:
one diff between the working jobs with containerd and the failing ones with docker is that the failing ones use this part of the code for reference, kube-proxy adds the NAT rules here for external IPs here
IIRC you are using kubenet for azure jobs, are you using docker as runtime, can you please check if this tests fails there too?
The problem happens on environment that are using the bridge plugin, seems I got wrong the feature will revert until we have the right test.

We use kubenet only for IPv6 and dualstack jobs. Rest of the tests use Azure CNI. Default is docker runtime for all jobs.


Question: } // Run tests in archive against host <del> func testHost(host, archive string) *TestResult { output, err := e2e_node.RunRemote(archive, host, *cleanup) </del> <ins> func testHost(host, archive string, deleteFiles bool) *TestResult { output, err := e2e_node.RunRemote(archive, host, deleteFiles) </ins> return &TestResult{ output: output, err:    err,

Looks like it maybe the ssh connect is getting killed
The relevant log snippet:
And another one, not just coreos.


Question: - [CPU and Memory manager are not working correctly for Guaranteed Pods with multiple containers](#cpu-and-memory-manager-are-not-working-correctly-for-guaranteed-pods-with-multiple-containers) - [`CSIMigrationvSphere` feature gate has not migrated to new CRD APIs](#csimigrationvsphere-feature-gate-has-not-migrated-to-new-crd-apis) - [Workloads that saturate nodes with pods may see pods that fail due to node admission](#workloads-that-saturate-nodes-with-pods-may-see-pods-that-fail-due-to-node-admission) <del> - [Etcd v3.5.[0-2] data corruption](#etcd-v350-2-data-corruption) </del> <ins> - [Etcd v3.5.[0-2] data corruption](#etcd-v350-2-data-corruption) </ins> - [Urgent Upgrade Notes](#urgent-upgrade-notes) - [(No, really, you MUST read this before you upgrade)](#no-really-you-must-read-this-before-you-upgrade) - [Changes by Kind](#changes-by-kind-15)

The section of the Changelog for 1.23.14 (and sibling bugfix releases) says that 1.23.13 both has the vuln and also has the fix. The Fix version should be 1.23.14 in the above and sibling changelogs. Load the link above. No response N/A N/A N/A N/A N/A N/A
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig docs
cc
For v1.23.14, the title is . This seems to be valid. means
You're looking at the wrong text. Follow the link and see .13 in both lists: kube-apiserver v1.25.0 - v1.25.3 kube-apiserver v1.24.0 - v1.24.7 kube-apiserver v1.23.0 - v1.23.13 kube-apiserver v1.22.0 - v1.22.15 kube-apiserver <= v1.21.? kube-apiserver v1.25.4 kube-apiserver v1.24.8 kube-apiserver v1.23.13 kube-apiserver v1.22.16
Got it. Thanks for pointing out the location.
I opened to fix it.
/reopen Can you please cherry-pick to the release branches (1.25-1.22)?
Reopened this issue. <detailsIn response to : /reopen Can you please cherry-pick to the release branches (1.25-1.22)? Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Of course. Done
Cherry-picks are done, thanks a lot /close
Closing this issue. <detailsIn response to : Cherry-picks are done, thanks a lot /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: // annotation stored in the given endpoints object or the "zero" time if the annotation wasn't set // or was set incorrectly. func getLastChangeTriggerTime(endpoints *v1.Endpoints) time.Time { <ins> if _, ok := endpoints.Annotations[v1.EndpointsLastChangeTriggerTime]; !ok { // It's possible that the Endpoints object won't have the EndpointsLastChangeTriggerTime // annotation set. In that case return the 'zero value', which is ignored in the upstream code. return time.Time{} } </ins> val, err := time.Parse(time.RFC3339Nano, endpoints.Annotations[v1.EndpointsLastChangeTriggerTime]) if err != nil { klog.Warningf("Error while parsing EndpointsLastChangeTriggerTimeAnnotation: '%s'. Error is %v",

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!--What happened: The kube- is full of the following log lines: W0213 00:00:40. 1 ] Error while parsing EndpointsLastChangeTriggerTimeAnnotation: ''. Error is parsing time "" as "2006-01-02T15:04:05.999999999Z07:00": cannot parse "" as "2006" This is caused by the recent merge of What you expected to happen: We shouldn't log warning when EndpointsLastChangeTriggerTimeAnnotation is not set How to reproduce it (as minimally and precisely as possible): Set-up cluster, create some endpoints objects, go to the any node and open the kube-
This was caused by so assigning sig/scalability /sig scalability


Question: // TODO: allow selection from command line. (e.g., just check links in the examples directory.) allMunges = []munge{ {"table-of-contents", updateTOC}, <ins> {"unversioned-warning", updateUnversionedWarning}, </ins> {"check-links", checkLinks}, {"blank-lines-surround-preformatted", checkPreformatted}, {"header-lines", checkHeaderLines}, <del> {"unversioned-warning", updateUnversionedWarning}, </del> {"analytics", checkAnalytics}, {"kubectl-dash-f", checkKubectlFileTargets}, }

If you try mimicking the release flow for a release on right now, it's busted: If I generate a dummy commit out of that failure, then run to see what gets generated, it's attempting to actually add the unversioned warnings back in: I was trying to write up a PR to enact something for , but this blocks the release piece for that. Which is fine, we have some time.
cc
Running gendocs should add the unversioned warnings, unless the doc has <!-- TAG IS_VERSIONED --. Do you know if mark-new-version that tag before failing?
Ah, I think I see what might be happening. This is the post- version of Notice the newline fun.
Sorry which newline are you talking about? :)
My PR fixes that
Ah, yes, cool. It would help if you flagged functional changes along more loudly. :)
Yea, sorry about that. Have updated the PR description to highlight each fix.


Question: - -k - --cache-size=1000 - --no-negcache <ins> - --dns-loop-detect </ins> - --log-facility=- - --server=/{{dns_domain}}/127.0.0.1#10053 - --server=/in-addr.arpa/127.0.0.1#10053

Is this a BUG REPORT or FEATURE REQUEST?: What happened: If hosts points to a local DNS server, or if it contains the cluster's DNS server (to enable looking up services from host), it'll cause DNS loop which can effectively disable kube-dns or make it very slow. What you expected to happen: At least, kube-dns should detect localhost or cluster DNS server entries in and not send queries to them. How to reproduce it (as minimally and precisely as possible): Use a system with local cashing DNS server, or add cluster's DNS server address to hosts Anything else we need to know?: According to documentation, adding should make dnsmasq detect that and disable the upstream DNS server which cause the loop. It should prevent DNS from becoming unresponsive in the first case (a better solution is to extract real upstream servers, which probably cannot be done for every configuration); and for the second case it should skip sending queries to itself and send to real upstream servers.
/sig network
Could you describe the use-case more? If the host's contains cluster DNS server and some other upstream server - like 8.8.8.8, will services lookups work reliably from the host? If the services lookup goes to the 8.8.8.8 server, it will return NXDOMAIN. If it hits the clusterDNS server, only then it would work, correct?
Also ref If the first server returns NXDOMAIN, the second server will not be tried. How do you make both cluster DNS name and non-cluster DNS name resolvable on the host?
If cluster DNS server comes before a real DNS server, it works fine. Also note that what man page says is the glibc (or libc) behavior, not other users (e.g. dnsmasq sends its queries to any one of them, but can be configured to query them in strict order ( flag), or to query all of them (). Anyway, consider when cluster DNS server comes before the real server: cluster DNS queries, the cluster DNS server responds with the IP of the cluster or pod. external names, the query is sent to cluster DNS server. It doesn't find the name, so it tries to send it to upstream DNS servers; which in my example are cluster DNS server itself and an external DNS server.

If dnsmasq sends it to external DNS server, it'll receive the answer and can serve the query. If it sends it to itself, there is a loop and the same process happens again until dnsmasq decides to query the other server (which might not happen!). WIth flag, when dnsmasq sends the query to itself, it'll recognize the loop and will remove its own address from the list of upstream DNS servers, so it'll always send external names to the external DNS server from now on.
Thanks for the explanation. I believe it might work with explicit configuration. Though adding a flag to upstream for fulfilling a not quite common use case seems a bit odd to me. Have you thought of configuring upstream DNS servers via kube-dns config [1] such that it will not send request to itself at all? [1] -
Note that the flag has two use cases, the other one is prevent kube-dns breakage when upstream DNS server is a local one (e.g. If you deploy to Ubuntu, which has local caching DNS server by default (e.g. in 18.04, it is systemd-resolved and contains 127.0.0.53)). Actually I guess there are a number of DNS related issues which might be solved by this flag too. So, while this flag will not make kubernetes magically find real upstream dns servers in such cases, it prevents kube-dns crash; which has caused frustration and some time for me to discover. So, either kubelet should error out and exit if the given contains a local address, or such flag should be used by kube-dns to prevent DNS loops. And, IIRC, the only reason that cluster's DNS server is not to host, at least by kubespray, is mentioned to be host specific in the docs. So, my use case seems to be an ideal one and it would be the default one if it was easy to setup for different platforms. About using kube-dns config, no. We're going to provide automatic kubernetes deployment for many environments, and so we don't know the upstream DNS servers upfront, and we don't want to hard code it. We want it to be the DNS servers used in the target network. So, configuring a specific upstream DNS server is a no-go.


Question: kube::golang::setup_env API_REFERENCE_DOCS_ROOT="${KUBE_ROOT}/docs/api-reference" <ins> # Use REPO_DIR if provided so we can set it to the host-resolvable path # to the repo root if we are running this script from a container with # docker mounted in as a volume. # We pass the host output dir to update-api-reference-docs.sh, but use # the regular one to compute diff (they will be the same if running this # test on the host, potentially different if running in a container). REPO_DIR=${REPO_DIR:-"${KUBE_ROOT}"} HOST_OUTPUT_DIR="${REPO_DIR}/_tmp/api-reference" </ins> TMP_OUTPUT_DIR="${KUBE_ROOT}/_tmp/api-reference" TMP_ROOT="${KUBE_ROOT}/_tmp" # Generate API reference docs in tmp. <del> "./hack/update-api-reference-docs.sh" "${TMP_OUTPUT_DIR}" </del> <ins> "./hack/update-api-reference-docs.sh" "${HOST_OUTPUT_DIR}" </ins> echo "diffing ${API_REFERENCE_DOCS_ROOT} against freshly generated docs" ret=0

In the pull request Jenkins, we're running up to 8 instances of verification checks, build, unit tests, integration tests, and e2e tests. Unfortunately, it seems like several of these components were not designed to run simultaneous to other tests on the same machine. In particular, several unit tests seem to write to static file names in /tmp (or equivalent), which is a fairly easy fix. Some of the verification checks and the integration test start up etcd, however, and the etcd helper checks that only a single etcd binary is running on the host (and uses a static port for it). For example: We might be able to fix the etcd issue separately, or we can just figure out how to containerize all of these tests so they can't possibly interfere with each other. Note that we're ignoring the exit status of the verification checks and unit/integration tests right now, so these etcd failures are effectively being ignored (though it is causing us to miss test coverage).
What percentage of tests are failing due to being non-hermetic like this?
I'm not sure about unit tests (I've anecdotally seen just a few fail), but the verification checks are failing fairly regularly, and the integration test is also similarly frequently not running due to this.
To be clear, I'm looking for real numbers, to decide whether it's feasible to temporarily disable a small number of tests until we can hermetify them. Otherwise we need to turn off all unit, integration and verification tests in Jenkins to avoid irritating invalid regression signals. We need to do one or the other this week.
I'll try to dig through the logs, figure out which ones have failed recently, disable them in Jenkins, and see whether the flakes abate.
I just turned off all unit, integration and verification checks in Jenkins, as a temporary measure to unstall the merge-bot.
As a point of information, I just found approximately 20 tests in which use a static tmp path with semi-random filenames which are highly likely to collide. We could keep searching for unit tests that are non-hermetic, but it'd probably be a better investment of time to fix the test infrastructure so test authors don't need to care (and so we don't need to go look at and fix ~2000 test cases).

Another few metrics, using the string "/tmp" as a likely anti-pattern: We'd need to update each of these 23 files to get them to use or if they aren't already.
Created to track cleaning up tempdir usage in unit tests.
cc


Question: <ins> /* Copyright 2017 The Kubernetes Authors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package group import ( "net/http" "k8s.io/apiserver/pkg/authentication/authenticator" "k8s.io/apiserver/pkg/authentication/user" ) // AuthenticatedGroupAdder adds system:authenticated group when appropriate type AuthenticatedGroupAdder struct { // Authenticator is delegated to make the authentication decision Authenticator authenticator.Request } // NewAuthenticatedGroupAdder wraps a request authenticator, and adds the system:authenticated group when appropriate. // Authentication must succeed, the user must not be system:anonymous, the groups system:authenticated or system:unauthenticated must // not be present func NewAuthenticatedGroupAdder(auth authenticator.Request) authenticator.Request { return &AuthenticatedGroupAdder{auth} } func (g *AuthenticatedGroupAdder) AuthenticateRequest(req *http.Request) (user.Info, bool, error) { u, ok, err := g.Authenticator.AuthenticateRequest(req) if err != nil || !ok { return nil, ok, err } if u.GetName() == user.Anonymous { return u, true, nil } for _, group := range u.GetGroups() { if group == user.AllAuthenticated || group == user.AllUnauthenticated { return u, true, nil } } return &user.DefaultInfo{ Name:   u.GetName(), UID:    u.GetUID(), Groups: append(u.GetGroups(), user.AllAuthenticated), Extra:  u.GetExtra(), }, true, nil } </ins>

When an auth proxy tries to proxy an anonymous request, the user.Info ends up with the group system:authenticated, which is wrong.


Question: # We don't join cockroachdb in order to avoid a node attempting # to join itself, which currently doesn't work # (https://github.com/cockroachdb/cockroach/issues/9625). <del> CRARGS+=("--join" "cockroachdb-public") </del> <ins> CRARGS+=("--join" "cockroachdb-0.cockroachdb,cockroachdb-1.cockroachdb,cockroachdb-2.cockroachdb") </ins> fi exec /cockroach/cockroach ${CRARGS[*]} # No pre-stop hook is required, a SIGTERM plus some time is all that's

Seems to be broken for at least the past two weeks. /sig apps /area test
can you take a look please?
Absolutely, I'll take a look today. Sorry guys!
thanks!


Question: <del> /* Copyright 2014 The Kubernetes Authors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package common import ( "fmt" "os" "path" "time" "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/util/uuid" "k8s.io/kubernetes/test/e2e/framework" imageutils "k8s.io/kubernetes/test/utils/image" . "github.com/onsi/ginkgo" . "github.com/onsi/gomega" ) var _ = Describe("[sig-storage] Projected", func() { // Part 1/3 - Secrets f := framework.NewDefaultFramework("projected") /* Release : v1.9 Testname: Projected Volume, Secrets, volume mode default Description: A Pod is created with a projected volume source ‘secret’ to store a secret with a specified key with default permission mode. Pod MUST be able to read the content of the key successfully and the mode MUST be -rw-r--r-- by default. */ framework.ConformanceIt("should be consumable from pods in volume [NodeConformance]", func() { doProjectedSecretE2EWithoutMapping(f, nil /* default mode */, "projected-secret-test-"+string(uuid.NewUUID()), nil, nil) }) /* Release : v1.9 Testname: Projected Volume, Secrets, volume mode 0400 Description: A Pod is created with a projected volume source ‘secret’ to store a secret with a specified key with permission mode set to 0x400 on the Pod. Pod MUST be able to read the content of the key successfully and the mode MUST be -r—-—————. */ framework.ConformanceIt("should be consumable from pods in volume with defaultMode set [NodeConformance]", func() { defaultMode := int32(0400) doProjectedSecretE2EWithoutMapping(f, &defaultMode, "projected-secret-test-"+string(uuid.NewUUID()), nil, nil) }) /* Release : v1.9 Testname: Project Volume, Secrets, non-root, custom fsGroup Description: A Pod is created with a projected volume source ‘secret’ to store a secret with a specified key. The volume has permission mode set to 0440, fsgroup set to 1001 and user set to non-root uid of 1000. Pod MUST be able to read the content of the key successfully and the mode MUST be -r—-r————-. */ framework.ConformanceIt("should be consumable from pods in volume as non-root with defaultMode and fsGroup set [NodeConformance]", func() { defaultMode := int32(0440) /* setting fsGroup sets mode to at least 440 */ fsGroup := int64(1001) uid := int64(1000) doProjectedSecretE2EWithoutMapping(f, &defaultMode, "projected-secret-test-"+string(uuid.NewUUID()), &fsGroup, &uid) }) /* Release : v1.9 Testname: Projected Volume, Secrets, mapped Description: A Pod is created with a projected volume source ‘secret’ to store a secret with a specified key with default permission mode. The secret is also mapped to a custom path. Pod MUST be able to read the content of the key successfully and the mode MUST be -r—-—————— on the mapped volume. */ framework.ConformanceIt("should be consumable from pods in volume with mappings [NodeConformance]", func() { doProjectedSecretE2EWithMapping(f, nil) }) /* Release : v1.9 Testname: Projected Volume, Secrets, mapped, volume mode 0400 Description: A Pod is created with a projected volume source ‘secret’ to store a secret with a specified key with permission mode set to 0400. The secret is also mapped to a specific name. Pod MUST be able to read the content of the key successfully and the mode MUST be -r—-—————— on the mapped volume. */ framework.ConformanceIt("should be consumable from pods in volume with mappings and Item Mode set [NodeConformance]", func() { mode := int32(0400) doProjectedSecretE2EWithMapping(f, &mode) }) It("should be able to mount in a volume regardless of a different secret existing with same name in different namespace [NodeConformance]", func() { var ( namespace2  *v1.Namespace err         error secret2Name = "projected-secret-test-" + string(uuid.NewUUID()) ) if namespace2, err = f.CreateNamespace("secret-namespace", nil); err != nil { framework.Failf("unable to create new namespace %s: %v", namespace2.Name, err) } secret2 := secretForTest(namespace2.Name, secret2Name) secret2.Data = map[string][]byte{ "this_should_not_match_content_of_other_secret": []byte("similarly_this_should_not_match_content_of_other_secretn"), } if secret2, err = f.ClientSet.CoreV1().Secrets(namespace2.Name).Create(secret2); err != nil { framework.Failf("unable to create test secret %s: %v", secret2.Name, err) } doProjectedSecretE2EWithoutMapping(f, nil /* default mode */, secret2.Name, nil, nil) }) /* Release : v1.9 Testname: Projected Volume, Secrets, mapped, multiple paths Description: A Pod is created with a projected volume source ‘secret’ to store a secret with a specified key. The secret is mapped to two different volume mounts. Pod MUST be able to read the content of the key successfully from the two volume mounts and the mode MUST be -r—-—————— on the mapped volumes. */ framework.ConformanceIt("should be consumable in multiple volumes in a pod [NodeConformance]", func() { // This test ensures that the same secret can be mounted in multiple // volumes in the same pod.  This test case exists to prevent // regressions that break this use-case. var ( name             = "projected-secret-test-" + string(uuid.NewUUID()) volumeName       = "projected-secret-volume" volumeMountPath  = "/etc/projected-secret-volume" volumeName2      = "projected-secret-volume-2" volumeMountPath2 = "/etc/projected-secret-volume-2" secret           = secretForTest(f.Namespace.Name, name) ) By(fmt.Sprintf("Creating secret with name %s", secret.Name)) var err error if secret, err = f.ClientSet.CoreV1().Secrets(f.Namespace.Name).Create(secret); err != nil { framework.Failf("unable to create test secret %s: %v", secret.Name, err) } pod := &v1.Pod{ ObjectMeta: metav1.ObjectMeta{ Name: "pod-projected-secrets-" + string(uuid.NewUUID()), }, Spec: v1.PodSpec{ Volumes: []v1.Volume{ { Name: volumeName, VolumeSource: v1.VolumeSource{ Projected: &v1.ProjectedVolumeSource{ Sources: []v1.VolumeProjection{ { Secret: &v1.SecretProjection{ LocalObjectReference: v1.LocalObjectReference{ Name: name, }, }, }, }, }, }, }, { Name: volumeName2, VolumeSource: v1.VolumeSource{ Projected: &v1.ProjectedVolumeSource{ Sources: []v1.VolumeProjection{ { Secret: &v1.SecretProjection{ LocalObjectReference: v1.LocalObjectReference{ Name: name, }, }, }, }, }, }, }, }, Containers: []v1.Container{ { Name:  "secret-volume-test", Image: imageutils.GetE2EImage(imageutils.Mounttest), Args: []string{ "--file_content=/etc/projected-secret-volume/data-1", "--file_mode=/etc/projected-secret-volume/data-1"}, VolumeMounts: []v1.VolumeMount{ { Name:      volumeName, MountPath: volumeMountPath, ReadOnly:  true, }, { Name:      volumeName2, MountPath: volumeMountPath2, ReadOnly:  true, }, }, }, }, RestartPolicy: v1.RestartPolicyNever, }, } f.TestContainerOutput("consume secrets", pod, 0, []string{ "content of file "/etc/projected-secret-volume/data-1": value-1", "mode of file "/etc/projected-secret-volume/data-1": -rw-r--r--", }) }) /* Release : v1.9 Testname: Projected Volume, Secrets, create, update delete Description: Create a Pod with three containers with secrets namely a create, update and delete container. Create Container when started MUST no have a secret, update and delete containers MUST be created with a secret value. Create a secret in the create container, the Pod MUST be able to read the secret from the create container. Update the secret in the update container, Pod MUST be able to read the updated secret value. Delete the secret in the delete container. Pod MUST fail to read the secret from the delete container. */ framework.ConformanceIt("optional updates should be reflected in volume [NodeConformance]", func() { podLogTimeout := framework.GetPodSecretUpdateTimeout(f.ClientSet) containerTimeoutArg := fmt.Sprintf("--retry_time=%v", int(podLogTimeout.Seconds())) trueVal := true volumeMountPath := "/etc/projected-secret-volumes" deleteName := "s-test-opt-del-" + string(uuid.NewUUID()) deleteContainerName := "dels-volume-test" deleteVolumeName := "deletes-volume" deleteSecret := &v1.Secret{ ObjectMeta: metav1.ObjectMeta{ Namespace: f.Namespace.Name, Name:      deleteName, }, Data: map[string][]byte{ "data-1": []byte("value-1"), }, } updateName := "s-test-opt-upd-" + string(uuid.NewUUID()) updateContainerName := "upds-volume-test" updateVolumeName := "updates-volume" updateSecret := &v1.Secret{ ObjectMeta: metav1.ObjectMeta{ Namespace: f.Namespace.Name, Name:      updateName, }, Data: map[string][]byte{ "data-1": []byte("value-1"), }, } createName := "s-test-opt-create-" + string(uuid.NewUUID()) createContainerName := "creates-volume-test" createVolumeName := "creates-volume" createSecret := &v1.Secret{ ObjectMeta: metav1.ObjectMeta{ Namespace: f.Namespace.Name, Name:      createName, }, Data: map[string][]byte{ "data-1": []byte("value-1"), }, } By(fmt.Sprintf("Creating secret with name %s", deleteSecret.Name)) var err error if deleteSecret, err = f.ClientSet.CoreV1().Secrets(f.Namespace.Name).Create(deleteSecret); err != nil { framework.Failf("unable to create test secret %s: %v", deleteSecret.Name, err) } By(fmt.Sprintf("Creating secret with name %s", updateSecret.Name)) if updateSecret, err = f.ClientSet.CoreV1().Secrets(f.Namespace.Name).Create(updateSecret); err != nil { framework.Failf("unable to create test secret %s: %v", updateSecret.Name, err) } pod := &v1.Pod{ ObjectMeta: metav1.ObjectMeta{ Name: "pod-projected-secrets-" + string(uuid.NewUUID()), }, Spec: v1.PodSpec{ Volumes: []v1.Volume{ { Name: deleteVolumeName, VolumeSource: v1.VolumeSource{ Projected: &v1.ProjectedVolumeSource{ Sources: []v1.VolumeProjection{ { Secret: &v1.SecretProjection{ LocalObjectReference: v1.LocalObjectReference{ Name: deleteName, }, Optional: &trueVal, }, }, }, }, }, }, { Name: updateVolumeName, VolumeSource: v1.VolumeSource{ Projected: &v1.ProjectedVolumeSource{ Sources: []v1.VolumeProjection{ { Secret: &v1.SecretProjection{ LocalObjectReference: v1.LocalObjectReference{ Name: updateName, }, Optional: &trueVal, }, }, }, }, }, }, { Name: createVolumeName, VolumeSource: v1.VolumeSource{ Projected: &v1.ProjectedVolumeSource{ Sources: []v1.VolumeProjection{ { Secret: &v1.SecretProjection{ LocalObjectReference: v1.LocalObjectReference{ Name: createName, }, Optional: &trueVal, }, }, }, }, }, }, }, Containers: []v1.Container{ { Name:    deleteContainerName, Image:   imageutils.GetE2EImage(imageutils.Mounttest), Command: []string{"/mounttest", "--break_on_expected_content=false", containerTimeoutArg, "--file_content_in_loop=/etc/projected-secret-volumes/delete/data-1"}, VolumeMounts: []v1.VolumeMount{ { Name:      deleteVolumeName, MountPath: path.Join(volumeMountPath, "delete"), ReadOnly:  true, }, }, }, { Name:    updateContainerName, Image:   imageutils.GetE2EImage(imageutils.Mounttest), Command: []string{"/mounttest", "--break_on_expected_content=false", containerTimeoutArg, "--file_content_in_loop=/etc/projected-secret-volumes/update/data-3"}, VolumeMounts: []v1.VolumeMount{ { Name:      updateVolumeName, MountPath: path.Join(volumeMountPath, "update"), ReadOnly:  true, }, }, }, { Name:    createContainerName, Image:   imageutils.GetE2EImage(imageutils.Mounttest), Command: []string{"/mounttest", "--break_on_expected_content=false", containerTimeoutArg, "--file_content_in_loop=/etc/projected-secret-volumes/create/data-1"}, VolumeMounts: []v1.VolumeMount{ { Name:      createVolumeName, MountPath: path.Join(volumeMountPath, "create"), ReadOnly:  true, }, }, }, }, RestartPolicy: v1.RestartPolicyNever, }, } By("Creating the pod") f.PodClient().CreateSync(pod) pollCreateLogs := func() (string, error) { return framework.GetPodLogs(f.ClientSet, f.Namespace.Name, pod.Name, createContainerName) } Eventually(pollCreateLogs, podLogTimeout, framework.Poll).Should(ContainSubstring("Error reading file /etc/projected-secret-volumes/create/data-1")) pollUpdateLogs := func() (string, error) { return framework.GetPodLogs(f.ClientSet, f.Namespace.Name, pod.Name, updateContainerName) } Eventually(pollUpdateLogs, podLogTimeout, framework.Poll).Should(ContainSubstring("Error reading file /etc/projected-secret-volumes/update/data-3")) pollDeleteLogs := func() (string, error) { return framework.GetPodLogs(f.ClientSet, f.Namespace.Name, pod.Name, deleteContainerName) } Eventually(pollDeleteLogs, podLogTimeout, framework.Poll).Should(ContainSubstring("value-1")) By(fmt.Sprintf("Deleting secret %v", deleteSecret.Name)) err = f.ClientSet.CoreV1().Secrets(f.Namespace.Name).Delete(deleteSecret.Name, &metav1.DeleteOptions{}) Expect(err).NotTo(HaveOccurred(), "Failed to delete secret %q in namespace %q", deleteSecret.Name, f.Namespace.Name) By(fmt.Sprintf("Updating secret %v", updateSecret.Name)) updateSecret.ResourceVersion = "" // to force update delete(updateSecret.Data, "data-1") updateSecret.Data["data-3"] = []byte("value-3") _, err = f.ClientSet.CoreV1().Secrets(f.Namespace.Name).Update(updateSecret) Expect(err).NotTo(HaveOccurred(), "Failed to update secret %q in namespace %q", updateSecret.Name, f.Namespace.Name) By(fmt.Sprintf("Creating secret with name %s", createSecret.Name)) if createSecret, err = f.ClientSet.CoreV1().Secrets(f.Namespace.Name).Create(createSecret); err != nil { framework.Failf("unable to create test secret %s: %v", createSecret.Name, err) } By("waiting to observe update in volume") Eventually(pollCreateLogs, podLogTimeout, framework.Poll).Should(ContainSubstring("value-1")) Eventually(pollUpdateLogs, podLogTimeout, framework.Poll).Should(ContainSubstring("value-3")) Eventually(pollDeleteLogs, podLogTimeout, framework.Poll).Should(ContainSubstring("Error reading file /etc/projected-secret-volumes/delete/data-1")) }) // Part 2/3 - ConfigMaps /* Release : v1.9 Testname: Projected Volume, ConfigMap, volume mode default Description: A Pod is created with projected volume source ‘ConfigMap’ to store a configMap with default permission mode. Pod MUST be able to read the content of the ConfigMap successfully and the mode on the volume MUST be -rw-r—-r—-. */ framework.ConformanceIt("should be consumable from pods in volume [NodeConformance]", func() { doProjectedConfigMapE2EWithoutMappings(f, 0, 0, nil) }) /* Release : v1.9 Testname: Projected Volume, ConfigMap, volume mode 0400 Description: A Pod is created with projected volume source ‘ConfigMap’ to store a configMap with permission mode set to 0400. Pod MUST be able to read the content of the ConfigMap successfully and the mode on the volume MUST be -r——-——-—-. */ framework.ConformanceIt("should be consumable from pods in volume with defaultMode set [NodeConformance]", func() { defaultMode := int32(0400) doProjectedConfigMapE2EWithoutMappings(f, 0, 0, &defaultMode) }) It("should be consumable from pods in volume as non-root with defaultMode and fsGroup set [NodeFeature:FSGroup]", func() { defaultMode := int32(0440) /* setting fsGroup sets mode to at least 440 */ doProjectedConfigMapE2EWithoutMappings(f, 1000, 1001, &defaultMode) }) /* Release : v1.9 Testname: Projected Volume, ConfigMap, non-root user Description: A Pod is created with projected volume source ‘ConfigMap’ to store a configMap as non-root user with uid 1000. Pod MUST be able to read the content of the ConfigMap successfully and the mode on the volume MUST be -rw—r——r—-. */ framework.ConformanceIt("should be consumable from pods in volume as non-root [NodeConformance]", func() { doProjectedConfigMapE2EWithoutMappings(f, 1000, 0, nil) }) It("should be consumable from pods in volume as non-root with FSGroup [NodeFeature:FSGroup]", func() { doProjectedConfigMapE2EWithoutMappings(f, 1000, 1001, nil) }) /* Release : v1.9 Testname: Projected Volume, ConfigMap, mapped Description: A Pod is created with projected volume source ‘ConfigMap’ to store a configMap with default permission mode. The ConfigMap is also mapped to a custom path. Pod MUST be able to read the content of the ConfigMap from the custom location successfully and the mode on the volume MUST be -rw—r——r—-. */ framework.ConformanceIt("should be consumable from pods in volume with mappings [NodeConformance]", func() { doProjectedConfigMapE2EWithMappings(f, 0, 0, nil) }) /* Release : v1.9 Testname: Projected Volume, ConfigMap, mapped, volume mode 0400 Description: A Pod is created with projected volume source ‘ConfigMap’ to store a configMap with permission mode set to 0400. The ConfigMap is also mapped to a custom path. Pod MUST be able to read the content of the ConfigMap from the custom location successfully and the mode on the volume MUST be -r-—r——r—-. */ framework.ConformanceIt("should be consumable from pods in volume with mappings and Item mode set [NodeConformance]", func() { mode := int32(0400) doProjectedConfigMapE2EWithMappings(f, 0, 0, &mode) }) /* Release : v1.9 Testname: Projected Volume, ConfigMap, mapped, non-root user Description: A Pod is created with projected volume source ‘ConfigMap’ to store a configMap as non-root user with uid 1000. The ConfigMap is also mapped to a custom path. Pod MUST be able to read the content of the ConfigMap from the custom location successfully and the mode on the volume MUST be -r-—r——r—-. */ framework.ConformanceIt("should be consumable from pods in volume with mappings as non-root [NodeConformance]", func() { doProjectedConfigMapE2EWithMappings(f, 1000, 0, nil) }) It("should be consumable from pods in volume with mappings as non-root with FSGroup [NodeFeature:FSGroup]", func() { doProjectedConfigMapE2EWithMappings(f, 1000, 1001, nil) }) /* Release : v1.9 Testname: Projected Volume, ConfigMap, update Description: A Pod is created with projected volume source ‘ConfigMap’ to store a configMap and performs a create and update to new value. Pod MUST be able to create the configMap with value-1. Pod MUST be able to update the value in the confgiMap to value-2. */ framework.ConformanceIt("updates should be reflected in volume [NodeConformance]", func() { podLogTimeout := framework.GetPodSecretUpdateTimeout(f.ClientSet) containerTimeoutArg := fmt.Sprintf("--retry_time=%v", int(podLogTimeout.Seconds())) name := "projected-configmap-test-upd-" + string(uuid.NewUUID()) volumeName := "projected-configmap-volume" volumeMountPath := "/etc/projected-configmap-volume" containerName := "projected-configmap-volume-test" configMap := &v1.ConfigMap{ ObjectMeta: metav1.ObjectMeta{ Namespace: f.Namespace.Name, Name:      name, }, Data: map[string]string{ "data-1": "value-1", }, } By(fmt.Sprintf("Creating projection with configMap that has name %s", configMap.Name)) var err error if configMap, err = f.ClientSet.CoreV1().ConfigMaps(f.Namespace.Name).Create(configMap); err != nil { framework.Failf("unable to create test configMap %s: %v", configMap.Name, err) } pod := &v1.Pod{ ObjectMeta: metav1.ObjectMeta{ Name: "pod-projected-configmaps-" + string(uuid.NewUUID()), }, Spec: v1.PodSpec{ Volumes: []v1.Volume{ { Name: volumeName, VolumeSource: v1.VolumeSource{ Projected: &v1.ProjectedVolumeSource{ Sources: []v1.VolumeProjection{ { ConfigMap: &v1.ConfigMapProjection{ LocalObjectReference: v1.LocalObjectReference{ Name: name, }, }, }, }, }, }, }, }, Containers: []v1.Container{ { Name:    containerName, Image:   imageutils.GetE2EImage(imageutils.Mounttest), Command: []string{"/mounttest", "--break_on_expected_content=false", containerTimeoutArg, "--file_content_in_loop=/etc/projected-configmap-volume/data-1"}, VolumeMounts: []v1.VolumeMount{ { Name:      volumeName, MountPath: volumeMountPath, ReadOnly:  true, }, }, }, }, RestartPolicy: v1.RestartPolicyNever, }, } By("Creating the pod") f.PodClient().CreateSync(pod) pollLogs := func() (string, error) { return framework.GetPodLogs(f.ClientSet, f.Namespace.Name, pod.Name, containerName) } Eventually(pollLogs, podLogTimeout, framework.Poll).Should(ContainSubstring("value-1")) By(fmt.Sprintf("Updating configmap %v", configMap.Name)) configMap.ResourceVersion = "" // to force update configMap.Data["data-1"] = "value-2" _, err = f.ClientSet.CoreV1().ConfigMaps(f.Namespace.Name).Update(configMap) Expect(err).NotTo(HaveOccurred(), "Failed to update configmap %q in namespace %q", configMap.Name, f.Namespace.Name) By("waiting to observe update in volume") Eventually(pollLogs, podLogTimeout, framework.Poll).Should(ContainSubstring("value-2")) }) /* Release : v1.9 Testname: Projected Volume, ConfigMap, create, update and delete Description: Create a Pod with three containers with ConfigMaps namely a create, update and delete container. Create Container when started MUST not have configMap, update and delete containers MUST be created with a ConfigMap value as ‘value-1’. Create a configMap in the create container, the Pod MUST be able to read the configMap from the create container. Update the configMap in the update container, Pod MUST be able to read the updated configMap value. Delete the configMap in the delete container. Pod MUST fail to read the configMap from the delete container. */ framework.ConformanceIt("optional updates should be reflected in volume [NodeConformance]", func() { podLogTimeout := framework.GetPodSecretUpdateTimeout(f.ClientSet) containerTimeoutArg := fmt.Sprintf("--retry_time=%v", int(podLogTimeout.Seconds())) trueVal := true volumeMountPath := "/etc/projected-configmap-volumes" deleteName := "cm-test-opt-del-" + string(uuid.NewUUID()) deleteContainerName := "delcm-volume-test" deleteVolumeName := "deletecm-volume" deleteConfigMap := &v1.ConfigMap{ ObjectMeta: metav1.ObjectMeta{ Namespace: f.Namespace.Name, Name:      deleteName, }, Data: map[string]string{ "data-1": "value-1", }, } updateName := "cm-test-opt-upd-" + string(uuid.NewUUID()) updateContainerName := "updcm-volume-test" updateVolumeName := "updatecm-volume" updateConfigMap := &v1.ConfigMap{ ObjectMeta: metav1.ObjectMeta{ Namespace: f.Namespace.Name, Name:      updateName, }, Data: map[string]string{ "data-1": "value-1", }, } createName := "cm-test-opt-create-" + string(uuid.NewUUID()) createContainerName := "createcm-volume-test" createVolumeName := "createcm-volume" createConfigMap := &v1.ConfigMap{ ObjectMeta: metav1.ObjectMeta{ Namespace: f.Namespace.Name, Name:      createName, }, Data: map[string]string{ "data-1": "value-1", }, } By(fmt.Sprintf("Creating configMap with name %s", deleteConfigMap.Name)) var err error if deleteConfigMap, err = f.ClientSet.CoreV1().ConfigMaps(f.Namespace.Name).Create(deleteConfigMap); err != nil { framework.Failf("unable to create test configMap %s: %v", deleteConfigMap.Name, err) } By(fmt.Sprintf("Creating configMap with name %s", updateConfigMap.Name)) if updateConfigMap, err = f.ClientSet.CoreV1().ConfigMaps(f.Namespace.Name).Create(updateConfigMap); err != nil { framework.Failf("unable to create test configMap %s: %v", updateConfigMap.Name, err) } pod := &v1.Pod{ ObjectMeta: metav1.ObjectMeta{ Name: "pod-projected-configmaps-" + string(uuid.NewUUID()), }, Spec: v1.PodSpec{ Volumes: []v1.Volume{ { Name: deleteVolumeName, VolumeSource: v1.VolumeSource{ Projected: &v1.ProjectedVolumeSource{ Sources: []v1.VolumeProjection{ { ConfigMap: &v1.ConfigMapProjection{ LocalObjectReference: v1.LocalObjectReference{ Name: deleteName, }, Optional: &trueVal, }, }, }, }, }, }, { Name: updateVolumeName, VolumeSource: v1.VolumeSource{ Projected: &v1.ProjectedVolumeSource{ Sources: []v1.VolumeProjection{ { ConfigMap: &v1.ConfigMapProjection{ LocalObjectReference: v1.LocalObjectReference{ Name: updateName, }, Optional: &trueVal, }, }, }, }, }, }, { Name: createVolumeName, VolumeSource: v1.VolumeSource{ Projected: &v1.ProjectedVolumeSource{ Sources: []v1.VolumeProjection{ { ConfigMap: &v1.ConfigMapProjection{ LocalObjectReference: v1.LocalObjectReference{ Name: createName, }, Optional: &trueVal, }, }, }, }, }, }, }, Containers: []v1.Container{ { Name:    deleteContainerName, Image:   imageutils.GetE2EImage(imageutils.Mounttest), Command: []string{"/mounttest", "--break_on_expected_content=false", containerTimeoutArg, "--file_content_in_loop=/etc/projected-configmap-volumes/delete/data-1"}, VolumeMounts: []v1.VolumeMount{ { Name:      deleteVolumeName, MountPath: path.Join(volumeMountPath, "delete"), ReadOnly:  true, }, }, }, { Name:    updateContainerName, Image:   imageutils.GetE2EImage(imageutils.Mounttest), Command: []string{"/mounttest", "--break_on_expected_content=false", containerTimeoutArg, "--file_content_in_loop=/etc/projected-configmap-volumes/update/data-3"}, VolumeMounts: []v1.VolumeMount{ { Name:      updateVolumeName, MountPath: path.Join(volumeMountPath, "update"), ReadOnly:  true, }, }, }, { Name:    createContainerName, Image:   imageutils.GetE2EImage(imageutils.Mounttest), Command: []string{"/mounttest", "--break_on_expected_content=false", containerTimeoutArg, "--file_content_in_loop=/etc/projected-configmap-volumes/create/data-1"}, VolumeMounts: []v1.VolumeMount{ { Name:      createVolumeName, MountPath: path.Join(volumeMountPath, "create"), ReadOnly:  true, }, }, }, }, RestartPolicy: v1.RestartPolicyNever, }, } By("Creating the pod") f.PodClient().CreateSync(pod) pollCreateLogs := func() (string, error) { return framework.GetPodLogs(f.ClientSet, f.Namespace.Name, pod.Name, createContainerName) } Eventually(pollCreateLogs, podLogTimeout, framework.Poll).Should(ContainSubstring("Error reading file /etc/projected-configmap-volumes/create/data-1")) pollUpdateLogs := func() (string, error) { return framework.GetPodLogs(f.ClientSet, f.Namespace.Name, pod.Name, updateContainerName) } Eventually(pollUpdateLogs, podLogTimeout, framework.Poll).Should(ContainSubstring("Error reading file /etc/projected-configmap-volumes/update/data-3")) pollDeleteLogs := func() (string, error) { return framework.GetPodLogs(f.ClientSet, f.Namespace.Name, pod.Name, deleteContainerName) } Eventually(pollDeleteLogs, podLogTimeout, framework.Poll).Should(ContainSubstring("value-1")) By(fmt.Sprintf("Deleting configmap %v", deleteConfigMap.Name)) err = f.ClientSet.CoreV1().ConfigMaps(f.Namespace.Name).Delete(deleteConfigMap.Name, &metav1.DeleteOptions{}) Expect(err).NotTo(HaveOccurred(), "Failed to delete configmap %q in namespace %q", deleteConfigMap.Name, f.Namespace.Name) By(fmt.Sprintf("Updating configmap %v", updateConfigMap.Name)) updateConfigMap.ResourceVersion = "" // to force update delete(updateConfigMap.Data, "data-1") updateConfigMap.Data["data-3"] = "value-3" _, err = f.ClientSet.CoreV1().ConfigMaps(f.Namespace.Name).Update(updateConfigMap) Expect(err).NotTo(HaveOccurred(), "Failed to update configmap %q in namespace %q", updateConfigMap.Name, f.Namespace.Name) By(fmt.Sprintf("Creating configMap with name %s", createConfigMap.Name)) if createConfigMap, err = f.ClientSet.CoreV1().ConfigMaps(f.Namespace.Name).Create(createConfigMap); err != nil { framework.Failf("unable to create test configMap %s: %v", createConfigMap.Name, err) } By("waiting to observe update in volume") Eventually(pollCreateLogs, podLogTimeout, framework.Poll).Should(ContainSubstring("value-1")) Eventually(pollUpdateLogs, podLogTimeout, framework.Poll).Should(ContainSubstring("value-3")) Eventually(pollDeleteLogs, podLogTimeout, framework.Poll).Should(ContainSubstring("Error reading file /etc/projected-configmap-volumes/delete/data-1")) }) /* Release : v1.9 Testname: Projected Volume, ConfigMap, multiple volume paths Description: A Pod is created with a projected volume source ‘ConfigMap’ to store a configMap. The configMap is mapped to two different volume mounts. Pod MUST be able to read the content of the configMap successfully from the two volume mounts. */ framework.ConformanceIt("should be consumable in multiple volumes in the same pod [NodeConformance]", func() { var ( name             = "projected-configmap-test-volume-" + string(uuid.NewUUID()) volumeName       = "projected-configmap-volume" volumeMountPath  = "/etc/projected-configmap-volume" volumeName2      = "projected-configmap-volume-2" volumeMountPath2 = "/etc/projected-configmap-volume-2" configMap        = newConfigMap(f, name) ) By(fmt.Sprintf("Creating configMap with name %s", configMap.Name)) var err error if configMap, err = f.ClientSet.CoreV1().ConfigMaps(f.Namespace.Name).Create(configMap); err != nil { framework.Failf("unable to create test configMap %s: %v", configMap.Name, err) } pod := &v1.Pod{ ObjectMeta: metav1.ObjectMeta{ Name: "pod-projected-configmaps-" + string(uuid.NewUUID()), }, Spec: v1.PodSpec{ Volumes: []v1.Volume{ { Name: volumeName, VolumeSource: v1.VolumeSource{ Projected: &v1.ProjectedVolumeSource{ Sources: []v1.VolumeProjection{ { ConfigMap: &v1.ConfigMapProjection{ LocalObjectReference: v1.LocalObjectReference{ Name: name, }, }, }, }, }, }, }, { Name: volumeName2, VolumeSource: v1.VolumeSource{ Projected: &v1.ProjectedVolumeSource{ Sources: []v1.VolumeProjection{ { ConfigMap: &v1.ConfigMapProjection{ LocalObjectReference: v1.LocalObjectReference{ Name: name, }, }, }, }, }, }, }, }, Containers: []v1.Container{ { Name:  "projected-configmap-volume-test", Image: imageutils.GetE2EImage(imageutils.Mounttest), Args:  []string{"--file_content=/etc/projected-configmap-volume/data-1"}, VolumeMounts: []v1.VolumeMount{ { Name:      volumeName, MountPath: volumeMountPath, ReadOnly:  true, }, { Name:      volumeName2, MountPath: volumeMountPath2, ReadOnly:  true, }, }, }, }, RestartPolicy: v1.RestartPolicyNever, }, } f.TestContainerOutput("consume configMaps", pod, 0, []string{ "content of file "/etc/projected-configmap-volume/data-1": value-1", }) }) // Part 3/3 - DownwardAPI // How long to wait for a log pod to be displayed const podLogTimeout = 2 * time.Minute var podClient *framework.PodClient BeforeEach(func() { podClient = f.PodClient() }) /* Release : v1.9 Testname: Projected Volume, DownwardAPI, pod name Description: A Pod is created with a projected volume source for downwardAPI with pod name, cpu and memory limits and cpu and memory requests. Pod MUST be able to read the pod name from the mounted DownwardAPIVolumeFiles. */ framework.ConformanceIt("should provide podname only [NodeConformance]", func() { podName := "downwardapi-volume-" + string(uuid.NewUUID()) pod := downwardAPIVolumePodForSimpleTest(podName, "/etc/podinfo/podname") f.TestContainerOutput("downward API volume plugin", pod, 0, []string{ fmt.Sprintf("%sn", podName), }) }) /* Release : v1.9 Testname: Projected Volume, DownwardAPI, volume mode 0400 Description: A Pod is created with a projected volume source for downwardAPI with pod name, cpu and memory limits and cpu and memory requests. The default mode for the volume mount is set to 0400. Pod MUST be able to read the pod name from the mounted DownwardAPIVolumeFiles and the volume mode must be -r—-—————. */ framework.ConformanceIt("should set DefaultMode on files [NodeConformance]", func() { podName := "downwardapi-volume-" + string(uuid.NewUUID()) defaultMode := int32(0400) pod := projectedDownwardAPIVolumePodForModeTest(podName, "/etc/podinfo/podname", nil, &defaultMode) f.TestContainerOutput("downward API volume plugin", pod, 0, []string{ "mode of file "/etc/podinfo/podname": -r--------", }) }) /* Release : v1.9 Testname: Projected Volume, DownwardAPI, volume mode 0400 Description: A Pod is created with a projected volume source for downwardAPI with pod name, cpu and memory limits and cpu and memory requests. The default mode for the volume mount is set to 0400. Pod MUST be able to read the pod name from the mounted DownwardAPIVolumeFiles and the volume mode must be -r—-—————. */ framework.ConformanceIt("should set mode on item file [NodeConformance]", func() { podName := "downwardapi-volume-" + string(uuid.NewUUID()) mode := int32(0400) pod := projectedDownwardAPIVolumePodForModeTest(podName, "/etc/podinfo/podname", &mode, nil) f.TestContainerOutput("downward API volume plugin", pod, 0, []string{ "mode of file "/etc/podinfo/podname": -r--------", }) }) It("should provide podname as non-root with fsgroup [NodeFeature:FSGroup]", func() { podName := "metadata-volume-" + string(uuid.NewUUID()) uid := int64(1001) gid := int64(1234) pod := downwardAPIVolumePodForSimpleTest(podName, "/etc/podinfo/podname") pod.Spec.SecurityContext = &v1.PodSecurityContext{ RunAsUser: &uid, FSGroup:   &gid, } f.TestContainerOutput("downward API volume plugin", pod, 0, []string{ fmt.Sprintf("%sn", podName), }) }) It("should provide podname as non-root with fsgroup and defaultMode [NodeFeature:FSGroup]", func() { podName := "metadata-volume-" + string(uuid.NewUUID()) uid := int64(1001) gid := int64(1234) mode := int32(0440) /* setting fsGroup sets mode to at least 440 */ pod := projectedDownwardAPIVolumePodForModeTest(podName, "/etc/podinfo/podname", &mode, nil) pod.Spec.SecurityContext = &v1.PodSecurityContext{ RunAsUser: &uid, FSGroup:   &gid, } f.TestContainerOutput("downward API volume plugin", pod, 0, []string{ "mode of file "/etc/podinfo/podname": -r--r-----", }) }) /* Release : v1.9 Testname: Projected Volume, DownwardAPI, update labels Description: A Pod is created with a projected volume source for downwardAPI with pod name, cpu and memory limits and cpu and memory requests and label items. Pod MUST be able to read the labels from the mounted DownwardAPIVolumeFiles. Labels are then updated. Pod MUST be able to read the updated values for the Labels. */ framework.ConformanceIt("should update labels on modification [NodeConformance]", func() { labels := map[string]string{} labels["key1"] = "value1" labels["key2"] = "value2" podName := "labelsupdate" + string(uuid.NewUUID()) pod := projectedDownwardAPIVolumePodForUpdateTest(podName, labels, map[string]string{}, "/etc/podinfo/labels") containerName := "client-container" By("Creating the pod") podClient.CreateSync(pod) Eventually(func() (string, error) { return framework.GetPodLogs(f.ClientSet, f.Namespace.Name, podName, containerName) }, podLogTimeout, framework.Poll).Should(ContainSubstring("key1="value1"n")) //modify labels podClient.Update(podName, func(pod *v1.Pod) { pod.Labels["key3"] = "value3" }) Eventually(func() (string, error) { return framework.GetPodLogs(f.ClientSet, f.Namespace.Name, pod.Name, containerName) }, podLogTimeout, framework.Poll).Should(ContainSubstring("key3="value3"n")) }) /* Release : v1.9 Testname: Projected Volume, DownwardAPI, update annotation Description: A Pod is created with a projected volume source for downwardAPI with pod name, cpu and memory limits and cpu and memory requests and annotation items. Pod MUST be able to read the annotations from the mounted DownwardAPIVolumeFiles. Annotations are then updated. Pod MUST be able to read the updated values for the Annotations. */ framework.ConformanceIt("should update annotations on modification [NodeConformance]", func() { annotations := map[string]string{} annotations["builder"] = "bar" podName := "annotationupdate" + string(uuid.NewUUID()) pod := projectedDownwardAPIVolumePodForUpdateTest(podName, map[string]string{}, annotations, "/etc/podinfo/annotations") containerName := "client-container" By("Creating the pod") podClient.CreateSync(pod) pod, err := podClient.Get(pod.Name, metav1.GetOptions{}) Expect(err).NotTo(HaveOccurred(), "Failed to get pod %q", pod.Name) Eventually(func() (string, error) { return framework.GetPodLogs(f.ClientSet, f.Namespace.Name, pod.Name, containerName) }, podLogTimeout, framework.Poll).Should(ContainSubstring("builder="bar"n")) //modify annotations podClient.Update(podName, func(pod *v1.Pod) { pod.Annotations["builder"] = "foo" }) Eventually(func() (string, error) { return framework.GetPodLogs(f.ClientSet, f.Namespace.Name, pod.Name, containerName) }, podLogTimeout, framework.Poll).Should(ContainSubstring("builder="foo"n")) }) /* Release : v1.9 Testname: Projected Volume, DownwardAPI, CPU limits Description: A Pod is created with a projected volume source for downwardAPI with pod name, cpu and memory limits and cpu and memory requests. Pod MUST be able to read the cpu limits from the mounted DownwardAPIVolumeFiles. */ framework.ConformanceIt("should provide container's cpu limit [NodeConformance]", func() { podName := "downwardapi-volume-" + string(uuid.NewUUID()) pod := downwardAPIVolumeForContainerResources(podName, "/etc/podinfo/cpu_limit") f.TestContainerOutput("downward API volume plugin", pod, 0, []string{ fmt.Sprintf("2n"), }) }) /* Release : v1.9 Testname: Projected Volume, DownwardAPI, memory limits Description: A Pod is created with a projected volume source for downwardAPI with pod name, cpu and memory limits and cpu and memory requests. Pod MUST be able to read the memory limits from the mounted DownwardAPIVolumeFiles. */ framework.ConformanceIt("should provide container's memory limit [NodeConformance]", func() { podName := "downwardapi-volume-" + string(uuid.NewUUID()) pod := downwardAPIVolumeForContainerResources(podName, "/etc/podinfo/memory_limit") f.TestContainerOutput("downward API volume plugin", pod, 0, []string{ fmt.Sprintf("67108864n"), }) }) /* Release : v1.9 Testname: Projected Volume, DownwardAPI, CPU request Description: A Pod is created with a projected volume source for downwardAPI with pod name, cpu and memory limits and cpu and memory requests. Pod MUST be able to read the cpu request from the mounted DownwardAPIVolumeFiles. */ framework.ConformanceIt("should provide container's cpu request [NodeConformance]", func() { podName := "downwardapi-volume-" + string(uuid.NewUUID()) pod := downwardAPIVolumeForContainerResources(podName, "/etc/podinfo/cpu_request") f.TestContainerOutput("downward API volume plugin", pod, 0, []string{ fmt.Sprintf("1n"), }) }) /* Release : v1.9 Testname: Projected Volume, DownwardAPI, memory request Description: A Pod is created with a projected volume source for downwardAPI with pod name, cpu and memory limits and cpu and memory requests. Pod MUST be able to read the memory request from the mounted DownwardAPIVolumeFiles. */ framework.ConformanceIt("should provide container's memory request [NodeConformance]", func() { podName := "downwardapi-volume-" + string(uuid.NewUUID()) pod := downwardAPIVolumeForContainerResources(podName, "/etc/podinfo/memory_request") f.TestContainerOutput("downward API volume plugin", pod, 0, []string{ fmt.Sprintf("33554432n"), }) }) /* Release : v1.9 Testname: Projected Volume, DownwardAPI, CPU limit, node allocatable Description: A Pod is created with a projected volume source for downwardAPI with pod name, cpu and memory limits and cpu and memory requests.  The CPU and memory resources for requests and limits are NOT specified for the container. Pod MUST be able to read the default cpu limits from the mounted DownwardAPIVolumeFiles. */ framework.ConformanceIt("should provide node allocatable (cpu) as default cpu limit if the limit is not set [NodeConformance]", func() { podName := "downwardapi-volume-" + string(uuid.NewUUID()) pod := downwardAPIVolumeForDefaultContainerResources(podName, "/etc/podinfo/cpu_limit") f.TestContainerOutputRegexp("downward API volume plugin", pod, 0, []string{"[1-9]"}) }) /* Release : v1.9 Testname: Projected Volume, DownwardAPI, memory limit, node allocatable Description: A Pod is created with a projected volume source for downwardAPI with pod name, cpu and memory limits and cpu and memory requests.  The CPU and memory resources for requests and limits are NOT specified for the container. Pod MUST be able to read the default memory limits from the mounted DownwardAPIVolumeFiles. */ framework.ConformanceIt("should provide node allocatable (memory) as default memory limit if the limit is not set [NodeConformance]", func() { podName := "downwardapi-volume-" + string(uuid.NewUUID()) pod := downwardAPIVolumeForDefaultContainerResources(podName, "/etc/podinfo/memory_limit") f.TestContainerOutputRegexp("downward API volume plugin", pod, 0, []string{"[1-9]"}) }) // Test multiple projections /* Release : v1.9 Testname: Projected Volume, multiple projections Description: A Pod is created with a projected volume source for secrets, configMap and downwardAPI with pod name, cpu and memory limits and cpu and memory requests. Pod MUST be able to read the secrets, configMap values and the cpu and memory limits as well as cpu and memory requests from the mounted DownwardAPIVolumeFiles. */ framework.ConformanceIt("should project all components that make up the projection API [Projection][NodeConformance]", func() { var err error podName := "projected-volume-" + string(uuid.NewUUID()) secretName := "secret-projected-all-test-volume-" + string(uuid.NewUUID()) configMapName := "configmap-projected-all-test-volume-" + string(uuid.NewUUID()) configMap := &v1.ConfigMap{ ObjectMeta: metav1.ObjectMeta{ Namespace: f.Namespace.Name, Name:      configMapName, }, Data: map[string]string{ "configmap-data": "configmap-value-1", }, } secret := &v1.Secret{ ObjectMeta: metav1.ObjectMeta{ Namespace: f.Namespace.Name, Name:      secretName, }, Data: map[string][]byte{ "secret-data": []byte("secret-value-1"), }, } By(fmt.Sprintf("Creating configMap with name %s", configMap.Name)) if configMap, err = f.ClientSet.CoreV1().ConfigMaps(f.Namespace.Name).Create(configMap); err != nil { framework.Failf("unable to create test configMap %s: %v", configMap.Name, err) } By(fmt.Sprintf("Creating secret with name %s", secret.Name)) if secret, err = f.ClientSet.CoreV1().Secrets(f.Namespace.Name).Create(secret); err != nil { framework.Failf("unable to create test secret %s: %v", secret.Name, err) } pod := projectedAllVolumeBasePod(podName, secretName, configMapName, nil, nil) pod.Spec.Containers = []v1.Container{ { Name:    "projected-all-volume-test", Image:   imageutils.GetE2EImage(imageutils.BusyBox), Command: []string{"sh", "-c", "cat /all/podname && cat /all/secret-data && cat /all/configmap-data"}, VolumeMounts: []v1.VolumeMount{ { Name:      "podinfo", MountPath: "/all", ReadOnly:  false, }, }, }, } f.TestContainerOutput("Check all projections for projected volume plugin", pod, 0, []string{ fmt.Sprintf("%s", podName), "secret-value-1", "configmap-value-1", }) }) }) func doProjectedSecretE2EWithoutMapping(f *framework.Framework, defaultMode *int32, secretName string, fsGroup *int64, uid *int64) { var ( volumeName      = "projected-secret-volume" volumeMountPath = "/etc/projected-secret-volume" secret          = secretForTest(f.Namespace.Name, secretName) ) By(fmt.Sprintf("Creating projection with secret that has name %s", secret.Name)) var err error if secret, err = f.ClientSet.CoreV1().Secrets(f.Namespace.Name).Create(secret); err != nil { framework.Failf("unable to create test secret %s: %v", secret.Name, err) } pod := &v1.Pod{ ObjectMeta: metav1.ObjectMeta{ Name:      "pod-projected-secrets-" + string(uuid.NewUUID()), Namespace: f.Namespace.Name, }, Spec: v1.PodSpec{ Volumes: []v1.Volume{ { Name: volumeName, VolumeSource: v1.VolumeSource{ Projected: &v1.ProjectedVolumeSource{ Sources: []v1.VolumeProjection{ { Secret: &v1.SecretProjection{ LocalObjectReference: v1.LocalObjectReference{ Name: secretName, }, }, }, }, }, }, }, }, Containers: []v1.Container{ { Name:  "projected-secret-volume-test", Image: imageutils.GetE2EImage(imageutils.Mounttest), Args: []string{ "--file_content=/etc/projected-secret-volume/data-1", "--file_mode=/etc/projected-secret-volume/data-1"}, VolumeMounts: []v1.VolumeMount{ { Name:      volumeName, MountPath: volumeMountPath, }, }, }, }, RestartPolicy: v1.RestartPolicyNever, }, } if defaultMode != nil { //pod.Spec.Volumes[0].VolumeSource.Projected.Sources[0].Secret.DefaultMode = defaultMode pod.Spec.Volumes[0].VolumeSource.Projected.DefaultMode = defaultMode } else { mode := int32(0644) defaultMode = &mode } if fsGroup != nil || uid != nil { pod.Spec.SecurityContext = &v1.PodSecurityContext{ FSGroup:   fsGroup, RunAsUser: uid, } } modeString := fmt.Sprintf("%v", os.FileMode(*defaultMode)) expectedOutput := []string{ "content of file "/etc/projected-secret-volume/data-1": value-1", "mode of file "/etc/projected-secret-volume/data-1": " + modeString, } f.TestContainerOutput("consume secrets", pod, 0, expectedOutput) } func doProjectedSecretE2EWithMapping(f *framework.Framework, mode *int32) { var ( name            = "projected-secret-test-map-" + string(uuid.NewUUID()) volumeName      = "projected-secret-volume" volumeMountPath = "/etc/projected-secret-volume" secret          = secretForTest(f.Namespace.Name, name) ) By(fmt.Sprintf("Creating projection with secret that has name %s", secret.Name)) var err error if secret, err = f.ClientSet.CoreV1().Secrets(f.Namespace.Name).Create(secret); err != nil { framework.Failf("unable to create test secret %s: %v", secret.Name, err) } pod := &v1.Pod{ ObjectMeta: metav1.ObjectMeta{ Name: "pod-projected-secrets-" + string(uuid.NewUUID()), }, Spec: v1.PodSpec{ Volumes: []v1.Volume{ { Name: volumeName, VolumeSource: v1.VolumeSource{ Projected: &v1.ProjectedVolumeSource{ Sources: []v1.VolumeProjection{ { Secret: &v1.SecretProjection{ LocalObjectReference: v1.LocalObjectReference{ Name: name, }, Items: []v1.KeyToPath{ { Key:  "data-1", Path: "new-path-data-1", }, }, }, }, }, }, }, }, }, Containers: []v1.Container{ { Name:  "projected-secret-volume-test", Image: imageutils.GetE2EImage(imageutils.Mounttest), Args: []string{ "--file_content=/etc/projected-secret-volume/new-path-data-1", "--file_mode=/etc/projected-secret-volume/new-path-data-1"}, VolumeMounts: []v1.VolumeMount{ { Name:      volumeName, MountPath: volumeMountPath, }, }, }, }, RestartPolicy: v1.RestartPolicyNever, }, } if mode != nil { //pod.Spec.Volumes[0].VolumeSource.Projected.Sources[0].Secret.Items[0].Mode = mode pod.Spec.Volumes[0].VolumeSource.Projected.DefaultMode = mode } else { defaultItemMode := int32(0644) mode = &defaultItemMode } modeString := fmt.Sprintf("%v", os.FileMode(*mode)) expectedOutput := []string{ "content of file "/etc/projected-secret-volume/new-path-data-1": value-1", "mode of file "/etc/projected-secret-volume/new-path-data-1": " + modeString, } f.TestContainerOutput("consume secrets", pod, 0, expectedOutput) } func doProjectedConfigMapE2EWithoutMappings(f *framework.Framework, uid, fsGroup int64, defaultMode *int32) { userID := int64(uid) groupID := int64(fsGroup) var ( name            = "projected-configmap-test-volume-" + string(uuid.NewUUID()) volumeName      = "projected-configmap-volume" volumeMountPath = "/etc/projected-configmap-volume" configMap       = newConfigMap(f, name) ) By(fmt.Sprintf("Creating configMap with name %s", configMap.Name)) var err error if configMap, err = f.ClientSet.CoreV1().ConfigMaps(f.Namespace.Name).Create(configMap); err != nil { framework.Failf("unable to create test configMap %s: %v", configMap.Name, err) } pod := &v1.Pod{ ObjectMeta: metav1.ObjectMeta{ Name: "pod-projected-configmaps-" + string(uuid.NewUUID()), }, Spec: v1.PodSpec{ SecurityContext: &v1.PodSecurityContext{}, Volumes: []v1.Volume{ { Name: volumeName, VolumeSource: v1.VolumeSource{ Projected: &v1.ProjectedVolumeSource{ Sources: []v1.VolumeProjection{ { ConfigMap: &v1.ConfigMapProjection{ LocalObjectReference: v1.LocalObjectReference{ Name: name, }, }, }, }, }, }, }, }, Containers: []v1.Container{ { Name:  "projected-configmap-volume-test", Image: imageutils.GetE2EImage(imageutils.Mounttest), Args: []string{ "--file_content=/etc/projected-configmap-volume/data-1", "--file_mode=/etc/projected-configmap-volume/data-1"}, VolumeMounts: []v1.VolumeMount{ { Name:      volumeName, MountPath: volumeMountPath, }, }, }, }, RestartPolicy: v1.RestartPolicyNever, }, } if userID != 0 { pod.Spec.SecurityContext.RunAsUser = &userID } if groupID != 0 { pod.Spec.SecurityContext.FSGroup = &groupID } if defaultMode != nil { //pod.Spec.Volumes[0].VolumeSource.Projected.Sources[0].ConfigMap.DefaultMode = defaultMode pod.Spec.Volumes[0].VolumeSource.Projected.DefaultMode = defaultMode } else { mode := int32(0644) defaultMode = &mode } modeString := fmt.Sprintf("%v", os.FileMode(*defaultMode)) output := []string{ "content of file "/etc/projected-configmap-volume/data-1": value-1", "mode of file "/etc/projected-configmap-volume/data-1": " + modeString, } f.TestContainerOutput("consume configMaps", pod, 0, output) } func doProjectedConfigMapE2EWithMappings(f *framework.Framework, uid, fsGroup int64, itemMode *int32) { userID := int64(uid) groupID := int64(fsGroup) var ( name            = "projected-configmap-test-volume-map-" + string(uuid.NewUUID()) volumeName      = "projected-configmap-volume" volumeMountPath = "/etc/projected-configmap-volume" configMap       = newConfigMap(f, name) ) By(fmt.Sprintf("Creating configMap with name %s", configMap.Name)) var err error if configMap, err = f.ClientSet.CoreV1().ConfigMaps(f.Namespace.Name).Create(configMap); err != nil { framework.Failf("unable to create test configMap %s: %v", configMap.Name, err) } pod := &v1.Pod{ ObjectMeta: metav1.ObjectMeta{ Name: "pod-projected-configmaps-" + string(uuid.NewUUID()), }, Spec: v1.PodSpec{ SecurityContext: &v1.PodSecurityContext{}, Volumes: []v1.Volume{ { Name: volumeName, VolumeSource: v1.VolumeSource{ Projected: &v1.ProjectedVolumeSource{ Sources: []v1.VolumeProjection{ { ConfigMap: &v1.ConfigMapProjection{ LocalObjectReference: v1.LocalObjectReference{ Name: name, }, Items: []v1.KeyToPath{ { Key:  "data-2", Path: "path/to/data-2", }, }, }, }, }, }, }, }, }, Containers: []v1.Container{ { Name:  "projected-configmap-volume-test", Image: imageutils.GetE2EImage(imageutils.Mounttest), Args: []string{"--file_content=/etc/projected-configmap-volume/path/to/data-2", "--file_mode=/etc/projected-configmap-volume/path/to/data-2"}, VolumeMounts: []v1.VolumeMount{ { Name:      volumeName, MountPath: volumeMountPath, ReadOnly:  true, }, }, }, }, RestartPolicy: v1.RestartPolicyNever, }, } if userID != 0 { pod.Spec.SecurityContext.RunAsUser = &userID } if groupID != 0 { pod.Spec.SecurityContext.FSGroup = &groupID } if itemMode != nil { //pod.Spec.Volumes[0].VolumeSource.ConfigMap.Items[0].Mode = itemMode pod.Spec.Volumes[0].VolumeSource.Projected.DefaultMode = itemMode } else { mode := int32(0644) itemMode = &mode } // Just check file mode if fsGroup is not set. If fsGroup is set, the // final mode is adjusted and we are not testing that case. output := []string{ "content of file "/etc/projected-configmap-volume/path/to/data-2": value-2", } if fsGroup == 0 { modeString := fmt.Sprintf("%v", os.FileMode(*itemMode)) output = append(output, "mode of file "/etc/projected-configmap-volume/path/to/data-2": "+modeString) } f.TestContainerOutput("consume configMaps", pod, 0, output) } func projectedDownwardAPIVolumePodForModeTest(name, filePath string, itemMode, defaultMode *int32) *v1.Pod { pod := projectedDownwardAPIVolumeBasePod(name, nil, nil) pod.Spec.Containers = []v1.Container{ { Name:    "client-container", Image:   imageutils.GetE2EImage(imageutils.Mounttest), Command: []string{"/mounttest", "--file_mode=" + filePath}, VolumeMounts: []v1.VolumeMount{ { Name:      "podinfo", MountPath: "/etc/podinfo", }, }, }, } if itemMode != nil { pod.Spec.Volumes[0].VolumeSource.Projected.Sources[0].DownwardAPI.Items[0].Mode = itemMode } if defaultMode != nil { pod.Spec.Volumes[0].VolumeSource.Projected.DefaultMode = defaultMode } return pod } func projectedDownwardAPIVolumePodForUpdateTest(name string, labels, annotations map[string]string, filePath string) *v1.Pod { pod := projectedDownwardAPIVolumeBasePod(name, labels, annotations) pod.Spec.Containers = []v1.Container{ { Name:    "client-container", Image:   imageutils.GetE2EImage(imageutils.Mounttest), Command: []string{"/mounttest", "--break_on_expected_content=false", "--retry_time=1200", "--file_content_in_loop=" + filePath}, VolumeMounts: []v1.VolumeMount{ { Name:      "podinfo", MountPath: "/etc/podinfo", ReadOnly:  false, }, }, }, } applyLabelsAndAnnotationsToProjectedDownwardAPIPod(labels, annotations, pod) return pod } func projectedDownwardAPIVolumeBasePod(name string, labels, annotations map[string]string) *v1.Pod { pod := &v1.Pod{ ObjectMeta: metav1.ObjectMeta{ Name:        name, Labels:      labels, Annotations: annotations, }, Spec: v1.PodSpec{ Volumes: []v1.Volume{ { Name: "podinfo", VolumeSource: v1.VolumeSource{ Projected: &v1.ProjectedVolumeSource{ Sources: []v1.VolumeProjection{ { DownwardAPI: &v1.DownwardAPIProjection{ Items: []v1.DownwardAPIVolumeFile{ { Path: "podname", FieldRef: &v1.ObjectFieldSelector{ APIVersion: "v1", FieldPath:  "metadata.name", }, }, { Path: "cpu_limit", ResourceFieldRef: &v1.ResourceFieldSelector{ ContainerName: "client-container", Resource:      "limits.cpu", }, }, { Path: "cpu_request", ResourceFieldRef: &v1.ResourceFieldSelector{ ContainerName: "client-container", Resource:      "requests.cpu", }, }, { Path: "memory_limit", ResourceFieldRef: &v1.ResourceFieldSelector{ ContainerName: "client-container", Resource:      "limits.memory", }, }, { Path: "memory_request", ResourceFieldRef: &v1.ResourceFieldSelector{ ContainerName: "client-container", Resource:      "requests.memory", }, }, }, }, }, }, }, }, }, }, RestartPolicy: v1.RestartPolicyNever, }, } return pod } func applyLabelsAndAnnotationsToProjectedDownwardAPIPod(labels, annotations map[string]string, pod *v1.Pod) { if len(labels) > 0 { pod.Spec.Volumes[0].VolumeSource.Projected.Sources[0].DownwardAPI.Items = append(pod.Spec.Volumes[0].VolumeSource.Projected.Sources[0].DownwardAPI.Items, v1.DownwardAPIVolumeFile{ Path: "labels", FieldRef: &v1.ObjectFieldSelector{ APIVersion: "v1", FieldPath:  "metadata.labels", }, }) } if len(annotations) > 0 { pod.Spec.Volumes[0].VolumeSource.Projected.Sources[0].DownwardAPI.Items = append(pod.Spec.Volumes[0].VolumeSource.Projected.Sources[0].DownwardAPI.Items, v1.DownwardAPIVolumeFile{ Path: "annotations", FieldRef: &v1.ObjectFieldSelector{ APIVersion: "v1", FieldPath:  "metadata.annotations", }, }) } } func projectedAllVolumeBasePod(podName string, secretName string, configMapName string, labels, annotations map[string]string) *v1.Pod { pod := &v1.Pod{ ObjectMeta: metav1.ObjectMeta{ Name:        podName, Labels:      labels, Annotations: annotations, }, Spec: v1.PodSpec{ Volumes: []v1.Volume{ { Name: "podinfo", VolumeSource: v1.VolumeSource{ Projected: &v1.ProjectedVolumeSource{ Sources: []v1.VolumeProjection{ { DownwardAPI: &v1.DownwardAPIProjection{ Items: []v1.DownwardAPIVolumeFile{ { Path: "podname", FieldRef: &v1.ObjectFieldSelector{ APIVersion: "v1", FieldPath:  "metadata.name", }, }, }, }, }, { Secret: &v1.SecretProjection{ LocalObjectReference: v1.LocalObjectReference{ Name: secretName, }, }, }, { ConfigMap: &v1.ConfigMapProjection{ LocalObjectReference: v1.LocalObjectReference{ Name: configMapName, }, }, }, }, }, }, }, }, RestartPolicy: v1.RestartPolicyNever, }, } return pod } </del>

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . If the matter is security related, please disclose it privately via --Is this a BUG REPORT or FEATURE REQUEST?: What happened: Running the non- conformance tests with produces unpredictable results, sometimes tests show up again with a on the end. This is due to the following tests having multiple tests with identical names: - - What you expected to happen: Tests should have unique names. How to reproduce it (as minimally and precisely as possible): run the conformance tests with a skip on and Anything else we need to know?: example here: https://k8s- /sig node /sig testing cc
er this is probably better /sig storage cc
/assign /assign
/area conformance
/milestone v1.13


Question: # Finds the master PD device find-master-pd() { <ins> if ( grep "/mnt/master-pd" /proc/mounts ); then echo "Master PD already mounted; won't remount" MASTER_PD_DEVICE="" return fi </ins> echo "Waiting for master pd to be attached" attempt=0 while true; do

We currently remount the master PD every boot, as part of the GCE instance-boot script. I'm wondering if this is safe: if we have services that start concurrently, should we put the master-pd into /etc/fstab instead? I think we'll have to do this on AWS, in any case.
any thoughts on whether we should also do this for GCE?
too for I don't pretend to understand the startup dance of the various master components like etcd, kube*, etc., though it does seem reasonable to add it to /fstab to me. I'm also interested in getting us to something other than containervm like trusty and friends and getting rid of as much of the instance-boot dance as possible.
helped build the master pd dance and may have some input as well.
Actually did the initial work there. The main problem is that GCE PDs (I don't know if this is different in AWS?) are unformatted, so at least on cluster creation, we have to come up, format and mount them. I don't think we can easily move it inside until after first boot.
If I understand correctly, new EBS volumes also start unformatted, and it looks as though something like would still let us automatically format the disk on initial startup because the fstab entry wouldn't exist on the first boot.
Although if there is some sort of race among services starting up on boot, we'll still hit that problem on the initial boot, and on the initial boot of each recreated master VM if we tear it down and bring it back up with the same disk/IP for upgrades.
ping - someone who understands this issue (not me) please confirm doing this for GCE is a 1.2 blocker, or else move it to next-candidate or remove milestone tag completely and demote to P2.
I wouldn't consider it a 1.2 blocker to do this for GCE -
I don't either. I'd actually rather not touch this path right now; we've lived with it this way for a while.
OK well we can see how it goes on AWS :-)
/sig cluster-lifecycle
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. Prevent issues from auto-closing with an comment.

If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/close


Question: // Prints summary metrics for request types with latency above threshold // and returns number of such request types. func HighLatencyRequests(c *client.Client, threshold time.Duration, ignoredResources util.StringSet) (int, error) { <ins> ignoredVerbs := util.NewStringSet("WATCHLIST", "PROXY") </ins> metrics, err := ReadLatencyMetrics(c) if err != nil { return 0, err

I saw the below failure in [Performance suite] should allow starting 30 pods per node. Some requests with verb:PROXY took more than 10 seconds, and are causing the test to fail. I think that proxy issues are a different test, right? (maybe testMonitoringUsingHeapsterInfluxdb?) And that this taking 10 seconds could be very reasonable, depending on what we're proxying. There is this note in If that is the case, sounds like we should either clear the metrics, or if that is hard, record them before we run our test and ignore the pre-existing ones.
Yeah we should clearly exclude proxy from those metrics. This test is still being actively worked on :)
+1 for excluding proxy metrics This should be very simple fix, because we're already exluding all "watches" from the metrics. Reassigning it to me - will fix it today.
is sent out for review.
thanks for fixing!


Question: // otherwise we leave it as it is. if savedCondition.LastTransitionTime != observedCondition.LastTransitionTime { glog.V(3).Infof("ReadyCondition for Node %s transitioned from %v to %v", node.Name, savedCondition.Status, observedCondition) transitionTime = nc.now() } else { transitionTime = savedNodeStatus.readyTransitionTimestamp

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: If node is unreachable, NodeController only set , , to unknown; it's better to also set NetworkUnavailable & InodePressure to unknown.
There are no sig labels on this issue. Please by: a sig: e.g., to notify the contributor experience sig, OR the label manually: e.g., to apply the label Note: Method 1 will trigger an email to the group. You can find the group list and label list . The in the method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals
/sig node
Isn't a known condition if node is unreachable?
is the status of container, e.g. CNI is not ready. If node is unreachable, it should be Unknown.
Thanks for explanation!
will not set NetworkUnavailable & InodePressure to unknown, refer to .


Question: if containers == nil || len(containers) != 1 || containers[0].Image != nginxImage { framework.Failf("Failed creating rc %s for 1 pod with expected image %s", rcName, nginxImage) } <ins> framework.WaitForRCToStabilize(c, ns, rcName, framework.PodStartTimeout) </ins> By("rolling-update to same image controller") framework.RunKubectlOrDie("rolling-update", rcName, "--update-period=1s", "--image="+nginxImage, "--image-pull-policy="+string(api.PullIfNotPresent), nsFlag)

The desired behavior is: Create spec.Replica pods As pods keep coming down watch, update status.Replicas, so users have pseudo real-time feedback If during this time someone submits a new update to spec.Replicas that bumps up the generation, dont update status.Generation to match Update Generation when pending creates/deletes are observed Without the third check, we could confuse a client who quicky scales up/down into thinking the current is for their spec (because spec will match status, but we still have pending creates/deletes). Observed in: Where we see: Because kubectl rolling-update deleted the rc soon after creation.
See discussion in Un-assigning myself since immediate issue of test flake is solved, and we need to think through how we're going to expose generation.
Issues go stale after 30d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. Prevent issues from auto-closing with an comment. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten /remove-lifecycle stale
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-testing, kubernetes/test-infra and/or . /close


Question: apiPass:    defaultAPIPassword, apiVersion: defaultAPIVersion, } <del> klog.V(4).Infof("Using default StorageOS API settings: addr %s, version: %s", apiCfg.apiAddr, defaultAPIVersion) </del> <ins> klog.V(4).Infof("using default StorageOS API settings: addr %s, version: %s", apiCfg.apiAddr, defaultAPIVersion) </ins> } api, err := storageosapi.NewVersionedClient(apiCfg.apiAddr, defaultAPIVersion)

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: When a PVC is created and the PV mounted into a Pod using the StorageOS in-tree volume driver, the StorageOS CLI & UI do not report that the volume is mounted. The PV is created and mounted into the Pod successfully, and all k8s functionality works as expected. However, since the correct mount status is not visible to administrators using the StorageOS UI or CLI, they may make changes to or delete a mounted volume without realising, and the safeguards (must use to delete a mounted volume) do not trigger. This is a regression caused by , released in 1.14.0. What you expected to happen: The StorageOS CLI should report volume mount information (node name, mount timestamp). How to reproduce it (as minimally and precisely as possible): StorageOS node containers with CSI disabled: application Pod with a PVC using the StorageOS StorageClass: StorageOS to list volumes: Anything else we need to know?: Environment: Kubernetes version (use ): 1.14.0 and later Cloud provider or hardware configuration: any OS (e.g: ): any linux (incl. Ubuntu 18.04) Kernel (e.g. ): any linux (incl. 4.15.0-33-generic) Install tools: any (incl. minikube) Network plugin and version (if this is a network-related bug): N/A Others: Requires StorageOS deployment /sig storage /assign


Question: maskBits := mask.GetBits() singleNUMAHint := len(maskBits) == 1 <del> // the node already in group with another node, it can not be used for the single NUMA node allocation if singleNUMAHint && len(machineState[maskBits[0]].Cells) > 1 { return } </del> totalFreeSize := map[v1.ResourceName]uint64{} totalAllocatableSize := map[v1.ResourceName]uint64{} <del> // calculate total free memory for the node mask </del> <ins> // calculate total free and allocatable memory for the node mask </ins> for _, nodeID := range maskBits { <del> // the node already used for the memory allocation if !singleNUMAHint && machineState[nodeID].NumberOfAssignments > 0 { // the node used for the single NUMA memory allocation, it can not be used for the multi NUMA node allocation if len(machineState[nodeID].Cells) == 1 { return } // the node already used with different group of nodes, it can not be use with in the current hint if !areGroupsEqual(machineState[nodeID].Cells, maskBits) { return } } </del> for resourceName := range requestedResources { if _, ok := totalFreeSize[resourceName]; !ok { totalFreeSize[resourceName] = 0

Setup: 3 Master nodes and 2 Worker Nodes. Version: v1.22.0-rc.0+ Memory Manager on Kubernetes Version (v1.22.0-rc.0+) and Nodes configured with Hugepages of size 2M on 2 Worker nodes. Worker node having 2 Numa Nodes (numa node0 and numa node1) of Hugepages of size 2M are 10 on each Numa node sh-4.4# cat /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nrhugepages 10 sh-4.4# cat /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nrhugepages 10 After configuring Memory , where Hugepages are 20M (2M * 10) on each Numa node. we create QoS pod1 pod which consumes 24Mi Hugepages (of size 2M) Pod Spec: List pods Memory Manager state file: Now create a QoS Guaranteed Pod2 requesting 16Mi . On the same worker node. Pod spec: List pods: Memory Manager state file after pod2 is deployed. Pod2 should be rejected. Steps provided above Kubernetes version (use ): v1.22.0-rc.0+ Cloud provider or hardware configuration: Openshift OS (e.g: ):Red Hat Enterprise Linux CoreOS release 4.9 Kernel (e.g. ): Linux 4.18.0-305.12.1.rt7.84.el84.x8664 SMP PREEMPTRT Thu Jul 29 14:18:12 EDT 2021 x8664 x8664 x8664 GNU/Linux Kubelet configuration
/triage accepted /priority important-longterm /sig node /area kubelet
/assign


Question: $fluentbit_parser_file = "$LOGGINGAGENT_ROOTconfparsers.conf" $PARSERS_CONFIG | Out-File -FilePath $fluentbit_parser_file -Encoding ASCII <ins> # Create directory for all the log position files. New-Item -Type Directory -Path "/var/run/google-fluentbit/pos-files/" </ins> Log-Output "Wrote logging config to $fluentbit_parser_file" }

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --The default fluent-bit configuration for GCE Windows nodes produces unparsed JSON logs and does not forward container logs. Container log output is produced. fluent-bit on a given windows image. a GCE Windows e2e cluster and view the stackdriver output for pods. Kubernetes version (use ): 1.20+ Cloud provider or hardware configuration: GKE/GCE OS (e.g: ): Windows Kernel (e.g. ): Any version Install tools: fluent-bit Network plugin and version (if this is a network-related bug): Others:
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig windows