Question: inkscape:groupmode="layer" id="layer1" transform="translate(-6.1051208,-305.36552)"> <del> <text xml:space="preserve" </del> <ins> <g </ins> style="font-size:40px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#ffffff;fill-opacity:1;stroke:none;font-family:Sans" <del> x="6.6551208" y="420.12552" id="text4373" sodipodi:linespacing="125%"><tspan sodipodi:role="line" id="tspan4375" x="6.6551208" y="420.12552" style="font-size:135px;font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;fill:#ffffff;fill-opacity:1;font-family:Ubuntu;-inkscape-font-specification:Ubuntu Bold">kubernetes</tspan></text> </del> <ins> id="text4373"> <path d="m 36.220121,374.76552 c 1.979968,-2.15995 4.004966,-4.40995 6.075,-6.75 2.159962,-2.33995 4.22996,-4.63494 6.21,-6.885 1.979956,-2.33994 3.824954,-4.54494 5.535,-6.615 1.799951,-2.06993 3.329949,-3.86993 4.59,-5.4 l 23.895,0 c -4.770071,5.49006 -9.450067,10.75506 -14.04,15.795 -4.500057,4.95005 -9.450053,10.08004 -14.85,15.39 2.69995,2.43004 5.489947,5.35503 8.37,8.775 2.879942,3.33003 5.669939,6.79502 8.37,10.395 2.699933,3.60002 5.174931,7.20001 7.425,10.8 2.249926,3.60001 4.139925,6.885 5.67,9.855 l -23.085,0 c -1.440052,-2.34 -3.105051,-4.905 -4.995,-7.695 -1.800047,-2.87999 -3.735045,-5.71499 -5.805,-8.505 -2.070041,-2.87998 -4.275039,-5.62498 -6.615,-8.235 -2.250034,-2.60997 -4.500032,-4.81497 -6.75,-6.615 l 0,31.05 -20.115,0 0,-101.52 20.115,-3.24 0,59.4" style="font-size:135px;font-variant:normal;font-weight:bold;font-stretch:normal;fill:#ffffff;font-family:Ubuntu;-inkscape-font-specification:Ubuntu Bold" id="path2985" /> <path d="m 153.44864,417.69552 c -3.42007,0.99 -7.83007,1.89 -13.23,2.7 -5.40005,0.9 -11.07005,1.35 -17.01,1.35 -6.03004,0 -11.07003,-0.81 -15.12,-2.43 -3.96002,-1.62 -7.11002,-3.87 -9.450004,-6.75 -2.340012,-2.96999 -4.005011,-6.47999 -4.995,-10.53 -0.990009,-4.04998 -1.485008,-8.50497 -1.485,-13.365 l 0,-39.555 20.115004,0 0,37.125 c -3e-5,6.48003 0.85497,11.16002 2.565,14.04 1.70996,2.88002 4.90496,4.32001 9.585,4.32 1.43995,10e-6 2.96995,-0.045 4.59,-0.135 1.61995,-0.17999 3.05995,-0.35999 4.32,-0.54 l 0,-54.81 20.115,0 0,68.58" style="font-size:135px;font-variant:normal;font-weight:bold;font-stretch:normal;fill:#ffffff;font-family:Ubuntu;-inkscape-font-specification:Ubuntu Bold" id="path2987" /> <path d="m 217.95754,384.08052 c -5e-5,-12.95995 -4.77005,-19.43995 -14.31,-19.44 -2.07004,5e-5 -4.14003,0.27005 -6.21,0.81 -1.98003,0.54005 -3.60003,1.21505 -4.86,2.025 l 0,36.585 c 0.98997,0.18001 2.24997,0.36001 3.78,0.54 1.52997,0.09 3.19497,0.13501 4.995,0.135 5.48996,10e-6 9.62995,-1.88998 12.42,-5.67 2.78995,-3.77998 4.18495,-8.77497 4.185,-14.985 m 20.52,0.54 c -7e-5,5.67003 -0.85507,10.80002 -2.565,15.39 -1.62007,4.59001 -4.00507,8.50501 -7.155,11.745 -3.15006,3.24 -7.02006,5.76 -11.61,7.56 -4.59005,1.71 -9.81004,2.565 -15.66,2.565 -2.43003,0 -4.99503,-0.135 -7.695,-0.405 -2.61003,-0.18 -5.22002,-0.45 -7.83,-0.81 -2.52002,-0.36 -4.95002,-0.765 -7.29,-1.215 -2.34001,-0.54 -4.41001,-1.08 -6.21,-1.62 l 0,-99.225 20.115,-3.24 0,35.37 c 2.24997,-0.98993 4.58997,-1.75493 7.02,-2.295 2.42996,-0.53993 5.03996,-0.80993 7.83,-0.81 5.03995,7e-5 9.49495,0.90007 13.365,2.7 3.86994,1.71007 7.10994,4.18506 9.72,7.425 2.60993,3.24006 4.58993,7.15505 5.94,11.745 1.34993,4.50004 2.02493,9.54004 2.025,15.12" style="font-size:135px;font-variant:normal;font-weight:bold;font-stretch:normal;fill:#ffffff;font-family:Ubuntu;-inkscape-font-specification:Ubuntu Bold" id="path2989" /> <path d="m 250.56215,385.16052 c 0,-6.29996 0.945,-11.78995 2.835,-16.47 1.97999,-4.76994 4.54499,-8.72994 7.695,-11.88 3.14998,-3.14993 6.74998,-5.53493 10.8,-7.155 4.13997,-1.61993 8.36997,-2.42993 12.69,-2.43 10.07995,7e-5 18.04494,3.10507 23.895,9.315 5.84993,6.12006 8.77493,15.16505 8.775,27.135 -7e-5,1.17003 -0.0451,2.47503 -0.135,3.915 -0.0901,1.35003 -0.18007,2.56503 -0.27,3.645 l -45.63,0 c 0.44997,4.14002 2.38497,7.42502 5.805,9.855 3.41997,2.43002 8.00996,3.64501 13.77,3.645 3.68995,10e-6 7.28995,-0.31499 10.8,-0.945 3.59994,-0.71998 6.52494,-1.57498 8.775,-2.565 l 2.7,16.335 c -1.08007,0.54 -2.52006,1.08 -4.32,1.62 -1.80006,0.54 -3.82506,0.99 -6.075,1.35 -2.16005,0.45 -4.50005,0.81 -7.02,1.08 -2.52005,0.27 -5.04004,0.405 -7.56,0.405 -6.39004,0 -11.97003,-0.945 -16.74,-2.835 -4.68002,-1.89 -8.59502,-4.455 -11.745,-7.695 -3.06001,-3.32999 -5.35501,-7.24499 -6.885,-11.745 -1.44,-4.49998 -2.16,-9.35997 -2.16,-14.58 m 47.25,-7.695 c -0.09,-1.70996 -0.40505,-3.37496 -0.945,-4.995 -0.45005,-1.61995 -1.21505,-3.05995 -2.295,-4.32 -0.99005,-1.25995 -2.29505,-2.29495 -3.915,-3.105 -1.53004,-0.80995 -3.46504,-1.21494 -5.805,-1.215 -2.25004,6e-5 -4.18503,0.40505 -5.805,1.215 -1.62003,0.72005 -2.97003,1.71005 -4.05,2.97 -1.08003,1.26005 -1.93503,2.74505 -2.565,4.455 -0.54003,1.62004 -0.94502,3.28504 -1.215,4.995 l 26.595,0" style="font-size:135px;font-variant:normal;font-weight:bold;font-stretch:normal;fill:#ffffff;font-family:Ubuntu;-inkscape-font-specification:Ubuntu Bold" id="path2991" /> <path d="m 375.57004,366.93552 c -1.80005,-0.44995 -3.91505,-0.89995 -6.345,-1.35 -2.43004,-0.53995 -5.04004,-0.80995 -7.83,-0.81 -1.26003,5e-5 -2.79003,0.13505 -4.59,0.405 -1.71003,0.18005 -3.01503,0.40505 -3.915,0.675 l 0,54.27 -20.115,0 0,-67.23 c 3.59999,-1.25993 7.82999,-2.42993 12.69,-3.51 4.94998,-1.16993 10.43997,-1.75493 16.47,-1.755 1.07996,7e-5 2.38496,0.0901 3.915,0.27 1.52996,0.0901 3.05996,0.27007 4.59,0.54 1.52995,0.18007 3.05995,0.45007 4.59,0.81 1.52995,0.27007 2.83495,0.63007 3.915,1.08 l -3.375,16.605" style="font-size:135px;font-variant:normal;font-weight:bold;font-stretch:normal;fill:#ffffff;font-family:Ubuntu;-inkscape-font-specification:Ubuntu Bold" id="path2993" /> <path d="m 389.72817,351.54552 c 3.41998,-0.98993 7.82998,-1.88993 13.23,-2.7 5.39997,-0.89993 11.06996,-1.34993 17.01,-1.35 6.02995,7e-5 11.02495,0.81007 14.985,2.43 4.04994,1.53007 7.24494,3.73507 9.585,6.615 2.33993,2.88006 4.00493,6.30006 4.995,10.26 0.98993,3.96005 1.48493,8.37004 1.485,13.23 l 0,40.095 -20.115,0 0,-37.665 c -5e-5,-6.47996 -0.85505,-11.06995 -2.565,-13.77 -1.71005,-2.69995 -4.90505,-4.04995 -9.585,-4.05 -1.44004,5e-5 -2.97004,0.0901 -4.59,0.27 -1.62003,0.09 -3.06003,0.22505 -4.32,0.405 l 0,54.81 -20.115,0 0,-68.58" style="font-size:135px;font-variant:normal;font-weight:bold;font-stretch:normal;fill:#ffffff;font-family:Ubuntu;-inkscape-font-specification:Ubuntu Bold" id="path2995" /> <path d="m 465.98207,385.16052 c 0,-6.29996 0.945,-11.78995 2.835,-16.47 1.97999,-4.76994 4.54499,-8.72994 7.695,-11.88 3.14998,-3.14993 6.74998,-5.53493 10.8,-7.155 4.13997,-1.61993 8.36997,-2.42993 12.69,-2.43 10.07995,7e-5 18.04495,3.10507 23.895,9.315 5.84993,6.12006 8.77493,15.16505 8.775,27.135 -7e-5,1.17003 -0.0451,2.47503 -0.135,3.915 -0.0901,1.35003 -0.18007,2.56503 -0.27,3.645 l -45.63,0 c 0.44998,4.14002 2.38497,7.42502 5.805,9.855 3.41997,2.43002 8.00996,3.64501 13.77,3.645 3.68995,10e-6 7.28995,-0.31499 10.8,-0.945 3.59994,-0.71998 6.52494,-1.57498 8.775,-2.565 l 2.7,16.335 c -1.08006,0.54 -2.52006,1.08 -4.32,1.62 -1.80006,0.54 -3.82506,0.99 -6.075,1.35 -2.16005,0.45 -4.50005,0.81 -7.02,1.08 -2.52004,0.27 -5.04004,0.405 -7.56,0.405 -6.39003,0 -11.97003,-0.945 -16.74,-2.835 -4.68002,-1.89 -8.59501,-4.455 -11.745,-7.695 -3.06001,-3.32999 -5.35501,-7.24499 -6.885,-11.745 -1.44,-4.49998 -2.16,-9.35997 -2.16,-14.58 m 47.25,-7.695 c -0.0901,-1.70996 -0.40505,-3.37496 -0.945,-4.995 -0.45005,-1.61995 -1.21505,-3.05995 -2.295,-4.32 -0.99005,-1.25995 -2.29504,-2.29495 -3.915,-3.105 -1.53004,-0.80995 -3.46504,-1.21494 -5.805,-1.215 -2.25003,6e-5 -4.18503,0.40505 -5.805,1.215 -1.62003,0.72005 -2.97003,1.71005 -4.05,2.97 -1.08003,1.26005 -1.93502,2.74505 -2.565,4.455 -0.54002,1.62004 -0.94502,3.28504 -1.215,4.995 l 26.595,0" style="font-size:135px;font-variant:normal;font-weight:bold;font-stretch:normal;fill:#ffffff;font-family:Ubuntu;-inkscape-font-specification:Ubuntu Bold" id="path2997" /> <path d="m 547.51996,331.43052 20.115,-3.24 0,20.925 24.165,0 0,16.74 -24.165,0 0,24.975 c -2e-5,4.23002 0.71997,7.60502 2.16,10.125 1.52997,2.52002 4.54497,3.78001 9.045,3.78 2.15996,10e-6 4.36496,-0.17999 6.615,-0.54 2.33996,-0.44998 4.45495,-1.03498 6.345,-1.755 l 2.835,15.66 c -2.43005,0.99 -5.13005,1.845 -8.1,2.565 -2.97004,0.72 -6.61504,1.08 -10.935,1.08 -5.49003,0 -10.03502,-0.72 -13.635,-2.16 -3.60002,-1.53 -6.48001,-3.6 -8.64,-6.21 -2.16001,-2.69999 -3.69001,-5.93999 -4.59,-9.72 -0.81,-3.77998 -1.215,-7.96498 -1.215,-12.555 l 0,-59.67" style="font-size:135px;font-variant:normal;font-weight:bold;font-stretch:normal;fill:#ffffff;font-family:Ubuntu;-inkscape-font-specification:Ubuntu Bold" id="path2999" /> <path d="m 602.03676,385.16052 c 0,-6.29996 0.94499,-11.78995 2.835,-16.47 1.97999,-4.76994 4.54499,-8.72994 7.695,-11.88 3.14998,-3.14993 6.74998,-5.53493 10.8,-7.155 4.13997,-1.61993 8.36997,-2.42993 12.69,-2.43 10.07995,7e-5 18.04494,3.10507 23.895,9.315 5.84993,6.12006 8.77493,15.16505 8.775,27.135 -7e-5,1.17003 -0.0451,2.47503 -0.135,3.915 -0.0901,1.35003 -0.18007,2.56503 -0.27,3.645 l -45.63,0 c 0.44997,4.14002 2.38497,7.42502 5.805,9.855 3.41997,2.43002 8.00996,3.64501 13.77,3.645 3.68995,10e-6 7.28995,-0.31499 10.8,-0.945 3.59994,-0.71998 6.52494,-1.57498 8.775,-2.565 l 2.7,16.335 c -1.08007,0.54 -2.52006,1.08 -4.32,1.62 -1.80006,0.54 -3.82506,0.99 -6.075,1.35 -2.16005,0.45 -4.50005,0.81 -7.02,1.08 -2.52005,0.27 -5.04004,0.405 -7.56,0.405 -6.39004,0 -11.97003,-0.945 -16.74,-2.835 -4.68002,-1.89 -8.59502,-4.455 -11.745,-7.695 -3.06001,-3.32999 -5.35501,-7.24499 -6.885,-11.745 -1.44001,-4.49998 -2.16,-9.35997 -2.16,-14.58 m 47.25,-7.695 c -0.09,-1.70996 -0.40505,-3.37496 -0.945,-4.995 -0.45005,-1.61995 -1.21505,-3.05995 -2.295,-4.32 -0.99005,-1.25995 -2.29505,-2.29495 -3.915,-3.105 -1.53004,-0.80995 -3.46504,-1.21494 -5.805,-1.215 -2.25004,6e-5 -4.18503,0.40505 -5.805,1.215 -1.62003,0.72005 -2.97003,1.71005 -4.05,2.97 -1.08003,1.26005 -1.93503,2.74505 -2.565,4.455 -0.54003,1.62004 -0.94503,3.28504 -1.215,4.995 l 26.595,0" style="font-size:135px;font-variant:normal;font-weight:bold;font-stretch:normal;fill:#ffffff;font-family:Ubuntu;-inkscape-font-specification:Ubuntu Bold" id="path3001" /> <path d="m 704.63465,405.68052 c 3.68997,10e-6 6.29997,-0.35999 7.83,-1.08 1.52996,-0.71998 2.29496,-2.11498 2.295,-4.185 -4e-5,-1.61998 -0.99004,-3.01498 -2.97,-4.185 -1.98003,-1.25998 -4.99503,-2.65497 -9.045,-4.185 -3.15002,-1.16997 -6.03002,-2.38497 -8.64,-3.645 -2.52001,-1.25997 -4.68001,-2.74497 -6.48,-4.455 -1.80001,-1.79996 -3.19501,-3.91496 -4.185,-6.345 -0.99001,-2.42996 -1.48501,-5.35495 -1.485,-8.775 -10e-6,-6.65994 2.47499,-11.92494 7.425,-15.795 4.94998,-3.86993 11.74498,-5.80493 20.385,-5.805 4.31996,7e-5 8.45996,0.40507 12.42,1.215 3.95995,0.72007 7.10995,1.53007 9.45,2.43 l -3.51,15.66 c -2.34005,-0.80995 -4.90505,-1.52995 -7.695,-2.16 -2.70004,-0.62994 -5.76004,-0.94494 -9.18,-0.945 -6.30003,6e-5 -9.45002,1.75505 -9.45,5.265 -2e-5,0.81005 0.13497,1.53005 0.405,2.16 0.26997,0.63005 0.80997,1.26005 1.62,1.89 0.80997,0.54005 1.88997,1.17005 3.24,1.89 1.43997,0.63004 3.23997,1.35004 5.4,2.16 4.40996,1.62004 8.05496,3.24004 10.935,4.86 2.87995,1.53004 5.12995,3.24003 6.75,5.13 1.70995,1.80003 2.87994,3.82503 3.51,6.075 0.71994,2.25002 1.07994,4.86002 1.08,7.83 -6e-5,7.02001 -2.65506,12.33001 -7.965,15.93 -5.22004,3.6 -12.64504,5.4 -22.275,5.4 -6.30002,0 -11.56502,-0.54 -15.795,-1.62 -4.14001,-1.08 -7.02,-1.98 -8.64,-2.7 l 3.375,-16.335 c 3.41999,1.35002 6.92999,2.43002 10.53,3.24 3.59998,0.72001 7.15498,1.08001 10.665,1.08" style="font-size:135px;font-variant:normal;font-weight:bold;font-stretch:normal;fill:#ffffff;font-family:Ubuntu;-inkscape-font-specification:Ubuntu Bold" id="path3003" /> </g> </ins> </g> </svg>

The logo (, ) font does not match the others. I believe the png/pdf versions are correct.
What other ones? It should be the same font - they look the same to me...


Question: ipv6Primary := kl.nodeIP != nil && utilnet.IsIPv6(kl.nodeIP) var iptClients []utiliptables.Interface <ins> var protocols []utiliptables.Protocol </ins> if maybeDualStack || !ipv6Primary { <ins> protocols = append(protocols, utiliptables.ProtocolIPv4) </ins> iptClients = append(iptClients, utiliptables.New(exec, utiliptables.ProtocolIPv4)) } if maybeDualStack || ipv6Primary { <ins> protocols = append(protocols, utiliptables.ProtocolIPv6) </ins> iptClients = append(iptClients, utiliptables.New(exec, utiliptables.ProtocolIPv6)) } <del> for _, iptClient := range iptClients { kl.syncNetworkUtil(iptClient) go iptClient.Monitor( utiliptables.Chain("KUBE-KUBELET-CANARY"), []utiliptables.Table{utiliptables.TableMangle, utiliptables.TableNAT, utiliptables.TableFilter}, func() { kl.syncNetworkUtil(iptClient) }, 1*time.Minute, wait.NeverStop, ) </del> <ins> for i := range iptClients { iptClient := iptClients[i] if kl.syncNetworkUtil(iptClient) { klog.Infof("Initialized %s iptables rules.", protocols[i]) go iptClient.Monitor( utiliptables.Chain("KUBE-KUBELET-CANARY"), []utiliptables.Table{utiliptables.TableMangle, utiliptables.TableNAT, utiliptables.TableFilter}, func() { kl.syncNetworkUtil(iptClient) }, 1*time.Minute, wait.NeverStop, ) } else { klog.Warningf("Failed to initialize %s iptables rules; some functionality may be missing.", protocols[i]) } </ins> } }

<!-- Please only use this template for submitting reports about continuously failing tests or jobs in Kubernetes CI --Which jobs are failing: ci-kubernetes-e2e-gci-gce-alpha-features Which test(s) are failing: Networking should recreate its iptables rules if they are deleted [Disruptive] Since when has it been failing: Commit Testgrid link: Reason for failure: Anything else we need to know: Possibly related to this? It's listed in the git diff /sig network
/assign /cc Thanks for detecting it so quicklly
most probably is related to
/unassign /assign he has already submitted a fix
/triage accepted
Reviewed. Marking as resolved on the CI Signal Board. Once was merged, failures stopped and they have not re-occurred in the last 8 days (Oct 10th)
Yay


Question: var containers []corev1.Container ctn := corev1.Container{ Name:  fmt.Sprintf("%s-ctn", w.Name()), <del> Image: "gcr.io/kubernetes-e2e-test-images/node-perf/npb-ep-amd64:1.0", </del> <ins> Image: "gcr.io/kubernetes-e2e-test-images/node-perf/npb-ep:1.0", </ins> Resources: corev1.ResourceRequirements{ Requests: corev1.ResourceList{ corev1.ResourceName(corev1.ResourceCPU):    resource.MustParse("15000m"),

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!--What happened: test/e2enode/perf tests are referring to adm only images, this need to be replaced with multi-arch fat manifest images. ref:
/assign /sig node


Question: volumeSpec *volume.Spec, uniquePodName volumetypes.UniquePodName, mountedVolumesForPod map[volumetypes.UniquePodName]map[string]cache.MountedVolume) { <del> if podVolume.PersistentVolumeClaim == nil { </del> <ins> if podVolume.PersistentVolumeClaim == nil || pvc == nil { </ins> // Only PVC supports resize operation. return }

The following test failures started happening on 30th March: Kubernetes e2e suite.[sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Inline-volume (default fs)] volumes should allow exec of files on the volume Kubernetes e2e suite.[sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Inline-volume (default fs)] volumes should store data Kubernetes e2e suite.[sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Inline-volume (ext4)] volumes should allow exec of files on the volume Kubernetes e2e suite.[sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Inline-volume (ext4)] volumes should store data See This seems related to the changes in kubelet fails with the following: Tests passing See the e2e logs above No response <detailsLatest CI build </details<detailsAWS </details<details</details<details</details<details</details<detailsEBS CSI driver 1.5.0 </details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/cc
/sig storage
/assign
not having debugging kops builds for awhile, are kubelet logs attached in prow? I don't see them...
Spoke with offline. It looks like we don't have kubelet logs captured in these tests. I need complete kubelet logs to make sense of these errors because testgrid shows inline volume tests as failing and code in question should not be called or inline volumes (because they can not be expanded). If that code fails, it would have been failing before too. I will wait for logs from and in the meanwhile try and see if I can reproduce these failures in single node AWS cluster setup that I can build manually.
thanks
Looks like this is happening for inline volumes if CSI migration is enabled for them. We construct a "fake" object on the fly for migration enabled plugins even though volume is inline, which is causing this condition to fail. It should be simple fix.
opened to fix the problem. The code change is just one line but has tests.


Question: fi APISERVER_LOG=${LOG_DIR}/kube-apiserver.log <del> ${CONTROLPLANE_SUDO} "${GO_OUT}/hyperkube" apiserver "${authorizer_arg}" "${priv_arg}" ${runtime_config}  </del> <ins> ${CONTROLPLANE_SUDO} "${GO_OUT}/hyperkube" kube-apiserver "${authorizer_arg}" "${priv_arg}" ${runtime_config}  </ins> ${cloud_config_arg}  "${advertise_address}"  "${node_port_range}"

Hyperkube supports a number of short aliases for commands, such as , , , , etc, rather than the longer , , , etc. There are comments indicating this is "unfortunate" and we still currently create these aliases in the hyperkube docker image: It'd be nice to simplify this, assuming we can find all uses of the short aliases and remove them. /kind cleanup Not sure which sig should own this, so tagging a few. /sig architecture cluster-lifecycle cli cc
interestingly, the command doesn't create the short aliases, though we also don't really use anywhere either.
I think we can just go ahead and start the deprecation cycle for this. WDYT? cc
cc as well
+1 to nuke it
/assign :wink: ok?
SGTM
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
/remove-lifecycle rotten
/help
This request has been marked as needing help from a contributor. Please ensure the request meets the requirements listed . If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : /help Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I would like to work on it!
It seems like this issue was already solved in ?
I don't think the hyperkube code was changed in that PR, only the image?
Yes, thanks for your comment, can i try to fix it ?
I made a PR here: . Review/feedback/suggestions appreciated! :)
/assign /unassign
GitHub didn't allow me to assign the following users: Rand01ph.

Note that only and repo collaborators can be assigned and that issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : /assign /unassign Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/remove-sig architecture cli /sig release Does anyone even still use hyperkube? Wouldn't they use kubeadm or k3s instead?
AFAIK, SAP (the company) still use hyperkube.
aks-engine deployer ( ) still uses hyperkube
Since this ticket is completed now. Can this todo be removed?
we are removing the whole thing :)


Question: "fmt" api "k8s.io/api/core/v1" <del> extensions "k8s.io/api/extensions/v1beta1" </del> metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" <ins> "k8s.io/apimachinery/pkg/labels" </ins> "k8s.io/kubernetes/pkg/security/apparmor" "k8s.io/kubernetes/test/e2e/framework" <ins> . "github.com/onsi/gomega" </ins> ) const ( appArmorProfilePrefix = "e2e-apparmor-test-" appArmorAllowedPath   = "/expect_allowed_write" appArmorDeniedPath    = "/expect_permission_denied" <ins> loaderLabelKey   = "name" loaderLabelValue = "e2e-apparmor-loader" </ins> ) // AppArmorDistros are distros with AppArmor support

The "AppArmor load AppArmor profiles should enforce an AppArmor profile" e2e test in failing on our 5k-node gce clusters - https://k8s- (run no. 8) due to timeout: Seems like the test is trying to load the profiles across all the nodes using a daemonset and then running a test pod to check that. It's passing on our 2k-node clusters (https://k8s-) but failing for 5k - suggesting that it's likely due to size of the cluster. I think it's the ds creation taking longer on large cluster. cc
Seems like you're the most relevant here. Could one of you take a look into it? And does increasing the timeout for the test-pod based on no. of nodes sound reasonable?
Assigning it you since is OOO. Feel free to reassign as apt.
This is part of the correctness suite and large cluster tests are release-blocking.
FYI
Reassigning to who is back on Tuesday. He wrote the AppArmor stuff and should be able to debug this quickly. /assign
Ref
Could you manage to take a look into this?
Sorry, I meant to reply to this! I have a fix, just need to confirm it works. I think you're right about the DaemonSet timing out. There's no reason this needs to load the profiles on every node, so the fix just runs a single loader, and uses pod affinity to co-schedule the test pod.
Smooth!
[MILESTONENOTIFIER] Milestone Labels Complete Issue label settings: sig/node sig/scalability: Issue will be escalated to these SIGs if needed. priority/important-soon: Escalate to the issue owners and SIG owner; move out of milestone after several unsuccessful escalation attempts. kind/bug: Fixes a bug discovered during the current release. <detailsAdditional instructions available <a href=""here</aThe commands available for adding these labels are documented <a href=""here</a</details


Question: }) ginkgo.It("should work with the pod containing more than 6 DNS search paths and longer than 256 search list characters", func(ctx context.Context) { <del> ginkgo.By("Getting the kube-dns IP") svc, err := f.ClientSet.CoreV1().Services("kube-system").Get(ctx, "kube-dns", metav1.GetOptions{}) framework.ExpectNoError(err, "Failed to get kube-dns service") kubednsIP := svc.Spec.ClusterIP </del> // All the names we need to be able to resolve. namesToResolve := []string{ "kubernetes.default",

an E2E test that assumes that a service named "kube-dns" exists in the "kube-system" namespace. This is a kubeadm-ism, and the test fails if the cluster DNS is not configured the way kubeadm configures it. Although this E2E test isn't a conformance test, making it functional on non-kubeadm clusters would increase its utility for distributors that may wish to run this E2E test as a conformance test. [sig-network] DNS HostNetwork should resolve DNS of partial qualified names for services on hostNetwork pods with dnsPolicy: ClusterFirstWithHostNet Since we ran it on an OpenShift cluster. No response The test makes assumptions about how the cluster DNS is configured. The test uses (not ), so it does not need to specify , as far as I can tell. Omitting obviates the need to look up the service. I believe that the service lookup was left in as an oversight during the development of the test when it was changed from specifying "None" to specifying "ClusterFirst": Cc: /sig network
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
responded on the pr.
It seems kubeadm inherited the status-quo, I can't find any place that mandates the Service name for DNS Asking for double checking, but the docs seems to mention that this is more a recommendation than a preescription If it seems that the test can be done in a more portable way we should do it


Question: const ( // Notice that if someday we also have minimum docker version requirement, this should also be updated. <del> fakeDockerVersion = "1.11.2" </del> <ins> fakeDockerVersion = "1.13.1" </ins> fakeImageSize = 1024

The current node e2e tests include images using Docker 1.11.x to , with the pending PR to test 18.06 () We should update the images to drop older docker versions. I think we can start by getting rid of 1.11 and 1.12, so that we still have three minor versions tested: , , and 1.13.x /cc (for cos and ubuntu)
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/assign Yang, could you take a look? Thanks


Question: } } <del> func TestHostsFileContent(t *testing.T) { </del> <ins> func TestNodeHostsFileContent(t *testing.T) { testCases := []struct { hostsFileName   string expectedContent string }{ { "hosts_test_file1", `# hosts file for testing. 127.0.0.1	localhost ::1	localhost ip6-localhost ip6-loopback fe00::0	ip6-localnet fe00::0	ip6-mcastprefix fe00::1	ip6-allnodes fe00::2	ip6-allrouters 123.45.67.89	some.domain `, }, { "hosts_test_file2", `# another hosts file for testing. 127.0.0.1	localhost ::1	localhost ip6-localhost ip6-loopback fe00::0	ip6-localnet fe00::0	ip6-mcastprefix fe00::1	ip6-allnodes fe00::2	ip6-allrouters 12.34.56.78	another.domain `, }, } for _, testCase := range testCases { tmpdir, err := writeHostsFile(testCase.hostsFileName, testCase.expectedContent) require.NoError(t, err, "could not create a temp hosts file") defer os.RemoveAll(tmpdir) actualContent, fileReadErr := nodeHostsFileContent(filepath.Join(tmpdir, testCase.hostsFileName)) require.NoError(t, fileReadErr, "could not create read hosts file") assert.Equal(t, testCase.expectedContent, string(actualContent), "hosts file content not expected") } } // writeHostsFile will write a hosts file into a temporary dir, and return that dir. // Caller is responsible for deleting the dir and its contents. func writeHostsFile(filename string, cfg string) (string, error) { tmpdir, err := ioutil.TempDir("", "kubelet=kubelet_pods_test.go=") if err != nil { return "", err } return tmpdir, ioutil.WriteFile(filepath.Join(tmpdir, filename), []byte(cfg), 0644) } func TestManagedHostsFileContent(t *testing.T) { </ins> testCases := []struct { hostIP          string hostName        string

Currently, kubelet mounts hosts file for pods. For hostNetwork pods, the behavior is different and instead, kubelet relies on docker (or whatever container runtime) to mount the host's hosts file. We should have kubelet handle hosts file for hostNetwork pods. This would have several benefits: control over hosts file in case the container runtime does something consistent behavior between hostNetwork and non-hostNetwork pods consistent behavior between different container runtimes -managed hosts file enables features like to be used for hostNetwork pods Implementation would require: kubelet's or its caller output different content based on hostNetwork. i. If it's not hostNetwork, use existing behavior. ii. If it's using hostNetwork, read the content from the host's hosts file. hostNetwork constraint in deciding whether to mount the file Open Questions: do we treat changes to the host's hosts file? should the file's content be static between container restarts of the same Pod? or should every container restart pick up the latest content of the hosts file? the former seems better for consistency, but letter might be more in line with other features that use host resources like host volumes. condition to manage hosts file also excludes windows even though windows also the use of hosts file and the same format. seems like this should be supported also. windows-support ships, should it have validation against windows platform? the tricky bit is the client, API, and minions' kubelet may be running on different platforms, and currently, there's no precedence to platform validation. /kind feature /sig node


Question: } } <ins> // While we only use a bounded number of nodes in the test. We need to know // the actual number of nodes in the cluster, to avoid running resourceMonitor // against large clusters. actualNodes, err := e2enode.GetReadySchedulableNodes(c) framework.ExpectNoError(err) </ins> // Start resourceMonitor only in small clusters. <del> if numNodes <= maxNodesToCheck { </del> <ins> if len(actualNodes.Items) <= maxNodesToCheck { </ins> resourceMonitor = e2ekubelet.NewResourceMonitor(f.ClientSet, e2ekubelet.TargetContainers(), containerStatsPollingInterval) resourceMonitor.Start() }

<!-- Please only use this template for submitting reports about failing tests in Kubernetes CI jobs --Which jobs are failing: https://k8s- Which test(s) are failing: [] [sig-node] kubelet [] [sig-node] Clean up pods on node kubelet should be able to delete 10 pods per node in 1m0s. Since when has it been failing: 2019-09-26 Testgrid link: Reason for failure: Anything else we need to know: /cc
is there someone who can pick this up since it's constantly failing in the master-informing? Thanks :)
Could you help us here?
Looked into the test logs, and it seems like the in was accidentally removed in , which caused its value to be 0. As the current log is showing: And last succeeded test log is: I'll send a PR to fix this. /cc
It seems that this PR didn't help. This run already contains this PR and it failed:
/milestone v1.17 /priority critical-urgent
I0926 12:39:42.660] I0926 12:35:26. ] Trace[]: "Reflector ListAndWatch" (started: 2019-09-26 12:33:26. +0000 UTC m=+1132.) (total time: 2m0.000301191s):
Seems like the test is timing out creating the pods and reflector to watch those pods.
Looks like the section of code taking a long time is
Checked KAS log and seeing about 7000 requests of the form I1010 12:26:59. 1 ] GET /api/v1/nodes/gce-scale-cluster-minion-group-1-4p2m:/proxy/stats/summary: (9.753712ms) 200 [ (linux/amd64) kubernetes/ -- [] [sig-node] kubelet [] [sig-node] Clean up pods on node kubelet should be able to delete 10 pods per node in 1m0s. 146.148.73.158:] between replicationcontroller being created and the pod list/watch request.
The is only supposed to run if there are less than 10 nodes in the cluster. However, the situation doesn't hold with this change: (or ) was changed to .

While the e2e test only uses 10 () nodes from the cluster, the resource monitor is querying all the nodes in the cluster. Thus the client qps limit was exceeded and caused the reflector to be throttled. I'll send a fix
/assign


Question: RestartPolicy: api.RestartPolicyAlways, DNSPolicy:     api.DNSClusterFirst, }, <ins> "bad supplementalGroups large than math.MaxInt32": { Containers: []api.Container{{Name: "ctr", Image: "image", ImagePullPolicy: "IfNotPresent"}}, SecurityContext: &api.PodSecurityContext{ HostNetwork:        false, SupplementalGroups: []int64{maxID, 1234}, }, RestartPolicy: api.RestartPolicyAlways, DNSPolicy:     api.DNSClusterFirst, }, "bad supplementalGroups less than 0": { Containers: []api.Container{{Name: "ctr", Image: "image", ImagePullPolicy: "IfNotPresent"}}, SecurityContext: &api.PodSecurityContext{ HostNetwork:        false, SupplementalGroups: []int64{minID, 1234}, }, RestartPolicy: api.RestartPolicyAlways, DNSPolicy:     api.DNSClusterFirst, }, "bad runAsUser large than math.MaxInt32": { Containers: []api.Container{{Name: "ctr", Image: "image", ImagePullPolicy: "IfNotPresent"}}, SecurityContext: &api.PodSecurityContext{ HostNetwork: false, RunAsUser:   &maxID, }, RestartPolicy: api.RestartPolicyAlways, DNSPolicy:     api.DNSClusterFirst, }, "bad runAsUser less than 0": { Containers: []api.Container{{Name: "ctr", Image: "image", ImagePullPolicy: "IfNotPresent"}}, SecurityContext: &api.PodSecurityContext{ HostNetwork: false, RunAsUser:   &minID, }, RestartPolicy: api.RestartPolicyAlways, DNSPolicy:     api.DNSClusterFirst, }, "bad fsGroup large than math.MaxInt32": { Containers: []api.Container{{Name: "ctr", Image: "image", ImagePullPolicy: "IfNotPresent"}}, SecurityContext: &api.PodSecurityContext{ HostNetwork: false, FSGroup:     &maxID, }, RestartPolicy: api.RestartPolicyAlways, DNSPolicy:     api.DNSClusterFirst, }, "bad fsGroup less than 0": { Containers: []api.Container{{Name: "ctr", Image: "image", ImagePullPolicy: "IfNotPresent"}}, SecurityContext: &api.PodSecurityContext{ HostNetwork: false, FSGroup:     &minID, }, RestartPolicy: api.RestartPolicyAlways, DNSPolicy:     api.DNSClusterFirst, }, </ins> "bad-active-deadline-seconds": { Volumes: []api.Volume{ {Name: "vol", VolumeSource: api.VolumeSource{EmptyDir: &api.EmptyDirVolumeSource{}}},

Help review this, thanks.
Are you asking for better validation on pod creation? I'm not really sure what you mean when you say "tip".
Yes, I mean it's better validate the pod when create. The tip mean the validation info response from server and show by client.


Question: KUBE_BUILD_CONFORMANCE=${KUBE_BUILD_CONFORMANCE:-y} KUBE_BUILD_PULL_LATEST_IMAGES=${KUBE_BUILD_PULL_LATEST_IMAGES:-y} <del> # The mondo test tarball is deprecated as of Kubernetes 1.14, and the default # will be set to 'n' in a future release. # See KEP sig-testing/20190118-breaking-apart-the-kubernetes-test-tarball KUBE_BUILD_MONDO_TEST_TARBALL=${KUBE_BUILD_MONDO_TEST_TARBALL:-y} </del> # Validate a ci version # # Globals:

What happened: On 2019-02-22 was merged with an indication that the mondo test tarball noted as deprecated as per But it did not include a timeline for eventual removal. What you expected to happen: It's been 7 months now. We should formalize a plan for removal. /sig testing /sig release /kind cleanup /priority important-soon
Enhancement tracking issue ref:
removes the deprecated mondo test tarball.


Question: KUBE_GCS_MAKE_PUBLIC=y KUBE_GCS_UPLOAD_RELEASE=y KUBE_GCS_RELEASE_BUCKET=kubernetes-release <del> KUBE_GCS_PROJECT=google-containers </del> KUBE_GCS_RELEASE_PREFIX=release/${KUBE_RELEASE_VERSION} KUBE_GCS_LATEST_FILE="release/latest.txt" KUBE_GCS_LATEST_CONTENTS=${KUBE_RELEASE_VERSION}

I cant bring up the monitoring addons on my local cluster after I turns out that thats because I am using an older version of docker which does not allow "-" in namespaces. I get error on running . suggested that we can change the path to use googlecontainers instead of google-containers since that works for all versions of docker. Using google-containers only works for docker versions 1.5+ Filling this issue to hear what others think about this.
SGTM
Fine with me. I'm working on moving the e2e images, can make this change at same time.
SGTM. Take note Let me know when we definitely decide to do this and I will re-generate the image names for the images that I have created and pushed.
please double and triple check to get everything :) , Satnam Singh wrote:
Thanks for heads-up. I am in the middle of pushing kubernetes/pause and etcd_server to Will make sure use the right path here.
fwiw GoogleCloudPlatform/kubernetes to complement already merged PRs...


Question: "//pkg/apis/coordination/install:go_default_library", "//pkg/apis/core:go_default_library", "//pkg/apis/core/install:go_default_library", <ins> "//pkg/apis/discovery/install:go_default_library", </ins> "//pkg/apis/events/install:go_default_library", "//pkg/apis/extensions/install:go_default_library", <ins> "//pkg/apis/flowcontrol/install:go_default_library", "//pkg/apis/imagepolicy/install:go_default_library", "//pkg/apis/networking/install:go_default_library", "//pkg/apis/node/install:go_default_library", </ins> "//pkg/apis/policy/install:go_default_library", "//pkg/apis/rbac/install:go_default_library", "//pkg/apis/scheduling/install:go_default_library",

What happened: Tried to convert a v1beta1 Ingress object to v1, and received the following error: What you expected to happen: The conversion to work. How to reproduce it (as minimally and precisely as possible): Then run: /sig cli /assign
Can i work on this issue?
/assign
/triage accepted


Question: <ins> rules: - selectorRegexp: k8s[.]io/kubernetes allowedPrefixes: - k8s.io/kubernetes/cmd/cloud-controller-manager/app - k8s.io/kubernetes/cmd/cloud-controller-manager/app/apis/config - k8s.io/kubernetes/cmd/cloud-controller-manager/app/apis/config/scheme - k8s.io/kubernetes/cmd/cloud-controller-manager/app/apis/config/v1alpha1 - k8s.io/kubernetes/cmd/cloud-controller-manager/app/config - k8s.io/kubernetes/cmd/cloud-controller-manager/app/options - k8s.io/kubernetes/cmd/controller-manager/app - k8s.io/kubernetes/cmd/controller-manager/app/options - k8s.io/kubernetes/pkg/api/legacyscheme - k8s.io/kubernetes/pkg/api/service - k8s.io/kubernetes/pkg/api/v1/pod - k8s.io/kubernetes/pkg/apis/apps - k8s.io/kubernetes/pkg/apis/autoscaling - k8s.io/kubernetes/pkg/apis/core - k8s.io/kubernetes/pkg/apis/core/helper - k8s.io/kubernetes/pkg/apis/core/install - k8s.io/kubernetes/pkg/apis/core/pods - k8s.io/kubernetes/pkg/apis/core/v1 - k8s.io/kubernetes/pkg/apis/core/v1/helper - k8s.io/kubernetes/pkg/apis/core/validation - k8s.io/kubernetes/pkg/apis/scheduling - k8s.io/kubernetes/pkg/capabilities - k8s.io/kubernetes/pkg/client/leaderelectionconfig - k8s.io/kubernetes/pkg/cloudprovider/providers - k8s.io/kubernetes/pkg/controller - k8s.io/kubernetes/pkg/controller/apis/config - k8s.io/kubernetes/pkg/controller/apis/config/v1alpha1 - k8s.io/kubernetes/pkg/controller/certificates/signer/config - k8s.io/kubernetes/pkg/controller/certificates/signer/config/v1alpha1 - k8s.io/kubernetes/pkg/controller/cloud - k8s.io/kubernetes/pkg/controller/daemon/config - k8s.io/kubernetes/pkg/controller/daemon/config/v1alpha1 - k8s.io/kubernetes/pkg/controller/deployment/config - k8s.io/kubernetes/pkg/controller/deployment/config/v1alpha1 - k8s.io/kubernetes/pkg/controller/endpoint/config - k8s.io/kubernetes/pkg/controller/endpoint/config/v1alpha1 - k8s.io/kubernetes/pkg/controller/endpointslice/config - k8s.io/kubernetes/pkg/controller/endpointslice/config/v1alpha1 - k8s.io/kubernetes/pkg/controller/garbagecollector/config - k8s.io/kubernetes/pkg/controller/garbagecollector/config/v1alpha1 - k8s.io/kubernetes/pkg/controller/job/config - k8s.io/kubernetes/pkg/controller/job/config/v1alpha1 - k8s.io/kubernetes/pkg/controller/namespace/config - k8s.io/kubernetes/pkg/controller/namespace/config/v1alpha1 - k8s.io/kubernetes/pkg/controller/nodeipam/config - k8s.io/kubernetes/pkg/controller/nodeipam/config/v1alpha1 - k8s.io/kubernetes/pkg/controller/nodelifecycle/config - k8s.io/kubernetes/pkg/controller/nodelifecycle/config/v1alpha1 - k8s.io/kubernetes/pkg/controller/podautoscaler/config - k8s.io/kubernetes/pkg/controller/podautoscaler/config/v1alpha1 - k8s.io/kubernetes/pkg/controller/podgc/config - k8s.io/kubernetes/pkg/controller/podgc/config/v1alpha1 - k8s.io/kubernetes/pkg/controller/replicaset/config - k8s.io/kubernetes/pkg/controller/replicaset/config/v1alpha1 - k8s.io/kubernetes/pkg/controller/replication/config - k8s.io/kubernetes/pkg/controller/replication/config/v1alpha1 - k8s.io/kubernetes/pkg/controller/resourcequota/config - k8s.io/kubernetes/pkg/controller/resourcequota/config/v1alpha1 - k8s.io/kubernetes/pkg/controller/route - k8s.io/kubernetes/pkg/controller/service - k8s.io/kubernetes/pkg/controller/service/config - k8s.io/kubernetes/pkg/controller/service/config/v1alpha1 - k8s.io/kubernetes/pkg/controller/serviceaccount/config - k8s.io/kubernetes/pkg/controller/serviceaccount/config/v1alpha1 - k8s.io/kubernetes/pkg/controller/statefulset/config - k8s.io/kubernetes/pkg/controller/statefulset/config/v1alpha1 - k8s.io/kubernetes/pkg/controller/testutil - k8s.io/kubernetes/pkg/controller/ttlafterfinished/config - k8s.io/kubernetes/pkg/controller/ttlafterfinished/config/v1alpha1 - k8s.io/kubernetes/pkg/controller/volume/attachdetach/config - k8s.io/kubernetes/pkg/controller/volume/attachdetach/config/v1alpha1 - k8s.io/kubernetes/pkg/controller/volume/persistentvolume/config - k8s.io/kubernetes/pkg/controller/volume/persistentvolume/config/v1alpha1 - k8s.io/kubernetes/pkg/features - k8s.io/kubernetes/pkg/fieldpath - k8s.io/kubernetes/pkg/kubelet/types - k8s.io/kubernetes/pkg/master/ports - k8s.io/kubernetes/pkg/security/apparmor - k8s.io/kubernetes/pkg/securitycontext - k8s.io/kubernetes/pkg/serviceaccount - k8s.io/kubernetes/pkg/util/configz - k8s.io/kubernetes/pkg/util/hash - k8s.io/kubernetes/pkg/util/node - k8s.io/kubernetes/pkg/util/parsers - k8s.io/kubernetes/pkg/util/taints </ins>

What would you like to be : Add a .import-restrictions file to cmd/cloud-controller-manager to prevent new imports to Why is this needed: As part of and the effort to remove cloud providers, we are moving cmd/cloud-controller-manager to staging (under ) so that external cloud providers don't have to vendor One of the perquisites to move cmd/cloud-controller-manager to staging is that it should not import anything else from An import-restrictions file would help ensure no new imports to are .
/sig cloud-provider
cc
/assign


Question: import ( "os" <del> "github.com/spf13/cobra" </del> "k8s.io/component-base/cli" <del> cliflag "k8s.io/component-base/cli/flag" </del> _ "k8s.io/component-base/logs/json/register" // for JSON log format registration _ "k8s.io/component-base/metrics/prometheus/restclient" _ "k8s.io/component-base/metrics/prometheus/version" // for version metric registration

When running kubelet, if the Run function fails, the returned error has no output, and the cause of the error cannot be located. such as After the error is returned, there is no position output, so the user cannot know the cause of the error. But previous versions had log output Run the Run function, when there is an error exit, print the error log Misconfigured cgroupRoot No response <details</details<details</details<details</details<details</details<details</details<details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign
/sig node
/cc


Question: func init() { api.Scheme.AddDefaultingFuncs( <ins> func(obj *ReplicationController) { if len(obj.DesiredState.ReplicaSelector) == 0 { obj.DesiredState.ReplicaSelector = obj.DesiredState.PodTemplate.Labels } if len(obj.Labels) == 0 { obj.Labels = obj.DesiredState.PodTemplate.Labels } }, </ins> func(obj *Volume) { if util.AllPtrFieldsNil(&obj.Source) { glog.Errorf("Defaulting volume source for %v", obj)

It doesn't make sense to define pod template with a set of labels that is strict subset of replicaSelector (or at least I can't find an use-case). For that reason I think that the labels in pod template part of replication controller should be optional and user should care about them only if he wants to define a superset of what replicaSelector already defines.
I've heard the same comments from a client. I think that it'd be more intuitive if we always specify pod labels in the template (so that we can copy&paste pod definition), and replicaSelector is optional (by default pod labels).
It would be reasonable for the selector to be a subset. I'm fine with intelligent defaulting but we need to be careful not to conflate the 2 concepts, which I think would increase the amount of confusion.
Setting RC selector from pod template labels SGTM. Any concerns It wouldn't have to work with separate templates, though it would be desirable if it did. It would be nice if we could add preconditions based on other objects (resource version of pod template in this case) to etcd transactions.
I think we agreed in another issue that defaulting the selector from the pod template labels is reasonable. No argument from me. One problem with pod templates is that we lose the strict relationship between selector and template, so if you change the template the RC needs to behave intelligently (I refuse to create more pods because the template selector doesn't match mine, etc)
We should do this, and we should default replicas to 1. We could also consider defaulting replication controller labels from pod template labels. That way, essentially the only thing that would need to be specified for a singleton RC would be the pod template.
I don't think we can default replicas to 1, since we can't tell the difference between an explicit zero and a default zero, and I think there are lots of reasonable cases where you want to set replicas == 0 on create. (esp. in the context of socket activation)
Responded on the PR: We can distinguish default from 0 if we make it a pointer.


Question: return cc.handler(csr) } <ins> // IgnorableError returns an error that we shouldn't handle (i.e. log) because // it's spammy and usually user error. Instead we will log these errors at a // higher log level. We still need to throw these errors to signal that the // sync should be retried. func IgnorableError(s string, args ...interface{}) ignorableError { return ignorableError(fmt.Sprintf(s, args...)) } type ignorableError string func (e ignorableError) Error() string { return string(e) } </ins>

Each node waiting for bootstrapping creates 1 line per 2-3s in the logs when the requesting client doesn't have access. It should not be logging at v(0), these are all v(4) level. We should be reporting these appropriately. I don't want a 5k node cluster showing this.
I'd like to fix it


Question: return cephfsVolume.plugin.host.GetPodVolumeDir(cephfsVolume.podUID, utilstrings.EscapeQualifiedNameForDisk(name), cephfsVolume.volName) } <ins> // GetKeyringPath creates cephfuse keyring path func (cephfsVolume *cephfs) GetKeyringPath() string { name := cephfsPluginName volumeDir := cephfsVolume.plugin.host.GetPodVolumeDir(cephfsVolume.podUID, utilstrings.EscapeQualifiedNameForDisk(name), cephfsVolume.volName) volumeKeyringDir := volumeDir + "~keyring" return volumeKeyringDir } </ins> func (cephfsVolume *cephfs) execMount(mountpoint string) error { // cephfs mount option ceph_opt := ""

The goal of this issue is to design a CephFS driver using FUSE client for Kubernetes.</brExisting Ceph kernel client has the following drawbacks:</br• Kernel client does not support client-enforced quotas. • Kernel CephFS doesn't support quota or capacity, capacity requested by PVC is not enforced or validated. • Kernel CephFS doesn't work with SELinux, setting SELinux label in Pod's security context will not work.
Just a detail. Since we will rely on ceph-fuse, we will need the ceph-fuse client to be available on all kubernetes nodes. We can add a parameter to allow the use of the kernel client as well. I would say better make the fuse client opt-in and keep the kernel one as the default option. The kernel client might support quotas in the future. Thanks
/sig storage
We are also inclined to use fuse instead of kernel client ,any progress on this issue?
I like to continue this job and prepare to send a PR to make kubernetes support cephfs fuse mounting.
/assign I think this task would be valuable. =)
waiting for the PR, do you have a plane to support s3 fuse ?
No, no plan for s3fs-fuse currently
also see and cc
Ceph-fuse support in the command to display the specified secret, it may take some time to support, because it involves LibFUSE(I will follow up this demand). On the other hand, ceph-fuse has stability, quota function, expansion, upgrade and other advantages. Although ceph-fuse only supports the secret file way, but if it can be pushed to the k8s community, there is a very meaningful thing.


Question: influxdb "github.com/influxdata/influxdb/client" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/labels" <ins> "k8s.io/apimachinery/pkg/util/wait" </ins> "k8s.io/kubernetes/pkg/client/clientset_generated/clientset" "k8s.io/kubernetes/test/e2e/framework"

The following E2E test is failing in the GCE 1.5-1.6 upgrade jobs. Please take a look and let me know ASAP if this is a 1.6 blocker or not (I think it may be a flake, but need you to confirm). https://k8s- Expected error: <*errors.errorString | 0xc421d5ef10: { s: "expected to find one replica for RC or deployment with label heapster but got 0", } expected to find one replica for RC or deployment with label heapster but got 0 not to have occurred
/assign
can you help get this looked at and triaged ASAP? Thanks.
I pinged by email. He's really the one who should look at this.
The problem seems to be not with the test itself but with Kubernetes or test framework, since the test failed on starting Heapster. I'll investigate more later today.
I haven't got time to investigate. This seems to be unrelated to monitoring itself, so any help during MTV day would be great. If not I'll try to fix this tomorrow
thanks
The problem is in , in function . The function is called immediately from , w/o waiting for system pods (heapster) to start. We should give some time for heapster to become ready, as it takes few minutes after cluster creation.
CC
thanks a lot for investigation! is going to fix it.


Question: "k8s.io/kubernetes/federation/apis/federation" "k8s.io/kubernetes/federation/client/clientset_generated/federation_release_1_3" "k8s.io/kubernetes/pkg/api" <ins> "k8s.io/kubernetes/pkg/api/errors" </ins> "k8s.io/kubernetes/pkg/api/v1" "k8s.io/kubernetes/pkg/client/clientset_generated/release_1_3" "k8s.io/kubernetes/pkg/client/restclient"

Federated service shards are created only in the clusters that have the specified namespace. To fix this problem we need to create the namespace on-demand while creating the federated service shards. cc
As discussed, we explicitly decided not to do it this way. In Kubernetes, when one creates an entity, if the namespace does not exist, the creation call fails. What we need to do for v1.3 is pre-create the test namespace in all of the underlying clusters as part of the e2e test turnup.


Question: configv1beta3 "k8s.io/kubernetes/pkg/scheduler/apis/config/v1beta3" ) <del> func loadConfigFromFile(logger klog.Logger, file string) (*config.KubeSchedulerConfiguration, error) { </del> <ins> // LoadConfigFromFile loads scheduler config from the specified file path func LoadConfigFromFile(logger klog.Logger, file string) (*config.KubeSchedulerConfiguration, error) { </ins> data, err := os.ReadFile(file) if err != nil { return nil, err

I want to be exported as a public function imports scheduler code and runs scheduler simulations for pods to decide which nodegroup should be scaled up. consider this case: a user is running a scheduler with custom scheduler config the user wants cluster-autoscaler to run scheduler simulations based on the custom scheduler config For ii default scheduler config is used in cluster-autoscaler scheduler simulations. Right now, it is not possible to use custom scheduler config for cluster-autoscaler scheduler simulations but I am working on a PR () to support this feature. For this feature, I need to parse scheduler config. . It would help if we can have exported as (just like ) so that programs which need to parse scheduler config can use the exported function directly instead of writing custom code.
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scheduling
PR:
/assign vadasambar
wdyt?
/sig autoscaling
LGTM, thanks.


Question: Buckets:   buckets, } vec := metrics.NewHistogramVec(HistogramOpts, labels) <del> registerMetrics.Do(func() { legacyregistry.MustRegister(vec) }) </del> <ins> // Use local registry var registry = metrics.NewKubeRegistry() var gather metrics.Gatherer = registry registry.MustRegister(vec) </ins> // Observe two metrics with same value for label1 but different value of label2. vec.WithLabelValues("value1-0", "value2-0").Observe(1.5) vec.WithLabelValues("value1-0", "value2-1").Observe(2.5) vec.WithLabelValues("value1-1", "value2-0").Observe(3.5) vec.WithLabelValues("value1-1", "value2-1").Observe(4.5) metricName := fmt.Sprintf("%s_%s_%s", HistogramOpts.Namespace, HistogramOpts.Subsystem, HistogramOpts.Name) <del> histogramVec, _ := GetHistogramVecFromGatherer(legacyregistry.DefaultGatherer, metricName, tt.lvMap) </del> <ins> histogramVec, _ := GetHistogramVecFromGatherer(gather, metricName, tt.lvMap) </ins> if diff := cmp.Diff(tt.wantVec, histogramVec); diff != "" { t.Errorf("Got unexpected HistogramVec (-want +got):n%s", diff) }

TestRecordOperation TestRecordOperation TestGetHistogramVecFromGatherer /kind failing-test <!-- If this is a flake, please add: /kind flake --<!-- Please assign a SIG using: /sig SIG-NAME --/sig testing This may be related with
/assign I will do some initial researches on this. is similar
/sig instrument
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : /sig instrument Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig instrumentation
/triage accepted /assign cc
To fix it gracefully, we may refactor the code in like The refactor would change a lot and need some confirmations or demo.(I prefer) A temporary workaround before refactoring it: is another try to ignore the dup error in but it is odd as should panic as designed if one register failed.
Three testings failed due to . Haven't found package promauto () being used in these testings. It might be other issues. These testings are: /filterwithonelabel
What are you guys doing here? What test is this? You cannot change the component-base metrics code base it is doing what it is supposed to be doing. Does this test suite involve multiple sequential runs of the same test?
I think this is odd to change the component-base metrics code base. But the refactor to use is another approach. You can try to reproduce it with is with
Oh, yes. This involves multiple sequential runs of the same test to . It uses
You can use to register the registry only once. It is important to reset the metrics, or next iteration in the test will fail
Thanks I other fixes in


Question: var _ = ginkgo.Describe("[sig-api-machinery] Events", func() { f := framework.NewDefaultFramework("events") <del> ginkgo.It("should ensure that an event can be fetched, patched, deleted, and listed", func() { </del> <ins> /* Release : v1.19 Testname: Event resource lifecycle Description: Create an event, the event MUST exist. The event is patched with a new message, the check MUST have the update message. The event is deleted and MUST NOT show up when listing all events. */ framework.ConformanceIt("should ensure that an event can be fetched, patched, deleted, and listed", func() { </ins> eventTestName := "event-test" ginkgo.By("creating a test event")

This issue proposes the writing of a new test for the API endponts , , , , , and with the intention of promoting it to Conformance. This issue contains a very basic mock test in order to start the conversation about what a good e2e test would look like for this endpoint. The code below is verified to be hitting the intended endpoint, show as per the queries in APIsnoop's live view of the cluster's audit logs. According to these query results from APIsnoop, we are lacking in tests for the following endpoints related to Events We chose the following 5 endpoints for this mock test: - - - Test outline: Event in namespace default with custom fields all events in all namespaces find event created in in response of the content of the event created in events by name in namespace default ensure that the event updated from event by name all events in namespace ensure event in namespace default was deleted This mock test uses client-go's standard functions for talking to the Event area of the API. Discover useragents Display endpoints hit by the test Show the change in coverage after the mock test If a test with these calls gets merged in, conformance will go up by 5 endpoints. /sig architecture /area conformance
lgtm according to the SIG-Arch conformance office hours 17th December 2019.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/reopen can you put the checkboxes at the top for your apisnoop org pr, the test, and link to the promotion PR? I think we are ready.
/open
Reopened this issue. <detailsIn response to : /reopen can you put the checkboxes at the top for your apisnoop org pr, the test, and link to the promotion PR? I think we are ready. Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/retitle Promote CoreV1Event test - +5 coverage
/assign
This test was written before the tickets were stored in APISnoop branches. Unsure what to do with it in regards to the APISnoop PR


Question: id: monitoring-heapster port: 80 containerPort: 8082 <ins> labels: name: heapster kubernetes.io/cluster-service: "true" </ins> selector: name: heapster

As per we should add a common label to all of our add-on pods: DNS, logging, monitoring, etc. The proposal is to add "": "true" for now.
FWIW "true" has to be quoted in YAML
Can you close this one once it's completed.


Question: {"single-stack ipv6, prefer v6", routeV6, ipv6NetworkInterface{}, preferIPv6, net.ParseIP("2001::200")}, {"dual stack", bothRoutes, v4v6NetworkInterface{}, preferIPv4, net.ParseIP("10.254.71.145")}, {"dual stack, prefer v6", bothRoutes, v4v6NetworkInterface{}, preferIPv6, net.ParseIP("2001::10")}, <ins> {"LLA and loopback with global, IPv4", routeV4, linkLocalLoopbackNetworkInterface{}, preferIPv4, net.ParseIP("10.1.1.1")}, {"LLA and loopback with global, IPv6", routeV6, linkLocalLoopbackNetworkInterface{}, preferIPv6, net.ParseIP("fd00:1:1::1")}, {"LLA and loopback with global, dual stack prefer IPv4", bothRoutes, linkLocalLoopbackNetworkInterface{}, preferIPv4, net.ParseIP("10.1.1.1")}, {"LLA and loopback with global, dual stack prefer IPv6", bothRoutes, linkLocalLoopbackNetworkInterface{}, preferIPv6, net.ParseIP("fd00:1:1::1")}, {"LLA and loopback with global, no routes", noRoutes, linkLocalLoopbackNetworkInterface{}, preferIPv6, nil}, {"interface and loopback with global, IPv4", routeV4, globalsNetworkInterface{}, preferIPv4, net.ParseIP("192.168.1.1")}, {"interface and loopback with global, IPv6", routeV6, globalsNetworkInterface{}, preferIPv6, net.ParseIP("fd00::200")}, {"interface and loopback with global, dual stack prefer IPv4", bothRoutes, globalsNetworkInterface{}, preferIPv4, net.ParseIP("192.168.1.1")}, {"interface and loopback with global, dual stack prefer IPv6", bothRoutes, globalsNetworkInterface{}, preferIPv6, net.ParseIP("fd00::200")}, {"interface and loopback with global, no routes", noRoutes, globalsNetworkInterface{}, preferIPv6, nil}, </ins> {"all LLA", routeV4, networkInterfaceWithOnlyLinkLocals{}, preferIPv4, nil}, {"no routes", noRoutes, validNetworkInterface{}, preferIPv4, nil}, {"fail get IP", routeV4, networkInterfaceFailGetAddrs{}, preferIPv4, nil},

What happened: When attempting to kubeadm join a control plane node or worker on a machine with global IPv4 unicast addresses on a loopback address (router to the host w/ BGP Unnumbered), the join fails with the following output: See this issue for details : Also: What you expected to happen: The correct IPv4 address should be selected from the loopback adapter, or we should be able to specify an IPv4 address to use for egress. The IPv6 addresses on the network adapters themselves are only used to establish upstream peering. IPv4 nexthops are fake 169.254.x.x addresses artificially programmed via RFC5549 into the ARP table. How to reproduce it (as minimally and precisely as possible): On a node with the following conditions: Routing to the host with FRR IPv6 link locals on physical network interfaces No IPv4 addresses on physical network interfaces Unicast IPv4 on the loopback adapter IPv4 default route present via RFC5549 Attempt to kubeadm join a node, and it will fail. Anything else we need to know?: We've been patching kubeadm with , which works as expected. There was a fix applied to allow initing control plane nodes w/ bind address 127.0.0.1 which would then select an IPv4 from the loopback, but it doesn't appear to have fixed the joining issues. Pinging and by request on kubeadm issues. Environment: Kubernetes version (use ): 1.17.12 Cloud provider or hardware configuration: baremetal OS (e.g: ): Centos 7 Kernel (e.g. ): 3.10.0-1127.19.1.el7.x86_64 Install tools: kubeadm Network plugin and version (if this is a network-related bug): Calico Others: FRR for peering with routers on physical network interfaces
/sig api-machinery
is defined but we should note the same function is used for the --bind-address of the kube-apiserver. /sig network
/assign
/triage accepted


Question: "describe.go", "drain.go", "edit.go", <ins> "edit_configmap.go", </ins> "exec.go", "explain.go", "expose.go",

Kubernetes version : What happened: When I try to display configmap content with yaml output (kubectl get configmap webapp-inifile -o yaml), the output is sometimes pretty, and sometimes not -depending newlines are effectively displayed or not - just n char- I can't find what's the difference between some file and some others. (limit on numer of lines ?) Configmap is created with --from-file option Ex : What you expected to happen: More human readable display, in order to be able to easily change parameter when kubectl edit congifmap How to reproduce it :
Hrm, this seems to me like the right answer is to special case 'kubectl edit' for configmaps, rather than try to hack around with the YAML decoder. I suppose we could use block quotes (e.g. ) if we see newlines but that seems error prone...
Closing this, pretty print is there, kubectl configMap edit support is there.
This still happens on kube 1.7 :-( I'm not 100% sure how to reproduce, but what I did was: have an existing configmap and edit a few lines exit the editor ... now I see the characters instead of actual newlines. even shows the characters
I'm also seeing this issue in kube 1.7. During the past few days I have edited a configmap ~110 times, and seen the formatting get destroyed (spaces messed up, characters visible) twice. I can't determine anything specific that I did in those two edits that would have triggered the formatting messup.
I'm still seeing this issue too. Pushed a lot of identical configmaps, some ones are displaying good, and some other bad. For example, I have a configmap with 3 files in ; they all are identical except one variable ; 2 of them display normally, and the last one displays with n ...
also seeing this issue on 1.7
can we reopen this issue? or open a new one?
I created a brand new configmap using and when I viewed the yaml with it's full of n characters and escape characters. This makes it very difficult to edit.
I am experiencing the same. n character is messing up a configmap I'm creating with --from-file option. Anyone found a reason why this is happening ?

Also failing in 1.8.4
I am also facing the same issue of formatting .. I have a application configuration of 400-500 lines. Tryied many times. For fist 40-50 lines the format is good ans easily readable.. However, when i try to create config for entire configuration file its very difficult to edit the config map..
Same issue here. No idea what makes one configmap show better than another.
This issue corrupted my aws-auth configMap that authenticates my user to access the EKS cluster and while trying to add other users to the configMap the formatting corrupted the configMap file and now I am locked out of my EKS cluster. Version: 1.10.3
I don't know if this will help somebody else but I was facing the same issue with escaped data in a config map edit and get (the config map had a single .conf file), and it seems that I had a unneeded extra space at the end of a single line in my data. When I removed that extra space the pretty printing was maintained in edit and get.
11 Still same
still same ( there is an example if install helm install stable/prometheus , you will find cm/prometheus-server )
I agree with - I can recreate it every time If you were to execute the following (ensuring no trailing spaces), the configmap remains "pretty" However, repeat the above (or edit the existing one with ), and put a trailing space after one of the key/value pairs, and that then messes up the configmap contents
Can we reopen this issue? I need this feature to change the yaml in cm, and I have been struggling with the intent and n for several days ...
Please reopen this ticket, it's incredibly frustrating to have to deal with this even on small configmaps
Got this also. I had to remove tabs from my config file to get the yaml multiline display.
Just created a messed up configmap on kuberntes 1.12.2.
Likewise.
OMG thank you!
Just ran into this in EKS (1.10.3) -- highly annoying. I have one config map that will come out OK if I use create directly, but if I pipe the output of to apply then it gets munged. Does seem like this is something that needs addressing.

Have the same issue and fixed by removing tabs
Have the same issue, don't have any tabs in file. This issue totally should be reopened.
This issue is valid and I am not sure why it was closed
Not only spaces. Trail symbols too. I uses vim for it
"n n" (spaces between end of lines) in one of my files also messed up the pretty display.
Created a new issue when I hit this again and was able to repro using the trailing whitespace
I've removed all the trailing whitespaces and tabs. The problem is gone.
Check any possible special chars. It solve my problem some time ago
I have seen this issue in K8 1.7 -1.13 kubectl create configmap NAMESPACE --from-file=$PATCHFILE --dry-run=true --output=yaml where PATCHFILE is attached file below. I renamed it from .json to .log so this webpage would let me include it.
Solve the issue by delete whitespace at each end of line.
The output of is ugly. Wether using json or yaml, k8s injects a whole bunch of escape characters and messes up the formatting of the original config file. It would be very nice to have a switch that tells it to dump the config out "normally"!
I a new line in a configMap using Tab for identation. After changing to Spaces instead of Tab, I was able to see the configmap as expected...
Same here, white space/s at the end of line ruin the output, everything back to pretty when white space/s removed.
This worked for me.
Since I've still met this on 1.16, I'll point a bullet list of the regular culprits for the configmap messing: spaces before newline:Tabs:Special characters like ãẽõ, áéó, öß will also mess your files. Look up for them in your comments, they might pass through!
An easy one liner to remove trailing whitespaces in a ConfigMap:
Please reopen. This issue still remains. This format makes eks aws-auth configmap very hard to maintain. Once the file is applied with trailing whitespaces the whole thing gets lots of n and . As a temporary measurement is there a way to reformat the configmap in the "readable" way? after the n are created?

if you use way didn't reformat the file for me.
Nurza's method works. But you may have a space before EOF as well: Or you may have CR's in there , in which case you should remove the CRs. Or you may have a TAB, which you should replace with one or more spaces.
No CR/TAB/new lines. I have a "clean" file that after I apply it all works great. (display as expected) After I add one trailing white space to it, it gets those n. I tried to run Nurza's method but it didn't work. Any other Ideas why Nurza's method is not working for me ? This a part of the output I'm getting with
has a space after it, although you don't see it. Because the LF is on the next line, there must be a space before it.
Correct but if I run Nurza's method the n are still there. Am I still missing anything? This is what I run
Nobody said that sed expression was perfect. Once you know this is your doing -- and you know how to fix it -- you're not bothered by this behaviour anymore.
This is scary shit :hankey: ... what is the industry practice to edit the file like these: aws-auth...? I managed to type 20+lines by hand :disappointed: and it worked on 3rd attempt :disappointed_relieved:
Cant' say about the 'like this', but aws-auth you can use eksctl to do the hard work for you:
If the formatting of cm goes wierd a simple hack to get it back to normal is : kubectl get cm configmapname -o yaml cm.yamlNow copy the contents of file and past it on is powerful tool to check the linting of yaml files. This will provide you with the configmap as expected with correct formatting. Paste the output in another yaml file (for e.g - )kubectl apply -f
Anyone knows why the issue is closed? It's not fixed as far as I can tell.

If you just want the raw output as it was read in when created --from-file, you can use jq to get the raw string (without escaped newlines etc) If you created a configmap from a file like this: Get the data:
Edited a cm, came back in, and had n all over. Something about white space in my previous edit? Not great overall :/ On kubectl version Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.5", GitCommit:"", GitTreeState:"clean", BuildDate:"2020-07-04T15:01:15Z", GoVersion:"go1.14.4", Compiler:"gc", Platform:"darwin/amd64"} Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.6", GitCommit:"", GitTreeState:"clean", BuildDate:"2019-11-13T11:11:50Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}
Awesome, very good website~
Thank You very much
I was able to fix this behavior by: 1) Don't use tabs, convert to spaces 2) To remove spaces before a newline character, use this: It will also convert CRLF to LF only, by removing .
Something small that exacerbates this issue is that in python's standard lib module , up to some version (I think python 3.4, see ), pretty printing adds a trailing space after every comma by default, unless you supply a list of separators yourself that omit the spaces (e.g. like ) So if you have old-ish glue (e.g. python 2.7) that parses a k8s configmap into a file for archival purposes, and you made that file readable by pretty-printing it, using that file to regenerate a new configmap results in this gross escape behaviour.
Hello There, One of our colleague has edited configmap with wrong data, now whole cluster inaccessible with any of role, need help on immediate basis and much appreciated. Error: through eksctl - Error: getting auth ConfigMap: Unauthorized through kubectl error: You must be logged in to the server (Unauthorized)
yaml marshallers will write a string to a multiline string e.g.

UNLESS string contains a character string terminates in a space character string contains a substring which matches the regex
I am in the same situation. Were you able to solve this?
Also reproduce in v1.22.3 :(
I occasionally encountered this problem in v1.15.6 or v1.21.2, but eventually I was able to deal with it by using edit and then using vim's own set list to back-propagate the original static configmap code configuration under gitlab to see if there were extra spaces, and then modifying the local configmap file based on the online configmap's set list display under edit. The state of the online configmap under edit, to modify the local configmap file, is ultimately able to deal with this problem, but if there is a better official way to do it would be good.
yes if you are working on linux
i would like to share an observation that helped me sort out the line merging and n. it appears to be related to the last line of the file you are using (--from-) if the last line ends with empty space or tab. the config map is merged up in one line with n as line feed if you remove the empty space from the last line, the ConfigMap will import file as is with +
I got the same error after editing the ConfigMap file. I used an online tool to remove the trailing white space that we cannot see in the text. After copying the content which removed trailing whitespaces I was able to resolve this issue FYI - (online tool which I used)
in v1.24 also i can see same issue. how we can edit cm with many lines.
Finally, I found that the absence of a value defined or the value is empty string in will cause the issue. data: {{ }} is not defined in my yaml just use quote to the value resolved
delete space before n or after n it works for me
did anyone find solution when we edit cm how we can see in proper format as for me really difficult to edit cm as we have many n char and I can still see this issue in k8s version 1.28


Question: q.AddUnschedulableIfNotPresent(q.newPodInfo(&unschedulablePod), q.SchedulingCycle()) q.AddUnschedulableIfNotPresent(q.newPodInfo(&highPriorityPod), q.SchedulingCycle()) q.MoveAllToActiveOrBackoffQueue("test") <ins> q.lock.RLock() defer q.lock.RUnlock() </ins> if q.activeQ.Len() != 1 { t.Error("Expected 1 item to be in activeQ") }

<!-- Please only use this template for submitting reports about flaky tests or jobs (pass or fail with no underlying change in code) in Kubernetes CI --Which jobs are flaking: pull-kubernetes-bazel-test Which test(s) are flaking: pkg/scheduler/internal/queue Testgrid link: Reason for failure: Anything else we need to know: /sig scheduling /assign
I just sent a pr for it according to the error message, but I'm not very sure if it will make things all right.
/assign


Question: var u map[string]interface{} var err error if unstr, ok := obj.(Unstructured); ok { <del> // UnstructuredContent() mutates the object so we need to make a copy first u = unstr.DeepCopyObject().(Unstructured).UnstructuredContent() </del> <ins> u = unstr.UnstructuredContent() </ins> } else { t := reflect.TypeOf(obj) value := reflect.ValueOf(obj)

What happened: In I introduced an optimization - do a deep copy instead of to and from JSON roundtrip to convert anything that implements . I then discovered that the method that is used there in both and may mutate the original object. This is problematic because previously (before ) there was no mutation and because this is unexpected and may lead to data races - it is bad behaviour to mutate original object when you just want a copy of it. This issue was fixed by . However the underlying issue is not fixed here - is brittle and dangerous. Method name does not imply that it mutates data when you call it. And godoc does not mention that either: Something needs to be done about it IMO. This method is misused in these places (mutation side effect is not intended): Also implementation in does not implement the behaviour required by godoc in . What you expected to happen: There should be a method that returns the current unstructured contents without mutation and perhaps not providing the write-through effect. /kind bug /sig api-machinery
/cc
Is this still required and could I work on it? I'm looking for something to work on as a new contributor :)
Yes and yes! :)
Perfect, I'll get started on it!
I've got out. I'd appreciate a review :)
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale


Question: - --memory={{ base_metrics_server_memory }} - --extra-memory={{ metrics_server_memory_per_node }}Mi - --threshold=5 <del> - --deployment=metrics-server-v0.4.4 </del> <ins> - --deployment=metrics-server-v0.5.0 </ins> - --container=metrics-server - --poll-period=30000 - --estimator=exponential

<!-- Please only use this template for submitting reports about flaky tests or jobs (pass or fail with no underlying change in code) in Kubernetes CI -- (gce-ubuntu-master-default) Kubernetes e2e suite: BeforeSuite (There are other flakes in this job, but they are failing less often and different test each time, so let's say this issue is only for BeforeSuite) is not starting: Spyglass: Triage: Mentioned in /cc <!-- Please see the deflaking doc () for more guidance! --
There are no sig labels on this issue. Please add an appropriate label by using one of the following commands: - Please see the for a listing of the SIGs, working groups, and committees available. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/milestone v1.22
/cc Since it is related to the metrics server. the metrics server keeps dying as the log shown in I cannot get more logs of metrics servers from the CI artifacts. Currently, we are using metric server 0.4.4 and the latest 0.5.0 is released. Are there some known bugs that may be related to this test failure?
I don't think so. /cc
The metrics-server fails to bind ~because there is another process listening~ on port 443 are some of the konnectivity pods listening on the port :443?
In the dockerfile of metrics-server if has ? Maybe it has permission problem?
ahh, right, nice catch, I didn't pay attention to the error and blindly assumed collision :sweat_smile: ... who can take an action on that? has this changed recently?
If the metrics-server version used after , I think it is correct. has no change recently.

I vaguely remember some changes recently to reduce privileges on pods on CI, tthis seems likely a permission problem, does this ring a bell? EDIT This PR introduced the change , any special reason why we want to run it in a privileged port if this is accessed using a Service? EDIT 2 let see if this is the problem
Yeah, Maybe you can refer to this, the release notes about v0.4.4 metrics-server . I think you have resolved this problem.
Release v0.3.X was already using privileged port 443, as v0.4.X has migrated to distroless image we needed to introduce to give it needed privileges. What's weird for me is that the test is flaking and not failing. Could it be related to node where MS is scheduled?
You mean, there may be other reasons? If so, may require in-depth analysis.
As far as I can tell, is never starting properly, even when the jobs are successful. If we were to take this job, that has succeeded: From the serial logs, we can see that the metrics-server pod is crash looping: But from the build logs, we can see that at some point the pod was deemed ready: To me it seems that there are 2 issues, one being that metrics-server doesn't have the capabilities it needs to run and another one that still needs to be investigated where metrics-server is sometimes marked ready even though it is in a crash looping state.
This is a problem with the metrics-server deployment on the CI jobs and the metrics tests I don't know which sig owns these components but it will be better to find first an owner, what is is clear is that the metrics tests are not reliable since once you set a readiness probe on the metrics-server it fails consistently , , however the tests pass cc
it seems the last try was the good one :smile:


Question: patch, delete, and deletecollection.' release: v1.20 file: test/e2e/common/node/runtimeclass.go <ins> - testname: Pod with the deleted RuntimeClass is rejected. codename: '[sig-node] RuntimeClass should reject a Pod requesting a deleted RuntimeClass [NodeConformance] [Conformance]' description: Pod requesting the deleted RuntimeClass must be rejected. release: v1.20 file: test/e2e/common/node/runtimeclass.go - testname: Pod with the non-existing RuntimeClass is rejected. codename: '[sig-node] RuntimeClass should reject a Pod requesting a non-existent RuntimeClass [NodeConformance] [Conformance]' description: The Pod requesting the non-existing RuntimeClass must be rejected. release: v1.20 file: test/e2e/common/node/runtimeclass.go - testname: RuntimeClass Overhead field must be respected. codename: '[sig-node] RuntimeClass should schedule a Pod requesting a RuntimeClass and initialize its Overhead [NodeConformance] [Conformance]' description: The Pod requesting the existing RuntimeClass must be scheduled. This test doesn't validate that the Pod will actually start because this functionality depends on container runtime and preconfigured handler. Runtime-specific functionality is not being tested here. release: v1.24 file: test/e2e/common/node/runtimeclass.go - testname: Can schedule a pod requesting existing RuntimeClass. codename: '[sig-node] RuntimeClass should schedule a Pod requesting a RuntimeClass without PodOverhead [NodeConformance] [Conformance]' description: The Pod requesting the existing RuntimeClass must be scheduled. This test doesn't validate that the Pod will actually start because this functionality depends on container runtime and preconfigured handler. Runtime-specific functionality is not being tested here. release: v1.20 file: test/e2e/common/node/runtimeclass.go </ins> - testname: Secrets, pod environment field codename: '[sig-node] Secrets should be consumable from pods in env vars [NodeConformance] [Conformance]'

Per the comment in this PR conformance tests will regress because of a newly Runtime Class GA v1 API is not covered with tests. Full comment: Full comment: /assign More links:
/sig node /triage accepted
KEP:
New to conformance tests. So the idea is to mark tests (or create new ones) that would call each of these methods at lease once? So if this test will be marked as conformance, it will cover and . Is there any additional requirements for the conformance tests? Maybe with the number of side effects of that test or a special execution environment?
Yes. You can make a single test that runs objects through CRUD operations. For the deletecollection one, I'd suggest labeling the objects under test and scoping the deletecollection to your test label to avoid breaking the cluster you're running on. See as an example
Great, will be working on it! Also just chatted with about it.
Just to get clarity on this - is this considered a blocker for Confirming bc grad criteria for it says "No release blocking feedback for API and functionality" should these tests be required to consider this stable? As a note: Code Freeze means that required tests must also be finished:
Here's possibly an easier link to track: !
This seems like something that should be tested generically, or at the very least have a table-driven test that covers all (or most) GA APIs.
! Just a quick note to show that currently the new tests are hitting all endpoints. I'd go ahead and create the Promotion PR in order to get any necessary feedback from #sig-arch
/reopen re: I disagree with the idea that a single test covering CRUD operations means RuntimeClass is now conformance tested. Generic CRUD on a resource's API is not sufficient to describe the intended behavior of the resource. That kind of test is at best verifying that api machinery is working. A vendor would able to pass this test, without implementing any of the behaviors runtimeclass enables. What does do? What kinds of behaviors are visible to an end-user? Are any of those tests in Conformance?
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository.

</details
I'm not familiar enough with runtimeclass to know whether these would be a good start:
Thank you for reopening. I agree with this in principle. But I really would appreciate some guidance here. Scheduling tests that takes runtime handler into account are great. However they don't hit any endpoints. I asked and I'm not clear whether those needs to be covered as they are not touching an API. As for actual scheduling, one limitation is posted . Which makes some other tests impossible to include to conformance. I next week, but there may not be all the right people there as it's a Thanksgiving short week. Do you have a suggestion?
I agree that exercising specific CRI implementations is not the place of Conformance testing. Conformance testing is intended to exercise end-user visible behaviors of a Kubernetes cluster, which basically means whatever you can see and do at the Control Plane API level (ref: ) The challenge we face with Conformance testing is that we don't have an easy way to enumerate and measure coverage of behaviors. We experimented with listing behaviors in .yaml so that humans and machines could populate and approve them independent of test implementation, but ultimately abandoned that effort due to its complexity (ref: ) So we use API coverage as an easily machine-measurable/enforceable proxy. Noticing the coverage was at 0% should have been a sign that we should look at the existing test coverage of RuntimeClass and decide which tests were eligible for promotion to Conformance, and whether any new tests needed to be written. In theory a solid test plan for a KEP would include a list of behaviors that need to be tested. That's not immediately clear to me from the KEP for RuntimeClass (ref: ) Based on my read of the KEP, some valid tests might be: tests that exercise the runtimeclass admission controller creating pods with a non-existent runtimeclass creating pods with an existing runtimeclass creating pods with scheduling rules that would clash with runtimeclass scheduling rules creating pods with scheduling rules that would merge with runtimeclass scheduling rules Tests that would be disallowed by conformance requirements because they would require pre-installation/configuration of the cluster under test: running pods with a runtimeclass (unless there's some way to create a functional "dummy" runtimeclass without touching kubelets) running pods with a runtimeclass with a specific handler such as containerd, crio, etc.

and expecting them to run
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-contributor-experience at . /lifecycle stale
sorry for the delay /remove-lifecycle stale
/area conformance
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-contributor-experience at . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-contributor-experience at . /lifecycle rotten
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-contributor-experience at . /close
Closing this issue. <detailsIn response to : Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen /remove-lifecycle rotten
Reopened this issue. <detailsIn response to : /reopen /remove-lifecycle rotten Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: ), }, expected: &DeviceRunContainerOptions{ <del> Annotations: []kubecontainer.Annotation{ {Name: "cdi.k8s.io/devicemanager_pod-container", Value: "vendor1.com/class1=device1,vendor2.com/class2=device2,vendor3.com/class3=device3,vendor4.com/class4=device4"}, }, </del> CDIDevices: []kubecontainer.CDIDevice{ {Name: "vendor1.com/class1=device1"}, {Name: "vendor2.com/class2=device2"},

Kubelet should stop using annotations to pass CDI device IDs to CRI runtimes. CDIDevices CRI field should be used for this purpose. This should be done when Kubelet that doesn't support CRI field reaches EOL and two major CRI runtimes support CRI field. Here is a PR comment that gives more details about this: Kubelet started using CDIDevices CRI field in . Previous version 1.27 will reach EOL .CRI-O started using it in . Previous version 1.27 will reach EOL together with Kubernetes 1.27.Containerd started using it in . Version 1.7 will reach EOL in a 6 months after Containerd 2.0 release Previous version 1.6 doesn't support CDI injections and should not be considered. So, safer option would be to wait until Containerd 1.7 reaches EOL. However it's relatively safe to do it earlier when 1.7.0 and 1.7.1 have little chances to be in real use. Containerd v1.7.0 and v1.7.1 is not used by the Linux distributions from , so it's safe to remove annotations support from the Kubelet in 1.31 in my opinion.
/sig node
/triage accepted /priority important-longterm
/assign


Question: if err != nil { return err } <del> if condition := findConditionByType(patchedStatus.Status.Conditions, customConditionType); condition != nil { </del> <ins> if condition := findConditionByType(statusToUpdate.Status.Conditions, customConditionType); condition != nil { </ins> condition.LastTransitionTime = now2 } else { framework.Failf("patched object does not have the required condition %v", customConditionType)

! /kind failing-test <!-- If this is a flake, please add: /kind flake --/sig apps
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign suspecting as cause
Ack, indeed, because this is the suite for alpha features enabled. We have this fix ready for the test itself: /cc
/cc It flakes in
/retitle [Flaking Test] [Conformance] [sig-apps] Job should apply changes to a job status
It used to fail with errors like After it fails with : [FAILED] patched object should have the applied condition with LastTransitionTime time.Date(2024, time.March, 7, 17, 58, 34, 0, time.UTC), got time.Date(2024, time.March, 7, 17, 58, 33, 0, time.Local) instead In [It] at: @ 03/07/24 17:58:34.109 Here I open the follow up fix: PTAL.


Question: pv.Status.Phase = v1.VolumeBound // bind pvc to "pv-$i" <del> // pvc.Spec.VolumeName = pv.Name </del> pvc.Status.Phase = v1.ClaimBound } else { pv.Status.Phase = v1.VolumeAvailable } <del> if err := CreatePersistentVolumeWithRetries(client, pv); err != nil { </del> <ins> // Create PVC first as it's referenced by the PV when the `bindVolume` is true. if err := CreatePersistentVolumeClaimWithRetries(client, namespace, pvc); err != nil { </ins> lock.Lock() defer lock.Unlock() <del> createError = fmt.Errorf("error creating PV: %s", err) </del> <ins> createError = fmt.Errorf("error creating PVC: %s", err) </ins> return } // We need to update statuses separately, as creating pv/pvc resets status to the default one. <del> if _, err := client.CoreV1().PersistentVolumes().UpdateStatus(context.TODO(), pv, metav1.UpdateOptions{}); err != nil { </del> <ins> if _, err := client.CoreV1().PersistentVolumeClaims(namespace).UpdateStatus(context.TODO(), pvc, metav1.UpdateOptions{}); err != nil { </ins> lock.Lock() defer lock.Unlock() <del> createError = fmt.Errorf("error updating PV status: %s", err) </del> <ins> createError = fmt.Errorf("error updating PVC status: %s", err) </ins> return } <del> if err := CreatePersistentVolumeClaimWithRetries(client, namespace, pvc); err != nil { </del> <ins> if err := CreatePersistentVolumeWithRetries(client, pv); err != nil { </ins> lock.Lock() defer lock.Unlock() <del> createError = fmt.Errorf("error creating PVC: %s", err) </del> <ins> createError = fmt.Errorf("error creating PV: %s", err) </ins> return } <del> if _, err := client.CoreV1().PersistentVolumeClaims(namespace).UpdateStatus(context.TODO(), pvc, metav1.UpdateOptions{}); err != nil { </del> <ins> // We need to update statuses separately, as creating pv/pvc resets status to the default one. if _, err := client.CoreV1().PersistentVolumes().UpdateStatus(context.TODO(), pv, metav1.UpdateOptions{}); err != nil { </ins> lock.Lock() defer lock.Unlock() <del> createError = fmt.Errorf("error updating PVC status: %s", err) </del> <ins> createError = fmt.Errorf("error updating PV status: %s", err) </ins> return }

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: Run integration test , for the case which needs PV or PVC, got the error which info us that PVC is not found, Furtherly, this will lead a inconsistent PV / PVC status, and fail the whole testcase. What you expected to happen: Run successfully. How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: The error message is thrown here, Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others: /sig scheduling /sig testing
/assign
/triage accepted


Question: return false } <del> if s.AnyOf == nil { s.AnyOf = []NestedValueValidation{ </del> <ins> if s.ValueValidation == nil { s.ValueValidation = &ValueValidation{} } if s.ValueValidation.AnyOf == nil { s.ValueValidation.AnyOf = []NestedValueValidation{ </ins> {ForbiddenGenerics: Generic{Type: "integer"}}, {ForbiddenGenerics: Generic{Type: "string"}}, } } else { <del> s.AllOf = append([]NestedValueValidation{ </del> <ins> s.ValueValidation.AllOf = append([]NestedValueValidation{ </ins> { ValueValidation: ValueValidation{ AnyOf: []NestedValueValidation{

What happened: API server panics handling requests for a CRD with OpenAPI validation with What you expected to happen: CRD is applied successfully. How to reproduce it (as minimally and precisely as possible): kind create cluster --image kindest/node:v1.16.1 then kubectl apply: Partial logs from API server: Anything else we need to know?: Environment: Kind Kubernetes version (use ):OS (e.g: ):
/sig api-machinery
/cc
where It seems that s.AnyOf was referencing some invalid address.
yes, which is very strange, given s was dereferenced earlier in the same method: is investigating
reproduced and has a fix
I had the same issue with prometheus (kube 1.16.1). Deploying is fine but when trying to delete, it crashes the api server. For those who can't upgrade to 1.16.2+ (fixed version) there is a workaround : using etcd to delete the resources that crash the api server : here I grep on coreos because I know it is prometheus that crashes the api server Remove every item in the list that causes the crash :
Note this is also resolved in 1.15.5+ and 1.16.2+, upgrading to 1.17.x is not required to resolve this