Documentation Run k8s conformance test using sonobuoy
Library OperatingSystem
Library BuiltIn
-Test Setup Check that k8s cluster is reachable
-Test Teardown Cleanup Sonobuoy
+Library Collections
+Library String
+Library SSHLibrary
+Library Process
+Library JSONLibrary
+Test Setup Run Keywords
+... Check that k8s cluster is reachable
+... Define Images
+... Onboard Images
+... Create Manifest File
+Test Teardown Run Keywords
+... Cleanup Sonobuoy
+... Close All Connections
*** Variables ***
${LOG} ${LOG_PATH}${/}${SUITE_NAME.replace(' ','_')}.log
+&{SONOBUOY} path=gcr.io/heptio-images
+... name=sonobuoy:v0.16.1
+&{E2E} path=akraino
+... name=Actual value set dynamically
+&{SYSTEMD_LOGS} path=akraino
+... name=validation:sonobuoy-plugin-systemd-logs-latest
+&{SONOBUOY_IMGS} sonobuoy=&{SONOBUOY}
+... e2e=&{E2E}
+... systemd_logs=&{SYSTEMD_LOGS}
+
+# Following tests assume DNS domain is "cluster.local"
+${DNS_DOMAIN_TESTS} SEPARATOR=
+... DNS should provide /etc/hosts entries for the cluster|
+... DNS should provide DNS for services|
+... DNS should provide DNS for ExternalName services|
+... DNS should provide DNS for the cluster|
+... DNS should provide DNS for pods for Subdomain|
+... DNS should provide DNS for pods for Hostname
+
+# Images listed by Sonobuoy but not available for downloading
+@{SKIP_IMGS} gcr.io/kubernetes-e2e-test-images/windows-nanoserver:v1
+... gcr.io/authenticated-image-pulling/windows-nanoserver:v1
+... gcr.io/authenticated-image-pulling/alpine:3.7
+... k8s.gcr.io/invalid-image:invalid-tag
+... invalid.com/invalid/alpine:3.1
+
*** Test Cases ***
Run Sonobuoy Conformance Test
# Start the test
Run kubectl apply -f ${CURDIR}${/}sonobuoy.yaml
- Sleep 5s
+ Sleep 20s
${rc} ${output}= Run And Return Rc And Output
- ... kubectl describe pod/sonobuoy -n heptio-sonobuoy
+ ... kubectl describe pod/sonobuoy -n sonobuoy
Append To File ${LOG} ${output}${\n}
# Wait until the test finishes execution
- Run while sonobuoy status | grep "Sonobuoy is still running"; do sleep 120; done
+ Run while sonobuoy status | grep "Sonobuoy is still running"; do sleep 180; done
Append To File ${LOG} "Sonobuoy has completed"${\n}
# Get the result and store the sonobuoy logs
# Make sure the pod is reachable with the local k8s client
${rc} ${output}= Run And Return Rc And Output
- ... kubectl get pods --all-namespaces
+ ... kubectl version
Append To File ${LOG} ${output}${\n}
- Should Contain ${output} kube-system
+ Should Contain ${output} Server Version: version.Info
Cleanup Sonobuoy
${rc} ${output}= Run And Return Rc And Output
Append To File ${LOG} ${output}${\n}
Sleep 3s
Should Contain ${output} service "sonobuoy-master" deleted
+
+Open Connection And Log In
+ Open Connection ${HOST}
+ Login With Public Key ${USERNAME} ${SSH_KEYFILE}
+
+Upload To Internal Registry
+ [Arguments] ${path} ${name}
+ ${rc}= Execute Command
+ ... docker pull ${path}/${name}
+ ... return_stdout=False return_rc=True
+ Should Be Equal As Integers ${rc} 0
+ ${rc}= Execute Command
+ ... docker tag ${path}/${name} ${INT_REG}/bluval/${name}
+ ... return_stdout=False return_rc=True
+ Should Be Equal As Integers ${rc} 0
+ ${rc}= Execute Command
+ ... docker push ${INT_REG}/bluval/${name}
+ ... return_stdout=False return_rc=True
+ Should Be Equal As Integers ${rc} 0
+
+Onboard Sonobuoy Images
+ FOR ${img} IN @{SONOBUOY_IMGS}
+ ${path}= Get From Dictionary ${SONOBUOY_IMGS['${img}']} path
+ ${name}= Get From Dictionary ${SONOBUOY_IMGS['${img}']} name
+ Upload To Internal Registry ${path} ${name}
+ Set To Dictionary ${SONOBUOY_IMGS['${img}']} path=${INT_REG}/bluval
+ END
+
+Onboard Kubernetes e2e Test Images
+ ${result}= Run Process sonobuoy images
+ Should Be Equal As Integers ${result.rc} 0
+ @{images}= Split String ${result.stdout}
+ FOR ${img} IN @{images}
+ Continue For Loop If $img in $SKIP_IMGS
+ ${path} ${name} Split String From Right ${img} / 1
+ Upload To Internal Registry ${path} ${name}
+ END
+
+Define Images
+ ${result}= Run Process kubectl version -o json
+ Should Be Equal As Integers ${result.rc} 0
+ ${versions}= Convert String To JSON ${result.stdout}
+ ${major}= Get Value From Json ${versions} $.serverVersion.major
+ ${minor}= Get Value From Json ${versions} $.serverVersion.minor
+ Set To Dictionary ${SONOBUOY_IMGS['e2e']} name=validation:kube-conformance-v${major[0]}.${minor[0]}
+
+Onboard Images
+ ${INT_REG}= Get Variable Value ${INTERNAL_REGISTRY} ${EMPTY}
+ Set Test Variable ${INT_REG}
+ Return From Keyword If $INT_REG == '${EMPTY}'
+ Open Connection And Log In
+ Onboard Sonobuoy Images
+ Onboard Kubernetes e2e Test Images
+
+Get Tests To Skip
+ ${flag}= Set Variable Aggregator|Alpha|\\[(Disruptive|Feature:[^\\]]+|Flaky)\\]
+ ${flag}= Run Keyword If '${DNS_DOMAIN}' != 'cluster.local'
+ ... Catenate SEPARATOR=| ${flag} ${DNS_DOMAIN_TESTS}
+ ... ELSE
+ ... Set Variable ${flag}
+ [Return] ${flag}
+
+Create Manifest File
+ ${skip}= Get Tests To Skip
+ @{flags}= Set Variable
+ ... --e2e-focus \\[Conformance\\\]
+ ... --e2e-skip ${skip}
+ ... --kube-conformance-image ${SONOBUOY_IMGS.e2e.path}/${SONOBUOY_IMGS.e2e.name}
+ ... --sonobuoy-image ${SONOBUOY_IMGS.sonobuoy.path}/${SONOBUOY_IMGS.sonobuoy.name}
+ ... --image-pull-policy Always
+ ... --timeout 14400
+ Run Keyword If $INT_REG != '${EMPTY}' Run Keywords
+ ... Append To List ${flags}
+ ... --e2e-repo-config ${CURDIR}${/}custom_repos.yaml
+ ... AND
+ ... Run Process sed -i s|{{ registry }}|${INT_REG}/bluval|g
+ ... ${CURDIR}${/}custom_repos.yaml
+ ${result}= Run Process sonobuoy gen @{flags}
+ Should Be Equal As Integers ${result.rc} 0
+ ${manifest}= Replace String ${result.stdout}
+ ... image: gcr.io/heptio-images/sonobuoy-plugin-systemd-logs:latest
+ ... image: ${SONOBUOY_IMGS.systemd_logs.path}/${SONOBUOY_IMGS.systemd_logs.name}
+ Create File ${CURDIR}${/}sonobuoy.yaml ${manifest}
Code Review / validation.git / blobdiff