Prerequisites

Single node kubernetes cluster with RBAC authorization mode enabled.

• check kube-apiserver has RBAC enabled

  prlctl exec scorpius "ps -aef | grep kube-apiserver | grep authorization-mode | sed 's/ --/\n--/g'"
  

  Resulting output for kube-apiserver process start command should contain --authorization-mode=RBAC. If authorization mode has other modes enabled but RBAC then RBAC would not work

  root        1624    1312  3 19:37 ?        00:00:42 kube-apiserver
  --advertise-address=10.211.55.49
  --allow-privileged=true
  --authorization-mode=Node,RBAC
  --client-ca-file=/etc/kubernetes/pki/ca.crt
  --enable-admission-plugins=NodeRestriction
  --enable-bootstrap-token-auth=true
  --etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt
  --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt
  --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key
  --etcd-servers=https://127.0.0.1:2379
  --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt
  --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key
  --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
  --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.crt
  --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.key
  --requestheader-allowed-names=front-proxy-client
  --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt
  --requestheader-extra-headers-prefix=X-Remote-Extra-
  --requestheader-group-headers=X-Remote-Group
  --requestheader-username-headers=X-Remote-User
  --secure-port=6443
  --service-account-issuer=https://kubernetes.default.svc.cluster.local
  --service-account-key-file=/etc/kubernetes/pki/sa.pub
  --service-account-signing-key-file=/etc/kubernetes/pki/sa.key
  --service-cluster-ip-range=10.96.0.0/12
  --tls-cert-file=/etc/kubernetes/pki/apiserver.crt
  --tls-private-key-file=/etc/kubernetes/pki/apiserver.key
  

Configure pod view access for user Bob in his namespace

• Create dedicated namespace for Bob with name bob-space

  prlctl exec scorpius --user cka "kubectl create namespace bob-space"
  

• Create role pod-viewer

  prlctl exec scorpius --user cka "kubectl -n bob-space create role pod-viewer --verb=get,list,watch --resource=pods"
  

• Create Role Binding ‘pod-viewer’ for user bob

  prlctl exec scorpius --user cka "kubectl -n bob-space create rolebinding --role=pod-viewer --user=bob pod-viewer"
  

• Use kubectl to test permissions

  # check Bob can view pods in his namespace
  prlctl exec scorpius --user cka "kubectl -n bob-space auth can-i get pods --as=bob"
  # yes
  
  # check Bob can view pods in default namespace
  prlctl exec scorpius --user cka "kubectl -n default auth can-i get pods --as=bob"
  # no
  
  $ check John can view pods in Bob's namespace
  prlctl exec scorpius --user cka "kubectl auth can-i get secrets --as=john"
  # no
  

• Another way of testing is to authenticate kubectl using different user. For that you need x509 cert for user bob which can be generated with kubeadm

  # generate credentials for Bob and save them into bob.conf 
  prlctl exec scorpius "kubeadm kubeconfig user --client-name=bob > /home/cka/.kube/bob.conf; chown cka:cka /home/cka/.kube/bob.conf"
  # generate credentials for John and save them into bob.conf 
  prlctl exec scorpius "kubeadm kubeconfig user --client-name=john > /home/cka/.kube/john.conf; chown cka:cka /home/cka/.kube/john.conf"
  

  config generated for Bob and John would look similar to this

  apiVersion: v1
    clusters:
    - cluster:
        certificate-authority-data: a3ViZXJuZXRlcy4iY2VydGlmaWNhdGUtYXV0aG9yaXR5LWRhdGEiCg==
        server: https://10.211.55.34:6443
      name: kubernetes
    contexts:
    - context:
        cluster: kubernetes
        user: bob
      name: bob@kubernetes
    current-context: bob@kubernetes
    kind: Config
    preferences: {}
    users:
    - name: bob
      user:
        client-certificate-data: Ym9iLnVzZXIuImNsaWVudC1jZXJ0aWZpY2F0ZS1kYXRhIgo=
        client-key-data: Ym9iLnVzZXIuImNsaWVudC1rZXktZGF0YSIK
  

• start some pods before the test

  # use admin permissions to create a new pod in Bob's namespace
  prlctl exec scorpius --user cka "kubectl run nginx --image=nginx -n bob-space"
  # use admin permissions to create a new pod in default namespace
  prlctl exec scorpius --user cka "kubectl run busybox --image=busybox -- sleep infinity"
  

• Get pods using bob credentials

  # use Bob's credentials to get pods in his own namespace
  prlctl exec scorpius --user cka "KUBECONFIG=~/.kube/bob.conf kubectl get pods -n bob-space"
  # NAME    READY   STATUS    RESTARTS   AGE
  # nginx   1/1     Running   0          81s
  
  # use Bob's credentials to get pods in default namespace
  prlctl exec scorpius --user cka "KUBECONFIG=~/.kube/bob.conf kubectl get pods"
  # Error from server (Forbidden): pods is forbidden: User "bob" cannot list resource "pods" in API group "" in the namespace "default"
  
  # use John's credentials to get pods in Bob's namespace
  prlctl exec scorpius --user cka "KUBECONFIG=~/.kube/john.conf kubectl get pods -n bob-space"
  # Error from server (Forbidden): pods is forbidden: User "john" cannot list resource "pods" in API group "" in the namespace "bob-space"
  

Configure pod view access for a service account

• ssh into scorpius

  ssh cka@scorpius
  

• create namespace antares

  kubectl create namespace antares
  

• create service account meteorite

  kubectl create sa meteorite -n antares
  

• create nginx pod

  kubectl run nginx --image=nginx -n antares
  

• create new role binding antares-view which binds existing cluster role viewer to the new service account ‘meteorite’

  kubectl create rolebinding antares-view --clusterrole=view \
  --serviceaccount=antares:meteorite -n antares"
  

• with antares-view role binding pods in antares namespace with meteorite permissions have access to view resources defined by cluster role view. Now create such pod in antares with curl utility onboard, set service account and a few environment variables

  kubectl run curl --image curlimages/curl \
  --overrides='{"spec":{"serviceAccountName":"meteorite"}}' \
  --env="APISERVER=https://kubernetes.default.svc" \
  --env="SERVICEACCOUNT=/var/run/secrets/kubernetes.io/serviceaccount" \
  --env="CACERT=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt" \
  -n antares -- sleep infinity
  

• check kubernetes REST API URIs documentation to understand how to run curl. The following list of request is expected to be forbidden since we’ve created role binding for a specific namespace antares

  • /api/v1/namespaces - forbidden
  • /api/v1/pods - forbidden
  • /apis/apps/v1/deployments - forbidden

  And resources in antares namespace should be accessible

  • /api/v1/namespaces/antares/pods - allowed
  • /apis/apps/v1/namespaces/antares/deployments - allowed
  • /apis/apps/v1/namespaces/antares/deployments/nginx - allowed

• test antares-view rolebinding

  # exec into curl pod
  kubectl -n antares exec curl -it -- sh
  

  Inside pod’s shell run

  # get service account namespace
  NAMESPACE=$(cat ${SERVICEACCOUNT}/namespace)
  # get service account token
  TOKEN=$(cat ${SERVICEACCOUNT}/token)
  # get pods in antares
  curl --cacert ${CACERT} --header "Authorization: Bearer ${TOKEN}" \
  ${APISERVER}/api/v1/namespaces/antares/pods
  # you should see full description of two pods `busybox` and `curl` in `json`
  # format
  # {
  #   "kind": "PodList",
  #   "apiVersion": "v1",
  #   "metadata": {
  #     "resourceVersion": "245637"
  #   },
  #   "items": [
  #     {
  #       "metadata": {
  #         "name": "curl",
  #         ...
  #       },
  #       ...
  #     },
  #     {
  #       "metadata": {
  #         "name": "busybox",
  #         ...
  #       },
  #       ...
  #     }
  #   ]
  # }
  
  # get pods in a different namespace
  curl --cacert ${CACERT} --header "Authorization: Bearer ${TOKEN}" \
  ${APISERVER}/api/v1/namespaces/default/pods
  # error
  # {
  #   "kind": "Status",
  #   "apiVersion": "v1",
  #   "metadata": {},
  #   "status": "Failure",
  #   "message": "pods is forbidden: User \"system:serviceaccount:antares:meteorite\" cannot list resource \"pods\" in API group \"\" in the namespace \"default\"",
  #   "reason": "Forbidden",
  #   "details": {
  #     "kind": "pods"
  #   },
  #   "code": 403
  # }
  

• exit pod Ctrl+D

• exit scorpius ssh Ctrl+D