# Kubernetes doet not create correct directory

**URL:** https://forum.scylladb.com/t/kubernetes-doet-not-create-correct-directory/4815
**Category:** ScyllaDB
**Tags:** error-message, troubleshooting, administration, kubernetes
**Created:** [May 8, 2025, 1:30pm UTC](https://forum.scylladb.com/t/kubernetes-doet-not-create-correct-directory/4815 "2025-05-08T13:30:07Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![Jaap\_Elst](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.scylladb.com/jaap_elst/32/1663_2.png) [@Jaap\_Elst](https://forum.scylladb.com/u/Jaap_Elst)
#### Post date: [May 8, 2025, 1:30pm UTC](https://forum.scylladb.com/t/kubernetes-doet-not-create-correct-directory/4815/1 "2025-05-08T13:30:07Z")

</div>

_Installation details_  
#kubernetes version: microk8s  
#ScyllaDB version: 2025.1  
#Cluster size: 3 nodes  
#os (RHEL/CentOS/Ubuntu/AWS AMI): Ubuntu

When I try to follow the gitops documentation for the scylla operator with a modified setup for the nodeconfig:

apiVersion: scylla{DOT}scylladb{DOT}com/v1alpha1  
kind: NodeConfig  
metadata:  
name: scylladb-pool-1  
spec:  
localDiskSetup:  
raids:  
- name: nvmes  
type: RAID0  
RAID0:  
devices:  
nameRegex: ^/dev/sdb$  
filesystems:  
- device: /dev/md/nvmes  
type: xfs  
mounts:  
- device: /dev/md/nvmes  
mountPoint: /var/lib/persistent-volumes  
unsupportedOptions:  
- prjquota  
placement:  
nodeSelector:  
scylla{DOT}scylladb{DOT}com/node-type: scylla  
tolerations:  
- effect: NoSchedule  
key: scylla-operator{DOT}scylladb{DOT}com/dedicated  
operator: Equal  
value: scyllaclusters

When a pod tries to use the local csi driver (installed like the instructions in the gitops documentation). I get the following error in pod describe while the pod stays on init indefinitely:  
MountVolume.SetUp failed for volume “pvc-9effa2df-4f8b-44b6-94a2-0697c2980a17” : rpc error: code = Internal desc = Failed to publish volume: can’t create target path at “/var/snap/microk8s/common/var/lib/kubelet/pods/ad3827af-2ee6-4913-9853-55205cd6f8c2/volumes/kubernetes{DOT}io~csi/pvc-9effa2df-4f8b-44b6-94a2-0697c2980a17/mount”: mkdir /var/snap/microk8s/common/var/lib/kubelet/pods/ad3827af-2ee6-4913-9853-55205cd6f8c2/volumes/kubernetes{DOT}io~csi/pvc-9effa2df-4f8b-44b6-94a2-0697c2980a17/mount: no such file or directory

All folders exist apart form the pvc-9effa2df-4f8b-44b6-94a2-0697c2980a17 one

When manually creating this folder it gets deleted when the pod tries to mount the volume again.

---

<div class="post-metadata">

### Author: ![Gabriel](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.scylladb.com/gabriel/32/60_2.png) [@Gabriel](https://forum.scylladb.com/u/Gabriel)
#### Post date: [May 11, 2025, 6:35pm UTC](https://forum.scylladb.com/t/kubernetes-doet-not-create-correct-directory/4815/2 "2025-05-11T18:35:14Z")

</div>

@mflendrich / @zimnx can you take a look ?

---

<div class="post-metadata">

### Author: ![Yehuda\_Lebi](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.scylladb.com/yehuda_lebi/32/1021_2.png) [@Yehuda\_Lebi](https://forum.scylladb.com/u/Yehuda_Lebi)
#### Post date: [May 12, 2025, 7:05am UTC](https://forum.scylladb.com/t/kubernetes-doet-not-create-correct-directory/4815/3 "2025-05-12T07:05:56Z")

</div>

On MicroK8s the kubelet root lives under  
`/var/snap/microk8s/common/var/lib/kubelet`, **not** `/var/lib/kubelet`.  
In the upstream manifest the three hostPath volumes that the pods rely on still point to `/var/lib/kubelet` (see [local-csi-driver/deploy/kubernetes/local-csi-driver/50\_daemonset.yaml at master · scylladb/local-csi-driver · GitHub](https://github.com/scylladb/local-csi-driver/blob/master/deploy/kubernetes/local-csi-driver/50_daemonset.yaml)).  
Because that directory is empty on a MicroK8s node, the driver container cannot see (or create)  
`/var/snap/…/kubelet/pods/<pod-uid>/volumes/kubernetes.io~csi/<pvc-uid>/mount` and fails exactly as you observed.

There are two options to fix it, quick and dirty, on every node you could create a symlink  
`sudo ln -s /var/snap/microk8s/common/var/lib/kubelet /var/lib/kubelet`, or patch the DaemonSet (or create your own overlay) so that all host paths reference the MicroK8s kubelet root.
