Usage

Hostdisk CSI can be used as an separate component that allows exposing manually formatted filesystem to kubernetes or together with Hostdisk Manager that allows for disk management kubernetes way using custom resources.

Usage with Hostdisk Manager

Display detected hostdisks

Hostdisk Manager will automatically detect disks connected to kubernetes nodes it's running on and will create Hostdisk resource for each of them.

kubectl get hostdisk

TIP

Only devices that don't have a file system will be detected

Hostdisk formatting

Set one of supported filesystem names in HostDisk specification to format it and create persistent volume. Filesystem can be set only once and can not be modified after setting. Supported filesystems: ext3, ext4, xfs.

kubectl edit hostdisks "<HOSTDISK NAME>"

In spec.format.filesystem.type set name of the filesystem you want to use to format disk:
```
[...]
spec:
  format:
    filesystem:
      type: xfs
```

Multiple disk formatting

You can format all disks with xfs using this script:

kubectl get hostdisks.hostdisk.sarkan.io | grep -v NAME | cut -d " " -f 1 | while IFS= read -r DISK; do kubectl patch hostdisks.hostdisk.sarkan.io $DISK --type merge -p "{\"spec\":{\"format\":{\"filesystem\": {\"type\": \"xfs\"}}}}"; done

Persistent Volume creation

After disk is formatted using Hostdisk Manager it will automatically create Persistent Volume for this disk. Persistent Volume name will be equal to filesystem UUID of this disk. Hostdisk filesystem UUID and Persistent Volume can be displayed using kubectl:

$ kubectl get hostdisks -o wide
NAME                                 NODE                      DEVICE     SIZE        FSTYPE   FSUUID                                 STATE   AGE
0x6000c291b0af68729901c9df8c853655   node01.lab.indevops.com   /dev/sde   1.00 TiB    xfs      18d224bb-9c5f-45aa-9540-5b5f563d006c   Ready   3d

$ kubectl get persistentvolumes
NAME                                   CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      AGE
18d224bb-9c5f-45aa-9540-5b5f563d006c   1Ti        RWO            Retain           Available   3d

TIP

If Persistent Volume associated to formatted disk is deleted Hostdisk Manager will create new Persistent Volume for this disk.

Changing disk filesystem

After Filesystem format is set, it's immutable to prevent incidental formatting and loss of data. To change filesystem you must delete hostdisk resource so it's no longer maintained by HostDisk manager and manually wipe filesystem on that disk. After new hostdisk resource will be created for empty disk that you can format as in previous section.

Wipe Hostdisk Manager managed disk:

Delete hostdisk resource and persistent volume associated to it
Directly on node wipe filesystem on selected disk

$ kubectl get hostdisks.hostdisk.sarkan.io 0x6000c291b0af68729901c9df8c853655 -o wide
NAME                                 NODE                      DEVICE     SIZE        FSTYPE   FSUUID                                 STATE   AGE
0x6000c291b0af68729901c9df8c853655   node01.lab.indevops.com   /dev/sde   1.00 TiB    xfs      18d224bb-9c5f-45aa-9540-5b5f563d006c   Ready   3d

$ kubectl delete hostdisk 0x6000c291b0af68729901c9df8c853655
$ kubectl delete pv 18d224bb-9c5f-45aa-9540-5b5f563d006c

$ ssh root@node01.lab.indevops.com wipefs -a /dev/sde

TIP

After disk is wiped it will be re-discovered within 60 seconds.

Manual usage (without hostdisk manager)

Node filesystem can be manually exposed by creating Persistent Volume with volumeHandle equal to filesystem UUID. It can be used if you don't want to use Hostdisk Manager or want to expose already formatted disk.

WARNING

Manually created Persistent Volumes won't work after disk is moved to other node and will have to be recreated. It's recommended to use Hostdisk Manager.

Preparing disk

Disk must be formatted to be exposed to Kubernetes. We recommend formatting using XFS without partition (proceed with caution, you will loose all existing data on the device):
```
mkfs.xfs /dev/sdx
```

Use the following command to obtain the filesystem UUID:

$ blkid --cache-file /dev/null /dev/sdx
/dev/sdx: UUID="b8b64172-6b47-4da8-b467-581fecc993b2" BLOCK_SIZE="512" TYPE="xfs"

Use the following command to obtain the filesystem size and type:

$ lsblk -o PATH,SIZE,FSTYPE /dev/sdx
PATH     SIZE FSTYPE
/dev/sdc   1T xfs

You will also need name of the Kubernetes node where the filesystem is available. Usually it's the same as the node's hostname which can be obtained using following command:
```
$ hostnamectl --static
node01.lab.com
```

Creating Persistent Volume

Create a PersistentVolume to represent this filesystem in Kubernetes:

metadata.name - name of the Persistent Volume. Must be unique across the cluster.
spec.csi.fsType - name of the filesystem disk is formatted to.
spec.csi.volumeHandle - UUID of the formatted disk.
spec.capacity.storage - size of the disk.
spec.nodeAffinity.required.nodeSelectorTerms - node affinity specifying on which node disk is located. Required for proper workload scheduling.

apiVersion: v1
kind: PersistentVolume
metadata:
  name: b8b64172-6b47-4da8-b467-581fecc993b2
spec:
  csi:
    driver: hostdisk.csi.sarkan.io
    fsType: xfs
    volumeHandle: b8b64172-6b47-4da8-b467-581fecc993b2
  capacity:
    storage: 1Ti
  storageClassName: ""
  accessModes:
    - ReadWriteOnce
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - node01.lab.com

Create a PersistentVolumeClaim for the PersistentVolume:

metadata.name - name of the Persistent Volume Claim. Must be unique in the namespace.
spec.volumeName - name of the Persistent Volume this claim should use.
spec.storageClassName - empty string must be explicitly set otherwise default StorageClass will be set.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: b8b64172-6b47-4da8-b467-581fecc993b2
  namespace: hostdisk-csi
spec:
  storageClassName: ""
  volumeName: b8b64172-6b47-4da8-b467-581fecc993b2
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Ti