Ubuntu ZFS pool refuses import after kernel upgrade: 'incompatible feature flags' and module version mismatch

Environment

Ubuntu 22.04 LTS server, ZFS-on-Linux (zfsutils-linux from distro packages), pool created on another machine with ZFS 2.1.x, kernel upgraded from 5.15 to 5.19 via HWE kernel upgrade, system uses DKMS zfs module build, machine has 8 TB NVMe pool.

Symptoms

zpool import fails with errors referencing 'incompatible pool version' and zfs module reports 'module spinlock mismatch' in dmesg. Boot drops to maintenance shell until pool import is attempted manually. Data appears intact on underlying devices (smartctl shows healthy drives).

Observations

dmesg contains: 'ZFS: vdev_dtl_reassess: module version mismatch' and 'ZFS: pool has feature flags not supported by this ZFS implementation'. zpool import -d /dev/nvme... -o readonly=on lists the pool but refuses to import read-write. zfs --version shows a different tag than the pool's feature set reported by zpool import -D on the other machine where pool was last active.

Already attempted

Tried apt-get install --reinstall zfs-dkms/zfsutils; DKMS build failed with gcc warnings tied to new kernel headers; rebooted into previous kernel (5.15) — on previous kernel the zfs module loads but zpool still warns about features; attempted zpool import -o compatibility=1 (rejected); exported pool on source machine and reimported remotely (not possible due to physical location).

Question

How can I safely import the ZFS pool on this machine after a kernel upgrade without risking data loss, and what steps revert to a compatible ZFS module version when DKMS won't build for the new kernel?

Post timeline

COMMUNITY RESPONSES

Verified community troubleshooting

0 replies

No answers yet..

Showing 0 of 0 community responses

AI DIAGNOSTICS

Incident analysis

Updated 1w ago• Trouble ID

Most likely causes

Pool was used on a host with newer ZFS feature flags enabled; current host's ZFS implementation lacks those features. 95%

Partial upgrade left zfsutils (userspace utilities) and kernel module out-of-sync (different versions). 90%

DKMS failed to build a kernel module against the new kernel headers, leaving an older or missing kernel module. 85%

Summary

The core issue is mismatch between the ZFS on Linux kernel module version and the on-disk pool feature flags. Kernel upgrade changed ABI or DKMS failed to build a matching module, leaving either an older ZFS userspace with incompatible on-disk features or an incomplete kernel module. Pools upgraded (feature flags set) on another host will refuse to import on older ZFS implementations. Attempting any write operations on an incompatible implementation risks corruption. Best path is to import read-only to examine metadata, then run matching ZFS tooling / kernel module that supports the pool's features.

Suggested Fixes

Diagnostic step 1

Do not attempt zpool import -f or any commands that may write metadata until you have a compatible ZFS environment; prefer read-only operations.

Diagnostic step 2

Boot the machine into the older kernel (5.15) where ZFS previously worked and attempt zpool import -o readonly=on to confirm data visibility; if it imports read-only, create a full backup (replicate dataset to another system) before further changes.

Diagnostic step 3

Examine zfs --version and zpool import -D output to discover specific feature flags required; on a temporary rescue system with matching ZFS version (or the original host that last used the pool), export the pool cleanly and then import here after resolving module mismatch.