Openai/6971d533-1acc-800b-a048-a162caba12d7

This conversation was exported from ChatGPT

gpt-5-2

User: Hi.

Hi. I'm on version TrueNAS-SCALE-25.04.2.6. I'm using 8 14tb HDD's in Raidz2. I do have my data backed up so I'm not super panicked, but I need to get to the bottom of this and get things back up and running.

I woke up this morning and server was on but not accessible via web GUI, and no HDMI output, so I had to hard power down. Now, when booting, I'm repeatedly getting stuck at: "Starting ix-zfs.service - Import ZFS pools... (1 of 2) Job ix-zfs.service start running (53s / 15min 26s)"

Assistant: You’re hitting this during ZFS pool import, which usually means ZFS is struggling with the pool hung I/O, device errors…

You’re hitting this during ZFS pool import, which usually means ZFS is struggling with the pool (hung I/O, device errors, or a corrupted import state). The hard power-off likely exposed an existing issue rather than caused it.

The good news:

• Your system is not frozen — ix-zfs.service waits up to 15+ minutes
• RAIDZ2 + backups = very recoverable situation

Below is a safe, structured way to diagnose and recover, starting with the least destructive options.

What’s most likely happening

One (or more) of these:

1. A disk is slow, timing out, or failing
2. ZFS import is blocked waiting on a bad drive
3. Previous unclean shutdown left the pool in a dirty state
4. Hardware issue (HBA, SATA cable, power)

ZFS is conservative — it will wait forever rather than risk corruption.

Step 1: Let it try once (important)

On the current boot:

• Wait at least 10–15 minutes total
• Watch for any disk activity LEDs

If it eventually proceeds → we move to cleanup. If it never completes → proceed below.

Step 2: Interrupt boot → get shell access

Reboot and do this:

1. At the GRUB menu, choose: `` Advanced options → TrueNAS SCALE → Recovery Mode ` OR press Esc during boot
2. Select: ` Drop to root shell

   You should now have a console prompt.
   
   ==== Step 3: Check if ZFS sees your pool ====
   
   Run:

   bash

zpool import

===== Possible outcomes: =====

====== A) Pool is listed normally ======

Example:

pool: media state: ONLINE

Good — try importing manually.

====== B) Pool shows DEGRADED / FAULTED ======

This usually points to one or two bad disks (still safe in RAIDZ2).

====== C) Command hangs ======

That strongly indicates a disk that is not responding.

==== Step 4: Try a safe manual import ====

First, try read-only (zero risk):

bash

zpool import -o readonly=on -f <poolname>

If that works → excellent, your data is intact.

If it hangs or fails, try:

bash

zpool import -f -m <poolname>

(-m ignores missing logs/cache devices)

==== Step 5: Identify bad or slow disks ====

Run:

bash

lsblk -o NAME,SIZE,MODEL,SERIAL

Then:

bash

dmesg | grep -Ei "ata|sas|scsi|error|timeout|reset"

You're looking for things like:
* I/O error
* link reset
* device offline
* failed command

⚠️ One bad disk can stall the entire import

==== Step 6: If import still hangs — isolate the offender ====

Since you’re backed up, this is safe.
# Power off
# Disconnect ONE drive
# Boot → try zpool import
# Repeat until import succeeds

RAIDZ2 allows two missing disks, so this is safe for diagnosis.

Once the pool imports, ZFS will clearly tell you which disk(s) are missing or failed.

==== Step 7: After successful boot (VERY IMPORTANT) ====

Once the system boots:

bash

zpool status -v

Then immediately:

bash

zpool scrub <poolname>

Also check SMART:

bash

smartctl -a /dev/sdX

(Replace sdX with the failing drive)

==== Common root causes I see with SCALE 25.x ====

Based on your setup and symptoms:
* SATA cable degradation
* Power splitter failure
* One HDD entering slow-death (timeouts, not total failure)
* HBA firmware issue (LSI cards especially)

Disks that are “half-alive” are the worst for ZFS.

==== What NOT to do (important) ====

❌ Do NOT:
* Reinstall TrueNAS yet
* Destroy or recreate the pool
* Run zpool clear blindly
* Panic — this is a classic ZFS recovery scenario

==== Next step ====

Reply with:
# Output of:

bash zpool import `

1. Whether it hangs or lists the pool
2. Any error lines from dmesg`

I’ll walk you through the exact recovery path from there.