For Oreo-based MIUI (Android 8.x), the location of "fstab.qcom" has been moved to /system/vendor/etc/fstab.qcom. You need to modify that file accordingly, by changing "encryptable=ice" to "fileencryption=ice".
Edit: If you already encrypted the data partition in Nougat/Oreo MIUI, there is no need to re-format the partition when flashing the new system. The partition will be kept encrypted. But you need a recovery (TWRP) with encryption support (try @raupe
's TWRP Build 20180124-0958 or later version).