EXT3 File System mini-HOWTO Rajesh Fowkar, draft, 23rd August 2001 Revision, 26th October 2001 (Changes suggested by Andreas Dilger) This document provides instructions on how to Convert an existing ext2 file sytem to ext3. It also describes some common reasons for using ext3 file system. 1. Introduction 2. Why to migrate to EXT3 3. Requirements 4. Upgrading Packages (deb & rpm) 5. Get set go... * 5.1 Kernel Compilation (with ext3 patch) * 5.2 Converting ext2 filesystem * 5.3 Creating new ext3 filesytem * 5.4 File System check intervals * 5.5 Changing /etc/fstab * 5.6 ext3 on LVM 6. Examples 7. Acknowledgements 1. Introduction ext3 is a Journalizing file system for Linux. It was written by Dr Stephen C. Tweedie for 2.2 kernels. The filesystem was ported to 2.4 kernels by Peter Braam, Andreas Dilger and Andrew Morton , with much valuable assistance from Stephen Tweedie. 2. Why to migrate to EXT3 ? Why do you want to migrate from ext2 to ext3? Four main reasons: availability, data integrity, speed, and easy transition. Here is why in the words of Michael K. Johnson : Availability: After an unclean system shutdown (unexpected power failure, system crash), each ext2 file system cannot be mounted until its consistency has been checked by the e2fsck program. The amount of time that the e2fsck program takes is determined primarily by the size of the file system, and for today's relatively large (many tens of gigabytes) file systems, this takes a long time. Also, the more files you have on the file system, the longer the consistency check takes. File systems several hundreds of gigabytes in size may take an hour or more to check. This severely limits availability. By contrast, ext3 does not require a file system check even after an unclean system shutdown, except for certain rare hardware failure cases (e.g. hard drive failures), because the data is written to disk in such a way that the file system is always consistent. The time to recover an ext3 file system after an unclean system shutdown does not depend on the size of the file system or the number of files; rather, it depends on the size of the "journal" used to maintain consistency. The default journal size takes about a second to recover (depends on the speed of the hardware). Data integrity: Using the ext3 file system can provide stronger guarantees about data integrity in case of an unclean system shutdown. You have a choice of how carefully to protect your data. Essentially, you can choose either to keep the file system consistent but allow for damage to data on the file system in the case of unclean system shutdown (for a modest speed up under some but not all circumstances) or to ensure that the data is consistent with the state of the file system (which means that you will never see garbage data in recently-written files after a crash.) The more safe choice to keep the data consistent with the state of the file system is the default. Since ext3 has the same on-disk format as ext2 it can use the very well tested and reliable e2fsck to ensure filesystem integrity and recover from errors, when this is needed. Speed: Despite writing some data more than once, ext3 is often faster (higher throughput) than ext2 because ext3's journaling optimizes hard drive head motion. You can choose from three journaling modes to optimize speed, optionally choosing to trade off some data integrity. One mode, data=writeback, limits the data integrity guarantees, allowing old data to show up in files after a crash, for a potential increase in speed under some circumstances. This mode, which is the default journaling mode for most journaling file systems, essentially provides the more limited data integrity guarantees of the ext2 file system and merely avoids the long file system check at boot time. The second mode, data=ordered (the default mode), guarantees that the data is consistent with the file system: recently-written files will never show up with garbage contents after a crash. The last mode, data=journal, requires a larger journal for reasonable speed in most cases and therefore takes longer to recover in case of unclean shutdown, but is sometimes faster for certain database operations, NFS, and synchronous MTA (mail server) operations. The default mode is recommended for all general-purpose computing needs. Easy transition: It is easy to change from ext2 to ext3 and gain the benefits of a robust journaling file system, without reformatting. That's right, no need to do a long, tedious, and error-prone backup, reformat, restore operation in order to experience the advantages of ext3. It is also possible to use an "ext3" filesystem with older kernels that only understand ext2, as long as the filesystem has been cleanly umounted or if e2fsck (1.20+) is run on it. 3. Requirements Make sure you have got the following : 1. Kernel Sources. ( 2.2.x or 2.4.x ) Check out /usr/src/linux for the presence of kernel sources. If not than install them from deb or rpm or tarball file in /usr/src. 2. ext3 patch for the Kernel Different patches are available for 2.2.x and 2.4.x kernels, make sure u download the correct patch. http://www.uow.edu.au/~andrewm/linux/ext3/ 3. The latest version of util-linux package. ( Version >= 2.11 ) Download the latest sources from http://www.kernel.org/pub/linux/utils/util-linux/ NOTE : the latest version of "util-linux" is NOT required unless you want to have "auto" in the /etc/fstab entries for ext3 filesystems. and compile. Follow the instructions given in INSTALL file which comes with the package. This package will contain the most important mount and unmount commands which support ext3 file system. 4. The latest version of e2fsprogs. ( Version >= 1.22 ) Download the latest sources from http://e2fsprogs.sourceforge.net/ and compile. Follow the instructions given in INSTALL file which comes with the package. This new version of e2fsprogs has got mke2fs which supports creation of ext3 file system on a linux native partition. 4. Upgrading Packages(deb & rpm) Upgrading util-linux and e2fsprogs packages on your System : * Debian Users : Debian users can apt-get e2fsprogs from the woody tree. Add the following in /etc/apt/sources.list : deb http://http.us.debian.org/debian woody main contrib non-free Now type the following : apt-get update apt-get install util-linux e2fsprogs This will install the latest versions of util-linux and e2fsprogs packages on your system from the woody tree. * Redhat / Mandrake Users Redhat / Mandrake users download the latest rpm's of util-linux and e2fsprogs from http://www.rpmfind.net You can also download up-to-date RPMs of e2fsprogs from http:// e2fsprogs.sourceforge.net 5. Get, Set, Go... 5.1 Kernel Compilation (with ext3 patch) $ cp ext3-2.4-0.x.y.patch.gz /usr/src $ gunzip ext3-2.4-0.x.y.patch.gz $ patch -p0 < ext3-2.4-0.x.y.patch $ cd /usr/src/linux $ make mrproper $ make menuconfig Under the filesystems menu, select ext3. Please also select "JBD debugging support", as it will produce useful diagnostics if something goes wrong. The filesystem may be compiled into the kernel or built as a module. Building it into the kernel can simplify the gathering of diagnostic information if something fails. WARNING : If u are converting your existing ext2 root file system to ext3 than compile ext3 support in the kernel since u will need it on startup :-) Debian Users : ------------ Once the above make menuconfig step is over, just type make-kpkg --revision=ext31.0 kernel-image to start compilation of the kernel. This will create a deb file in /usr/src with your customised kernel with ext3 support. Just type dpkg -i kernel-image* to install the new kernel :-) Edit your LILO (/etc/lilo.conf) or GRUB (/boot/grub/menu.lst) configuration file to reflect the new kernel. Update the MBR by giving the appropriate command for your boot loader. ( lilo -v for LILO ) Redhat Users : ------------ Redhat users are not so lucky as Debian users. You guys will have to type the following commands to compile and install the kernel: $ make dep $ make clean $ make bzImage $ make install $ make modules $ make modules_install Edit your LILO (/etc/lilo.conf) or GRUB (/boot/grub/menu.lst) configuration file to reflect the new kernel. Update the MBR by giving the appropriate command for your boot loader. ( lilo -v for LILO ) Voila !!! A major headache over .......... Hush............. 5.2 Converting ext2 filesystems An ext2 filesystem maybe converted to ext3 by creating a journal file on it. To do this, run tune2fs -j /dev/hdXX on the target filesystem. The filesystem is now ext3! Note that the filesystem need not be unmounted for this operation. 5.3 Creating new ext3 filesystems Simply run mke2fs -j /dev/hdXX to create a new ext3 filesystem on that device. 5.4 Filesystem check intervals A feature of e2fsck is that it will regularly force a check of a filesystem even if the filesystem is marked clean. Typically, this happens on every twentieth mount or every 180 days, whichever comes first. This still happens with ext3, and is quite possibly not what you want to happen - one of the reasons you chose ext3 was to avoid the downtime which is caused by a long fsck. So it is a good idea to turn this feature off for ext3. Use the command tune2fs -i 0 /dev/hdxx To disable the checking. 5.5 Change /etc/fstab : This is very important. Just change ext2 to ext3 in /etc/fstab For ex. : /dev/hda3 / ext2 defaults,errors=remount-ro 0 1 change this to /dev/hda3 / ext3 defaults,errors=remount-ro 0 1 5.6 ext3 on LVM : (Thanks to Andreas Dilger) Since an ext3 filesystem will not normally be checked at boot time, one option for long-running systems is to use the LVM snapshot feature to create a read-only snapshot of a filesystem and run a read-only e2fsck on the snapshot to ensure that your filesystem has not been corrupted by bad RAM, cables, disk drive, or kernel. You need to have your filesystem on an LVM Logical Volume, and you need the "LVM VFS Enhancement" patch applied for this to work properly (although it may work without the VFS enhancement patch on non-busy filesystems). For example, to run such a check on a Logical Volume (can be done at any time while the system is running, even from a cron script). This assumes that you will not change more than 64MB of data in a single Logical Volume during the course of the e2fsck, and that you have this much space free in each volume group you want to test): # simple but stupid LV listing (hardcoded LV names to check) #for LV in /dev/vg00/lv1 /dev/vg00/lv2 /dev/vg01/lvA /dev/vg01/lvB; do # this assumes that all LVs have ext2/ext3 filesystems on them for LV in `lvscan | sed -n -e '/ACTIVE/s/[^\"]*\"//' -e 's/\".*//p'`; do SNAP=lvtempsnap SNAPDEV=`dirname $lv`/$SNAP lvcreate -L 80M -s -n $SNAP $LV [ $? -ne 0 ] && echo "**** WARNING **** unable to check $LV" && continue e2fsck -fn $SNAPDEV [ $? -ne 0 ] && echo "**** ERROR **** problem with filesystem on $LV" lvremove -f $SNAPDEV [ $? -ne 0 ] && echo "**** WARNING **** unable to remove $SNAP" done 6. Examples THE COMMANDS AND THEIR OUTPUT (What I got on my system) : ******************************************************************** debian:~# mke2fs -j /dev/hda9 mke2fs 1.22, 22-Jun-2001 for EXT2 FS 0.5b, 95/08/09 Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) 134784 inodes, 269080 blocks 13454 blocks (5.00%) reserved for the super user First data block=0 9 block groups 32768 blocks per group, 32768 fragments per group 14976 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376 Writing inode tables: done Creating journal (8192 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 29 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. ******************************************************************** ******************************************************************** debian:~# tune2fs -i 0 /dev/hda9 tune2fs 1.22, 22-Jun-2001 for EXT2 FS 0.5b, 95/08/09 Setting interval between check 0 seconds ******************************************************************** 7. Acknowlegements I would like to thank: * Andreas Dilger specially for LVM snapshot tip and check on a Logical Volume as well as for suggesting some critical changes in other parts of this mini-howto Please visit : http://www.zip.com.au/~akpm/linux/ext3/ext3-usage.html NOTE : Some part of this HOWTO has been taken directly from this page since no better explanation could have been given than what was given on the above page. If you have any suggesstions, additions, corrections in this howto then please do mail them to < rfowkar@goatelecom.com> If you still have any queries, than please join ext3-user mailing list. The link is present on the above page. Please check the archieves of the list first before asking any questions. Archieves are present in .tar.gz format which is excellent. You can just gunzip the downloaded archive and read the mails directly in your mailer. The archives are in mbox format. Cooooooooool.