1. Partitions
A block storage device, e.g. hard disk, USB drive, SD card, etc. can be divided into several logical units, called partitions. Each partition can be formatted as a specific file system and may be installed with a different operating system. Most booting programs, e.g. GRUB, LILO, etc. can be configured to boot up different operating systems from the various partitions. The partition table is at the byte offset 446 (0x1BE) in the very first sector, which is called the Master Boot Record (MBR) of the device. The table has 4 entries, each defined by a 16-byte partition structure, which is
If a partition is EXTEND type (type number =5), it can be divided into more partitions. Assume that partition P4 is EXTEND type and it is divided into extend partitions P5, P6, P7. The extend partitions form a link list in the extend partition area, as shown in Fig. 7.2.
The first sector of each extend partition is a local MBR. Each local MBR also has a partition table at the byte offset 0x1BE, which contains only two entries. The first entry defines the start sector and size of the extend partition. The second entry points to the next local MBR. All the local MBR’s sector numbers are relative to P4’s start sector. As usual, the link list ends with a 0 in the last local MBR. In a partition table, the CHS values are valid only for disks smaller than 8GB. For disks larger than 8GB but fewer than 4G sectors, only the last 2 entries, start_sector and nr_sectors, are meaningful. In the following, we demonstrate fdisk and partitions by examples. Since working with real disks of a computer is very risky, we shall use a virtual disk image, which is just an ordinary file but looks like a real disk.
- Under Linux, e.g. Ubuntu, create a virtual disk image file named mydisk.
dd if=/dev/zero of=mydisk bs=1024 count=1440
dd is a program which writes 1440 (1KB) blocks of zeros to the target file mydisk. We choose count=1440 because it is the number of 1KB blocks of old floppy disks. If desired, the reader may specify a larger block number.
- . Run fdisk on the disk image file:
fdisk mydisk
fdisk is an interactive program. It has a help menu showing all the commands. It collects inputs from the user to create a partition table in memory, which is written to the MBR of the disk image only if the user enters the w command. While in memory, it allows the user to create, examine and modify the partitions. After writing the partition to disk, exit fdisk by the q command. The following figure shows the result of a fdisk session, which partitions the disk image into 3 primary partitions (P1 to P3) and an extend partition P4. The extend partition P4 is further divided into more partitions (P5 to P7), all within the extend partition area. The partition types are all default to Linux when first created. They can be changed to different file system types by the t command. Each file system type has a unique value in HEX, e.g. 0x07 for HPFS/NTFS, 0x83 for Linux, etc. which can be displayed by the t command (Fig. 7.3).
Exercise 7.1: Write a C program to display the partitions, including extend partitions, if any, of a virtual disk image in the same format as does fdisk.
Helps: For readers with less programming experiences, the following may be of help.
- Access partition tables in MBR:
#include <stdio.h>
#include <fcntl.h> char buf[512];
int fd = open(“mydisk”, O_RDONLY); // open mydisk for READ
read(fd, buf, 512); // read MBR into buf[512]
struct partition *p = (struct partition *)&buf[0x1BE];
printf(“p->start_sector = %d\n”, p->start_sector); // access p->start_sector
p+ + ; //advance p to point to the next partition table in MBR, etc.
- Assume that P4 is extend type (type=5) with start_sector = n.
lseek(fd, (long)(n*512), SEEK SET); // lseek to P4 begin
read(fd, char buf[ ], 512); // read local MBR
p = (struct partition *)&buf[0x1BE]); // access local ptable
- Extend partitions form a “link list”, which ends with a NULL “pointer”.
Conversely, we can also write a simple mkfs program to partition a virtual disk by the following steps.
- Open a virtual disk for read/write (O_RDWR).
- Read MBR into a char buf[512]; In a real disk, the MBR may contain the beginning part of a boot program, which must not be disturbed.
- Modify the partition table entries in buf[ ]: In each partition table entry, only the fields type, start_sector and nr_sectors are important. All other fields do not matter, so they can all be left as they are or set to zeros.
- Write buf[ ] back to the MBR.
Exercise 7.2: Write a C program myfdisk to divide a virtual disk into 4 primary partitions. After running the program, run Linux fdisk on the virtual disk to verify the results.
2. Format Partitions
fdisk merely divides a storage device into partitions. Each partition has an intended file system type, but the partition is not yet ready for use. In order to store files, a partition must be made ready for a specific file system first. The operation is traditionally called format a disk or disk partition. In Linux, it is called mkfs, which stands for Make File System. Linux supports many different kinds of file systems. Each file system expects a specific format on the storage device. In Linux, the command
mkfs -t TYPE [-b bsize] device nblocks
makes a file system of TYPE on a device of nblocks, each block of bsize bytes. If bsize is not specified, the default block size is 1KB. To be more specific, we shall assume the EXT2/3 file system, which used to be the default file systems of Linux. Thus,
mkfs -t ext2 vdisk 1440 OR mke2fs vdisk 1440
formats vdisk to an EXT2 file system with 1440 (1KB) blocks. A formatted disk should be an empty file system containing only the root directory. However, mkfs of Linux always creates a default lost +found directory under the root directory. After mkfs, the device is ready for use. In Linux, a new file system is not yet accessible. It must be mounted to an existing directory in the root file system. The / mnt directory is usually used to mount other file systems. Since virtual file systems are not real devices, they must be mounted as loop devices, as in
sudo mount -o loop vdisk /mnt
which mounts vdisk onto the /mnt directory. The mount command without any parameter shows all the mounted devices of the Linux system. After mounting, the mounting point /mnt becomes equivalent to the root directory of the mounted device. The user may change directory (cd) into /mnt, do file manipulations on the device as usual. After using the mounted device, cd out of /mnt, then enter
sudo umount /mnt OR sudo umount vdisk
to un-mount the device, detaching it from the root file system. Files saved on the device should be retained in that device.
It is noted that most current Linux systems, e.g. Slackware 14.1 and Ubuntu 15.10, can detect portable devices containing file systems and mount them directly. For example, when a Ubuntu user inserts a USB drive into a USB port, Ubuntu can detect the device, mount it automatically and display the files in a pop up window, allowing the user to access files directly. The user may enter the mount command to see where the device (usually /dev/sdb1) is mounted on, as well as the mounted file system type. If the user pulls out the USB drive, Ubuntu will detect it also and umount the device automatically. However, pulling out a portable device arbitrarily may corrupt data on the device. This is because the Linux kernel typically uses delayed writes of data to devices. To ensure data consistency, the user should umount the device first before detaching it.
The Linux mount command can mount partitions of real devices or an entire virtual disk, but it can not mount partitions of a virtual disk. If a virtual disk contains partitions, the partitions must be associated with loop devices first. The following example demonstrates how to mount partitions of virtual disks.
3. Mount Partitions
Man 8 losetup: display losetup utility command for system administration
- Create a virtual disk image by the dd command:
dd if=/dev/zero of=vdisk bs=1024 count=32768 #32K (1KB) blocks
- Run fdisk on vdisk to create a partition P1
fdisk vdisk
Enter the n (new) command to create a partition P1 by using default start and last sector numbers. Then, enter the w command to write the partition table to vdisk and exit fdisk. vdisk should contain a partition P1=[start=2048, end=65535]. The partition is 63488 sectors.
- Create a loop device on partition 1 of vdisk using sector numbers
losetup -o $(expr 2048 \* 512) –sizelimit $(expr 65535 \* 512) /dev/loop1 vdisk
losetup needs the start byte (start_sector*512) and end byte (end_sector*512) of the partition. The reader may compute these values by hand and use them in the losetup command. Loop devices for other partitions can be set up in a similar way. After creating a loop device, the reader may use the command
losetup -a
which displays all loop devices as /dev/loopN.
- Format /dev/loop1 an EXT2 file system
mke2fs -b 4096 /dev/loop1 7936 # mke2fs with 7936 4KB blocks
The size of the partition is 63488 sectors. The number of 4KB blocks is 63488 / 8 = 7936
- Mount the loop device
mount /dev/loop1 /mnt # mount as loop device
- Access mounted device as part of the file system
(cd /mnt; mkdir bin boot dev etc user) # populate with DIRs
- When finished with the device, umount it.
umount /mnt
- When finished with a loop device, detach it by the command
losetup –d /dev/loop1 # detach a loop device.
Exercise 7.3: Partition a vdisk into 4 partitions. Create 4 loop devices for the partitions. Format the partitions as EXT2 file systems, mount the partitions and copy files into them. Then umount the loop devices and detach the loop devices. If the reader knows how to write sh scripts, all the above steps can be done by a simple sh script.
Source: Wang K.C. (2018), Systems Programming in Unix/Linux, Springer; 1st ed. 2018 edition.