NTFS for Win98

Users Guide
Winternals Software LP 3101 Bee Caves Road, Suite 150 Austin, Texas 78746 (512) 330-9130 (512) 330-9131 Fax www.winternals.com
Copyright 1999 Winternals Software LP

Table of Contents

9 Introduction... 1 Requirements... 2 Approach.... 3 Limitations... 4 Recommended Configuration... 5 Installation... 6 CHKDSK... 8 Advanced Settings... 10 Technical Support... 11

Winternals Software LP

Page i

Introduction

Welcome to NTFS for Win98. NTFS for Win98 is a NTFS file system driver for Windows 95 and Windows 98. Once installed, any NTFS drives present on your system will be fully accessible as native Windows 98 volumes. With NTFS for Win98, users can share data in dual boot environments and facilitate migration to Windows NT/2000. Please read this entire guide before using NTFS for Win98.

Page 1

Requirements
NTFS for Win98 works with the following versions of NTFS: NTFS for Windows NT 4.0, SP3 and higher NTFS for Windows 2000, Beta 3, RC1, RC2 NTFS for Win98 works on the following versions of Windows: Windows 95 Windows 95 OSR2 Windows 98 Windows 98 SE

Page 2

Approach
NTFS for Win98 takes a unique approach to providing cross-platform file system support. Rather than implement code to read and write NTFS drives, NTFS for Win98 uses the NTFS and NTOSKRNL system files from a Windows NT or Windows 2000 configuration. NTFS for Win98 wraps the Windows NT/2000 NTFS driver in a run-time environment that simulates the Windows NT environment the NTFS driver is written to use. Thus, NTFS for Win98 does not rely on potentially unreliable reverse-engineered information about NTFS, provides ultimate compatibility with NTFS, and takes advantage of Microsoft NTFS bug fixes whenever you update the NTFS driver file NTFS for Win98 uses to a more recent version.

Figure 3-1

Figure 3-1 depicts NTFS for Win98s relationship with NTFS, NTOSKRNL and the Windows 95/98 file system. NTFS for Win98 provides an external interface that looks like a standard Windows 95/98 file system, while internally it presents an interface to NTFS that looks like the Windows NT/2000 environment. NTFS for Win98 uses a number of services within NTOSKRNL to help implement this environment. Note that NTFS for Win98 does not load NTFS or NTOSKRNL unless you have at least one NTFS drive on your system.

Page 3

Limitations
Because the Windows 95/98 environment is different than that of Windows NT/2000, the level of NTFS support that NTFS for Win98 provides is not equivalent to that provided on Windows NT/2000. For example, NTFS for Win98 does not enforce the Windows NT/2000 security model. All files and directories are accessible, regardless of their security permissions settings. In addition, NTFS for Win98 is not compatible with Windows 2000s Encrypting file System (EFS). Any files encrypted using EFS will not be accessible under Windows 98 using NTFS for Win98. NTFS for Win98 does not provide access to software-implemented NTFS volume sets, stripe sets or mirrored drives.

Page 4

Recommended Configuration
Do not convert your first partition, or your Windows 95/98 boot partition (the one with \windows on it), to NTFS as there is no support in Windows 95 or Windows 98 for reading NTFS drives during the boot sequence. For maximum compatibility in dual boot systems, the recommended partition configuration is to maintain a FAT partition as the first partition on the primary drive. This partition should contain Windows 95/98 and should not be used to store applications or data files. The rest of the primary disks, and any other disks that you wish to share between Windows 98 and Windows NT/2000 can be formatted with NTFS or other file system types (FAT16 or FAT32) that are understood by the operating systems from which you wish to access them. Your Windows NT/2000 system directory can be placed on either a FAT or NTFS drive (note that Windows NT 4.0 cannot be placed on a FAT32 drive, nor boot from a FAT32 drive). Table 5-1 shows a file system compatibility chart that can help you determine which file systems to use for compatibility in your dual boot environment. Windows 95 FAT16 FAT32 NTFS Windows 95/OSR2 Windows 98 Windows NT Windows 2000
Operating system can boot from the file system Operating system can access the file system, but not boot from it
With Winternals Softwares FAT32 for Windows NT 4.0 With Winternals Softwares NTFS for Win98

Table 5-1

Page 5

Installation

Before you run the NTFS for Win98 installation program, you must have access to a number of files (listed below) from the Windows NT/2000 installation you use to access your NTFS drives. This means that if the files are located on a NTFS drive you will have to copy them to a FAT drive accessible from Windows 98. During the NTFS for Win98 setup procedure you will be prompted for the location of these files. You may specify either the system directory of a Windows NT/2000 installation (e.g. c:\winnt), or a directory into which youve copied the necessary files. The files that you must make available to NTFS for Win98 are: NTFS.SYS NTOSKRNL.EXE AUTOCHK.EXE NTDLL.DLL C_437.NLS C_1252.NLS L_INTL.NLS this file is located at winnt>\system32\drivers\ntfs.sys this file is located at <winnt>\system32\ntoskrnl.exe this file is located at <winnt>\system32\autochk.exe this file is located at <winnt>\system32\ntdll.dll this file is located at <winnt>\system32\c_437.nls this file is located at <winnt>\system32\c_1252.nls this file is located at <winnt>\system32\l_intl.nls

<winnt> designates the system directory of the Windows NT/2000 installation that contains the NTFS driver you normally use to access your NTFS drives. The setup procedure allows you to assign drive letters to NTFS drives that NTFS for Win98 mounts. Simply enter a string in the drive-letter selection entry that designates, in order, the drive letters for NTFS for Win98 to assign. For example, if you want the first NTFS drive mounted to have a drive letter of D and the second to have a drive letter of T, you would enter dt (without the quotation marks). Note that the entry is case-insensitive. Leaving the entry blank has NTFS for Win98 assign the first available drive letter to each mounted NTFS drive.

Page 6

After the setup procedure is complete you are prompted to reboot your computer. The next time you boot the Windows 95 or 98 system on which you installed NTFS for Win98 you will have access to your computers NTFS volumes. You may rerun the configuration utility at any time to select different drive letters or a different NTFS file.

Page 7

CHKDSK
NTFS for Win98 includes a scandisk for NTFS drives named NTFSCHK. In the same way that NTFS for Win98 creates an NT-like environment for the NTFS driver, NTFSCHK creates an NT-like environment for the AUTOCHK.EXE (CHKDSK) program of a Windows NT/2000 installation. The files that make up this environment include AUTOCHK.EXE, NTDLL.DLL, and three character mapping files, C_437.NLS, C_1252.NLS, and L_INTL.NLS. The NTFS for Win98 setup program adds the execution of NTFSCHK to your AUTOEXEC.BAT file so that the consistency of your NTFS drives is checked every time you boot into Windows 95/98. You may also run NTFSCHK from DOS before Windows 95/98 starts, from a DOS-box command prompt, or by selecting the NTFSCHK icon in the NTFS for Win 98 program group in the Start menu. When NTFSCHK runs from AUTOEXEC.BAT and when you select the icon NTFSCHK runs in read-only mode. See the options below for running NTFSCHK in repair mode. In some cases NTFSCHK will indicate that it cannot lock the specified drive(s). This condition results when applications or Windows has active handles to open files on the NTFS drive for which NTFSCHK reports the condition. If the applications in question do not close their handles, or Windows keeps open handles, you must reboot the computer and run NTFSCHK before the drive is accessed by applications or Windows, or reboot into Windows NT/2000 and perform a CHKDSK operation there. While NTFSCHK scans a drive that drive will not be accessible. If you click on the drive icon in Explorer you will receive an access denied error message. If Explorer is opened to directories on a drive being checked and you attempt to navigate the directories you will receive errors indicating that the directories do not exist.

Page 8

NTFSCHKs command-line usage is as follows (selecting the NTFSCHK program icon results in a check of all the computers NTFS drives): ntfschk [/?] [/s] [/f] [/q] [ drive letter: | * ] /? /s Displays NTFSCHK usage. This switch has NTFSCHK display the list of detected NTFS drives. Use this command to determine which drive letters correspond to NTFS volumes. Unless you specify this flag NTFSCHK runs in read-only mode, where it detects and reports errors but does not correct them. Use this flag to fix drives for which NTFSCHK reports corruption. This switch has NTFSCHK perform a quick check of the specified NTFS drive(s) by not setting the NTFS drive dirty flag on the disk(s). Thus, NTFSCHK will only check the drive if it is marked as being in a possibly inconsistent state. Has NTFSCHK check all NTFS drives, or you can specify a specific drive letter (e.g. D:) to check.

Page 9

Advanced Settings
The following Registry configuration settings are used by NTFS for Win98. You should only change their default values for trouble-shooting purposes or upon direction from Winternals Software technical support. Certain values may result in poor performance or data loss on NTFS drives. All values reside under HKEY_LOCAL_MACHINE\Software\Winternals\NTFSforWin98. These settings are accessible from the Advanced dialog of the NTFS for Win98 configuration utility. Ntfs Ntoskrnl WriteThrough Specifies the full pathname of the NTFS.SYS file NTFS for Win98 loads. Specifies the full pathname of the NTOSKRNL.EXE file NTFS for Win98 loads. If set, NTFS for Win98 will immediately write any NTFS disk modifications to disk instead of buffering the operations in the Windows 98 file system cache. Default=0. If set, no disk modifications of NTFS drives occur, even though it will appear that they do. Note that performing a large amount of modifications on NTFS drives when this setting is enabled can lead to a crash, since NTFS will eventually encounter disk blocks with information inconsistent with what it thinks it has written. Default=0.

ReadOnly

CheckpointInterval NTFS is a logging file system, meaning that it maintains updates to metadata structures (like directories and the Master File Table) in a log file before the changes have been fully committed to disk. NTFS periodically commits modifications with an operation called a checkpoint. This setting specifies the number of seconds between checkpoints. Default=10. WritebackInterval This parameter specifies the period, in seconds, at which NTFS for Win98 writes back dirty disk blocks to disk. The longer the interval the better overall performance, but the more data that may be lost in the case of an abrupt system shutdown or crash. Default=1.

Page 10

Technical Support
If you encounter a problem while using NTFS for Win98 and wish to receive technical assistance, please visit http://www.winternals.com/es/support/getsupport.asp or send e-mail to support@winternals.com
To search the Winternals Knowledge Base, find answers to
Frequently Asked Questions, or download free updates to this product, please visit our Support pages at http://www.winternals.com/es/support
Telephone support is not available for NTFS for Win98.

Page 11

CSMC 412
Operating Systems Prof. Ashok K Agrawala
2006 Ashok Agrawala Set 11
Operating System Concepts
File System Implementation
File-System Structure File-System Implementation Directory Implementation Allocation Methods Free-Space Management Efficiency and Performance Recovery Log-Structured File Systems NFS Example: WAFL File System

Objectives

To describe the details of implementing local file systems and directory structures To describe the implementation of remote file systems To discuss block allocation and free-block algorithms and trade-offs

File System Structure

Primary device for File System Disks Direct Access Written in place

File-System Structure

File structure Logical storage unit Collection of related information File system resides on secondary storage (disks)
Direct Access Written in place
File system organized into layers File control block storage structure consisting of information about a file

Layered File System

Several disk and in memory structures are used to support the file system On disk the file system contains a variety of information Boot information Disk information No of blocks Directory Structure Free blocks Files their data
Boot Control Block per volume Information to boot the OS typically the first block of the disk. Boot Block in UFS Partition Boot Sector in NTFS Volume Control Block per volume Contains volume or partition details No of blocks. Size of clocks, free block count, free-block pointers, free FCB count Super Block in UFS Stored in Master File table in NTFS Directory Structure per file system In UFS INODE number and File name In NTFS Master File Table File Control Block per file
A Typical File Control Block

In Memory Information

Mount Table Directory Structure cache Information about recently accessed directories System-wide Open-file Table Copy of FCB of all open files Additional information Per-process open-file Table Pointers into System-wide Open File Table.
In-Memory File System Structures

File Operations

Creating a file Calls logical file system knows the format of the directory Allocates/creates an FCB Reads the directory into memory Updates it with the new file information Writes back to the disk Opening a file Call passes a file name Search System-Wide Open File Table to see if it is open If there create a per-process entry and point to it Search directory structure for the file name Copy FCB into System-Wide open file table Make an entry in per-process open file table Returns a pointer to per-process table entry
Read and Write Operations Use process open file table entry which also has the pointer This entry is called File Descriptor in UFS and File Handle in Windows Close Remove process table entry Decrease the open count in the System-wide table. If zero, write back any updated meta data to disk and remove the entry. Caching of information done for efficiency

Partitions and Mounting

Partitions or volumes Divide one disk to look like multiple disks Manage by having a partition table Each partition is treated as a raw disk dual boot Root partition Contains the OS kernel and other system files Mounted at Boot time Other volumes can be mounted if desired In Memory Mount Table

Virtual File Systems

Virtual File Systems (VFS) provide an object-oriented way of implementing file systems. VFS allows the same system call interface (the API) to be used for different types of file systems. The API is to the VFS interface, rather than any specific type of file system.
Schematic View of Virtual File System

Directory Implementation

Linear list of file names with pointer to the data blocks. simple to program time-consuming to execute Hash Table linear list with hash data structure. decreases directory search time collisions situations where two file names hash to the same location fixed size

Allocation Methods

An allocation method refers to how disk blocks are allocated for files: Contiguous allocation Linked allocation Indexed allocation

Contiguous Allocation

Each file occupies a set of contiguous blocks on the disk Simple only starting location (block #) and length (number of blocks) are required Random access Wasteful of space (dynamic storage-allocation problem) Files cannot grow
Mapping from logical to physical

Q LA/512 R

Block to be accessed = Q + starting address Displacement into block = R
Contiguous Allocation of Disk Space

Extent-Based Systems

Many newer file systems (I.e. Veritas File System) use a modified contiguous allocation scheme Extent-based file systems allocate disk blocks in extents An extent is a contiguous block of disks Extents are allocated for file allocation A file consists of one or more extents.

Linked Allocation

Each file is a linked list of disk blocks: blocks may be scattered anywhere on the disk.

pointer

Linked Allocation (Cont.)
Simple need only starting address Free-space management system no waste of space No random access Mapping Q LA/511 R
Block to be accessed is the Qth block in the linked chain of blocks representing the file. Displacement into block = R + 1 File-allocation table (FAT) disk-space allocation used by MS-DOS and OS/2.

File-Allocation Table

Indexed Allocation
Brings all pointers together into the index block. Logical view.

index table

Example of Indexed Allocation
Indexed Allocation (Cont.)
Need index table Random access Dynamic access without external fragmentation, but have overhead of index block. Mapping from logical to physical in a file of maximum size of 256K words and block size of 512 words. We need only 1 block for index table. Q LA/512 R Q = displacement into index table R = displacement into block
Indexed Allocation Mapping (Cont.)
Mapping from logical to physical in a file of unbounded length (block size of 512 words). Linked scheme Link blocks of index table (no limit on size).

Q1 LA / (512 x 511) R1

Q1 = block of index table R1 is used as follows:

R1 / 512

Q2 = displacement into block of index table R2 displacement into block of file:
Two-level index (maximum file size is 5123)

Mapping of a File Descriptor to an Inode
System calls that refer to open files indicate the file is passing a file descriptor as an argument. The file descriptor is used by the kernel to index a table of open files for the current process. Each entry of the table contains a pointer to a file structure. This file structure in turn points to the inode. Since the open file table has a fixed length which is only setable at boot time, there is a fixed limit on the number of concurrently open files in a system.
File-System Control Blocks

Disk Structures

The one file system that a user ordinarily sees may actually consist of several physical file systems, each on a different device. Partitioning a physical device into multiple file systems has several benefits. Different file systems can support different uses. Reliability is improved Can improve efficiency by varying file-system parameters. Prevents one program form using all available space for a large file. Speeds up searches on backup tapes and restoring partitions from tape.

Disk Structures (Cont.)

The root file system is always available on a drive. Other file systems may be mounted i.e., integrated into the directory hierarchy of the root file system. The following figure illustrates how a directory structure is partitioned into file systems, which are mapped onto logical devices, which are partitions of physical devices.
Mapping File System to Physical Devices

Implementations

The user interface to the file system is simple and well defined, allowing the implementation of the file system itself to be changed without significant effect on the user. For Version 7, the size of inodes doubled, the maximum file and file system sized increased, and the details of free-list handling and superblock information changed. In 4.0BSD, the size of blocks used in the file system was increased form 512 bytes to 1024 bytes increased internal fragmentation, but doubled throughput. 4.2BSD added the Berkeley Fast File System, which increased speed, and included new features. New directory system calls truncate calls Fast File System found in most implementations of UNIX.
Layout and Allocation Policy
The kernel uses a <logical device number, inode number> pair to identify a file. The logical device number defines the file system involved. The inodes in the file system are numbered in sequence. 4.3BSD introduced the cylinder group allows localization of the blocks in a file. Each cylinder gorup occupies one or more consecutive cylinders of the disk, so that disk accesses within the cylinder group require minimal disk head movement. Every cylinder group has a superblock, a cylinder block, an array of inodes, and some data blocks.

Three Major Layers of NFS Architecture
UNIX file-system interface (based on the open, read, write, and close calls, and file descriptors) Virtual File System (VFS) layer distinguishes local files from remote ones, and local files are further distinguished according to their file-system types The VFS activates file-system-specific operations to handle local requests according to their file-system types Calls the NFS protocol procedures for remote requests NFS service layer bottom layer of the architecture Implements the NFS protocol
Schematic View of NFS Architecture
NFS Path-Name Translation
Performed by breaking the path into component names and performing a separate NFS lookup call for every pair of component name and directory vnode To make lookup faster, a directory name lookup cache on the clients side holds the vnodes for remote directory names

NFS Remote Operations

Nearly one-to-one correspondence between regular UNIX system calls and the NFS protocol RPCs (except opening and closing files) NFS adheres to the remote-service paradigm, but employs buffering and caching techniques for the sake of performance File-blocks cache when a file is opened, the kernel checks with the remote server whether to fetch or revalidate the cached attributes Cached file blocks are used only if the corresponding cached attributes are up to date File-attribute cache the attribute cache is updated whenever new attributes arrive from the server Clients do not free delayed-write blocks until the server confirms that the data have been written to disk
Example: WAFL File System
Used on Network Appliance Filers distributed file system appliances Write-anywhere file layout Serves up NFS, CIFS, http, ftp Random I/O optimized, write optimized NVRAM for write caching Similar to Berkeley Fast File System, with extensive modifications

The WAFL File Layout

Snapshots in WAFL
File Systems supported by Windows NTFS Design Goals File System Driver Architecture NTFS Operation Windows File System On-Disk Structure NTFS File Compression
Windows File System - Terminology

Sectors:

hardware-addressable blocks on a storage medium Typical sector size on hard disks for x86-based systems is 512 bytes

File system formats:

Define the way data is stored on storage media Impact a file system features: permissions & security, limitations on file size, support for small/large files/disks

Clusters:

Addressable blocks that many file system formats use Cluster size is always a multiple of the sector size Cluster size tradeoff: space efficiency vs. access speed

Metadata:

Data stored on a volume in support of file system format management
Metadata includes the data that defines the placement of files and directories on a volume, for example
Typically not accessible to applications
Formats Supported by Windows
CD-ROM File System (CDFS) Universal Disk Format (UDF) File Allocation Table (FAT12, FAT16, and FAT32) New Technology File System (NTFS)
CDFS, or, is a relatively simple format that was defined in 1988 as the read-only formatting standard for CD-ROM media. Windows 2000 implements ISO 9660-compliant CDFS in \Winnt\System32\Drivers\Cdfs.sys, with long filename support defined by Level 2 of the ISO 9660 standard Because of its simplicity, the CDFS format has a number of restrictions Directory and file names must be fewer than 32 characters long Directory trees can be no more than eight levels deep CDFS is considered a legacy format because the industry has adopted the Universal Disk Format (UDF) as the standard for read-only media
OSTA (Optical Storage Technology Association) defined UDF in 1995 as a format to replace CDFS for magneto-optical storage media, mainly DVDROM The Windows 2000 UDF file system implementation is ISO 13346compliant and supports UDF versions 1.02 and 1.5 UDF file systems have the following traits: Filenames can be 255 characters long The maximum path length is 1023 characters Although the UDF format was designed with rewritable media in mind, the Windows 2000 UDF driver (\Winnt\System32\Drivers\Udfs.sys) provides read-only support
FAT (File Allocation Table) file systems are a legacy format that originated in DOS and Windows 9x Reasons why Windows supports FAT file systems: to enable upgrades from other versions of Windows compatibility with other operating systems in multiboot systems as a floppy disk format Windows FAT file system driver is implemented in \Winnt\System32\Drivers\Fastfat.sys Each FAT format includes a number that indicates the number of bits the format uses to identify clusters on a disk

Boot sector

File allocation table 1
File allocation table 2 (duplicate)
Root directory Other directories and all files

FAT format organization

Operating System Concepts 11.79
FAT12's 12-bit cluster identifier limits a partition to storing a maximum of 212 (4096) clusters Windows uses cluster sizes from 512 bytes to 8 KB in size, which limits a FAT12 volume size to 32 MB Windows uses FAT12 as the format for all 5-inch floppy disks and 3.5-inch floppy disks, which store up to 1.44 MB of data
FAT16, with a 16-bit cluster identifier, can address 216 (65,536) clusters On Windows, FAT16 cluster sizes range from 512 bytes (the sector size) to 64 KB, which limits FAT16 volume sizes to 4 GB The cluster size Windows uses depends on the size of a volume
FAT32 is the most recently defined FAT-based file system format it's included with Windows 95 OSR2, Windows 98, and Windows Millennium Edition FAT32 uses 32-bit cluster identifiers but reserves the high 4 bits, so in effect it has 28bit cluster identifiers Because FAT32 cluster sizes can be as large as 32 KB, FAT32 has a theoretical ability to address 8 TB volumes Although Windows works with existing FAT32 volumes of larger sizes (created in other operating systems), it limits new FAT32 volumes to a maximum of 32 GB FAT32's higher potential cluster numbers let it more efficiently manage disks than FAT16; it can handle up to 128-MB volumes with 512-byte clusters Unlike FAT12 and FAT16, root directory is not fixed size or location Largest file size on Windows is 4GB (largest on Win9x is 2G)
NTFS is the native file system format of Windows NTFS uses 64-bit cluster indexes Theoretical ability to address volumes of up to 16 exabytes (16 billion GB) Windows 2000 limits the size of an NTFS volume to that addressable with 32-bit clusters, which is 128 TB (using 64-KB clusters) Why use NTFS instead of FAT? FAT is simpler, making it faster for some operations, but NTFS supports: Larger file sizes and disks Better performance on large disks, large directories, and small files Reliability Security
CIFS the Common Internet File System
The standard Windows network file system The file sharing protocol at the heart of CIFS is an updated version of the Server Message Block (SMB) protocol dates back to the mid-1980s in 1996/97, Microsoft submitted draft CIFS specifications to the IETF The SMB protocol was originally developed to run over NetBIOS (Network Basic Input Output System) LANs Until Windows 2000, NetBIOS support was required for SMB transport The machine and service names visible in the Windows Network Neighborhood are, basically, NetBIOS addresses (Windows 2000 and later use DNS names) Windows 3.11 (WfW) introduced: service announcement and location system called Browsing The browser service provides the list of available file and print services presented in the Network Neighborhood Workgroup concept was expanded to create NT Domains

File System Format Compatibility
FAT12/FAT16 supported on all Microsoft OSs FAT32: Only Windows 2000/XP/2003 Winternals FAT32 driver for NT4 NTFS: Only Windows NT-based OSs Winternals NTFSDOS for DOS access Winternals NTFS for Windows 98 for Win9x/Me

NTFS Design Goals

Overcome limitations inherent in FAT / HPFS FAT (File Allocation Table) does not support large disks very well FAT16 (MS-DOS file system) supports only up to 216 clusters and 2 GB disks (with 64 Kb clusters!!) FAT / root directory represents single point of failure Number of entries in root directory is limited HPFS removed some of FATs limitations, but still did not support recoverability, security, data redundancy, and fault-tolerance (later versions of HPFS support up to 2TeraByte disks)

NTFS Recoverability

PC disk I/O in the old days: Speed was most important NTFS changes this view Reliability counts most: I/O operations that alter NTFS structure are implemented as atomic transactions Change directory structure, extend files, allocate space for new files Transactions are either completed or rolled back NTFS uses redundant storage for vital FS information Contrasts with FAT / HPFS on-disk structures, which have single sectors containing critical file system data Read error in these sectors -> volume lost
NTFS Security and Recoverability
NTFS security is derived from Windows object model
Open file is implemented as file object; security descriptor is stored on disk as part of the file NT security system verifies access rights when a process tries to open a handle to any object Administrator or file owner may set permissions
NTFS recoverability ensures integrity of FS structure
No guarantees for complete recovery of user files Layered driver model + FTDISK driver
Mirroring of data RAID level 1 Striping of data - RAID level 5 (one disk with parity info)
Large Disks and Large Files
Efficient support for large files and disks in NTFS FAT16:
16-bit wide table stores allocation status of disk Up to 65.536 clusters per volume (#files !!); adjustable cluster size

FAT32:

New in since Windows 2000 4kb clusters on volumes up to 8 GB Can relocate root directory / use backup copy of FAT Root directory is ordinary cluster chain no limits on #entries
HPFS (support dropped in NT 4.0):
32 bits to enumerate allocation units; maximum file size: 4GB Allocates disk space in terms of physical sectors of 512 bytes; problem with some disks (1024 bit sectors)
Large Disks and Large Files (contd.)
NTFS enumerates cluster with 64-bit numbers Up to 264 clusters of up to 64 Kbytes size Maximum file size: 264 bytes Cluster size is adjustable

Change Logging

Many types of applications, such as incremental backup utilities, need to monitor a volume for changes An obvious way to watch for changes is to perform a full scan Very performance inefficient There is a way for an application to wait on a directory and be told of notifications An application can miss changes since it must specify a buffer to hold them
With Windows 2000, NTFS introduces the change log, which is a sparse metadata file that records file system events (not enabled by default) As the file exceeds its maximum on-disk size, NTFS frees the disk space for the oldest portions marking them empty An application uses Win32 APIs to read events The log file is shared, and generally large enough that an application wont miss changes even during heavy file system activity

Per-User Volume Quotas

NTFS quota-management support allows for per-user specification of quota enforcement Can be configured to log an event indicating the occurrence to the system Event Log if a user surpasses his warning limit If a user attempts to use more volume storage then her quota limit permits, NTFS can log an event to the system Event Log and fail the application file I/O that would have caused the quota violation with a "disk full" error code User disk space is tracked on a per-volume basis by summing the logical sizes of all the files and directories that have the user as the owner in their security descriptors

11.100

Link Tracking
Several types of symbolic file links are used by layered applications Shell shortcuts allow users to place files in their shell namespace (on their desktop, for example) that link to files located in the file system namespace Object linking and embedding (OLE) links allow documents from one application to be transparently embedded in the documents of other applications In the past, these links were difficult to manage If someone moved a link source (what a link points to), the link broke Windows now has a link-tracking service, TrkWks (it runs in services.exe), tags link sources with a unique object ID NTFS can return the name of a file given a link, so if the link moves the service can query each of a systems volume for the object ID A distributed link-tracking service, TrkSvr, works to track link source movement across systems

11.101

Encryption
While NTFS implements security for files and directories, the security is ineffective if the physical security of the computer is compromised Can install a parallel copy of Windows NTFSDOS Encrypting File System (EFS) Like compression, its operation is transparent Also like compression, encryption is a file and directory attribute Files that are encrypted can be accessed only by using the private key of an account's EFS private/public key pair, and private keys are locked using an account's password While you might think that its implemented as a file system filter driver, its a driver thats tightly connected to NTFS

11.102

POSIX Support
POSIX support requires two file system features: Primary group in security descriptor Case-sensitive names

11.103

Defragmentation
Fragmentation: A file is fragmented if its data occupies discontiguous clusters

11.104

A common myth is that NTFS doesnt fragment, but it does Defragmentation APIs have been present since NT 4 Windows 2000 introduced a non-schedulable graphical defragmenter A command line interface was added in Windows XP

11.105

Compression and Sparse Files
NTFS supports transparent compression of files When a directories is marked compressed it means any files or subdirectories are marked compressed Compression is performed on 16-cluster blocks of a file Use Explorer or the compact tool to compress files (compact shows compression rations for compressed files) Sparse files are an application-controlled form of compression that define parts of a file as empty those areas dont occupy any disk space Applications use Windows APIs to define empty areas

11.106

NTFS File System Driver

Flush the log file

Log file service Write the cache Cache manager
Log the transaction Read/write the file
I/O manager NTFS driver Fault tolerant driver
Read/write a mirrored or striped volume Read/write the disk
Access the mapped file or flush the cache Virtual memory manager
Load data from disk into memory

Disk driver

11.107
Components related to NTFS

Cache Manager

System wide caching for NTFS and other file systems drivers Including network file system drivers (server and redirectors) Cached files are mapped into virtual memory Specialized interface from Cache Manager to NT virtual memory manager Memory manager calls NTFS to access disk driver and obtain file

Log File Service

2 copies of transaction logs Transaction log is flushed to disk before write-data is sent to disk Cache manager performs actual flush operation

Security descriptor data
Specifies who owns the file and who can access it Contents of the file; a file has one default unnamed data attribute; directory has no default data attrib. Three attributes used to implement filename allocation, bitmap index for large directories (dirs. only) List of attributes that make up the file and first reference of the MFT record in which the attribute is located (for files which require multiple MFT file records)

Index root, index

Attribute list

11.118

Attributes (contd.)
Each attribute in a file record has a name and a value NTFS identifies attributes: Uppercase name starting with $: $FILENAME, $DATA Attributes value: Byte stream The filename for $FILENAME The data bytes for $DATA Attribute names correspond to numeric typecodes File attributes in an MFT record are ordered by typecodes Some attribute types may appear more than once (e.g. Filename)

11.119

Filenames
POSIX: Case-sensitive, trailing periods & spaces NTFS namespace equiv. to POSIX space Win32: Long filenames, unicode names Multiple dots, embedded spaces, beginning dots MS-DOS: 8.3 names, case does not matter NTFS generates MS-DOS names for Win32 files automatically Fully functional aliases for NTFS names Stored in same directory as long names; dir /x

Namespaces

POSIX subsystem Win32 subsystem MS-DOS Win16 clients

11.120

MS-DOS filenames in NTFS
Standard info NTFS filename MS-DOS filename Security desc. Data
MFT file record with MS-DOS filename attribute
NTFS name and MS-DOS name are stored in same file record and refer to same file
Renaming changes both filenames Open, read, write, delete work with both names equally
POSIX hardlinks are implemented in similar way
Deleting a file with multiple names only decreases link count
Generation of MS-DOS names: 1. 2. 3.
Remove all illegal chars; remove all but one period; truncate to 6 chars Append ~1 to name; truncate extension to 3 chars; all uppercase Increment ~1 if filename duplicates an existing name in directory

11.121

Resident & Nonresident Attributes
Small files: All attributes and values fit into MFT Attribute with value in MFT is called resident All attributes start with header (always resident) Header contains offset to attr. value and length of value

Standard info

NTFS filename

Security desc.

header RESIDENT Offset: 8h Length: 14h

value MYFILE.DAT

11.122
Small directory: index root attribute contains index of file references for files and subdirectories
Standard info NTFS filename Security desc. Index root Index of files file1, file2, file3,. MFT file record for a small directory Empty
If file attribute does not fit into MFT:
NTFS allocates separate cluster (run, extent) to store the values NTFS allocates additional runs if an attributes value later grows Those attributes are called non-resident Header of non-resident attribute contains location info

11.123

Large files & directories
NTFS filename Security desc.

HPFS extended attr.

MFT record for large file with 2be non-resident Only attributes that can grow can data runs
Filename & standard info are always resident Index of files for directories forms B+ tree
NTFS filename Security desc. Index root Index allocation Index of files file4, file8

VCN-to-LCN mappings

Bitmap
MFT file record for a large directory with nonresident filename index file1, file2, file3
Operating System Concepts 11.124

file5, file6

Large files (contd.)
NTFS keeps track of runs by means of VCN
(Virtual Cluster Numbers)
Logical Cluster Numbers represent an entire volume Virtual Cluster Numbers represent clusters belonging to one file Attribute lists may extend over multiple runs (not only data)

Starting VCN

Starting LCN Number of clusters
VCN-to-LCN mappings for a nonresident data attribute

1355 1588

2 Data

6 Data

LCN 1357 1358
Operating System Concepts 11.125

LCN 1590 1591

Data Compression
NTFS supports compression
Per-file, per-directory, per-volume basis NTFS compression is performed on user data only, not NTFS metadata
Inspect files/volume via Winndows API: GetVolumeInformation(), GetCompressedFileSize() Change settings for files/directories: DeviceIoControl() with flags FSCTL_GET_COMPRESSION, FSCTL_SET_COMPRESSION

11.137

SMB Clients and Servers
Clients: Included in WfW 3.x, Win 95, Win98, Win ME and Windows NT/2000/XP/Server 2003/Vista. smbclient from Samba, smbfs for Linux, SMBlib Servers: Microsoft Windows for Workgroups 3.x, Win95, Win98, Win ME, Windows NT/2000/XP/Server 2003/Vista Samba (Linux, Solaris, SunOS, HP-UX, ULTRIX, DEC OSF/1, Digital UNIX, Dynix (Sequent), IRIX (SGI), SCO Open Server, DG-UX, UNIXWARE, AIX, BSDI, NetBSD, NEXTSTEP, A/UX) The PATHWORKS family of servers from Digital LAN Manager for OS/2, SCO, etc VisionFS from SCO Advanced Server for UNIX from AT&T (NCR?) LAN Server for OS/2 from IBM

11.138