[NTLUG:Discuss] Re: Hard drive question... -- less known things about NTFS ...

[Bryan J. Smith]

On Mon, 2004-10-18 at 08:06, Stephen Davidson wrote:
> Good morning, Douglas.
> If the destination OS is Linux, unless there is some issue with NTFS 
> Security Keys, the drive does not even have to come from the same OS as 
> the Destination.  I have used this technique once or twice for data 
> recovery myself from some Win98 HDs.  The only issue would be mounting 
> partitions, if you don't remember what they are (and have more than one
> ).

Many NTFS objects rely on meta-data that is either tied to the local SAM
(and its SID references) in the registry of the system that created it
and/or the [CIFS/ADS] domain (if applicable) SAM (and its SID
references) used on the network.  In fact, from a "controller" (not
browser) standpoint, that is really the definition of what a CIFS/ADS
"domain" is -- a network-based SAM.

This was originally designed to be a security feature.  But in the end,
it only ended up being a "false security" aspect of NTFS' design came
back to haunt Microsoft.

Prior to NT v5 (2000, XP, 2003+), Microsoft recommended you _never_ make
"changes" (as in change "meta-data" information) to a NTFS filesystem
from an installation of NT (or any OS for that matter) that did not
create the filesystem.  This was because the filesystem may have
"meta-data" that referenced information only found in the SAM of the
creating installation.

You could still use utilities that moved, resized, etc... NTFS (even the
Linux-LDM/NTFS projects provide "safe" utilities to do this on _older_
versions for/of NT/NTFS).  But when it comes to changing the filesystem
"on-line," if you touch something, you can not only trash files, but you
can leave the entire filesystem inconsistent because of meta-data
changes.  In fact, most of the newer, _kernel_-based Linux NTFS "write"
capability will only change very basic information.  E.g., typically,
you can change the file in a NTFS filesystem (add, remove, etc...), but
not its meta-data in the filesystem.

With NT v5 (2000, XP, 2003+), Microsoft introduced the new "Logical Disk
Manager (LDM)" "disk label" (aka partition table format) aka "Dynamic
Disk," to replace the PC-centric "DOS disk label" aka "Basic/Standard
Disk" (primary/logical partitioning).  For backward compatibility on the
legacy Int13h PC BIOS, a "Dynamic Disk" looks like a "Basic/Standard
Disk" with single primary partition of type 42h.

LDM does several things, including:  
 1.  Provide a non-PC centric format (e.g., works with IA-64/GPT too)
 2.  Provide a "disk label" journal (important for volume management)
 3.  Store LBA and several geometry parameters (solves lots of issues)
 4.  Store SID information outside the local SAM/registry

#4 is what makes NTFSv5 "safe" to write to from other NT v5 (2000, XP,
2003+) installations, as long as it is on a "Dyamaic Disk."  SID
information crucial to safely writing to a NTFSv5 filesystem from a
system that does not have access to those SIDs normally (i.e., they are
not in the "CIFS/ADS domain" SAM, but local to the installation that
created/wrote to the filesystem) is provided in hidden (but documented)
areas of the LDM disk label/slices (partition table/partitions).

Now it does not necessarily mean SID/meta-data will be "preserved" _if_
you write to it with another installation, but it _will_ mean it will
_prevent_ the NTFSv5 filesystem from getting _toasted_ with
inconsistencies.  But, again, you _must_ use a LDM disk label ("dynamic
disk" partition table) to do have this.  Linux/LDM-NTFS work continues
on supporting reading these portions of the LDM disk label for use in
"safely" writing to NTFS.

Until then, you can use a _user-space_ utility/mount approach (_not_
kernel-based NTFS support) for NTFS in Linux that _directly_ reads the
local SAM/registry of even NT 3.x-4.0 systems (as well as NT v5+) so it
can extract SID information to safely write to a NTFS filesystem that
was created by that NT installation.  The project is known as
"Captive."  Microsoft offers _no_ such equivalent capability for older
versions of NT (although a few 3rd parties do).

-- Bryan

P.S.  SIDE NOTE:  #3 would also solve a lot of issues for Linux
dual-boots.  Linux already has LDM disk label support (it will _not_
change with NT6 "Longhorn," LDM is a pretty fixed standard), but it
lacks the user-space tools and GRUB support.  But if Linux installers
would read a LDM disk label (partition table), they could find out
_exactly_ what the geometry of the disk is being "assumed" by NT v5 (and
not have to "guess" at it like it does today, and decide between LBA
255/63 and full Extended Int13h compliance which is the current delima
of Linux & co. in kernel 2.6).

That's why I'm a big proponent of GRUB and user-space utilities, as well
as distro installers, moving to support LDM Disk Labels.  I will
_finally_ "nip the geometry 'guessing game' in the 'butt' once and for
all!"  With Basic/Standard Disks, NT does _not_ store its _definitive_
geometry translation.  With Dynamic Disks, it does!  Unlike DOS (incl.
95/98/ME), which _requires_ that BIOS and partition table geometry
match, NT (incl. 2000, etc...) does _not_, and in the case of NT4 SP4
and later versions, it often "assumes" geometry that may very well be
_completely_ incompatible (e.g., LBA 255/63 on an older BIOS that does
not support Extended Int13h).

-- 
Bryan J. Smith                                  b.j.smith at ieee.org 
------------------------------------------------------------------ 
"Communities don't have rights. Only individuals in the community
 have rights. ... That idea of community rights is firmly rooted
 in the 'Communist Manifesto.'" -- Michael Badnarik