Performance issue?

[W8CCW]

Yesterday evening I discovered that I have not getting the backup that
I want out of SyncToy. To facilitate working on the problem I decided
to offload the current backup to a spare 1 TB drive in the same box. I
created a folder called Drive E, did a control A select in Win 7
Explorer, click on Copy, and select folder Drive, click on Paste and
here we are 19 hours later waiting for 250 G to be transfered.

Both drives are Seagate SATA on the same MSI MB. Is this normal?

 

[charlie]

Yesterday evening I discovered that I have not getting the backup that
I want out of SyncToy. To facilitate working on the problem I decided
to offload the current backup to a spare 1 TB drive in the same box. I
created a folder called Drive E, did a control A select in Win 7
Explorer, click on Copy, and select folder Drive, click on Paste and
here we are 19 hours later waiting for 250 G to be transfered.

Both drives are Seagate SATA on the same MSI MB. Is this normal?

"Is this normal?"
Not really.
I'd try a different method to copy.

Part of the issue may have to do with what SATA controller is used.
If both drives use the same controller, that can slow things down.
Another possibility has to do with how much space is available on the HD
drive used to "buffer" the copy, and the amount of installed RAM
available for the copy process.
Finally, all Windows copy methods are not equal in speed. (Why, I never
bothered to find out.)

 

[Paul]

Yesterday evening I discovered that I have not getting the backup that
I want out of SyncToy. To facilitate working on the problem I decided
to offload the current backup to a spare 1 TB drive in the same box. I
created a folder called Drive E, did a control A select in Win 7
Explorer, click on Copy, and select folder Drive, click on Paste and
here we are 19 hours later waiting for 250 G to be transfered.

Both drives are Seagate SATA on the same MSI MB. Is this normal?

To "bound" or tell you how bad a drive can be, consider the transfer
rate achieved when you defragment a drive.

One of my OSes, does that at around 1MB/sec. My best case is around 3MB/sec
(on another computer, with a different Windows OS). The reason those speeds
happen, is the transfer size is tiny, and there is a lot of head movement
per megabyte of data. During defrag, the disk head flies around like crazy.

If your drive has many small files, transfer rates are going to suck just
like defragmenting does.

If the drive was say, your "movie" drive, and there were a relatively
small number of gigabyte sized files, then something is seriously wrong.
On the one hand, it's possible to have extreme fragmentation. One way
to achieve that, would be to have a quad TV tuner in the PC, and record four
TV programs at the same time. That could cause the four, gigabyte sized
recorded files, to have thousands of fragments. Copying severely fragmented
files (just freshly recorded), would suck in the same way as our small file
example would, because the disk head has to move to find every fragment.
Smooth contiguous files, written with no other interference at the time,
can transfer at 125MB/sec+.

What I use for disk to disk transfer, is Robocopy. I use the command line,
but there is also a GUI. Note that, that tool is extremely dangerous, when
doing things like the "mirror" option. Mirror would erase any destination
folder contents not needed to mirror the source disk. If you accidentally
point the mirror operation at the wrong drive, you lose the contents
of the destination drive. I've had just one accident with Robocopy so
far, and had to modify my procedures a bit so it won't happen again.

http://technet.microsoft.com/en-us/magazine/2006.11.utilityspotlight.aspx

Robocopy does non-blocking I/O, and can issue read and write requests at
the same point in time (to two different drives). So it can overlap
commands. I find it generally good, and achieves an average of perhaps 40MB/sec
with my crappy $50 drives here. That's an average - transfer rate can vary
from 3MB/sec (doing the small files again) to 125MB/sec (copying my pagefile
or hiberfile on an OS partition). The long term average while copying
thousands of files, is in the 40MB/sec ballpark. If I copy A-->B then
B-->A, I usually get a *different* average value - don't ask me why.
That happened even when disk A is exactly the same model number as disk B.

If we do the math for you, 250GB is 250000MB in 19*60*60 seconds or
250000/68400 = 3.65MB/sec, which would be fine if you had a million
small files on the drive. You may find Robocopy does a little better -
I think it's a lot better than the file system copy routine, for
comparison.

Just for the record, this is a typical command line I use for moving the
entire contents from one partition to another. I run this in an MSDOS
(command prompt) window. I keep a logfile for each transfer, mainly
to compare the numbers at the end of the log. If copying a partition off
to another partition, then back again, I like to verify I got all the files.

robocopy L:\ E:\ /mir /copy:datso /dcopy:t /r:3 /w:2 /zb /np /tee /v /log:robocopy_ell_to_e.log

Typical log results:

****************************************
-------------------------------------------------------------------------------
ROBOCOPY :: Robust File Copy for Windows :: Version XP026
-------------------------------------------------------------------------------
Started : Tue Sep 06 20:23:59 2011
Source = L:\
Dest = E:\
Files : *.*
Options : *.* /V /TEE /S /E /COPYATSO /DCOPY:T /PURGE /MIR /ZB /NP /R:3 /W:2
------------------------------------------------------------------------------

NOTE : NTFS Security may not be copied - Source may not be NTFS.
NOTE : NTFS Security may not be copied - Destination may not be NTFS.

26 L:\
same 2.9 g hiberfil.sys
same 2.0 g pagefile.sys
...

Total Copied Skipped Mismatch FAILED Extras
Dirs : 14376 14375 1 0 0 0
Files : 178733 178731 2 0 0 0
Bytes : 39.895 g 34.896 g 4.999 g 0 0 0
Times : 0:29:50 0:16:06 0:00:00 0:13:44

Speed : 38778320 Bytes/sec. <--- my long term average copy rate
Speed : 2218.913 MegaBytes/min.

Ended : Tue Sep 06 20:53:50 2011
****************************************

Paul

 

[Yousuf Khan]

Yesterday evening I discovered that I have not getting the backup that
I want out of SyncToy. To facilitate working on the problem I decided
to offload the current backup to a spare 1 TB drive in the same box. I
created a folder called Drive E, did a control A select in Win 7
Explorer, click on Copy, and select folder Drive, click on Paste and
here we are 19 hours later waiting for 250 G to be transfered.

Both drives are Seagate SATA on the same MSI MB. Is this normal?

Use Resource Monitor and watch the Disk tab. Under the Storage heading
you'll see all of your drive letters listed. Make sure that disk queue
length on any one of them don't go over 1.00 during the transfer.

Yousuf Khan

 

[W8CCW]

I started a copy of the "documents" item as displayed in Power Desk 8
from C: to E: . It is a lengthy collection but I am not sure what it
is really copying. Whatever, it needs back up.

I don't think the displayed file system accurately displays the
physical file system. Any comments?

 

[Paul]

I started a copy of the "documents" item as displayed in Power Desk 8
from C: to E: . It is a lengthy collection but I am not sure what it
is really copying. Whatever, it needs back up.

I don't think the displayed file system accurately displays the
physical file system. Any comments?

I would agree with that sentiment.

After just about bricking my Windows 7 laptop yesterday, I'd recommend
you at least run the "full system backup" option built into Windows 7.
That creates a couple .vhd files, and you can find them by looking for
..vhd on your hard drive. My C: makes a 26GB .vhd file. After I've run
a complete system backup, I copy the two .vhd files to another
drive (for a rainy day). The .vhd files can be randomly accessed,
if you need to get at individual files.

That copy, has as much stuff as it is safe to copy. It would be
more complete than other backup methods.

Since I had that available to me (only two days old), I immediately
used that to clean up the mess I'd made.

It's really like tromping around in a mine field... Beware.

Who knows. Maybe when you create those .vhd files, the
copy operation will go faster ?

http://www.intowindows.com/how-to-create-system-image-backup-in-windows-7/

http://windows.microsoft.com/en-US/windows-vista/Restore-your-computer-from-a-system-image-backup

Another piece of trivia I learned in the last couple days. If the
"Set up backup" button doesn't work (which you wouldn't be using
in this case anyway), the command line equivalent of clicking
that button, is

sdclt.exe /configure

A number of my backup buttons don't work right (thanks to the Acer
added software...), but at least I've got a way to access that one.

http://answers.microsoft.com/en-us/...-happens/f75fb816-1bb1-40d7-9e58-1b92a663f753

HTH,
Paul

 

[Char Jackson]

After just about bricking my Windows 7 laptop yesterday, I'd recommend
you at least run the "full system backup" option built into Windows 7.
That creates a couple .vhd files, and you can find them by looking for
.vhd on your hard drive. My C: makes a 26GB .vhd file. After I've run
a complete system backup, I copy the two .vhd files to another
drive (for a rainy day). The .vhd files can be randomly accessed,
if you need to get at individual files.

I'm curious as to why you do a backup and then copy the image files to
another drive. Why not back up directly to that other drive? Do you
keep a .vhd handy as sort of a filestore and put away a second copy of
that same image for safe keeping?

That copy, has as much stuff as it is safe to copy. It would be
more complete than other backup methods.

I'd say it's equally complete as other image backups, no? Or what did
you mean?

<snip>

 

[Paul]

I'm curious as to why you do a backup and then copy the image files to
another drive. Why not back up directly to that other drive? Do you
keep a .vhd handy as sort of a filestore and put away a second copy of
that same image for safe keeping?


I'd say it's equally complete as other image backups, no? Or what did
you mean?

<snip>

The .vhd is an exact copy, as exact as they can make it. It's used
for doing P2V conversions, so it has to be able to handle any trick
present in NTFS. It should also use only as many sectors as are needed
to represent the file system.

As to why the backup was on my data partition on the laptop, that
was a "happy accident". When I made that backup, I was actually testing
the various "buttons" in my System Protection window (the window
partially broken by Acer). That one happened to work, so I said,
what the hell, let it run to completion, as I have room on the data partition.
And I also wanted to test, whether the data partition could get damaged or not,
in a later experiment. That file was going to be part of my "test data" set.

Little did I know, that later I would actually *need* that backup

So it wasn't an actual part of a backup strategy, just another
one of my stupid experiments. I'm a "learn by breaking things"
kind of guy.

Paul

 

[Char Jackson]

The .vhd is an exact copy, as exact as they can make it. It's used
for doing P2V conversions, so it has to be able to handle any trick
present in NTFS. It should also use only as many sectors as are needed
to represent the file system.

Ok, sounds like every other image backup. Good to know.

As to why the backup was on my data partition on the laptop, that
was a "happy accident". When I made that backup, I was actually testing
the various "buttons" in my System Protection window (the window
partially broken by Acer). That one happened to work, so I said,
what the hell, let it run to completion, as I have room on the data partition.
And I also wanted to test, whether the data partition could get damaged or not,
in a later experiment. That file was going to be part of my "test data" set.

Little did I know, that later I would actually *need* that backup

So it wasn't an actual part of a backup strategy, just another
one of my stupid experiments. I'm a "learn by breaking things"
kind of guy.

Got it. Thanks for indulging me.

Any relation to Jeff Liebermann over in alt.internet.wireless? His
motto (and his website) is "Learn By Destroying".

 

[W8CCW]

Thank you,
It looks like we are in the same situation.

The xcopy in Dos seemed to whistle right along until I got to a file
that was created by Thunderbird. The current release of Tbird does not
always display attachments as I expect. Those are necessary to
save/view and dispose. I created a directory named "1" in the Pictures
Folder to deal with these files. This particular file name was "Mom, I
gotta pee." The length was 0 bytes. Explorer would not delete it. Dos
saw it with as "Mom,"
Dos did allow me to delete the directory "1"
When I looked at the Pictures folder with Win Explorer the "1" folder
was still there! It did delete OK with Win Explorer.

I have now put back my "1" folder in the "Pictures" folder. I may need
to do this again. I will start a fresh xcopy later today. I am using
the command line of C:\User\John\xcopy /s /e on the E:drive and it
looks do be working.

Although the directory "Users" is accessible from the command line it
does not appear in WinExplorer.

If this Usenet group did not exist I would not have a clue as to where
to find useful advice on these problems.

 

[Jake]

I have now put back my "1" folder in the "Pictures" folder. I may need
to do this again. I will start a fresh xcopy later today. I am using
the command line of C:\User\John\xcopy /s /e on the E:drive and it
looks do be working.

If you're not afraid of xcopy, you might try xxcopy

http://www.xxcopy.com/xcpymain.htm

free for non commercial use. Very fast. Lots of options.

-J

 

[Roy Smith]

If you're not afraid of xcopy, you might try xxcopy

http://www.xxcopy.com/xcpymain.htm

free for non commercial use. Very fast. Lots of options.

And you definitely want to stay away from XXXcopy! It replaces you
copied files with porn... ;-)


--

Roy Smith
Windows 7 Home Premium 64-Bit
Thunderbird 6.0.2
Wednesday, September 21, 2011 2:48:06 PM

 

[R. C. White]

Hi, W8CCW.

Have you tried the Dir command with the /x switch?

This will produce a normal directory listing - but with an extra column
before the filename column. For every filename that is not a valid 8.3 SFN
(Short File Name) the SFN will appear before the LFN (Long File Name). Then
you can just Del that 8.3 filename.

To see this, just open a Command Prompt window (which we old-timers still
call a DOS window, even though we know it's not really DOS). At the C:\
prompt, type:

C:\>dir /x

In the directory listing, the entry for the Program Files folder will look
like this:
08/09/2011 10:50 PM <DIR> PROGRA~1 Program Files

Note that the Users and Windows folders do not display SFNs; their names are
already valid SFNs, so no conversion is required. But Program Files is not
a valid SFN because it contains a <space> character, so it gets converted to
PROGRA~1. To delete a FILE by this name, we would type:
Del progra~1

To Remove the Program Files FOLDER (Directory), use:
rd progra~1

You don't really want to remove your Program Files folder of course; this is
only an example. Windows won't let you do it anyhow unless you insist with
the /s switch, but please don't actually try it. You should be able to find
or create a sacrificial file to test this.

I tried your problem filename by renaming a small file. In my Dir /x it
looks like this:
05/06/2010 08:11 PM 97 MOM_IG~1 Mom, I gotta pee

So I could delete this file with:
del MOM_IG~1

I still use Xcopy.exe, too. The /c switch (Continues copying even if errors
occur) might work for you in this case. Of course, this switch must be used
with care and discretion.

Sometimes the old DOS methods get better results than we can get with the
Windows GUI. ;<)

RC
--
R. C. White, CPA
San Marcos, TX
(e-mail address removed)
Microsoft Windows MVP (2002-2010)
Windows Live Mail 2011 (Build 15.4.3538.0513) in Win7 Ultimate x64 SP1


"W8CCW" wrote in message
Thank you,
It looks like we are in the same situation.

The xcopy in Dos seemed to whistle right along until I got to a file
that was created by Thunderbird. The current release of Tbird does not
always display attachments as I expect. Those are necessary to
save/view and dispose. I created a directory named "1" in the Pictures
Folder to deal with these files. This particular file name was "Mom, I gotta
pee." The length was 0 bytes. Explorer would not delete it. Dos
saw it with as "Mom,"
Dos did allow me to delete the directory "1"
When I looked at the Pictures folder with Win Explorer the "1" folder
was still there! It did delete OK with Win Explorer.

I have now put back my "1" folder in the "Pictures" folder. I may need
to do this again. I will start a fresh xcopy later today. I am using
the command line of C:\User\John\xcopy /s /e on the E:drive and it
looks do be working.

Although the directory "Users" is accessible from the command line it
does not appear in WinExplorer.

If this Usenet group did not exist I would not have a clue as to where
to find useful advice on these problems.

 

[J. P. Gilliver (John)]

Hi, W8CCW.

Have you tried the Dir command with the /x switch?

This will produce a normal directory listing - but with an extra column
before the filename column. For every filename that is not a valid 8.3
SFN (Short File Name) the SFN will appear before the LFN (Long File
Name). Then you can just Del that 8.3 filename.

This switch (which is actually shown under dir /?, I'd just never
noticed it!) also works under XP. (This was originally posted in a W7
'group.)

 

[Peter Jason]

Yesterday evening I discovered that I have not getting the backup that
I want out of SyncToy. To facilitate working on the problem I decided
to offload the current backup to a spare 1 TB drive in the same box. I
created a folder called Drive E, did a control A select in Win 7
Explorer, click on Copy, and select folder Drive, click on Paste and
here we are 19 hours later waiting for 250 G to be transfered.

Both drives are Seagate SATA on the same MSI MB. Is this normal?

I have the Norton Ghost 15 and several HDDs to which I backup in
rotation. The backup is automatic and runs after hours. I can use
the firewire for small HDDs and the SATA 1Tb HDD that is in the
computer. If I lose a file on the main system then I can access it
in a backup HDD without separately converting the Ghost image file and
this is very useful.

 

[John Ferrell]

I have not abandoned the thread. As if I did not have enogh trouble, I
brought one one myself. I received copy of Popular Mechanics (that is
what the title said). I unrared it and it displayed an icon almost
like Adobe. I did not bother to positively verify the file type. When
I double clicked it, it turned out to be an .exe file and I had
executed the adh.2 trojan. I must have been among the lucky first
adopters because Norton Internet Security did not figure it out until
a day later. After many hours of effort, I regenerated the Win&
system. Fortunately it was not the system that this thread is
concerned with.

As far as the matter of the Performance issues, I will have to put
that off for a while. At least part of the problem is related to the
fact that the larger HDD's are only 5400 rpm.

Thanks to all that offerred suggestions!