Windows 7 disk queue length?

[Yousuf Khan]

I'm trying to investigate the source of some poor performance from my
boot drive. I ran the Windows 7 Resource Monitor, and found that the
disk queue length on this drive after bootup is over 5 and approaching
10! I don't know if disk queue length means the same thing as it does in
the Unix world, but in Unix, disk queue length measures how many
requests are coming in vs. how many request slots there are. So the disk
queue length should always be below 1, but it's showing 5 and sometimes
10. This means that the queue is bigger than the disk's ability to
handle it.

What can be done to get the disk queue under control?

Yousuf Khan

 

[Rod Speed]

Yousuf Khan wrote

I'm trying to investigate the source of some poor performance from my
boot drive. I ran the Windows 7 Resource Monitor, and found that the
disk queue length on this drive after bootup is over 5 and approaching
10! I don't know if disk queue length means the same thing as it does
in the Unix world, but in Unix, disk queue length measures how many
requests are coming in vs. how many request slots there are. So the disk queue length should always be below 1, but
it's showing 5 and sometimes 10. This means that the queue is bigger than the disk's ability to handle it.

What can be done to get the disk queue under control?

Is there any evidence that the drive is a real performance
limit like the hard drive led being on all the time when you
see that higher than what you expect queue length ?

 

[DevilsPGD]

In message <[email protected]> someone claiming to be Yousuf

I'm trying to investigate the source of some poor performance from my
boot drive. I ran the Windows 7 Resource Monitor, and found that the
disk queue length on this drive after bootup is over 5 and approaching
10! I don't know if disk queue length means the same thing as it does in
the Unix world, but in Unix, disk queue length measures how many
requests are coming in vs. how many request slots there are.

Sounds about right. The exact metrics probably vary slightly, but the
principle is the same, once you exceed 1 you're throwing more at your
drive than you can keep up with.

So the disk
queue length should always be below 1, but it's showing 5 and sometimes
10. This means that the queue is bigger than the disk's ability to
handle it.

What can be done to get the disk queue under control?

Cut back on requests to the disk, get hardware fast enough to meet your
needs, or accept that you're waiting on the disk.

Depending on what exactly you're doing that causes the problem,
defragmenting might help significantly, especially if your drive has a
high average latency.

 

[Rod Speed]

DevilsPGD wrote

Sounds about right. The exact metrics probably vary slightly, but the
principle is the same, once you exceed 1 you're throwing more at your
drive than you can keep up with.

Cut back on requests to the disk, get hardware fast enough
to meet your needs, or accept that you're waiting on the disk.

Depending on what exactly you're doing that causes
the problem, defragmenting might help significantly,

Unlikely with modern systems.

especially if your drive has a high average latency.

Even more unlikely.

 

[Yousuf Khan]

Yousuf Khan wrote



Is there any evidence that the drive is a real performance
limit like the hard drive led being on all the time when you
see that higher than what you expect queue length ?

Yes, that's exactly what was happening. LED on straight for more than a
half-hour (beyond the point where I could continue looking at it).

Anyways, it looks like I've found the source of my problems, it was my
anti-virus. I uninstalled AVG and replaced it with Avira, and it's all
settled down now.

I've had to restore my system from backups (another long story,
involving Windows 7's SP1 update). And the backup was from a period of
time when I was still running AVG. As I recall I had uninstalled AVG and
replaced it with Avira before too. The latest AVG's are just becoming
hogs, like McAfee and Norton had become long ago.

Yousuf Khan

 

[Yousuf Khan]

Cut back on requests to the disk, get hardware fast enough to meet your
needs, or accept that you're waiting on the disk.

Depending on what exactly you're doing that causes the problem,
defragmenting might help significantly, especially if your drive has a
high average latency.

Looks like the issue turned out to be my AVG anti-virus -- it was doing
something nutty. So I replaced it with Avira anti-virus.

Defrag'ing wouldn't have worked as the system was so slow that I
couldn't even start small simple apps like TaskManager without waiting
endlessly, let alone a large app like Defrag.

Yousuf Khan

 

[VanguardLH]

I'm trying to investigate the source of some poor performance from my
boot drive. I ran the Windows 7 Resource Monitor, and found that the
disk queue length on this drive after bootup is over 5 and approaching
10! I don't know if disk queue length means the same thing as it does in
the Unix world, but in Unix, disk queue length measures how many
requests are coming in vs. how many request slots there are. So the disk
queue length should always be below 1, but it's showing 5 and sometimes
10. This means that the queue is bigger than the disk's ability to
handle it.

What can be done to get the disk queue under control?

Yousuf Khan

Found mention of disk queue here:

http://support.microsoft.com/kb/146005
http://support.microsoft.com/kb/822219

So what I/O bound processes are you running perhaps in the background?
Tried disabling your anti-virus software for awhile to check for change
in disk traffic?

Got multiple anti-malware products installed that are concurrently
active and perhaps conflicting with each other? I've seen these cause
thrashing: one AV product monitors a change in a file which triggers
another AV product to do its check which triggers the 1st to recheck and
the cycle keeps repeating. I forget which anti-malware products were
involved back then but I recall noticing there were over 3900 disk
access per minute because of this software conflict. Shutting down just
one of the conflicting processes ceased the thrashing.

Got file indexing enabled? Are you running SQL on this problematic
host? If so, see http://support.microsoft.com/kb/167711.

You might want to use a file monitor to see what is doing all the file
accesses (opens, reads, writes), especially at times when you expect the
host should be relatively quiescent.

Did you install a "green" or "advanced" hard disk in your system? That
is, the type that uses 4KB sized sectors internally but maps to the
standard 512MB sectors on its interface? This misalignment can cause
lots of problems. Make sure you don't get WD's Green series of hard
drives or any that are Advanced Format Drives (AFDs) or Smart Align
Technology (Seagate). These work by using 4KB sector sizes on the
platter and translating in their interface to 512KB; however, there is a
problem is misalignment since some versions of Windows start a partition
at sector 63 instead of 64. You could end up with a very slow external
hard disk due to misalignment because of the need to do read-modify-
write instead of just a write.

Read:
http://www.anandtech.com/show/2888/2
http://dl.paragon-software.com/free/Paragon Alignment Tool - White Paper.pdf
http://www.paragon-software.com/technologies/components/partition-alignment/
http://support.wdc.com/product/download.asp?groupid=805

I'm not intimate with this issue to know if Vista/7 won't have problems
for migrations (i.e., where you upgraded to Vista/7 from a prior version
of Windows which would have a misalignment problem with a WDC green hard
disk). This possible cause of dragging forward problems into a new
version is why I always recommend doing fresh installs of newer versions
of an OS.

 

[Yousuf Khan]

Found mention of disk queue here:

http://support.microsoft.com/kb/146005
http://support.microsoft.com/kb/822219

So what I/O bound processes are you running perhaps in the background?
Tried disabling your anti-virus software for awhile to check for change
in disk traffic?

Turned out it was the anti-virus doing this to my system. Replaced AVG
with Avira, and problem is basically solved. Well, there is still a lot
of activity during boot up, and disk queue lengths are still well above
1, during this time. However, at least it doesn't stay in this condition
forever now. Now I'm wondering if there's something I can do to reduce
disk queue lengths during bootup too?

Yousuf Khan

 

[VanguardLH]

Turned out it was the anti-virus doing this to my system. Replaced AVG
with Avira, and problem is basically solved. Well, there is still a lot
of activity during boot up, and disk queue lengths are still well above
1, during this time. However, at least it doesn't stay in this condition
forever now. Now I'm wondering if there's something I can do to reduce
disk queue lengths during bootup too?

WinPatrol which lets me delay some startup items. This way, my host
isn't so choked with multiple concurrent startup programs contending for
the data bus from disk to memory that it actually becomes usable sooner.
After the startup programs that I wanted loaded immediately are done,
the delayed programs start loading. I can specify how long to wait for
each delayed program before it starts loading.

If you only get the free version of WinPatrol, be sure to reduce the
polling interval for all its monitors down to 1 minute; otherwise, you
won't remember what you did 5 minutes ago when it alerts you of a
change.

If the startup item has command-line parameters, you'll probably have to
enter them when you move a program from the startup list to WinPatrol's
delayed startup list. Conversely, if you later decide not to delay a
startup program and move it back to a normal startup program, WinPatrol
isn't smart enough to carry back the command-line parameters. So just
note what command-line parameters are specified to load the program when
moving to delayed or back to normal startup as you may have to reinstate
them.

 

[Yousuf Khan]

WinPatrol which lets me delay some startup items. This way, my host
isn't so choked with multiple concurrent startup programs contending for
the data bus from disk to memory that it actually becomes usable sooner.
After the startup programs that I wanted loaded immediately are done,
the delayed programs start loading. I can specify how long to wait for
each delayed program before it starts loading.

There is a new startup category for Windows services, called "Automatic
(delayed start)", which is new with Windows 7 (maybe even with Vista),
not available in XP. I had gone through and marked a few of the services
with this category, but I don't know if it made any difference.

Of course, this only works with Services, and not with just regular
startup programs. I had used another program back in the XP days called
Startup Delayer, which worked with regular startup programs. But I was
using that previously to fix another problem I was having with startup
programs not starting at all under XP. I might try that again under Win
7, if it still works with it.

Yousuf Khan

 

[Gene E. Bloch]

There is a new startup category for Windows services, called "Automatic
(delayed start)", which is new with Windows 7 (maybe even with Vista),
not available in XP. I had gone through and marked a few of the services
with this category, but I don't know if it made any difference.

Of course, this only works with Services, and not with just regular
startup programs. I had used another program back in the XP days called
Startup Delayer, which worked with regular startup programs. But I was
using that previously to fix another problem I was having with startup
programs not starting at all under XP. I might try that again under Win
7, if it still works with it.

Yousuf Khan

I recently tried to solve a problem by using Task Scheduler to start a
program at system startup plus a (specifiable) delay.

That part worked, but the problem wasn't solved

The program in question is called Stickies - I used it for on-screen
sticky notes, but every so often it lost its data. I tried the delayed
start to see if it was a start-up timing issue, to no avail.

 

[Bob I]

Turned out it was the anti-virus doing this to my system. Replaced AVG
with Avira, and problem is basically solved. Well, there is still a lot
of activity during boot up, and disk queue lengths are still well above
1, during this time. However, at least it doesn't stay in this condition
forever now. Now I'm wondering if there's something I can do to reduce
disk queue lengths during bootup too?

Yousuf Khan

SSD

 

[Yousuf Khan]

I recently tried to solve a problem by using Task Scheduler to start a
program at system startup plus a (specifiable) delay.

That part worked, but the problem wasn't solved

The program in question is called Stickies - I used it for on-screen
sticky notes, but every so often it lost its data. I tried the delayed
start to see if it was a start-up timing issue, to no avail.

Well, I'd say copy its database regularly, just in case.

Yousuf Khan

 

[Yousuf Khan]

SSD

Most probably that's the right solution, but a usefully-sized SSD is a
little too expensive for my tastes right now.

Yousuf Khan

 

[Gene E. Bloch]

Well, I'd say copy its database regularly, just in case.

Yousuf Khan

Or use an alternative program that doesn't have the problem.
That seems to me to be a better solution.

One problem is that every stickies program I've tried has some quirk
that bugs me almost as much as the one I mentioned - such as costing
money ;-)

Here's one: PNotes. When I update it, it is kind enough to ask me if I
want to keep my data, which of course I do, so I answer in the
affirmative. Then the program erases my data regardless of my answers.

Your solution for Stickies obviously applies here as well

 

[Bob I]

Most probably that's the right solution, but a usefully-sized SSD is a
little too expensive for my tastes right now.

Lots of us in that boat! <VBG>

 

[Gene E. Bloch]

Well, I'd say copy its database regularly, just in case.

Yousuf Khan

I confess that you shamed me into action.

With the help of XXCopy, I wrote a batch file to copy the appropriate
data files to a backup folder, but only if the files were of a
reasonable length[1] (to avoid copying the files after they were
clobbered). I scheduled it to run a few minutes after startup and hourly
after that.

I also am trying some new sticky note programs to see if there's one I
like better.

[1] I used minimum size = 10 for a very short file and = 1000 for two
files which are KB in length. I probably should have just used min size
= 1 for all, but I'm superstitious sometimes. The command line switch is
"/SZ:m-n" in its basic form, one of what looks like about 2,000,000
switches in XXCopy. (I omitted the n to imply no upper limit.)