Missing files and disk letter changed

[Fixer1234]

This is making my brain hurt!

We were talking about three drives, which appear in your original screen shots--a 250 GB internal, a 120 GB internal, and a 4 TB external. The 250 GB and 120 GB drives have all of their capacity accounted for. The 4 TB external had 4 TB available when you got it (you actually saw that capacity), and now 2 TB of that is missing. Nothing in your original system snapshots shows either a 2 TB or a 1 TB drive. If there was a 2 TB drive that was actually a partition of the external drive, this would all make sense. But what I'm trying to wrap my head around is that now there is still 2 TB missing from a 4 TB drive, and another 2 TB internal drive that didn't show up at all in your original screen shots, has since reappeared but is missing 1 TB. This is like something right out of Star Trek--a space-time anomaly where things drift in and out of existence.

Maybe it would help if you start with a physical description of your system. What are all of the internal and external storage devices--physical boxes and partitions. What was originally there and what do you see now--a before and after for each drive and partition accounting for all of the theoretical capacity.

And can you pass me some of that beer?

 

[Graham J Hadlington]

Sorry for messing-up your brain, not intended or sadistic, I promise

Right, let's start anew:

Drive C: My O/S drive and untouched as far as I can tell = 250GIG (SSD)
Drive F: My old O/S drive which was left in the PC when the above was fitted 120GB (SSD)
Drive X: This was my original D Drive which became I after the issue and I was advised to rename to X (internal WD drive)
Drives D,E, and H all contain System Reserved according to Computer.
Drive J: External WD My Book drive (4TB

Hope I have cleared that up and apologies again if there was any confusion.

I have enclosed another screenshot to show all the drives including the 4TB external which I have switched on.

I'm actually taking my better halk out tonight for a beer and will toast you for your patience
Graham

 

[Fixer1234]

OK, that's very different. In that screen grab, I see all the capacity that should be there accounted for--there are no missing drives or partitions. That means whatever is actually lost did not float away on a drive or partition that disappeared.

So let's get back to helping me understand what is there, what is missing, and what changed as a result of the problem.

When you started this thread, it sounded like you thought the contents of your C: drive had over-written and replaced the contents on every other drive/partition. As we're digging into this, it looks like my initial perception wasn't accurate. If I understand where we are, C:, D:, E:, and H: are completely unaffected. So help me understand the other drives:

• F: used to be your system drive. It contains C:-type folders, but is not an exact copy of your current C: drive. So what are we looking at?
  * Did you leave the original c: folders on there when you re-purposed the drive and that is what we're seeing now, or did you wipe the drive first, the problem incident copied stuff from your current C: drive onto it, and that's what we're seeing now?
  * Have you identified files that you knew used to be on F: that are now missing?
  
  
• So X: is what was previously labelled I:, and this was never a system drive.
  * Does it contain any C: files? If yes, is that because you were using it for backup and you expect to find those there or were they dumped there by the problem?
  * Have you identified files that you knew used to be on X: that are now missing?
  
  
• And the J: drive--often talked about but not previously seen. Glad to finally meet you.
  * Does it contain any C: files? If yes, is that because you were using it for backup and you expect to find those there or were they dumped there by the problem?
  * Have you identified files that you knew used to be on J: that are now missing?
  

If you can answer those questions for me, it may help me to stop drinking and we can start zeroing in on exactly what the problem is and how to fix it.

 

[Graham J Hadlington]

Right, after a great night seeing our friends from Canada who are visiting the UK, we sank a few drinks and enjoyed chewin the fat so now I feel refreshed and ready to get back to the job in hand

You ask:

>>F: used to be your system drive. It contains C:-type folders, but is not an exact copy of your current C: drive. So what are we looking at?
* Did you leave the original c: folders on there when you re-purposed the drive and that is what we're seeing now, or did you wipe the drive first, the problem incident copied stuff from your current C: drive onto it, and that's what we're seeing now?
* Have you identified files that you knew used to be on F: that are now missing?<<

The files and folders in F are the contents of my previous C drive which was taken over to the new SSD when it was fitted about 18 months ago. It is not a complete copy as new applications and files have been added since changing drives.This drive (F) has not been altered or used in any way since my newer SSD was installed.

You ask:

>>So X: is what was previously labelled I:, and this was never a system drive.
* Does it contain any C: files? If yes, is that because you were using it for backup and you expect to find those there or were they dumped there by the problem?
* Have you identified files that you knew used to be on X: that are now missing?<<

Yes, X was previously labelled I but was originally my D drive. It was never a system drive. It now contains C files which were dumped there by the problem yes.

If you mean can I see any of the missing files, sadly no

You ask:

>>And the J: drive--often talked about but not previously seen. Glad to finally meet you.
* Does it contain any C: files? If yes, is that because you were using it for backup and you expect to find those there or were they dumped there by the problem?
* Have you identified files that you knew used to be on J: that are now missing?<<

Yes, this contains the file structure of C now since the issue. I have not backed-up these files to the J drive, they have just appeared like the other drives AND I cannot find my backup file which is a NovaBAVKUP file.No, no files that should be on J drive have been identified.

Hope this helps and thanks.
Graham

 

[Graham J Hadlington]

Thought you might like to see the backup I took orginally of drive D (I and X since) you can see the backup was made.
Graham

 

[Fixer1234]

Sorry for the delay in getting back. My own computer crashed and I have been scrambling. So, I'm a little distracted. Let me just clarify a couple of things (they are probably clear but my brain isn't totally focused). I'll probably have to get back to you on the next steps. Not sure how long I will be in business here today until I can figure out what happened.

The files and folders in F are the contents of my previous C drive which was taken over to the new SSD when it was fitted about 18 months ago. It is not a complete copy as new applications and files have been added since changing drives.This drive (F) has not been altered or used in any way since my newer SSD was installed.

I'm not sure if this is a "yes" or a "no". Are you saying that F: appears to be unaffected by the problem (nothing obvious copied there and nothing apparently missing)?

Yes, X was previously labelled I but was originally my D drive. It was never a system drive. It now contains C files which were dumped there by the problem yes.

If you mean can I see any of the missing files, sadly no

So X now contains some flotsam. Tell me about the nature of the missing files:

• Are all of the old files missing or just some?
• Were any of the missing files "loose" in the root directory of X: or were they all contained in folders?
• How many original folders?
• Were any of the old folder names the same as the stuff that got copied there by the problem?

The next step is to do a careful look through the junk to see if your missing files are mixed in. Do a search for some of the file names you know used to be there. If you can find a hit on one, you may find the rest or get a clue as to what happened to them.

I forget at this point, did you say you already tried to run an undelete program to find the missing files? If so (which would obviously have been unsuccessful), which drives did you run it on? Did you try a "low level" recovery? There are two kinds of undeletes. If the file was deleted by a normal deletion process, the file remains there, just renamed in a special way and hidden. That can be undeleted by restoring its name and unhiding it. Otherwise, the file content remains on the drive but with nothing identifying it or pointing to it--just unknown data spread around the drive platters. A low level recovery goes looking for those snippets of raw data and tries to make sense of them. If nothing has over-written the pieces, the file can often be recovered, though often not identified as to what it was originally named or what type of file it was. The fun part is being a detective to look for clues as to what kind of file it was. Some of the recovery programs know the patterns of what common filetypes look like (things like unique file headers inside the file). From there, you figure out what it used to be called.

If you did not run a low level recovery scan, now would be the time to do it (before you clean up the junk, which an undelete program will just revover again and add to your work. Important: do any recovery to a drive that does not have missing data; writing the recovered files, which aren't guaranteed to be useful, will over-write any files that are missing.

After you have exhausted all methods of recovering the missing files, you might as well clean up the rubbish. One way to look and clean up at the same time is to start at the first folder and keep expanding it until there are no more embedded folders. If everything in the last (open) folder is trash (i.e., nothing you are missing), do a select all and delete. Go back up a level, hover over the now empty folder and verify that it is empty, then delete that folder. There are a couple of "extra" steps in this but they are a safeguard. As you go through tons of folders, you will start doing this automatically, and that is when you will miss something important and delete a folder you don't want to delete. After deleting a folder in its "parent" directory, if there is another folder there, expand it and repeat the process. Eventually, you will work your way back to the root directory, where you do the same thing with the next folder.

You may have to use some judgement and take some risk so you don't spend weeks doing this cleanup. If one of the root directories is, say, a Windows system directory that has hundreds of sub-folders, just satisfy yourself that none of your files are hiding there and streamline the process. If you are missing chunks of files in the 10s of MBs range, check the size of an entire folder "branch". Often there will be extensive Windows folder trees that are mostly empty or contain a few tiny files. If an entire sub-tree of folders contains less than a few MB and none of your missing files are in the surrounding vicinity, it is likely safe to prune that tree.

Yes, this contains the file structure of C now since the issue. I have not backed-up these files to the J drive, they have just appeared like the other drives AND I cannot find my backup file which is a NovaBAVKUP file.No, no files that should be on J drive have been identified.

So J: has flotsam copied from C:. You used this drive only for backup? And, all of the backup is zipped or compressed in a single file, which is missing? Are there any other files or folders that used to be on J: that are now missing? Was the missing backup the backup of the stuff that is also missing from X:, so you have no source to replace the missing stuff if we can't find it? No pressure.

Follow the same advice on J: as for X:. On both drives, make sure you follow the sequence described, which will protect anything that needs protecting.

 

[Fixer1234]

About a year ago, I had to do some undeleting and researched the free programs available at the time. I ended up downloading these three: Recuva, Pandora Recovery, and EaseUS Data Recovery Wizard. They had the best reviews. I know I used Recuva and can't remember which of the other two (maybe both). I think they all do low level recovery, but I forget now, which I had the most success with and what was best about each one. I also tried some others but they were not as good. They are free, so it can't hurt giving them a try. I don't know if they are better or worse than the commercially available programs, but they are generally well respected. This may save you some research time and trial and error.

 

[Graham J Hadlington]

Firstly: I am sorry to hear you are experiencing problems with your own PC The gods do take the micky at times, especially when you take so much time assisting others. Strange, life is at times.

I have demoed several data recovery applications; some make the job easy and construct 'tree' views which make a little sense while others (which cater for the professional more) bring back your files in which looks like a very gobbledygook fashion. Some trawl through your PC in a reasonably fast manner where others take hours, even with the easy file deletion option. I have tried and have two of your suggestions above and will give the other one a shot. Just trialed VirtualLAB but this takes a eternity just to locate your drives

Yes, F seems unaffected

Yes, I just used Drive J (4TB external) for backup only, nothing else.

I will start the 'detective' work you recommend today and get back with the results.

Speak later and thank you
Graham

 

[Fixer1234]

The first step is to look through what's there with a fine tooth comb to make sure the missing stuff isn't hiding in plain sight. If the problem was caused by your backup program, it shouldn't have deleted files at all. These programs don't delete or reformat before they write, and all writing is under OS control. That type of problem would just leave your original files there, surrounded by junk. If the folders that were copied as part of the junk duplicated any names that were already on the drive, Windows would have merged the folders. Your files could be buried within the junk. Doing a search for some of the missing filenames (using the option to include sub-folders), on the drives that became corrupted would be the fastest way to see if they are there.

For your files to have been deleted, there aren't too many possible ways that make sense. It would be something on the order of losing a partition or having the directory structure wiped off. However, you aren't missing any partitions and you can still access the drives, so that kind of damage didn't happen. Everything points to your old files almost having to still be there.

This thinking out loud did remind me of something strange I've seen before and nobody could figure out how it happened. It would actually be consistent with your problem. The issue was that all of the files on a drive had their attributes changed. In those other cases, it was to things like read-only. On top of that, the permissions were changed so that the owner of the files couldn't change it back. He could see his files sitting there but couldn't do anything with them.

If all of your old files had their attributes changed to "hidden", guess what? They would look like they disappeared and there wouldn't be any residual traces of other damage. Perhaps some underlying problem caused two symptoms at the same time--hiding you files and causing other junk to be copied to those drives. Off the top of my head, I would have said the odds against that are astronomical, but suppose in the whole history of computing this has happened only once, and you are the victim. Even with 1 in a gazillion odds of it happening, for whoever it does happen to, the odds are 100%.

Do two quick checks:

• Go into Windows Explorer, go to the root directory of the J drive, and select everything that is visible in the file detail (not in the navigation pane). Don't use "select all", but manually Ctrl-click on each folder and file you see. Then right-click on the aggregate, select properties, and look at the size of that collection of files/folders. Then right-click on the J drive, go to properties, and see how much space Windows says is being used on the drive. See if there is a discrepancy that would account for your missing stuff.
• For the root directory of J, go into the file display options and temporarily enable showing everything, including hidden and system files. Apply this to all sub-folders. Then browse around J and see if you can spot any of your missing stuff.

 

[Graham J Hadlington]

Hello again, I have got exhausted looking for those files now without any success I have also tried quite a lot of data recovery programs which have not found anything of relevance

Tried attaching the F Drive to my old laptop to see if it showed another collection of files/folders but, the machine is running Vista 32bit so wouldn't recognize 4TB I don't have access to another Win7 PC.

Would running a Linux boot-up disk help locate the files would you think? Really getting depressed now.

Thanks again for you excellent service and time.
Graham

 

[Graham J Hadlington]

Oh, FYI, here is a screenshot from the J Drive showing backup files but none from the first backup of the missing files!

 

[Fixer1234]

If there was something deleted there to find, the recovery programs would have found it, so your files probably were not deleted. There is no sign of "violence"--no missing partition or wiped file structure. Did you humor me with the check for hidden files? If they aren't there and hidden, I'm out of ideas. Well, there is one other explanation--somebody playing a very expensive and very evil prank, like swapping your hard disk for another identical one.

What happened to your computer was just one of those weird things that happen from time to time to time that makes no sense. Maybe you can take a little comfort that at least the hardware is intact and "all" you lost were files.

 

[Graham J Hadlington]

Morning again sir,

I will try the hidden files suggestion once more today. Could this be a 'hidden partition'? With you being a computer expert does this sound a stupid question or not?

Have tried many file recovery programs, those don't seem to shed any light on the missing files. Not sure if there is any software that traces hidden/missing files out there? If you know of anything I'm willing to give this a shot.

I have looked at disk recovery services and paying well over a £1000 sounds very extreme for this service I must admit.

Apologies for the negative state of mind yesterday, it had just got to me somewhat. Weather here for the 1st of June has turned sunny today which does brighten the mood somewhat
Graham

 

[Fixer1234]

This one just has me baffled. If anything tried to create a new partition, you would see evidence of it. For example, if you add Linux to your computer, it will create a new partition by stealing space from an existing one. Anything that was within that space is lost, and in Windows, it often just looks like your partition shrank and your disk got smaller. It isn't unusual for Windows to want to run chkdsk afterwards. typically, adding a partition in any automated fashion takes the space from the "unused" end of the drive if what is doing it is not malicious, which wouldn't have affected your files. The exception would be something like a recovery partition, but that is normally put on before the drive is used. Although it's about as possible as all of your files being made "hidden", I suppose a tiny new partition, could have been created at the "meat" end of your drive that ate your files and is not visible in Windows. The entire capacity is accounted for to within rounding error, so it would have to be pretty small. If your drive had a partition added and it ate your files, there would be nothing left to recover, so that possibility doesn't buy you anything except maybe some closure. If you've run low-level recovery and it didn't find anything resembling your files, there is probably nothing there to recover.

It's like your files were written in dry ice and just sublimated away without leaving a trace. This one is really beyond me. Maybe another member has some ideas.

 

[Graham J Hadlington]

Well first, let me thank you sincerely for the time you have spent on this issue, it is most appreciated and you have been a gent.

I too am completely baffled here and am not sure which way to proceed. Should I risk sending the internal disk to a retrieval company which may not be able to find these files or, should I just put this matter down to experience? Not really sure which road to take.

If you do stumble upon a solution please get back; who knows, that may be just the glimmer of hope that wiill take us forward.

Graham

 

[Fixer1234]

Those recovery services are very expensive and the price is for looking, not finding. You could spend a small fortune to learn that there was nothing there to find. If you've already run low level recovery software and it didn't find any sign of the missing files, I wouldn't be inclined to pay a service to look again if it were me. In all likelihood, it would be a lot of good money after bad.

For the future, you could consider some safeguards so it can't happen again. A few ideas:

• If you are using the 4 TB drive just for backup and you do it discretely (a "job" at intervals rather than continuously backing up in the background), leave it off or disconnected except when you are backing up or restoring.
  
  
• Make multiple backups and different types of backups--occasional full system images (exact "mirror" copy of the hard drive), plus regular full backup (backup of all the files), plus frequent incremental backup.
  
  You have some big drives and a system image will be the same size. You can reduce the size of system images by defragmenting the drive, including unused space, to move the stored files together and then shrink the partition to a reasonable size for what you expect to store there.
  
  
• Use backup software that allows you to specify a backup plan. Use it to specifically backup all of your important files (as distinguished from just copying whole directories that contain a few items of interest, like backing up all of the user directory tree or all of appdata). Then periodically archive a copy onto CDs or DVDs.
  
  
• Use backup software that stores the files individually and uncompressed. If something happens to a single composite backup file, you lose everything. If a composite backup file is compressed (like a zip file), and something gets corrupted so you can't open the file, you lose everything. If individual files are compressed you need to uncompress them before you can use them. With the size hard disks you have, you don't have to worry about saving storage space.

Anyway, good luck.

 

[Graham J Hadlington]

Thanks again Fixer.

I only carry-out intervals of backup and not continually so that sits well with me

Most UK-based data retrieval companies now support no fix no fee policies and I have found some less expensive alternatives since we spole last. Still not sure if it is worth the expense though as you point out. Using the RAW option on my file recovery software does produce results but, seeing all the results in what looks like gobbledygook leaves you in the position of not knowing which files are worth retrieving or not.

Graham

 

[Fixer1234]

In the words of Emily Litella (old SNL reference), "That's very different". Finding gobbledygook is something. If there is anything there to find, it is raw mode that will do it. The issue is making sense of the findings. If the results are at least identified by filetype, you are way ahead. Here's some thoughts on approaching the results:

• The first objective is just to determine whether any of the gobbledygook could be your missing files. Focus on trying to identify just one or two files.
  
  
• Start with file size. Ignore anything tiny. The only files that are likely to be ones you created that are tiny would be plain text files (with an extension like txt or bat). There is likely to be so many tiny files that it won't be worth trying to find the needle in the haystack.
  
  Anything that Windows classifies as a "document" (any type of file created by a Microsoft Office application), has a lot of meta-data that the application embeds in the file, so even a virtually empty document has a substantial minimum size.
  
  Anything humungous will be very time consuming just to load and look at. These are likely to be things like big programs, compressed files like zip files that contain many other files, or backups that are stored as a single aggregate file.
  
  
• If what you are looking for is huge aggregate-file backups and you find files that size, that would be the place to start. Try renaming it to change the extension to the one your backup program uses and see if the backup program will open it. This won't necessarily work because sometimes backup programs maintain an index of the files it creates and it won't recognize anything not in the index. If it is a good candidate file and this happens, see if there is a way, like a maintenance feature, to access files it can't identify in its index. If you can get the program to just say that the file isn't one created by that program, that is useful knowledge.
  
  If the backup program uses as standard compression file, like zip, see if you can get your stand-alone zip program to open the file (rename it with the appropriate extension for the type of file used).
  
  
• Next on the hierarchy would be to look for easily identifiable file types in the appropriate size range. If you expect to find individual documents created by one of the Microsoft Office applications, those would be good candidates. If you only create one type of document or one type is predominant, like say .doc files), just rename candidate files to that extension and try to open it. If it opens, you will see what the file is.
  
  Another way is to open the raw file in a text editor like notepad. You want a text editor because it won't try to interpret any code for you and because it saves files without adding any of its own junk. Big files may not open in notepad or only the first portion may fit. All you need is the beginning of the file. If the file is too big to open at all in notepad, use something like wordpad, just be careful not to save the file when you are done with it; just close it. Look at the first "paragraph" of header information at the beginning of the file. Much of it may be nonsense characters, but the name of the application is often embedded there. If it is, close the file and rename the extension to match the application. If the file was recovered successfully (all intact), you should be able to open it and see what it is. Actually, if you scroll through the garbage-looking stuff that appears to fill the file in the editor, you may find recognizable chunks of your content, which will tell you what it is.
  
  
• Images are another type of file that are easy to identify. You need a good image viewer that can open many image formats and that recognizes what type of image it is even if the name is wrong. Irfanview is good for this. Look for large images. If you look at a file in the editor and it doesn't look like a document (no snippets of text), rename the extension to the most common image type you use (like jpg). Open the file in the image viewer. In Irfanview, if it is an image but the extension is wrong, you will get a prompt asking if you want to correct it to the right one (say yes). You will be able to identify the image by looking at it.
  
  
• Those are the low hanging fruit.
  
  

 

[Graham J Hadlington]

Well, since talking last I have demoed two more data recovery apps using RAW and still have no folder structure to suggest anything is there? Looking back through this thread I see Trainableman suggested something about the library pointers getting corrupted(?) is this worth investigation perhaps?

Thanks
Graham

 

[Fixer1234]

I'm not sure if Trainableman was referring to the same thing I was or something different. I was talking generically about the file and directory structures. The file data is just scattered around the hard disk platters. When it it written there, Windows knows that it is starting with a specific file of a designated name, and associating it with a designated directory. As the file is written, the computer keeps track of where each snippet is located, where each piece starts and ends, and what the sequence of the pieces is. What makes those pieces your file is really just a list that has your filename on it, the directory it is associated with (also part of the list), and where to go looking for it. If that list gets lost or corrupted, the file's contents are still there, intermixed with snippets of other files.

The raw recovery programs go looking for anything that does not contain empty space as created when the drive was originally formatted. When it finds a snippet of something, there are several things it can be. It can be part of a file that was in use, it could be part of a file that was deleted, or it could be trash left over from when a "fast" reformat was performed. The fast reformat basically does the same thing you are worried might have happened on your hard disk--it just deletes the file list and leaves the contents sitting there available to be overwritten.

The snippet, itself, contains the pointer to the next piece of the file. The recovery program finds snippets and uses those pointers to link them together. When all of the pointers in a collection of snippets are accounted for, that might represent a file. It could be a whole, uncorrupted file. It could be a piece of a file and the rest of it wasn't found (sometimes a piece gives you partial contents, sometimes none of it is useful unless you have the whole thing). It could be a situation where a new file was written to the hard disk while the old one was just unaccounted-for snippets and part of the new file was written in the same location as a piece of the old file. In that case, following the trail of pointers gets you a portion of the old file apparently grafted onto a portion of the new file. The recovery programs can recognize which situation it is for each thing it recovers. It will tell you whether the raw gobbledygook it assembled appears to be a complete file that has not been overwritten. Generally, those are the only things worth dealing with.

If you are speaking literally, libraries are a little different. They are just views of multiple directories. They are also a list, but they don't actually "contain" (define), any content. If a library gets deleted, it doesn't affect the directories or files it pointed to. The library is more like a list of "fun places to see while you are here" on a tourist map. If you lose the map, the places are still there. If you lose the list of files and directories, what is still there is a pile of bricks from which you can rebuild everything.

I'm not sure if the libraries are stored in the same list or a different one. If it's the same list, corruption affects everything. I'm guessing that Trainableman either knows that everything is in the same list or was talking generically about the internal lists. Trainableman, jump in here if I'm wrong. I think the bottom line is that we were referring to the same thing and you've already covered that angle, so that reference isn't another thing to investigate.