NAS Recovery and NAS Crash Prevention Tips from Gillware Data Recovery

For a small business owner or a small business IT technician, nothing is more headache- and panic-inducing than a NAS server crash. NAS recovery after such a crash has its own issues to avoid making things worse and making subsequent data recovery more difficult. Unintended formatting of a NAS drive is just one serious example and yet so easy to do if you do the wrong thing. So make sure you read up on what to do before you start. At TheTechMentor.com we have a number of articles to help with hard drive data recovery, but a NAS failure is likely to be serious enough that you should enlist a professional data recovery team rather than do it yourself.

Professional data recovery company Gillware Inc. have dealt with countless NAS data recovery cases over the years, and below they give 7 tips for what to do when your NAS system crashes. They don’t just stop there as they also give a further 4 tips on how to avoid such problems in future. Read on, and good luck with recovering your data and making your system more robust.

The NAS devices we see range from everything from consumer-grade products like WD MyCloud, Iomega StorCenter, and Synology DiskStation to enterprise-level NAS servers like the Synology RackStation and Dell EqualLogic.

Over the years, we’ve seen NAS recovery cases that were made much more difficult for our engineers as a result of bad decisions made by the clients in the first hour or so after failure. Poor decisions, made in the heat of the moment, can cause irreversible data loss, which is ultimately bad for the client (and for us). When you’re dealing with a NAS server crash, what you do before you send it to a data recovery lab can make all the difference.

Some preventative care in setting up your NAS server can even help you skip the data recovery lab altogether. This is good for you, and we want what’s good for you (even if it doesn’t make us any money).

NAS Recovery: Keeping a Cool Head After Your NAS RAID Server Bites It

Whether you use an enterprise-grade NAS server or a repurposed consumer-grade NAS device for your small business, you need to account for the fact that these devices can and do fail. When they do, it can be entirely without warning, and you’ll want to get things up and running as soon as possible.

Sometimes you can do that by yourself; sometimes you need help from a professional data recovery company (link opens in a new tab). Either way, what you do when your NAS device fails has a huge impact on your likelihood of successfully recovering your data. Here are seven NAS recovery tips to make your chances of getting your data back quickly and (relatively) painlessly as high as possible:

NAS Recovery Tip #1: Don’t Panic

If your NAS device is mission-critical, getting it up and running again quickly and (if necessary, restoring from a backup) is paramount. Whether you’re a small business owner, or the IT guy, people come running to when there’s a problem, you’re under a lot of pressure to fix things so you can start serving your customers again with minimal downtime.

If you leap in and try to go for quick fixes, though, there’s a good chance you could just make the problem worse. You need to keep a level head and think carefully about what you need to do before you do it. In the end, taking two hours to do the right thing will serve you better than taking half an hour to make an even bigger mess.

So take some deep breaths, make some tea, put on some classical music, and consult your Hitchhiker’s Guide to the Galaxy for the best advice you’ll hear all day: “Don’t Panic.”

NAS Recovery Tip #2: Don’t Start Pushing Buttons, Read the Manual

NAS servers often use diagnostic codes and patterns of LED lights to explain what’s gone wrong. Your server’s manual will help you decipher these codes. Often you can rely on them to tell you exactly where your server went wrong, whether it’s a hard disk problem, a RAID controller problem, or some other issue.

If it’s a hard disk problem, it’s always better to confirm which drives have failed before you start yanking drives out. At Gillware, we’ve seen a surprising amount of server data recovery cases in which an IT technician meant to swap out the failed drive in a degraded RAID-5 server, but accidentally crashed the server by plucking a perfectly healthy drive instead of the one that had failed.

NAS Recovery Tip #3: Think Before You Try to Rebuild Your RAID

RAID-5 and RAID-6 arrays use parity data, spread across the disks in the array, to “fill in the blanks” when hard drives crash. RAID-5 can use this parity to reconstruct data if one hard drive crashes, but fails if multiple drives crash. RAID-6 can reconstruct data if two hard drives crash, but fails if more than two drives crash.

When you notice one drive failed in your RAID-5 array, or two drives failed in your RAID-6 array, removing them and inserting fresh drives will “rebuild” the array. The same parity data that picks up the slack for the missing drives works to copy the contents on the deceased drives onto the new drives.

Rebuilding a RAID array is far from a cure-all, though. A rebuild won’t resurrect an array that has crashed. Rebuilding only works when the array is still functional, within the limits of its fault tolerance—this is called a “degraded” array. Sticking fresh drives in a NAS that has already crashed and telling the RAID to rebuild itself will not fix your NAS. Rather, it can corrupt massive amounts of data and make your NAS server failure much harder to bounce back from.

Rebuilding even a degraded array can be risky. It puts the rest of the drives under tremendous stress, slowing the NAS’s performance. For this reason, some IT technicians might put off RAID rebuilds for as long as possible, Unfortunately, this can lead to the RAID never getting a chance to be rebuilt before another drive failure crashes the NAS. Worse, delaying a rebuild can make the rebuilding process much riskier. Putting it off can result in one or more drives failing under the strain of a rebuild, crashing the array and doing the very thing rebuilding is supposed to prevent.

NAS Recovery Tip #4: Remember, RAID Drives Work as a Team

Your NAS device’s drives work as a team. Aside from RAID-1, your server breaks up all of your data into chunks and distributes it across the hard drives inside it. Every hard drive only has a piece of the whole—not just in terms of the files you put on it, but the actual file system itself as well. The end result is that if you want to pull data off of a failed NAS server, hooking up the drives as individuals to another computer won’t cut it. Your computer won’t even have a complete file system to read, let alone any complete files (although you might be able to salvage a few files smaller than the RAID array’s block size).

NAS Recovery Tip #5: Don’t Force Drives Online

Some NAS devices and servers can force perfectly healthy hard drives offline for very stupid reasons. It might be tempting just force a server to bring a drive back online. However, in this case, you really don’t want to presume you know more than your RAID array, for two reasons.

First off, you have no idea if your NAS device rightfully took that drive offline. If there is something wrong with the hard drive, forcing it back online can exacerbate its problems, leading to situations like mangled read/write heads, scratched disk platters, or a stuck spindle motor.

Second, the drive could have been offline for a long time. When your NAS server kicks off a hard drive, it stops writing data to the drive. Depending on how much use your server sees, putting an offline drive back online weeks, days, or even mere hours after the fact is a bit like unfreezing Phillip J. Fry in the year 3000 in Futurama. Time has marched on, but the data on the “stale” drive hasn’t changed. Trying to force a stale drive into a degraded RAID array will result in widespread data corruption, as you can see in our case study about salvaging data from a RAID-5 array with one stale drive (opens in a new tab).

NAS Recovery Tip #6: Verify Your Backups

Of course, you have a backup (and if you don’t, you should absolutely set one up ASAP). But there’s more to a backup than having one. We’ve seen this story a thousand times in our data recovery lab—a small business owner had a backup system, but the backups had been made too infrequently. Either the data they needed hadn’t gone onto the backup system, or the system settings on the backup no longer matched the settings of the actual NAS device’s software or hardware, resulting in a phenomenon known as “configuration drift”.

In other words, they had a backup, but their backup was useless.

To avoid being lulled into a false sense of security, periodically test your backup systems and make sure they work as intended and work frequently, and that a failure won’t cost you weeks or months of data.

NAS Recovery Tip #7: If It Comes to That—Choose a Good Data Recovery Lab

NAS recovery isn’t easy, and sometimes you need some outside help. A NAS server crash can happen for a variety of reasons, and some crashes are more severe than others. When you can’t bounce back on your own, you need to get help from professional NAS data recovery experts.

Choosing a data recovery lab is very difficult. You’re under a lot of pressure to make a snap decision, which leaves you with little time to make an informed decision. Often, this leads to business owners going with whatever data recovery lab their IT consultant or managed service provider recommends. But for those who need to do their own research, it’s hard to choose the perfect data recovery company under such intense pressure.

Here are some questions to keep in mind when searching for data recovery labs:

– Do they have professional data recovery facilities? You should be looking for mention of an ISO-5 Class 100 or higher lab, where hard drives can be safely repaired. The lab’s security credentials, grade on the Better Business Bureau website, and social media presence are also key indicators of professionalism. A professional data recovery lab has clean and secure facilities and a history free of major disputes with customers.

– Is their lab staffed with NAS and RAID recovery experts? A professional data recovery lab has expert computer scientists with experience working with all sorts of NAS devices and RAID arrays. If the company publishes white papers or case studies, you can get a good sense of what their engineers are capable of.

– Do they offer expedited services for data recovery emergencies? To get your small business up and running again, time is of the essence. Too much downtime can put a huge dent in your revenue and even your reputation. A professional data recovery company can provide expedited emergency services with turnarounds as low as one or two business days.

– How affordable/risk-free are their data recovery services? You want a company that strikes a good balance between affordability and professionalism. Steer clear of flat-rate or per-gigabyte scammers, and labs that will take your money and run if they can’t recover your data. An affordable, customer-friendly data recovery lab will offer financially risk-free services with free evaluations, and won’t charge you unless they can meet your data recovery goals.

How to Head Off Future NAS Server Crash Crises

You can’t prevent every NAS server crash. But through careful planning, you can prevent most of them, and recover from the ones you can’t prevent much more easily. Whether you’re using an enterprise-grade NAS device or a repurposed consumer-grade NAS device, here are some tips to help keep your data just a little bit safer:

NA S Server Crash Prevention Tip #1: Use RAID-6, If Possible

This advice applies more to users of enterprise-grade equipment. Consumer-grade NAS devices typically only offer three possible RAID levels: 0, 1, and 5 (and sometimes 10). RAID-0 is disk striping with no redundancy. RAID-1 is disk mirroring with no gains in capacity or performance. RAID-5 uses parity to provide one disk’s worth of redundancy. Some consumer NAS devices support RAID-10, which is a combination of RAID-1 and RAID-0 that, in practice, functions similarly to RAID-5.

However, most enterprise-grade NAS servers can support RAID-6. RAID-6 uses extra parity computations, so that it can still run after as many as two drives fail. This provides extra cushioning in case of hard disk failure and a wider window of opportunity to replace failed disks and safely rebuild the array.

NAS Server Crash Prevention Tip #2: Configure Your NAS Properly

This is a problem that can affect both consumer-grade and enterprise-grade NAS devices. However, it tends to crop up more with consumer-grade NAS devices. NAS servers can be configured to send automatic email alerts regarding hard drive failures or other irregularities. However, they must be properly configured to do so. Many small business owners who pick up consumer-grade NAS devices expect them to be configured out-of-the-box—but unbeknownst to them, that work still needs to be done.

As a result, the NAS device will eventually fail without sending out any warning signs. Had the device been properly configured in the first place, a failed disk could have been replaced easily before future failures jeopardized the data on the NAS.

NAS Server Crash Prevention Tip #3: Automated, Incremental Backups

We’ve discussed the importance of backups in mitigating a NAS server crash. To make your backup as effective as possible and give it the best chance to save your bacon, it needs to be automated and run on a regular basis. This process could run at the end of the week, or better yet, the end of the day, when your NAS server would normally be idling (or otherwise experiencing low usage), and without any human input. If anything should go wrong with the backup process, your automated backup service should send out an alert.

NAS Server Crash Prevention Tip #4: Variety (In Hard Disk Models) Is the Spice of Life

The hard drives inside a NAS device often come off of the assembly line mere minutes or seconds after one another. They’ll often have similar life spans or manufacturing tolerances as a result. This might sound like a good thing—but it isn’t. It can actually make a NAS server crash more likely, because all of the drives in your RAID see roughly equal amounts of use, and could wear out and die within days or hours of each other. This makes rebuilds much riskier.

When possible, use hard drives with different manufacturing dates, and even different model numbers. This will decrease the likelihood that your hard drives will all last for roughly the same time. This way, when one hard drive fails, you can be reasonably more confident that trying to replace the failed disk and rebuild the array won’t result in further hard disk failures.

This tip, again, holds true mostly for enterprise NAS devices. Consumer-grade NAS devices like the WD MyCloud are often designed as closed systems, and will often only accept hard drives of a particular brand and model. Enterprise NAS devices usually lack these restrictions. Some NAS manufacturers even provide the option of buying the device sans hard drives, so the user can supply their own drives of their choosing instead of letting the manufacturer pick out drives for them.

Summary for these NAS Recovery and Crash Prevention tips

Hopefully, these NAS data recovery tips and server crash prevention tips can guide you in creating more robust NAS systems. All the same, though, all the NAS server crash prevention in the world can only take you so far. No matter how fault-tolerant your setup is, failure is a matter of when, not if. What you do to set up your NAS, as well as your behavior in the seconds and minutes after a NAS server crash, has a dramatic influence on your chances of a successful NAS recovery. To recover your own data at home if it is not that important (ie. be aware of the risks), check out our other articles on hard drive data recovery. Feel free to add your comments below if you have something more to share (note comments are manually moderated).