Exchange 2010 and Backups Pt. 3

Restoring Databases, Mailboxes and Deleted Messages
In Part 2 I described how to perform a backup (as opposed to using DAGs). Now its time to consider recovery. At some point you will want to perform the following types of data recovery:

• Entire mailbox databases
• Individual user mailboxes
• Single items in a mailbox

We will now look at these different scenarios
Recovering Databases with the Windows Server Backup Tool
A good visual walk-through on how to restore the database from backup is provided by the following link Database Restoration.
The following link provides a complete breakdown of WBAdmin commands WBAdmin Commands
and the recovery details are provided with this link WBAdmin Recovery Details
Recovering Mailboxes
Mailboxes can be recovered in the following ways:

1. Using the deleted mailbox retention
2. Using a recovery database
3. Using a lagged database copy

1. Using the Deleted Mailbox Retention
When a mailbox is removed from a database, it becomes disconnected from its active directory account. The disconnected mailbox stays disjoined for a period of time determined by the databases retention time (30 days by default). The act of removing  the mailbox will actually remove the active directory account from AD database. So, if you want to reconnect the mailbox back to the original AD account you will first have to bring the user 'back from the dead' perhaps by using the LDP tool. The general steps involve:

• Use the LDP tool to bring the deleted user object back from the deleted containers object
• Reconnect the mailbox with the original account (or another account if you prefer)

Of course this has to be done within thirty days of mailbox deletion. To determine which mailboxes are in a disconnected state type the following:

[PS] Get-MailboxStatistics -Server servername | Where-Object {$_.DisconnectDate -ne $null} | fl dis*,*guid

This will display all disconnected mailboxes, when they became disconnected and just for fun the mailbox GUID numbers. You can use the GUID identifier or the displayname to reconnect the mailbox:

[PS] Connect-Mailbox 46d34411-1c85-4f49-8772-7d8572a4dd1d -Database db2 -User "Rodica Woodcock" -Alias Rodica

As I mentioned, this has to be done within the 30 day retention time. You can extend this time if you wish by typing in the following cmdlet:

[PS] Set-MailboxDatabase DB2 -MailboxRetention 60.00:00:00

This command will extend the retention time by another 30 days to a total of 60 days

2. Using a Recovery Database
How would you recover a mailbox if the mailbox retention time has been exceeded? To do this you can create something called a recovery database. This is like a regular database but users cant access it with client software. Also mail cannot flow to and from a RDB nor can policies be applied to it.
The overall steps involve the following:

1. Create a RDB
2. Restore the normal database from backup
3. Point the RDB to the location of the restored backup and mount it
4. Restore the mailbox that was deleted

1. How to Create a Recovery Database (RDB)

[PS] New-MailboxDatabase RDB1 -Server srv215 –Recovery

2. Restore the Database From Backup
Refer back to the proceedure outlined above to do this. NB. You should restore the database to an alternate location. It is also suggested that you should repair the database (in case of corruption). The following command refers to a restored database to the alternative location of e:\rdbrestore. You must provide the full path to the .edb file

[PS] eseutil /p E:\rdbrestore\E_\db2\db2.edb

3. Point the RDB to the Restored Location

[PS] Move-DatabasePath RDB1 -EdbFilePath E:\rdbrestore\E_\db2\db2.edb -LogFolderPath E:\rdbrestore\E_\db2 –ConfigurationOnly

4. Mount the RDB

[PS] Mount-Database RDB1

Once mounted the recovery database can be used to recover a deleted mailbox. You do this with the Restore-Mailbox cmdlet.You will first need to enable the mailbox. If you removed the mailbox in practice, then the user account will have been deleted. Recreate the user in AD and then enable the mailbox.

3. Using a Lagged Database Copy

This involves replicating transaction logs to a passive database, waiting a specified amount of time and then replaying them into that copy. Since you build a delay into this model, when you recover from the copy you might incur some data loss since you will be deleting transactions that are valid.

The maximum lag time you can use is 14 days. To set a replay lag time of 7 days use the following cmdlet:

[PS] Set-MailboxDatabaseCopy SRV2\DB01 –ReplayLagTime 7.0:00:00

To recover from a lagged database copy, perform the following main actions:

• Suspend replication to the lagged copy (pausing database replication)
• Take a VSS snapshot of the volume that holds this copy
• Figure out the date/time you want to restore the database to and delete all transaction logs after that time
• Repair the database using ESEUtil
• Recover the data from the database
• Restore the VSS snapshot of the volume
• Resume replication to the lagged copy

Exchange 2010 and Backups Pt 2

Using Windows Server Backup with Exchange 2010
Exchange 2007 running on Server 2008 was not a happy marriage. Support for native exchange backups was removed. However here comes Exchange 2010 and with it the Microsoft Exchange Service Extension which allows you to backup up Exchange using Server 2008 Windows Server Backup. While the situation has improved (at least a backup is possible!) using Windows Server Backup has its limitations:

• Only full volumes can be backed up. This means only the whole, complete volume can be backed up. You cannot make independent backups of a database or logs files. Only the volumes these files reside on can be backed up. Because of this, both .edb and log files for any particular database must be available together in the same backup set inorder to restore the data.
• You cannot perform remote backups
• In order for log files to be trucated only full backups can be made
• All of the databases that are in your backup set will be restored. You cannot restore a single database if your backup set holds multiple databases.
• Only an active copy not passive copies of the database can be backed up 

Using WBAdmin to Perform Manual and Automated Backups
So, the first thing to do is to install the backup feature to your Server 2008 server:

[PS] ServerManagerCMD -i Backup-Features

Once the feature has been installed, you can perform a one-time backup using the following:
This link will provide a good visual to the process using the GUI.

how-to-backup-exchange-2010

The following will create a full backup to a network share. Note the use of -vssFull which should be used. Performs a full backup using the Volume Shadow Copy Service (VSS). Each file's history is updated to reflect that it was backed up. If this parameter is not used WBADMIN START BACKUP  makes a copy backup, but the history of files being backed up  is not updated.

[PS]  wbadmin Start Backup -backuptarget:\\ServerName\ShareName -include:c: -user:domainname\administrator -password:AccountPassword -vssFull

If you need to perform a scheduled backup you could try the following procedure. Note that the GUI wizards will only allow you to schedule backups to locally connected drives, USB drives/sticks and firewire. You can't backup to NAS drives. So how do you do it?

1. Save the above command line as a batch file
2. Schedule the script to run using the Windows Task Scheduler (found in Admin Tools). Create a basic task and end of the wizard ensure that in the task properties select Run whether the user is logged on or not, and check the Run with highest privileges checkbox. Also ensure that the user that the task is running as is an Administrator or part of the Administrators or Domain Admins group.

 

Exchange 2010 and Backups Pt 1

It has been argued that backup management in Exchange 2010 has changed dramatically. With the implementation of DAGs, replica databases and effective log shipping and replaying the need for database backup has disappeared. Apparently since moving onto Exchange 2010, Microsoft have eliminated the need to maintain data backups. Does this sound a little scary? Well consider the following and make your own mind up:

• Server Failure Add mailbox servers to a DAG and replicate databases to other servers in the DAG
• Disk Failure Place the edb file and log files on the same disk and ensure the database is replicated to at least three DAG members
• Database Corruption If a database goes bad you can set the ReplayLagTime so that copies of the database do not incorporate any dodgy transaction logs that caused the corruption on the active master. See this post. 
• Single Message Recovery This is an important consideration but easy to solve. Exchange now has single item recovery built in. You need to enable it and specify how long you want to keep deleted items for. 
• Mailbox Recovery Deleted mailboxes are maintained for 30 days by default which can be configured to be longer if required
• Site Failure You should stretch your DAG to include subnets in different sites.

One thing you would consider doing if going for a DAG based backup solution, is using circular logging. This will help keep the log sizes down (which would normally get truncated with 'normal' backups).
 
To enable circular logging of your log files, type in the following:

[PS] Set-MailboxDatabase DB01 -CircularLoggingEnabled $True

Redundency of Exchange 2010 Mailbox Database Servers

In an earlier post, I covered the creation of a DAG ( Database Availability Group) to provide mailbox database redundency. How does this differ to earlier methods of redundency used by Exchange 2007? Both CCR (Continuous Cluster Replication) used in Exchange 2007 and DAG used in Exchange 2010 are offshoots of Windows Failover Clustering (WFC).

CCR – as its name implies – works by allowing you to create a cluster during the installation of Exchange 2007.  This one is a bit easier to see as part of WFC, as you have to create a Failover Cluster first – specifically a Distributed Majority-Node File Share Witness Failover Cluster.  After that, when you install Exchange Server you can specify which server(s) will be the Active node(s) and which will be passive.  This creates the clustered Exchange resources for you, making the overall process of setting up clustering for Exchange a lot easier.


DAG will permit you to create the cluster itself from Exchange 2010 command sets, eliminating the need to pre-create the Failover Cluster prior to getting the Exchange installation rolling.  While this makes the process even easier than in 2007, it still requires that you have two or more servers capable of running Distributed Failover Clustering.  This means that not every version of Windows 2008 is going to be suitable for DAG, but also means that – under the hood – you still need to know how Distributed Failover Clustering works to properly manage the DAG systems.

How Does Exchange Determine How Databases Failover?

When the active database fails, failover to a replicated copy occurs automatically within a time period of aound 30 seconds. But which passive copy is chosen? The component that determines this is the Active Manager.

The Active Manager has essentially two parts , the SAM and PAM:

1. The Primary Active Manager which sits on just one of the DAG mailbox servers. This determines which replica should be active.
2. The Stanby Active Manager which sits on every other mailbox server in the DAG. This notifies the PAM of when a database fails and also the HTs  and CAS servers about which replica is active.

When the active copy fails the Active Manager makes a determination of which replica should be targeted to become active. This involves looking at the database copy status, health of the content index and length of the log copy queue. After determining potential targets, the database replica with the lowest Activation Preference Number will win.

To set this value on a database replica type the following:

[PS] Get-MailboxDatabase  -Server srv211 | fl name,act*

This cmdlet will show you the current activation preference numbers for the database replica (SRV211 hosts a replica database).

[PS] Set-MailboxDatabaseCopy -Identity DB01\SRV211 -ActivationPreference 2

Redundency of Exchange 2010 Client Access Servers

The redundency method used with CAS is Network Load Balancing or NLB. The cluster of up to 32 servers is called a client access array. Broadly speaking the three steps involved in creating an array are:

1. Install the NLB feature
2. Create the NLB cluster
3. Configure the client access array in Exchange 2010 

It should be noted that if a CAS is also hosting a mailbox server role which is in a DAG, the CAS cannot be part of a DAG.

To install the NLB feature on your CAS servers type the following:

[PS] ServerManagerCmd -I NLB

To create the NLB cluster on Server 2008 perform the following steps:

1. Log in at a CAS
2. Launch NLB Manager from Administrative Tools
3. Choose Cluster, New to start a new cluster wizard
4. On the Connect screen type in the IP address or FQDN of a CAS that will be in your CAS array. Try typing in the name of this CAS server, and click Connect.
5. You may see two NICs listed. Select the NIC that is used for sending/receiving client access traffic
6. On the Host Parameter screen, select the priority of the host. 
7. On the Cluster IP Address screen, click the Add button to add the IP address of the cluster. In the Add IP Address dialog box, type the IP address, mask and click OK.
8. On the cluster Parameters screen, type the FQDN of the cluster, e.g. nlb.yourdomainname.com.
9. In the cluster operation mode select multicast mode.
10. On the Port Rules Screen, select the port rule with the start port of 0 and the end port of 65535 and click the edit button
11. In the Add/Edit Port rule dialog box, change the Port Range From value 1024,and click OK. What does this do? Well, it will modify the load balancing port range to include the dynamic ports that can be assigned by the RPC endpoint mapper. 
12. Add another rule and set the from and to values to 135 using TCP. This makes sure that the RPC endpoint mapper is load balanced as well.
13. You will need to add any other ports that are to be load balanced by the CAS array. This of course will mean adding port 80, 443 (and ports for IMAP and POP3 but we will not be testing these services).
14. Click Finish to create the NLB cluster with the first node added.
15. Now the cluster has been created you will have to add the other nodes. Right click the cluster and select Add Host to Cluster from the Menu.
16. Type the FQDN of the CAS server you wish to add
17. Like before, select the correct NIC
18. Provide a priority number but you do not need to adjust the port rules.
19. Finish to close the wizard. 
20. It may take some time before the CAS NLB cluster fully converges. Don't worry. It will!

Now the NLB cluster has been created, in step three we configure the CAS array. 

The CAS array involves creating a site linked object in Active Directory. This means that all CAS servers in a particular site will be in this array. Mailbox servers will be logically tied to this array by virtue of their 'RpcClientAccessServer' attribute on the mailbox database. Remember this attribute on the database will reference the RpcClientAccess service running on the CAS servers which you just network load balanced. The attribute will point to the cluster which of course consists of at least 2 CAS servers.  

To create the CAS Array object and to assign it to your London site type the following:

[PS] New-ClientAccessArray -FQDN nlb.compulinxtraining.com -Site London

Thats it! Pretty simple really. However, we have the DB01 database created before the creation of the CAS array. This database will point to the CAS server before the creation of the NLB cluster. Try the following to determine this:

[PS] Get-MailboxDatabase DB01 | fl Name,RpcClientAccessServer 

This should display exactly which CAS server the database is currently looking at. To change it type the following:

[PS] Set-MailboxDatabase DB01 -RpcClientAccessServer nlb.compulinxtraining.com

So, thats how you configure redundency for CAS servers. Build an NLB cluster, build a CAS array object and then assign your database to use this object. Remember that any new databases will be tied to this array automatically.