How to Manage a Datacenter Failure or Disaster Recovery Scenario in Exchange 2010 – Part 3

Exchange 2010, High Availability

In Part 3 of this series, we are going to discuss how to failback to your primary site, after whatever condition that caused it to go offline in the first place occurred.

So when we last left off, we have activated all of our databases in our disaster recovery site. I showed you how had to stop the mailbox servers in the primary active directory site, which then allowed us to activate our disaster recovery site.

So now, let’s say the flood that occurred in the New York site has been fixed, all the water’s been removed, and thankfully, it did not damage our equipment. We are ready to move everything back to NYC. We begin by powering on the servers in NY. Since our DAG is in DAC mode, the NY servers do not try to mount their copies of the databases. Instead, the begin copying over any log files so as to bring their copies of the databases up to date with the DR site. Wait until all the database’s are reporting a status of healthy:

Now, remember, we told the DAG that the NY servers were down with the Stop-DatabaseAvailabilityGroup command from Part 2:

Notice how NYMB01 and NYMB02 are both listed as Stopped Mailbox Servers. You will get this error in the console if you try to activate the database on one of those two servers:

What we have to do is start these servers in the DAG, making them viable copies for activation again. We do this with the Start-DatabaseAvailabilityGroup command. Again, we could use with the –MailboxServer command and specify each server, separating them with a comma, or we can specify the whole site with the –ActiveDirectorySite option as below:

Start-DatabaseAvailabilityGroup DAG1 –ActiveDirectorySite NYC

Now if we run the Get-DatabaseAvailabilityGroup command, all servers are listed as started

Now, Microsoft recommends dismounting the databases that are going to be moved back to the primary datacenter site. This will mean that you will need to get a maintenance window to perform this action, as the databases will be offline. Keep in mind that you do not NEED to dismount the databases; it is just recommended by Microsoft.

Now, we can activate the Database. You can do this either by using the shell with the Move-ActiveMailboxDatabase MDB01 –ActivateOnServer NYMB02 command:

Or through the EMC with the Activate a Database Copy Wizard:

Now, the database should be activated. One issue that I have seen pop up is that the Catalog Index is corrupted. This is the index for the Full Text Index Search. If this is the case, you may have to reseed the Catalog Index with the following command:

Update-MailboxDatabaseCopy MDB02NYMB02 –SourceServer DRMB01 –CatalogOnly

This command will reseed the content index from the server DRMB01. You should now be able to activate the database copy.

Continue by moving all active database copies to their respective servers. If you dismounted the databases as stated above, now mount these databases.

Now, our clients are still connected, but they are still connecting through DRHT01.nygiants.com because it is set as the RpcClientAccessServer:

17-Dec09 15.08

We simply need to run the command Get-MailboxDatabase | Set-MailboxDatabase –RpcClientAccessServer NYHT01.nygiants.com to change this setting over:

If we check the properties of the databases, we see that NYHT01.nygiants.com is again set as the Rpc Client Access server:

Now, our Outlook clients will be connecting through NYHT01 for their access:

Well, that’s it!

In this three part series, I should you how to activate the backup datacenter, in the event that the primary datacenter became unavailable. We discussed the theory behind the process, mainly Datacenter Activation Coordinator, the actual work needed to do it, as well as how to failback when the primary site came back online.

If you have any questions, feel free to email me at ponzekap2 at gmail dot com.

Comments;

  1. Nabil Zahid

    Hi ,
    One Great thanks to you for this great job ! i tested these stepes and all is Gone Ok . i ve used the Microsoft documentation but none of theire procedure worked !! i started to make my own one based on the understanding of hwo DAG works ,when i searched for forcing the quorum i falled on your blog .
    So thanks for giving such help to Admins .
    (one last question is hwo to to switch the dag to use the primary witness server instead of the alternateOne when reswitching to the primary datacentre ?)

    Reply
    1. ponzekap2 Post author

      When the primary FSW comes back online and you restart the clustering services on the DAG members to move it back, that should re-activate the primary FSW

      Reply
  2. Rahamathullah

    Hi, Really it is good article.. The way it is explained is wonderful..

    I am working as a SME for last 6 years ..I have gone through N number of articles ..But this is one of the best explanation…Keep Rocking…

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *