APPAssureD – Dell AppAssure Backup Plugin For LabTech

appassured-logo600

A new plugin is under development here inside the Squidworks garage that allows Labtech MSPs to manage Dell Appassure Backup Cores servers from a common console inside of Labtech. The new plugin leverages Appassure REST API to mine data from each core and send that data back to the Labtech database. Using the same processes the plugin is able to send commands back to the cores to preform various tasks like start a new backup or replication.

The plugin provides a common view of each core at the location level console in Labtech. You can select one of many cores at any location and view the protected servers managed by that core. Selecting a protected server provides details about that server and the list of snapshots preformed on that system. We have many great features planed for this plugin so start following us now!

 Client Console5-20-look

Dashboard View

dashboard

The plugin is still under development and will we have a release  for early beta testers. Place a comment here you would like to be notified of updates and being included in the beta testing.

Get Version 1.0.1 Beta  here

download

You Will need Powershell 3 on all Cores, Here is a script to install Powershell 3 directly from LabTech

Get PS Installer Script Here

{Solved} Appassure 5 -> Error occurred pairing to agent at URL

Replay.Core.Contracts.Agents.AgentPairingFailedException: Error occurred pairing to agent at URL 

Here is several possible fixes for this type of error based on state of agent systems, Each fix is based on a given type of problem.  I will continue to post other fixes for this error type as I come across them.

 

New Agent Install on Windows DC or SBS server.

So you just installed an Appassure 5 core server and now are trying to protect a new Windows domain controller or maybe an Microsoft SBS server and you keep getting the following errors.

There is a quick fix that worked great for me when this happened to me during a deployment to a Microsoft SBS 2011 system. Found it to be an issue when the TLS client authentication fails between Unified Communications peers where the Configure Group Policy to ignore the list of trusted certification authorities on the computer that hosts the UC client is disabled.

 

Open up REGEDIT on the core and agent system and add a DWord  to following key.

HKLM\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL

 

Add Dword -> SendTrustedIssuerList with a value of  zero.

Now restart the core and agent services and retry to protect server.

It should connect and add new server to core as expected.

 ————————————————————————————————————–

Agents Previously Paired on Another Core

If after removing an agent from protection on a Core that protects other agents, you receive an error asking you to repair the orphaned agent when you attempt to re-add it to protection on a new Core. During the repair it fails with same error “Error occurred pairing to agent at URL”  and the stack trace shows the error below.

 

Stack Trace Error:

Call to service method https://localhost:8006/apprecovery/api/core/agents/pairing/repairOrphan POST failed: Error occurred pairing to agent at URL ‘https://anvil-hosting:8006/apprecovery/api/agent/’


Replay.Core.Contracts.Agents.AgentPairingFailedException: Error occurred pairing to agent at URL ‘https://anvil-hosting:8006/apprecovery/api/agent/’ —> WCFClientBase.HttpUnauthorizedRequestException: Call to service method https://anvil-hosting:8006/apprecovery/api/agent/pair/connect GET failed at WCFClientBase.ClientBase.HandleResponse(Uri uri, String method, HttpResponseMessage response) at WCFClientBase.ClientBase.GetResponse(Uri uri, String method, String knownEtag) at WCFClientBase.ClientBase.ExecuteServiceCall[T](Uri uri, String method) at Replay.Core.Implementation.Agents.AgentsService.VerifyPairedAgentConnectivity(AgentDescriptor agentDescriptor) at Replay.Core.Implementation.Agents.AgentsService.AddOrRepairProtectedAgent(AgentDescriptor agentDescriptor, Boolean isOrphaned)

 

Follow this process to to remove the connection information in the registry of the Cores and Agents involved in the moves.

On the old and new Core servers:

  1. Open the registry editor and navigate to HKEY_LOCAL_MACHINE\Software\AppRecovery
  2. Verify that the agent’s registry key no longer displays this specific agent ID.

On the agent systems:

  1. Stop the Agent Service.
  2. Open the registry editor and navigate to HKEY_LOCAL_MACHINE\Software\AppRecovery.
  3. Export and verify that the old Core pairing is still visible under the Pairing settings.
  4. Delete the entire agent’s key.
  5. Start the Agent Service.

 

Now Protect the agent once again after all services have been restarted. You should not get any orphan or repair warnings and the system should complete without errors.

Goodluck

 

Cubert 😎

 

[Solved] – AppAssure Replay 4.7 Log write failed for device \Device\HarddiskVolume (drive ?:).

AppAssure backups stop making snapshots and you start seeing a flood of log errors in the event viewer reporting Log write failed for device \Device\HarddiskVolume (drive ?:).   the failure status code is the last word of the data. an error event occurred.  eventid: 0xcaa12433

 

I came across a website called Merrit Solutions that has a great little how-to on fixing the dreaded Log write failed for device \Device\HarddiskVolume (drive ?:) error you get in the logs when you have a System Reserve Partition with excessive or oversized AALOGS.

Thanks  Merrit Solutiions for keeping this thread alive.

 

Here are the steps to quickly resolve the issue.

  • Assign a drive letter to the System Reserved Partition in diskmgmt.msc. (This is the 100 Mb partition that works with the operating system.)
  • Make Folders in this newly assigned drive letter visible in Explorer Folder Options (unhide hidden and system folders).
  • Grant your current login account full access to the System Volume Information folder through Explorer Security Tab.
  • Confirm the existence of excessive  AALOG files or file sizes in System Volume Information Folder. (I had one AALOG file that was 77Mb.)
  • There should only be 1 AALOG… file, usually only a few KB in size.
  • If you do not see excessive or oversized AALOGS, do not proceed.
  • Issue “fltmc unload aafsflt” command in elevated Command Prompt (Run As Administrator) to release all of the AALOGs.
  • In elevated command prompt change to the disk assigned above in Step 1) and “cd Sys*”.
  • “del aalog*.*”to delete them.
  • Before closing the Command Prompt run “fltmc load aafsflt” to reload the filter driver and re-create clean new AALOGs.
  • When I deleted the AALOGs I got an alert from my core warning me to reboot the Agent machine.
  • Snapshot the machine again (you should see a Base and incrementals thereafter.)
  • Remove the drive letter assigned in the first step.
  • Restart the server or atleast restart the replay agent on server and logs should stop.

[Solved] – Unable to take VSS Snapshot and transfer data for ‘C:’ Unable to take VSS Snapshot of volume(s) because the VSS subsystem is in a bad state.

App Asure Replay CoreAppAssure Replay backup reports Errors, ‘Unable to take VSS Snapshot of volume(s) because the VSS subsystem is in a bad state.’ or ‘Error taking snapshot of volume(s): AddToSnapshotSet failed when trying to add volume ‘C:’ – VSS Provider is not registered’

 

I found this to be a fairly common issue when deploying new Replay agents out to Windows 2003 servers. There are several things that should be addressed when deploying Replay agents to Windows 2003 Servers including repairing LSP, refreshing and / or verifying VSS states and removing conflicting software. I have had great success resolving these types of issues by following these simple steps.

#1  Remove Offending Software Conflicts

Offending Software:

Acronis Backup & Recovery 10 Agent

Symantec Backup Exec Remote Agent for Windows

any 3rd party backup software

 

 

#2 Reset VSS Writers

I use a simple bat file that re-registeres all my VSS writers and then I follow that up with a reboot. Below is the bat script commands, copy them to a file, name the file vss.bat, then run the file. It should shutdown a few services and re-register the VSS writers.  reboot is needed after this has completed.

net stop “System Event Notification”
net stop “COM+ Event System”
net stop “Microsoft Software Shadow Copy Provider”
net stop “Volume Shadow Copy”
cd /d %windir%\system32
net stop vss
net stop swprv
regsvr32 /s ole32.dll
regsvr32 /s oleaut32.dll
regsvr32 /s Vss_ps.dll
vssvc /register
regsvr32 /s /i swprv.dll
regsvr32 /s /i eventcls.dll
regsvr32 /s es.dll
regsvr32 /s stdprov.dll
regsvr32 /s vssui.dll
regsvr32 /s msxml.dll
regsvr32 /s msxml3.dll
regsvr32 /s msxml4.dll
net start “COM+ Event System”

 

#3 Repair the Layered Service Providers

Layered Service Provider (LSP) is a feature of the Microsoft Windows Winsock 2 Service Provider Interface (SPI). A Layered Service Provider is a DLL that uses Winsock APIs to insert itself into the TCP/IP protocol stack. Once in the stack, a Layered Service Provider can intercept and modify inbound and outbound Internet traffic. It allows processing of all the TCP/IP traffic taking place between the Internet and the applications that are accessing the Internet (such as a web browser,  email client or Replay agent.).

This is a common problem in Windows 2003 especially if you use Kaseya agents on these systems. The kaseyasp.dll causes some issues with transferring the file across the network. Luckily this is a really easy fix. Download and run LSPFix.exe on the Windows 2003 Server system. It will find the Kaseya Dll issue and repair it for you.

 

#4 Verify no Disk Errors are present

This often overlooked but a simple “CHKDSK C:” at the command prompt will scan your disk and verify if there are any errors on your disk. Using the “/F” switch you can fix the errors. Make sure you have clean disks.

 

By following these steps you should resolve 98% of your common backup issues with Replay clients failing snapshots or not transferring the snapshot once it has been taken.

 

I hope this helps fellow AppAssure users out there in cybertown.

 

Enjoy

Cubert 😎

[Solved] Appassure Replay snapshots or base image fails to copy from Agent to Core on or after new installs

So you may have a Windows 2003 server or servers that after installing the agents, configuring permissions and setting system to Protect on your core keeps failing to grab the first snapshot from your system.

This is quite common in Windows 2003 but may see it in 2008 as well. The issue is that the Winsock is messed up. Download and run LSPFix.exe on the client system and restart the agent services afterwards. Then go back to the core and select to force new snapshot to start the download. You should see the system start the download with in 5 minutes and no more errors in events.

 I am using Replay version 4.7.2 at the time of this post

Error that may show up when fix is needed:

Logger: tevolib.txtrack
Context: PG=VolsWithInterval60Minutes Volume=\\?\Volume{741d2f02-3eb0-11dd-8e1a-806e6f6e6963}\ DriveLetter=C:
Source Location: TransmissionTracking.cpp:115
Details:
Transmission of volume ‘C:’ started when another transmission of that volume is already in progress

Logger: tevoSource.snapshotHandler
Context: PG=VolsWithInterval60Minutes Volume=\\?\Volume{741d2f02-3eb0-11dd-8e1a-806e6f6e6963}\ DriveLetter=C:
Source Location: SnapshotHandler.cpp:992
Details:
Transfer failed on volume: C: Epoch: 204 Target: CNSCOLO-BDR Port: 8001 – Unspecified error

Logger: exceptions.seh
Context: PG=VolsWithInterval60Minutes
Source Location: exceptions\SehHandler.cpp:151
Details:
Encountered a serious error EXCEPTION_ACCESS_VIOLATION.  Exception dump logged to C:\Documents and Settings\cns\Application Data\AppAssure\ErrorDump-TevoSource.exe-2012-02-28-11-43-41-0625-41.dmp.bz2

 

Enjoy

Cubert 😎