Nakivo Backup and Replication

Cross-Platform Module Development

An enterprise-grade backup and replication platform supporting physical, virtual, and cloud workloads — acting as feature owner for multiple cross-platform recovery modules within a distributed product team at Nakivo.

Overview

Industry: Enterprise IT / Data Protection
Product Type: Backup & Replication Software
Architecture: Java 8 monolith with Spring backend, ExtJS frontend, cross-platform script execution layer for agent-side operations on Windows and Linux targets

The platform enabled backup, replication, and granular recovery of files, directories, and directory service objects across heterogeneous enterprise environments. Deployed across highly variable customer infrastructure — physical hosts, hypervisor clusters, and cloud-hosted workloads — the system required per-environment diagnostic analysis during failure scenarios rather than assuming uniform runtime behavior. Each module was owned end-to-end by a single engineer, from implementation through team delivery.

Architecture Context

Environment

  • Customer-premise and cloud deployments across highly variable infrastructure configurations
  • Target systems: physical Windows/Linux hosts, VMware vCenter, Microsoft Hyper-V, Amazon Web Services (EC2/S3), Microsoft Active Directory, Microsoft 365 (SharePoint Online, OneDrive for Business)
  • AWS deployments introduced significant IAM complexity: instance role permissions, cross-account access policies, and S3/EBS access control required per-environment diagnostic work
  • Script execution on target machines via PowerShell (Windows) and POSIX shell (Linux)
  • Backup storage: mounted as external disk on target host, with shadow copy support for VSS-aware workloads
  • Period: October 2017 — August 2021

Stack

  • Backend: Java 8, Spring Framework
  • Frontend: JavaScript, ExtJS
  • Scripting: PowerShell, POSIX shell
  • Cloud integration: Microsoft 365 REST API (SharePoint Online, OneDrive for Business)
  • Directory services: Microsoft Active Directory (LDAP, shadow copy, AD DS object model)

Features

  • Backup and recovery across physical hosts, VMware vCenter, Microsoft Hyper-V, and AWS EC2 environments
  • Active Directory object recovery from mounted backup images via shadow copy
  • Microsoft 365 backup and restore: SharePoint Online, OneDrive for Business
  • Unified cross-platform script execution model with deterministic logging and exit code handling
  • End-to-end module ownership: design, implementation, delivery, and team presentation

My Contribution

  • Took over and completed FileRecovery module: mount/unmount logic, directory tree retrieval, file and directory restoration via cross-platform scripts; delivered stabilization and bug fixes across the existing codebase
  • Designed and enforced a unified execution model for main and logging threads across all PowerShell and POSIX scripts — standardized launch procedure, deterministic exit codes, structured logging pipeline; reduced debugging overhead and support escalation time on subsequent script failures
  • Identified and resolved Linux interpreter mismatch at the script infrastructure level: migrated the entire Linux script surface from bash to strict POSIX sh, eliminating an entire class of environment-dependent failures across customer installations
  • Designed and developed Active Directory object recovery from scratch: backup mount as external disk, shadow copy extraction, LDAP object reconstruction with full attribute mapping
  • Owned cross-environment diagnostic work on AWS-hosted customer deployments: traced IAM policy evaluation failures spanning EC2 instance roles, S3 bucket policies, and EBS access control
  • Implemented Microsoft 365 backup and restore for SharePoint Online and OneDrive for Business via REST API: data extraction, local storage, and restore-back flows
  • Delivered bug fixes across FileRecovery, Active Directory, SharePoint, and OneDrive modules

Engineering Highlights

Linux interpreter mismatch — architectural fix, not a workaround. Scripts executing correctly in interactive sessions failed silently under the daemon user context. Root-cause analysis revealed that the OS assigned the dash interpreter to the service account, while interactive logins defaulted to bash — a behavioral divergence affecting variable scoping, built-in availability, and error propagation. Patching individual scripts or chasing the system-level assignment configuration would have addressed symptoms without resolving the underlying fragility. Migrating the entire Linux script surface to strict POSIX sh eliminated the interpreter dependency at the infrastructure level, making runtime behavior consistent regardless of execution context. This eliminated an entire class of environment-dependent failures across customer installations.

Unified script execution model. The existing script layer had no consistent pattern for managing the main execution thread and the logging thread independently, resulting in inconsistent failure surfaces and unpredictable exit codes across platforms. I designed and implemented a standardized launch procedure applied uniformly across all PowerShell and POSIX scripts: deterministic thread initialization order, structured logging pipeline with explicit flush points, and normalized exit code semantics. The result was consistent failure surfaces across Windows and Linux targets, reduced debugging overhead on support escalations, and a reusable pattern for all subsequent script development.

Active Directory object recovery — domain controller invariants. Classical AD recovery restores objects with their original objectGUID and objectSid — immutable identifiers controlled by the domain controller and unreplayable into a live directory from an external source. Root-cause analysis confirmed no supported path existed for true in-place restore without domain controller-level tooling. The implemented solution: mount the backup image as an external disk, extract the AD database shadow copy, read object attributes via LDAP tooling, and reconstruct objects in the live directory with identical attributes but new system-assigned identifiers. This solution respected domain controller invariants while providing practical business-level recovery capability. Group memberships and ACL bindings dependent on objectSid required post-recovery remediation — a known and documented limitation, communicated explicitly in the feature scope.

AWS environment — IAM policy evaluation as the actual failure layer. AWS-hosted workloads introduced a class of failures absent from on-premise deployments. Unlike physical or hypervisor-based targets where access fails visibly at the network or credential layer, AWS failures were frequently silent or returned misleading error surfaces — actual root cause buried in IAM policy evaluation: EC2 instance role scope, cross-account trust boundaries, S3 bucket policies, or EBS attachment permissions. Effective diagnostics required treating IAM policy resolution as a distinct layer — separate from application logic — and tracing failures through AWS SDK calls, CloudTrail, and policy simulation rather than through the backup agent logs alone.

Microsoft 365 REST API integration. SharePoint Online and OneDrive for Business backup required navigating Microsoft’s REST API surface: correct endpoint selection for site collections, drive items, and version history; handling pagination and throttling; and managing restore-back operations with preserved metadata. No significant architectural complexity — primary challenge was API surface correctness and round-trip fidelity of the backup/restore cycle.

Technologies

Java 8, Spring Framework, JavaScript, ExtJS, PowerShell, POSIX shell, VMware vCenter, Microsoft Hyper-V, Amazon Web Services (EC2, S3, IAM), Microsoft Active Directory, Microsoft 365 REST API, SharePoint Online, OneDrive for Business

Nakivo Backup & Replication
Nakivo Backup & Replication