Protecting Oracle Database deployed on EC2 Instance- Part I 3 months ago

ORACLE server is one of the key components in the data center. It is critical to protecting the database on a periodic basis. Backup and recovery refer to the process of backing up data in case of a loss and setting up systems that allow that data recovery due to data loss. Backing up data requires copying and archiving computer data so that it is accessible in case of data deletion or corruption. Below are the different types of backups of ORACLE supports.

Oracle – Types of backup:-

  • Cold Backup – A cold backup is done when there is no user activity going on with the system. Also called as an offline backup, is taken when the database is not running and no users are logged in. all files of the database are copied and no changes during the copy are made.
  • Hot Backup – A hot backup is taken when the database needs to run all the time. It is an online backup. All files of the database are copied and there may be changes to the database during the copy.

The benefit of taking a hot backup is that the database is available for use while the backup is occurring and one can recover the database to any point in time. The benefit of taking a cold backup is that it is typically easier to administer the backup and recovery process. For cold backups, the database does not require being in archive log mode and thus there will be a slight performance gain as the database is not cutting archive logs to disk.

Difference between Restore and Recovery:-

Restoring involves copying backup files from secondary storage (backup media) to disk. This can be done to replace damaged files or to copy/move a database to a new location. Recovery is the process of applying redo logs to the database to roll it forward.

Media recovery applies archived redo logs and online redo logs to recover the datafiles. Whenever a change is made to a data file, the change is first recorded in the online redo logs. Media recovery selectively applies the changes recorded in the online and archived redo logs to the restored datafile to roll it forward.

Unlike media recovery, Oracle performs crash recovery and instance recovery automatically after an instance failure. Crash and instance recovery recover a database to its transaction-consistent state just before instance failure. By definition, crash recovery is the recovery of a database in a single-instance configuration or an Oracle Real Application Clusters configuration in which all instances have crashed.

Basic Recovery steps (Rolling forward and Rolling back):-

 

For managing production database is it mandatory to put the database in archive log mode.

How to take “App Aware” Oracle DB backup files using Nimesa?

Nimesa takes the snapshot of EBS volumes. Below steps describes how to take application consistent snapshot of Oracle databases using Nimesa pre and post scripts.

The first step is to create an Oracle App aware policy through “Application Policy” screen.

  • Update below fields with appropriate data based on your solution deployment needs
    • Retention Copies – Number of retention copies required
    • Enable the pre & post DB script
    • Check Abort Backup id Pre/Post Script Fails
    • When enabling Pre/Post Scripts,  attach the respective DB scripts present in your local file system.

Upload the pre and post scripts.

“pre-db.sh” file Usage

Description:- Script is designed for DB to make it run on a hot backup mode. It is a pre-backup shell script which connects to the database, Quiesce and puts the database into “begin backup” mode.

Pre-req:-

    • Ensure your database is running in archivelog mode
    • Connect to DB and run the query  ARCHIVE LOG LIST; to check if DB is in archive log mode.
    • Run Alter database archivelog; command in the SQL prompt to enable the DB into archive log mode.
    • Better to have dbf file and log file destinations configured in a different location

“pre-db.sh” script modification:-

  • Modify Oracle User, Oracle Instance ID and Oracle Home in the script (marked as bold)
  • Run the script as “root” user after logging in to Oracle DB host server

###Connecting to the bash shell prompt

#!/bin/bash

###Connecting to oracle user, Oracle instance ID from root user

orauser=”oracle”

orasid=”dummy”

###Setting the Oracle home path, getting it as input from the user

orahome=”/u02/app/oracle/product/11.2.0/db_1″

###Swtich User to Oracle and set ORACLE_SID and ORACLE_HOME based on orasid and orahome parameters

su – ${orauser}   << BOF 2>&1

export ORACLE_SID=$orasid

export ORACLE_HOME=$orahome

###Connect to SQLPLUS as sysdba and execute oracle commands needed for backup the DB

sqlplus “/ as sysdba” << EOF 2>&1

alter system archive log current;

alter database begin backup;

exit;

EOF

BOF

“post-db.sh” file Usage

Description:- Script is designed for DB to end its hot backup mode. It is a post-backup shell script which connects to the database and puts the database into “end backup” mode.

“post-db.sh” script modification:-

  • Modify Oracle User, Oracle Instance ID and Oracle Home in the script (marked as bold)
  • Run the script as “root” user after login to Oracle DB host server

#!/bin/bash

orauser=”oracle”

orasid=”dummy”

orahome=”/u02/app/oracle/product/11.2.0/db_1″

#source /home/$orauser/$orasid.env

source /home/$orauser/.bash_profile

su – ${orauser}   << BOF 2>&1

export ORACLE_SID=$orasid

export ORACLE_HOME=$orahome

###Connecting to Oracle as sysdba and execute Oracle commands needs to end the DB backup

sqlplus “/ as sysdba” << EOF 2>&1

alter system archive log current;

alter database end backup;

alter system archive log current;

alter system switch logfile;

alter system switch logfile;

exit;

EOF

BOF

Create a backup group in Nimesa add the EC2 instance and assign the newly created policy. Trigger a scheduled or on demand backup as part of job progress the script excecution output will be seen.

Pre-Script Logs:

Post-Script Logs:

About Nimesa

Nimesa is an enterprise-class Application Aware data protection, cost management & copy data management Solution for applications running on AWS. It uses native AWS capabilities like EBS snapshots capabilities to automatically protect the environment. Nimesa provides simple policy based lifecycle management of snapshots and clones of EC2 instances

 

Protecting the SQL Database deployed on EC2 instance 5 months ago

SQL server is one of the key components in the data center. Admins take different backup strategy based on the criticality of the database and the rate of change data. Below are the different types of backups of SQL supports.

Full Backup:

A full backup contains all the data including all the transaction at the time of backup. A full backup helps to restore the database to the time of backup. This kind of backup is time-consuming as the complete database is getting backed up.

Differential Backup:

To reduce the time taken for backup admin can take periodic differential backup. These are changes in data since the last backup. These backups cannot be independently restored first the full backup needs to be restored and subsequently each differential backups needs to be restored.

Transaction log Backup:

Transaction log backups are taken when databases are in the Full or Bulk-Logged recovery model. Frequent transaction log backup helps in restoring the database to point in time or to a transaction. Truncation of logs while taking log backup removes the old transaction from the active log file.

Protecting SQL databases with Nimesa

Nimesa by integrating with EBS snapshots and VSS framework provides instant protection of database irrespective of the size of the database. To protect EC2 instances with SQL databases following things needs to be done

  • Application Policy needs to be created by opting for VSS aware processing and opting for SQL aware processing. In addition to this policy can also specify if log backup needs to be taken and at what frequency

  • Download and Install Nimesa Windows agent
  • Assign the Windows credentials to the Backup group. A user should be part of the local administrator group in the windows host and also should have sysadmin role in SQL database.

As part of backup Nimesa coordinated with the windows agent to take application aware VSS snapshot. Nimesa uses VSS system provider to take differential backups it coordinates with all the VSS writers in the host along with the volume to take application-consistent backup. Once the backup is complete it discovers all the SQL databases and SQL instances those are part of the EC2 instances. Since backups are snapshot based more frequent full backups can be taken and reduce the RTO and RPO to minutes. Post backup user can browse the backup to see the databases part of the backup, its recovery model and tables part of the database.

Operations Supported on the SQL Database backups:

Nimesa supports following operations on the SQL databases

  • Cloning the database to same EC2 instance – This option can be used as side by side restore of the database
  • Cloning the database to different EC2 instance – This option is used in use cases like Test and Dev, Analytics where periodic production data needs to be brought

  • Restore of Database – Database can be restored to a point in time with the help of full and transaction log backup

  • Table Restores – There are scenarios where a particular table needs to be restored and nimesa helps in restoring a particular set of tables

About Nimesa

Nimesa is an enterprise-class Application Aware data protection, cost management & copy data management Solution for applications running on AWS. It uses native AWS capabilities like EBS snapshots capabilities to automatically protect the environment. Nimesa provides simple policy based lifecycle management of snapshots and clones of EC2 instances

 

Identifying and managing underutilized (Idle Instance) EC2 instances 5 months ago

Every minute is a cost when using cloud services. Users create EC2 instances for test and development but they are never deleted. To identify Idle instance organizations needs tools that can help them identify and also have insight at the usage pattern of the resources. This information will provide them a better view on when to power on and off these resources. Nimesa helps you in identifying such resources and helps in the cost saving.

In this blog, I will walk you over how to configure and setup thresholds to detect the idle instances.

Configure Threshold

In Nimesa UI select the Nimesa Settings and select “AWS Settings”

Here the user can configure the following

  • CPU Utilization threshold -The value below the threshold value will be considered as not utilized or underutilized. All the calculations are considered for instance when it is power-on. Power off period will not be included.
  • A user can configure from 1 day to max of 30 days for computation period.
  • A granularity of the time window to gather CPU metrics – 15 min, 30 min etc…
  • Definition of Low and High utilization in % for the time span.

Inventory

In the inventory against each EC2 instance, Nimesa provides the utilization info based on the settings.  If there is no recommendation it means the EC2 instance was never powered on during the configured windows. This is a candidate for termination if the machine is not required. Admin can also take a backup using Nimesa to preserve the data before termination.

Admin can select “LOW/HIGH” to look at the data as a chart.

This provides them with a usage pattern and helps them in lowering the cost. This data helps them in planning the power on/off schedules. For more details on configuring power on/off schedule visit manage-aws-cost-using-nimesa.

About Nimesa

Nimesa is an enterprise-class Application Aware data protection, cost management & copy data management Solution for applications running on AWS. It uses native AWS capabilities like EBS snapshots capabilities to automatically protect the environment. Nimesa provides simple policy based lifecycle management of snapshots and clones of EC2 instances.

Manage AWS Cost Using Nimesa 12 months ago

To manage cost the first approach organization take is to power on/off the resources in non-business hours. It is not difficult to write a simple schedule to power on/off the resources at a specific time. There are many things that a simple script/scheduler cannot do. Business will need the following in addition to just Power on and off feature of the scripts.

  • Grouping – Not all resource needs to be powered on and off at the same time. There should be grouping based on business needs
  • Policy Based – Easy to suspend schedules when the team needs the resource. Resume the schedule once the job is over.
  • On Demand operations- Initiate an on/off operations outside the schedule
  • Reporting – Generate reports that give insight into the cost saving of the policy
  • Dashboard – Provides overall cost savings with the policies
  • Ease of use – Provides a simple UI that is usable by non-developers
  • Consistency – Every business unit takes their own approach and difficult to create a uniform approach across the company

In this blog, I will go over how Nimesa helps in solving the above challenges.

Creating Schedule

To create a power management schedule, select “SLA Policy” -> “Power Management” option. Nimesa offers multiple options like creating a schedule just to “Power ON” or “Power OFF” or both “Power ON and OFF”. Admin can group the resources based on Tags or he can pick the instances individually.

Once the policy is saved he can see the approximate savings in $ based on the resources he has selected.

  Managing the Schedules

Admin can pause the schedule when they don’t want the schedules to be applied.


They can also trigger on-demand operation to turn on or off the resources in the policy using “Run Now” option.

As you can see Nimesa is very simple to use and provides centralized policy based power management. You don’t need to maintain scripts to keep up with the changes in your environment and AWS public APIs.

About Nimesa

Nimesa is an enterprise-class Application Aware data protection, cost management & copy data management Solution for applications running on AWS. It uses native AWS capabilities like EBS snapshots capabilities to automatically protect the environment. Nimesa provides simple policy based lifecycle management of snapshots and clones of EC2 instances

Automate Backup of EC2 instances using AWS snapshots 12 months ago

AWS provides the capability to take the snapshot of EBS volumes, but there is some manual effort involved in the process which can result in human error. First approach companies take is to automate using scripts. With ever-growing resources in the infrastructure, it would be difficult to keep the scripts up to date to support the company needs. There is a need for robust automation that can be configured once and forgot.

It is very critical for the business to protect their data on regular basis, and EBS snapshots are an important part of the strategy. If you are running scripts, it would be very difficult to make sure the scripts are running consistently. Someone needs to verify/update the scripts, when there are changes to EC2 instance, like removal/addition of EBS volumes.

What if the script failed and it was detected only now? It is important to make sure everything is running as per the SLAs of the business requirement.

Nimesa offers flexible snapshot policies where say, if the business wanted to protect their business-critical instances on an hourly basis and would like to keep 1 week worth of snapshots, whereas for test and dev instances take a daily snapshot and retain for 2 weeks. Email notification is sent for all successful/failed operations with relevant details. Nimesa helps in bringing back the instances from the snapshot instantly when the EC2 instance is not in the desired state.

In this article, we will see how Nimesa can be used for automating snapshots for EC2 instances and related EBS volumes on a regular basis

First Step is to create application policy with required retention. In Nimesa go to “SLA Policy” menu and select “Application policy”.

Create a new “Schedule policy” from “SLA Policy” menu. In this example, we have created a fourth hourly schedule.

Create a new backup group by selecting the required instance and associate Application Policy and Schedule policy created in the previous step.

Our dashboard gives a comprehensive view of the number of instances in the account, how many of them are protected and are they meeting the business SLAs of snapshot schedules like an hourly snapshot. In case any of the backup schedules fails automatically it is flagged in the dashboard.

About Nimesa

Nimesa is an enterprise-class Application Aware data protection & copy data management Solution for applications running on AWS. It uses native AWS capabilities like EBS snapshots capabilities to automatically protect the environment. Nimesa provides simple policy based lifecycle management of snapshots and clones of EC2 instances

Contact Nimesa @ sales@nimesa.io

Restore AWS EC2 instance using Nimesa 12 months ago

In the previous blog, we saw how to use Nimesa to protect the data at a periodic interval that meets organizations SLA needs. Protecting the environment is just one part of the story, a company needs a different level of restore options for different scenarios. Manual restore of data is not impossible in AWS but it’s very time-consuming. IT admin needs to find the associated volumes and find the snapshot, create volume from the snapshot and recover the data. Once it’s done he also needs to delete the unused volumes so it does not add to the cost.  The whole process can be automated but IT admin needs to write a script and also maintain the script. Restore also involves different scenarios which you will be seeing below.

Here are some of the scenario encountered by the organization when they have to restore

  • One straightforward scenario is the whole instance got deleted or entered into an undesired state user needs to restore the whole instance.
  • One of the EBS volumes got corrupted. In this case, an admin would like to quickly replace the volume with a volume from a stable snapshot
  • Few files got corrupted (Like database files, which are big in size) where he would like to recover back those files. In this  admin would like to have the volume from the consistent snapshot presented to the EC2 instance where he can use the native tools to restore the files/applications

Let us see how Nimesa can help in restoring the above scenarios and get the restore completed in Minutes.

In Nimesa, select the backup group and select the appropriate backup select “Restore” option

Full Instance Restore:

Select Full restore option and proceed with the restoration. This will restore the volumes from the snapshot and delete the original detached volumes. Additionally, you can enable postscripts and execute scripts in the instance once the volumes are restored and attached with the instance

 

Single Volume Restore:

For cases where only single-volume needs to be replaced by the volume from the snapshot select volume restore. Nimesa catalogs all the EBS volumes that are associated with an EC2 instance as part of every backup. We can pick the volumes from the list that needs to be restored.

Side by Side Volume Restore:

In case, where the user wanted to manually restore the files within the EC2 instance they can opt for Volume Mount capability where a new volume is cloned from the Snapshot and attached to the instance. This option will add cost post manual restore the cloned volume needs to be deleted.

About Nimesa

Nimesa is an enterprise-class Application Aware data protection & copy data management Solution for applications running on AWS. It uses native AWS capabilities like EBS snapshots capabilities to automatically protect the environment. Nimesa provides simple policy based lifecycle management of snapshots and clones of EC2 instances

Change EC2 instance type with minimal downtime 1 year ago

There are many reasons where you would like to change the instance type.  Some of the scenarios are like a load to the system is high and you would like to increase size for performance and there are situations like your machine is underutilized where you would like to reduce the size to save cost. In this blog, we will see how Nimesa can help you in changing the instance type with minimal downtime.

AWS allows modifying the EC2 instance type from CLI as well as AWS console. Follow the following steps to manually change instance type.

  • In AWS console identify the EC2 instance for which instance type needs to be changed
  • Power Off the EC2 if it is Powered On
  • Select “Instance Settings” -> “Change Instance Type” from Action menu.
  • Select the required type and Apply
  • Power On the instance

Nimesa supports protection of AWS Instances backed by EBS volumes. It is advisable to take a snapshot of the EBS volumes before changing the instance type. In case if there are issues with the instance coming up, snapshots can be used to restore the instance back. Nimesa’s clone feature can be leveraged here to change the instance type verify the new instance and then discard the original instance.

Use Nimesa to backup the instance.

Select the instance and the backup and select clone option. select the instance type in clone instance properties and proceed with the clone. This will create a new instance from the snapshot with the specified instance type. Now you can verify the instance, once everything is working original instance can be discarded.

 

This downtime can be eliminated if the machines are part of ELB. Power off the instance which needs to changed disassociate the elastic IP. Clone the instance from the Snapshot using Nimesa to the new cloned instance associate the elastic IP of the original machine, add the instance to load balancing group.

About Nimesa

Nimesa is an enterprise-class Application Aware data protection & copy data management Solution for applications running on AWS. It uses native AWS capabilities like EBS snapshots capabilities to automatically protect the environment. Nimesa provides simple policy based lifecycle management of snapshots and clones of EC2 instances

Automating DR tasks using Nimesa 1 year ago

Disaster Recovery (DR) is a process that helps you prepare for any kind of unwanted disaster. Cloud platforms like AWS are well suited to meet your DR needs. The organization needs a DR plan that could be periodically tested and should scale with the growth of the data. In this article, we will discuss how to use Nimesa to achieve DR in AWS.

Traditionally organizations were creating Datacenters around the world to keep a backup of the critical data and servers. Amazon has datacenter around the globe with 18 regions and 55 Availablity zones. Just having regular backups will not be sufficient to protect you against the disaster. For setting up a DR plan first need to arrive at the RTO (Recovery Time Objective) and RPO (Recovery Point Objective).

RTO:  It is the time required to bring the environment up and running or in other words how long your company can be offline without incurring a significant financial loss.

RPO: It is the amount of data loss a company can incur with respect to time. This determines the frequency of the backup. In AWS as all services are pay as you use more aggressive backups will add cost to your bill

When we talk about DR, there are various options available for achieving different levels of DR on AWS including Pilot Light, Hot Standby, and Multi-site setup.

Backup and Restore:  It is a simple, straightforward, cost-effective method that backs up and restores data as needed. There is no standby copy, time to bring back the environment will be time-consuming.

Pilot Light:  This method replicates required data based on the RPO and keep minimal infrastructure up and running and bring the environment when required.

Warm Standby: In this method minimal fully functional standby is kept operational in case of DR complete environment can be brought up with seamlessly with minimum downtime.

Multi-Site Solution: Also known as a Hot Standby, this method complete environment is replicated along with data, application and kept running in two or more regions. In the case of DR, everything rerouted automatically with almost zero downtime. But running the environment in multiple regions will incur the cost.

If you need a Higher SLA, automate using Nimesa DR policy and make EBS snapshots available in multiple regions.

Automating DR tasks using Nimesa

DR Policy Creation:

Create a protection policy specifying the primary retention, regions where snapshots need to be copied, and secondary retention. As part of this policy, I have selected 2 regions.

Backup Group Creation:

Identify the critical resources, create a backup group specifying the policy and the schedule. Backup jobs get scheduled at the specified time and after a successful backup, a backup copy job is triggered to copy the snapshots to the specified region.


Job progress shows snapshots that are successfully copied to different regions

Restore:

Select the backup and execute a clone operation. Specify the region on which to be cloned and cloned instance properties. Nimesa will find the suitable image launch the instance and replace the root and the data volumes from the snapshot.

Select the region on which the instance to be restored. Nimesa will automatically pick the compatible AMIs and list. A company can keep a copy of the AMIs in the DR region to have an accurate match.

Additionally, settings like type, VPC, and Elastic IP can be associated with the cloned instance. If this is a test drill expiry can be set and the instance will be automatically deleted after the stipulated time.

Restore Job completed successfully.

 

About Nimesa

Nimesa is an enterprise-class Application Aware data protection & copy data management Solution for applications running on AWS. It uses native AWS capabilities like EBS snapshots capabilities to automatically protect the environment. Nimesa provides simple policy based lifecycle management of snapshots and clones of EC2 instances

 

Recover your datacenter instantly from Ransomware attack (ofcourse without paying) 1 year ago

Recently, An Indiana hospital paid a ransom of $55,000 to get rid of ransomware that had infected its systems and was hindering operations. when attackers breached the network of Hancock Health, a regional hospital in the city of Greenfield, Indiana.

Hancock Health CEO Steve Long said the hackers are believed to be in eastern Europe and may have used a type of ransomware called SamSam,

SamSam encrypted files and renamed them with the phrase “I’m sorry”, according to a local newspaper who broke the news.

The hospital said that despite having backups it opted to pay the ransom demand of 4 Bitcoin, which was worth around $55,000 at the time the hospital paid the sum. Hospital management told local press that restoring from backups was not a solution as it would have taken days and maybe even weeks to have all systems up and running. Hence, they decided to pay the ransom was quicker.

What is Ransomware

Ransomware is a type of malicious software from cryptovirology that threatens to publish the victim’s data or perpetually block access to it unless a ransom is paid. Ransomware is different from other attacks that it’s all about storage.

The hacker’s application locks up your storage by encrypting it and more advanced malware uses a technique called cryptoviral extortion, in which it encrypts the victim’s files, making them inaccessible, and demands a ransom payment to decrypt them and you either unravel the rogue code’s encryption or start learning about blockchain technology and shipping bitcoins into the ether.

Today, ransomware is one of the biggest cyber threats, according to McAfee Labs and Trend Micro. Rapid digitization of consumers’ lives and enterprise records will increase the cost of data breaches drastically. To avoid becoming a victim, users take action to protect their Computers. But waiting ( RTO ) could cost you money, hassle, and negative publicity.

Backup admins should have periodic backups and good data protection with great RPOs and lesser RTOs to survive these attack

Protecting your data from Ransomware

In case of cloud environments ( VMware / AWS ) this is much easier to recover but many times a simple traditional backup app doesn’t provide sufficient defence against ransomware because those applications will backup only the application data of your machine.

Another big concern about ransomware is traditional backup stores their backup on the media server, as they would be susceptible to the same attacks that go after the systems they’re backing up.

If you have a good copy of backup ( recovery point with the pre-ransomware copy of the data) you can recover your data but you still need everything related to operating system settings, application, application settings and the configurations to get up and run again in the production.

Is this sounds like it will take a lot of time to bring back production machine?

Introducing Nimesa

No, With Nimesa, A Best data protection technique which can help business by cloud-based disaster recovery, backup up data and replicate to another site. In case, ransomware blocks you on-prem data you can recover from another site.

 

As Nimesa backups based on the snapshot (snapshots are read-only ). Nimesa helps you recover the data from the good backup copy in minutes.

Nimesa backups the Virtual Machine by taking the snapshot of the Virtual Machine and the volume of datastore ( underlying storage snapshot capabilities) in case of VMware, it creates the snapshot of the Instance volumes in case AWS cloud. In both these cases, Nimesa replicated the snapshot to another site configured by user underlying storage / AWS EBS capabilities.

In case of an attack or data loss user has the option to recover the whole machine or individual disk or even the SQL database inside the Virtual Machine. Nimesa provides data protection with smaller RPO and faster RTO.

 





Your applications run in: