Storage snapshots for on-premises storage arrays have been around for a long time. Storage snapshots are incremental in nature i.e. they contain only the blocks that changed since the previous snapshot. So, they are fast and efficient. The snapshots of the storage volume are stored on the same storage array.
AWS EBS snapshots are also similar. They are incremental forever snapshots. But the one big difference from on-premises storage snapshots is that each snapshot of an EBS volume is stored in S3, which is an object storage tier. Thus recoveries from snapshots stored in S3 take a long time as the data has to be copied over from S3 to EBS.
EBS Fast Snapshot Restore
To solve the above restore problem, AWS announced a new feature called EBS Fast Snapshot Restore (FSR) that accelerates the data copy process from the snapshots stored in S3 to EBS. Behind the scenes, FSR engages more compute and memory to accelerate the copy process from S3 to EBS volume. AWS claims that enabling FSR can recover 1 TB per hour, which by no means is great throughput, but that is still better than what it did in the past.
Like everything else in AWS, there are charges for using FSR. FSR costs depend on the duration for which the feature is enabled at a price of $0.75 per hour. So it’s best to enable FSR just before you intend to mount from a snapshot and perform your recoveries. Thus, for example, if you mount an FSR enabled snapshot and it takes 4 hours to recover a 4 TB volume, you get charged for $0.75 per hour x 4 hours = $3.00.
FSR also allows you to create up to 10 initialized volumes per hour from each snapshot. So this enables users to create copies of production data from snapshots for Dev/QA/UAT testing. Thus if you need 10 copies of a 1 TB production volume, you can create 10 copies from an EBS snapshot using FSR in just 1 hour.
However, note that this will consume 10 TB of EBS volume. Assuming $100 per TB per month, it would cost 10 TB x $100 per TB = $1,000 per month.
Can FSR be used to Clone Databases in AWS?
Typically enterprises need copies or clones of production data while working with databases so that Dev, QA, UAT users can rapidly test their patches or latest builds against a real copy of the production database.
Let’s start with how a database backup works with EBS snapshot. When a snapshot of an EBS volume containing a database such as Oracle or SAP HANA is taken, it’s crash-consistent and not application-consistent. This means that the application doesn’t know that a snapshot was taken and hence there is a high chance of incomplete IOs written to disk. There is also the problem of maintaining write order fidelity when an EBS snapshot is taken in situations where database data is laid out on multiple disks, which typically is the case. Thus DBAs never recommend such crash-consistent recoveries.
To summarize –
- FSR is a great feature to accelerate restores from snapshots.
- Creating clones using FSR increases speed from what would have been possible without FSR, but it creates expensive EBS physical copies.
- EBS snapshots are not a good choice for databases such as Oracle, SAP HANA and other databases
An Ideal Data Management Solution in AWS
An ideal AWS data management solution would have the following key capabilities:
- For AWS EC2 instances without databases, leverage native EBS snapshots and use FSR to accelerate restores from such snapshots
- For AWS EC2 instances with databases, perform database aware, application-consistent incremental forever snapshots using change block technology
- Perform instant mount and recoveries from database backups that could be stored in EBS or S3 storage in the same or a different region.
- Reuse the backup copy in EBS or S3 to provision rapid multi-TB database clones on-demand EC2 machines for Dev/QA/UAT.
- Reuse backup copy to provision database clones to containers in AWS
To learn more details on how to reduce data management costs for mission-critical databases in AWS by more than 70%.