Storage - Migrate, Measure, Manage

Lesson 4 · Storage

Migrate, Measure, Manage

Once data lands in the cloud, three questions follow it everywhere: how did it get here, how fast can it move, and what does keeping it around actually cost.

4.4

Migrate Data to the Cloud

Cloud service providers offer various solutions for moving massive amounts of data from on-premises to cloud storage. One method relies on file transfers across the Internet using dedicated or temporary direct connections. The other transports data physically using devices delivered to the provider's data center.

Direct connection

Physical device

Each CSP has many migration tools. The above lists are not comprehensive.

4.5

Performance Implications

Various performance measurements exist for storage media — IOPS, throughput, latency, and queue depth. All these factors impact storage performance and should be assessed as a single system. Select a metric.

Comparing two Amazon EBS volume types — Throughput-Optimized HDD (st1) and Cold HDD (sc1) — st1 is built for active, data-intensive workloads; sc1 trades performance for the lowest storage cost:

Attribute	st1 · Throughput-Optimized	sc1 · Cold HDD
Use cases	Big data, data warehouses, log processing	Infrequently accessed data, lowest-cost storage
Volume size	125 GiB – 16 TiB	125 GiB – 16 TiB
Max IOPS / volume	500	250
Max throughput / volume	500 MiB/s	250 MiB/s

Both types share the same size range, but st1 runs roughly twice as fast on both measures — making it the better fit for active workloads, while sc1 serves as low-cost cold storage that still stays online.

GCP Cloud Monitoring surfaces average read IOPS, read/write latency, average queue depth, peak read/write operations, and disk reads/writes in bytes — using these tools is a major part of a cloud storage administrator's role.

4.6

Storage as a Service

Storage isn't only a requirement for other cloud services — it's a standalone solution. Standalone Storage as a Service (STaaS) is a practical requirement for businesses and end-users alike, offering performance and cost benefits.

Microsoft OneDrive

Dropbox

Box

Google Drive

Benefits

Cost savings, if correctly managed

Disaster recovery

Scalability

Accessibility from multiple device platforms, any location

Easy sharing and collaboration

Drawbacks

Security & privacy of data at rest

Security & privacy of data in transit

Storage capacity could become expensive

Data control and retention

High availability / downtime

Vendor lock-in

4.7

Software-Defined Storage

Software-defined storage (SDS) separates the management software from the storage hardware, sitting between the data request and the hardware to abstract it — combining SAN, NAS, and traditional storage arrays under one policy-based layer.

Compression

Represents data in fewer bits. Best for content accessed infrequently, since decompression costs CPU time.

Deduplication

Removes duplicate file chunks, replacing them with pointers. VM images commonly see 50–60% savings.

Provisioning

Thin bills for what's used; thick reserves and bills the full defined capacity up front.

Replication

Duplicates data across locations — storage, VM, CDN, distributed file system, or availability zone.

Scenario: a cloud administrator deploys a VM with 100 GB of defined storage capacity, and only 50 GB of data currently needs to be stored. Toggle the provisioning model:

Thin provisioning

Thick provisioning

0 GBDefined max: 100 GB

4.8

Cost Implications

Cloud storage offers real savings, but only if you understand what actually drives the bill: egress fees, unnecessary transfers, data lifecycle, tiering, and regional pricing differences.

Egress fees — uploading to cloud storage is typically free; pulling data out is what's charged. "Pulling" means using the data, not deleting it — employee downloads, customer access, and cross-storage moves all count.

User quotas control storage cost and capacity at the filesystem, device, or cloud level. Toggle enforcement style:

Soft quota

Hard quota

4.9

Hyperconverged Storage Resources

Hyperconverged storage combines compute resources — vCPU and memory — with storage capacity: "software-defined everything." AWS presents storage options alongside other instance configuration settings.

AWS M4 · general-purpose, balanced instances

Instance	vCPU	Mem (GiB)	Storage	Dedicated EBS Bandwidth
m4.large	2	8	EBS-only	450 Mbps
m4.xlarge	4	16	EBS-only	750 Mbps

AWS D2 · storage-optimized, dense local HDD

Instance	vCPU	Mem (GiB)	Storage
d2.xlarge	4	30.5	3 × 2 TB HDD
d2.2xlarge	8	61	6 × 2 TB HDD

GCP offers local solid-state drives for high-performance local block storage on compute instances — like AWS, it maintains many storage services optimized by anticipated workload.

§ 4.4–4.9 · Migrate, Measure, Manage