Because EC2's hypervisor was (when it was launched) lacking features (no hot swap, not shared block storage, no host movements, no online backup/clone, no live recovery, no Highly available host failover ) S3 had to step in to pick up some of the slack that proper block or file storage would have taken.
For better or for worse, people adopted the ephemeral style of computing, and used s3 as the state store.
S3 got away with it because it was the only practical object store in town.
The biggest drawback that is still has (and will likely always have) is that you can't write parts to a file. its either replace or nothing.
But that's by design.
So I suspect it'll stay like that.
For that purpose, I think S3's age is a killer feature. I won't be surprised when we see the HN post "google cloud storage has been sent to the graveyard"
Even if there is some technical reason why the data needs copying, S3 could at least pretend that the file is in the new place until it’s actually there.
This is true, but S3 does support replication (including deletion markers), and even 2-way replication, between two regions. Definitely not the same thing as a dual-region bucket, but it can satisfy many use cases where dual-region bucket would be used otherwise.
https://docs.aws.amazon.com/AmazonS3/latest/userguide/replic...
It's the actual federal hate crime that are the egress costs I could do without.
Being able to reuse the s3 command-line and existing S3 libraries has made the migration painless, so I'm thankful they've created a defacto standard that other S3 compatible object storage providers can implement.
Yes yes yes. However, DynamoDB can be expensive very quickly :]
Cloudflare supports CAS on copyObject and on putObject [1]. It doesn't support CAS on deleteObject.
I don't know about the others though (ABS, GCS, Tigris, MinIO).
https://developers.cloudflare.com/r2/api/s3/extensions/#cond...
[0] https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObje...
[1] https://developers.cloudflare.com/r2/api/s3/extensions/#cond...
the ideal design is to use s3 for immutable objects, then build a similar system on ec2 nvme for mutable/ephemeral data.
the one i use by default is s4[1].
(But this rarely happens with S3, because it's so simple.)
I just implemented a "posix-like" filesystem on top of it. Which means large object offload to s3 is not a problem or even an "ugly abstraction." In fact it looks quite natural once you get down to the layer that does this.
You also get something like regular file locks which extend to s3 objects, and if you're using cognito, you can simplify your permissions management and run it all through the file system as well.
Actual storage cost is "meh" but...
My god the absolute highway robbery of the bandwidth costs.
Is there a data center with many solid state disks?
Are tape drives used?
Adding features makes documentation more complicated, makes the tech harder to learn, makes libraries bigger, likely harms performance a bit, increases bug surface area, etc.
When it gets too out of hand, people will paper it over with a new, simpler abstraction layer, and the process starts again, only with a layer of garbage spaghetti underneath.
Show your age and be proud, Simple Storage Service.