• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

Amazon EFS...NFS storage on AWS

Jim Boy

Well-known member
#6
Finally got around to putting EFS into proper use today. Like any network attached storage it is slow and you would be crazy to use it for library files. Likewise attachment data is best served from S3 which is 10% of the cost of EFS.

Where it has solved a problem for though is using it for image caching, which resides within internal_data. Have got my user-data scripts mapping this drive at launch. I imagine it will hover around 1GB of data used, which is only 30 cents a month.
 

Moscato

Active member
#7
Finally got around to putting EFS into proper use today. Like any network attached storage it is slow and you would be crazy to use it for library files. Likewise attachment data is best served from S3 which is 10% of the cost of EFS.

Where it has solved a problem for though is using it for image caching, which resides within internal_data. Have got my user-data scripts mapping this drive at launch. I imagine it will hover around 1GB of data used, which is only 30 cents a month.
I wish you luck with keeping your data costs down. My site has been repeatedly ddos'd, and if I didn't use a cheaper host, like linode, or a CDN like cloudflare, it would be immensely expensive.
 

thedude

Well-known member
#8
Finally got around to putting EFS into proper use today. Like any network attached storage it is slow and you would be crazy to use it for library files. Likewise attachment data is best served from S3 which is 10% of the cost of EFS.

Where it has solved a problem for though is using it for image caching, which resides within internal_data. Have got my user-data scripts mapping this drive at launch. I imagine it will hover around 1GB of data used, which is only 30 cents a month.
What do you normally use for keeping your internal_data and data dirs sync'd between multiple ec2 instances?
 

Jim Boy

Well-known member
#9
What do you normally use for keeping your internal_data and data dirs sync'd between multiple ec2 instances?
I've found that internal_data hasn't had to be shared between servers, until image caching, so I let each server handle internal_data itself. Have never had a problem with it and I'll run up to ten servers concurrently.

For external data, it's a little more complex. I've registered S3 as a stream within config.php including credentials, then set the two external directives to point at S3 or cloudfront location for the end user. This handles things like avatars just fine. For attachments, I use "[bd] attachment store" which has been good. Could be better though if it would set additional metadata on upload, such as cache control headers. Having said that, its been super reliable and I recommend it.
 

Moscato

Active member
#12
There are many reasons why one would choose AWS instead of VPSes/Dedicated hosting.

Linode should be compared with Digital Ocean. AWS is different, if it's to be compared it should be with Azure or Google Cloud Platform.
AWS sells VPS hourly which can be used via script , as well as a dynamic database system, hourly. They also sell NFS storage.

Linode sells VPS hourly, which can be used via script. They also pool your VPS storage, so you can place most of your storage on a single VPS, which is similar to buying NFS space. They don't have a dynamic database system, but you can place linked databases on VPS via script easily enough.

The area where Amazon wins is where you have a large amount of data you rarely access.

Google Cloud Platform and Azure have a difference where you can actually rent workloads. Amazon doesn't do that. Amazon is closer to linode in that sense than it is to Azure or Google Cloud.

Linode is substantially cheaper, and gives you large pools of free outgoing bandwidth, also.

The only case where amazon wins, is where you have a unusually large pool of rarely accessed data, and a small computational and ram requirement.

Linode also gives you some unusual benefits, such as placing you on boxes with 40gbit incoming connections, incase you get hit with something nasty that you can rate limit and drop. On Amazon, a 10gbit ddos will result in a failed machine. On linode, a 10gbit ddos will result in a CPU spike.
 

eva2000

Well-known member
#14
Just remember the fixed known costs factor too - Linode you'd have known fixed costs while Amazon EC2 would have highly variable higher costs i.e. bandwidth at US$20/TB to another AWS region and/or US$90/TB to non-AWS regions while Linode for bandwidth you have option to upgrade VPS or spin up another VPS to increase your pooled bandwidth costs i.e. 4GB Linode has 4TB bandwidth, spin up a 2nd 4GB Linode and you have 8TB pooled at 2x US$40 = US$80/month

That same 8TB on Amazon EC2 = 7999GB x .09/GB = US$719.91 just for bandwidth

Amazon AWS has more flexibility to scale on demand. However, you better make sure your bank account is as flexible and can scale with AWS :D
 

Jeremy P

Well-known member
#15
And your account will be suspended.
In my experience they just nullroute your node IP and open a ticket to let you know, and remove the nullroute when the attack is over. I've never been threatened with a suspension. In fact, they've been pretty helpful and informative regarding attacks.
 

n00bsaibot

Well-known member
#16
If you're content goes viral a lot, having AWS on-demand scalability will be more profitable than the higher associated costs.

If you have the same steady flow of traffic, a fixed cost provider will probably make more sense.
 

RoldanLT

Well-known member
#17
In my experience they just nullroute your node IP and open a ticket to let you know, and remove the nullroute when the attack is over. I've never been threatened with a suspension. In fact, they've been pretty helpful and informative regarding attacks.
Probably just a very small attack.
One company that I work for just suspended this week because we receive a very large attack.




And we are down for 3 days because they are not giving us access on our data :(.
 

Jeremy P

Well-known member
#19
@RoldanLT Was this an out of the blue attack, or had there been a history of them? The attacks I've had weren't massive, but they still used the same verbiage about it being large enough to negatively impact the platform.



I could imagine with larger or more frequent attacks that they may require mitigation. Still, sorry to hear that happened. Are you guys migrating to a different service, pending the release of your data?

Which is essentially like a suspension... Your account is unusable until the attack stops.
My account is usable, the node being attacked is not. I know the line between the two is thin, but there's still a slight distinction. I can still login to the manager and even spin up new nodes, and I'm not being asked to leave the platform or take any action myself.
 

RoldanLT

Well-known member
#20
Was this an out of the blue attack, or had there been a history of them?
1st attack was happened October 25, 2015.
2nd and last attack happened November 1, 2015.. and they suspended us completely after that.

Are you guys migrating to a different service, pending the release of your data?
We migrated to OVH :).
But sadly we have to use a one week old backup data since they don't allow us to grab our latest data, until today at last they granted us after they know we're not using them anymore.

I can still login to the manager and even spin up new nodes, and I'm not being asked to leave the platform or take any action myself.
We can't do this anymore after the second attack.