Amazon EFS...NFS storage on AWS

[thedude]

http://aws.amazon.com/efs/

For those running XenForo on EC2 instances, this will let multiples instances access the same storage simultaneously (unlike EBS, which could only be attached to one instance at a time).

 

[Marcus]

It's currently possible with S3

 

[thedude]

It's currently possible with S3

s3fs-fuse isn't very stable, from what I've heard anecdotally. Is that what you're using?

 

[Marcus]

No i directly use s3. But I had to write a custom addon for it.

EFS will make our life much easier

 

[Moscato]

EFS is very expensive... how much data do you actually need to store?

Linode overall seems to be a better deal than AWS

 

[Jim Boy]

Finally got around to putting EFS into proper use today. Like any network attached storage it is slow and you would be crazy to use it for library files. Likewise attachment data is best served from S3 which is 10% of the cost of EFS.

Where it has solved a problem for though is using it for image caching, which resides within internal_data. Have got my user-data scripts mapping this drive at launch. I imagine it will hover around 1GB of data used, which is only 30 cents a month.

 

[Moscato]

Finally got around to putting EFS into proper use today. Like any network attached storage it is slow and you would be crazy to use it for library files. Likewise attachment data is best served from S3 which is 10% of the cost of EFS.

Where it has solved a problem for though is using it for image caching, which resides within internal_data. Have got my user-data scripts mapping this drive at launch. I imagine it will hover around 1GB of data used, which is only 30 cents a month.

I wish you luck with keeping your data costs down. My site has been repeatedly ddos'd, and if I didn't use a cheaper host, like linode, or a CDN like cloudflare, it would be immensely expensive.

 

[thedude]

Finally got around to putting EFS into proper use today. Like any network attached storage it is slow and you would be crazy to use it for library files. Likewise attachment data is best served from S3 which is 10% of the cost of EFS.

Where it has solved a problem for though is using it for image caching, which resides within internal_data. Have got my user-data scripts mapping this drive at launch. I imagine it will hover around 1GB of data used, which is only 30 cents a month.

What do you normally use for keeping your internal_data and data dirs sync'd between multiple ec2 instances?

 

[Jim Boy]

What do you normally use for keeping your internal_data and data dirs sync'd between multiple ec2 instances?

I've found that internal_data hasn't had to be shared between servers, until image caching, so I let each server handle internal_data itself. Have never had a problem with it and I'll run up to ten servers concurrently.

For external data, it's a little more complex. I've registered S3 as a stream within config.php including credentials, then set the two external directives to point at S3 or cloudfront location for the end user. This handles things like avatars just fine. For attachments, I use "[bd] attachment store" which has been good. Could be better though if it would set additional metadata on upload, such as cache control headers. Having said that, its been super reliable and I recommend it.

 

[Moscato]

May I ask why use amazon at all? They're overpriced

Linode is a much better deal

 

[Minecraftly]

sk why use amazon at all? They're overpriced

There are many reasons why one would choose AWS instead of VPSes/Dedicated hosting.

Linode should be compared with Digital Ocean. AWS is different, if it's to be compared it should be with Azure or Google Cloud Platform.

 

[Moscato]

There are many reasons why one would choose AWS instead of VPSes/Dedicated hosting.

Linode should be compared with Digital Ocean. AWS is different, if it's to be compared it should be with Azure or Google Cloud Platform.

AWS sells VPS hourly which can be used via script , as well as a dynamic database system, hourly. They also sell NFS storage.

Linode sells VPS hourly, which can be used via script. They also pool your VPS storage, so you can place most of your storage on a single VPS, which is similar to buying NFS space. They don't have a dynamic database system, but you can place linked databases on VPS via script easily enough.

The area where Amazon wins is where you have a large amount of data you rarely access.

Google Cloud Platform and Azure have a difference where you can actually rent workloads. Amazon doesn't do that. Amazon is closer to linode in that sense than it is to Azure or Google Cloud.

Linode is substantially cheaper, and gives you large pools of free outgoing bandwidth, also.

The only case where amazon wins, is where you have a unusually large pool of rarely accessed data, and a small computational and ram requirement.

Linode also gives you some unusual benefits, such as placing you on boxes with 40gbit incoming connections, incase you get hit with something nasty that you can rate limit and drop. On Amazon, a 10gbit ddos will result in a failed machine. On linode, a 10gbit ddos will result in a CPU spike.

 

[rdn]

On linode, a 10gbit ddos will result in a CPU spike.

And your account will be suspended.

 

[eva2000]

Just remember the fixed known costs factor too - Linode you'd have known fixed costs while Amazon EC2 would have highly variable higher costs i.e. bandwidth at US$20/TB to another AWS region and/or US$90/TB to non-AWS regions while Linode for bandwidth you have option to upgrade VPS or spin up another VPS to increase your pooled bandwidth costs i.e. 4GB Linode has 4TB bandwidth, spin up a 2nd 4GB Linode and you have 8TB pooled at 2x US$40 = US$80/month

That same 8TB on Amazon EC2 = 7999GB x .09/GB = US$719.91 just for bandwidth

Amazon AWS has more flexibility to scale on demand. However, you better make sure your bank account is as flexible and can scale with AWS

 

[Jeremy P]

And your account will be suspended.

In my experience they just nullroute your node IP and open a ticket to let you know, and remove the nullroute when the attack is over. I've never been threatened with a suspension. In fact, they've been pretty helpful and informative regarding attacks.

 

[n00bsaibot]

If you're content goes viral a lot, having AWS on-demand scalability will be more profitable than the higher associated costs.

If you have the same steady flow of traffic, a fixed cost provider will probably make more sense.

 

[rdn]

In my experience they just nullroute your node IP and open a ticket to let you know, and remove the nullroute when the attack is over. I've never been threatened with a suspension. In fact, they've been pretty helpful and informative regarding attacks.

Probably just a very small attack.
One company that I work for just suspended this week because we receive a very large attack.

And we are down for 3 days because they are not giving us access on our data .

 

[WSWD]

In my experience they just nullroute your node IP and open a ticket to let you know, and remove the nullroute when the attack is over.

Which is essentially like a suspension... Your account is unusable until the attack stops.

 

[Jeremy P]

@RoldanLT Was this an out of the blue attack, or had there been a history of them? The attacks I've had weren't massive, but they still used the same verbiage about it being large enough to negatively impact the platform.

I could imagine with larger or more frequent attacks that they may require mitigation. Still, sorry to hear that happened. Are you guys migrating to a different service, pending the release of your data?

Which is essentially like a suspension... Your account is unusable until the attack stops.

My account is usable, the node being attacked is not. I know the line between the two is thin, but there's still a slight distinction. I can still login to the manager and even spin up new nodes, and I'm not being asked to leave the platform or take any action myself.

 

[rdn]

Was this an out of the blue attack, or had there been a history of them?

1st attack was happened October 25, 2015.
2nd and last attack happened November 1, 2015.. and they suspended us completely after that.

Are you guys migrating to a different service, pending the release of your data?

We migrated to OVH .
But sadly we have to use a one week old backup data since they don't allow us to grab our latest data, until today at last they granted us after they know we're not using them anymore.

I can still login to the manager and even spin up new nodes, and I'm not being asked to leave the platform or take any action myself.

We can't do this anymore after the second attack.