1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Amazon EFS...NFS storage on AWS

Discussion in 'Server Configuration and Hosting' started by thedude, Apr 9, 2015.

  1. thedude

    thedude Well-Known Member


    For those running XenForo on EC2 instances, this will let multiples instances access the same storage simultaneously (unlike EBS, which could only be attached to one instance at a time).
    eva2000 and Marcus like this.
  2. Marcus

    Marcus Well-Known Member

    It's currently possible with S3 :)
  3. thedude

    thedude Well-Known Member

    s3fs-fuse isn't very stable, from what I've heard anecdotally. Is that what you're using?
  4. Marcus

    Marcus Well-Known Member

    No i directly use s3. But I had to write a custom addon for it.

    EFS will make our life much easier :)
  5. Moscato

    Moscato Active Member

    EFS is very expensive... how much data do you actually need to store?

    Linode overall seems to be a better deal than AWS
  6. Jim Boy

    Jim Boy Well-Known Member

    Finally got around to putting EFS into proper use today. Like any network attached storage it is slow and you would be crazy to use it for library files. Likewise attachment data is best served from S3 which is 10% of the cost of EFS.

    Where it has solved a problem for though is using it for image caching, which resides within internal_data. Have got my user-data scripts mapping this drive at launch. I imagine it will hover around 1GB of data used, which is only 30 cents a month.
    thedude likes this.
  7. Moscato

    Moscato Active Member

    I wish you luck with keeping your data costs down. My site has been repeatedly ddos'd, and if I didn't use a cheaper host, like linode, or a CDN like cloudflare, it would be immensely expensive.
  8. thedude

    thedude Well-Known Member

    What do you normally use for keeping your internal_data and data dirs sync'd between multiple ec2 instances?
  9. Jim Boy

    Jim Boy Well-Known Member

    I've found that internal_data hasn't had to be shared between servers, until image caching, so I let each server handle internal_data itself. Have never had a problem with it and I'll run up to ten servers concurrently.

    For external data, it's a little more complex. I've registered S3 as a stream within config.php including credentials, then set the two external directives to point at S3 or cloudfront location for the end user. This handles things like avatars just fine. For attachments, I use "[bd] attachment store" which has been good. Could be better though if it would set additional metadata on upload, such as cache control headers. Having said that, its been super reliable and I recommend it.
    thedude likes this.
  10. Moscato

    Moscato Active Member

    May I ask why use amazon at all? They're overpriced

    Linode is a much better deal
  11. Minecraftly

    Minecraftly Member

    There are many reasons why one would choose AWS instead of VPSes/Dedicated hosting.

    Linode should be compared with Digital Ocean. AWS is different, if it's to be compared it should be with Azure or Google Cloud Platform.
    Marcus likes this.
  12. Moscato

    Moscato Active Member

    AWS sells VPS hourly which can be used via script , as well as a dynamic database system, hourly. They also sell NFS storage.

    Linode sells VPS hourly, which can be used via script. They also pool your VPS storage, so you can place most of your storage on a single VPS, which is similar to buying NFS space. They don't have a dynamic database system, but you can place linked databases on VPS via script easily enough.

    The area where Amazon wins is where you have a large amount of data you rarely access.

    Google Cloud Platform and Azure have a difference where you can actually rent workloads. Amazon doesn't do that. Amazon is closer to linode in that sense than it is to Azure or Google Cloud.

    Linode is substantially cheaper, and gives you large pools of free outgoing bandwidth, also.

    The only case where amazon wins, is where you have a unusually large pool of rarely accessed data, and a small computational and ram requirement.

    Linode also gives you some unusual benefits, such as placing you on boxes with 40gbit incoming connections, incase you get hit with something nasty that you can rate limit and drop. On Amazon, a 10gbit ddos will result in a failed machine. On linode, a 10gbit ddos will result in a CPU spike.
  13. RoldanLT

    RoldanLT Well-Known Member

    And your account will be suspended.
    WSWD likes this.
  14. eva2000

    eva2000 Well-Known Member

    Just remember the fixed known costs factor too - Linode you'd have known fixed costs while Amazon EC2 would have highly variable higher costs i.e. bandwidth at US$20/TB to another AWS region and/or US$90/TB to non-AWS regions while Linode for bandwidth you have option to upgrade VPS or spin up another VPS to increase your pooled bandwidth costs i.e. 4GB Linode has 4TB bandwidth, spin up a 2nd 4GB Linode and you have 8TB pooled at 2x US$40 = US$80/month

    That same 8TB on Amazon EC2 = 7999GB x .09/GB = US$719.91 just for bandwidth

    Amazon AWS has more flexibility to scale on demand. However, you better make sure your bank account is as flexible and can scale with AWS :D
    MattW, Xon and RoldanLT like this.
  15. Jeremy P

    Jeremy P Well-Known Member

    In my experience they just nullroute your node IP and open a ticket to let you know, and remove the nullroute when the attack is over. I've never been threatened with a suspension. In fact, they've been pretty helpful and informative regarding attacks.
  16. n00bsaibot

    n00bsaibot Well-Known Member

    If you're content goes viral a lot, having AWS on-demand scalability will be more profitable than the higher associated costs.

    If you have the same steady flow of traffic, a fixed cost provider will probably make more sense.
  17. RoldanLT

    RoldanLT Well-Known Member

    Probably just a very small attack.
    One company that I work for just suspended this week because we receive a very large attack.


    And we are down for 3 days because they are not giving us access on our data :(.
  18. WSWD

    WSWD Well-Known Member

    Which is essentially like a suspension... Your account is unusable until the attack stops.
    Tracy Perry likes this.
  19. Jeremy P

    Jeremy P Well-Known Member

    @RoldanLT Was this an out of the blue attack, or had there been a history of them? The attacks I've had weren't massive, but they still used the same verbiage about it being large enough to negatively impact the platform.


    I could imagine with larger or more frequent attacks that they may require mitigation. Still, sorry to hear that happened. Are you guys migrating to a different service, pending the release of your data?

    My account is usable, the node being attacked is not. I know the line between the two is thin, but there's still a slight distinction. I can still login to the manager and even spin up new nodes, and I'm not being asked to leave the platform or take any action myself.
  20. RoldanLT

    RoldanLT Well-Known Member

    1st attack was happened October 25, 2015.
    2nd and last attack happened November 1, 2015.. and they suspended us completely after that.

    We migrated to OVH :).
    But sadly we have to use a one week old backup data since they don't allow us to grab our latest data, until today at last they granted us after they know we're not using them anymore.

    We can't do this anymore after the second attack.

    Jeremy P likes this.

Share This Page