Apantic Downtime Alert

[nanocode]

I do apologise for posting this here on XF.com, however we felt it would be best to inform customers about the events currently occurring at apantic.com and related Apantic services.

All services are currently down unexpectedly. A problem occurred on our nodes which completely bottlenecks and restricts PHP and nginx activity, so we are currently working on debugging this, which is a bit hard as it produces no errors, either, other than a 500 response code. This should not affect any customer sites, but it may produce some errors as they cannot connect to apantic.com to check branding related information.

We will provide further information when we have it. At worst, we plan to take a database backup, which we have, and rebuild the VPS from scratch.

Thanks for your understanding,
The Apantic Team

 

[nanocode]

Update #1: Apantic services are back online, possibly pending further maintenance.

 

[Blast]

Still can't access.

 

[Robust]

Still can't access.

Are you using Firefox? Use another browser.

 

[Roka]

From Firefox dont work, from another browsers open site

 

[Blast]

@Robust is it working on Firefox now?

 

[Robust]

Update: Apantic is not working on Firefox browsers, please see: https://www.apantic.com/community/threads/apantic-bugs-with-firefox-browsers.62/

Source of post:

Apantic's website (and related services) are not working with Firefox browsers.

We are working on debugging this.

The problem appears to be global and no Firefox user can seem to access apantic.com or related services.

If you use Firefox and are experiencing difficulties, please could you give us the following information:

• Computer OS
• Firefox version

Additionally, please provide information on when you first started experiencing this problem.

Thank you, and we hope to get this bug resolved soon.

Please provide the above information so we can get this debugged.

 

[Blast]

@Robust

Firefox 42.0
Windows 7 64-bit
Can't access since yesterday

Just to clarify, if the addon is installed in a site, will that malfunction for the member who uses Firefox?

 

[Robust]

Just to clarify, if the addon is installed in a site, will that malfunction for the member who uses Firefox?

No. apantic.com and related are the only sites affected by Firefox users. Your site's users do not communicate with apantic.com.

 

[Blast]

No. apantic.com and related are the only sites affected by Firefox users. Your site's users do not communicate with apantic.com.

Oh ok. I really hope this gets fixed ASAP. As I don't really like Chrome or any other browser.

 

[Robust]

Oh ok. I really hope this gets fixed ASAP. As I don't really like Chrome or any other browser.

Working on it. Really makes no sense to me, Linode, CloudFlare and Mozilla Community hasn't responded yet. The bug started after a Linode reboot, indicating a setting may have been changed. The nginx web server was upgraded and SPDY was replaced with HTTP2 - however, when I disable this it doesn't really "disable". Mozilla Firefox has beta support for HTTP2, nothing intense, so that could be a cause but disabling it from config still allows HTTP2 requests and doesn't do anything really.

 

[Blast]

Working on it. Really makes no sense to me, Linode, CloudFlare and Mozilla Community hasn't responded yet. The bug started after a Linode reboot, indicating a setting may have been changed. The nginx web server was upgraded and SPDY was replaced with HTTP2 - however, when I disable this it doesn't really "disable". Mozilla Firefox has beta support for HTTP2, nothing intense, so that could be a cause but disabling it from config still allows HTTP2 requests and doesn't do anything really.

I have visited sites with the same server configuration with you and there wasn't any issue. Are you using Centmin?

 

[Robust]

I have visited sites with the same server configuration with you and there wasn't any issue. Are you using Centmin?

Yeah, we're going to stop using it I think. It's making the debugging process much harder. I really only used it for... well making the job easier and not having to compile everything from scratch. Oh well.

All kinds of problems with the back-end right now. Connection refused, hitting max limits, overloading, Firefox blocking, WHMCS broken, for a few days everything broken. I'm considering an OS reinstall and going from scratch again. Centmin is great for non-experienced sysadmins but the debugging process with centmin is a damn pain in the arse.

 

[Robust]

BTW, with the help of Linode we identified that the problem began 4 days ago when centmin was updated to git repo and a few updates were done. @eva2000

 

[Blast]

I really hope you get this fixed ASAP Robust. Lots of people on Firefox. Good luck brother.

 

[Robust]

OS reinstallation to occur tomorrow @ 4PM GMT: https://www.apantic.com/community/threads/os-reinstallation-major-development-window.63/

 

[eva2000]

Working on it. Really makes no sense to me, Linode, CloudFlare and Mozilla Community hasn't responded yet. The bug started after a Linode reboot, indicating a setting may have been changed. The nginx web server was upgraded and SPDY was replaced with HTTP2 - however, when I disable this it doesn't really "disable". Mozilla Firefox has beta support for HTTP2, nothing intense, so that could be a cause but disabling it from config still allows HTTP2 requests and doesn't do anything really.

if it's ssl related check with ssllabs site test to see if you have ssl certificate/config misconfiguration

fyi nginx since 1.9.5 has removed spdy module, so if you disable HTTP/2 there is only plain ssl non-SPDY available.

Yeah, we're going to stop using it I think. It's making the debugging process much harder. I really only used it for... well making the job easier and not having to compile everything from scratch. Oh well.

All kinds of problems with the back-end right now. Connection refused, hitting max limits, overloading, Firefox blocking, WHMCS broken, for a few days everything broken. I'm considering an OS reinstall and going from scratch again. Centmin is great for non-experienced sysadmins but the debugging process with centmin is a damn pain in the arse.

tried nginx debug logging https://community.centminmod.com/posts/21637/ ?

 

[Robust]

if it's ssl related check with ssllabs site test to see if you have ssl certificate/config misconfiguration

Errors thrown everywhere. I might just go with a fresh installation and move data over. Data is very independent on our servers and is easily portable, that's how we take constant backups.

Problems started when I upgraded centmin from old versions to your new git thing (which was a cool idea, actually). Not sure if there was something wrong that happened in the process.

Pulling some parts from your git but mostly going with a custom installation now, just more familiar with that. I could learn how centmin does things (well, basically just where everything is) but yeah... Centmin makes the job easier, too easy for me lol, I'm basically relearning the ropes there.

 

[eva2000]

indeed firefox problem is your server ssl cipher misconfiguration https://dev.ssllabs.com/ssltest/analyze.html?d=apantic.com&hideResults=on

you're using blacklisted SSL ciphers which HTTP/2 do not support !

 

[eva2000]

Errors thrown everywhere. I might just go with a fresh installation and move data over. Data is very independent on our servers and is easily portable, that's how we take constant backups.

Problems started when I upgraded centmin from old versions to your new git thing (which was a cool idea, actually). Not sure if there was something wrong that happened in the process.

Pulling some parts from your git but mostly going with a custom installation now, just more familiar with that. I could learn how centmin does things (well, basically just where everything is) but yeah... Centmin makes the job easier, too easy for me lol, I'm basically relearning the ropes there.

were you on centmin mod version nginx with spdy prior to git update ? if so that is probably why as new nginx versions support HTTP/2 so they auto update all nginx vhosts to support HTTP/2 see http://centminmod.com/http2-nginx.html but you ssl ciphers do not support HTTP/2 and are black listed

you can use nginx vhost generator at http://centminmod.com/vhost.php and select self signed ssl check box and drop down menu for xenforo config and generator a template of latest ssl configured vhost for HTTP/2 and use those settings transplanted to your vhost (oh do not transplant commented out ocsp stapling settings as that only applies to self-signed ssl not to paid)