Donations for Shawns coffee pot...

[digitalpoint]

Lol... PM me your IP or something and I'll check if you want.

 

[MattW]

Running really slow at the minute Shawn. Loading find-new/threads taking about 45 seconds to render the page.

 

[IcEWoLF]

Yep, it's slow, but I believe he said there are some issues that still need to be addressed server side.

I am sure dp will take care of things soon. He just needs 2 hours of sleep now.

 

[Gladius]

2 hours? I heard that dp never sleeps for more than 20 minutes... a week!

 

[digitalpoint]

Running really slow at the minute Shawn. Loading find-new/threads taking about 45 seconds to render the page.

I could people mentioned it, but it's been difficult to track down because it's been fine for me...

 

[MattW]

Using Chrome, the biggest delay I'm getting seems to be from x.dpstatic.com. Taking 25 seconds to serve the avatars in a thread.

EDIT: and another new threads

 

[digitalpoint]

Yeah... I think it's about 50% packet loss somewhere upstream on some route... really confused right now

 

[MattW]

Yeah... I think it's about 50% packet loss somewhere upstream on some route... really confused right now

Yep, looks that way:

[root@vps2 XenTag]#traceroute x.dpstatic.com
traceroute to x.dpstatic.com (216.9.35.59), 30 hops max, 40 byte packets
1 range-1.gateway.pcsmarthosting.co.uk (95.154.207.1) 0.945 ms 1.092 ms 1.076 ms
2 87.117.211.198 (87.117.211.198) 0.853 ms 0.835 ms 0.978 ms
3 590.net2.north.dc5.as20860.net (62.233.127.214) 0.674 ms 0.695 ms 0.677 ms
4 610.core1.hex.as20860.net (62.233.127.181) 1.593 ms 1.580 ms 1.559 ms
5 TenGigabitEthernet7-1.ar4.LON3.gblx.net (64.211.1.189) 1.548 ms 1.529 ms 1.509 ms
6 ae3.scr4.LON3.gblx.net (67.17.111.142) 1.454 ms 1.489 ms 1.464 ms
7 xe2-3-2-10G.scr4.LAX1.gblx.net (67.16.165.69) 132.295 ms 132.337 ms 132.319 ms
8 po2.ar4.LAX1.gblx.net (67.16.132.214) 142.546 ms 142.528 ms 134.681 ms
9 ip-transit.ethernet16-4.ar6.lax1.gblx.net (159.63.49.174) 132.491 ms 132.477 ms 132.456 ms
10 ge5-2-6509-c.castleaccess.com (69.43.169.114) 136.074 ms 136.044 ms 136.021 ms
11 (69.43.187.34) 135.945 ms 135.930 ms 135.911 ms
12 vl98.sndg-cr-1.nethere.net (66.63.128.40) 137.987 ms 138.471 ms 137.267 ms
13 systemofadown.digitalpoint.com (216.9.35.59) 136.789 ms 137.431 ms 138.337 ms
[root@vps2 XenTag]#ping x.dpstatic.com
PING x.dpstatic.com (216.9.35.59) 56(84) bytes of data.
64 bytes from systemofadown.digitalpoint.com (216.9.35.59): icmp_seq=2 ttl=52 time=136 ms
64 bytes from systemofadown.digitalpoint.com (216.9.35.59): icmp_seq=4 ttl=52 time=136 ms

--- x.dpstatic.com ping statistics ---
29 packets transmitted, 2 received, 93% packet loss, time 27997ms
rtt min/avg/max/mdev = 136.903/136.914/136.926/0.370 ms

 

[MattW]

13 hops to each domain:

[root@vps2 XenTag]#traceroute x.dpstatic.com
traceroute to x.dpstatic.com (216.9.35.59), 30 hops max, 40 byte packets
1 range-1.gateway.pcsmarthosting.co.uk (95.154.207.1) 0.945 ms 1.092 ms 1.076 ms
2 87.117.211.198 (87.117.211.198) 0.853 ms 0.835 ms 0.978 ms
3 590.net2.north.dc5.as20860.net (62.233.127.214) 0.674 ms 0.695 ms 0.677 ms
4 610.core1.hex.as20860.net (62.233.127.181) 1.593 ms 1.580 ms 1.559 ms
5 TenGigabitEthernet7-1.ar4.LON3.gblx.net (64.211.1.189) 1.548 ms 1.529 ms 1.509 ms
6 ae3.scr4.LON3.gblx.net (67.17.111.142) 1.454 ms 1.489 ms 1.464 ms
7 xe2-3-2-10G.scr4.LAX1.gblx.net (67.16.165.69) 132.295 ms 132.337 ms 132.319 ms
8 po2.ar4.LAX1.gblx.net (67.16.132.214) 142.546 ms 142.528 ms 134.681 ms
9 ip-transit.ethernet16-4.ar6.lax1.gblx.net (159.63.49.174) 132.491 ms 132.477 ms 132.456 ms
10 ge5-2-6509-c.castleaccess.com (69.43.169.114) 136.074 ms 136.044 ms 136.021 ms
11 (69.43.187.34) 135.945 ms 135.930 ms 135.911 ms
12 vl98.sndg-cr-1.nethere.net (66.63.128.40) 137.987 ms 138.471 ms 137.267 ms
13 systemofadown.digitalpoint.com (216.9.35.59) 136.789 ms 137.431 ms 138.337 ms

[root@vps2 XenTag]#traceroute digitalpoint.com
traceroute to digitalpoint.com (216.9.35.56), 30 hops max, 40 byte packets
1 range-1.gateway.pcsmarthosting.co.uk (95.154.207.1) 0.578 ms 1.201 ms 1.190 ms
2 87.117.211.198 (87.117.211.198) 0.733 ms 0.722 ms 1.050 ms
3 590.net2.north.dc5.as20860.net (62.233.127.214) 0.455 ms 0.444 ms 0.429 ms
4 610.core1.hex.as20860.net (62.233.127.181) 1.600 ms 1.601 ms 1.579 ms
5 TenGigabitEthernet7-1.ar4.LON3.gblx.net (64.211.1.189) 1.584 ms 1.572 ms 1.561 ms
6 ae0.scr3.LON3.gblx.net (67.16.142.101) 1.501 ms 1.634 ms 1.471 ms
7 xe2-3-2-10G.scr3.LAX1.gblx.net (67.16.165.65) 132.260 ms 132.299 ms 132.279 ms
8 e5-1-40G.ar6.LAX1.gblx.net (67.17.111.65) 134.379 ms 134.364 ms 132.204 ms
9 ip-transit.ethernet16-4.ar6.lax1.gblx.net (159.63.49.174) 132.331 ms 132.315 ms 132.573 ms
10 ge5-2-6509-c.castleaccess.com (69.43.169.114) 136.114 ms 136.385 ms 136.369 ms
11 (69.43.187.34) 135.986 ms 135.968 ms 135.947 ms
12 vl98.sndg-cr-1.nethere.net (66.63.128.40) 139.275 ms 138.005 ms 137.801 ms
13 crystalmethod.digitalpoint.com (216.9.35.56) 136.822 ms 136.852 ms 136.840 ms

Hops 7 & 8 are different

7 xe2-3-2-10G.scr4.LAX1.gblx.net (67.16.165.69) 132.295 ms 132.337 ms 132.319 ms
8 po2.ar4.LAX1.gblx.net (67.16.132.214) 142.546 ms 142.528 ms 134.681 ms

 

[digitalpoint]

I think my switch in the data center might be dying... lol 50% packet loss was resolved by rebooting the switch... and by "resolved" I mean brought it down to 10% packet loss or so. Seriously... wtf?

 

[Mark3121]

For me the site this last few days was not slow but not fast either, around 2-4s per page when browsing around.

Today is a little better, not as fast as here yet, looks great though.

 

[MattW]

I think my switch in the data center might be dying... lol 50% packet loss was resolved by rebooting the switch... and by "resolved" I mean brought it down to 10% packet loss or so. Seriously... wtf?

Much faster for me now. Not sure how your switching network is set up, but could it be a faulty port on the switch that was causing the problem with access to one of the servers?

 

[SneakyDave]

Maybe you should be using the justinbieber.digitalpoint.com server?

 

[digitalpoint]

Much faster for me now. Not sure how your switching network is set up, but could it be a faulty port on the switch that was causing the problem with access to one of the servers?

I'm still not entirely sure... Rebooting switching DEFINITELY fixed the bulk of it, but still seeing some packet loss.

The uplink connection is fine... certain servers get no packet loss. But servers behing the load balancers do. We have a primary and hot backup load balancers (obviously on different ports on the switch)... Switching all traffic to the backup load balancer routes all traffic through different port/ethernet cable, but still the same amount of packet loss. Load balancers are not under high load at all. Whole thing is really weird.

 

[Slavik]

I'm still not entirely sure... Rebooting switching DEFINITELY fixed the bulk of it, but still seeing some packet loss.

The uplink connection is fine... certain servers get no packet loss. But servers behing the load balancers do. We have a primary and hot backup load balancers (obviously on different ports on the switch)... Switching all traffic to the backup load balancer routes all traffic through different port/ethernet cable, but still the same amount of packet loss. Load balancers are not under high load at all. Whole thing is really weird.

Did you try adding some magic?

 

[digitalpoint]

Did you try adding some magic?

Yeah... my magic isn't strong enough to fix it entirely though.

 

[Slavik]

Yeah... my magic isn't strong enough to fix it entirely though.

Time to call in magical trevor

 

[Digital Doctor]

Yeah... I think it's about 50% packet loss somewhere upstream on some route... really confused right now

There must have been some problems with the internets last night.
I was getting odd packet losses as well.

 

[digitalpoint]

There must have been some problems with the internets last night.
I was getting odd packet losses as well.

Not sure how rebooting my switch would have helped the whole Internet get fixed, but I'll just pretend it did.

 

[Digital Doctor]

rebooting my switch

How did you know your switch needed rebooting ?
My router need rebooting sometimes .... the only way I know the router needs rebooting is .... I can't access the internet at all.