Broadband
Dial-Up
Email
Hosting
Wireless
Security
Useful Tools
Quick Reference
Contact Us
Customer Portal
Site Search

Zen Internet Support Forum

Welcome to the Zen Internet community support forums.

Before posting we recommend you search our
extensive Knowledge Base or the forum archives
as an answer to your query may already be available.

Welcome to Zen Internet Support Forum Sign in | Join | Help
in
Forums Forum Rules

HTTP problems driving me crazy

Last post 21-05-2008, 2:25 PM by alewis. 38 replies.
Page 2 of 3 (39 items)   < Previous 1 2 3 Next >
Sort Posts: Previous Next
  •  29-04-2008, 10:12 AM 30413 in reply to 30370

    Re: HTTP problems driving me crazy

    If you were using XP I would have suggested checking the network connection preference and making sure that any other network interfaces didn't have DNS servers specified.

    The symptoms point to DNS issue, but it would be nice to confirm this.  When you know the connection wouldn't normally be working try pinging a domain (e.g.echo.zen.co.uk) and see if it resolves right away, if not try again and see if it does eventually resolve.  If it seems to resolve instantly then wouldn't point to a DNS issue and you should be checking if you are getting responses from your ping packets.  If it doesn't resolve instantly leave the connection again till the point you would expect it to no longer be working and then ping an IP (echo.zen.co.uk = 212.23.5.209) if this works instantly it would point to a DNS issue (granted back to square one but least we have a better idea).  If it doesn't respond or drops the first few packet it points to a lower level connection issue.

    If you let us know your results we can advise where to try next.  Oh the other machines that don't have the issue are they also Vista?  What router do you have?
     

  •  29-04-2008, 2:22 PM 30417 in reply to 30413

    Re: HTTP problems driving me crazy

    Hello once again,

    No.1 I have no idea why the second line of the hosts file is included nor what its purpose is as I have never seen anything like that in the hosts file. (sorry! I know thats no help really though).

    No.2 YES! I think its also a DNS issue but what precisely I'm not clear about.

    No.3 Also a further thought is I'm wondering if this is a problem related to your MTU (maximum transmission unit) settings?

    **SEE http://www.kitz.co.uk (for a very good explanation of MTU & how to precisely calculate your connections most appropriate values.) NOTE: In both XP & Vista the RWIN values are calculated dynamically so don't bother resetting these.  Just the MTU value.

    Regards Ivan

     

  •  30-04-2008, 2:31 AM 30427 in reply to 30417

    Re: HTTP problems driving me crazy

    cyteck:
    No.3 Also a further thought is I'm wondering if this is a problem related to your MTU (maximum transmission unit) settings?

    **SEE http://www.kitz.co.uk (for a very good explanation of MTU & how to precisely calculate your connections most appropriate values.) NOTE: In both XP & Vista the RWIN values are calculated dynamically so don't bother resetting these.  Just the MTU value.

    Regards Ivan

     Vista "Autotunes" MTU and RWIN settings "On the Fly" so using tools to tweak these values won't help at all.  This isn't unique to Vista either, there are some Linux Distributions that also can do this with good efficiency.Smile
     

  •  30-04-2008, 12:52 PM 30432 in reply to 30427

    Re: HTTP problems driving me crazy

    Hi,

    YES! of course you are correct, its just XP's MTU that can be tweaked with positive effect.

    Ivan
  •  30-04-2008, 1:38 PM 30433 in reply to 30417

    Re: HTTP problems driving me crazy

    The 2nd line in the hosts file of '::1' is the IPv6 notation for localhost and is innocuous.

    To the OP:

    1) When you get this failure do you get a generic failure page, or is it the browser specific error page ?

    2) What happens if you run "telnet www.some.website 80" from a command line ?

    3) If #2 connects easily the what do you get if you then type 'GET /blort HTTP1/1' and a few blank lines ?

    4) Are you positive that this is only affecting HTTP ? It is of course possible that other applications are retrying automatically so you're not actually seeing that they're struggling. For the ones that work, do the applications have any form of connection log to see what they've got to say ?


    --
    Jerry Nicholls
    Principal Systems Engineer
    perl -e '$_=q(print "perl -e \x27\$_=q($_);eval\x27\n");eval'
  •  03-05-2008, 7:09 PM 30459 in reply to 30433

    Re: HTTP problems driving me crazy

    Hello,

    I just came to the forums looking for advice about the exact same problem. I have lost count of the number of times over the last couple of weeks or so that pages have been coming up as 'not found' on the first page load and taking one or often several refreshes to finally load. This is happening on multiple machines on the network. The error page is generated locally as if I'd entered a non-existant URL which makes me suspect a DNS issue. Whilst the problem is most noticable via HTTP in the browser, I have also experienced some issues with other ports such as 110 (POP), and with Windows Live Foldershare which attempts to use a number of ports. I've also had an ongoing issue with JungleDisk for quite a while now which uses HTTP to communicate with Amazon S3 and JungleDisk's own servers and has been reporting issues resolving DNS. I had initially put that down to a problem with the JungleDisk software itself, but having begun to experience similar issues with web browsing I now suspect it's all related to the same problem.

    I've looked over my router configuration and can't find anything wrong, I've tried reconnecting, rebooting the router and the computers, and to no avail each time. I've checked the hosts file and there's nothing with that (I actively manage it myself so I'd know if it had been hijacked anyway). In terms of my network infrastructure, it hasn't changed for a while and was working fine up until the last couple of weeks so I doubt it is likely to be an issue with that.

    I'm therefore at a loss as to what's going on, but the fact that I'm not the only one experiencing this problem at least adds support to my belief that it's not a problem at my end! I'm wondering whether it might be related to this thread? http://forum.zensupport.co.uk/thread/30373.aspx

    Thanks in advance for any assistance,

  •  06-05-2008, 10:01 AM 30464 in reply to 30459

    Re: HTTP problems driving me crazy

    Ok, this is looking like it may be DNS related. HTTP is likely to show up such problems over other protocols due to the number of lookups that may be done per-page.

    Could you find out what DNS servers you're set to ? Under Windows that will be 'ipconfig /all' from a command prompt.

    The issue referred to in that link would have affected anyone set to 212.23.3.100, but 212.23.6.100 was fully functional and reachable so all that should have been seen is a delay in resolving records. The problem at that time, however, was network hardware failure and that has since been fixed so isn't causing any problems now.

    At this time I know of no issues with any part of our DNS platform but will investigate in case this is the first sign of any larger problems.

    If any one else is suffering this but has access to a Linux / OS X platform could they try a few 'dig some.web.site' commands and see what crops up ? 'dig' is very good at giving timings, what server resolved the address, and error messages so is great for this kind of diagnostics.


    --
    Jerry Nicholls
    Principal Systems Engineer
    perl -e '$_=q(print "perl -e \x27\$_=q($_);eval\x27\n");eval'
  •  06-05-2008, 9:57 PM 30465 in reply to 30464

    Re: HTTP problems driving me crazy

    This problem seems to have got worse the last couple of days; my online backups are completely screwed as they use Amazon S3 via REST, so there's obviously shedloads of HTTP calls being made, and it's dropping out after hardly any time at all. At least before it was only playing once every few days or so, now it's playing up within 5-10 minutes.

    My router's currently using the DNS servers you supply it with; currently these are set to 212.23.3.100 and 212.23.6.100 so seemingly no problems there.

    Couple of dig outputs for you:

     ; <<>> DiG 9.3.2 <<>> google.co.uk. NS
    ;; global options:  printcmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 230
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 4

    ;; QUESTION SECTION:
    ;google.co.uk.   IN NS

    ;; ANSWER SECTION:
    google.co.uk.  4581 IN NS ns2.google.com.
    google.co.uk.  4581 IN NS ns3.google.com.
    google.co.uk.  4581 IN NS ns1.google.com.
    google.co.uk.  4581 IN NS ns4.google.com.

    ;; ADDITIONAL SECTION:
    ns4.google.com.  34735 IN A 216.239.38.10
    ns1.google.com.  34079 IN A 216.239.32.10
    ns2.google.com.  34103 IN A 216.239.34.10
    ns3.google.com.  34584 IN A 216.239.36.10

    ;; Query time: 46 msec
    ;; SERVER: 192.168.0.1#53(192.168.0.1)
    ;; WHEN: Tue May 06 21:43:15 2008
    ;; MSG SIZE  rcvd: 176

     ; <<>> DiG 9.3.2 <<>> news.bbc.co.uk
    ;; global options:  printcmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1431
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 2, ADDITIONAL: 2

    ;; QUESTION SECTION:
    ;news.bbc.co.uk.   IN A

    ;; ANSWER SECTION:
    news.bbc.co.uk.  2216 IN CNAME newswww.bbc.net.uk.
    newswww.bbc.net.uk. 126 IN A 212.58.226.29

    ;; AUTHORITY SECTION:
    bbc.net.uk.  23294 IN NS ns0.thdo.bbc.co.uk.
    bbc.net.uk.  23294 IN NS ns0.rbsov.bbc.co.uk.

    ;; ADDITIONAL SECTION:
    ns0.rbsov.bbc.co.uk. 33972 IN A 212.58.227.47
    ns0.thdo.bbc.co.uk. 34077 IN A 212.58.224.20

    ;; Query time: 31 msec
    ;; SERVER: 192.168.0.1#53(192.168.0.1)
    ;; WHEN: Tue May 06 21:45:28 2008
    ;; MSG SIZE  rcvd: 157

    Can't see anything amiss there myself, but you may! One final piece of information - we use Zen at work and I have experienced the same problem there, although it happens less frequently. Adds support to the notion that it's a problem at your end though.

    Edit: Just realised those dig outputs are no good as that's my local router IP in there, sorry.

  •  07-05-2008, 10:57 AM 30466 in reply to 30465

    Re: HTTP problems driving me crazy

    I had a good look over things yesterday and still can't see anything apparently wrong at our end - all the servers look to be behaving as required.

    For those that are suffering these issues, what is the first DNS resolver listed - ie is it 3.100 or 6.100 ?


    --
    Jerry Nicholls
    Principal Systems Engineer
    perl -e '$_=q(print "perl -e \x27\$_=q($_);eval\x27\n");eval'
  •  07-05-2008, 6:49 PM 30467 in reply to 30466

    Re: HTTP problems driving me crazy

    3.100 and then 6.100

    Had the issue both at home and at work again today.

  •  09-05-2008, 1:20 PM 30470 in reply to 30467

    Re: HTTP problems driving me crazy

    I can't be certain, but I have just made some changes to our DNS servers that may fix the issues seen - I say 'may' as it wasn't conclusive the DNS was at fault, but I did spot an issue with our DNS servers whilst investigating.

    Some weeks back we upgraded from BIND9.3 to BIND9.4 to solve some other issues we were seeing. This upgrade did indeed make matters significantly better. However BIND9.4 introduced a new 'clients-per-query' value which could have a maximum setting. This value applies to the number of simultaneous queries for the same record during the time it took to resolve that record, eg the simultaneous number of lookups for www.bbc.co.uk. Most of the time this has been settling down in the 50-70 area, with the maximum allowed defaulting to 100, so not causing a problem. Whilst looking today I saw that we have been occasionally hitting the 100 mark over the past few days. When this happens subsequent queries (for the same record) are dropped as it it believed (by BIND) that there is an issue resolving that record if too many simultaneous queries occur. Thus, given the number of customers we have, the numbers of our DNS servers, the popularity of certain sites, and the number of lookups required per page (eg lots of images), it's understandable that the limit of 100 may be hit from time to time.

    As such, I have raised the limit to take into account these spikes we have seen and we'll continue to monitor to see if the limit needs to be adjusted further.

    Hopefully, if it was DNS at fault, that this will have fixed the issue.


    --
    Jerry Nicholls
    Principal Systems Engineer
    perl -e '$_=q(print "perl -e \x27\$_=q($_);eval\x27\n");eval'
  •  09-05-2008, 3:36 PM 30473 in reply to 30470

    Re: HTTP problems driving me crazy

    Thanks Jerry! That sounds really promising - although I hadn't really thought about it much before now, it did seem to be the larger sites that were having trouble (Google, Youtube, BBC etc) whilst smaller sites were generally fine. This ties in with what you have found because the bigger sites are obviously going to get more requests. What's more, the biggest issue I had was with JungleDisk as I mentioned before, which makes 100s of HTTP calls to Amazon S3's REST API in quick succession. This again ties in with what you have found.

    That said, whilst it was mainly the larger websites which caused problems, I have definitely had issues with some of my personal domain names which wouldn't have been receiving that volume of requests. So it may be that the issue isn't quite solved yet.

    I will keep a close eye on things over the weekend and let you know - thanks again!

  •  09-05-2008, 4:31 PM 30474 in reply to 30473

    Re: HTTP problems driving me crazy

    It seems to have been introduced as a 'burst' control for unresponsive domains, ie if the server received lots of queries in a short period of time for a slow DNS server it would just drop the excess requests. As you've seen, and I suggested, this would be very obvious with HTTP and popular sites. It's still possible that your personal domains may just have hit the limit if you managed to make a lot of queries in a very short period of time and the remote DNS server was slow to respond.

    Hitting this limit was obviously not going to help matters, and may certainly have been the cause of what people have been seeing - we shall have to see if between your reports and my investigations if we've managed to fix things before they started to cause a larger impact.

    On these forums it can often be difficult to identify what is a problem specific to just one (or a few) users and one that is something at our end. At first this felt to be a user-specific issue, but comments made here seemed to point at our DNS having a problem - which in the end there was certainly one configuration option that wasn't set to an appropriate value.


    --
    Jerry Nicholls
    Principal Systems Engineer
    perl -e '$_=q(print "perl -e \x27\$_=q($_);eval\x27\n");eval'
  •  09-05-2008, 7:53 PM 30475 in reply to 30474

    Re: HTTP problems driving me crazy

    Hi all,



    Ok, I'll respond in detail with the info asked for in trying to track down what is wrong but I can say that I am still getting the issue with HTTOP despite the measure introduced to eliminate/reduce it.

    I do appreciate the efforts being put in to determining what the issue might be and to try and resolve it though,

    Many thanks,

     

    SR 

  •  15-05-2008, 12:05 AM 30540 in reply to 30474

    Re: HTTP problems driving me crazy

    An update on this as promised. It certainly seems better - I've not had as many problems since you made the change. However, it hasn't quite been perfect - JungleDisk in particular has still hit the odd DNS issue, especially with large operations when it's making a lot of requests very quickly. I've also had a couple of HTTP requests go dead today in fairly quick succession, to Wikipedia and SlideShare, so I don't know whether maybe the new limits you've set are still too low?

    I'll keep monitoring the situation and advise accordingly.

Page 2 of 3 (39 items)   < Previous 1 2 3 Next >
View as RSS news feed in XML