Zen Internet Support Forum

Dear Zen

 As a long admirer and user of Zen's services I think it time you reviewed your service alerts process.

The problems of last weekend were anticipated and it would be much better if we could put a message on our websites to alert our users or email our sites' members in anticipation of a major outage. That would mean that you send an email to us to alert us of upcoming problems, before they happen, and not leave it to us to find that we have a problem and then to come scurrying to the support page to try to understand the not very friendly alerts notices.

I am sure with the technical skills at your disposal that you can move to a more pro-active approach and allow your customers to keep their customers better informed.

As it is, a new webservice I have introduced and which receives most interest at weekends was out for a considerably long period of time and there was no way we could pre-advise our users. Nor was there a friendly notice put up to apologise for the outage.

I look forward to an improvement in this part of what is normally a very good service from your company.

 

David Ryedale

Hello,

The problems last weekend were in no way anticipated or planned - they were the result of a major platform fault which occurred on Thursday, and is still ongoing (however the effects have been reduced due to the work that's taken place so far).  We attempted to restore service on the existing service on Thursday, but it became apparent that this was not possible - so the decision was made to immediately begin moving customers over to a new platform.  That platform was already in development, and we were expecting to begin moves to it in the next month; had it occurred in that manner than advance notification would have been possible.  As it was, our actions were in direct response to a serious issue which required immediate attention.  If we did do this then the potential impact on our customers was significantly greater than that experienced over the past week.

Had we the option available to us we would have much preferred to be able to provide reasonable notice to all our customers through our Service Alert function - and that is the manner in which future planned maintenance and unplanned outages will be published.

We recommend subscribing to the RSS feeds from our Service Alerts page - this will mean you get updates as they are posted - RSS is a far more reliable means of receiving alerts than e-mail (e.g. you can subscribe in multiple places and you won't miss alerts if, for example, your e-mail service is affected by a fault).  We are currently working on increasing visibility of the RSS feeds with a page on our Support site highlighting how to get them; along with an alert feed to the front page of the Support site, showing the titles for open alerts. I'll be speaking with our PR and products teams on how to increase visibility of our service notifications.

I'd like to finish by apologising on behalf of Zen for the problems you and other cPanel customers have seen lately - and assure you that the steps we've taken (and have planned) to improve the platform will result in a much improved level of service.

We welcome any further feedback.

kind regards,
Phil D.Long

---

--
Phil Long
Technical Support Manager - Zen Internet Ltd.

Still problems then? No email today at all, sent a test one to myself and it has bounced back!!!! But nothing showing in service alert, when is the alert going to be in real time?

Hi,

We're not aware of any general issues at present - without details of the bounce it's impossible for me to say what the cause of the problem you're experiencing may be.  

Do you know which cPanel server you're on (use the checker on the alert), can you provide details of the bounce message, and is your domain hosted with Zen or another provider ?  (e.g. if your other provider is having DNS issues then that may be a possible reason for the problem).

kind regards,

Phil. 

---

--
Phil Long
Technical Support Manager - Zen Internet Ltd.

All hosted with Zen, CP06 I think (ip is 82.71.204.9), no email from anyone today, family have sent a couple and they have not arrived, the one I sent bounced back as permenant error!! Here is the header:

From MAILER-DAEMON@yahoo.com Sat Mar  1 19:48:07 2008
Return-Path: <>
Authentication-Results: mta134.mail.ukl.yahoo.com  from=yahoo.com; domainkeys=pass (ok)
Received: from 217.146.182.123  (HELO web28103.mail.ukl.yahoo.com) (217.146.182.123)
  by mta134.mail.ukl.yahoo.com with SMTP; Sat, 01 Mar 2008 19:48:07 +0000
Received: (qmail 71865 invoked for bounce); 1 Mar 2008 19:48:07 -0000
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com;
  h=Date:From:To:Subject;
  b=sl7mXNPaigO+c5a7pkuSKtog+cZmcb6SEQmvvjTl0x8IyKxz+cqDc1A8SIYjCfqMUvo8nSt7vwEgpUGjjCJg1R2okYDVpVXA7RjOg3ieBJ7JvIk3W34+yEvSuAIQJfQ1WuAggIB3Za2+Oh5uT63N4PYAZWlsv/i4OZFwP+soxjI=  ;
Date: 1 Mar 2008 19:48:07 -0000
From: MAILER-DAEMON@yahoo.com
To: *********@yahoo.co.uk
Subject: failure notice
Content-Length: 2443

 Hope that helps you, it means very little to me!!!

Edit: it also had this in the rtn'd email:

82.71.24.9 does not like recipient.
Remote host said: 550 5.7.1 Unable to relay for *******@transmac.co.uk
Giving up on 82.71.24.9.
 

Great!!!!!!!

 Just called Tech Support, looks like it is a DNS error, someone inputting the wrong IP address, but, NOTHING can be done till Monday, so another day without email, another 24hrs of lost business. And it looks like all the emails that have been sent during this time are lost.

Yet, if the tech support had access to change the DNS then I would have only lost 24hrs of email, not 48. Thats pretty poor Phil, they are your team and yet you are not giving them the tools to resolve issues. Sorry to say that I will probably be moving my hosting after this issue. It's not that there are problems, we all know that occasionally problems occur, it is how they are resolved, I felt really quite sorry for the guy I have just spoke to, was not his fault at all, in fact he seemed to spot the issue very quickly, but just did not have the access to be able to reslove it.

Not a happy bunny this end!! Due to my type of customer most of my new business comes in over the weekend, so all this weekends will now be lost - we could have halfed this if your tech guys were given access to the tools they need to do their jobs properly - this is your dept. is it not Phil?
 

Hi there,

I've just taken a look at this and from what I can see the MX record is actually correct - there was no DNS error.  There was a configuration error with our mail platform not pointing at cPanel, but Technical Support contacted the on-call engineer who could correct this.  The note indicates this was fixed this morning.

I'm sorry that this information was not relayed to you and the problems this has caused you; I'll look into why this happened on Monday.

kind regards,

Phil D.Long

---

--
Phil Long
Technical Support Manager - Zen Internet Ltd.

Its still not working though!!

Problem is they have missed out a "0" apperantly. It is  xx.xx.24.xx and should be xx.xx.204.xx

Hi,

That is the error that has been corrected - I've checked the domain configuration for our mail servers and can see it is correct - i.e. 204  NOT 24.  If you're still having problems I'd suggest contacting support again to see whether there are any other issues with your account.

regards,

Phil 

---

--
Phil Long
Technical Support Manager - Zen Internet Ltd.

Still sat here tapping my fingers on the desk, counting all the lost potential business over the weekend, and now looking forward to the lost business today!!!!

Not the type of service we had come to expect from Zen to be honest. 

Exactly.

 

It's the type of service I was getting used to with Force 9 where I came here from, though I'm now wondering if it was a wise decision.

 

Think I may be looking for another move if this carries on. 

---

Regards,

Dave

Hi Phil

 The problem with RSS is that you get all feeds. There is no way of disaggregating the content to make it specific to the end user's individual server, as far as I can see.

 Could you please review matters and come up with a more targeted solution that your customers want, rather than what you want to deliver? Have you surveyed your customers on how they would like to be notified?

I would have thought, in these days of Web 2.0 and personalisation, a step in the direction of automatic and individual notification would be well within the realm of possibility.

I take your point about email, but most of us have multiple webmail services as well eg Googlemail and the like (which we an also read on our PDAs when out and about). After all, it would be the height of daftness to rely on an email service from a broken server to inform you that the server was broken. And if we can get your online service alerts we can get webmail.

Finally, the whole nature of the service alerts isn't exactly what I would call easy to understand. Clarity of the server affected, the dates downtime began, expected outage and problem resolution steps could be vastly improved and make the whole end-user customer experience so much better.

Best wishes

David Ryedale

davidryedale:

Finally, the whole nature of the service alerts isn't exactly what I would call easy to understand. Clarity of the server affected, the dates downtime began, expected outage and problem resolution steps could be vastly improved and make the whole end-user customer experience so much better.

 Yeah, pretty appalling communication, I mean if you're going to say further  updates will be posted then post some. I know and appreciate that time scales are not easy to even estimate when dealing with faults, however, it's not unreasonable for a service alert update to be posted say when cpanel06 work is completed prior to work starting on cpanel11. It's not acceptable to post:

cp06 - due to begin today
cp11 - due to begin when cp06 is completed

as your schedule for planned work and then not update the service alert appropriately at the right time!

I can appreciate you wanting to focus resources on the fault rectification, however, we the customers, need to know what's going on and quite frankly the service alerts are of poor quality, seem rushed, and undermine Zen's hard won reputation for professionalism and customer service.

As you can tell, I'm not impressed - so far - Sorry!

Addendum:

Last update on this "High profile incident," is 14:45 - that's acceptable?
 

Hi,

At present we consider the alert detail to be appropriate for planned moves that will only affect a single site at a time.  It's not possible to give an indication of time scale as the process can be interupted at times - and is highly dependant upon server load.  It would be setting false expectations to state specifics when it's not possible to provide such detail.  There is no update since 2.45pm as the situation is the same - cPanel 06 is being migrated, one user at a time.  Once that is completed we will post an update to indicate the beginning of the migration of users on cPanel 11.  Again, that will be one user at a time. 

There are no system-wide issues for cPanel 6 or 11 at present, so no other work is taking place that would warrant an alert.

Throughout the problems experienced with cPanel over the last week we have posted information as soon as we have it - and wherever possible we will provide timescales.  Unfortunately there are circumstances where a reliable time scale can not be provided. 

kind regards,

Phil D.Long 

 

---

--
Phil Long
Technical Support Manager - Zen Internet Ltd.

Hi Phil

"At present we consider the alert detail to be appropriate for planned moves that will only affect a single site at a time"

 Are you trained as a politician?

Unfortunately, your users don't agree with you. In the spirit of constructive feedback and criticism, I for one would like to see a vastly improved way in which I can quickly and easily identify problems that will, or already have, directly affected me and my customers.

Hastily-written posts that appear to be just going through the motions are not what we expect from a company that rightly gets many plaudits for the rest of its customer service.

Please re-consider the broader picture and not a narrow focus on just the problems of last week or so. It's just that these have helped to focus us on the nature of your alerts process.

Of course, if you're telling me that none of your servers is ever going to fall over again and take our sites with it, then you can forget the foregoing.

Best wishes

 

David Ryedale