We never stop working for you.  [well unless you are a customer with a dynamic IP that is…]


Companies can either handle a policy well by communicating in a great way, or they can totally blow it.  Handle it right, you build Customer Evangelists.  Handle it wrong and you end up with blog posts like this one.


Today Verizon proved that they are totally messing up by not notifying customers that they’ve changed their mail policies.  Eriq posts on his blog the troubleshooting steps and resolution that he had to do to overcome his issue [and it’s a great how to on checking mail issues by the way]





“AS OF JANUARY 31, 2005, VERIZON WILL NO LONGER ALLOW BUSINESS OR
RESIDENTIAL CUSTOMERS WITH DYNAMIC WAN IPS TO RELAY THROUGH THEIR SMTP
SERVERS.” 





Update 2/22/2005:  Please note it appears that if you don’t host your domain with Verizon even if you do have a Static IP they block your sending.  Your only alternative is to route your email through your domain mail hoster’s SMTP servers or investigate the third party mail relayers like Eriq used.

 

33 Responses to Verizon…. “We do stop working for you”

  1. Thanks for the link. The same thing just happened here with SBC. It blocked port 25 for all DSL customers last week. That was a surprise, even though the company claims it told customers via its newsletter last year. Right, let me go back to my catalog of SBC newsletters.

    However, thanks to a number of bloggers who’d had the same problem, we were able to resolve the issue in a day’s time.

    The irony of telephone companies being the worst communicators in the world is rich.

  2. Tim Long says:

    When will ISPs learn that all we need from them is a pipe to the internet with a specified bandwidth and NO INTERFERENCE.

    –Tim

  3. Steve says:

    Tim wrote "When will ISPs learn that all we need from them is a pipe to the internet with a specified bandwidth and NO INTERFERENCE. "

    Probably about the same time that their customers learn to keep their PCs clean of viruses, trojans, etc.

  4. serge says:

    got hit with the same problem. what else have people done to combat this issue. I know of dyndns.org Outbound MailHop. I tried that but it does not seem to wanna send to verizon customers, I believe Verizon has them blacklisted.

    So any other ideas on how to solve this?

    Thanks.

  5. Daniel says:

    We, and apparently other Verizon DSL customers, started having problems sending mail through outgoing.verizon.net a few days ago. Investigation appears to show that the problem is not an intentional block of any sort, but rather that the Verizon SMTP server is no longer accepting the following format of the SMTP AUTH command:

    AUTH LOGIN <encoded username>

    though they still accept

    AUTH LOGIN

    followed by the <username> in response to the encoded "Username:" challenge.

    Unfortunately, MS Exchange 2003 uses the one-line format to initiate the authentication as permitted by RFC 2554 (http://www.faqs.org/rfcs/rfc2554.html), so anyone trying to use Exchange 2003 to forward mail through Verizon’s SMTP server will no longer be able to do so.

    I have contacted Verizon today and I was told that the problem will be forwarded to engineering. I’ll post back if/when I hear anything.

    –Daniel

  6. Eriq Neale says:

    Daniel –

    I’m amazed that you were able to speak with someone who actually understood what you were talking about. I had thesame conversation with multiple Verizon support engineers on Monday, told them abou the problems with the AUTH LOGIN <username> versus AUTH LOGIN, and got absolutely nowhere with them. Could you please share the secret you used to get routed to somone who had a clue? Do you stillhav a contact name, because there are several of us who could call and veify the problem with them to help speed up the "resolution" process.

    Thank!

    -Eriq

  7. Daniel says:

    Eriq –

    I hate to disappoint you, but I was doing all the talking when I spoke with the Verizon tech support agent; I spelled out the problem in excruciating detail and I insisted that he note the RFC that was being violated when he wrote up the ticket, but I would not claim that he understood anything that I was saying.

    After I finished my detailed technical explanation of the problem, there was a long pause, and then he said, "…and you know this…how?" I had to go on to explain my reading of the RFC and my hand-testing of the Verizon SMTP server via Telnet (you can easily do this). I’m sure I lost him.

    Anyway, I still haven’t gotten the promised callback from Verizon engineering, so I called back on the ticket and I’m on hold right now. Updates as I get them.

    –Daniel

  8. Anne Stanton says:

    WOW! Two weeks ago I switched to Cable. Managed to miss that blow..

  9. G says:

    Eriq and Daniel,

    I started experiencing this same problem on Monday Feb 21.

    I’m using LotusNotes as my mail client and have been using it for over a year and a half without any issues sending e-mails

    In the SMTP debug log, I see that it is using the AUTH LOGIN <encoded username>

    format. On the next line of the log, I get ReceiveResponse: 501 5.5.0 Invalid input (Invalid authentication protocol).

    I just got off the phone with Verizon technical support who were pretty much clueless about this problem and stated that they don’t support LotusNotes.

    I still demanded that they log this issue and troubleshoot it because I should be able to use whatever e-mail client I want.

    Daniel, have you had any response regarding your ticket. I even asked if they had any other calls regarding this issue, and the support person said NO.

    G

  10. Daniel says:

    I called Verizon back after not hearing from ‘engineering’ as I was promised. Guess what? They’d closed my trouble ticket as soon as they’d gotten off the phone with me on the previous call, according to the agent ‘Vaughn’ I spoke with this time. He also said he had no idea what the first tech was talking about when he said he’d refer my call to ‘engineering’ — Vaughn said there is no such procedure.

    He was very helpful, hwoever, and he did admit that there had been problem reports (though he wasn’t specific) related to SMTP over the last week or so, and he claimed that it was being investigated. He took down all the technical info I gave him about the violation of RFC 2554, and he said he would refer the ticket to an ‘offline group’ who would look into it and contact me back in approximately 24 (business) hours. He urged me to call back on Monday to check the status. I told him that there were incresing numbers of reports on the Internet about this specific problem with Verizon Business DSL customers, and that we would going to drop their service if it was not resolved. He was very sympathetic and he agreed htat Verizon should take the situation seriously. FWIW, the number to call is 888-649-9500, and keep hitting ‘0’ to get out of the infernal voice-recognition maze in order to speak to a human being.

    Updates as they become available…

    –Daniel

  11. Daniel says:

    P.S. I NEVER mention what email app I’m using — otherwise you’ll get the door slammed with a "we don’t support <Notes|Exchange|WhateverYou’reUsing> — bye!". I just hammer the point that thei SMTP service, since 2/21/05, is now in violation of RFC2554. That’s the error that needs to be corrected. Saying "we’re using Exchange" is the kiss of death.

    –Daniel

  12. ca_verizon says:

    the problem is that the configuration has changed on the mail servers. they do not accept single line authentication any longer, which is what exhange server uses. this is being looked by our network eng.

  13. G says:

    I just spend another hour on the phone with Verizon support trying to follow up and explain this situation. They claim that there were no changes made to their servers that should affect the sending of e-mail. What a bunch of crap. I think I should switch to Cable.

  14. Julie says:

    This is all very helpful to me but is there an alternate solution that anyone has. I have too many horrid memories of dealing with Verizon just to move a DSL line connection to move from one street to the next to know that they are not going to get this taken care of anytime soon or quickly.

    Thanks

  15. Daniel says:

    Just got off a long call with Verizon Technical Support Supervisor "Ivan" (ID#1562). If anyone has any questions, I suggest you attempt to contact him (888-649-9500). You will need to have a *lot* of patience to get through a coherent conversation with him.

    He finally told me in no uncertain terms that Verizon is *not* bound by RFC 2554 (http://www.faqs.org/rfcs/rfc2554.html) because it is not a Verizon-authored document, and Verizon is not supporting the initial-response argument to the AUTH command (see third paragraph in "Discussion:" section of major section 4, "the AUTH command", of the RFC), and Verizon does not consider this a bug or a violation of the SMTP AUTH specification and therefore IT WILL NOT BE FIXED. He referred me to their Terms of Service as the only document they consider themselves bound by.

    So that’s the final word. We’re consultants, and we’ll be advising all of our Verizon Business DSL customers to terminate their service and switch to an ISP that’s less dysfunctional.

    –Daniel

  16. Eriq Neale says:

    Thanks for the update, Daniel. I wish I could say that I’m surprised, but I’m not. Verizon is not the only company out there that refuses to b RFC-compliant, bu this is going to potentially cause some problems. Fortunately, for my clients that have been affected, I removed the smathost routing altogether, and so far no one has refused to accept e-mail from them. We do have a bakup plan, however, should that become the case.

    I continue my search for alternate ISPs in the DFW area that can provide service over the Verizon trunks, bu so far I haven’t found anyone that can do better than double my current cost for lower bandwidth.

    -Eriq

  17. Tried a few outbound relays – had the same problem with Dyndns not delivering to quite a few ISP’s.

    The one we have chosen is Authsmtp.com – http://www.authsmtp.com – seems to be working 100% (even delivering to Verizon!!).

    We chose them because they were low cost, monthly (not daily) quotas, more secure and virus scanning of outgoing mail.

  18. Brian C says:

    I am using dyndns and have no problems with delivery to any domains, including verizon.

  19. Charlie says:

    Had the same problem. We were running Exchange 5.5 and were told that 5.5 does not support their new authentation protocol (require SMTP now). After several days of rebuilding/upgrading the server to exchange2000, I was still not able to get e-mails to go out. After several calls to Verizon, found out that a new server had been ID’ed as the mail sever… outgoing.west.verizon.net [IP 206.46.170.8]. This server requires SMTP authorization. User name and pswd.

    I am fearful that this server has not yet been upgraded with whatever they did on the weekend of Feb 20th and it is only a matter of time until this stops working as well.

  20. Merv Porter says:

    From the SBS 2003 newsgroup… (03/18/04 – Re: SMTP – The remote SMTP service rejected AUTH negotiation)

    THANKS Daniel for unveiling the mystery SMTP issue experienced with

    Verizon.

    I have since called Verizon and they have explained that they are now

    supporting SMTP AUTH – the new SMTP server is

    "outgoing.west.verizon.net" and of course you will need a username and

    password to authenticate.

    -Jill

  21. Merv Porter says:

    Maybe that should read: 03/18/05 🙂

  22. Merv Porter says:

    Just did some testing with outgoing.west.verizon.net as a smarthost and could not successfully send mail outbound.

  23. Carl Houseman says:

    I’m in the mid-Altantic area and outgoing.verizon.net was not allowing the auth login <username> as of a week or two ago. It does allow it now.

    My port 25 is also not blocked outbound. The posts that started this suggest that they are or will be. Has Verizon relented on this too (for those of you who noticed it previously)?

    I’m worried that after I sign up for a year contract to get the the decent price, they’ll break this or something else, for months at a time.

  24. DMS in Tampa says:

    No one can do their jobs anymore, nor do they care to……Verizon screwed up the shipment address on DSL equipment being sent to me UPS, which I discovered tracking the package on the UPS system. When I called Verizon, they wanted ME to call UPS to fix the problem they caused. The Verizon representative kept asking if I wanted him to call UPS, and it took me saying "YES" 5 TIMES before he understood (that I wanted him to call). When I hung up, he called me back to keep me on the line while he called UPS (I was thoroughly unnecessary, and thoroughly pissed off).

    They get my business because they are the only game in town……and will lose my business as soon as competition is available.

  25. Eder Pires says:

    all of a suddent nothing works with verizon DSL. not allowed to host web site, port 80 is blocked. I’m computer geek yet, but I used to be able to do VPN with my friends over the internet, set up iis web server. and now nothing works .. I’m realy lost in this world of computers. .. I can’t unerstand what’s my WAN IP .. no ip I give to my friend to try to connect to me .. works .. ..I realy need a vPN goin on .. any help will be apreciated guys.. you’re should be the one getting paid the big bucks to help people .. these "engeeners", most of them are dumb..

    thanks

  26. Kurt Miller says:

    I too have verizon business DSL in the mid-atlantic and just tried ‘again’ to use them as a smarthost and was denied.

    The crazy thing is that I host 6 domains on my SBS2003 server, and the first one works fine with its own DNS settings. Yahoo accepts it without being SPAM, but the others all get marked as SPAM.

  27. Kurt Miller says:

    Just found this link that says to use your full login name with verizon’s outgoing server. I tried it using SBS2003 and it WORKED!!!!

    Go figure.

    Here is the link for reading pleasure:

    http://www.winserverhelp.com/ftopic46350.html

    Look for the part from Merv Porter on 08/02/2005.

  28. Kurt Miller says:

    Ok, back to the drawing board. THe smarthost is not sending all of our email out. I use the blackberry webclient and auto-forward all of my email to it via exchange. I am not getting all of my email on the blackberry, which leads me to believe my users emails are all not being delivered as well….

  29. Darren Britsch says:

    I appear to be having the same problem with QWest DSL. The problem with using thier smart host, if it will work, is that anyone using reverse DNS on your MX records will fail (Hot mail, gmail, and many of my clients customers, etc)

    I believe I have to setup a provider as the highest priority MX and relay everything to and from it.

    Thoughts?

  30. attilahooper says:

    I traced my evolution smtp communications;

    looks like verizons outgoing smtp take an:

    AUTH PLAIN <username>::<pass>

    user and pass in base 64 of course.

  31. attilahooper says:

    when I do a base64 decode of the authentication string sent by my mail client (evolution) I get myusername {this character ?} mypassword

    what the hell base64 character is that ?

    Justtrying to figure out what evolution is sending outgoing.

  32. attilahooper says:

    darn, the character is UTF+28FF (Braille pattern dots-12345678) from the character map.

    Not a valid base64 number. what gives ?

  33. Curren says:

    Really funny…. I was having problems sending outbound mail after waiting 2 months for them to move my service (they really screwed that up).

    Best part of the whole thing, when I was on the phone with tech support I specifically asked if they blocked any ports…. Of course they said NO. Nice to know they lie to their customers. If it didn’t cost 4X as much I would have a cable connection by now.

    Thanks everyone, for all of the leg work.