Frankly, I don’t know, but here’s what I learnt.
It started with a customer reporting a problem
When we press the publish button on the website, we get a 403 error.
A co-worker of mine picked up the call. After trying many different things, he asks our Network team for help.
If it works on a standard Windows PC, we’re not interested in even having a look.
“What next?”, asked my colleague.
Never, ever volunteer; but the words sprang from my mouth,
“Perhaps I can be of assistance.”
Some network tracing was done, and the problem was SSL related. An SSL error was being thrown. The application server was throwing a SSL Malformed Packet back at us.
As SSL traffic is encrypted, you can’t tell much more than that, unless you turn off encryption. Not going to happen on a production system.
So what else do we see in the network trace? Ah… we’re getting packet fragmentation.
Told my colleague to look at the PMTU Discovery setting and to turn it on, to eliminate the packet fragmentation. That was wrong too, as it was already on.
But turning it OFF fixed the issue.
So what are the important “take-aways”?
- Sometimes your first, second, and even third guess, can be wrong.
- Sometimes you need to cut your losses with an issue, but in this particular case, we did not get to that stage.
IMHO, I thought we were close.
- Know what is “normal” behaviour, and what is “abnormal” behaviour. Spot the difference.
In this case, we had two network traces; a working one, and one which captured the problem.
Update 4th Oct 2009: Eric Law’s MSDN Blog post is worth a read: Internet Explorer Cannot Download https://something
Update 6th Dec 2009: Deb Shinder’s article is worth a read: SSL Acceleration and Offloading: What Are the Security Implications?
Update 17th Dec 2009: I had to revisit SSL sniffing in Internet Explorer has issues with session cookies, fancy that.