I figured someone here might be able to explain this to me better than the ISP (or tell me they're wrong).
A lot of domain registrars provide DNS hosting, but often only after the domain has transferred, and only after the DNS is set to use their servers. This means there is a delay between (a) switching to the new DNS and (b) adding the host records to it, resulting in potential downtime for anything that relies on the DNS.
My question: If the time between (a) and (b) can be reduced to almost zero (such as by making both changes in a single hit via an API), I assumed that there'd (almost certainly) never be an incorrect DNS response given and therefore no propagation delays should apply. However the company in question told me that there would still be a propagation delay but disappeared from the chat when I asked where.
Am I missing something, and if so what?
On Mon, 14 Mar 2022 18:58:12 +0000 Mark Rogers mark@more-solutions.co.uk allegedly wrote:
I figured someone here might be able to explain this to me better than the ISP (or tell me they're wrong).
A lot of domain registrars provide DNS hosting, but often only after the domain has transferred, and only after the DNS is set to use their servers. This means there is a delay between (a) switching to the new DNS and (b) adding the host records to it, resulting in potential downtime for anything that relies on the DNS.
My question: If the time between (a) and (b) can be reduced to almost zero (such as by making both changes in a single hit via an API), I assumed that there'd (almost certainly) never be an incorrect DNS response given and therefore no propagation delays should apply. However the company in question told me that there would still be a propagation delay but disappeared from the chat when I asked where.
Am I missing something, and if so what?
Mark
First point, do you actually /need/ to transfer the domain?
I did this a few years ago and suffered because of it. See
https://baldric.net/2009/08/02/dns-failure-a-cautionary-tale/
Second point. Do you actually /need/ to add new A (or any other) records to the domain immediately ater the transfer? If so, why not add the records now (with a long TTL) and then transfer? That way it would not matter over much if there were a delay.
Mick
--------------------------------------------------------------------- Mick Morgan gpg fingerprint: FC23 3338 F664 5E66 876B 72C0 0A1F E60B 5BAD D312 https://baldric.net/about-trivia ---------------------------------------------------------------------
On Mon, 14 Mar 2022 at 19:17, mick mbm@rlogin.net wrote:
First point, do you actually /need/ to transfer the domain?
Need is a strong word. But I have domains across a couple of registrars which have various flaws I wanted to get past. A lot of the domains are non-critical so I'm moving them first but using them as a test of procedure.
https://baldric.net/2009/08/02/dns-failure-a-cautionary-tale/
Those issues all sound familiar!
Second point. Do you actually /need/ to add new A (or any other) records to the domain immediately ater the transfer? If so, why not add the records now (with a long TTL) and then transfer? That way it would not matter over much if there were a delay.
The issue is that (at present) I'm using the DNS provided by the registrar. So the DNS records are all correct in the old registrar's DNS, but they will be deleted at some point after I move to the new one. So the records need migrating to the new host.
There is an obvious workaround: use a separate DNS. And this is something I have considered and will probably do at least for domains that matter. But for some domains, it's easier to use the provider's DNS because they also use basic functions from that provider such as web or email forwarding, which generally assume if not require that the DNS is set to use their own servers.
Moreover, this is really about filling a hole in my understanding of how DNS works.
Hi Mark,
If I remember correctly, there could be a delay of up to 12h when you transfer between DNS... In fact, now after searching, I recall the reduction of TTL can make it a tad easier. Look here: https://chrisburge.net/blog/avoid-dns-propagation-delay-when-updating-dns TTL will force clients to update the dns record, but it isn't going to change the speed at which the DNS to DNS communication works. I really doubt that it would take anywhere near the 5 days, 48 or even 12 hours. Last I tampered with a live DNS, propagation took literary minutes (alright, maybe a few dozen minutes ;) )
Another way would be to split webserver to host "old" and "new" location, so as the new entry gets advertised, users are visiting the new instance and not the old.
Hope that helps, Bart
pon., 14 mar 2022 o 18:58 Mark Rogers mark@more-solutions.co.uk napisał(a):
I figured someone here might be able to explain this to me better than the ISP (or tell me they're wrong).
A lot of domain registrars provide DNS hosting, but often only after the domain has transferred, and only after the DNS is set to use their servers. This means there is a delay between (a) switching to the new DNS and (b) adding the host records to it, resulting in potential downtime for anything that relies on the DNS.
My question: If the time between (a) and (b) can be reduced to almost zero (such as by making both changes in a single hit via an API), I assumed that there'd (almost certainly) never be an incorrect DNS response given and therefore no propagation delays should apply. However the company in question told me that there would still be a propagation delay but disappeared from the chat when I asked where.
Am I missing something, and if so what?
-- Mark Rogers // More Solutions Ltd (Peterborough Office) // 0344 251 1450 Registered in England (0456 0902) 21 Drakes Mews, Milton Keynes, MK8 0ER _______________________________________________ To unsubscribe send an email to main-leave@lists.alug.org.uk http://www.alug.org.uk/ Unsubscribe? See message headers or the web site above!
On Mon, 14 Mar 2022 at 19:36, B D dzidek23@gmail.com wrote:
TTL will force clients to update the dns record, but it isn't going to change the speed at which the DNS to DNS communication works.
It's the DNS to DNS bit that I'm not clear on.
Suppose a client is sat behind a router that uses Google's 8.8.8.8 DNS, and assume host1.example.com has just been added to the DNS (it never existed before and nothing has ever requested it before) with a TTL of 1hr.
The way I (possibly wrongly) understood it, at this point neither the router's DNS nor Google know anything about host1.example.com, and that remains the case until the client requests it in their web browser. Their router doesn't know so it asks Google, Google doesn't know so it asks the authoritative DNS, that does know and sens back an IP address with a TTL of 1hr. Google stores it, makes a note not to ask again for 1hr, and passes it to the client's router, which again caches it for an hour and passes it on to the client's browser, which might well cache it until the browser or PC is restarted.
In this scenario, any change to host1.example.com will not be visible to anyone else who uses Google's DNS until that hour is up, after which the next request for host1.example.com will trigger a fresh authoritative lookup. (And some DNS servers might not honour the 1hr TTL, and browsers are laws unto themselves.) But in principle this is where propagation delay comes from, *as I understand it - and I might be wrong*.
The key here, if I understand correctly, is that when host1.example.com is added to DNS it isn't published to Google (and all the other DNS servers); it just sits there until something asks for it that either doesn't already know the answer, or does but the cached answer has expired.
So if I update the domain to use a new DNS server, then provided I get records into the DNS before anything asks for them, then any requests will either be served from the cache values from the old server until they expire, or from the new server after that; the chance of asking the new server before the records are present can be made vanishingly small if the delay between switching DNS and adding the records is small enough.
HOWEVER: What I seem to be being told is that it doesn't work this way and that whether or not there are queries against the new DNS before it has records added, there will be a propagation delay. If this is correct then my understanding above is wrong.
(One potential issue would be if the new DNS didn't update immediately with new records, perhaps only showing them after an hourly database reload. But from some quick tests using dig @<authoritative DNS> they seem to be available from the authoritative DNS immediately they're added.)
Mark,
I think you are right in principal. The only issue is there's more than just two levels of DNS (the router and google). Have a look at https://en.wikipedia.org/wiki/Domain_Name_System
The problem is not only the number of caches involved, browser, operating system, router. You have root DNS and according to wikipedia there were 938 root servers. They are partially linked to the clients IP (global location). From there it is down to TLD etc.
My understanding is that if you host your domain here in the UK, with the server located next to it and your client comes from a UK based ISP they wont see any delay. However, if your ISP is in the USA, your DNS sits in Germany and client connects from Australia there's a good chance they might see some dns inconsistencies.
https://en.wikipedia.org/wiki/DNS_root_zone
On Mon, 14 Mar 2022 at 19:36, B D dzidek23@gmail.com wrote:
TTL will force clients to update the dns record, but it isn't going to change the speed at which the DNS to DNS communication works.
It's the DNS to DNS bit that I'm not clear on.
Suppose a client is sat behind a router that uses Google's 8.8.8.8 DNS, and assume host1.example.com has just been added to the DNS (it never existed before and nothing has ever requested it before) with a TTL of 1hr.
The way I (possibly wrongly) understood it, at this point neither the router's DNS nor Google know anything about host1.example.com, and that remains the case until the client requests it in their web browser. Their router doesn't know so it asks Google, Google doesn't know so it asks the authoritative DNS, that does know and sens back an IP address with a TTL of 1hr. Google stores it, makes a note not to ask again for 1hr, and passes it to the client's router, which again caches it for an hour and passes it on to the client's browser, which might well cache it until the browser or PC is restarted.
In this scenario, any change to host1.example.com will not be visible to anyone else who uses Google's DNS until that hour is up, after which the next request for host1.example.com will trigger a fresh authoritative lookup. (And some DNS servers might not honour the 1hr TTL, and browsers are laws unto themselves.) But in principle this is where propagation delay comes from, *as I understand it - and I might be wrong*.
The key here, if I understand correctly, is that when host1.example.com is added to DNS it isn't published to Google (and all the other DNS servers); it just sits there until something asks for it that either doesn't already know the answer, or does but the cached answer has expired.
So if I update the domain to use a new DNS server, then provided I get records into the DNS before anything asks for them, then any requests will either be served from the cache values from the old server until they expire, or from the new server after that; the chance of asking the new server before the records are present can be made vanishingly small if the delay between switching DNS and adding the records is small enough.
HOWEVER: What I seem to be being told is that it doesn't work this way and that whether or not there are queries against the new DNS before it has records added, there will be a propagation delay. If this is correct then my understanding above is wrong.
(One potential issue would be if the new DNS didn't update immediately with new records, perhaps only showing them after an hourly database reload. But from some quick tests using dig @<authoritative DNS> they seem to be available from the authoritative DNS immediately they're added.)
On Mon, 14 Mar 2022 at 22:55, BD dzidek23@gmail.com wrote:
I think you are right in principal. The only issue is there's more than just two levels of DNS (the router and google). Have a look at https://en.wikipedia.org/wiki/Domain_Name_System
Indeed, I know I simplified it.
But as I understand it it's multiple levels of the same: the authoritative DNS server knows the answer, and it isn't pushed anywhere from them, but pulled on request by a client (and then cached in multiple places which may or may not follow TTL).
Therefore if at any point the authoritative DNS is wrong (missing records, wrong records), and a request is made from it, there is a delay in any correction propagating due to those caches. However - and this is the crucial bit - if no requests are made while it is wrong, then no incorrect information ever makes it into those caches, so there is no correction to propagate.
To give a simple example: If you put a wrong entry into your DNS, access it using your web browser, realise your mistake, then correct the DNS, it will take some time before you'll be able to refresh your browser to see the correct result. However, if you discover the mistake and correct it before you access it (and before anyone else accesses it), the web browser will access it correctly on first attempt - there isn't a delay replacing the wrong information with the right information provided the wrong information was never accessed.
(Again: this is all as I understand it, what I am being told would make the above wrong. But it is both my understanding and my experience which I describe above so I'm having a hard time believing it is wrong without someone more authoritative on the subject than me telling me otherwise, and I can't automatically assume the person in a chat support session from a web hosting company is correct, nor am I arrogant enough to assume they are not.)
On Mon, Mar 14, 2022 at 06:58:12PM +0000, Mark Rogers wrote:
My question: If the time between (a) and (b) can be reduced to almost zero (such as by making both changes in a single hit via an API), I assumed that there'd (almost certainly) never be an incorrect DNS response given and therefore no propagation delays should apply. However the company in question told me that there would still be a propagation delay but disappeared from the chat when I asked where.
They're wrong or bad providers, there's a couple of ways of doing this.
The one I'm doing at the moment is I'm setting up a zone in the new provider which matches the records in the current provider and then I'm going to update the NS records so that they point to the new provider. Once this is done you can transfer the domain and tidy up any leftovers after.
There is another way where you can mix and match your NS records and setup one of the providers to be a secondary of the other so records will be copied. Then transfer the domain and tidy up the NS records and make the new provider the primary.
If the new provider can't setup a zone before you transfer the domain then I'd suggest you don't use them.
Adam --
On Tue, 15 Mar 2022 at 18:37, Adam Bower adam@thebowery.co.uk wrote:
The one I'm doing at the moment is I'm setting up a zone in the new provider which matches the records in the current provider and then I'm going to update the NS records so that they point to the new provider. Once this is done you can transfer the domain and tidy up any leftovers after.
This is how I would normally do it, but a lot of providers don't allow this which seems very odd; even after the transfer itself is complete (but the NS records still point at the old DNS) they won't let you add records to their DNS.
But if they have an API the delay between "change NS records" and "add records to DNS" can be in the order of milliseconds, which is why the question of whether there would be a propagation delay anyway matters.
If the new provider can't setup a zone before you transfer the domain then I'd suggest you don't use them.
Indeed, but I've not yet found a provider that ticks all the boxes that matter to me (where, I'll be honest, price is one of the biggest boxes, but certainly not the only one). My original plan was to separate domain registration/renewal from DNS, email, web hosting, etc so that domain renewals can be cheap and the others can be reliable. I'm still heading that way so to an extent this whole topic is moot, except that DNS is something I feel I should understand so if I have a misunderstanding I want to correct it.
On Wed, Mar 16, 2022 at 08:27:27AM +0000, Mark Rogers wrote:
If the new provider can't setup a zone before you transfer the domain then I'd suggest you don't use them.
Indeed, but I've not yet found a provider that ticks all the boxes that matter to me (where, I'll be honest, price is one of the biggest boxes, but certainly not the only one). My original plan was to separate domain registration/renewal from DNS, email, web hosting, etc so that domain renewals can be cheap and the others can be reliable.
I'd be talking to mythic beasts: https://www.mythic-beasts.com/domains#features
I don't really understand the price concern, DNS is far more important than the other services, if your DNS isn't reliable then all the other services won't be either. DNS is the one thing I would pay for reliability and service for, as then when the other things do go wrong you can change the back end services more easily.
Adam --
On Wed, 16 Mar 2022 at 11:17, Adam Bower adam@thebowery.co.uk wrote:
I don't really understand the price concern, DNS is far more important than the other services, if your DNS isn't reliable then all the other services won't be either.
Indeed, which is why I'm likely to not put that into the same basket as domain renewals.
One of my hosting providers (DigitalOcean) provides DNS services I can use freely with my account, for example. So really I could just move all the DNS there and then chop and change where the domains are registered pretty much based on price alone. One disadvantage of DO used to be that their DNS doesn't update quickly, although a quick test now suggests that's not an issue any more (or I misremember it).
The reason cost matters is that I have quite a lot of domains, a lot of them not heavily used (for example .uk domains that redirect to .co.uk) or just for email forwarding (which for the most part I'm happy to let the domain provider manage).
On Wed, 16 Mar 2022 at 11:17, Adam Bower adam@thebowery.co.uk wrote:
I don't really understand the price concern, DNS is far more important than the other services, if your DNS isn't reliable then all the other services won't be either.
Indeed, which is why I'm likely to not put that into the same basket as domain renewals.
One of my hosting providers (DigitalOcean) provides DNS services I can use freely with my account, for example. So really I could just move all the DNS there and then chop and change where the domains are registered pretty much based on price alone. One disadvantage of DO used to be that their DNS doesn't update quickly, although a quick test now suggests that's not an issue any more (or I misremember it).
The reason cost matters is that I have quite a lot of domains, a lot of them not heavily used (for example .uk domains that redirect to .co.uk) or just for email forwarding (which for the most part I'm happy to let the domain provider manage).
-
Adam Bower -
B D -
BD
-
Mark Rogers -
mick