Enhancing CoreDNS configuration customisation #475

I just like to throw out a third option:
3. Recommended path could be: "If you need advanced configuration of in-cluster DNS you can leverage the node-local-dns pattern. This would allow for full customization of DNS routing and rules, bonus cache layer and iptables conntrack utilization reduction, and we don't have to petition upstream for changes.

Maybe this is a near and mid-term work-around/solution? And petitioning upstream for configurable coredns at provision time is the long-term plan? It sure would be nice to have that.

Thanks for jumping into the discussion, Jason.

The upstream petitioning could be very cumbersome and a long journey, so your proposed path seems interesting.

I'd like to understand what you're expecting from Kamaji, tho: you would install CoreDNS manually, and then Kamaj would configure the NodeLocalDNS add-on as we're doing with kube-proxy, CoreDNS, and Konnectivity?

I didn't give it a lot of thought in terms of implementation just was thinking about what could accomplish the goal.
Personally I don't have a huge use-case for this so I'm not speaking to my own needs.

I think your idea is pretty solid but node-local-dns works on top of CoreDNS so I'm not sure why anything with CoreDNS needs to change from what you're doing now? (Referring to your comment about installing manually)
I think just like konnectivity it would make a lot of sense to have an optional additional container for node-local-dns with customizable config through the CRD.
Some challenges I see are:

• Do you support advanced HA configurations? If not node-local-dns becomes a single-point-of-failure per-node. Most people usually accept this but just figured I'd call it out.
• node-local-dns needs to be installed as a daemonset if its used
• It changes the way the kubelet is configured with the DNS flag, so end-users would have to be responsible for doing that