xml-diff — Compare XML Files

xml-diff compares two XML files after normalizing them: attributes and elements are sorted, so purely cosmetic ordering differences are not reported as changes.

A common database vendor's XML dump utility regularly produces output where the order of attributes changes between runs. That makes the command line diff utility useless for comparing two dumps. xml-diff sorts the attributes and elements first, then performs a diff on the results.

go install github.com/pschlump/xml-diff@latest

git clone https://github.com/pschlump/xml-diff
cd xml-diff
make build          # produces ./xml-diff

xml-diff -l left.xml -r right.xml [flags]

Following the diff(1) convention:

• 0 — the normalized files are identical
• 1 — the files differ
• 2 — an error occurred (missing file, invalid XML, bad config, …)

This makes xml-diff usable in scripts:

if xml-diff -l old.xml -r new.xml; then
	echo "no changes"
fi

For XML inputs

<?xml version="1.0" encoding="UTF-8"?>
<ConnectedApp xmlns="http://soap.sforce.com/2006/04/metadata">
	<contactEmail>foo@example.org</contactEmail>
	<label>WooCommerce</label>
	<oauthConfig>
		<scopes>Basic</scopes>
		<scopes>Api</scopes>
		<scopes>Web</scopes>
		<scopes>Full</scopes>
		<callbackUrl>https://login.salesforce.com/services/oauth2/callback</callbackUrl>
		<consumerKey>CLIENTID</consumerKey>
	</oauthConfig>
</ConnectedApp>

and

<?xml version="1.0" encoding="UTF-8"?>
<ConnectedApp xmlns="http://soap.sforce.com/2006/04/metadata">
	<contactEmail>foo@example.org</contactEmail>
	<label>WooCommerce</label>
	<oauthConfig>
		<callbackUrl>https://login.salesforce.com/services/oauth2/callback</callbackUrl>
		<consumerKey>OTHER</consumerKey>
		<scopes>Full</scopes>
		<scopes>Basic</scopes>
	</oauthConfig>
</ConnectedApp>

run:

./xml-diff -l ./testdata/left.xml -r ./testdata/right.xml

The output is:

Note that the re-ordered <scopes> elements are not reported; only the real change (CLIENTID → OTHER) is shown.

With the -byLine flag the diff is shown by entire lines:

./xml-diff -l ./testdata/left.xml -r ./testdata/right.xml -byLine

Some XML producers arbitrarily swap between representing a value as an attribute and as a child element. The -lcfg / -rcfg files normalize these representations before the comparison, and also control which tags keep their document order.

{
	"AttrsToValue": [
		{ "TagName": "tag", "AttrName": "k", "ValName": "k" }
	],
	"ValueToAttr": [
		{ "TagName": "tag", "AttrName": "k" }
	],
	"NoSort": [
		{ "TagName": "drawing" }
	]
}

• AttrsToValue — convert an attribute into a child element. With the rule above, <tag k="name" v="Neu Broderstorf"/> is normalized to <tag v="Neu Broderstorf"><k>name</k></tag>. The child element is named after ValName, or after AttrName when ValName is empty.
• ValueToAttr — the reverse: convert a child element into an attribute on its parent, so <tag v="Ya ya"><k>name</k></tag> normalizes to <tag k="name" v="Ya ya"/>.
• NoSort — children of the named tag keep their document order instead of being sorted (useful for formats like SVG where order is meaningful).

"*" may be used as a wild card for TagName or AttrName, and multiple rules may apply to the same tag.

Config files are JSON with //// comment support; __LINE__, __FILE__ and __ENV__:NAME substitutions are also available. See testdata/lcfg.json and testdata/lcfg_04.json for working examples, and make test03 / make test04 in the Makefile for how they are used.

The command line tool is in this directory; the package that does the work is xmllib:

import "github.com/pschlump/xml-diff/xmllib"

cfg, err := xmllib.ReadCfg("lcfg.json")       // optional config
buf, err := xmllib.ConvertXML(r, cfg)         // r is an io.Reader of XML
// buf contains the normalized XML

xmllib can also convert XML to JSON (xmllib.Convert).

The diff is based on the Myers algorithm (via go-diff), the most common approach to comparing differences between files.

The comparison is done on the re-serialized text rather than on the parsed node tree. This makes it possible to apply attribute↔value normalization before comparing, and the normalized XML can be saved (-lo / -ro) and fed to other tools — for example command line diff to produce patch files. See planned-changes.md for the design rationale.

It takes about 4.2 times the size of the XML file in heap to run. With 128 MB of memory you should be able to compare files of up to about 30 MB.

The XML read/parse/generate pipeline processes about 10 MB of XML per second; roughly 100 ms to compare 1 MB. Performance depends heavily on how much data has to be sorted.

make build      # build ./xml-diff
make test       # end-to-end tests against ref/ golden files
go test ./...   # unit tests
make vet        # go vet
make lint       # golangci-lint

See LICENSE.