Remove Duplicate Entry from XML using Xslt

Question

<LISTINGS>
<LISTING LISTING_ID="123456789">
    <NAME1>1</NAME1>
    <NAME1>1</NAME1>
    <NAME1>13</NAME1>
    <NAME1>13</NAME1>
    <NAME1>12</NAME1>
    <NAME1>100</NAME1>
    <NAME1>sumit is testing</NAME1>
    <NAME1>TEST IT</NAME1>
</LISTING>

<LISTING LISTING_ID="987654321">
    <NAME1>3</NAME1>
    <NAME1>3</NAME1>
    <NAME1>4</NAME1>
</LISTING>

<LISTING LISTING_ID="5656566565">
    <NAME1>3</NAME1>
    <NAME1>4</NAME1>
</LISTING>
</LISTINGS>

Output should be

<LISTINGS>
<LISTING LISTING_ID="123456789">
    <NAME1>1</NAME1>
    <NAME1>13</NAME1>
    <NAME1>12</NAME1>
    <NAME1>100</NAME1>
    <NAME1>sumit is testing</NAME1>
    <NAME1>TEST IT</NAME1>
</LISTING>

<LISTING LISTING_ID="987654321">
    <NAME1>3</NAME1>
    <NAME1>4</NAME1>
</LISTING>

<LISTING LISTING_ID="5656566565">
    <NAME1>3</NAME1>
    <NAME1>4</NAME1>
</LISTING>
</LISTINGS>

Good question, +1. See my answer for a demo of the Muenchian method for grouping -- short and efficient complete solution. — Dimitre Novatchev
– Dimitre Novatchev, Commented Dec 22, 2010 at 21:21

user357812 · Accepted Answer · 2010-12-22 21:14:58Z

This stylesheet:

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    <xsl:key name="kElementByListingIdAndValue"
             match="LISTING/*"
             use="concat(../@LISTING_ID,'+',.)"/>
    <xsl:template match="node()|@*">
        <xsl:copy>
            <xsl:apply-templates select="node()|@*"/>
        </xsl:copy>
    </xsl:template>
    <xsl:template match="LISTING/*[count(.|key('kElementByListingIdAndValue',
                                               concat(../@LISTING_ID,
                                                      '+',
                                                      .)
                                              )[1]) != 1]"/>
</xsl:stylesheet>

Output:

<LISTINGS>
    <LISTING LISTING_ID="123456789">
        <NAME1>1</NAME1>
        <NAME1>13</NAME1>
        <NAME1>12</NAME1>
        <NAME1>100</NAME1>
        <NAME1>sumit is testing</NAME1>
        <NAME1>TEST IT</NAME1>
    </LISTING>
    <LISTING LISTING_ID="987654321">
        <NAME1>3</NAME1>
        <NAME1>4</NAME1>
    </LISTING>
    <LISTING LISTING_ID="5656566565">
        <NAME1>3</NAME1>
        <NAME1>4</NAME1>
    </LISTING>
</LISTINGS>

Dimitre Novatchev · Accepted Answer · 2010-12-22 21:19:57Z

1

Using the Muenchian method for grouping:

<xsl:stylesheet version="1.0"
 xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
 <xsl:output omit-xml-declaration="yes" indent="yes"/>
 <xsl:strip-space elements="*"/>

 <xsl:key name="kListNameByVal" match="LISTING/NAME1"
  use="concat(generate-id(..),'+',.)"/>

 <xsl:template match="node()|@*">
  <xsl:copy>
   <xsl:apply-templates select="node()|@*"/>
  </xsl:copy>
 </xsl:template>

 <xsl:template match=
 "NAME1
   [not(generate-id()
       =
        generate-id(key('kListNameByVal',
                        concat(generate-id(..),'+',.)
                        )
                        [1]
                    )
        )
   ]
 "/>
</xsl:stylesheet>

when applied on the provided XML document:

<LISTINGS>
<LISTING LISTING_ID="123456789">
    <NAME1>1</NAME1>
    <NAME1>1</NAME1>
    <NAME1>13</NAME1>
    <NAME1>13</NAME1>
    <NAME1>12</NAME1>
    <NAME1>100</NAME1>
    <NAME1>sumit is testing</NAME1>
    <NAME1>TEST IT</NAME1>
</LISTING>

<LISTING LISTING_ID="987654321">
    <NAME1>3</NAME1>
    <NAME1>3</NAME1>
    <NAME1>4</NAME1>
</LISTING>

<LISTING LISTING_ID="5656566565">
    <NAME1>3</NAME1>
    <NAME1>4</NAME1>
</LISTING>
</LISTINGS>

the wanted, correct result is produced:

<LISTINGS>
   <LISTING LISTING_ID="123456789">
      <NAME1>1</NAME1>
      <NAME1>13</NAME1>
      <NAME1>12</NAME1>
      <NAME1>100</NAME1>
      <NAME1>sumit is testing</NAME1>
      <NAME1>TEST IT</NAME1>
   </LISTING>
   <LISTING LISTING_ID="987654321">
      <NAME1>3</NAME1>
      <NAME1>4</NAME1>
   </LISTING>
   <LISTING LISTING_ID="5656566565">
      <NAME1>3</NAME1>
      <NAME1>4</NAME1>
   </LISTING>
</LISTINGS>

answered Dec 22, 2010 at 21:19

Dimitre Novatchev

244k27 gold badges308 silver badges438 bronze badges

4 Comments

Flack Over a year ago

can you point the meaning of [1] here? I always thought that generate-id(), applied to a key, will always generate id of the first element, that matches key pattern. Also, your code works well without this predicate.

user357812 Over a year ago

@Flack: Muenchian method selects first in a kind, that's why you have to compare the node (in this case with the generate-id()) with the first one (in document order) node with the same key.

Flack Over a year ago

@Alejandro, thanks, I've got the point. Still, can you provide a link with an example, where necessity is more obvious. I just want to consolidate the knowledge a little bit.

Dimitre Novatchev Over a year ago

@Flack: Yes, but I always write [1] to make this fact absolutely clear. Many people forget or don't know the fact that when the argument of generate-id() is a node-set, the it takes only the first node, in document order, from that node-set. Exactly the same as <xsl:value-of>.

Collectives™ on Stack Overflow

Remove Duplicate Entry from XML using Xslt

2 Answers 2

Comments

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related