On 30 July 2013 06:47, Hady elsahar <[email protected]> wrote:
>
> Hi Jona ,
>
>>
>> > i noticed that the full version of IOUtils file is in the dump branch
>> > unlike
>> > the master branch
>> > https://github.com/dbpedia/extraction-framework/blob/dump/core/src/main/scala/org/dbpedia/extraction/util/IOUtils.scala
>> >
>> > so i pulled that file exclusively from the dump branch merged it to the
>> > local branch
>> > i written small test code to utilize the IOUtils when compiling i faced the
>> > following errors :
>> >
>> > object compress is not a member of package org.apache.commons
>> > import
>> > org.apache.commons.compress.compressors.bzip2.{BZip2CompressorInputStream,BZip2CompressorOutputStream}
>> > ^
>>
>> That's because on the master branch, core/pom.xml does not declare the
>> dependency on commons-compress.
>>
>
> in the master branch , it was declared in the outer Pom.xml file , isn't it
> supposed to define global dependencies ?
The parent pom has an element called <dependencyManagement>. The
dependencies in this element are not really dependencies, they just
define the version of a certain module that will be used if one of the
related poms defines an actual dependency (in a top-level
<dependencies> element). Thus, we can just include the ID of the
module and omit the version. This way, all modules will use the same
version. For example, all the child poms contain:
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
</dependency>
There's no version, but Maven doesn't complain because the parent pom
defines it in the <dependencyManagement>.
>
> it worked when i added it in the core/pom.xml
>
>
>>
>> > value charSet is not a member of java.nio.charset.Charset
>> > def writer(file: FileLike[_], charset: Charset = Codec.UTF8.charSet):
>> > Writer =
>> > ^
>> > value charSet is not a member of java.nio.charset.Charset
>> > def reader(file: FileLike[_], charset: Charset = Codec.UTF8.charSet):
>> > Reader =
>>
>> That's because the Scala people do not care much about backwards
>> compatibility (and neither do I, I must admit).
>>
>> In Scala 2.9.2 (used on master branch), Codec.UTF8 had type Charset:
>>
>> http://www.scala-lang.org/api/2.9.2/index.html#scala.io.Codec$
>> final val UTF8: Charset
>>
>> In Scala 2.10.1 (used on dump branch), Codec.UTF8 has type Codec:
>>
>> http://www.scala-lang.org/api/2.10.1/index.html#scala.io.Codec$
>> final val UTF8: Codec
>>
>
> that's correct the Codec.UTF-8 is of type Codec in scala.10 , but this
> wasn't the problem specifically
> actually i was still using scala 2.9 (depending on the master pom file)
> the problem was that Codec.UTF-8 didnt have charSet member
Yes, because in 2.9 it was of type Charset. In 2.10, it's of type
Codec, which has a charSet member. That's what I meant.
>
> this worked fine with me (scala 2.9):
> def writer(file: FileLike[_], charset: Charset = Codec.UTF8): Writer =
>
> and i guess it should be for scala 2.10 :
> def writer(file: FileLike[_], charset: Codec = Codec.UTF8): Writer =
Yes, it would, but I wanted charset be of type Charset, not Codec.
>
>
>> > it seems that there's something not included in the packages
>> > apache.commons
>> > and Code.UTF-8
>> > should i pull the whole dump branch from Github ?
>>
>> You could try that. It would mean that your code is incompatible with
>> the master branch until we merge my changes from the dump branch. Not
>> terribly bad, but not very good either.
>>
>> or this is because
>> > another thing ?
>>
>> You could also just copy & paste the new methods from IOUtils.scala to
>> your own class and add the commons-compress dependency to your
>> pom.xml
>
>
> merging would create conflicts , even it would be a little messy but i'd
> copy the methods
> even selectively pulling new updates in the core/IOUtils.scala would make
> conflicts because there's other IOUtils object in the scripts in the master
> branch, so let's copy the methods for now
That's fine.
>
>
>
> one last Question :
>
> i noticed IOUtils.inputStream method takes FileLike abstract object
> i used the implemented class RichFile , didn't know if that's it's main
> function but it worked fine , would that be a problem?
>
> val file = new RichFile(fileName)
> val in = org.dbpedia.extraction.util.IOUtils.inputStream(file)
> val lines = Source.fromInputStream(in,"UTF-8").getLines()
Scala uses implicit conversions, so instead of the explicit conversion
new RichFile(...) you can just import a method that does that
conversion and is declared as implicit. In this case,
RichReader.wrapReader. You can basically replace the three lines of
code above by one import and one statement:
import org.dbpedia.extraction.util.RichReader.wrapReader
IOUtils.readLines(file) { line =>
...
}
As another cosmetic change, you could also import IOUtils.readLines:
import org.dbpedia.extraction.util.IOUtils.readLines
import org.dbpedia.extraction.util.RichReader.wrapReader
readLines(file) { line =>
...
}
For more examples, see most of the scripts in
https://github.com/dbpedia/extraction-framework/tree/master/scripts/src/main/scala/org/dbpedia/extraction/scripts
.
I just searched for introductions to Scala implicits, and I think this
one is pretty good:
http://www.artima.com/pins1ed/implicit-conversions-and-parameters.html
JC
>
>
>
>
>
>
> -------------------------------------------------
> Hady El-Sahar
> Research Assistant
> Center of Informatics Sciences | Nile University
>
> email : [email protected]
> Phone : +2-01220887311
> http://hadyelsahar.me/
>
>
>
------------------------------------------------------------------------------
Get your SQL database under version control now!
Version control is standard for application code, but databases havent
caught up. So what steps can you take to put your SQL databases under
version control? Why should you start doing it? Read more to find out.
http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk
_______________________________________________
Dbpedia-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dbpedia-developers
