Reading from CSV file in scala

Question

I have a file in that has different data informations about a certain population.

Example of file format:

1880,Mary,F,7065
1880,Anna,F,2604
1880,Emma,F,2003
1880,Elizabeth,F,1939

We can interpret this data as “In the year 1880, 7065 female babies were born named Mary"

I have a function that reads from the file

fromFile(name:String):List[List[String]]

fromFile returns a list of lists:

List( List("1880","Mary", "F","7065"))

I am having trouble figuring out how to get the data and parsing it out for a function like this, which takes a nested list and a number,and returns a list of of entries of such year. For example if 'n' is 1880, then the return list would return all info about Mary.

 object readFile{
  val years = CSV.fromFile("my_file.csv") 

def yearIs(data: List[List[String]], n: Int): List[List[String]] = 
      ??
}

I trying to figure out how to access each element in the returned list and compare it to the given 'int', and return all the data.

You can split the line using the comma separator, it will give an array in which you have the line data. — alifirat
– alifirat, Commented Feb 9, 2016 at 16:05
if( n == line.split(",").map(._trim)), is this what you mean? — user_123945839432
– user_123945839432, Commented Feb 9, 2016 at 16:07
Your code seems a bit confused. yearIs takes a data parameter, but your code uses years.. From your comment, the line.split will return an array of entries, so you can't just compare to n. But it's not clear that you need to split at all, since fromFile seems to return a list of lists of strings, presumably a list of rows each with multiple strings? — The Archetypal Paul
– The Archetypal Paul, Commented Feb 9, 2016 at 16:14
You need to give a complete example of what you want, it will be more clear for us. — alifirat
– alifirat, Commented Feb 9, 2016 at 16:16
use tabulate nrinaudo.github.io/tabulate or Faster XML Jackson XML — oluies
– oluies, Commented Feb 9, 2016 at 19:35

dth · Accepted Answer · 2016-02-09 17:59:11Z

I would always recommend to first convert input data into a appropriate structure and doing all conversions and possibly error reporting and then do what you want to do on that.

So an appropriate structure for one record would be:

case class Record(year: Int, name: String, female: Boolean, count: Int)

Lets convert your data:

val data = CSV.fromFile("my_file.csv").map {
  case List(year, name, female, count) =>
    Record(year.toInt, name, female == "F", count.toInt)
}

You should catch a MatchError and a NumberFormatException here or try to detect these error, if you do care about error handling.

Now we can define your method yearIs in a type-safe and concise manner:

def yearIs(data: List[Record], year: Int) = data.filter(_.year == year)

You could also directly create a Map from years to list of recors:

val byYear: Map[Int, List[Record]] = data.record.groupBy(_.year)

Brian · Accepted Answer · 2016-02-09 16:42:20Z

1

I think the best way to get "a list of the years from n onward" is to compare n with the year or first element in the List using filter.

scala> def yearIs(data: List[List[String]], n: Int): List[List[String]] = {
     | data.filter(xs => xs.head.toInt > n)
     | }
yearIs: (data: List[List[String]], n: Int)List[List[String]]

scala> yearIs(data, 1880)
res6: List[List[String]] = List()

scala> yearIs(data, 1879)
res7: List[List[String]] = List(List(1880, Mary, F, 7065), List(1880, Anna, F, 2604), List(1880, Emma, F, 2003))

answered Feb 9, 2016 at 16:42

Brian

20.3k6 gold badges36 silver badges55 bronze badges

Collectives™ on Stack Overflow

Reading from CSV file in scala

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related