Skip to content

Added the ability to do a raster_join on proj_raster types.#420

Merged
vpipkt merged 1 commit intolocationtech:developfrom
s22s:fix/419
Nov 18, 2019
Merged

Added the ability to do a raster_join on proj_raster types.#420
vpipkt merged 1 commit intolocationtech:developfrom
s22s:fix/419

Conversation

@metasim
Copy link
Member

@metasim metasim commented Nov 18, 2019

Fixes #419.

val leftGeom = st_geometry(leftExtent)
val rightGeomReproj = st_reproject(st_geometry(rightExtent), rightCRS, leftCRS)
val joinExpr = st_intersects(leftGeom, rightGeomReproj)
val joinExpr = new Column(SpatialRelation.Intersects(leftGeom.expr, rightGeomReproj.expr))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is more of nicety of deferring to the RasterFrame Expression form of spatial relations that are rewritten into the call graph anyway to help with predicate push down, and eventually, optimization.

// On the RHS we collect result as a list.
val rightAggCtx = Seq(collect_list(rightExtent) as rightExtent2, collect_list(rightCRS) as rightCRS2)
val rightAggTiles = right.tileColumns.map(c => collect_list(c) as c.columnName)
val rightAggTiles = right.tileColumns.map(c => collect_list(ExtractTile(c)) as c.columnName)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The primary fix for #419

noException shouldBe thrownBy {
val joined1 = df1.rasterJoin(df2)
val joined2 = df2.rasterJoin(df1)
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was at a loss on how to really beef this up without having to work something out by hand.

@metasim metasim requested a review from vpipkt November 18, 2019 16:36
metasim added a commit to s22s/rasterframes that referenced this pull request Nov 18, 2019
@vpipkt vpipkt merged commit a5ed5ed into locationtech:develop Nov 18, 2019
@vpipkt vpipkt deleted the fix/419 branch November 18, 2019 19:27
metasim added a commit to s22s/rasterframes that referenced this pull request Nov 19, 2019
* develop: (79 commits)
  remove unused imports
  Python unit tests check read pipeline stages
  Cruft removal.
  Fix for ML transformer read/write
  Add failing unit tests for issue 425 ML custom transformer loading is broken
  PR feedback.
  Fixed `ReprojectToLayer` to work with `proj_raster`. Closes locationtech#357 Depends on locationtech#420.
  break out commented assert into skipped unit test around masking and deserialization
  register rf_local_extract_bit with SQL functions
  Added the ability to do a raster_join on proj_raster types. Fixes locationtech#419.
  Add landsat masking section to masking docs page
  Add mask bits  python api and unit test
  Extract bits should throw on non-integral cell types
  Fix for both masking by def and value; expand code comments; update tests
  Python regression.
  Masking improvements and unit tests.
  Regression
  Added (disabled) integration test for profiling spatial index effects. Reorganized non-evaluated documentationm files to `docs`.
  Add failing unit test for mask by value on 0
  Updated python tests against spatial index functions.
  ...
metasim added a commit to s22s/rasterframes that referenced this pull request Nov 22, 2019
* develop: (24 commits)
  remove unused imports
  Python unit tests check read pipeline stages
  Cruft removal.
  Fix for ML transformer read/write
  Add failing unit tests for issue 425 ML custom transformer loading is broken
  PR feedback.
  Fixed `ReprojectToLayer` to work with `proj_raster`. Closes locationtech#357 Depends on locationtech#420.
  break out commented assert into skipped unit test around masking and deserialization
  register rf_local_extract_bit with SQL functions
  Added the ability to do a raster_join on proj_raster types. Fixes locationtech#419.
  Add landsat masking section to masking docs page
  Add mask bits  python api and unit test
  Extract bits should throw on non-integral cell types
  Fix for both masking by def and value; expand code comments; update tests
  Python regression.
  Masking improvements and unit tests.
  Regression
  Added (disabled) integration test for profiling spatial index effects. Reorganized non-evaluated documentationm files to `docs`.
  Add failing unit test for mask by value on 0
  Updated python tests against spatial index functions.
  ...

# Conflicts:
#	core/src/main/scala/org/locationtech/rasterframes/expressions/DynamicExtractors.scala
#	core/src/main/scala/org/locationtech/rasterframes/expressions/transformers/XZ2Indexer.scala
#	core/src/main/scala/org/locationtech/rasterframes/extensions/Implicits.scala
#	core/src/test/scala/examples/CreatingRasterFrames.scala
#	core/src/test/scala/org/locationtech/rasterframes/RasterLayerSpec.scala
#	core/src/test/scala/org/locationtech/rasterframes/expressions/XZ2IndexerSpec.scala
#	datasource/src/main/scala/org/locationtech/rasterframes/datasource/raster/RasterSourceRelation.scala
#	datasource/src/test/scala/org/locationtech/rasterframes/datasource/raster/RasterSourceDataSourceSpec.scala
metasim added a commit to s22s/rasterframes that referenced this pull request Nov 26, 2019
* develop:
  Fix rf_local_extract_bits argument parsing
  remove unused imports
  Python unit tests check read pipeline stages
  Cruft removal.
  Fix for ML transformer read/write
  Add failing unit tests for issue 425 ML custom transformer loading is broken
  Fixed `ReprojectToLayer` to work with `proj_raster`. Closes locationtech#357 Depends on locationtech#420.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RasterJoin error on LHS projected raster tile

2 participants