473 add training data tool by okolekar · Pull Request #481 · GispoCoding/eis_toolkit

okolekar · 2025-01-10T13:15:59Z

Added the code to convert data points to raster. The base raster values will be 0 and the points will have the value as per the attribute selected.

…bounds

In this commit I have added the argument to the points_to_raster.py as per the user, value of the points will be displayed on the raster.

nmaarnio · 2025-01-20T06:25:26Z

Hi @okolekar , what's the status of this PR – is it ready for review?

okolekar · 2025-01-27T07:58:50Z

There are some changes required.

okolekar · 2025-01-31T08:24:14Z

Hi Niko,
the PR is ready for review.
Thankyou for your time and support.

nmaarnio

Hi @okolekar . Good work with this one, looks like a well-designed tool! However, the points_to_raster tool seems to me a lot like our rasterize tool. The rasterize tool allows also users to buffer around points, so I wonder what exactly do we want out of this tool that rasterize does not handle? I might be just missing something here. I did not yet have time to look into random_sampling.py, I will do that too soon hopefully

nmaarnio · 2025-02-10T09:31:38Z

.vscode/launch.json

This file should be removed from the PR

Added the file to the git ignore.

nmaarnio · 2025-02-10T09:44:38Z

eis_toolkit/training_data_tools/points_to_raster.py

+@beartype
+def save_raster(path: str, array: np.ndarray, meta: dict = None, overwrite: bool = False):
+    """Save the given raster NumPy array and metadata in a raster format at the provided path.
+
+    Args:
+        path: Path to store the raster.
+        array: Raster NumPy array.
+        meta: Raster metadata.
+        overwrite: overwrites the existing raster file if present, default false.
+    """
+
+    if os.path.exists(path) and overwrite is False:
+        print(f"File already exists: {os.path.basename(path)}.")
+        return
+    else:
+        if array.ndim == 2:
+            array = np.expand_dims(array, axis=0)
+
+    with rasterio.open(path, "w", compress="lzw", **meta) as dst:
+        dst.write(array)
+        dst.close()


This function should be removed as it does not relate to the implemented feature itself

nmaarnio · 2025-02-10T09:46:57Z

eis_toolkit/training_data_tools/points_to_raster.py

+def _point_to_raster(raster_array, raster_meta, positives, attribute, radius, buffer):
+    with MemoryFile() as memfile:
+        raster_meta["driver"] = "GTiff"
+        with memfile.open(**raster_meta) as datawriter:
+            datawriter.write(raster_array, 1)
+
+        with memfile.open() as memraster:
+            if not check_matching_crs(
+                objects=[memraster, positives],
+            ):
+                raise NonMatchingCrsException("The raster and geodataframe are not in the same CRS.")
+
+            # Select only positives that are within raster bounds
+            positives = positives.cx[
+                memraster.bounds.left : memraster.bounds.right,  # noqa: E203
+                memraster.bounds.bottom : memraster.bounds.top,  # noqa: E203
+            ]
+
+            if attribute is not None:
+                values = positives[attribute]
+            else:
+                values = [1]
+
+            positives_rows, positives_cols = rasterio.transform.rowcol(
+                memraster.transform, positives.geometry.x, positives.geometry.y
+            )
+            raster_array[positives_rows, positives_cols] = values
+
+            unique_values = list(set(values))
+
+            if radius is not None:
+                for target_value in unique_values:
+                    raster_array = _create_buffer_around_labels(raster_array, radius, target_value, buffer)
+
+    return raster_array, raster_meta


I would refactor this function and leave out the MemoryFile stuff.

Hi niko,
The MemoryFile stuff helps selecting the correct points form the geodataframe.
I tried to achieve the same output using the transform from the raster meta data however I could not achieve the same values of the raster bounds as I get it from the memory file. There is always some error associated.
Should I still remove the memoryfile?

nmaarnio · 2025-02-10T09:50:34Z

eis_toolkit/training_data_tools/points_to_raster.py

+    positives: geopandas.GeoDataFrame,
+    attribute: Optional[str] = None,
+    radius: Optional[int] = None,
+    buffer: Optional[str] = None,
+    template_raster: Optional[rasterio.io.DatasetReader] = None,
+    coord_west: Optional[Number] = None,
+    coord_north: Optional[Number] = None,
+    coord_east: Optional[Number] = None,
+    coord_south: Optional[Number] = None,
+    target_epsg: Optional[int] = None,
+    target_pixel_size: Optional[int] = None,
+    raster_width: Optional[int] = None,
+    raster_height: Optional[int] = None,
+    nodata_value: Optional[Number] = None,


I would harmonize the inputs of this function to match how we have implemented other EIS Toolkit tools. That is, when a base raster is needed, the user needs to give raster profile / metadata and the base raster is constructed from that, see e.g. distance_computation implementation. We will provide the user the option to give either a base raster or define pixel size + extent in QGIS if they wish to do so. So I suggest us to remove all parameters from template_raster up until nodata_value and simply put raster_profile parameter there instead

These ways to define a base raster you have defined here are anyway available as create_constant_raster_... tools separately

nmaarnio · 2025-02-10T09:53:07Z

eis_toolkit/training_data_tools/points_to_raster.py

+    the desired number of pixels for rows and columns.
+
+    Args:
+        positives: The geodataframe points set to be converted into raster.


I think the first parameter could be more generic like deposits or just geodataframe – unless this tool is meant just for positive point sample, in which case I think the tool's and Python module's names should reflect this more precisely.

…save raster function 3) Removed the Memoryfile function 4) Renamed the positives

okolekar · 2025-02-17T13:48:25Z

Hi @okolekar . Good work with this one, looks like a well-designed tool! However, the points_to_raster tool seems to me a lot like our rasterize tool. The rasterize tool allows also users to buffer around points, so I wonder what exactly do we want out of this tool that rasterize does not handle? I might be just missing something here. I did not yet have time to look into random_sampling.py, I will do that too soon hopefully

This tool allows us to select the radius size for the buffer and also allows to select the type of buffer (min max or average) In addition to the above mentioned points it would also be possible (in the future) to select the shape of the buffer to be applied like square or circular if requested.

nmaarnio

Hi Omkar, I took another look at this and I'd just make a small modification to one parameter type. Other than that, one test seems to be failing and the docs files missing (that go here) but after they are fixed, we can merge this!

nmaarnio · 2025-03-18T13:16:30Z

eis_toolkit/training_data_tools/points_to_raster.py

+    raster_profile: Union[profiles.Profile, dict],
+    attribute: Optional[str] = None,
+    radius: Optional[int] = None,
+    buffer: Optional[str] = None,


I'd recommend making this parameter a Literal type, since only certain values are accepted.

Hello Niko,
I have made the requested changes!

nmaarnio

Looks good now, merging!

okolekar added 5 commits December 27, 2024 17:19

Added the tool point to raster and tests

8eee70f

Corrected the plot function added the argument to plot the data with …

d862bcb

…bounds

Points to raster code now accepts geodataframe

acf0429

In this commit I have added the argument to the points_to_raster.py as per the user, value of the points will be displayed on the raster.

Made cosmetic changes

e6a1565

Renamed the plots

f802a12

Added the mark_negatives on the raster

4267db6

okolekar marked this pull request as draft January 27, 2025 09:38

okolekar added 3 commits January 31, 2025 09:16

Added the code for random negative sampling

0809ea1

Added test testing_random_sampling

90f5ac5

Some cosmetic changes done

c3929de

okolekar marked this pull request as ready for review January 31, 2025 08:23

nmaarnio requested changes Feb 10, 2025

View reviewed changes

Omkar added 2 commits February 14, 2025 12:21

Changes made: 1) Added launch.json file to gitignore. 2) Removed the …

f87a532

…save raster function 3) Removed the Memoryfile function 4) Renamed the positives

Refactored the points_to_raster.py according to the requested changes

b86be86

Delete .vscode/launch.json

5bde0f1

nmaarnio requested changes Mar 18, 2025

View reviewed changes

nmaarnio mentioned this pull request Mar 18, 2025

Add CLI functions for points_to_raster and generate_negatives tools #486

Closed

okolekar added 3 commits March 20, 2025 09:40

Added the Literal to points_to_raster

aff595f

fixed the problem with random_sampling_test.py

492d517

Added the required files

2727352

nmaarnio approved these changes Mar 25, 2025

View reviewed changes

nmaarnio merged commit 82bc314 into GispoCoding:master Mar 25, 2025
2 of 4 checks passed

nmaarnio linked an issue Mar 25, 2025 that may be closed by this pull request

[New tool] - training data tool required #473

Closed

nmaarnio mentioned this pull request Mar 25, 2025

[New tool] - training data tool required #473

Closed

Conversation

okolekar commented Jan 10, 2025

Uh oh!

nmaarnio commented Jan 20, 2025

Uh oh!

okolekar commented Jan 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

okolekar commented Jan 31, 2025

Uh oh!

nmaarnio left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

okolekar commented Feb 17, 2025

Uh oh!

nmaarnio left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nmaarnio left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

okolekar commented Jan 27, 2025 •

edited

Loading

nmaarnio left a comment •

edited

Loading

nmaarnio left a comment •

edited

Loading